big refactor (#171)v1.10.0-rc1

* use in-file macros rather than global funcs for registering dll load callbacks

* move more things to macros

* fix debug crashes

* move sqvm funcs to sq managers

* get rid of context file

* refactor some squirrel stuff and ingame compilation error message

* move tier0 and playlist funcs to namespaces

* uiscript_reset concommand: don't loop forever if compilation fails

* improve showing console for ui script compile errors

* standardise concommand func naming in c++

* use lambdas for dll load callbacks so intellisense shits itself less

* use cvar change callbacks for unescaping ns_server_name and ns_server_desc

* add proper helpstrings to masterserver cvars

* add cvar help and find

* allow parsing of convar flags from string

* normalise mod fs paths to be lowercase

* move hoststate to its own file and add host_init hooks

* better IsFlagSet def

* replace files in ReadFromCache

* rename g_ModManager to g_pModManager

* formatting changes

* make cvar print work on dedi, move demo fix stuff, add findflags

* add proper map autocompletes and maps command

* formatting changes

* separate gameutils into multiple r2 headers

* Update keyvalues.cpp

* move sqvm funcs into wrappers in the manager class

* remove unnecessary header files

* lots of cleanup and starting moving to new hooking macros

* update more stuff to new hook macros

* rename project folder (:tf: commit log)

* fix up postbuild commands to use relative dir

* almost fully replaced hooking lib

* completely remove old hooking

* add nsprefix because i forgot to include it

* move exploit prevention and limits code out of serverauthentication, and have actual defs for CBasePlayer

* use modular ServerPresence system for registering servers

* add new memory lib

* accidentally pushed broke code oops

* lots of stuff idk

* implement some more prs

* improve rpakfilesystem

* fix line endings on vcxproj

* Revert "fix line endings on vcxproj"

This reverts commit 4ff7d022d2602c2dba37beba8b8df735cf5cd7d9.

* add more prs

* i swear i committed these how are they not there

* Add ability to load Datatables from files (#238)

* first version of kinda working custom datatables

* Fix copy error

* Finish custom datatables

* Fix Merge

* Fix line endings

* Add fallback to rpak when ns_prefere_datatable_from_disk is true

* fix typo

* Bug fixess

* Fix Function Registration hook

* Set convar value

* Fix Client and Ui VM

* enable server auth with ms agian

* Add Filters

* FIx unused import

* Merge remote-tracking branch 'upsteam/bobs-big-refactor-pr' into datatables

Co-authored-by: RoyalBlue1 <realEmail@veryRealURL.com>

* Add some changes from main to refactor (#243)

* Add PR template

* Update CI folder location

* Delete startup args txt files

* Fix line endings (hopefully) (#244)

* Fix line endings (hopefully)

* Fix more line endings

* Update refactor (#250)

* Add PR template

* Update CI folder location

* Delete startup args txt files

* Add editorconfig file (#246)

* Add editorconfig file

It's a cross-editor compatible config file that defines certain editor
behaviour (e.g. adding/removing newline at end of file)

It is supported by major editors like Visual Studio (Code) and by
version control  providers like GitHub.

Should end the constant adding/removing of final newline in PRs

* More settings

- unicode by default
- trim newlines
- use tabs for indentation (ugh)

* Ignore folder rename (#245)

* Hot reload banlist on player join (#233)

* added banlist hotreload

* fix formatting

* didnt append, cleared whole file oopsie

* unfuckedunban not rewriting file

* fixed not checking for new line

Co-authored-by: ScureX <47725553+ScureX@users.noreply.github.com>

* Refactor cleanup (#256)

* Fix indentation

* Fix path in clang-format command in readme

* Refactor cleanup (some formatting fixes) (#257)

* Fix some formatting

* More formatting fixes

* add scriptdatatable.cpp rewrite

* Some formatting fixes (#260)

* More formatting stuff (#261)

* various formatting changes and fixes

* Fix changed icon (#264)

* clang format, fix issues with server registration and rpak loading

* fix more formatting

* update postbuild step

* set launcher directory and error on fail creating log files

* change some stuff in exploitfixes

* only unrestrict dev commands when commandline flag is present

* fix issues with cvar flag commit

* fixup command flags better and reformat

* bring up to date with main

* fixup formatting

* improve cvar flag fixup and remove temp thing from findflags

* set serverfilter better

* avoid ptr decay when setting auth token

* add more entity functions

* Fix the MS server registration issues. (#285)

* Port ms presence reporter to std::async

* Fix crash due to std::optional being assigned nullptr.

* Fix formatting.

* Wait 20 seconds if MS returns DUPLICATE_SERVER.

* Change PERSISTENCE_MAX_SIZE to fix player authentication (#287)

The size check added in the refactor was incorrect:

- 56306: expected pdata size based on the pdef
- 512: allowance for trailing junk (r2 adds 137 bytes of trailing junk)
- 100: for some wiggle room

Co-Authored-By: pg9182 <96569817+pg9182@users.noreply.github.com>

* change miscserverscript to use actual entity arguments rather than
player index jank

* Fix token clearing hook (#290)

A certain someone forgot to put an `0x` in front of their hex number, meaning the offset is wrong.
This would cause token to be leaked again

Co-authored-by: Maya <malte.hoermeyer@web.de>
Co-authored-by: RoyalBlue1 <realEmail@veryRealURL.com>
Co-authored-by: GeckoEidechse <40122905+GeckoEidechse@users.noreply.github.com>
Co-authored-by: ScureX <47725553+ScureX@users.noreply.github.com>
Co-authored-by: Erlite <ys.aameziane@gmail.com>
Co-authored-by: Emma Miler <emma.pi@protonmail.com>
Co-authored-by: pg9182 <96569817+pg9182@users.noreply.github.com>

1 files changed, 200 insertions, 0 deletions

diff --git a/NorthstarDLL/exploitfixes_utf8parser.cpp b/NorthstarDLL/exploitfixes_utf8parser.cpp
new file mode 100644
index 00000000..e2510765
--- /dev/null
+++ b/NorthstarDLL/exploitfixes_utf8parser.cpp
@@ -0,0 +1,200 @@
+#include "pch.h"
+
+AUTOHOOK_INIT()
+
+INT64(__fastcall* sub_F1320)(DWORD a1, char* a2);
+
+// Reimplementation of an exploitable UTF decoding function in titanfall
+bool __fastcall CheckUTF8Valid(INT64* a1, DWORD* a2, char* strData)
+{
+	DWORD v3; // eax
+	char* v4; // rbx
+	char v5; // si
+	char* _strData; // rdi
+	char* v7; // rbp
+	char v11; // al
+	DWORD v12; // er9
+	DWORD v13; // ecx
+	DWORD v14; // edx
+	DWORD v15; // er8
+	int v16; // eax
+	DWORD v17; // er9
+	int v18; // eax
+	DWORD v19; // er9
+	DWORD v20; // ecx
+	int v21; // eax
+	int v22; // er9
+	DWORD v23; // edx
+	int v24; // eax
+	int v25; // er9
+	DWORD v26; // er9
+	DWORD v27; // er10
+	DWORD v28; // ecx
+	DWORD v29; // edx
+	DWORD v30; // er8
+	int v31; // eax
+	DWORD v32; // er10
+	int v33; // eax
+	DWORD v34; // er10
+	DWORD v35; // ecx
+	int v36; // eax
+	int v37; // er10
+	DWORD v38; // edx
+	int v39; // eax
+	int v40; // er10
+	DWORD v41; // er10
+	INT64 v43; // r8
+	INT64 v44; // rdx
+	INT64 v45; // rcx
+	INT64 v46; // rax
+	INT64 v47; // rax
+	char v48; // al
+	INT64 v49; // r8
+	INT64 v50; // rdx
+	INT64 v51; // rcx
+	INT64 v52; // rax
+	INT64 v53; // rax
+
+	v3 = a2[2];
+	v4 = (char*)(a1[1] + *a2);
+	v5 = 0;
+	_strData = strData;
+	v7 = &v4[*((UINT16*)a2 + 2)];
+	if (v3 >= 2)
+	{
+		++v4;
+		--v7;
+		if (v3 != 2)
+		{
+			while (1)
+			{
+				if (!MemoryAddress(v4).IsMemoryReadable(1))
+					return false; // INVALID
+
+				v11 = *v4++; // crash potential
+				if (v11 != 92)
+					goto LABEL_6;
+				v11 = *v4++;
+				if (v11 == 110)
+					break;
+				switch (v11)
+				{
+				case 't':
+					v11 = 9;
+					goto LABEL_6;
+				case 'r':
+					v11 = 13;
+					goto LABEL_6;
+				case 'b':
+					v11 = 8;
+					goto LABEL_6;
+				case 'f':
+					v11 = 12;
+					goto LABEL_6;
+				}
+				if (v11 != 117)
+					goto LABEL_6;
+				v12 = *v4 | 0x20;
+				v13 = v4[1] | 0x20;
+				v14 = v4[2] | 0x20;
+				v15 = v4[3] | 0x20;
+				v16 = 87;
+				if (v12 <= 0x39)
+					v16 = 48;
+				v17 = v12 - v16;
+				v18 = 87;
+				v19 = v17 << 12;
+				if (v13 <= 0x39)
+					v18 = 48;
+				v20 = v13 - v18;
+				v21 = 87;
+				v22 = (v20 << 8) | v19;
+				if (v14 <= 0x39)
+					v21 = 48;
+				v23 = v14 - v21;
+				v24 = 87;
+				v25 = (16 * v23) | v22;
+				if (v15 <= 0x39)
+					v24 = 48;
+				v4 += 4;
+				v26 = (v15 - v24) | v25;
+				if (v26 - 55296 <= 0x7FF)
+				{
+					if (v26 >= 0xDC00)
+						return true;
+					if (*v4 != 92 || v4[1] != 117)
+						return true;
+
+					v27 = v4[2] | 0x20;
+					v28 = v4[3] | 0x20;
+					v29 = v4[4] | 0x20;
+					v30 = v4[5] | 0x20;
+					v31 = 87;
+					if (v27 <= 0x39)
+						v31 = 48;
+					v32 = v27 - v31;
+					v33 = 87;
+					v34 = v32 << 12;
+					if (v28 <= 0x39)
+						v33 = 48;
+					v35 = v28 - v33;
+					v36 = 87;
+					v37 = (v35 << 8) | v34;
+					if (v29 <= 0x39)
+						v36 = 48;
+					v38 = v29 - v36;
+					v39 = 87;
+					v40 = (16 * v38) | v37;
+					if (v30 <= 0x39)
+						v39 = 48;
+					v4 += 6;
+					v41 = ((v30 - v39) | v40) - 56320;
+					if (v41 > 0x3FF)
+						return true;
+					v26 = v41 | ((v26 - 55296) << 10);
+				}
+				_strData += (DWORD)sub_F1320(v26, _strData);
+			LABEL_7:
+				if (v4 == v7)
+					goto LABEL_48;
+			}
+			v11 = 10;
+		LABEL_6:
+			v5 |= v11;
+			*_strData++ = v11;
+			goto LABEL_7;
+		}
+	}
+LABEL_48:
+	return true;
+}
+
+// prevent utf8 parser from crashing when provided bad data, which can be sent through user-controlled openinvites
+// clang-format off
+AUTOHOOK(Rson_ParseUTF8, engine.dll + 0xEF670, 
+bool, __fastcall, (INT64* a1, DWORD* a2, char* strData)) // 48 89 5C 24 ? 48 89 6C 24 ? 48 89 74 24 ? 57 41 54 41 55 41 56 41 57 48 83 EC 20 8B 1A
+// clang-format on
+{
+	static void* targetRetAddr = CModule("engine.dll").FindPattern("84 C0 75 2C 49 8B 16");
+
+	// only call if we're parsing utf8 data from the network (i.e. communities), otherwise we get perf issues
+	void* pReturnAddress =
+#ifdef _MSC_VER
+		_ReturnAddress()
+#else
+		__builtin_return_address(0)
+#endif
+		;
+
+	if (pReturnAddress == targetRetAddr && !CheckUTF8Valid(a1, a2, strData))
+		return false;
+
+	return Rson_ParseUTF8(a1, a2, strData);
+}
+
+ON_DLL_LOAD("engine.dll", EngineExploitFixes_UTF8Parser, (CModule module))
+{
+	AUTOHOOK_DISPATCH()
+
+	sub_F1320 = module.FindPattern("83 F9 7F 77 08 88 0A").As<INT64(__fastcall*)(DWORD, char*)>();
+}