big refactor (#171)v1.10.0-rc1

* use in-file macros rather than global funcs for registering dll load callbacks

* move more things to macros

* fix debug crashes

* move sqvm funcs to sq managers

* get rid of context file

* refactor some squirrel stuff and ingame compilation error message

* move tier0 and playlist funcs to namespaces

* uiscript_reset concommand: don't loop forever if compilation fails

* improve showing console for ui script compile errors

* standardise concommand func naming in c++

* use lambdas for dll load callbacks so intellisense shits itself less

* use cvar change callbacks for unescaping ns_server_name and ns_server_desc

* add proper helpstrings to masterserver cvars

* add cvar help and find

* allow parsing of convar flags from string

* normalise mod fs paths to be lowercase

* move hoststate to its own file and add host_init hooks

* better IsFlagSet def

* replace files in ReadFromCache

* rename g_ModManager to g_pModManager

* formatting changes

* make cvar print work on dedi, move demo fix stuff, add findflags

* add proper map autocompletes and maps command

* formatting changes

* separate gameutils into multiple r2 headers

* Update keyvalues.cpp

* move sqvm funcs into wrappers in the manager class

* remove unnecessary header files

* lots of cleanup and starting moving to new hooking macros

* update more stuff to new hook macros

* rename project folder (:tf: commit log)

* fix up postbuild commands to use relative dir

* almost fully replaced hooking lib

* completely remove old hooking

* add nsprefix because i forgot to include it

* move exploit prevention and limits code out of serverauthentication, and have actual defs for CBasePlayer

* use modular ServerPresence system for registering servers

* add new memory lib

* accidentally pushed broke code oops

* lots of stuff idk

* implement some more prs

* improve rpakfilesystem

* fix line endings on vcxproj

* Revert "fix line endings on vcxproj"

This reverts commit 4ff7d022d2602c2dba37beba8b8df735cf5cd7d9.

* add more prs

* i swear i committed these how are they not there

* Add ability to load Datatables from files (#238)

* first version of kinda working custom datatables

* Fix copy error

* Finish custom datatables

* Fix Merge

* Fix line endings

* Add fallback to rpak when ns_prefere_datatable_from_disk is true

* fix typo

* Bug fixess

* Fix Function Registration hook

* Set convar value

* Fix Client and Ui VM

* enable server auth with ms agian

* Add Filters

* FIx unused import

* Merge remote-tracking branch 'upsteam/bobs-big-refactor-pr' into datatables

Co-authored-by: RoyalBlue1 <realEmail@veryRealURL.com>

* Add some changes from main to refactor (#243)

* Add PR template

* Update CI folder location

* Delete startup args txt files

* Fix line endings (hopefully) (#244)

* Fix line endings (hopefully)

* Fix more line endings

* Update refactor (#250)

* Add PR template

* Update CI folder location

* Delete startup args txt files

* Add editorconfig file (#246)

* Add editorconfig file

It's a cross-editor compatible config file that defines certain editor
behaviour (e.g. adding/removing newline at end of file)

It is supported by major editors like Visual Studio (Code) and by
version control  providers like GitHub.

Should end the constant adding/removing of final newline in PRs

* More settings

- unicode by default
- trim newlines
- use tabs for indentation (ugh)

* Ignore folder rename (#245)

* Hot reload banlist on player join (#233)

* added banlist hotreload

* fix formatting

* didnt append, cleared whole file oopsie

* unfuckedunban not rewriting file

* fixed not checking for new line

Co-authored-by: ScureX <47725553+ScureX@users.noreply.github.com>

* Refactor cleanup (#256)

* Fix indentation

* Fix path in clang-format command in readme

* Refactor cleanup (some formatting fixes) (#257)

* Fix some formatting

* More formatting fixes

* add scriptdatatable.cpp rewrite

* Some formatting fixes (#260)

* More formatting stuff (#261)

* various formatting changes and fixes

* Fix changed icon (#264)

* clang format, fix issues with server registration and rpak loading

* fix more formatting

* update postbuild step

* set launcher directory and error on fail creating log files

* change some stuff in exploitfixes

* only unrestrict dev commands when commandline flag is present

* fix issues with cvar flag commit

* fixup command flags better and reformat

* bring up to date with main

* fixup formatting

* improve cvar flag fixup and remove temp thing from findflags

* set serverfilter better

* avoid ptr decay when setting auth token

* add more entity functions

* Fix the MS server registration issues. (#285)

* Port ms presence reporter to std::async

* Fix crash due to std::optional being assigned nullptr.

* Fix formatting.

* Wait 20 seconds if MS returns DUPLICATE_SERVER.

* Change PERSISTENCE_MAX_SIZE to fix player authentication (#287)

The size check added in the refactor was incorrect:

- 56306: expected pdata size based on the pdef
- 512: allowance for trailing junk (r2 adds 137 bytes of trailing junk)
- 100: for some wiggle room

Co-Authored-By: pg9182 <96569817+pg9182@users.noreply.github.com>

* change miscserverscript to use actual entity arguments rather than
player index jank

* Fix token clearing hook (#290)

A certain someone forgot to put an `0x` in front of their hex number, meaning the offset is wrong.
This would cause token to be leaked again

Co-authored-by: Maya <malte.hoermeyer@web.de>
Co-authored-by: RoyalBlue1 <realEmail@veryRealURL.com>
Co-authored-by: GeckoEidechse <40122905+GeckoEidechse@users.noreply.github.com>
Co-authored-by: ScureX <47725553+ScureX@users.noreply.github.com>
Co-authored-by: Erlite <ys.aameziane@gmail.com>
Co-authored-by: Emma Miler <emma.pi@protonmail.com>
Co-authored-by: pg9182 <96569817+pg9182@users.noreply.github.com>

1 files changed, 98 insertions, 117 deletions

diff --git a/NorthstarDLL/filesystem.cpp b/NorthstarDLL/filesystem.cpp
index c17d813f..e6c6f49a 100644
--- a/NorthstarDLL/filesystem.cpp
+++ b/NorthstarDLL/filesystem.cpp
@@ -1,87 +1,69 @@
 #include "pch.h"
 #include "filesystem.h"
-#include "hooks.h"
-#include "hookutils.h"
 #include "sourceinterface.h"
 #include "modmanager.h"
 
 #include <iostream>
 #include <sstream>
 
-// hook forward declares
-typedef FileHandle_t (*ReadFileFromVPKType)(VPKData* vpkInfo, __int64* b, char* filename);
-ReadFileFromVPKType readFileFromVPK;
-FileHandle_t ReadFileFromVPKHook(VPKData* vpkInfo, __int64* b, char* filename);
+AUTOHOOK_INIT()
 
-typedef bool (*ReadFromCacheType)(IFileSystem* filesystem, char* path, void* result);
-ReadFromCacheType readFromCache;
-bool ReadFromCacheHook(IFileSystem* filesystem, char* path, void* result);
+using namespace R2;
 
-typedef void (*AddSearchPathType)(IFileSystem* fileSystem, const char* pPath, const char* pathID, SearchPathAdd_t addType);
-AddSearchPathType addSearchPathOriginal;
-void AddSearchPathHook(IFileSystem* fileSystem, const char* pPath, const char* pathID, SearchPathAdd_t addType);
+bool bReadingOriginalFile = false;
+std::string sCurrentModPath;
 
-typedef FileHandle_t (*ReadFileFromFilesystemType)(
-	IFileSystem* filesystem, const char* pPath, const char* pOptions, int64_t a4, uint32_t a5);
-ReadFileFromFilesystemType readFileFromFilesystem;
-FileHandle_t ReadFileFromFilesystemHook(IFileSystem* filesystem, const char* pPath, const char* pOptions, int64_t a4, uint32_t a5);
+ConVar* Cvar_ns_fs_log_reads;
 
-typedef VPKData* (*MountVPKType)(IFileSystem* fileSystem, const char* vpkPath);
-MountVPKType mountVPK;
-VPKData* MountVPKHook(IFileSystem* fileSystem, const char* vpkPath);
+// use the R2 namespace for game funcs
+namespace R2
+{
+	SourceInterface<IFileSystem>* g_pFilesystem;
 
-bool readingOriginalFile;
-std::string currentModPath;
-SourceInterface<IFileSystem>* g_Filesystem;
+	std::string ReadVPKFile(const char* path)
+	{
+		// read scripts.rson file, todo: check if this can be overwritten
+		FileHandle_t fileHandle = (*g_pFilesystem)->m_vtable2->Open(&(*g_pFilesystem)->m_vtable2, path, "rb", "GAME", 0);
 
-void InitialiseFilesystem(HMODULE baseAddress)
-{
-	g_Filesystem = new SourceInterface<IFileSystem>("filesystem_stdio.dll", "VFileSystem017");
-
-	// create hooks
-	HookEnabler hook;
-	ENABLER_CREATEHOOK(hook, (char*)baseAddress + 0x5CBA0, &ReadFileFromVPKHook, reinterpret_cast<LPVOID*>(&readFileFromVPK));
-	ENABLER_CREATEHOOK(
-		hook,
-		reinterpret_cast<void*>((*g_Filesystem)->m_vtable->ReadFromCache),
-		&ReadFromCacheHook,
-		reinterpret_cast<LPVOID*>(&readFromCache));
-	ENABLER_CREATEHOOK(
-		hook,
-		reinterpret_cast<void*>((*g_Filesystem)->m_vtable->AddSearchPath),
-		&AddSearchPathHook,
-		reinterpret_cast<LPVOID*>(&addSearchPathOriginal));
-	ENABLER_CREATEHOOK(hook, (char*)baseAddress + 0x15F20, &ReadFileFromFilesystemHook, reinterpret_cast<LPVOID*>(&readFileFromFilesystem));
-	ENABLER_CREATEHOOK(
-		hook, reinterpret_cast<void*>((*g_Filesystem)->m_vtable->MountVPK), &MountVPKHook, reinterpret_cast<LPVOID*>(&mountVPK));
-}
+		std::stringstream fileStream;
+		int bytesRead = 0;
+		char data[4096];
+		do
+		{
+			bytesRead = (*g_pFilesystem)->m_vtable2->Read(&(*g_pFilesystem)->m_vtable2, data, (int)std::size(data), fileHandle);
+			fileStream.write(data, bytesRead);
+		} while (bytesRead == std::size(data));
 
-std::string ReadVPKFile(const char* path)
-{
-	// read scripts.rson file, todo: check if this can be overwritten
-	FileHandle_t fileHandle = (*g_Filesystem)->m_vtable2->Open(&(*g_Filesystem)->m_vtable2, path, "rb", "GAME", 0);
+		(*g_pFilesystem)->m_vtable2->Close(*g_pFilesystem, fileHandle);
+
+		return fileStream.str();
+	}
 
-	std::stringstream fileStream;
-	int bytesRead = 0;
-	char data[4096];
-	do
+	std::string ReadVPKOriginalFile(const char* path)
 	{
-		bytesRead = (*g_Filesystem)->m_vtable2->Read(&(*g_Filesystem)->m_vtable2, data, (int)std::size(data), fileHandle);
-		fileStream.write(data, bytesRead);
-	} while (bytesRead == std::size(data));
+		// todo: should probably set search path to be g_pModName here also
 
-	(*g_Filesystem)->m_vtable2->Close(*g_Filesystem, fileHandle);
+		bReadingOriginalFile = true;
+		std::string ret = ReadVPKFile(path);
+		bReadingOriginalFile = false;
 
-	return fileStream.str();
-}
+		return ret;
+	}
+} // namespace R2
 
-std::string ReadVPKOriginalFile(const char* path)
+// clang-format off
+HOOK(AddSearchPathHook, AddSearchPath,
+void, __fastcall, (IFileSystem * fileSystem, const char* pPath, const char* pathID, SearchPathAdd_t addType))
+// clang-format on
 {
-	readingOriginalFile = true;
-	std::string ret = ReadVPKFile(path);
-	readingOriginalFile = false;
+	AddSearchPath(fileSystem, pPath, pathID, addType);
 
-	return ret;
+	// make sure current mod paths are at head
+	if (!strcmp(pathID, "GAME") && sCurrentModPath.compare(pPath) && addType == PATH_ADD_TO_HEAD)
+	{
+		AddSearchPath(fileSystem, sCurrentModPath.c_str(), "GAME", PATH_ADD_TO_HEAD);
+		AddSearchPath(fileSystem, GetCompiledAssetsPath().string().c_str(), "GAME", PATH_ADD_TO_HEAD);
+	}
 }
 
 void SetNewModSearchPaths(Mod* mod)
@@ -90,96 +72,84 @@ void SetNewModSearchPaths(Mod* mod)
 	// in the future we could also determine whether the file we're setting paths for needs a mod dir, or compiled assets
 	if (mod != nullptr)
 	{
-		if ((fs::absolute(mod->ModDirectory) / MOD_OVERRIDE_DIR).string().compare(currentModPath))
+		if ((fs::absolute(mod->m_ModDirectory) / MOD_OVERRIDE_DIR).string().compare(sCurrentModPath))
 		{
-			spdlog::info("changing mod search path from {} to {}", currentModPath, mod->ModDirectory.string());
+			spdlog::info("changing mod search path from {} to {}", sCurrentModPath, mod->m_ModDirectory.string());
 
-			addSearchPathOriginal(
-				&*(*g_Filesystem), (fs::absolute(mod->ModDirectory) / MOD_OVERRIDE_DIR).string().c_str(), "GAME", PATH_ADD_TO_HEAD);
-			currentModPath = (fs::absolute(mod->ModDirectory) / MOD_OVERRIDE_DIR).string();
+			AddSearchPath(
+				&*(*g_pFilesystem), (fs::absolute(mod->m_ModDirectory) / MOD_OVERRIDE_DIR).string().c_str(), "GAME", PATH_ADD_TO_HEAD);
+			sCurrentModPath = (fs::absolute(mod->m_ModDirectory) / MOD_OVERRIDE_DIR).string();
 		}
 	}
 	else // push compiled to head
-		addSearchPathOriginal(&*(*g_Filesystem), fs::absolute(GetCompiledAssetsPath()).string().c_str(), "GAME", PATH_ADD_TO_HEAD);
+		AddSearchPath(&*(*g_pFilesystem), fs::absolute(GetCompiledAssetsPath()).string().c_str(), "GAME", PATH_ADD_TO_HEAD);
 }
 
-bool TryReplaceFile(char* path, bool shouldCompile)
+bool TryReplaceFile(const char* pPath, bool shouldCompile)
 {
-	if (readingOriginalFile)
+	if (bReadingOriginalFile)
 		return false;
 
 	if (shouldCompile)
-		(*g_ModManager).CompileAssetsForFile(path);
+		g_pModManager->CompileAssetsForFile(pPath);
 
 	// idk how efficient the lexically normal check is
 	// can't just set all /s in path to \, since some paths aren't in writeable memory
-	auto file = g_ModManager->m_modFiles.find(fs::path(path).lexically_normal().string());
-	if (file != g_ModManager->m_modFiles.end())
+	auto file = g_pModManager->m_ModFiles.find(g_pModManager->NormaliseModFilePath(fs::path(pPath)));
+	if (file != g_pModManager->m_ModFiles.end())
 	{
-		SetNewModSearchPaths(file->second.owningMod);
+		SetNewModSearchPaths(file->second.m_pOwningMod);
 		return true;
 	}
 
 	return false;
 }
 
-FileHandle_t ReadFileFromVPKHook(VPKData* vpkInfo, __int64* b, char* filename)
+// force modded files to be read from mods, not cache
+// clang-format off
+HOOK(ReadFromCacheHook, ReadFromCache,
+bool, __fastcall, (IFileSystem * filesystem, char* pPath, void* result))
+// clang-format off
 {
-	// move this to a convar at some point when we can read them in native
-	// spdlog::info("ReadFileFromVPKHook {} {}", filename, vpkInfo->path);
-
-	// there is literally never any reason to compile here, since we'll always compile in ReadFileFromFilesystemHook in the same codepath
-	// this is called
-	if (TryReplaceFile(filename, false))
-	{
-		*b = -1;
-		return b;
-	}
-
-	return readFileFromVPK(vpkInfo, b, filename);
-}
-
-bool ReadFromCacheHook(IFileSystem* filesystem, char* path, void* result)
-{
-	// move this to a convar at some point when we can read them in native
-	// spdlog::info("ReadFromCacheHook {}", path);
-
-	if (TryReplaceFile(path, true))
+	if (TryReplaceFile(pPath, true))
 		return false;
 
-	return readFromCache(filesystem, path, result);
+	return ReadFromCache(filesystem, pPath, result);
 }
 
-void AddSearchPathHook(IFileSystem* fileSystem, const char* pPath, const char* pathID, SearchPathAdd_t addType)
+// force modded files to be read from mods, not vpk
+// clang-format off
+AUTOHOOK(ReadFileFromVPK, filesystem_stdio.dll + 0x5CBA0,
+FileHandle_t, __fastcall, (VPKData* vpkInfo, uint64_t* b, char* filename))
+// clang-format on
 {
-	addSearchPathOriginal(fileSystem, pPath, pathID, addType);
-
-	// make sure current mod paths are at head
-	if (!strcmp(pathID, "GAME") && currentModPath.compare(pPath) && addType == PATH_ADD_TO_HEAD)
+	// don't compile here because this is only ever called from OpenEx, which already compiles
+	if (TryReplaceFile(filename, false))
 	{
-		addSearchPathOriginal(fileSystem, currentModPath.c_str(), "GAME", PATH_ADD_TO_HEAD);
-		addSearchPathOriginal(fileSystem, GetCompiledAssetsPath().string().c_str(), "GAME", PATH_ADD_TO_HEAD);
+		*b = -1;
+		return b;
 	}
+
+	return ReadFileFromVPK(vpkInfo, b, filename);
 }
 
-FileHandle_t ReadFileFromFilesystemHook(IFileSystem* filesystem, const char* pPath, const char* pOptions, int64_t a4, uint32_t a5)
+// clang-format off
+AUTOHOOK(CBaseFileSystem__OpenEx, filesystem_stdio.dll + 0x15F50,
+FileHandle_t, __fastcall, (IFileSystem* filesystem, const char* pPath, const char* pOptions, uint32_t flags, const char* pPathID, char **ppszResolvedFilename))
+// clang-format on
 {
-	// this isn't super efficient, but it's necessary, since calling addsearchpath in readfilefromvpk doesn't work, possibly refactor later
-	// it also might be possible to hook functions that are called later, idk look into callstack for ReadFileFromVPK
-	if (!readingOriginalFile)
-		TryReplaceFile((char*)pPath, true);
-
-	return readFileFromFilesystem(filesystem, pPath, pOptions, a4, a5);
+	TryReplaceFile(pPath, true);
+	return CBaseFileSystem__OpenEx(filesystem, pPath, pOptions, flags, pPathID, ppszResolvedFilename);
 }
 
-VPKData* MountVPKHook(IFileSystem* fileSystem, const char* vpkPath)
+HOOK(MountVPKHook, MountVPK, VPKData*, , (IFileSystem * fileSystem, const char* pVpkPath))
 {
-	spdlog::info("MountVPK {}", vpkPath);
-	VPKData* ret = mountVPK(fileSystem, vpkPath);
+	spdlog::info("MountVPK {}", pVpkPath);
+	VPKData* ret = MountVPK(fileSystem, pVpkPath);
 
-	for (Mod mod : g_ModManager->m_loadedMods)
+	for (Mod mod : g_pModManager->m_LoadedMods)
 	{
-		if (!mod.Enabled)
+		if (!mod.m_bEnabled)
 			continue;
 
 		for (ModVPKEntry& vpkEntry : mod.Vpks)
@@ -189,13 +159,13 @@ VPKData* MountVPKHook(IFileSystem* fileSystem, const char* vpkPath)
 			{
 				// resolve vpk name and try to load one with the same name
 				// todo: we should be unloading these on map unload manually
-				std::string mapName(fs::path(vpkPath).filename().string());
+				std::string mapName(fs::path(pVpkPath).filename().string());
 				std::string modMapName(fs::path(vpkEntry.m_sVpkPath.c_str()).filename().string());
 				if (mapName.compare(modMapName))
 					continue;
 			}
 
-			VPKData* loaded = mountVPK(fileSystem, vpkEntry.m_sVpkPath.c_str());
+			VPKData* loaded = MountVPK(fileSystem, vpkEntry.m_sVpkPath.c_str());
 			if (!ret) // this is primarily for map vpks and stuff, so the map's vpk is what gets returned from here
 				ret = loaded;
 		}
@@ -203,3 +173,14 @@ VPKData* MountVPKHook(IFileSystem* fileSystem, const char* vpkPath)
 
 	return ret;
 }
+
+ON_DLL_LOAD("filesystem_stdio.dll", Filesystem, (CModule module))
+{
+	AUTOHOOK_DISPATCH()
+
+	R2::g_pFilesystem = new SourceInterface<IFileSystem>("filesystem_stdio.dll", "VFileSystem017");
+
+	AddSearchPathHook.Dispatch((*g_pFilesystem)->m_vtable->AddSearchPath);
+	ReadFromCacheHook.Dispatch((*g_pFilesystem)->m_vtable->ReadFromCache);
+	MountVPKHook.Dispatch((*g_pFilesystem)->m_vtable->MountVPK);
+}