Page 1 of 1

Clearing Model Cache

Posted: Sun Feb 07, 2016 10:22 pm
by D3CEPTION
Hello Fellows,

i am trying to figure out if there is a way to use SP ( maybe memory hook ) to clear model cache from the server and even client? ( clearing of a specific model OR even the entire cache)

lets say i am spawning many DIFFERENT models on the server and client, but are going towards the cache limit ( which for "ModelMesh" is 256MB, using cache_print in console ).
Now, some models have already faded and are technically not needed anymore on the server and client.
But now i am unable to spawn any new (DIFFERENT) models or else it will crash the game.

is there a methode to release/unload any "old"/unneeded models from the cache, so that i can keep spawning new models after old ones have faded?

right now i am looking into: https://github.com/alliedmodders/hl2sdk/blob/98fe5b5a34b3721fe4d60ec7ba3a28ade3512560/public/datacache/imdlcache.h
which shows that the engine itself has commands to do so. ( they are probably applied on mapchanges etc )

Posted: Mon Feb 08, 2016 6:29 pm
by Ayuto
You would probably want to call IMDLCache::Flush(). We haven't exposed it, but you can easily call it with the memory module. The global pointer for IMDLCache is already available through studio.cache.

Posted: Mon Feb 08, 2016 10:37 pm
by D3CEPTION
yeah, but the problem im facing is that i dont have any dissambler to find the correct function access. i could download this one:
http://x64dbg.com/#start
which looks pretty neat (like some advanced version of ollydbg, but assumingly without any community content).
also it doesn't seem to support a script engine, so customizing the program, dumping vtables etc could become a bit of a drudgery.

should i just buy ida pro?
thanks for helping

Posted: Mon Feb 08, 2016 11:18 pm
by L'In20Cible
You can do all you need with the free version of ida. Don't waste 10k on that, lol.

Posted: Tue Feb 09, 2016 3:59 am
by D3CEPTION
L'In20Cible wrote:You can do all you need with the free version of ida. Don't waste 10k on that, lol.

thanks for mentioning this. looking at the program ida seems like a piece of gold. i am going to take my time and look into it!

Posted: Wed Feb 10, 2016 10:20 am
by D3CEPTION
okay, ive stumbled upon a few problems here.
first off, i am completely new to accessing machine code,using sp etc so please correct me if my assumptions are going down a wrong track.

logically, to access a function it would make sense to me to route like this : binary -> function. now, a function will probably be routed through several pointers, depending on the hierarchical complexity..
so im wondering if i have to respect all the pointers manually in and via my route or does the identifier/signature that i can dump via ida represent this pointer-squence?

now, following ayutos suggestion to use model_cache as a pointer, i have discovered that it is only the pointer to my mainclass so i am wondering if this is even the correct approach to access IMDLCache?

CMDLCache
-CTier3AppSystem<class IMDLCache,0>
--CTier2AppSystem<class IMDLCache,0>
---CTier1AppSystem<class IMDLCache,0>
----CTier0AppSystem<class IMDLCache>
-----CBaseAppSystem<class IMDLCache>
------IMDLCache
-------IAppSystem
-IStudioDataCache
--IAppSystem
-CDefaultDataCacheClient
--IDataCacheClient


following the print it looks like "IMDLCache", is something like a 6th subclass of "CMDLCache"? so cant i access it directly?
there is only one xrefs in CMDLCache::Flush(MDLCacheFlush_t) with "rodata:00097F14" as the address, so im wondering if
there is some sort of obsfuscation here since i dont have any function reference here, rather than some "datablock reference"?
how would i know how to route from this just from looking at the machinecode? do routes have to respect some sort of "classorder" or
is it possible to access my function directly with the correct pointer? ( the optimal route probably does that anyway, but how do i find it? )

so trying to use the signature of flush to route directly to it always returns me "wrong signature". im using windows and the latest bild.
is there some sort of ida script which checks the fastest route by comparing and executing different possiblities/pointers ?

also, i cant seem to find any documentation in the forum on how to execute the function after having found the correct route..
i guess memory.callback is where im heading if i want to register and execute my function later from sp?
but then im wondering, why nobody gave me any infos on how to approach this? so i assume that i am missing something and its more obvious..

thanks for help


EDIT:
i just realized, that i wrote much too hasty, im not even accessing IMDLCache, so forget that part :D

Posted: Wed Feb 10, 2016 10:55 am
by Ayuto
Actually, it's pretty much the same like this one:
http://forums.sourcepython.com/showthread.php?1066-clientside-sv_cheats-commands-and-immune-to-triggers&p=6886&viewfull=1#post6886

For CS:S I get the following output:

Code: Select all

Inheritance Tree:
 CMDLCache
  CTier3AppSystem<IMDLCache, 0>
   CTier2AppSystem<IMDLCache, 0>
    CTier1AppSystem<IMDLCache, 0>
     CTier0AppSystem<IMDLCache>
      CBaseAppSystem<IMDLCache>
       IMDLCache
        IAppSystem
  IStudioDataCache
   IAppSystem
  CDefaultDataCacheClient
   IDataCacheClient

VTable for CMDLCache: (0, 0)
 Lin  Win Function
   0      CMDLCache::Connect(void * (*)(char  const*, int *))
   1      CMDLCache: :D isconnect(void)
   2      CMDLCache::QueryInterface(char  const*)
   3      CMDLCache::Init(void)
   4      CMDLCache::Shutdown(void)
   5    0 CMDLCache::SetCacheNotify(IMDLCacheNotify *)
   6    1 CMDLCache::FindMDL(char  const*)
   7    2 CMDLCache::AddRef(unsigned short)
   8    3 CMDLCache::Release(unsigned short)
   9    4 CMDLCache::GetRef(unsigned short)
  10    5 CMDLCache::GetStudioHdr(unsigned short)
  11    6 CMDLCache::GetHardwareData(unsigned short)
  12    7 CMDLCache::GetVCollide(unsigned short)
  13    8 CMDLCache::GetAnimBlock(unsigned short, int)
  14    9 CMDLCache::GetVirtualModel(unsigned short)
  15   10 CMDLCache::GetAutoplayList(unsigned short, unsigned short **)
  16   11 CMDLCache::GetVertexData(unsigned short)
  17   12 CMDLCache::TouchAllData(unsigned short)
  18   13 CMDLCache::SetUserData(unsigned short, void *)
  19   14 CMDLCache::GetUserData(unsigned short)
  20   15 CMDLCache::IsErrorModel(unsigned short)
  21   17 CMDLCache::Flush(MDLCacheFlush_t)
  22   16 CMDLCache::Flush(unsigned short, int)
  23   18 CMDLCache::GetModelName(unsigned short)
  24   19 CMDLCache::GetVirtualModelFast(studiohdr_t  const*, unsigned short)
  25   20 CMDLCache::BeginLock(void)
  26   21 CMDLCache::EndLock(void)
  27   22 CMDLCache::GetFrameUnlockCounterPtrOLD(void)
  28   23 CMDLCache::FinishPendingLoads(void)
  29   24 CMDLCache::GetVCollideEx(unsigned short, bool)
  30   25 CMDLCache::GetVCollideSize(unsigned short, int *)
  31   26 CMDLCache::GetAsyncLoad(MDLCacheDataType_t)
  32   27 CMDLCache::SetAsyncLoad(MDLCacheDataType_t, bool)
  33   28 CMDLCache::BeginMapLoad(void)
  34   29 CMDLCache::EndMapLoad(void)
  35   30 CMDLCache::MarkAsLoaded(unsigned short)
  36   31 CMDLCache::InitPreloadData(bool)
  37   32 CMDLCache::ShutdownPreloadData(void)
  38   33 CMDLCache::IsDataLoaded(unsigned short, MDLCacheDataType_t)
  39   34 CMDLCache::GetFrameUnlockCounterPtr(MDLCacheDataType_t)
  40   35 CMDLCache::LockStudioHdr(unsigned short)
  41   36 CMDLCache::UnlockStudioHdr(unsigned short)
  42   37 CMDLCache::PreloadModel(unsigned short)
  43   38 CMDLCache::ResetErrorModelStatus(unsigned short)
  44   39 CMDLCache::MarkFrame(void)
  45      CMDLCache::VerifyHeaders(studiohdr_t *)
  46      CMDLCache::CacheVertexData(studiohdr_t *)
  47      CMDLCache::HandleCacheNotification(DataCacheNotification_t  const&)
  48      CMDLCache::GetItemName(unsigned int, void  const*, char *, unsigned int)

VTable for IStudioDataCache: (1, 8)
 Lin  Win Function
T  0    0 CMDLCache::Connect(void * (*)(char  const*, int *))
T  1    1 CMDLCache: :D isconnect(void)
T  2    2 CMDLCache::QueryInterface(char  const*)
T  3    3 CMDLCache::Init(void)
T  4    4 CMDLCache::Shutdown(void)
T  5    5 CMDLCache::VerifyHeaders(studiohdr_t *)
T  6    6 CMDLCache::CacheVertexData(studiohdr_t *)

VTable for CDefaultDataCacheClient: (2, 12)
 Lin  Win Function
T  0    0 CMDLCache::HandleCacheNotification(DataCacheNotification_t  const&)
T  1    1 CMDLCache::GetItemName(unsigned int, void  const*, char *, unsigned int)


There are many examples here on the forums on how to call a function. The keyword you need to look for is "make_virtual_function". memory.Callback is something different. It can be used to create C++ functions which call a Python callback.

Posted: Wed Feb 10, 2016 11:13 am
by D3CEPTION
if it is that simple, that will make things very easy, but i am wondering how to execute a function? things in the forum mainly concern functions, that are used by the game and simply hook into it to modify, but i am trying to execute a function manually? am i still missing something?

edit: okay nvm i read the post now, i just have to "run" the function in python xD
my thought process is much too complicated. source python actually makes things very easy, that is great :D

Re: Clearing Model Cache

Posted: Tue May 10, 2016 4:11 pm
by D3CEPTION
somehow, when i flush a specific model with this code, no changes in the length of string_tables.modelprecache happen...

Code: Select all

from studio.cache import model_cache
model_cache_ptr = memory.get_object_pointer(model_cache)

def Flush_Model(index,flag):
   flush = model_cache_ptr.make_virtual_function(
      28,
      Convention.THISCALL,
      (DataType.POINTER, DataType.INT, DataType.UINT),
      DataType.VOID
   )

   flush(model_cache_ptr,index,flag)

now my guess was, that maybe "locked" models cant be flushed away, so right now im trying to call the "endlock" function, but get this error on the line datatype.void in the below endlock() function:

Code: Select all

TypeError: 'DataType' object is not iterable

Code: Select all

def EndLock():
   flush = model_cache_ptr.make_virtual_function(
      31,
      Convention.THISCALL,
      (DataType.POINTER),
      DataType.VOID,
   )

   flush(model_cache_ptr)

what am i doing wrong in endlock() ? also, has anybody in here ever done this? flush a model from server memory? i put this issue aside a few months ago, but now it became relevant to me again. also: interestingly "cache_print" in console cuts off by ~1000 entries when i call my flush function, but no changes happen to the actual cache dictionary! so i might even be doing something wrong in that functin as well? any help is much appreciated!

for further detail. this is how i send my flush function:

Code: Select all

Flush_Model(any_model_index,0xFFFFFFFF)


and this is a nice helper for the flushflags that ayuto created for me:

Code: Select all

class MDLCacheFlush_t(IntEnum):
   STUDIOHDR = 0x01
   STUDIOHWDATA = 0x02
   VCOLLIDE = 0x04
   ANIMBLOCK = 0x08
   VIRTUALMODEL = 0x10
   AUTOPLAY = 0x20
   VERTEXES = 0x40
   IGNORELOCK = 0x80000000
   ALL = 0xFFFFFFFF

Re: Clearing Model Cache

Posted: Tue May 10, 2016 4:37 pm
by iPlayer

Code: Select all

(DataType.POINTER)
equals

Code: Select all

DataType.POINTER

What you want is a tuple, and that will be

Code: Select all

(DataType.POINTER, )


Comma creates the tuple, and parentheses are here just to give it right priority.

Re: Clearing Model Cache

Posted: Tue May 10, 2016 4:42 pm
by D3CEPTION
thank you, yes ive tried that before, but got this error "RuntimeError: Access violation while reading address '337'."
so i disregarded the try. i gotta find out what this error means exactly for now. its definately function related. maybe im calling the wrong vtable index..

Re: Clearing Model Cache

Posted: Tue May 10, 2016 4:46 pm
by D3CEPTION
yeah my index was wrong, i knew it worked before 2 months ago. endlock index is 33 in csgo. my bad :D
i would still like to know if anybody had any success with this or knows how to flush models from server cache!
unluckily endlock wasnt the way to go

Re: Clearing Model Cache

Posted: Tue May 10, 2016 4:57 pm
by D3CEPTION
okay, let me rephrase everything :im trying to flush a specific model from server cache.
now i have a few questions:

1) how do i call the current modelcache length/size? e.g. 102/4096 active models ("cache_print" seems to be wrong? and stringtables.modelprecache might just be a inadequate separate printer?)

2) how do i correctly flush one? ( my attempts mostly make the server crash without giving me error returns to find the reasons myself, so someone knowledgable would be much better off here with proper server/memory understanding!)

Re: Clearing Model Cache

Posted: Tue May 10, 2016 8:14 pm
by D3CEPTION
ill probaby spam this entire thread now.
but i have more thoughts.. :D
1) why does tempentity.base have its own precache function?
2) i figured out an idea in which, maybe i can achieve my goal of the entire above process, without any big effort, it goes as follows:

can i use sendtables, to push a tempentity to my client withouth precaching its model on the server? will the client be able to cache the model on its own?

any help is much appreciated and will not be taken for granted :P