Clearing Model Cache

All other Source.Python topics and issues.
User avatar
D3CEPTION
Senior Member
Posts: 129
Joined: Tue Jan 26, 2016 1:24 pm
Location: Switzerland

Clearing Model Cache

Postby D3CEPTION » Sun Feb 07, 2016 10:22 pm

Hello Fellows,

i am trying to figure out if there is a way to use SP ( maybe memory hook ) to clear model cache from the server and even client? ( clearing of a specific model OR even the entire cache)

lets say i am spawning many DIFFERENT models on the server and client, but are going towards the cache limit ( which for "ModelMesh" is 256MB, using cache_print in console ).
Now, some models have already faded and are technically not needed anymore on the server and client.
But now i am unable to spawn any new (DIFFERENT) models or else it will crash the game.

is there a methode to release/unload any "old"/unneeded models from the cache, so that i can keep spawning new models after old ones have faded?

right now i am looking into: https://github.com/alliedmodders/hl2sdk/blob/98fe5b5a34b3721fe4d60ec7ba3a28ade3512560/public/datacache/imdlcache.h
which shows that the engine itself has commands to do so. ( they are probably applied on mapchanges etc )
User avatar
Ayuto
Project Leader
Posts: 2193
Joined: Sat Jul 07, 2012 8:17 am
Location: Germany

Postby Ayuto » Mon Feb 08, 2016 6:29 pm

You would probably want to call IMDLCache::Flush(). We haven't exposed it, but you can easily call it with the memory module. The global pointer for IMDLCache is already available through studio.cache.
User avatar
D3CEPTION
Senior Member
Posts: 129
Joined: Tue Jan 26, 2016 1:24 pm
Location: Switzerland

Postby D3CEPTION » Mon Feb 08, 2016 10:37 pm

yeah, but the problem im facing is that i dont have any dissambler to find the correct function access. i could download this one:
http://x64dbg.com/#start
which looks pretty neat (like some advanced version of ollydbg, but assumingly without any community content).
also it doesn't seem to support a script engine, so customizing the program, dumping vtables etc could become a bit of a drudgery.

should i just buy ida pro?
thanks for helping
User avatar
L'In20Cible
Project Leader
Posts: 1533
Joined: Sat Jul 14, 2012 9:29 pm
Location: Québec

Postby L'In20Cible » Mon Feb 08, 2016 11:18 pm

You can do all you need with the free version of ida. Don't waste 10k on that, lol.
User avatar
D3CEPTION
Senior Member
Posts: 129
Joined: Tue Jan 26, 2016 1:24 pm
Location: Switzerland

Postby D3CEPTION » Tue Feb 09, 2016 3:59 am

L'In20Cible wrote:You can do all you need with the free version of ida. Don't waste 10k on that, lol.

thanks for mentioning this. looking at the program ida seems like a piece of gold. i am going to take my time and look into it!
User avatar
D3CEPTION
Senior Member
Posts: 129
Joined: Tue Jan 26, 2016 1:24 pm
Location: Switzerland

Postby D3CEPTION » Wed Feb 10, 2016 10:20 am

okay, ive stumbled upon a few problems here.
first off, i am completely new to accessing machine code,using sp etc so please correct me if my assumptions are going down a wrong track.

logically, to access a function it would make sense to me to route like this : binary -> function. now, a function will probably be routed through several pointers, depending on the hierarchical complexity..
so im wondering if i have to respect all the pointers manually in and via my route or does the identifier/signature that i can dump via ida represent this pointer-squence?

now, following ayutos suggestion to use model_cache as a pointer, i have discovered that it is only the pointer to my mainclass so i am wondering if this is even the correct approach to access IMDLCache?

CMDLCache
-CTier3AppSystem<class IMDLCache,0>
--CTier2AppSystem<class IMDLCache,0>
---CTier1AppSystem<class IMDLCache,0>
----CTier0AppSystem<class IMDLCache>
-----CBaseAppSystem<class IMDLCache>
------IMDLCache
-------IAppSystem
-IStudioDataCache
--IAppSystem
-CDefaultDataCacheClient
--IDataCacheClient


following the print it looks like "IMDLCache", is something like a 6th subclass of "CMDLCache"? so cant i access it directly?
there is only one xrefs in CMDLCache::Flush(MDLCacheFlush_t) with "rodata:00097F14" as the address, so im wondering if
there is some sort of obsfuscation here since i dont have any function reference here, rather than some "datablock reference"?
how would i know how to route from this just from looking at the machinecode? do routes have to respect some sort of "classorder" or
is it possible to access my function directly with the correct pointer? ( the optimal route probably does that anyway, but how do i find it? )

so trying to use the signature of flush to route directly to it always returns me "wrong signature". im using windows and the latest bild.
is there some sort of ida script which checks the fastest route by comparing and executing different possiblities/pointers ?

also, i cant seem to find any documentation in the forum on how to execute the function after having found the correct route..
i guess memory.callback is where im heading if i want to register and execute my function later from sp?
but then im wondering, why nobody gave me any infos on how to approach this? so i assume that i am missing something and its more obvious..

thanks for help


EDIT:
i just realized, that i wrote much too hasty, im not even accessing IMDLCache, so forget that part :D
User avatar
Ayuto
Project Leader
Posts: 2193
Joined: Sat Jul 07, 2012 8:17 am
Location: Germany

Postby Ayuto » Wed Feb 10, 2016 10:55 am

Actually, it's pretty much the same like this one:
http://forums.sourcepython.com/showthread.php?1066-clientside-sv_cheats-commands-and-immune-to-triggers&p=6886&viewfull=1#post6886

For CS:S I get the following output:

Code: Select all

Inheritance Tree:
 CMDLCache
  CTier3AppSystem<IMDLCache, 0>
   CTier2AppSystem<IMDLCache, 0>
    CTier1AppSystem<IMDLCache, 0>
     CTier0AppSystem<IMDLCache>
      CBaseAppSystem<IMDLCache>
       IMDLCache
        IAppSystem
  IStudioDataCache
   IAppSystem
  CDefaultDataCacheClient
   IDataCacheClient

VTable for CMDLCache: (0, 0)
 Lin  Win Function
   0      CMDLCache::Connect(void * (*)(char  const*, int *))
   1      CMDLCache: :D isconnect(void)
   2      CMDLCache::QueryInterface(char  const*)
   3      CMDLCache::Init(void)
   4      CMDLCache::Shutdown(void)
   5    0 CMDLCache::SetCacheNotify(IMDLCacheNotify *)
   6    1 CMDLCache::FindMDL(char  const*)
   7    2 CMDLCache::AddRef(unsigned short)
   8    3 CMDLCache::Release(unsigned short)
   9    4 CMDLCache::GetRef(unsigned short)
  10    5 CMDLCache::GetStudioHdr(unsigned short)
  11    6 CMDLCache::GetHardwareData(unsigned short)
  12    7 CMDLCache::GetVCollide(unsigned short)
  13    8 CMDLCache::GetAnimBlock(unsigned short, int)
  14    9 CMDLCache::GetVirtualModel(unsigned short)
  15   10 CMDLCache::GetAutoplayList(unsigned short, unsigned short **)
  16   11 CMDLCache::GetVertexData(unsigned short)
  17   12 CMDLCache::TouchAllData(unsigned short)
  18   13 CMDLCache::SetUserData(unsigned short, void *)
  19   14 CMDLCache::GetUserData(unsigned short)
  20   15 CMDLCache::IsErrorModel(unsigned short)
  21   17 CMDLCache::Flush(MDLCacheFlush_t)
  22   16 CMDLCache::Flush(unsigned short, int)
  23   18 CMDLCache::GetModelName(unsigned short)
  24   19 CMDLCache::GetVirtualModelFast(studiohdr_t  const*, unsigned short)
  25   20 CMDLCache::BeginLock(void)
  26   21 CMDLCache::EndLock(void)
  27   22 CMDLCache::GetFrameUnlockCounterPtrOLD(void)
  28   23 CMDLCache::FinishPendingLoads(void)
  29   24 CMDLCache::GetVCollideEx(unsigned short, bool)
  30   25 CMDLCache::GetVCollideSize(unsigned short, int *)
  31   26 CMDLCache::GetAsyncLoad(MDLCacheDataType_t)
  32   27 CMDLCache::SetAsyncLoad(MDLCacheDataType_t, bool)
  33   28 CMDLCache::BeginMapLoad(void)
  34   29 CMDLCache::EndMapLoad(void)
  35   30 CMDLCache::MarkAsLoaded(unsigned short)
  36   31 CMDLCache::InitPreloadData(bool)
  37   32 CMDLCache::ShutdownPreloadData(void)
  38   33 CMDLCache::IsDataLoaded(unsigned short, MDLCacheDataType_t)
  39   34 CMDLCache::GetFrameUnlockCounterPtr(MDLCacheDataType_t)
  40   35 CMDLCache::LockStudioHdr(unsigned short)
  41   36 CMDLCache::UnlockStudioHdr(unsigned short)
  42   37 CMDLCache::PreloadModel(unsigned short)
  43   38 CMDLCache::ResetErrorModelStatus(unsigned short)
  44   39 CMDLCache::MarkFrame(void)
  45      CMDLCache::VerifyHeaders(studiohdr_t *)
  46      CMDLCache::CacheVertexData(studiohdr_t *)
  47      CMDLCache::HandleCacheNotification(DataCacheNotification_t  const&)
  48      CMDLCache::GetItemName(unsigned int, void  const*, char *, unsigned int)

VTable for IStudioDataCache: (1, 8)
 Lin  Win Function
T  0    0 CMDLCache::Connect(void * (*)(char  const*, int *))
T  1    1 CMDLCache: :D isconnect(void)
T  2    2 CMDLCache::QueryInterface(char  const*)
T  3    3 CMDLCache::Init(void)
T  4    4 CMDLCache::Shutdown(void)
T  5    5 CMDLCache::VerifyHeaders(studiohdr_t *)
T  6    6 CMDLCache::CacheVertexData(studiohdr_t *)

VTable for CDefaultDataCacheClient: (2, 12)
 Lin  Win Function
T  0    0 CMDLCache::HandleCacheNotification(DataCacheNotification_t  const&)
T  1    1 CMDLCache::GetItemName(unsigned int, void  const*, char *, unsigned int)


There are many examples here on the forums on how to call a function. The keyword you need to look for is "make_virtual_function". memory.Callback is something different. It can be used to create C++ functions which call a Python callback.
User avatar
D3CEPTION
Senior Member
Posts: 129
Joined: Tue Jan 26, 2016 1:24 pm
Location: Switzerland

Postby D3CEPTION » Wed Feb 10, 2016 11:13 am

if it is that simple, that will make things very easy, but i am wondering how to execute a function? things in the forum mainly concern functions, that are used by the game and simply hook into it to modify, but i am trying to execute a function manually? am i still missing something?

edit: okay nvm i read the post now, i just have to "run" the function in python xD
my thought process is much too complicated. source python actually makes things very easy, that is great :D
User avatar
D3CEPTION
Senior Member
Posts: 129
Joined: Tue Jan 26, 2016 1:24 pm
Location: Switzerland

Re: Clearing Model Cache

Postby D3CEPTION » Tue May 10, 2016 4:11 pm

somehow, when i flush a specific model with this code, no changes in the length of string_tables.modelprecache happen...

Code: Select all

from studio.cache import model_cache
model_cache_ptr = memory.get_object_pointer(model_cache)

def Flush_Model(index,flag):
   flush = model_cache_ptr.make_virtual_function(
      28,
      Convention.THISCALL,
      (DataType.POINTER, DataType.INT, DataType.UINT),
      DataType.VOID
   )

   flush(model_cache_ptr,index,flag)

now my guess was, that maybe "locked" models cant be flushed away, so right now im trying to call the "endlock" function, but get this error on the line datatype.void in the below endlock() function:

Code: Select all

TypeError: 'DataType' object is not iterable

Code: Select all

def EndLock():
   flush = model_cache_ptr.make_virtual_function(
      31,
      Convention.THISCALL,
      (DataType.POINTER),
      DataType.VOID,
   )

   flush(model_cache_ptr)

what am i doing wrong in endlock() ? also, has anybody in here ever done this? flush a model from server memory? i put this issue aside a few months ago, but now it became relevant to me again. also: interestingly "cache_print" in console cuts off by ~1000 entries when i call my flush function, but no changes happen to the actual cache dictionary! so i might even be doing something wrong in that functin as well? any help is much appreciated!

for further detail. this is how i send my flush function:

Code: Select all

Flush_Model(any_model_index,0xFFFFFFFF)


and this is a nice helper for the flushflags that ayuto created for me:

Code: Select all

class MDLCacheFlush_t(IntEnum):
   STUDIOHDR = 0x01
   STUDIOHWDATA = 0x02
   VCOLLIDE = 0x04
   ANIMBLOCK = 0x08
   VIRTUALMODEL = 0x10
   AUTOPLAY = 0x20
   VERTEXES = 0x40
   IGNORELOCK = 0x80000000
   ALL = 0xFFFFFFFF
User avatar
iPlayer
Developer
Posts: 590
Joined: Sat Nov 14, 2015 8:37 am
Location: Moscow
Contact:

Re: Clearing Model Cache

Postby iPlayer » Tue May 10, 2016 4:37 pm

Code: Select all

(DataType.POINTER)
equals

Code: Select all

DataType.POINTER

What you want is a tuple, and that will be

Code: Select all

(DataType.POINTER, )


Comma creates the tuple, and parentheses are here just to give it right priority.
Image /id/its_iPlayer
My plugins: Map Cycle • Killstreaker • DeadChat • Infinite Jumping • TripMines • AdPurge • Bot Damage • PLRBots • Entity AntiSpam

Hail, Companion. [...] Hands to yourself, sneak thief. Image
User avatar
D3CEPTION
Senior Member
Posts: 129
Joined: Tue Jan 26, 2016 1:24 pm
Location: Switzerland

Re: Clearing Model Cache

Postby D3CEPTION » Tue May 10, 2016 4:42 pm

thank you, yes ive tried that before, but got this error "RuntimeError: Access violation while reading address '337'."
so i disregarded the try. i gotta find out what this error means exactly for now. its definately function related. maybe im calling the wrong vtable index..
User avatar
D3CEPTION
Senior Member
Posts: 129
Joined: Tue Jan 26, 2016 1:24 pm
Location: Switzerland

Re: Clearing Model Cache

Postby D3CEPTION » Tue May 10, 2016 4:46 pm

yeah my index was wrong, i knew it worked before 2 months ago. endlock index is 33 in csgo. my bad :D
i would still like to know if anybody had any success with this or knows how to flush models from server cache!
unluckily endlock wasnt the way to go
User avatar
D3CEPTION
Senior Member
Posts: 129
Joined: Tue Jan 26, 2016 1:24 pm
Location: Switzerland

Re: Clearing Model Cache

Postby D3CEPTION » Tue May 10, 2016 4:57 pm

okay, let me rephrase everything :im trying to flush a specific model from server cache.
now i have a few questions:

1) how do i call the current modelcache length/size? e.g. 102/4096 active models ("cache_print" seems to be wrong? and stringtables.modelprecache might just be a inadequate separate printer?)

2) how do i correctly flush one? ( my attempts mostly make the server crash without giving me error returns to find the reasons myself, so someone knowledgable would be much better off here with proper server/memory understanding!)
User avatar
D3CEPTION
Senior Member
Posts: 129
Joined: Tue Jan 26, 2016 1:24 pm
Location: Switzerland

Re: Clearing Model Cache

Postby D3CEPTION » Tue May 10, 2016 8:14 pm

ill probaby spam this entire thread now.
but i have more thoughts.. :D
1) why does tempentity.base have its own precache function?
2) i figured out an idea in which, maybe i can achieve my goal of the entire above process, without any big effort, it goes as follows:

can i use sendtables, to push a tempentity to my client withouth precaching its model on the server? will the client be able to cache the model on its own?

any help is much appreciated and will not be taken for granted :P

Return to “General Discussion”

Who is online

Users browsing this forum: No registered users and 20 guests