D3D9 VMT Hooking [game device & d3d9 device]
Hey guys today I'll show you how to make a D3D9 VTable hook by replacing the pointers.
I will show you 2 methods:
- swapping the whole vtable [change the vtable ptr to point to your copied vtable]
- swapping single functions of a vtable [change the ptr at the vtable to the specified function to point to your modified]
First I will show you how to swap a function with the game device:
first you have to hook endscene/present/dip or whatever with detours/hwbp/veh/page guard or w/e to retrieve the return address,
to get the caller of the d3d9 function in your game executable. You can log the return address by using _ReturnAddress or logging
the ptr from the stack which is always located(after cleaning the stack (local vars & arguments)) at esp/ebp + 0x4. So just log it and save
it in a dword/uint_32t.
If you go to your dump in Ollydbg or IDA pro and follow your ret. addr you will see something like this[endscene]:
Now to find the vtable address you could easily breakpoint the start of this function to get pushed the caller of this subroutine
which is IN the vtable. So that means you can then easily find where ESI is set and with that you know what the address of the vtable is.
Now If you want to get a clearer structure of all here you could reclass everything to gather a class/struct which insists of pads, to tell which byte where is in the initialized class which is extremely convenient because you only have to get the base address of the memory structure which is xx bytes before your game device. Then you don't have to dereference 10x things. After that you can just define and declare you class and initialize it as follows:
some remarks:
-VP stands for standard virtual protect to give you access to write into the .text section
-tEndScene is the prototype to call endscene later in your hooked function (return oEndScene(pDevice)
and to call your vmthook function
to overwrite the original address.
//edit: had some time dependent problem, which causes me to write the rest of the tutorial tomorrow
I will show you 2 methods:
- swapping the whole vtable [change the vtable ptr to point to your copied vtable]
- swapping single functions of a vtable [change the ptr at the vtable to the specified function to point to your modified]
First I will show you how to swap a function with the game device:
first you have to hook endscene/present/dip or whatever with detours/hwbp/veh/page guard or w/e to retrieve the return address,
to get the caller of the d3d9 function in your game executable. You can log the return address by using _ReturnAddress or logging
the ptr from the stack which is always located(after cleaning the stack (local vars & arguments)) at esp/ebp + 0x4. So just log it and save
it in a dword/uint_32t.
If you go to your dump in Ollydbg or IDA pro and follow your ret. addr you will see something like this[endscene]:
Code:
MOV EAX,DWORD PTR DS:[ESI+0x10] // vtable ptr which is the game device ptr MOV ECX,DWORD PTR DS:[EAX] // vtable itself MOV EDX,DWORD PTR DS:[ECX+0xA8] // endscene offset which is relative to the vtable base address[start address] PUSH EAX // game device (pdevice) CALL EDX // d3d9 function call CMP DWORD PTR SS:[EBP+0xC],EBX // return addr place (call edx -> retn -> return address is most times the next asm line in the main routine where the subroutine is called.)
which is IN the vtable. So that means you can then easily find where ESI is set and with that you know what the address of the vtable is.
Now If you want to get a clearer structure of all here you could reclass everything to gather a class/struct which insists of pads, to tell which byte where is in the initialized class which is extremely convenient because you only have to get the base address of the memory structure which is xx bytes before your game device. Then you don't have to dereference 10x things. After that you can just define and declare you class and initialize it as follows:
Code:
class C_D3D9Device
{
public:
DWORD pDevice; // 0x00??
DWORD dwGetVMTCount( DWORD* dwVMT )
{
DWORD dwIndex = 0;
for ( dwIndex = 0; dwVMT[ dwIndex ]; dwIndex++ )
{
if ( IsBadCodePtr( ( FARPROC ) dwVMT[ dwIndex ] ) )
{
break;
}
}
return dwIndex;
}
DWORD GetVtableIndex(UINT Index)
{
if(!pDevice) return 0;
DWORD dwTmp = *(DWORD*)pDevice;
if(!dwTmp) return 0;
return dwTmp + (Index * 4); // padding of every vtable function distance (dword == 32 bits == 4 bytes)
}
};
C_D3D9Device* g_pD3D9device = NULL;
g_pD3D9device = (C_D3D9Device*)(*(DWORD*)0xclassBase);
bool IsHooked(DWORD dwHkFunc, DWORD dwFuncPtr)
{
if(*(DWORD*)dwFuncPtr != dwHkFunc)
return true;
return false;
}
DWORD VMTHook(DWORD dwFuncPtr, DWORD dwHkFunc)
{
//VP
void *orig_t = *(DWORD*)dwFuncPtr;
*(DWORD*)dwFuncPtr = (DWORD)dwHkFunc;
//VP
return pOrig;
}
void d3dmain(void *pArg)
{
while(true)
{
if(g_pD3D9device && g_pD3D9device->pDevice)
{
if(CheckHook(g_pD3D9device->GetDevice(42),(DWORD)&myEndScene))oEndScene = (tEndScene)VMTHook(g_pD3D9device->GetDevice(42),(DWORD)&myEndScene);
}
}
Sleep(200);
}
BOOL APIENTRY DllMain(HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpReserved)
{
if(fdwReason == DLL_PROCESS_ATTACH)
{
_beginthread(d3dmain, NULL, NULL); // for _beginthreadex you need to use a _stdcall thread/function
}
}
-VP stands for standard virtual protect to give you access to write into the .text section
-tEndScene is the prototype to call endscene later in your hooked function (return oEndScene(pDevice)
and to call your vmthook functionto overwrite the original address.
//edit: had some time dependent problem, which causes me to write the rest of the tutorial tomorrow
Originally Posted by Hitokiri~
