Hooking members of IDirect3DDevice9

Hello.

I think this is my second contribution to this section, hopefully it turns out to be useful.

I wrote up sort of a snippet on how to hook functions that are part of the IDirect3DDevice9 class. I won't be giving much explanation outside of the code tags since I commented mostly everything.

First I should let you know I'm using a detour function I found somewhere. I'll post it for you guys so you don't get confused, since I'm not using Microsoft's detour library.

Explanations are in the code, it's all commented.

Detour.h:
[highlight=cpp]
void *DetourFunc(BYTE *src, const BYTE *dst, const int len)
{
BYTE *jmp = (BYTE*)malloc(len+5);
DWORD dwBack;

VirtualProtect(src, len, PAGE_READWRITE, &dwBack);

memcpy(jmp, src, len);
jmp += len;
jmp[0] = 0xE9;

*(DWORD*)(jmp+1) = (DWORD)(src+len - jmp) - 5;
src[0] = 0x90; //50
src[1] = 0x90; // 58
src[2] = 0xE9;

*(DWORD*)(&src[3]) = (DWORD)(dst - src) - 7;

for (int i=7; i<len; i++) src[i] = 0x90;
VirtualProtect(src, len, dwBack, &dwBack);

return (jmp-len);
}
[/highlight]

And here's Main:
[highlight=cpp]
#include <windows.h>
#include "Detour.h"
#include <d3d9.h>

/* Type definition for our function to store the original EndScene, this way we don't
have to do a long annoying cast when detouring. */
typedef HRESULT(__stdcall* Real_EndScene)(LPDIRECT3DDEVICE9);

/* This is the function we are going to store the original EndScene in. We're going to
need the real function so that we can return properly at the end of our hooked function
otherwise we're most likely to get errors */
Real_EndScene oEndScene = NULL;

/* This is the function we are detouring the original function to. Before we call and
return normally, we can basically do whatever we want in between. This function is part
of the IDirect3DDevice9 class. Every function's first parameter in a class is always 'This'
So the first parameter is IDirect3DDevice9* or LPDIRECT3DDEVICE9. Now we can call other
functions of this class easily. */
HRESULT __stdcall hook_EndScene(LPDIRECT3DDEVICE9 pDevice)
{
MessageBox(NULL,"Hooked","Hooked",MB_OK);

/* Return using original function. */
return oEndScene(pDevice);
}

DWORD GetPointerDereference(int index)
{
/* Using the device pointer, we can easily find addresses for the functions that are
part of the class. There are 2 ways of doing this, the dereference operator or
inline assembly. Example 1: */

DWORD *DevicePointer = **(DWORD***)0x40CE08;
return DevicePointer[index];
}

DWORD GetPointerInlineAssembly(int index)
{
DWORD DevicePointer = 0x40CE08;

/* This method is a little more confusing, obviously. But with a bit of knowledge in
assembly this shouldn't be a problem. I actually find that dereferencing in assembly
is cleaner than C++, and less of an eye sore when trying to figure out what it does.

It basically does the same thing. First, dereferencing DevicePointer and putting the value
in EAX. Dereference EAX and store it in EBX, etc..

Addresses store 1 byte, since addresses are 4 bytes and we're looking for the address to
a function, we have to multiply the index number by 4, then add the result to ecx, then
dereference to get the address.

Example: If we were to do GetPointerInlineAssembly(42), we aren't adding 42 to ecx,
we're adding 168.

Note: The multiplication in assembly as shown below makes everything a little harder to
understand, you don't have to multiply inside the braces. You can multiply the
index before even starting the inline assembly.

index *= 4;
__asm
{

}
*/

__asm
{
mov eax,[DevicePointer]
mov ebx,[eax]
mov ecx,[ebx]

mov eax,index
mov ebx,4
mul ebx

mov edx,[ecx+eax]
mov DevicePointer,edx
}
return DevicePointer;
}

void Hook()
{
/* We can't hook Direct3D function if the module isn't loaded yet can we? (: */
while(!GetModuleHandle("d3d9.dll"))
{
Sleep(100);
}

/* Make sure the value isn't 0 when dereferencing ( in the GetPointer function )
otherwise you're function is going to be trying to read the value at address
0x00000000, most likely to get an access violation error.
*/

while( *(DWORD*)0x40CE08 == 0)
{
Sleep(100);
}

//Detouring
oEndScene = (Real_EndScene)DetourFunc((PBYTE)GetPointerInlineA ssembly(42),(PBYTE)hook_EndScene,5);

}

/* The usual DllMain function */
bool __stdcall DllMain(HINSTANCE hInstance,DWORD reason, void* useless)
{
if(reason == DLL_PROCESS_ATTACH)
{
CreateThread(0,0,(LPTHREAD_START_ROUTINE)Hook,0,0, 0);
}

if(reason == DLL_PROCESS_DETACH)
{
//clean up
}
return true;
}
[/highlight]

I didn't explain how to get the device pointer, maybe I'll make a small tutorial on how to get it, too bad a tutorial using olly would kind of have to be visual otherwise it's hard to understand.

Anyway, I did that from scratch and made it as simple as possible so that it would be the least confusing possible. I gave 2 methods on how to get the virtual addresses from the device pointer.

Note: Using this method, you need the device pointer, not the device address itself since it's dynamic. I used the d3d9 test environment, although it doesn't give you the device pointer, I was able to get it myself.

So yeah.. that's about it. Giving credits to Why06 for explaining the 'mul' instruction I used in the inline assembly.

If you find an mistakes, please notify me. Thanks.

Good luck.