use ida features carefully to correctly reverse the function
here is a step-by-step guide : (iam using a very old version of cshell.dll found here)
lets check the same function in this old cshell.dll :
Code:
int __cdecl sub_1001EF50()
{
int v1; // ecx@0
int v2; // esi@1
int result; // eax@1
v2 = v1;
if ( sub_10148B80(*(_BYTE *)(v1 + 29636)) )
LOBYTE(result) = *(_BYTE *)(1056 * *(_BYTE *)(v2 + 29636) + v2 + 29644);
else
LOBYTE(result) = -1;
return result;
}
the first line is confusing because v1 is used without being initalized :
but checking v1 declaration :
its passed to the function on ecx, which means the function is actually __thiscall (not __cdecl as stated)
correct the mistake by right clicking on the function name and selecting "Set item type"
also add a parameter "this"..
now it should look like this:
Code:
int __thiscall sub_1001EF50(int this)
for better code viewing, rename v2 to this2.
now we have this line :
Code:
if ( sub_10148B80(*(_BYTE *)(this + 29636)) )
lets split this one to several parts :
Code:
if ( sub_10148B80(...) )
follow this function and check if it has something to offet:
Code:
char __cdecl sub_10148B80(unsigned int a1)
{
return a1 <= 15;
}
it seems to check if its parameter is less than or equal 15, lets keep this in mind for now.
this one points to a member of the class at offset 29636.
this one dereferences the pointer, and the member appears to have the type of _BYTE (which is equivalent to unsigned char).
its time to tell ida about the new member..
since ida doesn't have support to c++, we are going to create a c struct instead.
press SHIFT+F1 to open "Local Types" subview,
now right click somewhere and select "Insert..."
and add this struct declaration :
Code:
struct mystruct
{
char _reserved0[29636];
unsigned char m_byte1;
char _align0[3];
char _reservedn[1000000];
};
the member _reserved0 isn't real, its only there to force the compiler to put the next member (i.e m_byte1) in the correct offset,
which is 29636 in this case..
and _align0 isn't real either, but is put to maintain a struct alignment of 4.
and member _reservedn is inserted to make sure our new struct has ATLEAST the size of original class.
you might want to configure default align by selecting "Options->Compiler..."
now return to pseudecode subview and change type of this and this2 from int to mystruct* and check the difference :
Code:
int __thiscall sub_1001EF50(mystruct *this)
{
mystruct *this2; // esi@1
int result; // eax@1
this2 = this;
if ( sub_10148B80(this->m_byte1) )
LOBYTE(result) = this2->_reservedn[1056 * this2->m_byte1 + 4];
else
LOBYTE(result) = -1;
return result;
}
now we have this part :
it stats that the right hand side has the type of char (or unsigned char)
but there is a second one two lines after, and the right hand side is a negative value
so its clear that this->_reservedn[n] has the type of signed char.
the problem now is with this part :
Code:
this2->_reservedn[1056 * this2->m_byte1 + 4];
notice the appearance of this pattern :
this usually identify an array where :
x is the size of each element, y is the index (or viseverse)
z is the offset
so we have an array of some type with size of 1056 starting at offset 4 from _reservedn.
and this array has 16 elements, this explains why m_byte1 is checked first.
we will have to create another struct with size of 1056 and its first member with type of char
so create it now:
Code:
struct mysecondstruct
{
char m_char1;
char _align0[3];
char _reserved0[1052];
};
again, _align0 is there to keep struct alignment and _reserved0 to make proper struct size of 1056.
modify mystruct (right click on it and select "Edit...") so it contain the array :
Code:
struct mystruct
{
char _reserved0[29636];
unsigned char m_byte1;
char _align0[3];
char _reserved1[4];
struct mysecondstruct m_second[16];
char _reservedn[1000000];
};
notice how we pushed _reservedn down and placed a new placeholder in its location.
now close the pseudecode subview and re-disassemble the same function :
Code:
int __thiscall sub_1001EF50(mystruct *this)
{
mystruct *this2; // esi@1
int result; // eax@1
this2 = this;
if ( sub_10148B80(this->m_byte1) )
LOBYTE(result) = this2->m_second[this2->m_byte1].m_char1;
else
LOBYTE(result) = -1;
return result;
}
one last thing to do is to trace "this" pointer.. now you know it points to CLTClientShell.
now the function is decompiled, but the names we used are not very friendly, so we will rename them after moving to c++ :
Code:
class CLTClientShell
{
public:
char _reserved0[29636];
unsigned char m_iLocalPlayerIndex;
char _align0[3];
char _reserved1[4];
struct PLAYER m_Player[16];
public:
int GetLocalPlayerIndex();
};
Code:
struct PLAYER
{
char m_iIndex;
char _align0[3];
char _reserved0[1052];
};
Code:
bool IsValidPlayerIndex(int index)
{
return index <= 15;
}
int CLTClientShell::GetLocalPlayerIndex()
{
int result;
if ( IsValidPlayerIndex(m_iLocalPlayerIndex) )
result = m_Player[m_iLocalPlayerIndex].m_iIndex;
else
result = -1;
return result;
}
note how _reservedn was stripped in the c++ version, as it is no longer needed.
also, its safe to remove _align[] members as the compiler can handle this job.
at last, iam sorry for using an old cshell, but i have uninstalled the game long time ago.
if you still have any questions, feel free to contact me.
regards.