A breakthrough I had

[Auxilium]

Just thought I would share a breakthrough that I had. You may think it's no big deal but I was stuck for an entire hour.

So I'm writing an IPS file patcher. Long story short, the IPS file stores its "records" in this format: (3 byte offset, 2 byte LENGTH, LENGTH bytes)
The problem is that the offset and length fields are stored as Big-Endian in the file.
So when you read the numbers from the file, it would read the numbers stored as Big-Endian as Little-Endian, which causes obvious issues.

2nd problem is, there are data types for 2 byte long data, but no such thing for data that is 3 bytes long.
So you have to use a larger data type (in this case a dword) to store it. This is fine, because if you put something smaller, it will just pad the most significant bits with zeros until the data begins.

Here is where the issue arises:
I use this to swap endians for a dword

dword swapEndian32(dword x)
{
	return (x >> 24) |
		((x << 8) & 0x00FF0000) |
		((x >> 8) & 0x0000FF00) |
		(x << 24);
}

This works if all of the bytes in x are taken, for example 0x1A2B3C4D. It will convert it to 0x4D3C2B1A.
However, like I said, since I am loading a 24 bit value into a 32 bit data type, the leftmost bits get padded with zeros.
If I load ABCDEF (in hexadecimal, obviously) into x, internally it will be stored as 0x00ABCDEF.

Thus, when the endian is swapped, it will result in 0xEFCDAB00. This is the issue. Because EFCDAB00 and EFCDAB are NOT the same thing. The former is a much larger number than intended.

So how do we convert the number back to 24 bits? I was stuck on this for an hour, until I finally thought of this:

dword _32to24(dword x)
{
	return ((x >> 8) & 0x00FFFFFF);
}

First, you shift x to the right 8 bits. This will cause the problematic zeros on the right to go away.
However, now the leftmost bits contain FF. So now we have 0xFFEFCDAB.
Now, to get rid of the garbage on the left, we use bitwise and (&) with 0x00FFFFFF to get rid of them, and now we have our precious desired result: 0x00EFCDAB.

Example:

[Hippo42]

However, now the leftmost bits contain FF. So now we have 0xFFEFCDAB.

Hey! I think at the point when you had "0xEFCDAB00" you were all set. The reason why you are getting ones in the most significant bits after the right shift is probably because you are using signed types. Signed types will propagate the sign bit. Try to use unsigned types. That way, you can probably shorten your code and simply stop after right shifting by 8. There is no need to zero out the upper bits with a mask.

Oops, I just looked at your data types and you are using dword after all, which should be unsigned. That is strange, because this should work if I understand your issue correctly:

DWORD d = 0xEFCDAB00;
d >>= 8; // 0x00efcdab

Also, the MSDN documentation says that DWORD is a 32bit data type, but I think they are neglecting to mention which architectures that is true for On 64 bit machines, as the name of the type suggests, it is 64 bit. It might be better to use types with a fixed size.

The following should work, and remain working no matter if you compile for 32 or 64 bit:

unsigned __int32 d = 0xEFCDAB00;
d >>= 8; // 0x00efcdab

I'm afraid that you might also be reinventing the wheel a bit. Swapping between little and big endian is a very common problem, so naturally others who have come before you have already written code to do so. Look for ntohl, htonl, ntohs, htons on POSIX systems or _byteswap_ulong under Visual C++. I'm afraid that I cannot think of any builtin method in the C or C++ standards off the top of my head.

- - - Updated - - -

I also just thought of another method. Since pointer have iterator semantics in C++, the following would also work, and might be more generic. Instead of relying on types that may or may not oddly fit the amount of data you want to store, you could just address the memory you want to reverse byte-wise and do something like this:

void reverse_memory(char* mem, size_t len) {
	std::reverse(mem, mem + len);
}

unsigned __int32 d = 0xEFCDAB00;
reverse_memory((char*)&d, sizeof(d)); // 0x00abcdef