Yea the one byte at a time thing was not a real solution at all, it just worked more often then the other way i was doing it lol.

Here's what I ended up going with

Code:
public byte[] ReadFixed(Socket sk, int bufferSize)
        {
            byte[] ret = new byte[bufferSize];
            int read = 0;
            while (read < bufferSize)
            {
                read += sk.Receive(ret, read, ret.Length - read, SocketFlags.None);
            }
            return ret;
        }
Pretty much the same as you posted. I call this function once for the first 5 bytes and then again for the rest of the packet once I've figured out the length. So far my project has been working out pretty great. I've gotten 6 clients running at once and moving around the nexus without any problems.