You're not working out enough. If you ever do serious road work, or train for a marathon then you'll understand how much you need to consume to repair excessively exhausted muscles. I say this anecdotally y but a lot of people I know including my self have massive gains on our legs just by running too much. The same goes for any part of the body, if you are working that muscle out to the point of no return and then refueling it you'll see some sort of gains within 24 hours if your body works anything like mine and most other peoples.

It takes a few weeks or months to get seriously buffed, from what I see you just don't work out enough though. You don't have enough definition to begin with, which leads one to believe you simply aren't working out any specific muscle enough. The main worry training to failure is literally not having the resources to repair your body with. So long as you have a sound diet you'll be fine with severe soreness for a day or two. Just don't workout when you're excessively sore and everything should workout.