Because, when you fight, people grab anything to stop you. They'll pull your shirt to gain leverage, use it to throw you on the ground, pull it over your head to make you fight blind, and all that.
But a lot of idiots use it as a form of intimidation.