Even if he had been "ok" with it, (putting aside its just rude) he has his reasons to decline, which neither you or myself know. Those reasons don't just dissipate even if he was forced into it.