Not being able to invite them will probably point that they are full, so there is no room for you.
If they are rejecting you, it can have multiple reasons.
My main reason I click away people if they invite me, is most people that do that (no offence) aren't so well known with the game, and mostly have failing builds. I prefer my heroes and henchman then, that listen to what I say.
Secondly the goal can be different. Sometimes people ask me to help with the mission or quest, but I'm about to do it in HM or going to vanquish the area near it.
Don't feel dumped, you'll get the hang of it.
+ exactly what Faure said. Its not rare people are snobbie and are going to shout bad things at you. And to prevent you from getting agro of those people, only attack what they attack. Never walk on from, never last person, just stay in the middle so it's never your folt

You better not get agro of some people as after your group failed, he is going to waste an hour of his life spamming <Name> is a noob, dont invite him! shouting in the outpost. Specially in alliance battles, some people have blacklists
Good luck in Guild Wars!