It's possible in some cases, but I've had situations where I've been left in the lobby and everyone else has gone into the game. Perhaps it could be caused by a variety of reasons rather than just one?