I agree, this has gone far enough

Simple choice here guys. If you want to play against them, do so. If not, leave the game whenever they join. That's as far as it needs to go.