I though you could only play 4 player ( with the backgrounds on ) with the expansion pack installed, I remember playing 4 player with just the track on a friends N64, but my N64 ( with expansion ) can have limited background scenery on.

Might be my memory though, havn't played 64 in about a year, although I really do need a few laps of Qoron VI