Reverse engineering of Wipeout XL for modding !?

[aybe]

This thread is dedicated to the eventual reverse-engineering of Wipeout XL, for educational purpose and fun.

@bigsnake suggested whether it'd be possible to mod the game, so I decided to start this thread to act as a 'hub' for posting and tracking progress of this project.


Current progress:

• it seems that simple file swapping for track does not crash the game, which would mean that track-specific logic is not in game code as I've suggested earlier, this opens the door to eventually replacing tracks

Current thoughts:


(from what I know from the work I did on the game, this will be inaccurate partly or entirely, i.e. lots of unverified/speculative comments)

(I will not refer to any particular platform, currently the preferred one is the PC since it's the easiest to try things on)

Requirements for track modding

Topic	Difficulty	Comments
design a cool track for the mod	hard	will have to fit in memory, playable and so on, see below
design track 3d model	medium	constraints in the numbers of polygons and so on
textures	easy/medium	will have to fit platform constraints, for PSX can be tough since they should all fit in VRAM, i.e. TimTool
wipeout-specific formats: ordering tables, TRS, VEW for the hardest parts	hard/very hard/inhuman	PC code is based on PSX code, where the logic employs ordering tables for the rendering, as well as view lists defined as pointers to know what to be rendered according player POV; one must exactly understand what's up and how it works, game will crash if it's wrong or slow down if it's partly right. this was a time where every CPU cycle was expensive and therefore required some magic to keep a consistent frame rate, therefore the use of 'pre-baked' stuff that was used at run-time. this IMO will be the hardest part of the project, because it will require to dig in game code disassembly, first to understand how it works, then to replicate it as will
packing assets	relatively easy	formats are well understood, shouldn't be too difficult to export 21st century assets to it, there might be tricky parts though, i.e. will require beta-testers to spot things out

Dilemma:

• ideal platform would be PSX but the lack of tools and the complexity for modifying does not make it a good candidate
• PC is the preferred one (IDA can produce pseudo C code to read and understand) but it does not run well on modern systems
• MAC version has lots of debugging information but no pseudo C code can be generated to understand and it's PowerPC assembly
• Amiga: requires a Frankenstein platform with 3D graphics, I don't have it + what applies to the Mac w/out function names
• PC DOS: not as easy as a Windows EXE and it's Wipeout 1
• PC ATI: is a Windows EXE but Wipeout 1

TODO:

• describe the virtual workspace required to get started, e.g. VirtualBox, game version, environment configuration etc ...
• private repository for contributors
• find a simpler way to edit this post
• TBD

That's it for now, personally I started deciphering the assembly in IDA, renaming and commenting things, I'm also here for trying to reply to any questions related to game formats.

Release date:

Not tomorrow, obviously

[bigsnake]

Other then being able to swap entire tracks in the PC version of the game, one thing I've been able to do is also replace textures and sounds. This was a little while ago so I can't remember the specifics for the sounds, but the sounds you export have to be of a certain bitrate and have to be mono otherwise the game basically ignores them. The textures are easy to replace as they are in BMP format, track textures for instance are stored in their atlas form on disk, so you can just open it up and change the tiles, first thing I did was swap out the track textures with textures from WOHD which worked out very well.

[aybe]

Myself I too have other duties so it's a rather a low-priority project, though a few hours here and there can already make it advance eventually significantly.

'dream mode on'

If we understand the whole code it means we've successfully reversed it; then why simply not recompiling it under a modern environment, we'd strip out old DX7 and fix the speed in the mean time ?? (slightly too optimistic IMO but it weighs quite a bit against modifying an old EXE with all the knowledge it requires that I simply don't have) (IMO understanding the code and rewriting it can be easier (though slower) than fixing an EXE).

Obviously in first case ideally the end-user should provide the original EXE which would be used for generation, the key would be reproducibility, like SharpDX does to generate C# assemblies from C++ files; i.e. some translator. At this point basically we'd be magicians ...

'dream-mode off'

A first good exercise would be probably to try diff-out the speed-slowdown patched EXEs from the original untouched EXE to try understand the thing a bit more.

Another simple exercise would be to try to make HD textures of current tracks and see how the game behaves, this should take an afternoon: write the WAD un-packer and polygon updater, though since it's U8 they can't be larger than 256*256 (still leaves room for improvement, i.e. 4x times bigger track tiles and increased horizontal resolution for things like the pillar) <- might probably give it a go on this since it's not too hard. (PSX is a no-go for this unfortunately)

2017-04-04 00_36_47-TRACK01.png

For the WAVs basically it's a piece of cake,,,

The rest, ships, UI are IMO clearly doable and not too hard to mod.

In the past I've been able to mod game resolution though 1920*1200 fails while 1080p works, could be an outdated way of computing some variables in the game,

what I'd like:
- understand floating speed fixes on the web, make my own against original EXE (they seem quite involved)
- change default resolutions
- fix joystick, currently with XB1 pad if I move the stick the game crashes
- kill the mouse steal once and for all
- get the full game to run in a VM, currently only the demo works because it's not D3D HAL
- maybe many of these things can be fixed with Microsoft ACT, should be tried first
(need to get hold of XP64 though as the version they provide has a 64-bit db,
i.e. must develop the fix on a 64-bit machine, copy it for testing -> very boring)

etc etc ,,, a never ending discussion actually

PS:

here's what it takes to modify an EXE, there you can play Wipeout for ATI CIF properly on Win10
https://github.com/ata4/glrage
the author did a pretty good job in getting older games to work on modern systems

also MS ACT should not be overlooked I think, tons of GOG games use it actually (I might be a bit too optimistic that it'd fix ALL issues though)

there's also DxWnd but frankly only the guy who wrote it pulls out incredible patches out of it, maybe we should open a topic there see if he'd give a go on WOXL

[bigsnake]

I've started disassembling the PC exe on my end here. I've found a fair amount of stuff so far, a bunch of file reading functions which then led me to finding out where the function to load the ships is. I'm currently going through the function for loading the ships and naming things, there's a lot of other function calls that I need to figure out first before I even start to comprehend most of the stuff that's going on though. In particular I need to go through that function I've named Something_Decompression in the screenshot below, it has an entire tree of functions that go all over the place so that's going to be annoying to figure out.

I'll pick back up on this in a couple of days, I want to try and figure out the higher level function calls first so I have more of an idea of how to look at the deeper stuff.

[aybe]

Been working on it actually !

2017-04-04 17_35_56-IDA - D__Downloads_WIPEOUT2_WIPEOUTXLDEMO.idb (wipeout2.exe).png

At this point we need to share our findings through a repo, no point in spending time trying to figure things out that someone else figured out already ...

Here I can only debug the XL demo through VirtualBox because it's the only one featuring software rendering that works in VirtualBox.

If you are able to debug the full game, explain how you did it so I can do it here, then we'll work against the same IDB

btw, just found this yesterday https://github.com/narzoul/DDrawCompat/releases , it runs the game in Win10 without any compatibility fix of any sort (still too fast though), in fact what happens is that older Win9x calls were more error-tolerant than today, what MS ACT does is fixing them to return appropriate values, skip some normal exceptions on Win9x, no 32-bit z buffer etc etc and so on; that DLL does that somehow.

EDIT: good point actually, I was after low-level stuff

[bigsnake]

Sweet finds! You've already found a lot more then I have, if we work off a repo it's probably best you create it since you already have a bunch of stuff there. Are IDA database files source control friendly by the way? Wouldn't want to loose huge changes if we happen to both be working on it at the same time and want to push changes.

[aybe]

there is this but I can't make it to use an https connection: https://github.com/CubicaLabs/IDASynergy
see if you can, then there'll be cvs, otherwise it can be used without svn but each time files must be committed by hand

- - - Updated - - -

actually this might be unneeded at all ! GHO's preliminary patch is already promising (esp. speed):
https://sourceforge.net/p/dxwnd/disc...?limit=25#b67f

[bigsnake]

Nice! It would be really useful to get it running on Win10 with a working alt tab, it would make debugging so much easier. I was having problems in my VM with the game, if I hooked the debugger into it then the game would cut to a black screen then do nothing else, this was on my laptop though which isn't really cut out for good virtualization, need to install everything on the VM I have here on my desktop at some point then continue from that.

Will try out that IDA plugin in a bit, hopefully it works

[aybe]

you still haven't told what VM and game version you're using so I could try here !

what I do currently here and always works in VBOX
- start the IDA listener utility in the VM
- load the IDB in desktop, configure and start remote debugging (make sure you copy WOXL folder from VM first else IDA will complain)
- there will be 1 exception occurring in ddraw, pass it to the app.
- debugging experience works well, rarely it will crash
- but only for the demo since it's HEL, couldn't get full ver. HAL to work in a VM
(btw not sure if it's really worth trying to absolutely get debugging of full ver. HAL, their modified date differs only by a week, but if you did then explain how you did )

For the SVN, see if you can get svn command-line utility to accept the certificate when you check out against a sourceforge repo, it used to work here but not anymore. If that works but the plugin still fails when updating from the repository (see IDA output window) then it means that the PySvn they use is outdated -> the only solution then is to manually commit as I've said, not a big deal but you lose this functionality straight from what the plugins adds to IDA file menu. Basically each one keeps a base IDB, plugin export names/comments/etc to DATs as text...