Page 1 of 15 1234511 ... LastLast
Results 1 to 10 of 146

Thread: Reverse engineering of Wipeout

  1. #1
    Join Date
    Apr 2015
    Location
    France, Paris
    Timezone
    GMT + 1
    Posts
    310

    Default Reverse engineering of Wipeout

    I am opening this new thread to keep things separated from Reverse engineering of Wipeout 3 SE.

    After successfully drawing textured models:

    snap0829.png

    I found that there's at least 2 new types of polygons : 0x14 and 0x16

    But there's also what seems to be a wrongly interpreted header since there are others that have outstanding values.

    (been working on COMMON folder)

    I'll be back once I made some progress !

  2. #2
    Join Date
    Mar 2013
    Location
    Germany, North-Rhine Westphalia
    Timezone
    GMT + 1
    PSN ID
    JonnydotB
    Posts
    1,294

    Default

    "Antique Repro"
    XD

  3. #3
    Join Date
    Apr 2015
    Location
    France, Paris
    Timezone
    GMT + 1
    Posts
    310

    Default

    Quote Originally Posted by Jonny View Post
    "Antique Repro"
    XD


    @all

    I will only post here stuff of interests, such as models and so on, for those fond of tech details I am reporting them here https://github.com/phoboslab/wipeout/issues as it's probably a better place.

  4. #4
    Join Date
    Mar 2015
    Posts
    56

    Default

    Hi there,

    Iím trying to understand the TRS/VEW files format used in original Wipeout/WipeoutXL game.
    I was wondering if somebody could give me some help (aybe maybe?).

    Most parts of TRS are now well understood. However the occlusion culling part is still a total mystery.

    Here is the content of TRS file. Only what is still unknown is the middle part ("unknown" variables).

    They are all, i think, related to occlusion culling or additional track information.

    Code:
    Wipeout.TrackSection = Struct.create(
    	Struct.int32('nextJunction'),
    	Struct.int32('previous'),
    	Struct.int32('next'),
    	Struct.int32('positionx'),
    	Struct.int32('positiony'),
    	Struct.int32('positionz'),
    
            Struct.uint16('unknown1'), //always = 8?
            Struct.skip(10), //always = 0 padding ?
            Struct.array('unknown2', Struct.uint32(), 15), //PSX RAM addresses to VEW file loaded in memory?
            Struct.array('unknown3', Struct.uint16(), 15), //always values like 2, 4, 6, 8. VEW file data offset/length ? 
    
    	Struct.array('unknown4', Struct.uint16(), 2),  //section indexes or scene objects (PRM) indexes, or offsets inside VEW file ? seems to start near zero then keep increasing
            Struct.array('unknown5', Struct.uint32(), 1),  //always = 0 padding?
            Struct.array('unknown6', Struct.uint16(), 2),  //very similar to "unknown4"
            Struct.int32('unknown7'), //always = 0 padding ?

    
    	Struct.uint16('firstFace'),
    	Struct.uint16('numFaces'),
    	Struct.int16('unknown8'), //always = -1?
            Struct.int16('unknown9'), //always = -1?
    	Struct.uint16('flags'),
    	Struct.skip(4) //always = 0 padding?
    );
    To debug/use it just copy that section into wipeout.js of Phoboslab model viewer.

    The VEW file seems to be simpler structure, I think contains only a bunch of indexes, all unsigned 16bit.
    Probably indexes to sections or PRM scene objects ? (view lists?)

    So it should be something like this :

    Code:
    Wipeout.TrackView = Struct.create(
    	Struct.uint16('unknown') //indexes to track section/PRM objects?
    );
    Also: there seems to have some reference to "track radius" and "east/south/west/north view lists" on game CD.

    Here is a short passage from BETA.OUT file (from TRACK02 folder)

    Starting track radius pre process

    Section 0 :: north 32 south 10 west 32 east 41 all 41
    Section 1 :: north 34 south 10 west 32 east 43 all 43
    Section 2 :: north 35 south 10 west 32 east 44 all 44
    Section 3 :: north 37 south 10 west 32 east 46 all 46
    Section 4 :: north 38 south 11 west 33 east 48 all 48
    Section 5 :: north 39 south 12 west 34 east 50 all 50
    Ö
    Again, some related stuff found in WIPEOUT.EXE file (left by compiler) :

    radius.c::TrackRadiusCheckNew: Global View List Full ( North )
    radius.c::TrackRadiusCheckNew: Global View List Full ( South )
    radius.c::TrackRadiusCheckNew: Global View List Full ( West )
    radius.c::TrackRadiusCheckNew: Global View List Full ( East )
    Track radius is, I think some information related to how large a given track section is (how far the track extends sideways).

    Any help will be appreciated.
    Last edited by tigrou; 12th May 2015 at 06:27 PM.

  5. #5
    Join Date
    Apr 2015
    Location
    France, Paris
    Timezone
    GMT + 1
    Posts
    310

    Default

    Hi!

    Yesterday I've been trying to look at it, the best pattern coming seems to be a 84 bytes length:

    snap0848.png

    However there seems to be variable-length structs as the picture above is skewed at times ... Next thing I did is to disasm Wipeout 2 for PC and found this:

    snap0849.png

    So that's what they call a view list, hintful but not so helpful ... I've found the sections were they load vertices and so on and I think I've seen were they unpack each of the component (4), for the VEW however there seems to be many more things being read but since I don't know ASM ... Also I've found they use the CDQ mnemonic (http://faydoc.tripod.com/cpu/cdq.htm) so there might be real numbers in there.

    There is another problem, recent Windows version, if someone could find a way to run WOXL on it, at least we could use a debugger and step through, I'm pretty sure with a memory viewer we could find how the values are being decoded. I've been trying this http://www.wipeoutzone.com/forum/sho...layer-tutorial, the game goes black screen then I do get couldn't initialize the desired resolution. This game is from the DirectX 3 era , consequently they rarely work like many pre-DirectX 7 games on newer PCs. Unfortunately there isn't much to be done as the best emulator for these versions do not work on Windows (WineD3D), go figure ! However the game will run fine under Wine in Linux. On a Win98 PCEM virtual machine, the game will load but PCEM will crash ... (unimplemented opcode or whatever). There is another hope : VirtualBOX which uses WineD3D for acceleration, needs to be tested though.

    I still do have my P200 + 3DFX where I used to play it, but the problem is rather is a software one, can we find a convenient debugger for Win98 ? BTW, I've used IDA. Otherwise have you tried no$psx ? It might help though it lacks many of the facilities IDA provides and targets MIPS instead of x86.

    Other than that don't you think we can do without it as we already have the essentials bits (TRF,TRV,TRS) ? I mean regarding culling and so on, we don't necessarily need to understand the VEW.

    PS: beside this failed attempt, I wrote a WAD unpacker to see whether the files in the PC version were identical to the PSX one and they are. So whatever progress you'll make through the PC version will apply to the PSX game data.

  6. #6
    Join Date
    Apr 2015
    Location
    France, Paris
    Timezone
    GMT + 1
    Posts
    310

    Default

    Some (not so good) news about the VEW format

    I've disassembled the code with IDA and CodeExplorer and here's where a VEW is parsed,

    Code:
    struct s0 {
        signed char[6] pad6;
        int16_t f6;
    };
    
    /* zFOPEN? */
    int32_t zFOPEN_(int32_t a1, int32_t a2, int32_t a3);
    
    int32_t ga2afa4 = -1;
    
    struct s1 {
        signed char[8] pad8;
        int32_t f8;
        signed char[8] pad20;
        void* f20;
        int32_t f24;
    };
    
    struct s1* ga25644 = (struct s1*)0xffffffff;
    
    void zFATAL_ERROR(int32_t a1, ...);
    
    void sub_411A1E(struct s0* a1, int32_t a2, signed char a3);
    
    void sub_401980(int32_t a1, signed char a2);
    
    void zTRACK_VEW(struct s0* a1) {
        uint48_t v1;
        uint56_t v2;
        int32_t edi3;
        int32_t eax4;
        int32_t eip5;
        signed char v6;
        int16_t v7;
        int32_t edx8;
        int32_t eax9;
        signed char v10;
        void* v11;
        int32_t v12;
        int16_t v13;
        int16_t v14;
        int32_t v15;
        int32_t v16;
        int32_t v17;
        int32_t v18;
        int32_t eax19;
    
        *(struct s0**)&v1 = a1;
        *(int32_t*)&v2 = *(int32_t*)((int32_t)*(struct s0**)&v1 + (*(struct s0**)&v1)->f6 * 4 + 8);
        eax4 = zFOPEN_(ga2afa4, *(int32_t*)&v2 + 32, edi3);
        ga25644->f24 = eax4;
        if (ga25644->f24 == 0) {
            zFATAL_ERROR("Track.c:InitViewList: Failed to allocate memory for view list", "Track.c:InitViewList: Failed to allocate memory for view list");
        }
        *(int32_t*)((int32_t)&v2 + 3) = eip5;
        sub_411A1E(*(struct s0**)&v1, ga25644->f24, v6);
        v7 = 0;
        while (*(int32_t*)&v2 - edx8 >> 1 > (int32_t)v7) {
            eax9 = (int32_t)v7;
            sub_401980(eax9 + eax9 + ga25644->f24, v10);
            v7 = (int16_t)(v7 + 1);
        }
        v11 = ga25644->f20;
        v12 = 0;
        v13 = 0;
        while (ga25644->f8 > (int32_t)v13) {
            v14 = 0;
            while ((int32_t)v14 < 3) {
                *(int32_t*)((int32_t)v11 + v14 * 4 + 36) = ga25644->f24 + v12 * 2;
                v15 = v12 + *(int16_t*)((int32_t)v11 + v14 * 2 + 96);
                *(int32_t*)((int32_t)v11 + v14 * 4 + 48) = ga25644->f24 + v15 * 2;
                v16 = v15 + *(int16_t*)((int32_t)v11 + v14 * 2 + 0x66);
                *(int32_t*)((int32_t)v11 + v14 * 4 + 60) = ga25644->f24 + v16 * 2;
                v17 = v16 + *(int16_t*)((int32_t)v11 + v14 * 2 + 0x6c);
                *(int32_t*)((int32_t)v11 + v14 * 4 + 72) = ga25644->f24 + v17 * 2;
                v18 = v17 + *(int16_t*)((int32_t)v11 + v14 * 2 + 0x72);
                *(int32_t*)((int32_t)v11 + v14 * 4 + 84) = ga25644->f24 + v18 * 2;
                v12 = v18 + *(int16_t*)((int32_t)v11 + v14 * 2 + 0x78);
                v14 = (int16_t)(v14 + 1);
            }
            v11 = (void*)((int32_t)v11 + 0x9c);
            v13 = (int16_t)(v13 + 1);
        }
        eax19 = v12;
        if (eax19 + eax19 != *(int32_t*)&v2) {
            zFATAL_ERROR("track.c::InitViewList():problem with view list", "track.c::InitViewList():problem with view list");
        }
        return;
    }
    It does really strange things, reads an offset then somewhere else in the file and so on, doesn't make much sense to me. Also the code does check for invalid data and will fail it finds some. Seems to emit a 16 bytes structs (s1) by reading 5 items ... Note that there also VEWs not only for tracks but also for objects. Could this be the path points of the camera when it is rendered ???

    I've been able to run WOXL with the latest dgVoodoo2 in Win8.1 However it will crash as soon as Alt-Tab is invoked so debugging is impossible... the other good news is that it runs fine in the latest VirtualBox though it is way too fast. One should try remote debugging that VM with IDA.

    The content of a VEW seems to be unsigned integers (indices ?), but for what ?

    I've also tried to figure out the texturing issue in Gare d'Europa but all my tests failed ... there is clearly something that is being done we're not aware of, maybe the TTF is misleading after all. There also the TRACK.TEX file which I've tried to grab things from, but there are outstanding values beyond texture count. Basically on the track there are 0xD textures, most of the bytes are in this range but rendering doesn't get any better. Also tried to use the unknown bytes from each face...

    EDIT: also, it seems that files inside TRACK.WAD are being used for WOXL, so maybe other files are misleading

    EDIT: I've debugged WOXL through VirtualBox, and it works quite well though an exception happens and you must pass it to the game and it will continue.

    And that's what I said, this thing reads somewhere with an offset, and write somewhere else with an offset, I couldn't understand any of what it was doing/writing
    Last edited by aybe; 15th May 2015 at 02:08 AM.

  7. #7
    Join Date
    Mar 2015
    Posts
    56

    Default

    @aybe :

    about WOXL gare d'europa and Spliskinanke texture issue : this has been fixed since a long time, see https://github.com/phoboslab/wipeout/pull/16

    about your VEW disassembly : thanks for this, this is quite helpful. Anyway i'm doing some good progress with understanding VEW files, something might come in days to come. Stay tuned !

  8. #8
    Join Date
    Apr 2015
    Location
    France, Paris
    Timezone
    GMT + 1
    Posts
    310

    Default

    I should have read better your commits, thanks, going to give it a try right now

    I'm pretty close to get my whole thing as featured as the WebGL experiment, got track/scene/texturing up, need to merge all this and I guess I'll be done

    It was not as straight-forward as I thought (especially implementing your booster animations), but it ended up well, basically I wrote a scene along meshes/material just like THREE does; it is clean IMO and both things are independent. Then I will use BEPU for the physics as it looks great and is the only viable/recent option BTW.

    Once I'm happy with the results, I will look at the next 2 things, the ability of picking scene elements and get the real data from game files so hopefully RE-ing will progress. My first target would be to get these billboards animations working.



    For IDA, I'll go and have another look, basically I ran through all flavors of Wipeout : 1/XL, demo/retail. There are two things of interest I've seen and will try to get and share : there are 4 variable names for the TRS in the strings, the length and the count of the VEW file. Hopefully these will help ! Interestingly, the DOS code produced is more pleasant to read but it cannot be debugged ...

    Also I've been trying to get the game running and made good progress, beside running it in VirtualBox : I've got the 2 demos running every time in Win8.1 without any funny stuff, thanks to the latest dgVoodoo. There are 2 issues now, game runs too fast and if you Alt-Tab it will crash. Other than that it works really well see @ Running Wipeout 2097/XL in Windows 8.1

    Next objective is to get the retail games running as they are behaving different, working on it



    Can't wait to see progress you're talking about !



    EDIT: just got retail ver working (see thread above for instructions), it does not crash anymore when using Alt-Tab, the mouse is hidden though ... we might eventually get debugging running on the same OS
    Last edited by aybe; 16th May 2015 at 09:14 AM.

  9. #9
    Join Date
    Apr 2015
    Location
    France, Paris
    Timezone
    GMT + 1
    Posts
    310

    Default



    EDIT 2

    I've been debugging a bit

    The instructions that I've put for running it in Win8.1 are pretty good as I've debugged locally

    Not sure but the exception that sometimes occur may come from dgVoodoo (unimplemented opcode), it occurs less often whenever I navigate with the gamepad...

    Anyway, I've got a log being output from the game the whenever it loads a track :

    Code:
    CDMusicStop
    CDMusicStop
    -Loop number-  3
    Mem load       35
    Total phys     -1
    Avail phys     -1
    Total pagefile -1
    Avail pagefile -1
    Total virtual  2147352576
    Avail virtual  1884729344
    d:\Wipeout 2097\track08\track.wad
    Track load start
    d:\Wipeout 2097\track08\tracktex.cmp
    attempting malloc
    Track load finish
    searching for pit lane
    creating new instance of light beam 
    trying to alloc mem for particles
    init particle table
    init particle done
    doing init explode
    init explode done
    terminated controlled sounds
    start controlled sound
    Playing track = 10
    So it seems like all we'll ever need to use is in the WAD (basically FYI I've checked a WAD content, it is identical to other files; maybe all these files are simply misleading after all...)

    Here's the content of one, haven't checked them but I'd guess they're all similar ...

    Code:
    library.cmp   
    library.ttf   
    scene.cmp     
    scene.prm     
    sky.cmp       
    sky.prm       
    track.trf     
    track.trs     
    track.trv     
    track.vew
    The format of the WAD is really simple if you'd want to base yourself on it in the experiment :

    Roughly, the name of each item is variable, you have to adjust accordingly, then the next 2 values should be packed/unpacked length -> I assert these are equals; finally extract each item.

    Code:
            private static void Unpack(Options options)
            {
                using (var stream = File.OpenRead(options.InputFile))
                using (var reader = new BinaryReader(stream))
                {
                    // read header
                    var itemCount = reader.ReadUInt16LE();
                    var items = new List<Tuple<string, uint, uint>>();
                    for (var i = 0; i < itemCount; i++)
                    {
                        var name = reader.ReadStringAsciiNullTerminated();
                        stream.Position += 16 - name.Length - 1;
                        var length1 = reader.ReadUInt32LE();
                        var length2 = reader.ReadUInt32LE();
                        stream.Position += 1;
                        items.Add(new Tuple<string, uint, uint>(name, length1, length2));
                    }
    
    
                    // check data
                    var all = items.All(s => s.Item3 == s.Item2);
                    if (!all)
                    {
                        throw new InvalidDataException("Compressed items not supported");
                    }
    
                    var sum = items.Sum(s => s.Item2);
                    if (sum > stream.Length - stream.Position)
                    {
                        throw new InvalidDataException("Not enough bytes left");
                    }
    
                    // extract
                    var outputDirectory = options.OutputDirectory;
                    if (string.IsNullOrEmpty(outputDirectory))
                    {
                        outputDirectory = Environment.CurrentDirectory;
                    }
    
                    var directoryInfo = new DirectoryInfo(outputDirectory);
                    if (!directoryInfo.Exists)
                    {
                        directoryInfo.Create();
                    }
    
                    foreach (var item in items)
                    {
                        var itemName = item.Item1;
                        var itemLength = item.Item2;
                        var itemBytes = reader.ReadBytes((int) itemLength);
                        var itemPath = Path.Combine(directoryInfo.FullName, itemName);
                        File.WriteAllBytes(itemPath, itemBytes);
                    }
                }
            }
    If by some magic we could lower the frame rate and play it ... actually I'm as impressed as I was 15 years ago when I used to play it with a 3dfx, it is so smooth and 480p compared to the PSX

  10. #10
    Join Date
    Mar 2015
    Posts
    56

    Default

    Hi there,

    I had a look at VEW files and get it working : see my commit https://github.com/tigrouind/wipeout...vew/wipeout.js

    Thanks for aybe disassembly, this was VERY useful to get it working.

    Notes :
    - This is not very efficient way to perform culling on today GPUs: the whole track (without objects around) is about 3000-5000 faces (triangles) which is nothing. This is maybe even slower than no culling at all (because we send new polygons to GPU every frame). A better way (if needed, eg : on low end hardware) would be to cut the track in big pieces, and the perform some frustum culling on it. This is something already implement by threejs (all objects are frustum culled by default), only needed is to cut the track in separate objects.
    - VEW files are only used for the track culling , not for other objects around (eg :train, bridge, trees, ...). For these ones another info (inside PRM file) is probably used. I suspect PRM file to contains some sort of bounding spheres/ boxes , organised as a tree (sort of bounding volume hierarchy).

    About VEW files (how it works) :

    For each track section, there 15 lists defined.

    north south west east all
    near x x x x x
    medium x x x x x
    far x x x x x

    - north, south, east, west lists : used for default game play, when camera is behind player. depending how player is oriented with the track, one list is selected. default is north list, when riding reverse, south is used.
    - all list : a combination of the 4 lists above. this is used for replays, for cameras around the track

    near, medium, far : used for mip mapping. for far sections, low quality texture are used.

    Each list contains visible track segments (so for a given section , the renderer only renders visible track segments).

    One example : player is in section 47, riding forward (not reverse)
    for section 47, in north near list, we have two segments : [400,4] [420,2]

    So we will render sections 400,401,402,403,420,421
    Last edited by tigrou; 19th May 2015 at 06:59 PM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •