About the .PBP file format

[aybe]

UPDATE 4

Take a look at notes in update 3 then,

https://github.com/aybe/WXX-Rebirth/releases/tag/PBPv2

• TIMs detection
• bug fixes

At this point you are getting pretty much the same picture as in the attached Excel file, what is not an image is game data; things are clearer.

There won't be anything new until quite some time now, things needs to be reverse-engineered.

---

Update 3

• it has been simplified, accepts one file and an output directory
• all bugs are fixed AFAIK
• better interface/reporting with colors
• shipped with an Unpack.bat to make your life easier

The older version was trying to be smart and failed somehow, now there are two tools that each do a very specific task and both do work well. I've ditched the multiple input files feature as this was limiting the output options and was cumbersome, with the current version you can write your own batch file and keep control of things as you wish.

Please disregard all previous versions.

There will be a last update (or unless bugs are found), the detection of TIM files; for the time being they are easy to spot : every file that has two sequence of numbers (e.g. TRACKM0A.PBP.00011.00000) is a TIM. You can easily group them within Explorer by sorting by modified date :



(in this example I've already renamed them to TIMs but they would pop in the exact same order)

grab it here

PS1: to preview TIM files in Explorer, grab SageThumbs
PS2: your AV might suspect these EXEs, there's nothing I can do about that and you have to allow them, also you can always check the code with JustDecompile

Waiting for your feedback now !

---

---

---

UPDATE 2

Two new tools (UnpackCMPEx and UnpackPBPEx)

1. searches for CMP files and unpacks them
2. unpacks a PBP (simplified version of the first tool, better suited for batch operations)

So far, it successfully unpacks a PBP and the CMPs inside it, no false-positive AFAIK



See USAGE.TXT for an example !

PS1: TIMs are not yet renamed to .TIM as explained in USAGE.TXT, working on another tool for that.
PS2: you can use SageThumbs to get a preview of TIMs within Explorer

---

UPDATE 1

Here is (finally) the first version of the unpacker : https://github.com/aybe/WXX-Rebirth/releases

I guess it works properly since it extracted 10990 items out of 197 files (every PBP file from every version: WO3E, WO3J, WO3SE, WO3U, WO3UDEMO)

NOTES:

• full instructions on how to use it are on the README.TXT
• archives of TIM images (aka CMPs) are not decompressed by the utility -> the UnpackCMP utility is on its way for this.
• in the web link above there is also an Excel file, it contains a couple of informations already figured from the unique PBP file in the Wipeout 3 US Demo Disc (highly recommended since most files will harbor this structure)
• the 'unofficial' preferred version of Wipeout 3 to study is the SE version, it is the most recent, the most featured and surprisingly there are less files

Let me know if you encounter any problem !

---

Never say never, all things have a beginning but if we don't start we have zero chance to succeed

I took a quick look at the format and here are my findings:





Header (in UINT32 units)

• at offset 80 it is the number of entries in the container
• at offset 88 starts the 12 bytes struct of each entry
• 1st value is absolute offset in file
• 2nd value is unknown
• 3rd is the length of the entry
• i've checked on multiple PBP and they fit perfectly with their length

We can already get the individual parts of these PBPs and try to get to next step

IMO these files are certainly compressed but not necessarily encrypted, there used to be some entropy detection software that could help but can't remember the name actually ...

EDIT

Just wrote a tiny unpacker, it swallowed every PBP out of WO3SE from \WIPEOUTC and \WIPEOUT3; there is always between 58 and 60 files in each PBP. With time and patience we'll decipher them

EDIT 2

I don't know where he got these infos from but they're hintful.

EDIT 3

Some progress !

There are VAG files in these containers: (ship thruster sound)



There are also a bunch of files which I suspect are textures since there are many entries in their header have a 0x40 value (64 pixels), not .TIM files though. We should prioritize in searching for files from the official formats in the SDK, if not then it could simply be a form of RLE encoding.

There are also files that looks like polygon data but it's a bit premature IMO :



The left-most pixel rarely changes and could indicate the polygon type as it can be seen in no$psx debugger.


Also many of these files are duplicate, basing myself on studying a group of files from the link posted in edit 2.

2 be continued

- - - Updated - - -

EDIT 5

Houston, we have TIMs from PBPs !






- - - Updated - - -

EDIT 6



- - - Updated - - -

EDIT 7

Houston, we're about to crash on vertex data !





basically, files with no headers are simply LZ77 streams

EDIT 8

• Large 2 groups on bottom : ship/team data and I guess textures
• Bigger files seems to be vertex data as they have some kind of stride

[aybe]

Last update for today

The content of the 8 last files :

[Xpand]

Wow, great work, man!

Just out of curiosity, what programs did you use? I only have an hex editor and gcc... xD Makes things so much tougher.

[tigrou]

Yep, great work. If you keep going that way we will have a Wipeout3 model/track viewer soon!

[aybe]

Wow, great work, man!

Just out of curiosity, what programs did you use? I only have an hex editor and gcc... xD Makes things so much tougher.

Only an hex editor but it's quite featured as you can see above (coloring, etc) and the infos from wipeout.js; the rest I wrote it in C#

- - - Updated - - -

Yep, great work. If you keep going that way we will have a Wipeout3 model/track viewer soon!

this is going to take time, I think I'll post my results somewhere so people can try by themselves.

- - - Updated - - -

To the admins:

Can we post a decompressor of these files at the forum, or not ?

[tigrou]

Only an hex editor but it's quite featured as you can see above (coloring, etc) and the infos from wipeout.js; the rest I wrote it in C#
To the admins:
Can we post a decompressor of these files at the forum, or not ?

What about creating a github repository ?

[aybe]

Yes, probably within the next days, there are a couple of things I would like to sort before:

- references because I've just been coding against Monogame and WPF and those utilities do drag lots of unneeded DLLs
- get my renderer on par with Dominic's one (textured objects + Wipeout 2), shouldn't be too hard.
- clean up the thing a bit, write a README etc ...
(those are imminent, easy tasks)

I'm done with the PBP/CMP/LZ77 unpackers, still trying to figure out whether I can properly detect LZ77 streams so one wouldn't have to bother to manually check whether it is encoded or not (to get a 1-click PBP unpack)

other things of interests (later)
- add viewers to the GUI app. basically you select a file and you can see 3d objects etc ...
- implement a viewer with triangle selection, we'd instantly know what object is what data hence help us understand the thing better
- begin the coding related to the game

regarding previous talks it seems that the way to go would be to have another name than Wipeout, been thinking about WXX-Rebirth (like http://www.dxx-rebirth.com/), what do you think, sounds cool isn't it ?

- - - Updated - - -

Just 'secured' the thing : https://github.com/aybe/WXX-Rebirth

I'll post the WO3 unpackers there ASAP ...