Wing Commander Toolbox

Yeah the file extensions are the same, but the file format is completely different. I have started looking into them and I am planning on adding extraction support for these in the near future.
 
I was not planning on making a release just yet, but I have noticed some interest in the container formats for console versions of WC games. Since I have working code to handle some of these files, I figured I would make it available even though I am in the middle of restructuring the toolbox and not yet done.

I have been making changes to improve large file handling, include support for the console versions of the games, and reduce the number of executables. I have also refactored some of the code and updated to .NET Framework version 4.6. See the first post for the download.


Improving large file handling was becoming more important. In earlier games, all the files are small by today's standards and I would simply load the file content into memory and then process it. But in later games, the file sizes have considerably increased and I now only load the necessary parts. As an example of the difference, in the previous version of the toolbox, the WC3 intro movie extraction would peek at around 220 MB of memory usage, now it's down to about 10 MB.

This release introduces support for console versions of the games. At the moment, it is mostly limited to container files, but hopefully support will grow. Before the toolbox can process these files, they need to be extracted from the disc images; there are various tools you can find online that do this.

I have also removed game specific console applications and consolidated all functionality into a single application. As a result, the command line interface has slightly changed and now all commands are of the form: "Game : Platform : Command" (without quotes or spaces), for example:

WCToolsCmd WC3:PC:ExtractTree CD1MOVIE.TRE -HashMappingFile:CD1MOVIE.TRE.txt​

You can consult the README file for a complete listing of all the games, platforms, and file types supported; or invoke a Help command:

WCToolsCmd WC3:PC:Help​

Code: 
Type WCToolsCmd WC3:PC:Help <command> for command line description.

Commands:

WC3:PC:ExtractAudio        Extracts the audio from a profile sound (IFF) file.
WC3:PC:ExtractFrame        Extracts all the frames from a movie (MVE) file.
WC3:PC:ExtractMovie        Extracts all the videos from a movie (MVE) file.
WC3:PC:ExtractPack         Extracts all the packets from a pack (PAK) file.
WC3:PC:ExtractPalette      Extracts an image from a palette (PAL) file.
WC3:PC:ExtractShape        Extracts all the images from a shape (SHP) file.
WC3:PC:ExtractSubtitle     Extracts all the subtitles from a movie (MVE) file.
WC3:PC:ExtractTree         Extracts all the files from a tree (TRE) archive.
WC3:PC:MakePack            Creates a pack (PAK) file from imported files.
WC3:PC:UnmakePack          Reverses a pack (PAK) file into extracted files.


WCToolsCmd WC3:3DO:Help​

Code: 
Type WCToolsCmd WC3:3DO:Help <command> for command line description.

Commands:

WC3:3DO:ExtractBundle      Extracts all the files from a bundle (BUN) archive.
WC3:3DO:ExtractStream      Extracts all the files from a stream (STR) archive.


WCToolsCmd WC3:3DO:Help ExtractStream​

Code: 
Extracts all the files from a stream (STR) archive.

WCToolsCmd WC3:3DO:ExtractStream source [destination] [-HashMappingFile:text]

source       Specifies the stream file to extract from.
destination  Specifies the directory for the extracted files.

-HashMappingFile  Specifies the text file to use for resolving file name hashes.
 
Last edited:
For WC3 on the PC, you can extract the audio from a profile file:

WCToolsCmd WC3:PC:ExtractAudio HOBBES.IFF​

Code: 
HOBBES.IFF-Radio0000.WAV
HOBBES.IFF-Radio0001.WAV
HOBBES.IFF-Radio0002.WAV
HOBBES.IFF-Radio0003.WAV
HOBBES.IFF-Radio0004.WAV
...


All the larger files probably have audio data in them.
 
Last edited:
For WC3 on the PlayStation, you can extract the files from a library (LIB) archive:

WCToolsCmd WC3:PSX:ExtractLibrary INFLIT1.LIB​

Code: 
a1spfei1.tai
a1spfei2.tai
a2spfrl1.tai
a2spfrl2.tai
a3spfrl1.tai
...


You can also convert the audio into a WAV file:

WCToolsCmd WC3:PSX:ExtractAudio *.TAI -SampleRate:11025​

You may need to specify the sampling rate of the audio since there are no indications in the source file. The default is set to 11025, which is used for inflight sounds; use 22050 for gameflow sounds.
 
Last edited:
For WC3 on the 3DO, you can extract the files from bundle (BUN) and stream (STR) archives:

WCToolsCmd WC3:3DO:ExtractBundle commov.bun -HashMappingFile:commov.bun.txt​

Code: 
wc3\movies\sc_200.mov
wc3\movies\sc_201.mov
wc3\movies\sc_202.mov
wc3\movies\sc_203.mov
wc3\movies\sc_205.mov
...


WCToolsCmd WC3:3DO:ExtractStream mission.STR -HashMappingFile:mission.STR.txt​

Code: 
wc3\objects\3DOLASER.IFF
wc3\objects\3DOMASS.IFF
wc3\objects\3DONEUT.IFF
wc3\objects\ARROW.IFF
wc3\objects\ARROW.TXM
wc3\objects\ASTFIGHT.IFF
wc3\objects\ASTFIGHT.TXM
...


You should reference the appropriate hash mapping text file in the "external\wc3-3do-hashes" directory, this will provide for file name resolution.
 
Does this mean we can read the WC3 3DO meshes? I'm really interested in studying them more closely (and getting them into a 3D viewer for the site!)
 
I don't know if the formats between the PC and 3DO are compatible; although they do seem similar. I am not familiar with the 3D meshes for the PC version so perhaps someone with more knowledge can answer.
 
UnnamedCharacter said:
I don't know if the formats between the PC and 3DO are compatible; although they do seem similar. I am not familiar with the 3D meshes for the PC version so perhaps someone with more knowledge can answer.
Click to expand...

I will take a look,I’m assuming atleast the endian would be different?

You mentioned 3DO str files, what about psx str files? Since the wc4 comms are in one that’d be huge
 
Last edited:
Bandit LOAF said:
Does this mean we can read the WC3 3DO meshes? I'm really interested in studying them more closely (and getting them into a 3D viewer for the site!)
Click to expand...

I had a quick look at them, everything except the textures seems to be the same. Whereas on PC the texture data is baked into the object iff here there is a seperate .TXM file. I could probably export the untextured models for you if that is useful, I don't know when I could get around to investigating the .TXM's.

Sadly the same is not true for the missions, I can't see any similarities between them and and the PSX/ PC versions so little hope of any integration there.
 
Pedro said:
You mentioned 3DO str files, what about psx str files?
Click to expand...

Same extension, but different file format. I certainly would like to be able to extract the files from all these container archives and I will take a closer look at it, but this is another case of having no file names; all the entries are using hashes.
 
Pedro said:
I had a quick look at them, everything except the textures seems to be the same. Whereas on PC the texture data is baked into the object iff here there is a seperate .TXM file. I could probably export the untextured models for you if that is useful, I don't know when I could get around to investigating the .TXM's.

Sadly the same is not true for the missions, I can't see any similarities between them and and the PSX/ PC versions so little hope of any integration there.
Click to expand...
The WC3 (and 4) IFFs all have the textures in a TXMS block with individual textures in TXMP chunks which all start with the actual texture name. The 3DO version seems to have simplified the object list significantly with a lot less objects and textures and everything just seems more basic at a glance. There doesn't seem to be separate debris objects and so on. Most objects on the 3DO only seem to have a single texture per object whereas there's often multiple textures per PC version of the same object. The 3DO version does seem to have at least 2 different LODs baked into the object mess file IFF.

Most of the 3DO textures have '3DOT' at the start of the file and only have what looks like one big image chunk. It doesn't look at a glance like it's the same as the CEL format but I honestly don't really know what I'm doing. The textures definitely don't seem interchangeable though and don't even have the same naming system as the PC version.
 
AD said:
The WC3 (and 4) IFFs all have the textures in a TXMS block with individual textures in TXMP chunks which all start with the actual texture name. The 3DO version seems to have simplified the object list significantly with a lot less objects and textures and everything just seems more basic at a glance. There doesn't seem to be separate debris objects and so on. Most objects on the 3DO only seem to have a single texture per object whereas there's often multiple textures per PC version of the same object. The 3DO version does seem to have at least 2 different LODs baked into the object mess file IFF.

Most of the 3DO textures have '3DOT' at the start of the file and only have what looks like one big image chunk. It doesn't look at a glance like it's the same as the CEL format but I honestly don't really know what I'm doing. The textures definitely don't seem interchangeable though and don't even have the same naming system as the PC version.
Click to expand...

Fortunately none of that will have any impact loading them (I already passed them through the converter and they didn’t have issues aside from the textures). The actual model data is identical except for the fact that I’m seeing models which are made up of quads only and no triangles (but again no assumptions there).

A single texture makes life simpler as for the pc models I adjusted the UVs and merged all of the textures into one. As for the format, one of the boasting points of the 3do was hardware texture mapping support; there’s a good chance it’s a standard format and the emulation scene is the place to look (I couldn’t find much on a quick google sadly).
 
UnnamedCharacter,

Are the file formats for WC1/2 published/described anywhere? I see that your WC Toolbox includes a DLL that can be reused, but I'm trying to interface with the data files using C on some non-Windows platforms.

Edit: I just found the XML file you provided in "BeBe.WCToolbox.Engines.xml". I'll go RTFM for a bit but it looks like the information I was seeking. Thanks!
 
Last edited:
I'm still woefully out of my depth but: the 3DO version has a built in texture viewer and that seems to suggest every ship has a bunch of smaller textures rather than one image. In case that helps anything! (It's accessible from the cheat menu.)
 
@Pedro

No, not yet. But there is plenty of information in the data to indicate multiple textures. Using the ARROW for example, and looking at the pretty-printed data bellow, it indicates a texture count of 60 and the texture index values in the mappings run from 0 to 59. Also, looking at the mapping coordinates, you will see the same values over-and-over again but referencing a different texture index, and the values are too small for one large texture.

Code: 
FORM.REAL
    FORM.APPR
        FORM.POLY
            FORM.TXMS
                INFO
                    StaticTextureCount: 60
                    AnimatedTextureCount:  0
            FORM.QUAD
                MAPS
                    Face  Texture  Mappings                       
                    ----  -------  -------------------------------
                       0        0    0,  0  15,  0  15, 15   0, 15
                       1        1    0,  0  15,  0  15, 15   0, 15
                       2        2    0,  0  15,  0  15, 15   0, 15
                       3        3    0,  0  15,  0  15, 15   0, 15
                       4        4    0,  0  31,  0  31, 31   0, 31
                       5        5    0,  0  15,  0  15, 15   0, 15
                       6        6    0,  0  15,  0  15, 15   0, 15
                       7        7    0,  0  31,  0  31, 31   0, 31
                       8        8    0,  0  15,  0  15, 15   0, 15
                       9        9    0,  0  15,  0  15, 15   0, 15
                      10       10    0,  0  15,  0  15, 15   0, 15
                     ...
                     121       55    0,  0  15,  0  15, 31   0, 31
                     122       56    0, 31   0,  0  15,  0  15, 31
                     123       57    0,  0  15,  0  15, 31   0, 31
                     124       58    0,  0  15,  0  15, 31   0, 31
                     125       59    0,  0  15,  0  15, 31   0, 31
                     126       53    0,  0  15,  0  15, 31   0, 31
                     127       50    0, 31   0,  0  15,  0  15, 31
                     128       51    0,  0   0, 15  15, 15  15,  0
                     129       52    0,  0   0, 31  15, 31  15,  0
                     130       51    0,  0   0, 15  15, 15  15,  0
 
Pedro said:
This sounds interesting, what are you doing?
Click to expand...

My goal is to re-implement the WC engine in C89 with shims for all the platform specific stuff so that I can easily port it to a variety of platforms like my Sun Ultra 2 and Blade 1000, an SGI Octane, and a homemade ARM-based platform that runs NuttX. I've also got a side project going for a homemade CPU design that I intend to target with a simple C compiler and I'm hoping to eventually get it running there as well, performance permitting.

So basically, my goal is a clean, open source, easy-to-port, re-implementation of the WC engine.

It's a project that's been on my mind for years, but I recently ran across my copy of "Secrets of the Wing Commander Universe" while cleaning my office, providing the motivation to finally put the plan into action.

As a child I dreamt up all sorts of add-on missions/campaigns for the game, so if all goes well I would like to implement some of those, but that's a project for later, after I have a better feel for how the internals work.
 
You must log in or register to reply here.
Back
Top