Originator

[AllTinker]

Here's version 0.2.04!

This update adds supports for a few more Wing Commander Prophecy formats; MGI streaming audio for music, bitmap images (should cover everything including textures), and I also added a handler for some of the text content. No models yet, but soon!



It also adds support for streaming/extracting Red Book audio from plain ISO images, i.e. music from Wing Commander 1/2 and Space Rogue for FM Towns and Wing Commander for SEGA CD.

I've also improved the hex viewer with some extra tools for searching and data inspection.



Here's a few more shots of content viewable since the last couple of updates:

 

[AllTinker]

Version 0.2.05 is mostly a bugfix update, but it does include one new feature - VGA aspect ratio compensation for the image viewer. This is currently done in a pretty basic way, but should look fine at zoom values around 2-3 or higher. Most images will auto-sense and turn this on as appropriate, with WC3/4 being the main exception for now.

Several issues have been fixed for existing RealSpace models and their textures, which were mirrored incorrectly and were found to render differently to original WC4 - which is hopefully not the case anymore.

 

[Rylex]

I quickly wrote some code to extract the complete set of Privateer/RF jump points, at least as far as I understand the data at this point. I can't 100% guarantee there isn't some other mechanism which adds jump points at runtime, but I haven't found any evidence that there is such a thing either...

...

Not sure if that helps at all.

Hi AllTinker,

At first I'd like to thank you for your help!
I've already analyzed the sector data with a normal HEX viewer up and down and so far your extracted data seems correct. However I've to add something for your tool. With SECTORS.IFF -> Sector Number -> SCTR -> PART -> JUMP

The jump target sector is missing on the first jump point in your tool. Take a look at my screenshot. It shows jump point 2 in Perry that originally leads to "2F". I changed that to "45" to go to the Test Bed. See my forum posts here or the news.

But you are missing this target location for the first Jump point, because it starts three positions before the word "JUMPTYPEJUMP".
I hope you understand what I mean with the help of my image. So perhaps you can update your tool and add the target jump location in this part?

I know that there is definitely at least ONE another hidden jump point in the game! It seems not to occur in the sector data. But I've seen it and I flew through it but I don't know which sector it was. It could have been Rikel, because I was really mad that day (it was more than 10 years ago) because I thought: "what a sh#t... just a shortcut jump from X to Perry". I didn't wrote the position down and where it ended in Perry. I've searched for months up to now to find it again, but no luck.
It was not a dream (how some friends of mine state) nor am I crazy. It really exists out there!

Best wishes

Rylex

 

[AllTinker]

It's definitely starting on the right offset, there's no missing byte there. If you find the start of the JUMP chunk in hex you'll see that the IFF chunk begins with JUMP (the IFF chunk tag name), then the standard 4 bytes (big endian) for the chunk size, then two bytes, then JUMPTYPEJUMP.

Each jump is 46 bytes, where the first two bytes I assume is an index (i.e. 16-bit little-endian, or possibly an index and then a zero byte), then JUMPTYPEJUMP - with the final byte of the jump giving the destination. So rather than changing the first byte of a jump, you were actually changing the last byte. It's much harder to tell in a raw hex editor, where you're looking at everything at once - which is part of the reason I made Originator.

EDIT: Also as I said it's quite possible that the game has jump points hard-coded in or otherwise generated by the executable code. It may have been a glitch... Stranger things have happened; probably in Privateer, definitely in other games. Without completely reverse engineering the executable or finding a reference elsewhere in the data it's difficult to say definitively either which way. If I find anything I'll certainly let everyone know!

 

[Rylex]

It's definitely starting on the right offset, there's no missing byte there. If you find the start of the JUMP chunk in hex you'll see that the IFF chunk begins with JUMP (the IFF chunk tag name), then the standard 4 bytes (big endian) for the chunk size, then two bytes, then JUMPTYPEJUMP.

Each jump is 46 bytes, where the first two bytes I assume is an index (i.e. 16-bit little-endian, or possibly an index and then a zero byte), then JUMPTYPEJUMP - with the final byte of the jump giving the destination. So rather than changing the first byte of a jump, you were actually changing the last byte. It's much harder to tell in a raw hex editor, where you're looking at everything at once - which is part of the reason I made Originator.

EDIT: Also as I said it's quite possible that the game has jump points hard-coded in or otherwise generated by the executable code. It may have been a glitch... Stranger things have happened; probably in Privateer, definitely in other games. Without completely reverse engineering the executable or finding a reference elsewhere in the data it's difficult to say definitively either which way. If I find anything I'll certainly let everyone know!

Shame on me, you're right with the jump destination. Thanks for pointing that out... it seems that I'm more overworked, than I thought.

You probably already knew this then too. Concerning the coordinates, radius and ids of the spheres for the navigation and jump points.
Here the example of the hidden Eden jump point (it's in the SPHR section). This time, however, I'm sure that my calculation are right.

So if you ever have trouble with asteroids, then just set the radius to a low number and they are gone^^. This is good for testing, too.

Oh, and one more thing. I'm sure you've taken a look at he Perry system?
There is no official jump point from Perry to Rikel...

...so why is there a JMPRIKE entry there?
Could this somehow be the indication of a hidden jump point?

As I said, I know that I ended up in Perry back then. But I can't remember from where. Only that it wasn't that far away from the other hidden jump point. That could have been indeed Rikel.

 

[AllTinker]

...so why is there a JMPRIKE entry there?
Could this somehow be the indication of a hidden jump point?

Interesting!

I will definitely be looking into this; I'd like to put together a Privateer sector viewer and consolidate all this data, so I'll see what I learn once I get into it!

 

[Rylex]

Interesting!

I will definitely be looking into this; I'd like to put together a Privateer sector viewer and consolidate all this data, so I'll see what I learn once I get into it!

That sounds indeed very, very interesting! Please keep me updated on this - I can't wait to hear more about it!

 

[AllTinker]

In looking into terrain stuff for Strike Commander thru WC4, I've accidentally taken a step forward on that Pacific Strike mystery. I don't believe it's encrypted anymore - just compressed; mostly likely an LZ(W) variant, but that doesn't help narrow it down too much...

The ".ZIP" terrain files (nothing to do with the common ZIP format) are either compressed IFF files, or containers with at least one IFF file in them. I found one decompressor via QuickBMS which gives me "FORM", "TERA" and "ATRI" in amongst mostly-garbage, but it's more coincidental than anything and is clearly a long way from a correct decompression. The investigation continues...

If anyone else wants to take a look I've attached an example terrain file.

 

[AllTinker]

Version 0.2.06 - mostly improvements to the existing RealSpace model support, including:

• Support for TXMV (inbuilt shape) and TXMK (external PAK shapes) textures.
• Added TXM texture support for terrain, both standalone (WC4) and in PAKs (Strike Commander).
• Added support for sprites along with loading/viewing untextured models.
• Added a tool in the model viewer to select the current frame for animated textures.

I think all 3D models from Armada, WC3 and WC4 should be viewable/exportable - let me know if not! There are some objects (specifically the LASR and BEAM types) which aren't actually 3D models, just info for the renderer. These might be viewable eventually perhaps, but are very low priority.

I'm in the process of doing a bunch of work with the terrain data, which will be viewable in a future version. Currently WC4 heightmaps are exportable again (load the IFF file, select File->Export), but after the new terrain stuff goes in WC3 and Strike Commander should also work (and with colours/textures, hopefully objects).

 

[AllTinker]

Version 0.3.00 is a fairly broad update.

Support has been added for Prophecy/Secret Ops and Privateer 2 models. The former took longer than I thought - I forgot many models included polygonal faces, which required triangulation - the details of which are quite complicated and probably not that interesting.



Prophecy models are quite modular; eventually I may add a specific Ship Viewer, which combines the various models for viewing - also handling things like the WC3/4 hangars. A task for another day!

I've added a "Global Search" mode (under Tools -> Global Search), which allows searching for hex or text strings across all loaded game files. Double-clicking on a result will jump to the IFF or raw hex where the instance was found.



I noticed there was some confusion about what disc image formats could be used for games (e.g. ports) which expect images. Originator will now search for .iso or .bin extensions, in addition to .img. Most PC (and 3DO) disc images should be detected and loadable, regardless of extension. Some proprietary formats (e.g. .nrg) aren't supported; nor are Mac images. Support for Mac images might happen eventually, but probably not any time soon.

I've also fixed a couple of things which broke at some point; namely 3DO STR support and the OBJ export. The generated model normals also had some issues which I've now fixed.


(The scale for Prophecy models was wrong above, which I've since fixed.)

I made some progress on Pacific Strike - seems like it used the PKWARE Data Compression Library; the text could be decoded via the "Explode" algorithm which is now integrated. Unfortunately this hasn't helped directly with the audio and terrain formats, but it's another step forward.

I've started hooking up palette logic for Strike Commander, but it's a work in progress; perhaps halfway done.



Summary of major changes:

• Added support for Prophecy / Secret Ops models, including triangulation of polygonal meshes.
• Added support for Privateer 2 textures/models.
• Added "Global Search" mode, which allows searching for text/hex across all loaded game files.
• Disc images will be detected with .iso or .bin extensions, in addition to .img.
• Added support for Pacific Strike text content decompression.
• Fixed the solid backdrop scaling for VGA aspect mode.
• Fixed 3DO STR container support.
• Improvements to FLX container support.
• Better support for partial palettes (mostly for Strike Commander).
• Some palette hookups for Strike Commander (work in progress).
• Fixed texture coordinates for OBJ export.
• Improved generation of normals for 3D models.
• Added support for Privateer 2's XMIDI music.
• Fixed terrain elevation export for WC4; more general terrain support will happen later.
• Some UI improvements, e.g. showing what's selected in tree navigation panes.

 

[AllTinker]

Quick sneaky update (v0.3.01) - I realised I could hook up Bioforge model viewing with minimal effort. Animations aren't currently viewable.

 

[Knight26-77]

Is the Devastator really that tiny? Or was there a scaling issue?

 

[AllTinker]

I imagine you came from the front page; I did mention in the update (below the image) that there was an issue with the scaling which I've fixed. My fault for not taking the time to re-export them and update the image.

Having said that, the next release refines the scaling slightly further; matching the wingspan of the Dralthi IV between WC4 and Prophecy - the rear of the ships are roughly lined up so this is a more fair size comparison, but not exactly perfectly precise:

I still need to refine the Privateer 2 scaling...

The next release also improves the Privateer 2 palettes. It's now using the "true" base palette and the BRender look-up tables properly, it's a lot less washed-out - although I do need to verify/refine my gamma curve. Here's a proof-of-concept for the palette swaps for the different faction colour schemes:

 

[AllTinker]

Version 0.3.02 is mostly a bugfix and quality-of-life release, but does have a bunch of new/improved features.

• Fixed palettes for FM Towns Wing Commander Armada.
• Removed hard requirement for Proving Grounds for PC Armada.
• Privateer 2 palette improvements; proper base palette and BRender LUT usage.
• Finished an initial pass on picking up the remaining Origin FX containers for Wing Commander 1/2.
• More text support for Bioforge and Wing Commander 2 (PC & FM Towns versions).
• Fixed a bug where changing the game path wasn't properly applying without restarting.
• Added the ability to unload games and manually-loaded files.
• Changing the path for a game optionally causes an automatic reload.
• Added the ability to see/configure the names for disc/other image files for games which use them.
• Added support for the Prophecy 3dfx Test.
• Fixed Bioforge model axis orientation.
• Added basic translation (lateral camera movement) to the model viewer - hold middle mouse.
• Overhauled manual loading of files quite extensively.
  • Files can be attached to a loaded game, improving viewing support.
  • Files can be loaded from any drive/directory.

The new camera movement makes examining models slightly easier; especially larger ones. I may change how the camera works in future...

Here's a quick example of the new loose file support; loading SM1.5 into WC1 (via the zip straight from wcpedia):

 

[AD]

Version 0.3.02 is mostly a bugfix and quality-of-life release, but does have a bunch of new/improved features.

• Fixed palettes for FM Towns Wing Commander Armada.
• Removed hard requirement for Proving Grounds for PC Armada.
• Privateer 2 palette improvements; proper base palette and BRender LUT usage.
• Finished an initial pass on picking up the remaining Origin FX containers for Wing Commander 1/2.
• More text support for Bioforge and Wing Commander 2 (PC & FM Towns versions).
• Fixed a bug where changing the game path wasn't properly applying without restarting.
• Added the ability to unload games and manually-loaded files.
• Changing the path for a game optionally causes an automatic reload.
• Added the ability to see/configure the names for disc/other image files for games which use them.
• Added support for the Prophecy 3dfx Test.
• Fixed Bioforge model axis orientation.
• Added basic translation (lateral camera movement) to the model viewer - hold middle mouse.
• Overhauled manual loading of files quite extensively.
  • Files can be attached to a loaded game, improving viewing support.
  • Files can be loaded from any drive/directory.

The new camera movement makes examining models slightly easier; especially larger ones. I may change how the camera works in future...

View attachment 13606

Here's a quick example of the new loose file support; loading SM1.5 into WC1 (via the zip straight from wcpedia):

View attachment 13607

Amazing! You're a rock star here. I'm blown away by how fast you're knocking out these revisions but maybe that's just years of stagnation showing through. This is such a boon to the community at large and I'm grateful for it. I can't wait to see what everyone gets up to with how much easier this makes fan projects in general!

 

[AllTinker]

Here's version 0.3.03:

• Fixed a couple of bugs in the loose file loading where it would fail to open containers/files.
• The file opener now remembers the last path, with a "Home" button to return to the Originator working directory.
• Added support for "inbuilt" external files; e.g. loading Kilrathi Saga saves from the Users directory.
• Added a savegame viewer (WC1 only currently; work-in-progress).

I've started working on some stuff I need for Confederation; in this case saved games for WC1. I plan to eventually add the ability to edit these within Originator (and those for other games) once I've finished researching/verifying the data.

 

[Bob McDob]

This is fantastic. I mentioned Privateer 2 more out of hope than expectation, but I never expected you to add it within weeks.

 

[AllTinker]

I'm chipping away at some longer-term features (and Confederation), but here's a new version (0.3.04) which is mostly focussed on Privateer 2.

I've added support for "BASE" images, which allowed me to complete the remaining palette hook-up logic. Hopefully everything is hooked up now, although I haven't checked exhaustively...

The major text formats used by the game are also now supported.



I've added support for the IFF-nested sample sets, which gives access to the majority of the remaining (non-movie) audio.

IFF shapes are also now (manually) exportable; along with a few miscellaneous bug fixes and such.

 

[AllTinker]

Oops, there were a couple of things I missed palette & text-wise upon further investigation; 0.3.04a fixes those.

 

[Bob McDob]

Beautiful! This will be a godsend for the wiki ― we have the booth images for instance but they're screenshots rather than asset rips and they're nowhere near as clean.