I've Created a WC4 HD Video Pack

[Rylex]

The quality of the intro is incredible! Really fantastic and a big step compared to the DVD version!
I mean... I own the original DVDs but I have to admit that especially the sound of the original DVDs is really bad compared to the "old" episodes that were available here on the CIC back in... (Golly!)... 2001 or 2002? I don't know why the publisher didn't take care of that sound problem. The sound is just too silent and not good quality. So I'm happy I still have the CIC video version here. 😎

 

[ODVS]

The quality of the intro is incredible! Really fantastic and a big step compared to the DVD version!

Thank you, @Rylex 🙂

The "CGI/Animation" upscale AI model is certainly a lot better at what it does than the film/video models - but it is working on far less visually complex material. Also worth considering is that because WCA is 4:3 and has no letterbox to remove, there's considerably more vertical resolution for the AI model to work with than in the 16:9 WC4 footage.

The main experimental part of this project was the use of DAIN to attempt to smooth out the animation quality and frame rate. Saturday Morning cartoons like this used to cut a lot of corners for budget/production time reasons. One of the methods is to not draw the full 24 frames per second on things like character/ship animations (where a whole new drawing is required for each frame). Usually only 12 frames per second are drawn.

Unfortunately, pan and zoom shots which were achieved literally by moving the snapshot camera between frames, or sequences where a single image can be animated across the screen by incrementally moving the cel, were almost always animated at full 24fps, because they're far simpler to achieve - so you get this weird mashup where some elements animate at full 24fps and others use duplicate frames to animate at 12fps (or lower) in the same actual sequence. This kind of thing really throws the AI off.

The sequence where the "camera" zooms around Blair in his cockpit is a perfect example. DAIN did a half decent job making that motion smoother than in the original source footage, but there are still some frames where the motion isn't quite right - and it produced a lot of false interpolation artefacts.

These are the kind of things I'm certain I could improve once I have the time spare to concentrate on this properly - and I'm sure I'll revisit this later to produce a better version 😁

 

[Rylex]

These are the kind of things I'm certain I could improve once I have the time spare to concentrate on this properly - and I'm sure I'll revisit this later to produce a better version 😁

Yeah, I know what you mean... I'm always trying to produce the best results for the projects I'm working on. But I'm afraid, I don't have a lot of time for projects and for Wing Commander anymore. Yeah, it feel's like time is running faster and faster as you're getting older. It still feels like yesterday when I was working on the Unknown Enemy project with Pedro and Quaro... ah... now I'm getting nostalgic. 🤧

I still hope that there might be a remastered "original" version of Privateer 1 one day (perhaps with new ships and an additional storyline after Righteous Fire). That would be the game I'd LOVE to play. I know that Privateer Gemini Gold has been published back in 2009 but it did not really feel as good as the original game.
Well... what did Blair say to Vaquero on the TCS Victory? ... "It's a nice dream Vaquero..." ... so I dream of a new Privateer 1...

 

[ODVS]

I've been quiet for a while, but I have been tinkering away and experimenting.

Specifically, I've experimented with using alternative AI methods. I know I've mentioned using DAIN (Depth-Aware frame INterpolation) AI models to increase the frame rate of the WC3 PSX videos - but recently I've been combining that with ESRGAN (Enhanced Super-Resolution Generative Adversarial Network) AI for the upscaling process.

If you haven't heard of ESRGAN before, it's the Neural Network AI tech used by a lot of game remastering enthusiasts to upscale textures in older games. If you've seen any of those original Doom mods with higher res textures or the Final Fantasy VII fan remaster that was doing the rounds a while ago, they use ESRGAN.

It's really best suited to textures and individual images rather than video enhancement - but being open source it does give me consideably more granular control over how it's actually implemented than Topaz Video Enhance AI does. I'd say the results are a mixed bag. It does a far better job of enhancing the CGI backgrounds and things like uniforms and hair, but it's not as good at faces. Prepare for the return of the crazy eyes!

Unfortunately, it's a far slower process, too. Not just from a processing perspective, but as a workload. I unfortunately can't automate the process as much as I can with the Topaz workflow - I have to manually select and process each video file, then leave it running. Given that there are 471 individual video files in WC3, it would take a very, very long time to create an entire HD pack using this process. If I left one per night processing, for example, that's about a year and three months - and most of it would be dead time, just waiting for me to kick off the next file.

Given that fact and the mixed results, I think this is something to return to later - but it's a very promising experiment, none the less.

I've also modified my AVIsynth scripts for the WC3 workflow to alter the brightness and contrast a bit. The PSX footage (in my opinion) is quite overblown on brightness, so I thought I'd address that moving forward.

Demo below, as usual. I decided to re-do some scenes I've used before, so you guys can get an idea of the difference between this workflow and my previous one.

 

[Shades2585]

I've heard good things about ESRGAN and believe they were using it for some of the SW Jedi Knight HD pack. I do like the lower brightness though sadly I don't think the quality came out as well as your other efforts. Keep up the great work though. I look forward to seeing more. 😀

 

[ODVS]

I've heard good things about ESRGAN and believe they were using it for some of the SW Jedi Knight HD pack. I do like the lower brightness though sadly I don't think the quality came out as well as your other efforts. Keep up the great work though. I look forward to seeing more.

Thanks, @Shades2585

Yeah, I think I'll continue using DAIN for the frame interpolation and the updated AVIsynth script for the colour correction, but stick with Topaz for the upscaling process for now. I'll revisit ESRGAN later, I think.

More to come - watch this space...

 

[Saturnyne]

Hey, I'm emerging from the ether to express my appreciation for this! Everything looks and works great in Wing Commander IV.

In Prophecy, the FMVs are stretched vertically for me instead of fitting the screen, so I can't see the top and bottom edges of them. Any idea what I might be doing wrong?

Thanks for all your hard work! Looking forward to Wing Commander III stuff.

 

[ODVS]

In Prophecy, the FMVs are stretched vertically for me instead of fitting the screen, so I can't see the top and bottom edges of them. Any idea what I might be doing wrong?

Hi @Saturnyne,

I suspect that's most likely a Codec issue. Although WC4 and WCP use different playback methods for the DVD+ video formats, I have seen that same issue in both games when I've been playing around with different encoder codecs to see if I can increase quality/redcuce file size.

It might help to check out this post and see if installing and configuring the K-Lite pack that way helps with your issue

EDIT - also, some people have found that following previous tweaks/hacks listed on the GOG forums (especially ones that have you manually alter the Windows registry) prevent the HD videos from playing back properly. If you've done anything like that, it might be worth undoing it to see if it helps too.

 

[Saturnyne]

All right, I'll do a little reconfiguring after work today and see what happens. Thanks!

 

[Saturnyne]

Okay, I've tried every setting K-Lite had for the H.264 codec, and while the movies play just fine, they're still vertically stretched, and there's a black band along the bottom of the screen. Also, my mouse pointer appears at the center of every FMV, and while I can move it around, it snaps back to center. XD I've included a screencap as an example, and halved the image's resolution for the sake of filesize.

I haven't done any registry editing, but I did get this version off of GOG. These are the steps I used, in case they're helpful:
- Installed the game.
- Moved nglide_config.exe, glide.dll, glide2x.dll and glide3x.dll to a backup folder.
- Added the Enhancement Pack files to the game folder, but did NOT run the installer exe.
- Installed the OpenGL patch. Configured it to run at 1920x1080, Anisotropy x4, no anti-aliasing.
- Overwrote the Enhancement Pack's wcphr.dll with the one from the DVD pack.
- Copied the 3.0 FMVs into their own Prophecy\movie folder.

In terms of codecs, I've been using the K-Lite Basic Codec pack.

Oh, and as a related note, WCIV works just fine, and looks great. Again, thanks for your great work! And for your patience in dealing with me. lol

 

[ODVS]

Hi @Saturnyne,

Thank you for the compliment 😄

I'm afraid the mouse pointer being stuck in the middle is a noted bug that affects even the DVD video pack and is due to the DirectShow video player that's used. Someone has suggested a fix for this on the GOG forums, but that's the very Registry edit I was talking about that others have found breaks the HD pack playback.

As for why you're getting the stretched videos, I'm afraid I'm at a loss at this point. I think the only thing I can suggest is to try downloading the Full or Mega versions of the K-Lite pack and configuring the tweak tool again to ensure the h264 settings are definitely set to "Microsoft".

That and the usual "please make sure your graphics drivers are up to date" (I get sick of hearing that one, too).

Has anyone else out there in the community come across this error and, if so, did you find a way around it?

 

[Panther1.0]

@Saturnyne,

Did you overwrite the Prophecy.exe file upon adding the HCL WCP enhancement package to the GOG version? Don't overwrite the .exe

move the new movie files to the Movie folder ( under the Prophecy insallation folder.)

then after installing K-lite codec pack run Prophecy with admin rights and disable windows 98 compatibility.

 

[ODVS]

As discussed in this post a few days ago, here's a replacement to that video using DAIN for frame rate interpolation and Topaz Video Ehance AI for the upscale instead of ESRGAN.

This workflow maintains the AVIsynth scripts I wrote to correct the brightness/contrast balance that I previously mentioned - and I'm happy enough with how it turned out that I'm going to base an entire WC3 HD Pack V2.0 around it. This will be the first time I've taken a proper, optimised run at a WC3 HD Pack, so I'll be concentrating on balancing the quality vs. file size this time around. I'm confident it will turn out pretty well

In the meantime, if anyone has any requests for scenes from WC3 you'd like to see in a V2.0 comparison video, just let me know - I should be able to knock them out pretty swiftly

 

[Panther1.0]

interesting, looks like it works

 

[Pedro]

It's now at the point where I'm absolutely certain I would rather play with these upscales than the original video.
Rollins teeth (did he have braces in filming? they look weird even without the upscaling) and eyes which are only a couple of pixels in size on the original suffer but other than that it looks amazing. I wonder if in future upscaling will be more involved where you can highlight areas which look odd and it will present you with a number of options to choose from.

 

[Saturnyne]

EDIT: I figured out the problem. The OpenGL driver install should be done ABSOLUTELY LAST, at least on my system. Once done, the movies play just fine, though to skip them now is Alt-F4, which thankfully doesn't exit the game. Thanks for putting up with me! lol

@ODVS
Wow, that looks great! Does it work with the DOS/GOG version, or is it KS only?

 

[Pedro]

Wow, that looks great! Does it work with the DOS/GOG version, or is it KS only?

Darkmage was talking about adding a patch for KS, currently no options for watching them in-game.
We are playing around with it (and other WC3 files) on the WC4 remake, but that's all years away from completion unless we get more programming support.

 

[Wedge009]

I agree the v2.0 video looks a lot better, looks much closer to what I've seen before in your other work. However, in the scene with Taggart and Blair, I noticed a black spot briefly appear on Blair's brow just as Taggart says 'since the Concordia went down' - about 2:20. Any idea where that came from?

If you haven't heard of ESRGAN before, it's the Neural Network AI tech used by a lot of game remastering enthusiasts to upscale textures in older games. If you've seen any of those original Doom mods with higher res textures or the Final Fantasy VII fan remaster that was doing the rounds a while ago, they use ESRGAN.

I was going to ask if you'd seen the Moguri Mod for FFIX and whether or not that also uses ESRGAN, but then I now see they say so right there on its front page... I hadn't heard of this kind of project until fairly recently and was quite impressed to see how much detail can be regenerated (a bit like those computer 'enhance' filters we often see in investigation scenes in the movies, except this in a real world context), and obviously static scenes are much easier to process and polish than videos. FFVII through FFIX, and even some scenes in FFX used pre-rendered backgrounds that make it easy for people to upscale like this, I understand players are begging for the same treatment for FFVIII because the recent 'remaster' was pretty lacking in consistency, with many players complaining that the high-res character models jar against the backdrop of blurry, unfiltered, original resolution backgrounds.

 

[ODVS]

...in the scene with Taggart and Blair, I noticed a black spot briefly appear on Blair's brow just as Taggart says 'since the Concordia went down' - about 2:20. Any idea where that came from?

Yeah, I clocked that too - don't worry, I'm going to manually fix it before compiling the pack :^)

I had a look and it is actually there in the original footage as just a slightly darker pixel on Blair's face for a couple of frames. It's just that once it's upscaled, it's obviously larger and more noticeable. The AI routine also seems to have latched onto it a bit and made it stand out more.

 

[ODVS]

EDIT: I figured out the problem. The OpenGL driver install should be done ABSOLUTELY LAST, at least on my system. Once done, the movies play just fine, though to skip them now is Alt-F4, which thankfully doesn't exit the game. Thanks for putting up with me! lol

Fantastic! I'm glad you're all sorted, @Saturnyne - thanks for letting me know. The "Alt-f4 to skip" thing is because the videos are technically playing in another app (a DirectShow player that the modified game launches when required to play a DVD/Remastered video file), since the base game never had DVD support. If you pop up the Windows Task Bar while a video is playing, you'll see it there.

@ODVS
Wow, that looks great! Does it work with the DOS/GOG version, or is it KS only?

Thank you Sadly, as Pedro mentioned, there currently isn't any way to play these back in-game - they're more a techinical Proof of Concept at the moment. I will be making them available to download soon, though, so any interested Wingnuts will be able to browse through/watch them. I also live in hope that someone more skilled in software modding/development than I will find a way :^)