Frank Reminisces About Playtesting At Origin

Normally we don't report on playtesting guides posted to the XNA site, but this one is a little different. It's written by Wing Commander veteran Frank Savage, who is a development manager for the XNA toolset. You can read the entire article here. To prove a point, he even delves into his Wing Commander past to pull up an anecdote from his time at Origin:

Got any good playtest stories?

I have dozens of great playtest stories from the trenches that could probably fill a book. I’ll give you my favorite two.

The first was from the Wing Commander era, and is a great example of an issue that didn’t really matter. In Wing Commander 1, when you returned to the Tiger’s Claw after a mission, the carrier was sometimes upside down, depending on your orientation when you were coming back. This was to be expected since the game was 3D and it was easy to get your ship into that state by simply rolling. A very persistent playtester at Origin kept filing bugs on this until the game shipped.

So, in Wing Commander 3, I purposely brought the carrier in upside down for several missions in the hopes of getting this bug again. Alas, no one complained about it. This is a great example of an issue that doesn’t really matter. You could land on the carrier regardless of its orientation in both games.

The second story is from the Strike Commander era, and is a great example of how to approach an issue when you think your game is done. We were running through the last set of tests before shipping the game to the duplicators, when I got a call that the cluster bombs in the training mission on the F-16 were exploding the instant you dropped them, which of course destroyed your plane. Since we had dropped hundreds, if not thousands, of cluster bombs during every other mission in the game, this bug was perplexing. I went into work and watched someone reproduce the bug and I noticed that they were flying over water in the training mission when they dropped them. This struck me as important because I had never seen the bombs dropped over water before. There was simply no reason to drop them over water as there were no targets in the water that a cluster bomb would damage severely.

I went and dug into this issue, only to discover that I had indeed put a bug in the code almost eight months earlier! The cluster bombs were supposed to detonate in the air and scatter their munitions over a wide area. Instead of code instructing the game to “detonate when you reach this altitude or go below it”, what I had written was code to “detonate if the terrain is ever lower than this altitude”! Since the terrain was always higher than the minimum altitude specified in the code in every other part of the game except over water, the bombs usually dropped fine and detonated when they hit the ground. This was not the desired effect, but we had tweaked the entire game to play well with the cluster bombs hitting the ground instead of going off in the air. No one knew the bug had been there for the last eight months.

Making the code “right” by changing it to act as I had intended would almost certainly have made the cluster bombs behave differently everywhere. This might have made them more effective or less effective, but only a complete play through would have been able to make sure this feature wasn’t broken, and we were scheduled to ship the game the next day!

We decided to remove the line of code that made the bombs detonate over water, instantly destroying your plane, and left the rest of the code the way it was. It wasn’t “accurate” but we knew we had a good game with the cluster bombs behaving the way they did over land. This also fixed any possible issues from a player inadvertently dropping the bombs over water and blowing themselves up! In this case, the right thing to do was to fix the small thing that was clearly broken, and leave the rest of the bomb’s behavior alone.