GDC 2006: Test Driven Development from High Moon Studios

Submitted by User ProfileRob on 9:08pm, Tuesday, 2005-12-20
I just got an email from GDC with a reference to their Programming Track Highlights. There was a session in there that I had meant to get back to - something on Test Driven Development. I'm not one to chase after every new programming methodology that someone thinks up, but I like to hear about the problems people are trying to solve through changes to their programming methodologies. Sometimes they make sense, sometimes they're very unrealistic. So the session at GDC is called "Backwards Is Forward: Making Better Games with Test-Driven Development" and it's put on by some people from High Moon Studios. I've already read the blurb at the GDC site, but I'd like to know more. First I'll dig in to find out more about TDD, then I'll see what I can find out about the speakers. Here's what Wikipedia has to say about Test Driven Development:
Test-driven development (TDD) is a Computer programming technique that involves writing tests first and then implementing the code to make them pass. The goal of test-driven development is to achieve rapid feedback and implements the "illustrate the main line" approach to constructing a program. This technique is heavily emphasized in Extreme Programming. Practitioners emphasize that test-driven development is primarily a method of designing software, not just a method of testing. TDD also comes in handy when fixing bugs: first, you write a test to reproduce the bug (so that the test fails due to the bug), then you implement a fix until the test passes.
The wiki page makes TDD sound like a useful technique for a subset of software that can be tested automatically. Wouldn't it be great if all software could be tested by automation? Test Driven has a more detailed overview. They emphasize TDD as a core practice of Extreme Programming. I think there's been a substantial backlash against XP. It seems that it's not cool anymore, which would only be reinforced by the fact that the GDC blurb about this TDD session doesn't mention Extreme Programming even once. On the other hand, I also think I'm part of a huge crowd that sees benefit in some of the XP practices. I'm not sold yet. It sounds like a nice feature in the first pass, and it'd be great if that were enough. I need to know how Test Driven Development works with code maintenance. What happens if I have an intermittent bug that I can't find the root cause of? I suppose that means I don't have a real solution for the that root cause. Unfortunately we know that in software maintenance such bugs are often patched over with a shotgun approach such that any one of many possible causes are avoided. Customers just don't wait for the complete solution to the problem which would require research or, worse yet, additional tools (debuggers, hardware that exhibits the bug, etc). While I don't believe in the maintenance case, I can still see where TDD may be effectively applied when building new software. Applying TDD or some other XP concepts would require some flexibility in the design and development environment. This could be easy to get for an independent game developer, so I could definitely see where I could get something out of the GDC session from High Moon. That brings me around to the next question I like to answer before showing up. Who are these guys? The speakers for this session are Noel Llopis and Sean Houghton. Noel Llopis, I was surprised to find, is already in my bookmarks. He writes at Game From Within. I seem to remember reading his "Care and Feeding of Pre-Compiled Headers" earlier this year. He's also the author of C++ For Game Programmers, a hefty text that's done very well in Amazon ratings. He seems like someone I'd like to hear talk on programming. Sean Houghton also seems to know what he's talking about, but I haven't been able to dig up much on him from Google (and maybe I'm just a little lazy by now). So after all that, I'm putting the Test-Driven Design session on my list. The list is really just the sessions that I'd like to attend, but I can never get to them all. The GDC web site doesn't help me out here, either, since there are no times posted for any sessions yet.
0
Your rating: None
At 11:35pm, Tuesday, 2005-12-20, Jeff Schiller said:

I'm willing to give them a listen. We should have the time.

Mind you, I've only learned about TDD from your post, but my first questions are: Who writes the test and who writes the code?

If it's the same guy it would ultimately seem to be pretty useless - the person testing should not be the person coding because you benefit from that detachment/differing perspective. i.e. I have to write a function that sufficiently shuffles an array of objects, so I write a test that feeds the function an array of ordered numbers and then I check the output to ensure that at least half of the elements are not adjacent to elements that they were originally. I write my code and I pass my own test. But later we find out that my function actually overwrites the original array with random numbers and the test case did not account for this possibility. Coders who write their own tests are naturally going to be influenced by their programming style and that they are thinking ahead to how they will code it.

At 11:49pm, Tuesday, 2005-12-20, Rob said:

I can definitely see that as a problem. I think they address that when they say that when you find a bug you have to write a test to catch that bug. The test should first be failed by the code with the bug, then the bug gets fixed and the test should pass. They have the mantra "red/green/refactor" which reflects the idea that you first fail (red), then pass (green) then clean up (refactor) the code.

The hole I see here is that if you don't test your tests then there's the possibility that they no longer apply or no longer test the bug they were looking for. Continuing in your example, assume a new test is made to catch your bug. Test 2 checks to make sure that the same array items that were passed in are also present in the output. First, how do I know that Test 2 is working after the bug is fixed? It's easy enough for the test to work with a shallow equality check the first time through, then I rework the function being tested and the shallow equality check always passes but I've diddled with the objects and they're not really equal.

I guess what it comes down to is that the tests themselves are actually dependent on the code they test. Changes to the code under test would require validation of all the tests that check that code. They must also have better disambiguating names for these things...

Anyhow, TDD sounds good partly because I can't come up with a better way to do what they're talking about. That is automated execution of tests on code that validates (at least some) requirements. So it should make for interesting discussion at the least, and at the best I'll learn something about how to design for test.

At 7:42pm, Thursday, 2005-12-22, Something Witty Goes Here » Blog Archive » GDC 200 said:

[...] Rob recently wrote about Test-Driven Development, a lecture that will be given at the Game Developer’s Conference (GDC) in 2006 from High Moon Studios. High Moon is also giving another talk about Agile Development and something called “scrum”. Curious, I decided to take a look. [...]

At 9:07pm, Friday, 2005-12-23, GDC2006: C++ on Next-Gen Consoles: Effective Code for New Ar said:

[...] I really enjoyed the talk a few years back on how they ported DirectX to the original Xbox and I also got a kick out of a GDC talk on advanced compiler optimizations for C++. I think this one could be right up my alley. Of course I like to be an informed attendee, so I’m going to do some research on the issue raised here and get some background. That should get me ready to get the most out of this lecture - if I get it into my schedule. The list of sessions I’m building up here goes along with Jeff’s post on Scrum yesterday and mine on TDD the other day. [...]

At 12:09am, Wednesday, 2006-01-18, Getting Psyched for the Game Developer’s Conference ~ said:

[...] Backwards Is Forward: Making Better Games with Test-Driven Development (GDC description) [...]