By the Books: Solid Software Engineering for GamesGDC2003 roundtable by Noel Llopis
Thanks again to everybody who attended the roundtable this year. It was great to see the amount of interest in the topic, as well as all the people who repeated from last year.
The roundtable was held at three different times, and even though the general topic was the same, all three sessions narrowed down on slightly different subjects. Here is a summary of what was discussed.
AttendanceAll three sessions ended up being packed, with a total of around 120 people (50, 35, and 35), which is pretty amazing considering the last roundtable was going up against Fred Brooks' keynote :-)
Of the people who attended, very few were straight managers (about 5-6%), but there were a lot of lead engineers (roughly 30%), and the rest regular programmers/engineers.
The split between consoles and PC development was a rough 50-50 with a tiny fraction of people doing handheld development.
Like last year, C++ was by far the dominant language of choice, with just a few people using straight C, Java, or even C# in the case of one person. Nobody this year admitted to using only assembly, so maybe things are moving in the right direction.
Software engineeringWhat exactly is software engineering and why do we care as game programmers? People pointed out that better software engineering doesn't necessarily make for a better game. As a matter of fact, there are plenty of examples of terrible development processes that end up creating a good game. What software engineering does is help make the development of the game faster, more predictable, and especially less painful for everybody involved.
People pointed out that the larger a team becomes, the more important software engineering becomes to keep development in track and keep everybody working together. With teams of 25+ programmers and schedules over 18 months, this is becoming more and more important every day. One person mentioned that there were 140 programmers in his company, split into 4-5 different projects, and a team that creates a common code base and tools for those teams.
The notes for last year's roundtable have a good section on general software engineering resources.
MethodologiesSome people associated software engineering with highly structured, pre-defined approaches, with lots of documentation and overhead. Something to be feared even. Clearly, not all software engineering approaches are that way, and some try to be quite at the other end of the spectrum (agile methods).
Mostly everybody described their development process as being somewhat unstructured and very iterative, although some claimed they were doing nothing more than plain “hack and slash”. Only a few people used a traditional waterfall model, and even so, with some iteration to it.
A few people were either using, or had tried in the past, agile methodologies like extreme programming. Not everybody adopts all the practices from extreme programming, and some of them always encounter a lot of resistance (pair programming, or no code ownership). Somebody mentioned they used to do XP in his company, but they had to stop because all the programmers hated it and threatened to quit.
Unlike last year, we didn’t get into agile methodologies on purpose because there was another roundtable dealing entirely with that topic.
Technical designMostly everybody did some sort of technical design. Only two or three people claimed they jumped into coding something without doing any prior analysis.
There was quite a range of how people did design though. For some, design meant running some ideas by a co-worker, maybe using a whiteboard and drawing some UML diagrams. On the other extreme, some people were very happy with tools like Together that allow for a UML design to be converted into code and then have the code changes reflected back in the UML diagram.
Most people fell somewhere in the middle. A lot of people used UML, but mostly as a communication aid rather than as a starting point for code. Most people did not keep the initial design documents up to date since their purpose was for the initial design only.
Some of the other UML tools mentioned were:
Sharing informationOne of the problems with trying to do any design with a team of people is the question of how to actually share that information, keep it up to date, and make sure everybody is aware of it.
Several people brought up code reviews, but the response to them was very mixed. Only about 15 people were doing some form of code reviews. Some thought it was a waste of time, but most of the people who were doing them found them useful. Most of the code reviews were done by a senior engineer on a one-on-one basis, or even reviewing the code alone and then emailing the original programmer or talking to him in person. One of the side benefits mentioned of code reviews is that people know somebody is going to be looking at their code, so they end up writing better quality code to start with.
Somewhat related to code reviews was the issue of pair programming. It was brought up that pair programming was also a way to share and spread knowledge about the design and the code base.
Weekly meetings were also mentioned as a good way to keep the team in synch, knowing what everybody else is working with, but it's not enough to explain all the details. One person said everybody in their team did everything together, in and outside of work, so they were always talking about work issues and information dissemination was never a problem.
Some specific ways of sharing information that were brought up were:
Code ownershipThis is another hot issue that nobody seems to manage to agree on. Most people expressed that they were using a system with fairly strong code ownership. There was one or two people who were clearly responsible for knowing a particular system, and were the people to go to if anything had to be changed.
On the other extreme, a few people preferred the no-ownership approach. Everybody can modify any files, make improvements to them, fix bugs etc. This has the advantage that knowledge about the code is spread among the whole team and that issues won't become a problem of egos, but problems with the code. It might also improve team spirit and communication. It also makes the approach to development much more flexible since there might be several people qualified to work in different areas.
On the other hand, strong code ownership might be more appropriate for areas where specialized knowledge is required (for example, physics code or pathfinding). Also, with strong code ownership it is possible for people to identify more with a particular system and do a better job with it as opposed to letting the broken windows syndrome settle in.
One team used an approach of no-ownership, but had a person full-time refactoring the code, fixing problems, and improving the overall quality.
Coding standardsA fairly large percentage of the people who attended had coding standards in their company; around 30-40%. However, probably only a bit over half of them actually followed them.
Like in the case of code reviews, the people who currently had coding standards found them invaluable, while the rest were mostly skeptical about their usefulness. Somebody described them painful to implement, but good in the long run. A coding standard can be particularly beneficial when bringing in new members.
Some people used macros in their environment to make it easier to follow a particular convention. For example, macros could create new skeletons for classes and function definitions. One person mentioned the tool Artistic Style to automatically format the code in the correct way for the company standard (although it might cause some problems with source control history).
A hotly debated issue is how much should a coding standard cover. Everybody agreed that it shouldn't be too nit-picky, and only concentrate on the things that matter. Unfortunately, for some a consistent bracing style matter a lot, while others are not willing to give up their own style. It seems that one of the most beneficial things that could be covered in a standard is not so much the formatting of the code, but the design principles behind it, and other high-level issues, but usually none of those things are part of a standard.
Source controlLike last year, everybody working in a team was using source control. Only two people who were working by themselves were not using it (but several other people who were also working by themselves were using source control regularly).
The source control programs that people were using were:
Interestingly, the large majority of the people only used source control to keep the history of the files. Only a small number of people (10%?) were taking full advantage of branching during development (virtually none of them were SourceSafe users).
Another source control strategy presented was that each programmer has a private branch. This gives great flexibility, but merging is more complicated. Some teams had scripts that would email specific people if files in a particular section of the code changes so they are aware of the changes.
Unit testingUnit testing was brought up briefly. Only a handful of people were actually doing any real unit tests, but they were really pleased with the results.
A common concern with unit tests is how to effectively test many of the systems involved in game development since they might not be tested easily in isolation, or they might need full levels to be tested. Also, testing high-level code is more complicated than low-level math libraries.
Testing of visual aspects is particular difficult with unit tests, but one team (working on a game engine) reported to use image diffs to verify that the engine was working correctly.
The beneficial side effect of serving as documentation for other people was brought up, as well as a brief mention of test-driven development.
Automated buildsAutomated builds were much more popular this year with a fair amount of people reporting that they had them at their companies (maybe 15%-20%).
The advantages cited for automated builds were:
A common problem was trying to avoid breaking the build. Some teams used tactics to discourage people from breaking the build such as having to contribute to a common fund (which would be used for ice cream at the end of the month), or wearing the “sombrero of shame.” Some teams had more complex procedures to prevent breaking the build, such as doing full local builds before committing the changes, or even submitting the changes automatically to a machine that would do a tentative build and email back the programmer with the results.
Everybody was using a custom build systems (in Python, Perl, plain batch files, or even in .NET). One person was using Ant, an open source, Java-based automated build system
There were different strategies for automated builds. Some people built once every day, some twice a day (once at night, once at lunch), some once every hour, and some as soon as one build was finished started another one.
|