A key benefit of a continuous integration system is that it enables developers to quickly address problems with builds or tests. In order to address those problems, developers must be kept appraised of a project's build health. Paul Duvall discusses various feedback mechanisms continuous integrations systems offer in an IBM developerWorks article.
In addition to automatically running build jobs and executing tests, continuous integration systems must also keep developers abreast of a project's build health. Thus, an important feature of continuous integration systems is developer feedback.
Paul Duvall penned a recently published article on Continuous Feedback for IBM's developerWorks web site. Duval points out that the feedback loop between a continuous integration system and a developer must be a two-way street:
Rapid feedback of build events is the central tenet of CI and one that causes many to describe a CI system as a continuous feedback mechanism... The sooner I know [of a problem], the more relevant the information and the better chance I have to fix the problem...
Feedback, however, is useless without action. In the case of CI, this action must be prompt because a broken build affects everyone; therefore, the feedback mechanism employed by a CI server must be timely. The beauty is that most, if not all, CI servers available today provide ... feedback mechanism... These various feedback mechanisms quickly facilitate jumpstarting the necessary people into action.
He lists six different types of feedback provided by popular continuous integration systems:
Every CI server ... provides some type of e-mail feedback mechanism... Most CI servers provide the capability for configuring who receives which message, based on a desired build event. For instance, a technical lead may always receive an e-mail regardless of build status, but the project manager only receives an e-mail when the build fails...
An RSS reader ... picks up the change [from the CI system] and creates a passive message indicating new content. So, if you'd rather not get bombarded with e-mail, try this approach instead... [The] RSS feed can be updated based on the latest build status.
Using SMS to send text messages can be an effective form of feedback for urgent build status messages... If you believe that keeping an integration build successful is one of the highest priorities on your project, you'll want to know the build status at any point in time... The same mechanism used to send e-mail can also be used to send SMS text messages.
There are a variety of X10 devices—LavaLamps, sirens, you name it—that can really enliven a project's environment. Adding these devices to a room is bound to create excitement, along with a clear indication of a build's status... You can control most any electrical device that can be enabled using special hardware that can be manipulated through the X10 protocol... Several X10 kits are commercially available and [for example] CruiseControl includes the Java COMM API so that you can use CruiseControl's config.xml to communicate with the device and turn it on or off.
CruiseControl.NET ... provides a monitoring mechanism called CCTray that works with both CruiseControl.NET and the traditional Java built CruiseControl and is run from the Windows taskbar (which means it works only on Windows machines)... You can use another client monitor called Nag on non-Windows machines and even on other CI servers, such as Apache Continuum.
Instant messages [are] another way to receive feedback quickly because of IM's ubiquity among developers... CI servers like Apache Continuum provide a simple way to send IMs regarding a build's status.
In your projects, how do you keep track of the current build and test status?