Summary
Passing or failing: these are the usual states of JUnit tests. But discover a third state that many use but only few acknowledge, and how you can elegantly manage these tests with JUnit-addons Runner.

---

Advertisement

Tests can either pass or fail. Thus they are binary.

Looking at several Open Source projects (and my own practice) I've noticed numerous tests whose names were altered or simply commented out only to exclude this particular test from the general execution.

But why should you ever exclude a test? Plenty of very good reasons:

• test reveals a bug that you can't fix (for example in an external library)
• test reveals a bug but you don't know how to fix it (or you simply don't have the time)
• test is broken but you don't have time to fix it
• ...

And why does it matter? After the build itself, tests are the best indicators of the code's correctness. If failing tests remain in the test suite, nobody can trust them anymore: it confuses other developers (did my modification break the tests or were they already broken?) and slowing the development process. You can of course comment out the failing tests but the standard JUnit runner won't remind you that they are waiting to be fixed.

One solution can be to create another test suite for these tests but my experience is that doing so is the exception, not the rule. Fortunately the JUnit-addons Runner offers a flexible and elegant solution to this problem.

One of the features of this runner is its ability to ignore a test by simply appending _ignored to its method's name (public void testDummy becomes public void testDummy_ignored). When executing the suite, the runner outputs:

    ****

    Elapsed time: 0.005 sec (4 tests + 1 ignored)

    IGNORED

    1) testDummy_ignored (junitx.example.IgnoredTestCase)

All your tests now pass and you (and other developers) are always reminded that there is still work to be done.

The JUnit-addons Runner is available at sourceforge.net/projects/junit-addons

Talk Back!

Have an opinion? Readers have already posted 15 comments about this weblog entry. Why not add yours?

RSS Feed

If you'd like to be notified whenever Vladimir Ritz Bossicard adds a new entry to his weblog, subscribe to his RSS feed.

Digg | del.icio.us | Reddit

About the Blogger

Vladimir Ritz Bossicard is a software engineer located in the Bay Area. He has a Master's degree in Computer Science from the EPFL and is interested in Open Source Software and pragmatic testing. Besides software engineering his main interest is his daughter Thea.