Summary
For some time I've been thinking about how TDD tests can be as simple, as expressive, and as elegant as possible. This article explores a bit about what it's like to make tests as simple and decomposed as possible: aiming for a single assertion in each test.
Advertisement
A while ago there was a bit of fuss on the testdrivendevelopment
Yahoo group about the idea of limiting yourself to one assertion per
test method, which is a guideline that others and I offer for TDD
work.
An address parser was the example of a situation where it was
argued that multiple assertions per test made sense. Date was
formatted with one address per line, each in one of the following
formats:
ADDR1$ADDR2$CSP$COUNTRY
ADDR1$ADDR2$CSP
ADDR1$CSP$COUNTRY
ADDR1$CSP
The poster went on to say:
My first inclination is/was to write a test like this:
a = CreateObject("Address")
a.GetAddressParts("ADDR1$ADDR2$CITY IL 60563$COUNTRY")
AssertEquals("ADDR1", a.Addr1)
AssertEquals("ADDR2", a.Addr2)
AssertEquals("CITY IL 60563", a.CityStatePostalCd)
AssertEquals("Country", a.Country)
They didn't see how to achieve this with one assertion per test, as
there are obviously four things to test in this case. I decided that
rather than simply reply, I would write some tests and code to
illustrate my view on the matter, and offer a solid response.
For this problem, I chose Squeak Smalltalk (see www.squeak.org) and Java. For the sake of
conciseness, I'll omit any required accessors.
So, where to start? Well, when doing TDD it often makes sense to
start with something simple to quickly and easily get some code
written and working. Then it can be extended and evolved in response
to further test driving. Here the simplest case is: ADDR1$CSP. There
are two requirements in the parsing of this example: that the ADDR1
was recognized, and that the CSP was recognized. Viewed this way, we
need two tests. We start with one for ADDR1:
public void testAddr1() throws Exception {
Address anAddress = new Address("ADDR1$CITY IL 60563");
assertEquals("ADDR1", anAddress.getAddr1());
}
To get this to pass we need an Address class and a from: factory
method, which creates an instance and has it parse the address string.
For brevity, I'll skip the "return literal" step.
public class Address {
private String addr1;
public Address(String aString) {
parse(aString);
}
private void parse(String aString) {
StringTokenizer parts = new StringTokenizer(aString, "$");
addr1 = parts.nextToken();
}
}
That's well & good. The next test is for CSP.
Squeak:
testCsp
| anAddress |
anAddress := Address from: 'ADDR1$CITY IL 60563'.
self assert: anAddress csp equals: 'CITY IL 60563'
Java:
public void testCsp() throws Exception {
Address anAddress = new Address("ADDR1$CITY IL 60563");
assertEquals("CITY IL 60563", anAddress.getCsp());
}
Address>>parse: will need to be extended (and we need to add a csp
instance variable and accessors):
Squeak:
parse: aString
| parts |
parts := aString findTokens: '$'.
addr1 := (parts at: 1).
csp := (parts at: 2)
Java:
private void parse(String aString) {
StringTokenizer parts = new StringTokenizer(aString, "$");
addr1 = parts.nextToken();
csp = parts.nextToken();
}
So. We have two tests for this one situation. Notice the duplication
in the tests... the creation of the instance of Address that is being
probed. This is the fixture. After refactoring, we have:
public class Addr1CspTests extends TestCase {
private Address anAddress;
protected void setUp() throws Exception {
anAddress = new Address("ADDR1$CITY IL 60563");
}
public void testAddr1() throws Exception {
assertEquals("ADDR1", anAddress.getAddr1());
}
public void testCsp() throws Exception {
assertEquals("CITY IL 60563", anAddress.getCsp());
}
}
So, a fixture that creates the Address instance from the string, and
very simple tests that focus on each aspect of that fixture.
The next simplest case is the obvious choice for the next fixture:
Squeak:
setUp
anAddress := Address from: 'ADDR1$CITY IL 60563$COUNTRY'
Java:
protected void setUp() throws Exception {
anAddress = new Address("ADDR1$CITY IL 60563$COUNTRY");
}
This set of tests will include ones for addr1 and csp as before
(refactoring this to remove that duplication is left to the reader) as
well as a new test for country:
Squeak:
testCountry
self assert: anAddress country equals: 'COUNTRY'
Java:
public void testCountry() throws Exception {
assertEquals("COUNTRY", anAddress.getCountry());
}
As before, an instance variable and associated accessors need to be
added to the Address class.
This drives Address>>parse: to evolve:
Squeak:
parse: aString
| parts |
parts := aString findTokens: '$'.
addr1 := (parts at: 1).
csp := (parts at: 2).
country := (parts at: 3 ifAbsent: [''])
Java:
private void parse(String aString) {
StringTokenizer parts = new StringTokenizer(aString, "$");
addr1 = parts.nextToken();
csp = parts.nextToken();
country = parts.hasMoreTokens() ? parts.nextToken() : "";
}
From here on, the evolution gets a bit more complex, as we add the
ADDR2 option to the mix.
Conclusion
So we took a situation that was thought to require multiple assertions
in a test and did it in such as way as to have only one assertion per
test.
The key is that instead of using a single TestCase subclass with a
complex (i.e. multiple assertion) tests for each situation, we made
each of those situations into a separate fixture. Each fixture is
implemented by a separate subclass of TestCase. Now each test focuses
on a very small, specific aspect of that particular fixture.
I'm convinced writing tests like this is a useful approach. One
advantage is that the resulting tests simpler and easier to
understand. Just as important, and maybe more so, is that by adding
the specification of the behavior one tiny piece at a time, you drive
toward evolving the code in small, controllable, understandable steps.
It also fits better into the test fixture centered approach that is
the recommended way to organize your tests. We set up the object to
test in the setUp method, and tested each aspect of it in individual
tests methods.
As I've been writing this, something clicked. I see these test
methods as specifications of tiny facets of the required behavior.
Thus, it makes sense to me to be as gradual as possible about it,
driving the evolution of the code in the smallest steps possible.
Striving for one assertion per test is a way to do that.
If, however, you view test methods as strictly performing
verification, then I can see how it might be seen to make sense to
invoke some code and then test all the postconditions. But this view
is not TDD, and doesn't buy you all of the benefits of TDD. I contend
that central to TDD is this notion of working in the smallest steps
possible, both for the finest-grained long-term verification, and for
the most flexible design evolution. Furthermore, this is best done by
striving to keep tests as small, focused and simple as possible.
Aiming for one assertion per test is one way to get there.
I have just been trying to polish up an extension to the Python unittest framework to make this kind of testing easier (at svn://colorstudy.com/trunk/DataTest) -- i.e., allow for small tests with different kinds of context. The next extension of your example is to feed a bunch of addresses and look for the expected results, but after adding this and other customizations to a TestCase subclass I find the framework becomes rather difficult to work with. I wrote about it a while ago (http://blog.colorstudy.com/ianb/weblog/2003/10/10.html), though I don't know how much applies to Java or Smalltalk (I got the impression that the Python implementation was modeled on those language's implementations, but maybe not).
There's really a wide variety of loops and other structures to a test, all of which would be nice to have accessible from outside, particularly when using TDD. When you start your code with lots of failing tests, you want to be able to focus on a few tests and knock them down one by one. The granularity of tests that I've worked with is usually at the level of individual subclasses of TestCase, but often I want to write twenty inputs for one test, and when all I have is one subclass, I end up rearranging the tests themselves so that the most interesting failure is the first failure. That's a pretty crude way to do it, obviously.
That said, writing tests can be a pain, and I don't want to burden myself with too much structure for tests. Disincentives too easily prey on my inclination to skip the tests ;) So decomposing the test into small pieces has to be easy and not require an excess of typing or clicking. Most unit testing examples have scary amounts of typing involved.
> I don't know how much applies to Java or Smalltalk > (I got the impression that the Python implementation was > modeled on those language's implementations, but maybe > not).
I've used PyUnit a bit but not unittest. If it's part of the family of "Beckian" test frameworks, then yes... it all started with Smalltalk (like so many good things). JUnit is the Java flavour... also authored initially by Kent Beck along with Erich Gamma (of GoF and now Eclipse fame).
> When you start your code with lots of failing tests
This is not the way to do TDD. You should only have one test failing at a time... the one you just wrote. If you have a bunch of tests failing, it means you've broken something and you are obliged to fix that before doing anything else (especially before checking in code)
> inputs for one test, and when all I have is one subclass, > I end up rearranging the tests themselves so that the most > interesting failure is the first failure.
One of the tenets of TDD is that tests are order independant.
Generally with TDD, you don't start with the most intertesting test, but rather the simplest... gradually specifying more complex behaviour until you have the functionallity you need.
> That said, writing tests can be a pain, and I don't want > to burden myself with too much structure for tests. > Disincentives too easily prey on my inclination to skip > p the tests ;) So decomposing the test into small pieces > has to be easy and not require an excess of typing or > clicking. Most unit testing examples have scary amounts > of typing involved.
Yes, writing tests is work. But a) writing them first is less work than writing them later, and b) writing them is less work than dealing with code that has no tests.
How do you refactor your test code to remove duplication that spawns several TestCases? I often find that I have more or less complex setUp-code that creates an environment for running my test. If I split my testcode in very many TestCases, as you describe, I will need the same setup-code for several TestCases
While this is good advice to follow, there may sometimes be exceptions.
One example would be assertions that make it easier to know what's happening with a failing test. For example, consider a test that verifies that a factory method works correctly. It will obviously check to see if the resulting object is set up as per spec, and this is the assertion that the test revolves around.
However, if the factory method isn't working, it may return a null object. Having a simple assertion in the test to verify it's not null (as well as a separate test to verify it's not null!) will help prevent a bit of mess if the factory method does break. After all, if the factory method breaks, you'll probably have several failing tests, and the "testFactoryDoesnNotReturnNull" test will be buried in all the rest of the noise.
Other than that (very minor) point, this is a great idea, Dave. Do you have this one covered in your book?
> I often find that I have more or less complex setUp-code > that creates an environment for running my test. If I > split my testcode in very many TestCases, as you describe, > I will need the same setup-code for several TestCases
Well, you could always create TestSuites to contain that setup code... or a test hierarchy, where you inherit setup code from a parent test case.
In another thread I recently posted an example of a small unit test from from I project I recently joined. Here's another (bigger but by no means the biggest). Is it a comprehensive test? Could be, but it's only one of several similar and overlapping tests in this particular TestCase. Am I going to refactor it? Absolutely not - I'm still getting my head around the eclipse editor they use here and besides I've got my own deadlines.
(It's a good indicator of the coding style in general though.)
Vince.
publicvoid testKioskObjectCompare()
{
System.out.println( "===testKioskCompare()===" );
// First a simple create
Kiosk test = new Kiosk("name","userid");
assert(test.getPrinterName().equals("userid"));
assert(test.getKioskName().equals("name"));
Kiosk test2 = new Kiosk("name","userid");
assert(test2.getPrinterName().equals("userid"));
assert(test2.getKioskName().equals("name"));
int i = test.compare(test,test2);
assert(i==0);
System.out.println("compare equal");
i=test.compareTo( test2 );
assert(i==0);
System.out.println("compareTo equal");
test2 = new Kiosk("aname","userid");
i = test.compare(test,test2);
assert(i>0);
i=test.compareTo( test2 );
assert(i>0);
System.out.println("compareTo greaterthan");
test2 = new Kiosk("zname","userid");
i = test.compare(test,test2);
assert(i<0);
i=test.compareTo( test2 );
assert(i<0);
System.out.println("compareTo lessthan");
test2 = new Kiosk("name","a");
i = test.compare(test,test2);
System.out.println("test="+test+" test2="+test2+" i="+i);
assert(i>0);
i=test.compareTo( test2 );
assert(i>0);
System.out.println("compareTo greaterthan");
test2.setPrinterName("z");
i = test.compare(test,test2);
assert(i<0);
i=test.compareTo( test2 );
assert(i<0);
System.out.println("compareTo lessthan");
System.out.println( "===after Kiosk compare test===" );
}
> One assertion per test! Read this and weep. > ... > Am I going to refactor it? Absolutely not - > I'm still getting my head around the eclipse editor they > use here and besides I've got my own deadlines.
Forget the test.
I can appreciate the kind of pressure that goes into "I've got my own deadlines" but, not knowing your situation, from here it sounds like you're new to the team or everyone thinks that way and there is no team. If the people around the project don't act like a team, its usually a bigger deal than any testing problem.
> One thing that would interest me: > > How do you refactor your test code to remove duplication > that spawns several TestCases? > I often find that I have more or less complex setUp-code > that creates an environment for running my test. If I > split my testcode in very many TestCases, as you describe, > I will need the same setup-code for several TestCases
You could factor common setup into a common baseclass.. sort of a super-fixture. Be careful not to put test methods there as they will be inherited by each concrete fixture class and run with each one.
Possibly. It's more of something to be thought provoking. I don't do this all the time in practice, but I do keep it in mind as a worthy goal and aspire to reach it as often as possible/reasonable.
> but I think > that asserting pre-conditions as well as post-conditions > can be valuable. e.g. > > AssertEquals(null, a.Addr1) > a.GetAddressParts("ADDR1$ADDR2$CITY IL 60563$COUNTRY") > AssertEquals("ADDR1", a.Addr1) > > It makes it clear which action, in this case > GetAddressParts, sets the value of Addr1.
I don't know how useful this is, but it can add communication value to the test. As you say it makes it very clear what the side effects of the call is. Maybe that's the key... it's good for making side effects explicit.
> While this is good advice to follow, there may sometimes > be exceptions.
Yes. There are always exceptions.
> However, if the factory method isn't working, it may > return a null object. Having a simple assertion in the > test to verify it's not null (as well as a separate test > to verify it's not null!) will help prevent a bit of mess > if the factory method does break. After all, if the > factory method breaks, you'll probably have several > failing tests, and the "testFactoryDoesnNotReturnNull" > test will be buried in all the rest of the noise.
I'm not completely convinced about the "buried in the rest of the noise" bit.
> Other than that (very minor) point, this is a great idea, > Dave.
Thanks.
> Do you have this one covered in your book?
Not explicitly.. well at least not at length (it's been a while since I wrote/read it :), but in I do talk about the idea of decomposing tests to a greater extent than most people do in practice.
> Am I going to refactor it? Absolutely not - > I'm still getting my head around the eclipse editor they > use here and besides I've got my own deadlines.
a) refactoring is easy/safe in Eclipse b) refactoring should improve the clarity of the tests and make it easier to understand. c) the author of this needs guidance on how to write tests... start some seminars.. official or not.
By aspiring to Ghandi's famous edict: "Be the change you're trying to create."
Nevertheless, if I had time to go and rewrite all the code that other's had written and I though I could improve, whilst still meeting my own deadlines then there'd be something seriously wrong with the deadlines. Not to mention the sheer vanity (and vainness) of joining a project and telling everyone else where they were going wrong whilst demonstrating an impressive incapability to get eclipse set up properly.
I'll need to have my feet under the table here a bit longer before I venture down that road.
Vince.
PS: Things I have had my fingers rapped for in the past: 1) Writing PL1 using lower case. PL1 code doesn't distinguish between upper and lower case (a significant failing in Java) but this project wrote everything in upper case. They were so used to it that they found the lower case code hard to read. I was even asked if I'd tried to compile it, they were so sure it was wrong. 2) (Same project) Writing PL1 functions that returned values. The project had never used functions only subroutines that didn't return values but modified one or more of the parameters passed in. I was accused showing off by writing deliberately obscure code!
So you'll maybe appreciate why I have "that kind of attitude".
Flat View: This topic has 22 replies
on 2 pages
[
12
|
»
]