Re: A Set of Unit Testing Rules
Posted: Sep 17, 2005 10:02 AM
> ... you'd probably feel really
> good about not testing your applications ability to handle
> partial failure. It may feel like it's slowing you down
> early on, but at some point, you really need to have real
> world tests that put your application under really heavy
> loads to test concurrent data structure access.
> You also
> need to have fault injection capabilities to stress the
> failure handling code to make sure idempotent behaviors
> are true to form. There's lots of reasons to interface to
> a real-world environment. It might not be in your unit
You're right (and this is Michael's point) when you say that you don't want load tests or integration tests running with your unit tests. Imaginge that every time you wanted to compile you had to set up a test database, some network services, etc., etc. You'd probably be able to compile once a day!
There is also a philosophical question you're touching on: When is testing a developer function, and when is it a software tester function?
Unit tests are meant to be written by a developer, and run numerous times a day, as the developer writes new code. The tests are a way for the developer to assert that the code they've written works the way they think it should. Obviously, the tests will be incomplete unless the developer thinks of every test case up front, and they often don't. Typically, the developer adds these new tests as he fixes the bugs found by the software tester.
A software tester will devote themselves to the more complex testing tasks, including setting up test environments that mimic real-world conditions. (Note that when I say software tester, I'm talking about someone who is as adept at breaking code as the developer is a writing it. I'm not talking about someone who's never programmed in their life, and who just clicks buttons all day.) Their tests will be far more complete than the developer's unit tests, and will be the true test of the software's quality.
As tools become more powerful, developers will find themselves with more time to do "true" testing, so the line between developer and software tester is blurring. Ultimately, I think it'd be great if more developers really understood the various ways their code could break, and I think that developing unit tests may help them learn this. For now, most programmers don't have the testers mindset and so a good software tester is necessary.
> ... but you should have a plan, during design and
> coding on how you are going to instrument and inject for
> real-world testing.
I think those who advocate TDD would say that code that is easy to unit test is also easier to test in other ways.