Weblogs Forum - Post-TDD

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

Weblogs Forum
Post-TDD

26 replies on 2 pages. Most recent reply: Sep 28, 2005 3:14 PM by Mike Petry

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 26 replies on 2 pages [ 1 2 | » ]

Christopher Diggins

Posts: 1215
Nickname: cdiggins
Registered: Feb, 2004

Post-TDD (View in Weblogs)

Posted: Aug 31, 2005 11:35 AM

Summary
Test Driven Development (TDD) represents the state of the art of development processes, but what's next?

It is hard to argue that Test Driven Development (TDD) is not the state of the art of development methodologies, but where do we go from there? Well you eliminate the tests of course.

If we look at a software design as satisfying a set of tests, then by simply reducing the number of tests neccessary is a logical way to streamline the development process. I propose that perhaps the evolutionary development of TDD is Test Elimination Driven Design (TEDD?).

Clearly tests are crucial, but they are also undeniably time-consuming to write, and execute. There is also a strong correlation between the number of tests needed to verify the correctness of a design, and the complexity of the design.

Reducing tests can only be done when you know something to be correct, or by reducing the design complexity. My approach for a long time has been to rely on code which is verifiably correct, and to use implicit testing techniques. For instance I would write code which reused small functions, even when it would lead to more code and less efficient designs. This put an extra strain on the function which would lead to an implicit "stress-test". This helped to reveal flaws more quickly.

I guess what I am getting at is that reuse of verified code (demonstrably correct through: explicit tests, implicit stress-tests, proofs, assertions, and simple inspection), reduces the need for certain tests and thus can streamline the development process.

Michael Feathers

Posts: 448
Nickname: mfeathers
Registered: Jul, 2003

Re: Post-TDD

Posted: Aug 31, 2005 12:00 PM

I've thought about this one a lot and I just don't think that tests will go away. There's the "double entry bookkeeping" aspect of them that seems hard to achieve in any other way.

I do think, though, that there is a next step and it involves altering languages and increasing the range of refactoring tools so that programs can become representation invariant. By that, I mean that you should be able to check out a piece of code and apply an incredible arsenal of manipulations against it, whatever it takes to make it easier to understand and better structured for the change you are about to make, and know that it is exactly the same program as the one you check out; it behaves the same.

When you have a test-infected project, one of the side effects is that you are able to treat structure and correctness orthogonally. I think there is more that we can do to push in that direction.

Mike Petry

Posts: 34
Nickname: mikepetry
Registered: Apr, 2005

Re: Post-TDD

Posted: Aug 31, 2005 1:56 PM

An intended side-effect of TDD is that it forces code to be testable. Code that is NUnit testable is usually decoupled. I personally think that the testability of system components is a chief criteria of a successful design and usually indicates that all the good "ilities" are present.

Michael Feathers

Posts: 448
Nickname: mfeathers
Registered: Jul, 2003

Re: Post-TDD

Posted: Aug 31, 2005 3:54 PM

> Reducing tests can only be done when you know something to
> be correct, or by reducing the design complexity. My
> approach for a long time has been to rely on code which is
> verifiably correct, and to use implicit testing
> techniques. For instance I would write code which reused
> small functions, even when it would lead to more code and
> less efficient designs. This put an extra strain on the
> function which would lead to an implicit "stress-test".
> This helped to reveal flaws more quickly.
>
> I guess what I am getting at is that reuse of verified
> code (demonstrably correct through: explicit tests,
> implicit stress-tests, proofs, assertions, and simple
> inspection), reduces the need for certain tests and thus
> can streamline the development process.

The thing is, we can skimp, but tests are at least as important tomorrow as they are today. The great thing about having them around is the mobility they enable. Repeatedly desk checking or inspecting code that we could have tests for is more of a waste of time than writing the tests. It's O(n) rather than O(1).

We're all in the automation business; it's great when we can serve ourselves too.

Vesa Karvonen

Posts: 116
Nickname: vkarvone
Registered: Jun, 2004

Re: Post-TDD

Posted: Aug 31, 2005 4:12 PM

> I guess what I am getting at is that reuse of verified
> code (demonstrably correct through: explicit tests,
> implicit stress-tests, proofs, assertions, and simple
> inspection) [...]

I'd like to point out that the term "verification" as in "program verification" is usually associated with rather strict requirements (formal specifications and proofs) that ad-hoc stuff like testing, assertions, (human) inspection, and even (casual) proofs do not quite fulfill.

(http://en.wikipedia.org/wiki/Program_verification)

P.S. I'm not criticizing TDD in any way here.

Vincent O'Sullivan

Posts: 724
Nickname: vincent
Registered: Nov, 2002

Re: Post-TDD

Posted: Sep 1, 2005 12:13 AM

> Clearly tests are crucial, but they are also undeniably
> time-consuming to write, and execute.

Its worth pointing out that these are coding time hits, not run time (since the tests are not part of the product). In any serious product development, coding time is a small percentage of the overall development. Taking a hit there to produce demonstrably correct code is not a worry (unlike the hit taken as a result of developing code that can't be shown to work).

> Reducing tests can only be done when you know something to
> be correct, or by reducing the design complexity.

No, you can't reduce the tests when you 'know something to be correct' - for two reasons: One, you never really know something is correct, you can only surmise it; two, Tests don't tell you something is correct only that it was correct (with respect to that test) at the time and under the circumstances when the test was run. Once the tests are removed, that test run date recedes into the past along with the reliance that can be placed upon it, the applicable circumstances, etc. Hence the TDD requirement to run all tests as frequently as possible.

> My
> approach for a long time has been to rely on code which is
> verifiably correct, and to use implicit testing
> techniques. For instance I would write code which reused
> small functions, even when it would lead to more code and
> less efficient designs. This put an extra strain on the
> function which would lead to an implicit "stress-test".

Is this different to 'optimistic coding' (i.e. treating the fact that something didn't go wrong as being equivalent to saying it is right)?

> I guess what I am getting at is that reuse of verified
> code (demonstrably correct through: explicit tests,
> implicit stress-tests, proofs, assertions, and simple
> inspection), reduces the need for certain tests and thus
> can streamline the development process.

Which tests qualify as the "certain tests" that can be removed? Once you have removed those tests, how do you demonstrate that changes elsewhere in the system do not impact on that code?

Interesting that you refer to 'simple inspection' as a test method. Surely once you've made such an inspection and corrected the code, if necessary, then simple inspection no longer works because you can no longer simply see any problems with the code. This therefore makes the tests more necessary - to catch the unseen problems.
Vince.

Kondwani Mkandawire

Posts: 530
Nickname: spike
Registered: Aug, 2004

Re: Post-TDD

Posted: Sep 1, 2005 4:17 AM

> An intended side-effect of TDD is that it forces code to
> be testable.

Is that necessarily a good thing? I mean seriously
why add extra parameters to a method which otherwise
would have less? For instance there are some JUnit
tests I have implemented in which I had to introduce
extra parameters - causing me to be unfond of JUnit,
I thought it was a drag, and added slighter bulk to
code, but that's just a matter of opinion.

Kondwani Mkandawire

Posts: 530
Nickname: spike
Registered: Aug, 2004

Re: Post-TDD

Posted: Sep 1, 2005 4:19 AM

> > An intended side-effect of TDD is that it forces code
> to
> > be testable.
>
> Is that necessarily a good thing?

My baad since you listed it as a side-effect, I guess
we share opinions to an extent.

Michael Feathers

Posts: 448
Nickname: mfeathers
Registered: Jul, 2003

Re: Post-TDD

Posted: Sep 1, 2005 4:45 AM

> > An intended side-effect of TDD is that it forces code
> to
> > be testable.
>
> Is that necessarily a good thing? I mean seriously
> why add extra parameters to a method which otherwise
> would have less? For instance there are some JUnit
> tests I have implemented in which I had to introduce
> extra parameters - causing me to be unfond of JUnit,
> I thought it was a drag, and added slighter bulk to
> code, but that's just a matter of opinion.

If you're game, there's a very good thread about this on the tdd mailing list (testdrivendevelopment@yahoogroups.com) The subject is: "Tests of objects working together - are they unit tests?"

The core, though, is that TDD is more about design than testing. When you write tests first, it makes coupling painful, so you avoid coupling. The sad thing is, most of us aren't tuned to coupling problems, so we couple and make our code a little more tangled than it could be.

In the long run, the things that we rationalize away, saying, "oh, that's no problem" are the things that make code hard to reuse or change latter." TDD makes you think more, as you're implementing, and often you end up creating more finely grained classes and methods. The end result is worthwhile.

Adding extra parameters to a method to make it testable is an "after the fact" testing trick. Often with TDD you don't have to use that trick, because you find a better ways to get the method what it needs or end up with a class breakdown where the problem doesn't come up.

John D. Mitchell

Posts: 244
Nickname: johnm
Registered: Apr, 2003

Re: Post-TDD

Posted: Sep 1, 2005 9:15 AM

> It is hard to argue that Test Driven Development (TDD) is
> not the state of the art of development methodologies, but
> where do we go from there?

Well, I guess that depends on where you draw the line between "state of the art" and "good, popular practice". :-)

> Well you eliminate the tests of
> course.

And how do you know which tests to eliminate? Heck, where did you come up with all of the tests in the first place that you can now talk about eliminating?

> If we look at a software design as satisfying a set of
> tests, then by simply reducing the number of tests
> neccessary is a logical way to streamline the development
> process. I propose that perhaps the evolutionary
> development of TDD is Test Elimination Driven Design
> (TEDD?).

Basically, all you're talking about here, if I understand you correctly, is devolving the development process to the creation of the necessary and sufficient "test" cases. That's the same thing that the MDA guys are trying to do from the "specification" side of the terminology fence.

> Clearly tests are crucial, but they are also undeniably
> time-consuming to write, and execute. There is also a
> strong correlation between the number of tests needed to
> verify the correctness of a design, and the complexity of
> the design.

You're conflating TDD with "testing". TDD is fundamentally a generative/constructivist approach to software design and implementation which has the direct side-effect of creating some level of quality assurance.

Correctness testing is almost orthogonal to TDD. Basically, they are complimentary -- correctness testing is primarily deconstructivist.

Of course, we need both the constructivist and deconstructivist.

John D. Mitchell

Posts: 244
Nickname: johnm
Registered: Apr, 2003

Checks and Balances

Posted: Sep 1, 2005 9:22 AM

> I've thought about this one a lot and I just don't think
> that tests will go away. There's the "double entry
> bookkeeping" aspect of them that seems hard to achieve in
> any other way.

Indeed. In fact, there's really three facets: code, tests, and specs. All three are needed.

This is part of why things like CASE/MDA are doomed to failure as they try to smear all three together. The whole point is that each of the three provides checks and balances against the other two parts of the trio.

> I do think, though, that there is a next step and it
> involves altering languages and increasing the range of
> refactoring tools so that programs can become
> representation invariant. By that, I mean that you
> should be able to check out a piece of code and apply an
> incredible arsenal of manipulations against it, whatever
> it takes to make it easier to understand and better
> structured for the change you are about to make, and know
> that it is exactly the same program as the one you check
> out; it behaves the same.

Hmm... You seem to be presuming that those languages allow for a multiplicity of forms to say the same thing. Why?

Doesn't the adage "Do one thing and do it well" apply to (domain specific) language design?

> When you have a test-infected project, one of the side
> effects is that you are able to treat structure and
> correctness orthogonally. I think there is more that we
> can do to push in that direction.

Indeed.

Christopher Diggins

Posts: 1215
Nickname: cdiggins
Registered: Feb, 2004

Re: Post-TDD

Posted: Sep 1, 2005 9:27 AM

> You're conflating TDD with "testing". TDD is fundamentally
> a generative/constructivist approach to software design
> and implementation which has the direct side-effect of
> creating some level of quality assurance.
>
> Correctness testing is almost orthogonal to TDD.
> Basically, they are complimentary -- correctness testing
> is primarily deconstructivist.
>
> Of course, we need both the constructivist and
> deconstructivist.

This is an interesting and accurate observation. I am conflating TDD and Correctness Testing. However the majority of people on both sides of the TDD advocacy fence make this mistake. However I believe that if TDD is not used in conjunction with CT then it becomes virtually useless. In other words if TDD is only really useful if aggressively used to pursue quality, otherwise an arbitrary ad-hoc development method may just as easily produce better results (for instance: an experienced and smart programmer just coding in their naturally evolved personal methodology).

John D. Mitchell

Posts: 244
Nickname: johnm
Registered: Apr, 2003

Testing vs. TDD

Posted: Sep 1, 2005 10:00 AM

[...TDD vs. Testing, constructivist and deconstructivist...]

> This is an interesting and accurate observation. I
> am conflating TDD and Correctness Testing. However
> the majority of people on both sides of the TDD advocacy
> fence make this mistake.

Just because stupidity is popular doesn't make it a good idea to promote it. :-( :-)

The distinction is critical to make if there's ever going to be a "post-TDD" evolution.

> However I believe that if TDD is
> not used in conjunction with CT then it becomes virtually
> useless.

Hogwash. TDD induces a lot of improvements over the status quo development methodologies including highly-iterativeness, use-driven design, granularity reduction, etc. along with getting a lot of developers to open up to the notion that tests have a direct, positive benefit to them.

> In other words if TDD is only really useful if
> aggressively used to pursue quality, otherwise an
> arbitrary ad-hoc development method may just as
> easily produce better results (for instance: an
> experienced and smart programmer just coding in their
> naturally evolved personal methodology).

Depends on that "personal methodology". There's a huge and hugely slippery slope in the generalization that you're making. Part of the problem is the so-called "Lake Wobegone Effect" -- every developer thinks their process is (way) above average. Of course, looking at the industry as a whole, it's obvious that not only is the average outcome dismal but that even the above average outcomes are crappy.

To tie this back into to your original question, part of the problem with "TDD" is the unfortunate name. It talks about "how" and that confuses a lot of people with all of the "what" that's really going on and the point of it all.

Michael Feathers

Posts: 448
Nickname: mfeathers
Registered: Jul, 2003

Re: Post-TDD

Posted: Sep 1, 2005 10:21 AM

> > Correctness testing is almost orthogonal to TDD.
> > Basically, they are complimentary -- correctness
> testing
> > is primarily deconstructivist.
> >
> > Of course, we need both the constructivist and
> > deconstructivist.
>
> This is an interesting and accurate observation. I
> am conflating TDD and Correctness Testing. However
> the majority of people on both sides of the TDD advocacy
> fence make this mistake. However I believe that if TDD is
> not used in conjunction with CT then it becomes virtually
> useless. In other words if TDD is only really useful if
> aggressively used to pursue quality, otherwise an
> arbitrary ad-hoc development method may just as
> easily produce better results (for instance: an
> experienced and smart programmer just coding in their
> naturally evolved personal methodology).

I think we have two points here.. one is that TDD is about Correctness Testing (CT) and the other is that TDD is only useful is when pursuing quality. I disagree with the first and agree with the second.

CT asks the question: does this software behave correctly, so the assumption is that the software is already there. TDD is really incremental specification rather than post-hoc testing.

I think there is a way in which TDD does CT, and that's at the derivative. When you have a large existing set of tests, they act as change detectors, and assuming the existing tests depict "correct" behavior, they act as CT checks.

For what it's worth, I think we'd be better, as an industry, if we concentrated more on change verification than program verification: http://www.artima.com/weblogs/viewpost.jsp?thread=79267

Michael Feathers

Posts: 448
Nickname: mfeathers
Registered: Jul, 2003

Re: Checks and Balances

Posted: Sep 1, 2005 10:34 AM

> > I've thought about this one a lot and I just don't
> think
> > that tests will go away. There's the "double entry
> > bookkeeping" aspect of them that seems hard to achieve
> in
> > any other way.
>
> Indeed. In fact, there's really three facets: code,
> tests, and specs. All three are needed.
>
> This is part of why things like CASE/MDA are doomed to
> failure as they try to smear all three together. The
> whole point is that each of the three provides checks and
> balances against the other two parts of the trio.

I wonder, though. There are many teams which merge specification and test and end up doing very well. The "double check seems" to do okay, so the "triple check" seems to need justification.

> > I do think, though, that there is a next step and it
> > involves altering languages and increasing the range of
> > refactoring tools so that programs can become
> > representation invariant. By that, I mean that
> you
> > should be able to check out a piece of code and apply
> an
> > incredible arsenal of manipulations against it,
> whatever
> > it takes to make it easier to understand and better
> > structured for the change you are about to make, and
> know
> > that it is exactly the same program as the one you
> check
> > out; it behaves the same.
>
> Hmm... You seem to be presuming that those languages allow
> for a multiplicity of forms to say the same thing. Why?

I think they all do.. I can say 1 + 3, or 2 + 2, and that's just a base case. I think every language provides an infinite number of ways of writing most programs.

> Doesn't the adage "Do one thing and do it well" apply to
> (domain specific) language design?

Yeah, but that's at the language level. Sort of, if a language allows a + 1 and a++, it isn't as simple as it could be.

I think that if there is a way of mutating among most of the many forms that the same computation can be expressed in, in a language, we'd have an incredible tool. You could take the nastiest code, dust it off, and resurrect it, knowing that it does exactly what it did before. I don't know how much of this is computationally feasible, though. I suspect there are some points 'a' and 'b' that you simply can not refactor across and know you've done it right via code analysis.

Flat View: This topic has 26 replies on 2 pages [ 1 2 | » ]

Previous Topic

Next Topic