The Artima Developer Community
Sponsored Link

Weblogs Forum
Is Test-First Development an Impediment to Creative Flow?

41 replies on 3 pages. Most recent reply: May 1, 2006 10:18 AM by piglet

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 41 replies on 3 pages [ « | 1 2 3 ]
Dan Perl

Posts: 28
Nickname: nanov
Registered: Sep, 2004

Re: Is Test-First Development an Impediment to Creative Flow? Posted: Feb 2, 2005 11:30 AM
Reply to this message Reply
Advertisement
> Is there a test to ensure the list's order? If so, then
> it is an enforced element of the features contract.
>
> If no test exists, the ordering is coincidental and cannot
> be relied upon.

It's been stated by other people but I will get my 2 cents in anyway. I disagree with you. Tests are not the only way to design or to ensure the correctness of the code. There is also code inspection, for instance. The real advantage of automatic tests is not in ensuring correctness but in providing confidence and "courage" for applying frequent refactorings to the code. However, confidence and courage can be achieved in other ways and frequent refactorings are not always necessary. In other words, TDD is important for code that you know that it will change often. It is also a great way to do bottom-up design if that approach is best for the problem. But it is not the only way.

Ged Byrne

Posts: 22
Nickname: gedb
Registered: Nov, 2003

Re: Is Test-First Development an Impediment to Creative Flow? Posted: Feb 2, 2005 12:15 PM
Reply to this message Reply
There are others ways, I don't believe anybody has disputed this.

I will stick my neck out and say that there is no way to ensure correctness with code alone.

You say that 'Code Inspection' can ensure correctness. This can only work if you have another statement of intent against which the code can be compared against.

On its own, Code is always correct. It always does exactly what it is told to do.

You therefore need to have something separate from the code that describes in another way what it is the code is supposed to do.

There are, of course, many ways to do this. In the past this alternative model was either captured in a written specification or within the minds of the developers.

The problem is that Developers move on, and written specs move out of sync. Even when the intent is expressed as comments in the source, there is no guarantee that the update always occurs on both sides.

The best solution is to have the intention described in two places that can be automatically compared. TDD achieves this by using tests. Another approach is Eiffel's Design By Contract. I'd love to learn of more ways.

This still leaves the problem that to be effective your tests or contracts must provide full coverage. Always writing the test or contract before writing the code is one way to do this. I can't think of any other way, but I'd love to hear of them.

Eric Armstrong

Posts: 207
Nickname: cooltools
Registered: Apr, 2003

Re: Is Test-First Development an Impediment to Creative Flow? Posted: Feb 2, 2005 12:58 PM
Reply to this message Reply
> The real advantage of automatic tests is ...
> in providing confidence and "courage" for
> applying frequent refactorings to the code.
>
Have to agree with that.

> However, confidence and courage can be achieved in
> other ways ...
>
I don't see how. I spent many years as a professional programmer, and I never found any other way to be totally confident that was working before was still working after making changes. More often, changes that seemed like a good idea at the time wound up having unforeseen consequences.

Of course, that was before there were objects, too, so there was no encapsulation and many dependencies. The state of the art may have improved in other ways, as well, so maybe there's something I don't know about it. But I can tell you that all coders of my generation approached working code with extreme reluctance. We didn't reoganize unless it was absolutely necessary. We called it a "rewrite", and knew that we would be back to square 1 on. To do that for a commerical application would require a 3 month trial period before we would trust it enough to use it in place of the new version.

Now, with Fowler's ground-breaking work in incremental refactoring, a wonderful stockpile of patterns, and (above all) a solid test suite that lets you test every step of the refactoring process, and *know* that what was working before is still working.

If there is some way other than unit tests to acquire confidence like that, I'd love to hear of it.

Ernie Varitimos

Posts: 38
Nickname: erntheburn
Registered: May, 2003

Re: Is Test-First Development an Impediment to Creative Flow? Posted: Feb 4, 2005 1:16 PM
Reply to this message Reply
The problem that most people seem to ignore with writing tests is that tests themselves are code that sometimes require creative solutions to implement. Sometimes it is impossible to write a test first without a complete understanding of potential solutions. That is, requirements hardly ever provide complete insight to the solution.

Eric Hodel

Posts: 660
Nickname: drbrain
Registered: Mar, 2006

Re: Is Test-First Development an Impediment to Creative Flow? Posted: Apr 27, 2006 11:00 PM
Reply to this message Reply
* The activity must present just the right amount of challenge: If the activity is too hard, we become frustrated; if too easy, boredom ensues.
* There must be clear and unambiguous goals in mind.
* There needs to be clear-cut and immediate feedback about the activity's success.
* Finally, our focus during the activity must center on the present - the activity itself.

My tests will quickly tell me when what I'm doing is too hard. Without tests I can run into too hard abruptly which is very un-fun. Tests also keep me away from too easy. If the tests I'm writing are too easy I'm probably testing incorrectly. (Or I need to refactor because I'm doing gruntwork.)

Make the next test pass is my clear and unambiguous goal.

If I don't get clear-cut and immediate feedback my tests are bad. Either they aren't the right tests or they run too slow.

Making the tests pass is the focus of my activity.

I've had longer, more productive instances of "Creative Flow" when doing TDD because of all the feedback tests give me to keep me "in the zone". Without I might be able to go a few hours. With TDD I've stayed in the zone for days.

Ryan Davis

Posts: 651
Nickname: zenspider
Registered: Oct, 2004

Re: Is Test-First Development an Impediment to Creative Flow? Posted: Apr 28, 2006 3:38 AM
Reply to this message Reply
I think most of this dialog is utter nonsense as they are quibbling over TDD vs not and ignoring the actual title of this article. Creative Flow (aka The Zone) wrt TDD is what is at question here. As Eric stated, he's had plenty of zone-time doing TDD sessions. So have I. So the answer to the question of the article is probably "it depends on the person". In the case of the author, it seems to be "yes" and for some others "no".

Maybe folks should try seeing past their religion in order to address the actual topic at hand.

Greg Jorgensen

Posts: 65
Nickname: gregjor
Registered: Feb, 2004

Re: Is Test-First Development an Impediment to Creative Flow? Posted: Apr 28, 2006 5:34 AM
Reply to this message Reply
I find that TDD sometimes distracts me from solving the programming problem. If I don't understand the specs writing tests first to nail them down is useful. Otherwise writing tests feels like multiplying the work and dividing my focus and energy.

The two or three times in my career that I've written something really cool or hard, it's always come out of one or a few programming sessions when the code just flowed out. That's always happened after I've spent considerable time thinking the problem through. My unit tests (written after the code) and later full tests reveal minor bugs, but I always recognize the cause of the bugs right away.

I suspect that most well-known programming achievements are the output of one or a small number of minds working in an extended flow state. I don't think the best programmers reject formal techniques; I think they've internalized the processes and don't always need to explicitly write the specs or the tests. And I think that is part of the creative flow: solving enough of the problem in your head so the code comes almost automatically.

I've learned from lots of programming techniques, going back to Yourdon and Wirth in the '70s and '80s and more recently from OOP and XP. There are always plenty of good ideas and tools, but watch out when they harden into rituals. Programming methodologies can be like diets: if you succeed it's because of the methodology, but if you don't it's because you didn't follow it carefully enough.

piglet

Posts: 63
Nickname: piglet
Registered: Dec, 2005

Fine-grained low-level tests considered harmful Posted: Apr 28, 2006 11:30 AM
Reply to this message Reply
The rule "Write your tests before you write your code", if taken literally, is bullshit (strictly speaking, it doesn't even make sense - after all, tests *are* code). You can't write tests (especially if you are talking of fine-grained, low-level tests) unless you already know your functional code structure in detail. IOW the interfaces must be known before they are coded. But getting the interfaces right is hard (often it is *the hard part*) and usually needs several cycles of trial and error. Unit tests are great to verify implementations but they don't help to get the interfaces right. I honestly don't understand how one can seriously work that way. And I do agree with Frank that programming is a creative process that simply isn,t compatible with that sort of rigid rules.

My tests are usually higher-level tests that test a recognizable chunk of functionality (usually at least on class level, not on method level) and that are themselves short. I don't believe that it makes sense to produce test code of the same or even greater order of magnitude than the functional code to be tested, for the well-known reasons:
- Test code needs to be debugged and maintained just like any other code.
- Fine grained tests are really testing implementation details, which is unnecessary and hinders refactoring.
- Test code is client code. If you need lots of client code just to *use* your functional classes, maybe your interfaces are not well-designed. If your functional classes expose lots of methods that each need to be tested, maybe there are redundancies.

I try to write the functional code in a way that already anticipates the testing. As an example, you often have operations that are invertible, e.g. serializing and deserializing objects from xml. The natural test then would be to make the operation, invert it (by calling the functional code) and test for equality. This test is extremely short and doesn't need to be modified each time some implementation detail changes.

piglet

Posts: 63
Nickname: piglet
Registered: Dec, 2005

Re: Fine-grained low-level tests considered harmful Posted: Apr 28, 2006 11:41 AM
Reply to this message Reply
To continue this line of thought, what I am trying to say is that testing can often focus on the invariants in the structure to be tested. In the above example, the invariant is that the inverse of an operation must bring you back to the starting point.

Keith Gaughan

Posts: 17
Nickname: kgaughan
Registered: Feb, 2003

Re: Fine-grained low-level tests considered harmful Posted: Apr 28, 2006 3:32 PM
Reply to this message Reply
piglet writes:

> The rule "Write your tests before you write your code", if
> taken literally, is bullshit (strictly speaking, it
> doesn't even make sense - after all, tests *are* code).

Yes, if taken literally, but you'd only do that if you suffered from severe Asperger's syndrome.

Anyway, tests, in its TDD meaning, are more like calibrations.

> You can't write tests (especially if you are talking of
> fine-grained, low-level tests) unless you already know
> your functional code structure in detail.

No. That assumes that testing is your aim rather than design. The regression suite you end up with at the end is simply a pleasant side-effect of the process. Sure, it's good to go in with a clear idea of the domain you're coding to, but "know[ing] your functional code structure in detail"? That's not a necessity. It's all about design. If you misstep, refactor.

> IOW the interfaces must be known before they are coded.

No. Only the part under test needs to be known. It's good to have a rough idea of what your interfaces will be, but until you make them public, they're far from fixed and are open to change.

The tests, are in part, an attempt to codify the interface.

> But
> getting the interfaces right is hard (often it is *the
> hard part*) and usually needs several cycles of trial and
> error. Unit tests are great to verify implementations but
> they don't help to get the interfaces right.

Again, you're mistaken if you're talking about TDD, which is a design activity. However, with post-hoc unit testing, you're correct.

Part of the point of TDD is to arrive at a point where you have easily testable interfaces, and easily testable interfaces are easily usable ("right") by definition.

> I honestly
> don't understand how one can seriously work that way.

Different strokes for different folk.

> And
> I do agree with Frank that programming is a creative
> process that simply isn,t compatible with that sort of
> rigid rules.

They're not that rigid! There's nobody out there saying that it's the one true way, it's just a practice that works well for many people in many contexts. If your context isn't suited to it, fine.

> My tests are usually higher-level tests that test a
> recognizable chunk of functionality (usually at least on
> class level, not on method level) and that are themselves
> short.

Ditto.

> I don't believe that it makes sense to produce test
> code of the same or even greater order of magnitude than
> the functional code to be tested, for the well-known
> reasons:
>
> - Test code needs to be debugged and maintained just like
> any other code.

It also tends to be more trivial, meaning it has a much lower maintenance overhead than regular code. If your test are complex, there's probably something wrong with your interfaces: it's a code smell.

> - Fine grained tests are really testing implementation
> details, which is unnecessary and hinders refactoring.

Then, by that token, whenever your code is used elsewhere in the codebase, refactoring is hindered.

But that's not true, is it.

And the same goes for tests. I rarely have to do any special refactoring for tests, and less than I have to do for regular code as my IDE takes care of most of the work automatically. And whenever I have to do more invasive refactoring by hand, the tests help me ensure that I don't get any nasty regressions, which saves me time I'd otherwise spend debugging.

It's a kind of proactive laziness.

> - Test code is client code. If you need lots of client
> code just to *use* your functional classes, maybe your
> interfaces are not well-designed. If your functional
> classes expose lots of methods that each need to be
> tested, maybe there are redundancies.

...and identifying that kind of thing is part of the point. I repeat, TDD is a design method, not a testing method.

> I try to write the functional code in a way that already
> anticipates the testing. As an example, you often have
> operations that are invertible, e.g. serializing and
> deserializing objects from xml. The natural test then
> would be to make the operation, invert it (by calling the
> functional code) and test for equality. This test is
> extremely short and doesn't need to be modified each time
> some implementation detail changes.

Yup, but you'd need tests that check the various edge cases (such as what happens when you, say serialise an array with no elements, one element, or many elements, or if you attempt to serialise null somewhere.

For me, a test is a test, no matter whether it's calibrating something high-level or low-level. If a piece of code is trivial, testing it's not quite as important, but if it does something complex, no matter how high or low-level it is, I want to make sure it works.

Keith Braithwaite

Posts: 1
Nickname: keithb
Registered: Apr, 2006

Re: Is Test-First Development an Impediment to Creative Flow? Posted: Apr 28, 2006 6:55 PM
Reply to this message Reply
Strange that my experience of test-first (and test-driven) programming as regards flow is the exact point-for-point opposite of Frank's, as I describe at http://peripateticaxiom.blogspot.com/2006/04/test-first-and-flow.html

piglet

Posts: 63
Nickname: piglet
Registered: Dec, 2005

Re: Fine-grained low-level tests considered harmful Posted: May 1, 2006 10:18 AM
Reply to this message Reply
Keith Gaughan:

"If your test are complex, there's probably something wrong with your interfaces: it's a code smell."

I agree with that. But remember that I was taking issue with the idea that it is a good thing to produce large volumes of test code (some say that test code should be several times the volume of the functional code). I say that test code volume should be small.

"> IOW the interfaces must be known before they are coded.

No. Only the part under test needs to be known. It's good to have a rough idea of what your interfaces will be, but until you make them public, they're far from fixed and are open to change."

"a rough idea of what your interfaces will be" isn't enough because you have to code to that interfaces. If you always write the test first, then whenever you change some signature, you have to refactor not only the functional code but also the tests. I just don't see why I should to twice the work just to fulfill some dogma. In the early stages of designing a certain interface, when changes are frequent, it may be a hindrance to already have to maintain test code, especially fine-grained low-level tests.

"Yes, if taken literally, but you'd only do that if you suffered from severe Asperger's syndrome."

Well, the rule is proclaimed as an absolute dogma isn't it. They don't say "write the tests first whenever you feel like it" or "it may sometimes be a good idea to start with writing a test". So don't blame me if I take issue with the rule as it stands.

Flat View: This topic has 41 replies on 3 pages [ « | 1  2  3 ]
Topic: Is Test-First Development an Impediment to Creative Flow? Previous Topic   Next Topic Topic: Prefactoring and Refactoring


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us