The Artima Developer Community
Sponsored Link

Weblogs Forum
Test Driven Development?!

25 replies on 2 pages. Most recent reply: Aug 18, 2007 11:25 PM by Greg Helton

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 25 replies on 2 pages [ « | 1 2 ]
Michael Feathers

Posts: 448
Nickname: mfeathers
Registered: Jul, 2003

Re: Test Driven Development?! Posted: Nov 23, 2005 6:36 PM
Reply to this message Reply
Advertisement
Newsflash: Brian Button posts that MS has removed the TDD page:

http://www.agileprogrammer.com/oneagilecoder/archive/2005/11/23/9769.aspx

John M Camara

Posts: 15
Nickname: camara
Registered: Dec, 2004

Re: Like liver, I don't like TDD Posted: Nov 28, 2005 11:49 PM
Reply to this message Reply
In my opinion anyone who feels that TDD is not for them needs to give it a try. If after given it I try and you still have the same feelings, give it a try again, and again until you find enlightenment. I guarantee that once you get it you wont look back.

> ...However I write better more flexible OO code with rapid iterative throw away
> prototypes... Thinking about the problem by writing tests first may make sense
> for some people, I just dont think that way.

Hmm.

think of some feature
+ write a test case
+ write code = rapid prototype

repeat again = rapid iterative prototype

repeat again
+ refactor = rapid iterative throw away prototype (not quite, but close enough)

> What I take from the premise of the original article was that "If you can write
> your tests first, but then your are on the slippery slope to solving the
> problem a certain way." That's why I view it as "less creative, and more
> methodical/scientific".

On a slippery slope when using TDD. This couldn't be further from reality. With TDD the solution evolves over time and changing directions are made as easy as trying an idea and and fixing all the test cases that fail. Without TDD, be ready for higher levels of stress, longer refactoring sessions, and those sneaky bugs that are introduced and will bite you in the future when you make such changes in directions.

> TDD seems to assume you have specific solution in mind (i.e. hypothesis) and you
> want to somewhat "scientifically prove" it works by finding all of the ways that it will
> break (In logic I belive this is the process of proof by contradiction)...

You have misconceptions about TDD as it is not proof by contradiction.

> The problem/inflexibility for me is it assumes "You have a solution/hypothesis in mind"...
> What if my original hypothesis was incorrect or needs to be tweaked... (this causes you
> to technically redo all of the work done to validate your original hypothesis)

Not true see my previous comment on slippery slope.

I don't believe TDD is some panacea but it is a powerful tool that all developers should learn to master. You know you are starting to master TDD when you finally realize that TDD is all about design with testing as a nice side effect. We all know that the natural order of the software development cycle is design, code, and test (plus refactor). So design is first (write test case) and test is last (run test cases). Design can also be achieved by modeling and quite often I do a little modeling and then do a little TDD and then more modeling followed by more TDD. When I model I tend to use a very light weight approach where I only show classes (just names) and associations without worrying about the low level details as TDD will fill in those details better than modeling alone.

John

Andy Dent

Posts: 165
Nickname: andydent
Registered: Nov, 2005

Re: Like liver, I don't like TDD Posted: Dec 13, 2005 9:22 AM
Reply to this message Reply
Doing TDD without having a code-coverage tool measure what's covered is like doing optimisation without running a profiler.

Somone who is good and understands the problem and the code will probably do the right things but there's also a chance of something significant being missed because of an assumption.

You don't know what your tests cover until you confirm they have covered it.

Matt Gerrans

Posts: 1153
Nickname: matt
Registered: Feb, 2002

Re: Like liver, I don't like TDD Posted: Dec 14, 2005 11:09 PM
Reply to this message Reply
Good point, Andy.

Maybe TTD should be generalized to Code Coverage Driven Development.

Ben St. John

Posts: 3
Nickname: jbstjohn
Registered: Aug, 2005

Re: Like liver, I don't like TDD Posted: Dec 15, 2005 9:08 AM
Reply to this message Reply
First, I'll try to back you up on the scientific vs. creative. The words are perhaps too inflammatory, but the idea (as I understand) is that in the "creative" case you go with what you think is the correct answer, quickly, perhaps skipping some tests or checks, and in the second you proceed according a methodology. Both methods have advantages and disadvantages. I believe in Code Complete Steve McConnell talks about them with respect to debugging, where he basically says, do it the intuitive way, at first, but set a time limit on how long you "putter about" before going scientific.

I would argue that almost all (intelligent) development has some trade-off between speed an certainty. If I run all the tests for my system (and since it's an engine this theoretically also means all apps that have ever caused a problem) it takes about a day. So, if I change a line of commenting, I might NOT RUN ANY TESTS! Even though, of course, it's possible that I accidentally did change something. Similarly, in daily work, I tend just to test the areas that I think (eep, intuitive, creative!) might be affected. And then every once in a while, I do the full deal. Actually even that's not true -- some of the tests are so long (I didn't get to write/choose all of them) and so apparently redundant that I rarely run them.

So that was the first point, but it ties in to others. I definitely like some of the ideas behind TDD, but I find it a bit too extreme. I like the idea of coding/designing a few tests at roughly the same time as coding/designing the system itself. I think that makes you certain your system is testable, and may clarify what "success" or "correctness" mean for you. I like having a standard suite of tests, to allow you to quickly check correctness, especially for refactoring. But I think some things are missing, over-emphasized, or wrong.

First, test have bugs in them too, and the more you write, the more time you'll also have to spend debugging. Does it save you time? Generally yes. If you have 30 tests for one line of code, probably not. I use assertions a lot, and I've definitely spent time find the errors in them.

You also have to maintain these tests, and the cost for that will never be zero. I also think the test suite can imbue a false sense of confidence -- just because you've passed your tests doesn't mean your code is correct. Even with 100% coverage (assuming line coverage, and not control flow coverage). And running them, for non-trivial cases, is a fair bit of time, which perhaps, perhaps, could be better spend doing something else.

If it were so easy, formal proofs of correctness would be much more widely used.

So, trade-offs are always present. Historically I'll gladly admit that the balance has lain much too far to the side of not testing. But put it this way -- every passed test run is essentially wasted time (apart from the sense of confidence in the code). You balance this "lost" time off against the (potentially massive but not infinite) amount of time you save by catching some bugs earlier. But I would suggest that if you're running your tests so often that you're passing them 99.99% of the time, you're testing too often.

M. Eric DeFazio

Posts: 10
Nickname: defazio
Registered: May, 2005

Re: Like liver, I don't like TDD Posted: Dec 15, 2005 4:07 PM
Reply to this message Reply
...Suppose using the words "scientific vs. creative" really did raise the ire of those in favor of TDD. (lighten up people). This debate has become more "political" than anything else, (I'm a moderate not a democrat or republican)...

It would be nice if someone who actually uses TDD in practice could make a convincing argument about how TDD
A) improved the code quality
B) reduced bugs
C) decreased time to create new features
(Any other benefits)

In general how valuable is "instant feedback"?

I'd be particularly interested in hearing about those individuals who have gone from a traditional programming shop to TDD who could talk about there experiences. Some questions I would ask would be:

1) What kind of software are you writing (Enterprise Business Applications, Shrink Wrapped Software)
2) Do you actually follow TDD? (write the test case first (TFD) before any other code) in ALL phases of system development (from design through maintenance).
3) Do you have 100% code coverage?
4) How often do you run your test cases? (do you use continuous integration?)
5) How do you handle writing test cases for integration pieces (i.e. Databases, Messaging, etc.) Do you write test harnesses for each, Mock Objects, etc.?
6) How much time is spent writing and refactoring test cases -verses- Coding?

I don't mind putting an unpopular opinion out there... Anyone can read the TDD manifesto and have an opinion or interpretation of what TDD means to them. I suppose I'm just having a hard time believing that people really follow what it says to the letter (just as most of the edicts of Extreme Programming, which tend to take any concept to the "Extreme").

Josh Friedman

Posts: 3
Nickname: jdfphilpa
Registered: Dec, 2005

Re: Test Driven Development?! Posted: Dec 15, 2005 7:49 PM
Reply to this message Reply
I have used TDD to varying degrees over the last two years, and I am very troubled by it. I have just run into too many problems and limitations for TDD to be useful to me. I grant that perhaps I have a misunderstanding of TDD, and also that the environments that I have worked in have not always been supportive of TDD, or agile programming in general.

First, there are the areas that others have raised, where TDD is difficult or impossible - distributed services and APIs, user interfaces, etc. How do you come up with a test suite for Swing? At some point, there has to be visual inspection of the results.

A bigger problem is that I spend 2 - 5 times as long writing the unit tests as I do the code being tested. Evangelists of TDD typically use small, self-contained examples, such as developing a factorial method. That's easy to write tests for, since the method depends only on its arguments. But, that is not the real world. At my current job, I am responsible for a batch process - yes, they still exist, even in the Java world - that reads from about 40 tables in MySQL, processes the data according to complex business rules, then generates output files in a complex, arcane format (the files are read by a legacy application). For reasons that are too long to get into, this is a plain old command-line app. It could perhaps have been written in multi-tier form, but that would not change the basic problem. In order to set up the inputs for a test, I need to condition the database. This can mean inserting records into many tables. In addition, most of the tables have auto-incrementing key columns, which must be passed onto the output files. So, I cannot pre-condition the database to the same exact state each time - even if I clean up the rows that I inserted from the test, the next time I run the test, the new rows will have different values for the key columns, making automated validation more difficult. It can be done - I can get back the auto-generated values - but it is more code to write in the test case.

So now, I have test cases that are themselves so complex that they themselves need to be tested! Sure, I could separate the app's data access code out into a separate layer that implements an interface, and then use mock objects to test the business logic. But, at some point, the data access code has to be tested. It doesn't really matter if I'm going directly through JDBC or through an ORM layer, such as Hibernate - DAO code has to be tested, and this means preconditioning the DB in the test case.

Similarly, code to validate the arcane output format will be similarly complex. I have to parse the generated file and compare the parsed result to hand-coded expected results (containing potentially hundreds of data items).

In the end, I wind up writing test cases for fairly self-contained logic, but for anything that involves preconditioning external data sources or examining output files, it is just not worth it. It is much faster to visually inspect the output file to verify correctness, even if I have to do it dozens of times. For this particular product, unit testing can cover less than half of the functionality.

Also, what happens if a test case needs refactoring? Perhaps setup code needs to be extracted to a setUp() method. Suppose that involves four tests. Do I now have to recalibrate each of those four tests? That greatly discourages refactoring test cases.

There's also the issue of boilerplate code, such as value objects, Struts form beans, etc. Classes such as these are usually just dumb data containers, with one-line getters and setters and a constructor. Indeed, I have used XSLT to generate value-object code from a small XML file. Using TDD for such code is pure drudgery, and also interferes with using and developing automated code generation tools.

Perhaps I am missing some basic concept. How can I get TDD to work, given these issues?

Vincent O'Sullivan

Posts: 724
Nickname: vincent
Registered: Nov, 2002

Re: Test Driven Development?! Posted: Dec 16, 2005 5:00 AM
Reply to this message Reply
I can broadly sympathize with your comments and would certainly agreed that TDD can double (and more) the amount of code that needs to be written to accomplish any particular task. This is a frustrating factor that is never mentioned. However, that being said TDD (using JUnit presumably) is very much a unit testing tool. Once you go beyond testing units, i.e. testing if and application and a database work together, then JUnit's capabilities are being stretched and you need other, more complex tools.

> ...dumb data containers, with one-line
> getters and setters and a constructor...
> ...Using TDD for such code is pure drudgery, and also
> interferes with using and developing automated code
> generation tools.

You are quite correct. You simply don't need to test dumb getter and setters. Only write a test for them if you intend to introduce any complexity into them. Of course, if you are using a code coverage tool to monitor your tests then it will complain but don't be a slave to the process, apply it intelligently.

Josh Friedman

Posts: 3
Nickname: jdfphilpa
Registered: Dec, 2005

Re: Test Driven Development?! Posted: Jan 18, 2006 2:52 PM
Reply to this message Reply
Ok, so I used TDD in developing 15 classes and associated test cases and wound up hitting a wall that required me to completely throw 4 days worth of work and start over. If I had put some previous thought into a design to start with, I could have avoided going down a blind hole. I fail to see where TDD, or XP for that matter, saves any time and produces better software. In this time, I am behind a deadline. So, goodbye to TDD.

Charles Sphar

Posts: 1
Nickname: chucksp
Registered: Jan, 2006

Re: Test Driven Development?! Posted: Jan 27, 2006 2:55 PM
Reply to this message Reply
Hmm. As I see XP methodology, you aren't going to begin coding, test-first or any other way, without a clear set of user stories to delineate your requirements. Those should guide you with TDD in much the same way they would without it, I'd think.

Greg Helton

Posts: 7
Nickname: greghelton
Registered: Aug, 2007

Re: Test Driven Development?! Posted: Aug 18, 2007 11:25 PM
Reply to this message Reply
<quote>something mysterious that happens in nearly all large organizations. Ideas get passed from person to person through a grapevine, and the form that they end up in is rarely the form they start in.</quote>

It's the same process Ron Jeffries describes in "We Tried Baseball and It Didn't Work"
http://www.xprogramming.com/xpmag/jatBaseball.htm

Flat View: This topic has 25 replies on 2 pages [ « | 1  2 ]
Topic: Test Driven Development?! Previous Topic   Next Topic Topic: Object Design


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us