The Artima Developer Community
Sponsored Link

Weblogs Forum
Pitching a Fit

4 replies on 1 page. Most recent reply: Dec 10, 2004 12:44 AM by Michael Feathers

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 4 replies on 1 page
Michael Feathers

Posts: 448
Nickname: mfeathers
Registered: Jul, 2003

Pitching a Fit (View in Weblogs)
Posted: Aug 26, 2004 10:57 AM
Reply to this message Reply
Summary
Have you heard about FIT yet? Even if you have, do you know what makes it signficantly different from other testing frameworks? Here is a re-introduction to FIT that highlights one of its most important features.
Advertisement

I had a weird feeling at the XP/Agile Universe conference in Calgary last week. I was looking through the sessions and the workshops, and I noticed how prominent FIT was at the conference. But that stood in stark contrast to what I've seen out in the field. FIT is nowhere near as well known as, say, JUnit. Fortunately, things are changing. More people are hearing about FIT, but sometimes I wonder whether they are hearing the most important things about it, the things that make it clear just how different it is from other testing frameworks. This blog entry is an attempt to introduce FIT to newcomers and put those differences in the foreground.

FIT stands for "Framework for Integrated Tests." It is a deceptively simple set of classes that Ward Cunningham developed to aid testing in a large variety of applications. On reading this you might say “Oh yes, well fine, why should I use it? I can call any vendor at random and get a site-license for an automated testing tool.” Well, you can, but FIT is free. You might reply, “Sure, but I can to go to SourceForge and download any number of automated testing tools.” True, but not only is FIT free, it is also significantly more useful than most mainstream testing tools. The reason is subtle and not widely appreciated. I’ll say more about that later, but first, here is a brief explanation of FIT for people who haven’t seen it before.

What FIT Does...

FIT is a testing tool, but at it’s core, FIT rewrites HTML documents. If you have a typical HTML document and you pass it to FIT, it will give you an exact copy, unmodified. The thing that makes FIT interesting is that it treats HTML tables in a special way. If you give it HTML with an embedded table that looks like this:

MarketEvaluation
Symbol Shares Price Portfolio Attempt Fill
IBM 100 6791 no yes
PLI 1100 3701 no yes
EFT 500 793 yes yes

It will give you back an HTML document that looks the same, but there will be one difference: the table will look like this:

MarketEvaluation
Symbol Shares Price Portfolio Attempt Fill
IBM 100 6791 no yes
PLI 1100 3701 no yes
EFT 500 793 yes yes

FIT interprets the first line of each table as the name of a fixture. A fixture is a piece of bridging code that interprets the table and talks to your application. It reads the table, row by row, and initiates actions against the application or passes it values; then it colors cells in the table based upon the results. In the example above, the cells containing “yes” in the “Attempt Fill” column matched values that were returned from the application for each row. Whenever a table value and the value returned by the application match, a cell is colored green, indicating a successful test.

Here is another example:

MarketEvaluation
Symbol Shares Price Portfolio Attempt Fill
PLI 1100 1560 no no

The table above hasn’t been run yet, but when we run it we'll be able to see a bug in our software. The table tells us that when we attempt to evaluate 1100 shares of PLI at a particular price, "Attempt Fill" should be "no." Let’s run it and see what happens:

MarketEvaluation
Symbol Shares Price Portfolio Attempt Fill
PLI 1100 1560 no no expected
yes actual

FIT shows us that the software returns "yes" instead. FIT changes the table so that we can see both the actual and the expected values for "Attempt Fill" of PLI and it colors the error red so that we can see it easily.

There is much more to FIT than what I’ve described so far. The example above only shows the use of one type of fixture. There are many other kinds. Some allow you to script against an API, others help you check results of database queries, etc. You can also insert markers in your document that will be replaced with summary information about all of the tests run. One of the key benefits of FIT is the fact that all of the text in the HTML you input is completely preserved. This means that you can embed actual tests in project documents. When you run them through FIT you end up with documents explain the software and actually show what it does.

What Makes FIT Powerful...

If you’ve read the section above or had some experience with FIT, you probably have a sense of how it compares to other automated testing tools. Superficially, FIT looks very useful, but there is something subtle about FIT that you only notice when you compare teams that use it with teams who’ve attempted to use another tool:

When you use FIT, tests are decoupled from the development process.

Here’s what I mean by that. When you use FIT, you can write tests before the code that needs to be tested. After you write them you don’t have to rewrite them later or re-enter them when the code is ready.

Is this significantly different from other testing tools? What about scripting tools? Couldn’t we write little scripts or program against code that doesn’t exist yet? We could but there are two downsides; first of all, to do that you have to be a programmer. FIT presents a high level interface that non-programmers can use. Secondly, you end up tying yourself to a specific application interface. When you use FIT, the fixture defines communication with the application. If the interface you want to test against changes, you can modify the fixtures and leave the tests the same. This moves FIT past the “volatile interface” problem that plagues GUI-based screen scraping tools. With FIT you are decoupled. Once business analysts or customers in your organization know how to design FIT tests, they can work on them at any point during the development process. They can coordinate with the developers to decide on the fixtures that they need for their tests to run completely, but they can still run the tests at any time. At some point, the tests will start to turn green. Nobody has to rewrite them or transliterate them from a test design document. You can write them early, in your documentation, and run them to see progress.

Recently, I’ve visited some teams that have run into trouble with their home-grown testing frameworks. They took significant time to create them, gained organizational buy-in and then they discovered that they had a bottleneck: Their testers couldn't work concurrently with their developers. They had to wait until developers solidified the user interface of the application before they could start their work. This limited the work that testers could do until they got the software, and it also skewed the work that developers did in an iteration: they had to stabilize the UI before they did anything else. It’s unfortunate that they discovered only later that decoupled testing could have helped them from the beginning.

You can get more information about FIT at (http://fit.c2.com) and FitNesse (a hosting environment for FIT tests) at (http://fitnesse.org). Both sites have plenty of information, and they are well worth a look.


Johannes Link

Posts: 3
Nickname: jlink
Registered: Sep, 2004

Re: Pitching a Fit Posted: Sep 29, 2004 3:22 AM
Reply to this message Reply
There's one thing I'd like to add about why FIT is different from other testing frameworks. Since there has to be someone who writes the fixture code, ie. who's actually doing programming work, it is obvious that integration testing requires - to some degree - development work and that the development team must therefore be involved. Capture&Replay tools give the illusion that test automation can be done by anyone (the business experts) on their own. In my opinion it's this attitude which makes many of the well-meant test automation efforts fail.

Hans Martin Kern

Posts: 1
Nickname: hmkern99
Registered: Nov, 2004

Re: Pitching a Fit Posted: Nov 10, 2004 9:28 AM
Reply to this message Reply
I can't find a C++ version of FIT. Is there one?

Steven E. Newton

Posts: 137
Nickname: cm
Registered: Apr, 2003

Re: Pitching a Fit Posted: Nov 10, 2004 2:17 PM
Reply to this message Reply
There are some challenges in implementing FIT in a language without reflection. There have been some attempts, as discussed in http://fit.c2.com/wiki.cgi?CppPlatformSpike.

Michael Feathers

Posts: 448
Nickname: mfeathers
Registered: Jul, 2003

Re: Pitching a Fit Posted: Dec 10, 2004 12:44 AM
Reply to this message Reply
I wrote on a couple of years ago. James Grenning took it, cleaned it up, added to it, and documented it. It is in the files section of the fitnesse yahoo group. I think we'll be making it fully compliant with what FIT has become recently soon. FIT has changed a little since the initial port.

I did the reflection using pointers to members and pointers to member functions wrapped in a macro.

Flat View: This topic has 4 replies on 1 page
Topic: Heron just got Slashdotted Previous Topic   Next Topic Topic: The Six Myths of Creativity


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us