The Artima Developer Community
Sponsored Link

Weblogs Forum
How Unit Testing and Refactoring Work Together

12 replies on 1 page. Most recent reply: Apr 24, 2006 1:20 PM by Eric Armstrong

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 12 replies on 1 page
Eric Armstrong

Posts: 207
Nickname: cooltools
Registered: Apr, 2003

How Unit Testing and Refactoring Work Together (View in Weblogs)
Posted: Apr 17, 2006 9:36 AM
Reply to this message Reply
Summary
Refactoring is a wonderful thing. It lets elegance evolve, instead of making you live with faulty initial design decisions for the life of the project. Unit testing makes it possible to refactor safely, for sure. That's an important reason for unit tests. But the need for unit tests goes well beyond that...
Advertisement

Unit tests record the important use cases -- use cases that you frequently discover as you're going along, and then forget about some number of weeks later after you've modified the code to handle them. You're now several months down the road, and you've got some humungous wad of code that badly needs refactoring.

I'm looking at one of those at the moment, in fact. It's a link-managing routine that takes an input link like http://xyx, ../xyz, xyz, or /xyz. It has to normalize relative links, take into account whether there was a "base" directive on the page it came from, and map the result to a new location, if a mapping has been supplied by the user. It also has to deal with http:/, https:/, ftp:/, and file:/ protocols, as well as plain directory paths.

Frankly, the code is a mess. It was patched in multiple places, each time to solve a problem introduced by some set of factors that I hand't orignally taken into account. Looking at it now, I can see code that can never be reached--a sure sign that it has grown too large and too complex for my feeble brain to manage.

Obviously, refactoring is needed. But what kind of refactoring? The answer, naturally enough, depends on what problems the code is trying to solve.

The question is, what were those problems?? They arrived over the course of a couple of years. The code shows the result of the attempted soulutions, but I've long since forgotten the problems I needed to solve.

One way to get such a list is to examine the version modification history. With sufficient scouring, they could be ferreted out. That's one useful reason for maintaining version-controlled sources.

A bug tracking system could also be examined. Out of all the problems that needed to be solved, it would be possible to extract that ones that involve this particular part of the code.

But a better answer is unit tests. Every time the code broke, my first task should have been to create a unit test that replicated the break. That speeds up the edit/debug cycle, too. A small test takes a lot less time to run than the real-life data that generates the error. That's another great reason for unit tests.

But more importantly, had those tests been constructed, they would now provide a complete list of the problems the code had to solve. And the list would be organized by section, with all the tests that involve this method in one place.

In addition, there would be a place to record new issues, as they arise. (An almost certain occurrence, since the history of the project has been one of finding out that the code had to deal with things that I never knew were possible.)

With that list of cases, I could be sure that when I refactor to solve the problem in front of me, I won't create a regression for some issue that I've completely forgotten about--something that the current code is handling successfully, no matter how ugly it is.

Over time, then, the unit tests you collect tells you what the code has to do. In effect, they give you a complete, detailed specification--a specification you can use in an automated way to ensure that your newly refactored design achieves all of the goals that have been identified for it, past and present.

So the moral here is that unit tests be very, very good. They set you up so you can refactor safely. They record the reasons for the patches you made over time. And they tell you when a new refactoring is successful. Taken all together, that's a heck of a lot of value.


Carl Manaster

Posts: 24
Nickname: cmanaster
Registered: Jun, 2003

It's bidirectional Posted: Apr 17, 2006 10:20 AM
Reply to this message Reply
Of course unit tests make refactoring safer and easier, but it's also the case that refactoring can make it easier to write unit tests. We tend to refactor into smaller, more cohesive units (classes and methods). The smaller the scope of a unit, the easier it is to write unit tests for it with good coverage.

Steven Deal

Posts: 3
Nickname: situboo
Registered: Jul, 2003

Re: How Unit Testing and Refactoring Work Together Posted: Apr 17, 2006 11:19 AM
Reply to this message Reply
Nice article. I've often felt that this was a natural take away from working with unit tests or TDD. I've been amazed though at how eyes glaze over when this is brought up. I hope you fare better with your evangelizing than I have.

Geoff Sobering

Posts: 13
Nickname: geoffs
Registered: Apr, 2003

Re: How Unit Testing and Refactoring Work Together Posted: Apr 17, 2006 1:23 PM
Reply to this message Reply
I don't want to seem flip, but my only response is "duh!".

I think the synergy between widespread unit-tests and refactoring has been around since at least the advent of XP (1999?). I don't have my copy of "XP: Explained", but I'm pretty sure the diagram of how the 12 practices interact and support each other descibes the postive feedback between testing and refactoring.

FWIW, I think your note is excellent. Anything that helps keep these notions at the forefront of the software-development mindset is the proverbial "good thing".

Cheers,

Geoff S.
http://geoff-s.net/

unmesh

Posts: 15
Nickname: unmesh
Registered: May, 2003

Re: How Unit Testing and Refactoring Work Together Posted: Apr 17, 2006 9:17 PM
Reply to this message Reply
I was discussing TDD and cost of change on the similar lines at
http://www.artima.com/forums/flat.jsp?forum=276&thread=155901

Calvin Mathew Spealman

Posts: 13
Nickname: ironfroggy
Registered: Aug, 2005

Re: How Unit Testing and Refactoring Work Together Posted: Apr 18, 2006 12:08 AM
Reply to this message Reply
I liked the points you made here. There is always so much information about how important unittests are in the initial development of a product, but relatively little is said about its longterm benefits. Good to remind us all.

Anuradha Athreya

Posts: 1
Nickname: kavz
Registered: Apr, 2006

Re: How Unit Testing and Refactoring Work Together Posted: Apr 18, 2006 2:59 AM
Reply to this message Reply
"But more importantly, had those tests been constructed, they would now provide a complete list of the problems the code had to solve. And the list would be organized by section, with all the tests that involve this method in one place."


This is one reason why i love to write unit tests. Very often, I have to come back to my classes and add more business logic. Unit tests make my life so simple!

Eric Armstrong

Posts: 207
Nickname: cooltools
Registered: Apr, 2003

Re: How Unit Testing and Refactoring Work Together Posted: Apr 18, 2006 12:32 PM
Reply to this message Reply
Thanks, Jeff. I have to agree. My forehead hurt so hard
from smacking it that I had to write up the note. I never saw that particular book, but I'm fascinated by systems diagrams that show how things interrelate. I'll have to get it.

Eric Armstrong

Posts: 207
Nickname: cooltools
Registered: Apr, 2003

Re: It's bidirectional Posted: Apr 18, 2006 12:51 PM
Reply to this message Reply
> ...it's also the case that refactoring can make it easier
> to write unit tests. We tend to refactor into smaller,
> more cohesive units (classes and methods). The smaller
> the scope of a unit, the easier it is to write unit tests
> for it with good coverage.
>
True. But the revelation for me was the realization that as "stuff happens"--as I find out more about the domain than I originally new--unit tests give me a way to record that knowledge. Because it's only when enough knowledge has accumulated that I can refactor intelligently--but if I'm not creating unit tests, I'm not accumulating that knowledge in the form of use cases.

Partly, the goal of writing the note was to motivate myself. The project I was working on was one in which problems tended to surface in a production setting. More often than not, I couldn't reproduce the problem with short tests, so I would up debugging in semi-production mode.
By the time I figured out what was going on, it was easier to jsut fix the code than it would have been to create a test that replicated it.

Refactoring to allow for better testing is another important concept, but the insight that burst upon me was that doing the extra work to set up a test is an important device for knowledge capture--especially since my knowledge is invariably limited in any given domain, and that my "learning opportunities" (aka bugs) can be separated by months, if not years.

Geoff Sobering

Posts: 13
Nickname: geoffs
Registered: Apr, 2003

Re: It's bidirectional Posted: Apr 18, 2006 1:45 PM
Reply to this message Reply
>> The smaller the scope of a unit, the easier it is to
>> write unit tests for it with good coverage.

> ... unit tests give me a way to record that knowledge.
> Because it's only when enough knowledge has accumulated
> that I can refactor intelligently...

So true. I've been trying to instill in the folks that I've been mentoring that tests are the perfect place to capture requirements. Functional/Acceptance tests *are* the system requiremens, and unit-tests document lower-level behavior.

IM(H)O, it's all "about the testing", or more conventionally said, "about the design". I think it's crucially important to remember that that "test first" and "test driven" initially affect the system's _design_. The residual unit-tests are very useful, (ex. for subsequent refactoring), but perhaps their greatest benefit is as a description of the design.

> ... it was easier to jsut fix the code than it would have
> been to create a test that replicated it.
> ... doing the extra work to set up a test is an important
> device for knowledge capture ...

That is a very significant observation. I forget where it was first esposed (probably XPE), but one of the best things you can do in the way of process improvement is capture *every* bug as a test or tests (most likely a functional-level test initially, but usually lower level tests as you track down the root cause).

In the spirit of this discssion, creating the test forces you to really understand and _capture_ what the bug is.

Subsequently, the test(s) give you two benefits: 1) you have a focused environment to fix the bug in, and 2) you know when you've fixed the bug (the test(s) pass); they also prevent the bug from ever reappearing (unoticed).

The epiphany comes when you realize that in many (most?) cases, the time/effort saved in fixing the bug can easily repay the effort to create the tests.

Cheers,

Geoff S.
http://c2.com/cgi/wiki?GeoffSobering

Eric Armstrong

Posts: 207
Nickname: cooltools
Registered: Apr, 2003

Re: It's bidirectional Posted: Apr 24, 2006 11:06 AM
Reply to this message Reply
Geoff Sobering wrote:
>
> Tests are the perfect place to capture requirements.
> Functional/Acceptance tests *are* the system
> requirements, and unit-tests document lower-level
> behavior.
>
That's one to engrave in gold and hang on the wall.

I went to a Ruby conference over the weekend and was
exposed to the latest testing concept: Behavior Driven Testing. It has a slightly more clear an O-O testing syntax, where you invoke a method on the object to compare it to an expected result, instead of sending both the result and the expected value to a method in a third object. So you see things like actual.should.equal(expected), instead of
assert_equal(actual, expected)
-- or is it the other way around?

Syntactically, it's a little nicer. But the deeper motivation was the idea of focusing on desired behaviors rather than objects. Paralleling your line of thinking, it embodies the notion of "test suite as specification", to the point that it's called RSpec, rather than some variant of "Test".

What was particularly fascinating was that Dave Astels originally floated the idea as a thought experiment, but Ruby makes it so easy to extend the behavior of existing classes that Steven Baker implemented the concept in a matter of hours.

Eric Armstrong

Posts: 207
Nickname: cooltools
Registered: Apr, 2003

Re: It's bidirectional Posted: Apr 24, 2006 11:38 AM
Reply to this message Reply
Reply, part II.

Geoff Sobering wrote
>
> creating the test forces you to really understand and
> _capture_ what the bug is.
>
Couldn't agree more. But to understand what was happening
well enough to write a test, I found myself using trace
statements and the debugger while running on production data, so I could see just what the heck what was going.
By that time I knew enough to write the test, I *also* knew
enough to solve the problem.

That situation existed, of course, because this particular program was written before I had bitten by the unit test bug. One of the joys of unit tests is that it keeps all bugs /shallow/. A bug is almost always immediately pinpointed to the method it's in. And with a test harness already set up, you can easily set up experiments to test your assumptions, in the form of more tests. So you're almost always puzzling out one routine, and thinking about what it should do.

That's "fun" because it's like solving crossword puzzles for a living. The problems aren't too hard, and they're not to easy. They're just right. On the other hand, diving through a ton of intertwining functions trying to find the source of the bug... well, that's hard.

So now, I'm at that point where I've been looking at legacy code (which I sadly happen to have perpetrated myself), and I finally see what's going on. But...

* there deadline pressure
* the unit test harness doesn't exist
* the code needs to be refactored to support testing

So now I'm facing one of those difficult decisions. Do I "fix the bug", or do I set up a test harness, do some refactoring to make testing possible, and create a test, and /then/ fix the bug.

> Subsequently, the test(s) give you two benefits: 1) you
> have a focused environment to fix the bug in, and 2) you
> know when you've fixed the bug (the test(s) pass); they
> also prevent the bug from ever reappearing (unoticed).
>
Ah, but in this case benefit #1 has *seemingly* disappeared. I no longer need an environment to fix the bug in, because I already know what I need to fix.

Of course, that appearance also disguises several very real risks. For one thing, I may not understand the problem as well as I think I do. For another, I may botch the fix. Even if I get it right, I may break something else--but lacking a test suite, I have no protection against that in any case--at least for the moment. (I do begin to acquire that protection in the future if I start collecting tests now.)

> The epiphany comes when you realize that in many (most?)
> cases, the time/effort saved in fixing the bug can easily
> repay the effort to create the tests.
>
Again, you speak the truth. But that traditional argument for unit testing (an argument I subscribe to and apply to all new projects) is one that doesn't apply well to legacy code--or spaghetti, as the case may be.

(Along those lines, the guys at the Ruby conference mentioned a must-read book by ___ called ___ Legacy Code. It's about retrofitting unit testing and refactoring in bits and pieces to help bring such dinosaurs under control.)

Anyway, the bottom line in all this was that "making it easir to fix the bug" wasn't a good argument for setting up a unit test, because once I knew enough to create the test, I already knew how to fix the bug. So it seemed like just so much "extra work".

Meanwhile, the argument of protection against future regressions was too abstract for me. I've just got way too much confidence in myself. Heck, I won't break things in the future. And if I do, I'll fix them. Hey, maybe its not the best of reactions. But it's something.

But the one argument that strikes me as irrefutable is that if I need to make a patch, it means that there is *something I didn't know*. Even if it's someone else'
s code, it means there was something the code didn't anticipate.

We can probably afford to rule out simple typos, here, like maybe I forgot a parenthesis or used A instead of B. But when I find myself adding code--especially if I
add new logic to handle some previously unexpected condition--I now have to realize that new item is the tip of an iceberg. The top is the new condition that surfaced. Underneath, lies the vast body of ignorance that inhabits the majority of my brain.

So I really *do* need write those unit tests, even if I've already fixed the bug. Thanks to your earlier note and the weekend seminar, I can now state the reason more succinctly:

I need to write a unit test for a bug, even if
I already know how to fix it, in order to
*capture the specification*.

Seen in that light, writing the unit test is essentially a documenation process--one that will guide later refactoring and provide a means for automated validation of the result.

Eric Armstrong

Posts: 207
Nickname: cooltools
Registered: Apr, 2003

Re: It's bidirectional Posted: Apr 24, 2006 1:20 PM
Reply to this message Reply
> (... the guys at the Ruby conference
> mentioned a must-read book by ___ called ___ Legacy Code.
> It's about retrofitting unit testing and refactoring in
> bits and pieces to help bring such dinosaurs under
> control.)
>
The book is Working Effectively with Legacy Code, by Michael Feathers.

Flat View: This topic has 12 replies on 1 page
Topic: How Unit Testing and Refactoring Work Together Previous Topic   Next Topic Topic: Embedded DSLs and Productivity


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us