A Conversation with Martin Fowler, Part VI

---

Over the last decade, Martin Fowler pioneered many software development techniques in the development of business information systems. He's well known for his work on object-oriented analysis and design, software patterns, Unified Modeling Language, agile software processes (particularly extreme programming), and refactoring. He is the author of Analysis Patterns (Oct. 1996), Refactoring (June 1999; coauthored with Kent Beck, et al.), UML Distilled (Aug. 1999; with Kendall Scott), Planning Extreme Programming (Oct. 2000; with Kent Beck), and the soon to be released Patterns of Enterprise Application Architecture (Nov. 2002), all published by Addison Wesley.

In this six-part interview, which is being published in weekly installments, Fowler gives his views on many topics, including refactoring, design, testing, and extreme programming. In Part I, Fowler makes the business case for refactoring and testing, and describes the interplay between refactoring, design, and reliability. In Part II, Fowler discusses design principles of avoiding duplication, separating presentation and domain logic, being explicit, and describes how refactoring depends on code ownership. In Part III, Fowler differentiates between planned and evolutionary design, suggests that focusing on superficial problems can lead to the discovery of substantial problems, and claims that doing a good job won't slow you down. In Part IV, Fowler discusses design decay, flexibility and reusability versus complexity, four criteria for a simple system, and interface design. In Part V, Fowler describes the unhurried quality of test-first design, defines monological thinking, and distinguishes between unit and functional testing. In this final installment, Fowler describes how to balance maintainability and efficiency and create tunable software, and discusses the role of patterns and the Agile Software Manifesto.

Balancing Maintainability with Efficiency

Bill Venners: I keep bumping into famous people at the Red Carpet Club in the Denver airport. This summer I ran into Calista Flockhart. Last year I ran into you. I was too star struck and afraid of Harrison Ford to talk to Calista, but you and I sat down and had a beer. One of the things you said to me over that beer is that you think serialized objects should be in a programmer-readable character format rather than binary. When I mentioned that a character format could be slower than a binary format, you said the binary approach makes things harder to maintain in the name of efficiency. I wonder if you could talk about the specific case of serialization forms, and in general, how do you balance maintainability with efficiency?

Martin Fowler: Efficiency always wins providing that you're looking at it properly. The problem is that a lot of people do things for efficiency reasons without for instance running a profiler. If you ever do something for efficiency reasons and you aren't running the profiler, then basically you are speaking out of your rear end.

The serialization case is a bit more problematic. One of the problems with binary serialization is you can't look at the result. This is particularly a problem if you are storing that serialized object. A classic problem with Java is if you then change the class and try to reread the serialized object, you can't. Similarly, if you have a client and a server communicating through serialized objects, and you update one end of the connection but not the other, it all falls down.

Now there is a trick you can use to get around that. Rather than serializing the object, pull the data out of the object and put it into a dictionary. Then serialize the dictionary. That gives you some resilience against change.

Bill Venners: It's not explicit, though.

Martin Fowler: No a dictionary isn't as explicit, but if you add a field to the class and pump an extra value into the dictionary, it doesn't matter. It is a less brittle mechanism. XML is often less brittle because, again, you can ignore data you don't know about. Much of the problem with binary serialization is its brittleness. I talk a bit more about serialization forms in my book, Patterns of Enterprise Application Architecture. For data transfer and storage in a database, for example, you have to think about the variations between character and binary serialization.

Creating Tunable Software

Bill Venners: You say in Refactoring, "The secret to fast software in all but hard real time contexts is to write tunable software first, and then to tune it for sufficient speed. How do I make software tunable?

Martin Fowler: By making it well-factored.

Bill Venners: Why?

Martin Fowler: Because then it's easier to change.

Bill Venners: So in this case it's not easier to change for adding new functionality, but for enhancing performance of existing functionality.

Martin Fowler: Well-factored software is easier to tune. You should focus on making software well-factored and well-designed, then go through performance optimization driven by profilers.

Programmer and Compiler Optimization

Martin Fowler: Another thing you have to bear in mind is that performance optimizations are quite implementation and release dependent. When you get a new version of Java, you really ought to undo every performance optimization you ever made, then reapply them to make sure they still make the program go faster. Often you'll find that a performance optimization you did in a previous version of the virtual machine (VM) or optimizing compiler will actually make you slower now. A performance optimization that helped before may actually defeat the optimizations the new compiler or VM is doing.

Bill Venners: It's not easy to remember what mutations you made to improve performance in the past.

Martin Fowler: Undoing and reapplying optimizations is really what you ought to do. I know it's not easy. It argues for keeping careful track of what changes you make for optimization. The thing to remember is the performance hit of old optimizations can sometimes be a big deal.

Craig Larman told me a story I still love. Craig gave a talk on performance optimization at JavaOne, and he mentioned two well-known techniques: object and thread pooling. Object pooling is reusing the same objects rather than creating new objects. Thread pooling is basically the same thing with threads. After the talk, two guys came up to him, both of which were designers of high performance VMs. One VM was Hotspot, and I think the other VM was JRocket. The one guy said, thread pooling works well, but you don't want to do object pooling on our VM because it will actually slow it down. And the other guy said the reverse. You can use object pooling, but not thread pooling, because it will slow us down.

So you could make a performance optimization in one VM, and then bring in Hotspot, and it will actually slow Hotspot down. You've got to be very wary of that. Object pooling is a good example. A lot of people are very enamored with object pooling, yet half the time people are not measuring to that to find out whether object pooling is any good. Object pooling was very important in the early days of Java, because garbage collection wasn't terribly good. When you've got generational garbage collection, object pooling becomes a lot less effective, because short-lived objects can be collected very cheaply. It's the long-lived objects, such as ones you might pool, that are expensive to garbage collect.

So the rules keep changing. That's why you've got to be very careful to profile. If you think you can predict from the source code what the machine is doing, you've got no chance. When you're in a world of optimizing compilers and VMs, you have to profile, because the compilers and VMs are doing things that you can't even imagine. So don't predict, just measure.

The Point of Patterns

Bill Venners: You say in your article Is Design Dead, "For me, patterns are still vitally important," in the light of XP. Where do patterns fit in now?

Martin Fowler: Patterns give us targets that we look at. I still do some up-front design. Patterns are about that. They also give us targets for refactorings. We know what we want to aim towards. Also, knowing patterns gives us a way to kind of tune our sense of design aesthetics, because patterns are at least put forward as examples of good design. You can learn a lot by looking at examples. So I still think patterns have a very important role to play. It's no surprise that most of the people who've been pushing the XP movement are also really big in the patterns community. There's a huge overlap between those two communities.

The Agile Software Manifesto

Bill Venners: The last question I have is, what the heck is the agile software manifesto?

Martin Fowler: A bunch of people who have been very heavily involved in what we now call agile software methodologies--approaches such as extreme programming, scrum, crystal, fix-driven developments, and DSDM --got together and compared notes. We realized we have a lot in common and decided to write down the things we agree on. We made the manifesto as a kind of rallying symbol for this style of methodology we're all in favor of. At the meeting we decided to call them agile methodologies. The manifesto is the closest thing you'll ever get to a definition of what agile software development is about. The manifesto is meant to be a rallying cry to say this is the way we think a large chunk of the industry should go.

Bill Venners: Let's talk about one of the four principles: "valuing Individuals and interactions over processes and tools." What does that mean?

Martin Fowler: That principle basically says that rather than focusing on processes and tools as a way to further your software development organization, it is more valuable to focus on the people on your team, on individuals and the way that they interact at a personal level.

Bill Venners: Do you mean improving their skills?

Martin Fowler: That principle means a whole bunch of things. It involves improving skills. It involves perhaps deciding to go to greater lengths to make developers happier so the abler ones will stay. It means taking personality conflicts more seriously. It means spending time dealing with people rather than figuring out the perfect software process and telling everyone they should follow that process. One of my connotations from that principle is that rather than choosing a process and then fitting the team to the process, teams should chose their software process. You should have the process fit the team, not the other way around.

Even though many of us at the meeting were expounding processes of our own and several of us were tools vendors, we all agreed that software processes and tools are second order effects as to a project's success. The people you have in the team, and how they work together on a human level, is a much bigger factor.

Resources

Refactoring: Improving the Design of Existing Code, by Martin Fowler with Kent Beck, John Brant, William Opdyke, and Don Roberts is at Amazon.com at:
http://www.amazon.com/exec/obidos/ASIN/0201485672/

Patterns of Enterprise Application Architecture, by Martin Fowler is at Amazon.com at:
http://www.amazon.com/exec/obidos/ASIN/0321127420/

The Manifesto for Agile Software Development, and its four principles are at:
http://agilemanifesto.org/

Is Design Dead, an article by Martin Fowler, is at:
http://www.martinfowler.com/articles/designDead.html

To Be Explicit, an article by Martin Fowler first published in IEEE Software:
http://www.martinfowler.com/articles/explicit.pdf

Public versus Published Interfaces, an article by Martin Fowler first published in IEEE Software:
http://www.martinfowler.com/articles/published.pdf

JesTer, the tool mentioned by Martin Fowler that finds places in your code not covered by a unit test:
http://www.xpdeveloper.com/cgi-bin/wiki.cgi?JesTer

The Pragmatic Programmer: From Journeyman to Master, by Andrew Hunt and David Thomas, is at Amazon.com at:
http://www.amazon.com/exec/obidos/ASIN/020161622X/

IntelliJ IDEA, a Java IDE with refactoring support:
http://www.intellij.com/idea/

Eclipse, an open source IDE with refactoring support:
http://www.eclipse.org/

A catalog of summaries of refactorings mentioned in the book, Refactoring:
http://www.refactoring.com/catalog/index.html

A refactoring portal maintained by Martin Fowler contains links to refactoring tools and other refactoring sites:
http://www.refactoring.com/

Martin Fowler's links to extreme programming resources:
http://martinfowler.com/links.html

Articles written by Martin Fowler about XP and agile methods:
http://martinfowler.com/articles.html#agile

UML Distilled: A Brief Guide to the Standard Object Modeling Language, by Martin Fowler and Kendall Scott is at Amazon.com at:
http://www.amazon.com/exec/obidos/ASIN/020165783X/

Planning Extreme Programming, by Kent Beck and Martin Fowler is at Amazon.com at:
http://www.amazon.com/exec/obidos/ASIN/0201710919/

Analysis Patterns: Reusable Object Models , by Martin Fowler is at Amazon.com at:
http://www.amazon.com/exec/obidos/ASIN/0201895420/

Martin Fowler's website contains many articles, book chapters, and other information from Martin:
http://www.martinfowler.com/

Talk back!

Have an opinion? Readers have already posted 3 comments about this article. Why not add yours?

About the author

Bill Venners is president of Artima Software, Inc. and editor-in-chief of Artima.com. He is author of the book, Inside the Java Virtual Machine, a programmer-oriented survey of the Java platform's architecture and internals. His popular columns in JavaWorld magazine covered Java internals, object-oriented design, and Jini. Bill has been active in the Jini Community since its inception. He led the Jini Community's ServiceUI project that produced the ServiceUI API. The ServiceUI became the de facto standard way to associate user interfaces to Jini services, and was the first Jini community standard approved via the Jini Decision Process. Bill also serves as an elected member of the Jini Community's initial Technical Oversight Committee (TOC), and in this role helped to define the governance process for the community. He currently devotes most of his energy to building Artima.com into an ever more useful resource for developers.