Summary
I recently returned from an intense, one-month marathon consulting tour to three different companies. I came away with a sense of satisfaction that everyone had benefitted significantly from the experience -- including me. The common sticking point was understanding the process of object-oriented design.

There were many side issues surrounding that idea, but everything always came back to objects and design. Over the years I've studied most of the OO design methods (and the procedural methods before the advent of OO) and have selected pieces to add to my own slowly-amalgamating method. However, this tour produced a number of ephiphanies about OO design.

First, it's really, really important that a design method be simple. Any bit of complexity that doesn't produce big results in the design process makes the method fuzzy and hard to remember. I believe you need to be able to hold the method in your head, which means that it must conform to the "seven plus-or-minus two" rule, regarding the number of things that humans can keep in their head at one time.

There are people that will argue with this, saying that it's generally a good idea but that because of this or that special case we must add these extra steps or that complexity (a big argument is: "so that the method will scale" -- this produces a method that promises to scale but that doesn't get used because it's too complex). In my experience this falls into the same trap as premature optimization or any other form of mixing abstraction levels that we, as programmers, are prone to.

This is the second point: people have a very difficult time separating abstraction levels. We are drawn to the complexity that we know. Once we conquer a particular bit of complexity, I believe we enjoy revisiting it to renew the sense of satisfaction. That's why it's so hard to give it up in order to step back and say: "but that isn't relevant here; we're trying to get the big picture right now."

As a result, attempts to develop the domain model end up in arguments about whether to use stored database procedures, or how to optimize a particular aspect of the system, or invent yet another framework in an attempt to create reusable code rather than just solving the problem at hand. Much of my coaching during this trip involved gentle prodding to help people stay at the right level of abstraction. There are a million ways to fall off the razor's edge, which is why it's so essential that a design method be simple.

The third big issue was about the UML. Everywhere I went, people were struggling with it, and I think the repetition of that confusion finally made me take a hard look at it. The UML is not only too complex, it seems to be incomplete (or at the very least, it takes far too many kinds of diagrams to express the design).

The reason for the complexity of the UML is, ironically, inappropriate mixing of abstraction levels. I'm certain this happened because, back when Rational was a viable company (before too many people asked "if Rational is supposed to teach us how to create good software, how come Rose is such a bad piece of software" and the company imploded, being "saved," if you can call it that, only because IBM bought them as they were spiraling down the drain), and they brought Booch, Jacobsen and Rumbaugh together so they would stop fighting and create a "Unified" model (the creation of peace was the reason for the "U", for those of you new to the story). When this happened, agendas inevitably got mixed together, and I suspect that Rational had its own agendas which got into the mix, design-by-committee style. At the time, C++ was king of the OO languages so you see a lot of C++-isms in the UML, which introduced innapropriate details. At least one player appeared to have grand ideas of taking this abstraction tool and giving it enough detail to produce code. Of course, we see this in most of the modeling tools, which can spit out framework code from the UML, but in recent years the spectre of Model-Driven Architecture (MDA) has appeared, which was probably inevitable given the implied promise that someday UML would be the one ring that binds them all. Fortunately, the MDA experiments seem to be happening in the open-source world, so we can discover (again) that programming-by-drawing-pictures is Goedel-incomplete, without having to pay hundreds of thousands for tools that don't work hawked by vendors selling faith-based solutions, as has been the case for every "revolution" that has popped up in the past.

Because of all these misdirections, it's no wonder people get confused about what to do with the UML. Is it for modeling, or is it for detailed design? My guess is that forces within Rational wanted it to be everything to everyone, but this goes back to my original point: a method must be really, really simple in order for people to use it without getting lost.

Two things became clear about the UML on this trip. First, when walking through use cases/user stories, the static structure diagram is what appears. This is not a revelation; CRC cards or simply hunting for nouns and verbs produce the same results. However, we found that a very simple form of structure diagram would suffice, and this seemed to be very helpful for people because they could see that it was only intended to capture the domain model, and not express every possible detail about the code. Without this guidance, people can waste a lot of time filling in all the methods, arguments and fields and lose sight of the big picture.

The second thing that became clear was that a UML structure chart is a very helpful step in expressing the domain model, but it lacks an essential ingredient: mechanism. The structure chart only expresses the relationships between classes, not how they interact. Sometimes this relationship is enough, along with some words and hand-waving, to understand how the system will work. But, I began to see, that is only in the case where the mechanism is so obvious that everyone can see it. (I am aware that the UML has a various number of different diagrams that show different aspects of mechanism, but none of them are able to sum it up.)

On the other hand, expressing mechanism down to the nuts and bolts is precisely the pitfall we are trying to avoid by using a model. What's needed is the essence of mechanism, just enough so that we can see how something should work without getting lost in detail.

Could this essence be design patterns? Might we be able to see the most obvious designs because we have been schooled in the most basic of patterns so effectively that we don't even consider them patterns, but just fundamental ways of doing things, and don't even have names for them? But when we have more complex interactions, we need a pattern to describe it?

During one session, I began to see the possibility of annotating a structure chart with the mechanisms that drove the program, and this seemed to work in some situations. However, in the following consulting session the idea of a separate diagram began to appear. I started calling this a "flow diagram" because it showed the flow of objects and messages through the system of objects. This diagram does not show composition and inheritance, but rather the relationship of higher-level objects and the way they accomplish their tasks. It also seems to show threading in the system. In general, it seems to focus on the way that objects are wired together to build a system and the dynamic of that system, rather than just examining the inheritance and composition relationships (which are certainly important, but are clearly only a step in the design process). I don't have a "finished" approach for flow diagrams, but the idea continues to reappear and evolve.

Finally, it has become quite clear to me that the software design process involves incomplete information at every stage. This can be a point of contention, one that appears to be rooted in your belief system about the world. Suggest that you don't ever really know everything about a system to someone who really wants things to be deterministic from beginning to end, and you'll get a response ranging from "huh?" to downright hostile, depending on how threatening the idea is (we could probably carry this same argument into the realms of the static/dynamic language debate, exceptions, developer testing, etc.). But my experience is that, if you insist that complete information must be available at every stage, you rapidly converge on analysis paralysis on the project level, and the waterfall approach on the method level.

On the other hand, accepting that information is incomplete allows a project to become remarkably nimble. At each stage you know you will only be able to mine some of the information about the project, and when a vein begins to dry up you move on rather than wasting time trying to pry out every possible nugget. From experience, you know that you'll move forward a lot faster by moving on, into a design pass, or even by pseudocoding your design (in a couple of instances we used Python to try out a design; this worked remarkably well in producing new insights about that design). In addition, you can "backfill" user stories and design details as you uncover new factors, so you don't need to feel like an incomplete design must hold you back. The experience I had was very rapid and liberating; we could often punch through into designs and insights that had been holding people back for significant periods.

Talk Back!

Have an opinion? Readers have already posted 37 comments about this weblog entry. Why not add yours?

RSS Feed

If you'd like to be notified whenever Bruce Eckel adds a new entry to his weblog, subscribe to his RSS feed.

Digg | del.icio.us | Reddit

About the Blogger

Bruce Eckel (www.BruceEckel.com) provides development assistance in Python with user interfaces in Flex. He is the author of Thinking in Java (Prentice-Hall, 1998, 2nd Edition, 2000, 3rd Edition, 2003, 4th Edition, 2005), the Hands-On Java Seminar CD ROM (available on the Web site), Thinking in C++ (PH 1995; 2nd edition 2000, Volume 2 with Chuck Allison, 2003), C++ Inside & Out (Osborne/McGraw-Hill 1993), among others. He's given hundreds of presentations throughout the world, published over 150 articles in numerous magazines, was a founding member of the ANSI/ISO C++ committee and speaks regularly at conferences.