I recently returned from an intense, one-month marathon consulting tour to three different companies. I came away with a sense of satisfaction that everyone had benefitted significantly from the experience -- including me.
The common sticking point was understanding the process of object-oriented design.
There were many side issues surrounding that idea, but everything always came back to objects and design. Over the years I've studied most of the OO design methods (and the procedural methods before the advent of OO) and have selected pieces to add to my own slowly-amalgamating method. However, this tour produced a number of ephiphanies about OO design.
First, it's really, really important that a design method be simple. Any bit of complexity that doesn't produce big results in the design process makes the method fuzzy and hard to remember. I believe you need to be able to hold the method in your head, which means that it must conform to the "seven plus-or-minus two" rule, regarding the number of things that humans can keep in their head at one time.
There are people that will argue with this, saying that it's generally a good idea but that because of this or that special case we must add these extra steps or that complexity (a big argument is: "so that the method will scale" -- this produces a method that promises to scale but that doesn't get used because it's too complex). In my experience this falls into the same trap as premature optimization or any other form of mixing abstraction levels that we, as programmers, are prone to.
This is the second point: people have a very difficult time separating abstraction levels. We are drawn to the complexity that we know. Once we conquer a particular bit of complexity, I believe we enjoy revisiting it to renew the sense of satisfaction. That's why it's so hard to give it up in order to step back and say: "but that isn't relevant here; we're trying to get the big picture right now."
As a result, attempts to develop the domain model end up in arguments about whether to use stored database procedures, or how to optimize a particular aspect of the system, or invent yet another framework in an attempt to create reusable code rather than just solving the problem at hand. Much of my coaching during this trip involved gentle prodding to help people stay at the right level of abstraction. There are a million ways to fall off the razor's edge, which is why it's so essential that a design method be simple.
The third big issue was about the UML. Everywhere I went, people were struggling with it, and I think the repetition of that confusion finally made me take a hard look at it. The UML is not only too complex, it seems to be incomplete (or at the very least, it takes far too many kinds of diagrams to express the design).
The reason for the complexity of the UML is, ironically, inappropriate mixing of abstraction levels. I'm certain this happened because, back when Rational was a viable company (before too many people asked "if Rational is supposed to teach us how to create good software, how come Rose is such a bad piece of software" and the company imploded, being "saved," if you can call it that, only because IBM bought them as they were spiraling down the drain), and they brought Booch, Jacobsen and Rumbaugh together so they would stop fighting and create a "Unified" model (the creation of peace was the reason for the "U", for those of you new to the story). When this happened, agendas inevitably got mixed together, and I suspect that Rational had its own agendas which got into the mix, design-by-committee style. At the time, C++ was king of the OO languages so you see a lot of C++-isms in the UML, which introduced innapropriate details. At least one player appeared to have grand ideas of taking this abstraction tool and giving it enough detail to produce code. Of course, we see this in most of the modeling tools, which can spit out framework code from the UML, but in recent years the spectre of Model-Driven Architecture (MDA) has appeared, which was probably inevitable given the implied promise that someday UML would be the one ring that binds them all. Fortunately, the MDA experiments seem to be happening in the open-source world, so we can discover (again) that programming-by-drawing-pictures is Goedel-incomplete, without having to pay hundreds of thousands for tools that don't work hawked by vendors selling faith-based solutions, as has been the case for every "revolution" that has popped up in the past.
Because of all these misdirections, it's no wonder people get confused about what to do with the UML. Is it for modeling, or is it for detailed design? My guess is that forces within Rational wanted it to be everything to everyone, but this goes back to my original point: a method must be really, really simple in order for people to use it without getting lost.
Two things became clear about the UML on this trip. First, when walking through use cases/user stories, the static structure diagram is what appears. This is not a revelation; CRC cards or simply hunting for nouns and verbs produce the same results. However, we found that a very simple form of structure diagram would suffice, and this seemed to be very helpful for people because they could see that it was only intended to capture the domain model, and not express every possible detail about the code. Without this guidance, people can waste a lot of time filling in all the methods, arguments and fields and lose sight of the big picture.
The second thing that became clear was that a UML structure chart is a very helpful step in expressing the domain model, but it lacks an essential ingredient: mechanism. The structure chart only expresses the relationships between classes, not how they interact. Sometimes this relationship is enough, along with some words and hand-waving, to understand how the system will work. But, I began to see, that is only in the case where the mechanism is so obvious that everyone can see it. (I am aware that the UML has a various number of different diagrams that show different aspects of mechanism, but none of them are able to sum it up.)
On the other hand, expressing mechanism down to the nuts and bolts is precisely the pitfall we are trying to avoid by using a model. What's needed is the essence of mechanism, just enough so that we can see how something should work without getting lost in detail.
Could this essence be design patterns? Might we be able to see the most obvious designs because we have been schooled in the most basic of patterns so effectively that we don't even consider them patterns, but just fundamental ways of doing things, and don't even have names for them? But when we have more complex interactions, we need a pattern to describe it?
During one session, I began to see the possibility of annotating a structure chart with the mechanisms that drove the program, and this seemed to work in some situations. However, in the following consulting session the idea of a separate diagram began to appear. I started calling this a "flow diagram" because it showed the flow of objects and messages through the system of objects. This diagram does not show composition and inheritance, but rather the relationship of higher-level objects and the way they accomplish their tasks. It also seems to show threading in the system. In general, it seems to focus on the way that objects are wired together to build a system and the dynamic of that system, rather than just examining the inheritance and composition relationships (which are certainly important, but are clearly only a step in the design process). I don't have a "finished" approach for flow diagrams, but the idea continues to reappear and evolve.
Finally, it has become quite clear to me that the software design process involves incomplete information at every stage. This can be a point of contention, one that appears to be rooted in your belief system about the world. Suggest that you don't ever really know everything about a system to someone who really wants things to be deterministic from beginning to end, and you'll get a response ranging from "huh?" to downright hostile, depending on how threatening the idea is (we could probably carry this same argument into the realms of the static/dynamic language debate, exceptions, developer testing, etc.). But my experience is that, if you insist that complete information must be available at every stage, you rapidly converge on analysis paralysis on the project level, and the waterfall approach on the method level.
On the other hand, accepting that information is incomplete allows a project to become remarkably nimble. At each stage you know you will only be able to mine some of the information about the project, and when a vein begins to dry up you move on rather than wasting time trying to pry out every possible nugget. From experience, you know that you'll move forward a lot faster by moving on, into a design pass, or even by pseudocoding your design (in a couple of instances we used Python to try out a design; this worked remarkably well in producing new insights about that design). In addition, you can "backfill" user stories and design details as you uncover new factors, so you don't need to feel like an incomplete design must hold you back. The experience I had was very rapid and liberating; we could often punch through into designs and insights that had been holding people back for significant periods.
Thanks for the great post. This is exactly how I feel about modern approaches to design. UML is too complex. There are too many diagrams and too many symbols in each diagram. Often it's just enough to use a flow chart or a mind map.
I also think too many people try to treat various design approaches as rules not as guidelines. That and hidden complexity prices often make design too complicated. It becomes harder than it should be, sometimes resulting in poor software quality.
There's also a wave trend. When a new approach or methodology appears it often keeps a low profile for a long time. If after a while it survived, and more and more people learn about it, it becomes really popular. People want to stick it everywhere (often though they try to adopt what they use to be more like the new hype, than actually learn to use the "hype" correctly) only to find out it doesn't fit all shapes and sizes. For example I think UML can be good at helping to see how existing code works, how objects interact, etc. if it is generated dynamically for the existing code. I don't think it's a good tool to use for the extensive design before coding starts.
Anyway keep up the good work of analyzing how to do designs easier! I know it's a very complicate topic and I'd love to see more posts on it.
> <p>The third big issue was about the UML. Everywhere I > went, people were struggling with it, and I think the > repetition of that confusion finally made me take a hard > look at it. The UML is not only too complex, it seems to > be incomplete (or at the very least, it takes far too many > kinds of diagrams to express the design).</p>
That sums up UML very well indeed. A few years back I took a UML course. I came away liking the 'simple' diagrams that showed class relationships but found I I couldn't find a use for all the other diagrams probably because few of them could be readily expressed as code.
Feeling I had missed something fundamental, I bought Martin Fowler's "UML Distilled", a 200 page 'brief' guide to UML. This book displays the key UML constructs on its inside covers. That was the useful bit. The rest of the book confirmed my suspicions that most UML diagrams actually introduce complexity into a system in order to justify their existance.
Your summary of UML - as being an unedited and inclusive mix of rival academic notations compounded with being a selling vehicle for a major commercial software development product - is spot on.
Now is the time for UML to be put back in its place, as just a useful and commonly agreed notation for books and diagrams, and to detach it from its commercial placement as a software development methodology.
> Finally, it has become quite clear to me that the software design process involves incomplete information at every stage
Have you read Alistair Cockburn's "Agile Software Development"? Your final couple of paragraphs remind me very much of that book. He characterises software as a 'cooperative game of invention and communication', but then points out that communication is always imperfect. He suggests that the real skill lies in knowing how hard to strive to get information across and determining when we've done enough. So no documentation at all is a bad thing, but too much documentation may be just as bad.
In his words I would be a UMLSketcher or a UMLBlueprinter. Not a UMLCoder. On the other hand I allways hated documents that seemed to state: "I said it my way in visio and I hope you figure out what those arrows mean.". So I actually highly value the fact that UML is just some standard. In points I also really like it. I seem to be able to express most things with it. But yes it took a while and yes I did also get lost in details quite often.
So the conflict, that Martin Fowler lays out, remains. The UML 2.0 is in essence targeted towards MDA that does not mingle well with agile development and light design. This has the tendency to break the legitimacy of the standard, eventhough the sheer fact that it is a standard is of great value in itself.
What the standardness of UML gives us:
1.) One can express allmost anything at allmost any level of abstraction with the UML in a way that people who have read a book on UML (many now have) can understand. If used well, it is much better than text.
2.) There are tools available to create UML diagrams with a certain sense of guidance. This is not the case for visio/powerpoint-style design, where everyone uses his own idea of arrows and boxes. UML ist 1000% better than that.
Your posting seems to show, just how right Martin Fowler was. The legitimacy of the standard is crumbling as there is no standard that enforces the subset of "sketching usage". Just to say that it is not necessary to include the possible details is not a valid argument either. Maybe someone remembers the "use strict" directive in Perl. One can voluntarily make the compiler enforce better coding standards. But the tendency with programmers would be not to do so. In the end Perl lost quite some ground to languages that allways enforced the desired simplicity and rigor.
So don't get me wrong. I'm not opposed to your article. I just feel hesitant to give in to a rising Anti-UML sentiment alltogether. It reminds me of the rising Anti EU-sentiment in these day's politics.
Perhaps it would be a good idea to propose a UML Sketching Dialect to the OMG. That could include your new diagram type (I would indeed be very curious to see more details on it) and could exclude quite some detail on UML 2.0 to make it less tempting for the Sketcher to get lost. AND we could avoid going back to the "Say it your way" days.
I've used role based design in the past to great success. Essentially, it is the process of finding the "actors" (your classes/components), and then determining how they work together (the verbs/methods). The process allows for a multi-pass approach where you paint with broad strokes. When I mock it up in UML, I merely create interface boxes with the components. It has the "who" without even deciding what gets done. The next pass adds the action names (methods), and as a result we have a more useful view of how classes work together at a conceptual level.
Later on as we get deeper into implementing the work of the basic design we can use the same approach again until the unit of work is small enough to "get it" without having to resort to modeling it.
I agree on many cases with the UML diagram. I'll be completely honest when I say I could care less about the inheritance structure of a class. That doesn't tell me what the class does or what role it plays in the system. It is an implementation detail. While it is useful to have that information, it isn't necessarily useful to see it in a diagram.
I like an article I read a while back on Continuous Design, which supports alot of the same ideas in your post. You're not going to know everything all the time, although you should know enough to get the next thing done. Design evolves, embracing that fact frees you up tremendously.
> Feeling I had missed something fundamental, I bought > Martin Fowler's "UML Distilled", a 200 page 'brief' guide > to UML. This book displays the key UML constructs on its > inside covers. That was the useful bit. The rest of the > book confirmed my suspicions that most UML diagrams > actually introduce complexity into a system in order to > justify their existance.
Note that Martin often suggests that only the first half of his book is the important part.
Interesting post. I'd second the request that I'd *really* like to see some of the diagrams you came up with.
UML looks to me like it has the curse of all *diagramming* methods. It loses a sense of time in a way that text somehow doesn't. Maybe because text still has a temporal dimension (or flow down the page) you sense time passing as you read it. Looking at diagrams, your eyes wander backwards and forwards across a static landscape, trying to find the movement.
I'm a Software Engineering student - graduating in a few weeks. My final project is a Statechart code generator + simulator for a high-tech company. Interestingly, I had two problems with UML while developing the software:
1. The designer of the statecharts I had to generate code from uses UML it in a very abstract manner. For example, in actions he doesn't call real functions - he just puts some text that will mean something to someone. That's because this is the company's first attempt at code generation - and the designer "knows" that a human will read the charts and translate them into code (and not a computer).
2. When, in the middle of the year, I had to prepare UML diagrams to document my own software, I found it impossible: most work (code generation) was done in XSLT, which is not OO - so I found no matching UML diagram. Instead, I came up with a DFD that shows what my software does. Since DFDs are not part of the UML, and because I found no open-source DFD diagramming tool, I had to use... PowerPoint!
For fixing the first problem, I wish UML tools had "design modes", something based on Fowler's UML Mode (http://martinfowler.com/bliki/UmlMode.html) referred-to earlier in this thread. That is, in the "sketch" mode, the tool will hide those detailed-design features of the UML (like composition/aggregation). When running in "programming" mode, users will be able to add these details to the model. Later, when the high-level designer edits the model again, those features will be hidden - but will remain in the model. I realize that doing this correctly is not trivial - it's in my wish-list nevertheless.
The second problem is different. UML is for OOAD, not procedural/functional design. But there are still projects that use the older methods. What diagrams should they use? In my college, they teach UML thoroughly, but say next to nothing about the other options - as if one should never develop non-OO software.
These days I think almost entirely in term of protocols and interaction models.
Classes don't interest me, neither does inheritance. Its an implementation detail.
I suspect this is the result of working mostly in Objective C and Smalltalk. If I want one object to use another object for somethin, I identify the protocol that is expected by the first object and implement it as a method category on the second.
How to express protocols? By documenting them as method categories.
Artifacts are ObjectiveC header files or simply implementations.
I see very little point in Class level design as I can create and test the classes faster than I can document them in some "notation".
System architecture is the finest level of granularity at which design documents make sense in my experience.
[The reason for the complexity of the UML is, ironically, inappropriate mixing of abstraction levels]
It mixes design and implementation. With UML 2.0, MDA and related stuff I turned away from the UML completely. At my first contact with the UML ( around 5-6 years ago ) the diagrams where introduced after object oriented analysis i.e. the decomposition of a textual descriptions ( design by documentation and requirement statements ) into nouns, verbs and adjectives and their relations. I did regard the UML as a good-enough translation method into mainstream languages but I was probably wrong and I'm proved to be wrong by more current developments.
Personally I would like to see people who take a fresh look on OOA but include prepositions, adverbs and such stuff into their considerations. I would like to regard diagrams as a way to clarify thoughts visually ( "a picture tells more than thousand words" ), but not as a crippled visual equivalent of Java programs - of course I understand that people do not want to program Java, but generate it by wizards that make them efficient. I also do understand the intentions of tool vendors that want to abstract from languages alltogether. But despite all new media hypes not only books are still living but alphanumeric scripture is still the state of the art for establishing and communicating precise thoughts.
> This is the second point: people have a very difficult > time separating abstraction levels.
This is the hard part of design and design discussion. If you have a very experience team, it's easier, but if you don't, chances are design discussion goes on with no directions and no end.
1) skip conceptual/logical design and go to physical design directly. It's not uncommon that at a design review meeting, the developer can't articulate clearly what a system/component is and does conceptually.
2) at an early stage, discuss database schema and implementation details/algorithms. I've often seen that at a design review meeting, the developer doesn't present the APIs, and people don't bother to ask.
I was wondering what the guidelines of the design process (from early design discussion all the way to design review and decision) would be? or is this important at all?
Flat View: This topic has 37 replies
on 3 pages