Common wisdom, especially in distributed computing, says that the right approach to all problems is to use a standard. This common wisdom has no basis in fact or history, and is curtailing innovation and rewarding bad behavior in our industry.
We are living in the era of standards. Technology that is a standard is
good because it is a standard; if something is blessed as a standard it can
be seen as open, non-proprietary, and gets the blessings of the IT managers
and the analysts. Something that is not a standard is closed, proprietary,
and to be avoided at all costs. This is the received wisdom in our
industry; it is so obviously true that we don't even question it, and we
seem to get uncomfortable around those who ask why standards should always
This kowtowing to the god of standards is, I believe, doing great damage
to our industry, or craft, and our science. It is stifling innovation. It
turns technical discussions into political debates. It misunderstands the
role that standards have played in the past. Worst of all, it is leading us
down absurd technological paths in the quest to follow standards which have
never been implemented and aren't the right thing for the problems at hand.
Let's get back to first principles...standards have historically played
two roles in our industry. The first role has been to codify what is
already common practice in the industry. Such standards are attempts to
capture what everyone is already doing, perhaps ironing out some minor
inconsistencies along the way. But such a standards effort is at base a
descriptive one. The original IP standard was such a beast; so was the ANSI
C standard. Neither attempted to invent much of anything; both simply tried
to make clear what everyone was already using. The standards being
described were already de facto standards; writing them down was a
documentation job that either got it right or missed the mark.
This is very different from a standards effort that attempts to invent
the standard from the ground up. Such a standards effort is not
descriptive, but rather an attempt to invent by committee. There is no
common practice to describe; instead the standards group is trying to tell
everyone what they should be doing in the future. The ANSI C++ committee,
which started out as a descriptive body, became one of these sorts of
standards groups when it was decided that they should "improve" the
language. Many of the Object Management Group (OMG) Common Object Request
Broker Architecture (CORBA) Common Services groups were this sort of
standards body. There are lots of others around now; I'm sure you can name
many of them so I won't. These are the de jure standards groups,
and they are the ruin of our industry.
Those of you who have been part of a de jure standard effort
know what I mean when I say that they turn all technical questions into
political ones. Discussions may appear to center around the technology, but
the real decisions are made either in setting up the requirements (beware
the use case, my son) or in meetings outside of the general discussions
between subsets of the committee who trade votes like Chicago precinct
bosses. The ultimate threat in such organizations is that some of the
founding (or important) members will split off and start a competing
standards group, which seems to be happening on a weekly basis in some of
these discussions. The real goal in all of this, of course, is to get your
competitors to (unknowingly) agree to some approach to technology that you
and your company already have in a product (either shipping or ready to
go). Then what would otherwise be proprietary can now be seen as an "open"
standard. Just an open standard that only you have.
What gets lost in all of this, of course, is whether the technology
being blessed by the standard actually helps the customer to solve a real
problem. Can the standard be implemented? Will the implementation perform
adequately? How much will it cost? All of these sorts of questions are not
appropriate in the era of de jure standards.
What is also forgotten in all of this is how fragile the de
jure standards have been in the past. I can't think of a single
standard that was invented by committee that has survived in the
marketplace. The long-standing standards are those that were first de
facto standards, and were described (no invented) by the standards
Such standards didn't start out in a standards body. They started out
solving problems. Because they solved the problems, people used them. The
use drove the standard, not the other way around. This allows innovation,
this allows technical progress. Things that work get used by people who are
trying to solve problems.
This does take the decision making power out of the hands of the
managers, and the IT departments, and the technical analysts. They aren't
trying to solve problems in new ways; they are trying to lead parades, or
keep their jobs, or show that they have influence. They aren't the
engineers that can actually understand the solutions, but they do (for the
most part) understand politics. Standards groups cater to their expertise,
not the expertise of the engineer.
Of course, this hard dichotomy is something of a caricature. There are
substantive discussion of technical merit in standards groups that are
trying to invent. But that isn't all that goes on. And certainly there is
little evidence that the best technology wins in such groups. Just look
around you for evidence of that.
So the next time you are talking to a manager and he or she tells you
that you have to use something "because it is a standard", push back. Ask
why only standards can be used. Ask if the standard has actually been
implemented, or if the standard will really solve the problem under
discussion. For that matter, ask if the manager really knows what the
standard is. If any of these questions can't be clearly answered, may the
standard isn't the way you should approach your problem.
I agree, but the process you decry does occasionally produce useful results.
Sometimes a standard is needed in order to allow companies to go off and build interoperable products. At the outset there may be a general understanding within the technical community of the approach(es) that may work, but nothing that can simply be adopted as "the standard".
MPEG-2 is the best example I can think of a de jure standard that turned out reasonably well. And of course, that same group then went on to create MPEG-4, which became unbelievably bloated with everyone trying to get something in so as to participate in the patent pool.
<p>I agree wih much of what Jim says, but I also think our industry is self-regulating. The examples he uses of bad standards (CORBA) have not gained traction. We may waste significant time and energy debating and experimenting with bad standards but in the end we go on to create something better. <p>In the end it is all down to creating better solutions to real problems. A way to evaluate standards is asking <i>"does this standard create a useful, new and distinct tool to solve a set of problems"</i>. This of course forces us to define the problem we are trying to solve. <p>We can all move the industry forward by articulating real problems we are trying to solve. Standards should serve the problem not the other way round.
> <p>I agree wih much of what Jim says, but I also think our > industry is self-regulating. The examples he uses of bad > standards (CORBA) have not gained traction. We may waste > significant time and energy debating and experimenting > with bad standards but in the end we go on to create > something better.
Hmm, well self-regulating we might be but what criteria does that self-regulation apply in coming to it's conclusions?
Sadly, I'd venture to suggest that the criteria used are mostly based on how much hype something gets and the various outrageous promises made about it's benefits. Worse, I think a number of the supposed "thought leaders" we allow to influence us are the least qualified to be making suggestions.
The fact that we then "waste significant time" playing with these bad standards is not a good thing. Might there not be something more useful to do?
How many better solutions have we lost/missed because we were busy wasting that time? How much better off could we be had we not taken all those blind alleys (and I'd happily cite some which I believe could have been avoided)?
>This kowtowing to the god of standards is, I believe, > , doing great damage > to our industry, or craft, and our science. It is > s stifling innovation. It > turns technical discussions into political debates. It > t misunderstands the > role that standards have played in the past. Worst of > f all, it is leading us > down absurd technological paths in the quest to follow > w standards which have > never been implemented and aren't the right thing for > r the problems at hand.
Yep - you've just described my opinion of the J2EE/EJB fiasco. A ridiculously overblown "architecture" that is a poor fit for most enterprise apps (which are typically CRUD on the web).
Now if we could get Sun & Co to shut up about it we could explore some decent architectures for putting apps on the web.
Jim has seen much more of the standards-setting environment that I have. The article is interesting, but it seems to be pulling its punches. It is less risky to point to CORBA because it is not in the limelight.
I'm asking this will all respect: it would be good to hear Jim name names with regard to the standards efforts currently drawing industry attention. I'd like to read more of Jim's insight as applied to concrete, present day examples.
The remedy Jim suggests of pushing back at management direction outright should not be undertaken without having your resume and cover letter in order.
A better (albeit more difficult) approach is to educate management about what is better for the development effort. This can be a political exercise, requiring patience. Pushing back outright will result, at the very least, in your being regarded as a stubborn technologist -- the end result being a glass ceiling on your career, and dust gathering on your suggestions.
Jim, correct me if I am wrong but CORBA itself would not be an example of a de jure standard but the CORBA services spec may well be. I would dispute the view that CORBA is dead because it is still the main thing in use in non Java circles. Java as well uses CORBA - RMI and the Java Orb are CORBA in essence. CORBA itself grew out of real efforts in distributed computing (i.e. there were many CORBA like products and they were trying to standardize on how distributed computing should be done).
Also the comment that these technological mistakes fail eventually misses the point. Of course they fail eventually but how many bad systems and how much energy will be spent before they do.
It might well be the case that official, de-jure standards are important for large organizations. But I found somewhat the opposite at smaller organizations: The worship of de facto standards, at the almost complete exclusion of anything else. A few years ago, why trying to suggest Java as a language for a project, I faced a similar response many times, "But don't you see that everyone uses Windows?" So people just wanted to see Win32-based apps. That's probably true even today when it comes to desktop clients. Companies with limited resources must support the de facto standard(s), and in general have little capabilities to venture into the lofty terrain of standards bodies and collective specification writing efforts (that just doesn't help pay the bills).
Another thing I noticed is that even larger companies have lately become more cautious in their support of de-jure standards. In the case of Web service standards, while on the PR level most big tech companies pray daily at the altar of SOAP, UDDI, etc., in reality few have actually put big resouces into coming up with useful Web services based on those standards. What I'm seeing in that space, instead, is a wait-and-see attitude, where small developers, often working on open-source projects, write simple Web services, and big companies try to watch that and learn from the early mistakes. What XML-based Web services does IBM, Sun, Oracle, or Microsoft provide? Not many.
Lately I also noticed that official standards bodies have become rather attuned to the real world. There is the case of J2EE, where an official standard exists. But just yesterday I read that JBoss no longer seems to care if they get J2EE certified, as they might already have a large-enough user-base to whom that official blessing matters less. And when it came for Apache to work with the JCP's servlets expert group, it was clear from the beginning that Apache had the real power, because they had the user base. If the JCP did not officially bless Tomcat as a servlet implementation, they could have just called it xlets or z-lets, or whatever, because their users would have just kept using whatever that API was called. If Apache pulled out of the JCP (the standards process), it would have hurt the JCP much more than it would have Apache.
Finally, even with the Jini ServiceUI standard, folks were using that API in shipping products long before it became an official Jini community standard. Now with the security APIs, which have passed the official Jini standards process, we'll have to see how they start showing up in Jini-based products.
So, I think the increasing pressure for enterprises to deliver shipping, working (and profitable) products on short time frames has swung the pendulum towards de-facto standards - tried and tested things that just work, and usually work pretty well.
Funny world. Years ago we found ourselfs in a world of incompatible systems and a bunch of software islands in the IT world, which was described as the crux of the computer science. The last years there were great efforts to solve this problem by establishing de facto standards. Living in a world of enterprise software development I must say this was a great step forward and solved a lot of our problems with this isolated applications.
Now standards are bad! Shall we go back into the seventies developing software with some proprietar language on some proprietar sytem which cannot coporate with some other proprietar system?
Here are some rules you cannot break. Now go break the mold! :-D
Standards should come into play after a particular technology has matured enough that it wants to communicate and work with other technologies, and they want to communicate with it.
Imposing a complete, consistent, overall standard is fine for a set of technologies that all have the same underlying design philosophy and aim, but really, that's not going to be innovation, it's renovation.
I couldn't agree more. It seems that the job of standards committees, who were tasked with documenting what was working, has been turned on its head.
And how could that not be. After all, a standard that has risen from the user-base is one that is proven. It "works". Making that practice a standard is simply recognizing what is.
But when you start by designing something by committee, establish it as a standard, then set that standard out into the real world, failure is almost guaranteed.
More philosophically, the difference we are talking about here is market vs. command driven. Something that becomes a standard is market driven. It has survived the Darwinian experience and thrived. But the command-based standard, one built by a committee, can expect no more success than any other command-derived item when having to compete in a market. History bears this out as well.
The Internet was not built based on standards. In "Where Wizards Never Sleep", the story of the founding of the Internet, not one standard was used to design and build ARPA-net. Standards eventually emerged, SMTP, IP, TCP, FTP, etc.. But they were after-the-fact.
"Fire in the Valley", the story of the birth of the personal computer industry, documents one of the most innovative periods in our history. Instead of looking for standards to build their computers around, Apple, Osborne, etc. created their products, from which eventually standards emerged. Again, after-the-fact.
Go back further in time. IBM was not the industry leader from the 1930's til the 1980's because of industry standards, but rather IBM's own, internal, proprietary standards.
Standards are a nice way to not have to innovate and think. Just use a standard! But is what is developed by a committee always the best thing from a time or technology standpoint? Isn't it a sort of socialist or communist type approach, a Borgian way of doing things?
> <p>What is also forgotten in all of this is how fragile > e the <em>de > jure</em> standards have been in the past. I can't think > k of a single > standard that was invented by committee that has > s survived in the > marketplace. The long-standing standards are those that > t were first <em>de > facto</em> standards, and were described (no invented) > ) by the standards > bodies.
I believe that MIME was entirely de jure, with some possible inheriting of some common practice. But you're right -- having participated in the creation of S/MIME, I know that de facto followed by de jure is an excellent progression. For a current example, see Jabber / XMPP.
The forces acting on standards are not in the least bit akin to those of natural selection. In fact the phrase 'natural selection' was chosen specifically by Darwin to highlight the difference between the proposed theory of evolution, and artificial selection which comes into play through reasoning agents.
Hence the selection and implementation of standards is artificial selection, and thus not Darwinian.
> Darwinian forces? > > The forces acting on standards are not in the least bit > akin to those of natural selection. In fact the phrase > 'natural selection' was chosen specifically by Darwin to > highlight the difference between the proposed theory of > evolution, and artificial selection which comes into play > through reasoning agents. > > Hence the selection and implementation of standards is > artificial selection, and thus not Darwinian. > > A small point of information, but a crucial one.
OK but go back and read his post again. He was calling Market forces Darwinian not standards committee output. Reasoning agents don't really drive the market in the sense that you need here to say it is artificial selection. In fact looking at what usually occurs I would deny that people who make technology decisions are reasoning agents ; )
Flat View: This topic has 17 replies
on 2 pages