Summary
In his speech to the 2006 MIT Emerging Technologies Conference, Amazon.com founder Jeff Bezos noted that about 70% of a project's time and effort is spent on tasks that, while important, do not differentiate a product or a business in any way from its competitors. Bezos suggested that such features, or "muck," may be outsourced to third parties.

An often-used criteria in measuring a developer's effectiveness is how quickly a developer can complete a given task. That's because most organizations are able to break down a bigger project, such as a new product release, into smaller tasks, and to the extent those tasks involve writing code, prefer to hire developers who can execute those tasks fast.

Thus, from a developer's point of view, getting something done quickly is a positive. While most developers would want to proceed with a task as quickly possible, many things slow a developer down on the way to the finish line. Apart from considerations of quality, and human factors—which are not the subject of this post—less glamorous forces can loosely be grouped under the term "infrastructure."

By infrastructure, I mean those aspects of a project that do not directly define a product or a service, but are nevertheless necessary ingredients of the solution, in the way a road is a necessary, but not sufficient, element in reaching a terrestrial destination.

While infrastructure elements are seldom defining components of a product, they have a big impact on the productivity of each developer working on that product or project. And that's not just because developers have to work with such infrastructure elements, but also, and mainly, because developers are often tasked with creating that infrastructure in the first place.

According to Amazon.com founder Jeff Bezos, on average we spend about 70% of our time on dealing with such tasks—or "muck," as he calls those chores—instead of developing the aspects of our products that really matter. In addressing the MIT Emerging Technologies Conference, Bezos noted that,

Everybody who ever started a successful company wants to ... go directly from [an] idea to a successful product. The reality is that there are a lot of obstacles between your idea and a successful product...

Your major bottleneck in developing your ideas and your successful products and service offerings turns out to be something that's actually not unique to your business at all. It's completely undifferentiated. This is certainly true at Amazon.com, and it's certainly been true from the very beginning from one person to twelve thousand people: that at least seventy percent of your time, energy and dollars, go into this back-end heavy-lifting infrastructure. I call this heavy-lifting, muck.

The thing about this muck is, you have to do it, unfortunately, at the highest quality level. Because if you don't do all of these things that have nothing to do with your business at superb quality levels, your idea will never succeed...

So you have your idea, and between you and your idea is a bunch of undifferentiated heavy lifting. And you have to cycle through that loop, and the winners are the ones who can cycle through that loop faster than others [can]. So you end up being slowed down.

I was not completely surprised to hear that 70% of the effort of a business is spent on non-differentiating infrastructure components. If you ever worked on user management, authentication, discussion forums, inventory management, and so on, many of those features might very well all belong to the 70% of necessary, but un-differentiating, product features. Many projects repeat core infrastructure-level features over and over, slowing that business down in delivering features that differentiate the business from its competitors.

It is inevitable that products and innovations will help reduce the time developers spend on non-differentiating tasks. The question is what form those services and tools will take.

In the past, such progress occurred by raising the level of abstraction developers work with. Economies of scale have allowed operating system vendors to completely automate tasks such as saving data to disk or shuffling bits between network nodes. While current tools attempt to make developers faster in developing code, if the past is any indication of things to come, products and tools will help entirely eliminate the need to develop large areas of a project's code, perhaps, again, by raising the abstraction level within the context of an application domain.

Spending some time learning Rails lately, I saw some of that already in the domain of Web applications. Many developers like Rails because it reduces the chunk of time spent on "muck," allowing a developer to shift that time to producing features that can differentiate a product or a business. Figuring out how to display error messages, or perform input validation, or even to sign up and manage users, are hardly factors that make or break or business. As Bezos said, those things also have to be done very well, but are alone insufficient in creating a compelling application.

While toolkits, such as Rails, reduce "muck" to a great extent, in the future network-based services may speed up a developer's work even more. Instead of creating a user administration framework, we might be able to just invoke a network service that helps manage users. Or, instead of writing code to save images to a file system, we might just invoke a network storage service, such as Amazon's S3 Web service.

Many companies and development teams have worked so closely with their applications, and for so long, that they have over time come to believe that almost every aspect of their application is unique, and that most every feature helps differentiate the application from the competition. In reality, only a few features or areas of functionality are truly crucial for a business' success. Because development resources and time are finite, the more honestly a business or development team can asses what those critical, differentiating feature are, the more likely that team will succeed. And, equally important, the more productive and speedy developers will feel.

But what to do with the rest of the application, with the necessary but non-differentiating "muck"? As countless hours are spent re-inventing the wheel and developing non-differentiating features, the pressure will increase to outsource such features to network-based services. That will especially be the case as such services evolve into viable alternatives in the coming years.

If you took an honest look at your application, what percentage of your work would you say is dedicated to truly differentiating features? And to what extent would you trust third-parties to handle your application's "muck"? Would you trust Amazon.com to, say, manage your application's users or your users' files? How much more productive do you think you would be if you could outsource those essential but non-differentiating functions?

Talk Back!

Have an opinion? Readers have already posted 21 comments about this weblog entry. Why not add yours?

RSS Feed

If you'd like to be notified whenever Frank Sommers adds a new entry to his weblog, subscribe to his RSS feed.

Digg | del.icio.us | Reddit

About the Blogger

Frank Sommers is a Senior Editor with Artima Developer. Prior to joining Artima, Frank wrote the Jiniology and Web services columns for JavaWorld. Frank also serves as chief editor of the Web zine ClusterComputing.org, the IEEE Technical Committee on Scalable Computing's newsletter. Prior to that, he edited the Newsletter of the IEEE Task Force on Cluster Computing. Frank is also founder and president of Autospaces, a company dedicated to bringing service-oriented computing to the automotive software market. Prior to Autospaces, Frank was vice president of technology and chief software architect at a Los Angeles system integration firm. In that capacity, he designed and developed that company's two main products: A financial underwriting system, and an insurance claims management expert system. Before assuming that position, he was a research fellow at the Center for Multiethnic and Transnational Studies at the University of Southern California, where he participated in a geographic information systems (GIS) project mapping the ethnic populations of the world and the diverse demography of southern California. Frank's interests include parallel and distributed computing, data management, programming languages, cluster and grid computing, and the theoretic foundations of computation. He is a member of the ACM and IEEE, and the American Musicological Society.