The Artima Developer Community
Sponsored Link

Java Community News
Amazon's Architecture and Lessons Learned

1 reply on 1 page. Most recent reply: Oct 23, 2007 5:46 AM by Steve Ghuman

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 1 reply on 1 page
Frank Sommers

Posts: 2642
Nickname: fsommers
Registered: Jan, 2002

Amazon's Architecture and Lessons Learned Posted: Sep 17, 2007 10:42 AM
Reply to this message Reply
Summary
Todd Hoff posted a description of Amazon.com's architecture and the lessons learned from building and operating the Amazon.com infrastructure.
Advertisement

Todd Hoff posted a short description of Amazon.com's internal architecture, and of the lessons learned from building and operating the Amazon.com infrastructure, on High Scalability.

Some of the more interesting points include:

  • You must change your mentality to build really scalable systems. Approach chaos in a probabilistic sense that things will work well. In traditional systems we present a perfect world where nothing goes down and then we build complex algorithms (agreement technologies) on this perfect world. Instead, take it for granted stuff fails, that's reality, embrace it. For example, go more with a fast reboot and fast recover approach. With a decent spread of data and services you might get close to 100%. Create self-healing, self-organizing lights out operations.
  • The only way to manage a large distributed system is to keep things as simple as possible. Keep things simple by making sure there are no hidden requirements and hidden dependencies in the design. Cut technology to the minimum you need to solve the problem you have. It doesn't help the company to create artificial and unneeded layers of complexity.
  • They build their own infrastructure for performance, reliability, and cost control reasons. By building it themselves they never have to say Amazon went down because it was company X's fault. Their software may not be more reliable than others, but they can fix, debug, and deployment much quicker than when working with a 3rd party.
  • Use measurement and objective debate to separate the good from the bad. I've been to several presentations by ex-Amazoners and this is the aspect of Amazon that strikes me as uniquely different and interesting from other companies. Their deep seated ethic is to expose real customers to a choice and see which one works best and to make decisions based on those tests.
  • Innovation can only come from the bottom. Those closest to the problem are in the best position to solve it. any organization that depends on innovation must embrace chaos. Loyalty and obedience are not your tools.
  • Getting rid of the influence of the HiPPO's, the highest paid people in the room. This is done with techniques like A/B testing and Web Analytics. If you have a question about what you should do code it up, let people use it, and see which alternative gives you the results you want.
  • If you have a new business idea or problem you want to solve you form a team. Limit the team to 8-10 people because communication hard. They are called two pizza teams. The number of people you can feed off two pizzas.
  • Between 100-150 services are accessed to build a page.
  • Amazon doesn't like middleware because it tends to be framework and not a tool. If you use a middleware package you get lock-in around the software patterns they have chosen. You'll only be able to use their software. So if you want to use different packages you won't be able to. You're stuck. One event loop for messaging, data persistence, AJAX, etc. Too complex. If middleware was available in smaller components, more as a tool than a framework, they would be more interested.
  • Eric Brewer's CAP Theorem or the Three properties of Systems... Three properties of a system: consistency, availability, tolerance to network partitions... You can have at most two of these three properties for any shared-data system... To scale you have to partition, so you are left with choosing either high consistency or high availability for a particular system. You must find the right overlap of availability and consistency.

What do you think of the key points mentioned in Hoff's post?


Steve Ghuman

Posts: 1
Nickname: stevegh
Registered: Oct, 2007

Re: Amazon's Architecture and Lessons Learned Posted: Oct 23, 2007 5:46 AM
Reply to this message Reply
I've created a small utility app that does that automatically takes whatever URL is in the clipboard , reformats it to be an Amazon referral URL and puts the result in the clipboard. Very easy and fast

Flat View: This topic has 1 reply on 1 page
Topic: NetBeans 6 Second Beta Released Previous Topic   Next Topic Topic: Jacob Kaplan-Moss Reviews CouchDB

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use