|
Re: Seven Lessons on Scalability from Reddit
|
Posted: May 19, 2010 1:09 PM
|
|
Hi John,
> Bill, > > Stop posting this garbage and putting "What do you think?" > questions at the end. Nobody will learn anything from > Reddit, the only two people who believed Paul Graham's > essays on Lisp on the web. It is not really surprising, > given this presentation, why their Lisp-based engine > failed; they are simply bad programmers. > Sorry you didn't like the presentation, but please keep in mind that the discussion here is a big part of the educational value we try to provide. The reason I posted it is that I was surprised by some of the things in the presentation, and wanted to hear what our audience had to say about it.
As far as the thing table goes, that sounded like the "no-sql" movement to me. It seems some high-profile, high-traffic websites have been going away from sql towards big-hash-table data stores. I think that's a good topic for conversation here. I myself was surprised that the Reddit folks felt they were spending too much time making data models, because data model design seems like it should be straightforward to me. But the other reason they went away from sql, which was interesting to me, was that they had trouble updating schemas on their database while it was in use. We have never had that problem, but we don't have such a high-volume site. But a lot of people who do have high volume seem to get along fine with sql databases. So I thought that would be a good issue to discuss here too.
I also thought it was interesting that they had deamon's automatically restart things when they grew too large. I've seen that before, and as I think it is a great technique. And he said to watch the logs, so that you find out when there probably is a memory leak or other problem so you can actually fix it. That all made sense to me.
His points on storing redundant data and doing as much work offline as possible made sense to me too, the goal being make the user wait as little as possible. I thought that was useful.
It sounded like they never thought about clustering until after they needed to cluster, so they had state in their app and no way to share it between app servers. That seemed short sighted to me, but on the other hand, since most ventures don't succeed anyway, maybe it's best to throw something out there quickly and worry about clustering it later if you have enough success. Still, I'd lean towards doing a two-app server cluster at the beginning.
|
|