Re: Twitter on Scala
Posted: Apr 5, 2009 8:00 AM
> This is the best writeup I've found about it:
I think an opportunity may have been missed here. First, there's some inaccuracy in some of these articles. For example, this one:
The company is leaving behind a programming language that has caused it much pain in the past, and instead embracing a new and somewhat obscure language called Scala.
That's not really correct I'd say. They aren't leaving behind Ruby, but still using it, just not for everything they used to use it for. What they are using it for is the Twitter website, and although that may not be the largest portion of Twitter's traffic compared to API traffic, it still must be pretty huge. And what they said in my interview that "Rails works great for doing front-end web development." This is at Twitter. So isn't that a good PR opportunity for Ruby?
Another thing is that I'm confused why it is such a concern to some Ruby folks that Twitter would opt for a different technology at this stage for part of their infrastructure. I thought that was part of the dynamic language story. First, Matz himself explained that the prime focus on Ruby was never performance. It was on making the programmer productive:
In my point of view, efficiency and productivity are the same thing, because we focus on programming efficiency, not runtime efficiency. I don't care about runtime efficiency. The computer goes faster and faster every year, so I don't really care about performance. Actually, I care about performance when I'm implementing the Ruby interpreter, but not when I'm designing Ruby. The implementer should care about performance, but the designer shouldn't. If you care about performance, you will focus on machines in design instead of on humans. So you shouldn't focus on performance in design.
And Guido van Rossum, the creator of Python, didn't seem at all bothered by people replacing a Python app with Java or C++ later. In this interview:
I'm not particularly worried by the fact that people say you can prototype more easily in Python, but eventually the Java version makes it easier to build a robust large system. You can prototype in Python. Once you've explored the space more, you can do the planning and design that the Java version requires. If you start writing in Java knowing as little as you did when you started writing the Python version, you'll waste way more time exploring than actually building the system you'll eventually build.
The Python world has some nice examples of this. Early on, when the Web was just becoming interesting for things like shopping online, a small company called eShop was developing various commerce servers. eShop had proprietary protocols and proprietary applications, and it realized that it should just be able to use a Web server and Web browser. The developers decided to do a prototype in Python. Because they used Python, the eShop developers were the first to release a beta version compared to many other startups doing exactly the same thing. They never actually released working code after their beta release, because the company was acquired by Microsoft. Microsoft spent two or three product revisions to eventually replace all the Python code with C++. But if eShop hadn't started in Python, it would never have released something interesting enough for Microsoft to acquire in the first place.
That also happened with Yahoo. Yahoo Mail started out as a successful Python application. Again, because the developers used Python, they could respond quickly to the user feedback. And that's an application that almost everybody can use. They saw many things wrong with their application, and they responded to that quickly and added new features. Because they were doing something new, they didn't know exactly what people would need from an email Web application. It is different from a program that has your email on your computer. Access times are different. All sorts of things are different. So they were learning about what those differences were. And again, I think Yahoo may now have replaced all the Python code with C++ or some other language, but the Python prototype was essential in order to get there.
So my take on Twitter was they were just doing this. There was a part of their application that they decided to reimplement so it would perform better.
As far as why they didn't do JRuby instead of Scala, I think frankly they wanted the threads, GC, etc., of the Java VM and static typing and a flexible language like Ruby. Well that's not JRuby, because JRuby is statically typed, but Scala fits that perfectly.
So anyway, I think part of the problem is the slant that the reporters took when writing their articles about this. This one seems pretty balanced:
And the text of this one seems OK, but the title is a bit sensational:
In the body of the article, they quote Alex Payne saying, "We're still happy with Rails for building user facing features... performance-wise, it's fine for people clicking around web pages." But then the title is "Twitter Jilts Ruby for Scala." The Register tries to be in your face in general, and so making a title like that isn't out of character, but I think it gives the wrong impression. My impression was that Twitter still loves Ruby and especially Rails, they just don't want to use it for everything. I wouldn't call that "jilting" Ruby.
The Unlimited Novelty blog seems to be basically saying, "It's not Ruby's fault, it's the fault of Twitter's engineers." Wouldn't calling out these misrepresentations in the articles, and countering that Twitter's continuing confidence in Rails for their website proves Ruby on Rails to be very scalable be a better response?