The Artima Developer Community
Sponsored Link

Java Community News
Cedric Beust's Verdict on Erlang

32 replies on 3 pages. Most recent reply: Oct 12, 2007 4:11 PM by Isaac Gouy

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 32 replies on 3 pages [ 1 2 3 | » ]
Frank Sommers

Posts: 2642
Nickname: fsommers
Registered: Jan, 2002

Cedric Beust's Verdict on Erlang Posted: Oct 3, 2007 4:31 PM
Reply to this message Reply
Summary
In a concluding piece about his investigations of the Erlang language, Cedric Beust is not convinced that Erlang lives up to all the claims its supporters make about the language.
Advertisement

In a follow-up to his earlier blog post reviewing Joe Armstrong's book, Programming Erlang, Cedric Beust discusses several claims Erlang advocates often make about the language. Beust's verdict on Erlang: the language does not live up to all the claims its supporters make about Erlang's effectiveness to create robust systems.

Regarding scalability, for instance, Beust notes that:

I have always had problems with this claim, and even after reading Armstrong's book, my skepticism has not abated. I can definitely see that the syntax of the language makes it easier to write message-based components which, I agree, are probably easier to scale than locking intensive languages such as Java, but syntax itself is not enough. Just because you write a program in Erlang doesn't mean your code will scale better on multiple processors than if you had written it in Java.

Erlang's notion of lock-free concurrency also does not escape Beust as a claim that's hard to live up to:

I am also beginning to question a few claims that are often tossed around when discussing about Erlang, such as the fact that since Erlang doesn't have any variable nor side effect, it is contention free, and therefore doesn't suffer from lock issues that traditional languages have...

There is one crucial bottleneck that's never even mentioned in the entire book: the message inbox.

The only way a process can modify the state of or talk to another process is by sending a message to that process. Regardless of how lightweight the implementation of this message passing is, there has to be a lock on all these inboxes to guarantee that when dozens of processes send messages to your own process, these messages are delivered and treated in the order they are received from.

In effect, Erlang seems to be moving locks scattered in various places throughout your code (e.g. synchronized blocks in Java) into one single bottleneck. No matter how efficient this bottleneck is, I'm having a hard time imagining that this approach scales as miraculously as Erlang advocates claim. At least, Java gives me complete control on the coarseness of the locks that I use, and also absolute freedom on where I want to place these locks. If an object can be accessed by only two processes, only these two processes will compete for this lock. In Erlang, these two processes will have to pass messages to the inbox of my process, thereby competing with *all* the processes of my application. And since Erlang programmers want you to create as many processes as you possibly can, this can represent a lot of contention and wait times.

Beust also notes that Erlang programs tend to be very slow, and that even when multiple processes are created, the entire computation can be performed only as fast as the slowest component in the system allows.

What do you think about Beust's criticisms of Erlang?


Shahzad Bhatti

Posts: 2
Nickname: bhatsha
Registered: May, 2005

Re: Cedric Beust's Verdict on Erlang Posted: Oct 3, 2007 6:37 PM
Reply to this message Reply
Though, I already replied to his comments on my blog at http://weblog.plexobject.com/?p=1586. But here are my thoughts on his comments:

Object Oriented
His first objection is that Joe is biased against OO and Erlang is not an object oriented language. I can’t say having OO is must for every good language. Erlang is fundamentally functional language with good support for concurrent programming. And as I wrote earlier on http://weblog.plexobject.com/?p=1572, Erlang is based on Actor model, where each object has a mailbox associated that receives messages. In this model, the message passing takes literal form rather than method invocation. Also, according to GoF author Ralph Johnson sequential Erlang is functional and concurrent Erlang is object oriented. And I admit method invocation based object oriented is easier than message passing, but it’s just matter of style. I believe abstraction, encapsulation, separation of concerns are more important than inheritance or combination of state/behavior.


Scalability
His second objection is Erlang claims to be scalable, but it only provides primitives to build scalable systems. Though, I agree building scalable systems requires scalable architecture and just using Erlang will not make your system scalable. I also agree that you can build scalable systems in any language. I have been building distributed systems for over ten years and I have used CORBA, EJB, messaging middlewares, JINI and all kind of RPC or Web Services. In a lot of ways OTP gives you comparable features, but all that is well integrated into Erlang platform. Also, one of basic requirement for distributed systems is building monitoring and fault tolerant system. Back in 90s when I worked on CORBA, I build pretty complexed monitoring system that allowed sysadmins to watch all servers on machines and see their health. The system also provided ability to automatic and manual start/stop of processes when they crashed. All that was a lot of work, but OTP gives you that capability right away. These days you can also get similar features from application servers and commercial monitoring systems, though open source app servers still lack in that area.


Performance
Cedric also raised recent performance problem that Tim Bray had processing large files in Erlang. Erlang was built for telecommunication systems and though it’s now being marketed as general purpose language, but it still lacks many of the features for general purpose language. Slow I/O or slow regular expression is one of them. I hear a lot of people dismiss performance by just saying computing cycles are cheap and human cycles matter more, and just throw another box. I think CPU cycles still count, especially when difference is in the order of magnitude than other solutions. I also heard from Ebay architect Dan Pritchett, how building large data centers are running into a wall because of power demands. Erlang works best when your application is not CPU-bound, so in that case I would not recommend using it. I think now that Erlang is under spotlight, the future releases of Erlang will address slow I/O, regular expressions and commonly used primitives.


Five 9s
In Erlang fault tolerance is very different from traditional approach in Java, where server tries to handle exceptions and keep running. In Erlang when an error occurs the process simply dies (if it chooses) and supervisor can restart it. Again, this can be built in any language. Erlang provides nice primitives for building fault tolerant systems, though I agree that Erlang does not give this for free, but you still have to build it. Cedric omitted hot code swap, which I have seen only in commercial application servers such as Weblogic and Websphere.


Lock free programming
I have been writing multi-threaded applications for over ten years and I like Erlang’s approach because I have spent too much time in fixing locking and synchronization problems. Cedric complains that Erlang uses locking internally and I am fine with that as long as it’s not exposed to the APIs. Though, I admit in certain cases locking can be useful. For example, one of the exercise from Joe’s book asks reader to implement a correct way to register a process when multiple processes trying to do that simultaneously. And I have seen a lot of clever solutions for that, which would have been easier with locking. I don’t like clever code because it becomes maintenance and debugging hell, so I would have liked simple locking in that case.


Distributed Data
In conclusion, Cedric mentions how distributing data is easier than distributing code. I agree and that is why I like Mnesia instead of centralized database approach. Our CTO Werner Vogels at Amazon has been talking for a while on CAP principle and how databases don’t scale. In one of the system we actually had a database that could not scale and was replaced by a collection of distributed berkley databases. Erlang gives you that capability with Mnesia.


Conclusion
I think Erlang is a specialized language that may not be suitable for all applications, and I have been trying to learn it myself and writing a few applications. I like skepticism that Cedric showed, I think it’s important to have a healthy skepticism when trying new thing. I see too often, a new thing is simply marketed as a Silver Bullet. Also, the authors or makers of new product have their own agenda. Though, you cannot judge or give a verdict just by reading a book. You have to try it yourself in a real project. You can only learn something by trying its limits and not by just reading a book. I think the new era of application development is multi-language, where you use right language for the right task and Erlang is just another tool that will only work for the kind of problems that it inherently was designed to support.

Nemanja Trifunovic

Posts: 172
Nickname: ntrif
Registered: Jun, 2004

Re: Cedric Beust's Verdict on Erlang Posted: Oct 4, 2007 5:36 AM
Reply to this message Reply
I don't think that just reading a book on a subject makes a person qualified to make verdicts.

nes

Posts: 137
Nickname: nn
Registered: Jul, 2004

Re: Cedric Beust's Verdict on Erlang Posted: Oct 4, 2007 7:30 AM
Reply to this message Reply
Cedric's Verdict on Erlang: The Verdict: Inane

nes

Posts: 137
Nickname: nn
Registered: Jul, 2004

Re: Cedric Beust's Verdict on Erlang Posted: Oct 4, 2007 8:48 AM
Reply to this message Reply
Just to clarify my previous comment. Cedric’s argument boils down to: everything you can do in Erlang I can do in Java also. It completely misses the point of using Erlang. Everything you can do in Java you can do in Assembly also.

The point of using Erlang is not that it automatically makes everything concurrent or that there is no contention over resources ever. The point is that it makes it easier to write concurrent applications that work properly.

Brad Schneider

Posts: 5
Nickname: snide
Registered: Jun, 2006

Re: Cedric Beust's Verdict on Erlang Posted: Oct 4, 2007 8:48 AM
Reply to this message Reply
Therein lies the Catch-22 of Erlang adoption: Developers should use Erlang because their intuitive approach using mainstream technologies is inherently flawed; Because their intuition is inherently flawed, developers will dismiss Erlang.

Thought exercises are worse than useless in evaluating a technology such as Erlang. Unfortunately, there are not a lot of real world, comparative case studies to lean on, so short of a leap of faith and substantial development effort, there is no way to really prove out the fundamental Erlang proposition.

Cedric Beust

Posts: 140
Nickname: cbeust
Registered: Feb, 2004

Re: Cedric Beust's Verdict on Erlang Posted: Oct 4, 2007 9:27 AM
Reply to this message Reply
Thanks for the thoughtful analysis, Bhatsha, it was an interesting read.

Nemanja: you're making assumptions on my level of involvement with Erlang. Why not focus on what I wrote and comment on it, like Bhatsha did?

My post raises concrete issues that I would love to see discussed... concretely, and not with vague hand waving.

--
Cedric

Nemanja Trifunovic

Posts: 172
Nickname: ntrif
Registered: Jun, 2004

Re: Cedric Beust's Verdict on Erlang Posted: Oct 4, 2007 10:56 AM
Reply to this message Reply
> Nemanja: you're making assumptions on my level of
> involvement with Erlang.

The very first sentence in your blog says: "Ever since I finished reading Joe Armstrong's "Programming Erlang", several details have been nagging me but I haven't been able to put my finger on them... until now."

That leads me to believe that your only involment with Erlang was reading the abovementioned book. But if I am wrong - I appologize. In any case, if you mentioned the number of years you spent developing with Erlang, it would certainly add to the credibility of your assesment.

Frank Sommers

Posts: 2642
Nickname: fsommers
Registered: Jan, 2002

Re: Cedric Beust's Verdict on Erlang Posted: Oct 4, 2007 3:34 PM
Reply to this message Reply
> That leads me to believe that your only involment with
> Erlang was reading the abovementioned book. But if I am
> wrong - I appologize. In any case, if you mentioned the
> number of years you spent developing with Erlang, it would
> certainly add to the credibility of your assesment.

I have to admit that I have not yet read Armstrong's book, but I'm still curious about some of the claims people make about Erlang. For instance, I'd like to understand how a language, per se, can lead one to create more scalable systems. I thought that scalability is an architectural issue, and that language has not much to do with scalability, except in the sense Cedric mentioned (languages that can result in better-performing applications can "scale more" in a narrow sense).

I can certainly see how a language can help one write better parallel programs, but parallelism at the code level and system scalability are not always in a cause-and-effect relationship. How does Erlang, or any language, help one write more scalable applications?

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Cedric Beust's Verdict on Erlang Posted: Oct 4, 2007 4:08 PM
Reply to this message Reply
> I can certainly see how a language can help one write
> better parallel programs, but parallelism at the code
> level and system scalability are not always in a
> cause-and-effect relationship. How does Erlang, or any
> language, help one write more scalable applications?

Maybe I'm the wrong person to answer this as I've never used Erlang either but as I understand it, Erlang does this by constraining the ways that one can develop code. The actor model (which I think I've used without knowing it) forces your threads (for lack of a better term) to communicate in a very specific way, one that Erlang is able to guarantee is safe.

It seems to me that the Actor model is a lot like message-oriented architecture at an application level. If you've ever worked with JMS or other queue-based architectures, it's easy to see why they are scalable. The number of listeners on a queue is (FAIAP) unbounded so you can easily allocate more listeners to backed-up queues as needed and as resources allow.

Of course, I could be spouting-off complete nonsense. If I'm confused, please disabuse me (without abusing me, please) o Erlang sages.

John Zabroski

Posts: 272
Nickname: zbo
Registered: Jan, 2007

Re: Cedric Beust's Verdict on Erlang Posted: Oct 5, 2007 4:43 AM
Reply to this message Reply
It seems like the flame war that has precipitated over Beust's last two Enterprise Developer Spotlights boils down to looking for a real simple question:

How can a language claim to be more fault-tolerant and reliable than another?

There isn't much literature on this subject, so it is not surprising people "don't get it" instantly. Edward A. Lee, a professor at Berkeley, has two position papers on concurrency. One, "The Problem With Threads", explains the difficulty with proving correctness using the threads model. (Not everyone agrees with Lee, of course. Notably, Sam Aiken at Stanford has acknowledged Lee's position in some recent papers. Sam is perhaps best known as the co-author of a static analysis race detection program for Java.)

My only criticism of Lee's position paper is that he doesn't delve much into the intuitive appeal of threads, which is why they're so popular. Threads present a programmer with an autonomous entity: set it and forget it. The garbage collector is a good example of how a thread enables not having to think heavily about memory availability and performance.

Personally, I think the Actor model is even more intuitive than threads, and, as Lee points out, more judicious than threads as well. The C++ Boost project accepted a Google Summer of Code 2006 proposal from Matthew Calabrese for an Actor model library implementation of concurrency in C++. The project is known as boost::act and it's quite different from the POSIX threads model.

Another resource that addresses the issue of concurrency from a modern perspective is Sumit Ghosh's "Algorithm Design for Networked Information Technology Systems". Ghosh tackles the notion of asynchronous, distributed decision-making algorithms. Such algorithms are inherently complex and also fertile ground for parallelization, in part because, as Ghosh points out, entities at any given level of abstraction have their own notion of timing.

nes

Posts: 137
Nickname: nn
Registered: Jul, 2004

Re: Cedric Beust's Verdict on Erlang Posted: Oct 5, 2007 9:30 AM
Reply to this message Reply
Maybe I was a bit harsh in my first assessment yesterday. I have not read Amstrong's book myself and I assumed people could not possibly be arguing that the language itself is more scalable, fault-tolerant, etc. because that makes no sense to me at all. Making a system scalable, fault-tolerant, reliable, secure etc. falls into the realm of architecture and not language. What a language can do is to provide conventions that make it easier to build such architecture. Bill in an article about thread safety http://www.artima.com/designtechniques/threadsafetyP.html
suggests making things immutable and providing synchronizing wrappers, Erlang makes all variables immutable and adds an inbox to each process (and by virtue of being a queue should not be a performance problem for most cases). The end result is that whatever process a random developer writes is almost instantaneously capable to be used concurrently. This does not automatically makes your system scalable but makes the life of whoever is putting the parts for a scalable system together much easier.

I hope that we leave the myth of the silver bullet behind. There are languages with bad tradeoffs and languages with good tradeoffs. In my humble opinion Erlang chooses some of the right tradeoffs to easily build concurrent applications on small systems. I can't criticize it for those choices (like immutability and an inbox).

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Cedric Beust's Verdict on Erlang Posted: Oct 5, 2007 9:46 AM
Reply to this message Reply
> Erlang makes all variables
> immutable and adds an inbox to each process (and by
> virtue of being a queue should not be a performance
> problem for most cases).

Is my description above fairly accurate?

Isaac Gouy

Posts: 527
Nickname: igouy
Registered: Jul, 2003

Re: Cedric Beust's Verdict on Erlang Posted: Oct 5, 2007 11:33 AM
Reply to this message Reply
Cedric Beust wrote

> Why not focus on what I wrote
> and comment on it, like Bhatsha did?
>
> My post raises concrete issues that I would love to see
> discussed... concretely, and not with vague hand waving.


For example paragraph 2: "Erlang was open sourced in 1998, but it's actually much more ancient than that" (referring to a "History of Erlang" webpage showing "1987, The first experiments with Erlang").

A massive 4 years later "Java technology was created as a programming tool in a small, closed-door project ... in 1991"

http://java.sun.com/features/1998/05/birthday.html


No doubt you also characterize Java as ancient technology.

Cedric Beust

Posts: 140
Nickname: cbeust
Registered: Feb, 2004

Re: Cedric Beust's Verdict on Erlang Posted: Oct 5, 2007 12:00 PM
Reply to this message Reply
> A massive 4 years later "Java technology was created as a
> programming tool in a small, closed-door project ... in
> 1991"
>
> http://java.sun.com/features/1998/05/birthday.html
>
>
> No doubt you also characterize Java as ancient technology.

I haven't really given this much thought, but yes, I would probably qualify Java as ancient technology as well.

What's your point?

--
Cedric

Flat View: This topic has 32 replies on 3 pages [ 1  2  3 | » ]
Topic: Nati Shalom: Why Are Most Large-Scale Web Sites Not Written in Java? Previous Topic   Next Topic Topic: What Would You Ask An EJB 3 Spec Lead?

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use