Java Community News - Cedric Beust's Verdict on Erlang

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

Java Community News
Cedric Beust's Verdict on Erlang

32 replies on 3 pages. Most recent reply: Oct 12, 2007 4:11 PM by Isaac Gouy

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 32 replies on 3 pages [ « | 1 2 3 | » ]

nes

Posts: 137
Nickname: nn
Registered: Jul, 2004

Re: Cedric Beust's Verdict on Erlang

Posted: Oct 5, 2007 12:06 PM

> > Erlang makes all variables
> > immutable and adds an inbox to each process (and by
> > virtue of being a queue should not be a performance
> > problem for most cases).
>
> Is my description above fairly accurate?

James, I have to confess I am no Erlang expert having only read the online documentation and played around with a few examples. I am extrapolating from having used immutable values and queues on Java in threaded programs to good effect.

If I understand correctly each queue is tied to one process. Many processes can write to a queue but only one can read. In practice it means that for tasks with slow consumers, you would have to either create a new consumer process per work request (you can create 200 million processes and should not run out) or use a proxy process that distributes load to a predefined pool of processes.

Now we are talking about the real issues. At this point some experienced Erlang programmer has to chime in or the next time I play around with Erlang I will have to remember to test both approaches and see how it goes.

Michael Hobbs

Posts: 51
Nickname: hobb0001
Registered: Dec, 2004

Re: Cedric Beust's Verdict on Erlang

Posted: Oct 5, 2007 2:03 PM

> > > Erlang makes all variables
> > > immutable and adds an inbox to each process (and by
> > > virtue of being a queue should not be a performance
> > > problem for most cases).
> >
> > Is my description above fairly accurate?
>
> James, I have to confess I am no Erlang expert having only
> read the online documentation and played around with a few
> examples. I am extrapolating from having used immutable
> values and queues on Java in threaded programs to good
> effect.

Having written an Erlang "emulator" for Python (http://candygram.sourceforge.net), I feel somewhat qualified to answer. Your descriptions are basically accurate. One point that hasn't been noted is that it is possible to link processes together as peers, so that if one process dies the others are either notified or killed, depending on the nature of the link. This is why people tout Erlang's fault-tolerance.

Also, the interface to a local process is exactly the same as the interface to a remote distributed process, which is why it is easier to scale an Erlang system. Though, in practice, you usually have to be careful to keep track of where processes are located so that you don't waste time sending a lot of transactions to a remote process.

I unfortunately have no knowledge about how well a single queue will scale. I've never tried to slam one with millions of simultaneous messages. I suspect that the CPU, memory, or network would become saturated before the queue's lock would.

As an aside, I have largely abandoned my "emulator" since the impedance mismatch between Erlang and Python is too high. Python values are not typically immutable (i.e., lists, dicts, and sets) and Python has little facility for pattern matching on incoming messages. In fact, using the dynamic dispatch of method calls is a much more natural way to process messages in Python, which is why I've lately been using an actor model in Python instead.

Isaac Gouy

Posts: 527
Nickname: igouy
Registered: Jul, 2003

Re: Cedric Beust's Verdict on Erlang

Posted: Oct 6, 2007 12:31 AM

Cedric Beust wrote

> I haven't really given this much thought, but yes, I would
> probably qualify Java as ancient technology as well.
>
> What's your point?

My point is that in fact you didn't characterize Java as ancient technology as well, instead you again opined that Erlang "language and tools feel so old".

The puzzle is that you started 2 or 3 weeks ago with supposedly a book review and that has somehow morphed into a tendentious verdict on a programming language and tools.

Isaac Gouy

Posts: 527
Nickname: igouy
Registered: Jul, 2003

Re: Cedric Beust's Verdict on Erlang

Posted: Oct 8, 2007 2:08 PM

Frank Sommers wrote

> ... I'm still curious about some of the claims people make
> about Erlang. For instance, I'd like to understand how a
> language, per se, can lead one to create more scalable
> systems.

I'm curious about who is supposed to have made that claim, and the context in which they made it - URI please.

Isaac Gouy

Posts: 527
Nickname: igouy
Registered: Jul, 2003

Cedric 's vague hand-waving skepticism

Posted: Oct 8, 2007 3:13 PM

Cedric Beust wrote

> My post raises concrete issues that I would love to see
> discussed... concretely, and not with vague hand waving.

Let's move on to the vague hand-waving in paragraph 4 -

When somebody creates a language after writing a "sucks" article, you know the yet-to-be-born language is already starting at a disadvantage because its creator is obviously not going to follow a "take what's best in all languages" approach while designing his new language. And unfortunately, Erlang fell right into this trap. ... Maybe if some OO had been thrown in Erlang from the start, we would indeed be facing a language that has the potential to become meaningful. As it stands, I still believe that Erlang is a non-starter for this reason, ...

The Internet Archive first notices that "Why OO sucks" page on July 23, 2002. The "Why OO sucks" page was written while Joe Armstrong was at Bluetail and Bluetail wasn't founded until 1998.

Your suggestion that the Erlang language was created after "Why OO sucks" was written is factually incorrect by at least 10 years - never let the facts get in the way of a good story!

I have always had problems with this claim ... my skepticism has not abated ... I am skeptical ... I am also beginning to question a few claims ... I'm having a hard time imagining ... I find the evidence that this approach is more robust than traditional ones hard to believe ...

You don't show who made "this claim" that you have problems with.

You don't let us see "the evidence" that you find hard to believe.

As far as we can tell the claims might be strawmen and your skepticism might be unreasonable in light of the evidence.

Isaac Gouy

Posts: 527
Nickname: igouy
Registered: Jul, 2003

Online chapters

Posted: Oct 8, 2007 3:35 PM

Frank Sommers wrote
> I have to admit that I have not yet read Armstrong's book

Incidentally, pdfs for

Chapter 2. Getting Started
Chapter 7. Concurrent Programming

are available on the book website
http://www.pragmaticprogrammer.com/titles/jaerlang/

David Pollak

Posts: 4
Nickname: dpp
Registered: Jan, 2007

Re: Cedric Beust's Verdict on Erlang

Posted: Oct 9, 2007 2:36 PM

Cedric is pretty clueless.

First he assails the Erlang implementation (gee, the locks must be someplace) without looking at the source code. There are plenty of ways to implement lock-free message passing for the normal case. The FJ library in Java is used by Scala's Actors to implement such a mechanism. If you own the VM (rather than relying on the abstractions of the JVM, e.g. Monitor Enter/Exit), you can do even more clever stuff.

Second, he trivializes the Actor paradigm to "syntax." It's a different way of solving the problem, just as Object Oriented programming is a different way of solving the problem. C++ was originally implemented as a C preprocessor (syntactic sugar on top of C), but it changed the way people approach problems. Without spending 12+ months using Erlang's Actors, he does not have the background to compare the two.

After using Scala Actors (patterned after Erlang's Actors), I have come to appreciate the power of message passing as a concurrency abstraction.

After spending a year doing functional programming, I state to a 98% degree of certainty that the majority of the bugs I've had in statically typed code (Java, C++) has come from unexpected mutability in state. Well defined "statefullness" leads to significantly more robust and testable code.

Isaac Gouy

Posts: 527
Nickname: igouy
Registered: Jul, 2003

multicore scalability

Posted: Oct 9, 2007 6:06 PM

Cedric Beust blogged
Just because you write a program in Erlang doesn't mean your code will scale better on multiple processors than if you had written it in Java.

Yes a sequential Erlang program is still sequential when it's on a multicore processor.

Now, if we have to think in terms of distribution from the very beginning of our design phase, Erlang's edge fades considerably. It seems pretty easy to me to design similar programs with Java using messaging API's, and with very little additional effort.

Java has built in support for objects and Java programs are ordinarily composed of many cooperating objects. As you noted in your review of "Programming Erlang" - Erlang does not have built in support for objects. Erlang has built in support for processes and Erlang programs are ordinarily composed of many cooperating processes. Those ordinary Erlang programs take advantage of multicore technology without modification.
http://www.ericsson.com/technology/opensource/erlang/news/archive/erlang_goes_multi_core.shtml

... Erlang itself, the language and the virtual machine, already runs orders of magnitude slower than Java (and even than Ruby), as Tim Bray's latest forays in the domain have shown ...

Are you really generalizing about the performance of Erlang systems based on Tim Bray's experiments with a file scripting task?
Are you going to generalize about how fast Erlang is if Tim Bray's experiments on 8 core processors provide a fast Erlang solution?

Meanwhile, some faster than C++ programs:

"High-level Distribution for the Rapid Production of Robust Telecoms Software: Comparing C++ and Erlang"
http://www.macs.hw.ac.uk/~trinder/papers/CPE2006.pdf

Isaac Gouy

Posts: 527
Nickname: igouy
Registered: Jul, 2003

myths and guaranteed message order

Posted: Oct 10, 2007 10:00 AM

Cedric Beust blogged
... there has to be a lock on all these inboxes to guarantee that when dozens of processes send messages to your own process, these messages are delivered and treated in the order they are received from.

Let's be clear, the only guaranteed message order is - when a process P1 dispatches two messages M1 and M2 to the same process P2, in that order, then message M1 will never arrive after M2 at the message queue of P2.

When different sender processes dispatch messages to the same receiver process, then the arrival order can vary.

See the Erlang spec
See slide 11 in these lecture notes
"An Introduction to Erlang: Concurrency"
http://www.it.uu.se/edu/course/homepage/projektDV/ht06/08-erlang-cont.pdf

"An Introduction to Erlang"
http://www.it.uu.se/edu/course/homepage/projektDV/ht06/07-erlang-intro.pdf

Cedric Beust blogged
... ETS and DETS (Erlang's version of databases) ...

Simply reading the table of contents of the book you reviewed "Programming Erlang" we learn

Ets and Dets — Large Data Storage Mechanisms
MNesia - The Erlang Database

Mnesia is "The Erlang Database"

"Mnesia A Distributed Robust DBMS for Telecommunications Applications"
http://www.erlang.se/publications/mnesia_overview.pdf

Isaac Gouy

Posts: 527
Nickname: igouy
Registered: Jul, 2003

five nines and completely missing the point of process isolation

Posted: Oct 10, 2007 1:35 PM

Cedric Beust blogged
Another Erlang claim to fame is that it makes it easy to create fault-tolerant system. This is usually ascribed to the fact that Erlang processes don't throw exceptions: they die... This doesn't strike me very different from any other languages, regardless of the failure mechanism they support (typically, exceptions).

The "Five nines" section is so utterly misleading that it's difficult to know where to begin!

Let's start with the knowledge that Cedric reviewed "Programming Erlang", let's look at the table of contents
http://media.pragprog.com/titles/jaerlang/toc.pdf

Chapter 4 Exceptions, Raising an Exception, try ... catch, catch, Improving Error Messages, Programming Style with try...catch, Catching Every Possible Exception, Old- and New-Style Exception Handling, Stack Traces

The fact seems to be that Erlang processes do throw exceptions.

So what about Erlang is "very different from any other languages"?

"Our applications are structured using large numbers of communicating parallel processes. We take this approach because:
...
3. Fault isolation — concurrent processes with no data sharing provide a strong measure of fault isolation. A software error in a concurrent process should not influence processing in the other processes in the system.
...
The third characteristic is essential for programming fault-tolerant systems. Each independent activity should be performed in a completely isolated process. Such processes should share no data, and only communicate by message passing. This is to limit the consequences of a software error."

p18 "Making reliable distributed systems in the presence of software errors"
http://www.sics.se/~joe/thesis/armstrong_thesis_2003.pdf

As for "it's up to a supervisor to detect this crash" - when a process dies, an exit signal is sent to all
linked processes.

See slides 20,21,22 "An Introduction to Erlang: Concurrency"
http://www.it.uu.se/edu/course/homepage/projektDV/ht06/08-erlang-cont.pdf

As for "you still need to implement the supervisors" - "To implement a process such as a supervisor, the user only has to implement the callback module which should export a pre-defined set of functions, the callback functions."

"OTP Design Principles : 1.2 Behaviours"
http://www.erlang.org/doc/design_principles/part_frame.html

Isaac Gouy

Posts: 527
Nickname: igouy
Registered: Jul, 2003

how does any language help write more scalable apps?

Posted: Oct 11, 2007 9:36 AM

Frank Sommers wrote
> How does Erlang, or any language, help one write more scalable applications?

James Watson's answer we might caricature as - by getting in the way - by not providing features that make it difficult to distribute across cores and machines.

The other answer we might caricature as - by not getting in the way - by providing features that make it easy to distribute across cores and machines.

For example, have the same syntax and semantics for sending a message to any process through its Pid – irrespective of whether that process is on the same OS thread, on a different OS thread and processor core, on a different computer, on a different computer and different OS ...

Cedric Beust

Posts: 140
Nickname: cbeust
Registered: Feb, 2004

Re: how does any language help write more scalable apps?

Posted: Oct 11, 2007 10:01 AM

> For example, have the same syntax and semantics for
> sending a message to any process through its Pid –
> irrespective of whether that process is on the same OS
> thread, on a different OS thread and processor core, on a
> different computer, on a different computer and different
> OS ...

Yes, but there is nothing inherent to a language syntax or philosophy to achieve this result.

Send a message with JMS in Java and you have absolutely no idea where it will go nor who declared interest in it, so Java achieves the same level of transparency as Erlang in that area.

--
Cedric

Isaac Gouy

Posts: 527
Nickname: igouy
Registered: Jul, 2003

Re: how does any language help write more scalable apps?

Posted: Oct 11, 2007 10:26 AM

Cedric Beust wrote
-snip-
> Send a message with JMS in Java and you have absolutely no
> idea where it will go nor who declared interest in it, so
> Java achieves the same level of transparency as Erlang in
> that area.

Is JMS even included with Java Platform, Standard Edition?

Cedric Beust

Posts: 140
Nickname: cbeust
Registered: Feb, 2004

Re: how does any language help write more scalable apps?

Posted: Oct 11, 2007 10:41 AM

> Cedric Beust wrote
> -snip-
> > Send a message with JMS in Java and you have absolutely
> no
> > idea where it will go nor who declared interest in it,
> so
> > Java achieves the same level of transparency as Erlang
> in
> > that area.
>
> Is JMS even included with Java Platform, Standard
> Edition?

No, it's not, but I don't see how this is relevant to my point.

--
Cedric

Isaac Gouy

Posts: 527
Nickname: igouy
Registered: Jul, 2003

Re: how does any language help write more scalable apps?

Posted: Oct 11, 2007 11:09 AM

afaict your point was ... Java achieves the same level of transparency as Erlang in that area.

I'm struggling to understand how that can be true when in Erlang that functionality is a fundamental part of the language, and in Java that functionality is a library that isn't even included in most distributions. (Turing complete?)

Incidentally, it would help if you said which statements in "Erlang: the verdict" you now accept were misunderstandings or mistakes, and which statements you stand by.

It would help if you showed us who made the claims you have problems with, and showed us the evidence you find hard to believe.

Flat View: This topic has 32 replies on 3 pages [ « | 1 2 3 | » ]

Previous Topic

Next Topic