Once upon a time, there was a language, it was called Scala.
People came to look at the language, and they announced that it was “pretty impressive”, but they thought that the feature set was changing too fast, and that nobody was using it yet outside of hobby projects. It sure did look good, but nobody wanted to risk their career on it, it was too young, it might not last.
Then something happened; Scala grew up. Twitter announced that they used it to replace some of their Ruby back-end, and SAP were using it, and EDF. The news spread and new developers also came to see that Scala was “pretty impressive”, whilst the early pilgrims began to look at it with renewed interest and a glint in their eye.
What they saw was a language that is now mature and ready to be used in anger. With the approach of version 2.8, Scala is coming of age, and it is indeed “pretty impressive”!
Languages evolve, they spawn children, and they give rise to distinct species. Much like the formation of life on a young earth, programming languages came to existence in a bubbling primordial soup of CPU instruction sets and mathematical ideas. Unlike the formation of life, there was no slime involved [1].
But this isn’t the vicious tooth and claw battle of natural selection, although it sometimes feels that way with the ongoing wars about tabs vs. spaces and exactly where you should stick your brackets. Languages are picked with as much care as a breeder of thoroughbreds will choose horses. Just as in biology, this artificial selection is not without its problems, and inbreeding is an ever-present risk. Fortunately, there is a flip-side to this coin, good husbandry and horticulture also takes advantage of hybrid vigour. This is where two parents are specifically chosen for their differences to create offspring that is stronger than either, the idea being that both sets of weaknesses will be cancelled out. The same thinking has been applied in the world of programming languages too [2], with many now fusing concepts from Object-Oriented and Functional styles to offer the programmer incredible power and expressiveness.
Most folk reading this will be Java programmers, so I’d like to explain some of the ideas behind functional programming before covering the sordid details of how functions and objects interact.
The internet is already full of better explanations that I can ever dream of writing, so I’ll keep it short.
In mathematics, a function is a thing that takes one value (the input) then uses it somehow to produce another value (the output). This definition covered almost every eventuality for a long time, even when mathematicians started seriously stretching the definition of what a number is; complex numbers, matrices, vectors, co-ordinates (both Polar and Cartesian), quaternions... Lots of things can be treated as a number if you squint in just the right way.
This was fine for a long time, then programmers were invented, and then computers were invented [3]. Once computers settled into the collective consciousness and matured a bit, the programmers began to think differently about them, one such thought was that looking at oversized print-outs with endless lists of three letter opcodes was a pretty good way to give oneself a headache.
If only they could group together some of these sequences and give them a name then we’d have a neat way to reuse stuff and all that time spent repeating things could be freed up to, hopefully, be spent down the pub (or looking at quaternions, or writing pong). Because so many of these programmers were also mathematicians, and because so many of their programs were working on mathematical problems, it was decided that the idea of a function fitted very nicely with the idea of a pre-packaged unit of behaviour, and so 2nd generation languages were born.
The fit wasn’t perfect though. There was the question of functions that needed to work with multiple inputs or (more frustratingly) multiple outputs. Fortunately some mathematicians already allowed for functions with multiple inputs, so this idea was adopted very early on and ways were figured out to return multiple values (usually by subverting the input parameters and writing back out to them). Other mathematicians (like Haskell) didn’t like multiple inputs and came up with the idea of higher-order functions instead, functions that could return other functions, or take functions as arguments, but these were harder to implement for the programmers and so got ignored at first.
Another discrepancy with the functions of programming was in side effects. A function might do a different thing every time it was used with the same input (such as when reading a file), or it could do something besides just returning a value (such as printing a line to the console). Even worse, it was possible that a value might have been changed after being used as an argument to a function! As a program needs to have side effects in order to do more than warm the CPU this wasn’t worried about very much, except by the mathematicians that didn’t especially like beer but still had free time on their hands.
So the functions of programming were not the same as the functions of mathematics. They had a new (and less precise) definition: A procedure, or bunch of instructions, with a name, that can optionally have one or more inputs and one or more outputs, some of which can be both at the same time, and might also do other stuff on the side.
Of course, a lot of mathematicians weren’t happy about the way functions were being undermined, so a new breed of programming language was created to give back some of what was lost and once again put this stuff on a solid theoretical framework.
Functions became first class entities and not just a name for a bunch of code. This allowed for Haskell's higher-level functions to be used in designing software. Language features also evolved to encourage immutable values, so that functions couldn’t muck about with their inputs. Currying was implemented and Tuples were used, so that functions could give a respectable facade of only having one input and one output. Interesting stuff was also being done [4] to restrict those pesky side-effects, if they couldn’t be totally eliminated, then they could certainly be put in their place! This family was known as “functional” programming languages, due to the importance placed on returning functions back to their mathematical roots. The family includes such languages as Lisp, Scheme, Caml, Erlang, F#, Clojure, etc.
Being a good example of sound engineering, functional languages are elegant in design, comprehensible [5], efficient and internally consistent. Also, as so often happens with Good Ideas™, they spent a long time being largely ignored by the mainstream. The programmers [6] were still very conscious of why they liked functions in the first place; the need to split their systems up into distinct, named units of behaviour that could be isolated and reused in different ways. This need was still itching and so the Object-Oriented paradigm was born as a way to scratch it [7]. For the time being, functional programming was delegated to being a hobby, and also a handy way to find cute new material for the occasional dissertation or thesis. It was believed that functional languages were slower than their imperative cousins, but this was mostly taken on faith and nobody really went out of their way to prove it. It was also believed that, although FP looked very neat for small programs and for demos, it wouldn’t be able to scale up to programs that were hundreds of thousands of lines long, it wouldn’t be maintainable.
But this stuff wasn’t just a hobby. In common with all the best revolutions it was just bubbling underground for a few years until the world was ready for it. Mainstream programmers were increasingly finding that there were things they could easily think of, but which couldn’t be easily expressed in their (object oriented) code. Even simple ideas like “take this list of strings, and turn it into a list of uppercase strings” would run into subtle problems. Sometimes the list of uppercase strings would be missing the first entry because they’d started counting from 1 instead of 0, sometimes another user would find that their list had all been turned to uppercase for them even though they didn’t want it to be, there were also a lot of null pointer exceptions.
Slowly, people began clamouring for closures, and continuations, and for other tricks to make their programs more robust and easier to maintain. But these weren’t things that could be built from objects, so enhanced for-loops were invented, and anonymous classes, and the visitor pattern, and the command pattern; none of them matched perfectly what the programmers were thinking, but they still helped and made a lot of problems more manageable [8] (even if they did create a small swamp of boilerplate code). The time was ripe for a paradigm shift, and functional programming was awaiting its grand entrance.
Via Erlang, Ericsson demonstrated that the FP paradigm could scale to massive systems. It could be made fast, maintainable, testable, and remarkably free of errors. This was FP outside of the labs and it was looking better than the object-oriented systems. Ericsson programmers were getting more beer time than ever before. The gauntlet had been thrown down!
On the other side of the fence, programmers were looking at FP with thoughtful expressions and a hint of jealousy. Java was getting bloated, and every new feature seemed to come with its own requirement for boilerplate code. Even small programs are now littered with annotations and template parameters and duplicate type declarations, big programs are even worse. To add insult to injury, the debate about how to add closures doesn’t look like reaching an early conclusion, and we still have the endless lists of get/set methods in java beans. Something had to give.
Despite this, Java is not without a LOT of strengths. The Virtual Machine is one of the most mature and optimised on the market and can be found everywhere, from washing machines and mobile phones through to countless web servers and desktop computers. The ecosystem with its range of open source libraries and frameworks for the language is just astounding, the paid-for stuff is pretty hot too. Under the Java umbrella, the world has been flooded with projects that flourished and matured following corporate investment and millennia of collective development time.
That’s an awful lot to give up just for a few new language features
What was needed was a way to keep all that juicy java goodness, but to have it a la carte with a refreshing order of FP at the same time. Scala stepped in to meet this need, although it wasn’t the only contender. Pizza came first, but it’s seen as a prototype for Scala nowadays.
Other languages learned to run on the JVM too, such as JavaFX, JRuby, Jython, Groovy etc. Most of them also had closures and other elements of FP, but in the nation of Java these newcomers weren’t quite native citizens, behaving more like immigrants with a shiny new work visa and a foreign accent.
The fashion for dynamic languages didn’t help either; by pretending that types don’t exist it becomes difficult to generate compiled code that “speaks” idiomatic Java. This can be big problem if you’re producing objects to be handed to a 3rd party library. In some cases the language barrier was so challenging that it needed an interpreter, such as the JSR233 Scripting API or the Bean Scripting Framework.
Scala came to the party with a new trick, it integrates so tightly with Java that it can speak types like a native. Instead of acting like a migrant worker this is a naturalised, passport carrying, citizen. From the outside world Java and Scala compile to classes that look identical [9], this is hardly surprising as it was an explicit goal in the design of the language. It’s even more impressive when considering that Scala is a fully-fledged FP language, embracing and unifying the twin worlds of objects and functions in a beautifully consistent manner.
Because of this tight integration, Scala can be used as a drop-in replacement for java and nothing forces you to write a single line of code in a functional style. Features like type inference, traits, clean property handling [10], and the fusion of member variables with constructor arguments all allow it to be used as a kind of java-with-a-cleaner-syntax. Even foregoing the benefits of FP, it can be argued that Scala is still a worthy successor by being much more object-oriented than Java:
Everything is an object, including number literals and functions. In Java, methods aren’t objects, nor are primitives. 2.toString is a valid statement in Scala.
It does away with static class members, these are mostly an anachronism dating back to Java’s heritage as a C++ successor. C++ was a hybrid language itself, with the goal of supporting the procedural style of C whilst also adding support for objects. Static members aren’t very object-oriented as they can’t implement an interface, or take advantage of polymorphism and overloading in the same manner as instance members. It’s also not possible to pass a class as an argument to a function that will then use its static members [11]. Instead, Scala offers singleton objects which have none of these failings. The new concept of companions allows a singleton to access the restricted members on instances of a same named class, thus duplicating the privileges of static members.
All properties on an object are implemented behind-the-scenes as a hidden field and a pair of methods to get and set the field [12]. No more public access to your object’s internal data that can be directly modified by anyone.
In the future, virtual classes may well be added to this list, making Scala’s support for objects even more impressive.
The world of functional programming is supported just as well:
tail-call optimisation of recursive functions
pattern matching
first class and higher order functions
partial functions (not defined for all possible input values)
currying and partial application of functions
closures
concise language syntax to declare immutable values, and a good library to work with immutable collections.
These methods don’t use the same cumbersome naming convention as Java’s bean accessors, although generation of such accessors can be explicitly requested by using the @BeanProperty annotation.
FP has proven itself, and a growing number of developers are now coming to recognise the problems that are best suited to this approach. Scala demonstrates how functional design patterns can be added to the programmers’ arsenal without needing to sacrifice the benefits of object-orientation. It also shows how the two styles can complement each other to produce a robust well-rounded language, without any hint of multiple personality disorder.
Once you get beyond basic syntax and concepts of closures, first class properties, higher order functions, traits, immutable refs, etc., the deeper potential of Scala reveals itself in the interplay between features. Some of the design choices made over the lifetime of the language have resulted in powerful synergies [13]; and learning this new generation of object-functional design patterns perhaps offers the key to being truly effective in Scala.
With future articles, I hope to further explain this exciting language by covering some of the ideas that are helping to spread its growing popularity.
I’m proud to rescue this word from its mindless enslavement by corporate jargon. Terminology is important to us developers, we should fight for our political prisoners!
I'm still fighting the compiler over correct generation of field setter delegates, when I've resolved this problem it'll be part of the write-up and I'll post my experience.
In the meantime, this is an article I wrote a couple of months ago, but never found the time to proof-read and format.
Nice posting. But let me ask the question: how many seasonal programmers are actually using Scala in their everyday work? Also, when do you think we'll see actual moves from Java to Scala in the commercial world? Does it imply that the vendors will need to announce the discontunuation of Java support?
I think the biggest barrier to Scala adoption right now is not the language, but the current level of tooling support. Right now there are plugins in active development for Eclipse, Netbeans and IntelliJ, all of them actively targeting stable versions to co-ordinate with the Scala 2.8 release.
Once the tool support is recognized to be stable, I think we'll see gradual usage across multiple projects, on a class-by-class basis and probably starting in the test suites. Absolutely nothing new needs to be installed on deployment machines, Scala will run on the same web servers and uses the same virtual machines as Java. This is much closer to the adoption pattern of a library (such as jodatime or commons-lang) than to the adoption pattern of a new language like Ruby or Python.
As for announcing "moves" or "discontinuation of Java" I just don't see that happening at all. What you will see is more work released in the style of Akka (http://wiki.github.com/jboner/akka). It's written in scala, and is arguably more usable in Scala, but also provides Java bindings thanks to the sheer strength of Scala's interoperability.
I can't think of a single Scala feature that isn't somehow accessible via Java, though it often isn't very pretty to try...
> I think the biggest barrier to Scala adoption right now is > not the language, but the current level of tooling > support. Right now there are plugins in active > development for Eclipse, Netbeans and IntelliJ, all of > them actively targeting stable versions to co-ordinate > with the Scala 2.8 release.
IDE support is important but I think there are other bigger issues. You touched on the first in your article: rapid feature set changes. It creates uncertainty and apprehension. Has Scala reached a point of relative stability in features?
The other thing that I think prevents Scala from gaining wider use is the way it is being promoted. Your article is decent but still focuses on a lot of academic esoterica that most people who make decisions about programming languages couldn't care less about.
P.S. At the time Ada Lovelace was born, computers already existed. 'Computer' was a job title and people had long been defining algorithms for computers to execute.
Functional Programming has had a shot at mainstream acceptance many times, and it has failed every time. Does that mean it will fail this time? Not necessarily. But I wouldn't bet on it.
More likely, IMO, is the useful features of FP will be integrated into mainstream imperative languages. Closures, yes. Monads, not so much (although particular Monads that address common problems might be added as syntactic one-offs.) The closer Scala stays to mainstream programming languages, the more acceptance it will get. Since it appears to be moving away from these languages into more theoretical areas, I think it will suffer lower adoption amongst the masses.
JMO, and I've been wrong many, many times. But betting against FP has been my most reliable prediction since I was introduced to it in undergrad.
For years I've read inspiring stuff about wonderful languages that I can't use. I work in embedded firmware where, if we're lucky, we may get to work on a project with a uC that can support C++. Otherwise we continue to write in C, and occasionally assembler. Considering how much of the total programming effort is spent creating embedded firmware, I think it's time we got some attention.
So please can I have Clojure, Ruby or Scala, that will run on a battery-operated 10 MHz ARM7 with 128k program FLASH and 8k RAM? Thank you.
> P.S. At the time Ada Lovelace was born, computers already > existed. 'Computer' was a job title and people had long > been defining algorithms for computers to execute.
I struggled with this one for a while myself, and couldn't think of another term that would unambiguously include everything from Babbage's difference engine through to the semiconductor revolution. In the end I just had to trust that context would make it all clear :)
> So please can I have Clojure, Ruby or Scala, that will run > on a battery-operated 10 MHz ARM7 with 128k program FLASH > and 8k RAM? Thank you. The Squawk VM will work on an ARM Cortex-M3 with 64K RAM and 512K flash. I don't expect it to shrink down to the 8K+128K you want.
It's no secret that Scala is actively targeting the needs of concurrency with both functional programming and the actors library, and I'm sure that the growing need to deal with multi-core processors will encourage adoption here.
In spite of this, aspects of Functional Programming are already gaining developer mindshare, even when not being marketed as FP:
C#'s LINQ can directly translate to map, flatMap and filter operations. Java will soon gain closures, and many JVM languages already have them. Google-Collections are encouraging the use of immutable structures. Scala just manages to combine the full set plus a few other tricks besides.
The revolution may be gradual and understated, but it really does seem to be upon us.
> Has Scala reached a point of relative stability in features?
That point is now close enough to be a light at the end of the tunnel, and stability is a *very* high priority for Scala's future.
Breaking changes, deprecation and regression failures are already fiercely argued on the mailing lists and won't be accepted without an incredibly strong justification. One of the reasons for delaying the 2.8 release is to make sure we "do the right thing" now with certain features, to avoid making changes later.
The recently announced 3-tier system of incubator/greenhouse/trunk will also ensure that new features can be explored and tested without impacting core stable functionality.
> The other thing that I think prevents Scala from gaining > wider use is the way it is being promoted. Your article > is decent but still focuses on a lot of academic esoterica > that most people who make decisions about programming > languages couldn't care less about.
Guilty as charged...
Right now Scala is of greatest appeal to early-adopters, many of whom will base their decisions on academically strong technical arguments. There are many aspects to 2.8 that I believe will make the language far more attractive to a wider audience. Hopefully the way it's promoted will evolve at that time to match this target market.
> Right now Scala is of greatest appeal to early-adopters, > many of whom will base their decisions on academically > strong technical arguments. There are many aspects to 2.8 > that I believe will make the language far more attractive > to a wider audience. Hopefully the way it's promoted will > evolve at that time to match this target market.
Microsoft may indirectly help Scala a lot by its attempt at pushing F# into the mainstream (in addition to pushing functional features into C# and VB). If F# becomes somewhat of a success in the .NET world, many more people will take notice of the usefulness of hybrid functional and OO languages, which will benefit Scala.
> > (OK, now someone reply about parallelism and FP.) > > Parallelism: "It won't catch on in this universe."
In the client/singleuser universe. It's been going great guns in the database engine universe for decades, without benefit of "new" languages.
We need, at minimum, compilers (not new languages) which parallelize linear client code. We need, fully, a new hardware architecture which accepts linear code and executes it in a parallel fashion. So far, no volunteers.
> parallelize linear client code. We need, fully, a new > hardware architecture which accepts linear code and > executes it in a parallel fashion. So far, no volunteers.
Many current processors do exactly that --- independent operations are executed in parallel on the available arithmetic units. Only really simple CPUs like the Intel Atom are completely in order.
> > parallelize linear client code. We need, fully, a new > > hardware architecture which accepts linear code and > > executes it in a parallel fashion. So far, no > volunteers. > > Many current processors do exactly that --- independent > operations are executed in parallel on the available > arithmetic units. Only really simple CPUs like the Intel > Atom are completely in order.
Granted. But this is at the assembler level.
But this does beg the question: what would a parallel hardware look like, if you could have it? What exists now are von Neumann units (if not full machines) stitched together where the coder is required to manage execution; define the boundaries of what are parallel/concurrent activities.
Thinking Machines, at least, wrote compilers for its massively parallel machines. It may be that compilers are sufficient to answer the question.
The compiler's job would be to analyze the work, decide where the data dependencies are, and generate appropriate code. The compiler would either need to know how many cores/processors to target, or have a (rather heavy) runtime to sort that out.
Flat View: This topic has 59 replies
on 4 pages
[
1234
|
»
]
Previous Topic
