Article Discussion
The Origins of Scala
Summary: Martin Odersky talks with Bill Venners about the history that led to the creation of the Scala programming language.
77 posts on 6 pages.      
The ability to add new comments in this discussion is temporarily disabled.
Most recent reply: May 21, 2009 4:21 AM by
Andrew
Posts: 18 / Nickname: 55548 / Registered: May 16, 2008 6:33 AM
Re: The Origins of Scala
May 5, 2009 3:32 PM      
> For the record, I think Meyer's example is, ultimately,
> poor.

yup, no argument there. the actual concrete example is odd in that it uses the inheritance graph and type system to encode what is effectively a policy about sharing rooms. i.e. the policy could easily change or be subject to more complex rules.

however, the example does serve to raise an important question (for me at least ;-) about how to write reusable code in the presence of abstract types.
James
Posts: 11 / Nickname: jiry / Registered: November 15, 2007 2:29 AM
Re: The Origins of Scala
May 5, 2009 4:45 PM      
> for example, how would you create a single method that
> took a list of either boys or girls and arranged it so
> that everyone in the list was sharing a room, without
> duplicating the code for boys and girls separately?

This is quite an interesting challenge. I'm not sure if this is the best way, but here's what I've got


// use an implicit to prove that the suitable roommate matches
def shareList[T <: Skier](list : List[T])(implicit f : T => Skier {type SuitableRoommate = T} ) = {
val length = list.length
(list take length/2) zip (list drop length/2) foreach {case (x, y) => f(x) share y}
}

// these two compile just fine
shareList(List(new Boy, new Boy, new Boy, new Boy))
shareList(List(new Girl, new Girl, new Girl, new Girl))

// this does not compile
shareList(List(new Girl, new Girl, new Girl, new Boy))


It does make me think that there must be a cleaner way to express the constraint than with an implicit. But I'm stymied about what it might be. The more direct approach "def shareList[T <: Skier {type SuitableRoommate = T}]" got me errors regarding cyclic references to type T.

Now, if the challenge was to take a List[Skier] and pair everybody up then we'll have to do something a bit different to recover static types using pattern matching. Here's one way to go about it.


def separateSkier(s : Skier) : Either[Boy, Girl] = s match {
case b : Boy => Left(b)
case g : Girl => Right(g)
}


With that I can take a list of Skiers and split it into Boys and a list of Girls using a generic split function


def split[T, A, B](lst : List[T], separate : T => Either[A, B]) = lst.foldLeft((List[A](), List[B]())) {
case ((ls, rs), x) => separate(x).fold({l => (l :: ls, rs)}, {r => (ls, r :: rs)})
}


And here's how to split a list


val (boys, girls) = split(List(new Boy, new Girl, new Boy, new Girl), separateSkier)


And now we can call pairList(boys) and pairList(girls).
James
Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
Re: The Origins of Scala
May 5, 2009 8:11 PM      
> Well, for me logic and mathematics is a science, it's just
> not a natural science.

Don't get me wrong. Logic is a pre-requisite of science and it's no less important than science. In fact, it's given that science could not exist without it. It's hugely important and I depend on it in every aspect of my life (so much so that it annoys my wife daily.)

But I think we need to realize that science is not the same as logic and mathematics. Any logical statement can be proven or disproven given the right axioms but the laws of gravity or electricity don't depend on what axioms we choose.

I don't point this out to demean any subject. And Mr. Odersky, (may I call you Martin?) I have great respect for what you have produced in, especially with regard to Scala. You clearly are a master in your field. No rational person could ever question that.

I'm a little intimidated but honored to even argue with you and I think your willingness to converse with me is a sign of your character.

I guess all I am asking is that you consider is that programmers deal with fiction (I'm falling back on physics metaphors here.) In physics, we usually ignored friction. But engineers can't. Having been on the both sides of this, I see why engineers thing physicists have their heads in the clouds. But none would for a second doubt the value of physics. It's just a different focus and different skills.

> And it's the only candidate
> discipline to underly our field of software engineering
> (maybe some people see that field rooted in social
> sciences, but I would have my doubts). As to falsifiable
> hypothesis: I would say, ``immutable datastructures can
> always be covariant'' (that's what you stated) is such a
> falsifiable hypothesis. That's not a matter of belief.

The point I was attempting to make was that it's not a law of computing that types should allow for covariance and contravariance. There are lot's of people that think that typing in general is nonsense. These are all valid opinions. Does your opinion carry more weight than the average programmer (myself included)? Absolutely. But please don't pretend that your opinion is a fact.

> Belief is: does it matter? Should I care in my work? But
> that's just the same as for Ohm's law. There are gifted
> electricians who do not care for Ohm's law; they just
> cobble their circuits together guided by experience and
> intuition. They might do great work. But they are not
> engineers. So for that reason I thought James' analogy was
> right on spot.

I can't imagine that there are many electricians that ignore Ohms law. That would be like designing an aqueduct and ignoring that water runs downhill. There isn't a rational argument for saying Ohms law is not true, at least not on a macro scale. You could hypothesize a universe where it is not true but that won't affect our reality. Reality has little concern for our thoughts.

I'd love to discuss this kind of stuff at length with you but I feel this is not the forum for that. If you want I can send you a direct email. I am still subscribed to the Scala email list. I'll understand, however if you are too busy to be bothered. I have been doing work with neural networks and think Scala could be a great fit for that work.
Bill
Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
Re: The Origins of Scala
May 5, 2009 10:29 PM      
> Andrew wrote:
> > for example, how would you create a single method that
> > took a list of either boys or girls and arranged it so
> > that everyone in the list was sharing a room, without
> > duplicating the code for boys and girls separately?
>
James Wrote:
> It does make me think that there must be a cleaner way to
> express the constraint than with an implicit. But I'm
> stymied about what it might be. The more direct approach
> "def shareList[T <: Skier {type SuitableRoommate = T}]"
> got me errors regarding cyclic references to type T.
>
I think part of the difficulty here is that this is a contrived example dreamed up by Bertrand Meyers over a decade ago. There's no real problem we're trying to solve, and in such cases, it's hard to know what would make a good solution.

However, I get the sense that Andrew's basic desire is to be able to call share on a variable of type Skier. I think the trouble is that if the goal is to have the compiler to statically determine that you're only passing in a suitable roommate each time you call share, and all you know about the object you're calling share on (at compile time) is that it is a Skier, then the compiler doesn't have enough information to compile the code. It just isn't defined at the Skier level of abstraction what type is suitable to be passed to share.

As a result, any time you call share, you will need to call it on either a Boy reference or a Girl reference, or a reference of some other subtype of Skier that has a concrete SuitableRoommate.

I think James Iry's solution is very nice. It is probably obscure to anyone not familiar with Scala, but the key that he uses an implicit function named f to ensure that shareList can only be called when passing Skier subtypes whose SuiteableRoommate type is the same type as the Skier subtype itself.

James, you don't need to actually call f, because by the time you get into the shareList method, T is already known to fulfill the constraint. So this works too (the difference from your code is instead of f(x) share y, it is just x share y):


// use an implicit to prove that the suitable roommate matches
def shareList[T <: Skier](list : List[T])(implicit f : T => Skier {type SuitableRoommate = T} ) = {
val length = list.length
(list take length/2) zip (list drop length/2) foreach {case (x, y) => x share y}
}

// these two compile just fine
shareList(List(new Boy, new Boy, new Boy, new Boy))
shareList(List(new Girl, new Girl, new Girl, new Girl))


For those not familiar with Scala's implicits, an implicit parameter is a parameter that's passed in implicitly without you having to type the code at the call site. You can also do it explicitly, and that would look like this:


shareList(List(new Boy, new Boy, new Boy, new Boy))((boy: Boy) => boy)
shareList(List(new Girl, new Girl, new Girl, new Girl))((girl: Girl) => girl)


Because the parameter is declared implicit in the shareList signature, clients don't need to specify it explicitly. But all it is is a function that takes a Boy and returns a Boy, or a Girl and returns a Girl, respectively, both of which adhere to the constraint James put on the implicit function. That constraint is that the input to the function be a T (which is already defined to be a Skier with the [T <: Skier] at the beginning of the shareList signature), and the output be a Skier whose SuiteableRoommate type is also T. That only works for Boy and Girl. You couldn't pass to shareList a List of Cat or Dog types, which I showed earlier, that had each other as SuitableRoommates.

The implicit function that gets picked up is the identity function, which Scala imports into every Scala source file. This is an implicit function from any type to itself. All this function does is convert a Boy to a Boy, or a Girl to a Girl, which does nothing in this case, except reduce the types that can be passed to shareList. In my version of the shareList method, I don't even ever invoke the function.

One last tweak that could be used to make the shareList function a bit more concise is to use a view bounds. This is just a shorthand for the implicit that James wrote out explicitly. That would look like this:


def shareList[T <% Skier {type SuitableRoommate = T}](list : List[T]) = {
val length = list.length
(list take length/2) zip (list drop length/2) foreach {case (x, y) => x share y}
}

// these two compile just fine
shareList(List(new Boy, new Boy, new Boy, new Boy))
shareList(List(new Girl, new Girl, new Girl, new Girl))


So there it is. I would probably use this code, though I'm not sure I would have thought of it. Not without a lot of trial and error, but maybe from now on I'll just email James Iry and let him have at it first.

If you want a more understandable description of implicits and view bounds, you may want to check out the Feel of Scala video on Parleys:

http://tinyurl.com/dcfm4c
Martin
Posts: 15 / Nickname: modersky / Registered: September 14, 2003 9:46 PM
Re: The Origins of Scala
May 6, 2009 2:41 AM      
@James Watson

Sure, please call me Martin!


> The point I was attempting to make was that it's not a law
> of computing that types should allow for covariance and
> contravariance. There are lot's of people that think that
> typing in general is nonsense. These are all valid
> opinions. Does your opinion carry more weight than the
> average programmer (myself included)? Absolutely. But
> please don't pretend that your opinion is a fact.
>
That's a good point,and I fully agree. Whether a language should have dynamic typing, static typing, or gradual typing as Fan seems to do, is a matter of taste, and nobody's opinion counts more than any other's.

I am a bit less happy with type systems that are unsound.
That's like saying: we know our theory inconsistent, but we do not care because empirically it works well. I think in the end better results are obtained by not being too complacent with such inconsistencies. But in the end that is also a matter of taste, as long as the limitations of a theory/type system are explained clearly.

What I was really objecting to is that (very common) sentiment that ``academic'' means ``impractical''. An engineering dicipline deserves that name only if it takes its underlying foundations (whether these are science or mathematics) seriously. I see that culture is largely lacking in CS.

Back to variance: I believe my examples have shown that
contravariance is essential as soon as we have higher-order functions (because arguments to such functions are naturally contravariant). I may well be that most Eiffel code isfirst order; then the problem does not arise so much. But once you become more functional you can't maintain the illusion that everything is covariant.
Andrew
Posts: 18 / Nickname: 55548 / Registered: May 16, 2008 6:33 AM
Re: The Origins of Scala
May 6, 2009 4:37 AM      
> I think James Iry's solution is very nice. It is probably
> obscure to anyone not familiar with Scala, but the key
> that he uses an implicit function named f to ensure that
> shareList can only be called when passing Skier subtypes
> whose SuiteableRoommate type is the same type as the Skier
> subtype itself.

yes, i agree it's nice. I also concur that it's obscure to someone not well versed in scala ;-)

> I think part of the difficulty here is that this is a
> contrived example dreamed up by Bertrand Meyers over a
> decade ago. There's no real problem we're trying to solve,
> and in such cases, it's hard to know what would make a
> good solution.

It's not such a far fetched example if we translate it to a slightly different domain.

Consider the currency example in your book: we have the Dollar and Euro classes for representing monetary amounts, say.

Now, i'd like to build a small accounting framework on top of these definitions. An account is single currency and I'd like to be able to ask for the sum in the currency of the account.

A very naive version (ok, i can't write anything else in scala ;-) would perhaps look like:

class Account
{
var entries = new ListBuffer[AbstractCurrency]

def addEntry(m : AbstractCurrency) = entries += m
def sum(): AbstractCurrency =
{
var sum = entries(0).make(0)
for (e <- entries) sum = sum + e
sum
}
}

of course, the "sum = sum + e" won't compile. I guess I could then add an abstract type into Account and create DollarAccount and EuroAccount, but this isn't really the way to structure a framework... I'll get an explosion of classes required.

is there another way, perhaps using generics?
James
Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
Re: The Origins of Scala
May 6, 2009 5:13 AM      
> What I was really objecting to is that (very common)
> sentiment that ``academic'' means ``impractical''. An
> engineering dicipline deserves that name only if it takes
> its underlying foundations (whether these are science or
> mathematics) seriously. I see that culture is largely
> lacking in CS.

I don't believe software engineering really exists as a real discipline the way mechanical engineering exists as a discipline. I think it's something that will need to be addressed eventually. I once watched a series of television programs about Isambard Kingdom Brunel and I was struck by how at the time, there were no official mechanical engineering degrees or certifications. It reminded me of how software engineering is done today. Engineers at the time used wildly different approaches and argued about them much the way we do today about things like typing systems.

But an engineering discipline requires empirical research. Software engineering would be based on CS but would have different goals much the way Physics and most Engineering disciplines are related. Before that happens, people in general need to care more about software quality. Right now, relatively few organizations are willing to pay for this kind of discipline.

> Back to variance: I believe my examples have shown that
> contravariance is essential as soon as we have
> higher-order functions (because arguments to such
> functions are naturally contravariant). I may well be that
> most Eiffel code isfirst order; then the problem does not
> arise so much. But once you become more functional you
> can't maintain the illusion that everything is covariant.

Perhaps as I understand functional programming more, I will see this too. What I struggle with now is that most developers and managers are not interested in understanding sophisticated language features and things like contravariance can cause havoc if you don't understand what you are doing.
James
Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
Re: The Origins of Scala
May 6, 2009 5:31 AM      
> What I was really objecting to is that (very common)
> sentiment that ``academic'' means ``impractical''.

I'll just say one more thing about this. It's common for engineers to have this kind of attitude about physicists and it's not that engineers don't take physics seriously. The perception of the engineers often that the physicist act as if engineering is easy. Having been in the physics world (briefly) I saw that there is something to this. The physicists did believe they could do what engineers do but better and if the engineers would just do things the way physicists said they should do it, things would be a lot better.

The reality is that most physicist could not be engineers without a lot of training and experience. I guess I would just ask that you consider that when developers say something isn't working for them, they might have valid reasons to think that. I've found in general that most people's complaints about my software are valid and I always gain a lot by taking the time to learn why.
Bill
Posts: 28 / Nickname: billpyne / Registered: January 29, 2007 4:12 AM
Re: The Origins of Scala
May 6, 2009 6:58 AM      
> > What I was really objecting to is that (very common)
> > sentiment that ``academic'' means ``impractical''.
>
> I'll just say one more thing about this. It's common for
> engineers to have this kind of attitude about physicists
> and it's not that engineers don't take physics seriously.
> The perception of the engineers often that the physicist
> t act as if engineering is easy. Having been in the
> physics world (briefly) I saw that there is something to
> this. The physicists did believe they could do what
> engineers do but better and if the engineers would just do
> things the way physicists said they should do it, things
> would be a lot better.
>
> The reality is that most physicist could not be engineers
> without a lot of training and experience. I guess I would
> just ask that you consider that when developers say
> something isn't working for them, they might have valid
> reasons to think that. I've found in general that most
> people's complaints about my software are valid and I
> always gain a lot by taking the time to learn why.

Not to hijack this topic but the practical vs. theoretical debate comes up so often it seems like there should be a separate topic just for that. At some point, field developers and academic researchers really need to communicate more. The "that's academic" reaction to less obvious ideas, while I wouldn't term as disgusting, it's certainly becoming a tired cliché. Likewise I'd like to disabuse anyone of the notion that field developers don't, as a whole, care about the quality of our product.
James
Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
Re: The Origins of Scala
May 6, 2009 7:22 AM      
> Not to hijack this topic but the practical vs. theoretical
> debate comes up so often it seems like there should be a
> separate topic just for that. At some point, field
> developers and academic researchers really need to
> communicate more.

I agree but it needs to be a two-way discussion.

> The "that's academic" reaction to less obvious ideas,
> while I wouldn't term as disgusting, it's certainly
> becoming a tired cliché.

Does it really matter what words people use to express their frustration? There's a real issue and dismissing it as 'disgusting', 'cliche', or 'silly' is not going to resolve anything.

> Likewise I'd like to
> disabuse anyone of the notion that field developers don't,
> as a whole, care about the quality of our product.

If you are referring to what I wrote above, I don't mean to say that developers don't care about quality but that consumers of software, by and large, are not concerned with quality and/or are unaware of it. MS Office software is mediocre at best while being some of the most popular software on the planet. In IT, I'm constantly confronted with an attitude that cheap software is preferred over quality software even if the cheap software doesn't solve the right problems. A lot of this has to do with the "Tragedy of the Commons" but there's a distinct attitude that good software is not worth the cost, especially when you are talking about the guts of the software.

There's nothing preventing software engineer certification right now. But any state or nation that required software engineers would immediately see development job disappear. Qualified people cost more and most software producing organizations won't receive any benefit from rigorous licensing.
Carson
Posts: 18 / Nickname: cgross / Registered: October 16, 2006 3:21 AM
Re: The Origins of Scala
May 6, 2009 7:50 AM      
> But once you become more functional you
> can't maintain the illusion that everything is covariant.

That's why GScript has a separate class of types for function types, where parameters are contravariant and return types are covariant. It leaves normal generic classes (99% of the time Collections) to be covariant, which is good enough for development work.

And we shouldn't assume that a covariant regime always means less type safety. In java generics (and, so far as I can tell, in scala) the following code is perfectly legal:


List<String> myListOfStrings = Arrays.asList( "foo", "bar" );

myListOfStrings.contains( Boolean.FALSE );


In GScript, that's a compilation error, because we've retyped contains() parameter to be T, rather than Object (or Any.) We can get away with that because of the covariant nature of generics.

Now, because we've adopted covariance of generic types you simply can't assume that a type variable type appearing at as a function parameter is correctly typed at runtime, and you must check that it is so if it matters to you. OTOH, for methods like Collection#contains() you don't really care, and you get additional call site type safety that neither java nor scala (again, as far as I can tell) provide.

I'm sure that, with sufficiently more complexity, this could be addressed in a more sound manner, butyou have to ask yourself: is the complexity worth it? Given how productive people are in dynamic langauges with no static typing whatsoever, I think the answer is probably not.

Worse is better?

Cheers,
Carson
Bill
Posts: 28 / Nickname: billpyne / Registered: January 29, 2007 4:12 AM
Re: The Origins of Scala
May 6, 2009 8:09 AM      
What I wrote wasn't pointed at anyone. Mostly I go between Artima, HackerNews, and Reddit with some LtU mixed in. At least weekly I see a similar debate go on. Daily I see someone shrug off discussion on language features as "too academic". Often enough I have read the complaint about field developers not taking software quality seriously enough as if there are no external factors influencing our (at times seemingly poor) choices.

We need a dedicated forum for discussing problems we see in the field and what researchers have to offer.

Poor quality isn't isolated to software. Fast food meets the goal of fast but is often enough not nutritious nor is it very tasty. Education - not so encouraging. From secondary to higher ed. I hear complaints from teachers about having to pass people through the system; about having to cover topics that would have been learned grades earlier a few decades ago. My point is that we have a culture of "lowest cost is good enough". Doing something well costs more upfront.

Again, I apologize for straying far off an interesting topic. James, you have my email address. If you'd like to discuss this further, please email me.
James
Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
Re: The Origins of Scala
May 6, 2009 8:13 AM      
> In java generics
> (and, so far as I can tell, in scala) the following code
> is perfectly legal:
>
>
> List<String> myListOfStrings = Arrays.asList( "foo",
> ", "bar" );
>
> myListOfStrings.contains( Boolean.FALSE );
>

>
> In GScript, that's a compilation error, because we've
> retyped contains() parameter to be T, rather than Object
> (or Any.) We can get away with that because of the
> covariant nature of generics.

I think an illustrative example is that during the beta period of java generics, there was a version of the 1.5 JDK where Map.get(Object) was defined as Map.get(K). It was then changed back to get(Object) generating a lot of questions. After 1.5 was released, it was common for people to say that Map.get() should be generic.

That's wrong and I don't think I need to explain why. As I understand it, in GScript you could cast that list to List<Object> and do the contains(Boolean) would then compile. That makes sense to me. There are times when it's reasonable to ask if an Object is in a list of Strings there are even times where it makes sense to ask if a banana is in a list of oranges but in the most common cases, you don't want that.
James
Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
Re: The Origins of Scala
May 6, 2009 8:15 AM      
> What I wrote wasn't pointed at anyone.

OK. I think we are in general agreement.
Fred
Posts: 5 / Nickname: fredgarvin / Registered: January 4, 2008 3:43 PM
Re: The Origins of Scala
May 6, 2009 9:27 AM      
Heh. It is interesting that a lot of the longer discussions in artima become a debate over the complications of generic variance. That alone indicates something is wrong. I've said this many times here: a feature of a type system is useful only if it *helps* **us** read and write code. If it's confusing us, it's not helping, thus it's not useful.

A lot of smart people have a difficult time grasping variance with respect to type parameters esp. with call site variance e.g., wildcards in Java. Definition site variance as discussed here (too) many times is also a hard pill to swallow. The difficulties arising from generics in both Java and Scala, in my opinion, indicate a failed experiment -- so far generics in the aggregate is not helping us.

I'm not suggesting we should abandon generics. We just need to be pragmatic about how we integrate generics into a language so that it's useful to most people who'll use the language. Since most people seem to have no problem understanding variance with respect to array component types, I'm inclined to believe that is the best option we have (so far). Thus the GScript and Eiffel approach seem like the way to go.
77 posts on 6 pages.