Article Discussion
The Origins of Scala
Summary: Martin Odersky talks with Bill Venners about the history that led to the creation of the Scala programming language.
76 posts.
The ability to add new comments in this discussion is temporarily disabled.
Most recent reply: May 21, 2009 4:21 AM by Gregor
    Bill
     
    Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
    The Origins of Scala
    May 3, 2009 9:00 PM      
    In this Artima interview, Bill Venners talks with Martin Odersky about the history that led to the creation of the Scala programming language:

    http://www.artima.com/scalazine/articles/origins_of_scala.html

    What do you think of Odersky's reasons for wanting to break source compatibility with Java?
    • Trond
       
      Posts: 2 / Nickname: tolsen / Registered: March 21, 2007 2:54 AM
      Re: The Origins of Scala
      May 6, 2009 7:32 PM      
      Would it be possible for Scala to also target system programming or would that a require major language revision?
    • Carson
       
      Posts: 18 / Nickname: cgross / Registered: October 16, 2006 3:21 AM
      Re: The Origins of Scala
      May 4, 2009 10:30 AM      
      Heh. Well, here's an interesting tweak on the generics reasoning: in our internal programming language we've actually gone the other way. Covariance, for all it's faults, is pretty simple to understand and use. Therefore we have covariance of generic types.

      It isn't sound but it is simple. You get 90% (95%? 99%? 30%?) of the benefits of generics with almost no mental costs. Given that java development was trucking along just fine without generics, it seems like the type-safety argument for them isn't as strong as is often suggested.

      Worse is better?

      Cheers,
      Carson
      • James
         
        Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
        Re: The Origins of Scala
        May 4, 2009 10:43 AM      
        > It isn't sound but it is simple. You get 90% (95%? 99%?
        > 30%?) of the benefits of generics with almost no mental
        > costs. Given that java development was trucking along
        > just fine without generics, it seems like the type-safety
        > argument for them isn't as strong as is often suggested.

        I think it comes down to theory over practice. In theory, covariance is not adequate. In practice, it's only potentially an issue when your covariant types are mutable and even then only in rare cases does it become an issue and I'm fairly sure there's always a workaround. You get most of the value without all the extra complexity. The real problem with more complicated variance is that it's often hard to see that the declarations you use now will limit you later and unwinding them is difficult.
        • Bill
           
          Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
          Re: The Origins of Scala
          May 4, 2009 3:39 PM      
          > > It isn't sound but it is simple. You get 90% (95%?
          > 99%?
          > > 30%?) of the benefits of generics with almost no mental
          > > costs. Given that java development was trucking along
          > > just fine without generics, it seems like the
          > type-safety
          > > argument for them isn't as strong as is often
          > suggested.
          >
          > I think it comes down to theory over practice. In theory,
          > covariance is not adequate. In practice, it's only
          > potentially an issue when your covariant types are mutable
          > and even then only in rare cases does it become an issue
          > and I'm fairly sure there's always a workaround. You get
          > most of the value without all the extra complexity. The
          > real problem with more complicated variance is that it's
          > often hard to see that the declarations you use now will
          > limit you later and unwinding them is difficult.
          >
          Are you talking primarily about Java's wildcard variance annotations? If so, I can see your point. For a long time I ignored them, and didn't try and learn or understand them, mostly because we just didn't have time to generify our app, even though we wanted to. There were just always more important things to do with our limited resources. So I didn't really tackle Java wildcards until I started learning Scala's variance stuff, and I do find wildcards confusing myself.

          Josh Bloch tried to make it easier by providing a mnemonic in his Effective Java 2nd edition book. He talked about it at Devoxx last December, and had a picture of our (Californian's) Governer, Arnold Schwarzenegger, as a younger, bare-chested man. Because the mnemonic was PECS, for Provide Extends Consumer Super. That might make it easier, but frankly I think it is a terribly big complexity burden to require anyone using any generic type in a method signature to try and figure out what the variance should be. So regardless of how you value the benefit of type soundness, the cost with wildcards seems high. And as you say, if someone gets this wrong, it can be difficult to unwind it and change it later, because any code that uses that method signature or subclasses, overrides it, etc., can break.

          I think declaration site variance that Scala uses is much easier on users, but I don't have enough experience to know how much easier. In other words, the cost is much lower than Java's wildcards, and I think also the cost of change is also mitigated. If a library designer tries to make something covariant or contravariant that isn't by nature covariant or contravariant, the program won't compile. Any nonvariant type will compile (absent other problems), but you can always change a nonvariant type parameter into a covariant or contravariant one without breaking any client code.

          I also think that nonvariant and covariant will not be too hard for most programmers to grasp. Only contravariance is counterintuitive, but hopefully with a bit of study and head scratching, most people will be able to grok it. But is Scala's variance simple enough to reduce the cost and make it worth the benefit? I'd be curious to hear what people who have actually used Scala think on that question.
        • Martin
           
          Posts: 15 / Nickname: modersky / Registered: September 14, 2003 9:46 PM
          Re: The Origins of Scala
          May 4, 2009 3:35 PM      
          > I think it comes down to theory over practice. In theory,
          > covariance is not adequate. In practice, it's only
          > potentially an issue when your covariant types are mutable
          > and even then only in rare cases does it become an issue
          > and I'm fairly sure there's always a workaround.

          Theory says this is wrong :-) Here's why:


          class Function[A, B] {
          def apply(x: A): B
          }

          class Adder extends Function[Int, Int] {
          def apply(x: Int): Int = x * x
          }

          object Test {
          def testWithString(f: Function[Any, Any]) {
          f("hi")
          }
          testWithString(Adder) // boom! we multiply a string
          }

          No mutability anywhere, yet if you let Function be covariant (which Scala does not do, I hasten to add),
          you get a runtime error. I claim this is relevant in practice because otherwise all your higher-order functions are potential type holes.

          So, sometimes, theory has a thing or two to teach to practice :-)
          • Martin
             
            Posts: 15 / Nickname: modersky / Registered: September 14, 2003 9:46 PM
            Re: The Origins of Scala
            May 4, 2009 4:11 PM      
            Slightly edited example, to make it less confusing:

            class Function[A, B] {
            def apply(x: A): B
            }

            class Squarer extends Function[Int, Int] {
            def apply(x: Int): Int = x * x
            }

            object Test {
            def testWithString(f: Function[Any, Any]) {
            f.apply("hi")
            }
            testWithString(Squarer) // In Scala, this is a variance error. Otherwise: boom! we square a string
            }
            • Carson
               
              Posts: 18 / Nickname: cgross / Registered: October 16, 2006 3:21 AM
              Re: The Origins of Scala
              May 5, 2009 9:03 AM      
              Martin,

              I think that most people here understand that covariance of generic types isn't sound. The question is, is fixing that worth the complexity involved?

              GScript has a separate class of types for function types, where argument types are contravariant and return types are covariant. This eliminates one big class of ugly problems with covariant generics. Is that enough? I think so, but then I don't do particularly complicated programming.

              It is, of course, a judgement call. I think that if you look at how collections are used in practice in java (and, in particular, note that there wasn't a scourge of ClassCastExceptions before generics came along) you can come to terms with covariance. It isn't correct, but it is useful.

              It is definitely a worse-is-better, simplicity-over-correctness argument, though.

              Cheers,
              Carson
          • James
             
            Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
            Re: The Origins of Scala
            May 5, 2009 6:08 AM      
            > No mutability anywhere, yet if you let Function be
            > covariant (which Scala does not do, I hasten to add),
            > you get a runtime error. I claim this is relevant in
            > practice because otherwise all your higher-order functions
            > are potential type holes.

            I see my error. I said mutability but what I really mean is that as long as you are not calling methods with covariant parameters and only using return values or otherwise only doing pulls instead of pushes, there are no issues with covariance. Of course, you might be able to show an example of such an issue that I am unaware of but my main point it is the unchanged.

            There's no doubt that the kind of situation you show in your example can come about but it's just not worth introducing the extra complexity over. It's a really drastic case of diminishing returns. The initial return of covariant types is huge with very little complexity. Adding soundness adds a lot of complexity for very little benefit.
            • Martin
               
              Posts: 15 / Nickname: modersky / Registered: September 14, 2003 9:46 PM
              Re: The Origins of Scala
              May 5, 2009 8:01 AM      
              > > No mutability anywhere, yet if you let Function be
              > > covariant (which Scala does not do, I hasten to add),
              > > you get a runtime error. I claim this is relevant in
              > > practice because otherwise all your higher-order
              > functions
              > > are potential type holes.
              >
              > I see my error. I said mutability but what I really mean
              > is that as long as you are not calling methods with
              > covariant parameters and only using return values or
              > otherwise only doing pulls instead of pushes, there are no
              > issues with covariance.

              Correct. And in that case you would naturally make your type parameters covariant. But for function arguments it really goes the other way round. How would you find a language where immutable collections such as Scala's lists are contravariant? I.e you could assign a List[Object] to a List[String] but not the other way round? You would find that a crazy idea, right? Yet for function parameters contravariance exactly as natural and correct as covariance is natural and correct for immutable collections. Making function parameters covariant is just as crazy as making immutable collections contravariant.

              Cheers

              -- Martin
              • James
                 
                Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
                Re: The Origins of Scala
                May 5, 2009 9:40 AM      
                > Correct. And in that case you would naturally make your
                > type parameters covariant. But for function arguments it
                > really goes the other way round. How would you find a
                > language where immutable collections such as Scala's lists
                > are contravariant? I.e you could assign a List[Object] to
                > a List[String] but not the other way round? You would find
                > that a crazy idea, right? Yet for function parameters
                > contravariance exactly as natural and correct as
                > covariance is natural and correct for immutable
                > collections. Making function parameters covariant is just
                > as crazy as making immutable collections contravariant.

                I see that. But I'm still not convinced it's worth the extra complications. That it's 'wrong' doesn't invalidate the fact that, in practice, it works just fine for a lot of people.

                My real problem with generics is not that it's difficult to understand covariance and contravariance. It's that it's hard to imagine how these declarations will affect what you can do with your APIs later.

                Here's an example of this that I ran into with Java: I had an observer interface defined for component types. I then declared that the component type were generic in terms of the observer type. That is, they could have a specialized observers. This worked well for a while because all the observers were created with full knowledge of the generic type of the component. But then I ran into a situation where I wanted to add an observer to all components based on the basic observer interface. I was stuck. I couldn't do that because the generic type of the component was not known. What I wanted was for the method to be covariant and in the case that the observer was not the generic type, allow it to receive the basic notifications. This is what I ended up doing but I had to add a new method. I could not use the generic one. So while you think this is 'crazy' this is actually perfectly fine for my needs. Even in the case where I wouldn't be able to handle the mismatch, a runtime error would be fine.
      • Bill
         
        Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
        Re: The Origins of Scala
        May 4, 2009 3:25 PM      
        > Heh. Well, here's an interesting tweak on the generics
        > reasoning: in our internal programming language we've
        > actually gone the other way. Covariance, for all it's
        > faults, is pretty simple to understand and use. Therefore
        > we have covariance of generic types.
        >
        > It isn't sound but it is simple. You get 90% (95%? 99%?
        > 30%?) of the benefits of generics with almost no mental
        > costs. Given that java development was trucking along
        > just fine without generics, it seems like the type-safety
        > argument for them isn't as strong as is often suggested.
        >
        > Worse is better?
        >
        I can't actually remember a specific instance where I actually got an ArrayStoreException in Java, though I have a vague feeling I saw one once or twice. But that's in over decade of relatively regular Java programming. So in the Java array case at least, the type unsoundness of array covariance didn't seem to be a very costly hack for me in practice. (Except perhaps for the "original sin" problem Martin pointed out of making it hard to do generics well in Java later.)

        I'm curious what the implication of assuming covariance for type parameter is in practice? Do you have to occasionally to casts if you need contravariance for example? Or how would that even work? I guess my question is, since you are actually using this approach, what is the practical downsides, and are there upsides aside from conceptual simplicity?
        • Andrew
           
          Posts: 18 / Nickname: 55548 / Registered: May 16, 2008 6:33 AM
          Re: The Origins of Scala
          May 4, 2009 11:35 PM      
          > I'm curious what the implication of assuming covariance
          > for type parameter is in practice?

          Eiffel allows covariant input parameters for methods. This approach is often "beaten up" by academic researchers, but it always seemed to be intuitive to me.

          I remember reading the excellent treatment of contra/covariance in A Theory Of Objects (Abadi, Cardelli) and thinking this only applies to the single record type dispatch model. As demonstrated in http://portal.acm.org/citation.cfm?id=203096 the conflict goes away with generic functions.
          • Jules
             
            Posts: 2 / Nickname: jules2 / Registered: March 4, 2006 9:24 AM
            Re: The Origins of Scala
            May 5, 2009 6:30 AM      
            Here's a link to the paper for those who do not have an ACM account: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.56.2799
          • Bill
             
            Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
            Re: The Origins of Scala
            May 5, 2009 11:10 AM      
            Hi Andrew,

            > > I'm curious what the implication of assuming covariance
            > > for type parameter is in practice?
            >
            > Eiffel allows covariant input parameters for methods.
            > This approach is often "beaten up" by academic
            > c researchers, but it always seemed to be intuitive to
            > me.
            >
            I looked up Bertrand Meyer's rationale in his book Object-Oriented Software Construction. I have the 2nd edition from 1997. The reason is I was reading some older posts on this subject on Artima and found an old blog by Bruce Eckel that quoted him:

            http://www.artima.com/weblogs/viewpost.jsp?thread=132041

            What Meyers shows is a hierarchy with a class Skier with two subclasses, Girl and Boy. (It's a high school ski team going to a meet.) Skier has a method share that takes another skier, and is used to model which skier will share a room with which other skier during the event. So in Java it might look like:

            abstract class Skier {
                abstract void share(other: Skier);
            }
            


            Then he wants to model a rule that only girls can share a room with girls, and boys can share a room with boys. So he says what you want to do in the subclasses is:

            class Boy extends Skier {
                void share(other: Boy) { ... }
            }
             
            class Girl extends Skier {
                void share(other: Girl) { ... }
            }
            


            In Eiffel I think the subclass share methods override (actually implement, since it is abstract) the share method in Skier, because this kind of covariance of parameter types is allowed. In Java, the subclass share methods would overload the superclass one, and you'd be told that you still need to implement share(Skier) in the subclasses.

            Meyers then later says, in a section entitled "polymorphic perversity", that "a harmful scheme is easy to make up," and gives this example (again translated to Java):

            Skier s;
            Boy b = new Boy;
            Girl g = new Girl;
            s = b;
            s.share(g);
            


            Sorry about the the one-char names, but that's what Meyer used. Basically, you assign a Boy to a variable of type Skier. You can then call share on that variable and pass in any Skier, because that's the parameter type of share in class Skier. Well, a Girl is a Skier, so I could pass a Girl to share a room with a Boy. Now what Meyer could have concluded about all this is "Cats and dogs living together! Mass Hysteria!" However, his take was:

            With contravariance one would not have these problems: as you specialize the target of a call (s in this example), you would generalize the argument. Contravariance, as a result, leads to simpler mathematical models of the inheritance-redefinition-polymorphism mechanism. For that reason a number of theoretical articles have advocated contravariance. But the argument is not very convincing, since, as we have seen and as the literature readily admits, contravariance is of essentially no practical use.

            An argument often encountered in the programming literature is that one should strive for techniques that have simple mathematical models. Mathematical elegance, however, is one one of several design criteria; we should not forget to make our designs realistic and useful too. In computing science as in other disciplines, it is after all much easier to device drastically simple theories if we neglect to make them agree with reality.

            So rather than trying to force a covariant body into a contravariant suit, we should accept the reality for what it is, covariant, and study ways to remove the unpleasant effects.


            What I think Meyer means when talking about reality is that you want to actually model that two kinds of Skiers are Boys and Girls, and that Skiers can share rooms, but only Boys can share with other Boys, and Girls with other Girls. So he suggests that the "rare but serious possibility of type violations when [covariance is] combined with polymorphism," as he put it, are something that can be managed and are worth getting the benefit of being able to model this kind of reality. He never said it was about covariance being simpler or easier to understand, just that he want to be able to model things like making sure Girl and Boy skiers stay in separate rooms.

            So fast forward 12 years, and here's how you could do this in Scala that keeps both the types and the high schoolers out of trouble. First, in Skier you add an abstract type as a member:


            abstract class Skier {
            type SuitableRoommate <: Skier
            def share(other: SuitableRoommate)
            }


            The <: means that type SuitableRoommate must be a subtype of Skier. The SuitableRoommate type is abstract, because there's no equals sign specifying it to be some other type. The share method is also abstract, as in the Java example I showed previously, because it has no definition. In Scala, you don't need to put the keyword abstract on these as in Java.

            OK, now in the subclasses, you define what the SuitableRoommate type is:

            I edited this later, because I had some bugs in the code below. Fixed now.


            class Boy extends Skier {
            type SuitableRoommate = Boy
            def share(other: Boy) {}
            }

            class Girl extends Skier {
            type SuitableRoommate = Girl
            def share(other: Girl) {}
            }


            For Boys, a suitable roommate is another Boy. And for Girls, a suitable roommate is another Girl. Now you can put Boy and Girl in the method signatures, and you get the behavior you want:


            scala> :load skier.scala
            Loading skier.scala...
            defined class Skier
            defined class Boy
            defined class Girl

            scala> val s: Skier = new Boy
            s: Skier = Boy@101afe

            scala> val g = new Girl
            g: Girl = Girl@cf63bb

            scala> s.share(g)
            <console>:10: error: type mismatch;
            found : Girl
            required: s.SuitableRoommate
            s.share(g)
            ^
            • James
               
              Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
              Re: The Origins of Scala
              May 5, 2009 0:25 PM      
              > What I think Meyer means when talking about reality is
              > that you want to actually model that two kinds of Skiers
              > are Boys and Girls, and that Skiers can share rooms, but
              > only Boys can share with other Boys, and Girls with other
              > Girls. So he suggests that the "rare but serious
              > possibility of type violations when [covariance is]
              > combined with polymorphism," as he put it, are something
              > that can be managed and are worth getting the benefit of
              > being able to model this kind of reality. He never said it
              > was about covariance being simpler or easier to
              > understand, just that he want to be able to model things
              > like making sure Girl and Boy skiers stay in separate
              > rooms.

              From what has been posted, I don't agree that this is what Meyer is saying at all. Unless I'm missing something he's saying that covariance doesn't work in this kind of situation but it's not worth including contravariance over. He saying he doesn't care about this situation much at all. That's what I read.
            • Andrew
               
              Posts: 18 / Nickname: 55548 / Registered: May 16, 2008 6:33 AM
              Re: The Origins of Scala
              May 5, 2009 0:33 PM      
              > He never said it was about covariance being simpler or easier to understand

              that's not my understanding of Mr Meyers position. from http://se.ethz.ch/~meyer/ongoing/covariance/recast.pdf , which admittedly only has him as a coauthor:

              "The cats-and-boats example is not contrived. It is typical of a common scheme:
              covariant redefinition of the type of a query (usually an attribute, but possibly a
              function) such as captain. The very first case of inheritance that many people see in
              elementary presentations tends to be (see e.g. [16]) something like a class MAMMAL
              inheriting from ANIMAL; if there is a query offspring, then its type should be
              redefined covariantly throughout, reflecting that the offspring of a mammal are
              mammals, not just animals."

              So, it appears to me as an intuition and expressiveness issue.
              • Andrew
                 
                Posts: 18 / Nickname: 55548 / Registered: May 16, 2008 6:33 AM
                Re: The Origins of Scala
                May 5, 2009 0:35 PM      
                and further from the same paper of Bertrand's:

                "The earlier discussions, and everyday examples, suggest that “the world is
                covariant”. In particular allowing covariant results but disallowing covariant
                arguments mean that we can’t associate setter procedures (such as sail or engender)
                with queries, or write object comparison functions such as is_equal with proper type
                signatures. Regrettably in light of its mathematical elegance, this scheme (contravariance) seems to
                have little practical applicability."
            • Andrew
               
              Posts: 18 / Nickname: 55548 / Registered: May 16, 2008 6:33 AM
              Re: The Origins of Scala
              May 5, 2009 0:10 PM      
              > What Meyers shows is a hierarchy with a class Skier with
              > two subclasses, Girl and Boy.

              haha. as an aside, i'm not a big fan of these types of scenarios as (1) they misuse inheritance, and (2) date very quickly. it kind of reminds me of the model in the sina composition filters paper that models Secretary as a subclass of Spouse (which is assumed to be female). Employees who have secretaries are assumed to be male of course!! http://trese.cs.utwente.nl/oldhtml/publications/reports/cf.pdf

              > In Eiffel I think the subclass share methods override
              > (actually implement, since it is abstract) the share
              > method in Skier, because this kind of covariance of

              yes, i believe these these override in eiffel. i can't even remember if eiffel has overloading.

              > Meyers then later says, in a section entitled "polymorphic
              > perversity", that "a harmful scheme is easy to make up,"
              > and gives this example (again translated to Java):

              yes, it's easy to make up a scenario where parameter covariance leads to runtime holes. whether it is a major problem in practice is another question.

              > contravariance is of essentially no practical use.

              I'd agree with bertrand here. I've found contravariance of input parameters to be useless. covariance is statically unsound, and therefore the java position of invariance seems sensible in a record type model. (Obviously contravariance of return types is a different story)

              > In computing science as in other disciplines, it is after
              > all much easier to device drastically simple theories if
              > we neglect to make them agree with reality.

              we just spend so much time trying to form the perfect type system that I wonder if it would be better to accept occasional holes for simplicity and clarity. It's the old 80/20 rule. To me Bertrand's crazy example sort of makes sense. You have a boy skier and a girl skier and they can't share a room. However, if you just tell someone you have 2 skiers and to put them in a room together, and you do it, someone's going to get in trouble ;-)

              > So he suggests that the "rare but serious
              > possibility of type violations when [covariance is]
              > combined with polymorphism," as he put it, are something
              > that can be managed and are worth getting the benefit of
              > being able to model this kind of reality. He never said it
              > was about covariance being simpler or easier to
              > understand, just that he want to be able to model things
              > like making sure Girl and Boy skiers stay in separate
              > rooms.

              fair enough. i find it simpler then to understand from a modeling perspective. for instance, if you have a Java class animal:

              class Animal { void eat(Food f) {}}

              and you want to subclass it to make Bear, you'd ideally want:

              class Bear extends Animal { void eat (Honey h) {} }

              where eat is overridden and Honey is a subclass of Food. To me this makes modeling sense, even though we can then ask the question as to what happens if we feed a bear weetabix...

              > So fast forward 12 years, and here's how you could do this
              > in Scala that keeps both the types and the high schoolers
              > out of trouble. First, in Skier you add an abstract type
              > as a member:
              >
              >

              > abstract class Skier {
              > type SuitableRoommate <: Skier
              > def share(other: SuitableRoommate)
              > }
              >

              >
              > The <: means that type SuitableRoommate must be a subtype
              > of Skier. The SuitableRoommate type is abstract, because
              > there's no equals sign specifying it to be some other
              > type. The share method is also abstract, as in the Java
              > example I showed previously, because it has no definition.
              > In Scala, you don't need to put the keyword abstract on
              > these as in Java.

              yes, you do it the same sort of way in Beta with virtual patterns. In Beta you still have type holes because the Boy and Girl would still be subclasses of Skier.

              So, in your scala example, does it mean that Boy and Girl are no longer subclasses of skier? if they are, what happens if you call Skier::share() on a boy as a Skier and pass in a girl as a Skier?

              So, the idea of putting the share() function in the base class must be so that Boy and Girl can be treated polymorphically from the perspective of the share() method at some point, surely? otherwise, why wouldn't you just omit it from the Skier base and just place share(Girl) on the Girl class and vice versa?

              And if Boy and Girl are no longer substitutable for Skier, then haven't we lost something major?

              Alternatively, i'm not understanding how Scala handles this.

              Cheers,
              Andrew
              • Bill
                 
                Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
                Re: The Origins of Scala
                May 5, 2009 1:33 PM      
                > yes, it's easy to make up a scenario where parameter
                > covariance leads to runtime holes. whether it is a major
                > problem in practice is another question.
                >
                Yes, I think its the same question with respect to covariance of arrays in Java. It is type unsound, but relatively rare that people get ArrayStoreExceptions in practice. They do get them occasionally, though.

                > > contravariance is of essentially no practical use.
                >
                > I'd agree with bertrand here. I've found contravariance
                > of input parameters to be useless. covariance is
                > statically unsound, and therefore the java position of
                > invariance seems sensible in a record type model.
                > (Obviously contravariance of return types is a different
                > t story)
                >
                I turned out to need contravariance in ScalaTest matchers design, but it is really also an example of function parameters being contravariant, because in ScalaTest a matcher is a function. Users don't really need to worry about it, or understand why it is contravariant. They just know that things they try that they think should work turn out to work as expected. If you're interested, I put some info on it in this page (search for variance):

                http://www.artima.com/scalatest/doc-0.9.5/org/scalatest/matchers/Matcher.html

                > > In computing science as in other disciplines, it is
                > after
                > > all much easier to device drastically simple theories
                > if
                > > we neglect to make them agree with reality.
                >
                > we just spend so much time trying to form the perfect type
                > system that I wonder if it would be better to accept
                > occasional holes for simplicity and clarity. It's the old
                > 80/20 rule. To me Bertrand's crazy example sort of makes
                > sense. You have a boy skier and a girl skier and they
                > can't share a room. However, if you just tell someone you
                > have 2 skiers and to put them in a room together, and you
                > do it, someone's going to get in trouble ;-)
                >
                I'd put the question slightly differently. Thinking about how to form the perfect type system is what academics are paid to do, and that's a worthy endeavor. Martin has come up with a pretty nice type system in Scala. The question I'd ask is whether the benefit offered by the type system is worth the cost of learning, understanding, and dealing with that type system. Also, what are the real benefits, and what are the real costs in practice?

                > fair enough. i find it simpler then to understand from a
                > modeling perspective. for instance, if you have a Java
                > class animal:
                >
                > class Animal { void eat(Food f) {}}
                >
                > and you want to subclass it to make Bear, you'd ideally
                > want:
                >
                > class Bear extends Animal { void eat (Honey h) {} }
                >
                > where eat is overridden and Honey is a subclass of Food.
                > To me this makes modeling sense, even though we can then
                > n ask the question as to what happens if we feed a bear
                > weetabix...
                >
                That's funny. In Programming in Scala we use this example, and show that covariant method parameters would lead to the abhorrent notion that you could feed fish to cows!

                > yes, you do it the same sort of way in Beta with virtual
                > patterns. In Beta you still have type holes because the
                > Boy and Girl would still be subclasses of Skier.
                >
                > So, in your scala example, does it mean that Boy and Girl
                > are no longer subclasses of skier? if they are, what
                > happens if you call Skier::share() on a boy as a Skier and
                > pass in a girl as a Skier?
                >
                Oops. I wondered why I got the wrong compiler error. My example was full of bugs. I forgot to make Boy and Girl extend Skier, and forgot to put an explicit type on s. Here's what I meant:


                abstract class Skier {
                type SuitableRoommate <: Skier
                def share(other: SuitableRoommate)
                }

                class Boy extends Skier {
                type SuitableRoommate = Boy
                override def share(other: Boy) {}
                }

                class Girl extends Skier {
                type SuitableRoommate = Girl
                override def share(other: Girl) {}
                }

                scala> :load skier.scala
                Loading skier.scala...
                defined class Skier
                defined class Boy
                defined class Girl

                scala> val s: Skier = new Boy
                s: Skier = Boy@101afe

                scala> val g = new Girl
                g: Girl = Girl@cf63bb

                scala> s.share(g)
                <console>:10: error: type mismatch;
                found : Girl
                required: s.SuitableRoommate
                s.share(g)
                ^


                > So, the idea of putting the share() function in the base
                > class must be so that Boy and Girl can be treated
                > polymorphically from the perspective of the share() method
                > at some point, surely? otherwise, why wouldn't you just
                > omit it from the Skier base and just place share(Girl) on
                > the Girl class and vice versa?
                >
                Yes, that's the point. Share is still in there on the base class.

                > And if Boy and Girl are no longer substitutable for Skier,
                > then haven't we lost something major?
                >
                > Alternatively, i'm not understanding how Scala handles
                > this.
                >
                Scala has something called path-dependent types, which means the actual type depends on the path you take to get to it. It highlights that objects actually do have members that are types. The type error is that they got a Girl where an s.SuitableRoommate was needed. The Scala compiler somehow keeps track that the object referenced by s, its type member is a Boy. So when you try to pass a Girl in there, you get the type error.
                • Mark
                   
                  Posts: 48 / Nickname: mthornton / Registered: October 16, 2005 11:22 PM
                  Re: The Origins of Scala
                  May 5, 2009 2:21 PM      
                  > That's funny. In Programming in Scala we use this example,
                  > and show that covariant method parameters would lead to
                  > the abhorrent notion that you could feed fish to cows!

                  The rather unfunny truth is that many animal products (including fish) really have been fed to cows. The consequences of this practice have been very expensive, especially here in the UK. So perhaps covariance models reality more accurately than you might have expected!
            • James
               
              Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
              Re: The Origins of Scala
              May 5, 2009 11:33 AM      
              > What I think Meyer means when talking about reality is
              > that you want to actually model that two kinds of Skiers
              > are Boys and Girls, and that Skiers can share rooms, but
              > only Boys can share with other Boys, and Girls with other
              > Girls.

              The way I read it, he says what he means by that in the text you have quoted: "contravariance is of essentially no practical use."
              • Bill
                 
                Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
                Re: The Origins of Scala
                May 5, 2009 11:50 AM      
                > > What I think Meyer means when talking about reality is
                > > that you want to actually model that two kinds of
                > Skiers
                > > are Boys and Girls, and that Skiers can share rooms,
                > but
                > > only Boys can share with other Boys, and Girls with
                > other
                > > Girls.
                >
                > The way I read it, he says what he means by that in the
                > text you have quoted: "contravariance is of essentially no
                > practical use."
                >
                Yes, I'm sure he meant that too. Remember this was 1997. What contravariance doesn't help you do is keep Girls and Boys in separate rooms. That was the practical benefit he saw in allowing covariant method parameters. What contravariance does do is exactly express the nature of overridden method parameters. It makes the types sound, which has some practical benefit I'd say. As Martin pointed out, once you have functions being passed around, it is handy that function parameter types are contravariant. Meyer may not have had that use case in mind back then.

                But my main point with the skier post is that there are alternative ways to get at the benefit Meyer seemed to be after when he compromised and made the parameter types covariant even though their true nature was contravariant. It's also about as simple conceptually.
                • James
                   
                  Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
                  Re: The Origins of Scala
                  May 5, 2009 0:05 PM      
                  > Yes, I'm sure he meant that too. Remember this was 1997.
                  > What contravariance doesn't help you do is keep Girls and
                  > Boys in separate rooms.

                  I'm at a disadvantage here in that I don't have the full text but the first line in what you quote is:

                  "With contravariance one would not have these problems" which seems to suggest that it does solve the problem of keeping the boys and girls in separate rooms. Maybe I'm misunderstanding but I thought the boy and girl skier example was meant to show the problem with covariance and then the following text was meant to argue that despite these types of problems, covariance is still the right choice.
                  • Bill
                     
                    Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
                    Re: The Origins of Scala
                    May 5, 2009 0:49 PM      
                    Hi James,

                    > > Yes, I'm sure he meant that too. Remember this was
                    > 1997.
                    > > What contravariance doesn't help you do is keep Girls
                    > and
                    > > Boys in separate rooms.
                    >
                    > I'm at a disadvantage here in that I don't have the full
                    > text but the first line in what you quote is:
                    >
                    > "With contravariance one would not have these problems"
                    > which seems to suggest that it does solve the problem of
                    > keeping the boys and girls in separate rooms. Maybe I'm
                    > misunderstanding but I thought the boy and girl skier
                    > example was meant to show the problem with covariance and
                    > then the following text was meant to argue that despite
                    > these types of problems, covariance is still the right
                    > choice.
                    >
                    Sorry, let me clarify. Meyer was pointing out that although covariance has a type soundness problem, it achieves his goal of modeling the use case of Skiers, Boys, Girls, and Rooms. He then points out that although contravariance does not have the type problem, contravariance doesn't let him model the use case of Skiers, Boys, Girls, and Rooms. He also goes further to say he doesn't know of any use cases that contravariance actually helps you implement. And then says that in a nod to the practical, he made type parameters covariant so he could support this kind of use case even though it has a type soundness hole.
                    • Martin
                       
                      Posts: 15 / Nickname: modersky / Registered: September 14, 2003 9:46 PM
                      Re: The Origins of Scala
                      May 5, 2009 1:22 PM      
                      The Eiffel example is very instructive. In the end, we want our types to model interesting properties of the application domain as faithfully as possible. Given our current knowledge, this is hard. Often, there's a temptation to give up, and just do a convenient shortcut, as in Eiffel covariance or Java arrays. But I fear that everytime one does this, it has really bad long-term consequences. I have experienced that first hand for Java arrays and suspect that Eiffel has suffered the same.

                      I do not advocate at all to ignore the needs of practical modelling. It's precisely this tension between modelling needs and logic which is fruitful in the long run. As an example, I claim that Scala has solved Eiffel's covariance problems by the introduction of abstract type members. If we had swept the problem under the carpet by adopting unsound rules, we would never have found the right solution.
                      • Andrew
                         
                        Posts: 18 / Nickname: 55548 / Registered: May 16, 2008 6:33 AM
                        Re: The Origins of Scala
                        May 5, 2009 1:35 PM      
                        > As an example, I claim that Scala has solved Eiffel's covariance
                        > problems by the introduction of abstract type members.

                        I must be missing something.

                        Given that Scala Boys and Girls are no longer subclasses of Skier, how do I write a function which can treat Boys and Girls as Skiers in Scala, thereby retaining the spirit of the original model?

                        i.e. def test(s: Skier) { println(s) }

                        Is there a type marking to put on the input parameter to indicate that it can accept covariant extensions to Skier?
                        • Andrew
                           
                          Posts: 18 / Nickname: 55548 / Registered: May 16, 2008 6:33 AM
                          Re: The Origins of Scala
                          May 5, 2009 1:37 PM      
                          > Given that Scala Boys and Girls are no longer subclasses
                          > of Skier, how do I write a function which can treat Boys
                          > and Girls as Skiers in Scala, thereby retaining the spirit
                          > of the original model?

                          ignore this, bill just answered my question as i posted!

                          cheers,
                          Andrew
                      • Andrew
                         
                        Posts: 18 / Nickname: 55548 / Registered: May 16, 2008 6:33 AM
                        Re: The Origins of Scala
                        May 5, 2009 1:50 PM      
                        > I claim that Scala has solved Eiffel's covariance
                        > problems by the introduction of abstract type members.

                        yes, i can see it is very powerful and resolves the type errors that I used to get in languages like Beta.

                        One question however: is it possible given Bill's Girl/Boy classes to write a function which takes 2 Skiers and makes them share a room, such that it works for 2 Boys or 2 Girls?

                        i.e. my first attempt doesn't compile:

                        def test[T <: Skier](s1: T, s2: T) { s1.share(s2) }

                        Cheers,
                        Andrew
                        • Bill
                           
                          Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
                          Re: The Origins of Scala
                          May 5, 2009 2:39 PM      
                          Hi Andrew,

                          > > I claim that Scala has solved Eiffel's covariance
                          > > problems by the introduction of abstract type members.
                          >
                          > yes, i can see it is very powerful and resolves the type
                          > errors that I used to get in languages like Beta.
                          >
                          > One question however: is it possible given Bill's Girl/Boy
                          > classes to write a function which takes 2 Skiers and makes
                          > them share a room, such that it works for 2 Boys or 2
                          > Girls?
                          >
                          > i.e. my first attempt doesn't compile:
                          >
                          > def test[T <: Skier](s1: T, s2: T) { s1.share(s2) }
                          >
                          Well one trouble with this attempt is that it isn't defined in Skier that the type of SuitableRoommate will always be the outer type. This is possible for example:


                          class Cat extends Skier {
                          type SuitableRoommate = Dog
                          override def share(other: Dog) {}
                          }

                          class Dog extends Skier {
                          type SuitableRoommate = Cat
                          override def share(other: Cat) {}
                          }


                          More generally, the compiler has to check the path dependent types at compile time, so it must base them on the type of the variable. What that means you can't do with Scala's abstract type approach that you can do (without safety) with Eiffel's covariant method parameters is pass either a Boy to a Skier that will happen to be a Boy at runtime. For example:


                          scala> val s: Skier = new Boy
                          s: Skier = Boy@995b29

                          scala> val g = new Girl
                          g: Girl = Girl@7aaee9

                          scala> val b = new Boy
                          b: Boy = Boy@43c919

                          scala> s.share(b)
                          <console>:9: error: type mismatch;
                          found : Boy
                          required: s.SuitableRoommate
                          s.share(b)
                          ^

                          scala> s.share(g)
                          <console>:10: error: type mismatch;
                          found : Girl
                          required: s.SuitableRoommate
                          s.share(g)
                          ^

                          scala> b.share(b)

                          scala> b.share(g)
                          <console>:10: error: type mismatch;
                          found : Girl
                          required: b.SuitableRoommate
                          b.share(g)
                          ^

                          scala> g.share(g)

                          scala> g.share(b)
                          <console>:10: error: type mismatch;
                          found : Boy
                          required: g.SuitableRoommate
                          g.share(b)
                          ^


                          So a Girl can share a room with a Girl, and a Boy with a Boy, and a Girl can't share a room with a Boy, nor can a Boy share a room with a Girl, but a Boy held from a Skier reference can also not share a room with a Boy. I think that may be what you'll run up against if you try to make a method like you're talking about. However I'm not convinced there isn't a way to do it until I try. I don't have time to try it now but later I'll see if I can come up with a solution.
                          • Andrew
                             
                            Posts: 18 / Nickname: 55548 / Registered: May 16, 2008 6:33 AM
                            Re: The Origins of Scala
                            May 5, 2009 3:08 PM      
                            > What that means you can't do with Scala's abstract type approach that you can do
                            > (without safety) with Eiffel's covariant method parameters
                            > is pass either a Boy to a Skier that will happen to be a
                            > Boy at runtime.

                            i'm sure there has to be a way. otherwise, it would restrict the type of framework-like code you could write in scala where abstract types were used? or perhaps there's another way using generic type parameters?

                            for example, how would you create a single method that took a list of either boys or girls and arranged it so that everyone in the list was sharing a room, without duplicating the code for boys and girls separately?
                            • James
                               
                              Posts: 11 / Nickname: jiry / Registered: November 15, 2007 2:29 AM
                              Re: The Origins of Scala
                              May 5, 2009 4:45 PM      
                              > for example, how would you create a single method that
                              > took a list of either boys or girls and arranged it so
                              > that everyone in the list was sharing a room, without
                              > duplicating the code for boys and girls separately?

                              This is quite an interesting challenge. I'm not sure if this is the best way, but here's what I've got


                              // use an implicit to prove that the suitable roommate matches
                              def shareList[T <: Skier](list : List[T])(implicit f : T => Skier {type SuitableRoommate = T} ) = {
                              val length = list.length
                              (list take length/2) zip (list drop length/2) foreach {case (x, y) => f(x) share y}
                              }

                              // these two compile just fine
                              shareList(List(new Boy, new Boy, new Boy, new Boy))
                              shareList(List(new Girl, new Girl, new Girl, new Girl))

                              // this does not compile
                              shareList(List(new Girl, new Girl, new Girl, new Boy))


                              It does make me think that there must be a cleaner way to express the constraint than with an implicit. But I'm stymied about what it might be. The more direct approach "def shareList[T <: Skier {type SuitableRoommate = T}]" got me errors regarding cyclic references to type T.

                              Now, if the challenge was to take a List[Skier] and pair everybody up then we'll have to do something a bit different to recover static types using pattern matching. Here's one way to go about it.


                              def separateSkier(s : Skier) : Either[Boy, Girl] = s match {
                              case b : Boy => Left(b)
                              case g : Girl => Right(g)
                              }


                              With that I can take a list of Skiers and split it into Boys and a list of Girls using a generic split function


                              def split[T, A, B](lst : List[T], separate : T => Either[A, B]) = lst.foldLeft((List[A](), List[B]())) {
                              case ((ls, rs), x) => separate(x).fold({l => (l :: ls, rs)}, {r => (ls, r :: rs)})
                              }


                              And here's how to split a list


                              val (boys, girls) = split(List(new Boy, new Girl, new Boy, new Girl), separateSkier)


                              And now we can call pairList(boys) and pairList(girls).
                              • Bill
                                 
                                Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
                                Re: The Origins of Scala
                                May 5, 2009 10:29 PM      
                                > Andrew wrote:
                                > > for example, how would you create a single method that
                                > > took a list of either boys or girls and arranged it so
                                > > that everyone in the list was sharing a room, without
                                > > duplicating the code for boys and girls separately?
                                >
                                James Wrote:
                                > It does make me think that there must be a cleaner way to
                                > express the constraint than with an implicit. But I'm
                                > stymied about what it might be. The more direct approach
                                > "def shareList[T <: Skier {type SuitableRoommate = T}]"
                                > got me errors regarding cyclic references to type T.
                                >
                                I think part of the difficulty here is that this is a contrived example dreamed up by Bertrand Meyers over a decade ago. There's no real problem we're trying to solve, and in such cases, it's hard to know what would make a good solution.

                                However, I get the sense that Andrew's basic desire is to be able to call share on a variable of type Skier. I think the trouble is that if the goal is to have the compiler to statically determine that you're only passing in a suitable roommate each time you call share, and all you know about the object you're calling share on (at compile time) is that it is a Skier, then the compiler doesn't have enough information to compile the code. It just isn't defined at the Skier level of abstraction what type is suitable to be passed to share.

                                As a result, any time you call share, you will need to call it on either a Boy reference or a Girl reference, or a reference of some other subtype of Skier that has a concrete SuitableRoommate.

                                I think James Iry's solution is very nice. It is probably obscure to anyone not familiar with Scala, but the key that he uses an implicit function named f to ensure that shareList can only be called when passing Skier subtypes whose SuiteableRoommate type is the same type as the Skier subtype itself.

                                James, you don't need to actually call f, because by the time you get into the shareList method, T is already known to fulfill the constraint. So this works too (the difference from your code is instead of f(x) share y, it is just x share y):


                                // use an implicit to prove that the suitable roommate matches
                                def shareList[T <: Skier](list : List[T])(implicit f : T => Skier {type SuitableRoommate = T} ) = {
                                val length = list.length
                                (list take length/2) zip (list drop length/2) foreach {case (x, y) => x share y}
                                }

                                // these two compile just fine
                                shareList(List(new Boy, new Boy, new Boy, new Boy))
                                shareList(List(new Girl, new Girl, new Girl, new Girl))


                                For those not familiar with Scala's implicits, an implicit parameter is a parameter that's passed in implicitly without you having to type the code at the call site. You can also do it explicitly, and that would look like this:


                                shareList(List(new Boy, new Boy, new Boy, new Boy))((boy: Boy) => boy)
                                shareList(List(new Girl, new Girl, new Girl, new Girl))((girl: Girl) => girl)


                                Because the parameter is declared implicit in the shareList signature, clients don't need to specify it explicitly. But all it is is a function that takes a Boy and returns a Boy, or a Girl and returns a Girl, respectively, both of which adhere to the constraint James put on the implicit function. That constraint is that the input to the function be a T (which is already defined to be a Skier with the [T <: Skier] at the beginning of the shareList signature), and the output be a Skier whose SuiteableRoommate type is also T. That only works for Boy and Girl. You couldn't pass to shareList a List of Cat or Dog types, which I showed earlier, that had each other as SuitableRoommates.

                                The implicit function that gets picked up is the identity function, which Scala imports into every Scala source file. This is an implicit function from any type to itself. All this function does is convert a Boy to a Boy, or a Girl to a Girl, which does nothing in this case, except reduce the types that can be passed to shareList. In my version of the shareList method, I don't even ever invoke the function.

                                One last tweak that could be used to make the shareList function a bit more concise is to use a view bounds. This is just a shorthand for the implicit that James wrote out explicitly. That would look like this:


                                def shareList[T <% Skier {type SuitableRoommate = T}](list : List[T]) = {
                                val length = list.length
                                (list take length/2) zip (list drop length/2) foreach {case (x, y) => x share y}
                                }

                                // these two compile just fine
                                shareList(List(new Boy, new Boy, new Boy, new Boy))
                                shareList(List(new Girl, new Girl, new Girl, new Girl))


                                So there it is. I would probably use this code, though I'm not sure I would have thought of it. Not without a lot of trial and error, but maybe from now on I'll just email James Iry and let him have at it first.

                                If you want a more understandable description of implicits and view bounds, you may want to check out the Feel of Scala video on Parleys:

                                http://tinyurl.com/dcfm4c
                                • Andrew
                                   
                                  Posts: 18 / Nickname: 55548 / Registered: May 16, 2008 6:33 AM
                                  Re: The Origins of Scala
                                  May 6, 2009 4:37 AM      
                                  > I think James Iry's solution is very nice. It is probably
                                  > obscure to anyone not familiar with Scala, but the key
                                  > that he uses an implicit function named f to ensure that
                                  > shareList can only be called when passing Skier subtypes
                                  > whose SuiteableRoommate type is the same type as the Skier
                                  > subtype itself.

                                  yes, i agree it's nice. I also concur that it's obscure to someone not well versed in scala ;-)

                                  > I think part of the difficulty here is that this is a
                                  > contrived example dreamed up by Bertrand Meyers over a
                                  > decade ago. There's no real problem we're trying to solve,
                                  > and in such cases, it's hard to know what would make a
                                  > good solution.

                                  It's not such a far fetched example if we translate it to a slightly different domain.

                                  Consider the currency example in your book: we have the Dollar and Euro classes for representing monetary amounts, say.

                                  Now, i'd like to build a small accounting framework on top of these definitions. An account is single currency and I'd like to be able to ask for the sum in the currency of the account.

                                  A very naive version (ok, i can't write anything else in scala ;-) would perhaps look like:

                                  class Account
                                  {
                                  var entries = new ListBuffer[AbstractCurrency]

                                  def addEntry(m : AbstractCurrency) = entries += m
                                  def sum(): AbstractCurrency =
                                  {
                                  var sum = entries(0).make(0)
                                  for (e <- entries) sum = sum + e
                                  sum
                                  }
                                  }

                                  of course, the "sum = sum + e" won't compile. I guess I could then add an abstract type into Account and create DollarAccount and EuroAccount, but this isn't really the way to structure a framework... I'll get an explosion of classes required.

                                  is there another way, perhaps using generics?
                            • Morgan
                               
                              Posts: 37 / Nickname: miata71 / Registered: March 29, 2006 6:09 AM
                              Re: The Origins of Scala
                              May 5, 2009 3:29 PM      
                              Has anybody noticed that all Artima discussions degenerate into debates over Java generics? :-)

                              For the record, I think Meyer's example is, ultimately, poor. There are lots of reasons for Skiers not to room together other than sex. Does he propose to add AntiSmokingZealotGirl and LazySlobSmokingBoy subclasses? No, all skiers (or a superclass, Person) should have something like a canRoomWith(Skier other) method.

                              Use the classes to prevent Skiers from rooming with Skis and Avalanches.
                              • Andrew
                                 
                                Posts: 18 / Nickname: 55548 / Registered: May 16, 2008 6:33 AM
                                Re: The Origins of Scala
                                May 5, 2009 3:32 PM      
                                > For the record, I think Meyer's example is, ultimately,
                                > poor.

                                yup, no argument there. the actual concrete example is odd in that it uses the inheritance graph and type system to encode what is effectively a policy about sharing rooms. i.e. the policy could easily change or be subject to more complex rules.

                                however, the example does serve to raise an important question (for me at least ;-) about how to write reusable code in the presence of abstract types.
                                • Fred
                                   
                                  Posts: 5 / Nickname: fredgarvin / Registered: January 4, 2008 3:43 PM
                                  Re: The Origins of Scala
                                  May 6, 2009 9:27 AM      
                                  Heh. It is interesting that a lot of the longer discussions in artima become a debate over the complications of generic variance. That alone indicates something is wrong. I've said this many times here: a feature of a type system is useful only if it *helps* **us** read and write code. If it's confusing us, it's not helping, thus it's not useful.

                                  A lot of smart people have a difficult time grasping variance with respect to type parameters esp. with call site variance e.g., wildcards in Java. Definition site variance as discussed here (too) many times is also a hard pill to swallow. The difficulties arising from generics in both Java and Scala, in my opinion, indicate a failed experiment -- so far generics in the aggregate is not helping us.

                                  I'm not suggesting we should abandon generics. We just need to be pragmatic about how we integrate generics into a language so that it's useful to most people who'll use the language. Since most people seem to have no problem understanding variance with respect to array component types, I'm inclined to believe that is the best option we have (so far). Thus the GScript and Eiffel approach seem like the way to go.
                                  • Bill
                                     
                                    Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
                                    Re: The Origins of Scala
                                    May 6, 2009 11:08 AM      
                                    Hi Fred,

                                    > Heh. It is interesting that a lot of the longer
                                    > discussions in artima become a debate over the
                                    > complications of generic variance. That alone indicates
                                    > something is wrong. I've said this many times here: a
                                    > feature of a type system is useful only if it *helps*
                                    > **us** read and write code. If it's confusing us, it's not
                                    > helping, thus it's not useful.
                                    >
                                    I agree with the general thrust here, but might say it differently. I think you need to strive to make features as simple as possible. But sometimes a feature, though it be hard to learn, can be still worth it. In other words, even if the cost is great, it can still be worthwhile so long as the benefit is greater.

                                    > A lot of smart people have a difficult time grasping
                                    > variance with respect to type parameters esp. with call
                                    > site variance e.g., wildcards in Java. Definition site
                                    > variance as discussed here (too) many times is also a hard
                                    > pill to swallow. The difficulties arising from generics in
                                    > both Java and Scala, in my opinion, indicate a failed
                                    > experiment -- so far generics in the aggregate is not
                                    > helping us.
                                    >
                                    I've met a lot of people who have expressed the same sentiment about Java's wildcards, and I myself have that feeling. Although the basic concept of a wildcard I pretty much grok, I find the subtleties of wildcards quite confusing and challenging myself. By contrast, I find Scala's declaration site variance quite easy to grok. The most challenging part for me was contravariance, but with some study I got it.

                                    You made a blanket statement that smart people find definition site variance a hard pill to swallow. I've know many dynamic typing leaning folks who don't want to swallow the pill, but I must say so far I've never heard someone who actually programs in Scala complain that Scala's variance is too hard to understand. They all have complaints, as do I, but the complexity of the type system just doesn't pop up much as a complaint.

                                    That said, I think it is too early to tell. The people coming to Scala so far are early adopters and may not represent the mainstream well. They may also be people who are comfortable with type systems, etc. So what I'm curious about is to what extent this will become a complaint as more people try Scala. It also tells me that maybe I should publish some things that try and explain definition site variance in easy to understand terms.

                                    > I'm not suggesting we should abandon generics. We just
                                    > need to be pragmatic about how we integrate generics into
                                    > a language so that it's useful to most people who'll use
                                    > the language. Since most people seem to have no problem
                                    > understanding variance with respect to array component
                                    > types, I'm inclined to believe that is the best option we
                                    > have (so far). Thus the GScript and Eiffel approach seem
                                    > like the way to go.
                                    >
                                    Eiffel, we've been guessing, didn't really let you pass functions around. So contravariance wasn't an issue there as much. But most of today's emerging languages have first class functions. That's one of the trends.

                                    If I understand Carson correctly, GScript does have contravariance of function parameters. So GScript has contravariance, the hard to understand one. It just only has it in one place, like Fan's only letting you put type parameters on Map, List, or Func, or Java only letting you use + on String. Scala just makes contravariance a general feature. You can make your own types contravariant if you want. The place where I've seen contravariance used in Scala is primarily function parameters. The place where I used it was in Matcher, which is also a function, and contravariance is for the function parameter. It was nice that I could do that.

                                    What GScript does then, if I understand it, is everything else is covariant, a la Eiffel. In Scala, other things will be either nonvariant or covariant. The Scala compiler will only let you make something covariant if it is type sound. It won't let you make a mistake. Also, if you're not sure, just start out nonvariant. Later you can make it covariant without breaking any client code. There are no wildcards to think about, and with those out of the way, I really don't think the conceptual complexity of grasping this stuff is much greater in Scala than GScript.

                                    But as I said before, time will tell. I'm not sure Scala's early adopters are a good measure of how palatable the masses would find Scala's approach to variance.
                        • James
                           
                          Posts: 11 / Nickname: jiry / Registered: November 15, 2007 2:29 AM
                          Re: The Origins of Scala
                          May 5, 2009 2:30 PM      
                          > One question however: is it possible given Bill's Girl/Boy
                          > classes to write a function which takes 2 Skiers and makes
                          > them share a room, such that it works for 2 Boys or 2
                          > Girls?
                          >
                          > i.e. my first attempt doesn't compile:
                          >
                          > def test[T <: Skier](s1: T, s2: T) { s1.share(s2) }


                          def test(s1 : Skier)(s2 : s1.SuitableRoommate) = s1 share s2

                          If you don't write it in curried form Scala complains about an illegal dependent method type. I'm not sure why. I tested this using the 2.7.4 compiler.
                          • Andrew
                             
                            Posts: 18 / Nickname: 55548 / Registered: May 16, 2008 6:33 AM
                            Re: The Origins of Scala
                            May 5, 2009 2:33 PM      
                            > def test(s1 : Skier)(s2 : s1.SuitableRoommate) = s1 share s2
                            >
                            > If you don't write it in curried form Scala complains
                            > about an illegal dependent method type. I'm not sure why.
                            > I tested this using the 2.7.4 compiler.

                            thanks james. yes, i got the illegal dependent type error also when I used the non-curried form also. In fact, I didn't know that it was possible to use a curried form ;-)

                            Andrew
                            • Bill
                               
                              Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
                              Re: The Origins of Scala
                              May 6, 2009 10:41 AM      
                              Hi Andrew,

                              > > def test(s1 : Skier)(s2 : s1.SuitableRoommate) = s1
                              > share s2
                              > >
                              > > If you don't write it in curried form Scala complains
                              > > about an illegal dependent method type. I'm not sure
                              > why.
                              > > I tested this using the 2.7.4 compiler.
                              >
                              > thanks james. yes, i got the illegal dependent type error
                              > also when I used the non-curried form also. In fact, I
                              > didn't know that it was possible to use a curried form
                              > ;-)
                              >
                              OK I got an answer on this one. Dependent methods are currently an experimental feature of the Scala compiler. So to see them in action you need to add a -Xexperimental to the command line when you run a script or compile a .scala file. If you do that, it works as James showed. You do need to use the curried form. I also don't know why. But it basically lets you do what you want. Given these types:


                              abstract class Skier {
                              type SuitableRoommate <: Skier
                              def share(other: SuitableRoommate)
                              }

                              class Boy extends Skier {
                              type SuitableRoommate = Boy
                              override def share(other: Boy) {}
                              }

                              class Girl extends Skier {
                              type SuitableRoommate = Girl
                              override def share(other: Girl) {}
                              }

                              class Dog extends Skier {
                              type SuitableRoommate = Cat
                              override def share(other: Cat) {}
                              }

                              class Cat extends Skier {
                              type SuitableRoommate = Dog
                              override def share(other: Dog) {}
                              }


                              You can write a method like this:


                              def placeInRoom(skier: Skier)(roommate: skier.SuitableRoommate) { skier share roommate }


                              And call it like this:


                              placeInRoom(new Dog)(new Cat)
                              placeInRoom(new Cat)(new Dog)
                              placeInRoom(new Girl)(new Girl)
                              placeInRoom(new Boy)(new Boy)


                              These all work, because the roommate passed is suitable for the skier. If you try and pass in an unsuitable roommate, you'll get a compiler error:


                              placeInRoom(new Dog)(new Dog)

                              (fragment of skier.scala):33: error: type mismatch;
                              found : this.Dog
                              required: this.Dog#SuitableRoommate
                              placeInRoom(new Dog)(new Dog)
                              ^
                              one error found


                              Now by the way, this still doesn't violate the rule that the Scala compiler will need a specific reference (i.e., more specific than Skier) to call share on, but it moves the requirement to the call site. You can only call placeInRoom if the type of the variable or expression you specify for the first argument is more specific than Skier. Here's an example where it isn't. Even though I'm holding onto a Dog, the type of my variable is Skier, and when I try to pass a Dog in that way, I get a compiler error:


                              val skierDog: Skier = new Dog
                              placeInRoom(skierDog)(new Dog)

                              (fragment of skier.scala):35: error: type mismatch;
                              found : this.Dog
                              required: this.Skier#SuitableRoommate
                              placeInRoom(skierDog)(new Dog)
                              ^
                              one error found


                              Anyway, the -Xexperimental stuff is where EPFL puts things that aren't quite ready or proven. If this does mature and work out, then they would likely put it in the regular Scala. But it is possible that it doesn't work out, in which case it will never get into regular Scala. What you can do in that case, and in the mean time, is add the methods you want to add to the specific classes themselves. Inside those classes, the SuitableRoommate is defined and usable. But that gives you the code duplication I think you are mainly trying to avoid. In some cases you can avoid it with the view bounds trick demonstrated by James Iry's example.
                              • Andrew
                                 
                                Posts: 18 / Nickname: 55548 / Registered: May 16, 2008 6:33 AM
                                Re: The Origins of Scala
                                May 6, 2009 11:14 AM      
                                > Anyway, the -Xexperimental stuff is where EPFL puts things
                                > that aren't quite ready or proven.

                                many thanks for that, Bill. that's exactly what I was looking for.

                                >In some cases you can avoid it with the view bounds trick
                                > demonstrated by James Iry's example.

                                i'm still struggling with the Account example I posted earlier. The Skiing analog would be to make a class (say called Booker) which has a member variable which is a listbuffer of Skiiers, with a method called bookRooms() which pairs up the skiiers in the list and shares() them. Obviously the assumption is that only compatible (i.e. the same type of) skiiers can be added to the list in the 1st place. I also want to avoid creating BoyBooker and GirlBooker classes, and pushing the type parameter higher.

                                I'll try James' clever approach next.

                                Which brings me to something bugging me a bit: the scala type system is clearly very advanced, and i expect that someone has proven it sound. What i'm trying to work out is to what extent are all reasonably practical cases covered (I realise this is very vague). Or does having a more powerful type system actually prevent you from doing things which could be done via the casting and other covariant holes in a less powerful (e.g. Java) type system? At this point, I'm sort of getting an insight into why dynamic language proponents claim typing restricts them even though I've not been previously sympathetic in any way...

                                Cheers,
                                Andrew
                                • Admin
                                   
                                  Posts: 15 / Nickname: admin / Registered: January 17, 2002 3:57 PM
                                  Re: The Origins of Scala
                                  May 6, 2009 11:26 AM      
                                  Hi Andrew,

                                  > Which brings me to something bugging me a bit: the scala
                                  > type system is clearly very advanced, and i expect that
                                  > someone has proven it sound. What i'm trying to work out
                                  > is to what extent are all reasonably practical cases
                                  > covered (I realise this is very vague). Or does having a
                                  > more powerful type system actually prevent you from doing
                                  > things which could be done via the casting and other
                                  > covariant holes in a less powerful (e.g. Java) type
                                  > system? At this point, I'm sort of getting an insight
                                  > into why dynamic language proponents claim typing
                                  > restricts them even though I've not been previously
                                  > sympathetic in any way...
                                  >
                                  I think doing things in a statically type safe way often makes things more complicated than doing it in a dynamic way, and can often require a bit more code as well. (In the case of Scala, it is a bit more code. In the case of Java, a lot more code.) The benefit you get for dealing with that extra code and complexity are the benefits of static typing, which include deterministic refactoring, documentation of programmer intent, possibly runtime performance (though often this is not relevant), better IDE support (code completion, etc.).

                                  But what I always try and point out is that just because you're in a static language doesn't mean you can't use dynamic techniques when they are a better tradeoff. In ScalaTest, I have a couple matchers that are using dynamic techniques. When you use them, you lose deterministic refactoring, but you gain that you need to write a bit less code. Since this is testing, you'll hopefully find out the next time you run the tests what broke when you made a change.

                                  One other thing is that when you can't express a constraint in the type system, you may be able in Scala to make a compiler plugin that catches the constraint. (In Java, you can do this only with an annotations processor, so there needs to be an annotation. But in Scala, you get the whole AST to play with.) I demonstrate this kind of stuff in my Feel of Scala talk on Parleys:

                                  http://tinyurl.com/dcfm4c
                                • James
                                   
                                  Posts: 11 / Nickname: jiry / Registered: November 15, 2007 2:29 AM
                                  Re: The Origins of Scala
                                  May 6, 2009 11:45 AM      
                                  > Or does having a
                                  > more powerful type system actually prevent you from doing
                                  > things which could be done via the casting and other
                                  > covariant holes in a less powerful (e.g. Java) type
                                  > system?

                                  Scala's type system is essentially Java's type system plus stuff. The big exceptions are that arrays are not covariant and Scala doesn't have "raw types." In other words, if you have a type design that works well for Java then it will probably work well in Scala (although there might be a better solution available).

                                  > At this point, I'm sort of getting an insight
                                  > into why dynamic language proponents claim typing
                                  > restricts them even though I've not been previously
                                  > sympathetic in any way...

                                  I don't follow. You've started going down this path to explore abstract type members and path dependent types. It's an interesting and sometimes useful approach to things. But it's not always the right answer. If abstract type members aren't the right answer (and frankly I rarely use them) then don't use them. Use something else.

                                  If you want static checking then maybe "generic" type parameters are the right answer. In fact, I think that until there's a larger body of experience with path dependent types, the default solution to most such "real" static typing problems should be type parameters.

                                  And if static checking isn't the right answer and you want to dynamically check compatible skiers or currency or whatever, Scala gives powerful tools for that as well. You can use OO style polymorphism or pattern matching, or both.

                                  Finally, I want to point out that Scala has an escape mechanism in its type system. You can cast anything to anything at anytime. That's a big gaping hole as far as type systems are concerned, but a pragmatic and useful choice sometimes.
                                  • Andrew
                                     
                                    Posts: 18 / Nickname: 55548 / Registered: May 16, 2008 6:33 AM
                                    Re: The Origins of Scala
                                    May 6, 2009 0:03 PM      
                                    > > At this point, I'm sort of getting an insight
                                    > > into why dynamic language proponents claim typing
                                    > > restricts them even though I've not been previously
                                    > > sympathetic in any way...
                                    >
                                    > I don't follow. You've started going down this path to
                                    > explore abstract type members and path dependent types.
                                    > It's an interesting and sometimes useful approach to
                                    > o things. But it's not always the right answer. If
                                    > abstract type members aren't the right answer (and frankly
                                    > I rarely use them) then don't use them. Use something
                                    > else.

                                    let me clarify a bit. I'm not criticising Scala's type system per se. It seems very powerful and I can't really critique it as I literally understand 10% of it. i guess i just had a brief insight into the lengths to which it is sometimes necessary to delve into and understand the type system in order to express something in Scala which I have otherwise found intuitive in other languages (despite them being unsound). This type of expertise in type systems is not required with dynamic typing, although clearly this has severe disadvantages with regard to checking and refactoring.

                                    (i'm no dynamic typing proponent however. i worked commercially in smalltalk on a medium sized team, and found the lack of typing to be very unpleasant)

                                    > If you want static checking then maybe "generic" type
                                    > parameters are the right answer. In fact, I think that
                                    > until there's a larger body of experience with path
                                    > dependent types, the default solution to most such "real"
                                    > static typing problems should be type parameters.

                                    Yes, sure, i'd tend to use generics also. I guess I was just following a path of thought from the Scala book. I've also been influenced somewhat by Beta perhaps, which has no generics and instead uses virtual patterns (path dependent types, I guess) to express this type of flexibility.

                                    > And if static checking isn't the right answer and you want
                                    > to dynamically check compatible skiers or currency or
                                    > whatever, Scala gives powerful tools for that as well.
                                    > You can use OO style polymorphism or pattern matching, or
                                    > r both.

                                    Sure.

                                    > Finally, I want to point out that Scala has an escape
                                    > mechanism in its type system. You can cast anything to
                                    > anything at anytime. That's a big gaping hole as far as
                                    > type systems are concerned, but a pragmatic and useful
                                    > choice sometimes.

                                    Hmm, that is interesting. And without wishing to be flippant, I think the presence of that "hole" would be very important in some programs. I expect there will always be things that are difficult or impossible to express in a practical type system.

                                    Andrew
                                    • Bill
                                       
                                      Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
                                      Re: The Origins of Scala
                                      May 6, 2009 1:04 PM      
                                      > > > At this point, I'm sort of getting an insight
                                      > > > into why dynamic language proponents claim typing
                                      > > > restricts them even though I've not been previously
                                      > > > sympathetic in any way...
                                      > >
                                      > > I don't follow. You've started going down this path to
                                      > > explore abstract type members and path dependent types.
                                      > > It's an interesting and sometimes useful approach to
                                      > > o things. But it's not always the right answer. If
                                      > > abstract type members aren't the right answer (and
                                      > frankly
                                      > > I rarely use them) then don't use them. Use something
                                      > > else.
                                      >
                                      > let me clarify a bit. I'm not criticising Scala's type
                                      > system per se. It seems very powerful and I can't really
                                      > critique it as I literally understand 10% of it. i guess
                                      > i just had a brief insight into the lengths to which it is
                                      > sometimes necessary to delve into and understand the type
                                      > system in order to express something in Scala which I have
                                      > otherwise found intuitive in other languages (despite them
                                      > being unsound). This type of expertise in type systems is
                                      > not required with dynamic typing, although clearly this
                                      > has severe disadvantages with regard to checking and
                                      > refactoring.
                                      >
                                      I think that's a reasoned way to look at it. There's no free lunch. Dynamic typing can indeed remove some complexity and bulk from your code, but in exchange complexity pops up elsewhere, such as needing to write more tests, less assurance when refactoring, less clarity when trying to figure out what you can count on about the types or services offered by objects being passed to a method, etc.. Static typing lets gives you deterministic refactoring, documentation about the types being passed to methods, eliminates the need to write tests for things caught by the type system, etc., but in exchange you have to learn and deal with a more complex type system compared to the dynamic languages and write a bit more code. (A bit more in Scala. In Java, you have to write a lot more code, but that isn't really static typing's fault.)

                                      > > If you want static checking then maybe "generic" type
                                      > > parameters are the right answer. In fact, I think that
                                      > > until there's a larger body of experience with path
                                      > > dependent types, the default solution to most such
                                      > "real"
                                      > > static typing problems should be type parameters.
                                      >
                                      > Yes, sure, i'd tend to use generics also. I guess I was
                                      > just following a path of thought from the Scala book.
                                      > I've also been influenced somewhat by Beta perhaps, which
                                      > h has no generics and instead uses virtual patterns (path
                                      > dependent types, I guess) to express this type of
                                      > flexibility.
                                      >
                                      I also tend to use generics. It may be because I come from Java and that's what I'm used to, but they seem easier for me to think about.

                                      > > And if static checking isn't the right answer and you
                                      > want
                                      > > to dynamically check compatible skiers or currency or
                                      > > whatever, Scala gives powerful tools for that as well.
                                      > > You can use OO style polymorphism or pattern matching,
                                      > or
                                      > > r both.
                                      >
                                      > Sure.
                                      >
                                      > > Finally, I want to point out that Scala has an escape
                                      > > mechanism in its type system. You can cast anything to
                                      > > anything at anytime. That's a big gaping hole as far
                                      > as
                                      > > type systems are concerned, but a pragmatic and useful
                                      > > choice sometimes.
                                      >
                                      > Hmm, that is interesting. And without wishing to be
                                      > flippant, I think the presence of that "hole" would be
                                      > very important in some programs. I expect there will
                                      > always be things that are difficult or impossible to
                                      > express in a practical type system.
                                      >
                                      Well first there's pattern matching, which effectively does safe casts for you. Martin doesn't like calling this a cast though. In Scala jargon, "cast" means unsafe cast from the statically provable perspective. And in Scala you can pretty much cast anything to anything. Even things that could be statically known to be sure to fail at runtime are allowed, if I'm remembering correctly. I think Java will catch these as compiler errors. So casting with asInstanceOf is very much discouraged in Scala, but it is there when you need it.

                                      In ScalaTest I have used it on several occasions. I did a grep to see why. Looks like I primarily use it when I'm using Java reflection:


                                      Macintosh-10:scalatest bv$ fgrep asInstanceOf *.scala
                                      Assertions.scala: case Some(e) => e.asInstanceOf[T] // I know this cast will succeed, becuase iSAssignableFrom succeeded above
                                      Assertions.scala: case Some(e) => e.asInstanceOf[T] // I know this cast will succeed, becuase iSAssignableFrom succeeded above
                                      Assertions.scala: val clazz = manifest.erasure.asInstanceOf[Class[T]]
                                      Assertions.scala: case Some(e) => e.asInstanceOf[T] // I know this cast will succeed, becuase iSAssignableFrom succeeded above
                                      Assertions.scala: "hi".asInstanceOf[T]
                                      PrivateMethodTester.scala: for ((arg, paramType) <- zipped if !paramType.isAssignableFrom(arg.asInstanceOf[AnyRef].getClass)) yield arg
                                      PrivateMethodTester.scala: case anyVal: AnyVal => anyVal.asInstanceOf[AnyRef]
                                      PrivateMethodTester.scala: privateMethodToInvoke.invoke(target, anyRefArgs.toArray: _*).asInstanceOf[T]
                                      SuiteRerunner.scala: val suite = suiteClass.newInstance().asInstanceOf[Suite]
                                      TestRerunner.scala: val suite = suiteClass.newInstance.asInstanceOf[Suite]
                                      • James
                                         
                                        Posts: 11 / Nickname: jiry / Registered: November 15, 2007 2:29 AM
                                        Re: The Origins of Scala
                                        May 6, 2009 1:18 PM      
                                        > runtime are allowed, if I'm remembering correctly. I think
                                        > Java will catch these as compiler errors.

                                        Java just makes you jump through an extra hoop


                                        public class TestJ {
                                        static class Foo {}
                                        static class Bar {}

                                        public static void main(String[] args) {
                                        Object it = new Bar();
                                        Foo foo = (Foo)it;
                                        }
                                        }


                                        Scala would let you just write

                                        val foo = (new Bar).asInstanceOf[Foo]


                                        Of course, either way gets you

                                        java.lang.ClassCastException: Bar cannot be cast to Foo
                                        • Bill
                                           
                                          Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
                                          Re: The Origins of Scala
                                          May 6, 2009 1:50 PM      
                                          > > runtime are allowed, if I'm remembering correctly. I
                                          > think
                                          > > Java will catch these as compiler errors.
                                          >
                                          > Java just makes you jump through an extra hoop
                                          >
                                          >
                                          > public class TestJ {
                                          > static class Foo {}
                                          > static class Bar {}
                                          >
                                          > public static void main(String[] args) {
                                          > Object it = new Bar();
                                          > Foo foo = (Foo)it;
                                          > }
                                          > }
                                          >

                                          >
                                          > Scala would let you just write
                                          >
                                          > val foo = (new Bar).asInstanceOf[Foo]
                                          >

                                          >
                                          > Of course, either way gets you
                                          >
                                          > java.lang.ClassCastException: Bar cannot be cast to Foo
                                          >
                                          That's funny. Even for unsafe code Java is more verbose than Scala.
                                          • James
                                             
                                            Posts: 11 / Nickname: jiry / Registered: November 15, 2007 2:29 AM
                                            Re: The Origins of Scala
                                            May 6, 2009 2:13 PM      
                                            > That's funny. Even for unsafe code Java is more verbose
                                            > than Scala.

                                            Well, you can help Java out a bit if you want.


                                            public class Perversion {
                                            static class Foo {}
                                            static class Bar {}

                                            @SuppressWarnings("unchecked")
                                            public static <T> T cast(Object x) {
                                            return (T)x;
                                            }

                                            public static void main(String[] args) {
                                            Foo f = cast(new Bar());
                                            }
                                            }
                                          • Morgan
                                             
                                            Posts: 37 / Nickname: miata71 / Registered: March 29, 2006 6:09 AM
                                            Re: The Origins of Scala
                                            May 7, 2009 8:15 PM      
                                            > > Scala would let you just write
                                            > >
                                            > > val foo = (new Bar).asInstanceOf[Foo]
                                            > >

                                            > >
                                            > > Of course, either way gets you
                                            > >
                                            > > java.lang.ClassCastException: Bar cannot be cast to Foo
                                            > >
                                            > That's funny. Even for unsafe code Java is more verbose
                                            > than Scala.

                                            How about String s = (String)new Date();

                                            I just tried this in Eclipse, it doesn't give a syntax error, it gives a perfectly reasonable "Cannot cast from Date to String" error.

                                            Once again, a claim that Java is way more verbose than another language is refuted.
                                            • Andrew
                                               
                                              Posts: 18 / Nickname: 55548 / Registered: May 16, 2008 6:33 AM
                                              Re: The Origins of Scala
                                              May 8, 2009 2:41 AM      
                                              > How about String s = (String)new Date();
                                              >
                                              > I just tried this in Eclipse, it doesn't give a syntax
                                              > error, it gives a perfectly reasonable "Cannot cast from
                                              > Date to String" error.

                                              I don't think anyone said it was a syntax error. It is a compiler error though, in the standard Java compiler and the Eclipse one. Eclipse catches it with a little error symbol in the sidebar.

                                              You can of course do this to avoid the error, and push it into runtime:

                                              String s = (String) (Object) new Date();
                                            • Carson
                                               
                                              Posts: 18 / Nickname: cgross / Registered: October 16, 2006 3:21 AM
                                              Re: The Origins of Scala
                                              May 8, 2009 10:07 AM      
                                              > Once again, a claim that Java is way more verbose than
                                              > another language is refuted.

                                              Yeah. The example given is really just a case where the library utilities in Scala are better than the ones available in Java.

                                              Still, you have to admit that closures and type inference clean up statically typed languages a lot.

                                              Cheers,
                                              Carson
        • Stephen
           
          Posts: 1 / Nickname: scolebourn / Registered: February 10, 2007 9:48 AM
          Re: The Origins of Scala
          May 4, 2009 3:39 PM      
          This article and discussion emphasises the key debate over generics and arrays, and which approach works best. I've had a variety of conversations with language designers down the years, and one thing I've noticed is that there is often a strong emphasis on proving the type system. As part of this, there is often a extremely strong dislike of arrays because of their 'unsound' covariant nature.

          However, in the real world, I find that all the developers I talk to have no problem whatsoever with the covariant nature of arrays, and even find it more intuitive than the restrictions imposed when its done 'correctly' (generics). I can only conclude that this is just a difference in how (academic) language designers and developers think about the languages they write/use.

          If you want to see what a JavaNG language looks like where you can freely switch between a List of Numbers and a List of Integers, take a look at Fan - http://fandev.org . Where Scala takes Java's type system and strengthens it, Fan goes the opposite direction, and takes a slightly more dynamic approach. Given the success of fully-dynamic languages over the past few years, it seems to me that Scala's super-static typing isn't the way for the industry to go.
          • Daniel
             
            Posts: 3 / Nickname: 56653 / Registered: July 4, 2008 9:51 PM
            Re: The Origins of Scala
            May 4, 2009 11:25 PM      
            > ...
            > If you want to see what a JavaNG language looks like where
            > you can freely switch between a List of Numbers and a List
            > of Integers, take a look at Fan - http://fandev.org .
            > Where Scala takes Java's type system and strengthens it,
            > Fan goes the opposite direction, and takes a slightly more
            > dynamic approach. Given the success of fully-dynamic
            > languages over the past few years, it seems to me that
            > Scala's super-static typing isn't the way for the industry
            > to go.

            I think this is a common misconception. This is what an average programmer "sees":
            - Java types are restrictive and the syntax is horrible
            - Scala types look scary, especially the generics (with all that +, -, <:, <%...)
            Of course this leads to the "conclusion" to avoid that kind of trouble.

            However:
            - Scala solved the syntax problem so well that still some people think it is a dynamic language
            - Statical correctness can give you a confidence you'll never get with unit tests. This is even more true when using functional style. I found myself thinking "if it compiles, it is correct" in several situations, and so it was.
            - When you're not able to express your ideas using types, your ideas are usually "fuzzy" - which *might* still work, but doesn't scale well
          • Bill
             
            Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
            Re: The Origins of Scala
            May 4, 2009 4:16 PM      
            > This article and discussion emphasises the key debate over
            > generics and arrays, and which approach works best. I've
            > had a variety of conversations with language designers
            > down the years, and one thing I've noticed is that there
            > is often a strong emphasis on proving the type system. As
            > part of this, there is often a extremely strong dislike of
            > arrays because of their 'unsound' covariant nature.
            >
            > However, in the real world, I find that all the developers
            > I talk to have no problem whatsoever with the covariant
            > nature of arrays, and even find it more intuitive than the
            > restrictions imposed when its done 'correctly' (generics).
            > I can only conclude that this is just a difference in how
            > (academic) language designers and developers think about
            > the languages they write/use.
            >
            I suspect the dislike is on array covariance, not arrays per se. Is that correct? Because I'd expect nonvariant arrays in a language with generics wouldn't bother anyone, academic or otherwise.

            > If you want to see what a JavaNG language looks like where
            > you can freely switch between a List of Numbers and a List
            > of Integers, take a look at Fan - http://fandev.org .
            > Where Scala takes Java's type system and strengthens it,
            > Fan goes the opposite direction, and takes a slightly more
            > dynamic approach. Given the success of fully-dynamic
            > languages over the past few years, it seems to me that
            > Scala's super-static typing isn't the way for the industry
            > to go.
            >
            I've been curious about Fan's approach. My understanding is users can't make generic types, but that three built in types (Map, List, and Func) take type parameters. I assume Map takes 2, List 1, and Func a return type plus 0 to many parameter types.

            First, are these type parameters on Map, List, and Func treated as nonvariant, covariant, contravariant? Or some combination? I see no mention of variance in their documentation.

            Also, what do people usually do when they want to make some other collection besides List or Map? Do they cast, or perhaps use the -> to invoke mehods dynamically on objects pulled out of collections?
            • Brian
               
              Posts: 2 / Nickname: brianfrank / Registered: December 16, 2008 1:56 AM
              Re: The Origins of Scala
              May 5, 2009 3:53 AM      
              > I assume Map takes 2, List 1, and Func a return type plus 0 to many parameter types.

              That is correct - each one uses a custom syntax. For example Fan's List parametrization looks like the Java array syntax. Because generics are used with only built-in types, it allows Fan to easily maintain the parametrization at runtime for reflection. For example given a list of Str, the type signature would be Str[], and you could reflect this type from the list at runtime. That becomes important for meta-programming and serialization.

              > First, are these type parameters on Map, List, and Func treated as nonvariant, covariant, contravariant? Or some combination? I see no mention of variance in their documentation.

              Technically they are covariant similar to Java arrays (although from the setter method perspective that allows contravariance). But it is sort of moot issue in Fan. Fan is mostly statically typed, however the compiler only flags type errors which are guaranteed to be never be correct - for example passing a Str where an Int is expected. However any type check which could pass at runtime will generate a synthetic cast to keep the JVM/CLR happy. For example passing a Num where an Int is expected doesn't require an explicit cast.

              It seems sort of crazy, but it turns out to be a great compromise. Almost all type errors still end up getting caught at compile time, we get Java-like performance, yet it avoids all the nasty syntax that clutters up Java code. So Fan's philosophy has been to use the type system as a useful tool whenever it makes sense, but break out of it whenever it gets into the way.

              > Also, what do people usually do when they want to make some other collection besides List or Map? Do they cast, or perhaps use the -> to invoke mehods dynamically on objects pulled out of collections?

              I actually have seen it done very much - most all the code I've seen just uses the built-in types. But you would basically design it to work with Obj just like the pre-generics Java collections. The difference is you don't actually do the casting (it gets done for you). Although I use -> for duck typing things myself too.
              • Bill
                 
                Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
                Re: The Origins of Scala
                May 5, 2009 10:18 AM      
                > > I assume Map takes 2, List 1, and Func a return type
                > plus 0 to many parameter types.
                >
                > That is correct - each one uses a custom syntax. For
                > example Fan's List parametrization looks like the Java
                > array syntax. Because generics are used with only
                > built-in types, it allows Fan to easily maintain the
                > parametrization at runtime for reflection. For example
                > given a list of Str, the type signature would be Str[],
                > and you could reflect this type from the list at runtime.
                > That becomes important for meta-programming and
                > d serialization.
                >
                Is List mutable or immutable? Sorry I can't seem to find something like a FanDoc on the website. Is there a URL of something like the Java API JavaDoc? The reason I ask is that this looks like the syntax for Java's arrays. So I'm curious if it is implemented using Arrays underneath, and if not, how do I use Java arrays in Fan? (Like if I have to pass an array to a pre-existing Java method.)

                > > First, are these type parameters on Map, List, and Func
                > treated as nonvariant, covariant, contravariant? Or some
                > combination? I see no mention of variance in their
                > documentation.
                >
                > Technically they are covariant similar to Java arrays
                > (although from the setter method perspective that allows
                > contravariance). But it is sort of moot issue in Fan.
                > Fan is mostly statically typed, however the compiler only
                > y flags type errors which are guaranteed to be never be
                > correct - for example passing a Str where an Int is
                > expected. However any type check which could pass at
                > runtime will generate a synthetic cast to keep the JVM/CLR
                > happy. For example passing a Num where an Int is expected
                > doesn't require an explicit cast.
                >
                Can you give an example of a "type check which could pass at runtime that generates a synthetic cast?

                > It seems sort of crazy, but it turns out to be a great
                > compromise. Almost all type errors still end up getting
                > caught at compile time, we get Java-like performance, yet
                > it avoids all the nasty syntax that clutters up Java code.
                > So Fan's philosophy has been to use the type system as a
                > a useful tool whenever it makes sense, but break out of it
                > whenever it gets into the way.
                >
                > > Also, what do people usually do when they want to make
                > some other collection besides List or Map? Do they cast,
                > or perhaps use the -> to invoke mehods dynamically on
                > objects pulled out of collections?
                >
                > I actually have seen it done very much - most all the code
                > I've seen just uses the built-in types. But you would
                > basically design it to work with Obj just like the
                > pre-generics Java collections. The difference is you
                > don't actually do the casting (it gets done for you).
                > Although I use -> for duck typing things myself too.
                >
                Ah, you do a kind of cast inference on the Obj type. I see that now in the docs. That's interesting.
                • Brian
                   
                  Posts: 2 / Nickname: brianfrank / Registered: December 16, 2008 1:56 AM
                  Re: The Origins of Scala
                  May 5, 2009 10:43 AM      
                  > Is List mutable or immutable? Sorry I can't seem to find something like a FanDoc on the website.

                  Lists can be mutable, readonly (shallow copy-on-write), or immutable (guaranteed to store immutable items). Fan has a first-class notion of immutability using the const modifier, and doesn't allow threads (actors) to share mutable state.

                  The "ro" and "toImmutable" methods are used on List/Map to convert. You can see the Fandoc for List at http://fandev.org/doc/sys/List.html.

                  > So I'm curious if it is implemented using Arrays underneath, and if not, how do I use Java arrays in Fan?

                  List is a class backed by an array (like ArrayList in Java). True Java arrays are accessed with the Java FFI: http://fandev.org/doc/docLang/JavaFFI.html#arrays.

                  > Can you give an example of a "type check which could pass at runtime that generates a synthetic cast?

                  Take a look at: http://fandev.org/doc/docLang/TypeSystem.html#implicitCasts.
          • James
             
            Posts: 11 / Nickname: jiry / Registered: November 15, 2007 2:29 AM
            Re: The Origins of Scala
            May 5, 2009 7:02 AM      
            > However, in the real world, I find that all the developers
            > I talk to have no problem whatsoever with the covariant
            > nature of arrays, and even find it more intuitive than the
            > restrictions imposed when its done 'correctly' (generics).
            > I can only conclude that this is just a difference in how
            > (academic) language designers and developers think about
            > the languages they write/use.


            Then allow me to be your one counter data point. I have no graduate degrees. My undergraduate schooling was at a state college. I have 17 years of programming experience varying in "enterprise" systems, web startups, shrink wrap commercial software, and open source projects. The vast majority of my code has been in languages like Java, C++, SQL, and Javascript. I'm a "real world" programmer, not an academic.

            I find Java's covariant arrays irritating and problematic. They interact poorly with the rest of Java. If used extensively for non-primitives I do get ArrayStoreExceptions. If I could wave a magic wand and fix this one aspect of Java I would. In practice I just mostly avoid using arrays.

            Making everything covariant is deeply unintuitive and confusing to me. It violates Liskov substitution. It violates the old protocol guideline "be strict in what you send, be lenient in what you accept." It's counter to the way method calls work.

            So, please, for the love of all that is good, please stop trying to make things into academic vs real world debates. It's disgusting. You don't see electrical engineers running around saying "we don't use Ohm's law here, it's too academic."
            • James
               
              Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
              Re: The Origins of Scala
              May 5, 2009 8:49 AM      
              > So, please, for the love of all that is good, please stop
              > trying to make things into academic vs real world debates.
              > It's disgusting. You don't see electrical engineers
              > s running around saying "we don't use Ohm's law here, it's
              > too academic."

              There's a big difference between a physical law that has been confirmed innumerable times and the recommendations of computer scientists to programmers.

              "It's disgusting"? That's a bit over the top, don't you think?

              My father is a mechanical engineer with a masters in thermodynamics and has been working with Phd engineers on a special project recently. He tells me many stories of the problems the Phds cause by not understanding how theory and practicality can differ.
              • Martin
                 
                Posts: 15 / Nickname: modersky / Registered: September 14, 2003 9:46 PM
                Re: The Origins of Scala
                May 5, 2009 10:24 AM      
                > There's a big difference between a physical law that has
                > been confirmed innumerable times and the recommendations
                > of computer scientists to programmers.
                >
                You mean, logic is a recommendation by computer scientists to programmers?

                > "It's disgusting"? That's a bit over the top, don't you
                > think?
                >
                Frankly I have to side with James there. These discussions of academic vs practical are silly and annoying (and particularly funny in the context of Scala, since people seem to just disregard who's behind the language and what their practical credentials are).

                We are doing ourselves a disservice as a field with these allusions. Engineering is defined as the application of science to a technical problem. If we feel it's cool to disregard or doubt science in computing we put ourselves on the same level as simple technicians, and we should not complain if we will in the future be paid on this level.
                • James
                   
                  Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
                  Re: The Origins of Scala
                  May 5, 2009 10:48 AM      
                  > > There's a big difference between a physical law that
                  > has
                  > > been confirmed innumerable times and the
                  > recommendations
                  > > of computer scientists to programmers.
                  > >
                  > You mean, logic is a recommendation by computer scientists
                  > to programmers?

                  No. I didn't say anything like that.

                  > > "It's disgusting"? That's a bit over the top, don't
                  > you
                  > > think?
                  > >
                  > Frankly I have to side with James there. These discussions
                  > of academic vs practical are silly and annoying (and
                  > particularly funny in the context of Scala, since people
                  > seem to just disregard who's behind the language and what
                  > their practical credentials are).

                  Since when are 'silly' and 'annoying' synonyms for 'disgusting'?

                  If you are saying that people must believe whatever those with the proper credentials say, then I reject that outright. The lack of a piece of paper is not the lack of intelligence, ability or knowledge.

                  > We are doing ourselves a disservice as a field with these
                  > allusions. Engineering is defined as the application of
                  > science to a technical problem. If we feel it's cool to
                  > disregard or doubt science in computing we put ourselves
                  > on the same level as simple technicians, and we should not
                  > complain if we will in the future be paid on this level.

                  Language design is not a science. I studied Physics before I studied computer science and computer science is more like mathematics than it is like a hard science. Science requires falsifiable hypothesis. Logic is not science. If you feel you can propose such a hypothesis I'm open to it.
                  • Martin
                     
                    Posts: 15 / Nickname: modersky / Registered: September 14, 2003 9:46 PM
                    Re: The Origins of Scala
                    May 5, 2009 1:04 PM      
                    > > > There's a big difference between a physical law that
                    > > has
                    > > > been confirmed innumerable times and the
                    > > recommendations
                    > > > of computer scientists to programmers.
                    > > >
                    > > You mean, logic is a recommendation by computer
                    > scientists
                    > > to programmers?
                    >
                    > No. I didn't say anything like that.
                    >
                    > > > "It's disgusting"? That's a bit over the top, don't
                    > > you
                    > > > think?
                    > > >
                    > > Frankly I have to side with James there. These
                    > discussions
                    > > of academic vs practical are silly and annoying (and
                    > > particularly funny in the context of Scala, since
                    > people
                    > > seem to just disregard who's behind the language and
                    > what
                    > > their practical credentials are).
                    >
                    > Since when are 'silly' and 'annoying' synonyms for
                    > 'disgusting'?
                    >
                    > If you are saying that people must believe whatever those
                    > with the proper credentials say, then I reject that
                    > outright. The lack of a piece of paper is not the lack of
                    > intelligence, ability or knowledge.
                    >
                    > > We are doing ourselves a disservice as a field with
                    > these
                    > > allusions. Engineering is defined as the application of
                    > > science to a technical problem. If we feel it's cool to
                    > > disregard or doubt science in computing we put
                    > ourselves
                    > > on the same level as simple technicians, and we should
                    > not
                    > > complain if we will in the future be paid on this
                    > level.
                    >
                    > Language design is not a science. I studied Physics
                    > before I studied computer science and computer science is
                    > more like mathematics than it is like a hard science.
                    > Science requires falsifiable hypothesis. Logic is not
                    > t science. If you feel you can propose such a hypothesis
                    > I'm open to it.

                    Well, for me logic and mathematics is a science, it's just not a natural science. And it's the only candidate discipline to underly our field of software engineering (maybe some people see that field rooted in social sciences, but I would have my doubts). As to falsifiable hypothesis: I would say, ``immutable datastructures can always be covariant'' (that's what you stated) is such a falsifiable hypothesis. That's not a matter of belief.

                    Belief is: does it matter? Should I care in my work? But that's just the same as for Ohm's law. There are gifted electricians who do not care for Ohm's law; they just cobble their circuits together guided by experience and intuition. They might do great work. But they are not engineers. So for that reason I thought James' analogy was right on spot.
                    • James
                       
                      Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
                      Re: The Origins of Scala
                      May 5, 2009 8:11 PM      
                      > Well, for me logic and mathematics is a science, it's just
                      > not a natural science.

                      Don't get me wrong. Logic is a pre-requisite of science and it's no less important than science. In fact, it's given that science could not exist without it. It's hugely important and I depend on it in every aspect of my life (so much so that it annoys my wife daily.)

                      But I think we need to realize that science is not the same as logic and mathematics. Any logical statement can be proven or disproven given the right axioms but the laws of gravity or electricity don't depend on what axioms we choose.

                      I don't point this out to demean any subject. And Mr. Odersky, (may I call you Martin?) I have great respect for what you have produced in, especially with regard to Scala. You clearly are a master in your field. No rational person could ever question that.

                      I'm a little intimidated but honored to even argue with you and I think your willingness to converse with me is a sign of your character.

                      I guess all I am asking is that you consider is that programmers deal with fiction (I'm falling back on physics metaphors here.) In physics, we usually ignored friction. But engineers can't. Having been on the both sides of this, I see why engineers thing physicists have their heads in the clouds. But none would for a second doubt the value of physics. It's just a different focus and different skills.

                      > And it's the only candidate
                      > discipline to underly our field of software engineering
                      > (maybe some people see that field rooted in social
                      > sciences, but I would have my doubts). As to falsifiable
                      > hypothesis: I would say, ``immutable datastructures can
                      > always be covariant'' (that's what you stated) is such a
                      > falsifiable hypothesis. That's not a matter of belief.

                      The point I was attempting to make was that it's not a law of computing that types should allow for covariance and contravariance. There are lot's of people that think that typing in general is nonsense. These are all valid opinions. Does your opinion carry more weight than the average programmer (myself included)? Absolutely. But please don't pretend that your opinion is a fact.

                      > Belief is: does it matter? Should I care in my work? But
                      > that's just the same as for Ohm's law. There are gifted
                      > electricians who do not care for Ohm's law; they just
                      > cobble their circuits together guided by experience and
                      > intuition. They might do great work. But they are not
                      > engineers. So for that reason I thought James' analogy was
                      > right on spot.

                      I can't imagine that there are many electricians that ignore Ohms law. That would be like designing an aqueduct and ignoring that water runs downhill. There isn't a rational argument for saying Ohms law is not true, at least not on a macro scale. You could hypothesize a universe where it is not true but that won't affect our reality. Reality has little concern for our thoughts.

                      I'd love to discuss this kind of stuff at length with you but I feel this is not the forum for that. If you want I can send you a direct email. I am still subscribed to the Scala email list. I'll understand, however if you are too busy to be bothered. I have been doing work with neural networks and think Scala could be a great fit for that work.
                      • Martin
                         
                        Posts: 15 / Nickname: modersky / Registered: September 14, 2003 9:46 PM
                        Re: The Origins of Scala
                        May 6, 2009 2:41 AM      
                        @James Watson

                        Sure, please call me Martin!


                        > The point I was attempting to make was that it's not a law
                        > of computing that types should allow for covariance and
                        > contravariance. There are lot's of people that think that
                        > typing in general is nonsense. These are all valid
                        > opinions. Does your opinion carry more weight than the
                        > average programmer (myself included)? Absolutely. But
                        > please don't pretend that your opinion is a fact.
                        >
                        That's a good point,and I fully agree. Whether a language should have dynamic typing, static typing, or gradual typing as Fan seems to do, is a matter of taste, and nobody's opinion counts more than any other's.

                        I am a bit less happy with type systems that are unsound.
                        That's like saying: we know our theory inconsistent, but we do not care because empirically it works well. I think in the end better results are obtained by not being too complacent with such inconsistencies. But in the end that is also a matter of taste, as long as the limitations of a theory/type system are explained clearly.

                        What I was really objecting to is that (very common) sentiment that ``academic'' means ``impractical''. An engineering dicipline deserves that name only if it takes its underlying foundations (whether these are science or mathematics) seriously. I see that culture is largely lacking in CS.

                        Back to variance: I believe my examples have shown that
                        contravariance is essential as soon as we have higher-order functions (because arguments to such functions are naturally contravariant). I may well be that most Eiffel code isfirst order; then the problem does not arise so much. But once you become more functional you can't maintain the illusion that everything is covariant.
                        • James
                           
                          Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
                          Re: The Origins of Scala
                          May 6, 2009 5:13 AM      
                          > What I was really objecting to is that (very common)
                          > sentiment that ``academic'' means ``impractical''. An
                          > engineering dicipline deserves that name only if it takes
                          > its underlying foundations (whether these are science or
                          > mathematics) seriously. I see that culture is largely
                          > lacking in CS.

                          I don't believe software engineering really exists as a real discipline the way mechanical engineering exists as a discipline. I think it's something that will need to be addressed eventually. I once watched a series of television programs about Isambard Kingdom Brunel and I was struck by how at the time, there were no official mechanical engineering degrees or certifications. It reminded me of how software engineering is done today. Engineers at the time used wildly different approaches and argued about them much the way we do today about things like typing systems.

                          But an engineering discipline requires empirical research. Software engineering would be based on CS but would have different goals much the way Physics and most Engineering disciplines are related. Before that happens, people in general need to care more about software quality. Right now, relatively few organizations are willing to pay for this kind of discipline.

                          > Back to variance: I believe my examples have shown that
                          > contravariance is essential as soon as we have
                          > higher-order functions (because arguments to such
                          > functions are naturally contravariant). I may well be that
                          > most Eiffel code isfirst order; then the problem does not
                          > arise so much. But once you become more functional you
                          > can't maintain the illusion that everything is covariant.

                          Perhaps as I understand functional programming more, I will see this too. What I struggle with now is that most developers and managers are not interested in understanding sophisticated language features and things like contravariance can cause havoc if you don't understand what you are doing.
                        • James
                           
                          Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
                          Re: The Origins of Scala
                          May 6, 2009 5:31 AM      
                          > What I was really objecting to is that (very common)
                          > sentiment that ``academic'' means ``impractical''.

                          I'll just say one more thing about this. It's common for engineers to have this kind of attitude about physicists and it's not that engineers don't take physics seriously. The perception of the engineers often that the physicist act as if engineering is easy. Having been in the physics world (briefly) I saw that there is something to this. The physicists did believe they could do what engineers do but better and if the engineers would just do things the way physicists said they should do it, things would be a lot better.

                          The reality is that most physicist could not be engineers without a lot of training and experience. I guess I would just ask that you consider that when developers say something isn't working for them, they might have valid reasons to think that. I've found in general that most people's complaints about my software are valid and I always gain a lot by taking the time to learn why.
                          • Bill
                             
                            Posts: 28 / Nickname: billpyne / Registered: January 29, 2007 4:12 AM
                            Re: The Origins of Scala
                            May 6, 2009 6:58 AM      
                            > > What I was really objecting to is that (very common)
                            > > sentiment that ``academic'' means ``impractical''.
                            >
                            > I'll just say one more thing about this. It's common for
                            > engineers to have this kind of attitude about physicists
                            > and it's not that engineers don't take physics seriously.
                            > The perception of the engineers often that the physicist
                            > t act as if engineering is easy. Having been in the
                            > physics world (briefly) I saw that there is something to
                            > this. The physicists did believe they could do what
                            > engineers do but better and if the engineers would just do
                            > things the way physicists said they should do it, things
                            > would be a lot better.
                            >
                            > The reality is that most physicist could not be engineers
                            > without a lot of training and experience. I guess I would
                            > just ask that you consider that when developers say
                            > something isn't working for them, they might have valid
                            > reasons to think that. I've found in general that most
                            > people's complaints about my software are valid and I
                            > always gain a lot by taking the time to learn why.

                            Not to hijack this topic but the practical vs. theoretical debate comes up so often it seems like there should be a separate topic just for that. At some point, field developers and academic researchers really need to communicate more. The "that's academic" reaction to less obvious ideas, while I wouldn't term as disgusting, it's certainly becoming a tired cliché. Likewise I'd like to disabuse anyone of the notion that field developers don't, as a whole, care about the quality of our product.
                            • James
                               
                              Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
                              Re: The Origins of Scala
                              May 6, 2009 7:22 AM      
                              > Not to hijack this topic but the practical vs. theoretical
                              > debate comes up so often it seems like there should be a
                              > separate topic just for that. At some point, field
                              > developers and academic researchers really need to
                              > communicate more.

                              I agree but it needs to be a two-way discussion.

                              > The "that's academic" reaction to less obvious ideas,
                              > while I wouldn't term as disgusting, it's certainly
                              > becoming a tired cliché.

                              Does it really matter what words people use to express their frustration? There's a real issue and dismissing it as 'disgusting', 'cliche', or 'silly' is not going to resolve anything.

                              > Likewise I'd like to
                              > disabuse anyone of the notion that field developers don't,
                              > as a whole, care about the quality of our product.

                              If you are referring to what I wrote above, I don't mean to say that developers don't care about quality but that consumers of software, by and large, are not concerned with quality and/or are unaware of it. MS Office software is mediocre at best while being some of the most popular software on the planet. In IT, I'm constantly confronted with an attitude that cheap software is preferred over quality software even if the cheap software doesn't solve the right problems. A lot of this has to do with the "Tragedy of the Commons" but there's a distinct attitude that good software is not worth the cost, especially when you are talking about the guts of the software.

                              There's nothing preventing software engineer certification right now. But any state or nation that required software engineers would immediately see development job disappear. Qualified people cost more and most software producing organizations won't receive any benefit from rigorous licensing.
                              • Bill
                                 
                                Posts: 28 / Nickname: billpyne / Registered: January 29, 2007 4:12 AM
                                Re: The Origins of Scala
                                May 6, 2009 8:09 AM      
                                What I wrote wasn't pointed at anyone. Mostly I go between Artima, HackerNews, and Reddit with some LtU mixed in. At least weekly I see a similar debate go on. Daily I see someone shrug off discussion on language features as "too academic". Often enough I have read the complaint about field developers not taking software quality seriously enough as if there are no external factors influencing our (at times seemingly poor) choices.

                                We need a dedicated forum for discussing problems we see in the field and what researchers have to offer.

                                Poor quality isn't isolated to software. Fast food meets the goal of fast but is often enough not nutritious nor is it very tasty. Education - not so encouraging. From secondary to higher ed. I hear complaints from teachers about having to pass people through the system; about having to cover topics that would have been learned grades earlier a few decades ago. My point is that we have a culture of "lowest cost is good enough". Doing something well costs more upfront.

                                Again, I apologize for straying far off an interesting topic. James, you have my email address. If you'd like to discuss this further, please email me.
                                • James
                                   
                                  Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
                                  Re: The Origins of Scala
                                  May 6, 2009 8:15 AM      
                                  > What I wrote wasn't pointed at anyone.

                                  OK. I think we are in general agreement.
                        • Carson
                           
                          Posts: 18 / Nickname: cgross / Registered: October 16, 2006 3:21 AM
                          Re: The Origins of Scala
                          May 6, 2009 7:50 AM      
                          > But once you become more functional you
                          > can't maintain the illusion that everything is covariant.

                          That's why GScript has a separate class of types for function types, where parameters are contravariant and return types are covariant. It leaves normal generic classes (99% of the time Collections) to be covariant, which is good enough for development work.

                          And we shouldn't assume that a covariant regime always means less type safety. In java generics (and, so far as I can tell, in scala) the following code is perfectly legal:


                          List<String> myListOfStrings = Arrays.asList( "foo", "bar" );

                          myListOfStrings.contains( Boolean.FALSE );


                          In GScript, that's a compilation error, because we've retyped contains() parameter to be T, rather than Object (or Any.) We can get away with that because of the covariant nature of generics.

                          Now, because we've adopted covariance of generic types you simply can't assume that a type variable type appearing at as a function parameter is correctly typed at runtime, and you must check that it is so if it matters to you. OTOH, for methods like Collection#contains() you don't really care, and you get additional call site type safety that neither java nor scala (again, as far as I can tell) provide.

                          I'm sure that, with sufficiently more complexity, this could be addressed in a more sound manner, butyou have to ask yourself: is the complexity worth it? Given how productive people are in dynamic langauges with no static typing whatsoever, I think the answer is probably not.

                          Worse is better?

                          Cheers,
                          Carson
                          • James
                             
                            Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
                            Re: The Origins of Scala
                            May 6, 2009 8:13 AM      
                            > In java generics
                            > (and, so far as I can tell, in scala) the following code
                            > is perfectly legal:
                            >
                            >
                            > List<String> myListOfStrings = Arrays.asList( "foo",
                            > ", "bar" );
                            >
                            > myListOfStrings.contains( Boolean.FALSE );
                            >

                            >
                            > In GScript, that's a compilation error, because we've
                            > retyped contains() parameter to be T, rather than Object
                            > (or Any.) We can get away with that because of the
                            > covariant nature of generics.

                            I think an illustrative example is that during the beta period of java generics, there was a version of the 1.5 JDK where Map.get(Object) was defined as Map.get(K). It was then changed back to get(Object) generating a lot of questions. After 1.5 was released, it was common for people to say that Map.get() should be generic.

                            That's wrong and I don't think I need to explain why. As I understand it, in GScript you could cast that list to List<Object> and do the contains(Boolean) would then compile. That makes sense to me. There are times when it's reasonable to ask if an Object is in a list of Strings there are even times where it makes sense to ask if a banana is in a list of oranges but in the most common cases, you don't want that.
                            • Carson
                               
                              Posts: 18 / Nickname: cgross / Registered: October 16, 2006 3:21 AM
                              Re: The Origins of Scala
                              May 6, 2009 9:40 AM      
                              > That's wrong and I don't think I need to explain why. As
                              > I understand it, in GScript you could cast that list to
                              > List<Object> and do the contains(Boolean) would then
                              > compile. That makes sense to me. There are times when
                              > it's reasonable to ask if an Object is in a list of
                              > Strings there are even times where it makes sense to ask
                              > if a banana is in a list of oranges but in the most common
                              > cases, you don't want that.

                              In java you have to declare contains() to take Object, or you couldn't call contains() on lists with wildcards in their parameterization. E.g.:


                              List<? extends CharSequence> listOfChars = ...
                              listOfChars.contains( "foo" );


                              That would horribly hamstring wildcard collections.

                              An interesting note, just to show how hard generics are for average joe programmers (like myself) to understand: I don't think that even the authors of the O'Reilly book "Java Generics and Collections" totally understood this. On page 26 they argue that the designers of java generics made a mistake by making these methods take Object, without ever going into the ugly ramification wrt wildcards if they had.

                              Worse is better?

                              Cheers,
                              Carson
                              • James
                                 
                                Posts: 128 / Nickname: watson / Registered: September 7, 2005 3:37 AM
                                Re: The Origins of Scala
                                May 6, 2009 11:20 AM      
                                > In java you have to declare contains() to take
                                > Object, or you couldn't call contains() on lists with
                                > wildcards in their parameterization. E.g.:
                                >
                                > An interesting note, just to show how hard generics are
                                > for average joe programmers (like myself) to understand: I
                                > don't think that even the authors of the O'Reilly book
                                > "Java Generics and Collections" totally understood this.
                                > On page 26 they argue that the designers of java generics
                                > s made a mistake by making these methods take Object,
                                > without ever going into the ugly ramification wrt
                                > wildcards if they had.

                                Right. This is the point I am trying to make. A lot of people say that Java generics isn't that hard but there are all these kinds of issues where you can paint yourself into a corner and not realize it.
                  • Carson
                     
                    Posts: 18 / Nickname: cgross / Registered: October 16, 2006 3:21 AM
                    Re: The Origins of Scala
                    May 5, 2009 11:04 AM      
                    As a side note, is anyone else shocked that Java doesn't have ImmutableCollection, ImmutableList, ImmutableSet and ImmutableMap interfaces? They could be easily added with no backwards compatability issues and would make loads of API's much, much more obvious.

                    Cheers,
                    Carson
                    • Gregor
                       
                      Posts: 6 / Nickname: gregor / Registered: August 3, 2005 4:18 AM
                      Re: The Origins of Scala
                      May 21, 2009 4:21 AM      
                      > As a side note, is anyone else shocked that Java doesn't
                      > have ImmutableCollection, ImmutableList, ImmutableSet and
                      > ImmutableMap interfaces? They could be easily added with
                      > no backwards compatability issues and would make loads of
                      > API's much, much more obvious.
                      Yes, I just realized that when I started using google collections. They are not the same as the immutable list of fan, because they can contain mutable objects, though.