> That's wrong and I don't think I need to explain why. As
> I understand it, in GScript you could cast that list to
> List<Object> and do the contains(Boolean) would then
> compile. That makes sense to me. There are times when
> it's reasonable to ask if an Object is in a list of
> Strings there are even times where it makes sense to ask
> if a banana is in a list of oranges but in the most common
> cases, you don't want that.

In java you have to declare contains() to take Object, or you couldn't call contains() on lists with wildcards in their parameterization. E.g.:


List<? extends CharSequence> listOfChars = ...
listOfChars.contains( "foo" );


That would horribly hamstring wildcard collections.

An interesting note, just to show how hard generics are for average joe programmers (like myself) to understand: I don't think that even the authors of the O'Reilly book "Java Generics and Collections" totally understood this. On page 26 they argue that the designers of java generics made a mistake by making these methods take Object, without ever going into the ugly ramification wrt wildcards if they had.

Worse is better?

Cheers,
Carson

Hi Andrew,

> > def test(s1 : Skier)(s2 : s1.SuitableRoommate) = s1
> share s2
> >
> > If you don't write it in curried form Scala complains
> > about an illegal dependent method type. I'm not sure
> why.
> > I tested this using the 2.7.4 compiler.
>
> thanks james. yes, i got the illegal dependent type error
> also when I used the non-curried form also. In fact, I
> didn't know that it was possible to use a curried form
> ;-)
>
OK I got an answer on this one. Dependent methods are currently an experimental feature of the Scala compiler. So to see them in action you need to add a -Xexperimental to the command line when you run a script or compile a .scala file. If you do that, it works as James showed. You do need to use the curried form. I also don't know why. But it basically lets you do what you want. Given these types:

abstract class Skier {
  type SuitableRoommate <: Skier
  def share(other: SuitableRoommate)
}

class Boy extends Skier {
  type SuitableRoommate = Boy
  override def share(other: Boy) {}
}

class Girl extends Skier {
  type SuitableRoommate = Girl
  override def share(other: Girl) {}
}

class Dog extends Skier {
  type SuitableRoommate = Cat
  override def share(other: Cat) {}
}

class Cat extends Skier {
  type SuitableRoommate = Dog
  override def share(other: Dog) {}
}

You can write a method like this:

def placeInRoom(skier: Skier)(roommate: skier.SuitableRoommate) { skier share roommate }

And call it like this:

placeInRoom(new Dog)(new Cat)
placeInRoom(new Cat)(new Dog)
placeInRoom(new Girl)(new Girl)
placeInRoom(new Boy)(new Boy)

These all work, because the roommate passed is suitable for the skier. If you try and pass in an unsuitable roommate, you'll get a compiler error:

placeInRoom(new Dog)(new Dog)

(fragment of skier.scala):33: error: type mismatch;
 found   : this.Dog
 required: this.Dog#SuitableRoommate
placeInRoom(new Dog)(new Dog)
                      ^
one error found

Now by the way, this still doesn't violate the rule that the Scala compiler will need a specific reference (i.e., more specific than Skier) to call share on, but it moves the requirement to the call site. You can only call placeInRoom if the type of the variable or expression you specify for the first argument is more specific than Skier. Here's an example where it isn't. Even though I'm holding onto a Dog, the type of my variable is Skier, and when I try to pass a Dog in that way, I get a compiler error:

val skierDog: Skier = new Dog
placeInRoom(skierDog)(new Dog)

(fragment of skier.scala):35: error: type mismatch;
 found   : this.Dog
 required: this.Skier#SuitableRoommate
placeInRoom(skierDog)(new Dog)
                       ^
one error found

Anyway, the -Xexperimental stuff is where EPFL puts things that aren't quite ready or proven. If this does mature and work out, then they would likely put it in the regular Scala. But it is possible that it doesn't work out, in which case it will never get into regular Scala. What you can do in that case, and in the mean time, is add the methods you want to add to the specific classes themselves. Inside those classes, the SuitableRoommate is defined and usable. But that gives you the code duplication I think you are mainly trying to avoid. In some cases you can avoid it with the view bounds trick demonstrated by James Iry's example.

Hi Fred,

> Heh. It is interesting that a lot of the longer
> discussions in artima become a debate over the
> complications of generic variance. That alone indicates
> something is wrong. I've said this many times here: a
> feature of a type system is useful only if it *helps*
> **us** read and write code. If it's confusing us, it's not
> helping, thus it's not useful.
>
I agree with the general thrust here, but might say it differently. I think you need to strive to make features as simple as possible. But sometimes a feature, though it be hard to learn, can be still worth it. In other words, even if the cost is great, it can still be worthwhile so long as the benefit is greater.

> A lot of smart people have a difficult time grasping
> variance with respect to type parameters esp. with call
> site variance e.g., wildcards in Java. Definition site
> variance as discussed here (too) many times is also a hard
> pill to swallow. The difficulties arising from generics in
> both Java and Scala, in my opinion, indicate a failed
> experiment -- so far generics in the aggregate is not
> helping us.
>
I've met a lot of people who have expressed the same sentiment about Java's wildcards, and I myself have that feeling. Although the basic concept of a wildcard I pretty much grok, I find the subtleties of wildcards quite confusing and challenging myself. By contrast, I find Scala's declaration site variance quite easy to grok. The most challenging part for me was contravariance, but with some study I got it.

You made a blanket statement that smart people find definition site variance a hard pill to swallow. I've know many dynamic typing leaning folks who don't want to swallow the pill, but I must say so far I've never heard someone who actually programs in Scala complain that Scala's variance is too hard to understand. They all have complaints, as do I, but the complexity of the type system just doesn't pop up much as a complaint.

That said, I think it is too early to tell. The people coming to Scala so far are early adopters and may not represent the mainstream well. They may also be people who are comfortable with type systems, etc. So what I'm curious about is to what extent this will become a complaint as more people try Scala. It also tells me that maybe I should publish some things that try and explain definition site variance in easy to understand terms.

> I'm not suggesting we should abandon generics. We just
> need to be pragmatic about how we integrate generics into
> a language so that it's useful to most people who'll use
> the language. Since most people seem to have no problem
> understanding variance with respect to array component
> types, I'm inclined to believe that is the best option we
> have (so far). Thus the GScript and Eiffel approach seem
> like the way to go.
>
Eiffel, we've been guessing, didn't really let you pass functions around. So contravariance wasn't an issue there as much. But most of today's emerging languages have first class functions. That's one of the trends.

If I understand Carson correctly, GScript does have contravariance of function parameters. So GScript has contravariance, the hard to understand one. It just only has it in one place, like Fan's only letting you put type parameters on Map, List, or Func, or Java only letting you use + on String. Scala just makes contravariance a general feature. You can make your own types contravariant if you want. The place where I've seen contravariance used in Scala is primarily function parameters. The place where I used it was in Matcher, which is also a function, and contravariance is for the function parameter. It was nice that I could do that.

What GScript does then, if I understand it, is everything else is covariant, a la Eiffel. In Scala, other things will be either nonvariant or covariant. The Scala compiler will only let you make something covariant if it is type sound. It won't let you make a mistake. Also, if you're not sure, just start out nonvariant. Later you can make it covariant without breaking any client code. There are no wildcards to think about, and with those out of the way, I really don't think the conceptual complexity of grasping this stuff is much greater in Scala than GScript.

But as I said before, time will tell. I'm not sure Scala's early adopters are a good measure of how palatable the masses would find Scala's approach to variance.

> Anyway, the -Xexperimental stuff is where EPFL puts things
> that aren't quite ready or proven.

many thanks for that, Bill. that's exactly what I was looking for.

>In some cases you can avoid it with the view bounds trick
> demonstrated by James Iry's example.

i'm still struggling with the Account example I posted earlier. The Skiing analog would be to make a class (say called Booker) which has a member variable which is a listbuffer of Skiiers, with a method called bookRooms() which pairs up the skiiers in the list and shares() them. Obviously the assumption is that only compatible (i.e. the same type of) skiiers can be added to the list in the 1st place. I also want to avoid creating BoyBooker and GirlBooker classes, and pushing the type parameter higher.

I'll try James' clever approach next.

Which brings me to something bugging me a bit: the scala type system is clearly very advanced, and i expect that someone has proven it sound. What i'm trying to work out is to what extent are all reasonably practical cases covered (I realise this is very vague). Or does having a more powerful type system actually prevent you from doing things which could be done via the casting and other covariant holes in a less powerful (e.g. Java) type system? At this point, I'm sort of getting an insight into why dynamic language proponents claim typing restricts them even though I've not been previously sympathetic in any way...

Cheers,
Andrew

> In java you have to declare contains() to take
> Object, or you couldn't call contains() on lists with
> wildcards in their parameterization. E.g.:
>
> An interesting note, just to show how hard generics are
> for average joe programmers (like myself) to understand: I
> don't think that even the authors of the O'Reilly book
> "Java Generics and Collections" totally understood this.
> On page 26 they argue that the designers of java generics
> s made a mistake by making these methods take Object,
> without ever going into the ugly ramification wrt
> wildcards if they had.

Right. This is the point I am trying to make. A lot of people say that Java generics isn't that hard but there are all these kinds of issues where you can paint yourself into a corner and not realize it.

Hi Andrew,

> Which brings me to something bugging me a bit: the scala
> type system is clearly very advanced, and i expect that
> someone has proven it sound. What i'm trying to work out
> is to what extent are all reasonably practical cases
> covered (I realise this is very vague). Or does having a
> more powerful type system actually prevent you from doing
> things which could be done via the casting and other
> covariant holes in a less powerful (e.g. Java) type
> system? At this point, I'm sort of getting an insight
> into why dynamic language proponents claim typing
> restricts them even though I've not been previously
> sympathetic in any way...
>
I think doing things in a statically type safe way often makes things more complicated than doing it in a dynamic way, and can often require a bit more code as well. (In the case of Scala, it is a bit more code. In the case of Java, a lot more code.) The benefit you get for dealing with that extra code and complexity are the benefits of static typing, which include deterministic refactoring, documentation of programmer intent, possibly runtime performance (though often this is not relevant), better IDE support (code completion, etc.).

But what I always try and point out is that just because you're in a static language doesn't mean you can't use dynamic techniques when they are a better tradeoff. In ScalaTest, I have a couple matchers that are using dynamic techniques. When you use them, you lose deterministic refactoring, but you gain that you need to write a bit less code. Since this is testing, you'll hopefully find out the next time you run the tests what broke when you made a change.

One other thing is that when you can't express a constraint in the type system, you may be able in Scala to make a compiler plugin that catches the constraint. (In Java, you can do this only with an annotations processor, so there needs to be an annotation. But in Scala, you get the whole AST to play with.) I demonstrate this kind of stuff in my Feel of Scala talk on Parleys:

http://tinyurl.com/dcfm4c

> Or does having a
> more powerful type system actually prevent you from doing
> things which could be done via the casting and other
> covariant holes in a less powerful (e.g. Java) type
> system?

Scala's type system is essentially Java's type system plus stuff. The big exceptions are that arrays are not covariant and Scala doesn't have "raw types." In other words, if you have a type design that works well for Java then it will probably work well in Scala (although there might be a better solution available).

> At this point, I'm sort of getting an insight
> into why dynamic language proponents claim typing
> restricts them even though I've not been previously
> sympathetic in any way...

I don't follow. You've started going down this path to explore abstract type members and path dependent types. It's an interesting and sometimes useful approach to things. But it's not always the right answer. If abstract type members aren't the right answer (and frankly I rarely use them) then don't use them. Use something else.

If you want static checking then maybe "generic" type parameters are the right answer. In fact, I think that until there's a larger body of experience with path dependent types, the default solution to most such "real" static typing problems should be type parameters.

And if static checking isn't the right answer and you want to dynamically check compatible skiers or currency or whatever, Scala gives powerful tools for that as well. You can use OO style polymorphism or pattern matching, or both.

Finally, I want to point out that Scala has an escape mechanism in its type system. You can cast anything to anything at anytime. That's a big gaping hole as far as type systems are concerned, but a pragmatic and useful choice sometimes.

> > At this point, I'm sort of getting an insight
> > into why dynamic language proponents claim typing
> > restricts them even though I've not been previously
> > sympathetic in any way...
>
> I don't follow. You've started going down this path to
> explore abstract type members and path dependent types.
> It's an interesting and sometimes useful approach to
> o things. But it's not always the right answer. If
> abstract type members aren't the right answer (and frankly
> I rarely use them) then don't use them. Use something
> else.

let me clarify a bit. I'm not criticising Scala's type system per se. It seems very powerful and I can't really critique it as I literally understand 10% of it. i guess i just had a brief insight into the lengths to which it is sometimes necessary to delve into and understand the type system in order to express something in Scala which I have otherwise found intuitive in other languages (despite them being unsound). This type of expertise in type systems is not required with dynamic typing, although clearly this has severe disadvantages with regard to checking and refactoring.

(i'm no dynamic typing proponent however. i worked commercially in smalltalk on a medium sized team, and found the lack of typing to be very unpleasant)

> If you want static checking then maybe "generic" type
> parameters are the right answer. In fact, I think that
> until there's a larger body of experience with path
> dependent types, the default solution to most such "real"
> static typing problems should be type parameters.

Yes, sure, i'd tend to use generics also. I guess I was just following a path of thought from the Scala book. I've also been influenced somewhat by Beta perhaps, which has no generics and instead uses virtual patterns (path dependent types, I guess) to express this type of flexibility.

> And if static checking isn't the right answer and you want
> to dynamically check compatible skiers or currency or
> whatever, Scala gives powerful tools for that as well.
> You can use OO style polymorphism or pattern matching, or
> r both.

Sure.

> Finally, I want to point out that Scala has an escape
> mechanism in its type system. You can cast anything to
> anything at anytime. That's a big gaping hole as far as
> type systems are concerned, but a pragmatic and useful
> choice sometimes.

Hmm, that is interesting. And without wishing to be flippant, I think the presence of that "hole" would be very important in some programs. I expect there will always be things that are difficult or impossible to express in a practical type system.

Andrew

> > > At this point, I'm sort of getting an insight
> > > into why dynamic language proponents claim typing
> > > restricts them even though I've not been previously
> > > sympathetic in any way...
> >
> > I don't follow. You've started going down this path to
> > explore abstract type members and path dependent types.
> > It's an interesting and sometimes useful approach to
> > o things. But it's not always the right answer. If
> > abstract type members aren't the right answer (and
> frankly
> > I rarely use them) then don't use them. Use something
> > else.
>
> let me clarify a bit. I'm not criticising Scala's type
> system per se. It seems very powerful and I can't really
> critique it as I literally understand 10% of it. i guess
> i just had a brief insight into the lengths to which it is
> sometimes necessary to delve into and understand the type
> system in order to express something in Scala which I have
> otherwise found intuitive in other languages (despite them
> being unsound). This type of expertise in type systems is
> not required with dynamic typing, although clearly this
> has severe disadvantages with regard to checking and
> refactoring.
>
I think that's a reasoned way to look at it. There's no free lunch. Dynamic typing can indeed remove some complexity and bulk from your code, but in exchange complexity pops up elsewhere, such as needing to write more tests, less assurance when refactoring, less clarity when trying to figure out what you can count on about the types or services offered by objects being passed to a method, etc.. Static typing lets gives you deterministic refactoring, documentation about the types being passed to methods, eliminates the need to write tests for things caught by the type system, etc., but in exchange you have to learn and deal with a more complex type system compared to the dynamic languages and write a bit more code. (A bit more in Scala. In Java, you have to write a lot more code, but that isn't really static typing's fault.)

> > If you want static checking then maybe "generic" type
> > parameters are the right answer. In fact, I think that
> > until there's a larger body of experience with path
> > dependent types, the default solution to most such
> "real"
> > static typing problems should be type parameters.
>
> Yes, sure, i'd tend to use generics also. I guess I was
> just following a path of thought from the Scala book.
> I've also been influenced somewhat by Beta perhaps, which
> h has no generics and instead uses virtual patterns (path
> dependent types, I guess) to express this type of
> flexibility.
>
I also tend to use generics. It may be because I come from Java and that's what I'm used to, but they seem easier for me to think about.

> > And if static checking isn't the right answer and you
> want
> > to dynamically check compatible skiers or currency or
> > whatever, Scala gives powerful tools for that as well.
> > You can use OO style polymorphism or pattern matching,
> or
> > r both.
>
> Sure.
>
> > Finally, I want to point out that Scala has an escape
> > mechanism in its type system. You can cast anything to
> > anything at anytime. That's a big gaping hole as far
> as
> > type systems are concerned, but a pragmatic and useful
> > choice sometimes.
>
> Hmm, that is interesting. And without wishing to be
> flippant, I think the presence of that "hole" would be
> very important in some programs. I expect there will
> always be things that are difficult or impossible to
> express in a practical type system.
>
Well first there's pattern matching, which effectively does safe casts for you. Martin doesn't like calling this a cast though. In Scala jargon, "cast" means unsafe cast from the statically provable perspective. And in Scala you can pretty much cast anything to anything. Even things that could be statically known to be sure to fail at runtime are allowed, if I'm remembering correctly. I think Java will catch these as compiler errors. So casting with asInstanceOf is very much discouraged in Scala, but it is there when you need it.

In ScalaTest I have used it on several occasions. I did a grep to see why. Looks like I primarily use it when I'm using Java reflection:

Macintosh-10:scalatest bv$ fgrep asInstanceOf *.scala
Assertions.scala:      case Some(e) => e.asInstanceOf[T] // I know this cast will succeed, becuase iSAssignableFrom succeeded above
Assertions.scala:      case Some(e) => e.asInstanceOf[T] // I know this cast will succeed, becuase iSAssignableFrom succeeded above
Assertions.scala:    val clazz = manifest.erasure.asInstanceOf[Class[T]]
Assertions.scala:      case Some(e) => e.asInstanceOf[T] // I know this cast will succeed, becuase iSAssignableFrom succeeded above
Assertions.scala:    "hi".asInstanceOf[T]
PrivateMethodTester.scala:              for ((arg, paramType) <- zipped if !paramType.isAssignableFrom(arg.asInstanceOf[AnyRef].getClass)) yield arg
PrivateMethodTester.scala:            case anyVal: AnyVal => anyVal.asInstanceOf[AnyRef]
PrivateMethodTester.scala:        privateMethodToInvoke.invoke(target, anyRefArgs.toArray: _*).asInstanceOf[T]
SuiteRerunner.scala:      val suite = suiteClass.newInstance().asInstanceOf[Suite]
TestRerunner.scala:      val suite = suiteClass.newInstance.asInstanceOf[Suite]

> runtime are allowed, if I'm remembering correctly. I think
> Java will catch these as compiler errors.

Java just makes you jump through an extra hoop


public class TestJ {
static class Foo {}
static class Bar {}

public static void main(String[] args) {
Object it = new Bar();
Foo foo = (Foo)it;
}
}


Scala would let you just write

val foo = (new Bar).asInstanceOf[Foo]


Of course, either way gets you

java.lang.ClassCastException: Bar cannot be cast to Foo

> > runtime are allowed, if I'm remembering correctly. I
> think
> > Java will catch these as compiler errors.
>
> Java just makes you jump through an extra hoop
>
>
> public class TestJ {
> static class Foo {}
> static class Bar {}
>
> public static void main(String[] args) {
> Object it = new Bar();
> Foo foo = (Foo)it;
> }
> }
>
>
> Scala would let you just write
>
> val foo = (new Bar).asInstanceOf[Foo]
>
>
> Of course, either way gets you
>
> java.lang.ClassCastException: Bar cannot be cast to Foo
>
That's funny. Even for unsafe code Java is more verbose than Scala.

> That's funny. Even for unsafe code Java is more verbose
> than Scala.

Well, you can help Java out a bit if you want.

public class Perversion {
   static class Foo {}
   static class Bar {}

   @SuppressWarnings("unchecked")
   public static <T> T cast(Object x) {
      return (T)x;
   }

   public static void main(String[] args) {
      Foo f = cast(new Bar());
   }	
}

Would it be possible for Scala to also target system programming or would that a require major language revision?

> > Scala would let you just write
> >
> > val foo = (new Bar).asInstanceOf[Foo]
> >
> >
> > Of course, either way gets you
> >
> > java.lang.ClassCastException: Bar cannot be cast to Foo
> >
> That's funny. Even for unsafe code Java is more verbose
> than Scala.

How about String s = (String)new Date();

I just tried this in Eclipse, it doesn't give a syntax error, it gives a perfectly reasonable "Cannot cast from Date to String" error.

Once again, a claim that Java is way more verbose than another language is refuted.

> How about String s = (String)new Date();
>
> I just tried this in Eclipse, it doesn't give a syntax
> error, it gives a perfectly reasonable "Cannot cast from
> Date to String" error.

I don't think anyone said it was a syntax error. It is a compiler error though, in the standard Java compiler and the Eclipse one. Eclipse catches it with a little error symbol in the sidebar.

You can of course do this to avoid the error, and push it into runtime:

String s = (String) (Object) new Date();