The Artima Developer Community
Sponsored Link

Weblogs Forum
Getting Dynamic Productivity in a Static Language

78 replies on 6 pages. Most recent reply: Apr 5, 2009 3:19 PM by Cedric Beust

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 78 replies on 6 pages [ « | 1 2 3 4 5 6 | » ]
Michele Simionato

Posts: 222
Nickname: micheles
Registered: Jun, 2008

Re: Getting Dynamic Productivity in a Static Language Posted: Mar 30, 2009 9:26 PM
Reply to this message Reply
Advertisement
> 2. Quick feedback loop. Given dynamic languages (at least
> Ruby, Python, and Groovy, which I was considering as
> alternatives to Scala) usually run (or can run) by
> interpreting right from source code, you don't have a
> compile cycle to wait for.

This is true for small scripts. However, if you are
refactoring a large application, you will likely have
to run a large test suite, so that you will have to wait
even when using a dynamic language. It is possible
that with a static language you will have to wait a bit more
since you have the compilation time on top of
the running time, but I am not sure if this make
a significant difference in practice.

Michele Simionato

Posts: 222
Nickname: micheles
Registered: Jun, 2008

Re: Getting Dynamic Productivity in a Static Language Posted: Mar 30, 2009 9:40 PM
Reply to this message Reply
> Hi James,
>
> > scala> def flatten(xs : List[Any]) : List[Any] = xs
> > flatMap {
> > | case y :: ys => flatten(y :: ys)
> > | case x => List(x)
> > | }
> > flatten: (List[Any])List[Any]
> >
> Thanks for this contribution. Much simpler.
>
> Now, it's not like we're keeping score here, but I count 8
> lines of code in the Python version and 4 lines of code in
> the Scala one!

Well, then Scheme only needs two lines:

(define (flatten lst)
(fold-right (lambda (x a) (if (list? x) (append (flatten x) a) (cons x a))) '() lst))

But we are not counting lines here ;) To be honest, the
version I like the most is your original Scala code.

Fred Garvin

Posts: 52
Nickname: fredgarvin
Registered: Jan, 2008

Re: Getting Dynamic Productivity in a Static Language Posted: Mar 31, 2009 1:08 AM
Reply to this message Reply
>> While I agree with the specific examples you give, it
>> doesn't change that the conceptual simplicity of
>> learning the language itself is greater when you have
>> the Smalltalk/Ruby/Python/Groovy etc. model of method
>> invocations are messages sent between objects.
>> ...

I'm not feeling it. How does "message passing" simplify the mental model? A programmer still needs to understand the concept of types, even if they are dynamic. Do I not have classes in Ruby? Are there not function types as well? That's the rub. A dynamic type system has types, but you are not supposed to admit that in your programs. Somehow you are supposed to know what "messages" are ok to send and what type of information should be in them. Call me crazy, but I think it's beneficial to have a declarative type system around to help me (and my tools) with these things.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Getting Dynamic Productivity in a Static Language Posted: Mar 31, 2009 3:11 AM
Reply to this message Reply
Hi Fred,

> >> While I agree with the specific examples you give, it
> >> doesn't change that the conceptual simplicity of
> >> learning the language itself is greater when you have
> >> the Smalltalk/Ruby/Python/Groovy etc. model of method
> >> invocations are messages sent between objects.
> >> ...
>
> I'm not feeling it. How does "message passing" simplify
> the mental model? A programmer still needs to understand
> the concept of types, even if they are dynamic. Do I not
> have classes in Ruby? Are there not function types as
> well? That's the rub. A dynamic type system has types, but
> you are not supposed to admit that in your programs.
> Somehow you are supposed to know what "messages" are ok to
> send and what type of information should be in them. Call
> me crazy, but I think it's beneficial to have a
> declarative type system around to help me (and my tools)
> with these things.
>
Well maybe I'm wrong about this. What "simple"means isn't so easy to get at. But one example is collections. When you use dynamic typing, you can put anything into a collection, then take it out and call anything on it. If the thing you pull out has the method you call, it will work. Else it will throw an exception at runtime. That's about all you need to explain it.

Now when you go to static typing, you'll find you want to talk about type parameterization. In Python a list is an ordered sequence of elements. In Scala, a list is an ordered sequence of elements of a particular type. List in Scala therefore takes a type parameter, for example, a List[String] would contain strings. This not only enforces that only strings go in there, but more importantly makes it possible to statically invoke string methods on the objects you pull out of it. So already it is a bit more complicated, but the plot thickens.

At some point you may have a List[String] and want to pass it to a method that takes a List[Object]. Can you? Well that brings up the question of variance. It turns out that you can in this case, because List in Scala is covariant in its type parameter. But there are also nonvariant type parameters, where there is no inheritance relationship provided by the type parameter, and contravariant, where the inheritance relationship is upside down. So this adds more complexity to understanding the language, even though this bit of language complexity here may actually simplify your life later when you're trying to refactor a program.

Andrew McVeigh

Posts: 29
Nickname: 55548
Registered: May, 2008

Re: Getting Dynamic Productivity in a Static Language Posted: Mar 31, 2009 3:12 AM
Reply to this message Reply
> One of the most commonly cited productivity benefits of
> dynamic
> languages is that they enable much more concise programs
> compared to
> Java.

> Another productivity benefit of dynamic languages is that,
> via a
> feature called open classes in Ruby or Python, you can add
> a method to
> a class and then call it. You can't do this in a static
> language.

i really like how you've listed actual reasons why you feel dynamically typed languages are more productive. i tend to agree that they feel more productive, certainly i feel that python more closely matches my mental model of the way programs work than Java does.

However, I remember in Richard P. Gabriel's book "Patterns of Software" he worked with some C++ programmers and actually quantified that the productivity of Lisp over C++ was at most something like 15%. I haven't read the book recently, but I recall that he (as a staunch lisp advocate) was very surprised by the small-ish disparity. He expected it to be much larger, i.e. 4-10x. I was impressed by his balanced approach, as it must have been hard for him to see these figures.

And that was for C++, a language without auto memory mgmt, and without significant class libraries... My point is that I think much of the imagined productivity benefits disappear when working on large systems. Certainly, I don't feel too much less productive working on my large phd project (in Java) versus when I do work in python or smalltalk...

Has there been actual scientific studies that show what the productivity benefit is in practice? i.e. not just impressionistic viewpoints?

Andrew

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Getting Dynamic Productivity in a Static Language Posted: Mar 31, 2009 3:17 AM
Reply to this message Reply
> I see. So basically you are saying that Scala has both
> static typing AND dynamic typing (the Any type there).
> Given that, it is not surprising that you can easily
> implement the flatten function.
> But I see this as a concession to the dynamic
> typing camp i.e. realizing that for certain classes of
> problems where types are unknown in advance you must use
> dynamic typing.
> Personally I am not in the dynamic camp nor
> in the static camp; I have much more experience with
> dynamic languages, nevertheless I have always liked
> static typing, at least from the time of Pascal. I also
> have some experience with SML for what concerns type
> inference. I like the idea. For me it is more of a matter
> of practicality. If I can find a readable language with
> small learning curve, clean design and decent libraries
> I have nothing against it, no matter if it is statically
> or dynamically typed.
> Having said that, we do not use Java at work,
> nor we have to interact with the Java platform,
> so I will likely stay ignorant about Scala
> for a while and just read one article or two of
> yours ;)
>
Actually the Any type doesn't signify dynamic typing. It is just the root type in Scala's inheritance hierarchy. It means you can put any object in a List[Any], but when you pull those objects out, you can only call methods on them that are defined in class Any, unless you cast them to a more specific class first.

But you can indeed use all the dynamic techniques in static languages, though how clunky it feels will vary from static language to static language. In the Feel of Scala video I show several cases of duck typing, for example, and in Scala they all look pretty decent.

Cedric Beust

Posts: 140
Nickname: cbeust
Registered: Feb, 2004

Re: Getting Dynamic Productivity in a Static Language Posted: Mar 31, 2009 9:51 AM
Reply to this message Reply
Hi Bill,

> While I agree with the specific examples you give, it
> doesn't change that the conceptual simplicity of learning
> the language itself is greater when you have the
> Smalltalk/Ruby/Python/Groovy etc. model of method
> invocations are messages sent between objects.

I don't really understand why the passing of messages is conceptually simpler than method invocation. It's actually worse because you will receive much less assistance from the IDE to tell you which messages are actually valid.

> When he asked me what would
> be a good language to learn to do some stuff on the side,
> I sent him a Python book.

... and? You're leaving out the essential part: did he like it?

Anyway, this is a bit of a tangent: he probably liked it if all he needed to do was write a program that's less than 100 lines. For this, emacs or even Notepad is probably enough and the punishment for misspelling a word won't be too dire (a runtime error).

You are basically arguing that Python is great for beginners, which looks pretty accurate to me. But that's not what we are discussing...

--
Cedric

Cedric Beust

Posts: 140
Nickname: cbeust
Registered: Feb, 2004

Re: Getting Dynamic Productivity in a Static Language Posted: Mar 31, 2009 9:55 AM
Reply to this message Reply
> Now when you go to static typing, you'll find you
> want to talk about type parameterization

Not always, you can still stick with raw types if you want to.

Computer science history has shown that it's a bad idea, but if that's what you want, Java gives you the option.

Dynamically typed languages don't (this has been a long-time beef of mine with Ruby: sometimes, I *do* want to give a type to variables or parameters. I know what they are, it's just a few more keys to type and it makes the code much more readable. But Ruby won't let me).

--
Cedric

Martin Odersky

Posts: 84
Nickname: modersky
Registered: Sep, 2003

Re: Getting Dynamic Productivity in a Static Language Posted: Mar 31, 2009 10:39 AM
Reply to this message Reply
> I see. So basically you are saying that Scala has both
> static typing AND dynamic typing (the Any type there).

No. Any is just the root of the type hierarchy. It's like Object in Java, except that it also encompasses ``value types'' such as Int or Boolean.

> Given that, it is not surprising that you can easily
> implement the flatten function.
> But I see this as a concession to the dynamic
> typing camp i.e. realizing that for certain classes of
> problems where types are unknown in advance you must use
> dynamic typing.

This is true only in an indirect sense. There is no dynamic typing in Scala, even though pattern matching gives some of the same advantages dynamic typing gives, in the sense that it allows to recover some type information at run time.
But it does so in a more controlled way. For instance, the unchecked warning in Bill's first solution would have told you that you might have missed a case in the pattern match.

James Iry's solution did not have that problem, so you did not see a warning.

Michele Simionato

Posts: 222
Nickname: micheles
Registered: Jun, 2008

Re: Getting Dynamic Productivity in a Static Language Posted: Mar 31, 2009 10:45 AM
Reply to this message Reply
> Anyway, this is a bit of a tangent: he probably liked it
> if all he needed to do was write a program that's less
> than 100 lines. For this, emacs or even Notepad is
> probably enough and the punishment for misspelling a word
> won't be too dire (a runtime error).

At work we have a code base of 250,000+ lines of Python code
written by us, plus probably something like 750,000
lines of code from the various frameworks/libraries we use.
We manage it pretty well. I am actually convinced
that it would be more difficult to manage an equivalent
code base in C++, even not taking in account the fact
that Python code is much more dense than C++ or Java
code, and 1 million of lines in Python are doing much
more than 1 million of lines of code of Java or C++.

We use emacs and vi, which have strong support for
Python and we never felt the need for an IDE.
We use tools like pylint and pyflakes to check for
syntax errors, mispellings and other trivial errors.
We use tests a lot. Not everything is perfect, and
static typing has some advantage, that's true, but
still it is perfectly possible to manage *huge* projects
with a dynamic language such as Python.
This has been proved many and many times, and I am not
the only on who could provide such an evidence based on
real life experience (I have being using Python as
my main programming language for 7 years now).

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Getting Dynamic Productivity in a Static Language Posted: Mar 31, 2009 10:50 AM
Reply to this message Reply
> > Given that, it is not surprising that you can easily
> > implement the flatten function.
> > But I see this as a concession to the dynamic
> > typing camp i.e. realizing that for certain classes of
> > problems where types are unknown in advance you must
> use
> > dynamic typing.
>
> This is true only in an indirect sense. There is no
> dynamic typing in Scala, even though pattern matching
> gives some of the same advantages dynamic typing gives, in
> the sense that it allows to recover some type information
> at run time.
> But it does so in a more controlled way. For instance, the
> unchecked warning in Bill's first solution would have told
> you that you might have missed a case in the pattern
> match.
>
And to clarify, in the video I show that you can use dynamic *techniques* in Scala, even though it is a statically typed language. Usually this involves reflection. The private method invocation mechanism that I show uses reflection, and is the same technique as duck typing in Ruby or Python. So even though Scala is statically typed, and not dynamically typed, you can still when you want use dynamic techniques, and you can get the code to look pretty concise.

But I also show that often in Scala there are better, static ways to get the same benefit that's really being derived from the dynamic technique. For example, structural subtyping gives you the same ability that duck typing offers in a dynamic language of passing objects that don't share a common supertype to a method, which treats them uniformly. But structural types are static types, checked at compile time, even though reflection is used at runtime to invoke the methods.

Cedric Beust

Posts: 140
Nickname: cbeust
Registered: Feb, 2004

Re: Getting Dynamic Productivity in a Static Language Posted: Mar 31, 2009 11:19 AM
Reply to this message Reply
I think Structural Typing has a very narrow scope in which it's actually useful, typically in a case where you need a structure that you won't be using more than a couple of times in your entire code base.

If you will be needing this structure more, you are much better off creating a real class.

For a more in-depth discussion of the pros and cons of Structural Typing, see this:

http://beust.com/weblog/archives/000476.html

James Iry

Posts: 85
Nickname: jiry
Registered: Nov, 2007

Re: Getting Dynamic Productivity in a Static Language Posted: Mar 31, 2009 1:03 PM
Reply to this message Reply
> Well, then Scheme only needs two lines:
>
> (define (flatten lst)
> (fold-right (lambda (x a) (if (list? x) (append (flatten
> en x) a) (cons x a))) '() lst))


Why stop there? The Scheme version can be one line and we can squish out the extra whitespace for some real shrinkage. Scheme doesn't care.
(define(flatten lst)(fold-right(lambda(x a)(if(list? x)(append(flatten x)a)(cons x a)))'()lst))


While we're at it I can make the Scala version one line and squish out unnecessary whitespace

def flatten(xs:List[Any]):List[Any]=xs flatMap{case y::ys=>flatten(y::ys);case x=>List(x)}


It's smaller than the squished Scheme version. Yay! I win the cookie! But wait, the Scheme version could be smaller by using the same technique as the Scala version (append-map is the same as flatMap on Scala lists, same as >>= on Haskell lists).

(define(flatten lst)(append-map(lambda(x)(if(list? x)(flatten x)(list x)))lst))


Yay, I win another cookie!

Morals of the story -
1) playing golf is fun
2) you can format incomprehensible code in any language
3) prefer higher level functions over folds
4) high level code is generally more concise than low level code regardless of typing discipline

Andrew McVeigh

Posts: 29
Nickname: 55548
Registered: May, 2008

Re: Getting Dynamic Productivity in a Static Language Posted: Mar 31, 2009 2:41 PM
Reply to this message Reply
> I think Structural Typing has a very narrow scope in which
> it's actually useful, typically in a case where you need a
> structure that you won't be using more than a couple of
> times in your entire code base.
>
> If you will be needing this structure more, you are much
> better off creating a real class.

this only works if you can modify the classes to implement your new interface. if you want to implement a new method that is suppose to accept as a parameter one of 2 classes, that share a common method signature but have no common subclass or subinterface, what alternative is there to structural typing?

James Iry

Posts: 85
Nickname: jiry
Registered: Nov, 2007

Re: Getting Dynamic Productivity in a Static Language Posted: Mar 31, 2009 3:05 PM
Reply to this message Reply
> this only works if you can modify the classes to implement
> your new interface. if you want to implement a new method
> that is suppose to accept as a parameter one of 2 classes,
> that share a common method signature but have no common
> subclass or subinterface, what alternative is there to
> structural typing?

The "adapter pattern." To the Java-mobile!


class Something {
public void close() {...}
}

class SomethingElse {
public void close() {...}
}

interface Closeable {
void close();
}

class Wrappers {
public static Closeable closeable(Something s) {
return new Closeable() {public void close() {s.close();}}

public static Closeable closeable(SomethingElse s) {
return new Closeable() {public void close() {s.close();}}
}

// imagine this is somewhere with appropriate imports and more boilerplate
public void foo(Closeable closeable) {
// ...
closeable.close();
}

// and somewhere else
import static Wrappers.*
// blah blah blah, yada yada yada
x.foo(closeable(new Something()));
x.foo(closeable(new SomethingElse()));


Not saying it's a BETTER alternative, mind you, I'm just saying it's an alternative. It also illustrates why the notion that nominative typing is somehow "stronger" than structural typing is so much hogwash. By typing a few pounds of boilerplate I've achieved the same effect without altering the type system. (people who argue about how strong a type system is rarely know what the hell they're talking about). Contrast this with the eminently more pleasant Scala


def foo(closeable : {def close}) {
// ...
closeable.close
}

// and now
x.foo(new Something)
x.foo(new SomethingElse)


Same thing, 1/100th the boilerplate, and works without typing more boilerplate to handle other closeable things. The type system is still doing its job. Also, nothing about the structurally typed version precludes using an adapter (maybe even an implicit one) to handle things with "shutdown" methods instead of "close" or whatever.

Flat View: This topic has 78 replies on 6 pages [ « | 1  2  3  4  5  6 | » ]
Topic: AntiSocial Networking Previous Topic   Next Topic Topic: Mixins considered harmful/3

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use