The Artima Developer Community
Sponsored Link

Weblogs Forum
An Alternative to Closure Conversion and to Restricted Closures

30 replies on 3 pages. Most recent reply: Jan 13, 2008 5:12 AM by Howard Lovatt

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 30 replies on 3 pages [ 1 2 3 | » ]
Howard Lovatt

Posts: 321
Nickname: hlovatt
Registered: Mar, 2003

An Alternative to Closure Conversion and to Restricted Closures (View in Weblogs)
Posted: Jan 2, 2008 5:03 PM
Reply to this message Reply
Summary
The Closure proposal from Neal Gafter et al. introduces something that is like an object, a closure, but not quite. To enable interoperation with normal objects a form of autoboxing is used, this autoboxing has two forms, normal closure conversion and restricted closure conversion. This blog examines an alternative.
Advertisement

Disclosure and Gripe

I should begin by disclosing that I have presented an alternative to the closures proposal, C3S that is based on objects. So obviously I don't whole heartedly agree with the closures proposal as it stands. However I would like a shorter syntax for creating one off instances of interfaces and classes.

Now the gripe: I have tried to have a discussion about alternatives to the BGGA proposal on Neal's blog. Neal chooses to moderate his blog and what typically happens is that I make a comment, which he allows, then he replies with a rebuttal, I disagree with the rebuttal and post a second comment. This second comment is then moderated. I am not sure if Neal is doing this deliberately or not, but either way the result appears that a question was raised and addressed and the issue has gone away. From my point of view this is far from the truth. The issue of this blog, closure conversions, is one such case and hence my own blog entry.

I have invited Neal to have a right of reply and I sincerely hope he does state how he sees the moderating of comments on his blog.

As an aside: an un-moderated blog has the advantage of providing a record of discussions. With a moderated blog you can never be certain that you are seeing the 'whole truth'.

Closure Conversion

Closures are structurally typed not name typed, for example suppose you had a military application:

interface Missile { boolean fire( long timeInMS ); }
interface Ship { boolean depart( long timeInMS ); }

You have two distinct methods with different names Missile.fire and Ship.depart, even though they have the same signature boolean xxx( long ). With a closure there are no names so a closure with the right signature can gets converted via a closure conversion to either of these:

Missile m = { long timeInMS => ... }; // closure get boxed into a Missile
Ship s = { long timeInMS => ... }; // closure gets boxed into a Ship

Note a closure only gets converted once, i.e. you can't say s = m and if you say { long => void } c = { long timeInMS => ... } then a closure conversion to a synthetic interface goes on behind the scenes and hence you can't say m = c. This automatic boxing is necessary to allow interoperation of closures with interfaces. Closure conversion only works with interfaces that have exactly one method. If there are two methods, or if there is a class then there are no closure conversions. Therefore not all existing APIs can be use directly with closures via closure conversions, e.g. AWT and Swing: AbstractAction, MouseListener, KeyListener, etc. There are some suitable interfaces in AWT like ActionListener, but it is annoying that it is not universally possible to use a closure to define abstract methods.

Restricted Closure Conversion

So what's the problem other than incompatible existing APIs? Well if the closure is stored somewhere and then executed at a later time there are a number of features that don't work. Closures allow non-local returns, for example. If an each method were added to java.util.Collections then:

static <T> boolean isKeyPresent( List<T> list, T key ) {
  Collections.each( list, {T x => if ( x.equals( key ) ) return true; } );
  return false;
}

There is an alternative syntax for things like each methods, but to keep this blog simple I didn't use that syntax in the above example.

The return statement inside the closure not only returns from the closure but also from the enclosing method, isKeyPresent.

Now if a closure is to be executed after it is made then an exception can result when the code is run, e.g. if a closure is passed to another thread, stored in a data structure for execution later, or made by another method:

static { => void } makeMethod() {
  return { =>
    ...
    return { => ...; }; // Error not caught by the compiler! Returns from makeMethod and the closure
  };
}

static void main( String[] notUsed ) { makeMethod().invoke(); }

This is an error since the return inside the closure inside makeMethod tries to return from makeMethod (and the closure) but the closure is executed inside main, not inside makeMethod, and hence the code throws an exception when run. The above example is an error — the return keyword should be omitted inside the closure (but with an inner class you would need the return and therefore this is likely to be a common error since people will have to mix closures with inner classes).

To try and prevent these exceptions occurring a marker interface, RestrictedFunction, can be used to mark an interface but not the closure. Therefore if the above example is rewritten to introduce an interface the error can be prevented:

interface VoidMethod extends RestrictedFunction { void invoke(); }

static VoidMethod makeMethod() {
  return { =>
    ...
    return { => ...; }; // Error now caught by the compiler
  };
}

But to prevent the error you need to modify interfaces to extend RestrictedFunction and this may not be possible if you don't control all the source, i.e. you will be relying on all the Java libraries, including third party, to be modified to be closure compatible.

An Alternative

An alternative to RestrictedFunction is given in C3S and is to make a closure a normal object that behaves like an instance of an inner class and for the special features, like non-local returns, to require named qualification, e.g. the examples above:

static { => void } makeMethod() {
  return { =>
    ...
    return { => ...; }; // Error! The closure returns a void
  };
}

or

static { => void } makeMethod() {
  return { =>
    ...
    makeMethod.return { => ...; };
  }; // Error non-local return used inside an un-executed closure
}

and

static <T> boolean isKeyPresent( List<T> list, T key ) {
  Collections.each( list, {
    T x => if ( x.equals( key ) ) isKeyPresent.return true; // Qualified return
  } );
  return false;
}

I.E. explicitly say what is to be returned from if it isn't the immediately enclosing scope.

For name scoping retain the current inner class rules, i.e. if a name is inherited and also exists in an outer scope then the inherited name takes precedence and the outer scope name needs to be qualified. The only inherited names in a closure are method names from Object and the method name invoke, therefore name clashes will be rare. If from within a closure you wanted to refer to the enclosing classes toString method, for example, then just like an inner class you would write [class name].toString().

Tennent's Correspondence Principle

Neal Gafter, one of the proposers of closures for Java, has suggested that Tennent's Correspondence Principle is applied to design closures. Tennent mostly demonstrated his principle by enclosing some code inside an inner procedure (void method in Java speak). The principle is that if you enclose an expression inside an inner procedure and execute the procedure then the meaning of the expression shouldn't change. This principle precludes the return statement and is therefore problematic to apply to Java closure proposals since Java already has return. In Java you can't have inner methods but if you could then:

int outer() {
  return 1; // Expression to be enclosed
}

Then enclose the statement in an inner method and execute the method (assuming Java had inner methods):

int outer() {
  void inner() { return 1; } // Error! inner is a void method
  inner();
} // Error outer doesn't return a value

The solution from Tennent is to ban return statements and have the name of the method as the return value, e.g.:

int outer() {
  outer = 1; // Expression to be enclosed (equivalent of return)
}

Becomes:

int outer() {
  void inner() { outer = 1; } // OK
  inner();
} // OK outer is 1

With name qualified returns it is better than standard Java unqualified return, though far from perfect:

int outer() {
  void inner() { outer.return 1; } // OK
  inner();
  return 0; // Dummy return to keep compiler happy
}

Therefore named returns help, but are not perfect. Surprisingly, considering that Tennent's Principle is often quoted when discussing closures, the present closure proposal has the same problem as named returns:

int outer() {
  { => return 1; }.invoke();
  return 0; // Dummy return to keep compiler happy
}

The compiler cannot in general follow control flow inside the closure and therefore cannot prove that outer always returns a value (at least for complex closures — unlike the example which the compiler may be able to do).

Summary

Closure conversions, both normal and restricted, have a number of catches, a bit like autoboxing of primitives does. The problem is that a closure, like a primitive, is a bit like an object but not quite and this difference is patched over by boxing the closure within an object. I think we should be striving for a more consistency, i.e. everything as an object, not introducing more none-object types.

I have presented an alternative that retains the power of the closure but without the downside, in particular I have presented a closure that is an object and therefore doesn't require any boxing. What do others think?


Neal Gafter

Posts: 12
Nickname: gafter
Registered: Oct, 2003

Re: An Alternative to Closure Conversion and to Restricted Closures Posted: Jan 2, 2008 5:50 PM
Reply to this message Reply
My blog is not intended to be an open forum for discussion about whatever topic you want. Generally, if a post is discussing the contents of my blog, I approve it. If you want to discuss alternative proposals in detail, without even referring to the blog post you are commenting on, I suggest you use your own blog.

Your blog post here demonstrates basic misunderstandings about the BGGA proposal and the way the type system works. Your very first example will not even compile with the BGGA prototype, and never will:


interface Missile { boolean fire( long timeInMS ); }
interface Ship { boolean depart( long timeInMS ); }
{ long => boolean } closure = { long timeInMS => ... };
Missile m = closure;
Ship s = closure;


The problem is that the variable "closure" is a variable of a function type, and function types are interface types. A reference to a variable is never a closure (even though a closure has been assigned to it), even if the variable has been misleadingly named, and so the closure conversion does not apply.

Howard Lovatt

Posts: 321
Nickname: hlovatt
Registered: Mar, 2003

Re: An Alternative to Closure Conversion and to Restricted Closures Posted: Jan 2, 2008 6:57 PM
Reply to this message Reply
@Neil,

Thankyou for posting

> My blog is not intended to be an open forum for discussion
> about whatever topic you want. Generally, if a post is
> discussing the contents of my blog, I approve it. If you
> want to discuss alternative proposals in detail, without
> even referring to the blog post you are commenting on, I
> suggest you use your own blog.

I would have thought an alternative to RestrictedFunction was on topic and I did try and post on your blog but my second post following your reply was moderated.

> Your blog post here demonstrates basic misunderstandings
> about the BGGA proposal and the way the type system works.
> Your very first example will not even compile with the
> BGGA prototype, and never will:
>
>
> interface Missile { boolean fire( long timeInMS ); }
> interface Ship { boolean depart( long timeInMS ); }
> { long => boolean } closure = { long timeInMS => ... };
> Missile m = closure;
> Ship s = closure;
>

>
> The problem is that the variable "closure" is a variable
> of a function type, and function types are interface
> types. A reference to a variable is never a closure (even
> though a closure has been assigned to it), even if the
> variable has been misleadingly named, and so the closure
> conversion does not apply.

Thanks for clearing that up - I have modified the blog.

As an aside you could allow the example with your proposal (I had assumed you would since there is some advantage in doing so). The statement { long => boolean } closure = { long timeInMS => ... }; gets boxed:

interface Function0<R, throws E> { // system-generated
    R invoke() throws E;
}
Function0<? extends Void, ? extends RuntimeException> closure = { long timeInMS => ... };


The above is a cut and past (with minor modification) from your spec. As the spec. notes Function0 is synthesized by the compiler, therefore you know Function0 is closure therefore you could allow further boxing to an arbitrary interface, just like you do for a closure literal;

Ricky Clarkson

Posts: 63
Nickname: rclarkson
Registered: Jul, 2006

Re: An Alternative to Closure Conversion and to Restricted Closures Posted: Jan 2, 2008 7:29 PM
Reply to this message Reply
> <p>Now the gripe: I have tried to have a discussion about
> alternatives to the BGGA proposal on Neil's blog.

I doubt this is behind the apparently selective moderation, but it would help if you spelled his name right sometimes. ;)

> <p>As an aside: an un-moderated blog has the advantage of
> providing a record of discussions. With a moderated blog
> you can never be certain that you are seeing the 'whole
> truth'.</p>

I happen to agree; I'd rather move off-topic content to a separate page and link to it from the blog post in a footnote than to delete it outright. I had similar behaviour from some bloggers in the "Wall of Erasure" blogs.

> conversions, e.g. most of AWT and Swing:
> <code>MouseListener</code>, <code>KeyListener</code>,

MouseListener and KeyListener are probably not used as often as ActionListener is, by a few orders of magnitude. I can imagine writing simple wrappers for those to make using closures with them attractive.

> <code>JButton</code>, etc.</p>

I'm not sure how JButton is relevant.
> <p>The <code>return</code> statement inside the closure
> not only returns from the closure but also from the
> enclosing method, <code>isKeyPresent</code>. Now if the
> closure is to be executed after it is made then an
> exception can result when the code is run, e.g. if the
> closure is passed to another thread, stored in a data
> structure for execution later, or made by another
> method:</p>

That's very true. In what case would an 'each' method store the closure for later execution?

> <i>The above example is deliberate
> error — the <code>return</code> keyword should be
> omitted inside the closure.</i></p>

Is your concern that you would get it wrong, or that others would? Here you had to do it deliberately.

> <p>But to prevent the error you need to modify interfaces
> to extend <code>RestrictedFunction</code> and this may not
> be possible if you don't control all the source

A very good point.

> and for the
> special features, like non-local returns, to require named
> qualification, e.g. the examples above:</p>

I'm sure you're aware of Neal's "Tennent's Correspondence Principle" argument. It's a shame you neglected to comment on that here.

Imagine that foreach did not exist already. Yes, this is an academic exercise, but may be specialised to other constructs. Here is how the calling code would look:

foreach(String s: listOfStrings)
{
    if (s.equals(someString))
        callingMethod.return s;
}


I'm not sure why we're having to name the method we're in when we return. If we can get over the mental hurdle that is "closures are blocks, not anonymous functions" then the labelled return is not necessary.

> <p>I have presented an alternative that retains the power
> of the closure but without the downside, in particular I
> have presented a closure that <i>is</i> an object and
> therefore doesn't require any boxing. What do others
> think?</p>

I asked Neal to provide a use case where a closure is both used for its value and for its side-effects. He hasn't shown one, perhaps it exists with exceptions. Here's what I mean. Note that it's really not a use case, because I can't think of one, but a syntax example:

{=> int} perhaps={ => if (Math.random()<0.5) return "hello"; else 5 };


When this is evaluated, sometimes it will cause the enclosing method to return "hello", sometimes the evaluation of the closure will yield 5. I think a simple rule to make most complaints about non-local return disappear is that a non-void closure should not be able to use return. That violates Tennent's Correspondence Principle, it is less pure than BGGA how it is now, but I can't think of a case that it negatively impacts.

Carson Gross

Posts: 153
Nickname: cgross
Registered: Oct, 2006

Re: An Alternative to Closure Conversion and to Restricted Closures Posted: Jan 2, 2008 8:54 PM
Reply to this message Reply
We meet again, Howard Lovatt.

Cheers,
Carson

Howard Lovatt

Posts: 321
Nickname: hlovatt
Registered: Mar, 2003

Re: An Alternative to Closure Conversion and to Restricted Closures Posted: Jan 2, 2008 10:05 PM
Reply to this message Reply
@Ricky,

Thanks for your comments - I have modified the blog to address most of your comments and will give a second reply for the remaining comment.

> but it would help if you spelled his name
> right sometimes. ;)

Oops - I have corrected the blog - sorry NeAl

> MouseListener and KeyListener are probably not used as
> often as ActionListener is, by a few orders of magnitude.

I have qualified my comments

> That's very true. In what case would an 'each' method
> store the closure for later execution?

I have slightly modified the wording to be clearer

> Is your concern that you would get it wrong, or that
> others would? Here you had to do it deliberately.

I have explicitly stated my concerns

> I'm sure you're aware of Neal's "Tennent's Correspondence
> Principle" argument. It's a shame you neglected to
> comment on that here.

I have added a section

Thanks again for you comments - very constructive.

Howard Lovatt

Posts: 321
Nickname: hlovatt
Registered: Mar, 2003

Re: An Alternative to Closure Conversion and to Restricted Closures Posted: Jan 2, 2008 10:17 PM
Reply to this message Reply
@Ricky,

This post addresses the last point you made in your post.

> Imagine that foreach did not exist already.
> ...
> I think a simple
> rule to make most complaints about non-local return
> disappear is that a non-void closure should not be able to
> use return.

The problem is inferring the return type if you don't have an interface that the closure is assigned to:

int outer() {
  { => return 1; }.invoke();
}


Is the closure an int or a void?

Another option is to say that non-local return etc. can only be used in control blocks (which use a keyword - e.g. for to mark them as special), the FCM extension proposes this solution (as I am sure you know - are you one of the proposers of this?)

Charles H

Posts: 5
Nickname: chhum
Registered: Feb, 2007

Re: An Alternative to Closure Conversion and to Restricted Closures Posted: Jan 3, 2008 6:20 AM
Reply to this message Reply
This seems as good as place as any to post this.

I've been trying to work out why the only closure proposal I like is CICE/ARM and I think it comes down to syntax. The reason almost all my development these days is done in Java is because I find the language profoundly productive. A lot of this comes down to the fact that the code is easy to write, easy to read and inherently predictable. There is one exception to this which is the generics syntax introduced as part of Java 5. I don't get it, and suspect I never will though I do find this a bit of an embarrassing admission. Closures suffers from the same problem. I look at example code and feel like I'm coming out in a rash. I just don't know what it does – it isn't obvious, in the way that Java generally is, and therefore isn't consistent or predictable, it feels wrong in some way I can't easily define.

I think in general people make a mistake when they compare Java to C#. C# is much closer to Scala in terms of its audience than it is to Java. Visual Basic would be a much better point of comparison since both are roughly equally wodely used languages (research my company has done would suggest that C# is a similar audience size to Ruby - about 500,000 developers. Not small by any means but roughly 1/12 of the number of developers using either Java or VB). What is interesting about this was that I was looking at the way closures where achieved in Visual Basic 9 and it seemed much more syntactically consistent with VB than any of the Java closure proposals (CICE excluded) seem to be with Java. So what I'd really like someone who gets this stuff more than I do to do would be to either
a) Explain to me why the VB9 implementation is inadequate for the purposes of Java or
b) Have a go at coming up with a syntax that is more like the VB one. And for this I would start with CICE/ARM and see if it can be extended to cover the use cases that you consider vital rather than starting again.

A very quick example from http://jroller.com/scolebourne/entry/java_7_comparing_closure_proposals

BGGA:
public void init(JButton button) {
button.addActionListener({ActionEvent ev => handleButtonPress(ev);
});
}

FCM:
public void init(JButton button) {
button.addActionListener(#(ActionEvent ev) {
handleButtonPress(ev);
});
}

CICE:
public void init(JButton button) {
button.addActionListener(ActionListener(ActionEvent ev) {
handleButtonPress(ev); // a method in the class
});
}

ug!

Roughly what I'd like:
button.addActionListener(handleButtonPress);

I would do this myself but I'm not a language designer by any stretch of the imagination as you can probably tell...

Jesse Kuhnert

Posts: 24
Nickname: jkuhnert
Registered: Aug, 2006

Re: An Alternative to Closure Conversion and to Restricted Closures Posted: Jan 3, 2008 8:13 AM
Reply to this message Reply
Just to be sure people had an alternate understanding of closures from languages that are currently using them and not proposing them:

http://ducktyper.com/2007/12/28/closures

> A very quick example from
> http://jroller.com/scolebourne/entry/java_7_comparing_closu
> re_proposals
>

Carson Gross

Posts: 153
Nickname: cgross
Registered: Oct, 2006

Re: An Alternative to Closure Conversion and to Restricted Closures Posted: Jan 3, 2008 5:24 PM
Reply to this message Reply
Just for the record, despite the fact that Howard said it couldn't work, the following GScript works beautifully:



var strings = new ArrayList<String>(){"abc", "ab", "a"}
var sortedStrings = strings.sortBy( \ s -> s.length )



All the symbol types are inferred, type safe, and support good ol' code completion.

Hmmmm.

Cheers,
Carson

Howard Lovatt

Posts: 321
Nickname: hlovatt
Registered: Mar, 2003

Re: An Alternative to Closure Conversion and to Restricted Closures Posted: Jan 3, 2008 6:28 PM
Reply to this message Reply
@Charles & Jesse,

The examples you give (Jesse from duck_typer) in my alternative closures proposal C3S:

http://www.artima.com/weblogs/viewpost.jsp?thread=182412

are:
final list = ArrayList.new( 1, 2, 3 ); 
reduce list, method( x1, x2 ) { x1 + x2 }; 
 
with lock, method { out.println "Hello" }; 
 
with File.open( "foo.txt", "w" ), method( file ) { doSomething };
 
button.addActionListener method( ev ) { handleButtonPress ev };

Howard Lovatt

Posts: 321
Nickname: hlovatt
Registered: Mar, 2003

Re: An Alternative to Closure Conversion and to Restricted Closures Posted: Jan 3, 2008 6:42 PM
Reply to this message Reply
@Carson,

In C3S (http://www.artima.com/weblogs/viewpost.jsp?thread=182412):
final strings = ArrayList.new "abc", "ab", "a";
final sortedStrings = strings.sortBy method( s ) { s.length };

Collin Fagan

Posts: 2
Nickname: aberrant
Registered: Jan, 2008

Re: An Alternative to Closure Conversion and to Restricted Closures Posted: Jan 3, 2008 11:10 PM
Reply to this message Reply
Wait isn't FCM's second syntax the kind of thing you are looking for?

button.addActionListener(this#handleButtonPress(ActionEvent));

I'm sure one could even remove the explicit "this" and have just the following.

button.addActionListener(#handleButtonPress(ActionEvent));

Is that close enough to the VB9 syntax? I haven't had a chance to do any VB since VB6.

Howard Lovatt

Posts: 321
Nickname: hlovatt
Registered: Mar, 2003

Re: An Alternative to Closure Conversion and to Restricted Closures Posted: Jan 3, 2008 11:33 PM
Reply to this message Reply
@Collin,

I haven't used VB9 but looking at examples wouldn't VB9 be more like:

button.addActionListener( Function( ByVal ev As ActionEvent ) handleButtonPress( ev ) )

Collin Fagan

Posts: 2
Nickname: aberrant
Registered: Jan, 2008

Re: An Alternative to Closure Conversion and to Restricted Closures Posted: Jan 3, 2008 11:39 PM
Reply to this message Reply
Thanks, I was trying to respond to Charles H. I was unaware that posts are flat and do not nest. Good article by the way.

Flat View: This topic has 30 replies on 3 pages [ 1  2  3 | » ]
Topic: An Alternative to Closure Conversion and to Restricted Closures Previous Topic   Next Topic Topic: Looking for Memories of Python Old-Timers


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us