The Artima Developer Community
Sponsored Link

Weblogs Forum
Lexical Scope of Java Inner Classes and Closures

23 replies on 2 pages. Most recent reply: Jun 24, 2007 4:51 AM by Jules Jacobs

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 23 replies on 2 pages [ 1 2 | » ]
Howard Lovatt

Posts: 321
Nickname: hlovatt
Registered: Mar, 2003

Lexical Scope of Java Inner Classes and Closures (View in Weblogs)
Posted: May 26, 2007 9:40 PM
Reply to this message Reply
Summary
One of the key differences between the different Java inner class and closure proposals is the lexical scope of names, this blog compares the scoping of the major proposals (CICE + ARM, C3S, FCM + JCA, and BGGA in both its forms)
Advertisement
Lexical Scope of Inner Classes and Closures

One of the key differences between the different Java inner class and closure proposals is the lexical scope of names, this blog compares the scoping of the major proposals (CICE + ARM, C3S, FCM + JCA, and BGGA in both its forms). The order in the comparison table below, from left to right, is how large a change is made to Java. There are two main change groups: CICE and C3S do not change the semantics of inner class methods, whereas FCM and BGGA do. All the proposals add syntactic changes to make the declaration of inner classes shorter. The comparison is similar to that given by Stephen Colebourne in his blog, but includes the inherited scope as well as the enclosing scope. (Stephen Colebourne was going to post the table below, or at least a similar one, on his blog but was travelling at the time.)

A more detailed comparison was given in a previous post, but the table below is a good summary of an important difference (lexical scope).

Property

Inner class

CICE

ARM Extension to CICE

C3S

FCM

JCA Extensions to FCM

Restricted Version of BGGA Closure Conversions

BGGA

Access to names from enclosing scope

Y

(Local variables must be final)

Y

(Local variables must be public to give write access)

Y

Y

Y

Y

Y

(Local variables must be final)

Y

Access to names from inherited classes/interfaces

Y

Y

N

(No inheritance)

Y

N

N

N

N

Inherited names hide enclosing names

Y

Y

N

(No inheritance)

Y

N

N

N

N

Break and continue into the enclosing scope from within the inner class/closure method

N

N

Y

(No inheritance)

Y

(Must be a labelled break or continue to break or continue into the enclosing method)

N

Y

N

Y

Return from the enclosing method (as opposed to returning from the inner class/closure method)

N

N

Y

(No inheritance)

Y

(Must be qualified with method name to return from the enclosing method)

N

Y

N

(Can only return via going of the end of the code block – no return statement allowed)

Y

(Return statement returns from the enclosing method and going of the end of a code block returns from the inner class/closure method)


Jules Jacobs

Posts: 119
Nickname: jules2
Registered: Mar, 2006

Re: Lexical Scope of Java Inner Classes and Closures Posted: Jun 1, 2007 10:49 AM
Reply to this message Reply
Are there any restrictions on "Break and continue into the enclosing scope from within the inner class/closure method" and "Return from the enclosing method (as opposed to returning from the inner class/closure method)" in the proposals with an y in those columns? Any sane proposal does have restrictions, because it's very hard to implement a return from a method that's no longer active.

For example (in Javascript syntax):

function test(){
  return function(){ 
    return 4 // return from the anonymous function or from test?
  }
}


For example, what happens if we do this:

func = test();
func();


Clearly, if the return in the anonymous function returns from the enclosing function (test), this should work like this:

1. the test() function returns the anonymous function
2. we call the anonymous function
3. we execute the anonymous function's return, returning from test *again*!
4. now we're back at line 1, test returns 4 now
5. the next line produces an error because 4 is not a function

It's strange to return from functions twice, but it possible in some languages (Scheme and Ruby, for example) by using first-class continuations:

def test
  callcc {|cc| return cc}
  return 4
end
 
func = test()
func.call()


And we can do things like:

def mystery(n)
  callcc do |cc| 
    return n > 0 ? cc : lambda{}, n
  end
  
  mystery(n-1)
end
 
go, n = mystery(5)
print n
go.call


So, is it possible, according to one of these proposals, to do these things with returns from inner classes/closures?

Howard Lovatt

Posts: 321
Nickname: hlovatt
Registered: Mar, 2003

Re: Lexical Scope of Java Inner Classes and Closures Posted: Jun 1, 2007 5:58 PM
Reply to this message Reply
As people say when someone asks a difficult question - good question.

The proposals behave differently. In C3S syntax a modification of your example would be:
method boolean test() {
  return !method { return false }
}

And it would return true, i.e the inner return is from from the inner method (like inner classes). However:
method boolean test() {
  return !method { test.return false }
}

Would return false since the inner return is qualified with the method to return from. The above code is translated into:
method boolean test() {
  class Test$ReturnException extends ReturnException {
    Test$ReturnException( Boolean value ) { super( value ) }
  }
  try {
    return !method { throw new Test$ReturnException( false ) }
  } catch ( Test$ReturnException e ) {
    return (boolean)e.getValue()
  }
}

You can pass the anonymous method to another method but that other method must throw ReturnException which is a checked exception.

With other proposals things are a bit different. With CICE and FCM you always return from the inner method. With ARM and JCA you always return from the outer method. With BGGA returning from the inner method is:
boolean test() {
  return !{=> false }
}

Outer method:
boolean test() {
  return !{=> return false; }
}

Both
boolean test() {
  return !{=> false; }
}

and
boolean test() {
  return !{=> return false }
}

are syntax errors in BGGA. With BGGA if you pass the annonymous method to another method then it is unchecked so you might get a runtime exception if you execute it in the wrong context. Unlike C3S this passing of an anonymous function in BGGA isn't checked by the compiler (in C3S it is a checked exception).

Note the return from the outer method is via an exception in BGGA, JCA, and C3S but not ARM. In ARM the annonymous method isn't a method but a block like a block in a for loop etc.

Jules Jacobs

Posts: 119
Nickname: jules2
Registered: Mar, 2006

Re: Lexical Scope of Java Inner Classes and Closures Posted: Jun 2, 2007 2:56 AM
Reply to this message Reply
I'm not sure if I understand this:

return !method{ return true }


Does this code type-check? method{ return true } is a method object, right? So !method{ return true } would be an error because method{ return true } is not a boolean. Or does method{ return true } run the code inside the brackets? Don't we need a .call() somewhere as in:

x = method{ return true }; // a closure object
x.call(); // true


If the code between the brackets is executed immediately (i.e. we don't need to .call()), what's the point of putting the code in a method{ ... }?

The example in Javascript:

function test(){
  return !function(){ return false }
}


This doesn't work because we need to call the anonymous function like this:

function test(){
  return !function(){ return false }() // note the two parens
}


Or am I completely wrong and is the ! sytax for calling a method?

Howard Lovatt

Posts: 321
Nickname: hlovatt
Registered: Mar, 2003

Re: Lexical Scope of Java Inner Classes and Closures Posted: Jun 2, 2007 5:40 PM
Reply to this message Reply
Oops you are right my example should have used call() - sorry. Here are some alternatives:
static boolean test() {
  return !method { return false }.call()
}
 
or
 
static boolean test() {
  return not method { return false }
}
 
where not is:
 
static Boolean not( Predicate p ) {
  return !p.call()
}

Jules Jacobs

Posts: 119
Nickname: jules2
Registered: Mar, 2006

Re: Lexical Scope of Java Inner Classes and Closures Posted: Jun 3, 2007 1:11 PM
Reply to this message Reply
OK, but then you didn't answer my question. My question is far more evil than that ;-)

method boolean test() {
  return method { test.return 4 }
}
 
x = test(); // returns a Closure object
x.call(); // what happens here?


When we do x.call(), the call to test has already returned (on the first line, it returned a Closure object). The code says test.return 4, so when we do x.call(), we should return 4 from test(), not from x.call(). So do we jump back to the first line, returning 4 this time? (this is what happens if we use first class continuations, like the Ruby example above).

Another way to say this is: What is the output of this:

method boolean test() {
  return method { test.return 4 }
}
 
x = test();
System.out.print("foo"); 
x.call();


The output could be "ERROR!" or "foofoo", because we return twice from test(). What do these proposals say about this?

Howard Lovatt

Posts: 321
Nickname: hlovatt
Registered: Mar, 2003

Re: Lexical Scope of Java Inner Classes and Closures Posted: Jun 3, 2007 6:25 PM
Reply to this message Reply
> OK, but then you didn't answer my question. My question is
> far more evil than that ;-)

As I said - a good question :). The short answer is that it behaves just like current inner classes would; it is just syntax sugar, no new semantics for inner classes/closures. So no callcc like behaviour, you would use an object with state for this (a state machine).

>
method boolean test() {
>   return method { test.return 4 }
> }
> 
> x = test(); // returns a Closure object
> x.call(); // what happens here?


You can't write the example quite like that because everything is an object (not a function), the functions (call methods) are instances of classes with a method called call. You could write:

final test = Method0< Method0< Integer > >.method { // instance of an anonymous class
  return Method0< Integer >.method { return 4 } // instance of a second anonymous class
}
 
final x = test.call; // returns an instance of the second class (calls the class's constructor)
out.println x.call; // prints 4


But that is probably not what you were driving at since there is only a return from the inner methods, no non-local return.

Maybe this is closer to what you are trying to demonstrate:

int test() { // normal method
  return increment method { test.return 4 } // make new instance and execute when test is called
}
 
out.println test; // prints 4


But again the behaviour is not the same as your example. As I said I would code callcc with the State Machine design pattern - just like you would at present in Java.

Assuming there is only one instance of the state machine required then enum is convenient:

enum FlipFlop implements Block {
  FLIP {
    method {
      out.print this;
      state = FLOP
    }
  },
  FLOP {
    method {
      out.print this;
      state = FLIP
    }
  };
  private static declare state = FLIP;
  public method void callcc() { state.call }
}

Jules Jacobs

Posts: 119
Nickname: jules2
Registered: Mar, 2006

Re: Lexical Scope of Java Inner Classes and Closures Posted: Jun 4, 2007 8:28 AM
Reply to this message Reply
>
> final test = Method0< Method0< Integer > >.method { //
> instance of an anonymous class
> return Method0< Integer >.method { return 4 } //
> // instance of a second anonymous class
> }
> 
> final x = test.call; // returns an instance of the second
> class (calls the class's constructor)
> out.println x.call; // prints 4
> 


OK, so what happens if we do test.return instead of return:

>
> final test = Method0< Method0< Integer > >.method { //
> instance of an anonymous class
> return Method0< Integer >.method { test.return 4 } //
> // instance of a second anonymous class
> }
> 
> final x = test.call; // returns an instance of the second
> class (calls the class's constructor)
> out.println x.call; // prints 4
> 


This doesn't work, of course, but I hope you get the idea. I've read some of the proposals, but I either don't understand them or they don't specify what happens if you do this, using the proposed closure syntax.

This is what happens in Common Lisp:

(defun test () (lambda () (return-from test 4)))
(funcall (test))
ERROR: attempt to RETURN-FROM a block or GO to a tag that no longer exists


>
> int test() { // normal method
> return increment method { test.return 4 } // make new
> ew instance and execute when test is called
> }
> 
> out.println test; // prints 4
> 


What does increment do?

> But again the behaviour is not the same as your example.

Yes, the (big) difference is that test() executes the anonymous method instead of returning it.

> As I said I would code callcc with the State Machine
> design pattern - just like you would at present in Java.
>
> Assuming there is only one instance of the state machine
> required then enum is convenient:
>
>
> enum FlipFlop implements Block {
>   FLIP {
>     method {
>       out.print this;
>       state = FLOP
>     }
>   },
>   FLOP {
>     method {
>       out.print this;
>       state = FLIP
>     }
>   };
>   private static declare state = FLIP;
>   public method void callcc() { state.call }
> }
> 


I don't think that's like callcc at all...maybe you're trying to emulate coroutines? (but it isn't like coroutines either).

Howard Lovatt

Posts: 321
Nickname: hlovatt
Registered: Mar, 2003

Re: Lexical Scope of Java Inner Classes and Closures Posted: Jun 4, 2007 1:41 PM
Reply to this message Reply
> >
> > final test = Method0< Method0< Integer > >.method { //
> > instance of an anonymous class
> OK, so what happens if we do test.return instead of
> return:
> 
> > [java]
> > final test = Method0< Method0< Integer > >.method { //
> > instance of an anonymous class
> > return Method0< Integer >.method { test.return 4 } //
> > // instance of a second anonymous class
> > }
> > 
> > final x = test.call; // returns an instance of the
> second
> > class (calls the class's constructor)
> > out.println x.call; // prints 4
> > 


This is a syntax error for two reasons:

1. There is no method called test - there is an object called test with a method called call

2. test.call returns a Method0< Integer > not an int

> >
> > int test() { // normal method
> > return increment method { test.return 4 } // make new
> > ew instance and execute when test is called
> > }
> > 
> > out.println test; // prints 4
> > 

>
> What does increment do?

For the purpose of the example, it is any method to cause the anonymous class instance to be created and to cause its call method to be invoked.

I think the enum example does behave like a callcc. Suppose that we had yield in Java then the example could be:

void callcc() {
  for (;;) {
    out.println( "FLIP" );
    yield;
    out.println( "FLOP" );
    yield;
  }
}

Jules Jacobs

Posts: 119
Nickname: jules2
Registered: Mar, 2006

Re: Lexical Scope of Java Inner Classes and Closures Posted: Jun 6, 2007 6:09 AM
Reply to this message Reply
> This is a syntax error for two reasons:
>
> 1. There is no method called test - there is an object
> called test with a method called call
>
> 2. test.call returns a Method0< Integer > not an int

OK, ok, but this is not important. What's important is the return from a closure to the enclosing function. Suppose we had a normal test() method, and when we call this method it puts this function in an instance variable: method{ test.return 4 }. Then, later, in another method we do the_instance_variable.call(). It still isn't clear to me what will happen.

> For the purpose of the example, it is any method to cause
> the anonymous class instance to be created and to cause
> its call method to be invoked.

Yes, but the point is that the call() method is *not* called in test(), but later.

> I think the enum example does behave like a callcc.
> Suppose that we had yield in Java then the example could
> be:
>
>
> void callcc() {
>   for (;;) {
>     out.println( "FLIP" );
>     yield;
>     out.println( "FLOP" );
>     yield;
>   }
> }
> 


What does your yield() do? Many programming languages have yield, and they do different things (eg. in Lua it's for coroutines and in Ruby it executes a block). I fail to see how this is in any way like callcc. The yield() keyword/function can be implemented using callcc (like other control flow constructs), but not the other way around.

Howard Lovatt

Posts: 321
Nickname: hlovatt
Registered: Mar, 2003

Re: Lexical Scope of Java Inner Classes and Closures Posted: Jun 6, 2007 5:59 PM
Reply to this message Reply
@Jules,

Thanks for keeping on asking questions I find it very helpful to explain ideas - it focuses my thoughts. We are discussing two separate examples, first what happens if you store the inner class and then use a non-local return:

method void test() {
  final thread = Thread.method run() { test.return }; // **Compile** time error, Thread.run does not throw test$ReturnException
  thread.start
}


The equivalent example in BGGA would not be caught by the compiler - instead a RuntimeException would be generated.

The second example is callcc, is this closer to what you had in mind:

static method< Throwable... Es > void callcc( final Block1< Block< Es >, Es > block, final Block< Es > continuation ) {
  block.call continuation;
}
 
callcc method( continuation ) {
  out.println "Before";
  continuation.call;
  out.println "Not reached"
}, method {
  callcc.return
};

Howard Lovatt

Posts: 321
Nickname: hlovatt
Registered: Mar, 2003

Re: Lexical Scope of Java Inner Classes and Closures Posted: Jun 7, 2007 7:05 PM
Reply to this message Reply
@Jules,

Hope you are still listning because I have a question for you, would the Scheme/Ruby equivalent of this behave as indicated below:

callcc method( continuation ) {
  out.println "Outer";
  callcc method( continuation ) {
    out.println "Inner";
    continuation.call;
    out.println "Not reached"
  }, continuation;
  out.println "Back to outer"
}, method {
  callcc.return
};


In particular, there is a recursive call to callcc and the continuation exits the inner only, not the whole lot. Is that what other languages would do?

Jules Jacobs

Posts: 119
Nickname: jules2
Registered: Mar, 2006

Re: Lexical Scope of Java Inner Classes and Closures Posted: Jun 10, 2007 12:25 PM
Reply to this message Reply
> @Jules,
>
> Hope you are still listning because I have a question for
> you, would the Scheme/Ruby equivalent of this behave as
> indicated below:
>
>
> callcc method( continuation ) {
>   out.println "Outer";
>   callcc method( continuation ) {
>     out.println "Inner";
>     continuation.call;
>     out.println "Not reached"
>   }, continuation;
>   out.println "Back to outer"
> }, method {
>   callcc.return
> };
> 

>
> In particular, there is a recursive call to callcc and the
> continuation exits the inner only, not the whole lot. Is
> that what other languages would do?

Well, callcc takes only one parameter.

callcc do |continuation|
  print "Outer"
  callcc do |continuation|
    print "Inner"
    continuation.call
    print "Not reached"
  end
  print "Back to outer"
end


This should print Outer, Inner, Back to outer, so the continuation.call jumps to the end of the inner block.

However, your exception-based callcc is far less powerful than a real callcc, because you can only go up the stack using exceptions. You can also jump back to functions that have already returned using callcc.

A continuation contains local variables (like a closure), and the return stack. "return"-stack is a wrong name because the return stack doesn't contain the places to return to, but the places to go next: it's a todo-list. This todo-list is saved in a continuation. When you call it, the current stack is replaced with the stack stored in the continuation. By calling a continuation we can replace the stack with a new stack, possible getting a bigger or completely different stack. With exceptions you can only go up the stack (removing items from it).

A way to think about it:

callcc do |return|
  // if we do return.call(...) we jump to
end  // <-- here, and we return the value passed to call(...)
 
// So:
 
x = callcc do |return|
  return.call(3)
end
 
// x = 3 now
 


And an infinite loop:

x = callcc do |return|
  return
end // x.call(something) goes to the end of this block, and puts something into x
 
puts "hi"
 
x.call(x) // go back to the end of the callcc block, putting x back into x again. 
// If we just used x.call(), then x would contain nil (null) on the next run, because x.call() is equivalent to x.call(nil).


Callcc() returns multiple times in this example (the return value is assigned to x). x.call() returns from the callcc() again. Yes, it's weird.

But is it clear?

Common Lisps condition system is interesting because it's more powerful than exceptions, and less powerful than continuations. When throwing an exception in CL we don't remove items from the stack yet: we leave them in place. This allows us to return to the point where we threw the exception in the exception handler. We cannot go back to a function that has already returned, however.

Howard Lovatt

Posts: 321
Nickname: hlovatt
Registered: Mar, 2003

Re: Lexical Scope of Java Inner Classes and Closures Posted: Jun 11, 2007 6:23 AM
Reply to this message Reply
@Jules,

Thanks for your reply


> callcc do |continuation|
>   print "Outer"
>   callcc do |continuation|
>     print "Inner"
>     continuation.call
>     print "Not reached"
>   end
>   print "Back to outer"
> end

>
> This should print Outer, Inner, Back to outer, so the
> continuation.call jumps to the end of the inner block.

That's good - I thought this would be the answer - the same as my example sans the extra argument. I think you can simulate all the other behaviour and note the inner classes fully close over the local variables so that aspect is also the same.

abstract class< T > Callcc {
  T value;
  private static class CallccReturn extends RuntimeException {}
  private static final premade = new CallccReturn();
  abstract method void first();
  method void exit() {}
  method void next() {}
  method void call() {
    try {
      first
    } catch ( CallccReturn notUsed ) {
      exit
    }
    next
  }
  method void callccReturn( final T value ) { 
    this.value = value;
    throw premade
  }
  method void callccReturn() { callccReturn null }
  method void callNext( final T value ) { 
    this.value = value;
    next
  }
  method void callNext() { repeatNext null }
}


The syntax is not quite the same, but similar. EG your examples:

final x = Callcc.method {
  callccReturn( 3 )
}
 
// x.value = 3 now
 


And an infinite loop:

final x = Callcc.new {
  method {}
  method next {
    out.println "hi"
    callNext
  }
}

Howard Lovatt

Posts: 321
Nickname: hlovatt
Registered: Mar, 2003

Re: Lexical Scope of Java Inner Classes and Closures Posted: Jun 12, 2007 8:19 PM
Reply to this message Reply
The above code was written late a night - here is a better version written is straight Java (i.e. I have tested this version):

public class Callcc< T > {
  private static class CallccReturn extends RuntimeException {}
  private static final CallccReturn preMade = new CallccReturn();
  public T value = null;
  public void first() { callccContinue(); }
  public void iteration() {}
  public void repeatable() {}
  public final void callccContinue( final T value ) {
    this.value = value;
    throw preMade;
  }
  public final void callccContinue() { callccContinue( null ); }
  public Callcc< T > call() {
    try {
      first();
    } catch ( CallccReturn notUsed ) {
      iteration();
    }
    for ( ;; ) {
      try {
        repeatable();
        return this;
      } catch ( CallccReturn notUsed ) {
        iteration();
      }
    }
  }
  @Override public String toString() { return "Callcc[" + value + "]"; }
}


And usage examples written in C3S but based on tested versions:

    // Return a value
    final x = Callcc< String >.new {
      method first { callccContinue "A" }
    }.call;
    out.println "x = " + x;
    
    // Close over a variable to return a result
    final is = String.new[ 3 ];
    Callcc< Void >.new {
      declare local = { "i", "ii", "iii" };
      method iteration { arraycopy local, 0, is, 0, local.length }
    }.call;
    out.println "is = " + Arrays.toString( is );
    
    // Repeat
    Callcc< Void >.new {
      int count = 1;
      method repeatable {
        out.println count;
        count++;
        if ( count <= 3 ) { callccContinue }
      }
    }.call;

Flat View: This topic has 23 replies on 2 pages [ 1  2 | » ]
Topic: Lexical Scope of Java Inner Classes and Closures Previous Topic   Next Topic Topic: Appreciative Inquiry

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use