The Artima Developer Community
Sponsored Link

Weblogs Forum
A Hazard of Covariant Return Types and Bridge Methods

4 replies on 1 page. Most recent reply: Jan 10, 2019 12:14 PM by Brian Goetz

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 4 replies on 1 page
Ian Robertson

Posts: 68
Nickname: ianr
Registered: Apr, 2007

A Hazard of Covariant Return Types and Bridge Methods (View in Weblogs)
Posted: Sep 25, 2013 10:25 AM
Reply to this message Reply
Summary
A combination of bridge methods, covariant return types and dynamic dispatch can lead to some surprising and unfortunate results.
Advertisement

A Hazard of Covariant Return Types and Bridge Methods

This week at JavaOne, Joe Darcy pointed out to me an interesting difficulty he ran into recently when trying to change various JDK classes to use covariant return types for clone(). It turns out that changing the return type of an overridden method on a class to be more specific can break behavior compatibility for child classes which themselves had already created an override of the same method. The original discussion is on the OpenJdk Core Libs Dev mailing list. To try to make it a touch easier to follow, I'll give a bit of background, and construct a stand-alone example.

Method invocation in the JVM

The Java language (as most OO languages) offers some flexibility when making method calls. Given the following class[1]:

import java.util.*;

public class Wrapper {
  public Collection wrap(Object o) {
    List l = new ArrayList();
    l.add(o);
    return l;
  }
}
it is possible to call this method as follows:
public class WrapperClient {
  public static void main(String args[]) {
    Object wrappped = new Wrapper().wrap("x");
  }
}

This is the Liskov Substitution Principal in action. While wrap is expecting an Object, it's fine to pass in a subtype of Object (in this case, String). Similarly, while the return type of wrapped is of type Object, it's fine to assign a subtype of Object to it. As we know, Java respects this, and makes it work. What is not as well known is exactly how Java makes this work.

To see what is happening under the hood, we need to understand how method calls look in the Java Virtual Machine. Rather than show disassembled byte code the way javap -c would, I'll introduce a bit of pseudocode syntax. When a method is called, I'm going to add the precise signature of the method after the method name, in brackets. In this pseudo code, the above call to wrap now looks like:

Object wrapped = new Wrapper().wrap[Object->Collection]("x");

The JVM honors Liskov Substution, in the sense that it is perfectly willing to invoke wrap[Object->Collection], even thought the type being passed in is not Object, but a subtype of Object. Similarly, it's willing to assign the result of this method invocation (declared to be a Collection) to a variable of type Object, a supertype of Collection.

There's a subtle issue here which is easy to overlook. While the JVM honors Liskov Substution perfectly well when it comes to accepting subtypes of what a method call or assignment operator expects, it will not look for method signatures that would work in the place of the signature asked for. To see this in action, recompile Wrapper.java with the following source code:

public class Wrapper {
  public Collection wrap(String o) {
    Collection c = new ArrayList();
    c.add(o);
    return c;
  }
}

If WrapperClient is run again against the newly compiled Wrapper class without also recompiling WrapperClient.java, a NoSuchMethodError will be thrown. This is because Wrapper is looking for a method with signature wrap[Object->Collection], but the only wrap method present in Wrapper is wrap[String->Collection]. While this would be an acceptable substitution, the JVM will not make it for us.

Covariant returns and Bridge methods

Prior to Java 5, overrides of a method in a subclass could not change the return type of the method. If WrapperChild were to extend Wrapper, it would only have one option for the return type of wrapper, namely Collection:

import java.util.*;
public class WrapperChild extends Wrapper {
  @Override
  public Collection wrap(Object o) {
    return super.wrap(o);
  }
}

Starting in Java 5, support for covariant return types was introduced added. This means that we can now subclass WrapperChild, and override the wrap method to return a type more specific than Collection:

import java.util.*;
public class WrapperGrandchild extends WrapperChild {
  @Override
  public List wrap(Object o) {
    return (List) (super.wrap(o));
  }
}
Suppose we have code calling wrap on a variable declared to be of type Wrapper which is actually of type WrapperGrandchild. How does Java avoid a NoSuchMethodError? It turns out that the work is done not by the JVM, but by javac. When WrapperGrandchild is compiled, a bridge method is created with the signature of the parent wrap method which forwards to the new wrap method. The resulting class looks like:
import java.util.*;
public class WrapperGrandchild extends WrapperChild {
  public List wrap(Object o) {
    return (List) (super.wrap[Object->Collection](o));
  }

  // bridge method created by javac
  public Collection wrap(Object o) {
    return this.wrap[Object->List].wrap(o));
  }
}

Thus, a client which has an instance with declared type WrapperChild, but actual type WrapperGrandchild, can still successfully invoke WrapperGrandchild.wrap[Object->Collection].

The Trap

Now that we understand covariant overrides and bridge methods, we can understand the problem that Joe ran into. Suppose that while Wrapper and WrapperChild are distributed in the same jar, WrapperGrandchild is distributed in a separate jar which has a separate release schedule. Suppose further that the maintainer of WrapperChild decides to change the signature of its wrap method to return List instead of Collection. Because the original Wrapper class still is defining wrap to return Collection, WrapperChild.class must now contain a bridge method:

import java.util.*;
public class WrapperChild extends Wrapper {
  public List wrap(Object o) {
    return (List) (super.wrap[Object->Collection](o));
  }

  // bridge method created by javac
  public Collection wrap(Object o) {
    return this.wrap[Object->List].wrap(o));
  }
}

Suppose that WrapperGrandchild is not recompiled against the new version of WrapperChild. Consider what happens if someone calls:

new WrapperGrandchild().wrap("x")
First, the wrap[Object->List] method on WrapperGrandchild is invoked. This calls super.wrap[Object->Collection] (the only signature that was available in the first version of WrapperChild). However, WrapperChild's wrap[Object->Collection] method is now a bridge method that forwards to wrap[Object->List]. Due to dynamic dispatch, the most specific override of wrap[Object->List] is invoked. Unfortunately, this is the orginal wrap[Object->List] method on WrapperGrandchild that we first called! We now have an infinite loop (or more precisely, a StackOverflowError). The combination of bridge methods, dynamic dispatch and partial recompilation has led us into a corner.

The good news is that this is a rather obscure edge case that most of us won't hit in practice. It requires three levels of inheritance for a method, with each child invoking super. It also requires a very specific combination of covariant return types at each inheritance level, and a specifc sequence of releases. The danger remains, however, especially for environments with multiple layers of dependencies which evolve on different time lines, or for widely used libraries (such as the core JDK libraries).

  1. Of course the code here should properly use generics. I've ommitted them for conciseness, and to avoid causing the impression that the issue described here is related to generics


Valery Silaev

Posts: 5
Nickname: vsilaev
Registered: Feb, 2007

Re: A Hazard of Covariant Return Types and Bridge Methods Posted: Sep 27, 2013 12:37 PM
Reply to this message Reply
Another issue with bridge methods is that they don't inherit annotations of the original methods. There is no mechanism to say that an annotation must be propagated to the bridge method, there is no easy way even to find a correspondence between original<->bridge methods neither via byte code attributes nor via the reflection API. Curious readers may check Spring sources - what a deal of the non-trivial code is necessary to "reconstruct" annotations info for bridge method.

Ian Robertson

Posts: 68
Nickname: ianr
Registered: Apr, 2007

Re: A Hazard of Covariant Return Types and Bridge Methods Posted: Sep 30, 2013 9:04 AM
Reply to this message Reply
> Another issue with bridge methods is that they don't
> inherit annotations of the original methods. There is no
> mechanism to say that an annotation must be propagated to
> the bridge method, there is no easy way even to find a
> correspondence between original<->bridge methods neither
> via byte code attributes nor via the reflection API.
> Curious readers may check Spring sources - what a deal of
> the non-trivial code is necessary to "reconstruct"
> annotations info for bridge method.

Intrestingly, it seems that the implementation of getMethod(String name, Class<?>... parameterTypes) in java.lang.Class has to do some similar hoop-jumping.

kill dill

Posts: 1
Nickname: killdill04
Registered: Nov, 2014

Re: A Hazard of Covariant Return Types and Bridge Methods Posted: Nov 21, 2014 1:40 AM
Reply to this message Reply
Due to dynamic dispatch, the most specific override of wrap[Object->List] is invoked. Unfortunately, this is the orginal wrap[Object->List] method on WrapperGrandchild that we first called! We now have an infinite loop (or more precisely, a StackOverflowError). The combination of bridge methods, dynamic dispatch and partial recompilation has led us into a corner.??





_______________________
GuL

Brian Goetz

Posts: 6
Nickname: briangoetz
Registered: Sep, 2005

Re: A Hazard of Covariant Return Types and Bridge Methods Posted: Jan 10, 2019 12:14 PM
Reply to this message Reply
Here's a simpler example:

Start with this:

class Parent implements Cloneable {
protected Object clone() { return (Parent)null; }
}

class Child extends Parent {
protected Parent clone() { return (Parent)super.clone(); }
}


Then you change Parent to this:

class Parent implements Cloneable {
protected Parent clone() { return (Parent)null; }
}

and recompile only Parent.

Then you call clone on child and you get a loop.

Flat View: This topic has 4 replies on 1 page
Topic: The Myth of Paradigms and TMTOWTDI Previous Topic   Next Topic Topic: The fate of reduce() in Python 3000

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use