Java Library Evolution Puzzlers
Upcoming SlideShare
Loading in...5
×
 

Java Library Evolution Puzzlers

on

  • 346 views

A collection of puzzlers demonstrating the difference between source and binary compatibility in Java.

A collection of puzzlers demonstrating the difference between source and binary compatibility in Java.

Statistics

Views

Total Views
346
Views on SlideShare
345
Embed Views
1

Actions

Likes
1
Downloads
1
Comments
0

1 Embed 1

https://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Java Library Evolution Puzzlers Java Library Evolution Puzzlers Presentation Transcript

  • Java Library Evolution Puzzlers Jens Dietrich1 1Massey University School of Engineering and Advanced Technology Palmerston North, New Zealand https://sites.google.com/site/jensdietrich/ Email: j.b.dietrich /at/ massey.ac.nz August 31, 2014 1
  • Revision History Revision Date Remarks 1.0 13 Sept 13 initial version 2.0 10 Feb 14 added bridge (synthetic methods generated by compiler) 3.0 14 Feb 14 added generics3 (changing the order of multiple type parameter bounds) 4.0 31 Aug 14 added static* (static vs non-static) 2
  • Table of Contents Introduction Modifying Interfaces Modifying Method Signatures Static vs Non-Static Primitive vs Wrapper Types Using Generic Parameter Types Changing the Values of Constants Modifying Exceptions 3
  • Introduction - Deploying Java Programs I Java programs are usually built (ant,maven,gradle,..) with all libraries they use, and then deployed I if the program or a library changes, the program is rebuilt and redeployed I the build step includes V&V: compiling and (automated regression) testing I partial library upgrades are becoming more and more popular, example: OSGi bundle updates I the puzzlers described here show the di erence between these two deployment modes 4
  • Introduction - Source vs Binary Compatibility I a program is source compatible with a library lib.jar if the program uses the library, and compilation succeeds: javac -cp ..,lib.jar,.. ... I source compatibility is checked by the compiler, incompatibility results in compilation errors I a program is binary compatible with a library lib.jar if it links and runs with this library: java -cp ..,lib.jar,.. .. [JLS, ch. 13] I binary compatibility is checked by the JVM, incompatibility results in (linkage) errors 5
  • The Limitations of Binary Compatibility I in the JLS, a very narrow de
  • nition of binary (in)compatibility is used: A change to a type is binary compatible with pre-existing binaries if pre-existing binaries that previously linked without error will continue to link without error." [JLS, ch. 13.2] I binary compatibility is de
  • ned w.r.t. to what the linker can detect by means of static analysis, failure results in errors (not exceptions) 6
  • Introduction - Evolution Problems I assume that a program references code de
  • ned in lib-1.0.jar I assume that the program can be compiled successfully and can be executed with lib-1.0.jar without causing an error or exception I then the library evolves to lib-2.0.jar 7
  • Introduction - Evolution Problems Questions: 1. does the program link with lib-2.0.jar - i.e., is it binary compatible with lib-2.0.jar? 2. does replacing the library change the behaviour of the program - i.e., is the library change binary behavioural compatible? 3. does the program compile against lib-2.0.jar - i.e., is it source compatible with lib-2.0.jar? 4. does recompiling the program against the changed library change the behaviour of the program - i.e., is the library change source behavioural compatible? 8
  • Introduction - Running Experiments I check out code: hg clone https://bitbucket.org/jensdietrich/ java-library-evolution-puzzlers I each example has a program with a main class aPackage.Main, and two versions of classes de
  • ned in a separate library I cd to folder and run ant as follows: ant -Dpackage=aPackage I this will do the following: 1. compile the two versions of the library and build lib-1.0.jar and lib-2.0.jar 2. compile and run the program with lib-1.0.jar 3. compile the program with lib-1.0.jar , but run it with lib-2.0.jar 4. re-compile and run the program with lib-2.0.jar 9
  • Adding a Method to an Interface lib-1.0.jar package lib.addtointerface; public interface Foo f public void foo(); g + lib-2.0.jar package lib.addtointerface; public interface Foo f public void foo(); public void bar(); g program package addtointerface; import lib.addtointerface.; public class Main implements Foo f @Override public void foo() f System.out.println("foo"); g public static void main(String[] args) f new Main().foo(); g g I the interface Foo is extended by adding bar() I but the client class implements the old interface I is this still binary compatible with lib-2.0.jar? 10
  • Adding a Method to an Interface Solution I running the program with library version 2.0 succeeds - the client program is not using the method added to the interface! I i.e., the program (compiled with lib-1.0.jar) is binary compatible with lib-2.0.jar I but recompilation fails as Main does not implement bar() I i.e., the program is source incompatible with lib-2.0.jar 11
  • Removing a Method from an Interface 1 lib-1.0.jar package lib.removefrominterface1; public interface Foo f public void foo(); public void bar(); g + lib-2.0.jar package lib.removefrominterface1; public interface Foo f public void foo(); g program package removefrominterface1; import lib.removefrominterface1.; public class Main implements Foo f @Override public void foo() f System.out.println("foo"); g @Override public void bar() f System.out.println("bar"); g public static void main(String[] args) f new Main().foo(); new Main().bar(); g g I the method bar() is removed from the interface Foo I but the client class implements the old interface 12
  • Removing a Method from an Interface 1 Solution I running the program with library version 2.0 succeeds ! I i.e., the program (compiled with lib-1.0.jar) is binary compatible with lib-2.0.jar I but recompilation fails as Main.bar() does not override a method! I i.e., the program is source incompatible with lib-2.0.jar 13
  • Removing a Method from an Interface 2 lib-1.0.jar package lib.removefrominterface2; public interface Foo f public void foo(); public void bar(); g + lib-2.0.jar package lib.removefrominterface2; public interface Foo f public void foo(); g program package removefrominterface2; import lib.removefrominterface2.; public class Main implements Foo f public void foo() f System.out.println("foo"); g public void bar() f System.out.println("bar"); g public static void main(String[] args) f new Main().foo(); new Main().bar(); g g I this is almost identical to the previous example I but this time the @Override annotation is not used 14
  • Removing a Method from an Interface 2 Solution I as before, the program (compiled with lib-1.0.jar) is binary compatible with lib-2.0.jar I but recompilation also succeeds as the compiler does not check whether Main.bar() overrides a method I i.e., the program is also source compatible with lib-2.0.jar 15
  • Removing a Method from an Interface 3 lib-1.0.jar package lib.removefrominterface3; public interface Foo f public void foo(); public void bar(); g + lib-2.0.jar package lib.removefrominterface3; public interface Foo f public void foo(); g program package removefrominterface3; import lib.removefrominterface3.; public class Main implements Foo f public void foo() f System.out.println("foo"); g public void bar() f System.out.println("bar"); g public static void main(String[] args) f Foo f = new Main(); f.foo(); f.bar(); g g I this is similar to the previous example I note the declaration of f in main 16
  • Removing a Method from an Interface 3 Solution I this time the program is binary incompatible with lib-2.0.jar: a linkage error (NoSuchMethodError) occurs as the linker now tries to
  • nd bar() in Foo (the declared type of f), not in Main (the actual type) I compilation against lib-2.0.jar fails for the same reason - the compiler also fails to
  • nd bar() in Foo I i.e., the program is source incompatible with lib-2.0.jar as well 17
  • Specialising Return Types 1 lib-1.0.jar package lib.specialiseReturnType1; public class Foo f public static java.util.Collection getColl() f return new java.util.ArrayList(); g g + lib-2.0.jar package lib.specialiseReturnType1; public class Foo f public static java.util.List getColl() f return new java.util.ArrayList(); g g program package specialiseReturnType1; import lib.specialiseReturnType1.Foo; public class Main f public static void main(String[] args) f java.util.Collection coll = Foo.getColl(); System.out.println(coll); g g I return type is replaced by a subtype I i.e., postconditions are strengthened (method guarantees more) I program should run with lib-2.0.jar ! 18
  • Specialising Return Types 1 Solution I running the program with library version 2.0 fails ! I inspecting byte code (javap -c Main.class) shows that main references getColl as getColl()Ljava/util/Collection; - and this descriptor has changed I the result is a linkage error (NoSuchMethodError) I recompiling (and then running) the program with lib-2.0.jar succeeds I i.e., the program (compiled with lib-1.0.jar) is binary incompatible but source compatible with lib-2.0.jar 19
  • Specialising Return Types 2 lib-1.0.jar package lib.specialiseReturnType2; public class Foo f public static long getAnswer() f return 42L; g g + lib-2.0.jar package lib.specialiseReturnType2; public class Foo f public static int getAnswer() f return 42; g g program package specialiseReturnType2; import lib.specialiseReturnType2.Foo; public class Main f public static void main(String[] args) f long i = Foo.getAnswer(); System.out.println(i); g g I return type is narrowed from long to int I similar to specialising reference types 20
  • Specialising Return Types 2 Solution I again, this is binary incompatible, but source compatible I i.e., the problem can easily be
  • xed through recompilation I clients can safely widen the int to a long 21
  • Specialising Return Types 3 lib-1.0.jar package lib.specialiseReturnType3; import java.util.; public class Foo f public Collection getColl() f return new ArrayList(); g g + lib-2.0.jar package lib.specialiseReturnType3; import java.util.; public class Foo f public List getColl() f return new ArrayList(); g g program package specialiseReturnType3; import lib.specialiseReturnType3.Foo; import java.util.; public class Main extends Foo f public static void main(String[] args) f Foo f = new Main(); Collection c = f.getColl(); System.out.println(c); g @Override public Collection getColl() f return new HashSet(); g g I return type Collection is replaced by subtype List I but getColl() is now overridden in Main ! 22
  • Specialising Return Types 3 Solution I as before, the program is binary incompatible with lib-2.0.jar: java.lang.NoSuchMethodError: lib.specialiseReturnType3.Foo.getColl() Ljava/util/Collection I recompilation with lib-2.0.jar fails as well: compiler error: return type Collection is not compatible with List" I i.e., the program is neither binary nor source compatible with lib-2.0.jar I when overriding a method, the return type can only be specialised (co-variant return types [JLS, 8.4.5]), but this does not apply here as the overridden method itself has specialised its return type 23
  • Specialising Return Types 4 lib-1.0.jar package lib.specialiseReturnType4; import java.util.; public class Foo f public Collection getColl() f return new ArrayList(); g g + lib-2.0.jar package lib.specialiseReturnType4; import java.util.; public class Foo f public List getColl() f return new ArrayList(); g g program package specialiseReturnType4; import lib.specialiseReturnType4.Foo; import java.util.; public class Main extends Foo f public static void main(String[] args) f Main f = new Main(); Collection c = f.getColl(); System.out.println(c); g @Override public Collection getColl() f return new HashSet(); g g I minor change: f is now declared as Main, not Foo I what impact does this have? 24
  • Specialising Return Types 4 Solution I the program runs but does not compile with lib-2.0.jar ! I i.e., the program is binary compatible but source incompatible I to
  • nd out why, inspect byte code I Specialising Return Types 4: getColl() is referenced as getColl:()Ljava/util/Collection; - reference to local method that hasn't changed I Specialising Return Types 3: getColl() is referenced as lib/specialiseReturnType3/Foo.getColl:() Ljava/util/Collection; - reference to inherited method that has changed 25
  • Generalising Parameter Types 1 lib-1.0.jar package lib.generaliseParamType1; public class Foo f public static void doIt(java.util.List coll) f System.out.println(coll); g g + lib-2.0.jar package lib.generaliseParamType1; public class Foo f public static void doIt(java.util.Collection coll) f System.out.println(coll); g g program package generaliseParamType1; import lib.generaliseParamType1.Foo; public class Main f public static void main(String[] args) f Foo.doIt(new java.util.ArrayList()); g g I param type List is replaced by supertype Collection I this can be seen as weakened precondition (expects less) I should be compatible ! 26
  • Generalising Parameter Types 1 Solution I running the program with library version 2.0 fails ! I similar to changing return types, the descriptor changes, resulting in a linkage error (NoSuchMethodError) I recompiling (and then running) the program with library version 2.0 succeeds I i.e., the program (compiled with lib-1.0.jar) is binary incompatible but source compatible with lib-2.0.jar 27
  • Generalising Parameter Types 2 Interface1 Interface2 implements Class1 Class2 28
  • Generalising Parameter Types 2 lib-1.0.jar package lib.generaliseParamType2; public class Foo f public static void doIt(Class1 c) f System.out.println("C1"); g public static void doIt(Interface2 c) f System.out.println("I2"); g g + lib-2.0.jar package lib.generaliseParamType2; public class Foo f public static void doIt(Interface1 c) f System.out.println("I1"); g public static void doIt(Interface2 c) f System.out.println("I2"); g g program package generaliseParamType2; import lib.generaliseParamType2.; public class Main f public static void main(String[] args) f Foo.doIt(new Class1()); g g I doIt is overloaded I can the compiler select a method after generalising the parameter type? 29
  • Generalising Parameter Types 2 Solution I running the program with library version 2.0 fails ! I the descriptor changes, resulting in a linkage error (NoSuchMethodError) I recompiling fails as well - the compiler cannot select the most speci
  • c method [JLS, 15.12]: Error: reference to doIt is ambiguous, both method doIt(Interface1) in Foo and method doIt(Interface2) in Foo match. I i.e., the program (compiled with lib-1.0.jar) is neither binary nor source compatible with lib-2.0.jar 30
  • Generalising Parameter Types 3 lib-1.0.jar package lib.generaliseParamType3; public class Foo f public static boolean isEven(int i) f return i%2==0; g g + lib-2.0.jar package lib.generaliseParamType3; public class Foo f public static boolean isEven( oat i) f return i%2==0; g g program package generaliseParamType3; import lib.generaliseParamType3.Foo; public class Main f public static void main(String[] args) f int n = Integer.MAX VALUE; System.out.println(Foo.isEven(n)); g g I is the program binary and source compatible? I what is printed on the console? 31
  • Generalising Parameter Types 3 Solution I the program is not binary compatible with lib-2.0.jar, but seems to be source compatible - it can be recompiled and then executed I however, the output changes: while the original program prints true, the recompiled program prints false I the type parameter change changes the semantics of the program - although the method body is not changed ! I the change is source compatible, but source behavioural incompatible I the problem is that the widening conversion from int to float results in loss of precision [JLS, ch. 5.1.2] 32
  • Change a Method from Static to Non-Static lib-1.0.jar package lib.static1; public class Foo f public static void foo() f System.out.println("foo"); g g + lib-2.0.jar package lib.static1; public class Foo f public void foo() f System.out.println("foo"); g g program package static1; import lib.static1.Foo; public class Main f public static void main(String[] args) f Foo.foo(); g g I remove the static modi
  • er from foo() 33
  • Change a Method from Static to Non-Static Solution I the change is source incompatible: a non-static method cannot be referenced from a static context I an instance must be created to invoke a non-static method I but what about binary compatibility ? I lets consider the reverse scenario
  • rst 34
  • Change a Method from Non-Static to Static lib-1.0.jar package lib.static2; public class Foo f public void foo() f System.out.println("foo"); g g + lib-2.0.jar package lib.static2; public class Foo f public static void foo() f System.out.println("foo"); g g program package static2; import lib.static2.Foo; public class Main f public static void main(String[] args) f new Foo().foo(); g g I add a static modi
  • er to foo() 35
  • Change a Method from Static to Non-Static Solution I the change is source compatible I many IDEs will generate a warning: static methods should be references using static context (Foo.foo()) I but the change is still binary incompatible: a java.lang.IncompatibleClassChangeError is thrown I the same happens in the previous scenario 36
  • Static vs Non-Static Solution I the reason is the use of di erent byte code instructions: I static methods are invoked using invokestatic, for non-static methods, invokevirtual is used instead I the JVM checks the method type during linking, and creates an IncompatibleClassChangeError if a unexpected type is encountered [JVMS, ch. 5.4] I the same applies for
  • eld (read and write) access: there are di erent byte code instructions for accessing static and non-static
  • elds: getfield, putfield, getstatic, putstatic 37
  • Primitive vs Wrapper Types 1 lib-1.0.jar package lib.primwrap1; public class Foo f public static int MAGIC = 42; g + lib-2.0.jar package lib.primwrap1; public class Foo f public static Integer MAGIC = new Integer(42); g program package primwrap1; import lib.primwrap1.Foo; public class Main f public static void main(String[] args) f int i = Foo.MAGIC; System.out.println(i); g g I the
  • eld type int is replaced by its wrapper type Integer I is this transparent to the client program? 38
  • Primitive vs Wrapper Types 1 Solution I running the program with library version 2.0 fails ! I the descriptors have types, resulting in a linkage error (NoSuchFieldError) I recompiling (and then running) the program with lib-2.0.jar succeeds - the compiler applies unboxing [JLS, 5.1.8] I i.e., the program is binary incompatible but source compatible with lib-2.0.jar 39
  • Primitive vs Wrapper Types 2 lib-1.0.jar package lib.primwrap2; public class Foo f public static Integer MAGIC = new Integer(42); g + lib-2.0.jar package lib.primwrap2; public class Foo f public static int MAGIC = 42; g program package primwrap2; import lib.primwrap2.Foo; public class Main f public static void main(String[] args) f Integer i = Foo.MAGIC; System.out.println(i); g g I the
  • eld type Integer is replaced by the respective primitive type int I is this transparent to the client program? 40
  • Primitive vs Wrapper Types 2 Solution I running the program with library version 2.0 fails ! I the descriptors have di erent types, resulting in a linkage error (NoSuchFieldError) I recompiling (and then running) the program with lib-2.0.jar succeeds - the compiler applies boxing [JLS, 5.1.7] I i.e., the program is binary incompatible but source compatible with lib-2.0.jar 41
  • Generics 1 lib-1.0.jar package lib.generics1; import java.util.; public class Foo f public static List<String> getList() f List<String> list = new ArrayList<String>(); list.add("42"); return list; g g + lib-2.0.jar package lib.generics1; import java.util.; public class Foo f public static List<Integer> getList() f List<Integer> list = new ArrayList<Integer>(); list.add(42); return list; g g program package generics1; import lib.generics1.; public class Main f public static void main(String[] args) f java.util.List<String> list = Foo.getList(); System.out.println(list.size()); g g I generic type parameter in method return type is changed I does this matter? 42
  • Generics 1 Solution I this is binary compatible due to type erasure in Java I however, this is not source compatible - the compiler cannot assign a list of integers to a variable declared as a list of strings 43
  • Generics 2 lib-1.0.jar package lib.generics2; import java.util.; public class Foo f public static List<String> getList() f List<String> list = new ArrayList<String>(); list.add("42"); return list; g g + lib-2.0.jar package lib.generics2; import java.util.; public class Foo f public static List<Integer> getList() f List<Integer> list = new ArrayList<Integer>(); list.add(42); return list; g g program package generics2; import lib.generics2.; public class Main f public static void main(String[] args) f java.util.List<String> list = Foo.getList(); for (String s:list) f System.out.println(s); g g g I note that only the way the generic type is used has changed I the program iterates over the strings in the list 44
  • Generics 2 Solution I this is binary compatible acc. to the JLS I when the elements are accessed inside the loop, a cast instruction (checkcast) is inserted by the compiler I this cast fails when the list is changed to a list of integers, and a runtime exception is thrown I the change is therefore binary behavioural incompatible I this is not source compatible either 45
  • Generics 3 lib-1.0.jar package lib.generics3; import java.io.Serializable; public class Foo<T extends Serializable & Comparable> f public void foo(T t) f t.compareTo(""); System.out.println(t); g g + lib-2.0.jar package lib.generics3; import java.io.Serializable; public class Foo<T extends Comparable & Serializable>f public void foo(T t) f t.compareTo(""); System.out.println(t); g g program package generics3; import lib.generics3.; public class Main implements java.io.Serializable f public static void main(String[] args) f Main m = new Main(); new Foo().foo(m); g g I Main only implements Serializable, but not Comparable I can Main even be compiled ? I what is the impact of changing the order of the interfaces de
  • ning the bounds of the type parameter? 46
  • Generics 3 Solution I the program compiles and links with lib-1.0.jar, despite not implementing both interfaces! I however, executing the program with lib-1.0.jar leads to a ClassCastException I the reason for this is how erasure works: only the leftmost bound is used [JLS, ch. 4.6] I i.e., foo(T) is referenced as foo(Serializable) I before compareTo is invoked (at runtime!), the parameter is cast to Comparable, and this fails 47
  • Generics 3 Solution ctd I changing the order of the interfaces in lib-2.0.jar is binary and source incompatible I now the leftmost bound is Comparable, i.e., foo(T) is referenced as foo(Comparable) I this incompatibility is detected by both the compiler and the linker 48
  • Constants 1 lib-1.0.jar package lib.constants1; public class Foo f public static
  • nal int MAGIC = 42; g + lib-2.0.jar package lib.constants1; public class Foo f public static
  • nal int MAGIC = 43; g program package constants1; import lib.constants1.; public class Main f public static void main(String[] args) f System.out.println(Foo.MAGIC); g g I now the question is: what does this program print? 49
  • Constants 1 Solution I the program prints 42 when it is executed with lib-1.0.jar as expected I but the program still prints 42 when executed with lib-2.0.jar ! I the compiler inlines the constant value into the client class I this is binary compatibility but binary behavioural incompatible 50
  • Constants 2 lib-1.0.jar package lib.constants2; public class Foo f public static
  • nal String MAGIC = "42"; g + lib-2.0.jar package lib.constants2; public class Foo f public static
  • nal String MAGIC = "43"; g program package constants2; import lib.constants2.; public class Main f public static void main(String[] args) f System.out.println(Foo.MAGIC); g g I inlining is applied to primitive data types, but what about strings? I what will be printed to the console? 51
  • Constants 2 Solution I the program still prints 42 when it is executed with lib-2.0.jar I constant inlining is still applied when strings are used I strings are immutable objects, and in many cases can be treated like primitive types I this is binary compatibility but binary behavioural incompatible 52
  • Constants 3 lib-1.0.jar package lib.constants3; public class Foo f public static
  • nal int MAGIC = 40+2; g + lib-2.0.jar package lib.constants3; public class Foo f public static
  • nal int MAGIC = 40+3; g program package constants3; import lib.constants3.; public class Main f public static void main(String[] args) f System.out.println(Foo.MAGIC); g g I now the constant value is de
  • ned by an expression I what will be printed to the console? 53
  • Constants 3 Solution I the program still prints 42 when it is executed with lib-2.0.jar I the compiler applies constant folding I this does not seem to be speci
  • ed in [JLS] and might therefore be a compiler-speci
  • c optimisation I i.e., the expression is evaluated at compile time I this is binary compatibility but binary behavioural incompatible 54
  • Constants 4 lib-1.0.jar package lib.constants4; public class Foo f public static
  • nal Integer MAGIC = 42; g + lib-2.0.jar package lib.constants4; public class Foo f public static
  • nal Integer MAGIC = 43; g program package constants4; import lib.constants4.; public class Main f public static void main(String[] args) f System.out.println(Foo.MAGIC); g g I now the constant is de
  • ned using the wrapper type I note that assignment is safe due to autoboxing I what will be printed to the console? 55
  • Constants 4 Solution I now 43 is printed as expected when the program runs with lib-2.0.jar ! I i.e., simply by using wrapper types, constant inlining can be prevented (at least using the current version of the compiler) I A variable of primitive type or type String, that is
  • nal and initialized with a compile-time constant expression (15.28), is called a constant variable. [JLS, ch. 4.12.4] I it is unclear why wrapper types are excluded, they are immutable as well! I this may explain why many constants in projects like velocity are de
  • ned using wrapper types - to prevent inlining 56
  • Exceptions I methods can declare exceptions I for checked exceptions, the compiler forces callers to handle or rethrow the exception I at runtime, when an exception occurs the JVM searches the invocation chain of the method (stack) for a suitable exception handler [JVMS, ch. 2.10] I the compiler treats exceptions as part of the method declaration - what about the JVM? I i.e., are certain changes to exceptions (removing or specialising exceptions) binary incompatible but source compatible? 57
  • Adding a Runtime Exception lib-1.0.jar package lib.exceptions1; public class Foo f public static void foo() fg g + lib-2.0.jar package lib.exceptions1; public class Foo f public static void foo() throws UnsupportedOperationException f throw new UnsupportedOperationException(); g g program package exceptions1; public class Main f public static void main(String[] args) f lib.exceptions1.Foo.foo(); g g I in lib-2.0.jar , an UnsupportedOperation- Exception is declared and thrown I note that this is a runtime (unchecked) exception 58
  • Adding a Runtime Exception Solution I the program is source compatible with lib-2.0.jar: UnsupportedOperationException is a runtime exception, and whether it is declared or not makes no di erence I however, while the program is binary compatible, it is binary behavioural incompatible - but this is only because the exception is actually thrown in lib-2.0.jar I if the throw statement was removed, the program would become binary behavioural compatible although the exception is still declared I it would still be possible to compile the modi
  • ed program - the compiler cannot
  • gure out that a declared unchecked exception is never thrown I this makes sense - runtime exceptions are thrown implicitly (null pointers, failed casts etc) and it is too dicult for the compiler to check this 59
  • Adding a Checked Exception lib-1.0.jar package lib.exceptions2; public class Foo f public static void foo() f g g + lib-2.0.jar package lib.exceptions2; import java.io.IOException; public class Foo f public static void foo() throws IOException f throw new IOException(); g g program package exceptions2; public class Main f public static void main(String[] args) f lib.exceptions2.Foo.foo(); g g I in lib-2.0.jar , an IOException is declared or thrown I this is a checked exception 60
  • Adding a Checked Exception Solution I not surprisingly, the change is source incompatible I however, the program is binary compatible with lib-2.0.jar I i.e., the declared exception is not detected during linking as this is not part of the method descriptor I the change is binary behavioural incompatible as the exception is thrown but not caught 61
  • Generalising a Checked Exception lib-1.0.jar package lib.exceptions3; import java.io.IOException; public class Foo f public static void foo() throws IOException f throw new IOException(); g g + lib-2.0.jar package lib.exceptions3; public class Foo f public static void foo() throws Exception f throw new Exception(); g g program package exceptions3; import java.io.IOException; public class Main f public static void main(String[] args) f try f lib.exceptions3.Foo.foo(); g catch (IOException x) f System.err.println("Caught it"); g g g I in lib-2.0.jar , the IOException is replaced by its super type Exception I the client program only handles the IOException 62
  • Generalising a Checked Exception Solution I the program is again source incompatible but binary compatible with lib-2.0.jar I but as before, the program behaviour changes as the exception is not caught, i.e. the change is binary behavioural incompatible 63
  • Specialising a Checked Exception lib-1.0.jar package lib.exceptions4; import java.io.IOException; public class Foo f public static void foo() throws Exception f throw new IOException(); g g + lib-2.0.jar package lib.exceptions4; import java.io.IOException; public class Foo f public static void foo() throws IOException f throw new IOException(); g g program package exceptions4; import java.io.IOException; public class Main f public static void main(String[] args) f try f lib.exceptions4.Foo.foo(); g catch (Exception x) f System.err.println("Caught it"); g g g I in lib-2.0.jar , the Exception is replaced by its sub type IOException I this is similar to specialising the return type 64
  • Specialising a Checked Exception Solution I the program is source and binary compatible with lib-2.0.jar I this is (surprisingly) di erent to specialising return types I the exceptions are not part of the method descriptor used to references method when linking 65
  • Removing a Checked Exception 1 lib-1.0.jar package lib.exceptions5; import java.io.IOException; public class Foo f public static void foo() throws IOException f throw new IOException(); g g + lib-2.0.jar package lib.exceptions5; public class Foo f public static void foo() f g g program package exceptions5; import java.io.IOException; public class Main f public static void main(String[] args) f try f lib.exceptions5.Foo.foo(); g catch (IOException x) f System.err.println("Caught it"); g g g I in lib-2.0.jar , the IOException is removed from the method I but note the exception handler in main 66
  • Removing a Checked Exception 1 Solution I the program is binary compatible but source incompatible with lib-2.0.jar I the compiler infers that the catch statement is not reachable because the updated foo() does not throw an exception: exception IOException is never thrown in body of corresponding try statement [JLS, ch. 14.21] 67
  • Removing a Checked Exception 2 lib-1.0.jar package lib.exceptions6; public class Foo f public static void foo() throws Exception f throw new Exception(); g g + lib-2.0.jar package lib.exceptions6; public class Foo f public static void foo() f g g program package exceptions6; public class Main f public static void main(String[] args) f try f lib.exceptions6.Foo.foo(); g catch (Exception x) f System.err.println("Caught it"); g g g I in lib-2.0.jar , the Exception is removed from the method I but note the exception handler in main 68
  • Removing a Checked Exception 2 Solution I surprisingly, the program is binary compatible and source compatible with lib-2.0.jar I the compiler still considers the catch clause as reachable I this makes sense, as Exception includes runtime exceptions 69
  • Removing a Checked Exception 2 ctd Solution I however, it seems to contradict the reachability rules: A catch block C is reachable i both of the following are true: Either the type of C's parameter is an unchecked exception type or Throwable; or some expression or throw statement in the try block is reachable and can throw a checked exception whose type is assignable to the parameter of the catch clause C. [JLS, ch. 14.21] but RuntimeException and all its subclasses are, collectively, the runtime exception classes. .. The unchecked exception classes are the runtime exception classes and the error classes. [JLS, ch. 11.1.1]. I i.e., one would expect a reachability compiler error! I this has been report as a bug in the JLS, and will be
  • xed in JLS-8 (email communication with Alex Buckley) 70
  • Ghost lib-1.0.jar package lib.ghost; public class Foo f public static class Bar f public static void foo() f System.out.println("foo"); g g g + lib-2.0.jar package lib.ghost.Foo; public class Bar f public static void foo() f System.out.println("foo"); g g program package ghost; public class Main f public static void main(String[] args) f lib.ghost.Foo.Bar.foo(); g g I now the question is: what does this program print? 71
  • Ghost Solution I the program is not binary compatible with lib-2.0.jar: java.lang.NoClassDefFoundError: lib/ghost/Foo$Bar I however, the program is source compatible I the problem is that the reference lib.ghost.Foo.Bar can either refer to an inner class Bar within the class lib.ghost.Foo, or a top-level class Bar in the package lib.ghost.Foo I the byte code representation however di ers: lib/ghost/Foo$Bar vs lib/ghost/Foo/Bar 72
  • Bridge lib-1.0.jar package lib.bridge1; public class Foo f private int foo = 0; public class Inner f @Override public String toString() f return "Inner[foo="+foo+"]"; g g g + lib-2.0.jar package lib.bridge1; public class Foo f int foo = 0; public class Inner f @Override public String toString() f return "Inner[foo="+foo+"]"; g g g program package bridge1; import lib.bridge1.; import java.lang.re ect.Method; public class Main f public static void main(String[] args) f Method[] mm = Foo.class.getDeclaredMethods(); for (Method m:mm) f System.out.println(m); g g g I re ection is used to
  • nd out how many methods Foo has I can this be changed by only changing the access modi
  • er of the
  • eld foo from private to default? 73
  • Bridge Solution I the program is binary and source compatible with lib-2.0.jar I however, the behaviour changes I in lib-1.0.jar, a synthetic bridge method static int lib.bridge1.Foo.access$000(lib.bridge1.Foo) is generated by the compiler to enable access to the private
  • eld by the inner class I this method is not necessary if access to the
  • eld in changed to non-private I synthetic methods [JLS, ch. 13.1] are widely used, for instances when overriding methods with generic parameter types, and co-variant return types 74
  • Summary I binary compatibility does not imply source compatibility (example: addtointerface) I binary compatibility does not imply binary behavioural compatibility (example: generics2) I source compatibility does not imply binary compatibility (example: specialiseReturnType1) I source compatibility does not imply source behavioural compatibility (example: generaliseParamType3) 75
  • Ongoing Research and Open Questions I Empirical study on Qualitas Corpus on whether and how often these problems occur in real-world programs. Proceedings IEEE CSMR-WCRE 2014. Preprint: https://sites.google.com/site/jensdietrich/ publications/preprints I Quiz developers to
  • nd out whether they are aware of this. Over 400 developers have responded,
  • rst results here: https://sites.google.com/site/jensdietrich/ java-developer-survey-2013, preprint on arxiv: http://arxiv.org/pdf/1408.2607v1.pdf I Build better tools (better than clirr) to check library compatibility, infer semantic versioning info. Some ongoing work (Uni of Western Bohemia), more planned for later 2014. 76
  • Acknowledgements I would like to thank Kamil Jezek who contributed Generics 1 and Static 1 and 2, Hussain Al Mutawa who pointed me to ghost references used in Ghost, and Alex Buckley for his comments and for contributing Adding a Method to an Interface. This work was inspired by Java Puzzlers by Joshua Bloch and Neal Gafter [PUZZ]. 77
  • References JAPI Jim des Rivieres: Evolving Java-based APIs. http://wiki.eclipse.org/Evolving_Java-based_APIs JLS James Gosling, Bill Joy, Guy Steele, Gilad Bracha and Alex Buckley: The JavaTMLanguage Speci
  • cation 7th Edition. JVMS Tim Lindholm, Frank Yellin, Gilad Bracha, and Alex Buckley. The JavaTMVirtual Machine Speci
  • cation - JavaTMSE 7 Edition. PUZZ Joshua Bloch, Neal Gafter. Java Puzzlers. Addison-Wesley 2005. 78