Java Library Evolution Puzzlers

921 views

Published on

A collection of puzzlers demonstrating the difference between source and binary compatibility in Java.

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
921
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
11
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Java Library Evolution Puzzlers

  1. 1. Java Library Evolution Puzzlers Jens Dietrich1 1Massey University School of Engineering and Advanced Technology Palmerston North, New Zealand https://sites.google.com/site/jensdietrich/ Email: j.b.dietrich /at/ massey.ac.nz August 31, 2014 1
  2. 2. Revision History Revision Date Remarks 1.0 13 Sept 13 initial version 2.0 10 Feb 14 added bridge (synthetic methods generated by compiler) 3.0 14 Feb 14 added generics3 (changing the order of multiple type parameter bounds) 4.0 31 Aug 14 added static* (static vs non-static) 2
  3. 3. Table of Contents Introduction Modifying Interfaces Modifying Method Signatures Static vs Non-Static Primitive vs Wrapper Types Using Generic Parameter Types Changing the Values of Constants Modifying Exceptions 3
  4. 4. Introduction - Deploying Java Programs I Java programs are usually built (ant,maven,gradle,..) with all libraries they use, and then deployed I if the program or a library changes, the program is rebuilt and redeployed I the build step includes V&V: compiling and (automated regression) testing I partial library upgrades are becoming more and more popular, example: OSGi bundle updates I the puzzlers described here show the dierence between these two deployment modes 4
  5. 5. Introduction - Source vs Binary Compatibility I a program is source compatible with a library lib.jar if the program uses the library, and compilation succeeds: javac -cp ..,lib.jar,.. ... I source compatibility is checked by the compiler, incompatibility results in compilation errors I a program is binary compatible with a library lib.jar if it links and runs with this library: java -cp ..,lib.jar,.. .. [JLS, ch. 13] I binary compatibility is checked by the JVM, incompatibility results in (linkage) errors 5
  6. 6. The Limitations of Binary Compatibility I in the JLS, a very narrow de
  7. 7. nition of binary (in)compatibility is used: A change to a type is binary compatible with pre-existing binaries if pre-existing binaries that previously linked without error will continue to link without error. [JLS, ch. 13.2] I binary compatibility is de
  8. 8. ned w.r.t. to what the linker can detect by means of static analysis, failure results in errors (not exceptions) 6
  9. 9. Introduction - Evolution Problems I assume that a program references code de
  10. 10. ned in lib-1.0.jar I assume that the program can be compiled successfully and can be executed with lib-1.0.jar without causing an error or exception I then the library evolves to lib-2.0.jar 7
  11. 11. Introduction - Evolution Problems Questions: 1. does the program link with lib-2.0.jar - i.e., is it binary compatible with lib-2.0.jar? 2. does replacing the library change the behaviour of the program - i.e., is the library change binary behavioural compatible? 3. does the program compile against lib-2.0.jar - i.e., is it source compatible with lib-2.0.jar? 4. does recompiling the program against the changed library change the behaviour of the program - i.e., is the library change source behavioural compatible? 8
  12. 12. Introduction - Running Experiments I check out code: hg clone https://bitbucket.org/jensdietrich/ java-library-evolution-puzzlers I each example has a program with a main class aPackage.Main, and two versions of classes de
  13. 13. ned in a separate library I cd to folder and run ant as follows: ant -Dpackage=aPackage I this will do the following: 1. compile the two versions of the library and build lib-1.0.jar and lib-2.0.jar 2. compile and run the program with lib-1.0.jar 3. compile the program with lib-1.0.jar , but run it with lib-2.0.jar 4. re-compile and run the program with lib-2.0.jar 9
  14. 14. Adding a Method to an Interface lib-1.0.jar package lib.addtointerface; public interface Foo f public void foo(); g + lib-2.0.jar package lib.addtointerface; public interface Foo f public void foo(); public void bar(); g program package addtointerface; import lib.addtointerface.; public class Main implements Foo f @Override public void foo() f System.out.println(foo); g public static void main(String[] args) f new Main().foo(); g g I the interface Foo is extended by adding bar() I but the client class implements the old interface I is this still binary compatible with lib-2.0.jar? 10
  15. 15. Adding a Method to an Interface Solution I running the program with library version 2.0 succeeds - the client program is not using the method added to the interface! I i.e., the program (compiled with lib-1.0.jar) is binary compatible with lib-2.0.jar I but recompilation fails as Main does not implement bar() I i.e., the program is source incompatible with lib-2.0.jar 11
  16. 16. Removing a Method from an Interface 1 lib-1.0.jar package lib.removefrominterface1; public interface Foo f public void foo(); public void bar(); g + lib-2.0.jar package lib.removefrominterface1; public interface Foo f public void foo(); g program package removefrominterface1; import lib.removefrominterface1.; public class Main implements Foo f @Override public void foo() f System.out.println(foo); g @Override public void bar() f System.out.println(bar); g public static void main(String[] args) f new Main().foo(); new Main().bar(); g g I the method bar() is removed from the interface Foo I but the client class implements the old interface 12
  17. 17. Removing a Method from an Interface 1 Solution I running the program with library version 2.0 succeeds ! I i.e., the program (compiled with lib-1.0.jar) is binary compatible with lib-2.0.jar I but recompilation fails as Main.bar() does not override a method! I i.e., the program is source incompatible with lib-2.0.jar 13
  18. 18. Removing a Method from an Interface 2 lib-1.0.jar package lib.removefrominterface2; public interface Foo f public void foo(); public void bar(); g + lib-2.0.jar package lib.removefrominterface2; public interface Foo f public void foo(); g program package removefrominterface2; import lib.removefrominterface2.; public class Main implements Foo f public void foo() f System.out.println(foo); g public void bar() f System.out.println(bar); g public static void main(String[] args) f new Main().foo(); new Main().bar(); g g I this is almost identical to the previous example I but this time the @Override annotation is not used 14
  19. 19. Removing a Method from an Interface 2 Solution I as before, the program (compiled with lib-1.0.jar) is binary compatible with lib-2.0.jar I but recompilation also succeeds as the compiler does not check whether Main.bar() overrides a method I i.e., the program is also source compatible with lib-2.0.jar 15
  20. 20. Removing a Method from an Interface 3 lib-1.0.jar package lib.removefrominterface3; public interface Foo f public void foo(); public void bar(); g + lib-2.0.jar package lib.removefrominterface3; public interface Foo f public void foo(); g program package removefrominterface3; import lib.removefrominterface3.; public class Main implements Foo f public void foo() f System.out.println(foo); g public void bar() f System.out.println(bar); g public static void main(String[] args) f Foo f = new Main(); f.foo(); f.bar(); g g I this is similar to the previous example I note the declaration of f in main 16
  21. 21. Removing a Method from an Interface 3 Solution I this time the program is binary incompatible with lib-2.0.jar: a linkage error (NoSuchMethodError) occurs as the linker now tries to
  22. 22. nd bar() in Foo (the declared type of f), not in Main (the actual type) I compilation against lib-2.0.jar fails for the same reason - the compiler also fails to
  23. 23. nd bar() in Foo I i.e., the program is source incompatible with lib-2.0.jar as well 17
  24. 24. Specialising Return Types 1 lib-1.0.jar package lib.specialiseReturnType1; public class Foo f public static java.util.Collection getColl() f return new java.util.ArrayList(); g g + lib-2.0.jar package lib.specialiseReturnType1; public class Foo f public static java.util.List getColl() f return new java.util.ArrayList(); g g program package specialiseReturnType1; import lib.specialiseReturnType1.Foo; public class Main f public static void main(String[] args) f java.util.Collection coll = Foo.getColl(); System.out.println(coll); g g I return type is replaced by a subtype I i.e., postconditions are strengthened (method guarantees more) I program should run with lib-2.0.jar ! 18
  25. 25. Specialising Return Types 1 Solution I running the program with library version 2.0 fails ! I inspecting byte code (javap -c Main.class) shows that main references getColl as getColl()Ljava/util/Collection; - and this descriptor has changed I the result is a linkage error (NoSuchMethodError) I recompiling (and then running) the program with lib-2.0.jar succeeds I i.e., the program (compiled with lib-1.0.jar) is binary incompatible but source compatible with lib-2.0.jar 19
  26. 26. Specialising Return Types 2 lib-1.0.jar package lib.specialiseReturnType2; public class Foo f public static long getAnswer() f return 42L; g g + lib-2.0.jar package lib.specialiseReturnType2; public class Foo f public static int getAnswer() f return 42; g g program package specialiseReturnType2; import lib.specialiseReturnType2.Foo; public class Main f public static void main(String[] args) f long i = Foo.getAnswer(); System.out.println(i); g g I return type is narrowed from long to int I similar to specialising reference types 20
  27. 27. Specialising Return Types 2 Solution I again, this is binary incompatible, but source compatible I i.e., the problem can easily be
  28. 28. xed through recompilation I clients can safely widen the int to a long 21
  29. 29. Specialising Return Types 3 lib-1.0.jar package lib.specialiseReturnType3; import java.util.; public class Foo f public Collection getColl() f return new ArrayList(); g g + lib-2.0.jar package lib.specialiseReturnType3; import java.util.; public class Foo f public List getColl() f return new ArrayList(); g g program package specialiseReturnType3; import lib.specialiseReturnType3.Foo; import java.util.; public class Main extends Foo f public static void main(String[] args) f Foo f = new Main(); Collection c = f.getColl(); System.out.println(c); g @Override public Collection getColl() f return new HashSet(); g g I return type Collection is replaced by subtype List I but getColl() is now overridden in Main ! 22
  30. 30. Specialising Return Types 3 Solution I as before, the program is binary incompatible with lib-2.0.jar: java.lang.NoSuchMethodError: lib.specialiseReturnType3.Foo.getColl() Ljava/util/Collection I recompilation with lib-2.0.jar fails as well: compiler error: return type Collection is not compatible with List I i.e., the program is neither binary nor source compatible with lib-2.0.jar I when overriding a method, the return type can only be specialised (co-variant return types [JLS, 8.4.5]), but this does not apply here as the overridden method itself has specialised its return type 23
  31. 31. Specialising Return Types 4 lib-1.0.jar package lib.specialiseReturnType4; import java.util.; public class Foo f public Collection getColl() f return new ArrayList(); g g + lib-2.0.jar package lib.specialiseReturnType4; import java.util.; public class Foo f public List getColl() f return new ArrayList(); g g program package specialiseReturnType4; import lib.specialiseReturnType4.Foo; import java.util.; public class Main extends Foo f public static void main(String[] args) f Main f = new Main(); Collection c = f.getColl(); System.out.println(c); g @Override public Collection getColl() f return new HashSet(); g g I minor change: f is now declared as Main, not Foo I what impact does this have? 24
  32. 32. Specialising Return Types 4 Solution I the program runs but does not compile with lib-2.0.jar ! I i.e., the program is binary compatible but source incompatible I to
  33. 33. nd out why, inspect byte code I Specialising Return Types 4: getColl() is referenced as getColl:()Ljava/util/Collection; - reference to local method that hasn't changed I Specialising Return Types 3: getColl() is referenced as lib/specialiseReturnType3/Foo.getColl:() Ljava/util/Collection; - reference to inherited method that has changed 25
  34. 34. Generalising Parameter Types 1 lib-1.0.jar package lib.generaliseParamType1; public class Foo f public static void doIt(java.util.List coll) f System.out.println(coll); g g + lib-2.0.jar package lib.generaliseParamType1; public class Foo f public static void doIt(java.util.Collection coll) f System.out.println(coll); g g program package generaliseParamType1; import lib.generaliseParamType1.Foo; public class Main f public static void main(String[] args) f Foo.doIt(new java.util.ArrayList()); g g I param type List is replaced by supertype Collection I this can be seen as weakened precondition (expects less) I should be compatible ! 26
  35. 35. Generalising Parameter Types 1 Solution I running the program with library version 2.0 fails ! I similar to changing return types, the descriptor changes, resulting in a linkage error (NoSuchMethodError) I recompiling (and then running) the program with library version 2.0 succeeds I i.e., the program (compiled with lib-1.0.jar) is binary incompatible but source compatible with lib-2.0.jar 27
  36. 36. Generalising Parameter Types 2 Interface1 Interface2 implements Class1 Class2 28
  37. 37. Generalising Parameter Types 2 lib-1.0.jar package lib.generaliseParamType2; public class Foo f public static void doIt(Class1 c) f System.out.println(C1); g public static void doIt(Interface2 c) f System.out.println(I2); g g + lib-2.0.jar package lib.generaliseParamType2; public class Foo f public static void doIt(Interface1 c) f System.out.println(I1); g public static void doIt(Interface2 c) f System.out.println(I2); g g program package generaliseParamType2; import lib.generaliseParamType2.; public class Main f public static void main(String[] args) f Foo.doIt(new Class1()); g g I doIt is overloaded I can the compiler select a method after generalising the parameter type? 29
  38. 38. Generalising Parameter Types 2 Solution I running the program with library version 2.0 fails ! I the descriptor changes, resulting in a linkage error (NoSuchMethodError) I recompiling fails as well - the compiler cannot select the most speci
  39. 39. c method [JLS, 15.12]: Error: reference to doIt is ambiguous, both method doIt(Interface1) in Foo and method doIt(Interface2) in Foo match. I i.e., the program (compiled with lib-1.0.jar) is neither binary nor source compatible with lib-2.0.jar 30
  40. 40. Generalising Parameter Types 3 lib-1.0.jar package lib.generaliseParamType3; public class Foo f public static boolean isEven(int i) f return i%2==0; g g + lib-2.0.jar package lib.generaliseParamType3; public class Foo f public static boolean isEven( oat i) f return i%2==0; g g program package generaliseParamType3; import lib.generaliseParamType3.Foo; public class Main f public static void main(String[] args) f int n = Integer.MAX VALUE; System.out.println(Foo.isEven(n)); g g I is the program binary and source compatible? I what is printed on the console? 31
  41. 41. Generalising Parameter Types 3 Solution I the program is not binary compatible with lib-2.0.jar, but seems to be source compatible - it can be recompiled and then executed I however, the output changes: while the original program prints true, the recompiled program prints false I the type parameter change changes the semantics of the program - although the method body is not changed ! I the change is source compatible, but source behavioural incompatible I the problem is that the widening conversion from int to float results in loss of precision [JLS, ch. 5.1.2] 32
  42. 42. Change a Method from Static to Non-Static lib-1.0.jar package lib.static1; public class Foo f public static void foo() f System.out.println(foo); g g + lib-2.0.jar package lib.static1; public class Foo f public void foo() f System.out.println(foo); g g program package static1; import lib.static1.Foo; public class Main f public static void main(String[] args) f Foo.foo(); g g I remove the static modi
  43. 43. er from foo() 33
  44. 44. Change a Method from Static to Non-Static Solution I the change is source incompatible: a non-static method cannot be referenced from a static context I an instance must be created to invoke a non-static method I but what about binary compatibility ? I lets consider the reverse scenario
  45. 45. rst 34
  46. 46. Change a Method from Non-Static to Static lib-1.0.jar package lib.static2; public class Foo f public void foo() f System.out.println(foo); g g + lib-2.0.jar package lib.static2; public class Foo f public static void foo() f System.out.println(foo); g g program package static2; import lib.static2.Foo; public class Main f public static void main(String[] args) f new Foo().foo(); g g I add a static modi
  47. 47. er to foo() 35
  48. 48. Change a Method from Static to Non-Static Solution I the change is source compatible I many IDEs will generate a warning: static methods should be references using static context (Foo.foo()) I but the change is still binary incompatible: a java.lang.IncompatibleClassChangeError is thrown I the same happens in the previous scenario 36
  49. 49. Static vs Non-Static Solution I the reason is the use of dierent byte code instructions: I static methods are invoked using invokestatic, for non-static methods, invokevirtual is used instead I the JVM checks the method type during linking, and creates an IncompatibleClassChangeError if a unexpected type is encountered [JVMS, ch. 5.4] I the same applies for
  50. 50. eld (read and write) access: there are dierent byte code instructions for accessing static and non-static
  51. 51. elds: getfield, putfield, getstatic, putstatic 37
  52. 52. Primitive vs Wrapper Types 1 lib-1.0.jar package lib.primwrap1; public class Foo f public static int MAGIC = 42; g + lib-2.0.jar package lib.primwrap1; public class Foo f public static Integer MAGIC = new Integer(42); g program package primwrap1; import lib.primwrap1.Foo; public class Main f public static void main(String[] args) f int i = Foo.MAGIC; System.out.println(i); g g I the
  53. 53. eld type int is replaced by its wrapper type Integer I is this transparent to the client program? 38
  54. 54. Primitive vs Wrapper Types 1 Solution I running the program with library version 2.0 fails ! I the descriptors have types, resulting in a linkage error (NoSuchFieldError) I recompiling (and then running) the program with lib-2.0.jar succeeds - the compiler applies unboxing [JLS, 5.1.8] I i.e., the program is binary incompatible but source compatible with lib-2.0.jar 39
  55. 55. Primitive vs Wrapper Types 2 lib-1.0.jar package lib.primwrap2; public class Foo f public static Integer MAGIC = new Integer(42); g + lib-2.0.jar package lib.primwrap2; public class Foo f public static int MAGIC = 42; g program package primwrap2; import lib.primwrap2.Foo; public class Main f public static void main(String[] args) f Integer i = Foo.MAGIC; System.out.println(i); g g I the
  56. 56. eld type Integer is replaced by the respective primitive type int I is this transparent to the client program? 40
  57. 57. Primitive vs Wrapper Types 2 Solution I running the program with library version 2.0 fails ! I the descriptors have dierent types, resulting in a linkage error (NoSuchFieldError) I recompiling (and then running) the program with lib-2.0.jar succeeds - the compiler applies boxing [JLS, 5.1.7] I i.e., the program is binary incompatible but source compatible with lib-2.0.jar 41
  58. 58. Generics 1 lib-1.0.jar package lib.generics1; import java.util.; public class Foo f public static ListString getList() f ListString list = new ArrayListString(); list.add(42); return list; g g + lib-2.0.jar package lib.generics1; import java.util.; public class Foo f public static ListInteger getList() f ListInteger list = new ArrayListInteger(); list.add(42); return list; g g program package generics1; import lib.generics1.; public class Main f public static void main(String[] args) f java.util.ListString list = Foo.getList(); System.out.println(list.size()); g g I generic type parameter in method return type is changed I does this matter? 42
  59. 59. Generics 1 Solution I this is binary compatible due to type erasure in Java I however, this is not source compatible - the compiler cannot assign a list of integers to a variable declared as a list of strings 43
  60. 60. Generics 2 lib-1.0.jar package lib.generics2; import java.util.; public class Foo f public static ListString getList() f ListString list = new ArrayListString(); list.add(42); return list; g g + lib-2.0.jar package lib.generics2; import java.util.; public class Foo f public static ListInteger getList() f ListInteger list = new ArrayListInteger(); list.add(42); return list; g g program package generics2; import lib.generics2.; public class Main f public static void main(String[] args) f java.util.ListString list = Foo.getList(); for (String s:list) f System.out.println(s); g g g I note that only the way the generic type is used has changed I the program iterates over the strings in the list 44
  61. 61. Generics 2 Solution I this is binary compatible acc. to the JLS I when the elements are accessed inside the loop, a cast instruction (checkcast) is inserted by the compiler I this cast fails when the list is changed to a list of integers, and a runtime exception is thrown I the change is therefore binary behavioural incompatible I this is not source compatible either 45
  62. 62. Generics 3 lib-1.0.jar package lib.generics3; import java.io.Serializable; public class FooT extends Serializable Comparable f public void foo(T t) f t.compareTo(); System.out.println(t); g g + lib-2.0.jar package lib.generics3; import java.io.Serializable; public class FooT extends Comparable Serializablef public void foo(T t) f t.compareTo(); System.out.println(t); g g program package generics3; import lib.generics3.; public class Main implements java.io.Serializable f public static void main(String[] args) f Main m = new Main(); new Foo().foo(m); g g I Main only implements Serializable, but not Comparable I can Main even be compiled ? I what is the impact of changing the order of the interfaces de
  63. 63. ning the bounds of the type parameter? 46
  64. 64. Generics 3 Solution I the program compiles and links with lib-1.0.jar, despite not implementing both interfaces! I however, executing the program with lib-1.0.jar leads to a ClassCastException I the reason for this is how erasure works: only the leftmost bound is used [JLS, ch. 4.6] I i.e., foo(T) is referenced as foo(Serializable) I before compareTo is invoked (at runtime!), the parameter is cast to Comparable, and this fails 47
  65. 65. Generics 3 Solution ctd I changing the order of the interfaces in lib-2.0.jar is binary and source incompatible I now the leftmost bound is Comparable, i.e., foo(T) is referenced as foo(Comparable) I this incompatibility is detected by both the compiler and the linker 48
  66. 66. Constants 1 lib-1.0.jar package lib.constants1; public class Foo f public static
  67. 67. nal int MAGIC = 42; g + lib-2.0.jar package lib.constants1; public class Foo f public static
  68. 68. nal int MAGIC = 43; g program package constants1; import lib.constants1.; public class Main f public static void main(String[] args) f System.out.println(Foo.MAGIC); g g I now the question is: what does this program print? 49
  69. 69. Constants 1 Solution I the program prints 42 when it is executed with lib-1.0.jar as expected I but the program still prints 42 when executed with lib-2.0.jar ! I the compiler inlines the constant value into the client class I this is binary compatibility but binary behavioural incompatible 50
  70. 70. Constants 2 lib-1.0.jar package lib.constants2; public class Foo f public static
  71. 71. nal String MAGIC = 42; g + lib-2.0.jar package lib.constants2; public class Foo f public static
  72. 72. nal String MAGIC = 43; g program package constants2; import lib.constants2.; public class Main f public static void main(String[] args) f System.out.println(Foo.MAGIC); g g I inlining is applied to primitive data types, but what about strings? I what will be printed to the console? 51
  73. 73. Constants 2 Solution I the program still prints 42 when it is executed with lib-2.0.jar I constant inlining is still applied when strings are used I strings are immutable objects, and in many cases can be treated like primitive types I this is binary compatibility but binary behavioural incompatible 52
  74. 74. Constants 3 lib-1.0.jar package lib.constants3; public class Foo f public static
  75. 75. nal int MAGIC = 40+2; g + lib-2.0.jar package lib.constants3; public class Foo f public static
  76. 76. nal int MAGIC = 40+3; g program package constants3; import lib.constants3.; public class Main f public static void main(String[] args) f System.out.println(Foo.MAGIC); g g I now the constant value is de
  77. 77. ned by an expression I what will be printed to the console? 53
  78. 78. Constants 3 Solution I the program still prints 42 when it is executed with lib-2.0.jar I the compiler applies constant folding I this does not seem to be speci
  79. 79. ed in [JLS] and might therefore be a compiler-speci
  80. 80. c optimisation I i.e., the expression is evaluated at compile time I this is binary compatibility but binary behavioural incompatible 54
  81. 81. Constants 4 lib-1.0.jar package lib.constants4; public class Foo f public static
  82. 82. nal Integer MAGIC = 42; g + lib-2.0.jar package lib.constants4; public class Foo f public static
  83. 83. nal Integer MAGIC = 43; g program package constants4; import lib.constants4.; public class Main f public static void main(String[] args) f System.out.println(Foo.MAGIC); g g I now the constant is de
  84. 84. ned using the wrapper type I note that assignment is safe due to autoboxing I what will be printed to the console? 55
  85. 85. Constants 4 Solution I now 43 is printed as expected when the program runs with lib-2.0.jar ! I i.e., simply by using wrapper types, constant inlining can be prevented (at least using the current version of the compiler) I A variable of primitive type or type String, that is
  86. 86. nal and initialized with a compile-time constant expression (15.28), is called a constant variable. [JLS, ch. 4.12.4] I it is unclear why wrapper types are excluded, they are immutable as well! I this may explain why many constants in projects like velocity are de
  87. 87. ned using wrapper types - to prevent inlining 56
  88. 88. Exceptions I methods can declare exceptions I for checked exceptions, the compiler forces callers to handle or rethrow the exception I at runtime, when an exception occurs the JVM searches the invocation chain of the method (stack) for a suitable exception handler [JVMS, ch. 2.10] I the compiler treats exceptions as part of the method declaration - what about the JVM? I i.e., are certain changes to exceptions (removing or specialising exceptions) binary incompatible but source compatible? 57
  89. 89. Adding a Runtime Exception lib-1.0.jar package lib.exceptions1; public class Foo f public static void foo() fg g + lib-2.0.jar package lib.exceptions1; public class Foo f public static void foo() throws UnsupportedOperationException f throw new UnsupportedOperationException(); g g program package exceptions1; public class Main f public static void main(String[] args) f lib.exceptions1.Foo.foo(); g g I in lib-2.0.jar , an UnsupportedOperation- Exception is declared and thrown I note that this is a runtime (unchecked) exception 58
  90. 90. Adding a Runtime Exception Solution I the program is source compatible with lib-2.0.jar: UnsupportedOperationException is a runtime exception, and whether it is declared or not makes no dierence I however, while the program is binary compatible, it is binary behavioural incompatible - but this is only because the exception is actually thrown in lib-2.0.jar I if the throw statement was removed, the program would become binary behavioural compatible although the exception is still declared I it would still be possible to compile the modi
  91. 91. ed program - the compiler cannot
  92. 92. gure out that a declared unchecked exception is never thrown I this makes sense - runtime exceptions are thrown implicitly (null pointers, failed casts etc) and it is too dicult for the compiler to check this 59
  93. 93. Adding a Checked Exception lib-1.0.jar package lib.exceptions2; public class Foo f public static void foo() f g g + lib-2.0.jar package lib.exceptions2; import java.io.IOException; public class Foo f public static void foo() throws IOException f throw new IOException(); g g program package exceptions2; public class Main f public static void main(String[] args) f lib.exceptions2.Foo.foo(); g g I in lib-2.0.jar , an IOException is declared or thrown I this is a checked exception 60
  94. 94. Adding a Checked Exception Solution I not surprisingly, the change is source incompatible I however, the program is binary compatible with lib-2.0.jar I i.e., the declared exception is not detected during linking as this is not part of the method descriptor I the change is binary behavioural incompatible as the exception is thrown but not caught 61
  95. 95. Generalising a Checked Exception lib-1.0.jar package lib.exceptions3; import java.io.IOException; public class Foo f public static void foo() throws IOException f throw new IOException(); g g + lib-2.0.jar package lib.exceptions3; public class Foo f public static void foo() throws Exception f throw new Exception(); g g program package exceptions3; import java.io.IOException; public class Main f public static void main(String[] args) f try f lib.exceptions3.Foo.foo(); g catch (IOException x) f System.err.println(Caught it); g g g I in lib-2.0.jar , the IOException is replaced by its super type Exception I the client program only handles the IOException 62
  96. 96. Generalising a Checked Exception Solution I the program is again source incompatible but binary compatible with lib-2.0.jar I but as before, the program behaviour changes as the exception is not caught, i.e. the change is binary behavioural incompatible 63
  97. 97. Specialising a Checked Exception lib-1.0.jar package lib.exceptions4; import java.io.IOException; public class Foo f public static void foo() throws Exception f throw new IOException(); g g + lib-2.0.jar package lib.exceptions4; import java.io.IOException; public class Foo f public static void foo() throws IOException f throw new IOException(); g g program package exceptions4; import java.io.IOException; public class Main f public static void main(String[] args) f try f lib.exceptions4.Foo.foo(); g catch (Exception x) f System.err.println(Caught it); g g g I in lib-2.0.jar , the Exception is replaced by its sub type IOException I this is similar to specialising the return type 64
  98. 98. Specialising a Checked Exception Solution I the program is source and binary compatible with lib-2.0.jar I this is (surprisingly) dierent to specialising return types I the exceptions are not part of the method descriptor used to references method when linking 65
  99. 99. Removing a Checked Exception 1 lib-1.0.jar package lib.exceptions5; import java.io.IOException; public class Foo f public static void foo() throws IOException f throw new IOException(); g g + lib-2.0.jar package lib.exceptions5; public class Foo f public static void foo() f g g program package exceptions5; import java.io.IOException; public class Main f public static void main(String[] args) f try f lib.exceptions5.Foo.foo(); g catch (IOException x) f System.err.println(Caught it); g g g I in lib-2.0.jar , the IOException is removed from the method I but note the exception handler in main 66
  100. 100. Removing a Checked Exception 1 Solution I the program is binary compatible but source incompatible with lib-2.0.jar I the compiler infers that the catch statement is not reachable because the updated foo() does not throw an exception: exception IOException is never thrown in body of corresponding try statement [JLS, ch. 14.21] 67
  101. 101. Removing a Checked Exception 2 lib-1.0.jar package lib.exceptions6; public class Foo f public static void foo() throws Exception f throw new Exception(); g g + lib-2.0.jar package lib.exceptions6; public class Foo f public static void foo() f g g program package exceptions6; public class Main f public static void main(String[] args) f try f lib.exceptions6.Foo.foo(); g catch (Exception x) f System.err.println(Caught it); g g g I in lib-2.0.jar , the Exception is removed from the method I but note the exception handler in main 68
  102. 102. Removing a Checked Exception 2 Solution I surprisingly, the program is binary compatible and source compatible with lib-2.0.jar I the compiler still considers the catch clause as reachable I this makes sense, as Exception includes runtime exceptions 69
  103. 103. Removing a Checked Exception 2 ctd Solution I however, it seems to contradict the reachability rules: A catch block C is reachable i both of the following are true: Either the type of C's parameter is an unchecked exception type or Throwable; or some expression or throw statement in the try block is reachable and can throw a checked exception whose type is assignable to the parameter of the catch clause C. [JLS, ch. 14.21] but RuntimeException and all its subclasses are, collectively, the runtime exception classes. .. The unchecked exception classes are the runtime exception classes and the error classes. [JLS, ch. 11.1.1]. I i.e., one would expect a reachability compiler error! I this has been report as a bug in the JLS, and will be
  104. 104. xed in JLS-8 (email communication with Alex Buckley) 70
  105. 105. Ghost lib-1.0.jar package lib.ghost; public class Foo f public static class Bar f public static void foo() f System.out.println(foo); g g g + lib-2.0.jar package lib.ghost.Foo; public class Bar f public static void foo() f System.out.println(foo); g g program package ghost; public class Main f public static void main(String[] args) f lib.ghost.Foo.Bar.foo(); g g I now the question is: what does this program print? 71
  106. 106. Ghost Solution I the program is not binary compatible with lib-2.0.jar: java.lang.NoClassDefFoundError: lib/ghost/Foo$Bar I however, the program is source compatible I the problem is that the reference lib.ghost.Foo.Bar can either refer to an inner class Bar within the class lib.ghost.Foo, or a top-level class Bar in the package lib.ghost.Foo I the byte code representation however diers: lib/ghost/Foo$Bar vs lib/ghost/Foo/Bar 72
  107. 107. Bridge lib-1.0.jar package lib.bridge1; public class Foo f private int foo = 0; public class Inner f @Override public String toString() f return Inner[foo=+foo+]; g g g + lib-2.0.jar package lib.bridge1; public class Foo f int foo = 0; public class Inner f @Override public String toString() f return Inner[foo=+foo+]; g g g program package bridge1; import lib.bridge1.; import java.lang.re ect.Method; public class Main f public static void main(String[] args) f Method[] mm = Foo.class.getDeclaredMethods(); for (Method m:mm) f System.out.println(m); g g g I re ection is used to
  108. 108. nd out how many methods Foo has I can this be changed by only changing the access modi
  109. 109. er of the
  110. 110. eld foo from private to default? 73
  111. 111. Bridge Solution I the program is binary and source compatible with lib-2.0.jar I however, the behaviour changes I in lib-1.0.jar, a synthetic bridge method static int lib.bridge1.Foo.access$000(lib.bridge1.Foo) is generated by the compiler to enable access to the private
  112. 112. eld by the inner class I this method is not necessary if access to the
  113. 113. eld in changed to non-private I synthetic methods [JLS, ch. 13.1] are widely used, for instances when overriding methods with generic parameter types, and co-variant return types 74
  114. 114. Summary I binary compatibility does not imply source compatibility (example: addtointerface) I binary compatibility does not imply binary behavioural compatibility (example: generics2) I source compatibility does not imply binary compatibility (example: specialiseReturnType1) I source compatibility does not imply source behavioural compatibility (example: generaliseParamType3) 75
  115. 115. Ongoing Research and Open Questions I Empirical study on Qualitas Corpus on whether and how often these problems occur in real-world programs. Proceedings IEEE CSMR-WCRE 2014. Preprint: https://sites.google.com/site/jensdietrich/ publications/preprints I Quiz developers to
  116. 116. nd out whether they are aware of this. Over 400 developers have responded,
  117. 117. rst results here: https://sites.google.com/site/jensdietrich/ java-developer-survey-2013, preprint on arxiv: http://arxiv.org/pdf/1408.2607v1.pdf I Build better tools (better than clirr) to check library compatibility, infer semantic versioning info. Some ongoing work (Uni of Western Bohemia), more planned for later 2014. 76
  118. 118. Acknowledgements I would like to thank Kamil Jezek who contributed Generics 1 and Static 1 and 2, Hussain Al Mutawa who pointed me to ghost references used in Ghost, and Alex Buckley for his comments and for contributing Adding a Method to an Interface. This work was inspired by Java Puzzlers by Joshua Bloch and Neal Gafter [PUZZ]. 77
  119. 119. References JAPI Jim des Rivieres: Evolving Java-based APIs. http://wiki.eclipse.org/Evolving_Java-based_APIs JLS James Gosling, Bill Joy, Guy Steele, Gilad Bracha and Alex Buckley: The JavaTMLanguage Speci
  120. 120. cation 7th Edition. JVMS Tim Lindholm, Frank Yellin, Gilad Bracha, and Alex Buckley. The JavaTMVirtual Machine Speci
  121. 121. cation - JavaTMSE 7 Edition. PUZZ Joshua Bloch, Neal Gafter. Java Puzzlers. Addison-Wesley 2005. 78

×