Your SlideShare is downloading. ×
  • Like
Managing Binary Compatibility in Scala (Scala Lift Off 2011)
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Managing Binary Compatibility in Scala (Scala Lift Off 2011)

  • 943 views
Published

Slides of my Scala Lift Off 2011 talk. The content of the presentation is mostly similar to the one presented at Scala Days 2011, with a few additions. Particularly, lazy values are discussed.

Slides of my Scala Lift Off 2011 talk. The content of the presentation is mostly similar to the one presented at Scala Days 2011, with a few additions. Particularly, lazy values are discussed.

Published in Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
943
On SlideShare
0
From Embeds
0
Number of Embeds
5

Actions

Shares
Downloads
11
Comments
0
Likes
4

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Managing Binary Compatibility in Scala Mirco Dotta Typesafe October 13, 2011 Mirco Dotta Managing Binary Compatibility in Scala
  • 2. Introduction Sources of Incompatibility ConclusionOutline Introduction Example Scala vs. Java Sources of Incompatibility Type Inferencer Type Parameters Trait Lazy Fields Conclusion Mirco Dotta Managing Binary Compatibility in Scala
  • 3. Introduction Sources of Incompatibility Conclusion Example Scala vs. JavaExample Assume Analyzer is part of a library we produce. We decide that its API has to evolve as follows:class Analyzer { // old version class Analyzer { // new version def analyze(issues: HashMap[ , ]) {...} def analyze(issues: Map[ , ]) {...}} } Further, assume the next expression was compiled against the old library new Analyzer().analyze(new HashMap[Any,Any]) Would the compiled code work if run against the new library? The answer lies in the bytecode... Mirco Dotta Managing Binary Compatibility in Scala
  • 4. Introduction Sources of Incompatibility Conclusion Example Scala vs. Java Example: Bytecode Let’s have a look at the method signature for the two versions:class Analyzer { // old version class Analyzer { // new version analyze(Lscala/collection/immutable/HashMap);V analyze(Lscala/collection/immutable/Map);V} } The expression compiled against the old library would look like: ... invokevirtual #9;// #9 == Analyzer.analyze:(Lscala/collection/immutable/HashMap;)V ⇒ The method’s name has been statically resolved at compile-time. Running it against the new library would result in the JVM throwing a NoSuchMethodException. ⇒ The evolution of class Analyzer breaks compatibility with pre-existing binaries. Mirco Dotta Managing Binary Compatibility in Scala
  • 5. Introduction Sources of Incompatibility Conclusion Example Scala vs. JavaIs Binary Compatibility a Scala issue? The short answer is No. The discussed example can be easily ported in Java or other languages targeting the JVM. Scala shares with Java many sources of binary incompatibility. But Scala offers many language features not available in Java: First-class functions. Type Inferencer. Multiple inheritance via mixin composition (i.e., traits). Lazy values. . . . Just to cite a few. ⇒ Scala code has new “unique” sources of binary incompatibility. Mirco Dotta Managing Binary Compatibility in Scala
  • 6. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy FieldsType Inferencer: Member Signature Does the following evolution break binary compatibility? class TextAnalyzer { // old version class TextAnalyzer { // new version def analyze(text: String) = { def analyze(text: String) = { val issues = Map[String,Any]() val issues = new HashMap[String,Any] // ... // ... issues issues }} }} Question What is the inferred return type of analyze? Let’s compare the two methods’ signature.class TextAnalyzer { // old version class TextAnalyzer { // new version public scala.collection.immutable.Map public scala.collection.immutable.HashMap analyze(java.lang.String); analyze(java.lang.String);} } Mirco Dotta Managing Binary Compatibility in Scala
  • 7. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy FieldsType Inferencer: Member Signature (2) Question Can we prevent the method’s signature change? That’s easy! The method’s return type has to be explicitly declared: class TextAnalyzer { // bytecode compatible new version def analyze(text: String): Map[String,Any] = { val issues = new HashMap[String,Any] // ... issues }} Take Home Message Always declare the member’s type. If you don’t, you may inadvertently change the members’ signature. Mirco Dotta Managing Binary Compatibility in Scala
  • 8. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy FieldsType Parameters & Type Erasure Assume NodeImpl <: Node. Does the following evolution break binary compatibility? class Tree[T <: NodeImpl] { // old version class Tree[T <: Node] { // new version def contains(e: T): Boolean = {...} def contains(e: T): Boolean = {...} } } Type parameters are erased during compilation, but the erasure of Tree.contains is different for the two declarations. boolean contains(NodeImpl) {...} boolean contains(Node) {...} Remember that the JVM looks up methods based on the textual representation of the signature, along with the method name. Take Home Message Type parameters may change the members’ signature and hence break binary compatibility. Mirco Dotta Managing Binary Compatibility in Scala
  • 9. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy FieldsTrait Compilation Traits are a powerful language construct that enables multiple-inheritance on top of a runtime – the JVM – that does not natively support it. Understanding how traits are compiled is crucial if you need to ensure release-to-release binary compatibility. So, how does the Scala compiler generate the bytecode of a trait? There are two key elements: A trait is compiled into an interface plus an abstract class containing only static methods. “Forwarder” methods are injected in classes inheriting traits. Mirco Dotta Managing Binary Compatibility in Scala
  • 10. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy FieldsTrait Compilation Explained An example will help visualize how traits get compiled:// declared in a librarytrait TypeAnalyzer { def analyze(prog: Program) { // client code // the trait’s method impl code class TypingPhase extends TypeAnalyzer }} The following is the (pseudo-)bytecode generated by scalac:interface TypeAnalyzer { class TypingPhase implements TraitAnalyzer { void analyze(Program prog); // forwarder method injected by scalac} void analyze(Program prog) {abstract class TypeAnalyzer$class { // delegates to implementation static void analyze(TypeAnalyzer $this, TypeAnalyzer$class.analyze(this,prog) Program prog) { } // the trait’s method impl code } }} Mirco Dotta Managing Binary Compatibility in Scala
  • 11. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy FieldsTrait: Adding a concrete method Question Can we add a member in a trait without breaking compatibility with pre-existing binaries? Mirco Dotta Managing Binary Compatibility in Scala
  • 12. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy FieldsTrait: Adding a concrete methodtrait TypeAnalyzer { // new version // compiled against the old version def analyze(prog: Program) {...} class TypingPhase implements TraitAnalyzer { def analyze(clazz: ClassInfo) {...} // forwarder method injected by scalac} void analyze(Program prog) { // delegates to implementation TypeAnalyzer$class.analyze(this,prog)//TypeAnalyzer trait compiled }interface TypeAnalyzer { // missing concrete implementation! void analyze(Program prog); ??analyze(ClassInfo clazz)?? void analyze(ClassInfo clazz); }}abstract class TypeAnalyzer$class { static void analyze(TypeAnalyzer $this, Take Home Message Program prog) {...} static void analyze(TypeAnalyzer $this, Adding a concrete method in a trait ClassInfo clazz) {...} breaks binary compatibility.} Mirco Dotta Managing Binary Compatibility in Scala
  • 13. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy FieldsLazy values decompiled Lazy values are a Scala language feature that has no correspondent in the JVM. ⇒ Their semantic has to be encoded in terms of the available JVM’s primitives. A lazy val is encoded into a private field and a getter plus a bitmap. The bitmap is used by the getter for ensuring that the private field gets initialized only once. The bitmap is of type Int, meaning that a maximum of 32 lazy value can be correctly handled by it. The bitmap is shared with subclasses, so that a new bitmap field is created only if strictly necessary. Mirco Dotta Managing Binary Compatibility in Scala
  • 14. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy FieldsLazy to Eager Question Can we transform a lazy field into an eager one? Mirco Dotta Managing Binary Compatibility in Scala
  • 15. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy FieldsLazy to Eager Let’s take a simple example and look at the generated bytecode: class ClassAnalyzer { // old version class ClassAnalyzer { // new version lazy val superclasses: List[Class] = { val superclasses: List[Class] = { // ... // ... } } } } And here is the bytecode: class ClassAnalyzer { // old version class ClassAnalyzer { // new version private List[Class] superclasses; private final List[Class] superclasses; volatile int bitmap$0 public int superclasses(); public int superclasses(); // ... // ... } } Take Home Message Transforming a lazy value into an eager one does preserve compatibility with pre-existing binaries. Mirco Dotta Managing Binary Compatibility in Scala
  • 16. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy FieldsEager to Lazy Question Can we transform an eager field into a lazy one? Mirco Dotta Managing Binary Compatibility in Scala
  • 17. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy FieldsEager to Lazy The previous example demonstrates that lazy and eager fields are syntactically equivalent, from a client perspective. Does that mean that the transformation is safe? It depends. If you are in a closed world, then the transformation is safe (e.g., your class is final). Otherwise, you are taking a high risk on breaking the semantic of lazy, and no good can come out of that. Remember that the bitmap is shared between subclasses. Take Home Message Transforming an eager value into a lazy one may not preserve compatibility with pre-existing binaries. Mirco Dotta Managing Binary Compatibility in Scala
  • 18. Introduction Sources of Incompatibility Conclusion Conclusion Future Work Scala Migration ManagerConclusion Ensuring release-to-release binary compatibility of Scala libraries is possible. Though, sometimes it can be difficult to tell if a change in the API of a class/trait will break pre-existing binaries. In the discussed examples we have seen that: Type inferencer may be at the root of changes in the member’s signature. Type parameters may also modify members’ signature. Traits are a sensible source of binary incompatibilities. Transforming an eager field into a lazy one can break semantic. It really looks like library’s maintainers’ life ain’t that easy... Mirco Dotta Managing Binary Compatibility in Scala
  • 19. Introduction Sources of Incompatibility Conclusion Conclusion Future Work Scala Migration ManagerScala Migration Manager (MiMa) A few months ago we released the Scala Migration Manager! It’s free!! It can tell you, library maintainers, if your next release is binary compatible with the current one. It can tell you, libraries users, if two releases of a library are binary compatible. MiMa can collect and report all sources of “syntactic” binary incompatibilities between two releases of a same library. “Syntactic” means NO LinkageError (e.g., NoSuchMethodException) will ever be thrown at runtime. Now, it’s time for a short demo! Mirco Dotta Managing Binary Compatibility in Scala
  • 20. Introduction Sources of Incompatibility Conclusion Conclusion Future Work Scala Migration ManagerFuture Work Reporting binary incompatibilities is only half of the story. We are working on a “companion” tool that will help you migrate binary incompatibilities. For the reporting there are many ideas spinning around. Your feedback will help us decide what brings you immediate value One that I believe is useful: Maven/Sbt integration. Mirco Dotta Managing Binary Compatibility in Scala
  • 21. Introduction Sources of Incompatibility Conclusion Conclusion Future Work Scala Migration ManagerScala Migration Manager Visit http://typesafe.com/technology/migration-manager More information about the Migration Manager Download it and try it out, it’s free! We want to hear back from you. Success stories Request new features Report bugs Want to know more, make sure to get in touch! email: mirco.dotta@typesafe.com, twitter: @mircodotta Mirco Dotta Managing Binary Compatibility in Scala
  • 22. Introduction Sources of Incompatibility Conclusion Conclusion Future Work Scala Migration ManagerOne last thing I didn’t mention... We will make the sources public! Mirco Dotta Managing Binary Compatibility in Scala