The document discusses managing binary compatibility when evolving Scala libraries. It provides examples of how changes to traits and type inference can cause incompatibilities. The author introduces a new tool called the Scala Migration Manager, which helps library maintainers determine if changes break binary compatibility and assist in migrating incompatible changes. Feedback on the tool will help prioritize future work.

1. Managing Binary Compatibility in Scala
Mirco Dotta
Typesafe
June 3, 2011
Mirco Dotta Managing Binary Compatibility in Scala

2. Introduction Sources of Incompatibility Conclusion
Outline
Introduction
Example
Scala vs. Java
Sources of Incompatibility
Type Inferencer
Trait
Conclusion
Mirco Dotta Managing Binary Compatibility in Scala

3. Introduction Sources of Incompatibility Conclusion Example Scala vs. Java
Example
Assume Analyzer is part of a library we produce. We decide that
its API has to evolve as follows:
class Analyzer { // old version class Analyzer { // new version
def analyze(issues: HashMap[ , ]) {...} def analyze(issues: Map[ , ]) {...}
} }
Further, assume the next expression was compiled against the old
library
new Analyzer().analyze(new HashMap[Any,Any]())
Would the compiled code work if run against the new library?
The answer lies in the bytecode...
Mirco Dotta Managing Binary Compatibility in Scala

4. Introduction Sources of Incompatibility Conclusion Example Scala vs. Java
Example: Bytecode
Let’s take look at the generated bytecode for the two versions:
class Analyzer { // old version class Analyzer { // new version
analyze(Lscala/collection/immutable/HashMap);V analyze(Lscala/collection/immutable/Map);V
}} }}
The expression compiled against the old library would look like:
...
invokevirtual #9;// #9 == Analyzer.analyze:(Lscala/collection/immutable/HashMap;)V
⇒ The method’s name has been statically resolved at
compile-time.
Running it against the new library would result in the JVM
throwing a NoSuchMethodException.
⇒ The evolution of class Analyzer breaks compatibility with
pre-existing binaries.
Mirco Dotta Managing Binary Compatibility in Scala

5. Introduction Sources of Incompatibility Conclusion Example Scala vs. Java
Is Binary Compatibility a Scala issue?
The short answer is No. The discussed example can be easily
ported in Java.
Scala shares with Java many sources of binary incompatibility.
But Scala offers many language features not available in Java:
Type Inferencer
First-class functions
Multiple inheritance via mixin composition (i.e., traits)
. . . Just to cite a few.
⇒ Scala code has new “unique” sources of binary incompatibility.
Mirco Dotta Managing Binary Compatibility in Scala

6. Introduction Sources of Incompatibility Conclusion Type Inferencer Trait
Type Inferencer: Member Signature
Does the following evolution break binary compatibility?
class TextAnalyzer { // old version class TextAnalyzer { // new version
def analyze(text: String) = { def analyze(text: String) = {
val issues = Map[String,Any]() val issues = new HashMap[String,Any]()
// ... // ...
issues issues
}} }}
Question
What is the inferred return type of analyze?
Let’s compare the two methods’ signature.
class TextAnalyzer { // old version class TextAnalyzer { // new version
public scala.collection.immutable.Map public scala.collection.immutable.HashMap
analyze(java.lang.String); analyze(java.lang.String);
} }
Mirco Dotta Managing Binary Compatibility in Scala

7. Introduction Sources of Incompatibility Conclusion Type Inferencer Trait
Type Inferencer: Member Signature (2)
Question
Can we prevent the method’s signature change?
That’s easy! The method’s return type has to be explicitly
declared:
class TextAnalyzer { // bytecode compatible new version
def analyze(text: String): Map[String,Any] = {
val issues = new HashMap()
// ...
issues
}}
Take Home Message
Always declare the member’s type. If you don’t, you may
inadvertently change the member’s signature.
Mirco Dotta Managing Binary Compatibility in Scala

8. Introduction Sources of Incompatibility Conclusion Type Inferencer Trait
Trait Compilation
Traits are a powerful language construct that enables
multiple-inheritance on top of a runtime – the JVM – that does
not natively support it.
Understanding how traits are compiled is crucial if you need to
ensure release-to-release binary compatibility.
So, how does the Scala compiler generate the bytecode of a trait?
There are two key elements:
A trait is compiled into an interface plus an abstract class
containing only static methods.
“Forwarder” methods are injected in classes inheriting traits.
Mirco Dotta Managing Binary Compatibility in Scala

9. Introduction Sources of Incompatibility Conclusion Type Inferencer Trait
Trait Compilation Explained
An example will help visualize how traits get compiled:
// declared in a library
trait TypeAnalyzer {
def analyze(prog: Program) {...}
}
// client code
class TypingPhase extends TypeAnalyzer
The following is the (pseudo-)bytecode generated by scalac:
interface TypeAnalyzer { class TypingPhase implements TraitAnalyzer {
void analyze(prog: Program); // forwarder method injected by scalac
} void analyze(prog: Program) {
abstract class TypeAnalyzer$class { // delegates to implementation
static void analyze($this: TypeAnalyzer, TypeAnalyzer$class.analyze(this,prog)
prog: Program) { }
// the trait’s method impl code }
}
}
Mirco Dotta Managing Binary Compatibility in Scala

10. Introduction Sources of Incompatibility Conclusion Type Inferencer Trait
Trait: Adding a concrete method
Question
Can we add a member in a trait without breaking compatibility
with pre-existing binaries?
trait TypeAnalyzer { // new version // compiled against the old version
def analyze(prog: Program) {...} class TypingPhase implements TraitAnalyzer {
def analyze(clazz: ClassInfo) {...} // forwarder method injected by scalac
} void analyze(prog: Program) {
// delegates to implementation
TypeAnalyzer$class.analyze(this,prog)
//TypeAnalyzer trait compiled }
interface TypeAnalyzer { // missing concrete implementation!
void analyze(prog: Program); ??analyze(clazz: ClassInfo)??
void analyze(clazz: ClassInfo); }
}
abstract class TypeAnalyzer$class {
static void analyze($this: TypeAnalyzer, Take Home Message
prog: Program{...}
static void analyze($this: TypeAnalyzer, Adding a concrete method in a trait
clazz: ClassInfo) {...} breaks binary compatibility.
}
Mirco Dotta Managing Binary Compatibility in Scala

11. Introduction Sources of Incompatibility Conclusion Conclusion Future Work Scala Migration Manager
Conclusion
Ensuring release-to-release binary compatibility of Scala
libraries is possible.
Though, sometimes it can be difficult to tell if a change in the
API of a class/trait will break pre-existing binaries.
In the discussed examples we have seen that:
Type inferencer may be at the root of changes in the signature
of a method.
Traits are a sensible source of binary incompatibilities.
It really looks like library’s maintainers’ life ain’t that easy...
Mirco Dotta Managing Binary Compatibility in Scala

12. Introduction Sources of Incompatibility Conclusion Conclusion Future Work Scala Migration Manager
Introducing the Scala Migration Manager (MiMa)
Today we release the Scala Migration Manager! (beta)
It’s free!!
It will tell you, library maintainers, if your next release is
binary compatible with the current one.
It will tell you, libraries users, if two releases of a library are
binary compatible.
MiMa can collect and report all sources of “syntactic” binary
incompatibilities between two releases of a same library.
“Syntactic” means NO LinkageError (e.g.,
NoSuchMethodException) will ever be thrown at runtime.
Now it’s time for a demo!
Mirco Dotta Managing Binary Compatibility in Scala

13. Introduction Sources of Incompatibility Conclusion Conclusion Future Work Scala Migration Manager
Future Work
Reporting binary incompatibilities is only half of the story. We are
already working on a “companion” tool that will help you migrate
binary incompatibilities.
For the reporting there are many ideas spinning around. Your
feedback will help us decide what brings you immediate value
One that I believe is useful:
Maven/Sbt integration.
Mirco Dotta Managing Binary Compatibility in Scala

14. Introduction Sources of Incompatibility Conclusion Conclusion Future Work Scala Migration Manager
Scala Migration Manager
Visit http://typesafe.com/technology/migration-manager
More information about the Migration Manager
Download it and try it out, it’s free!
We want to hear back from you.
Success stories
Request new features
Report bugs
Want to know more, make sure to get in touch!
Mirco Dotta, email: mirco.dotta@typesafe.com, twitter: @mircodotta
Mirco Dotta Managing Binary Compatibility in Scala