SlideShare a Scribd company logo
1 of 37
Download to read offline
The Binary Compatibility 
Challenge 
Martin Odersky 
Typesafe and EPFL
The Problem in a Nutshell 
• Binary compatibility has been an issue ever since Scala 
became popular. 
• Causes grief when building, friction for upgrading. 
• The community has learned to deal with this by 
becoming more conservative. 
• But this makes it harder to innovate and improve. 
Break your client’s builds vs Freeze, and stop improving 
Is there no third way? 
2
What is Binary Compatibility? 
Binary compatibility ≠ Source compatibility 
Source & binary incompatible 
object 
Client 
{ 
msg.length 
} 
object 
Server 
{ 
val 
msg 
= 
“abc” 
} 
object 
Server 
{ 
val 
msg 
= 
Some(“abc”) 
}
What is Binary Compatibility? 
Binary compatibility ≠ Source compatibility 
Source incompatible, binary compatible: 
object 
Client 
{ 
import 
a, 
b 
val 
x: 
String 
= 
1 
} 
object 
a 
{ 
implicit 
def 
f(x: 
Int): 
String 
= 
x.toString 
} 
object 
b 
object 
a 
{ 
implicit 
def 
f(x: 
Int): 
String 
= 
x.toString 
} 
object 
b 
{ 
implicit 
def 
g(x: 
Int): 
String 
= 
”abc” 
}
What is Binary Compatibility? 
Binary compatibility ≠ Source compatibility 
Source compatible, binary incompatible: 
object 
Apple 
extends 
Edible 
{ 
def 
joules 
= 
500000 
} 
trait 
Edible 
{ 
def 
joules: 
Double 
} 
trait 
Edible 
{ 
def 
joules: 
Double 
def 
calories 
= 
joules 
* 
4.184 
} 
 
èNeed to recompile on Java 1-7 
In Java 8 it’s more complex but fundamentally the same.
What is Binary Compatibility? 
Binary compatibility ≠ Source compatibility 
Source compatible, binary incompatible: 
object 
Apple 
extends 
Edible 
{ 
def 
joules 
= 
500000 
} 
trait 
Edible 
{ 
def 
joules: 
Double 
def 
calories: 
Double 
} 
trait 
Edible 
{ 
object 
def 
Edible$joules: 
class 
Double 
{ 
} 
def 
calories($this: 
Edible): 
Double 
= 
$this.joules 
* 
4.184 
trait 
Edible 
{ 
def 
joules: 
Double 
def 
calories 
= 
joules 
* 
4.184 
} 
} 
object 
Apple 
extends 
Edible 
{ 
def 
joules 
= 
500000.0 
def 
calories: 
Double 
= 
Edibl$class.calories(this) 
} 
 
èNeed to recompile on Java 1-7 
In Java 8 it’s more complex but fundamentally the same.
Other Issues 
Compiler optimizations and bug fixes can affect binary 
compatibility. 
Example: Implementation of lazy values. 
trait 
Edible 
{ 
def 
joules: 
Double 
lazy 
val 
def 
calories 
= 
joules 
* 
4.184 
} 
object 
Apple 
extends 
Edible 
{ 
def 
joules 
= 
500000 
} 
object 
Apple 
extends 
Edible 
{ 
def 
joules 
= 
500000.0 
private 
var 
initFlags: 
BitSet 
private 
var 
cals: 
Int 
= 
_ 
def 
calories 
= 
{ 
if 
(!initFlags(N)) 
{ 
cals 
= 
Edible$class.initCals(this) 
initFlags(N) 
= 
true 
} 
cals 
}} 
Previously: 
1 bit per lazy val 
To avoid deadlocks: 
2 bits. 
è all offsets change!
Compiler Pipeline 
Parser 
Typer 
SyntheticMethods 
SuperAccessors 
RefChecks 
ElimRepeated 
ElimLocals 
ExtensionMethods 
TailRec 
PatternMatcher 
ExplicitOuter 
Erasure 
Mixin 
Memoize 
LazyVals 
CapturedVars 
Constructors 
LambdaLift 
Flatten 
RestoreScopes 
Cleanup 
more GenBCode 
phases 
Source 
Symbols 
JVM 
Byte-code 
Lots of 
scope for 
things to 
go wrong!
Where It Breaks 
A.class 
C.class 
C.class 
C.scala C.scala 
(binary incompatible 
source change)
Why Is This Such a Big Problem? 
MyApplication 
DustyLegacyLib 
Scala 
Library 
2.10 
X can’t upgrade to 
(too old, can’t rebuild) 
Scala 
Library 
2.11 
Seq.scala Seq.scala 
(binary incompatible 
source change) 
Scala 2.11!
Not Just A Problem with Scala-Library 
MyApplication 
DustyLegacyLib 
Akka 
3.2 
X can’t upgrade to 
(can’t rebuild) 
Akka 
3.3 
Actor.scala Actor.scala 
(binary incompatible 
source change) 
Akka 3.3!
Not Just A Problem with Scala-Library 
MyApplication 
DustyLegacyLib 
shapeless 
2.0 
X can’t upgrade to 
(can’t rebuild) 
shapeless 
2.1 
unions.scala unions.scala 
(binary incompatible 
source change) 
shapeless 2.1!
Dealing With It So Far 
“MiMa” tool can detect binary incompatibilities. 
Scala release policy: 
– Minor versions need to be (forwards and backwards) binary 
compatible. 
– Major versions are allowed to break binary compatibility 
– Major versions are released rarely (+18 months between them). 
Problem: 
– 3rd party libraries need similar policies but often don’t enforce 
them. 
– Innovation is stifled. 
– Simple fixes have to wait for a long time to get in. 
– Lots of dev cycles spent on dealing with binary compatibility.
What Do Others Do? 
Java: 
• Language close to JVM bytecode. 
• Innovation happens on JVM level. 
– Either in the JVM itself or through reflection. 
– E.g. Java 8 lambdas, default methods. 
• Libraries are frozen when they appear. 
– E.g. java.util.Date 
• Language is restricted in terms of extensibility 
– E.g. nterface1, interface2, ... interface7 in Eclipse.
What Do Others Do? 
OSGI: Allow multiple versions of a library in an application 
MyApplication 
DustyLegacyLib 
Scala 
Library 
2.10 
MyApplication 
Scala 
Library 
2.11 
rebuild 
• Fragile, requires serious classloader magic. 
• Few frameworks beyond Eclipse have bought in.
What Do Others Do? 
C/C++: 
• Relies on Linker for more flexibility in interfaces. 
• Not that great a story either (c.f. DLL Hell).
What Do Others Do? 
Clojure: 
• Builds from source.
What Do Others Do? 
Javascript: 
• Builds from source.
What Do Others Do? 
Python: 
• Builds from source.
What Do Others Do? 
Ruby: 
• Builds from source.
What Do Others Do? 
Go: 
• Builds from source.
Why Can’t Scala Build from Source? 
No standard Build Tool 
Should we standardize on SBT, Gradle, Maven, Ivy, Ant? 
Reproducible builds are rare. 
Chicken and egg problem: 
Because everyone is used to binary builds, nobody* invests in 
making builds reproducible 
*Not quite true: Typesafe has invested in community build, can now 
build more than 1M lines of community projects. But it’s a huge 
effort.
What We Need 
• An interchange format 
that captures the essence 
of Scala dependencies. 
• This cannot be the JVM 
bytecode format 
• Nor can it be source 
23
The Idea 
Use Typed Trees as an 
interchange format. 
– More robust than source. 
– More stable than JVM 
bytecode. 
– Efficient?
New Compiler Pipeline 
Parser 
Typer 
SyntheticMethods 
SuperAccessors 
RefChecks 
ElimRepeated 
ElimLocals 
ExtensionMethods 
TailRec 
PatternMatcher 
ExplicitOuter 
Erasure 
Mixin 
Memoize 
LazyVals 
CapturedVars 
Constructors 
LambdaLift 
Flatten 
RestoreScopes 
Cleanup 
GenBCode 
Typed Trees 
Typed Trees 
Source 
Frontend Backend Bytecode
How To Build 
A.class 
| 
ATree 
| 
HashA1 
B.class 
| 
BTree 
| 
HashB1 
C.class 
| 
CTree 
| 
HashC1 
A.class 
| 
ATree 
| 
HashA2 
B.class 
| 
BTree 
| 
HashB2 
C.class 
| 
CTree 
| 
HashC2 
(rebuild from 
BTree) 
C.scala C.scala 
(source change) 
(rebuild)
More Robust Than Source 
Source 
Parser 
Typer 
SyntheticMethods 
SuperAccessors 
Typed Trees 
Frontend 
Ø Resolve Names 
Ø scan packages 
Ø handle imports 
Ø establish implicit scopes 
Ø Resolve Overloading 
Ø Find Implicits 
Ø Apply Conversions 
Ø Infer Type Parameters 
Ø Assign Types to Trees 
(5374 lines) 
A lot can go wrong here!
More Robust Than Source 
RefChecks 
ElimRepeated 
ElimLocals 
ExtensionMethods 
TailRec 
PatternMatcher 
ExplicitOuter 
Erasure 
Mixin 
Memoize 
LazyVals 
CapturedVars 
Constructors 
LambdaLift 
Flatten 
RestoreScopes 
Cleanup 
GenBCode 
Typed Trees 
Backend Bytecode 
Assign Types 
(311 lines)
More Resilient Than Bytecode 
Can 
– add fields and methods to traits 
– add lazy vals anywhere 
– change compilation scheme in any way necessary. 
None of these would be binary compatible! 
Can also 
– add or remove implicits 
– add methods anywhere 
– change imports 
All of these could be source incompatible!
Efficient? 
Can typed trees be efficient enough to build million+ line 
systems? 
Possible issues: 
• Size of trees 
– on disk 
– in memory 
• Transformation time 
30
Potential Issue: Tree Size 
xs.filter(_ 
= 
0) 
becomes: Apply 
Select 
Ident 
“filter” 
“xs” 
:: 
Block 
DefDef 
“anonfun” 
:: 
:: 
ValDef 
“$x” 
Nil 
Nil 
Nil 
TypeTree: 
Int 
Apply 
Select 
Ident 
“$x” 
“=” 
Literal 
0 
Literal 
“anonfun” 
:: 
16 Nodes, Nil 
not counting types 
17 chars
Back of the Envelope Calculation: 
16 nodes 
Average size of node: 32 bytes 
512 bytes total. 
Double that to include type info. 
è 16 bytes source à 1KB tree (factor 64 blow-up). 
For a 1M line system 
30MB source à 2GB trees. 
32
A More Compact Representation 
Apply 
(34) 
SelectTermWithSig 
(9) 
Ident 
(3) 
“xs” 
“filter” 
“Function1 
-­‐ 
Boolean” 
Closure 
(23) 
ParamDef 
(7) 
“x” 
TypeRef 
“scala.Int” 
Apply 
(14) 
SelectTermWithSig 
(9) 
Ident 
(3) 
“x$” 
“=“ 
“Integer 
-­‐ 
Boolean” 
Literal 
(3) 
0 
33 
Still navigable, 
because inner nodes 
contain size of total tree 
derived from them 
Types or symbols given 
at the leaves. 
Types of inner nodes are 
reconstituted using the 
TypeAssigner.
Speed 
Transformation + byte-code generation amounts to ~ 60% 
of total compile time. 
We can speed this up by 
– fusing phases, reducing amount of intermediate trees, 
– using a fast type assigner, instead of a slow typer, 
– building different files in parallel. 
Besides, can use incremental compilation. 
– Compile only this units that depend on changed libraries. 
– Need to do that only once. 
34
Other Benefits 
Optimization 
– Typed trees are a great format for interprocedural analyses 
– Inlining across compilation units made simple 
– Inlining without binary compatibility issues 
Program Analysis 
– Types trees are close to source, but easy to traverse 
– Ideal for context-dependent program analyses such as FindBugs 
– Ideal for instrumentation 
Portability 
– Typed trees allow retargeting to different backends, as long as 
dependencies exist. 
– Allow libraries to be used on JVM, JS, LLVM... without needing 
explicit recompilation. 
35
Common Intermediate Format 
36 
New 
Backend 
dotc 
Frontend 
scalac 
Frontend 
Old 
Backend 
GenBCode 
Bytecode
Conclusion 
37 
Typed 
trees 
can fix the 
binary 
compatibility 
problem and they 
offer lots of 
other benefits, too. 
Let’s start the 
work to make them 
real!

More Related Content

What's hot

Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009Martin Odersky
 
What To Leave Implicit
What To Leave ImplicitWhat To Leave Implicit
What To Leave ImplicitMartin Odersky
 
Introduction to Scala
Introduction to ScalaIntroduction to Scala
Introduction to ScalaRahul Jain
 
Scala Days San Francisco
Scala Days San FranciscoScala Days San Francisco
Scala Days San FranciscoMartin Odersky
 
Compilers Are Databases
Compilers Are DatabasesCompilers Are Databases
Compilers Are DatabasesMartin Odersky
 
Introduction to Functional Programming with Scala
Introduction to Functional Programming with ScalaIntroduction to Functional Programming with Scala
Introduction to Functional Programming with Scalapramode_ce
 
Haskell vs. F# vs. Scala
Haskell vs. F# vs. ScalaHaskell vs. F# vs. Scala
Haskell vs. F# vs. Scalapt114
 
What To Leave Implicit
What To Leave ImplicitWhat To Leave Implicit
What To Leave ImplicitMartin Odersky
 
Scala : language of the future
Scala : language of the futureScala : language of the future
Scala : language of the futureAnsviaLab
 
Advanced Functional Programming in Scala
Advanced Functional Programming in ScalaAdvanced Functional Programming in Scala
Advanced Functional Programming in ScalaPatrick Nicolas
 
A Brief Intro to Scala
A Brief Intro to ScalaA Brief Intro to Scala
A Brief Intro to ScalaTim Underwood
 
The Evolution of Scala / Scala進化論
The Evolution of Scala / Scala進化論The Evolution of Scala / Scala進化論
The Evolution of Scala / Scala進化論scalaconfjp
 
Introduction to Scala
Introduction to ScalaIntroduction to Scala
Introduction to ScalaSaleem Ansari
 
Functional Programming In Practice
Functional Programming In PracticeFunctional Programming In Practice
Functional Programming In PracticeMichiel Borkent
 

What's hot (19)

Quick introduction to scala
Quick introduction to scalaQuick introduction to scala
Quick introduction to scala
 
Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009
 
What To Leave Implicit
What To Leave ImplicitWhat To Leave Implicit
What To Leave Implicit
 
Introduction to Scala
Introduction to ScalaIntroduction to Scala
Introduction to Scala
 
Preparing for Scala 3
Preparing for Scala 3Preparing for Scala 3
Preparing for Scala 3
 
Scala Days San Francisco
Scala Days San FranciscoScala Days San Francisco
Scala Days San Francisco
 
Compilers Are Databases
Compilers Are DatabasesCompilers Are Databases
Compilers Are Databases
 
Introduction to Functional Programming with Scala
Introduction to Functional Programming with ScalaIntroduction to Functional Programming with Scala
Introduction to Functional Programming with Scala
 
Haskell vs. F# vs. Scala
Haskell vs. F# vs. ScalaHaskell vs. F# vs. Scala
Haskell vs. F# vs. Scala
 
What To Leave Implicit
What To Leave ImplicitWhat To Leave Implicit
What To Leave Implicit
 
Scala : language of the future
Scala : language of the futureScala : language of the future
Scala : language of the future
 
Scala Intro
Scala IntroScala Intro
Scala Intro
 
Simplicitly
SimplicitlySimplicitly
Simplicitly
 
Advanced Functional Programming in Scala
Advanced Functional Programming in ScalaAdvanced Functional Programming in Scala
Advanced Functional Programming in Scala
 
A Brief Intro to Scala
A Brief Intro to ScalaA Brief Intro to Scala
A Brief Intro to Scala
 
The Evolution of Scala / Scala進化論
The Evolution of Scala / Scala進化論The Evolution of Scala / Scala進化論
The Evolution of Scala / Scala進化論
 
Scala Days NYC 2016
Scala Days NYC 2016Scala Days NYC 2016
Scala Days NYC 2016
 
Introduction to Scala
Introduction to ScalaIntroduction to Scala
Introduction to Scala
 
Functional Programming In Practice
Functional Programming In PracticeFunctional Programming In Practice
Functional Programming In Practice
 

Viewers also liked

Implementing Higher-Kinded Types in Dotty
Implementing Higher-Kinded Types in DottyImplementing Higher-Kinded Types in Dotty
Implementing Higher-Kinded Types in DottyMartin Odersky
 
Scala - the good, the bad and the very ugly
Scala - the good, the bad and the very uglyScala - the good, the bad and the very ugly
Scala - the good, the bad and the very uglyBozhidar Bozhanov
 
Introduction to OpenCV (with Java)
Introduction to OpenCV (with Java)Introduction to OpenCV (with Java)
Introduction to OpenCV (with Java)Luigi De Russis
 
これからはじめるCoda2とSublime Text 2
これからはじめるCoda2とSublime Text 2これからはじめるCoda2とSublime Text 2
これからはじめるCoda2とSublime Text 2masaaki komori
 
new Objctive-C literal syntax
new Objctive-C literal syntaxnew Objctive-C literal syntax
new Objctive-C literal syntaxWataru Kimura
 
Next Generation Web Application Architecture
Next Generation Web Application ArchitectureNext Generation Web Application Architecture
Next Generation Web Application ArchitectureKoji SHIMADA
 
blogサービスの全文検索の話 - #groonga を囲む夕べ
blogサービスの全文検索の話 - #groonga を囲む夕べblogサービスの全文検索の話 - #groonga を囲む夕べ
blogサービスの全文検索の話 - #groonga を囲む夕べMasahiro Nagano
 
Rubyはとても「人間的」
Rubyはとても「人間的」Rubyはとても「人間的」
Rubyはとても「人間的」Kazuhiro Serizawa
 
Ruby1.9のfiberのかっこいい使い方
Ruby1.9のfiberのかっこいい使い方Ruby1.9のfiberのかっこいい使い方
Ruby1.9のfiberのかっこいい使い方Kindai University
 
本格的に始めるzsh
本格的に始めるzsh本格的に始めるzsh
本格的に始めるzshHideaki Miyake
 
いまさら聞けないDocker - 第5回コンテナ型仮想化の情報交換会@大阪
いまさら聞けないDocker - 第5回コンテナ型仮想化の情報交換会@大阪いまさら聞けないDocker - 第5回コンテナ型仮想化の情報交換会@大阪
いまさら聞けないDocker - 第5回コンテナ型仮想化の情報交換会@大阪Kunihiro TANAKA
 
プログラマとして仕事をするために勉強すること
プログラマとして仕事をするために勉強することプログラマとして仕事をするために勉強すること
プログラマとして仕事をするために勉強することなおき きしだ
 
受託開発時におけるAWSクラウド活用術
受託開発時におけるAWSクラウド活用術受託開発時におけるAWSクラウド活用術
受託開発時におけるAWSクラウド活用術Hiroshi Koyama
 
Code as data as code.
Code as data as code.Code as data as code.
Code as data as code.Mike Fogus
 
ARC環境で メモリリークを起こす 7つの方法
ARC環境で メモリリークを起こす 7つの方法ARC環境で メモリリークを起こす 7つの方法
ARC環境で メモリリークを起こす 7つの方法ushiostarfish _
 

Viewers also liked (20)

From DOT to Dotty
From DOT to DottyFrom DOT to Dotty
From DOT to Dotty
 
Implementing Higher-Kinded Types in Dotty
Implementing Higher-Kinded Types in DottyImplementing Higher-Kinded Types in Dotty
Implementing Higher-Kinded Types in Dotty
 
Flatmap
FlatmapFlatmap
Flatmap
 
Scala - the good, the bad and the very ugly
Scala - the good, the bad and the very uglyScala - the good, the bad and the very ugly
Scala - the good, the bad and the very ugly
 
Introduction to OpenCV (with Java)
Introduction to OpenCV (with Java)Introduction to OpenCV (with Java)
Introduction to OpenCV (with Java)
 
これからはじめるCoda2とSublime Text 2
これからはじめるCoda2とSublime Text 2これからはじめるCoda2とSublime Text 2
これからはじめるCoda2とSublime Text 2
 
Sinatraのススメ
SinatraのススメSinatraのススメ
Sinatraのススメ
 
new Objctive-C literal syntax
new Objctive-C literal syntaxnew Objctive-C literal syntax
new Objctive-C literal syntax
 
Next Generation Web Application Architecture
Next Generation Web Application ArchitectureNext Generation Web Application Architecture
Next Generation Web Application Architecture
 
blogサービスの全文検索の話 - #groonga を囲む夕べ
blogサービスの全文検索の話 - #groonga を囲む夕べblogサービスの全文検索の話 - #groonga を囲む夕べ
blogサービスの全文検索の話 - #groonga を囲む夕べ
 
Rubyはとても「人間的」
Rubyはとても「人間的」Rubyはとても「人間的」
Rubyはとても「人間的」
 
Ruby1.9のfiberのかっこいい使い方
Ruby1.9のfiberのかっこいい使い方Ruby1.9のfiberのかっこいい使い方
Ruby1.9のfiberのかっこいい使い方
 
本格的に始めるzsh
本格的に始めるzsh本格的に始めるzsh
本格的に始めるzsh
 
いまさら聞けないDocker - 第5回コンテナ型仮想化の情報交換会@大阪
いまさら聞けないDocker - 第5回コンテナ型仮想化の情報交換会@大阪いまさら聞けないDocker - 第5回コンテナ型仮想化の情報交換会@大阪
いまさら聞けないDocker - 第5回コンテナ型仮想化の情報交換会@大阪
 
プログラマとして仕事をするために勉強すること
プログラマとして仕事をするために勉強することプログラマとして仕事をするために勉強すること
プログラマとして仕事をするために勉強すること
 
Project Lambdaの基礎
Project Lambdaの基礎Project Lambdaの基礎
Project Lambdaの基礎
 
受託開発時におけるAWSクラウド活用術
受託開発時におけるAWSクラウド活用術受託開発時におけるAWSクラウド活用術
受託開発時におけるAWSクラウド活用術
 
Code as data as code.
Code as data as code.Code as data as code.
Code as data as code.
 
ARC環境で メモリリークを起こす 7つの方法
ARC環境で メモリリークを起こす 7つの方法ARC環境で メモリリークを起こす 7つの方法
ARC環境で メモリリークを起こす 7つの方法
 
ES6 at PayPal
ES6 at PayPalES6 at PayPal
ES6 at PayPal
 

Similar to Scalax

NoSQL Introduction, Theory, Implementations
NoSQL Introduction, Theory, ImplementationsNoSQL Introduction, Theory, Implementations
NoSQL Introduction, Theory, ImplementationsFirat Atagun
 
http://www.hfadeel.com/Blog/?p=151
http://www.hfadeel.com/Blog/?p=151http://www.hfadeel.com/Blog/?p=151
http://www.hfadeel.com/Blog/?p=151xlight
 
MySQL And Search At Craigslist
MySQL And Search At CraigslistMySQL And Search At Craigslist
MySQL And Search At CraigslistJeremy Zawodny
 
Linq To The Enterprise
Linq To The EnterpriseLinq To The Enterprise
Linq To The EnterpriseDaniel Egan
 
Introducing Scala to your Ruby/Java Shop : My experiences at IGN
Introducing Scala to your Ruby/Java Shop : My experiences at IGNIntroducing Scala to your Ruby/Java Shop : My experiences at IGN
Introducing Scala to your Ruby/Java Shop : My experiences at IGNManish Pandit
 
Front Range PHP NoSQL Databases
Front Range PHP NoSQL DatabasesFront Range PHP NoSQL Databases
Front Range PHP NoSQL DatabasesJon Meredith
 
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...Jose Quesada (hiring)
 
Java Serialization Facts and Fallacies
Java Serialization Facts and FallaciesJava Serialization Facts and Fallacies
Java Serialization Facts and FallaciesRoman Elizarov
 
Tensors Are All You Need: Faster Inference with Hummingbird
Tensors Are All You Need: Faster Inference with HummingbirdTensors Are All You Need: Faster Inference with Hummingbird
Tensors Are All You Need: Faster Inference with HummingbirdDatabricks
 
Low latency in java 8 v5
Low latency in java 8 v5Low latency in java 8 v5
Low latency in java 8 v5Peter Lawrey
 
Linq 1224887336792847 9
Linq 1224887336792847 9Linq 1224887336792847 9
Linq 1224887336792847 9google
 
Architecture by Accident
Architecture by AccidentArchitecture by Accident
Architecture by AccidentGleicon Moraes
 
The Why and How of Scala at Twitter
The Why and How of Scala at TwitterThe Why and How of Scala at Twitter
The Why and How of Scala at TwitterAlex Payne
 
JSR 335 / java 8 - update reference
JSR 335 / java 8 - update referenceJSR 335 / java 8 - update reference
JSR 335 / java 8 - update referencesandeepji_choudhary
 
ShaREing Is Caring
ShaREing Is CaringShaREing Is Caring
ShaREing Is Caringsporst
 
Object- Relational Persistence in Smalltalk
Object- Relational Persistence in SmalltalkObject- Relational Persistence in Smalltalk
Object- Relational Persistence in SmalltalkESUG
 
Rails Tips and Best Practices
Rails Tips and Best PracticesRails Tips and Best Practices
Rails Tips and Best PracticesDavid Keener
 
Matt Franklin - Apache Software (Geekfest)
Matt Franklin - Apache Software (Geekfest)Matt Franklin - Apache Software (Geekfest)
Matt Franklin - Apache Software (Geekfest)W2O Group
 
Peyton jones-2011-parallel haskell-the_future
Peyton jones-2011-parallel haskell-the_futurePeyton jones-2011-parallel haskell-the_future
Peyton jones-2011-parallel haskell-the_futureTakayuki Muranushi
 
Simon Peyton Jones: Managing parallelism
Simon Peyton Jones: Managing parallelismSimon Peyton Jones: Managing parallelism
Simon Peyton Jones: Managing parallelismSkills Matter
 

Similar to Scalax (20)

NoSQL Introduction, Theory, Implementations
NoSQL Introduction, Theory, ImplementationsNoSQL Introduction, Theory, Implementations
NoSQL Introduction, Theory, Implementations
 
http://www.hfadeel.com/Blog/?p=151
http://www.hfadeel.com/Blog/?p=151http://www.hfadeel.com/Blog/?p=151
http://www.hfadeel.com/Blog/?p=151
 
MySQL And Search At Craigslist
MySQL And Search At CraigslistMySQL And Search At Craigslist
MySQL And Search At Craigslist
 
Linq To The Enterprise
Linq To The EnterpriseLinq To The Enterprise
Linq To The Enterprise
 
Introducing Scala to your Ruby/Java Shop : My experiences at IGN
Introducing Scala to your Ruby/Java Shop : My experiences at IGNIntroducing Scala to your Ruby/Java Shop : My experiences at IGN
Introducing Scala to your Ruby/Java Shop : My experiences at IGN
 
Front Range PHP NoSQL Databases
Front Range PHP NoSQL DatabasesFront Range PHP NoSQL Databases
Front Range PHP NoSQL Databases
 
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
A full Machine learning pipeline in Scikit-learn vs in scala-Spark: pros and ...
 
Java Serialization Facts and Fallacies
Java Serialization Facts and FallaciesJava Serialization Facts and Fallacies
Java Serialization Facts and Fallacies
 
Tensors Are All You Need: Faster Inference with Hummingbird
Tensors Are All You Need: Faster Inference with HummingbirdTensors Are All You Need: Faster Inference with Hummingbird
Tensors Are All You Need: Faster Inference with Hummingbird
 
Low latency in java 8 v5
Low latency in java 8 v5Low latency in java 8 v5
Low latency in java 8 v5
 
Linq 1224887336792847 9
Linq 1224887336792847 9Linq 1224887336792847 9
Linq 1224887336792847 9
 
Architecture by Accident
Architecture by AccidentArchitecture by Accident
Architecture by Accident
 
The Why and How of Scala at Twitter
The Why and How of Scala at TwitterThe Why and How of Scala at Twitter
The Why and How of Scala at Twitter
 
JSR 335 / java 8 - update reference
JSR 335 / java 8 - update referenceJSR 335 / java 8 - update reference
JSR 335 / java 8 - update reference
 
ShaREing Is Caring
ShaREing Is CaringShaREing Is Caring
ShaREing Is Caring
 
Object- Relational Persistence in Smalltalk
Object- Relational Persistence in SmalltalkObject- Relational Persistence in Smalltalk
Object- Relational Persistence in Smalltalk
 
Rails Tips and Best Practices
Rails Tips and Best PracticesRails Tips and Best Practices
Rails Tips and Best Practices
 
Matt Franklin - Apache Software (Geekfest)
Matt Franklin - Apache Software (Geekfest)Matt Franklin - Apache Software (Geekfest)
Matt Franklin - Apache Software (Geekfest)
 
Peyton jones-2011-parallel haskell-the_future
Peyton jones-2011-parallel haskell-the_futurePeyton jones-2011-parallel haskell-the_future
Peyton jones-2011-parallel haskell-the_future
 
Simon Peyton Jones: Managing parallelism
Simon Peyton Jones: Managing parallelismSimon Peyton Jones: Managing parallelism
Simon Peyton Jones: Managing parallelism
 

Recently uploaded

Optimizing Operations by Aligning Resources with Strategic Objectives Using O...
Optimizing Operations by Aligning Resources with Strategic Objectives Using O...Optimizing Operations by Aligning Resources with Strategic Objectives Using O...
Optimizing Operations by Aligning Resources with Strategic Objectives Using O...OnePlan Solutions
 
Jax, FL Admin Community Group 05.14.2024 Combined Deck
Jax, FL Admin Community Group 05.14.2024 Combined DeckJax, FL Admin Community Group 05.14.2024 Combined Deck
Jax, FL Admin Community Group 05.14.2024 Combined DeckMarc Lester
 
OpenChain Webinar: AboutCode and Beyond - End-to-End SCA
OpenChain Webinar: AboutCode and Beyond - End-to-End SCAOpenChain Webinar: AboutCode and Beyond - End-to-End SCA
OpenChain Webinar: AboutCode and Beyond - End-to-End SCAShane Coughlan
 
INGKA DIGITAL: Linked Metadata by Design
INGKA DIGITAL: Linked Metadata by DesignINGKA DIGITAL: Linked Metadata by Design
INGKA DIGITAL: Linked Metadata by DesignNeo4j
 
A Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data MigrationA Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data MigrationHelp Desk Migration
 
Lessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdfLessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdfSrushith Repakula
 
Sourcing Success - How to Find a Clothing Manufacturer
Sourcing Success - How to Find a Clothing ManufacturerSourcing Success - How to Find a Clothing Manufacturer
Sourcing Success - How to Find a Clothing ManufacturerWave PLM
 
The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)Roberto Bettazzoni
 
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdf
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdfImplementing KPIs and Right Metrics for Agile Delivery Teams.pdf
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdfVictor Lopez
 
Community is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea GouletCommunity is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea GouletAndrea Goulet
 
Crafting the Perfect Measurement Sheet with PLM Integration
Crafting the Perfect Measurement Sheet with PLM IntegrationCrafting the Perfect Measurement Sheet with PLM Integration
Crafting the Perfect Measurement Sheet with PLM IntegrationWave PLM
 
Reinforcement Learning – a Rewards Based Approach to Machine Learning - Marko...
Reinforcement Learning – a Rewards Based Approach to Machine Learning - Marko...Reinforcement Learning – a Rewards Based Approach to Machine Learning - Marko...
Reinforcement Learning – a Rewards Based Approach to Machine Learning - Marko...Marko Lohert
 
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdfThe Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdfkalichargn70th171
 
Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024Andreas Granig
 
how-to-download-files-safely-from-the-internet.pdf
how-to-download-files-safely-from-the-internet.pdfhow-to-download-files-safely-from-the-internet.pdf
how-to-download-files-safely-from-the-internet.pdfMehmet Akar
 
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...naitiksharma1124
 
The Strategic Impact of Buying vs Building in Test Automation
The Strategic Impact of Buying vs Building in Test AutomationThe Strategic Impact of Buying vs Building in Test Automation
The Strategic Impact of Buying vs Building in Test AutomationElement34
 
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024MulesoftMunichMeetup
 
KLARNA - Language Models and Knowledge Graphs: A Systems Approach
KLARNA -  Language Models and Knowledge Graphs: A Systems ApproachKLARNA -  Language Models and Knowledge Graphs: A Systems Approach
KLARNA - Language Models and Knowledge Graphs: A Systems ApproachNeo4j
 

Recently uploaded (20)

Optimizing Operations by Aligning Resources with Strategic Objectives Using O...
Optimizing Operations by Aligning Resources with Strategic Objectives Using O...Optimizing Operations by Aligning Resources with Strategic Objectives Using O...
Optimizing Operations by Aligning Resources with Strategic Objectives Using O...
 
Jax, FL Admin Community Group 05.14.2024 Combined Deck
Jax, FL Admin Community Group 05.14.2024 Combined DeckJax, FL Admin Community Group 05.14.2024 Combined Deck
Jax, FL Admin Community Group 05.14.2024 Combined Deck
 
OpenChain Webinar: AboutCode and Beyond - End-to-End SCA
OpenChain Webinar: AboutCode and Beyond - End-to-End SCAOpenChain Webinar: AboutCode and Beyond - End-to-End SCA
OpenChain Webinar: AboutCode and Beyond - End-to-End SCA
 
INGKA DIGITAL: Linked Metadata by Design
INGKA DIGITAL: Linked Metadata by DesignINGKA DIGITAL: Linked Metadata by Design
INGKA DIGITAL: Linked Metadata by Design
 
A Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data MigrationA Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data Migration
 
Lessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdfLessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdf
 
Sourcing Success - How to Find a Clothing Manufacturer
Sourcing Success - How to Find a Clothing ManufacturerSourcing Success - How to Find a Clothing Manufacturer
Sourcing Success - How to Find a Clothing Manufacturer
 
What is an API Development- Definition, Types, Specifications, Documentation.pdf
What is an API Development- Definition, Types, Specifications, Documentation.pdfWhat is an API Development- Definition, Types, Specifications, Documentation.pdf
What is an API Development- Definition, Types, Specifications, Documentation.pdf
 
The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)
 
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdf
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdfImplementing KPIs and Right Metrics for Agile Delivery Teams.pdf
Implementing KPIs and Right Metrics for Agile Delivery Teams.pdf
 
Community is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea GouletCommunity is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea Goulet
 
Crafting the Perfect Measurement Sheet with PLM Integration
Crafting the Perfect Measurement Sheet with PLM IntegrationCrafting the Perfect Measurement Sheet with PLM Integration
Crafting the Perfect Measurement Sheet with PLM Integration
 
Reinforcement Learning – a Rewards Based Approach to Machine Learning - Marko...
Reinforcement Learning – a Rewards Based Approach to Machine Learning - Marko...Reinforcement Learning – a Rewards Based Approach to Machine Learning - Marko...
Reinforcement Learning – a Rewards Based Approach to Machine Learning - Marko...
 
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdfThe Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
 
Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024
 
how-to-download-files-safely-from-the-internet.pdf
how-to-download-files-safely-from-the-internet.pdfhow-to-download-files-safely-from-the-internet.pdf
how-to-download-files-safely-from-the-internet.pdf
 
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
 
The Strategic Impact of Buying vs Building in Test Automation
The Strategic Impact of Buying vs Building in Test AutomationThe Strategic Impact of Buying vs Building in Test Automation
The Strategic Impact of Buying vs Building in Test Automation
 
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
 
KLARNA - Language Models and Knowledge Graphs: A Systems Approach
KLARNA -  Language Models and Knowledge Graphs: A Systems ApproachKLARNA -  Language Models and Knowledge Graphs: A Systems Approach
KLARNA - Language Models and Knowledge Graphs: A Systems Approach
 

Scalax

  • 1. The Binary Compatibility Challenge Martin Odersky Typesafe and EPFL
  • 2. The Problem in a Nutshell • Binary compatibility has been an issue ever since Scala became popular. • Causes grief when building, friction for upgrading. • The community has learned to deal with this by becoming more conservative. • But this makes it harder to innovate and improve. Break your client’s builds vs Freeze, and stop improving Is there no third way? 2
  • 3. What is Binary Compatibility? Binary compatibility ≠ Source compatibility Source & binary incompatible object Client { msg.length } object Server { val msg = “abc” } object Server { val msg = Some(“abc”) }
  • 4. What is Binary Compatibility? Binary compatibility ≠ Source compatibility Source incompatible, binary compatible: object Client { import a, b val x: String = 1 } object a { implicit def f(x: Int): String = x.toString } object b object a { implicit def f(x: Int): String = x.toString } object b { implicit def g(x: Int): String = ”abc” }
  • 5. What is Binary Compatibility? Binary compatibility ≠ Source compatibility Source compatible, binary incompatible: object Apple extends Edible { def joules = 500000 } trait Edible { def joules: Double } trait Edible { def joules: Double def calories = joules * 4.184 } èNeed to recompile on Java 1-7 In Java 8 it’s more complex but fundamentally the same.
  • 6. What is Binary Compatibility? Binary compatibility ≠ Source compatibility Source compatible, binary incompatible: object Apple extends Edible { def joules = 500000 } trait Edible { def joules: Double def calories: Double } trait Edible { object def Edible$joules: class Double { } def calories($this: Edible): Double = $this.joules * 4.184 trait Edible { def joules: Double def calories = joules * 4.184 } } object Apple extends Edible { def joules = 500000.0 def calories: Double = Edibl$class.calories(this) } èNeed to recompile on Java 1-7 In Java 8 it’s more complex but fundamentally the same.
  • 7. Other Issues Compiler optimizations and bug fixes can affect binary compatibility. Example: Implementation of lazy values. trait Edible { def joules: Double lazy val def calories = joules * 4.184 } object Apple extends Edible { def joules = 500000 } object Apple extends Edible { def joules = 500000.0 private var initFlags: BitSet private var cals: Int = _ def calories = { if (!initFlags(N)) { cals = Edible$class.initCals(this) initFlags(N) = true } cals }} Previously: 1 bit per lazy val To avoid deadlocks: 2 bits. è all offsets change!
  • 8. Compiler Pipeline Parser Typer SyntheticMethods SuperAccessors RefChecks ElimRepeated ElimLocals ExtensionMethods TailRec PatternMatcher ExplicitOuter Erasure Mixin Memoize LazyVals CapturedVars Constructors LambdaLift Flatten RestoreScopes Cleanup more GenBCode phases Source Symbols JVM Byte-code Lots of scope for things to go wrong!
  • 9. Where It Breaks A.class C.class C.class C.scala C.scala (binary incompatible source change)
  • 10. Why Is This Such a Big Problem? MyApplication DustyLegacyLib Scala Library 2.10 X can’t upgrade to (too old, can’t rebuild) Scala Library 2.11 Seq.scala Seq.scala (binary incompatible source change) Scala 2.11!
  • 11. Not Just A Problem with Scala-Library MyApplication DustyLegacyLib Akka 3.2 X can’t upgrade to (can’t rebuild) Akka 3.3 Actor.scala Actor.scala (binary incompatible source change) Akka 3.3!
  • 12. Not Just A Problem with Scala-Library MyApplication DustyLegacyLib shapeless 2.0 X can’t upgrade to (can’t rebuild) shapeless 2.1 unions.scala unions.scala (binary incompatible source change) shapeless 2.1!
  • 13. Dealing With It So Far “MiMa” tool can detect binary incompatibilities. Scala release policy: – Minor versions need to be (forwards and backwards) binary compatible. – Major versions are allowed to break binary compatibility – Major versions are released rarely (+18 months between them). Problem: – 3rd party libraries need similar policies but often don’t enforce them. – Innovation is stifled. – Simple fixes have to wait for a long time to get in. – Lots of dev cycles spent on dealing with binary compatibility.
  • 14. What Do Others Do? Java: • Language close to JVM bytecode. • Innovation happens on JVM level. – Either in the JVM itself or through reflection. – E.g. Java 8 lambdas, default methods. • Libraries are frozen when they appear. – E.g. java.util.Date • Language is restricted in terms of extensibility – E.g. nterface1, interface2, ... interface7 in Eclipse.
  • 15. What Do Others Do? OSGI: Allow multiple versions of a library in an application MyApplication DustyLegacyLib Scala Library 2.10 MyApplication Scala Library 2.11 rebuild • Fragile, requires serious classloader magic. • Few frameworks beyond Eclipse have bought in.
  • 16. What Do Others Do? C/C++: • Relies on Linker for more flexibility in interfaces. • Not that great a story either (c.f. DLL Hell).
  • 17. What Do Others Do? Clojure: • Builds from source.
  • 18. What Do Others Do? Javascript: • Builds from source.
  • 19. What Do Others Do? Python: • Builds from source.
  • 20. What Do Others Do? Ruby: • Builds from source.
  • 21. What Do Others Do? Go: • Builds from source.
  • 22. Why Can’t Scala Build from Source? No standard Build Tool Should we standardize on SBT, Gradle, Maven, Ivy, Ant? Reproducible builds are rare. Chicken and egg problem: Because everyone is used to binary builds, nobody* invests in making builds reproducible *Not quite true: Typesafe has invested in community build, can now build more than 1M lines of community projects. But it’s a huge effort.
  • 23. What We Need • An interchange format that captures the essence of Scala dependencies. • This cannot be the JVM bytecode format • Nor can it be source 23
  • 24. The Idea Use Typed Trees as an interchange format. – More robust than source. – More stable than JVM bytecode. – Efficient?
  • 25. New Compiler Pipeline Parser Typer SyntheticMethods SuperAccessors RefChecks ElimRepeated ElimLocals ExtensionMethods TailRec PatternMatcher ExplicitOuter Erasure Mixin Memoize LazyVals CapturedVars Constructors LambdaLift Flatten RestoreScopes Cleanup GenBCode Typed Trees Typed Trees Source Frontend Backend Bytecode
  • 26. How To Build A.class | ATree | HashA1 B.class | BTree | HashB1 C.class | CTree | HashC1 A.class | ATree | HashA2 B.class | BTree | HashB2 C.class | CTree | HashC2 (rebuild from BTree) C.scala C.scala (source change) (rebuild)
  • 27. More Robust Than Source Source Parser Typer SyntheticMethods SuperAccessors Typed Trees Frontend Ø Resolve Names Ø scan packages Ø handle imports Ø establish implicit scopes Ø Resolve Overloading Ø Find Implicits Ø Apply Conversions Ø Infer Type Parameters Ø Assign Types to Trees (5374 lines) A lot can go wrong here!
  • 28. More Robust Than Source RefChecks ElimRepeated ElimLocals ExtensionMethods TailRec PatternMatcher ExplicitOuter Erasure Mixin Memoize LazyVals CapturedVars Constructors LambdaLift Flatten RestoreScopes Cleanup GenBCode Typed Trees Backend Bytecode Assign Types (311 lines)
  • 29. More Resilient Than Bytecode Can – add fields and methods to traits – add lazy vals anywhere – change compilation scheme in any way necessary. None of these would be binary compatible! Can also – add or remove implicits – add methods anywhere – change imports All of these could be source incompatible!
  • 30. Efficient? Can typed trees be efficient enough to build million+ line systems? Possible issues: • Size of trees – on disk – in memory • Transformation time 30
  • 31. Potential Issue: Tree Size xs.filter(_ = 0) becomes: Apply Select Ident “filter” “xs” :: Block DefDef “anonfun” :: :: ValDef “$x” Nil Nil Nil TypeTree: Int Apply Select Ident “$x” “=” Literal 0 Literal “anonfun” :: 16 Nodes, Nil not counting types 17 chars
  • 32. Back of the Envelope Calculation: 16 nodes Average size of node: 32 bytes 512 bytes total. Double that to include type info. è 16 bytes source à 1KB tree (factor 64 blow-up). For a 1M line system 30MB source à 2GB trees. 32
  • 33. A More Compact Representation Apply (34) SelectTermWithSig (9) Ident (3) “xs” “filter” “Function1 -­‐ Boolean” Closure (23) ParamDef (7) “x” TypeRef “scala.Int” Apply (14) SelectTermWithSig (9) Ident (3) “x$” “=“ “Integer -­‐ Boolean” Literal (3) 0 33 Still navigable, because inner nodes contain size of total tree derived from them Types or symbols given at the leaves. Types of inner nodes are reconstituted using the TypeAssigner.
  • 34. Speed Transformation + byte-code generation amounts to ~ 60% of total compile time. We can speed this up by – fusing phases, reducing amount of intermediate trees, – using a fast type assigner, instead of a slow typer, – building different files in parallel. Besides, can use incremental compilation. – Compile only this units that depend on changed libraries. – Need to do that only once. 34
  • 35. Other Benefits Optimization – Typed trees are a great format for interprocedural analyses – Inlining across compilation units made simple – Inlining without binary compatibility issues Program Analysis – Types trees are close to source, but easy to traverse – Ideal for context-dependent program analyses such as FindBugs – Ideal for instrumentation Portability – Typed trees allow retargeting to different backends, as long as dependencies exist. – Allow libraries to be used on JVM, JS, LLVM... without needing explicit recompilation. 35
  • 36. Common Intermediate Format 36 New Backend dotc Frontend scalac Frontend Old Backend GenBCode Bytecode
  • 37. Conclusion 37 Typed trees can fix the binary compatibility problem and they offer lots of other benefits, too. Let’s start the work to make them real!