This document discusses incremental compilation techniques in Zinc, the Scala compiler's incremental compiler. It describes how name hashing works to prevent invalidations from spreading across the entire dependency graph when only a small change is made. Name hashing collects signatures during initial compilation and compares them after subsequent compilations to determine if APIs have changed. This allows Zinc to only recompile the minimum necessary sources.

1. Analysis of Zinc
Eugene Yokota (@eed3si9n)
Lightbend

2. everything changes

3. separate compilation

4. separate compilation
A.scala
A.class B.scala
B.class

5. separate compilation
A.scala
A.jar B.scala
B.class

6. A.scala
A.class B.scala
B.class
A.sig
"Types are a property of expressions
and thus exist [only] at compile time"
— Creative Scala, Gurnell and Welsh

7. A.scala
A.class B.scala
B.class
A.sig
"I turned myself into a pickle, Morty!"
— Rick and Morty

8. incremental
(re)compilation =
minimality
• compiles sources at most once per build
only if they transitively depend on
sources that changed since the previous
build

9. naïve incremental
(re)compilation
A.scala B.scala C1.scala
C3.scala
C2.scala
C4.scala
• step 1

10. naïve incremental
(re)compilation
• step 2
A.scala B.scala C1.scala
C3.scala
C2.scala
C4.scala
initial invalidation

11. naïve incremental
(re)compilation
• step 3
A.scala B.scala C1.scala
C3.scala
C2.scala
C4.scala
transitive invalidation

12. naïve incremental
(re)compilation
• step 3
A.scala B.scala C1.scala
C3.scala
C2.scala
C4.scala
problem: results to compiling
all/most sources in the graph

13. name hashing

14. name hashing
• created by Grzegorz Kossakowski in
2013 (sbt/sbt#1026)

15. name hashing
• we need to prevent invalidation from
spreading like a wildfire

16. name hashing
• step 1 (initial compilation)
A.scala B.scala C1.scala
C3.scala
C2.scala
C4.scala
• inheritance (strong)
• member ref (weak)

17. name hashing
• step 2
A.scala B.scala C1.scala
C3.scala
C2.scala
C4.scala
• collect method signatures and
other information

18. name hashing
• step 3 (second compilation)
• compile initial invalidation
• update signature info for A
A.scala B.scala C1.scala
C3.scala
C2.scala
C4.scala

19. name hashing
• step 4
API changes: compare the
signature information before and
after compilation
A.scala B.scala C1.scala
C3.scala
C2.scala
C4.scala

20. name hashing
• step 5
member ref: transitively invalidated
only if a "used name" changed its
signature
A.scala B.scala C1.scala
C3.scala
C2.scala
C4.scala

21. name hashing
• step 4
inheritance: always transitively
invalidated
A.scala B.scala C1.scala
C3.scala
C2.scala
C4.scala

22. name hashing
package example
class B {
def foo: Unit = {
val a = new A
a.something
}
}
package example
class A {
def something: Int = 0
}

23. name hashing
used names:
example.B -> "A
example.B -> "AnyRef
example.B -> "B
example.B -> "Int
example.B -> "Object
example.B -> "Unit
example.B -> "a
example.B -> "example
example.B -> "example;A;init;
example.B -> "java;lang;Object;init;
example.B -> "scala
example.B -> "something

24. name hashing
package example
class B {
def foo: Unit = {
val a = new A
a.something
}
}
package example
class A {
// was 0
def something: Int = 2
}
• body change doesn't matter

25. name hashing
package example
class B {
def foo: Unit = {
val a = new A
a.something
}
}
package example
class A {
def something: Int = 0
// new
def somethingElse: Int = 1
}
• unused names do not matter

26. name hashing
package example
class C extends A {
def something: Int = 1
}
package example
class A {
def something: Int = 0
// new
def somethingElse: Int = 1
}
• inheritance propagates
invalidation

27. name hashing
• implementation body changes
• changes to unused methods / vals
prevents invalidation spread based on

28. dependency origins
subprojects local library JARs
• we only need
signature (API)
changes from
upstream

29. next steps

30. liberation from a machine

31. liberation from a machine
2 items
/Users/eed3si9n/work/zinc-example/A.scala -> /Users/eed3si9n/work/zinc-
example/target/scala-2.13/classes/example/A.class
/Users/eed3si9n/work/zinc-example/B.scala -> /Users/eed3si9n/work/zinc-
example/target/scala-2.13/classes/example/B.class
library dependencies:
4 items
/Users/eed3si9n/work/zinc-example/A.scala -> /Users/eed3si9n/.coursier/
cache/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.1/
scala-library-2.13.1.jar
/Users/eed3si9n/work/zinc-example/A.scala -> /Users/eed3si9n/.sdkman/
candidates/java/8.0.222.hs-adpt/jre/lib/rt.jar
/Users/eed3si9n/work/zinc-example/B.scala -> /Users/eed3si9n/.coursier/
cache/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.1/
scala-library-2.13.1.jar
/Users/eed3si9n/work/zinc-example/B.scala -> /Users/eed3si9n/.sdkman/
candidates/java/8.0.222.hs-adpt/jre/lib/rt.jar
library class names:
2 items
/Users/eed3si9n/.coursier/cache/v1/https/repo1.maven.org/maven2/org/scala-
lang/scala-library/2.13.1/scala-library-2.13.1.jar -> scala.Int
/Users/eed3si9n/.sdkman/candidates/java/8.0.222.hs-adpt/jre/lib/rt.jar ->
java.lang.Object

32. repeatable build = build as function
sbt/zinc#712

33. (modular) incremental build pipelining
sbt/zinc#744
subprojects local library JARs
parser
etc
typer, pickler etc jvm
refcheck,
erasure
parser
etc
typer, pickler etc jvm
refcheck,
erasure
parser
etc
typer, pickler etc jvm
refcheck,
erasure
early/foo.jar
early/bar.jar

34. Thank You and stay healthy!

35. 2020.03 mixtape
COVID-19 real-time update in US (https://
coronavirus.1point3acres.com/en)
Coronavirus update and guidance (https://
www.flattenthecurve.com/)