Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Analysis of Zinc (nescala 2020)

187 views

Published on

A quick overview of how the Scala's incremental compilation works.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Analysis of Zinc (nescala 2020)

  1. 1. Analysis of Zinc Eugene Yokota (@eed3si9n)
 Lightbend
  2. 2. everything changes
  3. 3. separate compilation
  4. 4. separate compilation A.scala A.class B.scala B.class
  5. 5. separate compilation A.scala A.jar B.scala B.class
  6. 6. A.scala A.class B.scala B.class A.sig "Types are a property of expressions and thus exist [only] at compile time" — Creative Scala, Gurnell and Welsh
  7. 7. A.scala A.class B.scala B.class A.sig "I turned myself into a pickle, Morty!" — Rick and Morty
  8. 8. incremental (re)compilation = minimality • compiles sources at most once per build
 only if they transitively depend on sources that changed since the previous build
  9. 9. naïve incremental (re)compilation A.scala B.scala C1.scala C3.scala C2.scala C4.scala • step 1
  10. 10. naïve incremental (re)compilation • step 2 A.scala B.scala C1.scala C3.scala C2.scala C4.scala initial invalidation
  11. 11. naïve incremental (re)compilation • step 3 A.scala B.scala C1.scala C3.scala C2.scala C4.scala transitive invalidation
  12. 12. naïve incremental (re)compilation • step 3 A.scala B.scala C1.scala C3.scala C2.scala C4.scala problem: results to compiling all/most sources in the graph
  13. 13. name hashing
  14. 14. name hashing • created by Grzegorz Kossakowski in 2013 (sbt/sbt#1026)
  15. 15. name hashing • we need to prevent invalidation from spreading like a wildfire
  16. 16. name hashing • step 1 (initial compilation) A.scala B.scala C1.scala C3.scala C2.scala C4.scala • inheritance (strong) • member ref (weak)
  17. 17. name hashing • step 2 A.scala B.scala C1.scala C3.scala C2.scala C4.scala • collect method signatures and other information
  18. 18. name hashing • step 3 (second compilation) • compile initial invalidation • update signature info for A A.scala B.scala C1.scala C3.scala C2.scala C4.scala
  19. 19. name hashing • step 4 API changes: compare the signature information before and after compilation A.scala B.scala C1.scala C3.scala C2.scala C4.scala
  20. 20. name hashing • step 5 member ref: transitively invalidated only if a "used name" changed its signature A.scala B.scala C1.scala C3.scala C2.scala C4.scala
  21. 21. name hashing • step 4 inheritance: always transitively invalidated A.scala B.scala C1.scala C3.scala C2.scala C4.scala
  22. 22. name hashing package example class B { def foo: Unit = { val a = new A a.something } } package example class A { def something: Int = 0 }
  23. 23. name hashing used names: example.B -> "A example.B -> "AnyRef example.B -> "B example.B -> "Int example.B -> "Object example.B -> "Unit example.B -> "a example.B -> "example example.B -> "example;A;init; example.B -> "java;lang;Object;init; example.B -> "scala example.B -> "something
  24. 24. name hashing package example class B { def foo: Unit = { val a = new A a.something } } package example class A { // was 0 def something: Int = 2 } • body change doesn't matter
  25. 25. name hashing package example class B { def foo: Unit = { val a = new A a.something } } package example class A { def something: Int = 0 // new def somethingElse: Int = 1 } • unused names do not matter
  26. 26. name hashing package example class C extends A { def something: Int = 1 } package example class A { def something: Int = 0 // new def somethingElse: Int = 1 } • inheritance propagates invalidation
  27. 27. name hashing • implementation body changes • changes to unused methods / vals prevents invalidation spread based on
  28. 28. dependency origins subprojects local library JARs • we only need signature (API) changes from upstream
  29. 29. next steps
  30. 30. liberation from a machine
  31. 31. liberation from a machine 2 items /Users/eed3si9n/work/zinc-example/A.scala -> /Users/eed3si9n/work/zinc- example/target/scala-2.13/classes/example/A.class /Users/eed3si9n/work/zinc-example/B.scala -> /Users/eed3si9n/work/zinc- example/target/scala-2.13/classes/example/B.class library dependencies: 4 items /Users/eed3si9n/work/zinc-example/A.scala -> /Users/eed3si9n/.coursier/ cache/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.1/ scala-library-2.13.1.jar /Users/eed3si9n/work/zinc-example/A.scala -> /Users/eed3si9n/.sdkman/ candidates/java/8.0.222.hs-adpt/jre/lib/rt.jar /Users/eed3si9n/work/zinc-example/B.scala -> /Users/eed3si9n/.coursier/ cache/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.1/ scala-library-2.13.1.jar /Users/eed3si9n/work/zinc-example/B.scala -> /Users/eed3si9n/.sdkman/ candidates/java/8.0.222.hs-adpt/jre/lib/rt.jar library class names: 2 items /Users/eed3si9n/.coursier/cache/v1/https/repo1.maven.org/maven2/org/scala- lang/scala-library/2.13.1/scala-library-2.13.1.jar -> scala.Int /Users/eed3si9n/.sdkman/candidates/java/8.0.222.hs-adpt/jre/lib/rt.jar -> java.lang.Object
  32. 32. repeatable build = build as function sbt/zinc#712
  33. 33. (modular) incremental build pipelining sbt/zinc#744 subprojects local library JARs parser etc typer, pickler etc jvm refcheck, erasure parser etc typer, pickler etc jvm refcheck, erasure parser etc typer, pickler etc jvm refcheck, erasure early/foo.jar early/bar.jar
  34. 34. Thank You and stay healthy!
  35. 35. 2020.03 mixtape COVID-19 real-time update in US (https:// coronavirus.1point3acres.com/en) Coronavirus update and guidance (https:// www.flattenthecurve.com/)

×