Successfully reported this slideshow.
Your SlideShare is downloading. ×

Analysis of Zinc (ScalaSphere 2019)

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 50 Ad

More Related Content

Slideshows for you (20)

Similar to Analysis of Zinc (ScalaSphere 2019) (20)

Advertisement

Recently uploaded (20)

Advertisement

Analysis of Zinc (ScalaSphere 2019)

  1. 1. Analysis of Zinc Eugene Yokota (@eed3si9n)
 Lightbend
  2. 2. previously in ScalaSphere 2017...
  3. 3. separate compilation
  4. 4. separate compilation A.scala A.class B.scala B.class
  5. 5. separate compilation A.scala A.jar B.scala B.class
  6. 6. A.scala A.class B.scala B.class A.sig "Types are a property of expressions and thus exist [only] at compile time" — Creative Scala, Gurnell and Welsh
  7. 7. A.scala A.class B.scala B.class A.sig "I turned myself into a pickle, Morty!" — Rick and Morty
  8. 8. A.scala A.class B.scala B.class A.sig def foo: Unit = { val a = new A a.something }
  9. 9. A.scala A.class B.scala B.class A.sig def foo: Unit = { val a = new A a.something } scala> :javap example.B ... public void foo(); descriptor: ()V flags: ACC_PUBLIC Code: stack=2, locals=2, args_size=1 0: new #12 // class example/A 3: dup 4: invokespecial #15 // Method example/A."<init>" 7: astore_1 8: aload_1 9: invokevirtual #19 // Method example/A.somethin 12: pop 13: return
  10. 10. incremental (re)compilation = minimality
  11. 11. incremental (re)compilation = minimality • compiles sources at most once per build
 only if they transitively depend on sources that changed since the previous build
  12. 12. naïve incremental (re)compilation A.scala B.scala C1.scala C3.scala C2.scala C4.scala • step 1
  13. 13. naïve incremental (re)compilation • step 2 A.scala B.scala C1.scala C3.scala C2.scala C4.scala initial invalidation
  14. 14. naïve incremental (re)compilation • step 3 A.scala B.scala C1.scala C3.scala C2.scala C4.scala transitive invalidation
  15. 15. naïve incremental (re)compilation • step 3 A.scala B.scala C1.scala C3.scala C2.scala C4.scala problem: results to compiling all/most sources in the graph
  16. 16. name hashing
  17. 17. name hashing • created by Grzegorz Kossakowski in 2013 (sbt/sbt#1026)
  18. 18. name hashing • we need to prevent invalidation from spreading like a wildfire
  19. 19. name hashing • step 1 A.scala B.scala C1.scala C3.scala C2.scala C4.scala • inheritance (strong) • member ref (weak)
  20. 20. name hashing • step 2 A.scala B.scala C1.scala C3.scala C2.scala C4.scala • collect method signatures and other information
  21. 21. name hashing • step 3 compile initial invalidation A.scala B.scala C1.scala C3.scala C2.scala C4.scala
  22. 22. name hashing • step 4 A.scala B.scala C1.scala C3.scala C2.scala C4.scala inheritance: invalidate dependents
  23. 23. name hashing • step 5 API changes: compare the signature information before and after compilation A.scala B.scala C1.scala C3.scala C2.scala C4.scala
  24. 24. name hashing • step 6 member ref: invalidate only if a "used name" is affected A.scala B.scala C1.scala C3.scala C2.scala C4.scala
  25. 25. name hashing package example class B { def foo: Unit = { val a = new A a.something } } package example class A { def something: Int = 0 }
  26. 26. name hashing used names: example.B -> "A example.B -> "AnyRef example.B -> "B example.B -> "Int example.B -> "Object example.B -> "Unit example.B -> "a example.B -> "example example.B -> "example;A;init; example.B -> "java;lang;Object;init; example.B -> "scala example.B -> "something
  27. 27. name hashing package example class B { def foo: Unit = { val a = new A a.something } } package example class A { // was 0 def something: Int = 1 } • body change doesn't matter
  28. 28. name hashing package example class B { def foo: Unit = { val a = new A a.something } } package example class A { def something: Int = 0 // new def somethingElse: Int = 1 } • unused names do not matter
  29. 29. name hashing • implementation body changes • changes to unused methods / vals prevents invalidation spread based on
  30. 30. compiler bridge
  31. 31. compiler bridge build tool Zinc compiler interface (xsbti) compiler bridge (xsbt) Scala Compiler (scala.tools.nsc) compiler bridge source JAR
  32. 32. compiler bridge parser etc typer, patmat etc Analyzer API Extractor Dependency refchecks etc jvm pickler
  33. 33. compiler bridge typer, pickler, etc patmat, erasure, etc jvm4.72% 0.00% 25.00% 50.00% 75.00% 100.00% compiler phases
  34. 34. dependency origins
  35. 35. dependency origins subprojects local library JARs
  36. 36. dependency origins subprojects local library JARs • we have the associated Analysis to perform 
 name hashing
  37. 37. dependency origins subprojects local library JARs • we only need signature (API) changes from upstream
  38. 38. next steps
  39. 39. liberation from a machine
  40. 40. liberation from a machine 2 items /Users/eed3si9n/work/zinc-example/A.scala -> /Users/eed3si9n/work/zinc- example/target/scala-2.13/classes/example/A.class /Users/eed3si9n/work/zinc-example/B.scala -> /Users/eed3si9n/work/zinc- example/target/scala-2.13/classes/example/B.class library dependencies: 4 items /Users/eed3si9n/work/zinc-example/A.scala -> /Users/eed3si9n/.coursier/ cache/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.1/ scala-library-2.13.1.jar /Users/eed3si9n/work/zinc-example/A.scala -> /Users/eed3si9n/.sdkman/ candidates/java/8.0.222.hs-adpt/jre/lib/rt.jar /Users/eed3si9n/work/zinc-example/B.scala -> /Users/eed3si9n/.coursier/ cache/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.1/ scala-library-2.13.1.jar /Users/eed3si9n/work/zinc-example/B.scala -> /Users/eed3si9n/.sdkman/ candidates/java/8.0.222.hs-adpt/jre/lib/rt.jar library class names: 2 items /Users/eed3si9n/.coursier/cache/v1/https/repo1.maven.org/maven2/org/scala- lang/scala-library/2.13.1/scala-library-2.13.1.jar -> scala.Int /Users/eed3si9n/.sdkman/candidates/java/8.0.222.hs-adpt/jre/lib/rt.jar -> java.lang.Object
  41. 41. liberation from time 2 items /Users/eed3si9n/work/zinc-example/target/scala-2.13/classes/example/A.class -> lastModified(1570453146449) /Users/eed3si9n/work/zinc-example/target/scala-2.13/classes/example/B.class -> lastModified(1570453146456)
  42. 42. repeatable build = build as function
  43. 43. compiler integration
  44. 44. compiler integration • incremental compilation as a first-class feature
  45. 45. compiler integration • incremental compilation as a first-class feature • test for specific bridge • no more source downloading • evolve bridge together with the language
  46. 46. compiler integration parser etc typer, patmat etc Analyzer API Extractor Dependency refchecks etc jvm pickler • API Extractor could probably go away if the bridge was integrated with the compiler, saving both time and memory
  47. 47. compiler integration subprojects local library JARs • incremental compilation with any JAR?
  48. 48. compiler integration subprojects local library JARs outline types • we only need signature (API) changes from upstream
  49. 49. compiler integration subprojects local library JARs outline typer typer, pickler etc jvm refcheck, erasure outline typer typer, pickler etc jvm refcheck, erasure outline typer typer, pickler etc jvm refcheck, erasure outline/foo.jar outline/bar.jar
  50. 50. Thank You

×