The document discusses using Bazel for building Scala projects. It begins with an overview of Bazel and how it uses small targets and rules to build code more incrementally and in parallel compared to tools like Maven and Gradle. It then covers the rules_scala plugin, which provides rules for compiling Scala code into JARs and running tests. Features of rules_scala like dependency management and support for multiple Scala versions are also summarized. Overall the document promotes Bazel and rules_scala as enabling significantly faster builds of large Scala codebases.

1. Natan Silnitsky
Software dev in CI team, Wix.com
Building Scala with Bazel
natans@wix.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil

2. Maven
45
Bazel
BAZEL IS FAST
avg build time
(minutes) 1

3. WIX BACKEND
~250 developers
~750 micro-services
>10M LoC, mostly Scala
some Java, Node.js
K’s of builds every day

4. Maven +
Teamcity
Bazel +
Google Cloud
WE MIGRATED
Infra change

5. Maven +
Teamcity
Bazel +
Google Cloud
WE MIGRATED
hours
few
minutesInfra change

6. AGENDA
Bazel
fast, correct, and
scalable
Bazel for Scala
faster Scala builds

7. In Bazel
You control the level of granularity
to build.
scala_project
├── WORKSPACE
└── src
└── main
└── scala
└── com
├── example
│ ├── A.scala
│ ├── B.scala
│ └── C.scala
└── example2
├── D.scala
├── E.scala
└── F.scala

8. The basic build unit is
called a target
scala_project
├── WORKSPACE
└── src
└── main
└── scala
└── com
├── example
│ ├── A.scala
│ ├── B.scala
│ ├── BUILD
│ └── C.scala
└── example2
├── D.scala
├── E.scala
└── F.scala
Target
Targets are instances
of rules.

9. A RULE
f(A,B,C)f( )
Source files
env’ variables
dependencies
A
B
C
scala library | C++ binary | python test | shell script | many many more...
wix

10. A RULE
f(A,B,C)f( )
A
B
C
INPUTs
source files | dependencies | tools configurations

11. A RULE
A
B
C
f( ) f(A,B,C)
Sandbox
source files | dependencies | tools configurations

12. A RULE
f(A,B,C)
OUTPUTs
f( )
A
B
C
f(A,B,C)
f( )
A
B
C
SAME

13. SBT /
Gradle /
Maven
BAZEL
MANY SMALL TARGETS

14. MUCH MORE PARALLEL
SBT /
Gradle /
Maven
BAZEL

15. MUCH MORE PARALLEL
SBT /
Gradle /
Maven

16. ✔ ✔ ✔
✔ ✔ ✔
✔ ✔ ✔ ✔
BAZEL
INCREMENTAL

17. ✔ ✔ ✔
✔ ✔ ✔
✔ ✔ ✔ ✔
GOOGLE CLOUD / OSS
BAZEL
Cache
REMOTE EXECUTION
100s
Workers
CI/local tradeoff

18. maven local clean 45 min
IT’S WORTH IT

19. 45 minmaven local clean
20 minbazel local clean
8.5 minbazel remote clean
IT’S WORTH IT
bazel fully cached 0.1 min

20. IT’S WORTH IT
IF
startup
Code becomes
complex
velocity
is hurt
mess
in CI

21. RULES_SCALA
Bazel plugin for Scala
Written in Scala and Starlark
OSS
co-maintainers

22. SBT /
Gradle /
Maven
scala_project
├── build.sbt / build.gradle / pom.xml
└── src
├── main
├── test

23. scala_project
├── WORKSPACE
└── src
├── main
│ └── scala
│ └── com
│ └── example
│ ├── A.scala
│ ├── B.scala
│ ├── BUILD FILE
│ ├── C.scala
│ └── Example.scala
└── test
SBT /
Gradle /
Maven
BAZEL
scala_project
├── build.sbt / build.gradle / pom.xml
└── src
├── main
├── test

24. #1 SCALA_LIBRARY
.JAR
Compiler
Calls the Scala

25. scala_project
├── WORKSPACE
└── src
├── main
│ └── scala
│ └── com
│ └── example
│ ├── A.scala
│ ├── B.scala
│ ├── BUILD FILE
│ ├── C.scala
│ └── Example.scala
└── test
A
C
A.scala -> scala_library -> a.jar
scala_library(
name="a",
srcs=[
"A.scala",
],
deps = [":c"])
$ bazel build :a
#1
SCALA_LIBRARY

26. #2 SCALA_BINARY
.JAR + sh script:
run MAIN CLASS on JVM
Generate
Shell Script

27. scala_project
├── WORKSPACE
└── src
├── main
│ └── scala
│ └── com
│ └── example
│ ├── A.scala
│ ├── B.scala
│ ├── BUILD FILE
│ ├── C.scala
│ └── Example.scala
└── test
B
B.scala -> scala_binary ->
B.jar + app_runner.sh
scala_binary(
name="b",
srcs=[
"B.scala",
],
main_class="com.example.Foo")
$ bazel run :b
#1
SCALA_LIBRARY

28. Tests.JAR + sh script:
with SCALA TEST RUNNER
TEST
files
Generate
Shell Script
#3 SCALA_TEST

29. ExampleTest.scala -> scala_test ->
example_test.jar + test_runner.sh
scala_test(
name="example_test",
srcs=glob(["*Test.scala"]))
$ bazel test :example_test
scala_project
├── WORKSPACE
└── src
├── main
│ └── scala
│ └── com
│ └── example
│ ├── A.scala
│ ├── B.scala
│ ├── BUILD
│ ├── C.scala
│ └── Example.scala
└── test
└── scala
└── com
└── example
├── BUILD
├── ExampleTest.scala
└── Example2Test.scala
example_test
#3
SCALA_TEST

30. Working with rules_scala
Demo

31. f(A,B,C)f( )
A
B
C
Junit test | specs2 test | thrift lib | proto lib | more... | add your own!
AND MANY MORE
So… build a library | run scala test

32. RULES_SCALA
Included features

33. FEATURES
E2E
hash1
hash2
Extensive E2E suite
Verifies Reproducibility
Own targets Build

34. FEATURES
JavaInfo provider
common jvm rules outputs
LANG INTEROP
Java
Scala
Groovy
Kotlin

35. FEATURES
SCALA TOOLCHAIN
Globally set scala configuration
scala_toolchain(
name = "my_toolchain_impl",
scalacopts = ["-Ywarn-unused"],
unused_dependency_checker_mode = "off",
visibility = ["//visibility:public"]
)

36. --unused_dependency_checker_mode=WARN|ERROR
...
error: Target '//some_package:unused_dep' is specified as a dependency to //path/to:foo but isn't used, please
remove it from the deps.
You can use the following buildozer command:
buildozer 'remove deps //some_package:unused_dep' //path/to:foo
FEATURES
RUN THE SCALA COMPILER WITH FLAGS
DEPENDENCY MGMT
foo
B unused

37. --strict_java_deps=WARN|ERROR
...
Target '@cats//jar' is used but isn't explicitly declared, please add it to the deps.
You can use the following buildozer command:
buildozer 'add deps @cats//jar' //path/to:foo
FEATURES
RUN THE SCALA COMPILER WITH FLAGS
DEPENDENCY MGMT
foo
specs2
cats

38. --strict_java_deps=WARN|ERROR
...
Target '@cats//jar' is used but isn't explicitly declared, please add it to the deps.
You can use the following buildozer command:
buildozer 'add deps @cats//jar' //path/to:foo
Based on Jason Zaugg’s Classpath Shrinker plugin
FEATURES
RUN THE SCALA COMPILER WITH FLAGS
DEPENDENCY MGMT
foo
specs2
cats

39. Automatically fix dependencies in BUILD files
by parsing errors from bazel build result log + index server.
FEATURES
OSS SOON
DepFixer
Internal
JARs
External
JARs
Build result log

40. Automatically fix dependencies in BUILD files
by parsing errors from bazel build result log (+ index server.)
Inspired by Pants
FEATURES
OSS SOON
DepFixer
Internal
JARs
External
JARs
Build result log

41. FEATURES
scala_toolchain(
name = "wix_plus_one_global_toolchain_impl",
scalacopts = [
"-unchecked",
...
],
plus_one_deps_mode = "on",
scalac_provider_attr = "@core_server_build_tools//toolchains:scalac_default",
visibility = ["//visibility:public"],
)
DEPS + 1
DEPENDENCY MGMT

42. build --strategy=Scalac=worker
FEATURES
FOR A ‘WARM’ SCALA COMPILER
PERSISTENT WORKERS
rules_scala annex (alternative plugin) has a persistent worker
too that uses zinc - stateful incremental compiler.
github.com/higherkindness/rules_scala
scalac
Workers

43. Demo
PERSISTENT WORKERS

44. Default is Scala 2.11.12.
For other versions:
scala_repositories(("2.12.6", {
"scala_compiler": "3023b07cc02f2b0217b2c04f8e636b396130b3a8544a8dfad498a19c3e57a863",
"scala_library": "f81d7144f0ce1b8123335b72ba39003c4be2870767aca15dd0888ba3dab65e98",
"scala_reflect": "ffa70d522fc9f9deec14358aa674e6dd75c9dfa39d4668ef15bb52f002ce99fa"
}))
FEATURES
CROSS-VERSION SUPPORT
BY THE END OF 2019

45. https://github.com/bazelbuild/rules_scala
RULES_SCALA

46. Migration Stories
How to successfully migrate to Bazel - My talk @Devoxx UA 2018
Migrating to Bazel? 5 crucial questions... - My Blog Series on Medium
Speedy Scala Builds with Bazel at Databricks

47. Faster Builds
Ecosystem is becoming better
rules_scala is mature and
battle tested
BAZEL
IS WORTH IT

48. Thank You
natans@wix.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil
vote.scalaua.com

49. Q&A
natans@wix.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil

50. Slides & More
slideshare.net/NatanSilnitsky
medium.com/@natansil
twitter.com/NSilnitsky