Sbt baby steps

© 2018 The Quantium Group Pty Ltd. In Commercial Confidence
> agenda
2
[info] build definition
[info] key, task
[info] resolvers
[info] multi-project build
[info] scope
[info] scalacOptions
[info] plugins we use
[maybe[success]] Expected time: 45 min

–Wikipedia
“sbt (Simple Build Tool) is an open-source build tool
for Scala and Java projects…”

500 pages of docs

> features of SBT
[info] little or no configuration required for simple
projects
[info] scala-based build definition
[info] incremental recompilation
[info] packages and publishes jars, generates docs
[info] supports testing with ScalaCheck, specs, ScalaTest
[info] modularization
[info] external project support (list a git repo as a
dependency)
[info] parallel task/test execution
[info] library management support
[info] more
[success] Total time: 0 s, completed 27/01/2018 6:35:31 PM

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
[error] (subs/*:projectDescriptors) Could not create file
/home/svc_jenkins/jenkins_workspace/Substitutability/subs-build-
pr/subs/target/streams/$global/projectDescriptors/$global/streams/outjava.io.IOExcepti
on: No such file or directory
[error] Total time: 0 s, completed Jan 15, 2018 2:39:24 AM
Build step 'Build using sbt' changed build result to FAILURE
Build step 'Build using sbt' marked build as failure
[CHECKSTYLE] Skipping publisher since build result is FAILURE
[WARNINGS] Skipping publisher since build result is FAILURE
Publishing Scoverage XML and HTML report ...
ERROR: Step ?Publish Scoverage Report? aborted due to exception:
java.io.IOException: /home/svc_jenkins/jenkins_workspace/Substitutability/subs-build-
pr/target/scala-2.11/scoverage-report not exists
at
org.jenkinsci.plugins.scoverage.ScoveragePublisher.copyReport(ScoveragePublisher.java:
114)
at
org.jenkinsci.plugins.scoverage.ScoveragePublisher.perform(ScoveragePublisher.java:66)
……
Notified Stash for commit with id f83c0fecc73c477c18843208317f9523a9b17882
Finished: FAILURE

> the basics
[info] build defined in objects extending Build
[info] builds are composed from Projects.
May be local (file) or remote (git url)
[info] projects are a sequence of Settings
[info] settings are key-initialization pairs

sbt.version = 0.13.16
build.properties

import sbt._
import sbt.Keys._
object ExampleBuild extends Build {
val dependencies = Seq(
"org.scalatest" %% "scalatest" % "1.9.1" % "test"
)
lazy val exampleProject =
Project("example", file(".")) settings(
version := "0.1",
organization := "com.companyname",
scalaVersion := "2.12.4",
libraryDependencies ++= dependencies
)
}
Extending the SBT Build object

name := "example"
version := "0.1"
organization := "com.companyname"
scalaVersion := "2.12.4"
libraryDependencies +=
"org.scalatest" %%
"scalatest" % "1.9.1" % "test"
build.sbt file

name := "example"
version := "0.1"
libraryDependencies +=
"org.scalatest" %%
"scalatest" % "1.9.1" % "test"
build.sbt file
{ }
——— —-————
key operator body

build.sbt file
:= replace existing
+= add one element
++= add Seq[T](…)

> keys
[info] SettingKey[T]
For static values
name, version, organization
[info] TaskKey[T]
For (re-)computed values
sources, compile
[info] InputKey[T]
For generating a Task from user input
run-main my.main.class

Custom Key
lazy val hello = taskKey[Unit]("An example task")
lazy val root = (project in file("."))
.settings(
hello := { println("Hello!") }
)

Definition of Task
lazy val hello = taskKey[Unit]("An example task")
.settings(
hello := { println("Hello!") }
)

> reload
[info] Loading project definition from
/Users/marina/IdeaProjects/example/project
[info] Set current project to example (in build
file:/Users/marina/IdeaProjects/example/)
> hello
Hello!
[success] Total time: 0 s, completed 21/01/2018
9:40:41 PM
>

Int Task
lazy val hello = taskKey[Int]("An example task")
.settings(
hello := {
println(“Hello!")
2
}
)

> show hello
Hello!
[info] 2
[success] Total time: 0 s, completed 21/01/2018 9:40:41 PM
> inspect hello
[info] Task: Int
[info] Description:
[info] An example task
[info] Provided by:
[info] {file:/Users/marina/IdeaProjects/example/}root/*:hello
[info] Defined at:
[info] /Users/marina/IdeaProjects/example/build.sbt:19
[info] Delegates:
[info] *:hello
[info] {.}/*:hello
[info] */*:hello
>
> show, inspect

> show package
[info] Packaging
/Users/marina/IdeaProjects/example/target/scala-
2.12/example_2.12-0.1.jar ...
[info] Done packaging.
[info]
/Users/marina/IdeaProjects/example/target/scala-
2.12/example_2.12-0.1.jar
[success] Total time: 0 s, completed 20/01/2018
10:04:01 PM
>
> show package

import sbt._
object Dependencies {
object Versions {
val scala = "2.12.4"
val scalatest = "3.0.1"
val shapeless = "2.3.2"
}
val testLibs = Seq(
"org.scalatest" %% "scalatest" % Versions.scalatest
).map(_ % "test")
def mainLibs = Seq(
"com.chuusai" %% "shapeless" % Versions.shapeless,
"org.scala-lang" % "scala-reflect" % Versions.scala
)
}
Dependencies.scala

import Dependencies._
name := "example"
version := "0.1"
libraryDependencies ++= mainLibs ++ testLibs
build.sbt
libraryDependencies ++= Seq(
groupID % artifactID % revision,
groupID % otherID % otherRevision
)
< artifactID
< revision
< groupID

import Dependencies._
name := "example"
version := "0.1"
organization := “com.companyname"
resolvers += "Java.net Maven2 Repository" at
"http://download.java.net/maven/2/"
build.sbt
https://www.scala-sbt.org/0.13/docs/Resolvers.html

Proxy Repositories
Nexus Artifactory Archiva
OR

-Dsbt.repository.config=<path-to-your-repo-file>
OR
Proxy Repositories
Nexus Artifactory Archiva

–Wikipedia
Apache Ivy is a transitive package manager.
Build tools and continuous integration servers regularly
support or include Ivy:
• sbt (included in dependency management)
• Jenkins
•…

Ivy Maven
sbt

MacBook-Pro:IdeaProjects marina$ cd example
MacBook-Pro:example marina$ cd ~/.ivy2
MacBook-Pro:.ivy2 marina$ ls
cache exclude_classifiers exclude_classifiers.lock
MacBook-Pro:.ivy2 marina$ cd cache/
MacBook-Pro:cache marina$ ls
args4j
com.chuusai
com.eed3si9n
com.github.ben-manes.caffeine
com.github.cb372
com.google
com.google.code.findbugs
com.google.code.gson
com.google.guava
com.google.protobuf
com.googlecode.json-simple
com.jcraft
com.lihaoyi
com.lmax
com.squareup.okhttp3
com.squareup.okio
com.thoughtworks.paranamer
com.trueaccord.lenses
com.trueaccord.scalapb
com.typesafe

val setts = Seq(
version := "0.1",
scalaVersion := "2.12.4",
)
lazy val root =
Project(
id = "example",
base = file("."),
dependencies = Seq(
domain,
magnolia
)
)
.settings(setts)
.settings(libraryDependencies ++= mainLibs ++ testLibs)
lazy val magnolia = ProjectRef(
uri("git://github.com/etaty/scalacheck-magnolia.git"),
"scalacheck-magnolia"
)
lazy val domain = Project(
id = "example-domain",
base = file("example-domain"))
.settings(setts)
.settings(moduleName := "domain")
build.sbt

Build
root
domain
magnolia

Cloning into ‘/Users/marina/.sbt/0.13/staging/8924b22f06679a54873e/scalacheck-magnolia'...
[info] Loading project definition from /Users/marina/IdeaProjects/example/project
/Users/marina/.sbt/0.13/staging/8924b22f06679a54873e/scalacheck-magnolia/project
[info] Set current project to example (in build file:/Users/marina/IdeaProjects/example/)
[info] Updating {git://github.com/etaty/scalacheck-magnolia.git}scalacheck-magnolia...
[info] downloading
https://repo1.maven.org/maven2/com/propensive/magnolia_2.12/0.6.1/magnolia_2.12-0.6.1.jar
...
[info] [SUCCESSFUL ] com.propensive#magnolia_2.12;0.6.1!magnolia_2.12.jar (2011ms)
[info] downloading
https://repo1.maven.org/maven2/org/scalacheck/scalacheck_2.12/1.13.4/scalacheck_2.12-
1.13.4.jar ...
[info] [SUCCESSFUL ] org.scalacheck#scalacheck_2.12;1.13.4!scalacheck_2.12.jar (4767ms)
[info] Done updating.
[info] Updating {file:/Users/marina/IdeaProjects/example/}example...
[info] Compiling 2 Scala sources to
/Users/marina/.sbt/0.13/staging/8924b22f06679a54873e/scalacheck-magnolia/target/scala-
2.12/classes...
[info] 'compiler-interface' not yet compiled for Scala 2.12.4. Compiling...
[info] Compilation completed in 12.795 s
[success] Total time: 28 s, completed 19/01/2018 11:35:13 AM
> compile

Cloning into ‘/Users/marina/.sbt/0.13/staging/8924b22f06679a54873e/scalacheck-magnolia'...
[info] Loading project definition from /Users/marina/IdeaProjects/example/project
/Users/marina/.sbt/0.13/staging/8924b22f06679a54873e/scalacheck-magnolia/project
[info] Set current project to example (in build file:/Users/marina/IdeaProjects/example/)
[info] Updating {git://github.com/etaty/scalacheck-magnolia.git}scalacheck-magnolia...
[info] downloading
https://repo1.maven.org/maven2/com/propensive/magnolia_2.12/0.6.1/magnolia_2.12-0.6.1.jar
...
[info] [SUCCESSFUL ] com.propensive#magnolia_2.12;0.6.1!magnolia_2.12.jar (2011ms)
[info] downloading
https://repo1.maven.org/maven2/org/scalacheck/scalacheck_2.12/1.13.4/scalacheck_2.12-
1.13.4.jar ...
[info] [SUCCESSFUL ] org.scalacheck#scalacheck_2.12;1.13.4!scalacheck_2.12.jar (4767ms)
[info] Updating {file:/Users/marina/IdeaProjects/example/}example...
[info] Compiling 2 Scala sources to
/Users/marina/.sbt/0.13/staging/8924b22f06679a54873e/scalacheck-magnolia/target/scala-
2.12/classes...
[info] 'compiler-interface' not yet compiled for Scala 2.12.4. Compiling...
[info] Compilation completed in 12.795 s
[success] Total time: 28 s, completed 19/01/2018 11:35:13 AM
> compile

> Project

> scope
1. Keys can have different values in
different scopes
• sources in Compile
• sources in Test
2. Keys can be scoped by tasks or
configuration
3. Configurations can extend from each other
• val Test = config("test") extend Compile

The subproject axis
The configuration axis
The task axis
> scope

> scope
Compile, Test
task keys compile, package, run
Project(id = "example-domain")

> referring to scopes in a build definition
.settings(
name := "hello"
)
name in Compile := “hello"
name in packageBin := “hello"
name in (Compile, packageBin) := “hello"

.settings(
name := "hello"
)
name in Compile := "hello"
name in packageBin := "hello"
name in (Compile, packageBin) := “hello"

.settings(
name := "hello"
)
name in Compile := "hello"
name in packageBin := "hello"
name in (Compile, packageBin) := "hello"

> scalacOptions
scalacOptions ++= Seq(
"-encoding", "UTF-8",
"-unchecked",
"-deprecation",
"-Ywarn-dead-code"
)

> scalacOptions
"-unchecked"
"-deprecation",
"-Ywarn-dead-code"
)
def printListType(lst: List[AnyVal]) = lst match {
case strings: List[String] => println(s"List of Strings
$strList")
case integers: List[Int] => println(s"List of Strings
$intList")
}

> scalacOptions
"-deprecation"
"-Ywarn-dead-code"
)
NO OPTION
[warn] there were three deprecation warnings;
re-run with -deprecation for details
WITH OPTION
[warn] Foo.scala:451: method listAll
in class Bar is deprecated: Don't use this method.
Its super broken.
[warn] Bar.listAll(user)

> scalacOptions
"-Ywarn-dead-code"
scala> def bar (a: Int): Int = {
| return 0
| a + 10
| }
:12: warning: dead code following this construct
return 0
^
bar: (a: Int)Int

> plugins.sbt
addSbtPlugin("com.eed3si9n" %
"sbt-man" % "0.1.0"
) // look up scaladoc
"sbt-dirty-money" % "0.1.0")
// clean Ivy2's cache
"sbt-assembly" % "0.14.6")
// create a fat JAR of your project
// with all of its dependencies

> assembly
• Merge Strategy
• Shading
• Excluding JARs and files

> assembly
•Merge Strategy
Shading
Excluding JARs and files
MergeStrategy.deduplicate
verify all files have the same contents & error out otherwise
MergeStrategy.first
picks the first of the matching files in classpath order
MergeStrategy.last picks the last one
MergeStrategy.singleOrError
bails out with an error message on conflict
MergeStrategy.concat
concatenates all matching files and includes the result
MergeStrategy.filterDistinctLines
also concatenates, but leaves out duplicates along the way
MergeStrategy.rename
renames the files originating from jar files
MergeStrategy.discard discards matching files

•Merge Strategy
Shading
> assembly
lazy val root =
Project(
id = "example",
base = file("."),
dependencies = Seq(
domain,
magnolia
)
)
.settings(setts)
.settings(libraryDependencies ++= mainLibs ++ testLibs)
.settings(assemblyJarName in assembly := "example.jar")
.settings(
assemblyMergeStrategy in assembly := {
case "application.conf" => MergeStrategy.concat
case x =>
val oldStrategy = (assemblyMergeStrategy in assembly).value
oldStrategy(x)
}
)

•Merge Strategy
Shading
> assembly
> assembly
[info] Including: scala-library-2.12.4.jar
[info] Including: scala-compiler-2.12.4.jar
…
[info] Merging files...
[warn] Merging 'META-INF/MANIFEST.MF' with strategy 'discard'
[warn] Merging 'META-INF/maven/jline/jline/pom.properties'
with strategy 'discard'
[warn] Merging 'META-INF/maven/jline/jline/pom.xml' with
strategy 'discard'
[warn] Merging 'application.conf' with strategy 'concat'
[warn] Merging 'rootdoc.txt' with strategy 'concat'
[warn] Strategy 'concat' was applied to 2 files
[warn] Strategy 'discard' was applied to 3 files
[info] SHA-1: 642c5daa2bfc30a825f08fa72a51a990cac441fc
[info] Packaging /Users/marina/IdeaProjects/example/target/scala-2.12/example.
[info] Done packaging.

Merge Strategy
•Shading
> assembly
ShadeRule.rename("x.**" -> "y.@1", …)
ShadeRule.zap("a.b.c")
any matched class to be removed from
the resulting jar file
ShadeRule.keep("x.**")
rule marks all matched classes
as "roots"

Merge Strategy
•Shading
> assembly
.settings(
assemblyShadeRules in assembly := Seq(
ShadeRule.rename("com.google.**" -> "org.moogle.@1").inProject
)
)

•Merge Strategy
•Shading
•Excluding JARs and files
> assembly
libraryDependencies ++= Seq(
("org.apache.spark" %% "spark-core" % "0.8.0-incubating")
.exclude("commons-logging", "commons-logging")
)
assemblyMergeStrategy in assembly := {
case PathList("about.html") => MergeStrategy.rename
case x =>
val oldStrategy = (assemblyMergeStrategy in assembly).value
oldStrategy(x)
}

> summary
[info] Build -> Project -> Settings
[info] s & tasks
[info] Proxy Resolvers
[info] Multi-project build
[info] Scope
[info] scalacOptions
[info] Plugins we use
[success] Total time: 45 min

Sbt baby steps

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Sbt baby steps

Similar to Sbt baby steps (20)

Recently uploaded

Recently uploaded (20)

Sbt baby steps