Custom deployments with sbt-native-packager

Custom Deployments
with SBT-Native-Packager
Gary Coady 
gcoady@gilt.com
Twitter: @ﬁadliel

Why Native Packager?
• Java = “run anywhere”
• Native Packager = “run Java anywhere”
• Classpath, JVM parameters, command-line
arguments, environment variables, behaviour on
quit, …

$
bin/my-‐first-‐app
-‐h

Usage:

[options]

-‐h
|
-‐help

print
this
message

-‐v
|
-‐verbose

this
runner
is
chattier

-‐d
|
-‐debug

set
sbt
log
level
to
debug

-‐no-‐version-‐check

Don't
run
the
java
version
check.

-‐main
<classname>

Define
a
custom
main
class

-‐jvm-‐debug
<port>

Turn
on
JVM
debugging,
open
at
the
given
port.

#
java
version
(default:
java
from
PATH,
currently
java
version
"1.8.0_45")

-‐java-‐home
<path>

alternate
JAVA_HOME

#
jvm
options
and
output
control

JAVA_OPTS

environment
variable,
if
unset
uses
""

-‐Dkey=val

pass
-‐Dkey=val
directly
to
the
java
runtime

-‐J-‐X

pass
option
-‐X
directly
to
the
java
runtime

(-‐J
is
stripped)

#
special
option

-‐-‐

To
stop
parsing
built-‐in
commands
from
the
rest
of
the
command-‐line.

e.g.)
enabling
debug
and
sending
-‐d
as
app
argument

$
./start-‐script
-‐d
-‐-‐
-‐d

In
the
case
of
duplicated
or
conflicting
options,
basically
the
order
above

shows
precedence:
JAVA_OPTS
lowest,
command
line
options
highest
except
"-‐-‐".

Setting[T]
Computation run once, returning T
>
set
name
:=
{
println("hello
world!”);
"name"
}

[info]
Defining
*:name

[info]
Reapplying
settings...

hello
world!

[info]
Set
current
project
to
name
(in
build
file:/Users/gcoady/my-‐project/)

>
name

[info]
name

Task[T]
Computation run every time value is needed, returning T
>
set
run
:=
{
println("hello
world");
()
}

[info]
Defining
*:run

[info]
The
new
value
will
be
used
by
no
settings
or
tasks.

[info]
Reapplying
settings...

blah

[info]
Set
current
project
to
name
(in
build
file:/Users/gcoady/my-‐project/)

>
run

hello
world

[success]
Total
time:
0
s,
completed
11-‐Feb-‐2016
14:59:18

>
run

hello
world

[success]
Total
time:
0
s,
completed
11-‐Feb-‐2016
14:59:19

Dependencies
• Tasks can depend on other tasks and settings
• Settings can depend on other settings

Keys
Typed identiﬁer & documentation for settings/tasks
>
inspect
name

[info]
Setting:
java.lang.String
=
name

[info]
Description:

[info]

Project
name.

[info]
Provided
by:

[info]

{file:/Users/gcoady/my-‐project/}my-‐project/*:name

AutoPlugin Ecosystem
SbtNativePackager
UniversalPlugin
DockerPlugin
LinuxPlugin
WindowsPluginDebianPlugin RpmPlugin
JavaAppPackaging

Universal Conﬁguration
• Provided by Universal Plugin
• Platform-independent layout
• Staging
• Package zip & tgz ﬁles
• Depended on by other deployment formats

Universal Conﬁguration
val
mappings
=
TaskKey[Seq[(File,
String)]](

"mappings",

"Defines
the
mappings
from
a
file
to
a

path,
used
by
packaging,
for
example." 
)

Starting your plugin
sbtPlugin := true 
 
// The Typesafe repository 
resolvers += "Typesafe repository" at "https://repo.typesafe.com/typesafe/releases/" 
 
addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.0.3" % "provided")

Starting your plugin
object MyPlugin extends AutoPlugin { 
 
val MyPluginConfig = config("myplugin") extend Universal 
 
object autoImport { 
val myPluginSetting = settingKey[Seq[String]](“A custom setting") 
val myPluginTask = taskKey[File](“A custom task") 
} 
 
import autoImport._ 
 
override val requires = JavaAppPackaging 
 
override val projectSettings = inConfig(MyPluginConfig)(Seq(
// Configure settings for project here 
)) 
}

Case Study
• YourKit Proﬁler: 
An awesome CPU and memory Java Proﬁler
• Lots of annoying steps to install

Plugin Requirements
• Add platform-specific shared library
• Add flags to use library as Java agent
• Allow changes to configuration at deployment time

Adding resources
$
find
src/main/resources
-‐type
f

src/main/resources/yjp-‐2015-‐build-‐15068/bin/aix-‐ppc-‐32/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/aix-‐ppc-‐64/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/freebsd-‐x86-‐32/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/freebsd-‐x86-‐64/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/hpux-‐ia64-‐32/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/hpux-‐ia64-‐64/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/linux-‐aarch64/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/linux-‐armv5-‐sf/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/linux-‐armv7-‐hf/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/linux-‐ppc-‐32/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/linux-‐ppc-‐64/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/linux-‐ppc64le/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/linux-‐x86-‐32/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/linux-‐x86-‐64/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/mac/libyjpagent.jnilib

src/main/resources/yjp-‐2015-‐build-‐15068/bin/solaris-‐sparc-‐32/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/solaris-‐sparc-‐64/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/solaris-‐x86-‐32/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/solaris-‐x86-‐64/libyjpagent.so

src/main/resources/yjp-‐2015-‐build-‐15068/bin/win32/yjpagent.dll

src/main/resources/yjp-‐2015-‐build-‐15068/bin/win64/yjpagent.dll

Adding resources
val
stream
=
Option(getClass.getResourceAsStream(path)) 
stream
match
{ 

case
Some(s)
=> 

val
tempFile
=
targetDir
/
"yourkit"
/
p
/
s"yourkit.$ext" 

tempFile.getParentFile.mkdirs() 

IO.transferAndClose(s,
new
java.io.FileOutputStream(tempFile))

Adding resources
mappings in Universal ++=
yourKitAgents.value.map(agent =>
agent.sourceFile -> agent.targetPath
)

Java entry point generation
• Provided by JavaAppPackaging
• Creates start script for Java applications
• Arbitrary bash code can be injected
• Adding Java agents to Java binaries
• Environment discovery & injection

Injecting arbitrary code
val
bashScriptExtraDefines
=
TaskKey[Seq[String]](

“bashScriptExtraDefines",

"A
list
of
extra
definitions
that
should
be
written
to
the

bash
file
template.")

Bash script helpers
• ${app_path}/…/ — root directory of distribution
• addJava — adds a Java argument
• addApp — adds an argument to your application
• addResidual — adds an argument to your
application, goes after non-residuals

Injecting bash code
def
startYourKitScript(defaultStartupOptions:
String):
String
=
""" 
if
[[
-‐z
"$YOURKIT_AGENT_DISABLED"
]];
then 

if
[[
-‐z
"$YOURKIT_AGENT_STARTUP_OPTIONS"
]];
then 

YOURKIT_AGENT_STARTUP_OPTIONS=""""
+
defaultStartupOptions
+
"""" 

export
YOURKIT_AGENT_STARTUP_OPTIONS 

fi 
"""

bashScriptExtraDefines
+=

"""addJava
"-‐agentpath:${app_home}/../"""
+

mapping
+

"""=${YOURKIT_AGENT_STARTUP_OPTIONS}""""

Trying it out
addSbtPlugin("com.gilt.sbt" % "sbt-yourkit" % "0.0.2")
enablePlugins(PlayScala,
YourKit)

Debugging
• Universal conﬁguration IS universal
• universal:stage presents application layout

Online Examples
• https://github.com/gilt/sbt-yourkit
• https://github.com/gilt/sbt-newrelic 
Downloads New Relic agent with Ivy 
Generates NR conﬁguration from SBT settings 
Adds agent, conﬁguration, and appropriate startup
arguments
• https://github.com/gilt/sbt-aspectjweaver 
Downloads AspectJWeaver agent with Ivy 
Adds agent and appropriate startup arguments

Native Packager Supported
Formats
• Zip/TGZ Archives
• Windows MSI
• OSX DMG
• Debian DEB
• Red Hat / Fedora RPM
• Docker

Case Study: ACI Images
• Used by RKT (Docker alternative)
• TAR format
• Optional compression, GPG signatures
• /manifest: Image metadata
• /rootfs/: Image contents

Deﬁning required keys
object autoImport {
val aciDependencies = settingKey[Seq[String]](“ACI Dependencies") 
val aciManifest = taskKey[File]("ACI Manifest") 
}

Reusing existing keys
val Aci = config("aci") extend Universal

Create ﬁle/path mappings
mappings
:=
(

renameDests((mappings
in
Universal).value,
"rootfs")
++

Seq(aciManifest.value
-‐>
“manifest")

) 
def
renameDest(originalPath:
String,
dest:
String)
= 

"%s/%s"
format
(dest,
originalPath) 
 
def
renameDests(from:
Seq[(File,
String)],
dest:
String)
= 

for
{ 

(f,
path)
<-‐
from 

}
yield
(f,
renameDest(path,
dest))

Create target ﬁle
packageBin := 
Archives.makeTarball(Archives.gzip, “.aci")(
target.value, normalizedName.value, mappings.value, None)

Online Code
• https://github.com/ﬁadliel/sbt-aci

Summary
• Extending SBT Native Packager is easy
• Alter program environment
• Create alternative formats for distribution

Custom deployments with sbt-native-packager

More Related Content

What's hot

Viewers also liked

Similar to Custom deployments with sbt-native-packager

Recently uploaded

Custom deployments with sbt-native-packager