WTF is Takari?
Jason van Zyl
http://takari.io
@takari_io

Agenda
‣
‣
‣
‣
‣
Incremental build
Aggressive parallelization
Generations
Polyglot for Maven
Shell

It’s too hard for organizations
to efficiently deliver
JVM-based software

The problem with delivery
What is an application and how do you reason about it? Why you might be here today?
External Services
Internal Components
3rd Party Components
Configuration
Machine
v1.0

Software Delivery Chain
Developer checks code into the SCM
IDE
Developer requests components
from Repoman and the components
are delivered to the developer
2
CI checks code out of the SCM
SCM
CI
3
CI instructs build tool to
perform a build
4
1
Repository
Central
Build
5
Build tool deploys components
to repository manager
Provisioning system requests components
and configuration from Nexus.
6
Provisioner
7
Agent
Provisioning system provisions apps and
components to the agents
managing runtimes
Agent
Agent

Agenda
‣
‣
‣
‣
‣
Incremental build
Aggressive parallelization
Generations
Polyglot for Maven
Shell

Incremental build

Incremental build
We’ve supported incremental build in m2eclipse for quite a while
‣
‣
‣
‣
m2eclipse is capable of operating on a single resource with a single Mojo call -- very efficient
We implemented incremental build for Maven on the CLI
We implemented a custom Guice scope for Maven’s core to @Inject what we call a BuildContext
We Implemented versions the Mojos for the standard lifecycle including and incremental CLI compiler based
on Eclipse JDT

public class BuildAvoidanceDemo {
private final BuildContext context;
private final File inputDir;
private final File outputDir;
private final Set<String> includes;
private final Set<String> excludes;
//
//
//
//
//
//
performed automatically by the framework
1.
2.
3.
4.
delete all stale or orphaned output files produced during previous build(s)
persist BuildContext state such that it can be used during the next build
replay any messages not cleared from previous build(s)
raise build error if any of the inputs processed in this or previous builds had SEVERITY_ERROR message
@Inject
public BuildAvoidanceDemo(BuildContext context, File inputDir, Set<String> includes, Set<String> excludes, File outputDir) {
this.context = context;
this.inputDir = inputDir;
this.includes = includes;
this.excludes = excludes;
this.outputDir = outputDir;
}
public void execute() throws IOException {
// determine all input files, regardless of their up-to-date status
// in this particular case use basedir with ant-style includes/excludes patterns
FileSet inputs = new FileSetBuilder(inputDir).addIncludes(includes).addExcludes(excludes).build();
// context.registerInputs registers specified inputs FileSets with the build context, which is necessary to
// enable cleanup of stale outputs. it returns collection of input files that require processing
for (File inputFile : context.registerInputs(inputs)) {
// context.addProcessedInput marks beginning of processing of the input file
// it clears all output files and messages associated with the input file for this build
context.addProcessedInput(inputFile);
// mapping between input and output files is outside of scope of BuildContext API
File outputFile = determineOuputFile(outputDir, inputFile);
// context.newOutputStream associates input and output files, which is necessary to determine input files
// that require processing and to perform cleanup of stale output files.
OutputStream os = context.newOutputStream(inputFile, outputFile);
try {
// generation of output file contents is outside of scope of BuildContext API
// likewise, Message object, if any, is application specific and is not part of BuildContext API
Collection<Message> messages = generateOutput(inputFile, os);
}
}
}
// record error and warning messages generated during input file processing
for (Message message : messages) {
context.addMessage(inputFile, message.getLine(), message.getColumn(), message.getText(), BuildContext.SEVERITY_ERROR, null);
}
} finally {
os.close();
}

Aggressive parallelization

Existing parallel mode
Upstream

Aggressive parallel mode
Upstream

Aggressive parallel mode
with smart scheduling
Upstream

Aggressive parallel mode
with smart scheduling
Build Telemetry
Critical path

Generations & Workspaces

SCM
revision 100:branch X
Generation 100

SCM
SCM
SCM
SCM
revision 100:branch X
revision 101:branch X
revision 102:branch X
revision 103:branch X
Generation 100
Generation 101
Generation 102
Generation 103
As time passes and your product gets better (I hope)

Workspaces
How does a workspace relate to a generation?
provisio-webserver
javax.inject
jetty-server
javax.servlet
jetty-continuation
jetty-http
jetty-io
jetty-util
jetty-webapp
jetty-xml
jetty-util
jetty-deploy
jetty-webapp
jetty-util
jetty-jmx
jetty-util
jetty-util

What if generations were supported across the delivery chain?
IDE
2
SCM
CI
3
4
1
Consuming a new generation
Repository
Build
5
Producing a new generation
6
Provisioner
7
Agent
Agent
Agent

Generations collaboration between build and repository manager

Polyglot for Maven

Polyglot for Maven
Maven can support different formats and even DSLs
‣
A lot of work has been done recently on the Ruby and Scala DSLs, but there is support for Groovy, Atom,
YAML, and Clojure as well
‣
‣
We cut the 0.9.0 release yesterday and it’s available to try
The code and instructions are available at https://github.com/tesla/tesla-polyglot

project 'Polyglot Tesla :: Aggregator' do
model_version '4.0.0'
id 'io.tesla.polyglot:tesla-polyglot:0.0.1-SNAPSHOT'
inherit 'io.tesla:tesla:4'
packaging 'pom'
properties( 'sisuInjectVersion' => '0.0.0.M2a',
'teslaVersion' => '3.1.0' )
modules [ 'tesla-polyglot-common',
'tesla-polyglot-atom',
'tesla-polyglot-ruby',
'tesla-polyglot-groovy',
'tesla-polyglot-yaml',
'tesla-polyglot-clojure',
'tesla-polyglot-scala',
'tesla-polyglot-cli',
'tesla-polyglot-maven-plugin' ]
overrides do
jar 'org.eclipse.sisu:org.eclipse.sisu.inject:${sisuInjectVersion}'
jar 'org.eclipse.sisu:org.eclipse.sisu.plexus:${sisuInjectVersion}'
jar 'org.apache.maven:maven-model-builder:3.1.0'
jar 'org.apache.maven:maven-embedder:3.1.0'
jar( 'junit:junit:4.11', :scope => 'test' )
end
plugin 'org.codehaus.plexus:plexus-component-metadata:1.5.4' do
execute_goals 'generate-metadata', 'generate-test-metadata'
end
build do
execute("first", :validate) do |context|
puts "Hello from JRuby!"
end
end
end

project {
modelVersion '4.0.0'
parent('io.tesla:tesla:4')
groupId 'io.tesla.polyglot'
artifactId 'tesla-polyglot'
version '0.0.1-SNAPSHOT'
packaging 'pom'
name 'Polyglot Tesla :: Aggregator'
modules([
'tesla-polyglot-common',
'tesla-polyglot-atom',
'tesla-polyglot-ruby',
'tesla-polyglot-groovy',
'tesla-polyglot-yaml',
'tesla-polyglot-clojure',
'tesla-polyglot-scala',
'tesla-polyglot-cli',
'tesla-polyglot-maven-plugin',
])
properties {
sisuInjectVersion '0.0.0.M2a'
teslaVersion '3.1.0'
}
dependencyManagement {
dependencies {
dependency('org.eclipse.sisu:org.eclipse.sisu.inject:${sisuInjectVersion}')
dependency('org.eclipse.sisu:org.eclipse.sisu.plexus:${sisuInjectVersion}')
dependency('org.apache.maven:maven-model-builder:3.1.0')
dependency('org.apache.maven:maven-embedder:3.1.0')
dependency('junit:junit:4.11:test')
}
}
build {
$execute(id: 'hello', phase: 'validate') {
println ""
println "hello, I am Groovy inside Maven. What? What am I doing here?? I thought he was my arch nemesis! I'm confused."
println ""
}
}
}
plugins {
plugin('org.codehaus.plexus:plexus-component-metadata:1.5.4') {
executions {
execution(goals: ['generate-metadata', 'generate-test-metadata'])
}
}
}

import org.sonatype.maven.polyglot.scala.model._
implicit val scalaVersion = ScalaVersion("2.10.2")
ScalaModel(
"" % "tesla-polyglot-scala",
name = "Polyglot Tesla :: Scala",
contributors = Seq(
Contributor(
name = "Christopher Hunt",
organization = "Typesafe",
organizationUrl = "http://typesafe.com"
)
),
parent = Parent("io.tesla.polyglot" % "tesla-polyglot" % "0.0.1-SNAPSHOT"),
repositories = Seq(
Repository(
snapshots = RepositoryPolicy(enabled = false),
id = "sonatype-public-grid",
url = "http://repository.sonatype.org/content/groups/sonatype-public-grid/"
)
),
dependencies = Seq(
"io.tesla.polyglot" % "tesla-polyglot-common" % "0.0.1-SNAPSHOT",
"com.twitter" %% "util-eval" % "6.3.8",
"com.googlecode.kiama" %% "kiama" % "1.5.1",
"org.specs2" %% "specs2" % "2.1.1" % "test",
"junit" % "junit" % "" % "test"
),
build = Build(
plugins = Seq(
Plugin(
"io.tesla.maven.plugins" % "tesla-license-plugin",
configuration = Config(
header = "../license-header.txt"
)
)
),
tasks = Seq(Task("someTaskId", "verify") {
ec => println("I'm Scala running during the verify phase. The ec passed in allows me to access the project")
})
),
modelVersion = "4.0.0"
)

modelVersion:
4.0.0
parent:
{artifactId:
tesla,
groupId:
io.tesla,
relativePath:
../pom.xml,
version:
'4'}
groupId:
io.tesla.polyglot
artifactId:
tesla-­‐polyglot
version:
0.0.1-­‐SNAPSHOT
packaging:
pom
name:
'Polyglot
Tesla
::
Aggregator'
properties:
{sisuInjectVersion:
0.0.0.M2a,
teslaVersion:
3.1.0}
build:
plugins:
-­‐
artifactId:
plexus-­‐component-­‐metadata
executions:
-­‐
goals:
[generate-­‐metadata,
generate-­‐test-­‐metadata]
id:
default
inherited:
true
priority:
0
extensions:
false
groupId:
org.codehaus.plexus
inherited:
true
version:
1.5.4
modules:
[
tesla-­‐polyglot-­‐common,
tesla-­‐polyglot-­‐atom,
tesla-­‐polyglot-­‐ruby,
tesla-­‐polyglot-­‐groovy,
tesla-­‐polyglot-­‐yaml,
tesla-­‐polyglot-­‐clojure,
tesla-­‐polyglot-­‐scala,
tesla-­‐polyglot-­‐cli,
tesla-­‐polyglot-­‐maven-­‐plugin]
dependencyManagement:
dependencies:
-­‐
{artifactId:
org.eclipse.sisu.inject,
groupId:
org.eclipse.sisu,
optional:
false,
type:
jar,
version:
'${sisuInjectVersion}'}
-­‐
{artifactId:
org.eclipse.sisu.plexus,
groupId:
org.eclipse.sisu,
optional:
false,
type:
jar,
version:
'${sisuInjectVersion}'}
-­‐
{artifactId:
maven-­‐model-­‐builder,
groupId:
org.apache.maven,
optional:
false,
type:
jar,
version:
3.1.0}
-­‐
{artifactId:
maven-­‐embedder,
groupId:
org.apache.maven,
optional:
false,
type:
jar,
version:
3.1.0}
-­‐
{artifactId:
junit,
groupId:
junit,
optional:
false,
scope:
test,
type:
jar,
version:
'4.11'}

project "Polyglot Tesla :: Aggregator" @ "" as pom
id: io.tesla.polyglot:tesla-polyglot:0.0.1-SNAPSHOT
inherits: io.tesla:tesla:4
properties: [ sisuInjectVersion: "0.0.0.M2a"
teslaVersion: "3.1.0" ]
overrides: [ org.eclipse.sisu:org.eclipse.sisu.inject:${sisuInjectVersion}
org.eclipse.sisu:org.eclipse.sisu.plexus:${sisuInjectVersion}
org.apache.maven:maven-model-builder:3.1.0
org.apache.maven:maven-embedder:3.1.0
junit:junit:4.11 ]
modules: [ tesla-polyglot-common
tesla-polyglot-atom
tesla-polyglot-ruby
tesla-polyglot-groovy
tesla-polyglot-yaml
tesla-polyglot-clojure
tesla-polyglot-scala
tesla-polyglot-cli
tesla-polyglot-maven-plugin ]
plugin
id: org.codehaus.plexus:plexus-component-metadata:1.5.4

Mavenize like a champ.
Introduction to Maven Virtual Training
Thursday, February 13, 2014
1:00PM-7:30PM EST (UTC -5 hours)
575CAD/student
Small classes, an accomplished instructor, top-notch training…all from
the comfort of your own pajamas office. Learn more at www.takari.io