Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
WTF is Takari?
Jason van Zyl
http://takari.io
@takari_io
Agenda

‣
‣
‣
‣
‣

Incremental build
Aggressive parallelization
Generations
Polyglot for Maven
Shell
It’s too hard for organizations
to efficiently deliver
JVM-based software
The problem with delivery
What is an application and how do you reason about it? Why you might be here today?

External Se...
Software Delivery Chain
Developer checks code into the SCM

IDE

Developer requests components
from Repoman and the compon...
Agenda

‣
‣
‣
‣
‣

Incremental build
Aggressive parallelization
Generations
Polyglot for Maven
Shell
Incremental build
Incremental build
We’ve supported incremental build in m2eclipse for quite a while

‣
‣
‣
‣

m2eclipse is capable of opera...
public class BuildAvoidanceDemo {
private final BuildContext context;
private final File inputDir;
private final File outp...
Aggressive parallelization
Existing parallel mode

Upstream
Aggressive parallel mode

Upstream
Aggressive parallel mode
with smart scheduling

Upstream
Aggressive parallel mode
with smart scheduling

Build Telemetry
Critical path
Generations & Workspaces
SCM

revision 100:branch X

Generation 100
SCM

SCM

SCM

SCM

revision 100:branch X

revision 101:branch X

revision 102:branch X

revision 103:branch X

Generation...
Workspaces
How does a workspace relate to a generation?
provisio-webserver
javax.inject
jetty-server
javax.servlet
jetty-c...
What if generations were supported across the delivery chain?

IDE

2

SCM

CI

3

4

1

Consuming a new generation
Reposi...
Generations collaboration between build and repository manager
Polyglot for Maven
Polyglot for Maven
Maven can support different formats and even DSLs

‣

A lot of work has been done recently on the Ruby ...
project 'Polyglot Tesla :: Aggregator' do
model_version '4.0.0'
id 'io.tesla.polyglot:tesla-polyglot:0.0.1-SNAPSHOT'
inher...
project {
modelVersion '4.0.0'
parent('io.tesla:tesla:4')
groupId 'io.tesla.polyglot'
artifactId 'tesla-polyglot'
version ...
import org.sonatype.maven.polyglot.scala.model._
implicit val scalaVersion = ScalaVersion("2.10.2")
ScalaModel(
"" % "tesl...
modelVersion:	
  4.0.0
parent:	
  {artifactId:	
  tesla,	
  groupId:	
  io.tesla,	
  relativePath:	
  ../pom.xml,	
  versi...
project "Polyglot Tesla :: Aggregator" @ "" as pom
id: io.tesla.polyglot:tesla-polyglot:0.0.1-SNAPSHOT
inherits: io.tesla:...
Mavenize like a champ.
Introduction to Maven Virtual Training
Thursday, February 13, 2014
1:00PM-7:30PM EST (UTC -5 hours)...
 Maven just won't die -- WTF is Takari?
 Maven just won't die -- WTF is Takari?
 Maven just won't die -- WTF is Takari?
 Maven just won't die -- WTF is Takari?
Upcoming SlideShare
Loading in …5
×

Maven just won't die -- WTF is Takari?

3,661 views

Published on

In this webinar, we touch on:

- Polyglot support: Ruby, Groovy, and Scala DSLs. These have all been actively worked on in the recent past, especially the Ruby and Scala DSLs.

- Full incremental support: the complete Maven lifecycle including an incremental command line compiler based on JDT, all with m2e integration

- Aggressive parallelization: a new parallelization mode that also optimizes scheduling based on critical path analysis

- Generations: a new form of continuous delivery for Maven -- Smart delta protocol and no more SNAPHOTs!

- Shell: long-lived CLI process for Maven

Much of this work is functional, and the new parallelization mode and generations support are actively being used in production. We are still iterating on these specific features but they show a lot of promise. Where all of this code eventually lands is a question for the Maven development community. All of this work was developed outside of Apache, and how easy it is to integrate back into the Maven project remains to be seen. At the very least there is a lot of very interesting work and we wanted to start the dialog because Maven just isn't going to die :).

Published in: Technology

Maven just won't die -- WTF is Takari?

  1. 1. WTF is Takari? Jason van Zyl http://takari.io @takari_io
  2. 2. Agenda ‣ ‣ ‣ ‣ ‣ Incremental build Aggressive parallelization Generations Polyglot for Maven Shell
  3. 3. It’s too hard for organizations to efficiently deliver JVM-based software
  4. 4. The problem with delivery What is an application and how do you reason about it? Why you might be here today? External Services Internal Components 3rd Party Components Configuration Machine v1.0
  5. 5. Software Delivery Chain Developer checks code into the SCM IDE Developer requests components from Repoman and the components are delivered to the developer 2 CI checks code out of the SCM SCM CI 3 CI instructs build tool to perform a build 4 1 Repository Central Build 5 Build tool deploys components to repository manager Provisioning system requests components and configuration from Nexus. 6 Provisioner 7 Agent Provisioning system provisions apps and components to the agents managing runtimes Agent Agent
  6. 6. Agenda ‣ ‣ ‣ ‣ ‣ Incremental build Aggressive parallelization Generations Polyglot for Maven Shell
  7. 7. Incremental build
  8. 8. Incremental build We’ve supported incremental build in m2eclipse for quite a while ‣ ‣ ‣ ‣ m2eclipse is capable of operating on a single resource with a single Mojo call -- very efficient We implemented incremental build for Maven on the CLI We implemented a custom Guice scope for Maven’s core to @Inject what we call a BuildContext We Implemented versions the Mojos for the standard lifecycle including and incremental CLI compiler based on Eclipse JDT
  9. 9. public class BuildAvoidanceDemo { private final BuildContext context; private final File inputDir; private final File outputDir; private final Set<String> includes; private final Set<String> excludes; // // // // // // performed automatically by the framework 1. 2. 3. 4. delete all stale or orphaned output files produced during previous build(s) persist BuildContext state such that it can be used during the next build replay any messages not cleared from previous build(s) raise build error if any of the inputs processed in this or previous builds had SEVERITY_ERROR message @Inject public BuildAvoidanceDemo(BuildContext context, File inputDir, Set<String> includes, Set<String> excludes, File outputDir) { this.context = context; this.inputDir = inputDir; this.includes = includes; this.excludes = excludes; this.outputDir = outputDir; } public void execute() throws IOException { // determine all input files, regardless of their up-to-date status // in this particular case use basedir with ant-style includes/excludes patterns FileSet inputs = new FileSetBuilder(inputDir).addIncludes(includes).addExcludes(excludes).build(); // context.registerInputs registers specified inputs FileSets with the build context, which is necessary to // enable cleanup of stale outputs. it returns collection of input files that require processing for (File inputFile : context.registerInputs(inputs)) { // context.addProcessedInput marks beginning of processing of the input file // it clears all output files and messages associated with the input file for this build context.addProcessedInput(inputFile); // mapping between input and output files is outside of scope of BuildContext API File outputFile = determineOuputFile(outputDir, inputFile); // context.newOutputStream associates input and output files, which is necessary to determine input files // that require processing and to perform cleanup of stale output files. OutputStream os = context.newOutputStream(inputFile, outputFile); try { // generation of output file contents is outside of scope of BuildContext API // likewise, Message object, if any, is application specific and is not part of BuildContext API Collection<Message> messages = generateOutput(inputFile, os); } } } // record error and warning messages generated during input file processing for (Message message : messages) { context.addMessage(inputFile, message.getLine(), message.getColumn(), message.getText(), BuildContext.SEVERITY_ERROR, null); } } finally { os.close(); }
  10. 10. Aggressive parallelization
  11. 11. Existing parallel mode Upstream
  12. 12. Aggressive parallel mode Upstream
  13. 13. Aggressive parallel mode with smart scheduling Upstream
  14. 14. Aggressive parallel mode with smart scheduling Build Telemetry Critical path
  15. 15. Generations & Workspaces
  16. 16. SCM revision 100:branch X Generation 100
  17. 17. SCM SCM SCM SCM revision 100:branch X revision 101:branch X revision 102:branch X revision 103:branch X Generation 100 Generation 101 Generation 102 Generation 103 As time passes and your product gets better (I hope)
  18. 18. Workspaces How does a workspace relate to a generation? provisio-webserver javax.inject jetty-server javax.servlet jetty-continuation jetty-http jetty-io jetty-util jetty-webapp jetty-xml jetty-util jetty-deploy jetty-webapp jetty-util jetty-jmx jetty-util jetty-util
  19. 19. What if generations were supported across the delivery chain? IDE 2 SCM CI 3 4 1 Consuming a new generation Repository Build 5 Producing a new generation 6 Provisioner 7 Agent Agent Agent
  20. 20. Generations collaboration between build and repository manager
  21. 21. Polyglot for Maven
  22. 22. Polyglot for Maven Maven can support different formats and even DSLs ‣ A lot of work has been done recently on the Ruby and Scala DSLs, but there is support for Groovy, Atom, YAML, and Clojure as well ‣ ‣ We cut the 0.9.0 release yesterday and it’s available to try The code and instructions are available at https://github.com/tesla/tesla-polyglot
  23. 23. project 'Polyglot Tesla :: Aggregator' do model_version '4.0.0' id 'io.tesla.polyglot:tesla-polyglot:0.0.1-SNAPSHOT' inherit 'io.tesla:tesla:4' packaging 'pom' properties( 'sisuInjectVersion' => '0.0.0.M2a', 'teslaVersion' => '3.1.0' ) modules [ 'tesla-polyglot-common', 'tesla-polyglot-atom', 'tesla-polyglot-ruby', 'tesla-polyglot-groovy', 'tesla-polyglot-yaml', 'tesla-polyglot-clojure', 'tesla-polyglot-scala', 'tesla-polyglot-cli', 'tesla-polyglot-maven-plugin' ] overrides do jar 'org.eclipse.sisu:org.eclipse.sisu.inject:${sisuInjectVersion}' jar 'org.eclipse.sisu:org.eclipse.sisu.plexus:${sisuInjectVersion}' jar 'org.apache.maven:maven-model-builder:3.1.0' jar 'org.apache.maven:maven-embedder:3.1.0' jar( 'junit:junit:4.11', :scope => 'test' ) end plugin 'org.codehaus.plexus:plexus-component-metadata:1.5.4' do execute_goals 'generate-metadata', 'generate-test-metadata' end build do execute("first", :validate) do |context| puts "Hello from JRuby!" end end end
  24. 24. project { modelVersion '4.0.0' parent('io.tesla:tesla:4') groupId 'io.tesla.polyglot' artifactId 'tesla-polyglot' version '0.0.1-SNAPSHOT' packaging 'pom' name 'Polyglot Tesla :: Aggregator' modules([ 'tesla-polyglot-common', 'tesla-polyglot-atom', 'tesla-polyglot-ruby', 'tesla-polyglot-groovy', 'tesla-polyglot-yaml', 'tesla-polyglot-clojure', 'tesla-polyglot-scala', 'tesla-polyglot-cli', 'tesla-polyglot-maven-plugin', ]) properties { sisuInjectVersion '0.0.0.M2a' teslaVersion '3.1.0' } dependencyManagement { dependencies { dependency('org.eclipse.sisu:org.eclipse.sisu.inject:${sisuInjectVersion}') dependency('org.eclipse.sisu:org.eclipse.sisu.plexus:${sisuInjectVersion}') dependency('org.apache.maven:maven-model-builder:3.1.0') dependency('org.apache.maven:maven-embedder:3.1.0') dependency('junit:junit:4.11:test') } } build { $execute(id: 'hello', phase: 'validate') { println "" println "hello, I am Groovy inside Maven. What? What am I doing here?? I thought he was my arch nemesis! I'm confused." println "" } } } plugins { plugin('org.codehaus.plexus:plexus-component-metadata:1.5.4') { executions { execution(goals: ['generate-metadata', 'generate-test-metadata']) } } }
  25. 25. import org.sonatype.maven.polyglot.scala.model._ implicit val scalaVersion = ScalaVersion("2.10.2") ScalaModel( "" % "tesla-polyglot-scala", name = "Polyglot Tesla :: Scala", contributors = Seq( Contributor( name = "Christopher Hunt", organization = "Typesafe", organizationUrl = "http://typesafe.com" ) ), parent = Parent("io.tesla.polyglot" % "tesla-polyglot" % "0.0.1-SNAPSHOT"), repositories = Seq( Repository( snapshots = RepositoryPolicy(enabled = false), id = "sonatype-public-grid", url = "http://repository.sonatype.org/content/groups/sonatype-public-grid/" ) ), dependencies = Seq( "io.tesla.polyglot" % "tesla-polyglot-common" % "0.0.1-SNAPSHOT", "com.twitter" %% "util-eval" % "6.3.8", "com.googlecode.kiama" %% "kiama" % "1.5.1", "org.specs2" %% "specs2" % "2.1.1" % "test", "junit" % "junit" % "" % "test" ), build = Build( plugins = Seq( Plugin( "io.tesla.maven.plugins" % "tesla-license-plugin", configuration = Config( header = "../license-header.txt" ) ) ), tasks = Seq(Task("someTaskId", "verify") { ec => println("I'm Scala running during the verify phase. The ec passed in allows me to access the project") }) ), modelVersion = "4.0.0" )
  26. 26. modelVersion:  4.0.0 parent:  {artifactId:  tesla,  groupId:  io.tesla,  relativePath:  ../pom.xml,  version:  '4'} groupId:  io.tesla.polyglot artifactId:  tesla-­‐polyglot version:  0.0.1-­‐SNAPSHOT packaging:  pom name:  'Polyglot  Tesla  ::  Aggregator' properties:  {sisuInjectVersion:  0.0.0.M2a,  teslaVersion:  3.1.0} build:    plugins:    -­‐  artifactId:  plexus-­‐component-­‐metadata        executions:        -­‐  goals:  [generate-­‐metadata,  generate-­‐test-­‐metadata]            id:  default            inherited:  true            priority:  0        extensions:  false        groupId:  org.codehaus.plexus        inherited:  true        version:  1.5.4 modules:  [    tesla-­‐polyglot-­‐common,      tesla-­‐polyglot-­‐atom,      tesla-­‐polyglot-­‐ruby,      tesla-­‐polyglot-­‐groovy,    tesla-­‐polyglot-­‐yaml,      tesla-­‐polyglot-­‐clojure,      tesla-­‐polyglot-­‐scala,      tesla-­‐polyglot-­‐cli,    tesla-­‐polyglot-­‐maven-­‐plugin] dependencyManagement:    dependencies:    -­‐  {artifactId:  org.eclipse.sisu.inject,  groupId:  org.eclipse.sisu,  optional:  false,  type:  jar,  version:  '${sisuInjectVersion}'}    -­‐  {artifactId:  org.eclipse.sisu.plexus,  groupId:  org.eclipse.sisu,  optional:  false,  type:  jar,  version:  '${sisuInjectVersion}'}    -­‐  {artifactId:  maven-­‐model-­‐builder,  groupId:  org.apache.maven,  optional:  false,  type:  jar,  version:  3.1.0}    -­‐  {artifactId:  maven-­‐embedder,  groupId:  org.apache.maven,  optional:  false,  type:  jar,  version:  3.1.0}    -­‐  {artifactId:  junit,  groupId:  junit,  optional:  false,  scope:  test,  type:  jar,  version:  '4.11'}
  27. 27. project "Polyglot Tesla :: Aggregator" @ "" as pom id: io.tesla.polyglot:tesla-polyglot:0.0.1-SNAPSHOT inherits: io.tesla:tesla:4 properties: [ sisuInjectVersion: "0.0.0.M2a" teslaVersion: "3.1.0" ] overrides: [ org.eclipse.sisu:org.eclipse.sisu.inject:${sisuInjectVersion} org.eclipse.sisu:org.eclipse.sisu.plexus:${sisuInjectVersion} org.apache.maven:maven-model-builder:3.1.0 org.apache.maven:maven-embedder:3.1.0 junit:junit:4.11 ] modules: [ tesla-polyglot-common tesla-polyglot-atom tesla-polyglot-ruby tesla-polyglot-groovy tesla-polyglot-yaml tesla-polyglot-clojure tesla-polyglot-scala tesla-polyglot-cli tesla-polyglot-maven-plugin ] plugin id: org.codehaus.plexus:plexus-component-metadata:1.5.4
  28. 28. Mavenize like a champ. Introduction to Maven Virtual Training Thursday, February 13, 2014 1:00PM-7:30PM EST (UTC -5 hours) 575CAD/student Small classes, an accomplished instructor, top-notch training…all from the comfort of your own pajamas office. Learn more at www.takari.io

×