SlideShare a Scribd company logo
Why your build matters
Peter Ledbrook
e:	

 peter@cacoethes.co.uk	

w:	

 http://www.cacoethes.co.uk	

t:	

 @pledbrook
When did it all begin?
• Programmable
machines have a long
history	

• Mechanised
instruments	

• Looms
Punched
cards
First used for looms in 18th Century France - Jacquard Loom 1801
IBM introduced punched card format 1928 - 80 columns!

 Text mode column width for DOS, Unix etc.
Imagine creating by hand
Keypunch machines (a little like typewriters) for creating them

 Errors mean throwing away the card and doing it again
Fortran and Cobol started with punched cards
Fixing card decks
What if your cards are dropped on the floor?
This is an IBM 082 Sorter
Automation of a very error-prone manual process
Magnetism saves the day!
Finally, R/W data and code. The dawn of text editors.
C/C++
Multi-platform
builds!
Compiler
Linker
Source
.so/.dll & .exe
Make
*.c: *.o	
	 	 cc …	
!
myapp: app.o mylib.o …	
	 	 ld …
+ manual custom dependencies
Focus mainly on compilation and linking
Make has powerful model based on file dependencies
Maintenance was a lot of work
Make
*.c: *.o	
	 	 cc …	
!
myapp: app.o mylib.o …	
	 	 ld …
What’s this?
+ manual custom dependencies
Tabs were the bane of Make file writers
Java
• Compiler handles .java to .class
dependencies	

• JAR as ‘module’ unit	

• Several standard deployment types	

- Applet	

- WAR	

- Standalone app
A standard build
Compile Test Package Publish?
A simplification from the days of C
Java build evolved
Compile Test Package Publish?
Dev setup Run
QA
SCM Tags
Deployment
DocsGenerators
JS + CSS
Builds are incorporating more and more steps
What will builds look like in 5
years time?
What is common across all these systems?
To understand that, let’s look at a non-software process
Making tea
Boil kettle
Pour water
into pot
Empty pot
Put tea in pot
Brew for n
seconds
Fill kettle
Clear that it’s a series of steps (or tasks)
Some tasks require others to complete first
Others can run in parallel
It’s an acyclic graph of tasks!
Making tea (my way)
Boil water on
hob
Add tea to pan
Brew for n
seconds
Measure water
into pan
A different way to make tea
Less standard, more appropriate for me
A standard process does not
mean an exclusive one
Many things can go wrong
Boil kettle
Pour water
into pot
Empty pot
Put tea in pot
Brew for n
seconds
Fill kettle
Forget to
empty pot
Brew for
too long
Put too little or
too much water in
Automate to eliminate human
error
People are fallible
Muscle memory can help

 e.g. tying shoe laces

 Not usually applicable in software development
What if your job depends on
perfect tea every time?
Inputs & parameters
Boil kettle
Pour water
into pot
Empty pot
Put tea in pot
Brew for n
seconds
Fill kettle
What type
of tea?
How much?
How long?
How much
water?
Build inputs:

 * water quality

 * tea bags/loose leaf

 * tea type
Build parameters:

 * quantity of tea

 * quantity of water

 * brew time
Same inputs should result in
same outputs
In other words, repeatable builds
Environment should not be a factor

 - boiling kettle at top of mountain reduces temperature of water
Consider “works for me” with local Maven cache
How many cups of tea do you
need to make a day?
Speed and efficiency are useful
Task avoidance
Boil kettle
Pour water
into pot
Empty pot
Put tea in pot
Brew for n
seconds
Fill kettle What if
pot already
empty?
What if hot
water already
available?
Doing all steps slows you down if some are unnecessary
Output of “boil kettle” is hot water
If hot water already exists, no need to boil
Incremental build (up-to-date
checks) saves time
Often quicker to check whether a task is up to date rather than run it regardless
A build should be…
Automated
Repeatable
As fast as possible
The
Way
Automation
Custom build
publishPdf
publishGuide
apiDocs
fetchGrailsSource
grails-doc
project
expensive,
so optional
A build to generate the Grails user guide
Automation of all steps
No standard tasks
Task graph is a suitable model
Custom tasks in Gradle
task	 publishGuide	 <<	 {

	 	 	 	 //	 Generate	 HTML	 from	 .gdoc

}
buildSrc/src/groovy/pkg/PublishGuideTask.groovy
pkg.PublishGuideTask in a JAR
Optional task
apiDocs.onlyIf	 {

	 	 	 !System.getProperty("disable.groovydocs")

}
Add custom tasks to standard
Groovy/Java projects
Even “standard” builds often have custom steps, e.g.

 integration tests

 deployment

 documentation
Stop using scripts or other tools separate from the build!
Repeatable builds
Depends on tasks
Are tasks environment-dependent?
Do system properties or environment variables have an effect?
What about number of cores?
Order of tests?
Gradle can’t help much with this - except for task ordering
Task ordering
• mustRunAfter/shouldRunAfter	

• No task dependencies
task	 integTest	 {

	 	 	 	 ...

}

!
task	 packageReports	 {

	 	 	 	 mustRunAfter	 integTest

	 	 	 	 ...

}

`packageReports` does not depend on `integTest`
But if both are executed, `packageReports` must come after `integTest`
shouldRunAfter is less strict - mostly to optimise feedback
Task ordering
• finalizedBy	

• Ensures execution of the finalizer task
task	 integTest	 {

	 	 	 	 finalizedBy	 packageReports

	 	 	 	 ...

}

!
task	 packageReports	 {

	 	 	 	 ...

}

`packageReports` does not depend on `integTest`
If `integTest` runs, then `packageReports` will run too, always after `integTest`
Dependency issues
Maven cache pollution (“works for me” syndrome)
Different remote repository configurations (where is the dependency coming from?)
Version resolution, eviction, and failed eviction

 Multiple JARs on the classpath and order counts
Gradle cache
• Origin checks	

• Artifact checksums	

• Concurrency safe	

• Avoid mavenLocal()!
Artifacts are stored with source repo URL (allows origin checks)
Artifact checksums protect against different binaries with same name & version
Doesn’t solve the version conflicts issue
Resolution strategy
configurations.all	 {

	 	 resolutionStrategy	 {

	 	 	 	 failOnVersionConflict()

!
	 	 	 	 force	 'asm:asm-all:3.3.1',

	 	 	 	 	 	 	 	 	 	 'commons-io:commons-io:1.4'

}
Override version to use
for specific dependencies
Fail the build if any
version conflicts
Defaults to ‘newest’ strategy
Fine-grained control over dependency versions
Automatic conflict resolution error-prone
Resolution strategy
configurations.all	 {

	 	 eachDependency	 {	 details	 ->

	 	 	 	 if	 (details.requested.name	 ==	 'groovy-all')	 {

	 	 	 	 	 	 details.useTarget(

	 	 	 	 	 	 	 	 	 	 group:	 details.requested.group,

	 	 	 	 	 	 	 	 	 	 name:	 'groovy',

	 	 	 	 	 	 	 	 	 	 version:	 details.requested.version)

	 	 	 	 }

	 	 }

}
Control modules with
different names, same classes
A painful problem to debug
Debugging dependencies
configurations.all.files.each	 {	 File	 f	 ->

	 	 println	 f.name

}
gradle	 dependencies
gradle	 dependencyInsight	 --dependency	 ...
`dependencies` gives you all dependencies in your project
`dependencyInsight` shows why/how a given dependency is included in the project
`<conf>.files` lists all the files and directories that will be on configuration’s classpath
Fast build execution
Depends on tasks
How long do individual tasks take?
No parallel execution of tasks currently
Can build decoupled projects in parallel
Incremental build
:lazybones-app:compileJava

:lazybones-app:compileGroovy	 UP-TO-DATE

:lazybones-app:processResources	 UP-TO-DATE

:lazybones-app:classes	 UP-TO-DATE

:lazybones-app:jar	 UP-TO-DATE

:lazybones-app:startScripts	 UP-TO-DATE

:lazybones-app:installApp	 UP-TO-DATE

!
BUILD	 SUCCESSFUL

!
Total	 time:	 2.178	 secs
UP-TO-DATE
Don’t execute tasks you don’t have to
Task inputs & outputs
class	 BintrayGenericUpload	 extends	 DefaultTask	 {

	 	 	 	 @InputFile	 File	 artifactFile

!
	 	 	 	 @Input	 	 	 	 	 String	 artifactUrlPath

!
	 	 	 	 @Optional

	 	 	 	 @Input	 	 	 	 	 String	 repositoryUrl

	 	 	 	 …⋯

}
Use of annotations makes task support incremental build
Inputs and outputs can be values, files, directories
Who needs to use the build?
What do they need to use it?
Will they use the whole build?
How long does it take?
Developers, QA, production types
How much software needs to be set up before someone can use the build? How long is your wiki page?
User categories
compile
test
package/install
publish
project
lead only
Lazybones
project
If the project is published, require credentials
Fail fast
Other users don’t require credentials
Interrogate task graph
gradle.taskGraph.whenReady	 {	 graph	 ->

	 	 	 	 if	 (graph.hasTask(":lazybones-app:uploadDist"))	 {

	 	 	 	 	 	 	 	 verifyProperty(project,	 'repo.url')

	 	 	 	 	 	 	 	 verifyProperty(project,	 'repo.username')

	 	 	 	 	 	 	 	 verifyProperty(project,	 'repo.apiKey')

!
	 	 	 	 	 	 	 	 uploadDist.repositoryUrl	 =	 project.'repo.url'

	 	 	 	 	 	 	 	 uploadDist.username	 =	 project.'repo.username'

	 	 	 	 	 	 	 	 uploadDist.apiKey	 =	 project.'repo.apiKey'

	 	 	 	 }

}
Only fail fast if `uploadTask` will be executed
Fail slow would require integration tests to run - adding minutes to deployment
Other uses
• Include class obfuscation conditionally	

• Limit visibility of tasks based on role	

• Update version based on ‘release’ task
Finally…
A build should have…
• Features	

• Error reporting	

• A model of the process
Just like any other piece of
software!
Summary
• End-to-end process is the build	

• 80-20 rule (80% standard 20% custom)	

• Automate everything == save time	

• Invest in build as if it’s part of your code
base	

• Building software requires a rich model

More Related Content

What's hot

Managing big test environment and running tests with Jenkins, Jenkins Job bui...
Managing big test environment and running tests with Jenkins, Jenkins Job bui...Managing big test environment and running tests with Jenkins, Jenkins Job bui...
Managing big test environment and running tests with Jenkins, Jenkins Job bui...
Timofey Turenko
 
Exploring the power of Gradle in android studio - Basics & Beyond
Exploring the power of Gradle in android studio - Basics & BeyondExploring the power of Gradle in android studio - Basics & Beyond
Exploring the power of Gradle in android studio - Basics & Beyond
Kaushal Dhruw
 
Introduction of React.js
Introduction of React.jsIntroduction of React.js
Introduction of React.js
Jyaasa Technologies
 
Cool JVM Tools to Help You Test
Cool JVM Tools to Help You TestCool JVM Tools to Help You Test
Cool JVM Tools to Help You Test
Schalk Cronjé
 
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012
kennethaliu
 
Continuous Integration using Cruise Control
Continuous Integration using Cruise ControlContinuous Integration using Cruise Control
Continuous Integration using Cruise Control
elliando dias
 
Gradle in 45min - JBCN2-16 version
Gradle in 45min - JBCN2-16 versionGradle in 45min - JBCN2-16 version
Gradle in 45min - JBCN2-16 version
Schalk Cronjé
 
Idiomatic gradle plugin writing
Idiomatic gradle plugin writingIdiomatic gradle plugin writing
Idiomatic gradle plugin writing
Schalk Cronjé
 
Workshop React.js
Workshop React.jsWorkshop React.js
Workshop React.js
Commit University
 
Dsc Charusat Learning React Part 1
Dsc Charusat Learning React Part 1 Dsc Charusat Learning React Part 1
Dsc Charusat Learning React Part 1
JainamMehta19
 
Introducing Playwright's New Test Runner
Introducing Playwright's New Test RunnerIntroducing Playwright's New Test Runner
Introducing Playwright's New Test Runner
Applitools
 
Cool Jvm Tools to Help you Test - Aylesbury Testers Version
Cool Jvm Tools to Help you Test - Aylesbury Testers VersionCool Jvm Tools to Help you Test - Aylesbury Testers Version
Cool Jvm Tools to Help you Test - Aylesbury Testers Version
Schalk Cronjé
 
The Gist of React Native
The Gist of React NativeThe Gist of React Native
The Gist of React Native
Darren Cruse
 
Gradle - Build system evolved
Gradle - Build system evolvedGradle - Build system evolved
Gradle - Build system evolved
Bhagwat Kumar
 
Building an Extensible, Resumable DSL on Top of Apache Groovy
Building an Extensible, Resumable DSL on Top of Apache GroovyBuilding an Extensible, Resumable DSL on Top of Apache Groovy
Building an Extensible, Resumable DSL on Top of Apache Groovy
jgcloudbees
 
Gradle: The Build System you have been waiting for!
Gradle: The Build System you have been waiting for!Gradle: The Build System you have been waiting for!
Gradle: The Build System you have been waiting for!
Corneil du Plessis
 
Gradle by Example
Gradle by ExampleGradle by Example
Gradle by Example
Eric Wendelin
 
Automated Testing in Angular Slides
Automated Testing in Angular SlidesAutomated Testing in Angular Slides
Automated Testing in Angular Slides
Jim Lynch
 
Lunch and learn: Cucumber and Capybara
Lunch and learn: Cucumber and CapybaraLunch and learn: Cucumber and Capybara
Lunch and learn: Cucumber and Capybara
Marc Seeger
 
Angular 2 Migration - JHipster Meetup 6
Angular 2 Migration - JHipster Meetup 6Angular 2 Migration - JHipster Meetup 6
Angular 2 Migration - JHipster Meetup 6
William Marques
 

What's hot (20)

Managing big test environment and running tests with Jenkins, Jenkins Job bui...
Managing big test environment and running tests with Jenkins, Jenkins Job bui...Managing big test environment and running tests with Jenkins, Jenkins Job bui...
Managing big test environment and running tests with Jenkins, Jenkins Job bui...
 
Exploring the power of Gradle in android studio - Basics & Beyond
Exploring the power of Gradle in android studio - Basics & BeyondExploring the power of Gradle in android studio - Basics & Beyond
Exploring the power of Gradle in android studio - Basics & Beyond
 
Introduction of React.js
Introduction of React.jsIntroduction of React.js
Introduction of React.js
 
Cool JVM Tools to Help You Test
Cool JVM Tools to Help You TestCool JVM Tools to Help You Test
Cool JVM Tools to Help You Test
 
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012
Modularizing your Grails Application with Private Plugins - SpringOne 2GX 2012
 
Continuous Integration using Cruise Control
Continuous Integration using Cruise ControlContinuous Integration using Cruise Control
Continuous Integration using Cruise Control
 
Gradle in 45min - JBCN2-16 version
Gradle in 45min - JBCN2-16 versionGradle in 45min - JBCN2-16 version
Gradle in 45min - JBCN2-16 version
 
Idiomatic gradle plugin writing
Idiomatic gradle plugin writingIdiomatic gradle plugin writing
Idiomatic gradle plugin writing
 
Workshop React.js
Workshop React.jsWorkshop React.js
Workshop React.js
 
Dsc Charusat Learning React Part 1
Dsc Charusat Learning React Part 1 Dsc Charusat Learning React Part 1
Dsc Charusat Learning React Part 1
 
Introducing Playwright's New Test Runner
Introducing Playwright's New Test RunnerIntroducing Playwright's New Test Runner
Introducing Playwright's New Test Runner
 
Cool Jvm Tools to Help you Test - Aylesbury Testers Version
Cool Jvm Tools to Help you Test - Aylesbury Testers VersionCool Jvm Tools to Help you Test - Aylesbury Testers Version
Cool Jvm Tools to Help you Test - Aylesbury Testers Version
 
The Gist of React Native
The Gist of React NativeThe Gist of React Native
The Gist of React Native
 
Gradle - Build system evolved
Gradle - Build system evolvedGradle - Build system evolved
Gradle - Build system evolved
 
Building an Extensible, Resumable DSL on Top of Apache Groovy
Building an Extensible, Resumable DSL on Top of Apache GroovyBuilding an Extensible, Resumable DSL on Top of Apache Groovy
Building an Extensible, Resumable DSL on Top of Apache Groovy
 
Gradle: The Build System you have been waiting for!
Gradle: The Build System you have been waiting for!Gradle: The Build System you have been waiting for!
Gradle: The Build System you have been waiting for!
 
Gradle by Example
Gradle by ExampleGradle by Example
Gradle by Example
 
Automated Testing in Angular Slides
Automated Testing in Angular SlidesAutomated Testing in Angular Slides
Automated Testing in Angular Slides
 
Lunch and learn: Cucumber and Capybara
Lunch and learn: Cucumber and CapybaraLunch and learn: Cucumber and Capybara
Lunch and learn: Cucumber and Capybara
 
Angular 2 Migration - JHipster Meetup 6
Angular 2 Migration - JHipster Meetup 6Angular 2 Migration - JHipster Meetup 6
Angular 2 Migration - JHipster Meetup 6
 

Similar to Why your build matters

Why Gradle?
Why Gradle?Why Gradle?
Why Gradle?
Peter Ledbrook
 
Stopping the Rot - Putting Legacy C++ Under Test
Stopping the Rot - Putting Legacy C++ Under TestStopping the Rot - Putting Legacy C++ Under Test
Stopping the Rot - Putting Legacy C++ Under Test
Seb Rose
 
Having Fun with Play
Having Fun with PlayHaving Fun with Play
Having Fun with Play
Clinton Dreisbach
 
Introduction to node.js GDD
Introduction to node.js GDDIntroduction to node.js GDD
Introduction to node.js GDD
Sudar Muthu
 
Writing native Linux desktop apps with JavaScript
Writing native Linux desktop apps with JavaScriptWriting native Linux desktop apps with JavaScript
Writing native Linux desktop apps with JavaScript
Igalia
 
C++ Windows Forms L01 - Intro
C++ Windows Forms L01 - IntroC++ Windows Forms L01 - Intro
C++ Windows Forms L01 - Intro
Mohammad Shaker
 
Pain Driven Development by Alexandr Sugak
Pain Driven Development by Alexandr SugakPain Driven Development by Alexandr Sugak
Pain Driven Development by Alexandr Sugak
Sigma Software
 
Dependent things dependency management for apple sw - slideshare
Dependent things   dependency management for apple sw - slideshareDependent things   dependency management for apple sw - slideshare
Dependent things dependency management for apple sw - slideshare
Cavelle Benjamin
 
DevOPS training - Day 2/2
DevOPS training - Day 2/2DevOPS training - Day 2/2
DevOPS training - Day 2/2
Vincent Mercier
 
Turku loves-storybook-styleguidist-styled-components
Turku loves-storybook-styleguidist-styled-componentsTurku loves-storybook-styleguidist-styled-components
Turku loves-storybook-styleguidist-styled-components
James Stone
 
The "Holy Grail" of Dev/Ops
The "Holy Grail" of Dev/OpsThe "Holy Grail" of Dev/Ops
The "Holy Grail" of Dev/Ops
Erik Osterman
 
Dev ops meetup
Dev ops meetupDev ops meetup
Dev ops meetup
Bigdata Meetup Kochi
 
Build automation best practices
Build automation best practicesBuild automation best practices
Build automation best practices
Code Mastery
 
Php on the Web and Desktop
Php on the Web and DesktopPhp on the Web and Desktop
Php on the Web and Desktop
Elizabeth Smith
 
OpenDaylight Developer Experience 2.0
 OpenDaylight Developer Experience 2.0 OpenDaylight Developer Experience 2.0
OpenDaylight Developer Experience 2.0
Michael Vorburger
 
(Declarative) Jenkins Pipelines
(Declarative) Jenkins Pipelines(Declarative) Jenkins Pipelines
(Declarative) Jenkins Pipelines
Steffen Gebert
 
10 Ways To Improve Your Code
10 Ways To Improve Your Code10 Ways To Improve Your Code
10 Ways To Improve Your Code
ConSanFrancisco123
 
Demystifying Maven
Demystifying MavenDemystifying Maven
Demystifying Maven
Mike Desjardins
 
Usability in the GeoWeb
Usability in the GeoWebUsability in the GeoWeb
Usability in the GeoWeb
Dave Bouwman
 
North east user group tour
North east user group tourNorth east user group tour
North east user group tour
10n Software, LLC
 

Similar to Why your build matters (20)

Why Gradle?
Why Gradle?Why Gradle?
Why Gradle?
 
Stopping the Rot - Putting Legacy C++ Under Test
Stopping the Rot - Putting Legacy C++ Under TestStopping the Rot - Putting Legacy C++ Under Test
Stopping the Rot - Putting Legacy C++ Under Test
 
Having Fun with Play
Having Fun with PlayHaving Fun with Play
Having Fun with Play
 
Introduction to node.js GDD
Introduction to node.js GDDIntroduction to node.js GDD
Introduction to node.js GDD
 
Writing native Linux desktop apps with JavaScript
Writing native Linux desktop apps with JavaScriptWriting native Linux desktop apps with JavaScript
Writing native Linux desktop apps with JavaScript
 
C++ Windows Forms L01 - Intro
C++ Windows Forms L01 - IntroC++ Windows Forms L01 - Intro
C++ Windows Forms L01 - Intro
 
Pain Driven Development by Alexandr Sugak
Pain Driven Development by Alexandr SugakPain Driven Development by Alexandr Sugak
Pain Driven Development by Alexandr Sugak
 
Dependent things dependency management for apple sw - slideshare
Dependent things   dependency management for apple sw - slideshareDependent things   dependency management for apple sw - slideshare
Dependent things dependency management for apple sw - slideshare
 
DevOPS training - Day 2/2
DevOPS training - Day 2/2DevOPS training - Day 2/2
DevOPS training - Day 2/2
 
Turku loves-storybook-styleguidist-styled-components
Turku loves-storybook-styleguidist-styled-componentsTurku loves-storybook-styleguidist-styled-components
Turku loves-storybook-styleguidist-styled-components
 
The "Holy Grail" of Dev/Ops
The "Holy Grail" of Dev/OpsThe "Holy Grail" of Dev/Ops
The "Holy Grail" of Dev/Ops
 
Dev ops meetup
Dev ops meetupDev ops meetup
Dev ops meetup
 
Build automation best practices
Build automation best practicesBuild automation best practices
Build automation best practices
 
Php on the Web and Desktop
Php on the Web and DesktopPhp on the Web and Desktop
Php on the Web and Desktop
 
OpenDaylight Developer Experience 2.0
 OpenDaylight Developer Experience 2.0 OpenDaylight Developer Experience 2.0
OpenDaylight Developer Experience 2.0
 
(Declarative) Jenkins Pipelines
(Declarative) Jenkins Pipelines(Declarative) Jenkins Pipelines
(Declarative) Jenkins Pipelines
 
10 Ways To Improve Your Code
10 Ways To Improve Your Code10 Ways To Improve Your Code
10 Ways To Improve Your Code
 
Demystifying Maven
Demystifying MavenDemystifying Maven
Demystifying Maven
 
Usability in the GeoWeb
Usability in the GeoWebUsability in the GeoWeb
Usability in the GeoWeb
 
North east user group tour
North east user group tourNorth east user group tour
North east user group tour
 

More from Peter Ledbrook

Groovy for Java Developers
Groovy for Java DevelopersGroovy for Java Developers
Groovy for Java Developers
Peter Ledbrook
 
Application Architectures in Grails
Application Architectures in GrailsApplication Architectures in Grails
Application Architectures in Grails
Peter Ledbrook
 
Open source-and-you-gr8conf-us-2013
Open source-and-you-gr8conf-us-2013Open source-and-you-gr8conf-us-2013
Open source-and-you-gr8conf-us-2013
Peter Ledbrook
 
Groovy & Grails for Spring/Java developers
Groovy & Grails for Spring/Java developersGroovy & Grails for Spring/Java developers
Groovy & Grails for Spring/Java developers
Peter Ledbrook
 
Grails & the World of Tomorrow
Grails & the World of TomorrowGrails & the World of Tomorrow
Grails & the World of Tomorrow
Peter Ledbrook
 
Migrating to Cloud Foundry
Migrating to Cloud FoundryMigrating to Cloud Foundry
Migrating to Cloud Foundry
Peter Ledbrook
 
Grails 2.0 Update
Grails 2.0 UpdateGrails 2.0 Update
Grails 2.0 Update
Peter Ledbrook
 
Grails and the World of Tomorrow
Grails and the World of TomorrowGrails and the World of Tomorrow
Grails and the World of Tomorrow
Peter Ledbrook
 
Cloud Foundry for Java devs
Cloud Foundry for Java devsCloud Foundry for Java devs
Cloud Foundry for Java devs
Peter Ledbrook
 

More from Peter Ledbrook (9)

Groovy for Java Developers
Groovy for Java DevelopersGroovy for Java Developers
Groovy for Java Developers
 
Application Architectures in Grails
Application Architectures in GrailsApplication Architectures in Grails
Application Architectures in Grails
 
Open source-and-you-gr8conf-us-2013
Open source-and-you-gr8conf-us-2013Open source-and-you-gr8conf-us-2013
Open source-and-you-gr8conf-us-2013
 
Groovy & Grails for Spring/Java developers
Groovy & Grails for Spring/Java developersGroovy & Grails for Spring/Java developers
Groovy & Grails for Spring/Java developers
 
Grails & the World of Tomorrow
Grails & the World of TomorrowGrails & the World of Tomorrow
Grails & the World of Tomorrow
 
Migrating to Cloud Foundry
Migrating to Cloud FoundryMigrating to Cloud Foundry
Migrating to Cloud Foundry
 
Grails 2.0 Update
Grails 2.0 UpdateGrails 2.0 Update
Grails 2.0 Update
 
Grails and the World of Tomorrow
Grails and the World of TomorrowGrails and the World of Tomorrow
Grails and the World of Tomorrow
 
Cloud Foundry for Java devs
Cloud Foundry for Java devsCloud Foundry for Java devs
Cloud Foundry for Java devs
 

Recently uploaded

"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
Fwdays
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Antonios Katsarakis
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
operationspcvita
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
LizaNolte
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
Ivo Velitchkov
 
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin..."$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
Fwdays
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
Fwdays
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Pitangent Analytics & Technology Solutions Pvt. Ltd
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
Safe Software
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
Enterprise Knowledge
 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
DanBrown980551
 
Christine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptxChristine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptx
christinelarrosa
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Neo4j
 

Recently uploaded (20)

"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
 
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin..."$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
 
Christine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptxChristine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptx
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
 

Why your build matters

  • 1. Why your build matters Peter Ledbrook e: peter@cacoethes.co.uk w: http://www.cacoethes.co.uk t: @pledbrook
  • 2. When did it all begin? • Programmable machines have a long history • Mechanised instruments • Looms
  • 3. Punched cards First used for looms in 18th Century France - Jacquard Loom 1801 IBM introduced punched card format 1928 - 80 columns! Text mode column width for DOS, Unix etc. Imagine creating by hand Keypunch machines (a little like typewriters) for creating them Errors mean throwing away the card and doing it again Fortran and Cobol started with punched cards
  • 4. Fixing card decks What if your cards are dropped on the floor? This is an IBM 082 Sorter Automation of a very error-prone manual process
  • 5. Magnetism saves the day! Finally, R/W data and code. The dawn of text editors.
  • 7. Make *.c: *.o cc … ! myapp: app.o mylib.o … ld … + manual custom dependencies Focus mainly on compilation and linking Make has powerful model based on file dependencies Maintenance was a lot of work
  • 8. Make *.c: *.o cc … ! myapp: app.o mylib.o … ld … What’s this? + manual custom dependencies Tabs were the bane of Make file writers
  • 9. Java • Compiler handles .java to .class dependencies • JAR as ‘module’ unit • Several standard deployment types - Applet - WAR - Standalone app
  • 10. A standard build Compile Test Package Publish? A simplification from the days of C
  • 11. Java build evolved Compile Test Package Publish? Dev setup Run QA SCM Tags Deployment DocsGenerators JS + CSS Builds are incorporating more and more steps
  • 12. What will builds look like in 5 years time?
  • 13. What is common across all these systems? To understand that, let’s look at a non-software process
  • 14. Making tea Boil kettle Pour water into pot Empty pot Put tea in pot Brew for n seconds Fill kettle Clear that it’s a series of steps (or tasks) Some tasks require others to complete first Others can run in parallel It’s an acyclic graph of tasks!
  • 15. Making tea (my way) Boil water on hob Add tea to pan Brew for n seconds Measure water into pan A different way to make tea Less standard, more appropriate for me
  • 16. A standard process does not mean an exclusive one
  • 17. Many things can go wrong Boil kettle Pour water into pot Empty pot Put tea in pot Brew for n seconds Fill kettle Forget to empty pot Brew for too long Put too little or too much water in
  • 18. Automate to eliminate human error People are fallible Muscle memory can help e.g. tying shoe laces Not usually applicable in software development
  • 19. What if your job depends on perfect tea every time?
  • 20. Inputs & parameters Boil kettle Pour water into pot Empty pot Put tea in pot Brew for n seconds Fill kettle What type of tea? How much? How long? How much water? Build inputs: * water quality * tea bags/loose leaf * tea type Build parameters: * quantity of tea * quantity of water * brew time
  • 21. Same inputs should result in same outputs In other words, repeatable builds Environment should not be a factor - boiling kettle at top of mountain reduces temperature of water Consider “works for me” with local Maven cache
  • 22. How many cups of tea do you need to make a day? Speed and efficiency are useful
  • 23. Task avoidance Boil kettle Pour water into pot Empty pot Put tea in pot Brew for n seconds Fill kettle What if pot already empty? What if hot water already available? Doing all steps slows you down if some are unnecessary Output of “boil kettle” is hot water If hot water already exists, no need to boil
  • 24. Incremental build (up-to-date checks) saves time Often quicker to check whether a task is up to date rather than run it regardless
  • 25. A build should be… Automated Repeatable As fast as possible
  • 28. Custom build publishPdf publishGuide apiDocs fetchGrailsSource grails-doc project expensive, so optional A build to generate the Grails user guide Automation of all steps No standard tasks Task graph is a suitable model
  • 29. Custom tasks in Gradle task publishGuide << { // Generate HTML from .gdoc } buildSrc/src/groovy/pkg/PublishGuideTask.groovy pkg.PublishGuideTask in a JAR
  • 30. Optional task apiDocs.onlyIf { !System.getProperty("disable.groovydocs") }
  • 31. Add custom tasks to standard Groovy/Java projects Even “standard” builds often have custom steps, e.g. integration tests deployment documentation Stop using scripts or other tools separate from the build!
  • 33. Depends on tasks Are tasks environment-dependent? Do system properties or environment variables have an effect? What about number of cores? Order of tests? Gradle can’t help much with this - except for task ordering
  • 34. Task ordering • mustRunAfter/shouldRunAfter • No task dependencies task integTest { ... } ! task packageReports { mustRunAfter integTest ... } `packageReports` does not depend on `integTest` But if both are executed, `packageReports` must come after `integTest` shouldRunAfter is less strict - mostly to optimise feedback
  • 35. Task ordering • finalizedBy • Ensures execution of the finalizer task task integTest { finalizedBy packageReports ... } ! task packageReports { ... } `packageReports` does not depend on `integTest` If `integTest` runs, then `packageReports` will run too, always after `integTest`
  • 36. Dependency issues Maven cache pollution (“works for me” syndrome) Different remote repository configurations (where is the dependency coming from?) Version resolution, eviction, and failed eviction Multiple JARs on the classpath and order counts
  • 37. Gradle cache • Origin checks • Artifact checksums • Concurrency safe • Avoid mavenLocal()! Artifacts are stored with source repo URL (allows origin checks) Artifact checksums protect against different binaries with same name & version Doesn’t solve the version conflicts issue
  • 38. Resolution strategy configurations.all { resolutionStrategy { failOnVersionConflict() ! force 'asm:asm-all:3.3.1', 'commons-io:commons-io:1.4' } Override version to use for specific dependencies Fail the build if any version conflicts Defaults to ‘newest’ strategy Fine-grained control over dependency versions Automatic conflict resolution error-prone
  • 39. Resolution strategy configurations.all { eachDependency { details -> if (details.requested.name == 'groovy-all') { details.useTarget( group: details.requested.group, name: 'groovy', version: details.requested.version) } } } Control modules with different names, same classes A painful problem to debug
  • 40. Debugging dependencies configurations.all.files.each { File f -> println f.name } gradle dependencies gradle dependencyInsight --dependency ... `dependencies` gives you all dependencies in your project `dependencyInsight` shows why/how a given dependency is included in the project `<conf>.files` lists all the files and directories that will be on configuration’s classpath
  • 42. Depends on tasks How long do individual tasks take? No parallel execution of tasks currently Can build decoupled projects in parallel
  • 43. Incremental build :lazybones-app:compileJava :lazybones-app:compileGroovy UP-TO-DATE :lazybones-app:processResources UP-TO-DATE :lazybones-app:classes UP-TO-DATE :lazybones-app:jar UP-TO-DATE :lazybones-app:startScripts UP-TO-DATE :lazybones-app:installApp UP-TO-DATE ! BUILD SUCCESSFUL ! Total time: 2.178 secs UP-TO-DATE Don’t execute tasks you don’t have to
  • 44. Task inputs & outputs class BintrayGenericUpload extends DefaultTask { @InputFile File artifactFile ! @Input String artifactUrlPath ! @Optional @Input String repositoryUrl …⋯ } Use of annotations makes task support incremental build Inputs and outputs can be values, files, directories
  • 45. Who needs to use the build? What do they need to use it? Will they use the whole build? How long does it take? Developers, QA, production types How much software needs to be set up before someone can use the build? How long is your wiki page?
  • 46. User categories compile test package/install publish project lead only Lazybones project If the project is published, require credentials Fail fast Other users don’t require credentials
  • 47. Interrogate task graph gradle.taskGraph.whenReady { graph -> if (graph.hasTask(":lazybones-app:uploadDist")) { verifyProperty(project, 'repo.url') verifyProperty(project, 'repo.username') verifyProperty(project, 'repo.apiKey') ! uploadDist.repositoryUrl = project.'repo.url' uploadDist.username = project.'repo.username' uploadDist.apiKey = project.'repo.apiKey' } } Only fail fast if `uploadTask` will be executed Fail slow would require integration tests to run - adding minutes to deployment
  • 48. Other uses • Include class obfuscation conditionally • Limit visibility of tasks based on role • Update version based on ‘release’ task
  • 50. A build should have… • Features • Error reporting • A model of the process
  • 51. Just like any other piece of software!
  • 52. Summary • End-to-end process is the build • 80-20 rule (80% standard 20% custom) • Automate everything == save time • Invest in build as if it’s part of your code base • Building software requires a rich model