Zen and-the-art-of-build-script-maintenance-skillsmatter


Published on

Build scripts are an essential art in any software project. And yet they are so often fragile, brittle and unportable things, hard to understand and harder to maintain. In this talk, we cover what constitutes a good build script, and look at a few of the essential rules in writing one.

John Smart, creator of the Java Power Tools Bootcamp, will look at general techniques that are applicable to any build scripting technology, as well as some technology-specific tips for Ant and Maven.

Published in: Technology, Sports
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Zen and-the-art-of-build-script-maintenance-skillsmatter

  1. 1. Who is this guy, anyway? John Ferguson Smart Consultant, Trainer, Mentor, Author, Speaker Specialities: Java, Groovy/Grails Software Development Lifecycle Agile stuff (CI, TDD, BDD,...) Open Source
  2. 2. Agenda What are we discussing today? What makes a good build script? Smelly build scripts Choosing your tools Maven tips Ant tips
  3. 3. Introduction Quality build scripts - why bother Maintenance costs Learning curve Turn-over Portability Automation
  4. 4. Build quality - quality builds What makes a good build script? Gold Standard Portable Reproducible Standard Maintainable
  5. 5. Build quality - quality builds Gold Standard Reference build process Reference binaries
  6. 6. Build quality - quality builds Portable Runs anywhere Runs on any OS No local dependencies Environment-specific configurations Specially-installed software or databases ...
  7. 7. Build quality - quality builds Reproducible “Play it again, Sam”
  8. 8. Build quality - quality builds Standard Knowing what to expect
  9. 9. Build quality - quality builds Maintainable Think of the next dude
  10. 10. Smelly builds So what makes a poor build script? 1) The hard coded build 2) The OS-specific build 3) The IDE-only build 4) The Magic Machine build 5) The Oral Tradition build 6) The Nested build 7) The Messy Dependencies build 8) The DYI build 9) The untrustworthy build 10) The slow build
  11. 11. Paths .. 1/ URLs Smelly builds 9. c- gi lo 1 eb 00 Passwords /w :7 om ea .c /b me C: ac r. ve er ts es .> /t ".. :/ ger "ti tp rd= The hard coded build ht swo pas tt" sco e=" nam ser n u <sv /> M E }" S_HO JBOS env. = "$ { alue s" v Environment jbos dir. me=" variables y na pert <pro
  12. 12. Smelly builds The hard coded build <target name="checkstyle"> Hard-coded directories <delete dir="./reports" quiet="true" /> <mkdir dir="./reports" /> <checkstyle config="./docs/sun_checks.xml"> <formatter type="xml" tofile="./reports/checkstyle.xml"/> <fileset dir="./src" includes="**/*.java"/> </checkstyle> <style in="./reports/checkstyle.xml" out="./reports/checkstyle.html" style="checkstyle.xsl"/> </target>
  13. 13. Smelly builds The hard coded build <target name="checkstyle"> Project-relative directory <property name=”reports.checkstyle.dir” <delete dir="./reports" quiet="true" /> value=”${basedir}/reports”/> <mkdir dir="./reports" /> DRY <checkstyle config="./docs/sun_checks.xml"> <target name="checkstyle"> <formatter type="xml" tofile="./reports/checkstyle.xml"/> <delete dir="${reports.checkstyle.dir}" quiet="true" /> <mkdir<fileset dir="./src" includes="**/*.java"/> dir="${reports.checkstyle.dir}" /> </checkstyle> <checkstyle config="./docs/sun_checks.xml"> <formatter type="xml" <style in="./reports/checkstyle.xml" tofile="${reports.checkstyle.dir}/checkstyle.xml"/> <fileset out="./reports/checkstyle.html" dir="./src" includes="**/*.java"/> style="checkstyle.xsl"/> </checkstyle> </target> <style in="${reports.checkstyle.dir}/checkstyle.xml" out="${reports.checkstyle.dir}/checkstyle.html" style="checkstyle.xsl"/> </target>
  14. 14. Smelly builds The hard coded build <target name="war" > <war destfile="c:/tomcat/jakarta-tomcat-5.0.19/webapps/app.war" webxml="${src}/app.xml" basedir="${bin}" /> </target> Hard-coded directories
  15. 15. Smelly builds The hard coded build <property name="wardir" location="c:/tomcat/jakarta-tomcat-5.0.19/webapps"/> <target name="war" > <war destfile="${wardir}" webxml="${src}/app.xml" basedir="${bin}" /> </target> Still hard-coded
  16. 16. Smelly builds The hard coded build <svn username="scott" password="tiger"> <checkout url="http://subversion.acme.com/myapp/trunk" destPath="${subproject.dir}" /> </svn> Hard-coded username/password
  17. 17. Smelly builds The hard coded build <property environment="env"/> <property name="dir.jboss" value="${env.JBOSS_HOME}"/> Environment variable
  18. 18. Smelly builds The OS-specific build <exec command="grep "@" ${build.dir} | wc -l" outputproperty="token.count"/>
  19. 19. Smelly builds The OS-specific build ... CALL PAUSE.CMD ... build.cmd ... :: Check for a non-existent IP address :: Note: this causes a small extra delay! IF NOT DEFINED NonExist SET NonExist= PING %NonExist% -n 1 -w 100 2>NUL | FIND "TTL=" >NUL ... pause.cmd
  20. 20. Smelly builds The IDE-only build
  21. 21. Smelly builds The Magic Machine build Directories App servers Databases Configuration files Environment variables Installed software or tools
  22. 22. Smelly builds The Magic Machine build <proprerty weblogic.dir="/u01/app/bea/weblogic-9.1"/> Directories App servers Databases Configuration files Environment variables Installed software or tools
  23. 23. Smelly builds The Oral Tradition build
  24. 24. Smelly builds The Nested Build #! /bin/sh ANT_HOME=/u01/app/tools/ant-1.7.1 ... $ANT_HOME/ant $1 project/tools/ant.sh
  25. 25. Smelly builds The Nested Build <target name="build-subproject"> <svn username="scott" password="tiger"> <checkout url="http://subversion.acme.com/someproject/trunk" destPath="${subproject.dir}" /> </svn> <ant dir="${subproject.dir}" target="build-all" /> </target> build.xml
  26. 26. Smelly builds The Messy Dependencies build JAR files in the SCM Unversioned JAR files Unclear dependencies
  27. 27. Smelly builds The DYI build “Not invented here” DYI dependencies DYI deployments DYI Maven releases ...
  28. 28. Smelly builds The untrustworthy build <junit fork="yes" haltonfailure="false" dir="${basedir}"> <classpath refid="test.class.path" /> <classpath refid="project.class.path"/> <formatter type="plain" usefile="true" /> <formatter type="xml" usefile="true" /> <batchtest fork="yes" todir="${logs.junit.dir}"> <fileset dir="${test.unit.dir}"> <patternset refid="test.sources.pattern"/> </fileset> </batchtest> </junit>
  29. 29. Smelly builds The slow build
  30. 30. Choosing your tools Flexibility verses Convention What’s better: flexibility or standards? It depends what you’re doing...
  31. 31. Choosing your tools Encourage/enforce Standards and Conventions Support standards standards 3 Easy to read 2 Make up your own standards No standards Hard to read Ad-hoc scripting Standards and Conventions
  32. 32. Choosing your tools Flexibility and expressiveness 3 Do whatever you Easy to read want 2 Encourage/enforce standards Hard to read Makes you stick to conventions Easy to do whatever you want
  33. 33. Choosing your tools Flexibility verses Convention Build Scripting Rule 1 “A build script will tend to reflect the personality of it’s developer”
  34. 34. Choosing your tools Flexibility verses Convention Build Scripting Rule 2 “The more flexible a build script, the more likely it is to become unmaintainable”
  35. 35. Choosing your tools Flexibility verses Convention Flexibility is great for some jobs: Ad-hoc tasks Some deployment tasks “Out-of-the-box” stuff
  36. 36. Choosing your tools Flexibility verses Convention But too much flexibility is hard to maintain
  37. 37. Ant tips Better Ant scripts Consistent conventions Declare your dependencies Make it readable Tidy up your mess Avoid long scripts
  38. 38. Ant tips Be consistent Standardize target names Document your public targets
  39. 39. Ant tips Declare your dependencies Use an Enterprise Repository Manager Several tool choices: Maven Ant Tasks Ivy
  40. 40. Ant tips Using the Maven Ant Tasks Declare dependencies Deploy to a Maven Enterprise Repository <artifact:dependencies pathId="dependency.classpath"> <dependency groupId="junit" artifactId="junit" version="3.8.2" scope="test"/> <dependency groupId="javax.servlet" artifactId="servlet-api" version="2.4" scope="provided"/> </artifact:dependencies>
  41. 41. Ant tips Make it readable Write a build script like your source code... Avoid long targets Avoid long build scripts Use descriptive target names
  42. 42. Ant tips Tidy up your mess Always define a ‘clean’ target.
  43. 43. Ant tips Move to Maven ;-)
  44. 44. Maven tips Better Maven scripts Simple Portable Reproducible Clean Automated
  45. 45. Maven tips Keep it simple Use modules Use an organization-level POM
  46. 46. Maven tips Keep it portable No hard-coding Define sensible defaults for properties and profiles Avoid resource filtering for production code
  47. 47. Maven tips Keep it reproducible Avoid external snapshots Specify plugin versions Use consistent environments
  48. 48. Maven tips Consistent environments Enforcing a minimum Maven version <?xml version="1.0"?> <project...> <modelVersion>4.0.0</modelVersion> <groupId>com.ciwithhudson.gameoflife</groupId> <artifactId>gameoflife</artifactId> <version>0.0.1-SNAPSHOT</version> Minimum Maven version <name>gameoflife</name> <prerequisites> <maven>2.2.1</maven> </prerequisites>
  49. 49. Maven tips Consistent environments Use the same version of Maven Use a “standard” Maven installation across the organization Use a global settings.xml file Store a copy in SCM Enforce a minimum Maven version in your projects
  50. 50. Maven tips Enforcing consistency with the enforcer plugin Maven version JDK version Snapshots Plugin versions OS ...
  51. 51. Maven tips Enforce the Maven version <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-enforcer-plugin</artifactId> <version>1.0-beta-1</version> <executions> <execution> <id>enforce-maven-version</id> <goals> <goal>enforce</goal> </goals> Minimum Maven version <configuration> <rules> <requireMavenVersion> <version>2.2.1</version> </requireMavenVersion> </rules> </configuration> </execution> </executions> </plugin>
  52. 52. Maven tips Enforce the JDK version All developers should be using the same JDKs Incompatible bytecode Different XML parsers Different Maven behaviour
  53. 53. Maven tips Enforce the JDK version <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-enforcer-plugin</artifactId> <version>1.0-beta-1</version> <executions> <execution> <id>enforce-jdk-version</id> <goals> <goal>enforce</goal> </goals> Authorized JDK versions <configuration> <rules> <requireJavaVersion> <version>[1.5.0,1.6.0)</version> </requireJavaVersion> </rules> </configuration> </execution> </executions> </plugin>
  54. 54. Maven tips Specify your plugin versions Undeclared version numbers are bad Inconsistent builds across different machines Non-repeatable builds Plugin changes can break the build Don’t use SNAPSHOT plugins either
  55. 55. Maven tips Specify your plugin versions <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-enforcer-plugin</artifactId> <version>1.0-beta-1</version> <executions> <execution> <id>enforce-versions</id> <goals> <goal>enforce</goal> </goals> Plugin versions must be defined <configuration> <rules> <requirePluginVersions/> </rules> </configuration> </execution> </executions> </plugin>
  56. 56. Maven tips Keep it clean Keep tabs on your dependencies: What dependencies are you actually using? What dependencies do you really need?
  57. 57. Maven tips Dependency list What dependencies are you actually using? $ mvn dependency:list [INFO] Scanning for projects... mvn dependency:list [INFO] Searching repository for plugin with prefix: 'dependency'. [INFO] ------------------------------------------------------------------------ [INFO] Building babble-core [INFO] task-segment: [dependency:list] [INFO] ------------------------------------------------------------------------ [INFO] [dependency:list] [INFO] [INFO] The following files have been resolved: [INFO] antlr:antlr:jar:2.7.6:compile ... [INFO] commons-collections:commons-collections:jar:2.1.1:compile [INFO] commons-logging:commons-logging:jar:1.0.4:compile [INFO] dom4j:dom4j:jar:1.6.1:compile [INFO] javax.persistence:persistence-api:jar:1.0:compile [INFO] javax.transaction:jta:jar:1.0.1B:compile [INFO] junit:junit:jar:4.5:test [INFO] net.sf.ehcache:ehcache:jar:1.2:compile [INFO] org.hamcrest:hamcrest-all:jar:1.1:compile [INFO] org.hibernate:hibernate:jar:3.2.0.ga:compile [INFO] org.hibernate:hibernate-annotations:jar:3.2.0.ga:compile [INFO] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESSFUL [INFO] ------------------------------------------------------------------------
  58. 58. Maven tips Dependency tree Where do they come from? $ mvn dependency:tree [INFO] Scanning for projects... mvn dependency:tree [INFO] Searching repository for plugin with prefix: 'dependency'. [INFO] ------------------------------------------------------------------------ [INFO] Building babble-core [INFO] task-segment: [dependency:tree] [INFO] ------------------------------------------------------------------------ [INFO] [dependency:tree] [INFO] com.sonatype.training:babble-core:jar:1.0-SNAPSHOT [INFO] +- org.hibernate:hibernate:jar:3.2.0.ga:compile [INFO] | +- net.sf.ehcache:ehcache:jar:1.2:compile [INFO] | +- javax.transaction:jta:jar:1.0.1B:compile [INFO] | +- commons-logging:commons-logging:jar:1.0.4:compile [INFO] | +- asm:asm-attrs:jar:1.5.3:compile [INFO] | +- dom4j:dom4j:jar:1.6.1:compile [INFO] | +- antlr:antlr:jar:2.7.6:compile [INFO] | +- cglib:cglib:jar:2.1_3:compile [INFO] | +- asm:asm:jar:1.5.3:compile [INFO] | - commons-collections:commons-collections:jar:2.1.1:compile [INFO] +- org.hibernate:hibernate-annotations:jar:3.2.0.ga:compile [INFO] | - javax.persistence:persistence-api:jar:1.0:compile [INFO] +- junit:junit:jar:4.5:test [INFO] - org.hamcrest:hamcrest-all:jar:1.1:compile [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESSFUL [INFO] ------------------------------------------------------------------------
  59. 59. Maven tips Dependencies in Eclipse Eclipse has an equivalent screen
  60. 60. Maven tips Dependency analyse What dependencies do you really need? $ mvn dependency:analyze [INFO] Scanning for projects... mvn dependency:analyse [INFO] Searching repository for plugin with prefix: 'dependency'. [INFO] ------------------------------------------------------------------------ [INFO] Building babble-core [INFO] task-segment: [dependency:analyze] [INFO] ------------------------------------------------------------------------ [INFO] Preparing dependency:analyze ... Used but not declared [INFO] [dependency:analyze] [WARNING] Used undeclared dependencies found: [WARNING] javax.persistence:persistence-api:jar:1.0:compile [WARNING] Unused declared dependencies found: [WARNING] org.hibernate:hibernate-annotations:jar:3.2.0.ga:compile [WARNING] org.hibernate:hibernate:jar:3.2.0.ga:compile [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESSFUL Declared but not used [INFO] ------------------------------------------------------------------------
  61. 61. Maven tips Excluding dependencies What if you don’t want a dependency? <dependencies> <dependency> <groupId>org.springframework</groupId> <artifactId>spring</artifactId> Don’t include JMS <version>2.5.5</version> <exclusions> <exclusion> <groupId>javax.jms</groupId> <artifactId>jms<artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.apache.geronimo.specs</groupId> <artifactId>geronimo-jms_1.1_spec</artifact> <version>1.1</version> </dependency> <dependencies>
  62. 62. Maven tips Standardizing versions Use dependencyManagement for consistency <dependencyManagement> Parent pom <dependencies> <dependency> ! <groupId>mysql</groupId> ! <artifactId>mysql-connector-java</artifactId> ! <version>5.1.6</version> </dependency> <dependency> ! <groupId>postgres</groupId> ! <artifactId>postgres</artifactId> ! <version>7.3.2</version> <dependencies> Child pom </dependency> <dependency> </dependencies> !<groupId>mysql</groupId> </dependencyManagement> <artifactId>mysql-connector-java</artifactId> </dependency> </dependencies>
  63. 63. Maven tips Keep it automated Plan your release strategy Use a Repository Manager Automatic snapshot deployments Automated releases
  64. 64. Maven tips Maven best practices for CI builds Use batch mode (-B) Always check or snapshot updates (-U) Use a repository per project Print test failures to stdout (-Dsurefire.useFile=false)
  65. 65. Maven tips Know when to script it Groovy or Ant scripting is easy in Maven Call external scripts when appropriate
  66. 66. Maven tips Know when to script it It’s pretty easy in Maven 2... <project> <build> <plugins> <plugin> <groupId>org.codehaus.groovy.maven</groupId> <artifactId>gmaven-plugin</artifactId> <version>1.0-rc-5</version> <executions> <execution> <phase>compile</phase> <goals> <goal>execute</goal> </goals> <configuration> <source> println "Hi there I’m compiling ${project.name}" </source> </configuration> </execution> </executions> </plugin> </plugins> </build> ...
  67. 67. Maven tips Know when to script it It’s even easier in Maven 3... project { build { $execute(id: 'compilation-script', phase: 'compile') { println "Hi there I’m compiling ${project.name}" } $execute(id: 'validation-script', phase: 'validate') { println "Hi there I’m validating ${project.name}" } ... } }
  68. 68. Java Power Tools Bootcamp London - October 13-17 2010