Zen and-the-art-of-build-script-maintenance

2,610 views
2,477 views

Published on

Published in: Technology, Education
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,610
On SlideShare
0
From Embeds
0
Number of Embeds
175
Actions
Shares
0
Downloads
63
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

Zen and-the-art-of-build-script-maintenance

  1. 1. Tuesday, 22 June 2010
  2. 2. Tuesday, 22 June 2010
  3. 3. Agenda What are we discussing today? What makes a good build script? Smelly build scripts Choosing your tools Maven tips Ant tips Tuesday, 22 June 2010
  4. 4. Introduction Quality build scripts - why bother Maintenance costs Learning curve Turn-over Portability Automation Tuesday, 22 June 2010
  5. 5. Build quality - quality builds What makes a good build script? Gold Standard Portable Reproducible Standard Maintainable Tuesday, 22 June 2010
  6. 6. Build quality - quality builds Gold Standard Reference build process Reference binaries Tuesday, 22 June 2010
  7. 7. Build quality - quality builds Portable Runs anywhere Runs on any OS No local dependencies Environment-specific configurations Specially-installed software or databases ... Tuesday, 22 June 2010
  8. 8. Build quality - quality builds Reproducible “Play it again, Sam” Tuesday, 22 June 2010
  9. 9. Build quality - quality builds Standard Knowing what to expect Tuesday, 22 June 2010
  10. 10. Build quality - quality builds Maintainable Think of the next dude Tuesday, 22 June 2010
  11. 11. Smelly builds So what makes a poor build script? 1) The hard coded build 2) The OS-specific build 3) The IDE-only build 4) The Magic Machine build 5) The Oral Tradition build 6) The Nested build 7) The Messy Dependencies build 8) The DYI build 9) The untrustworthy build 10) The slow build Tuesday, 22 June 2010
  12. 12. Paths .. 1/ URLs Smelly builds 9. c- gi lo 1 eb 00 Passwords /w :7 om ea .c /b me C: ac r. ve er ts es .> /t ".. :/ ger "ti tp rd= The hard coded build ht swo pas tt" sco e=" nam ser n u <sv /> M E }" S_HO JBOS env. = "$ { alue s" v Environment jbos dir. me=" variables y na pert <pro Tuesday, 22 June 2010
  13. 13. Smelly builds The hard coded build <target name="checkstyle"> Hard-coded directories <delete dir="./reports" quiet="true" /> <mkdir dir="./reports" /> <checkstyle config="./docs/sun_checks.xml"> <formatter type="xml" tofile="./reports/checkstyle.xml"/> <fileset dir="./src" includes="**/*.java"/> </checkstyle> <style in="./reports/checkstyle.xml" out="./reports/checkstyle.html" style="checkstyle.xsl"/> </target> Tuesday, 22 June 2010
  14. 14. Smelly builds The hard coded build <target name="checkstyle"> Project-relative directory <property name=”reports.checkstyle.dir” <delete dir="./reports" quiet="true" /> value=”${basedir}/reports”/> <mkdir dir="./reports" /> DRY <checkstyle config="./docs/sun_checks.xml"> <target name="checkstyle"> <formatter type="xml" tofile="./reports/checkstyle.xml"/> <delete dir="${reports.checkstyle.dir}" quiet="true" /> <mkdir<fileset dir="./src" includes="**/*.java"/> dir="${reports.checkstyle.dir}" /> </checkstyle> <checkstyle config="./docs/sun_checks.xml"> <formatter type="xml" <style in="./reports/checkstyle.xml" tofile="${reports.checkstyle.dir}/checkstyle.xml"/> <fileset out="./reports/checkstyle.html" dir="./src" includes="**/*.java"/> style="checkstyle.xsl"/> </checkstyle> </target> <style in="${reports.checkstyle.dir}/checkstyle.xml" out="${reports.checkstyle.dir}/checkstyle.html" style="checkstyle.xsl"/> </target> Tuesday, 22 June 2010
  15. 15. Smelly builds The hard coded build <target name="war" > <war destfile="c:/tomcat/jakarta-tomcat-5.0.19/webapps/app.war" webxml="${src}/app.xml" basedir="${bin}" /> </target> Hard-coded directories Tuesday, 22 June 2010
  16. 16. Smelly builds The hard coded build <property name="wardir" location="c:/tomcat/jakarta-tomcat-5.0.19/webapps"/> <target name="war" > <war destfile="${wardir}" webxml="${src}/app.xml" basedir="${bin}" /> </target> Still hard-coded Tuesday, 22 June 2010
  17. 17. Smelly builds The hard coded build <svn username="scott" password="tiger"> <checkout url="http://subversion.acme.com/myapp/trunk" destPath="${subproject.dir}" /> </svn> Hard-coded username/password Tuesday, 22 June 2010
  18. 18. Smelly builds The hard coded build <property environment="env"/> <property name="dir.jboss" value="${env.JBOSS_HOME}"/> Environment variable Tuesday, 22 June 2010
  19. 19. Smelly builds The OS-specific build <exec command="grep "@" ${build.dir} | wc -l" outputproperty="token.count"/> Tuesday, 22 June 2010
  20. 20. Smelly builds The OS-specific build ... CALL PAUSE.CMD ... build.cmd ... :: Check for a non-existent IP address :: Note: this causes a small extra delay! IF NOT DEFINED NonExist SET NonExist=10.255.255.254 PING %NonExist% -n 1 -w 100 2>NUL | FIND "TTL=" >NUL ... pause.cmd Tuesday, 22 June 2010
  21. 21. Smelly builds The IDE-only build Tuesday, 22 June 2010
  22. 22. Smelly builds The Magic Machine build Directories App servers Databases Configuration files Environment variables Installed software or tools Tuesday, 22 June 2010
  23. 23. Smelly builds The Magic Machine build <proprerty weblogic.dir="/u01/app/bea/weblogic-9.1"/> Directories App servers Databases Configuration files Environment variables Installed software or tools Tuesday, 22 June 2010
  24. 24. Smelly builds The Oral Tradition build Tuesday, 22 June 2010
  25. 25. Smelly builds The Nested Build #! /bin/sh ANT_HOME=/u01/app/tools/ant-1.7.1 ... $ANT_HOME/ant $1 project/tools/ant.sh Tuesday, 22 June 2010
  26. 26. Smelly builds The Nested Build <target name="build-subproject"> <svn username="scott" password="tiger"> <checkout url="http://subversion.acme.com/someproject/trunk" destPath="${subproject.dir}" /> </svn> <ant dir="${subproject.dir}" target="build-all" /> </target> build.xml Tuesday, 22 June 2010
  27. 27. Smelly builds The Messy Dependencies build JAR files in the SCM Unversioned JAR files Unclear dependencies Tuesday, 22 June 2010
  28. 28. Smelly builds The DYI build “Not invented here” DYI dependencies DYI deployments DYI Maven releases ... Tuesday, 22 June 2010
  29. 29. Smelly builds The untrustworthy build <junit fork="yes" haltonfailure="false" dir="${basedir}"> <classpath refid="test.class.path" /> <classpath refid="project.class.path"/> <formatter type="plain" usefile="true" /> <formatter type="xml" usefile="true" /> <batchtest fork="yes" todir="${logs.junit.dir}"> <fileset dir="${test.unit.dir}"> <patternset refid="test.sources.pattern"/> </fileset> </batchtest> </junit> Tuesday, 22 June 2010
  30. 30. Smelly builds The slow build Tuesday, 22 June 2010
  31. 31. Choosing your tools Flexibility verses Convention What’s better: flexibility or standards? It depends what you’re doing... Tuesday, 22 June 2010
  32. 32. Choosing your tools Encourage/enforce Standards and Conventions Support standards standards 3 Easy to read 2 Make up your own standards No standards Hard to read Ad-hoc scripting Standards and Conventions Tuesday, 22 June 2010
  33. 33. Choosing your tools Flexibility and expressiveness 3 Do whatever you Easy to read want 2 Encourage/enforce standards Hard to read Makes you stick to conventions Easy to do whatever you want Tuesday, 22 June 2010
  34. 34. Choosing your tools Flexibility verses Convention Build Scripting Rule 1 “A build script will tend to reflect the personality of it’s developer” Tuesday, 22 June 2010
  35. 35. Choosing your tools Flexibility verses Convention Build Scripting Rule 2 “The more flexible a build script, the more likely it is to become unmaintainable” Tuesday, 22 June 2010
  36. 36. Choosing your tools Flexibility verses Convention Flexibility is great for some jobs: Ad-hoc tasks Some deployment tasks “Out-of-the-box” stuff Tuesday, 22 June 2010
  37. 37. Choosing your tools Flexibility verses Convention But convention pay off in lower maintenance costs Tuesday, 22 June 2010
  38. 38. Ant tips Better Ant scripts Consistent conventions Declare your dependencies Make it readable Tidy up your mess Avoid long scripts Tuesday, 22 June 2010
  39. 39. Ant tips Be consistent Standardize target names Document your public targets Tuesday, 22 June 2010
  40. 40. Ant tips Declare your dependencies Use an Enterprise Repository Manager Several tool choices: Maven Ant Tasks Ivy Tuesday, 22 June 2010
  41. 41. Ant tips Using the Maven Ant Tasks Declare dependencies Deploy to a Maven Enterprise Repository <artifact:dependencies pathId="dependency.classpath"> <dependency groupId="junit" artifactId="junit" version="3.8.2" scope="test"/> <dependency groupId="javax.servlet" artifactId="servlet-api" version="2.4" scope="provided"/> </artifact:dependencies> Tuesday, 22 June 2010
  42. 42. Ant tips Make it readable Write a build script like your source code... Avoid long targets Avoid long build scripts Use descriptive target names Tuesday, 22 June 2010
  43. 43. Ant tips Tidy up your mess Always define a ‘clean’ target. Tuesday, 22 June 2010
  44. 44. Ant tips Move to Maven ;-) Tuesday, 22 June 2010
  45. 45. Maven tips Better Maven scripts Simple Portable Reproducible Clean Automated Tuesday, 22 June 2010
  46. 46. Maven tips Keep it simple Use modules Use an organization-level POM Tuesday, 22 June 2010
  47. 47. Maven tips Keep it portable No hard-coding Define sensible defaults for properties and profiles Avoid resource filtering for production code Tuesday, 22 June 2010
  48. 48. Maven tips Keep it reproducible Avoid external snapshots Specify plugin versions Use consistent environments Tuesday, 22 June 2010
  49. 49. Maven tips Consistent environments Enforcing a minimum Maven version <?xml version="1.0"?> <project...> <modelVersion>4.0.0</modelVersion> <groupId>com.ciwithhudson.gameoflife</groupId> <artifactId>gameoflife</artifactId> <version>0.0.1-SNAPSHOT</version> Minimum Maven version <name>gameoflife</name> <prerequisites> <maven>2.2.1</maven> </prerequisites> Tuesday, 22 June 2010
  50. 50. Maven tips Consistent environments Use the same version of Maven Use a “standard” Maven installation across the organization Use a global settings.xml file Store a copy in SCM Enforce a minimum Maven version in your projects Tuesday, 22 June 2010
  51. 51. Maven tips Enforcing consistency with the enforcer plugin Maven version JDK version Snapshots Plugin versions OS ... Tuesday, 22 June 2010
  52. 52. Maven tips Enforce the Maven version <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-enforcer-plugin</artifactId> <version>1.0-beta-1</version> <executions> <execution> <id>enforce-maven-version</id> <goals> <goal>enforce</goal> </goals> Minimum Maven version <configuration> <rules> <requireMavenVersion> <version>2.2.1</version> </requireMavenVersion> </rules> </configuration> </execution> </executions> </plugin> Tuesday, 22 June 2010
  53. 53. Maven tips Enforce the JDK version All developers should be using the same JDKs Incompatible bytecode Different XML parsers Different Maven behaviour Tuesday, 22 June 2010
  54. 54. Maven tips Enforce the JDK version <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-enforcer-plugin</artifactId> <version>1.0-beta-1</version> <executions> <execution> <id>enforce-jdk-version</id> <goals> <goal>enforce</goal> </goals> Authorized JDK versions <configuration> <rules> <requireJavaVersion> <version>[1.5.0,1.6.0)</version> </requireJavaVersion> </rules> </configuration> </execution> </executions> </plugin> Tuesday, 22 June 2010
  55. 55. Maven tips Specify your plugin versions Undeclared version numbers are bad Inconsistent builds across different machines Non-repeatable builds Plugin changes can break the build Don’t use SNAPSHOT plugins either Tuesday, 22 June 2010
  56. 56. Maven tips Specify your plugin versions <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-enforcer-plugin</artifactId> <version>1.0-beta-1</version> <executions> <execution> <id>enforce-versions</id> <goals> <goal>enforce</goal> </goals> Plugin versions must be defined <configuration> <rules> <requirePluginVersions/> </rules> </configuration> </execution> </executions> </plugin> Tuesday, 22 June 2010
  57. 57. Maven tips Keep it clean Keep tabs on your dependencies: What dependencies are you actually using? What dependencies do you really need? Tuesday, 22 June 2010
  58. 58. Maven tips Dependency list What dependencies are you actually using? $ mvn dependency:list [INFO] Scanning for projects... mvn dependency:list [INFO] Searching repository for plugin with prefix: 'dependency'. [INFO] ------------------------------------------------------------------------ [INFO] Building babble-core [INFO] task-segment: [dependency:list] [INFO] ------------------------------------------------------------------------ [INFO] [dependency:list] [INFO] [INFO] The following files have been resolved: [INFO] antlr:antlr:jar:2.7.6:compile ... [INFO] commons-collections:commons-collections:jar:2.1.1:compile [INFO] commons-logging:commons-logging:jar:1.0.4:compile [INFO] dom4j:dom4j:jar:1.6.1:compile [INFO] javax.persistence:persistence-api:jar:1.0:compile [INFO] javax.transaction:jta:jar:1.0.1B:compile [INFO] junit:junit:jar:4.5:test [INFO] net.sf.ehcache:ehcache:jar:1.2:compile [INFO] org.hamcrest:hamcrest-all:jar:1.1:compile [INFO] org.hibernate:hibernate:jar:3.2.0.ga:compile [INFO] org.hibernate:hibernate-annotations:jar:3.2.0.ga:compile [INFO] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESSFUL [INFO] ------------------------------------------------------------------------ Tuesday, 22 June 2010
  59. 59. Maven tips Dependency tree Where do they come from? $ mvn dependency:tree [INFO] Scanning for projects... mvn dependency:tree [INFO] Searching repository for plugin with prefix: 'dependency'. [INFO] ------------------------------------------------------------------------ [INFO] Building babble-core [INFO] task-segment: [dependency:tree] [INFO] ------------------------------------------------------------------------ [INFO] [dependency:tree] [INFO] com.sonatype.training:babble-core:jar:1.0-SNAPSHOT [INFO] +- org.hibernate:hibernate:jar:3.2.0.ga:compile [INFO] | +- net.sf.ehcache:ehcache:jar:1.2:compile [INFO] | +- javax.transaction:jta:jar:1.0.1B:compile [INFO] | +- commons-logging:commons-logging:jar:1.0.4:compile [INFO] | +- asm:asm-attrs:jar:1.5.3:compile [INFO] | +- dom4j:dom4j:jar:1.6.1:compile [INFO] | +- antlr:antlr:jar:2.7.6:compile [INFO] | +- cglib:cglib:jar:2.1_3:compile [INFO] | +- asm:asm:jar:1.5.3:compile [INFO] | - commons-collections:commons-collections:jar:2.1.1:compile [INFO] +- org.hibernate:hibernate-annotations:jar:3.2.0.ga:compile [INFO] | - javax.persistence:persistence-api:jar:1.0:compile [INFO] +- junit:junit:jar:4.5:test [INFO] - org.hamcrest:hamcrest-all:jar:1.1:compile [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESSFUL [INFO] ------------------------------------------------------------------------ Tuesday, 22 June 2010
  60. 60. Maven tips Dependencies in Eclipse Eclipse has an equivalent screen Tuesday, 22 June 2010
  61. 61. Maven tips Dependency analyse What dependencies do you really need? $ mvn dependency:analyze [INFO] Scanning for projects... mvn dependency:analyse [INFO] Searching repository for plugin with prefix: 'dependency'. [INFO] ------------------------------------------------------------------------ [INFO] Building babble-core [INFO] task-segment: [dependency:analyze] [INFO] ------------------------------------------------------------------------ [INFO] Preparing dependency:analyze ... Used but not declared [INFO] [dependency:analyze] [WARNING] Used undeclared dependencies found: [WARNING] javax.persistence:persistence-api:jar:1.0:compile [WARNING] Unused declared dependencies found: [WARNING] org.hibernate:hibernate-annotations:jar:3.2.0.ga:compile [WARNING] org.hibernate:hibernate:jar:3.2.0.ga:compile [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESSFUL Declared but not used [INFO] ------------------------------------------------------------------------ Tuesday, 22 June 2010
  62. 62. Maven tips Excluding dependencies What if you don’t want a dependency? <dependencies> <dependency> <groupId>org.springframework</groupId> <artifactId>spring</artifactId> Don’t include JMS <version>2.5.5</version> <exclusions> <exclusion> <groupId>javax.jms</groupId> <artifactId>jms<artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.apache.geronimo.specs</groupId> <artifactId>geronimo-jms_1.1_spec</artifact> <version>1.1</version> </dependency> <dependencies> Tuesday, 22 June 2010
  63. 63. Maven tips Standardizing versions Use dependencyManagement for consistency <dependencyManagement> Parent pom <dependencies> <dependency> ! <groupId>mysql</groupId> ! <artifactId>mysql-connector-java</artifactId> ! <version>5.1.6</version> </dependency> <dependency> ! <groupId>postgres</groupId> ! <artifactId>postgres</artifactId> ! <version>7.3.2</version> <dependencies> Child pom </dependency> <dependency> </dependencies> !<groupId>mysql</groupId> </dependencyManagement> <artifactId>mysql-connector-java</artifactId> </dependency> </dependencies> Tuesday, 22 June 2010
  64. 64. Maven tips Keep it automated Plan your release strategy Use a Repository Manager Automatic snapshot deployments Automated releases Tuesday, 22 June 2010
  65. 65. Maven tips Maven best practices for CI builds Use batch mode (-B) Always check or snapshot updates (-U) Use a repository per project Print test failures to stdout (-Dsurefire.useFile=false) Tuesday, 22 June 2010
  66. 66. Maven tips Know when to script it Groovy or Ant scripting is easy in Maven Call external scripts when appropriate Tuesday, 22 June 2010
  67. 67. Maven tips Know when to script it It’s pretty easy in Maven 2... <project> <build> <plugins> <plugin> <groupId>org.codehaus.groovy.maven</groupId> <artifactId>gmaven-plugin</artifactId> <version>1.0-rc-5</version> <executions> <execution> <phase>compile</phase> <goals> <goal>execute</goal> </goals> <configuration> <source> println "Hi there I’m compiling ${project.name}" </source> </configuration> </execution> </executions> </plugin> </plugins> </build> ... Tuesday, 22 June 2010
  68. 68. Maven tips Know when to script it It’s even easier in Maven 3... project { build { $execute(id: 'compilation-script', phase: 'compile') { println "Hi there I’m compiling ${project.name}" } $execute(id: 'validation-script', phase: 'validate') { println "Hi there I’m validating ${project.name}" } ... } } Tuesday, 22 June 2010
  69. 69. Tuesday, 22 June 2010

×