Introduction to Software Build Technology

  • 1,868 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,868
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
17
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Introduction to Software Build Technology Philip Johnson Collaborative Software Development Laboratory Information and Computer Sciences University of Hawaii Honolulu HI 96822
  • 2. Objectives
    • Understand motivation for build and packaging technologies.
      • Why was running your classmate ’s Robocode robot so hard?
    • Become familiar with a sample of build technologies.
    • Get introduced to the build technologies and standards to be used in this class.
  • 3. Motivation
    • Modern team-based, cross-platform development involves:
      • Multiple developers and users.
      • System is developed/used on multiple platforms:
        • Win2K, Linux, Win98, Solaris
      • Developers/users have different environments:
        • c:jdk1.5.0_08, d:jdk1.5, c:myjava
      • Differences between user and developer distributions:
        • Source distribution for developers
        • Binary distribution for users
      • Developers may prefer different IDEs:
        • Eclipse, JBuilder, Emacs, RationalRose
      • Users may not have any IDE at all.
      • Frequent releases and distributions.
      • Dependence on third party libraries (which may themselves be released frequently).
    • These issues are not unique to Java development!
  • 4. Motivation (cont.)
    • Build/packaging techniques should allow:
      • Users to build and test the system without IDE.
      • Developers to build and test the system using their IDE of choice.
      • Developers to build and test the way users build and test.
      • Configuration to different platforms and different environments within a platform.
      • Multiple release configurations
      • Automated build to prevent errors
      • Rapid release, deployment, testing
      • Simplified installation and updating of third-party libraries
    • Our (Java-specific) solution:
      • Ant
      • Ivy
      • Build/packaging standards
  • 5. A quick tour of build tools
  • 6.  
  • 7. Make
    • ‘ Make’ is the canonical build tool.
    • Make features:
      • File-level dependency analysis
        • Example: compile only changed files
      • Unix-based (but ported to other platforms)
      • Language independent (Java, C++, etc.)
      • ASCII configuration file syntax
        • Tab characters required—Yikes!
      • Supports invocation of any shell command.
      • Does not support library-level dependency management.
      • Related: imake, cmake, scons
  • 8.  
  • 9. Ant
    • Java-based build tool by Apache
      • http://ant.apache.org/
    • Features:
      • Java-based
      • Cross-platform
      • Extensible
      • XML configuration files
      • Open source
      • Defacto standard for Java projects
        • Maven is making headway.
      • File-level, but not library-level dependency management.
  • 10. Ant vs. Make
    • Advantages of Make (and related tools):
      • More popular and widespread
      • Supports any programming language.
      • Provides ‘glue’ for invoking any shell command.
    • Advantages of Ant:
      • Java-based, active development community
      • Built-in support for Java and its tools.
      • build.xml simpler and easier than Makefile.
      • Simplifies cross-platform Java development
      • Easier to extend in a portable manner.
    • Current recommendation:
      • Always use Ant over Make for Java-based development
      • Use Make for other development, at least for now.
  • 11.  
  • 12. Maven
    • “ Project Management” technology
      • Build based on Ant.
      • Documentation and reporting plugins.
      • File-level dependency management
      • Library-level dependency management
      • Library repository definition & management
    • “ Convention over configuration”
      • Maven builds in conventions for project layout and workflow to simplify use.
        • Great if you like them.
  • 13.  
  • 14. Ivy
    • A “standalone” library-level dependency management tool.
    • Ant + Ivy =
      • Much of what Maven offers.
        • No built-in reporting/documentation
      • Without Maven “conventions”.
      • Better representations for dependencies
        • configurations
  • 15. Ant+Ivy vs. Maven
    • Very strong feelings on both sides.
    • Maven advocates:
      • Maven is the future; deal with it.
    • Ant+Ivy advocates:
      • Maven is a black box; customization is much harder than Mavenites portray.
    • My opinion:
      • If you can live with Maven conventions, it provides more built-in functionality.
  • 16. Our approach
    • We will use Ant+Ivy
      • Ant is still more popular than Maven, so it is useful for you to gain experience with it.
      • Moving from Ant+Ivy -> Maven should not be too bad.
    • You will also learn some “best practices” for Ant-based projects.
      • Support for useful third party tools.
      • Ivy-based library-level dependency management.
  • 17. Basic Ant concepts
    • Every project requires an Ant build file
      • named build.xml by default.
    • Each build file is composed of targets .
      • Typical targets: compile, junit, etc.
    • Each target is composed of tasks
      • Ex: copy .java files to build/src, invoke javac...
    • Targets can have dependencies
      • Ex: modified sources must be recompiled before testing.
  • 18. Ant targets
    • Dependent targets are executed exactly once.
      • Example:
        • test depends upon compile
        • deploy depends upon compile
        • all depends upon test, deploy
      • Compile will be executed only once.
    • Some targets are executed only when needed.
      • Sources recompiled only if changed.
  • 19. Example Ant target
    • <target name=&quot;compile&quot;
    • depends= “init&quot;
    • description=&quot;Compiles code.&quot;>
    • <javac srcdir=&quot;${src.dir}&quot;
    • destdir=&quot;${build.dir}/classes “/>
    • </target>
  • 20. Best practices
    • Ant provides a simple, powerful language for building Java-based systems.
    • How do we use this language effectively?
    • The next section describes my best approach so far on how to employ Ant to achieve the Three Prime Directives of Open Source Software Engineering.
    • The “robocode-pmj-dacruzer” example project provides you with a template structure to get you off and running quickly.
  • 21. robocode-pmj-dacruzer: An example Ant-based build system
    • Implements a pretty lame Robocode robot.
      • Not very interesting.
    • It ’s the build system that’s interesting:
      • Provides a template
        • Easy to adapt to your own projects.
      • Integrates third-party build tools
        • Checkstyle, PMD, FindBugs, etc.
      • Is modular and extensible
        • You can grow or shrink it to your needs.
      • Well suited to build automation and project hosting:
        • Continuous integration (Hudson)
      • Uses Ivy to download/install/manage third-party libraries.
    • Result is Maven-ish, but without the “black box”
      • If you want to change something, it ’s just Ant code.
  • 22. Top-level directory contents
    • build.xml
      • top-level build file.
    • *.build.xml
      • “ module” build files, generally one per tool.
    • src/
      • the system source files
    • lib/
      • library files: maintained by Ivy.
    • build/
      • derived files: generated by build process
  • 23. Build operations
    • compile
      • compile system
    • jar
      • create jar file
    • junit
      • run junit tests
    • javadoc
      • build javadoc docs
    • clean
      • delete build dir.
    • dist
      • create zip dist file.
    • checkstyle
      • analyze source code
    • findbugs
      • analyze byte code
    • pmd
      • analyze source code
    • sclc
      • compute size of system
    • jacoco
      • compute coverage of test cases.
    • (Others to be added later in semester!)
  • 24. Build System Scalability Issues
    • Modularity:
      • putting all targets in one build.xml file creates a very large build.xml file that is difficult to understand and maintain.
    • Incrementality:
      • you may not want to require all developers to have to install every possible third party tool.
    • Coupling:
      • dependencies between target and property definitions should be minimal and easily visible.
  • 25. Build system architecture build.xml pmd.build.xml findbugs.build.xml dist.build.xml checkstyle.build.xml javadoc.build.xml sclc.build.xml *.build.xml build.xml contains most project- specific definitions. *.build.xml files import the build.xml file and provide an implementation of a single extension. common.build.xml boilerplate code
  • 26. build.xml
    • build.xml provides most (but unfortunately not all) of the project-specific definitions:
      • version number
      • dependency libraries
      • project name
      • etc.
    • Most other *.build.xml files can be used with little to no changes.
      • Typical exceptions: junit.build.xml, emma.build.xml, jar.build.xml
  • 27. *.build.xml responsibilities
    • Implements support for a single tool/function:
      • Checkstyle, Jar, JavaDoc, PMD, Distribution, etc.
    • Typical tasks (assume a tool called 'foo'):
      • Download foo using Ivy, install in lib/foo
      • Download a configuration file for foo, store in lib/configfiles.
      • Provide three standard tasks:
        • foo.tool: runs foo tool, generates data.
        • foo.report: creates HTML file.
        • foo: runs foo.tool and foo.report.
      • Make 'foo' the default task.
  • 28. User Interface
    • To do initial installation and compilation:
      • ant
    • For other operations, use -f <filename>:
      • ant -f dist.build.xml
      • ant -f junit.build.xml
      • ant -f checkstyle.build.xml
      • ant -f pmd.build.xml
      • ant -f emma.build.xml
      • etc.
  • 29. Considerations
    • This approach provides the following:
      • Minimal coupling: *.build.xml files depend only on definitions in build.xml
      • Modularity: individual *.build.xml files are small.
      • Reporting: &quot;report&quot; targets provide details on what the &quot;tool&quot; targets generated.
    • One problem:
      • How to &quot;verify&quot; that all analyses &quot;passed&quot;?
  • 30. verify.build.xml
    • Provides “one stop shop” for system analysis:
      • Imports not just build.xml, but other *.build.xml files as well.
      • Runs all &quot;tool&quot; analyses, failing build if they generate any warnings/errors.
        • Compile, Junit, PMD, Checkstyle, JavaDoc, FindBugs
    • User interface:
      • ant -f verify.build.xml
  • 31. The lib/ directory
    • Contains third-party libraries required by project:
      • robocode
      • checkstyle
      • pmd
      • findbugs
      • etc.
    • Contains configuration files for analysis tools.
      • lib/configfiles/checkstyle.modules.xml
      • lib/configfiles/pmd.rulesets.xml
      • lib/configfiles/findbugs.filter.xml
  • 32. The build/ directory
    • Contains files derived from build process.
    • Can be deleted and reconstructed.
    • Not included in version control or distribution .zip.
    • Build subdirs: Targets creating them:
      • build/javadoc/ javadoc
      • build/classes/ compile
      • build/dist/ dist
      • build/sclc/ size
      • build/junit/ junit
  • 33. build/ vs. bin/
    • Both Ant and Eclipse (or any other IDE) compiles files.
    • It causes problems if Ant and Eclipse are using the same output directories for compilation/etc.
    • Therefore, Eclipse writes to the bin/ directory, not the build/ directory.
      • Set in Project | Properties | Java Build Path | Source
  • 34. Testing Notes
    • There are two kinds of “tests” in this system:
      • Manual QA using JUnit
        • Ensures your code satisfies its specifications (if you write good test cases.)
      • Automated QA using Checkstyle, FindBugs, PMD
        • Ensures your code obeys best practices for coding style.
    • Encourages incremental testing and formatting.
  • 35. Traditional release numbering
    • Traditional approach (i.e. jakarta-tomcat-5.1.0.zip)
      • MM.mm.bb, where:
        • MM = Major release number (major redesign or incompatible changes)
        • mm = Minor release number (minor enhancements, backward compatible)
        • bb = Bugfix release number (bug fix, no new functionality)
      • All numbers maintained manually.
    • Advantage:
      • Release number comparison indicates degree of change.
    • Disadvantage:
      • Requires manual editing of release value every time which leads to errors (easy to forget to do this.)
  • 36. Our release numbering
    • Our approach (i.e. robocode-pmj-dacruzer-1.3.2011.09.11.12.32.zip)
      • Major.minor.timestamp, where:
        • Major = major release number
        • Minor= minor release number
        • Timestamp= time of bugfix release (yyyy.mm.dd.hh.mm)
      • Major and minor release number maintained manually.
      • Bugfix timestamp generated automatically.
    • Advantages:
      • Automatic except when incrementing major/minor release.
      • Release number provides timestamp of release.
    • Disadvantages:
      • Number of bugfix releases not indicated by release number.
      • Very long version number.
  • 37. IDE integration
    • Important to maintain independence from IDE.
      • All build functions should be possible without any IDE.
    • But it ’s fine to provide IDE-specific files to facilitate development.
      • Example: .class and .project files in top-level directory for Eclipse.
    • Provide additional files/directories for other IDEs if desired.
  • 38. robocode-pmj-dacruzer as a standard
    • From now on, all projects must be developed, packaged, and submitted using the standard build conventions embodied in this package.
  • 39. robocode-pmj-dacruzer as a template
    • You can use robocode-pmj-dacruzer as a template for your own projects:
      • Download a fresh version.
      • Remove dacruzer-specific info
      • Add your project info.
    • See the template instruction page in the course website for step-by-step instructions.
  • 40. Ant/Eclipse integration
    • Eclipse has nice support for Ant, but:
      • You will also need to tell Eclipse about ANT_HOME:
        • Window | Preferences | Ant | Runtime | Ant Home…
      • Make sure Eclipse writes to the bin/ directory
        • Project | Properties | Java Build Path | Source
  • 41. Limitations of example
    • The robocode-pmj-dacruzer template provides a framework plus basic build functionality, but has limitations:
      • No SVN integration
      • No support for web app development.
      • No support for automated metrics collection.
    • We will enhance this initial template during the semester.
  • 42. Meta-level goal
    • Ant provides an efficient solution to satisfying both
      • PD#2 (users can install system) and
      • PD#3 (developers can understand and enhance system)
    • Other approaches exist:
      • InstallAnywhere for PD#2 (more work)
      • IDE-specific solution for PD#3 (limited)
    • With good installation documentation, I believe Ant-based build can satisfy both PD#2 and PD#3.
  • 43.