Build server

Documentation
Build ServerLast Edit: 08/11/2010
Credits:
Christophe Vanlancker
Tom Van Goethem

Index
1 Preface
2 Installing the server
3 Installing the tools
3.1 Java
3.2 Mercurial
3.3 Maven
3.4 Junit
3.5 Jdepend
3.6 Hudson
4 Adding developers
5 Configuring Hudson
5.1 Plugins
Post-Installation
6 Securing Hudson
7 Push Mercurial enabled Projects to Hudson
8 Adding jobs to Hudson
9 Divide build process over 3 jobs
A) Syllabus

1. Preface
This documentation will guide you through setting up a build-server using popular tools
like Jdepend, Junit, Mercurial, Maven and Hudson.
Each section will contain a brief explanation of each tool and how it relates to the
building process.
The following process will enable developers to commit code from their IDE of choice to
the central build-server where it will be built, tested and analyzed.

2. Installing the server
The choice of operating system to use on any server should be motivated by the wide
availability of applications, good support, regular releases of both bug fixes as critical
security fixes.
Intermediate knowledge and understanding of the underlying operating system is also
required in order to troubleshoot.
For the reasons above ( and because of the requirements of this project ) we have
chosen to use the latest version of Ubuntu, a GNU/Linux distribution.
Getting Ubuntu
1. Surf to http://www.ubuntu.com/server/get-ubuntu/download
2. Follow the guide on the website to chose the appropriate architecture
version of Ubuntu and how to boot the system up to start the installation.
Installing
The installation is pretty straight forward following the instructions.
You may want to prefer selecting a few things during the installation:
1. At the Select and install software screen, chose to manage upgrades by
Install security updates automatically.

2. At the Software selection screen, chose to install the OpenSSH server.
This will make it easier to access the server remotely in case direct physical
access is cumbersome.
Update
It's advisable to update your newly installed system.
Bugs and security leaks will surely be discovered and fixed after the official
release.
This can be performed by logging into the system and by issuing the following
command:
sudo apt-get update && sudo apt-get upgrade
Note – it's possible that during the upgrade, a new kernel version has been
released. In that case, it's advised to reboot the system in order for the system to
use the new kernel. To reboot the system, issue the following command:
sudo shutdown -r now

3. Installing the tools
We are going to cover the installation of all the tools we are going to use on the build-
server. A brief explanation will be provided for each.
3.1 Java
Java is a general-purpose, concurrent, class-based, object-oriented language that
is specifically designed to have as few implementation dependencies as possible.
It is intended to let application developers "write once, run anywhere". Java is
currently one of the most popular programming languages in use, and is widely
used from application software to web applications.
We are going to use Java because Hudson, Maven, Jdepend and Junit are Java
tools.
It's note worthy that in Ubuntu there are 2 kinds of Java packages: sun-java6-jdk
and openjdk-6-jdk.
OpenJDK is an open-source implementation of Java. Even-though it would be
possible to use openjdk, practice shows that there can be some inconveniences
using it. Inconveniences, mainly caused by missing core classes due to the
proprietary code ownership of Oracle, as some minor incompatibilities.
Sun-Java is the official proprietary Java implementation supported by Oracle.
We will be using this Java stack.
Note – Ubuntu has a policy to not ship proprietary packages by default. However,
they have made it easy to add them should the user want to. Sun-Java is an
example of such package.
Enable the Partner Repository
Sun-Java is located in the Partner repository of Ubuntu. The repository is
commented out by default. We will have to remove the comment sign and update

in order for the Sun-Java package to be available for download.
1. From the command-line, issue the following command to open a text-
editor:
sudo nano /etc/apt/sources.list
2. Search for the 2 lines regarding the 'Partner' repository, and remove the '#'
symbol to enable them.
deb http://archive.canonical.com/ubuntu maverick partner
deb-src http://archive.canonical.com/ubuntu maverick partner
3. Now you need to refresh the sources list. Ubuntu keeps a local copy of all
available packages from all the different enabled repositories. Issue the
following command to update the source list:
sudo apt-get update
Install sun-java6-jdk
We need the jdk version of Java instead of the regular binary one because we
need an extended subset of the Java Software Development Kit (SDK) to be able
to compile Java applications among many other things.
1. Issue the following command to install Java:
sudo apt-get install sun-java6-jdk
Note – During the installation, you are required to accept a license. Accept
it to proceed with the installation.
2. Verify that Java is installed, issue the following command:
java -version

3.2 Mercurial
Mercurial is a cross-platform, distributed revision control tool for software
developers. It's mayor design goals include high performance and scalability,
decentralized, fully distributed collaborative development, robust handling of
both plain text and binary files, and advanced branching and merging capabilities,
while remaining conceptually simple. It includes an integrated web interface.
Developers will be able to upload their code to the build-server though a very
simple yet powerful tool which keeps track of different versions of the code. It
enables them to follow these changes with added commentary by other
developers working on the same project with the added capability of reverting
back to previously working code.
Install Mercurial
1. Install from the repository, issue the following command:
sudo apt-get install mercurial
Usage
If you are used to working with other revision control tools like Git, you'll find
that the usage is quite similar.
Every Mercurial command starts with hg followed by the action you want to
undertake.
1. Initializing a repository within a project folder
hg init
Note – this will create a hidden .hg folder where you initialized the
repository. Mercurial will store everything about your project in there.
2. Adding files to the repository (here will be shown how to include
everything in one step):
hg add *
3. Committing files, basically taking a snapshot of what currently has
changed can be done like this:
hg commit -m ' description of change '

4. Cloning a Mercurial repository, taking an identical copy of another
repository:
hg clone http://example.org/repos/project/
hg clone ssh://build-server.local/repos/project/
hg clone ftp://build-server.local/repos/project/
Note – The usage of ssh or ftp may require authentication. Please note
that on some systems, the username of the current user is implied during
the authentication.
5. Pushing your own changes after a commit happen with a similar syntax as
cloning:
hg push protocol://destination/path/to/project
6. Pulling new commits from a repository can be done like this:
hg pull protocol://destination/path/to/project

3.3 Maven
Maven is a software tool for project management and build automation.
Maven uses a construct known as a Project Object Model (POM) to describe the
software project being built, its dependencies on other external modules and
components, and the build order. It comes with predefined targets for
performing certain well defined tasks such as compilation of code and its
packaging.
Maven is built using a plugin-based architecture that allows it to make use of any
application controllable through standard input. Theoretically, this would allow
anyone to write plugins to interface with build tools (compilers, unit test tools,
etc.) for any other language.
Maven will be used as a kind of container for Java projects, in addition, describing
what needs to happen with the code.
It will form the basis of the automated building process.
Install Maven
1. To install Maven, issue the following command, dependencies will be
resolved automatically by apt:
sudo apt-get install maven2
2. Verify that Maven is installed:
mvn -v
Usage
Here are a few Maven commands which are mostly used.
1. Create a new project:
mvn archetype:generate -DgroupId=com.mycompany.app
-DartifactId=my-app
2. Compile:
mvn compile
3. Install the generated output to the respective repository:
mvn install

4. Package as .jar or .war:
mvn package
5. Clean target directory:
mvn clean
6. Run the Unit tests:
mvn test
7. Generate JavaDocs:
mvn javadoc:javadoc

3.4 JUnit
Junit is a unit testing framework for the Java programming language. JUnit has
been important in the development of test-driven development, and is one of a
family of unit testing frameworks collectively known as xUnit that originated
with SUnit.
JUnit is linked as a JAR at compile-time; the framework resides under packages
junit.framework for JUnit 3.8 and earlier and under org.junit for JUnit 4 and
later.
JUnit will be invoked by Maven through the building process to execute all tests
written by developers to ensure that all previously written code behaves just as
expected.
Install JUnit
1. Install JUnit:
sudo apt-get install junit

3.5 JDepend
JDepend traverses Java class file directories and generates design quality metrics
for each Java package. JDepend allows you to automatically measure the quality
of a design in terms of its extensibility, re-usability, and maintainability to
manage package dependencies effectively.
After a project has been built, Jdepend will run and collect the metrics of the
packages so to give insight on the design quality.
Install JDepend
1. Install JDepend
sudo apt-get install libjdepend-java

3.6 Hudson
Hudson monitors executions of repeated jobs, such as building a software project
or jobs run by cron.
It provides an easy-to-use so-called continuous integration system, making it
easier for developers to integrate changes to the project, and making it easier for
users to obtain a fresh build. The automated, continuous build increases the
productivity.
Hudson will play the central part of the whole building process.
Hudson will combine all the tools together, and make it easy for everyone to
monitor the building process.
Install Hudson
Hudson isn't present in any repository of Ubuntu. However, following the
installation here, it will be added into the package source file of Ubuntu, so you'll
be able to get updates when available.
1. Install the daemon package. Hudson won't be installed through apt, so we
need to install the dependency manually:
sudo apt-get install daemon
2. Download and add the repository secure key, to insure you are
downloading software from a respected source:
wget -O /tmp/key http://hudson-ci.org/debian/hudson-ci.org.key
sudo apt-key /tmp/key
3. Download Hudson and manually install it:
wget -O /tmp/hudson.deb http://hudson-ci.org/latest/debian/hudson.deb
sudo dpkg --install /tmp/hudson.deb
Usage
Hudson should start automatically on startup of the server.
1. To start/stop/ Hudson, you can issue the following command, Hudson will
start a built-in web-server reachable on http://localhost:8080 ( and
http://server-ip:8080 ):
sudo service hudson start/stop

4. Adding Developers
Adding new developers to the build-server should be made very easy so that they can
start working as soon a possible. Group management helps giving developers access
and rights efficiently. Also, this provides consistent and predictable usage as well as
spreading responsibility to different people during the build process.
For simplicity, we are going to use the already present Unix Account system, which
provides local login and grouping management.
We will couple this to Hudson to setup the developers policies.
Create the Developers Group
We will create a group to which we will add all our normal developers.
All users added to this group will have equal rights and policies applied.
1. Create the developers group:
sudo addgroup developers
Create a new User
Every user will have his own home directory where they can store their files.
1. Create the user 'joe':
sudo adduser joe
Add user to the developer group
Now we are going to add the user to our previously created group.
1. Add the user 'joe' to the developers group
sudo adduser joe developers
Note – For the developers to share access to the same repository, the project's
group access should be set to the group of the developers which are working on
it. This can be done with the chmod command.

5. Configuring Hudson
At this point, you should have all tools installed and a few developer accounts set. It's
time to focus on Hudson and it's configuration.
Accessing Hudson
Hudson's web-interface can be reached by visiting the following Url in a browser
on the server itself in a graphical environment.
http://127.0.0.1:8080
This approach however is not very professional. Ideally the server must be
reachable on the network through either an externally available IP address or a
domain name.
The Ubuntu server can be reached by using the hostname of machine as if it were
a domain name.
As an example, the hostname of our build-server has been intuitively been called
'build-server'.
Depending on the operating system run on the developer's client machine, they
either have to add '.local' as a suffix to the hostname of the build-server in their
browser, or rely on a DNS infrastructure which is beyond the scope of this
documentation.
Either way, in this example, our build-server is reachable from a Linux based
client through the following address:
http://build-server.local:8080

Configuration Screen
From the main page of Hudson, you can start by pressing the “Manage Hudson”
link. This will take you to another screen, from which you select “Configure
System”.
JDK
• Name: Java-JDK
• JAVA_HOME: /usr/lib/jvm/java-6-sun/
Note – /usr/lib/jvm/java-6-sun/ is a symbolic link. Every time there's an
update to Java, the symbolic link will be updated to point to the latest
installed update. Pointing it to some more specific version of Java will
present the risk of compiling projects with old core libraries, left exposed
to known security vulnerabilities and can also cause sudden breakages
should old versions of Java be removed.
Mercurial
Note – You need to activate the Mercurial Plugin before you can configure
these options, see section 5.1 Plugins
• Name: Mercurial
• Installation Directory: /usr/bin/
• Executable: hg

Maven
• Name: Maven
• MAVEN_HOME: /usr/share/maven2/
5.1 Plugins
Hudson can be extended through the use of Plugins.
There are many available, this documentation will only cover the ones needed for
the basic functioning and purpose of this build-server.
Go back to the home page of Hudson and just as described at the top of this
section ( 5. Configuring Hudson ), click on the link to “Manage Hudson”, and
select “Manage Plugins”.
Check the check-boxes of the following plugins:
• Hudson JDepend Plugin
• JUnit Attachments Plugin
• Hudson Mercurial plugin
Apply by pressing the button at the bottom of the page called “Install”.
Hudson will be restarted ( eventually after jobs have been completed should
there already be some running at the moment ).

Post-Installation
6. Securing Hudson
Security is an important aspect of anything. Security through policies prevents
accidental mis-configurations by non responsible people.
Security becomes even more critical when the build-server is exposed to the Internet,
preventing tampering of code from unknown sources.
We have prepared the Unix accounting system, creating accounts and adding them to a
group to easily manage them. We are now going to setup Hudson to utilize this as the
basis for managing users and their roles within Hudson.
Add Hudson to the shadow file
User passwords (hashes) are stored within a file at /etc/shadow.
By default, this file is owner by root and the shadow group.
We will have to add Hudson's Unix username “hudson” to this group in order for
Hudson to check the users credentials.
1. Add Hudson to the shadow group:
sudo adduser hudson shadow
2. Restart Hudson so that it reloads it's run-time permissions:
sudo service hudson restart
Activate Hudson's Security features
Access the Configuration screen of Hudson as previously described in section 5.
Configuring Hudson.
• Select the check-box “Enable Security”.
• In the “Access Control” section, select “Unix user/group database” in the
“Security Realm” division.
• In the “Authorization” section, you can chose among things, Matrix-based
security, and Project-based Matrix Authorization Strategy.

Matrix-based security, is handy when it comes to managing the overall
access for every user to all available projects. Different users and user
groups will have the same authority over any project.
Project-based Matrix Authorization Strategy is an extension to Matrix-
based security, which enables authorization on a per project basis.
In our current building infrastructure, we have chosen for “Matrix-based
security” out of simplicity.

7. Push Mercurial enabled Projects to Hudson
In a realistic scenario, a developer will be working on a project on his workstation in an
IDE such as Netbeans, Eclipse, or any other programming environment. He will have to
sync his code to the central repository of a project he's working on with a team of other
developers.
Here we will illustrate how to start a new project and perform a first push to the build-
server, and how to setup Hudson to automatically start the building process with the
necessary checks.
We suggest you find out how to perform the following steps within your IDE of choice.
Create a new Project
Start a new Maven Project. Either through your IDE, or by executing Maven's
corresponding command which can be found in this documentation in section 3.3
Maven.
Initialize a Mercurial Repository for your Project
Find the corresponding actions to initialize a Mercurial repository for your
project, either through your IDE, or through the command-line ( see section 3.2
Mercurial ).
Compile the project for the first time
Compiling for the first time will make sure that all necessary files will be
generated and ready for being pushed to the build-server.
Either use your IDE to start building your project, or the relative command in
Maven ( see section 3.3 Maven ).
Make your first Commit
Committing ( as mentioned in section 3.2 Mercurial ) means taking a snapshot of
changes made to the code. Because this will be the first time, it's custom to add
“Initial commit” to describe the commit.

Note – IDE's will usually take care of adding all the changed files to
Mercurial before committing. If committing manually, remember to add
the changed files first before committing.
Create Project folder and Repository on the Build-server
On the build-server, a folder should be prepared to accommodate the project
itself.
Access to the folder should be prepared accordingly.
Do not forget to initialize a mercurial repository on the build-server too.
Any Mercurial transaction will happen between 2 Mercurial folders.
If either local or destination doesn't have a mercurial repository, the transaction
will fail.
Push the Commit to the Build-server
Please refer to section 3.2 Mercurial regarding the pushing of a commit.
After performing all these steps, depending on the folder rights, it should be possible
for users to clone that repository on their local machine and have an exact copy of it.

8. Adding jobs to Hudson
After you've setup a repository on the build-server's filesystem, it's time to link the
repository to Hudson for monitoring and building.
This documentation will be based on a Mercurial enabled repository with a Maven
Project.
Further more, it will describe how to:
• make Hudson regularly scan the repository for changes to trigger building
• generate javadoc
• archive the artifact
• collect test results from JUnit
• generate JDepend results
Creating a new Job
• On the home page of Hudson, select the “New Job” link.
• Give a name to your new Job, and eventually a description. For the sake
of intuitiveness, name the Job after your Project's name.
• Select the radio button corresponding to the type of project. In our case,
select “Build a maven2 project”.
• Press the “Ok” button.
Configure the Job
After the creation of your new Job, the Job configuration screen should be
presented.
There are many options available. The options shown from here on will be the
only ones configured. All other options will be left as is.
Source Code Management
• Select the radio button “Mercurial”.
• Enter the location of the mercurial repository. Example:
/var/repositories/MyProject/

Note – It is imperative that the location is valid. Entering an invalid
path will result in Hudson crashing during a build attempt.
To avoid this, make sure that there is a hidden .hg folder present in
the project's folder you are pointing to.
Build Triggers
• Select the checkbox “Poll SCM” to make Hudson periodically check
for changes in the repository to trigger the build process.
• In the “Schedule” text field, enter “* * * * * “ without quotes. This is
a cron job syntax which means that Hudson will scan for changes
every minute.
Build
• In “Goals and options”, enter “javadoc:javadoc”. This will cause
Maven to generate JavaDoc when building.
Post-build Actions
• Select “Archive the artifacts”. A text field will appear. Within it, you
enter “target/*.jar”. This means that Hudson will archive any jar file
within the “target” folder. These are usually snapshots of your
project.
Note – If you haven't compiled your project once before starting the
build process, this might cause the build to fail because of the
absence of any jar file within the “target” folder. Should that be the
case, you way want to disable this option at first, and re-enable it
when the file has been created.
• Select “Aggregate downstream test results”.
• Select “Additional test report features”, and the underlying option
“Publish test attachments”.
• Finally, select “Report JDepend”.
• Press the “Save” button.

Starting your first Build
You can now start your build by pressing the “Build Now” button.
If all goes well, a blue co loured ball should appear next to your Job, provided that your
project doesn't contain any errors.
In the unlucky event that Hudson reports an error in the build, you can check the cause
of the error by going to the failed build, and take a look at the “Console Output”.
JDepend results can be found within the specific build screen as a link.
JavaDoc can be found at the Project's Page within the “Module” link.

9. Divide build process over 3 jobs
At some stage, specially with big projects that require a big amount of time to process
through the build-server, it becomes evident that there's a need to streamline the build
process to resemble the pipelining strategy of a CPU.
The idea is to chop the build process into smaller stages. Every stage from different
projects gets it's turn after waiting in the queue.
We will configure 3 separate jobs:
• Builder
• Stats and Tests
• Archiver
Before we begin, you should install an additional Plugin for Hudson:
• Hudson Clone Workspace SCM Plug-in. ( see section 5.1 Plugins )
Explanation of why will follow.
Job 1 – Builder
• Create a new Job ( see section 8. Adding jobs to Hudson for guidelines ).
Give it an intuitive name like “ProjectName - Builder”
• Select it to be a Maven project
• Configure the Mercurial Repository.
• Set it up to Poll Mercurial for changes.
• In “Goals and options”, enter “compile”. This will cause Maven to just
compile instead of running the Tests which will be performed by the next
Job.
• Select the “Build other projects” option and enter the name of the job it
has to trigger upon completion, example: “ProjectName – Stats and Tests”.
• Additionally, select the “Trigger even if build is unstable” option.
• Finally, select the “Archive for Clone Workspace SCM” option and add “
** ” without quotes in the corresponding text field.

What we have done now, is create a Job which will solely build the project, and
trigger the next Job “ProjectName – Stats and Tests”.
We have also indicated that the workspace of this Job should be cloned.
Cloning the workspace has the advantage of it being used as the source
repository, in a way, acting as our Mercurial repository. Additionally, the next job
will be able to use the same build in case another build job would have been
triggered in between.
Job 2 – Stats and Tests
We will create a 2nd
Job. This one will be triggered by our first Job, the Builder.
It will use Job 1's cloned workspace as the source for which to analyze and test
through JDepend and JUnit.
Give it an intuitive name like “ProjectName – States and Tests”
• Instead of choosing Mercurial, select “Clone Workspace” and
“ProjectName – Build” as the parent.
• You should notice that “Build after other projects are built” is already
selected and pointing to the correct parent Job name.
• Select “Additional test report features” and “Publish test attachments”.
• Select “Report JDepend”.
Job 3 – Archiver
This Job will run every 15 minutes and automatically archive our project.
It will run independently of our previous 2 Jobs.
Give it an intuitive name like “ProjectName - Archiver”
• Configure the Mercurial Repository.
• Select “Build periodically”, and add “ */15 * * * * ” without quotes in the
Schedule field. This will cause Hudson to start the build every 15minutes.
• In “Goals and options”, enter “javadoc:aggregate”. This will cause Maven
to generate JavaDoc found anywhere within the project when building.
• Select “Archive the artifacts” and enter “ ** ” without quotes in the

“Files to archive” field. This will make Hudson archive everything within
the project.
You may now try to start the building process by either pushing a new
Mercurial Commit , or by triggering the building process of your Builder Job manually.
As expected, the 2nd
Job should appear in the queue shortly after the completion of the
Builder Job. After about 15minutes, the Archiver Job should have a run.

A) Syllabus
Artifact
An artifact is one of many kinds of tangible by-product produced during the development of
software. Some artifacts (example: use cases, class diagrams, and other UML models,
requirements and design documents) help describe the function, architecture, and design of
software. Other artifacts are concerned with the process of development itself - such as project
plans, business cases, and risk assessments.
Build-server
Server deployed to provide a facility to support Continuous Integration.
Continuous Integration
Implementation of a continuous process of applying quality control - small pieces of effort,
applied frequently. Continuous integration aims to improve the quality of software, and to reduce
the time taken to deliver it, by replacing the traditional practice of applying quality control after
completing all development.
POM
See Project Object Model
Project Object Model
Provides all the configuration for a single project. General configuration includes the project's
name, its owner and its dependencies on other projects. One can also configure individual phases
of the build process, which are implemented as plugins. For example, one can configure the
compiler-plugin to use Java version 1.5 for compilation, or specify that project can be packaged
even if some unit test fails. Larger projects should be divided into several modules, or sub-
projects, each with its own POM. One can then write a root POM through which one can compile
all the modules with a single command. POMs can also inherit configuration from other POMs. All
POMs inherit from the Super POM by default. Super POM provides default configuration, such as
default source directories, default plugins and so on.
Repository
Central place in which an aggregation of data is kept and maintained in an organized way, usually
in computer storage. The term is from the Latin repositorium, a vessel or chamber in which things
can be placed, and it can mean a place where things are collected. Depending on how the term is
used, a repository may be directly accessible to users or may be a place from which specific
databases, files, or documents are obtained for further relocation or distribution in a network. A
repository may be just the aggregation of data itself into some accessible place of storage or it
may also imply some ability to selectively extract data.

Revision Control
The management of changes to documents, programs, and other information stored as computer
files. It is most commonly used in software development, where a team of people may change the
same files. Changes are usually identified by a number or letter code, termed the "revision
number", "revision level", or simply "revision".
SCM
Short for Software Configuration Management. Synonym for Revision Control
All information in the Syllabus has been partially or entirely been copied from wikipedia

Build server

More Related Content

What's hot

Similar to Build server

Recently uploaded

Build server