From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Implementing a Continuous Integration System for Software Design
1. Introduction First Design Second Design
Implementing a Continuous Integration System
for Software Design
Vincent Borchardt
Department of Computer Science
University of Minnesota, Morris
May 1, 2013
2. Introduction First Design Second Design
Objective
Make a continuous integration system to support the new web-based
software projects for software design.
Three main components of the system:
Tomcat: Web server for hosting student projects and/or the other
components
Jenkins: Automated build system for projects in Subversion
Sonar: Code analysis tool
3. Introduction First Design Second Design
Tomcat
A web server that hosts .war files (Java web container):
Student projects
Other utilities (Jenkins and Sonar)
4. Introduction First Design Second Design
Jenkins
An automated build system:
Checks if the build compiles
Runs the tests
Keeps track of all previous builds over time
Runs either by itself or on a web server
5. Introduction First Design Second Design
Sonar
A code analysis tool:
Checks for duplication
Calculates technical debt
Checks test coverage
6. Introduction First Design Second Design
Initial Objective
Became TA for Spring 12 offering of CSci 3601 Software Design
Many changes to the class this year:
Designing a web-based application (using Groovy and Grails)
instead of a desktop application (using Java and Swing)
New Agile textbook brings new practices (Inception Deck,
Story-Building Workshop)
Projects use continuous integration
7. Introduction First Design Second Design
System Design
One virtual machine (csci3601sp12)
Runs a Tomcat server that holds Jenkins, Sonar, and the students’
projects
Simplest possible execution
8. Introduction First Design Second Design
Problems
Overall:
Managing memory was always an issue, even as the RAM
available to the VM was increased over time
Which version of the software to use was always a problem
Authentication was not a priority
Jenkins:
Very easy to upgrade (download a new WAR file, upload to
Tomcat)
Easy to integrate with LDAP in the lab
Initially setup with no authentication system, but was "hacked"
halfway through the project
Most successful of the three components
9. Introduction First Design Second Design
Problems
Overall:
Managing memory was always an issue, even as the RAM
available to the VM was increased over time
Which version of the software to use was always a problem
Authentication was not a priority
Tomcat:
Started with Tomcat 6–memory leaks occurred whenever a WAR
file was removed from the server
Jenkins uploaded a new WAR of each project on each successful
build, so many WAR files were added and removed over time
Upgrading to Tomcat 7 fixed the memory leak problem–the
upgrade was not much trouble
Only I had access to Tomcat, but Jenkins needed access as well
I manually entered the password to access Tomcat for each project
10. Introduction First Design Second Design
Problems
Overall:
Managing memory was always an issue, even as the RAM
available to the VM was increased over time
Which version of the software to use was always a problem
Authentication was not a priority
Sonar:
Used Sonar 2.12–wasn’t compatible with LDAP authentication
Used the trial built-in database through the whole course:
Builds took extremely long
Sonar could not be updated
11. Introduction First Design Second Design
System Design
Three virtual machines (jenkins, sonar, tomcat)
Each component is running by itself, instead of all running on a
Tomcat server.
Services are backed up with database support if necessary
12. Introduction First Design Second Design
Jenkins
Installed with yum instead of being deployed on a Tomcat server
Uses Sonar Runner to transfer information from Jenkins to Sonar
Mostly unchanged from the first design
13. Introduction First Design Second Design
Sonar
Installed with yum instead of being deployed on Tomcat
Used version 3–LDAP works.
Backed up with a MySQL database for handling project information:
Allows for upgrading Sonar
Builds take less time
14. Introduction First Design Second Design
Tomcat
Installed manually
Started with version 7
Backed up with a MySQL database so students’ projects can be
connected to real databases in production
15. Introduction First Design Second Design
Current Problems
Overall:
Memory still a concern
Sonar doesn’t support Grails/Groovy perfectly
Tomcat has many quirks
Jenkins:
Test success/code coverage is being kept track of in Jenkins
instead of Sonar
16. Introduction First Design Second Design
Current Problems
Overall:
Memory still a concern
Sonar doesn’t support Grails/Groovy perfectly
Tomcat has many quirks
Tomcat:
Grails projects leave a Timer thread when undeployed
Causes a large memory leak; Tomcat must be occasionally
restarted (by restarting the whole VM)
Tomcat must be restarted manually when the VM starts
Errors in projects deployed on Tomcat don’t give meaningful error
messages by default
A logging.properties file in the Grails project allows stack traces to
be shown in Tomcat–but that file is deleted when tests are run on
the Grails project for some reason.
17. Introduction First Design Second Design
Current Problems
Overall:
Memory still a concern
Sonar doesn’t support Grails/Groovy perfectly
Tomcat has many quirks
Sonar:
Sonar cannot process code coverage information produced by
Grails through our current build process
Lots of information available in Sonar; difficult to find the
important information
18. Introduction First Design Second Design
Future Plans
Implement a larger-scale build process for more than just Grails
projects:
Sonar works with plain Java/Groovy projects, but tests aren’t run
Jenkins and Sonar have plugins for many languages
Confirm each service scales:
Do we want to leave each project on Jenkins/Sonar/Tomcat
forever?
How big must each VM be?
Determine which features of each tool are most important, and which
are missing