This is an overview of continuous integration, deployment and testing. It's a half day presentation, given in Orlando at Agile Dev Conference East.

1. Continuous
Testing
J a r e d R i c h a r d s o n
A g i l e A r t i s a n s
@ J a r e d R i c h a r d s o n
1
Who am I?
2
Jared Richardson
CoAuthored Ship It! and Career 2.0
Screencast editor for PragProg.com
Started AgileRTP in 2007
2nd public signatory of the Agile Manifesto
Agile coach
AgileArtisans.com
3
Who are you?
4

2. What would you say you
do here?
Roles?
Responsibilities?
Why are you here?
What are your tests for this session?
5
Objectives
Understand the Continuous Paradigm
See how it applies to your teams
Understand the basics of Jenkins
Understand an end-to-end Continuous workflow
Collect practical tips and tricks
6
Outline
Continuous paradigm
Continuous Integration (benefits & workflow)
Introduction to Jenkins
Continuous Testing (benefits & workflow)
Jenkins as a continuous deployment/testing tool
Integration testing tool
7
Breaks
Half-way (90 minutes)?
Each hour ?
8

3. Continuous
Paradigm
9
What Is Software?
10
Feedback Loops
11
What Is Agile?
12

4. AgileManifesto.org
13
Feedback Loops
14
flickr.com/photos/bobowen/4501875907
flickr.com/photos/30691679@N07/3808395213
flickr.com/photos/kevino/1438502858
15
Feedback Loops are
Fractal
flickr.com/photos/dancentury/4579906149
16

5. Feedback Loops
Inside Feedback Loops
Pairing
Test First
Demos
Daily Meetings
???
17
Agile Is...
Tighter Feedback Loops
18
Minimize Dark Times
Code Change
Validation
http://flickr.com/photos/bogenfreund/556656621/
19
Fast
feedback
leads to
fast fixes
20

6. Different Levels
Good at the bottom
Great at the top
Ignore the middle
21
Continuous Feedback
Continuous Integration
Continuous Delivery
Continuous Testing
22
Exercise
How long is your SDLC?
What are your SDLC feedback bottlenecks?
How long do you have to wait for feedback?
Write down 3 bottlenecks
Gather into groups of 5 to 7
Share and discuss (solved problems?)
Turn in your top 5 items (for the group)
Take ten minutes
23
Continuous Integration
24

7. Continuous Integration
Watch Code
Build product
Run tests
Publish results
25
Pros
Fast feedback
Known state
Easily consumed history
Shines a light
26
26
Cons
Slows you down
Nags you after every code change
Exposes problems
Shares issues
Forces you to deal with problems
Smoke alarm
flickr.com/photos/117168856@N06/1242867837493
27
28
Perception of Problem Accumulation
Work
Time
28

8. 29
Work
Actual Problem Accumulation
Time
29
30
Work
Continuous Problem Resolution
Time
30
GPS
Atlas
Maps
Printed maps!
Navigation software
What’s better?
31
31
Feedback Loops
32

9. Steps
Use source code management (SCM)
Automate the build (script)
Acquire a build box (isolated)
Install a build environment (dev tools)
!
33
33
Java Tooling
Publish results
34
Continuous Integration
Servers
Many
Jenkins
Cruise Control
Bamboo
Pulse
TFS
35
35
Jenkins
Open source
Established
Split from Hudson
Java application
WAR file
Runs on any application server
Self contained
XML & text
36
36

10. Example
Publish results
javac (Oracle & IBM)
message broker
etc
ws ejb deploy
message broker tools
sshexec (to scripts)
JUnit
HP's UFT
Selenium
Web page
RSS
Email
37
Dependent Builds
Data Layer
Services
Client/GUI
38
Dependent Builds
Data Layer
Data Services
Rich Client
Common Libs
Web Portal
Biz Services
39
Chained Continuous Integration
Team 1 Team 2 Team 3
Integration
build
40

11. Platform Testing
Team 1 Team 2 Team 3 Team 4 Team 5
Integration Build
Platform1 Plat 2 Plat 3 Plat 4 Plat 5
41
Demo
Sample project
Build script
Check into SCM
Add to Jenkins
Edit/Commit/Repeat
42
42
Continuous Testing
43
Continuous Integration
Watch Code
Build product
Run tests
Publish results
44

12. Continuous Testing
Watch CI
Deploy
product
Integration
tests
Publish results
45
Continuous Testing
Build in CI (compile & unit tests)
Deploy to dev (staging)
Run integration tests
46
What does this tell us?
If it works
(quickly)
47
Continuous deployment is controversial.
Most people ... think I’m advocating
low-quality code
or an undisciplined
cowboy-coding
development process.
ERIC RIES
radar.oreilly.com/2009/03/continuous-deployment-5-eas.html
48

13. Language is a Tool
Use it
49
What's this look like?
50
Continuous Testing
CI CT
Compile &
unit test
Deploy &
integration
test
51
Multiplatform Deploys
CI
CD Win7
CD Linux
CD AIX
52

14. Websphere Deploys
53
Demo
Add a new Jenkins job
Monitor the compile job
Deploy the artifact
Run integration tests
54
54
Exercise
Same as before...
What breaks (in your experience)?
Top 3 bug types?
Groups of 5 to 7
Gather the top 5 for your group
55
55
As We Explore a
Continuous World...
56
56

15. What Concepts Emerge?
57
57
Nuke and Pave
Reset (completely)
Start over
Completely Scripted
58
58
Binaries
Nuke & Pave
Delete
Recompile
Deploy
Use
flickr.com/photos/x-ray_delta_one/4152356464
59
59
Data
Nuke & Pave
Delete schema
Drop tables
Recreate
Restock
Rails
Liquibase
flickr.com/photos/x-ray_delta_one/4152356464
60
60

16. Rails ActiveRecord
CREATE
TABLE
products
(
id
int(11)
NOT
NULL
auto_increment,
name
varchar(255),
PRIMARY
KEY
(id)
);
61
61
Rails ActiveRecord
change_table :card_deck do |table|
tablet.remove :description, :name
tablet.string :first_name
tablet.index :part_number
tablet.rename :list_name, :last_name
end
62
62
Rails ActiveRecord
rake db:migrate
rake db:migrate VERSION=0
rake db:reset
63
63
ActiveRecord Demo
rake db:reset
Drops schema
Recreates schema
64
64

17. Operating System
Nuke & Pave
Virtualize
Recreate
flickr.com/photos/x-ray_delta_one/4152356464
65
65
Vagrant
vagrant init lnx http://lnx/lnx.box
vagrant up
vagrant ssh
vagrant reload --provision
!
Command line operating system
provisioning
66
66
Vagrant
config.vm.network "forwarded_port", guest: 80, host: 8080
config.vm.synced_folder "../data", “/vagrant_data"
67
67
Steps for a New OS
Install VirtualBox
Install Vagrant
vagrant init hashicorp/precise32
vagrant up
68
68

18. Output
Downloading: https://vagrantcloud.com/hashicorp/boxes/precise32/versions/1/providers/
virtualbox.box
!
69
69
Demo
Start a Vagrant virtual machine
70
70
Benefits
Scriptable
Reproducible environment
Solid results
Fast results
Repeatable results
71
When?
Day one
Easiest
72

19. “If you don’t do it on Day One,
you probably won’t”
!
Andy Hunt
andy.pragprog.co7m3
73
Easy?
No
Difficult
Necessary
Code
74
Costs Move to Zero
Once scripted…
What changes?
75
What Other Ideas?
76

20. Devs write
code normally
Developers
Code is
shared via a
source code
management
tool
SCM
Continuous
Testing
Providing
immediate
feedback and a
shared
environment for a
QA “First Look”
QA team
members trigger
this deploy
Triggered
Deploy
to QA
A stable
environment for
manual and
exploratory testing
Manual
Testing
77
Developers SCM
Watch
Build
Unit Tests
Deploy
to Dev
Integration
Tests
First
Look
Continuous
Testing
On
Demand
Deploy
to QA
Manual
Testing
Rapid feedback
Pair Dev & QA
78
First Look
• QA can see what’s been coded immediately
• Differences in opinions can be discussed quickly
• Fewer problems during the larger testing cycles
• Keeps QA & Dev working together as partners
• Faster feedback for QA
79
Continuous Testing
• Includes QA in the continuous feedback cycle
• Retains a traditional QA stable testing environment
• Dev & QA should both review all tests (unit & int.)
• QA requests trigger the QA env. deployment
80

21. The Traditional Approach
Write Code
Requirements
Demo
Discuss
Rewrite
Typically a
confrontation &
argument
The demo
forces a
discussion of the
requirements
An isolated event,
primarily worked by
the customer proxy
81
Demo
Write Code
The Agile Approach
Rewrite
Requirements
Discuss before
significant time is
invested
Include the
Golden Triad
Discuss
82
Alternatives
83
Work in a vacuum
flickr.com/photos/lightsinthedark/5810724330/
84

22. Ruthlessly
Automate!
Builds
Deploys
Tests
flickr.com/photos/pmillera4/9317924474
85
Test Automation
Unit tests
Acceptance Tests
86
Unit Tests
One class
One method
Fast
Illuminating
87
Integration Tests
High level
Lots of coverage
No detail
88

23. Continuous Feedback
Large amounts of code
Small amounts of change
Tests run after every change
Requires frequent code checkins
89
Continuous Testing
CI CT
Compile &
unit test
Deploy &
integration
test
90
What's Better?
Unit or Integration?
91
91
Best Is...
Automated
Reliable
Frequently executed
92
92

24. Test Lifecycle
Set up state
Use product
Verify results
(Clean up)
93
93
Demo
Explore Unit tests
Explore Selenium
94
94
Set Your Teams Up
Builds work
Functionality works
Biggest risk?
Requirements
95
The Team
Developers Testers
Customers
96

25. Grid Driven
Development
Remove technical barriers
Everyone speaks “Excel”
97
Demo
Write Code
The Agile Approach
Rewrite
Requirements
Discuss before
significant time is
invested
Include the
Golden Triad
Discuss
98
Testing Strategies
Test First is best
Defect Driven Testing
Blitzkrieg Testing
99
IN Summary...
100

26. Culture Change
Fundamental change in development
101
Steps
Automate builds
Continuous Integration
Add unit tests
Automate deployments
Continuous Delivery
Add integration tests
flickr.com/photos/untitlism/2547423465
102
Your move
103
Exercise
Two 3x5 cards
3 reasons for Continuous Testing
3 barriers to adoption
Groups of 5 to 7
Brainstorm on solutions
flickr.com/photos/curtisperry/6119618990
104

27. Tips
Open standards...
Junit or Junit compatible
Avoid vendor lock in
Glue tools
Time matters....
Fast servers
Incremental change
105
Script Tips
Save scripts in source code management
Modular
Basic encapsulation
One target => One function
106
Script Targets
all depends_on clean, compile, test-unit
clean # deletes build artifacts
compile # builds the product (Ruby?)
test-unit # run your tests
deploy_locally # to your dev environment
deploy_dev (?) # to a shared dev environment
test-integration # run your integration tests
107
Resources
JUnit JUnit.org
Jenkins Jenkins-ci.org
BDD Dannorth.net/introducing-bdd
Cucumber Cukes.info
Continuous Deployment en.wikipedia.org/wiki/Continuous_delivery
108

28. Devops Resources
VagrantUp.com
GetChef.com/chef
PuppetLabs.com
Ansible.com
VirtualBox.org
109
Data Resources
Guides.RubyOnRails.org/active_record_basics.html
Liquibase.org
FlywayDB.org
110
Exercise
Useful or not...
Will you pursue or not...
Continuous Integration
Continuous Testing
Test automation
How will these ideas help or hurt your org?
Write down your answers and pass them up
111
Retrospective
You're the customers
Were your requirements met?
What else needs to be added to this class?
What can we cut out?
112

29. 113