Continuous
Testing
J a r e d R i c h a r d s o n
A g i l e A r t i s a n s
1
Who are you?
2
Jared Richardson
CoAuthored Ship It! and Career 2.0	

Screencast editor for PragProg.com	

Started AgileRTP in 2007	

2nd public signatory of the Agile Manifesto	

Agile coach	

AgileArtisans.com
3
What Is Agile?
4
AgileManifesto.org
5
Feedback Loops
6
Feedback Loops are
Fractal
flickr.com/photos/dancentury/4579906149
7
flickr.com/photos/bobowen/4501875907
flickr.com/photos/30691679@N07/3808395213
flickr.com/photos/kevino/1438502858
8
Pairing
Test First
Daily Meetings
Demos
???
Feedback Loops
Inside Feedback Loops
9
Different Levels
Good at the bottom	

Great at the top	

Ignore the middle
10
Continuous Feedback
Continuous Integration	

Continuous Delivery	

Continuous Testing
11
Continuous Integration
Watch Code
Build product
Run tests
Publish results
12
Continuous Testing
Watch CI
Deploy
product
Integration
tests
Publish results
13
Continuous Testing
Build in CI (compile & unit tests)	

Deploy to dev (staging)	

Run integration tests
14
What does this tell us?
If it works
15
Continuous deployment is controversial. 



Most people ... think I’m advocating 

low-quality code 

or an undisciplined 

cowboy-coding 

development process.
ERIC RIES
radar.oreilly.com/2009/03/continuous-deployment-5-eas.html
16
Language is a Tool
Use it
17
What's this look like?
18
Java Tooling
Publish results
19
Example
Publish results
javac (Oracle & IBM)
message broker
etc
ws ejb deploy
message broker tools
sshexec (to scripts)
JUnit
HP's UFT
Selenium
Web page
RSS
Email
20
Continuous Testing
CI CT
Compile &
unit test
Deploy &
integration
test
21
Multiplatform Deploys
CI
CD Win7
CD Linux
CD AIX
22
Binaries
Nuke & Pave
Delete
Recompile
Deploy
Use
23
flickr.com/photos/x-ray_delta_one/4152356464
23
Data
Nuke & Pave
Delete schema
Drop tables
Recreate
Restock
Rails
Liquibase
24
flickr.com/photos/x-ray_delta_one/4152356464
24
Rails ActiveRecord
CREATE	
  TABLE	
  products	
  (	
  
	
  	
  	
  id	
  int(11)	
  NOT	
  NULL	
  auto_increment,	
  
	
  	
  	
  name	
  varchar(255),	
  
	
  	
  	
  PRIMARY	
  KEY	
  	
  (id)	
  
);
25
25
Rails ActiveRecord
change_table :card_deck do |t|

t.remove :description, :name 

t.string :first_name

t.index :part_number

t.rename :list_name, :last_name

end
26
26
Rails ActiveRecord
rake db:migrate
rake db:migrate VERSION=0
rake db:reset
27
27
Operating System
Nuke & Pave
Virtualize
Recreate
28
flickr.com/photos/x-ray_delta_one/4152356464
28
Vagrant
vagrant init lnx http://lnx/lnx.box	
vagrant up	
vagrant ssh	
vagrant reload --provision
29
29
Vagrant
Vagrant.configure("2") do |config|	
config.vm.box = "lnx"	
config.vm.provision :shell, :path => "bootstrap.sh"	
end
30
30
Benefits
Reproducible environment	

Solid results	

Fast results	

Repeatable results
31
Fast
feedback
leads to
fast fixes
32
Minimize Dark Times
Code Change
Validation
http://flickr.com/photos/bogenfreund/556656621/
33
When?
Day one	

Easiest
34
Easy?
No	

Difficult	

Necessary	

Code	

35
Alternatives
36
flickr.com/photos/lightsinthedark/5810724330/
Work in a vacuum
37
Ruthlessly
Automate!
Builds	

Deploys	

Tests
flickr.com/photos/pmillera4/9317924474
38
Steps
Automate builds	

Continuous Integration	

Add unit tests	

Automate deployments	

Continuous Delivery	

Add integration tests
flickr.com/photos/untitlism/2547423465
39
Your move
40
Exercise
Two 3x5 cards	

3 reasons for Continuous Testing	

3 barriers to adoption	

Pass cards to the next table	

Brainstorm on solutions
flickr.com/photos/curtisperry/6119618990
41
Tips
Open standards...	

Junit or Junit compatible	

Avoid vendor lock in	

Glue tools	

Time matters....	

Fast servers	

Incremental change
42
Resources
JUnit JUnit.org	

Jenkins Jenkins-ci.org	

BDD Dannorth.net/introducing-bdd	

Cucumber Cukes.info	

Continuous Deployment en.wikipedia.org/wiki/Continuous_delivery
43
Devops Resources
VagrantUp.com	

GetChef.com/chef	

PuppetLabs.com	

Ansible.com	

VirtualBox.org
44
Data Resources
Guides.RubyOnRails.org/active_record_basics.html	

Liquibase.org	

FlywayDB.org
45 46

Continuous Testing

  • 1.
    Continuous Testing J a re d R i c h a r d s o n A g i l e A r t i s a n s 1 Who are you? 2 Jared Richardson CoAuthored Ship It! and Career 2.0 Screencast editor for PragProg.com Started AgileRTP in 2007 2nd public signatory of the Agile Manifesto Agile coach AgileArtisans.com 3 What Is Agile? 4
  • 2.
    AgileManifesto.org 5 Feedback Loops 6 Feedback Loopsare Fractal flickr.com/photos/dancentury/4579906149 7 flickr.com/photos/bobowen/4501875907 flickr.com/photos/30691679@N07/3808395213 flickr.com/photos/kevino/1438502858 8
  • 3.
    Pairing Test First Daily Meetings Demos ??? FeedbackLoops Inside Feedback Loops 9 Different Levels Good at the bottom Great at the top Ignore the middle 10 Continuous Feedback Continuous Integration Continuous Delivery Continuous Testing 11 Continuous Integration Watch Code Build product Run tests Publish results 12
  • 4.
    Continuous Testing Watch CI Deploy product Integration tests Publishresults 13 Continuous Testing Build in CI (compile & unit tests) Deploy to dev (staging) Run integration tests 14 What does this tell us? If it works 15 Continuous deployment is controversial. 
 
 Most people ... think I’m advocating 
 low-quality code 
 or an undisciplined 
 cowboy-coding 
 development process. ERIC RIES radar.oreilly.com/2009/03/continuous-deployment-5-eas.html 16
  • 5.
    Language is aTool Use it 17 What's this look like? 18 Java Tooling Publish results 19 Example Publish results javac (Oracle & IBM) message broker etc ws ejb deploy message broker tools sshexec (to scripts) JUnit HP's UFT Selenium Web page RSS Email 20
  • 6.
    Continuous Testing CI CT Compile& unit test Deploy & integration test 21 Multiplatform Deploys CI CD Win7 CD Linux CD AIX 22 Binaries Nuke & Pave Delete Recompile Deploy Use 23 flickr.com/photos/x-ray_delta_one/4152356464 23 Data Nuke & Pave Delete schema Drop tables Recreate Restock Rails Liquibase 24 flickr.com/photos/x-ray_delta_one/4152356464 24
  • 7.
    Rails ActiveRecord CREATE  TABLE  products  (        id  int(11)  NOT  NULL  auto_increment,        name  varchar(255),        PRIMARY  KEY    (id)   ); 25 25 Rails ActiveRecord change_table :card_deck do |t|
 t.remove :description, :name 
 t.string :first_name
 t.index :part_number
 t.rename :list_name, :last_name
 end 26 26 Rails ActiveRecord rake db:migrate rake db:migrate VERSION=0 rake db:reset 27 27 Operating System Nuke & Pave Virtualize Recreate 28 flickr.com/photos/x-ray_delta_one/4152356464 28
  • 8.
    Vagrant vagrant init lnxhttp://lnx/lnx.box vagrant up vagrant ssh vagrant reload --provision 29 29 Vagrant Vagrant.configure("2") do |config| config.vm.box = "lnx" config.vm.provision :shell, :path => "bootstrap.sh" end 30 30 Benefits Reproducible environment Solid results Fast results Repeatable results 31 Fast feedback leads to fast fixes 32
  • 9.
    Minimize Dark Times CodeChange Validation http://flickr.com/photos/bogenfreund/556656621/ 33 When? Day one Easiest 34 Easy? No Difficult Necessary Code 35 Alternatives 36
  • 10.
    flickr.com/photos/lightsinthedark/5810724330/ Work in avacuum 37 Ruthlessly Automate! Builds Deploys Tests flickr.com/photos/pmillera4/9317924474 38 Steps Automate builds Continuous Integration Add unit tests Automate deployments Continuous Delivery Add integration tests flickr.com/photos/untitlism/2547423465 39 Your move 40
  • 11.
    Exercise Two 3x5 cards 3reasons for Continuous Testing 3 barriers to adoption Pass cards to the next table Brainstorm on solutions flickr.com/photos/curtisperry/6119618990 41 Tips Open standards... Junit or Junit compatible Avoid vendor lock in Glue tools Time matters.... Fast servers Incremental change 42 Resources JUnit JUnit.org Jenkins Jenkins-ci.org BDD Dannorth.net/introducing-bdd Cucumber Cukes.info Continuous Deployment en.wikipedia.org/wiki/Continuous_delivery 43 Devops Resources VagrantUp.com GetChef.com/chef PuppetLabs.com Ansible.com VirtualBox.org 44
  • 12.