SUCCESFUL TESTING
THE CONTINUOUS DELIVERY PROCESS
@mieldonkers
codecentric Nederland BV
INTRODUCTION
Miel Donkers
Developer
@mieldonkers

codecentric Nederland BV
TYPICAL
•  As a tester, do you need to wait for an
acceptable release from developers?
•  Loose time doing all repetitive ...
WHY CONTINUOUS DELIVERY

codecentric Nederland BV
codecentric Nederland BV
WHAT IS CONTINUOUS DELIVERY

•  Continuous Delivery pipeline
•  Automation
•  Manual control
•  Feedback

codecentric Nede...
GOALS
•  Predictability
•  Quality
•  Speed

codecentric Nederland BV
WHAT ARE THE MAIN STEPS OF CONTINUOUS DELIVERY?
•  Build – compile, unit test, version, package
•  Quality – metrics, docu...
PUSH VS. PULL

codecentric Nederland BV
STOP THE LINE!

codecentric Nederland BV
ROLE OF THE TESTER
Prod
Acc
Local
Build

Integration
server

Release
build

Test
Users

DEV

codecentric Nederland BV

DEV...
ANATOMY OF A DEPLOYMENT PIPELINE

Fast Feedback

Commit

Automated
Acceptance Test

Automated
Capacity Test

Manual
Testin...
COST OF TESTING

COST

FEATURES BUILT
codecentric Nederland BV
ASK YOURSELF
•  Which parts are tested how?
•  What remains to be tested?

codecentric Nederland BV
APPLICATION STACK

Apps / Services
/ Components

Application
Configuration

Middleware

Middleware
Configuration

Operatin...
codecentric Nederland BV
WHAT DO WE TEST WHERE AND WHEN?
•  Automation is key
•  Do what we do best
•  Team effort

codecentric Nederland BV
MANUAL TESTING
Still required:
•  "Show me"
•  Exploratory testing
•  User acceptance testing

codecentric Nederland BV
Examples

codecentric Nederland BV
OUR SETUP

https://github.com/...

Production Cluster
Blue / Green

User Acceptance
Test Server
CD Server

CD Slave

codec...
EXAMPLE – DELIVERY PIPELINE 1/2
Provision-Deploy
Acceptance

Commit
Push to Git Repo
Trigger CD Pipeline
Build Source Code...
EXAMPLE – DELIVERY PIPELINE 2/2
Performance
Test
Run JMeter

Provision-Deploy
UAT
1

Select Version
Start EC2 Image
Provis...
STEPS VISUALIZED

codecentric Nederland BV
codecentric Nederland BV
codecentric Nederland BV
EXAMPLE

§  Elke stage een slide.

codecentric Nederland BV
ANATOMY IN DETAIL
Env.&App
Config

Source
Code

Version Control

Testers
Self-service
deployments

Developers
See code met...
codecentric Nederland BV
codecentric Nederland BV
codecentric Nederland BV
TOOLING THAT CAN HELP YOU

#!/bin/sh!

codecentric Nederland BV
codecentric Nederland BV

Angelo Storari of Ancona, Italy
HIGHLIGHTS

• 
• 
• 
• 

Fast Feedback & Dashboards
Manual testing is (still) necessary
But try to automate as much as pos...
THANK YOU FOR YOUR ATTENTION – ANY QUESTIONS?

codecentric Nederland BV
CONTACT INFORMATION
miel.donkers@codecentric.nl
@mieldonkers

See our website for more information:
http://www.codecentric...
TEST QUADRANT
Brian Marick

codecentric Nederland BV
CONTINUOUS IMPROVEMENT CYCLE
Requires:
Discipline &
Responsibility

codecentric Nederland BV
WHAT ARE THE KEY BENEFITS OF CONTINUOUS DELIVERY?
•  Low-risk releases
•  Faster return on investment in software projects...
EXAMPLE – TECHNOLOGY STACK
• 
• 
• 
• 
• 
• 
• 

Amazon EC2 (Virtualization, Cloud)
Jenkins (CI Server)
Git/Github (Versio...
Upcoming SlideShare
Loading in …5
×

Successful testing continuous delivery - coding serbia 2013

635 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
635
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Successful testing continuous delivery - coding serbia 2013

  1. 1. SUCCESFUL TESTING THE CONTINUOUS DELIVERY PROCESS @mieldonkers codecentric Nederland BV
  2. 2. INTRODUCTION Miel Donkers Developer @mieldonkers codecentric Nederland BV
  3. 3. TYPICAL •  As a tester, do you need to wait for an acceptable release from developers? •  Loose time doing all repetitive manual stuff? •  Maintain different environments and software versions manually, over-and-over-and-over… •  Didn't we fix this issue three months ago? •  Is this really release 1.6.5? All sounds familiar? codecentric Nederland BV
  4. 4. WHY CONTINUOUS DELIVERY codecentric Nederland BV
  5. 5. codecentric Nederland BV
  6. 6. WHAT IS CONTINUOUS DELIVERY •  Continuous Delivery pipeline •  Automation •  Manual control •  Feedback codecentric Nederland BV
  7. 7. GOALS •  Predictability •  Quality •  Speed codecentric Nederland BV
  8. 8. WHAT ARE THE MAIN STEPS OF CONTINUOUS DELIVERY? •  Build – compile, unit test, version, package •  Quality – metrics, documentation •  Test – acceptance-, regression- and performance tests •  Provision environments – deployment to test- and staging environment •  Production – green/blue deployment to production codecentric Nederland BV
  9. 9. PUSH VS. PULL codecentric Nederland BV
  10. 10. STOP THE LINE! codecentric Nederland BV
  11. 11. ROLE OF THE TESTER Prod Acc Local Build Integration server Release build Test Users DEV codecentric Nederland BV DEV DEV TEST
  12. 12. ANATOMY OF A DEPLOYMENT PIPELINE Fast Feedback Commit Automated Acceptance Test Automated Capacity Test Manual Testing Done Release Trust in Application Stability •  •  •  •  Every change results in a trigger of the deployment process Software is build once and only once The same deployment process for every environment Deployment in production-like environments codecentric Nederland BV
  13. 13. COST OF TESTING COST FEATURES BUILT codecentric Nederland BV
  14. 14. ASK YOURSELF •  Which parts are tested how? •  What remains to be tested? codecentric Nederland BV
  15. 15. APPLICATION STACK Apps / Services / Components Application Configuration Middleware Middleware Configuration Operating System OS Configuration Hardware codecentric Nederland BV TESTING
  16. 16. codecentric Nederland BV
  17. 17. WHAT DO WE TEST WHERE AND WHEN? •  Automation is key •  Do what we do best •  Team effort codecentric Nederland BV
  18. 18. MANUAL TESTING Still required: •  "Show me" •  Exploratory testing •  User acceptance testing codecentric Nederland BV
  19. 19. Examples codecentric Nederland BV
  20. 20. OUR SETUP https://github.com/... Production Cluster Blue / Green User Acceptance Test Server CD Server CD Slave codecentric Nederland BV Test Server
  21. 21. EXAMPLE – DELIVERY PIPELINE 1/2 Provision-Deploy Acceptance Commit Push to Git Repo Trigger CD Pipeline Build Source Code Run Unit Tests Run Code Analyse Version Git Repo Release WAR codecentric Nederland BV 1 Start EC2 Image 2 Provisioning 3 DB Create Update 4 Deploy WAR 5 Start Tomcat 6 7 Acceptance Test 1 2 3 4 5 Start Selenium Run AATs 1 2
  22. 22. EXAMPLE – DELIVERY PIPELINE 2/2 Performance Test Run JMeter Provision-Deploy UAT 1 Select Version Start EC2 Image Provisioning DB Create Update Deploy WAR Start Tomcat codecentric Nederland BV Provision-Deploy PRODBLUE/GREEN 1 2 3 4 5 6 Select Version Start EC2 Image Provisioning DB Update Deploy WAR Start Tomcat 1 2 3 4 5 6
  23. 23. STEPS VISUALIZED codecentric Nederland BV
  24. 24. codecentric Nederland BV
  25. 25. codecentric Nederland BV
  26. 26. EXAMPLE §  Elke stage een slide. codecentric Nederland BV
  27. 27. ANATOMY IN DETAIL Env.&App Config Source Code Version Control Testers Self-service deployments Developers See code metrics and test failures Commit Stage Configure Environment Deploy Binaries Smoke Test Acceptance Tests UAT Configure Environment Deploy Binaries Smoke Test Acceptance Stage Compile Commit Tests Assemble Code Analysis Env.&App Config reports binaries metadata binaries reports metadata Operations Perform push Button releases binaries Artifact Repository codecentric Nederland BV Capacity Stage Configure Environment Deploy Binaries Smoke Test Run Capacity Tests Production Configure Environment Deploy Binaries Smoke Test reports metadata
  28. 28. codecentric Nederland BV
  29. 29. codecentric Nederland BV
  30. 30. codecentric Nederland BV
  31. 31. TOOLING THAT CAN HELP YOU #!/bin/sh! codecentric Nederland BV
  32. 32. codecentric Nederland BV Angelo Storari of Ancona, Italy
  33. 33. HIGHLIGHTS •  •  •  •  Fast Feedback & Dashboards Manual testing is (still) necessary But try to automate as much as possible Learn from each other codecentric Nederland BV
  34. 34. THANK YOU FOR YOUR ATTENTION – ANY QUESTIONS? codecentric Nederland BV
  35. 35. CONTACT INFORMATION miel.donkers@codecentric.nl @mieldonkers See our website for more information: http://www.codecentric.nl/portfolio/continuous-delivery/ codecentric Nederland BV
  36. 36. TEST QUADRANT Brian Marick codecentric Nederland BV
  37. 37. CONTINUOUS IMPROVEMENT CYCLE Requires: Discipline & Responsibility codecentric Nederland BV
  38. 38. WHAT ARE THE KEY BENEFITS OF CONTINUOUS DELIVERY? •  Low-risk releases •  Faster return on investment in software projects •  Improvement of competitiveness and responsiveness •  Quality improvement of new software versions codecentric Nederland BV
  39. 39. EXAMPLE – TECHNOLOGY STACK •  •  •  •  •  •  •  Amazon EC2 (Virtualization, Cloud) Jenkins (CI Server) Git/Github (Version Control) Sonar (Code Quality) jUnit (Unit Tests) jBehave (Acceptancetests) Selenium (UI Tests) codecentric Nederland BV •  •  •  •  •  •  •  Puppet (Provisioning) Tomcat (Application Server) Artifactory (Artifact Repository) Maven (Build Tool) jMeter (Performance Tests) AppDynamics (Performance Tests) Liquibase (DB Versioning)

×