Succesful Testing The Continuous Delivery Process


Published on

Many organisations want to shorten their time-to-market. By implementing agile they try to shorten the delivery time. Agile projects use practices like continuous integration, automated deployment, (acceptance) test-driven development and regression test automation to get to continuous delivery. In this process build, integration, test, and deployment automation plays an important role.

The delivery of software has to become fast and reliable. But what does it take to get to continuous delivery? What are the testing challenges we face in a continuous delivery? Which test strategies can we apply be able to test continuous delivery?

This talk zooms in on fully automated unit testing and a high degree of automation in the business facing testing to get to continuous delivery. It focusses on solutions (tooling) within a continuous delivery model from a testing perspective. It gives the participants insights in the use of requirements management tooling, test solutions built with tooling like Selenium and Fitnesse, JMeter, Appdynamics and other tooling. How can deployment processes be implemented with the use of the right (often lightweight) tooling? Which test strategy fits in this process and how to organize collaboration between developers, testers and operations?

Finally this talk gives practical examples of projects were these solutions were used.

What will be the learning moments for visitors?
* What is continuous delivery?
* What test challenges do we face in continuous delivery?
* Which test and tool strategies can we apply to continuous integration?
* What makes these strategies successful?
* Lessons learned in testing continuous delivery: what worked, what was the context and what didn't work so well?

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Succesful Testing The Continuous Delivery Process

  1. 1. codecentric Nederland BV @huibschoots & @mieldonkers SUCCESFUL TESTING THE CONTINUOUS DELIVERY PROCESS
  2. 2. codecentric Nederland BV INTRODUCTION Miel Donkers Developer @mieldonkers Huib Schoots Tester @huibschoots
  3. 3. codecentric Nederland BV • Experience with Continuous Delivery? • As a tester, do you need to wait for an acceptable release from developers? • Loose time doing all repetitive manual stuff? • Maintain different environments and software versions manually, over-and-over-and-over… • Is this really release 1.6.5? • All sounds familiar? TYPICAL
  4. 4. codecentric Nederland BV Prod BUILDING A RELEASE TEST Local Build Integration server Release build DEV Acc Test Users DEVDEV
  5. 5. codecentric Nederland BV WHY CONTINUOUS DELIVERY
  6. 6. codecentric Nederland BV WHAT IS CONTINUOUS DELIVERY •  Continuous Delivery pipeline •  Automation •  Manual control •  Feedback
  7. 7. codecentric Nederland BV GOALS •  Predictability •  Quality •  Speed
  8. 8. codecentric Nederland BV WHAT ARE THE KEY BENEFITS OF CONTINUOUS DELIVERY? •  Low-risk releases •  Faster return on investment in software projects •  Improvement of competitiveness and responsiveness •  Quality improvement of new software versions
  9. 9. codecentric Nederland BV CONTINUOUS IMPROVEMENT CYCLE Requires: Discipline & Responsibility
  10. 10. codecentric Nederland BV STOP THE LINE!
  11. 11. codecentric Nederland BV PUSH VS. PULL
  12. 12. codecentric Nederland BV ANATOMY IN DETAIL Source Code Commit Stage Compile Commit Tests Assemble Code Analysis Artifact Repository Acceptance Stage Configure Environment Deploy Binaries Smoke Test Acceptance Tests UAT Configure Environment Deploy Binaries Smoke Test Capacity Stage Configure Environment Deploy Binaries Smoke Test Run Capacity Tests Production Configure Environment Deploy Binaries Smoke Test Env.&App Config Env.&App Config Version Control Testers Self-service deployments Operations Perform push Button releases Developers See code metrics and test failures reports binaries metadata binaries reports metadata binaries reports metadata
  13. 13. codecentric Nederland BV WHAT ARE THE MAIN STEPS OF CONTINUOUS DELIVERY? •  Build – compile, unit test, version, package •  Quality – metrics, documentation •  Test – acceptance-, regression- and performance tests •  Provision environments – deployment to test- and staging environment •  Production – green/blue deployment to production
  14. 14. codecentric Nederland BV MAIN STEPS TO CREATE A CONTINUOUS DELIVERY PIPELINE? •  Getting started with virtual environments, e.g. Amazon EC2 •  Configure your Continuous Integration Server •  Provisioning your Test, Staging and Production environments •  Configure your Continuous Delivery Pipeline •  Create a Dashboard of your Systems
  15. 15. codecentric Nederland BV EXAMPLE § Elke stage een slide.
  17. 17. codecentric Nederland BV TOOLING THAT CAN HELP YOU
  18. 18. codecentric Nederland BV EXAMPLE – TECHNOLOGY STACK •  Amazon EC2 (Virtualization, Cloud) •  Jenkins (CI Server) •  Git/Github (Version Control) •  Sonar (Code Quality) •  jUnit (Unit Tests) •  jBehave (Acceptancetests) •  Selenium (UI Tests) •  Puppet (Provisioning) •  Tomcat (Application Server) •  Artifactory (Artifact Repository) •  Maven (Build Tool) •  jMeter (Performance Tests) •  AppDynamics (Performance Tests) •  Liquibase (DB Versioning)
  19. 19. codecentric Nederland BV OUR SETUP CD Server Test Server User Acceptance Test Server Production Cluster Blue / Green CD Slave
  20. 20. codecentric Nederland BV EXAMPLE – DELIVERY PIPELINE 1/2 Push to Git Repo 1 Provision-Deploy Acceptance Commit Trigger CD Pipeline 2 Build Source Code 3 Run Unit Tests 4 Start EC2 Image 1 Provisioning 2 DB Create Update 3 Deploy WAR 4 Start Tomcat 5 Start Selenium 1 Acceptance Test Run AATs 2 Version Git Repo 6 Release WAR 7 Run Code Analyse 5
  21. 21. codecentric Nederland BV EXAMPLE – DELIVERY PIPELINE 2/2 Run JMeter 1 Provision-Deploy UAT Start EC2 Image 2 Provisioning 3 DB Create Update 4 Deploy WAR 5 Start Tomcat 6 Performance Test Select Version 1 Provision-Deploy PRODBLUE/GREEN Start EC2 Image 2 Provisioning 3 DB Update 4 Deploy WAR 5 Start Tomcat 6 Select Version 1
  22. 22. codecentric Nederland BV ASK YOURSELF •  Which parts are tested how? •  What remains to be tested?
  23. 23. codecentric Nederland BV APPLICATION STACK Apps / Services / Components Application Configuration Middleware Middleware Configuration Operating System OS Configuration Hardware TESTING
  24. 24. codecentric Nederland BV WHAT DO WE TEST WHERE AND WHEN? •  Automation is key •  Do what we do best •  Team effort
  25. 25. codecentric Nederland BV MANUAL TESTING Still required: •  “Show me” •  Exploratory testing •  User acceptance testing
  26. 26. codecentric Nederland BV ANATOMY OF A DEPLOYMENT PIPELINE Release •  Every change results in a trigger of the deployment process •  Software is build once and only once •  The same deployment process for every environment •  Deployment in production-like environments Commit Automated Acceptance Test Automated Capacity Test Manual Testing Fast Feedback Trust in Application Stability Done
  27. 27. codecentric Nederland BV EXAMPLE § Elke stage een slide.
  28. 28. codecentric Nederland BV SUMMARY •  Continuous delivery is a way to create a stable release process •  Automated testing makes continuous delivery possible •  Manual testing is (still) necessary •  Pull vs Push •  Dashboard
  29. 29. codecentric Nederland BV THANK YOU FOR YOUR ATTENTION – ANY QUESTIONS?
  30. 30. codecentric Nederland BV +31 (0) 6 24 64 10 33 @huibschoots +31 (0) 6 51 19 77 38 @mieldonkers See our website for more information: CONTACT INFORMATION