DevOps: Where in the World Is Test?


Published on

As the world of software development changes, software testing organizations are challenged to be more innovative to match the speed at which software releases are being deployed. The new software industry buzzword is DevOps; so you might wonder if your software testing organization is still important and how it fits in to this new industry trend. Erik Stensland shares his research into what the DevOps model is, the three ways of implementing DevOps, testing solutions for DevOps, and the benefits of DevOps. Erik discusses the major challenges of a DevOps test team and offers solutions to elevate your own testing automation to become part of the daily-automated deployment process. With a real-world example, see how Erik helped Pearson’s engineering team transform itself through technology and new ideas to successfully build a DevOps team that focuses on reliability, repeatability, and quality of features released to market.

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

DevOps: Where in the World Is Test?

  1. 1. T12 Special Topics 5/8/2014 11:15:00 AM DevOps: Where in the World Is Test? Presented by: Erik Stensland Pearson Learning Technology Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ ∙
  2. 2. Erik Stensland Pearson Learning Technology Software quality architect for Pearson Learning Technologies, Erik Stensland is leading the technical testing efforts with a concentration on functional, performance, and integration testing. In the software test and quality engineering field for more than twenty years, Erik focuses on implementing innovative ideas around process and tools to help teams become more effective and efficient. His previous roles have included senior QA management positions and QA consultant for Internet startups and DoD industries. A board member for the Software Quality Association of Denver, Erik has been an international presenter at leading testing conferences in Europe and the United States. Follow Erik at and on Twitter @erikstensland60.
  3. 3. 4/26/2014 1 DevOps: Where in the World is Test? StarEast 2014 Conference Date: May 8th, 2014 Author: Erik Stensland A Little about me Blog: LinkedIn : Twitter : email:
  4. 4. 4/26/2014 2 What “IF” ? Agenda • What is DevOps • DevOps 1, 2, 3 • Challenges for QA • Take Away’s • Case Study
  5. 5. 4/26/2014 3 First Thought! The Backyard brawlers Then Later .
  6. 6. 4/26/2014 4 Dev Ops
  7. 7. 4/26/2014 5 TEST/QA Finally!!!
  8. 8. 4/26/2014 6 What is DevOps Where did it come from?
  9. 9. 4/26/2014 7 DevOps Principles Benefits of DevOps Faster time to market (reduced cycle times and higher deploy rates) Increased Quality (i.e., increased availability, increased change success rate, fewer failures) Increased organizational effectiveness (increased time spent on value adding activities vs. waste, increased amount of value being delivered to the customer).
  10. 10. 4/26/2014 8 Questions? The Challenge for QA High deployment rates typically associated with DevOps work streams will often put enormous pressure on QA.
  11. 11. 4/26/2014 9 No longer to find defects, necessarily.
  12. 12. 4/26/2014 10 Prevent defects Methodology
  13. 13. 4/26/2014 11 Automation/Measurement Culture / People
  14. 14. 4/26/2014 12 Keys to Test/QA in DevOps Your purpose is to Prevent Defects. Focus lies on Risks to the product instead of just the functionality of the software changes. QA must be testing early and often using “Walk Away Automation” to Integrate QA into the automated deployment flow. Team members need to have the “Right Attitude” The Case Study: Code Name - Manhattan
  15. 15. 4/26/2014 13 What is Manhattan YESTERDAY
  16. 16. 4/26/2014 14 Software Engineering • Few Builds a week. ~1 Deployment every week or two. • Limited Unit testing • Manually Build and Deploy to Pre-Prod Software Quality Engineering • Very little architecture review for testability. • Heavy Manual testing. • Limited UI automated tests, Little web services automated tests. • Automated test manually kicked off. • Some performance testing. IT Operations • Manual recompiling of Code for different source code repository • Manual Deployments and Configurations of Production servers • Limited monitoring of systems , No monitoring of Applications • No review of Systems before Code is delivered for Deployment Yesterday’s Environment The “Discussion”
  17. 17. 4/26/2014 15 Extend Eng into Production • Created a Continuous Integration and Release process using Jenkins. • Deeply integrating QA into the release process by creating “walk away automation”. • Ensure production readiness of the Code as well as the Environment by spinning up new environments before code deployment and testing.
  18. 18. 4/26/2014 16 Production Feedback into Eng • Implemented Systems and Application Monitoring in pre-prod and prod. • Use one incident reporting system for Dev, QA and Ops to monitor during production releases and incidents. • Created blameless postmortems for all releases and incidents. Embed Eng into IT Ops • Development and QA resources added to the rotation for production support. Embed IT Ops into Eng • Created reusable stories for non-functional such as Monitoring, Environment creations.
  19. 19. 4/26/2014 17 TODAY – Software Eng • 100’s of Builds a day. • Large amounts of Unit tests • Multiple Deploys a day to pre-prod • Push Button Deploys to all pre-prod environments. TODAY – Test/QA • Same Repository for Tests as Code • Large sets of Automated tests. • New feature Exploratory testing. • Low maintenance automation tests. • Continuously running all Automated Tests in All Pre-Prod environments.
  20. 20. 4/26/2014 18 TODAY – IT Operations • Same Repository for Code and Environment scripts. • Push Button Deployments to Production. • Monitoring at system and application on all new deployments. Ask Yourself What “IF” ?
  21. 21. 4/26/2014 19 Give away questions? What is the name of the Band I showed in the Video? How much did the Krupp Bagger 288 weigh?