Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Continuous Testing: Preparing for DevOps

1,229 views

Published on

by Ingo Philipp, Distinguished Evangelist, Tricentis at STeP-IN SUMMIT 2018 - 15th International Conference on Software Testing on August 30, 2018 at Taj, MG Road, Bengaluru

Published in: Technology
  • Be the first to comment

Continuous Testing: Preparing for DevOps

  1. 1. Continuous Testing Digital Transformation Requires Continuous Testing Ingo Philipp
  2. 2. Businesses must continuously exploit digital technologies to both create new sources of customer value and increase operational agility in service of customers. © Forrester Research
  3. 3. Digital Disruption Across industries, companies face the challenge of software-led transformation
  4. 4. Plan Code Version Build Provision Test Deploy Monitor © Melissa Perri The product is a solution. If the problem isn't solved, the product doesn't work. The goal is to accelerate this value exchange Identify Needs Receive Money Build Solution Your Company Your Customer
  5. 5. Requirements Design Implementation Testing Acceptance Deployment Inception Elaboration Construction Transition Development Operations DevOps Agile Horizon Digital Singularity Incremental Rational Unified Process 4 Agile Scrum, Kanban 2 Continuous DevOps 1 6+ Big Bang Waterfall Development Response Flexible Framework Rigid Rules Following a Plan Contract Negotiation Copious Documentation Processes & Tools Individuals & Interactions Working Software Customer Collaboration Responding to Change
  6. 6. It’s not the strongest that survive, nor the most intelligent, but the one most responsive to change Charles Darwin Because software is not written, it is rewritten
  7. 7. 80% Manual Testing 20% Test Automation The reliance on manual testing is the top technical challenge in app development World Quality Report, 2015/16 Test automation requires developers UI LoadAPI Present Traditional Testing
  8. 8. Future Continuous Testing +85% Test Automation Exploratory Tests Automated UI Tests API Tests Load Tests 80% Manual Testing 20% Test Automation Present Traditional Testing Orchestrated Service Virtualization
  9. 9. 100% 0% Sprint Start Sprint End API UI Start testing early, shift left test automation Degree of Completion Time Efficiency Gain +4x Creation +6x Maintenance +20x Execution
  10. 10. *voke, market snapshot report on service virtualization - 2012 On average, organizations require access to 33systems for development or testing. 18Average # of systems with unrestricted access 96% of testers have restricted test lab access Testing Present Enterprise system landscapes are alike disease gene networks. Welcome to the tester’s hell
  11. 11. Automation doesn't make testing easy, it makes testing possible Lesson Learned © Wolfgang Platz
  12. 12. A B Visiting London ~100 Billion Billion Billion Billion Billion Billion Billion Combinations = 50
  13. 13. Testing harder isn’t the answer, testing smarter is! Lesson Learned © Wolfgang Platz
  14. 14. Test Case
  15. 15. Passing Test Case
  16. 16. Passing Test Case @iamdevloper
  17. 17. © BBC
  18. 18. Testing is exactly like washing a pig. Because it’s messy. It has no rules. No clear beginning, middle, or end. It’s kind of a pain in the ass, and when you’re done you’re not sure if the pig is really clean or even why you were washing a pig in the first place. © Luke Sullivan © BBC
  19. 19. Testing is exactly like washing a pig. Because it’s messy. It has no rules. No clear beginning, middle, or end. It’s kind of a pain in the ass, and when you’re done you’re not sure if the pig is really clean or even why you were washing a pig in the first place. © Luke Sullivan That’s what we can know
  20. 20. Testing is exactly like washing a pig. Because it’s messy. It has no rules. No clear beginning, middle, or end. It’s kind of a pain in the ass, and when you’re done you’re not sure if the pig is really clean or even why you were washing a pig in the first place. © Luke Sullivan That’s what we can know That’s what we know That’s what we check Problem Knowledge Gap © Richard Bradshaw
  21. 21. © Cem Kaner © Richard Bradshaw The purpose of testing is to close the knowledge gap The goal is information, not gratuitous automation Testing is and always will be a search for information That’s what we can know That’s what we know That’s what we check Problem Knowledge Gap
  22. 22. That’s what we can know That’s what we know That’s what we check Problem Knowledge Gap Checking « Requires Processing » © Rich Rogers Verify through Instructions Pay attention to Deviations Create Test Cases Follow Procedure Examine Requirements Factory Process
  23. 23. » Here You Monitor Known Risks That’s what we can know That’s what we know That’s what we check Problem Knowledge Gap Confirmatory Testing « Requires Processing » © Rich Rogers Verify through Instructions Pay attention to Deviations Create Test Cases Follow Procedure Examine Requirements Factory Process MechanicalProcess
  24. 24. Confirmatory Testing « Requires Processing » © Rich Rogers Verify through Instructions Pay attention to Deviations Create Test Cases Follow Procedure Examine Requirements Factory Process Exploratory Testing « Requires Thinking » Investigate through Experiments Pay attention to Oracles Create Test Ideas Follow Clues Examine Risks Adaptive Investigation » Here You Analyze Potential Risks CognitiveProcess MechanicalProcess » Here You Monitor Known Risks
  25. 25. Checking Efficient Confirmatory Testing Exploring Effective Exploratory Testing Testing Thorough Testing + = Agile Testing Equation © Elisabeth Hendrickson
  26. 26. If this doesn’t make an impact, then its absence doesn’t make a difference Our Proof © Trey Smith
  27. 27. 100% Manual Testing 0% Automated Testing Testing Present There’s a better way to do it. Find it! Project Charter 11 Manual Testers 4755 Manual Test Cases ? Unknown Coverage 10 Weeks Execution
  28. 28. Risk Coverage Optimization Test Data Management UI/API Test Automation Continuous Integration & Distributed Execution 48 Hours Execution Single Agent 8 Hours Execution Multiple Agents 11 Manual Testers 4755 Manual Test Cases ? Unknown Coverage 10 Weeks Execution 89% Business Risk Coverage 1193 Manual Test Cases 37% Business Risk 5 Weeks Execution 2.5 Weeks Execution 92% Automated Test Cases 53% Business Risk Coverage Smoke Testing 34 Minutes Execution Smoke Testing Test Case Count 7% 75% Redundancy 50% Effort Test Data 64% API Tests
  29. 29. 8 Hours Execution Multiple Agents 72% Critical Defect Reduction Production Defects 89% Business Risk Coverage 1193 Manual Test Cases 5 Weeks Execution 2.5 Weeks Execution 92% Automated Test Cases 34 Minutes Execution Smoke Testing 53% Business Risk Coverage Smoke Testing Test Case Count 7% Test Case Design & Exploratory Testing 64% API Tests Risk Coverage Optimization Test Data Management UI/API Test Automation Continuous Integration & Distributed Execution
  30. 30. The future doesn’t just happen, it gets happened, so make it happen Conclusion © Paul Tagliablue
  31. 31. Questions The show is over. It’s your turn. ?

×