Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Accelerating Your Test Execution Pipeline

136 views

Published on

Our very own Bria Grangard will take you through the ways in which you can speed up your testing process. Check it out to learn about test frameworks, automation, parallel testing and more.

Published in: Software
  • Be the first to comment

Accelerating Your Test Execution Pipeline

  1. 1. Accelerating Your Test Execution Pipeline Thursday, May17th PorteShed,Galway
  2. 2. @Bria_Grangard Who Am I? • SmartBear Software • Automated UI functional testing tools & test management solutions • Stay in Touch! • @Bria_Grangard • Education • Went to Dartmouth: AB in Engineering, BE in Biomedical Engineering, MEM with a healthcare focus • What do I love to do? • Run, dance, play board games (Settlers of Catan anyone?)
  3. 3. Page Today’s Agenda @Bria_Grangard
  4. 4. We provide tools for development, testing, and operations teams to create great software, faster than ever. AccelerateSDLCWorkflows | ImproveQualityatEveryStage | RealizeRapidTime-to-Value • HQ in Boston, MA, USA, with 7 offices globally • Founded in 2009 • Open Source Innovator (Swagger & SoapUI) 6.5M+ Users 194 Countries 22K+ Companies TestComplete SoapUI Pro SwaggerHub CrossBrowserTesting QAComplete AlertSite
  5. 5. UI API Create Great Software, Without Tradeoffs Perform Code & Doc Review Collaborator Design, Develop, & Document APIs SwaggerHub DEV TEST OPS Code Create Automated UI Functional Tests (Web, Desktop, Mobile) TestComplete Script Run Tests On Real Devices in the Cloud CrossBrowserTesting Create Web Load Tests LoadComplete Spec Create Automated API Functional Tests (REST, SOAP, and more) SoapUI Pro Script Virtualize API & Database Services ServiceV Pro Create API Load Tests LoadUI Pro Monitor Web & API Performance, Availability, & Functional Correctness AlertSite Manage Manual & Automated Tests QAComplete Integrations …100 +
  6. 6. | SB Test Page What’s Going on in the Testing World? @Bria_Grangard BDD AI Machine Learning DevOps Shift Left Agile Automation
  7. 7. Page There are bottlenecks in today’s development processes. • Irontriangletrade-off • Teams today are constantly feeling pressure to deliver software faster, without compromising quality • Automationmightrampup • There is only a certain point as to how scalable automation can be • Testenvironmentsareoftentheroot-causeofthebottlenecks • They are very time consuming and costly @Bria_Grangard
  8. 8. Page @Bria_Grangard The promise of the new software delivery cycle WaterfallAgile Design Build Test Implement DevOps Week1 Week2 Week3 Week4
  9. 9. Page @Bria_Grangard Time Consuming Nature Of Web Testing #ofTests 0 200 400 600 800 1000 1200 MVP Feature Set 1 Feature Set 2 Feature Set 3 V2 More Features = More Testing Age Of Product #ofBrowsers 0 2 4 6 8 10 12 MVP Feature Set 1 Feature Set 2 Feature Set 3 V2 Popularity of Product
  10. 10. Page @Bria_Grangard 1 2 3 4 Test Less Hire More Testers Diversify Your Testing Run Tests In Parallel More Bugs Increased Cost
  11. 11. | SB Test Page Now… How Can We Go Faster? @Bria_Grangard 1. Test Frameworks 2. Automation 3. Parallel Testing
  12. 12. Level 1: Test Frameworks @Bria_Grangard
  13. 13. Page The Basics of a Test Framework @Bria_Grangard Requirements Tests Defects What do we make and how should it behave Make sure it works as stated in the Requirement Definition Sets Environments Actual Results do not equal Expected Results
  14. 14. Page @Bria_Grangard What is a Test Framework? A Test Framework: Links tests to other SDLC items Is NOT a Test Automation Framework but often contains one Allows for rapid creation of tests from reusable components Separates data from logic (REUSABILITY) Provides a standardized test “language” and reporting structure for an application under test
  15. 15. Page @Bria_Grangard Elements of a Test Framework • Library: A repository of all your decomposed scripts, separated into their components • Test Data Sources: A repository of all data sources • Helper Functions: A repository of all decomposed test scripts, automated or manual, that are inputs or checks • Test Environments: A list of all covered testing environments, broken out by type (OS, browsers…) • Modules: The combination of library items with any helper functions and test data sources–plus environments • Structure / Hierarchies: The “folder” structure of modules
  16. 16. Level 2: Figure Out What Tests Should be Automated @Bria_Grangard
  17. 17. Page There are many types of testing that need to be done… @Bria_Grangard Browser: Chrome HTML5, Angular JS Network Service/API/Database End-to-End Testing
  18. 18. Page @Bria_Grangard Automated Testing Pyramid Approach UI Service Unit Manual
  19. 19. Page @Bria_Grangard A Little Manual v Automated Math Product v2 # of Test Cases # of Browsers Supported Total Test Cases Avg Test Run Time Product v2 1,000 10 10,000 .5 Min Total Test Time 83 Hrs With 2 QA engineers, That is 1 week of testing # of Test Cases # of Browsers Supported Total Test Cases Avg Test Run Time 1,000 10 10,000 4 Min Total Test Time 666 Hrs With 5 manual testers, that is 3.5 weeks of testing
  20. 20. Page Decide on What to Automate • Environment Setup/Teardown • DataEntry • FormFilling • Varyingdatainputs inarepetitive process • Exposingbackend data(APIs,DBtable,etc.) • Repetitive/boring tasksthat arepronetoinattention errors • Taskswithhighreusevalue acrossmanyworkflows • Testswithtimingorscreenresponsivenessasacriteriaforsuccess • Manynon-functionaltesttypes,suchasperformancetesting • Capturing Results @Bria_Grangard
  21. 21. Page 21 @Bria_Grangard Speeding Up Your Pipeline Longer Time To Test Behind Releases Shorter Manual Testing Record & Replay Unit Testing POM Atomic Testing ContinuousTesting
  22. 22. PagePage But my Dev team says I have days to test, not weeks… @Bria_Grangard
  23. 23. Level 3: Go Faster. @Bria_Grangard
  24. 24. Page Let’s Go Faster! We can,withparalleltesting @Bria_Grangard Test 2Test 2Test 2 Test 3 Test 3 Test 3 Running tests sequentially, we were able to run our tests in 1 week Test 1Test 1 Test 1 Test 1 Test 1 Test 1 Test 2 Test 2 Test 2 Test 3 Test 3 Test 3 With 20 Parallel executions, we can run our entire test suite in only 4 hours
  25. 25. Page Types of test to run in parallel Testingacrossdifferentbrowsersanddevicesisoneofthemosttime consuming aspectsoftestingthefrontendofyourwebsiteorweb application. Runmoretests,against morebrowserconfigurations by running theminparallel. According tothetestingpyramid,Unittestsshould beyourmost abundant testtypeinyourentiretestingsuite.Because ofthis,running 14,000 unittestsinunderanhourisreallyonlypossible withamassive parallel testinginfrastructure investment. Needtogetyourminimumtestingdoneinthenext20minuteswhileyou pushahotfix?Onlywaytodothatistoruntheminparallel, allowing you togetthemosttestingdoneintheshortestamountoftime. Because deployments arehappening atsucharapidpace,regression testingisonofthebestwaystohaveatypeof“testingversioncontrol” makingsurethefunctionality ofthenewbuild, matchesthatofthelast stablebuild. Running thesetestsinparallel allows moretobetested. Cross Browser Testing Unit Testing Regression Testing Smoke Testing Most effectivetests tosee ROI fromfor paralleltesting @Bria_Grangard
  26. 26. Page Benefits of Parallel Testing 1. Quick Deployments 2. Faster Feedback 3. Cross Browser Testing 4. Better Test coverage 5. Saves Valuable Time @Bria_Grangard
  27. 27. | SB Test Page Key Takeaways 1. Testframeworksareimportantandakeytosuccess. 2. Whenautomating—choose whichteststoautomatewisely 3. Paralleltestinglet’syougofaster. @Bria_Grangard
  28. 28. Thank You! @Bria_Grangard

×