Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

5 Steps to Jump Start Your Test Automation

339 views

Published on

With the acceleration of software creation and delivery, test activities must align to the new tempo. Developers need immediate feedback to be efficient and correct defects as those are introduced. The path to achieving this vision is to build a reliable and scalable continuous test solution.

All beginnings are hard. Having a well-defined plan outlining the approach for your organization to create test automation is key to ensure long term success. Join Diego Molina, Senior Software Engineer at Sauce Labs as he discusses:

The importance of setting up the team correctly from the start
Choosing the right Testing Framework for your organization
Identifying the right scenarios and workflows to test
Learning to avoid common pitfalls at the beginning of the transformation journey

Published in: Technology
  • As a single mother every little bit counts! This has been such a great way for me to earn extra money. As a single mother every little bit counts! Finally, a vehicle for making some honest to goodness real money to make life easier and happier now that I don't have to pull my hair out budgeting every penny every day.Thanks for the rainbow in my sky. ■■■ http://ishbv.com/goldops777/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Your GCSE Maths program has helped me immensely in maths. I am much more confident with this subject and I'm striving for better grades. I really appreciate the time you took in making this program because it has boosted many students self-confidence with their exams. ◆◆◆ http://t.cn/AirraVnG
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1url.pw/Gj6fT ◀ ◀ ◀ ◀
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

5 Steps to Jump Start Your Test Automation

  1. 1. 5 STEPS TO JUMP-START YOUR TEST AUTOMATION DIEGO MOLINA, SENIOR SOFTWARE ENGINEER MAY 28, 2019
  2. 2. Who Am I? • OSS & Selenium Enthusiast • Love testing • Selenium contributor & committer • Co-creator and maintainer docker-selenium and Zalenium projects • Sr. Software Engineer @ diemol diegofmolina • Will be a the next Appium Conference in Bangalore, June 13-15 2
  3. 3. E D C B A Team Setup Testing Framework Test the Right Thing When to Run Tests Get People to Join You
  4. 4. STORY TIME Photo by Ehud Neuhaus on Unsplash
  5. 5. My Story 5 Testing was done by end users Wrote tests for every single use case Oh, I have too many tests! Built my own test infrastructure Hmm, I can add more tests! Oh, I have too many tests!
  6. 6. Why Automate? 6
  7. 7. Why Automate? 7 Software Development has changed, we are faster now Proprietary Single Stack On-Premise Open Source Best of Breed Cloud-based Tools Process Waterfall Fast Waterfall Continuous Integration Continuous Delivery Speed of Releases On DemandDailyWeeklyMonthlyQuarterlyAnnually Desktop/Packaged Apps Thick-Client Apps Web Apps Mobile Apps Desktop Web Responsive Web Apps
  8. 8. Photo by Mpho Mojapelo on Unsplash TEAM SETUP
  9. 9. Testers are in a separate team • Languages with an easy setup and a simple entry level, like Python or Ruby 9 Jumping into automation? What programming language should we use? PYPL PopularitY of Programming Language
  10. 10. Testers are in a separate team • Languages with an easy setup and a simple entry level, like Python or Ruby 10 Jumping into automation? What programming language should we use? Top languages over time - GitHub Octoverse StackOverflow Survey 2019
  11. 11. Testers are in a separate team • Aim for great communication with the Product Owner and the Dev team • Get access the story|task|feature definitions to create tests • Get business context from Product Owners to prioritise activities • Get as much information as possible (tech stack, system architecture, etc…) • Go step by step when implementing the first automated tests • Be in synch with the Dev team • Define a bug report flow • Agree on a working pace 11
  12. 12. Testing Role is part of the team • One or more team members have the role of guiding the team in testing • App code and test code use the same programming language • Leverages team collaboration • Testing has the good practices used in the software development lifecycle • Test automation is also a software project • Test code is in the same repository where the App code is • Simplifies flows to write tests for features that are not yet in production 12
  13. 13. Independently from the team setup • Invest time looking for a test framework that helps the team move faster • More about this on the next slides • Test design and implementation are a team task • Document how: • Tests are structured • Run, add and remove tests • Pair program and do code reviews • Treat test code as production code • Continuously evaluate if the existing test cases deliver enough value 13
  14. 14. How the Team Setup should evolve 14 Continuous Integration Full adoption of Agile Automated testing dominates; manual for debugging or exploratory Dev. and QA collaborate closely Fast Waterfall Initial adoption of Agile Automated testing begins Dev. & QA start communicate Waterfall Traditional sequential design model Manual testing dominates Dev. & QA completely separate Process People Tools Continuous Delivery Fully automated Development process Automated testing core to Dev. & Delivery Manual for exploratory Dev. and QA functions merge
  15. 15. TESTING FRAMEWORK Photo by Dayne Topkin on Unsplash
  16. 16. Why a testing framework? Reduce maintenance costs Be Faster Be Reactive 16
  17. 17. 17 Assertions on Actions Initialization and Cleanup Data Modeling/Mocking Configuration Site Modeling Abstractions Wrappers and Helpers API Usage Future ready features Local and Cloud setups Speed Debugging features Cross browser Simulators-Emulators/Real Devices Built in reporting or easy to plug in What features should my testing framework have?
  18. 18. How do I build my testing framework? By using Open Source WebDriverIO Watir Selenide Nerodia Allure GHP Reporter WireMock Mockoon Json-Server Mimesis Faker(Ruby, Python, JS) Mockaroo (and many others) 18
  19. 19. How to choose an Open Source tool? Programming language support Check ★s and forks Check activity (issues/PRs) Help/Support channels Documentation Involve the whole team Do a Proof of Concept 19
  20. 20. Photo by Matt Briney on Unsplash TEST THE RIGHT THING
  21. 21. Should we automate everything? 21 https://xkcd.com/1319/
  22. 22. What is the Right Thing to test? • Identify main application flows, the ones that must always work • A clear understanding of the flows is needed, inputs/outputs must be clear. • How bad is it if this feature/behaviour breaks? • How much value does the test have? How big is the risk that mitigates? • Analytics can help you to get more answers 22
  23. 23. 23 Is login working? Is registration working? Are errors shown for wrong passwords? Are ads displayed? Are emails being sent? Can items be added to the basket? Can users change their information? Can users pay? Can users add new payment methods? Are product images displayed? Is the page responsive? Can users logout? Example Online Shopping Site
  24. 24. 24 Is login working? Is registration working? Are errors shown for wrong passwords? Are ads displayed? Are emails being sent? Can items be added to the basket? Can users change their information? Can users pay? Can users add new payment methods? Are product images displayed? Is the page responsive? Can users logout? Example Online Shopping Site
  25. 25. How Analytics could help 25 Browsers
  26. 26. How Analytics could help 26 Browsers/Platforms
  27. 27. How Analytics could help 27 Browser Versions
  28. 28. How Analytics could help 28 Browser/Screen Resolution
  29. 29. How Analytics could help 29 Device Details / Platforms
  30. 30. How many tests are enough? • To start, we identified 5 main flows. Automating them gives us: • A lot of value because most of the users go through them • Security because they lower the risk of breaking something important • Based on Analytics, we will run them in: • 3 different browsers (Chrome, Firefox, Safari) • 2 different OS (Windows, OSX) • 3 different screen resolutions • 7 different browser versions (Chrome 71-74, Firefox 64-65, Safari 12) • 120 + 60 + 15 = 195 tests • Automate high value tests 30
  31. 31. Use good practices • Atomic and focused tests • Readable, faster and more stable • Sauce Labs Continuous Testing Benchmark report 31 https://info.saucelabs.com/sauce-labs-continuous-testing-benchmark-report-M.html
  32. 32. Use good practices • Focus on reusability and maintainability • Avoid code duplication across tests and helper classes/methods • Every test must be autonomous • Tests can run in any order without depending on each other • Write tests and code only for the current requirements • Avoid complex designs that consider potential future use cases • Overengineering is overkilling • Get familiar with software design patterns • They can benefit automation testing as well • Base your work on testing plans and/or strategies, not on tools 32
  33. 33. Photo by Maximilian Weisbecker on Unsplash WHEN (AND WHERE) TO RUN TESTS
  34. 34. It all depends on the stage where we are 34 Continuous Integration Full adoption of Agile Automated testing dominates; manual for debugging or exploratory Dev. and QA collaborate closely Fast Waterfall Initial adoption of Agile Automated testing begins Dev. & QA start communicate Waterfall Traditional sequential design model Manual testing dominates Dev. & QA completely separate Process People Tools Continuous Delivery Fully automated Development process Automated testing core to Dev. & Delivery Manual for exploratory Dev. and QA functions merge
  35. 35. Early stages... 35 Process People Tools When Big pre-release session Bug hunting sessions Regular validation cycles Where Locally on desktops or in the cloud Shared testing environments Why Not enough time to test more often due to the nature of manual testing Waterfall Traditional sequential design model Manual testing dominates Dev. & QA completely separate
  36. 36. Early automation stage... 36 Process People Tools One or more times per day Triggered manually or on schedule based Combined with manual before releasing Locally on desktops, own Grid or in the cloud Initial setup of a CI server to run on schedule Shared testing environments Fast Waterfall Initial adoption of Agile Automated testing begins Dev. & QA start communicate When Where Why While gaining confidence in automation, results need to be triaged and use that to improve test resilience and confidence. Automation starts to get relevant but it is not part of the main pipeline yet.
  37. 37. Automation dominates... 37 Process People Tools On every commit and pull/merge request After merging Cloud CI server integrated to development flow Managed testing environment for SUT Continuous Integration Full adoption of Agile Automated testing dominates; manual for debugging or exploratory Dev. and QA collaborate closely When Where Why Fast feedback through automated testing Automated testing is now part of the Quality Gate Cloud - infrastructure is vital for success. A malfunctioning infrastructure slows down the development process.
  38. 38. Automation in everything... 38 Process People Tools When On pre-commit, commit, and merge/pull requests After merging After release Cloud CI server integrated to development flow Dedicated testing environment for SUT Production Continuous Delivery Fully automated Development process Automated testing core to Dev. & Delivery Manual for exploratory Dev. and QA functions merge When Where Why Automated tests are fully resilient and trustable. Can be used in all environments. Cloud - infrastructure is vital for success. A robust infrastructure is key for confidence.
  39. 39. Photo by Priscilla Du Preez on Unsplash GET PEOPLE TO JOIN YOU
  40. 40. How do I motivate my coworkers? You Team/Department Company 40 Getting your team, manager, department and company onboard
  41. 41. Motivate yourself Risks • Not moving in a constantly changing industry: testing • Missing opportunities to improve testing of the product • Getting overwhelmed by repetitive tasks Actions • Get a % of your time for training, with the purpose of spreading it internally • Join online testing chats to get ideas and support • Be the agent that triggers change in your team and organisation 41
  42. 42. Motivating my Team/Department Risks • Dev vs. QA brings confusion and low motivation • Silos break communication, brings different expectations for testing • Low trust in automation Actions • Test earlier, pair with devs to find better ways to test features • Create a testing framework based on OSS • Show progress through beautiful reports and dashboards • Create simple guidelines on what to test and what to expect from testing • Perform internal training sessions to help people improve 42
  43. 43. Motivating my Company Risks • Slow organisations hardly evolve at the same pace as the marketplace • Testing can become a bottleneck, which could lead to unstable products • User experience and reliability are hurt if testing is treated as an afterthought Actions • Connect business success metrics with testing • How did a bug in production reduce revenue? • PoC with testing as a business enabler • Increase release frequency and speed • Improve product reliability 43
  44. 44. Photo by Cathryn Lavery on Unsplash RECAP
  45. 45. E D C B A Team Setup Testing Framework Test the Right Thing When to Run Tests Get People to Join You
  46. 46. The Continuous Testing Journey The Continuous Testing Journey LEGACY TESTING CONTINUOUS TESTING ● Executive Sponsor ● Org Structure ● Development Methodology ● Methodology, Framework & Infrastructure ● Authoring Plan ● Skills Development Plan ● Cloud execution & parallelization ● Pipeline integration ● Adoption Plan ● Analytics ● Shift Left ● Shift Right PROOF OF AUTOMATION VALUE ● Release faster ● Lower risks ● Increase developer productivity ● Delight your customers
  47. 47. THANK YOU Photo by Stefan Widua on Unsplash
  48. 48. Resources 48 Sauce Labs Continuous Testing Benchmark report https://info.saucelabs.com/sauce-labs-continuous-testing-benchmark-report-M.html The Inquiry Method for Test Planning https://testing.googleblog.com/2016/06/the-inquiry-method-for-test-planning.html Sauce Labs free trainings https://training.saucelabs.com/ Selenium Channel https://seleniumhq.herokuapp.com/ Appium Channel https://gitter.im/appium/appium Ministry of Testing Channel https://www.ministryoftesting.com/slack_invite Software Testers Channel https://software-testers.herokuapp.com/
  49. 49. 5 STEPS TO JUMP-START YOUR TEST AUTOMATION DIEGO MOLINA, SENIOR SOFTWARE ENGINEER MAY 28, 2019 Q/A
  50. 50. 50 It is never too late Diffusion of Innovation (developed by E.M. Rogers, 1962) Innovators 2.5% Early Adopters 13.5% Early Majority 34% Late Majority 34% Laggards 16%

×