Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Making Test Automation Visible


Published on

keynote presented at Agile & Automation in Krakow Poland 2018

Published in: Technology
  • Be the first to comment

Making Test Automation Visible

  2. 2. WHO AM I? • Karen N. Johnson is a longtime contributor to the software testing community. • Contributing author to the book, "BeautifulTesting" by O’Reilly publishers. • She has published numerous articles,blogs and tweets about her experiences. • Find her onTwitter as @karennjohnson (note the two n’s) • And her website: • Karen was previously an independent consultantbut is now employed at JAMF as a Software Engineering Director. See:
  3. 3. MAKE ITVISIBLE “Visible” test automation doesn’t mean hosting test automation demos - where people physically watch automation. “Visibility” is about creating transparency, knowledge and ultimately helping people to understand what has been built, what has not been built and the reasoning behind those decisions.
  4. 4. WHY
  5. 5. #1WHY: OPEN THE DOOR TO COLLABORATION • Developers can extend functions,offer code reviews and tackle some of the more technical aspects that might be out of reach for a test automation person. • Manual testers can pick up where automation leaves off. • In some cases, other teams might be able to find an opportunity to re-use what your team has built.All possible from sharing.
  6. 6. #2WHY: BUILD TEAM OWNERSHIP • When an automation test suite fails, people will care more about resolving the issue if they know what the automation suite contains and accomplishes. • People will not engage with something they don't understand. • People will not own what they don’t know.
  7. 7. #3WHY: GAIN MANAGEMENT SUPPORT Chances are there is a manager down the hall or perhaps in a different country,and when budget season rolls around as it always does, it’s best that your manager has some understanding of what you’re doing and the value the automation you’re building brings to the team.
  8. 8. #4WHY: GETTINGTO CI/CD A sense of team ownership of the automation is helpful as a team works towards deploying in a CI/CD environment.This awareness increases the possibility of other people on the team (other than the test automation people) getting involved when test automation fails.Think about it - why would you care if automation fails,if you don’t know what is lost when the automation is not working.
  9. 9. WHEN
  10. 10. #1WHEN: HOST ATESTAUTOMATION OPEN HOUSE • Showcase your work. • Automate based on a risk analysis and then share that analysis. • Explain the decision-making process. • The remaining work is another way of indicating the decision-making process that goes into automation as well as possible limitations - whether those limits are skills, tools or time.
  11. 11. #2WHEN: RECORDA SHORTVIDEO • The beauty of making a recording is you have the chance to say exactly what you want to say. • A short video could provide a multisensory explanation of the topic – which is particularly useful for complicated subjects. For example,you could be showing the testing pipeline while you're explaining the same information helping to show automation.
  12. 12. #3WHEN: CREATE A QUARTERLY AUTOMATION UPDATE Consider writing an email that provides a brief on the state of automation.An email that is sent frequently enough to be helpful without sending an email so often,it is overlooked.An email update gives you the opportunity to highlight the current state of automation.
  13. 13. #4WHEN: SHARE AT SPRINT REVIEWS One benefit of sharing automation updates at the sprint review is that the automation naturally becomes seen and understood as part of the product.Not as separate from the product.
  14. 14. WHAT
  15. 15. #1WHAT: FEATURE TESTING • When a new feature is being pushed into production use,knowing what has been automated provides not only insight into what testing has been done but some reassurance that if the automation is added to a regression suite, issues will be not only detected but potentially prevented. • When automation lags product delivery (which happens),it still helpful to know what has been built and what work remains to be done.
  16. 16. #2WHAT: REGRESSION • The regression suite is often the primary reason automation is built,so a clear understanding of the regression suite is essential to understanding the automation effort. • The importance of choosing what is included in the regression suite gives tremendous insight into the thinking behind the automation.
  17. 17. #3WHAT: ROADMAP Providing visibility into automation shows how testing is working towards the overall product roadmap.
  18. 18. #4WHAT: THE “ILITIES” A test automation suite for a product is more than the sum of feature testing and should be even larger than a full regression suite. Somewhere in the mix of automation scripts,there should be tests designed to cover some of the“ilities” of testing such as scalability,maintainability and a host of other "ilities."These are tests that are not run on a regular basis but are often the tests that cannot be achieved outside of automation.These tests are some of the most compelling reasons to even have automation.
  19. 19. HOW
  20. 20. #1 HOW: THE CHECKLIST • Keep it simple. • List the automation built. • Show the alignment of product delivery and the automation suite. • Explain (high level) the process of running automation in connection to shipping to production. • Identify what has been built then let others ask for more details.
  21. 21. #2 HOW: THE REALITIES • Nearly every stakeholder knows automation tools have limitations; share what’s been built as well as explaining (when it makes sense to) where the tool may have imposed limitations on what could be built. • Equally most stakeholders realize that test automation people may have limitations themselves as to what they are able to build, there’s nothing wrong in sharing the limitations of what you’re able to build.
  22. 22. #3 HOW: THE BACKLOG • When a feature ships to production, there may be more automation work to be done.It is perfectly ok to let your stakeholders know what work remains. • In addition to feature automation, there will be regression testing and possibly "ility" automation tests that continually need maintenance. Sharing what backlog of automation tasks remain is helpful for stakeholders to understand the current state of automation.
  23. 23. #4 HOW: THE MAP ANALOGY • Have you ever used a map that showed every road, body of water and other terrain information making the map unreadable? • Detailing too low level of information about test automation can generate the same result – over information creating under-informed people. • On the other hand,too little information equally leaves people under-informed. • Strike the balance.Find out what people want to hear about to provide a “map of information.”
  24. 24. Note:During presenting,I showed a physical map with many roads and markings to the point where the map was unreadable.I then showed this map as an example of how filtering and in some cases limiting information makes a map more readable and helpful. The credit for this map belongs to Rick Steves.