Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Wait, That's Not Right by Troy Walsh

84 views

Published on

This SauceCon 2019 presentation from Troy Walsh focuses on creating automated tests that fail better. He discusses logging and how it can make our failures valuable; explores error handling and how it can be both a blessing and a curse, and also covers test data management and how it can make failing far more useful.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Wait, That's Not Right by Troy Walsh

  1. 1. Wait, That’s Not Right! AKA - Failing better Troy Walsh 04.25.19
  2. 2. 2
  3. 3. 3 Effective Test Data Management » Sufficient » Representative » Reproducible » Automated » Deterministic? » *Legal/ethical/safe » Credible
  4. 4. 4 » Collaboration › QA › Dev › Operations » *Tools › Informatica - Test Data Management › CA - Datamaker › Compuware - Test Data Management › IBM - InfoSphere Optim Test Data Management › Many many more What Does It Take
  5. 5. 5
  6. 6. 6 » Is it a bug › “Works as designed” › “Cannot reproduce” › “Works on my machine” » What caused it › Data › Application › Integrations › Etc. » Is this really the problem › Root cause Challenges
  7. 7. 7 » Summary » Context › Browser, OS, Resolution, Time (UTC), user, etc. » Steps to reproduce » Expected results » Actual results » Artifacts › Screenshots, videos, logs, etc. Reporting
  8. 8. 8
  9. 9. 9 Plan To Fail
  10. 10. 10 Logging » Built in logging » Context » Useful messages » Testing artifact
  11. 11. 11 Event Logging » Captures interactions » Readable logs » Very powerful
  12. 12. 12 » Can be taken at any/multiple times » Multiple formats » *Integrates with many reporting frameworks Screenshots
  13. 13. 13 Browser Logs » Script errors » Nonvisible errors » Developer console *Does not work with every browser
  14. 14. 14
  15. 15. 15 » Videos » Extended debugging » Build server integration Even Better With Sauce
  16. 16. 16 Azure DevOps Special Sauce
  17. 17. 17 Jenkins Special Sauce
  18. 18. 18 » Have a test data management strategy » Write automation as though you expect it to fail » Have logging » Take advantage of advanced logging capabilities › Event firing › Screenshots › Console logs » If you have a service like Sauce Labs, take advantage of it › DevOps integration › Video › Extended debugging Takeaways
  19. 19. 19 » https://docs.microsoft.com/en- us/dotnet/api/microsoft.visualstudio.testtools.unittesting.testcontext.writeline?view=mstest-net- 1.3.2#Microsoft_VisualStudio_TestTools_UnitTesting_TestContext_WriteLine_System_String_ » https://docs.microsoft.com/en- us/dotnet/api/microsoft.visualstudio.testtools.unittesting.testcontext.addresultfile?view=mstest-net-1.3.2 » https://testng.org/doc/documentation-main.html#logging » https://github.com/SeleniumHQ/selenium/blob/db1c2add0b47d845b7d0d7d67b65cee92d9bb323/ja va/client/test/org/openqa/selenium/support/events/EventFiringWebDriverTest.java#L161 » https://github.com/SeleniumHQ/selenium/blob/07a18746ff756e90fd79ef253a328bd7dfa9e6dc/dotn et/test/support/Events/EventFiringWebDriverTest.cs#L38 » https://wiki.saucelabs.com/display/DOCS/Platform+Configurator#/ » https://wiki.saucelabs.com/display/DOCS/Test+Configuration+Options » https://wiki.saucelabs.com/display/DOCS/Setting+Up+CI+Platform+Integrations+with+Sauce+Plugins » https://github.com/Magenic/MAQS Helpful links
  20. 20. 20
  21. 21. 877.277.1044 / magenic.com // 21 THANK YOU

×