Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Escape from the automation hell

62 views

Published on

We build software that runs on 21+ platform with millions of users and billions of deployments. Our software engineers have to keep the high pace of quality. More than 500k tests are executed daily. In this talk I would like to share my experience of dealing with unstable and long running tests, collecting, aggregating and analysis test execution data.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Escape from the automation hell

  1. 1. COPYRIGHT 2014 @ UNITY TECHNOLOGIES ESCAPE FROM THE AUTOMATION HELL • By Yan Drugalya
  2. 2. COPYRIGHT 2014 @ UNITY TECHNOLOGIES KID DREAMS
  3. 3. COPYRIGHT 2014 @ UNITY TECHNOLOGIES WHO I BECAME
  4. 4. COPYRIGHT 2014 @ UNITY TECHNOLOGIES HOW OTHERS SEE ME
  5. 5. COPYRIGHT 2014 @ UNITY TECHNOLOGIES NOW
  6. 6. COPYRIGHT 2014 @ UNITY TECHNOLOGIES NOW
  7. 7. COPYRIGHT 2014 @ UNITY TECHNOLOGIES AUTOMATION
  8. 8. COPYRIGHT 2014 @ UNITY TECHNOLOGIES
  9. 9. COPYRIGHT 2014 @ UNITY TECHNOLOGIES
  10. 10. COPYRIGHT 2014 @ UNITY TECHNOLOGIES CHALLENGES • Slow tests • Unstable tests • Hard to run tests locally • Hard to reproduce failure locally
  11. 11. COPYRIGHT 2014 @ UNITY TECHNOLOGIES FIRST THERE WERE THE SCOUTS
  12. 12. COPYRIGHT 2014 @ UNITY TECHNOLOGIES AFTER ½ YEARS OF SCOUTING
  13. 13. COPYRIGHT 2014 @ UNITY TECHNOLOGIES THE HOPE
  14. 14. COPYRIGHT 2014 @ UNITY TECHNOLOGIES
  15. 15. COPYRIGHT 2014 @ UNITY TECHNOLOGIES UNIFIED TEST RUNNER • A Simple command line that changed a lot test -suite=<runtime|integration…>
  16. 16. COPYRIGHT 2014 @ UNITY TECHNOLOGIES TDD IN PERL?
  17. 17. COPYRIGHT 2014 @ UNITY TECHNOLOGIES ISSUES • Slow tests • Unstable tests • Hard to run tests locally • Hard to reproduce failure locally
  18. 18. COPYRIGHT 2014 @ UNITY TECHNOLOGIES NEXT STEP • Build farm started to use UTR
  19. 19. COPYRIGHT 2014 @ UNITY TECHNOLOGIES ISSUES • Slow tests • Unstable tests • Hard to run tests locally • Hard to reproduce failure locally
  20. 20. COPYRIGHT 2014 @ UNITY TECHNOLOGIES HOARDER
  21. 21. COPYRIGHT 2014 @ UNITY TECHNOLOGIES BuildAgents&Humans Hoarder WS
  22. 22. COPYRIGHT 2014 @ UNITY TECHNOLOGIES
  23. 23. COPYRIGHT 2014 @ UNITY TECHNOLOGIES
  24. 24. COPYRIGHT 2014 @ UNITY TECHNOLOGIES NICE DISCOVERIES • 16 559 749 tests run in Aug 2015! • Never failed tests • Compare performance regressions • How people use UTR to run tests locally • Automatic discovery of unstable test • ....
  25. 25. COPYRIGHT 2014 @ UNITY TECHNOLOGIES FUTURE • Immediately get an overview of the state of our test automation • Automatically find and assign test failures • Extend the REST API interface to make it easier to integrate with other tools we use at Unity

×