Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Loading time testing and results visualisation of web games

Доклад Andrejs Kalnacs на SQA Days-22. 17-18 ноября 2017. Санкт-Петербург, Россия

  • Be the first to comment

  • Be the first to like this

Loading time testing and results visualisation of web games

  1. 1. Loading time testing and results visualisation of web games Andrejs Kalnačs
  2. 2. who am i 2 Lead Software Developer in Test @ Evolution Gaming 10 years of industry experience in QA Many different roles Wide business domains experience Last 6 years – testing automation Java, Groovy, Bash, Perl, Tcl Evolution Gaming, Nokia, Accenture, Nokia Siemens Networks Riga Testing Automation Club meet-ups co-organiser English, Russian and Latvian language proficiency @TesterFromRiga
  3. 3. Evolution Gaming 3 The World leader in live dealer gaming 3 development locations – Riga, Tallinn, Amsterdam B2B product company
  4. 4. Agenda 4 Web based loading time testing Practical guide in test data points visualisation
  5. 5. Use case of Evolution Gaming 5
  6. 6. Use case of Evolution Gaming 6 16 different games 3 test environments 2 browser cache states
  7. 7. Creation and execution of web based loading time testing
  8. 8. Browser automation using WebDriver API 8
  9. 9. Custom metric collection from client 9 Loading time Intermediate loading metrics Component/service loading metrics Proprietary API
  10. 10. Interception & archiving of HTTP transactions 10 Proxy System under test HTTP, HTTPS, WebSocket
  11. 11. HTTP archive format 11 JSON Libraries Many viewers Storage solutions Diff solutions
  12. 12. Chrome 62 12
  13. 13. OWASP ZAP proxy 13 HTTPS support WebSocket support API Open Source Cross platform (Linux, Mac OS X, Windows)
  14. 14. Bandwidth throttling 14 Virtual or physical machine with two network interfaces Linux tc tcconfig tcset --device eth1 --rate 800k --direction outgoing tcset --device eth0 --rate 800k --direction incoming
  15. 15. Cloud? 15 Virtual hardware for desktop browsers Traffic throating only at Sauce Labs & AWS No HAR produced
  16. 16. Local desktop & device farm 16 Support, maintenance & update of infrastructure, hardware & software Stability of devices Complexity of iOS testing automation
  17. 17. Execution from CI server 17 Selenium Hub Appium iOS node Appium Android node Proxy Bandwidth throttler Docker host Docker host
  18. 18. Tests results 18 + Information provided by API Calculated & aggregated metrics from HAR
  19. 19. Test data storage 19 Jenkins (job artefacts) Attachments to report File storage (FTP, SAMBA, WebDAV, or anything else) Dedicated .har storage (harstorage) SQL or NoSQL database
  20. 20. Results visualization
  21. 21. Why? 21 Show stack holders how good or bad user experience is Compare your products Visualise regression Understand how new feature is affecting performance
  22. 22. Ops & finance tools 22
  23. 23. Use-case of performance tests results 23
  24. 24. 24 Prometheus Prometheus Push gateway Test job pushget (interval based)
  25. 25. 25
  26. 26. 26
  27. 27. 27 Templating SCREENSHOT TO BE ADDED
  28. 28. 28 Annotations
  29. 29. 29 Alerts
  30. 30. simplest-test-dashboard 30 Saves web page with content to disc and calculates it size
  31. 31. Grafana dashboard version control (git) 31 Edit dashboard Copy in IDE Diff & throw out unneeded things Push to branch Merge request Merge to master Grafana dashboard update via API
  32. 32. Key takeaways 32 Selenium can be used for performance tests Application it self can provide a lot of insides for testers Visualisation is excellent for displaying results in interactive, understandable and easily comparable way Mastering data visualisation tools is not a rocket science
  33. 33. Thank you! Any questions? andrejska Andrejs Kalnačs @TesterFromRiga