Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Synthetic web performance testing with Selenium

4,858 views

Published on

Another way to monitor performance of your site with Selenium and log analyzer

Published in: Technology
  • Be the first to comment

Synthetic web performance testing with Selenium

  1. 1. Synthetic performance monitoring with Selenium Andriy Samilyak OpsWay CEO
  2. 2. Who is talking? Working in IT since 2005 ● Since 2008 - Smile Ukraine CTO ● Since 2013 – OpsWay CEO ● Coaching with SmartMe.com.ua ●
  3. 3. You know this... “Metrics show that a 0.1 second delay in page rendering can translate into a 1% drop in customer activity” Amazon research papers
  4. 4. Projects are changed daily ● Application code ● Product database ● Marketing tags ● Server configuration ● External services ● etc … and no QA engineers at all...
  5. 5. User checkout cycle under control Add to cart Select product Success page! Login ~30-50 sec Billing/Shipping options Payment
  6. 6. Is it still running smoothly? ● Passive checks – analyzing end-user activity – – Webserver log analysis – ● NewRelic Web analytics (real user monitoring) Active checks – run test and record results – Gomez GPN – Uptrends – Pingdom
  7. 7. NewRelic Cool, sexy, not so cheap and not really what we need
  8. 8. GoogleAnalytics with UserTiming https://github.com/opsway/magento-mod-rum … but we need immediate alerts!
  9. 9. Server-side log analysis ● Complex models: – 2 add to cart requests timing – 1 login step timing – 5 checkout steps timings ● not comparable side by side ● not representative when user activity is low
  10. 10. Server-side logs: false-positive 3 mins to checkout? Are you serious?
  11. 11. Pingdom/Uptrends ● Transaction monitoring via scripting ● Fancy scripting language ● ● Robust, popular but not so cheap for multiple projects Fine for public sites, but not for intranet
  12. 12. We liked Compuware Gomez!
  13. 13. Compuware Gomez is cool
  14. 14. But we have already Selenium! Selenium Splunk 1. Load product page Send timestamp via UDP 2. Add to cart Send timestamp via UDP 3. Shopping cart Send timestamp via UDP 4. Address 5. Payment 6. Success page ...
  15. 15. It looks like this in code...
  16. 16. Splunk combines to transaction...
  17. 17. We can chart transactions
  18. 18. … and alert on abnormal behavior Slow transaction Failed Transaction!
  19. 19. Why did transaction fail? ● Selenium reported timeout – not always sufficient ● Rarely reproducible (many servers) ● We needed screenshots!
  20. 20. Capturing a screenshot
  21. 21. Saving screenshot with Jenkins
  22. 22. Jenkins build with errors We got it!
  23. 23. Why it took so long? ● ● We don't know always what http request blocked transaction Inspired by Gomez we integrated HAR recording of test sessions
  24. 24. Inspired by Gomez ...
  25. 25. HAR is not perfect, but ...
  26. 26. HAR proxy ● Based on BrowserMob proxy server http://bmp.lightbody.net/ HAR file Launching WebDriver Selenium Grid Proxy on port 64001 Request passes through proxy www.mysite.com Selenium node (Firefox)
  27. 27. Jenkins build with errors Here we have HAR
  28. 28. Issues we have ● Supporting test scripts takes a lot of time ● Support test infrastructure is tedious ● False-positives ● Still not enough debug information
  29. 29. Andriy Samilyak samilyak@gmail.com skype: samilyaka http://opsway.com

×