Upcoming SlideShare
QA Fest 2018. Slavik Pashanin. Analytics Data Driven Testing
- 1. Slavik Pashanin slavikp@wix.com twitter@slavik_pashanin github.com/slavikpa Data-driven Testing
- 2. Let’s play a game
- 3. Testing your product
- 4. All Most Common
- 5. How BI* Works *Business Intelligence
- 6. Implement BI events (methods)
- 7. example Implement BI events (methods)
- 8. Implement BI events (methods) Collect data
- 9. Implement BI events (methods) Collect data Query for information
- 10. Make decisions past present future
- 11. Make decisions past present future historic
- 12. Make decisions past present future real time
- 13. Make decisions past present future predictive
- 14. Why BI can help your testing
- 15. Without data, you are just another person with an opinion. W. Edwards Deming
- 16. Checkout form KPI (Key Performance Indicator) # of hits on the checkout button
- 17. Checkout form # of hits recorded in the database Success rate
- 18. Checkout form Error rate # of hits not recorded in the database
- 19. Getting BI into testing
- 20. In productionFeature Release Feature development
- 21. In production Feature Release Feature development
- 22. In productionFeature development Feature Release
- 23. 3 ways to get BI into your testing 1. Google Analytics 2. Internal tools 3. TPAs
- 24. Google Analytics SETUP Get the tracking number e.g., UA-XXXXX-Y and add the code <!-- Google Analytics --> <script> (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); ga('create', 'UA-XXXXX-Y', 'auto'); ga('send', 'pageview'); </script> <!-- End Google Analytics --> downloads analytics.js allows you to schedule commands to be run
- 25. Google Analytics ADD EVENTS Event hits are sent by using the Send command and specifying a hitType <button id="addButton"> Click me </button> <script type="text/javascript"> const button = document.getElementById('addButton') button.addEventListener('click',function (e) { ga('send', { hitType: 'event', eventCategory: 'buttonClick', eventAction: 'mainButtonClick', eventLabel: 'nameOfThePage' }); }); </script>
- 26. Google Analytics ▪ Page tracking ▪ Event tracking (user interactions) ▪ Social interactions ▪ User timing ▪ Error tracking
- 27. Example Mobile devices
- 28. Example Users’ common flows
- 29. Without big data analytics, companies are blind and deaf, wandering out onto the web like deer on a freeway. Geoffrey Moore, partner at MDV
- 30. Google Analytics ▪ It’s free ▪ Big community ▪ Fast integration ▪ Custom reports - upgrading is $150,000 - may be overwhelming
- 31. Internal BI Tools ▪ Tailored events ▪ Track history or real-time ▪ Trends AND users
- 32. Catalog for all Wix BI events
- 33. User explorer (history)
- 34. User explorer (real-time)
- 35. Quix (stored / custom queries)
- 36. Internal BI Tools ▪ Flexible & scalable ▪ TPAs Integration ▪ Single user flows ▪ Time frames - Takes time to develop and define
- 37. TPAs ▪ Anomaly detection ▪ Reports ▪ Automatic notifications ▪ Server-side status ▪ Client-side status
- 38. Anodot Real-time anomaly detection
- 39. Tableau reports
- 40. Slack automatic messages
- 41. 1. Drop in success rate 2. Automatic message sent to Slack channel 3. Dev teams looking for new GA’s open experiments 4. Rollback/ closing new experiments 5. Fix issue and GA 6. Back to normal
- 42. New Relic Server-side status
- 43. Grafana Client-side status
- 44. TrackJS Client-side status
- 45. Sentry.io Fix crashes in real-time
- 46. BI focuses you on the right things to test. As part of any feature development, you should rely on BI data to make better decisions.
- 47. Thank You slavikp@wix.com twitter@slavik_pashanin github.com/slavikpa
- 48. Q&A slavikp@wix.com twitter@slavik_pashanin github.com/slavikpa
Login to see the comments