This document discusses web analytics testing and the opportunities in testing IoT and big data solutions. It provides an overview of web analytics, why testing is needed, and common web analytics solutions like Google Analytics. It then describes challenges with manual testing and introduces WAAT (Web Analytics Automation Testing) as a solution to automate testing. WAAT supports various plugins and has Java and Ruby implementations. The document concludes by discussing how IoT and big data are generating large amounts of data and new opportunities for testing to validate data collection, analysis, and optimization of business value.

1. THE WHAT, WHY &
HOW OF (WEB)
ANALYTICS TESTING
Anand Bagmar
Software Quality Evangelist

2. @BagmarAnand
about.me/anand.bagmar
ABOUT ME

3. What do you expect
from this session?

4. Why do we do
Testing?

6. HOW DO WE VALIDATE?

7. What are Heuristics?

9. Web Analytics

10. WHAT IS WEB ANALYTICS?

12. WHY DO WE NEED WEB ANALYTICS?

13. WEB ANALYTICS SOLUTIONS
Google Analytics
SiteCatalyst
AWStats
WebTrends
….

14. Testing Web Analytics

18. LEARN & OPTIMIZE

19. TESTING AT THE REPORT LEVEL
Pros
¨ Ensure report is setup correctly
Cons
¨ Licensing
¨ Reports not yet setup
¨ Validate all requests are sent / captured

20. Biggest Problem
It is TOO LATE!

21. The Solution

25. Web Analytics Testing
Challenges

26. MANUAL

27. REPEATING OVER-AND-OVER AGAIN

29. WAAT - Web Analytics
Automation Testing

30. WHAT DOES WAAT DO FOR ME?
q Plugs into existing test framework
q With minimal changes
q Web Analytic tool independent
q UI Driver framework independent
q Browser independent

31. FLAVORS

32. ARCHITECTURE

33. WAAT-JAVA
q Original flavor
q Supports 3 plugins
q Omniture Debugger
q HttpSniffer
q JsSniffer
q Available on github

34. WAAT-RUBY
q Ruby gem implemented over WAAT-Java
q Uses RJB – Ruby-Java-Bridge
q Supports 2 plugins
q HttpSniffer
q JsSniffer
q Available on github and rubygems.org

35. OMNITURE DEBUGGER

36. OMNITURE DEBUGGER
Pros
q OS independent
q Run using the regular test-user
Cons
q Browser dependent – need to implement ScriptRunner
for the UI-driver in use
q Web-Analytic solution dependent – Adobe Marketing
Cloud / Omniture SiteCatalyst

37. HTTPSNIFFER
Pros
q Web-analytic solution independent
q Browser independent
q UI-driver independent
Cons
q 3
rd
party libraries are OS Dependent
q HTTPs not supported out-of-the-box
q Run tests as ‘root’

38. JSSNIFFER
Pros
q Web-analytic solution independent
q Browser independent
q HTTPs supported out-of-the-box
q No dependency on any 3
rd
party library
Cons
q Need to write JavaScript to get the URL from the browser
q UI-driver dependent

40. Is that all to
Web Analytics?

41. The new “kids” in town
IoT & Big Data

42. Some popular use cases

43. IOT – INTERNET OF THINGS
Opportunity to
¨ Create new value propositions
¨ Be Innovative
¨ Be Creative

44. IOT – HOW TO DELIVER VALUE?
¨ Automate the manual processes
¨ Integrate data capabilities
¨ Collect – integrate from various sources
¨ Repeat collection – automate the collection
¨ Analyze – manual & machine learning
¨ Optimize / Pivot
¨ Repeat

45. IOT – CHALLENGES
¨ Federated devices
¨ Different types of networks
¨ Different communication channels
¨ Physical (hardware) & Virtual (software)

46. IOT – BIGGER CHALLENGES
¨ Too many devices
¨ Lots of data

47. BIG DATA …. IS GETTING BIGGER
¨ Volumes of data generated
¨ A jet engine generates 1TB of data per flight.
¨ A large refinery generates 1TB of raw data per day.
¨ As cars get smarter, the number of sensors is projected to reach as many as 200
per car.
¨ Sensors of all types will generate immense amounts of data. In fact, analysts
estimate that by 2020, 40 percent of all data will come from sensors.
¨ IoT leads to massive volumes of data
http://www.cisco.com/web/solutions/trends/iot/docs/iot-data-analytics-white-paper

48. IoT is about Data!

49. TO GET VALUE FROM IOT …
¨ Collect
¨ Analyze
¨ Predict
¨ Plan

50. What does this mean
for Testing?

51. TESTING OPPORTUNITIES
We need to build capabilities to validate –
¨ Data collection is working well
¨ From all sources
¨ Analyzing data, capturing patterns and trends
¨ Optimize business value
¨ Create new opportunities and value propositions

52. RESOURCES
¨ https://github.com/anandbagmar/waat
¨ https://github.com/anandbagmar/waat-ruby
¨ http://essenceoftesting.blogspot.in/search/label/waat
¨ http://www.cisco.com/web/solutions/trends/iot/docs/
iot-data-analytics-white-paper
¨ http://www.dezyre.com/article/how-big-data-analysis-
helped-increase-walmart-s-sales-turnover/109
5

53. @BagmarAnand
about.me/anand.bagmar
THANK YOU