Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The what, why and how of web analytics testing

3,842 views

Published on

Slides from my talk in UNICOM's Next Generation Testing Conference on 13th December in Bangalore on "The What, Why and How of Web Analytics Testing". This is based on my open-source tool - WAAT.

More information about the talk is available here: http://goo.gl/FxISG

Information about WAAT is available here: http://goo.gl/oUNHU

Published in: Technology
  • Be the first to comment

  • Be the first to like this

The what, why and how of web analytics testing

  1. 1. The  What,  Why  &  How   of    Web  Analy3cs  Tes3ng   Anand  Bagmar   So#ware  Quality  Evangelist   Principal  Consultant,  ThoughtWorks  
  2. 2. Why  do  we  do  Tes3ng?  
  3. 3. What  is  Web  Analy3cs?  
  4. 4. Why  do  we  need  it?  
  5. 5. Adobe  Marke*ng   Cloud  /  Omniture   SiteCatalyst   …   WebTrends   Web  Analy*cs   Solu*on   Providers  StatCounter   AWStats   Google  Analy*cs  
  6. 6. How  to  test  Web  Analy3cs?  
  7. 7. The  end  report  
  8. 8. Tes3ng  at  the  report  level  Pros  •  Ensure  report  is  setup  correctly    Cons  •  Licensing  •  Reports  not  yet  setup  •  Validate  all  requests  are  sent  /  captured  
  9. 9. At  the  origin!  
  10. 10. The  problem  
  11. 11. Manual  
  12. 12. The  challenge  
  13. 13. Repea3ng  over-­‐and-­‐over  again  
  14. 14. The  solu3on  
  15. 15. What  does  WAAT  do  for  me?  •  Minimal  changes  to  exisFng  test  framework    •  Does  not  depend  on  junit  /  testng  /  …  •  Web  AnalyFc  tool  independent  •  UI  Driver  framework  independent  •  Browser  independent  
  16. 16. Flavors  
  17. 17. Architecture  
  18. 18. WAAT  -­‐  Java  •  Original  flavor  •  Source  code  +  jar    •  Supports  3  plugins     •  Omniture  Debugger   •  HUpSniffer   •  JsSniffer  •  Available  on  github  
  19. 19. WAAT  -­‐  Ruby  •  Ruby  gem  implemented  over  WAAT-­‐Java  •  Uses  RJB  –  Ruby-­‐Java-­‐Bridge  •  Supports  2  plugins   •  HUpSniffer   •  JsSniffer  •  Available  on  github  and  rubygems.org  
  20. 20. Omniture  Debugger  
  21. 21. Omniture  Debugger  How  does  it  work?    Pros  •  OS  independent  •  Run  using  the  regular  test-­‐user    Cons  •  Browser  dependent  –  need  to  implement   ScriptRunner  for  the  UI-­‐driver  in  use  •  Web-­‐AnalyFc  soluFon  dependent  –  Adobe  MarkeFng   Cloud  /  Omniture  SiteCatalyst    
  22. 22. HHpSniffer  How  does  it  work?    Pros  •  Web-­‐analyFc  soluFon  independent  •  Browser  independent  •  UI-­‐driver  independent    Cons   rd•  3  party  libraries  are  OS  Dependent  •  HTTPs  not  supported  out-­‐of-­‐the-­‐box  •  Run  tests  as  ‘root’    
  23. 23. JsSniffer  How  does  it  work?    Pros  •  Web-­‐analyFc  soluFon  independent  •  Browser  independent  •  HTTPs  supported  out-­‐of-­‐the-­‐box   rd•  No  dependency  on  any  3  party  library      Cons  •  Need  to  write  javascript  to  get  the  URL  from  the   browser  •  UI-­‐driver  dependent  
  24. 24. How  to  use  WAAT?  •  Download  zip  from  github  /  rubygems.org  •  Setup  •  jpcap  /  ScriptRunner  setup    •  Update  exis*ng  tests  
  25. 25. Update  exis3ng  Tests  •  Import  relevant  packages  /  gems  •  Define  &  provide  parameter  values  •  Ini*alize  Engine  •  Enable  Web  Analy*cs  Tes*ng  •  Define  Test  Data  •  Verify  Web  Analy*cs  repor*ng  using  WAAT  •  Valida*ons  •  Disable  Web  Analy*cs  Tes*ng  
  26. 26. Define  Test  Data  <Section !! !actionName="OpenUpcomingPage_HttpSniffer"! !numberOfEventsTriggered="1"! !tagList="pageName=upcoming:index||! ! ! !cc=USD||! ! ! !ch=upcoming||! ! ! !events=event2||"/>!
  27. 27. Ini3alize  Engine   !engine = getInstance(!! ! !webAnalyticTool, !! ! !inputFileType, !! ! !keepLoadedFileInMemory, !!! ! !log4jPropertiesAbsoluteFilePath!! ! !);!
  28. 28. Enable  Web  Analy3cs  Tes3ng   engine.enableWebAnalyticsTesting();!
  29. 29. Verify  Web  Analy3cs  Repor3ng   Cart.checkout();!!Result verificationResult = !!! !engine.verifyWebAnalyticsData (!! ! !inputDataFileName, !! ! !actionName, !! ! !urlPatterns, !! ! !minimumNumberOfPackets);!
  30. 30. Sample  Valida3ons  assertNotNull(verificationResult.getVerificationStatus(),"Verification status should NOT be NULL"); !!assertNotNull(verificationResult.getListOfErrors(),"Failure details should NOT be NULL");logVerificationErrors(verificationResult); !!assertEquals(verificationResult.getVerificationStatus(),Status.PASS, "Verification status should be PASS"); !!assertEquals(verificationResult.getListOfErrors().size(),0, "Failure details should be empty");!
  31. 31. Disable  Web  Analy3cs  Tes3ng   engine.disableWebAnalyticsTesting();!
  32. 32. Anand.Bagmar@thoughtworks.com                        Abagmar@gmail.com        @BagmarAnand   hUp://essenceo#esFng.blogspot.com/search/label/waat   hUps://github.com/anandbagmar/   hUp://rubygems.org/gems/WAAT  

×