Slides from my talk in UNICOM's Next Generation Testing Conference on 13th December in Bangalore on "The What, Why and How of Web Analytics Testing". This is based on my open-source tool - WAAT. More information about the talk is available here: http://goo.gl/FxISG Information about WAAT is available here: http://goo.gl/oUNHU

1. The
What,
Why
&
How
of
Web
Analy3cs
Tes3ng
Anand
Bagmar
So#ware
Quality
Evangelist
Principal
Consultant,
ThoughtWorks

2. Why
do
we
do
Tes3ng?

3. What
is
Web
Analy3cs?

4. Why
do
we
need
it?

5. Adobe
Marke*ng
Cloud
/
Omniture
SiteCatalyst
…
WebTrends
Web
Analy*cs
Solu*on
Providers
StatCounter
AWStats
Google
Analy*cs

6. How
to
test
Web
Analy3cs?

7. The
end
report

10. Tes3ng
at
the
report
level
Pros
• Ensure
report
is
setup
correctly
Cons
• Licensing
• Reports
not
yet
setup
• Validate
all
requests
are
sent
/
captured

11. At
the
origin!

14. The
problem

15. Manual

16. The
challenge

17. Repea3ng
over-­‐and-­‐over
again

19. The
solu3on

20. What
does
WAAT
do
for
me?
• Minimal
changes
to
exisFng
test
framework
• Does
not
depend
on
junit
/
testng
/
…
• Web
AnalyFc
tool
independent
• UI
Driver
framework
independent
• Browser
independent

21. Flavors

22. Architecture

23. WAAT
-­‐
Java
• Original
flavor
• Source
code
+
jar
• Supports
3
plugins
• Omniture
Debugger
• HUpSniffer
• JsSniffer
• Available
on
github

24. WAAT
-­‐
Ruby
• Ruby
gem
implemented
over
WAAT-­‐Java
• Uses
RJB
–
Ruby-­‐Java-­‐Bridge
• Supports
2
plugins
• HUpSniffer
• JsSniffer
• Available
on
github
and
rubygems.org

25. Omniture
Debugger

26. Omniture
Debugger
How
does
it
work?
Pros
• OS
independent
• Run
using
the
regular
test-­‐user
Cons
• Browser
dependent
–
need
to
implement
ScriptRunner
for
the
UI-­‐driver
in
use
• Web-­‐AnalyFc
soluFon
dependent
–
Adobe
MarkeFng
Cloud
/
Omniture
SiteCatalyst

27. HHpSniffer
How
does
it
work?
Pros
• Web-­‐analyFc
soluFon
independent
• Browser
independent
• UI-­‐driver
independent
Cons
rd
• 3
party
libraries
are
OS
Dependent
• HTTPs
not
supported
out-­‐of-­‐the-­‐box
• Run
tests
as
‘root’

28. JsSniffer
How
does
it
work?
Pros
• Web-­‐analyFc
soluFon
independent
• Browser
independent
• HTTPs
supported
out-­‐of-­‐the-­‐box
rd
• No
dependency
on
any
3
party
library
Cons
• Need
to
write
javascript
to
get
the
URL
from
the
browser
• UI-­‐driver
dependent

29. How
to
use
WAAT?
• Download
zip
from
github
/
rubygems.org
• Setup
• jpcap
/
ScriptRunner
setup
• Update
exis*ng
tests

30. Update
exis3ng
Tests
• Import
relevant
packages
/
gems
• Define
&
provide
parameter
values
• Ini*alize
Engine
• Enable
Web
Analy*cs
Tes*ng
• Define
Test
Data
• Verify
Web
Analy*cs
repor*ng
using
WAAT
• Valida*ons
• Disable
Web
Analy*cs
Tes*ng

31. Define
Test
Data
<Section !!
!actionName="OpenUpcomingPage_HttpSniffer"!
!numberOfEventsTriggered="1"!
!tagList="pageName=upcoming:index||!
! ! !cc=USD||!
! ! !ch=upcoming||!
! ! !events=event2||"/>!

32. Ini3alize
Engine
!engine = getInstance(!
! ! !webAnalyticTool, !
! ! !inputFileType, !
! ! !keepLoadedFileInMemory, !!
! ! !log4jPropertiesAbsoluteFilePath!
! ! !);!

33. Enable
Web
Analy3cs
Tes3ng
engine.enableWebAnalyticsTesting();!

34. Verify
Web
Analy3cs
Repor3ng
Cart.checkout();!
!Result verificationResult = !!
! !engine.verifyWebAnalyticsData (!
! ! !inputDataFileName, !
! ! !actionName, !
! ! !urlPatterns, !
! ! !minimumNumberOfPackets);!

35. Sample
Valida3ons
assertNotNull(verificationResult.getVerificationStatus(),
"Verification status should NOT be NULL"); !
!
assertNotNull(verificationResult.getListOfErrors(),
"Failure details should NOT be NULL");
logVerificationErrors(verificationResult); !
!
assertEquals(verificationResult.getVerificationStatus(),
Status.PASS, "Verification status should be PASS"); !
!
assertEquals(verificationResult.getListOfErrors().size(),
0, "Failure details should be empty");!

36. Disable
Web
Analy3cs
Tes3ng
engine.disableWebAnalyticsTesting();!

37. Anand.Bagmar@thoughtworks.com
Abagmar@gmail.com
@BagmarAnand
hUp://essenceo#esFng.blogspot.com/search/label/waat
hUps://github.com/anandbagmar/
hUp://rubygems.org/gems/WAAT