This is the presentation I presented at superweek.in (Mumbai - Jan 2017) http://superweek.in/. In the field of data management and analytics, one statement we often get to hear is - "The data is not accurate". This presentation is based on this statement, and I explore scenarios that might lead to such statements, reasons and remedies, expectation management (with various functional teams) and business definitions.

1. “Errors using inadequate data are much less than those using no data at all”
- Charles Babbage
The data is never accurate – and that’s OK!
By: Chanpreet Singh| twitter.com/chanpreets

2. Part 1: Why is data
not accurate?
Part 2: How to
ensure data sanity?
Part 3: How do we
work with this kind
of data?
Jan 13, 2017 Chanpreet 2

3. Tracking tool
Google Analytics
• Integrate an Analytics tool – like Google Analytics (GA)
Jan 13, 2017 Chanpreet 3

4. Part 1: Why is data
not accurate?
Jan 13, 2017 Chanpreet 4

5. What is the meaning of Accurate
Towards care
• (especially of information, measurements,
or predictions) correct in all details; exact.
• “Done with care”
• Digital measurement - ‘care’ unfortunately
is not enough
Jan 13, 2017 Chanpreet 5

6. Why would data not be Accurate
Factors responsible
• Digital Analytics programs do not
collect exact number of hits or queries
• Designed to give ‘valuable insights’
• Digital Analytics is not an Audit
• Inherent uncertainties
• Human nature
Jan 13, 2017 Chanpreet 6

7. Not designed to collect exact hits or
queries
JS limitations
• Different
layout engines
render JS
differently
Jan 13, 2017 Chanpreet 7
Layout Engine Browser
Trident
Internet Explorer for Windows
Maxthon
Netscape 8.1
Tasman Internet Explorer 5 for Macintosh
Gecko
Firefox
Netscape 6 and later
Mozilla
Camino
K-Meleon
SeaMonkey
Epiphany 2.20 and before
Galeon
KHTML Konqueror
Tkhtml Html Viewer
Layout
Engine
Browser
WebKit
Safari
Chrome
iCab 4 and later
Epiphany 2.26 and
later
Maxthon 3
OmniWeb
Shiira
Midori
Presto Opera
iCab iCab 3 and before
Tkhtml Html Viewer

8. Not designed to collect exact hits or
queries
Cookies
• Cookies
blocked or
disabled by
users
Jan 13, 2017 Chanpreet 8

9. Not designed to collect exact hits or
queries
Cookies
• Cookie
deletion
Jan 13, 2017 Chanpreet 9

10. Not designed to collect exact hits or
queries
<noscript>
• <noscript> tag
Jan 13, 2017 Chanpreet 10

11. Insights through % of data collected
Digital landscape
• Digital landscape is ever changing
• Multiple platforms
• Basic functional mobile phones
• Networks, CDNs and other systems
• Considerations of Outliers and Bots
• Sample data to extrapolate performance
Jan 13, 2017 Chanpreet 11

12. Not an Audit
Audit involves
• Series of checks and balances
• Accuracy
• Standardized metrics and
methodology
• Consistency of process
• Transparency of results
Jan 13, 2017 Chanpreet 12

13. Inherent uncertainties
Uncertainties of Digital Analytics
• Algorithms
• Models
• Data itself
• Outcomes
Jan 13, 2017 Chanpreet 13

14. Human Nature
Human factor at play
• Humans think differently
• Big organization, big teams, multiple functions
• Cross functional/ team dependencies
• Dependencies on partners
• Lack of proper understanding
Jan 13, 2017 Chanpreet 14

15. Human Nature
Tracking Code
• No tracking code
Jan 13, 2017 Chanpreet 15

16. Human Nature
Tracking Code
• Exclusion/ removal of tracking code
Jan 13, 2017 Chanpreet 16

17. Human Nature
Human factor at play
• Some users disable JavaScript
Jan 13, 2017 Chanpreet 17

18. Human Nature
Communication
• Inter team communication gap
Jan 13, 2017 Chanpreet 18

19. Human Nature
Team conflict
• Product and technology teams
Jan 13, 2017 Chanpreet 19

20. Human Nature
UTM
• Campaign tagging
Jan 13, 2017 Chanpreet 20
http://www.visitpa.com/pa-hotels-motels?
utm_campaign=EVG_PA_Primary_Lodging&utm_source=google&utm_medium=cpc&utm_term=places-to-
stay&utm_content=family-ad

21. Human Nature
Events
• Event tagging
Jan 13, 2017 Chanpreet 21

22. Human Nature
Access
• Access levels and profile view referred
Jan 13, 2017 Chanpreet 22
Account
Property
View

23. Part 2: How to
ensure data sanity?
Jan 13, 2017 Chanpreet 23

24. Awareness
Be Aware of differences
• Discrepancies vs Differences
• Different tools - different data
• Sampling
• Purpose of tools
• Metrics used
Jan 13, 2017 Chanpreet 24

25. Different tools different data – GA vs
AdWords
Metrics usage and nature differs
• Clicks vs Sessions
• AdWords filters invalid clicks, Analytics shows
all data
• Landing page might redirect to another
• Users browser preferences
• Users return during the lifetime of campaign
• Users return to your site via bookmarks
Jan 13, 2017 Chanpreet 25

26. Different tools different data – GA vs
DFP
Metrics usage and nature differs
• DFP counts ad clicks at the source; GA counts pageviews or
sessions when a user hits the site
• Metrics are counted at different points in the click-referral cycle
Jan 13, 2017 Chanpreet 26

27. Different tools different data –
Real time GA vs Another (Chartbeat)
Common to use multiple tools
• Chartbeat checks in for visitors every
few secs.
• GA checks in once for visitors every
5 mins.
• GA > Chartbeat = concurrent users
are high, but not staying long
• GA < Chartbeat = most users stay
longer than 5 mins.
Jan 13, 2017 Chanpreet 27

28. Different tools different data – GA vs
ComScore
Competition reporting
• ComScore
• Eliminates/ignores machine generated traffic
• Measures people
• Eliminates visits from multiple browsers,
multiple screens
• Extrapolates numbers based on a panel users
• 3 second rule
Jan 13, 2017 Chanpreet 28

29. Limitations within the tool – GA vs GA
GA Sampling
• GA sampling and limitations
• Standard vs Ad-Hoc reports
• GA Standard vs Premium
• High-cardinality dimensions
• Report Query limit - 1 million rows of data
• Data limits - single day/ multi day processed tables
• Multi channel funnel report > 1 million conversions
• Flow visualization report more than 100k sessions
Jan 13, 2017 Chanpreet 29

30. Part 3: How do we
work with this kind
of data?
Jan 13, 2017 Chanpreet 30

31. Best attempt to ensure better data
Plan> Implement> configure> report> Interpret
• Coding
• Admin Panel
• Product
• Interface usage and reporting
• Interpretation
Jan 13, 2017 Chanpreet 31

32. Ensuring correct implementation - Code
Coding
• Updated version (UA)
• Correct UAID
• Correct domain/ multi domain coding
• Ensure code is not Commented out
• Check for code Breakages
• Customizations (session duration, session end)
• Placement of code
• Duplication of code
• Conflict with another code (other tool), events
Jan 13, 2017 Chanpreet 32

33. Ensuring correct implementation -
Interface
Admin Panel
• Properties and Profile creation
• Correct filters
• Limitations/ Free_vs_Paid
• Custom Dimensions & Metrics
• Goals
• Cross tool integrations/ agreements
(change)
Jan 13, 2017 Chanpreet 33

34. Ensuring correct implementation -
Product
Product Sanity (Website/ Mobile Apps)
• URL/ Screen pattern and duplications
• UTM tagging
• Conflict with tech and tools
• Version control
• Audience platforms - Desktop vs Mobile
• Cache management/ deletion/
exclusions
• Bots/ malware
• Geo location of audience and network
• Firewalls
Jan 13, 2017 Chanpreet 34

35. Ensuring correct Reporting
Interface usage and reporting
• Report/ data pull/ interpretation
• Sampling
• Time period/ timelines
• Data influencing Events
• Data Cleansing - Incomplete/ inaccurate data
• Interpretations Differ
• Blind to our own biases
• Critical points that might influence another outcome
• Working with Limitations
Jan 13, 2017 Chanpreet 35

36. Same data – different interpretation
Interpretation matters
• Same data can yield wildly different
results
• Redundant data stored in multiple
systems and in different formats
• Data transformation is also important
Jan 13, 2017 Chanpreet 36

37. Finally
Directional Guidance
• Digital Analytics should be about providing directional guidance
Jan 13, 2017 Chanpreet 37

38. Thank you
Reach me twitter/@chanpreets or
email: chanpreet.mus@gmail.com
Jan 13, 2017 Chanpreet 38