Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Counting on confusion
Why no one knows how many
people are visiting their websites
What if everyone is wrong?
The mystery emerges
• Boston.com,
BostonHerald.com
complain of lowballing
The mystery emerges
• Boston.com,
BostonHerald.com
complain of lowballing
• Internal counts are
three times higher
than Ni...
The mystery emerges
• Boston.com,
BostonHerald.com
complain of lowballing
• Internal counts are
three times higher
than Ni...
Panel-based metrics
• Nielsen and comScore
sample what sites
audience is visiting
Panel-based metrics
• Nielsen and comScore
sample what sites
audience is visiting
• Afternoon traffic is
undercounted, loc...
Panel-based metrics
• Nielsen and comScore
sample what sites
audience is visiting
• Afternoon traffic is
undercounted, loc...
comScore is now the standard
• About 2 million users
have tracking code on
their computers
comScore is now the standard
• About 2 million users
have tracking code on
their computers
• comScore uses polling
techniq...
comScore is now the standard
• About 2 million users
have tracking code on
their computers
• comScore uses polling
techniq...
Server-side metrics
• An actual count of
incoming traffic.
What could go wrong?
Server-side metrics
• An actual count of
incoming traffic.
What could go wrong?
• Visitors are counted by
device. Less acc...
Server-side metrics
• An actual count of
incoming traffic.
What could go wrong?
• Visitors are counted by
device. Less acc...
What is a unique visitor?
• Most agree it is
someone who visited
once during time span
What is a unique visitor?
• Most agree it is
someone who visited
once during time span
• Unique visitors per
month is most...
What is a unique visitor?
• Most agree it is
someone who visited
once during time span
• Unique visitors per
month is most...
Dearth of free data
• SimilarWeb works like
comScore except its
data are more open
Dearth of free data
• SimilarWeb works like
comScore except its
data are more open
• Quantcast gets good
reviews from some...
Dearth of free data
• SimilarWeb works like
comScore except its
data are more open
• Quantcast gets good
reviews from some...
Was newspaper data better?
• No one knows how
many people actually
open up the paper
Was newspaper data better?
• No one knows how
many people actually
open up the paper
• Multipliers of “readers”
versus “su...
Was newspaper data better?
• No one knows how
many people actually
open up the paper
• Multipliers of “readers”
versus “su...
Is there a better way?
• Tony Haile of Chartbeat
is pushing for ad prices
based on time spent
Is there a better way?
• Tony Haile of Chartbeat
is pushing for ad prices
based on time spent
• The Washington Post
has ad...
Is there a better way?
• Tony Haile of Chartbeat
is pushing for ad prices
based on time spent
• The Washington Post
has ad...
Credits
• Much of this presentation is based on
“Confusion Online: Faulty Metrics and the
Future of Digital Journalism,” b...
Upcoming SlideShare
Loading in …5
×

The trouble with numbers

Why no one knows how many people are visiting their websites.

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

  • Be the first to like this

The trouble with numbers

  1. 1. Counting on confusion Why no one knows how many people are visiting their websites
  2. 2. What if everyone is wrong?
  3. 3. The mystery emerges • Boston.com, BostonHerald.com complain of lowballing
  4. 4. The mystery emerges • Boston.com, BostonHerald.com complain of lowballing • Internal counts are three times higher than Nielsen ratings
  5. 5. The mystery emerges • Boston.com, BostonHerald.com complain of lowballing • Internal counts are three times higher than Nielsen ratings • Whose count is accurate? Agencies’ or content-providers’?
  6. 6. Panel-based metrics • Nielsen and comScore sample what sites audience is visiting
  7. 7. Panel-based metrics • Nielsen and comScore sample what sites audience is visiting • Afternoon traffic is undercounted, local sites are a challenge
  8. 8. Panel-based metrics • Nielsen and comScore sample what sites audience is visiting • Afternoon traffic is undercounted, local sites are a challenge • Nielsen once gave washingtonpost.com 10m and comScore 17m
  9. 9. comScore is now the standard • About 2 million users have tracking code on their computers
  10. 10. comScore is now the standard • About 2 million users have tracking code on their computers • comScore uses polling techniques to weight and adjust its sample
  11. 11. comScore is now the standard • About 2 million users have tracking code on their computers • comScore uses polling techniques to weight and adjust its sample • Some users may have had code installed without permission
  12. 12. Server-side metrics • An actual count of incoming traffic. What could go wrong?
  13. 13. Server-side metrics • An actual count of incoming traffic. What could go wrong? • Visitors are counted by device. Less accurate if they clear cookies.
  14. 14. Server-side metrics • An actual count of incoming traffic. What could go wrong? • Visitors are counted by device. Less accurate if they clear cookies. • No way of separating out robotic hits from search engines
  15. 15. What is a unique visitor? • Most agree it is someone who visited once during time span
  16. 16. What is a unique visitor? • Most agree it is someone who visited once during time span • Unique visitors per month is most common Internet traffic measure
  17. 17. What is a unique visitor? • Most agree it is someone who visited once during time span • Unique visitors per month is most common Internet traffic measure • Other measures: return visits, number of page views, time spent
  18. 18. Dearth of free data • SimilarWeb works like comScore except its data are more open
  19. 19. Dearth of free data • SimilarWeb works like comScore except its data are more open • Quantcast gets good reviews from some, but code must be installed
  20. 20. Dearth of free data • SimilarWeb works like comScore except its data are more open • Quantcast gets good reviews from some, but code must be installed • Alexa, now owned by Amazon, offers an alternative approach
  21. 21. Was newspaper data better? • No one knows how many people actually open up the paper
  22. 22. Was newspaper data better? • No one knows how many people actually open up the paper • Multipliers of “readers” versus “subscribers” is highly suspect
  23. 23. Was newspaper data better? • No one knows how many people actually open up the paper • Multipliers of “readers” versus “subscribers” is highly suspect • Chicanery to make numbers look higher is not unusual
  24. 24. Is there a better way? • Tony Haile of Chartbeat is pushing for ad prices based on time spent
  25. 25. Is there a better way? • Tony Haile of Chartbeat is pushing for ad prices based on time spent • The Washington Post has adopted A/B testing and “lead measures”
  26. 26. Is there a better way? • Tony Haile of Chartbeat is pushing for ad prices based on time spent • The Washington Post has adopted A/B testing and “lead measures” • In the age of distributed journalism, the metrics challenge has grown
  27. 27. Credits • Much of this presentation is based on “Confusion Online: Faulty Metrics and the Future of Digital Journalism,” by Lucas Graves, John Kelly and Marissa Cluck, Tow Center for Digital Journalism, Columbia University Graduate School of Journalism, September 2010. • Updated on December 6, 2016.

×