The trouble with numbers

2,985 views

Published on

Why no one knows how many people are visiting their websites.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,985
On SlideShare
0
From Embeds
0
Number of Embeds
2,389
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

The trouble with numbers

  1. 1. Counting on confusion Why no one knows how many people are visiting their websites
  2. 2. What if everyone is wrong?
  3. 3. The mystery emerges • Boston.com, BostonHerald.com complain of lowballing
  4. 4. The mystery emerges • Boston.com, BostonHerald.com complain of lowballing • Internal counts are three times higher than Nielsen ratings
  5. 5. The mystery emerges • Boston.com, BostonHerald.com complain of lowballing • Internal counts are three times higher than Nielsen ratings • Whose count is accurate? Agencies’ or content-providers’?
  6. 6. Panel-based metrics • Nielsen and comScore sample what sites audience is visiting
  7. 7. Panel-based metrics • Nielsen and comScore sample what sites audience is visiting • Afternoon traffic is undercounted, local sites are a challenge
  8. 8. Panel-based metrics • Nielsen and comScore sample what sites audience is visiting • Afternoon traffic is undercounted, local sites are a challenge • Nielsen once gave washingtonpost.com 10m and comScore 17m
  9. 9. comScore is now the standard • About 2 million users have tracking code on their computers
  10. 10. comScore is now the standard • About 2 million users have tracking code on their computers • comScore uses polling techniques to weight and adjust its sample
  11. 11. comScore is now the standard • About 2 million users have tracking code on their computers • comScore uses polling techniques to weight and adjust its sample • Some users may have had code installed without permission
  12. 12. Server-side metrics • An actual count of incoming traffic. What could go wrong?
  13. 13. Server-side metrics • An actual count of incoming traffic. What could go wrong? • Visitors are counted by device. Less accurate if they clear cookies.
  14. 14. Server-side metrics • An actual count of incoming traffic. What could go wrong? • Visitors are counted by device. Less accurate if they clear cookies. • No way of separating out robotic hits from search engines
  15. 15. What is a unique visitor? • Most agree it is someone who visited once during time span
  16. 16. What is a unique visitor? • Most agree it is someone who visited once during time span • Unique visitors per month is most common Internet traffic measure
  17. 17. What is a unique visitor? • Most agree it is someone who visited once during time span • Unique visitors per month is most common Internet traffic measure • Other measures: return visits, number of page views, time spent
  18. 18. Dearth of free data • SimilarWeb works like comScore except its data are more open
  19. 19. Dearth of free data • SimilarWeb works like comScore except its data are more open • Quantcast gets good reviews from some, but code must be installed
  20. 20. Dearth of free data • SimilarWeb works like comScore except its data are more open • Quantcast gets good reviews from some, but code must be installed • Alexa, now owned by Amazon, offers an alternative approach
  21. 21. Was newspaper data better? • No one knows how many people actually open up the paper
  22. 22. Was newspaper data better? • No one knows how many people actually open up the paper • Multipliers of “readers” versus “subscribers” is highly suspect
  23. 23. Was newspaper data better? • No one knows how many people actually open up the paper • Multipliers of “readers” versus “subscribers” is highly suspect • Chicanery to make numbers look higher is not unusual
  24. 24. Is there a better way? • Tony Haile of Chartbeat is pushing for ad prices based on time spent
  25. 25. Is there a better way? • Tony Haile of Chartbeat is pushing for ad prices based on time spent • The Washington Post has adopted A/B testing and “lead measures”
  26. 26. Is there a better way? • Tony Haile of Chartbeat is pushing for ad prices based on time spent • The Washington Post has adopted A/B testing and “lead measures” • In the age of distributed journalism, the metrics challenge has grown
  27. 27. Credits • Much of this presentation is based on “Confusion Online: Faulty Metrics and the Future of Digital Journalism,” by Lucas Graves, John Kelly and Marissa Cluck, Tow Center for Digital Journalism, Columbia University Graduate School of Journalism, September 2010. • Updated on December 6, 2016.

×