Facts,  Figures &  Fictions Presentation to the Numis Securities Media Conference, London April 14 th  2011  by John Billett
<ul><li>7 * 7 = 48 (!) </li></ul>
Data has been around for ages, but there’s much that’s “new” <ul><li>Scale:   </li></ul><ul><ul><li>“ We are moving from s...
Loads of data <ul><li>We struggle with the unforeseen dilemma of what to do with the unimaginable quantities of data produ...
Loads of views & loads of data  <ul><li>“ Data are becoming a new type of raw material that’s on a par with capital & labo...
Our freedom to get it wrong <ul><li>These changes do  not  affect our ability to misinterpret data  </li></ul><ul><li>Inde...
<ul><li>4 + 4 = ? </li></ul><ul><li>Depends if you’re buying or selling </li></ul>
Buying or selling? 121 153 Japan  1.49 1.82 US  1.06 1.31 Euro Selling Buying Gatwick Currency Exchange
Buying or selling? Spread 23%-26%!! The “average” doesn’t exist! 121 153 Japan Yen 1.49 1.82 US $ 1.06 1.31 Euro Selling B...
Beware the Average <ul><li>The parable of the statistician who drowned  </li></ul><ul><li>in a pond 5cms deep -  on averag...
Beware the Average <ul><li>Data Set #1 </li></ul><ul><li>Average =  50 . Spread/dispersion 49-51 </li></ul><ul><li>Average...
Confusion of Cause & Effect <ul><li>Advertising  Sales </li></ul><ul><li>Massive FMCG advertiser uses “case rate” budget s...
Confusion of Cause & Effect <ul><li>Price of TV  Advertiser demand </li></ul><ul><li>Our work for ISBA & ITV proves that t...
False Accuracy <ul><li>There are no more than two significant figures </li></ul><ul><ul><li>87.6 & 88.4 are both 88 </li><...
Journalists love data fiction <ul><li>“ Tesco’s pre Xmas 2010 sales have declined” </li></ul><ul><li>“ The rate of growth ...
The Direct Marketing Data Fallacy – Certainty in Fiction #1 <ul><li>“ For the same money ad. X generated twice the respons...
The Direct Marketing Data Fallacy – Certainty in Fiction #2 <ul><li>“ The Google -charging for last click- proposition, eq...
The most common flaw –  “The Ecological Fallacy” <ul><li>“ The drawing of inferences about individual consumers based on a...
Conspiracy Theory, Common Sense, Coincidence & Cock Up <ul><li>The Spillers Flour Graders ads never appeared in the key ba...
Conspiracy Theory, Common Sense, Coincidence & Cock Up <ul><li>P&G grew market share & advertised in London only Monday-Th...
Attribution lies in the eye of  the beholder <ul><li>Chicago </li></ul><ul><li>5 year time series   (Republican claim) </l...
Words implying “precision” whose reality can be subject to large variation <ul><li>AVERAGE </li></ul><ul><li>BENCHMARK </l...
Measurement & Importance <ul><li>You can “measure”  what you can measure, but that may  not  be important </li></ul><ul><u...
Reject consumers’ claims & accept consumers’ behaviour <ul><li>“ We assume what we are being told is not the truth”   (Ame...
Four widespread data facts <ul><li>Chance events are often conspicuously misinterpreted as accomplishments or failure </li...
An arduous route from data to action <ul><li>“ Data & insights achieve nothing unless they reach the decision-makers and a...
Facts, Figures & Fictions  Check List <ul><li>#1 Averages are misleading unless you know the range, spread & dispersion of...
Facts, Figures & Fictions  Check List <ul><li>#3 When examining any data, always appreciate who commissioned & paid for th...
Facts, Figures & Fictions  Check List <ul><li>#5 Investigate & encourage time series data, not isolated one offs </li></ul...
Facts, Figures & Fictions  Check List <ul><li>#7 Are the observations fit for your purpose? Is there unexplained variabili...
Facts,  Figures &  Fictions Presentation to the Numis Securities Media Conference, London April 14 th  2011  by John Billett
References & Acknowledgements <ul><li>Patrick Barwise : London Business School </li></ul><ul><li>“ Freakonomics”: Steven L...
<ul><li>Johnbillett.com </li></ul><ul><li>22A Vincent House </li></ul><ul><li>Vincent Square </li></ul><ul><li>London SW1P...
Upcoming SlideShare
Loading in …5
×

Facts, Figures & Fictions

302 views

Published on

Presentation on the uses & misues of data, embracing illustrations & examples, as presented to the Numis Securities Media Conference in London April 2011

Published in: Business, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
302
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
7
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Facts, Figures & Fictions

  1. 1. Facts, Figures & Fictions Presentation to the Numis Securities Media Conference, London April 14 th 2011 by John Billett
  2. 2. <ul><li>7 * 7 = 48 (!) </li></ul>
  3. 3. Data has been around for ages, but there’s much that’s “new” <ul><li>Scale: </li></ul><ul><ul><li>“ We are moving from samples to audits” </li></ul></ul><ul><li>Speed: </li></ul><ul><ul><li>“ Last year to yesterday” </li></ul></ul><ul><li>Interactive: </li></ul><ul><ul><li>“ Output becomes input” </li></ul></ul><ul><li>Intensity: </li></ul><ul><ul><li>“ More pressure for immediate results” </li></ul></ul>
  4. 4. Loads of data <ul><li>We struggle with the unforeseen dilemma of what to do with the unimaginable quantities of data produced & received every year </li></ul><ul><li>2011 will generate data requiring the storage of 2,700 billion iPads </li></ul><ul><li>A single USB stick holds the equivalent of 20 tonnes of printed paper </li></ul>
  5. 5. Loads of views & loads of data <ul><li>“ Data are becoming a new type of raw material that’s on a par with capital & labour” (Bain & Company) </li></ul><ul><li>“ We shall soon be collecting data about each & every movement, transaction and more. There will be no excuse for not living lives of organised moderation every hour of every day” (Karen Canty: Enders Analysis) </li></ul>
  6. 6. Our freedom to get it wrong <ul><li>These changes do not affect our ability to misinterpret data </li></ul><ul><li>Indeed the apparent speed and alleged accuracy of data could even increase that misinterpretation </li></ul><ul><li>The fundamental “rules” of data have not changed </li></ul>
  7. 7. <ul><li>4 + 4 = ? </li></ul><ul><li>Depends if you’re buying or selling </li></ul>
  8. 8. Buying or selling? 121 153 Japan 1.49 1.82 US 1.06 1.31 Euro Selling Buying Gatwick Currency Exchange
  9. 9. Buying or selling? Spread 23%-26%!! The “average” doesn’t exist! 121 153 Japan Yen 1.49 1.82 US $ 1.06 1.31 Euro Selling Buying Gatwick Currency Exchange
  10. 10. Beware the Average <ul><li>The parable of the statistician who drowned </li></ul><ul><li>in a pond 5cms deep - on average </li></ul>
  11. 11. Beware the Average <ul><li>Data Set #1 </li></ul><ul><li>Average = 50 . Spread/dispersion 49-51 </li></ul><ul><li>Average Useful </li></ul><ul><li>Data Set #2 </li></ul><ul><li>Average = 50 . Spread/dispersion 25-75 </li></ul><ul><li>Average Crap </li></ul>
  12. 12. Confusion of Cause & Effect <ul><li>Advertising Sales </li></ul><ul><li>Massive FMCG advertiser uses “case rate” budget setting, with sales last year setting ad budget this year </li></ul><ul><li>Advertising Sales </li></ul>
  13. 13. Confusion of Cause & Effect <ul><li>Price of TV Advertiser demand </li></ul><ul><li>Our work for ISBA & ITV proves that the better value of TV the less advertisers spend </li></ul><ul><ul><li>Company profitability </li></ul></ul><ul><ul><li>Cash Flow </li></ul></ul><ul><ul><li>Consumer spending </li></ul></ul><ul><ul><li>Confidence </li></ul></ul><ul><li>Price of TV Advertiser demand </li></ul>
  14. 14. False Accuracy <ul><li>There are no more than two significant figures </li></ul><ul><ul><li>87.6 & 88.4 are both 88 </li></ul></ul><ul><li>% calculation is meaningless on a data base smaller than 100 observations </li></ul>
  15. 15. Journalists love data fiction <ul><li>“ Tesco’s pre Xmas 2010 sales have declined” </li></ul><ul><li>“ The rate of growth fell from 3.6% to 2.9%” </li></ul><ul><li>(Source: BBC Today Programme - Radio 4) </li></ul><ul><li>Tesco’s growth of 2.9% (lower than competitors’ growth rates) on significantly high sales the previous year meant Tesco had increased its market share </li></ul>
  16. 16. The Direct Marketing Data Fallacy – Certainty in Fiction #1 <ul><li>“ For the same money ad. X generated twice the response of ad. Y so its twice as cost effective” </li></ul><ul><li>This classic flawed data analysis overlooks:- </li></ul><ul><ul><li>Previous exposure patterns </li></ul></ul><ul><ul><li>Competitive activity </li></ul></ul><ul><ul><li>Other influences </li></ul></ul><ul><ul><li>Consumer experiences </li></ul></ul>
  17. 17. The Direct Marketing Data Fallacy – Certainty in Fiction #2 <ul><li>“ The Google -charging for last click- proposition, equates to cost per sales creation” </li></ul><ul><li>This classic flawed data analysis overlooks:- </li></ul><ul><ul><li>Previous exposure patterns </li></ul></ul><ul><ul><li>Contacts “lost” before last click </li></ul></ul><ul><ul><li>Value of non- internet exposure </li></ul></ul><ul><ul><li>Whole journey approach </li></ul></ul>
  18. 18. The most common flaw – “The Ecological Fallacy” <ul><li>“ The drawing of inferences about individual consumers based on aggregate level data analysis, results in the correct conclusions only by accident” (Robinson, Clancy, Berger et all) </li></ul><ul><ul><li>Market segmentation; targeting; clusters; mapping; promotions; etc. </li></ul></ul><ul><ul><li>16-34; ABC1; High Income: etc </li></ul></ul>
  19. 19. Conspiracy Theory, Common Sense, Coincidence & Cock Up <ul><li>The Spillers Flour Graders ads never appeared in the key baking season from Nov to Jan! </li></ul><ul><ul><li>Effectiveness? </li></ul></ul><ul><ul><li>Ad Stock? </li></ul></ul><ul><ul><li>Decay? </li></ul></ul><ul><ul><li>Financial Year Starts February </li></ul></ul>
  20. 20. Conspiracy Theory, Common Sense, Coincidence & Cock Up <ul><li>P&G grew market share & advertised in London only Monday-Thursday with Unilever only at Weekends </li></ul><ul><ul><li>Consumer receptivity? </li></ul></ul><ul><ul><li>Cheaper ad costs? </li></ul></ul><ul><ul><li>Econometric model insight? </li></ul></ul><ul><ul><li>Insufficient airtime </li></ul></ul>
  21. 21. Attribution lies in the eye of the beholder <ul><li>Chicago </li></ul><ul><li>5 year time series (Republican claim) </li></ul><ul><li>Police numbers increased Teenage crime reduced </li></ul><ul><li>20 year time series (Democrat claim) </li></ul><ul><li>New abortion laws High take up in poorer families Fewer Criminals </li></ul><ul><li>Teenage crime reduced </li></ul>
  22. 22. Words implying “precision” whose reality can be subject to large variation <ul><li>AVERAGE </li></ul><ul><li>BENCHMARK </li></ul><ul><li>DATA BASE </li></ul><ul><li>INDEX </li></ul><ul><li>NORM </li></ul><ul><li>STANDARD </li></ul><ul><li>SCORING </li></ul><ul><li>TARGET </li></ul>
  23. 23. Measurement & Importance <ul><li>You can “measure” what you can measure, but that may not be important </li></ul><ul><ul><li>Consumers’ claims about behaviour </li></ul></ul><ul><ul><li>Consumers’ intentions to act </li></ul></ul><ul><li>What is important may not be “measurable” in conventional terms </li></ul><ul><ul><li>Mixed media exposure & influence </li></ul></ul>
  24. 24. Reject consumers’ claims & accept consumers’ behaviour <ul><li>“ We assume what we are being told is not the truth” (Amelia Torode: VCCP) </li></ul><ul><li>“ Data based on claimed use of digital boxes indicated the end of live viewing and death of TV advertising. Reality is viewing +15% & more ads viewed” (Jeremy Tester BSkyB) </li></ul>
  25. 25. Four widespread data facts <ul><li>Chance events are often conspicuously misinterpreted as accomplishments or failure </li></ul><ul><li>The connection between actions & results is not as direct as we might like to believe </li></ul><ul><li>We benefit from looking beyond superficial explanations </li></ul><ul><li>Never have so many data been taken out of files & left un analysed or misread </li></ul>
  26. 26. An arduous route from data to action <ul><li>“ Data & insights achieve nothing unless they reach the decision-makers and are then acted upon. Fear, politics, denial etc in most companies make this a non-trivial issue” (Patrick Barwise London Business School) </li></ul>
  27. 27. Facts, Figures & Fictions Check List <ul><li>#1 Averages are misleading unless you know the range, spread & dispersion of observations around that average </li></ul><ul><li>#2 Demand to know the number of observations in any data base plus the scale of any claimed “norms” or “benchmarks” </li></ul>
  28. 28. Facts, Figures & Fictions Check List <ul><li>#3 When examining any data, always appreciate who commissioned & paid for the study & adjust accordingly </li></ul><ul><li>#4 Know the issues & challenges of the business whose data is under examination </li></ul>
  29. 29. Facts, Figures & Fictions Check List <ul><li>#5 Investigate & encourage time series data, not isolated one offs </li></ul><ul><li>#6 Check the data you have is complete, representative & meaningful. Avoid false attribution. </li></ul>
  30. 30. Facts, Figures & Fictions Check List <ul><li>#7 Are the observations fit for your purpose? Is there unexplained variability? Randomness, bias, error & chance are alive & well </li></ul><ul><li>#8 If it ain’t simple to understand – don’t believe it. Reject Google-de-goop. </li></ul>
  31. 31. Facts, Figures & Fictions Presentation to the Numis Securities Media Conference, London April 14 th 2011 by John Billett
  32. 32. References & Acknowledgements <ul><li>Patrick Barwise : London Business School </li></ul><ul><li>“ Freakonomics”: Steven Levitt & Stephen Dubner </li></ul><ul><li>“ The Drunkards Walk”: Leonard Mlodinow </li></ul><ul><li>“ The Ecological Fallacy”: Clancy, Berger & Magliozzi. Journal of Advertising Research </li></ul><ul><li>“ Facts from Figures”: Michael Moroney </li></ul><ul><li>Graeme Hutton: Universal McCann NY </li></ul><ul><li>“ Market Segmentation: Fallacies & Faults”: Tonks & Farr University of Lancaster. Marketing Review </li></ul><ul><li>Media Tel UK : MediaPost NY </li></ul><ul><li>Irene Pantazis: US Association of National Advertisers </li></ul><ul><li>Jeremy Tester: BSkyB </li></ul><ul><li>Andrew Walmsley </li></ul>
  33. 33. <ul><li>Johnbillett.com </li></ul><ul><li>22A Vincent House </li></ul><ul><li>Vincent Square </li></ul><ul><li>London SW1P 2NB </li></ul><ul><li>[email_address] </li></ul><ul><li>07836 200 321 </li></ul><ul><li>www.johnbillett.com </li></ul>

×