What Happens When You Set It Free? - Brian O'Leary

2,173 views

Published on

Success Metrics for Experimentation with Free Distribution - BookNet Canada Tech Forum 2009 - How do you know if your experiments with free digital content are working? Brian O'Leary, O'Reilly Media and Random House began a project in 2008 to characterize the “free” universe, catalog and assess recent experiments, and to establish ways to measure the benefit or cost of free distribution. In this session, Brian will report on the initial phase of this ongoing study, including a preliminary model of where and when free distribution works.

Published in: Education, Technology
  • Be the first to comment

What Happens When You Set It Free? - Brian O'Leary

  1. 1. Challenging notions of “free” BookNet Canada Tech Forum Brian O’Leary March 12, 2009
  2. 2. “ Free” is not “new” … <ul><li>A long and successful history </li></ul><ul><li>Galleys, ARCs, blads, sample chapters </li></ul><ul><li>Digital sampling on the rise </li></ul><ul><li>… but only a small set of experiments using fully “free” content </li></ul>
  3. 3. Why look at this topic now?
  4. 4. Our research approach The research is data-driven, open (without compromising publisher data) and structured to share knowledge. <ul><li>Collate experiments </li></ul><ul><li>Segment attributes </li></ul><ul><li>Identify data gaps </li></ul><ul><li>Use a consistent data source (POS feeds) </li></ul><ul><li>Measure pre- and post-release </li></ul><ul><li>Populate a structured matrix </li></ul><ul><li>Look at combined results </li></ul><ul><li>Initial impact of “free” </li></ul><ul><li>Initial take on the impact of pirated content </li></ul><ul><li>Share the initial analysis </li></ul><ul><li>Invite discussion </li></ul><ul><li>Grow the test sample </li></ul>
  5. 5. A value in structured testing … <ul><li>A robust set of variables </li></ul><ul><li>Appropriate segmentation </li></ul><ul><li>Captures content characteristics </li></ul><ul><li>Can collate like experiments </li></ul><ul><li>Can develop and test specific hypotheses and track results over time </li></ul>
  6. 6. Primary research findings <ul><li>The value of “free” is not binary </li></ul><ul><li>P2P “threat” may be overstated </li></ul><ul><ul><li>Low incidence </li></ul></ul><ul><ul><li>Significant lag </li></ul></ul><ul><ul><li>Technical skills are not commonly held </li></ul></ul>
  7. 7. Proposing a more nuanced model “ White” market “ Gray” market “ Back channel” <ul><li>Print sales </li></ul><ul><li>DRM-protected digital sales </li></ul><ul><li>“ Trialware” </li></ul><ul><li>Unprotected digital sales </li></ul><ul><li>Galleys, ARCs </li></ul><ul><li>“ Free” promotions </li></ul><ul><li>Unauthorized duplication </li></ul><ul><li>Pirated content </li></ul>Our current question: what impact does “free” have on sales?
  8. 8. What we tested … Testing free (Random House) Free PDF downloads (three for 1 day, one for 3 days, one for three weeks) Free PDF excerpt (3 weeks) Free ebook download (1 day) Free PDF, MP3 and ebook downloads (3 weeks) Monitoring P2P (O’Reilly) Sample set for peer-to-peer piracy Monitored three BitTorrent sites; only one (PirateBay) had more than one O’Reilly title posted Tracked activity of seeds (uploads) and leeches (downloads) for any 2008 O’Reilly front list titles found on these sites
  9. 9. What we found… Testing free (Random House) 8 titles, 12 formats tested in the first half of 2008 Sales up 19.1% during promotional period Sales up 6.5% during promotional and post-promotional periods Ranged from 155% up to 74% down Monitoring P2P (O’Reilly) 8 titles that were posted O’Reilly front list in 4Q 2008 Average post-seed sales were 6.5% higher in the four weeks after Ranged from 18.2% up to 33.1% down Low seed and leech volume Average first seeds appeared 20 weeks after publication date
  10. 10. Primary research findings <ul><li>The value of “free” is not binary </li></ul><ul><li>P2P “threat” may be overstated </li></ul><ul><ul><li>Low incidence </li></ul></ul><ul><ul><li>Significant lag </li></ul></ul><ul><ul><li>Technical skills are not commonly held </li></ul></ul>
  11. 11. The number of seeds peaks quickly
  12. 12. The number of leeches peaks immediately and quickly declines
  13. 13. Lag time before seeding varies Average = 20 weeks
  14. 14. Some research surprises… <ul><li>Number and range of “under the radar” free experiments available for analysis </li></ul><ul><li>Strong interest among trade publishers </li></ul><ul><li>Low volume of P2P incidence </li></ul><ul><li>Lag time on P2P seeding </li></ul>
  15. 15. Three useful cautions <ul><li>Correlation isn’t causality </li></ul><ul><li>Larger data sets may uncover a sample skew </li></ul><ul><li>What works today may not work as well at some future date </li></ul>
  16. 16. Next steps <ul><li>Additional tests are in work now </li></ul><ul><li>We continue to monitor P2P activity </li></ul><ul><li>More publishers can help fill in the test matrix </li></ul><ul><li>Now gathering feedback </li></ul><ul><li>Will continue to refine the analysis </li></ul>
  17. 17. For more information <ul><li>“ Rough Cut” research paper coming soon </li></ul><ul><ul><li>Includes research covered here </li></ul></ul><ul><ul><li>Also provides background on free and P2P </li></ul></ul><ul><ul><li>E-mail alert: http://toc.oreilly.com/reports/p2p/ </li></ul></ul><ul><li>[email_address] </li></ul>

×