Advertisement
Advertisement

More Related Content

Advertisement

Personalization - 10 Lessons Learned from Netflix

  1. PERSONALIZATION 10 LESSONS LEARNED from NETFLIX SUCCESS (and how to apply them to your video services)
  2. Personalization 10 Lessons Learned from Netflix’s Success (and how to apply them to video services)
  3. Greetings! Kauser Kanji Pancrazio Auteri Managing Editor, VOD Professional CTO, Contentwise
  4. Today’s objectives 2 1 Share the findings of observing how Netflix uses personalization as a competitive advantage in engaging and retaining users and planning content acquisition Show how you can connect the dots and take advantage of those findings for your online video services
  5. So what is personalization?
  6. ph. Thinkstock Not like this Personalized like this
  7. It’s about the pleasure of choice and the abundance of options It’s understanding needs and narrowing the options to make choosing a pleasant experience
  8. Personalization can go wrong
  9. It’s not just high-tech Brian & Doreen remember customers’ taste, curate and organize shelves, plan promotions, update the display windows, listen to requests and recommend products and let customers browse their shop in Somerset, UK
  10. Netflix Hulu Amazon Home page (web) Pure S-VOD S-VOD Prime Free, ad-based VOD Upsell S-VOD Hulu+ T-VOD
  11. So what’s the problem? Broad range of user types and tastes Fragmentation of content sources and applications: too many places to look at Content availability can be very dynamic over time Most UI solutions rely on drill-down and user efforts User’s attention span and screen real estate are very limited You name it!
  12. See the opportunity? Consumption increases Habits formation occurs Users feel more engaged Things can happen when people have a truly personalized experience Your service becomes a destination for unified discovery People talk about your brand with passion Subscribers perceive the value and the abundance of what you offer
  13. Let’s start!
  14. 10 lessons learned from Netflix The information, interpretations, advice and recommendations contained in this presentation are not endorsed in any way by Netflix and are based on information publicly shared by Netflix or its employees.
  15. 10 lessons learned from Netflix 1. Set objectives and pick metrics
  16. 10 lessons learned from Netflix 1. Set objectives and pick metrics Netflix: maximize member satisfaction and month-to-month subscription retention Example Metrics
  17. 10 lessons learned from Netflix 1. Set objectives and pick metrics Example Metrics Canceled subscriptions per month Interactive sessions resulting in a playback Played minutes per user per month Fully watched playbacks Interaction time before starting a playback Returning users
  18. 10 lessons learned from Netflix 2. Consider UX as mission-critical
  19. 10 lessons learned from Netflix 2. Consider UX as mission-critical secret sauce best practices innovative functionalities well-tuned business rules UX Engine Users Content offer Audiovisual quality Delivery infrastructure Editorial curation Customer support
  20. 10 lessons learned from Netflix 2. Consider UX as mission-critical UX Engine = Code Users Rules Widgets Behaviors Configurations Management tools
  21. 10 lessons learned from Netflix 2. Consider UX as mission-critical Changes in UI behavior can have a dramatic impact on key metrics. Multiscreen? Make sure behavior is consistent across devices Pay special attention to cross-screen consistency of Welcome screens Frequent user actions User “lost” actions Leverage the UX Engine to control UI behavior across all screens
  22. 10 lessons learned from Netflix 3. Personalize UX as much as possible
  23. 10 lessons learned from Netflix 3. Personalize UX as much as possible At Netflix, more than 75% of views come from some sort of recommendation or personalized ranking Views % from personalized ranking source: Netflix 0 25 50 75 100
  24. 10 lessons learned from Netflix 3. Personalize UX as much as possible User’s attention span is very limited The first 8-12 seconds are critical Weinreich et al. - ACM 2008 Desired outcomes - find something to watch - engage in some sort of exploration
  25. 10 lessons learned from Netflix 3. Personalize UX as much as possible Screen real estate is very limited too Ideally user should find relevant content in the first screen
  26. 10 lessons learned from Netflix 3. Personalize UX as much as possible A Ineffective sort criteria Effective criteria B Alphabetic C By release year Personalized order By ingestion order D Canned categories Even “computed” lists such as E Most popular Most viewed F Recently added and My list!
  27. 10 lessons learned from Netflix 3. Personalize UX as much as possible There are two “folds” Netflix personalizes in both directions 1. ranking of items in a carousel 2. ranking of carousels in the layout 2 Real estate “above-the-fold” 1
  28. 10 lessons learned from Netflix 3. Personalize UX as much as possible Featured content Resume play + My list { Most likely actions Popular Top picks for you Recently added Main genres Pseudo-genres Because you watched… Friends watching… Watch again Displayed in a personalized order" " Some of them disappear for a while if never “touched” " or because of A/B Testing
  29. 10 lessons learned from Netflix 3. Personalize UX as much as possible Personalized reordering may be disorienting for some people Indeed some Netflix users complain about this; but it seems to be effective and we’ll show you how to handle it
  30. Aha! My Aha! moment with NETFLIX
  31. 10 lessons learned from Netflix 4. Understand user’s lifestyle and context
  32. 10 lessons learned from Netflix 4. Understand user’s lifestyle and context Netflix mines usage data to extract behavior patterns Personalization may be affected by context elements Device type Time of the day Day of the week Season of the year User is at-home or out-of-home Geo-location (traveling, commuting, weekend-home…) Local weather Popular news Other users in close proximity (phones/wearables)
  33. 10 lessons learned from Netflix 5. Use interaction data then ask for feedback Priority on high-value usage events Playback start/stop/resume View asset details Add to personal list Other interactions Trick-play control Search Sharing Navigation paths… Ask for feedback 5-stars Like Dislike Love it!
  34. 10 lessons learned from Netflix 6. Let users know how the service is adapting to their tastes
  35. 10 lessons learned from Netflix 6. Let users know how the service is adapting to their tastes Promote trust in the system Encourage users to give feedback Better personalization
  36. 10 lessons learned from Netflix 6. Let users know how the service is adapting to their tastes Use meaningful labels referring to past behavior user can recognize Because you watched Breaking Bad Because of your interest for Time Travel Because you loved Kill Bill Vol.1
  37. 10 lessons learned from Netflix 7. Ensure metadata captures content nuances and is consistent
  38. 10 lessons learned from Netflix 7. Ensure metadata captures content nuances and is consistent Actors, Directors, Writers Genres Synopsis Release Year Duration Country Studio Language Characters Topics Themes Moods Locations Time Periods Keywords - Microtags “NETFLIX QUANTUM THEORY” A set of best practices for manual micro-tagging of video content Social acceptability of the lead character
  39. 10 lessons learned from Netflix 7. Ensure metadata captures content nuances and is consistent Let users search for content you don’t have
  40. 10 lessons learned from Netflix 7. Ensure metadata captures content nuances and is consistent With richer content metadata you can use analytics to understand content performance and drive content acquisition (or even original production) And add meaning to user profiles
  41. Movie Iron Man 3 Data from Gracenote-TMS
  42. TV Series Breaking Bad Data from Gracenote-TMS
  43. 10 lessons learned from Netflix 8. Give reasons to come back often Refresh catalog frequently - OR - Let the UX Engine do it for you (virtually)
  44. 10 lessons learned from Netflix 8. Give reasons to come back often Re-shuffle top items to periodically change the ones above-the-fold Items outside the first screen are still highly relevant for the user User perceives novelty and will be keen to return more often
  45. 10 lessons learned from Netflix 9. Run frequent UI experiments There is no “perfect way” and there are many types of users: experiments and adaptation seem to be the most effective ways Identify the UI elements on the path to the key goals Roll-out the variations and look at 2-5 metrics Run the experiments for two weeks or until statistical validity Design and plan experiments not to interfere with each other Experiments consume interaction events: make sure there is enough activity to feed all of the active variations
  46. 10 lessons learned from Netflix 10. Close the loop, base decisions upon data
  47. 10. Close the loop, base decisions upon data Netflix was the only network that said “We believe in you. We’ve run our data, and it tells us that our audience would watch this series. We don’t need you to do a pilot” Kevin Spacey, actor and producer Listen to Kevin saying this (video)
  48. 10 lessons learned from Netflix 10. Close the loop, base decisions upon data Netflix uses analytics to heavily influence the content acquisition policy Netflix proved to be agile and effective in rolling out variations and track several metrics across hundreds of client platforms Netflix team is very disciplined on reporting UI events. This enables full visibility in analytics and higher ROI Yes. At Netflix they go nuts for analytics! And they look to be right
  49. 10 Lessons from Netflix - Recap 1. Set objectives, pick metrics and share them with the team 2. Consider UX as mission-critical 3. Personalize UX as much as possible 4. Understand user’s lifestyle and context 5. Use interaction data then ask for feedback 6. Let users know your service is adapting to their tastes 7. Ensure metadata captures content nuances and is consistent 8. Give reasons to come back often 9. Run frequent UI experiments 10. Close the loop and base your decisions upon data
  50. Netflix solutions are applicable (and applied) at… Netflix Other services may include S-VOD as well as Linear TV, DVR, Transactional VOD, Pay TV, Pay-per view, music videos, sports highlights, Advertising or User-generated Content… We need a way to turn these lessons into practice touching all the stakeholders in our projects
  51. " UIDO A set of checklists to guide you while introducing personalization in your video service
  52. What you deliver How you start User Experience Integrator Experience UX IX DX OX Developer Experience Operator Experience How you build it Tools to manage " UIDO
  53. UX User Experience What you deliver
  54. UX User Experience What you deliver Content types ✓ Movies Aggregates ✓ Collections ✓ Series ✓ Episodes ✓ Extras ✓ Music videos ✓ Playlists ✓ News ✓ Sports events ✓ Sports highlights ✓ Scheduled programs ✓ Channels ✓ _____________________ ✓ Seasons ✓ Channel bundles ✓ Movie bundles ✓ Sports Team bundles ✓ Sports League bundles ✓ __________________
  55. UX User Experience What you deliver Key UX features ✓ Manually curated collections ✓ Search results ✓ Search suggestions while you type (single/multi-type) ✓ Search refine with smart filters (facets) ✓ Similar content ✓ Personalized picks for user ✓ Critics-based feed (Rotten Tomatoes, Metacritic…) ✓ Series you watch (with next-episode) ✓ VOD bookmarking (resume playback) ✓ User’s list ✓ Predictive browsing (surfacing folders) ✓ Personalized pseudo-genres
  56. Reference UI " Showing most of the personalization use cases supported by ContentWise
  57. UX User Experience What you deliver Key UX features (cont’d) ✓ Social graph (e.g. friends, followers) ✓ Sharing actions ✓ Content can be embedded ✓ Co-watching (blended profiles) ✓ Profile explanation with content metadata ✓ User can rate content (stars, like, dislike, love, etc.)
  58. UX User Experience What you deliver For kids ✓ Parental ratings ✓ Kids mode ✓ Specialized metadata (e.g. Commonsense) ✓ Editorial curation ✓ Curation by parents ✓ Analytics for parents
  59. UX User Experience What you deliver Content sources ✓ Linear schedule (line-ups) ✓ Start-over TV system ✓ VOD Catalog ✓ Local DVR ✓ Network DVR ✓ Reverse EPG (catch-up) ✓ ______________
  60. UX User Experience What you deliver Device types ✓ Phone ✓ Tablet ✓ PC ✓ TV ✓ Watch Access models ✓ S-VOD ✓ T-VOD ✓ Ad-VOD ✓ Free-Linear ✓ Pay-Linear ✓ PPV Profile types ✓ Personal ✓ Household ✓ Main account powers ✓ Blended ✓ Personas templates ✓ Personal on device Access locations ✓ At-home, OOH ✓ On-net, off-net
  61. UX User Experience What you deliver Entitlements ✓ S-VOD packages ✓ Rented movies ✓ Purchased movies ✓ Purchased seasons ✓ Purchased episodes ✓ Subscribed channels ✓ Subscribed bundles (e.g. Channel + S-VOD) ✓ ______________________
  62. Explaining a recommendation Because you liked these other movies Affinity between the user’s taste and the recommended movie (using the tag structure) ContentWise Reference UI
  63. OX Operator Experience How to manage
  64. OX Operator Experience How to manage ✓ Managing UI Elements with UX Engine ✓ Creating and updating editorial lists ✓ Generating and curating pseudo-genres ✓ Accessing analytics ✓ Content planning using analytics ✓ Managing variations and experiments for A/B Testing ✓ Understanding the impact of business rules on key metrics
  65. Personalized pseudo-genres INTENSE ACTION MOVIES mood genre type 2000s AUSTRALIAN THRILLER MOVIES release prod genre type year country AMERICAN DRAMA MOVIES STARRING TOM HANKS ContentWise Reference UI
  66. The magic of richer metadata MOVIES FROM FEMALE DIRECTORS type person role MOVIES STARRING A ROCKSTAR type gender from semantic enrichment looking into actors person role from semantic enrichment ContentWise Reference UI
  67. Curation of Pseudo-genres Metadata fields considered for labels Status of the pseudo-genre Type: Editorial or Computed ContentWise Management Console
  68. Driving from the UX Engine Rendered by UI code Configured by UX Engine ContentWise Management Console
  69. UX Engine - Personalized Order of Carousels
  70. Content planning - Choosing items to retire Find movies with a small number of “estimated” residual views and are “expiring” Automatically create a business rule The rule can be used in A/B Testing to anticipate the impact of removing these movies from the catalog. ContentWise Management Console
  71. A/B/C Testing Biz rule #1 Biz rule #N Variation A Biz rule #1 Biz rule #N Variation B Experiment Group A Group B Control Group Results Metrics Normal behavior
  72. Launching an experiment This is an experiment Treated user base Variations
  73. Monitoring experiment results Variations Affected users Metrics ContentWise Management Console
  74. DX Developer Experience How to build
  75. DX Developer Experience How to build UI Element ✓ Carousel ✓ Group of carousel ✓ Item ✓ Item attribute ✓ __________
  76. DX Developer Experience How to build UI Events to be reported ✓ Item displayed ✓ Item selected ✓ Item details accessed ✓ User rating submitted ✓ Explanation displayed ✓ Tuned-in ✓ Tuned-out ✓ Playback started ✓ Playback resumed ✓ Playback paused ✓ Playback completed ✓ Playback stopped ✓ Item saved to list ✓ Item removed from list ✓ Preview playback started
  77. IX Integrator Experience How to start
  78. IX Integrator Experience How to start ✓ Content model map ✓ Event model map ✓ User ID map ✓ Data refresh policy ✓ Bulk ingestion automation ✓ Delta updates automation ✓ Client applications map ✓ UI elements to be managed from UX Engine
  79. Thank you! For more information, please visit our website or contact us pancrazio.kauser.kanji@vodprofessional.com auteri@contentwise.tv Digital TV. Personalized www.vodprofessional.com www.contentwise.tv
Advertisement