1. Netflix uses innovative techniques in their user interface to make finding movies an easy and personalized experience for their 5.7 million members. 2. They take a quantitative and qualitative approach to testing new design concepts and features through multiple prototypes and with real user data before full release. 3. Key features discussed include the star rating widget which predicts member ratings and recommendations, using Ajax techniques to simplify browsing and search, and testing new interactions to keep users engaged on the site.

1. Developing a Great User Interface A Netflix Case Study Sean Kane Director, User Interface Engineering UIE Web Application Summit - Monterey January 2007 Innovative techniques Netflix uses to make finding movies an easy, personalized experience.

2. Background Designer, Developer, Usability Research Product Development Presentation-Layer Architect UI Engineering Lead Ajax Developer

3. Connecting people to the movies they’ll love Unexpected value Design, technology, insight and culture Context Approach Examples

4. Quick Facts World’s largest online movie rental service 5.7 million members 70,000 titles – 200 Genres 41 Shipping Centers Over 1.4 million daily ships – 42 Million DVDs >95% of all titles are rented each quarter Two release week cycle

5. Development Approach Qualitative & Quantitative

6. EPIC Easy Personalized Innovative Cinematic

7. Intuition is only a Start Prove it Define metrics Qualitative and quantitative testing Find gems in tests Prove with real world data Don’t become emotionally involved Total Stars Added Number of Interactions Time to task completion

8. Lifecycle Concept Paper Prototype Usability Testing Functional Prototype Usability Testing Production-Ready Prototype Quantitative Testing Full Release Refine

9. Starbar Rating Widget Connect people with the movies they’ll love.

10. Stars Rental patterns != strong signal of taste Red Stars Average rating Predictive rating Yellow Stars Member rating Marks movie as seen Refresh page to update prediction Note change In red stars

11. Simplify

12. Star Power Over 1.5 billion ratings Users get yellow stars Rate anywhere Could be better Red stars are perceived as average

13. Back of the Box Using Ajax to make finding the right movie simple

14. Watching the User Patterns Repeating paths User requests Nonmember experience

15. What About BOB? More detail fast Prototype Mostly a success

16. The New Path Fascinating qualitative test Very favorable response Largest traffic swing Some glitches

17. Queue Add Confirmation Layer Keeping context

18. A Successful Page Browse Site Add Movie Queue Add Confirmation Return to Starting Point? User Pattern

19. New Interaction Keep people in context Deliver same content

20. Strong Opinions, One Winner Concept debate Qualitative testing Quantitative testing Launched with confidence Room for improvement

21. Recommendations Layer Not everything works as expected

22. Goals Establish rating/personalization connection Show personalization in action

24. Because you loved Toy Story… We recommend A Bug’s Life Close x Seen it? Rate it. If not Add It

28. Weak correlation Moving focus Timing Inconsistent behavior

29. Friends Recommendation Layer Obscurity

33. Release Day

34. If Untested Reactive Defensive Time wasted looking at bad data Quick fixes not tested Left guessing Seasonal obfuscation

35. Previews The textbook case-study

36. Variants No Control User Select Full Control Whack-a-mole

45. Easy

47. More Information Netflix is Hiring www.netflix.com/jobs Senior User Interface Engineers HTML, CSS, JavaScript, Java, Flash Senior User Interface Designers Senior Manager, Consumer Behavior and Analysis Sean Kane Blog: seankane.wordpress.com