Developing a Great User Interface

Slide 1: Developing a Great User Interface A Netflix Case Study Innovative techniques Netflix uses to make finding movies an easy, personalized experience. Sean Kane Director, User Interface Engineering UIE Web Application Summit - Monterey January 2007

Slide 2: Background • Designer, Developer, Usability Research • Product Development • Presentation-Layer Architect • UI Engineering Lead • Ajax Developer 2

Slide 3: • Connecting people to the movies they’ll love • Unexpected value • Design, technology, insight and culture • Context • Approach • Examples 3

Slide 4: Quick Facts • World’s largest online movie rental service • 5.7 million members • 70,000 titles – 200 Genres • 41 Shipping Centers • Over 1.4 million daily ships – 42 Million DVDs • >95% of all titles are rented each quarter • Two release week cycle 4

Slide 5: Development Approach Qualitative & Quantitative

Slide 6: EPIC Easy Personalized Innovative Cinematic 6

Slide 7: Intuition is only a Start • Prove it • Define metrics • Qualitative and quantitative testing •Total Stars Added •Number of Interactions • FindTime to task completion • gems in tests • Prove with real world data • Don’t become emotionally involved 7

Slide 8: Lifecycle Concept Paper Prototype Usability Testing Refine Functional Prototype Usability Testing Production-Ready Prototype Quantitative Testing Full Release 8

Slide 9: Starbar Rating Widget Connect people with the movies they’ll love.

Slide 10: Stars • Rental patterns != strong signal of taste • Red Stars – Average rating – Predictive rating • Yellow Stars – Member rating – Marks movie as seen Note change In red stars Refresh page to update prediction 10

Slide 11: Simplify 11

Slide 12: Star Power • Over 1.5 billion ratings • Users get yellow stars • Rate anywhere Could be better • Red stars are perceived as average 12

Slide 13: Back of the Box Using Ajax to make finding the right movie simple

Slide 14: Watching the User Patterns Repeating paths User requests Nonmember experience 14

Slide 15: What About BOB? More detail fast Prototype Mostly a success 15

Slide 16: The New Path Fascinating qualitative test Very favorable response Largest traffic swing Some glitches 16

Slide 17: Queue Add Confirmation Layer Keeping context

Slide 18: A Successful Page User Pattern Browse Site Add Movie Queue Add Confirmation Return to Starting Point? 18

Slide 19: New Interaction Keep people in context Deliver same content 19

Slide 20: Strong Opinions, One Winner Concept debate Qualitative testing Quantitative testing Launched with confidence Room for improvement 20

Slide 21: Recommendations Layer Not everything works as expected

Slide 22: Goals • Establish rating/personalization connection • Show personalization in action 22

Slide 23: 23

Slide 24: Because you loved We recommend A Bug’s Life Toy Story… Seen it? Rate it. If not Add It Close x 24

Slide 25: 25

Slide 26: 26

Slide 27: 27

Slide 28: • Weak correlation • Moving focus • Timing • Inconsistent behavior 28

Slide 29: Friends Recommendation Layer Obscurity

Slide 30: 30

Slide 31: 31

Slide 32: 32

Slide 33: Release Day 33

Slide 34: If Untested Reactive Defensive Time wasted looking at bad data Quick fixes not tested Left guessing Seasonal obfuscation 34

Slide 35: Previews The textbook case-study

Slide 36: Variants 1. No Control 2. User Select 3. Full Control 4. Whack-a-mole 36

Slide 37: 37

Slide 38: 38

Slide 39: 39

Slide 40: 40

Slide 41: 41

Slide 42: 42

Slide 43: 43

Slide 44: 44

Slide 45: Easy 45

Slide 46: 46

Slide 47: More Information Netflix is Hiring www.netflix.com/jobs Senior User Interface Engineers HTML, CSS, JavaScript, Java, Flash Senior User Interface Designers Senior Manager, Consumer Behavior and Analysis Sean Kane Blog: seankane.wordpress.com