Developing a Great User Interface A Netflix Case Study Sean Kane Director, User Interface Engineering UIE Web Application Summit - Monterey January 2007 Innovative techniques Netflix uses to make finding movies an easy, personalized experience.

Background Designer, Developer, Usability Research Product Development Presentation-Layer Architect UI Engineering Lead Ajax Developer

Designer, Developer, Usability Research

Product Development

Presentation-Layer Architect

UI Engineering Lead

Ajax Developer

Connecting people to the movies they’ll love Unexpected value Design, technology, insight and culture Context Approach Examples

Connecting people to the movies they’ll love

Unexpected value

Design, technology, insight and culture

Context

Approach

Examples

Quick Facts World’s largest online movie rental service 5.7 million members 70,000 titles – 200 Genres 41 Shipping Centers Over 1.4 million daily ships – 42 Million DVDs >95% of all titles are rented each quarter Two release week cycle

World’s largest online movie rental service

5.7 million members

70,000 titles – 200 Genres

41 Shipping Centers

Over 1.4 million daily ships – 42 Million DVDs

>95% of all titles are rented each quarter

Two release week cycle

Development Approach Qualitative & Quantitative

EPIC Easy Personalized Innovative Cinematic

Intuition is only a Start Prove it Define metrics Qualitative and quantitative testing Find gems in tests Prove with real world data Don’t become emotionally involved Total Stars Added Number of Interactions Time to task completion

Prove it

Define metrics

Qualitative and quantitative testing

Find gems in tests

Prove with real world data

Don’t become emotionally involved

Total Stars Added

Number of Interactions

Time to task completion

Lifecycle Concept Paper Prototype Usability Testing Functional Prototype Usability Testing Production-Ready Prototype Quantitative Testing Full Release Refine

Starbar Rating Widget Connect people with the movies they’ll love.

Connect people with the movies they’ll love.

Stars Rental patterns != strong signal of taste Red Stars Average rating Predictive rating Yellow Stars Member rating Marks movie as seen Refresh page to update prediction Note change In red stars

Rental patterns != strong signal of taste

Red Stars

Average rating

Predictive rating

Yellow Stars

Member rating

Marks movie as seen

Simplify

Star Power Over 1.5 billion ratings Users get yellow stars Rate anywhere Could be better Red stars are perceived as average

Over 1.5 billion ratings

Users get yellow stars

Rate anywhere

Could be better

Red stars are perceived as average

Back of the Box Using Ajax to make finding the right movie simple

Watching the User Patterns Repeating paths User requests Nonmember experience

Repeating paths

User requests

Nonmember experience

What About BOB? More detail fast Prototype Mostly a success

More detail fast

Prototype

Mostly a success

The New Path Fascinating qualitative test Very favorable response Largest traffic swing Some glitches

Fascinating qualitative test

Very favorable response

Largest traffic swing

Some glitches

Queue Add Confirmation Layer Keeping context

A Successful Page Browse Site Add Movie Queue Add Confirmation Return to Starting Point? User Pattern

New Interaction Keep people in context Deliver same content

Keep people in context

Deliver same content

Strong Opinions, One Winner Concept debate Qualitative testing Quantitative testing Launched with confidence Room for improvement

Concept debate

Qualitative testing

Quantitative testing

Launched with confidence

Room for improvement

Recommendations Layer Not everything works as expected

Goals Establish rating/personalization connection Show personalization in action

Establish rating/personalization connection

Show personalization in action

Because you loved Toy Story… We recommend A Bug’s Life Close x Seen it? Rate it. If not Add It

Weak correlation Moving focus Timing Inconsistent behavior

Weak correlation

Moving focus

Timing

Inconsistent behavior

Friends Recommendation Layer Obscurity

Release Day

If Untested Reactive Defensive Time wasted looking at bad data Quick fixes not tested Left guessing Seasonal obfuscation

Reactive

Defensive

Time wasted looking at bad data

Quick fixes not tested

Left guessing

Seasonal obfuscation

Previews The textbook case-study

Variants No Control User Select Full Control Whack-a-mole

No Control

User Select

Full Control

Whack-a-mole

Easy

More Information Netflix is Hiring www.netflix.com/jobs Senior User Interface Engineers HTML, CSS, JavaScript, Java, Flash Senior User Interface Designers Senior Manager, Consumer Behavior and Analysis Sean Kane Blog: seankane.wordpress.com