Heuristic Evaluation

2
TABLE OF CONTENTS
UX Goals
User Focused Outcomes
Landscape
Competition
Scorecard Summary

UX GOALS
+ PERSONAL & FRESH CONTENT
+ MORE DISCOVER, LESS BROWSE
+ MORE ORGANIZATION, LESS FILTERING
+ SEAMLESS AND ANYWHERE
+ “SNACKABLE” ENTERTAINMENT
+ TAILORED EXPERIENCE PER DEVICE
+ SIGNATURE MOBILE AND TABLET APPLICATIONS
+ COMPANION EXPERIENCE
+ EXTEND REACH OF CORE FEATURE SETS, E.G. DOWNLOAD ETC
+ EASE OF RESUMING AND WATCHING NEWLY OWNED CONTENT
+ REDESIGN HOME PAGE
+ NDA REMINDERS FOR NEW EPISODES
+ MAKE IT EASY FOR THE USER TO LEARN MORE ABOUT THEIR OWN TASTE
+ SHORT FORM, FREE CONTENT ON MOBILE DEVICES
+ “GRACEFUL DERESOLUTION” FOR SLOWER DEVICES
3

5
USER FOCUSED OUTCOMES
It’s easy
to personalize
Tailor
content
to my device
I like to discover
exciting
content
I always
discover new
features
I like
short
form
content
Work well on my high
speed & low speed devices
It doesn’t clutter
up my TV
It’s easy to
play/resume
media across my devices

Apple | Exploration + Discovery Desktop
Notes
1) Snackable content (clips) on the film detail
page enables users to make informed decis-
sions pre rental/purchase.
2) Related tab has the following filters “View-
ers also bought”, “More by these actors”,
“More by this director”, and “Top movies by
genre”.
3) Movie homepage content is organized by
genres, promotions, lists and
popularity. Discovering content within a bucket
is difficult because it relies heaviliy on a
horizontal scroll. The content buckets do not
have a horizontal scroll requiring signifigant
mouse travel to explore.
4) The quick links jump menu is a handy tool
for content discovery. It includes “ Top All-Time
Movies”, “Oscar Winners” and “New
Releases”.
The “Top Movies” & “Top Indie Movies” section
pushes trending movies based on popularity.
Below “Top Movies” & “Top Indie Movies”,
iTunes shows “Browse by Rating” and “Collec-
tions”.
Summary
iTunes has a large content catalog with
powerful functionality. Related content is
realitively easy to find. Browsing sets of
content is cumbersome because Apple only
supports horizontal scrolling. This content
pagination method requires significant mouse
travel.
Score
B
2
2
4
3
1
Top Line Summary
What’s Working
• Large catalog
• Short form content like extras, clips and trailers
• Genius recommendations (not perfect)
• Editorial programs
• Exploration filters
• How they merchandize trailers
• Device continuity
What’s Not Working
• Lack of algorithmic discovery
• Thick content navigational hierarchy. Cumbersome to traverse and explore TV & Movie content
• Lack of personalization
Score
B-
9

10
Apple | Exploration + Discovery Desktop
Notes
page enables users to make informed deci-
sions pre rental/purchase.
2) Related tab has the following filters “View-
ers also bought”, “More by these actors”,
“More by this director”, and “Top movies by
genre”.
3) Movie homepage content is organized by
genres, promotions, lists and
popularity. Discovering content within a bucket
is difficult because it relies heavily on a hori-
zontal scroll. The content buckets do not have
a horizontal scroll requiring significant mouse
travel to explore.
4) The quick links jump menu is a handy tool
for content discovery. It includes “ Top All-Time
Movies”, “Oscar Winners” and “New
Releases”.
The “Top Movies” & “Top Indie Movies” section
pushes trending movies based on popularity.
Below “Top Movies” & “Top Indie Movies”,
iTunes shows “Browse by Rating” and “Collec-
tions”.
Summary
iTunes has a large content catalog with
powerful functionality. Related content is
relatively easy to find. Browsing sets of
content is cumbersome because Apple only
supports horizontal scrolling. This content
pagination method requires significant mouse
travel.
Score
B
2
2
4
3
1

11
Apple | Exploration + Discovery Mobile
Notes
1) Featured content is organized by genres,
promotions, lists, and popularity. Promotion tile
rotates featured titles.
2) Under the “More” menu users can access the
“Apple Genius” recommendations feature.
3) Chart page sorts movie titles by popularity.
Users can filter popular films by genre.
4) “Related” tab displays one content bucket
“Customers Also Bought”.
page enables users to make informed decisions
pre rental/purchase.
Missed opportunities
• Surface genius recommendations on
the root media page.
• Phones are more conducive to short form
content, iTunes could improve user
engagement by scattering snackable
content throughout the app.
• Apple doesn’t have a concept of a mixed
aggregated launch page or a “My media”. All
content is siloed into Music, Movies, and TV
Shows
What’s cool
• Short form content like trailers, extras, clips,
and featurettes.
• Search. Users can search by soundtrack, title,
genre, actor, and editorial categories
Summary
iTunes app is light and device appropriate.
Some desktop functionality has been removed
to accommodate the small screen. Content
browsing feels easier on the app vs the desktop.
Score
B-
1
2
3
4
5

12
Apple | Exploration + Discovery Living Room
Notes
1) People that watched X also watched Z
2) More button simply scrolls the content into
hidden tabs (reviews, actor info, director info,
producer info, screenwriters)
3) By selecting a actor Apple TV displays all of
the actors filmography.
4) By selecting Director, Producer or Screen-
writer, Apple TV displays associated media.
• Genius recommendations are buried
home>movies>genius. A better solution would
be to merchandize personalized recommenda-
tions on the home screen.
• The search experience does not allow users
to search across all media types – i.e. users
would not be able to search for an actor and
get TV and Movie results.
• Home screen is heavily focused on apps vs
personalized content. A better solution would
be to truncate apps and maximize media
content.
• The search experience feels like it can be
simplified. For example, the caps, special
characters, and recent tab feels overly com-
plex relative to other LR UX.
Summary
Overall Apple TV is a good experience. Apple
TV shines in a few key areas – airplay, related
content on the media detail page, large movie
posters on the categorical filtering, and lack of
clutter.
1
2
4
3
Score
C

13
Apple | Continuity Across Device
Notes
Desktop
The desktop UX offers more in terms of
functionality. Horizontal scrolling content
paginating is unique, all other platforms have
a horizontal carousel.
Tablet
For the most part the iOS tablet app is
consistent with the desktop and mobile UX.
Smartphone
Generally speaking, the iOS app is consistent
with the desktop and tablet. Some features
are not available on the iOS app. For
example, the wish list feature is not available.
Living Room
Apple TV UX is the most unique.
Noticable difference include
• Nomenclature
• Related movies by actors, related movies by
director, related movies by producer, related
movies by screenwriter,
• Adding/removing wish list
• Minimized subnav menu
• Pagination carousel and “•••• “ panel
position indicator
• Different visual design affordances.
Score
A

Amazon | Exploration + Discovery Desktop
Notes
1) The “Most Popular” section is powerful and
allows users to filter content by media type, new
releases, purchase type, genre, actor, director,
customer review rating, price, and decade
2) Navigtation is organized by popular genres,
exclusives, etc.
3) Content merchandised in center column,
organized in a series of horizontal carousels.
4) Lists e.g IMDB Top Prime Movies, Top Prime
TV Seasons.
5) Related content is hidden in a hover context
menu.
6) Movie detail page includes recommendations
like Customers that watched X also watched Z.
• Desktop UX is not optimized for media brows-
ing the UI feels very e-commerce driven and
transactional.
• Movie recommendations are not presented in a
systematic way.
• Too many call to actions on the content detail
page. In a worse case scenario I’ve seen 5 call
to actions buttons ranging from “Watch Now” to
4 different versions of the“1-Click to buy”
buttons.
What’s cool
• Rich UGC reviews. Amazon shines with
reviews.
• Large content catalog and original content.
• Recommendations are very relevant, my guess
is that at some point they might mine my
e-commerce transactions and wish lists.
• Great performance.
• Offers VOD and Prime.
Summary
Amazon has a large content catalog with
powerful filtering. Explicit searches are very
good, conducting general browsing and media
discovery can be frustrating and time consum-
ing.
Score
C
2 3
4
2
6
3
1
5
Top Line Summary
What’s Working
• Prime and VOD
• Large catalog including original content
• Rich exploration filters (mainly on desktop UX)
• Search & reviews
• Cloud and device streaming
• Lack of algorithmic discovery
• Lack of personalization (except on desktop UX)
• Lacking short form content
• Desktop UX (optimized for checkout vs entertainment)
• Difficult to resume media across platform
• UX continuity
Score
C+
15

16
Amazon | Exploration + Discovery Desktop
Notes
1) The “Most Popular” section is powerful and
allows users to filter content by media type, new
releases, purchase type, genre, actor, director,
customer review rating, price, and decade
2) Navigtation is organized by popular genres,
exclusives, etc.
3) Content merchandised in center column,
organized in a series of horizontal carousels.
4) Lists e.g IMDB Top Prime Movies, Top Prime
TV Seasons.
5) Related content is hidden in a hover context
menu.
6) Movie detail page includes recommendations
like Customers that watched X also watched Z.
• Desktop UX is not optimized for media brows-
ing the UI feels very e-commerce driven and
transactional.
• Movie recommendations are not presented in a
systematic way.
• Too many call to actions on the content detail
page. In a worse case scenario I’ve seen 5 call
to actions buttons ranging from “Watch Now” to
4 different versions of the“1-Click to buy”
buttons.
What’s cool
• Rich UGC reviews. Amazon shines with
reviews.
• Large content catalog and original content.
• Recommendations are very relevant, my guess
is that at some point they might mine my
e-commerce transactions and wish lists.
• Great performance.
• Offers VOD and Prime.
Summary
Amazon has a large content catalog with
powerful filtering. Explicit searches are very
good, conducting general browsing and media
discovery can be frustrating and time consum-
ing.
Score
C
2 3
4
2
6
3
1
5

17
Amazon | Exploration + Discovery Mobile
Notes
1) Content recommendations, editorial features,
and promotions are surfaced on the homepage.
Users can easily flick through media via horizon-
tal carousels.
2) From home>Your Watch list users can toggle
filter Watch list by Movies or TV. Additionally
users can sort by Most Popular, Date Added, or
Title.
3) The sort by Genre features basically acts like
a main navigation.
• Amazon mobile doesn’t really have smart
recommendations based on your media
consumption. Many of the “Featured” categories
are pre-canned.
• There is a lack of snackable short form
content. It would be good to merchandise this
type of content on the home screen.
• Lacks personalization. The desktop UX has
“Recently Watched” and recommendations
based on my content consumption.
• The navigation is somewhat confusing. If a
user navigates from home>Your Watch list and
then selects Genre they can traverse to other
areas of the app. This is confusing because the
other push buttons to the right filter content
within your Watch list. It would be better to
re-label Genres to something that sounds more
like a main navigation interaction or use some-
thing like a 3 bar icon.
What’s cool
• Search. Users can search by title, genre, actor,
and editorial categories.
• Filtering is straight forward and powerful.
• Toggle between cloud and device library.
• UGC i.e. reviews.
Summary
iTunes app is light and device appropriate.
Some desktop functionality has been removed
to accommodate the small screen. Content
browsing feels easier on the app vs the desktop.
1
2
3
Score
B

18
Amazon | Exploration + Discovery Living Room
Notes
Home screen displays search, your watch list,
prime instant videos, movies, TV, video library,
deals and recommendations.
1) Movie page displays popular content,
editorial content, filtering by editor’s picks,
filtering by genre.
2) Movies page on the Playstation displays
categories like popular movies, popular prime
movies, editors choice, new releases top
rentals and recently watched.
• Search results are not optimized for the
screen real estate. On both Roku and Playsta-
tion results display in 1 long row.
What’s cool
• Easy to resume TV and movies content
• UGC i.e. reviews.
Summary
Overall the Amazon Living Room UX is well
done. Recommendations, editor’s Picks,
recently watched media and the content
filtering make discovery and exploration
intuitive and fast.
1
2
Score
B

19
Amazon | Continuity Across Device
Notes
Desktop
Across the Amazon ecosystem the living room &
desktop UX are the most unique. The desktop
UX has an entirely different visual language and
in some cases a
different content taxonomy. The video pages
are very dense with a loose visual hierarchy.
Tablet
The Amazon tablet UX is closely aligned with the
smartphone UX. Minor difference include
episodic content surfaced in a modal on tablet,
tablet supports portrait and landscape
orientation and nav positioning.
Smartphone
The Amazon smartphone UX is closely aligned
with the tablet UX. Minor difference would be
that the smart phone does not support land-
scape orientation.
Living Room
Across the Amazon ecosystem the living room &
desktop UX are the most unique. On the Roku
browsing is similar to Netflix. Content is in a 4
column layout with multiple long rows. The UX
on PS3 and XBOX 360 are very similar but very
different than the Roku UX.
Score
B

Netﬂix | Exploration + Discovery Desktop
Notes
1) On the Netflix homepage Netflix prominently
shows the last movie a user watched, prompts
them to review the title, then Netflix surfaces up
3 movie recommendations.
The first half of the Netflix homepage is centered
around personalization and user content
consumption, the last half of the page is dedi-
cated to canned categories like new releases,
popular titles, and genres.
2) Related content is surfaced at the bottom of
the media detail page. Users paginate through
sets of content via a mouse hover.
3) Users can filter catalog by clicking cast,
genres, director and categories.
4) Users can read and write reviews here.
5) Within a genre users can filter content by
different dimensions.
• On the content detail page, provide a way to
sort reviews by dimensions like lowest/highest,
recency etc
• Horizontal carousels are cumbersome because
they are invoked via a mouse click.
• The type ahead search displays movies that
are not in the current Netflix catalog. This
doesn’t happen on other netflix platforms. It
seems like an oversight.
What’s cool
recency etc
Summary
The Netflix desktop app is a great tool for
discovering new media content. Personalized
recommendations are based on viewing habits
and are prominently displayed on the
homepage.
Score
A
1
2
3
4
5
Top Line Summary
What’s Working
• Personalization
• Algorithmic discovery
• Taste profile
• Desktop exploration filters
• Resume media across platform
• UX continuity
• Snackable content e.g. trailers, clips, featurettes
• Small catalog
• Desktop search (they display titles not in catalog)
• Taste profile tweaking (only available on desktop)
• Difficult to “see more” personalized genres on certain devices
• Living room search UX (results persist across sessions)
Score
B+
21

22
Netﬂix | Exploration + Discovery Desktop
Notes
1) On the Netflix homepage Netflix prominently
shows the last movie a user watched, prompts
them to review the title, then Netflix surfaces up
3 movie recommendations.
The first half of the Netflix homepage is centered
around personalization and user content
consumption, the last half of the page is dedi-
cated to canned categories like new releases,
popular titles, and genres.
2) Related content is surfaced at the bottom of
the media detail page. Users paginate through
sets of content via a mouse hover.
3) Users can filter catalog by clicking cast,
genres, director and categories.
4) Users can read and write reviews here.
5) Within a genre users can filter content by
different dimensions.
recency etc
• Horizontal carousels are cumbersome because
they are invoked via a mouse click.
• The type ahead search displays movies that
are not in the current Netflix catalog. This
doesn’t happen on other netflix platforms. It
seems like an oversight.
What’s cool
recency etc
Summary
The Netflix desktop app is a great tool for
discovering new media content. Personalized
recommendations are based on viewing habits
and are prominently displayed on the
homepage.
Score
A
1
2
3
4
5

23
Netﬂix | Exploration + Discovery Mobile
Notes
1) At the top of the home screen Netflix displays
the last 4 unfinished titles.
2) Home screen currates content based on the
following dimensions instant queue, last
watched, popularity, genre, subject
matter+genre, because you watch X, new
releases, recently added, and popular on Face-
book.
3) Netflix does not have trailers, so anytime a
user presses the play icon the movie will start.
4) Instead of saying “Related” Netlix says “More
Like : Movie Title”. TV or Movie detail page is
surfaced in a pop up, closing returns the user to
the homepage or category.
5) Main navigation consists of standards genres
and TV shows.
• Search is not as powerful as other competitors.
Netflix does not allow you to search by genre,
actor, or director.
• Lack of short form content i.e. trailers, extras,
featurettes, and clips.
• No way to see more unstructured categories
like “Critically-acclaimed comedies or Inspiring
Biographical Documentaries.
What’s cool
• The ”Continue Watching” feature is very
responsive and handy when transporting media.
• Recommendations aid users in discovering
new content. The irreverent categories can be
fun.
Summary
Because Netflix has such a limited catalog,
personalization becomes very important. Netflix
does a great job of merchandising categories
and introducing users to new content.
2
1
4
3
5
Score
A

24
Netﬂix | Exploration + Discovery Living Room
Notes
1) Netflix on the Playstation is very different than
some of the other LR UX. The left jump menu
allows users to filter wall content. Pressing X on
the game control opens up a submenu. Playsta-
tion LR is very similar to Netflix desktop in that it
allows users to do robust browsing e.g. genre
and sub genre filtering.
2) Netflix on the XBOX is also very different. The
content is organized in horizontal rows. By
hovering over a media title, users get a preview
still of the TV/Movie title and a description.
3) Netflix layout on the Roku closely resembles
Netflix iPad and Netflix on the desktop. Poster
tiles include search results, instant queue,
genres, popularity etc. When a title is in focus,
system displays synopsis, year, review, rating,
quality and # of episodes.
• The search function is buried across all LR
platforms. On the Roku, the search button and
results appear on the second content row. On
the Xbox users have to select “Y” from the game
control
• On the Roku, search results are not presented
well. Search results are positioned on the 2nd
content row, replacing content. Results persist
until a new search is performed or when the
user closes the app. On the Xbox results are
displayed in 1 row, not utilizing TV screen real
estate.
Summary
Overall the Netflix liviing room UX is decent. All
platforms present movies in some sort of poster
tile format. The Roku UI is the most basic where
the Playstation UI has the most functionality.
Score
B
2
3
1

25
Netﬂix | Continuity Across Device
Notes
Desktop
Netflix desktop elegantly presents recommenda-
tions based on your consumption taste graph.
The desktop is the only experience where you
can modify your taste graph. Taste preferences
can be modified directly via the homepage or
under the “Taste Profile” section. The desktop
UX appears to have the most comprehensive
browsing/searching/discovery functionality.
Tablet
The Netflix tablet UX closely aligns with the
smartphone UX. The only noticeable difference
being that the film detail page is surfaced via
modal. Other than that, it appears that both
experiences were built from the same tech
stack.
Smartphone
The Netflix smartphone UX closely aligns with
the tablet UX. It appears that both experiences
were built from the same tech stack.
Living Room
The Netflix UX on XBOX, Roku and PS3 all use
a poster tile grid layout. Many of the LR
platforms display the posters different for
instance XBOX displays posters 3 dimensionally
in rows of 8 titles vs Playstation and Roku
displaying content in rows of 5.
Score
A

Hulu | Exploration + Discovery Desktop
Notes
1). Large rectangular media stills merchan-
dised on the homepage. Users can paginate
through 15 titles.
2) On the recommendation page, users can
filter content by TV, Movies, and TV & Movies.
Additional filters include Captions, Hulu Plus,
and HD
mark if they are interested in a title. If the user
clicks yes, then system prompts user to watch
now. If the users clicks no, the title is removed
and replaced.
4) If the user clicks the genre title, a massive
drop down appears with standard and non
standard genres. If user selects a new genre,
the content refreshes.
5) Staff picks span anything from what’s
trending to editorial categories.
• Search seems limited to genre and title. It
would be nice if allowed users to search by
director and actor.
What’s cool
• Some of the social commenting features are
pretty neat. Users can comment from their
feed and auto publish to hulu.com.
• Exploration is a snap with the recommenda-
tion section and the curated content.
• Hulu has a well of short form content
Summary
The Hulu desktop UX is powerful with a focus
on discovery and exploration. Although the
catalog isn’t as deep as other competitors you
feel as though there is a deep content well.
Score
A
1
2
3
4
5
Top Line Summary
What’s Working
• Personalization
• Algorithmic discovery
• Lean back continuous play for clips
• Content thumbnails & editorial layout
• Desktop exploration filters on desktop
• Resume media across platform
• UX continuity across desktop, mobile and tablet
• Living room search UX
• Living room content grid (content is cut off in weird spots, see appendix for examples)
• Smaller catalog mostly episodic content
Score
A
27

28
Hulu | Exploration + Discovery Desktop
Notes
1). Large rectangular media stills merchan-
dised on the homepage. Users can paginate
through 15 titles.
filter content by TV, Movies, and TV & Movies.
Additional filters include Captions, Hulu Plus,
and HD
mark if they are interested in a title. If the user
clicks yes, then system prompts user to watch
now. If the users clicks no, the title is removed
and replaced.
4) If the user clicks the genre title, a massive
drop down appears with standard and non
standard genres. If user selects a new genre,
the content refreshes.
5) Staff picks span anything from what’s
trending to editorial categories.
• Search seems limited to genre and title. It
would be nice if allowed users to search by
director and actor.
What’s cool
• Some of the social commenting features are
pretty neat. Users can comment from their
feed and auto publish to hulu.com.
• Exploration is a snap with the recommenda-
tion section and the curated content.
• Hulu has a well of short form content
Summary
The Hulu desktop UX is powerful with a focus
on discovery and exploration. Although the
catalog isn’t as deep as other competitors you
feel as though there is a deep content well.
Score
A
1
2
3
4
5

29
Hulu | Exploration + Discovery Mobile
Notes
1) Large TV/Movie stills stacked vertically
instead of being housed within a carousel.
2) Popular tab is global, defaults to episodic
content.
3) Media resuming message persist across all
sections.
4) Related videos display below the media
detail page.
5) More tab allows users to filter content
anytime..
• The only real issue I found is that search
does not support searching for actor, genre,
director etc.
What’s cool
• Large thumbnails and rectangular
Movie/TV stills
• Popular tab on main navigation.
Drives users to explore new content.
• Persistent media resuming message.
• Shows you watch under the more tab.
Summary
Hulu mobile is very discovery and exploration
focused. Their app seems targeted to TV
consumption, with certain filters defaulting to
TV>episodes. The short form content works
well in mobile context.
2
3
1
2
3
1
4
Score
A

30
Hulu | Exploration + Discovery Living Room
Notes
1) Large media stills auto rotates through
content. Trailers, TV, Movies, Clips and
recommendations display on the home
screen.
• The top carousel persists on many pages
and is not a good use of TV real estate.
• Virtually all screen functions are invoked
within a row and a rectangle, which can kind
of be confusing because there is a lack of
visual hierarchy.
What’s cool
• Lots of snackable content
• History of shows that I watch
• Top 100 clips
• Movie Trailers Opening Week
• Recommendations by category
• Beautiful title detail page
Summary
Overall Hulu Plus is a good experience. Hulu
has great recommendations, and it’s relatively
easy to discover new content. Hulu could
improve their UX by optimizing the screen real
estate and tightening up their visual hierarchy.
1
Score
B

31
Hulu | Continuity Across Device
Notes
Desktop
Hulu desktop is a beautiful experiences. The
feature set is powerful and is tailored to explora-
tion and discovery. The desktop UX is most like
the tablet UX. Noticeable difference between the
tablet UX and the desktop UX would be hidden
menus, social Facebook commenting, quick
resume message and search.
Tablet
Overall the Hulu tablet app is pretty easy to use.
The carousels are personalized and great for
content discovery. Many of the UX affordances
are consistent with the other platforms. Content
browsing is largely dependent on carousels.
Content layout is editorial and personalized.
Method for content filtering “TV > Popular” is
similar to the living room UX.
Smartphone
The smartphone app is unique in that the
content is not contained in carousels, instead
content is formatted in list views. Some of the
main navigation items are housed in the more
tab. Similar to the tablet UX but with a different
taxonomy. Most of the recommendations are
short form content i.e TV shows, clips and
trailers.
Living Room
Overall the Hulu living room UX is rich with
content but difficult to navigation. The layout is
consistent with other platforms in that content is
housed in horizontal carousels. The taxonomy is
consistent with other platforms.
Score
A

33
SCORECARD SUMMARY
Apple
Amazon
Netflix
Hulu
Exploration +
Discovery
Continuity
Across Device
Overall
UX
C+
C+
B+
B+
A
B
A
A
B-
C+
B+
A

Heuristic Evaluation

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Heuristic Evaluation

Similar to Heuristic Evaluation (20)

Recently uploaded

Recently uploaded (20)

Heuristic Evaluation