Netflix's ability to easily switch subtitle languages was found to be its strongest feature compared to KT IPTV. A study had participants perform tasks on both IPTV and Netflix to evaluate ease of use and satisfaction. Netflix was rated higher for changing subtitles and parental controls, while navigation was rated similarly. Participants noted that easy language switching on Netflix would help children learn English and attract users interested in foreign language learning.

1. IPTV vs Netflix: Introduction
In 2016, The advent of Netflix was a huge threat to Korea Telecom (KT), the top IPTV provider.
To compare usability between IPTV and Netflix, KT held a company-wide research competition.
LIVE VIDEO
KIDS
• Live cable and VOD streaming are served simultaneously.
• Kids only service is available for parents.
• Mobile watching is not available.
MY CONTRIBUTION
BACKGROUND
IPTV ?
VS
Internet Protocol Television
SET-TOP BOX (STB)
• The KT’ IPTV, OLLEH TV, was No.1 in Korea.
• Netflix could be a huge risk for KT due to the multi-
screen feature and reasonable price.
• Hence, KT held a research competition to
investigate the relative weakness of IPTV
compared to Netflix for future defensive strategies.
• Led research team (size: 4)
• Planned research protocols.
• Conducted heuristic evaluation and task analysis
with think-aloud and analyzed user study data.
• Won 1st prize in the competition

2. IPTV vs Netflix: Research Scope
We conducted a heuristic evaluation to narrow research scope down and found three scenarios
where users feel serious inconvenient.
HEURISTIC EVALUATION
PURPOSE
• TARGET: YOUNG ADULTS (AGE 25 to 34)
They are potential users because the time
to select the first TV service is when they find
a new house for their new job or marriage.
• KT’S HYPOTHESIS: Young adults may prefer Netflix than IPTV
since they are generally sensitive to new
technology and new IT service.
• PURPOSE: a) Embody KT’s hypothesis into concrete
problems and research questions.
b) Outline the usage scenario of IPTV
c) Identify the pain points of KT IPTV
Figure1. The ranks of usability under each scenario
• EVALUATORS: - 4 of coworkers (age 25 to 34).
- Have used IPTV more than 1 year.
- Be familiar with diverse video platforms (i.e. YouTube).
• METHOD: - Compared 3 of IPTV in the market (KT, Competitor A and B).
- Organized the general usage scenarios.
- Ranked usability of all services per each scenario.
• RESULT: KT provided the lowest usability in the 3 scenarios.
(i.e. searching VOD, watching VOD, and using parental control)

3. IPTV vs Netflix: Research Question
In scenarios, we identified four discomforts that might have negative impacts on choosing our service and
set a research question for scrutinizing which one is the most critical.
Can’t Change Subtitle Languages
Too Deep to Explore
Parental Control still Exposes R-rated Movie Posters
Skips Too Far
Depth 1 2 3 4 5 6 Avg
KT 9 96 330 246 192 30 3.6
A 15 121 399 26 - - 2.8
B 6 74 431 - - - 2.7
Figure 2. Tree-Structured Menus Figure3. The number of menus per depth
• All IPTV used three-structured menus.
• ↔️ keys were used to access several sub-menus or go back to home, and
↕️ keys were used to select a particular menu.
• KT had the most deep structure and it forced to press buttons too many times.
• 5 min skip was too long for users who are
familiar with 10 sex skip of mobile streaming.
• It was inconvenient to find particular
moments users want to watch.
Subtitle
Dubbing
without
subtitle
Figure 4. Two Frozen VOD
distinguished by existence of subtitle
• American videos were sold as two versions with
English subtitle and without subtitle.
• Once users bought one of them, they couldn’t
change the subtitle settings.
RQ: Among these, what is the most critical inconvenience that would keep the users from using our IPTV service?
Figure 5. The poster UI of movies rated R
• R-rated film posters were often exposed to children.
• Although the parental control was activated,
posters were covered only with small lock image
and couldn’t hide violent image or nudity.

4. IPTV vs Netflix: Study
We conducted a task analysis by facilitating participants to simultaneously perform tasks on two TV services
and to evaluate their satisfaction by using Think-aloud and Survey.
• PURPOSE: - Verify the pain points we identified when using KT IPTV.
- Figure out if Netflix causes similar discomforts during same tasks.
• PARTICIPANTS: - 6 of users, age 25 to 34, married
- 3 of them have pre-school children
- They have used IPTV for more than 1 year
• METHOD: - Designed 4 tasks to test the 4 pain points previously identified
based on scenarios of watching foreign, R-rated, and kids’ movies.
- Asked to conduct tasks with KT IPTV and Netflix at the same time
by using think-aloud and fill out survey after each task was finished
to demonstrate their satisfaction qualitatively and quantitatively.
- Conducted tasks in participants’ house especially for those with
pre-school children by bring a smart television with Netflix installed
to mitigate the reluctance to visit laboratory due to children care.
Figure 6. The study environment in a living room
at a participant’s house
MAIN TASK SURVEY (Questions for Likert Scale) PURPOSE
1. Select up to three foreign movies
you want to watch.
How easy was it to search new
videos you want to watch?
To outline what is the pain points users
experience in the process of searching for
movie.
2. Play one of the movies you chose and find
the first scene where a hero appears.
How easy was it to navigate to the
scene that you were trying to find?
To examine if users can easily move to the
scene where they want to see.
3. Watch “Sofia Little Princess”, a foreign
kids’ movie with English subtitle.
How easy was it to change the
subtitle language?
To unpack if users can find the way to
watch foreign movies with English subtitle.
4. Turn on the Parental Controls and
search for an R-rated film.
How reliable was Parent Control
to protect your children?
To uncover if the Parental Control
filtered R-rated film correctly.
Figure 7. The list of tasks and questions

5. IPTV vs Netflix: Findings & Conclusion
Netflix’s strongest feature over IPTV was that it allows users to switch the subtitle language easily,
and it attracted users who were watching videos to learn foreign language.
USER SURVEY RESULT
QUOTES FROM
THINK-ALOUD
CONCLUSION
Task 1
Selecting a movie
Task 2
Navigating to a scene
Task 3
Changing subtitles
Task 4
Parental control
KT Netflix KT Netflix KT Netflix KT Netflix
P1 3 4 3 3 1 4 3 4
P2 4 3 2 4 1 5 2 3
P3 3 5 3 5 0 5 1 4
P4 3 3 2 5 1 5 0 5
P5 3 4 2 4 1 4 2 4
P6 2 4 2 4 0 5 0 5
Avg. 3.00 3.83 2.33 4.17 0.67 4.67 1.33 4.17
Diff. 0.83 1.83 4.00 2.83
• Participants admitted that IPTV has a bad interaction with video navigation, but it wasn’t a huge issue.
Given that KT’s IPTV provides the largest number of videos, inconvenience with navigation was ignorable.
• But they still noted us that they can consider changing to another TV service for specific purposes such as learning English and kids protection.
• Especially, participants were interested in learning English by watching English content for their kids as well as their own learning. Thus, easy
language settings of Netflix looked most attractive to them.
“I love Kids profile on
Netflix. If I can set a
password, I can rely
on it to block
hazardous content
from my kids.”
“This task is not difficult for
me since I already know
where each genre is
displayed based on my long
experience with this TV.”
“I usually watch American movies on
PC because I want to see English
subtitle and it is difficult to control
subtitle settings in KT IPTV. So I
prefer Netflix because I can change
subtitle language easily.”
“5 min skip is too long to
control, but it doesn’t matter. I
know KT has the largest
number of movies and TV
series, so I will keep using this
although UI is unwieldy.”
“If I watch animations with
kids in English on Netflix, it’ll
be more helpful for
language learning because
it’s easy to switch subtitles
to another language.”