3. 3
Executive
Summary
Test
Objective
The
objective
for
this
user
research
test
was
to
determine
whether
a
video
becomes
less
effective
in
terms
of
recall
ability
with
extended
duration
instead
of
more
informative.
I
wanted
to
investigate
a
possible
connection
between
the
length
of
a
technical
instruction
YouTube
video
and
the
users
ability
to
recall
information.
Methodology
To
conduct
my
user
research
I
recruited
six
participants
to
view
a
technical
instructional
YouTube
video.
The
steps
are
as
follows:
1. Participants
began
with
a
background
questionnaire
to
provide
demographic
information,
experience
with
the
Internet,
experience
with
instructional
videos,
and
knowledge
of
basketball.
2. Participants
were
split
into
two
groups;
one
group
was
shown
a
short
video
(2:40
sec)
and
the
other
group
was
shown
a
longer
video
(5:38
sec).
3. After
watching
their
respected
videos
the
participants
were
administered
a
five-‐
question
quiz.
4. Following
the
quiz
was
a
brief
interview
to
understand
user
preferences.
Results
The
results
from
my
testing
indicate
that
the
longer
video
was
more
effective
for
user
retention
and
recall
of
information.
Average
quiz
scores
were
3.667/5
for
the
short
video
group
and
4.667/5
for
the
long
video
group.
The
interview
responses
from
those
who
watched
longer
videos
stated
that
they
liked
the
additional
description
and
use
of
professional
players
in
the
videos,
which
may
have
led
to
the
higher
test
scores.
4. 4
Introduction
Purpose
The
purpose
of
this
project
is
to
gain
information
about
how
effective
YouTube
instructional
videos
are
for
users.
I
seek
to
gain
an
understanding
about
how
long
videos
can
be
before
users
lose
focus
and
the
video
ceases
to
provide
adequate
knowledge.
Through
this
project
I
hope
to
notice
a
connection
between
the
length
of
a
video
and
the
users
ability
to
recall
information
from
the
video.
Current
research
in
the
area
of
technical
instructional
videos
has
shown
that
shorter
videos
are
more
effective
for
user
retention.
A
research
project
into
technical
instructional
videos
by
Matt
Morain
and
Jason
Swarts
(2012)
at
North
Carolina
State
University
looked
into
only
the
pacing
of
videos
and
not
the
overall
duration
but
found,
“Videos
that
were
too
slow
tempted
viewers
to
skip
ahead
and
thus
risk
missing
content”
(Morain
and
Swarts,
2012,
pg.
10).
I
believe
that
after
a
certain
length
of
time
the
user
will
no
longer
be
able
to
recall
important
information
from
the
technical
instructional
YouTube
video.
Research
Question
Does
the
duration
of
a
technical
instructional
video
affect
the
users
ability
to
recall
information?
I
want
to
determine
whether
a
video
becomes
less
effective
in
terms
of
recall
ability
with
extended
duration
instead
of
more
informative.
Methodology
Testing
Method
5. 5
To
conduct
my
user
research
I
recruited
six
participants
to
view
a
technical
instructional
YouTube
video.
The
steps
are
as
follows:
1. Participants
began
with
a
background
questionnaire
to
provide
demographic
information,
experience
with
the
Internet,
experience
with
instructional
videos,
and
knowledge
of
basketball.
2. Participants
were
split
into
two
groups;
one
group
was
shown
a
short
video
(2:40
sec)
and
the
other
group
was
shown
a
longer
video
(5:38
sec).
3. After
watching
their
respected
videos
the
participants
were
administered
a
five-‐
question
quiz.
4. Following
the
quiz
was
a
brief
interview
to
understand
user
preferences.
Measurements
The
measurements
I
used
to
conduct
my
user
research
on
technical
instructional
videos
include:
a
5-‐question
quiz
to
evaluate
the
ability
for
participants
to
recall
specific
information
from
the
video,
and
a
follow-‐up
interview
to
learn
the
participant’s
opinions
and
preferences.
The
quiz
questions
came
directly
from
the
information
provided
in
the
video
and
will
serve
as
a
baseline
measurement
to
assess
user
recall
for
my
discussion.
The
interview
regarding
the
users
preferences
also
provided
me
with
information
regarding
the
effectiveness
of
the
videos
in
question.
I
gained
an
understanding
of
how
focused
the
participants
were
during
the
entire
duration
of
the
video,
or
if
participants
lost
focus
after
a
certain
amount
of
time.
The
user
feedback
greatly
helped
me
understand
what
the
users
were
thinking
during
the
video.
Results
6. 6
Background
Questionnaire
The
background
questionnaire
revealed
that
all
of
the
participants
had
background
experience
with
using
the
Internet
and
YouTube.
Some
of
the
participants
stated
they
were
quite
knowledgeable
on
the
topic
of
basketball,
but
when
compared
to
the
quiz
scores
background
knowledge
did
not
equate
to
higher
scores
than
the
less
knowledgeable
participants.
See
Figure
1
below
for
the
background
questionnaire
responses.
P1
P2
P3
P4
P5
P6
Q1:
Gender
female
female
male
male
male
female
Q2:
Age
10-‐17
30-‐39
18-‐29
50-‐59
18-‐29
18-‐29
Q3:
Occupation
student
management
student
sales
director
student
marketing
assoaciate
Q4:
How
do
you
most
often
use
a
computer?
(check
all
that
apply)
entertainment,
music,
internet
work,
personal
email,
internet
work,
word
processing,
personal
email,
entertainment,
music,
internet
work,
personal
email,
internet
word
processing,
entertainment,
music,
internet
work,
word
processing,
personal
email,
music,
photo/video,
internet
Q5:
How
often,
on
average,
do
you
use
a
computer?
(check
one)
weekly
daily
daily
daily
daily
daily
Q6:
How
long
have
you
been
using
a
computer?
(check
one)
6-‐10
10+
10+
10+
10+
10+
Q7:
How
often
do
you
use
the
Internet
(not
including
email)?
(check
one)
7. 7
weekly
daily
daily
daily
daily
daily
Q8:
How
comfortable
are
you
with
using
the
Internet
(not
including
email)?
5
5
5
5
5
5
Q9:
How
long
have
you
been
an
Internet
user?
(check
one)
3-‐5
10+
10+
10+
10+
10+
Q10:
How
often
do
you
use
YouTube?
(check
one)
weekly
weekly
monthly
weekly
daily
weekly
Q11:
How
often
do
you
view
technical
instructional
videos
on
YouTube?
(check
one)
more
than
once
more
than
once
more
than
once
once
regularly
more
than
once
Q12:
How
much
knowledge
and
experience
do
you
have
with
basketball?
3
1
4
4
4
2
Quiz
Data
The
results
from
the
quiz
are
illustrated
in
below.
The
orange
corresponds
to
participants
in
the
short
video
group
and
blue
corresponds
to
participants
in
the
long
video
group.
The
red
fill
represents
an
incorrect
answer
to
the
quiz
question.
The
results
are
as
follows:
Short
Video
Group
• Participant
1
scored
3
out
of
5
• Participant
3
scored
5
our
of
5
• Participant
5
scored
3
out
of
5
Long
Video
Group
• Participant
2
scored
5
out
of
5
• Participant
4
scored
5
out
of
5
• Participant
6
scored
4
out
of
5
Figure
1
8. 8
The
scores
from
each
participant
and
the
average
score
for
each
video
group
is
illustrated
in
Figure
2
below.
The
average
score
for
the
short
video
group
was
3.667
and
the
average
score
for
the
long
video
group
was
4.667,
an
entire
point
higher.
Interview
Data
The
responses
the
participants
provided
to
the
interview
questions
greatly
helped
to
understand
the
discrepancy
between
the
quiz
scores.
The
responses
about
the
videos
the
participants
were
shown
are
all
similar.
All
of
the
participants
felt
that
the
length
of
the
video
their
were
shown
was
appropriate.
The
difference
in
answers
between
the
short
video
group
and
the
long
video
group
is
that
the
long
video
group
all
mentioned
they
enjoyed
the
demonstrations
for
each
step
and
the
inclusion
of
a
clip
of
a
professional
3.667
4.667
0
1
2
3
4
5
P1
P3
P5
Average
(short)
Average
(long)
P2
P4
P6
Score
(out
of
5)
Short
Video
Group
Long
Video
Group
Quiz
Scores
Figure
2
9. 9
basketball
player.
The
fact
that
all
three
of
the
participants
in
the
long
video
group
stated
they
liked
the
professional
may
have
led
to
their
higher
quiz
scores.
The
interview
response
are
listed
in
Figure
3
below:
P1
P2
P3
P4
P5
P6
Q1:
On
a
scale
of
1
to
5
(with
5
being
very
appropriate
and
1
being
very
inappropriate)
do
you
feel
about
the
length
of
the
video?
5
5
4
4
5
4
Q2:
Did
the
video
grab
your
attention?
yes,
very
informative
yes,
learned
something
new
yeah,
instructor
&
demonstrator,
different
camera
angles
yes,
upfront
content,
casual
manner
if
I
was
looking
to
learn
then
yes
yeah
Q3:
Were
you
focused
throughout
the
duration
of
the
video?
yes
yes
no,
I'm
tired
yes
yes
yeah
definitely
(I
think
knowing
I
was
getting
quizzed
played
into
that)
Q4:
Was
there
anything
you
liked/disliked
about
the
video?
very
informational
in
describing
steps
liked
shots
of
NBA,
different
angles,
5
steps
beginning/r
ecap
liked-‐
explained
steps
quantitative.
Disliked-‐
more
broad
(other
styles
or
methods)
liked-‐
video
stopped
and
broke
down
separate
parts,
discussion
dip
v.
not
dip
ray
allen
liked-‐
execution
of
examples,
enough
example
and
elaboration.
Disliked-‐
distracted
by
bouncing
balls
in
background
liked-‐
how
laid
out
steps,
support,
protips
Discussion
The
results
from
the
quiz
and
the
interview
questions
were
surprising
and
informative.
The
current
research
into
the
area
of
technical
instructional
videos
shows
that
the
shorter
the
video
the
better.
Long
videos
with
a
more
formal
tone
are
less
effective;
Figure
3
10. 10
videos
with
slow
pacing
a
tempt
viewers
to
skip
ahead
and
miss
content
(Pflugfelder,
2013,
pg.
133)
(Morain
and
Swarts,
2012,
pg.
10).
Users
want
to
receive
the
information
in
as
little
time
as
possible,
but
the
results
from
my
user
research
into
technical
instructional
videos
provided
a
different
result.
I
found
that
the
longer
video
(5:45
sec)
was
more
effective
at
teaching
participants
the
steps
of
shooting
a
jump
shot
in
basketball.
My
research
question
inquired
whether
a
longer
instructional
video
becomes
ineffective
because
of
possible
attention
loss,
which
would
lead
to
lower
rates
of
retention.
The
data
collected
from
the
quiz,
to
measure
retention,
and
the
interviews
showed
that
the
longer
video
was
more
effective
than
the
shorter
video.
Both
the
quiz
scores
and
the
interview
responses
lead
to
the
conclusion
that
the
longer
video
was
more
effective
and
lead
to
higher
rates
of
retention.
There
may
be
multiple
factors
that
lead
to
higher
quiz
scores
in
the
longer
video
group
that
I
will
discuss
below
Background
Questionnaire
Discussion
The
background
questionnaire
provided
an
understanding
of
the
demographics
of
the
participants.
All
of
the
participants
stated
that
they
are
very
comfortable
with
using
the
Internet
and
have
at
least
some
experience
with
technical
instructional
videos
on
YouTube.
The
background
understanding
of
the
Internet
and
YouTube
may
have
led
to
higher
scores
on
the
quizzes
than
non-‐internet
users.
When
comparing
the
quiz
scores
to
the
background
information
P2
stands
out
because
they
responded
to
question
12
(How
much
knowledge
and
experience
do
you
have
with
basketball?)
with
a
1
meaning
very
little
knowledge,
but
after
watching
the
long
video
scored
5/5
on
the
quiz.
P6
responded
to
have
little
knowledge
about
basketball
(2)
but
was
11. 11
able
to
score
4/5
on
the
quiz
after
watching
the
long
video.
P1
stated
to
be
somewhat
knowledgeable
(3)
but
scored
3/5
on
the
quiz
after
viewing
the
short
video.
P3
responded
to
be
knowledgeable
about
basketball
(4)
and
scored
5/5
on
the
quiz
after
viewing
the
short
video.
I
believe
that
the
extended
demonstrations
and
further
discussion
about
the
steps
in
the
long
video
are
more
beneficial
to
users
with
less
knowledge
and
experience
with
basketball.
Quiz
Results
Discussion
The
average
scores
from
the
two
groups
shows
that
the
longer
video
was
more
effective
in
terms
of
participant’s
ability
to
recall
the
information.
The
short
video
group
received
an
average
score
of
3.667;
the
long
video
group
received
a
score
of
4.667.
The
group
of
participants
that
viewed
the
longer
video
scored
on
average
1
point
higher
on
the
quiz.
This
data
supports
the
conclusion
that
the
longer
video
was
more
effective.
The
results
from
the
quiz
scores
showed
a
clear
distinction
between
the
two
videos.
P1
and
P5
scored
3
out
of
5
on
the
quiz
and
both
got
question
1
(What
percent
of
jump
shots
does
the
video
say
you
can
make
with
this
form?)
and
question
4
(How
far
forward
should
your
shooting
foot
be
(in
shoe
lengths)?)
wrong.
It
might
be
the
case
that
the
answers
to
those
specific
questions
were
not
covered
in
as
much
depth
as
with
the
longer
video.
P6
also
got
one
of
the
same
questions
wrong
on
the
quiz
(Q4),
which
could
indicate
that
this
was
a
difficult
question
that
required
further
elaboration
that
was
not
present
in
the
shorter
video.
The
longer
video
breaks
down
the
step
with
still
images
of
how
much
further
forward
the
shooting
foot
should
be.
Neither
video
includes
written
words
of
how
much
this
distance
should
be,
which
may
have
lowered
the
retention
of
this
piece
of
information.
12. 12
The
shorter
video
contained
the
same
amount
of
content
as
the
longer
but
lacked
the
extended
demonstration
and
explanation
of
the
process.
My
the
quiz
data
suggests
that
for
those
who
were
less
familiar
with
basketball
would
have
higher
levels
of
retention
from
a
longer
instructional
video
that
goes
more
in
depth.
Interview
Results
Discussion
The
results
from
the
interviews
also
suggest
that
the
longer
video
was
more
effective
because
of
the
use
of
more
examples.
All
but
one
of
the
participants
responded
that
they
were
focused
throughout
the
duration
of
their
respective
videos.
P3,
who
watched
the
short
video,
responded
to
the
question
of
being
focused
throughout
with,
“no,
I’m
tired”.
The
participants
energy
level
was
not
important
to
the
discussion
of
length
of
a
video
because
P3
got
all
of
the
answers
right
on
the
quiz.
P3
did
state
that
they
were
knowledgeable
about
basketball
on
the
background
questionnaire,
which
may
have
helped
them
score
high
on
the
quiz.
The
interesting
thing
about
the
interview
responses
from
P3
do
not
match
up
with
the
video
they
were
shown.
When
asked
about
the
appropriateness
length
of
the
video
P3
rated
the
video
4
on
a
scale
with
5
being
very
appropriate
and
1
being
very
inappropriate,
and
the
participant
also
stated,
“good
length
(shorter
is
better),
maybe
more
animations”.
Later
in
the
interview
P3
stated
that
he
disliked
the
lack
of
explanation
of
the
steps
in
the
video
and
wanted
other
possible
styles
or
at
least
reasons
why
the
selected
method
is
the
best
option.
These
statements
actually
support
the
findings
that
the
longer
video
was
more
effective
for
this
audience.
All
three
of
the
participants
in
the
long
video
group
(P2,
P4,
and
P6)
stated
that
they
liked
the
demonstrations
and
explanations
for
each
step
of
the
process
of
shooting
a
13. 13
basketball
jump
shot.
All
three
also
stated
that
they
liked
the
discussion
of
the
steps
and
the
inclusion
of
the
NBA
player,
Ray
Allen.
P2
said
in
the
interview
that
she,
“liked
shots
of
NBA”.
Because
all
three
of
the
participants
stating
that
they
liked
the
real
footage
of
a
professional
player,
the
additional
information
and
credibility
from
the
clips
of
Ray
Allen
helped
the
participants
retain
the
information.
Likert
Scale
Response
Discussion
The
participants
were
asked
after
viewing
the
video
to
rate
how
appropriate
they
felt
the
video
was
on
a
scale
of
1
to
5
(1
being
very
inappropriate
and
5
being
very
appropriate).
The
data
collected
from
this
response
was
inconclusive.
Both
videos
received
high
scores.
The
average
score
of
the
short
video
group
was
4.667
compared
to
the
average
score
of
the
long
video
group
being
4.333.
Both
scores
indicate
that
the
participants
thought
the
length
of
the
video
was
appropriate
for
the
given
task
and
purpose.
The
short
video
group
did
rate
the
appropriateness
of
the
length
of
the
video
higher
even
though
they
received
lower
scores
on
the
quizzes.
The
short
video
group
may
have
rated
the
appropriateness
higher
because
viewers
seek
to
watch
as
short
of
a
video
as
possible
with
an
equal
amount
of
information.
While
both
videos
go
over
the
same
amount
of
information
for
the
technical
process
of
shooting
a
basketball
jump
shot,
the
longer
video
goes
into
more
discussion
and
demonstration
of
each
step.
The
participants
in
the
long
video
group
might
have
desired
to
watch
a
shorter
video
on
the
same
subject
but
received
higher
scores
because
of
the
reinforcement
of
the
information.
The
longer
video
was
more
effective
for
retention
but
might
be
less
effective
for
the
intended
users
because
they
might
not
watch
as
long
of
a
video.
14. 14
I
believe
the
high
ratings
for
both
videos
may
be
artificially
high
because
all
of
the
participants
have
a
relationship
with
me
and
wish
to
please
me.
Because
I
was
the
subject
of
the
videos
and
the
moderator
I
feel
that
the
participants
may
not
have
provided
an
accurate
response
to
the
appropriateness
of
the
length
of
the
videos.
If
the
participants
and
moderator
had
been
neutral
the
scores
may
have
been
different.
15. 15
Appendix
A
YouTube
Instructional
Video
User
Research
Project
Plan
Brad
Arrington
University
of
Minnesota
Spring
2015
17. 17
Intro
and
Summary
The
purpose
of
this
project
is
to
gain
information
about
how
effective
YouTube
instructional
videos
are
for
users.
I
seek
to
gain
an
understanding
about
how
long
videos
can
be
before
users
lose
focus
and
the
video
ceases
to
provide
adequate
knowledge.
Through
this
project
I
hope
to
notice
a
connection
between
the
length
of
a
video
and
the
users
ability
to
recall
information
from
the
video.
I
believe
that
after
a
certain
length
of
time
the
user
will
no
longer
be
able
to
recall
important
information
from
the
technical
instructional
YouTube
video.
This
project
only
compares
video
instructions
and
does
not
compare
to
print
medium,
however
a
lot
of
current
research
between
the
two
media
have
shown
that
technical
instructional
videos
lead
to
higher
rates
of
retention
than
print.
Kara
Poe
Alexander
(2013)
at
Baylor
University
found,
“[Users]
also
retained
more
information
about
the
task
after
they
had
watched
a
video
than
after
they
viewed
the
print
instructions”
(Alexander,
2013,
pg.
251).
Continuing
with
the
idea
that
users
have
higher
rates
of
information
retention
from
video
instructions
over
print,
I
want
to
gain
information
about
whether
duration
of
the
video
affects
retention
among
users.
Another
research
project
into
technical
instructional
videos
by
Matt
Morain
and
Jason
Swarts
(2012)
at
North
Carolina
State
University
looked
into
only
the
pacing
of
videos
and
not
the
overall
duration
but
found,
“Videos
that
were
too
slow
tempted
viewers
to
skip
ahead
and
thus
risk
missing
content”
(Morain
and
Swarts,
2012,
pg.
10).
Their
findings
of
users
skipping
ahead
in
the
video
due
to
slow
pacing
could
also
be
a
factor
if
users
feel
that
a
video,
with
adequate
pacing,
is
simply
too
long
and
decide
to
skip
ahead.
18. 18
Ehren
Helmut
Pflugfelder
(2013)
makes
interesting
contrasts
between
video
manuals
and
web
apps.
He
defines
large
technical
instructional
videos
as
manuals,
which
provide
great
detail,
visuals,
and
numerous
chapters.
These
are
large
manuals
such
as
a
GMC
owner’s
manual.
Pflugfelder
uses
the
example
of
the
website
Dropbox
to
compare
the
difference
to
video
manuals
to
a
web
app,
which
in
this
case,
uses
a
casual
tone
and
provides
a
general
overview.
The
Dropbox
web
app
provides
the
necessary
information
within
its
two-‐minute
playing
time
(Pflugfelder,
2013,
pg.
133).
The
distinctions
Pflugfelder
makes
provide
a
place
to
start
while
analyzing
the
retention
and
preferences
of
users
for
technical
instructional
videos.
Do
users
prefer
a
causal-‐tone
short
video
or
a
formal-‐voice
longer
video
that
provides
more
detail
into
the
subject
matter?
Research
question
Does
the
duration
of
a
technical
instructional
video
affect
the
users
ability
to
recall
information?
I
want
to
determine
whether
a
video
becomes
less
effective
in
terms
of
recall
ability
with
extended
duration
instead
of
more
informative.
Methodology
To
perform
this
test
of
the
usability
in
terms
of
learnability
for
a
technical
instructional
video
I
will
invite
6
participants
to
view
an
instructional
video.
The
first
group
will
watch
a
longer
video
that
is
over
5
½
minutes
long
and
explains
the
process
of
shooting
a
perfect
jump
shot
in
basketball.
The
second
group
will
view
an
instructional
19. 19
video
on
the
same
topic
of
shooting
a
perfect
jump
shot
but
the
video
will
be
half
as
long,
2
¾
minutes
in
duration.
I
will
divide
participants
into
2
groups
(Group
A
and
Group
B).
Participants
in
Group
A
will
be
shown
the
longer
video
and
participants
in
Group
B
will
be
shown
the
shorter
video.
After
participants
in
both
groups
finish
their
respective
videos
I
will
administer
a
brief
5-‐question
quiz
and
then
follow-‐up
with
a
few
interview
questions
Measurements
The
measurements
I
will
use
to
conduct
my
user
research
on
technical
instructional
videos
include:
the
5
question
quiz
to
evaluate
the
ability
for
participants
to
recall
specific
information
from
the
video,
and
the
follow
up
interview
to
learn
of
the
participants
opinions
and
preferences.
The
questions
will
come
directly
from
the
information
given
in
the
video
and
will
be
a
baseline
measurement
to
assess
user
recall.
The
interview
regarding
the
users
preferences
will
also
provide
me
with
information
regarding
the
effectiveness
of
the
videos
in
question.
It
will
provide
me
with
an
understanding
of
how
focused
the
participants
are
during
the
entire
duration
of
the
video,
or
if
participants
lose
focus
after
a
certain
amount
of
time.
The
user
feedback
will
greatly
help
me
to
understand
what
the
users
are
thinking
during
the
video,
and
if
the
longer
video
becomes
less
useful
because
information
is
not
retained.
I
hope
to
determine
which
video
participants
believe
to
be
the
appropriate
length.
Participants
Background
20. 20
The
five
participants
all
have
different
demographics,
and
unfortunately
because
of
the
subject
matter
of
the
videos
knowledge
if
the
sport
of
basketball
may
affect
the
results.
I
will
try
to
ask
questions
that
pertain
to
the
videos
but
do
not
focus
on
basketball
as
much
as
possible.
Participant
Gender
Age
Occupation
1
Male
58
Sales
Director
2
Female
35
Receptionist
3
Female
21
Marketing
4
Male
22
College
Sophomore
5
Female
12
Student
6
Male
21
College
Senior
Dates
of
Testing
I
will
perform
the
testing
between
March
14th
and
March
21st.
Participant
1
will
be
tested
on
March
14th,
participant
2
and
5
will
be
tested
on
March
15th,
participant
4
will
be
tested
on
March
18th,
and
participant
3
will
be
tested
on
March
21st.
The
testing
will
be
performed
at
two
locations
with
the
first
being
Black
Coffee
and
Waffle
Bar
in
Minneapolis,
MN,
and
the
second
will
be
from
my
father’s
house
in
Maple
Grove,
MN.
21. 21
References
Alexander,
K.
P.
(2013).
The
usability
of
print
and
online
video
instructions.
Technical
Communication
Quarterly,
22(3),
237-‐259.
doi:10.1080/10572252.2013.775628
Morain,
M.,
&
Swarts,
J.
(2012).
YouTutorial:
A
framework
for
assessing
instructional
online
video.
Technical
Communication
Quarterly,
21(1),
6-‐24.
doi:10.1080/10572252.2012.626690
Pflugfelder,
E.
H.
(2013).
The
minimalist
approach
to
online
instructional
videos.
Technical
Communication,
60(2),
131-‐146.
22. 22
Appendix
A1
Bill of Rights for Usability Evaluators
You are not being evaluated
The term “usability evaluation” refers to evaluating a software product (an application or Web
service), not a person. The usability team considers you a partner in the design of the product.
Any difficulties you encounter during this session help us identify which parts of the product
need to be improved.
You may leave at any time
You are taking part in the evaluation as a volunteer. You have the right to interrupt the
evaluation or withdraw from it at any time, for any reason. You do not need to give a reason.
Your identity will be kept confidential
Usability team members have signed Code of Conduct agreements that contain their promise to
keep your identity confidential. Any reports resulting from this session will protect your
anonymity. Any records from this session that might identify you will be kept confidential by the
usability team. These include forms, notes, and recordings which could identify you. You may be
recognizable on video or audio recordings, but this session will not be recorded without your
written consent. You have the right to be informed what the session records will be used for. The
records may not be used for purposes other than those you have agreed to unless your additional
written consent is obtained.
Video and Audio Recordings will be destroyed upon request
When no longer needed for design purposes, any video or audio recordings containing your
picture or voice from this session will be destroyed. You have the right to request that the
recordings from the session be destroyed, whether or not you complete the session.
You will be treated with respect
You have the right to be treated politely and with respect during the evaluation. Humor is
allowed, but you are invited to set the tone that you feel most comfortable with.
If
you
have
any
concerns
about
the
fulfillment
of
these
stated
intentions,
please
contact
David Rosen, Usability Services Manager, at 612-624-9365 or send email to
rose0414@umn.edu.
23. 23
Appendix
A2
Consent Form
Title of Study: Usability of Software or Internet Products
You
are
invited
to
be
in
a
research
study
in
which
you
are
asked
to
test
the
ease
of
use
of
a
particular
product,
Technical
Instructional
Videos
on
Youtube.
You
have
been
identified
as
a
possible
participant
because
you
are
a
potential
user.
I
ask
that
you
consider
this
research
opportunity
and
ask
any
questions
you
may
have
before
agreeing
to
be
in
the
study.
This
study
is
being
conducted
by
students
in
WRIT
4501:
Usability
and
Human
Factors
in
Technical
Communication
at
the
University
of
Minnesota
under
the
instruction
of
Professor
Lee-‐Ann
Kastman
Breuch
in
the
Department
of
Writing
Studies.
Background Information
The
purpose
of
this
study
is
to
observe
persons
interacting
with
a
product
to
gain
information
about
the
ease
of
use
of
that
product.
Procedures
If
you
agree
to
be
in
this
study,
your
participation
will
consist
of
a
background
questionnaire
about
your
experience
with
the
software
product,
viewing
of
a
technical
instructional
YouTube
video,
a
short
quiz,
and
a
short
debriefing
interview
about
your
experience
viewing
the
video.
Your
participation
will
not
exceed
30
minutes
of
your
time.
Risks and Benefits
There
are
no
foreseeable
risks
associated
with
your
participation
in
this
study.
Participation
in
this
study
may
benefit
you
by
encouraging
you
to
think
about
the
ways
in
which
web
sites
and
software
products
could
be
improved
to
suit
user
needs
and
preferences.
Your
participation
will
also
help
students,
program
developers,
and
instructors
learn
more
about
product
design
improvements.
Voluntary Nature of the Study
Your
participation
is
strictly
voluntary,
and
you
are
not
required
to
participate
in
this
study.
You
can
withdraw
from
the
study
at
any
time.
Your
decision
to
withdraw
will
have
no
affect
on
your
relationship
with
the
University
of
Minnesota.
24. 24
Confidentiality
The
records
of
this
study
will
be
kept
private.
In
any
sort
of
report
that
may
be
published,
no
information
will
be
included
that
will
make
it
possible
to
identify
a
subject.
Pseudonyms
or
numbers
will
be
used
in
place
of
your
real
name
to
protect
your
identity.
If
you
agree
to
participate
in
interviews,
your
interviews
will
be
recorded
(either
audio
or
videotape).
Only
student
researchers
will
have
access
to
these
data.
Contacts and Questions
The
researcher
conducting
this
study
is
Brad
Arrington.
You
may
ask
any
questions
you
have
about
the
study
now.
If
you
have
questions
later,
you
may
contact
me
through
e-‐mail:
arrin007@umn.edu
Phone:
(763)
923-‐8200
You
will
be
given
a
copy
of
this
form
to
keep
for
your
files.
Professor
Lee-‐Ann
Kastman
Breuch
is
instructor
of
this
class.
You
can
contact
Professor
Breuch
with
any
further
questions
at
lkbreuch@umn.edu
or
(612)
624-‐6727.
If you have any questions or concerns regarding this study and would
like to talk to someone other than the researcher(s), you are encouraged
to contact the Research Subjects’ Advocate Line, D528 Mayo, 420
Delaware St. Southeast, Minneapolis, Minnesota 55455; (612) 625-
1650.
Statement of Consent
I
have
read
the
above
information.
I
have
asked
questions
and
have
received
answers
to
my
questions.
I
consent
to
participate
in
the
study.
Signature
Date
Signature
of
Investigator
Date
25. 25
Appendix
A3
Evaluator Briefing Script Example
1) Introductions (Thanks for coming in today, did you find everything okay in the process so
far, etc.)
2) Agreement Form: You just completed our agreement form for usability evaluators. Did you
have any questions about anything on the form?
3) Who I am: I’m from Brad Arrington, and I’m conducting research for a Senior Project. My
research involves user’s experience with technical instructional videos on YouTube.
4) Give intro to design: Today you’ll be watching a YouTube instructional video and helping
to evaluate its usability.
5) Bill of Rights - You read your Bill of Rights? Did you have any questions about it?
a) The most important thing is that you are doing the evaluating and you are not
being evaluated.
b) If you happen to have any trouble your experience will help us to identify usability issues
so they can be fixed and other users won’t have to struggle later.
Other parts of the Bill of Rights – you’re here voluntarily and may leave, if you need to. I
will be observing your session and we’ll be making video recordings of your
responses. But your participation is confidential with the usability team, the
audio and video recordings will be kept confidential, and notes and reports about
the evaluation will not reveal your identity.
6) Is everything set up okay for you at the computer -- chair a comfortable height, the mouse
on the right side for you?
7) Your task is to simply view the instructional YouTube video and I will follow up with a brief
quiz and some interview question. Do you have any question or concerns before we begin?
8) Now that you’ve viewed the video I will administer the short quiz. Remember I am testing the
video and not you.
9) Thank you for completing the quiz I just have a few more questions to ask before the testing
is complete
a. On a scale of 1 to 5 (with 5 being very appropriate and 1 being very inappropriate) do
you feel about the length of the video?
b. Did the video grab your attention?
c. Were you focused throughout the duration of the video?
d. Was there anything you liked/disliked about the video?
10) Thank you for your participation in my research project!
26. 26
Appendix
A4
Background
Questionnaire
Thank you for helping us evaluate our client website. Before we begin, please tell us a little
about yourself.
1. Gender:
Male
Female
2. Age:
10-‐17
18-‐29
30-‐39
40-‐49
50-‐59
60 +
3. Occupation:
4. How do you most often use a computer? (Check all that apply)
I don’t use a computer
Work
Word Processing
Personal Email
Entertainment and Games
Music
Photo/video
Internet
Other – Please specify:
5. How often, on average, do you use a computer? (Check one)
I have never used a computer
Yearly
Monthly
Weekly
Daily
6. How long have you been using a computer? (Check one)
I have never used a computer
Less than 1 year
1-‐2 years
27. 27
3-‐5 years
6-‐10 years
10 + years
7. How often do you use the Internet (not including email)? (Check one)
I have never used the Internet
Yearly
Monthly
Weekly
Daily
8. How comfortable are you with use the internet (not including email)?
Not comfortable Comfortable
1 2 3 4 5
9. How long have you been an internet user? (Check one)
I have never used the internet
Less than 1 year
1-‐2 years
3-‐5 years
6-‐10 years
10 + years
10. How often do you use YouTube? (Check one)
I have never used the Internet
Yearly
Monthly
Weekly
Daily
11. How often do you view technical instructional videos on YouTube? (Check one)
I have never viewed an instructional video on YouTube
Once
More than once
Regularly
12. How much knowledge and experience do you have with basketball?
Not Very Knowledgeable Very Knowledgeable
1 2 3 4 5
28. 28
Appendix
A5
Interview
Questions
1. On
a
scale
of
1
to
5
(with
5
being
very
appropriate
and
1
being
very
inappropriate)
do
you
feel
about
the
length
of
the
video?
2. Did
the
video
grab
your
attention?
3. Were
you
focused
throughout
the
duration
of
the
video?
4. Was
there
anything
you
liked/disliked
about
the
video?
29. 29
Appendix
A6
Quiz
Name:
1. What
percent
of
jump
shots
does
the
video
say
you
can
make
with
this
form?
2. How
many
steps
are
there
to
a
perfect
jump
shot?
3. What
is
step
3
of
the
perfect
jump
shot?
4. How
far
forward
should
your
shooting
foot
be
(in
shoe
lengths)?
5. What
percentage
more
time
does
the
dip
take
up?
30. 30
Appendix
B
Background
Questionnaire
Thank you for helping us evaluate our client website. Before we begin, please tell us a little
about yourself.
1. Gender:
Male
Female
2. Age:
10-‐17
18-‐29
30-‐39
40-‐49
50-‐59
60 +
3. Occupation:
4. How do you most often use a computer? (Check all that apply)
I don’t use a computer
Work
Word Processing
Personal Email
Entertainment and Games
Music
Photo/video
Internet
Other – Please specify:
5. How often, on average, do you use a computer? (Check one)
I have never used a computer
Yearly
Monthly
Weekly
Daily
6. How long have you been using a computer? (Check one)
I have never used a computer
Less than 1 year
1-‐2 years
31. 31
3-‐5 years
6-‐10 years
10 + years
7. How often do you use the Internet (not including email)? (Check one)
I have never used the Internet
Yearly
Monthly
Weekly
Daily
8. How comfortable are you with use the internet (not including email)?
Not comfortable Comfortable
1 2 3 4 5
9. How long have you been an internet user? (Check one)
I have never used the internet
Less than 1 year
1-‐2 years
3-‐5 years
6-‐10 years
10 + years
10. How often do you use YouTube? (Check one)
I have never used the Internet
Yearly
Monthly
Weekly
Daily
11. How often do you view technical instructional videos on YouTube? (Check one)
I have never viewed an instructional video on YouTube
Once
More than once
Regularly
12. How much knowledge and experience do you have with basketball?
Not Very Knowledgeable Very Knowledgeable
1 2 3 4 5
32. 32
Background
Questionnaire
Responses
P1
P2
P3
P4
P5
P6
Q1:
Gender
female
female
male
male
male
female
Q2:
Age
10-‐17
30-‐39
18-‐29
50-‐59
18-‐29
18-‐29
Q3:
Occupation
student
management
student
sales
director
student
marketing
assoaciate
Q4:
How
do
you
most
often
use
a
computer?
(check
all
that
apply)
entertainment,
music,
internet
work,
personal
email,
internet
work,
word
processing,
personal
email,
entertainment,
music,
internet
work,
personal
email,
internet
word
processing,
entertainment,
music,
internet
work,
word
processing,
personal
email,
music,
photo/video,
internet
Q5:
How
often,
on
average,
do
you
use
a
computer?
(check
one)
weekly
daily
daily
daily
daily
daily
Q6:
How
long
have
you
been
using
a
computer?
(check
one)
6-‐10
10+
10+
10+
10+
10+
Q7:
How
often
do
you
use
the
Internet
(not
including
email)?
(check
one)
weekly
daily
daily
daily
daily
daily
Q8:
How
comfortable
are
you
with
using
the
Internet
(not
including
email)?
5
5
5
5
5
5
Q9:
How
long
have
you
been
an
Internet
user?
(check
one)
3-‐5
10+
10+
10+
10+
10+
33. 33
Q10:
How
often
do
you
use
YouTube?
(check
one)
weekly
weekly
monthly
weekly
daily
weekly
Q11:
How
often
do
you
view
technical
instructional
videos
on
YouTube?
(check
one)
more
than
once
more
than
once
more
than
once
once
regularly
more
than
once
Q12:
How
much
knowledge
and
experience
do
you
have
with
basketball?
3
1
4
4
4
2
34. 34
Appendix
C
Quiz
Questions
6. What
percent
of
jump
shots
does
the
video
say
you
can
make
with
this
form?
7. How
many
steps
are
there
to
a
perfect
jump
shot?
8. What
is
step
3
of
the
perfect
jump
shot?
9. How
far
forward
should
your
shooting
foot
be
(in
shoe
lengths)?
10. What
percentage
more
time
does
the
dip
take
up?
P1
P2
P3
P4
P5
P6
Q1:
What
percent
of
jump
shots
does
the
video
say
you
can
make
with
this
form?
5%
50%
50%
50%
5%
50%
Q2:
How
many
steps
are
there
to
a
perfect
jump
shot?
5
steps
5
5
steps
5
5
5
Q3:
What
is
step
3
of
the
perfect
jump
shot?
dip
dip
dip
dip
dip
dip
Q4:
How
far
forward
should
your
shooting
foot
be
(in
shoe
lengths)?
slightly
½
shoe
lendth
½
½
shoe
length
shoe
length
1
foot
Q5:
What
percentage
more
time
does
the
dip
take
up?
5%
5%
5%
or
more
5%
5%
5%
Total
3
5
5
5
3
4
35. 35
Appendix
D
Interview
Questions
5. On
a
scale
of
1
to
5
(with
5
being
very
appropriate
and
1
being
very
inappropriate)
do
you
feel
about
the
length
of
the
video?
6. Did
the
video
grab
your
attention?
7. Were
you
focused
throughout
the
duration
of
the
video?
8. Was
there
anything
you
liked/disliked
about
the
video?
Interview
Responses
P1
P2
P3
P4
P5
P6
Q1:
On
a
scale
of
1
to
5
(with
5
being
very
appropriate
and
1
being
very
inappropriate)
do
you
feel
about
the
length
of
the
video?
5
5
4
4
5
4
Q2:
Did
the
video
grab
your
attention?
yes,
very
informative
yes,
learned
something
new
yeah,
instructor
&
demonstrator,
different
camera
angles
yes,
upfront
content,
casual
manner
if
I
was
looking
to
learn
then
yes
yeah
Q3:
Were
you
focused
throughout
the
duration
of
the
video?
yes
yes
no,
I'm
tired
yes
yes
yeah
definitely
(I
think
knowing
I
was
getting
quizzed
played
into
that)
Q4:
Was
there
anything
you
liked/disliked
about
the
video?
very
informational
in
describing
steps
liked
shots
of
NBA,
different
angles,
5
steps
beginning/r
ecap
liked-‐
explained
steps
quantitative.
Disliked-‐
more
broad
(other
styles
or
methods)
liked-‐
video
stopped
and
broke
down
separate
parts,
discussion
dip
v.
not
dip
ray
allen
liked-‐
execution
of
examples,
enough
example
and
elaboration.
Disliked-‐
distracted
by
bouncing
balls
in
background
liked-‐
how
laid
out
steps,
support,
protips
36. 36
Appendix
E
Short
Technical
Instruction
Video
Link
https://www.youtube.com/watch?v=uinjgadPHbQ&feature=youtu.be
Long
Technical
Instruction
Video
Link
https://www.youtube.com/watch?v=WdY6KgRERvI
