This document describes a case study conducted to evaluate the Vimeo website using the 5 second test methodology. Two test formats were used - memory dump to measure memorability and attitudinal to measure visual appeal. Tests were conducted both online and in-person. Results found that most users recognized the video streaming purpose but few identified sharing features. While visual appeal was rated highly, the logo was rarely remembered. Recommendations included emphasizing upload/sharing features and reworking the logo design. Insights highlighted best practices for crafting instructions, image optimization, question wording and using both moderated and unmoderated testing.

1. 5 Second Test
Mar 13, 2017
H561 Research Method
Case Study
Sandeep Jagtap

2. 2
INTRODUCTION
Five second test is type of survey methodology in which
participant is exposed to web page/design image for five
second.
The image is then removed and participant is asked to recall
specifics of webpage with the help of short survey questions.
The results are analyzed to evaluate how well design
communicate the purpose and content within.
First implemented by Christine Perfetti (Founding Principal, Perfetti Media) in 2007
Perfetti Christine "5-Second Tests: Measuring Your Site’s Content Pages." UIE Articles, 11 Sept. 2007,
www.articles.uie.com/five_second_test/

3. The method reports visual prominence and memorability of content of
design under test.
Viewing the test image within the 5 s limit represents a large emphasis on perception, or
the organization, identification, and interpretation of sensory information—in this case,
visual information—in order to understand and make sense of the design
In General this method helps to answer following questions
▸ which elements of the page stand out in general memory?
▸ whether specific visual targets on the page are easily discerned or remembered?
▸ whether the design of a page elicits a specific emotional response?
▸ whether the design communicates values that the designer wishes to be
represented?
3
5 Second Test

4. It is mostly used to evaluate design in formative stage.
It can be used right after early low-fidelity prototypes or high-fidelity prototype or final
design of the product. It is advised to evaluate design in early stages to fix them as early
possible.
4
When ?

5. A test participant is given a set of instructions, views an image of a design
for a few seconds, and answers questions about it. That's it
The test can be conducted in either of following direction based on purpose of
evaluation. The instruction and questions for each scenario will be slightly different than
other
5
HOW TO DO IT?
MEMORY
DUMP
TARGET
IDENTIFICATION
ATTITUDINAL
What is most remembered about design ?
Whether a target is noticeable /or specific aspects of that
target are memorable
What people like, believe, or perceive about a design.

6. 6
HOW TO DO IT?
MEMORY DUMP
Instructions
"You'll see a screen for a few seconds—try to remember as much as you can about what
you see."
"After viewing a design for 5 s, be prepared to tell us what you recall about it."
Survey Questions
What do you remember most about the page you saw?
What do you think this page was about?
Emptying of the respondents’ short-term memory, resulting in a list of things
remembered about a design. It can can help confirm whether specific elements
stand out in support of the business goals—or, conversely, it can help identify
elements that need design attention

7. 7
HOW TO DO IT?
TARGET IDENTIFICATION
This format focuses on specific "targets"—visual elements or information—in a
design. Questions in this type of test aim to directly challenge a respondent's
recall of one or more targets—unlike the memory dump. As a result, the researcher
can learn not just whether a target is noticeable, but also whether specific aspects of
that target are memorable
Instructions
"You'll see a screen for a few seconds—pay attention to the “Specific feature” in the image
"After viewing a design for 5 s, we'll ask about ‘specific feature’."
Survey Questions
Where was the phone number for contacting the call center located?
Between what hours is phone support available?
Where on the page was the link to the chat functionality?

8. 8
HOW TO DO IT?
ATTITUDINAL
This format focuses on what people like, believe, or perceive about a design
such as aesthetic appeal, the emotional response, degree of satisfaction,
trustworthiness and/or credibility and whether a page's purpose is perceived as
intended.
Instructions
"You'll see a screen for a few seconds—pay attention to the general aesthetic and visual
appeal of the design."
"After viewing a design for 5 s, we'll ask for your opinions on its look and feel."
Survey Questions
What word(s) would you use to describe look and feel of the page you just saw?
Rate the overall visual appeal of the design on a scale of 1–10:1=‘not at all appealing,’ 10:
‘extremely appealing’."

9. GENERAL STEPS
9
Overall flow of
execution
1. Evaluate which page you would like to test.
2. Capture an image of the page
3. Identify test participants
4. Choose test appropriate format (memory dump/
target identification/attitudinal)
5. Prepare instruction and survey questions
6. Conduct test
7. Analyze result

10. Content
To evaluate content
pages for memorability
and organization i
Visual Design
To verify design’s
aesthetic appeal and
emotional response,
Primary Purpose
To verify whether design
communicate the
intentional purpose
effectively
Quick
It takes around 2-5
minutes to conduct test
including in person.
Generated large amount
of data in short span
Flexible
Can be done In Person or
Online (Crowdsourcing)
Using printout or portable
computer device or just
URL.
Inexpensive
Easy to administer
Does not require lot of
money or resources to
conduct test
Why do it ?
10
What advantages does it offers...
When you need to evaluate either of these..

11. CASE STUDY
Bring the attention of your audience over a key concept using
icons or illustrations
11

12. Place your screenshot here
12
PROJECT
VIMEO
The goal of the study was
to report both visual
prominence, memorability
and emotional appeal of
video sharing the website
(www.vimeo.com) using
five second method.

13. Design
Context
13 The design under test is lesser own
website called
http://www.vimeo.com/
The website’s primary function is to let
users share and discover quality videos.
Vimeo was founded by a group of
filmmakers who wanted to share their
creative work and personal moments from
their lives.
Website aims to provide a create fun and
friendly environment that will fuel creativity
and inspires users to contribute by
uploading professional videos.
The user base of website consist of
professional video content creator
and general public to view/share
videos

14. The goal of study was to record visual prominence, memorability and
emotional response of website using memory dump and attitudinal format
of five second test.
In General aim was to answer following questions
▸ which elements of the page stand out in general memory?
▸ whether specific visual targets on the page are easily discerned or remembered?
▸ whether the design of a page elicits a specific emotional response?
14
Research
Questions

15. Two test format namely memory dump and attitudinal were used.
Individual format had different instruction and survey question. However
same image was used for both test. Each participant was subjected to
either of one test
To record memorability (Memory Dump) following instructions and
questions were used
Instructions:
“You will have only 5 SECONDS to view the Design. After viewing a design, be prepared to
tell us what you recall about it”
Questions:
1. What do you remember most about the page you saw?
2.What else do you remember?
3.What do you think this page was about?
15
Test
Instructions
and
Questions

16. Two test format namely memory dump and attitudinal were used.
Individual format had different instruction and survey question. However
same image was used for both test. Each participant was subjected to
either of one test
To record visual appeal (Attitudinal) following instructions and questions
were used
Instructions:
“You will have only 5 SECONDS to view the Design. After viewing a design, we'll ask for
your opinions on its Aesthetics, look and feel.”
Questions:
1.What word(s) would you use to describe look and feel of the page you just saw?
2.Rate the overall visual appeal of the design on a scale of 1–10: ( 1=not at all appealing
10: ‘extremely appealing)
3.Give a rating from 0 to 5: ( 0= ‘unprofessional, 5= ‘Professional)
4.Give a rating from 0 to 5: (0= ‘dated’, 5= ‘Modern’)
5.Give a rating from 0 to 5: (0= ‘Simple’,5= ‘Complex’)
16
Test
Instructions
and
Questions

17. GENERAL Steps
17
Execution
Flow
1. For study purpose image of home page of vimeo site was used.
2. Test was conducted both without ( online survey) and with moderation ( in-
person)
3. Unmoderated test was conducted using online survey site
(www.usabilityhub.com) while moderated test was done in university library
4. Moderated test was run using black-white and color print out of home page
5. Each participant were subjected to either of memory dump or attitudinal test
6. Moderated test was run with 10 participant for each format
7. Unmoderated test resulted into 50 participants response
8. The results of moderated and unmoderated test were analyzed to publish
findings and make design recommendations.

18. The online version was prepared using https://usabilityhub.com . The
survey link was published on social platform like Facebook and WhatsApp
to collect response.
Memorability Survey Link
https://usabilityhub.com/preview/fb4284163c3e?continue=close
Visual Prominence Survey Link
https://usabilityhub.com/preview/fb4284163c3e?continue=close
18
Unmoderated
Test Artifacts
Instructions QuestionsWelcome information

19. The moderated test was conducted in university library with print out of
home page of vimeo website
19
Moderated
Test Artifacts
Participant recording response
Black-white /color print out and survey questions

20. Result and
Analysis
20
Memorability Test
● Most of the users recognized
the key purpose of website as
video, movies or music
streaming website.
● The most prominent thing user
observed was video
thumbnails and various
categories of videos
● Strangely some user mistaken
website for News and Article
website.
● Only two users out of 20 users
could recognize the website
logo
Visual Prominence
● Almost all user rated website as visually
appealing
● Most of the website described website
aesthetic in following words bright,
informative, consistent, neat, sharp,
organized
● Average rating for visual appearance
was 7.5/ 10
● Average rating for professional outlook
was 3.5 /5
● Average rating for modern design was
4.5/5
● Average rating for simplicity was 1/5
Response to describe page

21. 21
Recommendations
● As most of the user identified part of website
purpose which is to watch videos but almost
no user identified sharing and uploading
video functionality. A prominent upload
button and sharing button can be useful.
● Also, very few users identified site logo, so
designer might need to work on the brand
and identity communication.
● A visually appealing logo can be help to
promote brand
● Many users were disguised by carousal
image on the webpage. Designer can make
decrease emphasize on that to highlight
other functionality of web page

22. Insights
Guidelines based on insights
22

23. 23
Insights
1. Focus on the Specific Design Aspect(s) You Want to Test and Employ the Appropriate Test
Format
Depending upon the research question make use of either memory dump/ target identification/
attitudinal test format. Do not try to use mix of this method else you will get mixed results
2.Devote Time to Crafting Proper Instructions so as to minimize bias and the reporting of
false data.
Always write clear and concise instruction to give participants enough reasonable
expectation so they can provide the kind of data you're looking for.
3.Optimize the test Image so that Scrolling is Eliminated.
Surprisingly simple step can spoil result if not paid attention to. Make sure user can see
image without scroll as there are only five second for the evaluation

24. 24
Insights
4. Ask few question and order them optimally.
Since participant response are based on short term memory, asking only 2-3 questions is better.
Also, it is advised to ask questions that require recall of specifics first like target identification
first than general response to page
5.Pay Careful Attention to How the Questions are Worded.
This is applicable to all studies, like asking double double barrel, leading or biased question
is poor practice which will lead in false data.
6.Open-Ended Feedback Requests Carry a High Risk of Non responses
In contrast to other method this method works better with pointed question which can
produce factual data instead of open ended question. There is not enough time for user to
answer open ended question based on five second interaction with design.

25. 25
Insights
7.Whenever possible, use both moderated and unmoderated test
I learned this one hard way. The most important factor of this test surprise factor. I published test
with wrong test link and couldn't capture data. Due to lack of surprise, I lost many participant for
next round of study. To mitigate that you can run test in moderated environment which also let
participant ask question in case of confusion.
8.Always do pilot study before proceeding
This is one of the key principle to any research study. My pilot study help me to decide size
and quality of image under test before sending it to large audience.
9.This test can't tell you everything
Iust because this method is quick and easy, do not use everywhere. Use right tool for right
job. This method only gives supplies one piece of large usability problem do not replace this
with full fledged usability test

26. 26
References

27. More info about five second test can be found at
1. Lindgaard, G., Fernandes, G., Dudek, C., and Brown, J. “Attention Web Designers: You
Have 50 Milliseconds to Make a Good First Impression!” Behaviour & Information
Technology 25, 2 (2006), 115–126.
2. Doncaster, Paul. The UX Five-Second Rules:Guidelines for User Experience Design's
Simplest Testing Technique, 2014.
3. "Designing good five-second tests: An interview with Paul Doncaster." uxpod.com, 30
Sept. 2014, www.uxpod.com/designing-good-five-second-tests-an-interview-with-paul-
doncaster/.
4. “D4D 2014 Keynote Rethinking UX Research Best Practices for Successfully Evaluating
and Improving.” YouTube, uploaded by Design 4 Drupal, Boston, 21 Aug 2014,
https://youtu.be/Nv-sx3a7WTY/.
5. Perfetti Christine "5-Second Tests: Measuring Your Site’s Content Pages." UIE Articles,
11 Sept. 2007, www.articles.uie.com/five_second_test/.
6. The usability hub website. UsabilityHub 2008-2017, www.usabilityhub.com/. Accessed
13 Mar 2017
27
References

28. Thanks
Any questions?
Please email me at sajagtap@iu.edu
Visit my website at www.sandeepjagtap.com
28