USABILITY TESTING WORKSHOP
ESRA YALMAN
02.08.2016
• Why usability testing?
• Definition
• Planning
• Ethics
• Scenarios
• Test
• Reporting
WATERFALL WORKFLOW
REQUIREMENTS DESIGN IMPLEMENTATION
LEAN UX
Jevnaker, Ingjerd Straand. "Why Lean Facilitates Exceptional User Experiences - Making Waves Blog." Making Waves Blog. N.p., 9 Apr. 2014. Web. 01 Aug. 2016.
DEFINITION OF USABILITY
Usability is a quality attribute that assesses how easy user
interfaces are to use. The word "usability" also refers to
methods for improving ease-of-use during the design process.
Nielsen, Jacop. "Usability 101: Introduction to Usability." Usability 101: Introduction to Usability. N.p., 2 Jan. 2012. Web. 01 Aug. 2016.
DEFINITION OF USABILITY
Usability is defined by 5 quality components:
Learnability
Efficiency
Memorability
Errors
Satisfaction
Nielsen, Jacop. "Usability 101: Introduction to Usability." Usability 101: Introduction to Usability. N.p., 2 Jan. 2012. Web. 01 Aug. 2016.
USABILITY TESTING METHODS
USABILITY TESTING METHODS
Usability-Lab Studies
Ethnographic Field Studies
Focus Groups
Eye tracking, Heatmap
Remote Usability Studies
Card Sorting
Tree testing
A/B Testing (Split Test), Multivariate Test
Guerrilla usability testing
Paper prototype
First-click testing
Usability-Lab Studies
Interrogation Room
Ethnographic Field Studies
Focus Groups
Eye tracking, Heatmap
"Eye Tracking." Usability.gov. N.p., n.d. Web. 01 Aug. 2016.
Remote Usability Studies
A/B Testing (Split Test), Multivariate Test
Guerrilla usability testing
Paper prototype
First-click testing
Card Sorting
Tree Testing
FIRST CLICK TESTING
A participant who clicks down the right path on the first click
will complete their task successfully 87% of the time.
A participant who clicks down the wrong-path on the first click,
tends to only successfully complete their task 46% of the time.
Sauro, Jeff. "Getting the First Click Right." : MeasuringU. N.p., 19 Oct. 2011. Web. 01 Aug. 2016.
PLANNING A USABILITY TEST
"Planning a Usability Test." Usability.gov. N.p., n.d. Web. 01 Aug. 2016.
PLANNING A USABILITY TEST
You will need to include these elements in the usability test plan.
Scope
Purpose
Schedule & Location
Sessions
Equipment
Participants
Scenarios
Quantitative metrics
Roles
"Planning a Usability Test." Usability.gov. N.p., n.d. Web. 01 Aug. 2016.
THE ETHICS OF USABILITY TESTING
Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
THE ETHICS OF USABILITY TESTING
• Don’t allow managers to watch usability tests if their staff are the
participants.
• Emphasize that you’re testing the website or application, not the test
participant.
• Make the first task incredibly simple.
• Observe strictly your own rules for handling video tapes.
Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
SELECTING GOOD TEST TASKS
AND SCENARIOS
Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
SELECTING GOOD TEST TASKS AND SCENARIOS
• Focus your test on core tasks, rather than on what’s new or fun or easy to test
• Let test participants define their own tasks.
• Write scenarios — not just tasks.
• Avoid tasks you think are humorous.
• Avoid hidden clues in task descriptions.
• Give the test participant a goal but avoid describing the steps.
Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
RUNNING THE TEST
Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
RUNNING THE TEST
• Sit with the test participants.
• Let test participants explore the site for one or two minutes initially.
• Ask the test participants to tell you if there are any graphics that they wouldn’t expect to
be links.
• Be diplomatic when test participants blame themselves.
• Wait a few minutes before you help.
• Help immediately if the participant is struggling with something that you already have
sufficient data about.
• Watch your tongue & your body language.
• Show your appreciation for any participant suggestions.
Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
REPORTING TEST RESULTS
PROCESS
Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
REPORTING TEST RESULTS PROCESS
• Ask your customer about reporting requirements
• Do a quality and usability check of your test report.
• Let the project team comment on the usability report
before you show it to others.
• Setup a database of reports that is easily accessible.
Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
REPORTING TEST RESULTS
REPORT FORMAT
Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
REPORTING TEST RESULTS REPORT FORMAT
• Make it short.
• Consider the following usability test report format
• Executive summary (1 page)
• Test participant profiles (1 page)
• Test results (5–7 pages)
• Appendix: Test script, including test scenarios
• Appendix: Screenshots annotated with key issues.
• Include a one-page executive summary
• Top 3 successes
• Top 3 problems
• Your conclusions and recommendations.
Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
REPORTING TEST RESULTS REPORT FORMAT
• Classify all comments
• Problems
• Positive findings
• Suggestions from test participants
• Functional bugs
• Usage scenarios, where test participants describe their work.
• Include quantitative data
• The number of people who successfully completed each task.
• The number of people who experienced a particular problem.
• A breakdown of problems by experience level.
Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
REPORTING TEST RESULTS REPORT FORMAT
• Develop a consistent report format

• Use an attractive and usable professional layout

• Stress that problem descriptions should be considered generic
problem descriptions
Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
• www.uxbooth.com
• www.interaction-design.org
• www.uxservices.com
• blog.invisionapp.com
• www.thoughtworks.com/insights/experience-design
• www.nngroup.com/articles
• www.optimalworkshop.com/blog
• www.usertesting.com/blog
• www.uxpin.com/studio/blog
• www.usability.gov
• usabilitygeek.com
• www.userspots.com/category/blog
• www.thinkwithgoogle.com
• blog.designedbysherpa.com
• www.smashingmagazine.com
• www.webdesignernews.com
• www.uxmatters.com
ESRA YALMAN
@esrayetis

Usability Testing Workshop

  • 1.
  • 2.
    • Why usabilitytesting? • Definition • Planning • Ethics • Scenarios • Test • Reporting
  • 3.
  • 4.
    LEAN UX Jevnaker, IngjerdStraand. "Why Lean Facilitates Exceptional User Experiences - Making Waves Blog." Making Waves Blog. N.p., 9 Apr. 2014. Web. 01 Aug. 2016.
  • 5.
    DEFINITION OF USABILITY Usabilityis a quality attribute that assesses how easy user interfaces are to use. The word "usability" also refers to methods for improving ease-of-use during the design process. Nielsen, Jacop. "Usability 101: Introduction to Usability." Usability 101: Introduction to Usability. N.p., 2 Jan. 2012. Web. 01 Aug. 2016.
  • 6.
    DEFINITION OF USABILITY Usabilityis defined by 5 quality components: Learnability Efficiency Memorability Errors Satisfaction Nielsen, Jacop. "Usability 101: Introduction to Usability." Usability 101: Introduction to Usability. N.p., 2 Jan. 2012. Web. 01 Aug. 2016.
  • 7.
  • 8.
    USABILITY TESTING METHODS Usability-LabStudies Ethnographic Field Studies Focus Groups Eye tracking, Heatmap Remote Usability Studies Card Sorting Tree testing A/B Testing (Split Test), Multivariate Test Guerrilla usability testing Paper prototype First-click testing
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
    "Eye Tracking." Usability.gov.N.p., n.d. Web. 01 Aug. 2016.
  • 15.
  • 16.
    A/B Testing (SplitTest), Multivariate Test
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 23.
    FIRST CLICK TESTING Aparticipant who clicks down the right path on the first click will complete their task successfully 87% of the time. A participant who clicks down the wrong-path on the first click, tends to only successfully complete their task 46% of the time. Sauro, Jeff. "Getting the First Click Right." : MeasuringU. N.p., 19 Oct. 2011. Web. 01 Aug. 2016.
  • 24.
    PLANNING A USABILITYTEST "Planning a Usability Test." Usability.gov. N.p., n.d. Web. 01 Aug. 2016.
  • 25.
    PLANNING A USABILITYTEST You will need to include these elements in the usability test plan. Scope Purpose Schedule & Location Sessions Equipment Participants Scenarios Quantitative metrics Roles "Planning a Usability Test." Usability.gov. N.p., n.d. Web. 01 Aug. 2016.
  • 26.
    THE ETHICS OFUSABILITY TESTING Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
  • 27.
    THE ETHICS OFUSABILITY TESTING • Don’t allow managers to watch usability tests if their staff are the participants. • Emphasize that you’re testing the website or application, not the test participant. • Make the first task incredibly simple. • Observe strictly your own rules for handling video tapes. Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
  • 28.
    SELECTING GOOD TESTTASKS AND SCENARIOS Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
  • 29.
    SELECTING GOOD TESTTASKS AND SCENARIOS • Focus your test on core tasks, rather than on what’s new or fun or easy to test • Let test participants define their own tasks. • Write scenarios — not just tasks. • Avoid tasks you think are humorous. • Avoid hidden clues in task descriptions. • Give the test participant a goal but avoid describing the steps. Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
  • 30.
    RUNNING THE TEST Molich,Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
  • 31.
    RUNNING THE TEST •Sit with the test participants. • Let test participants explore the site for one or two minutes initially. • Ask the test participants to tell you if there are any graphics that they wouldn’t expect to be links. • Be diplomatic when test participants blame themselves. • Wait a few minutes before you help. • Help immediately if the participant is struggling with something that you already have sufficient data about. • Watch your tongue & your body language. • Show your appreciation for any participant suggestions. Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
  • 32.
    REPORTING TEST RESULTS PROCESS Molich,Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
  • 33.
    REPORTING TEST RESULTSPROCESS • Ask your customer about reporting requirements • Do a quality and usability check of your test report. • Let the project team comment on the usability report before you show it to others. • Setup a database of reports that is easily accessible. Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
  • 34.
    REPORTING TEST RESULTS REPORTFORMAT Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
  • 35.
    REPORTING TEST RESULTSREPORT FORMAT • Make it short. • Consider the following usability test report format • Executive summary (1 page) • Test participant profiles (1 page) • Test results (5–7 pages) • Appendix: Test script, including test scenarios • Appendix: Screenshots annotated with key issues. • Include a one-page executive summary • Top 3 successes • Top 3 problems • Your conclusions and recommendations. Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
  • 36.
    REPORTING TEST RESULTSREPORT FORMAT • Classify all comments • Problems • Positive findings • Suggestions from test participants • Functional bugs • Usage scenarios, where test participants describe their work. • Include quantitative data • The number of people who successfully completed each task. • The number of people who experienced a particular problem. • A breakdown of problems by experience level. Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
  • 37.
    REPORTING TEST RESULTSREPORT FORMAT • Develop a consistent report format
 • Use an attractive and usable professional layout
 • Stress that problem descriptions should be considered generic problem descriptions Molich, Rolf, and Jakob Nielsen. How to Conduct Usability Studies. USA: Nielsen Norman Group, n.d. PDF.
  • 38.
    • www.uxbooth.com • www.interaction-design.org •www.uxservices.com • blog.invisionapp.com • www.thoughtworks.com/insights/experience-design • www.nngroup.com/articles • www.optimalworkshop.com/blog • www.usertesting.com/blog • www.uxpin.com/studio/blog • www.usability.gov • usabilitygeek.com • www.userspots.com/category/blog • www.thinkwithgoogle.com • blog.designedbysherpa.com • www.smashingmagazine.com • www.webdesignernews.com • www.uxmatters.com
  • 39.