Measuring and Testing Website Usability

Measuring and Testing Website Usability
Weimin Hou
Senior UX Specialist
whou@userworks.com
MavenMini: Web User Experience (UX)
August 11, 2016
1738 Elton Rd., Suite 138, Silver Spring, Maryland 20903
@userworks; 301-431-0500; www.userworks.com

Agenda
 A brief overview of my background
 Website usability basics
 Methods to measure and test website usability and related tools/resources
‣ Moderated usability testing
• Remote and in-person
• Mobile testing
‣ Heuristic and accessibility evaluation
‣ Other evaluation methods and resources
 Usability testing books and additional online resources
 Q & A
Note: Thought-provoking & informative rather than instructional or how-to
MavenMini: Web User Experience
@UserWorks 8/11/2016
2

About Me
 Ph.D. in information systems, Human-Computer Interaction track, UMBC
‣ Now the Human-Centered Computing (HCC) program, Master of Science and Doctor of Philosophy
 Started working at UserWorks in 2008
 UserWorks, Inc.: A local user experience design and usability research firm
‣ Founded by Dr. Dick Horst in 1989
‣ Served over 250 clients: private sectors, government, non-profits, and academic institutions
‣ Has 11 staff members; 8 technical, with backgrounds in
• Cognitive psychology
• Computer science
• Human computer interaction
• Human factors engineering
• Information architecture and interface design
• Information technology
‣ Has participant databases with 6,000+ local / 12,000+ nation-wide volunteers
3

Variety of User-Centered Design Methods
Research
 User interviews
 User and task analysis
 Stakeholder interviews
 Focus groups
 Online surveys
 Web analytics
 Ethnographic/ contextual
inquiries
Design
 Workflow mapping
 User personas
 Card sorting
 Wireframes
 Prototyping
 Visual design
 Plain language content
 Style guides
Evaluation
 Usability testing
 Heuristic/expert reviews
 Accessibility evaluation
 Eye tracking
 Readability analysis and
comprehension testing
4

Website Usability Basics
 Usability definition
‣ The extent to which a product can be used by specified users to achieve specified goals with
effectiveness, efficiency and satisfaction in a specified context of use. (International Organization for
Standardization 9241-11)
 Important to:
‣ Recruit the right participants; consider:
• New or returning visitors; children or adults; professionals or general public; doctors or patients/caregivers;
people with disabilities
‣ Focus on the tasks that users seek to complete
‣ Observe actual user behavior
• Nielsen Norman Group article: First Rule of Usability? Don't Listen to Users
5

Methods to Measure and Test Website Usability
@userworks.com 8/11/2016

One-on-one Moderated Usability Testing
 Researcher interacts with the participant throughout the entire testing session
 Good for testing all website aspects:
 Can evaluate an entire website or can focus on certain functionality, features, or
sections
 Can be conducted in-person or remotely:
‣ In-person - researcher and participant interact face-to-face
‣ Remote - researcher and participant are not at the same location, but share researcher’s computer
screen using web conferencing software (such as GoToMeeting or WebEx); they talk though computer
audio or phone
7
‣ information architecture (IA)
‣ content (readability and comprehension)
‣ navigation
‣ search and other functionalities
‣ accessibility
‣ visual design, layout, etc.

Typical Moderated Usability Test Procedures
1) Technical setup (for remote sessions):
2) Study introduction
3) Background questions
4) Pre-defined task scenarios
‣ Observe participant’s behavior
‣ Use “Think aloud” protocol and appropriate probing/follow up questions
5) Post-task questionnaire to collect user satisfaction, such as System Usability Scale
(SUS)
6) Debrief on participant’s overall experience, the usefulness of the content, and the
participant’s suggestions for improvement
8

Video: Remote Moderated Usability Testing (1:46)
9

10
Moderated Mobile Testing
 Can be conducted in-person or remotely, but both require specific setup
‣ Useful apps and software tools: AirPlay, Zoom, Join.me, Mobizen, etc.
Possible setup for an in-person mobile usability test Remote mobile usability test in session

11
Moderated Mobile Testing Resources
 In-person mobile testing:
‣ Usability Testing Of Mobile Applications: A Step-By-Step Guide, by Justin Mifsud
 Remote mobile testing:
‣ An article by two of my colleagues: Dana Douglas and Tristan Wilson, Hold the Phone: A Primer
on Remote Mobile Usability Testing, User Experience Magazine, 2016.
‣ Their presentation at the annual UXPA International conference in June 2016 : Tips and Tools for
Testing Mobile Interactions Remotely (and On a Budget!)

Books on Moderating Usability Tests
 Usability Testing Essentials: Ready, Set...Test!, by Carol Barnum, 2010
 Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability
Problems, by Steve Krug, 2009
 Moderating Usability Tests: Principles and Practices for Interacting, by Joseph S. Dumas and
Beth A. Loring, 2008
 Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests, by Jeffrey
Rubin and Dana Chisnell, 2008
 A Practical Guide to Usability Testing, by Joseph S. Dumas and Janice C. Redish, 1999
12

Heuristic and Accessibility Evaluation
 Conducted by UX professionals who are familiar with industry guidelines, standards,
and best practices; no real website users involved
 Usually performed before usability testing to get familiar with the product and to
identify the usability tasks to use during testing
 Accessibility evaluation can be included in overall heuristic evaluation
‣ Use automated browser and software tools, such as the WAVE toolbar
‣ Use assistive technologies (screen readers, magnifiers, voice recognition software)
‣ Can be completed by an assistive technology user
13

Other Evaluation Methods and Resources
 Eye tracking
‣ Eye Tracking in User Experience Design, edited by Jennifer Romano Bergstrom and Andrew Schall, 2014
‣ Eye Tracking the User Experience: A Practical Guide to Research, by Aga Bojko, 2013
 Unmoderated usability testing
‣ Researcher not present when participants complete tasks or provide feedback
‣ Participant choosing the time and location convenient to them, following automated scripts and procedures
‣ Good for:
• Testing simple and straightforward tasks
• Collecting data from a large number of participants
• Evaluating specific website aspects, such as IA and layout
‣ Including online card sorting, tree testing, first-click testing, A/B testing, etc.
‣ See a comprehensive list of remote usability and UX research tools
 Web analytics
‣ Google Analytics, including search reports
‣ American Customer Satisfaction Index (ACSI)
‣ Crazy Egg
‣ Inspectlet
14

Additional Resources on Website Usability
 Usability.gov
 User Experience Professionals Association (UXPA)
 Boxes and Arrows
 Nielson Norman Group articles
 Smashing Magazine
 UsabilityGeek
 User Interface Engineering (UIE) Articles
 UXmatters
 WebAIM accessibility articles
15

Thank You!
16
 Contact Information
‣ Weimin Hou: whou@userworks.com
 UserWorks
‣ @userworks
‣ www.facebook.com/userworks
‣ Slides: www.userworks.com/slides.html

Measuring and Testing Website Usability

More Related Content

What's hot

Similar to Measuring and Testing Website Usability

More from UserWorks

Recently uploaded

Measuring and Testing Website Usability