Weimin Hou Web Content Mavens DC 2016

1. Measuring and Testing Website Usability
Weimin Hou
Senior UX Specialist
whou@userworks.com
MavenMini: Web User Experience (UX)
August 11, 2016
1738 Elton Rd., Suite 138, Silver Spring, Maryland 20903
@userworks; 301-431-0500; www.userworks.com

2. Agenda
 A brief overview of my background
 Website usability basics
 Methods to measure and test website usability and related tools/resources
‣ Moderated usability testing
• Remote and in-person
• Mobile testing
‣ Heuristic and accessibility evaluation
‣ Other evaluation methods and resources
 Usability testing books and additional online resources
 Q & A
Note: Thought-provoking & informative rather than instructional or how-to
Measuring and Testing Website Usability
MavenMini: Web User Experience
@UserWorks 8/11/2016
2

3. About Me
 Ph.D. in information systems, Human-Computer Interaction track, UMBC
‣ Now the Human-Centered Computing (HCC) program, Master of Science and Doctor of Philosophy
 Started working at UserWorks in 2008
 UserWorks, Inc.: A local user experience design and usability research firm
‣ Founded by Dr. Dick Horst in 1989
‣ Served over 250 clients: private sectors, government, non-profits, and academic institutions
‣ Has 11 staff members; 8 technical, with backgrounds in
• Cognitive psychology
• Computer science
• Human computer interaction
• Human factors engineering
• Information architecture and interface design
• Information technology
‣ Has participant databases with 6,000+ local / 12,000+ nation-wide volunteers
Measuring and Testing Website Usability
MavenMini: Web User Experience
@UserWorks 8/11/2016
3

4. Variety of User-Centered Design Methods
Measuring and Testing Website Usability
MavenMini: Web User Experience
@UserWorks 8/11/2016
Research
 User interviews
 User and task analysis
 Stakeholder interviews
 Focus groups
 Online surveys
 Web analytics
 Ethnographic/ contextual
inquiries
Design
 Workflow mapping
 User personas
 Card sorting
 Wireframes
 Prototyping
 Visual design
 Plain language content
 Style guides
Evaluation
 Usability testing
 Heuristic/expert reviews
 Accessibility evaluation
 Eye tracking
 Readability analysis and
comprehension testing
4

5. Website Usability Basics
 Usability definition
‣ The extent to which a product can be used by specified users to achieve specified goals with
effectiveness, efficiency and satisfaction in a specified context of use. (International Organization for
Standardization 9241-11)
 Important to:
‣ Recruit the right participants; consider:
• New or returning visitors; children or adults; professionals or general public; doctors or patients/caregivers;
people with disabilities
‣ Focus on the tasks that users seek to complete
‣ Observe actual user behavior
• Nielsen Norman Group article: First Rule of Usability? Don't Listen to Users
Measuring and Testing Website Usability
MavenMini: Web User Experience
@UserWorks 8/11/2016
5

6. Methods to Measure and Test Website Usability
Measuring and Testing Website Usability
MavenMini: Web User Experience
@userworks.com 8/11/2016

7. One-on-one Moderated Usability Testing
 Researcher interacts with the participant throughout the entire testing session
 Good for testing all website aspects:
 Can evaluate an entire website or can focus on certain functionality, features, or
sections
 Can be conducted in-person or remotely:
‣ In-person - researcher and participant interact face-to-face
‣ Remote - researcher and participant are not at the same location, but share researcher’s computer
screen using web conferencing software (such as GoToMeeting or WebEx); they talk though computer
audio or phone
Measuring and Testing Website Usability
MavenMini: Web User Experience
@UserWorks 8/11/2016
7
‣ information architecture (IA)
‣ content (readability and comprehension)
‣ navigation
‣ search and other functionalities
‣ accessibility
‣ visual design, layout, etc.

8. Typical Moderated Usability Test Procedures
1) Technical setup (for remote sessions):
2) Study introduction
3) Background questions
4) Pre-defined task scenarios
‣ Observe participant’s behavior
‣ Use “Think aloud” protocol and appropriate probing/follow up questions
5) Post-task questionnaire to collect user satisfaction, such as System Usability Scale
(SUS)
6) Debrief on participant’s overall experience, the usefulness of the content, and the
participant’s suggestions for improvement
Measuring and Testing Website Usability
MavenMini: Web User Experience
@UserWorks 8/11/2016
8

9. Video: Remote Moderated Usability Testing (1:46)
Measuring and Testing Website Usability
MavenMini: Web User Experience
@UserWorks 8/11/2016
9

10. 10
Moderated Mobile Testing
 Can be conducted in-person or remotely, but both require specific setup
‣ Useful apps and software tools: AirPlay, Zoom, Join.me, Mobizen, etc.
Measuring and Testing Website Usability
MavenMini: Web User Experience
@UserWorks 8/11/2016
Possible setup for an in-person mobile usability test Remote mobile usability test in session

11. 11
Moderated Mobile Testing Resources
 In-person mobile testing:
‣ Usability Testing Of Mobile Applications: A Step-By-Step Guide, by Justin Mifsud
 Remote mobile testing:
‣ An article by two of my colleagues: Dana Douglas and Tristan Wilson, Hold the Phone: A Primer
on Remote Mobile Usability Testing, User Experience Magazine, 2016.
‣ Their presentation at the annual UXPA International conference in June 2016 : Tips and Tools for
Testing Mobile Interactions Remotely (and On a Budget!)
Measuring and Testing Website Usability
MavenMini: Web User Experience
@UserWorks 8/11/2016

12. Books on Moderating Usability Tests
 Usability Testing Essentials: Ready, Set...Test!, by Carol Barnum, 2010
 Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability
Problems, by Steve Krug, 2009
 Moderating Usability Tests: Principles and Practices for Interacting, by Joseph S. Dumas and
Beth A. Loring, 2008
 Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests, by Jeffrey
Rubin and Dana Chisnell, 2008
 A Practical Guide to Usability Testing, by Joseph S. Dumas and Janice C. Redish, 1999
Measuring and Testing Website Usability
MavenMini: Web User Experience
@UserWorks 8/11/2016
12

13. Heuristic and Accessibility Evaluation
 Conducted by UX professionals who are familiar with industry guidelines, standards,
and best practices; no real website users involved
 Usually performed before usability testing to get familiar with the product and to
identify the usability tasks to use during testing
 Accessibility evaluation can be included in overall heuristic evaluation
‣ Use automated browser and software tools, such as the WAVE toolbar
‣ Use assistive technologies (screen readers, magnifiers, voice recognition software)
‣ Can be completed by an assistive technology user
Measuring and Testing Website Usability
MavenMini: Web User Experience
@UserWorks 8/11/2016
13

14. Other Evaluation Methods and Resources
 Eye tracking
‣ Eye Tracking in User Experience Design, edited by Jennifer Romano Bergstrom and Andrew Schall, 2014
‣ Eye Tracking the User Experience: A Practical Guide to Research, by Aga Bojko, 2013
 Unmoderated usability testing
‣ Researcher not present when participants complete tasks or provide feedback
‣ Participant choosing the time and location convenient to them, following automated scripts and procedures
‣ Good for:
• Testing simple and straightforward tasks
• Collecting data from a large number of participants
• Evaluating specific website aspects, such as IA and layout
‣ Including online card sorting, tree testing, first-click testing, A/B testing, etc.
‣ See a comprehensive list of remote usability and UX research tools
 Web analytics
‣ Google Analytics, including search reports
‣ American Customer Satisfaction Index (ACSI)
‣ Crazy Egg
‣ Inspectlet
Measuring and Testing Website Usability
MavenMini: Web User Experience
@UserWorks 8/11/2016
14

15. Additional Resources on Website Usability
 Usability.gov
 User Experience Professionals Association (UXPA)
 Boxes and Arrows
 Nielson Norman Group articles
 Smashing Magazine
 UsabilityGeek
 User Interface Engineering (UIE) Articles
 UXmatters
 WebAIM accessibility articles
Measuring and Testing Website Usability
MavenMini: Web User Experience
@UserWorks 8/11/2016
15

16. Thank You!
16
Measuring and Testing Website Usability
MavenMini: Web User Experience
@UserWorks 8/11/2016
 Contact Information
‣ Weimin Hou: whou@userworks.com
 UserWorks
‣ @userworks
‣ www.facebook.com/userworks
‣ Slides: www.userworks.com/slides.html