Beyond Usability Testing: Assessing the Usefulness of Your DesignPresentation Transcript
Beyond Usability Testing: Assessing the Usefulness of Your Design UPA Boston Mini-Conference 2011 Prepared by: Michael Hawley – Chief Design Officer Daniel Berlin – Experience Research Director May 25, 2011
Have You Run Usability Tests?
Usability Testing Participants attempt to complete a set of defined tasks. Researchers learn what to improve by observing and interpreting think-aloud.
Virzi, R.A., Refining the Test Phase of Usability Evaluation: How Many Subjects is Enough? Human Factors, 1992. 34(4): p. 457-468.
Trend Business sponsors turn to us as UX professionals with questions that are not about usability problems. Rather, their questions are about overall user experience strategy, value and usefulness.
Usability: Find interruptions in workflow that prevent users from performing tasks quickly and efficiently.
Usefulness: Inform a re-structure of the application to best align with workflow. Determine where to position productivity tips and help buttons within the application for best utilization. Find optimal level of personalization and customization that users would take advantage of.
Usability: Assess effectiveness of navigation system in guiding users to desired pages. Evaluate descriptiveness and clarity of links. Gauge ability of page layouts to orient users to relevant content.
Usefulness: Identify content that is missing which will help overcome objections or answer critical questions? Understand how branded labels and content themes contribute to the overall experience or detract from it. Determine level/types of promotions and interstitials that are acceptable to users. Understand how different audience personas prefer to consume information for the particular domain.
Usability: Determine optimal level of difficulty to encourage advancement to multiple levels of the game. Assess discoverability of game features and controls.
Usefulness: Find the optimal rate of point accumulation and alignment with prize levels. Understand best use of social media within or around the game. Determine the threshold for ads, interstitials and registration for game play.
Usability: Assess if users can figure out how to add a comment, share content, or use a tagging mechanism to find what they are looking for.
Usefulness: Determine the most compelling and appealing topics or categories for conversation. Understand the level of involvement the sponsoring company should have in the social experience, if any. Balance branded or non-branded experience for optimal trust of the site. Determine the elements or attributes that should allow comment and review.
Usability: Find any confusion points or interruptions that prevent users from registering. Find misleading or ambiguous terminology.
Usefulness: Determine the most persuasive elements that will compel the target audience to register. Understand a design’s impact a user’s perception of the brand. Position the offering and messaging against the company’s competitors. Determine missing content that can help target audience make an informed decision about the product.
Natural Reaction Turn to what we know: Usability Testing (one-on-one interviews, design artifact, and tasks)
Are You Forgetting Contextual Inquiry and Foundational Research? Discovery research and needs analysis is valid, but:
Time and budget for separate research is not always an option
Many participants need design artifacts to elicit appropriate reaction and commentary
Our Goal Leverage the strengths of usability testing but adjust our approach when objectives differ from finding usability problems.