Quantify your concept (Poster & associated flyer, Ryan Kasper)


Published on

A key tool in the design process for UX professionals is concept testing. Traditionally, concepts for new products or design approaches are tested qualitatively by showing potential users visual sketches or wireframes during an interview session. While this approach can generate rich information about individual reactions to the concept, it is difficult to extrapolate to understand reactions from a larger potential user base. This poster will outline the methods we have used to enhance concept evaluation by including quantitative user reaction surveys that combine UX and market research techniques, as well as describe how this methodology can motivate further concept development.

Published in: Design
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Quantify your concept (Poster & associated flyer, Ryan Kasper)

  1. 1. Quantify your concept: Combining market research and UX survey methods to evaluate new ideas How do you test out a new idea? Ryan Kasper Idea Talk to People Qualitative Survey the Market Quantitative Part 1 Show Concept Part 2 Ask UX Questions User tests of new ideas can provide rich qualitative information about how the idea can impact users, but it takes additional research to understand how strong that impact could be in the larger market. To gauge the impact on a larger scale, you can use a concept evaluation survey, which allows you to show and describe concepts while gathering quantitative data. Concept Evaluation Survey Part 3 Market Research Reaction ratings Multiple choice Open-ended A/B testing System Usability Scale Net Promoter Score Jobs-To-Be-Done Likehood to purchase Pricing sensitivity Demographics @Dr_RyanKasper ryan.kasper@citrix.com
  2. 2. Survey Example Part 1. Show Concept Part 2. UX Questions Part 3. Market Research “This is your phone. You open the app and arrive on this screen...” Reaction Ratings 1 2 3 4 5 6 7 Would you like to use this feature? Multiple Choice How would you share with a friend? Other Multiple Choice I think I would........... ...... ....... “How would you use this saving option?” 1 2 3 4 5 a) I would like to use this tool frequently ... k) I needed to learn a lot before using the tool System Usability Scale Would you recommend this to a friend?Net Promoter Score 1 2 3 4 5 6 7 8 9 10 How likely would you be to purchase this tool?Likehood to Purchase 1 2 3 4 5 6 7 Please enter the price point at which this tool: a) is too expensive: b) is so inexpensive you question the quality: c) is getting expensive, but you still consider it: d) is a bargain: Pricing Sensitivity Minimize my effort to find a photograph a) How important is this need to you? b) How well is this need currently satisified? Jobs-To-Be-Done 1 2 3 4 5 6 7 8 9 10 Please select your industry Healthcare Law Other Demographics
  3. 3. Survey Method Details A/B Testing Show a group of people one version, and the other group a different version. Then, compare their answers to certain questions. Version A Q1 avg = 3.7 Version B Q1 avg = 5.9 Part 1. Show Concept This part of the survey focuses on showing and describing the concept. You can use the same images as in 1:1 sessions. One interesting approach to comparing alternative concept versions is to do an A/B test. Part 2. UX Questions System Usability Scale (SUS)¹ If you want to gauge how usable your concept seems, consider using the SUS. The SUS is a 10-question tool that gives you a perceived usability measure. SUS Analysis SUS Score 100 0 Industry Average Concept Ask UX questions along the way to understand how people react to the concept. Some standard survey questions include reaction ratings, multiple choice, and open-ended questions. References 1. Brooke, J. (1996) Usability Evaluation in Industry. 2. Reichheld, F. (2003) Harvard Business Review 3. Van Westendorp, P. (1976) Proceedings of the ESOMAR Congress. Pricing Sensitivity3 To determine the potential monetary value of the concept, use the Van Westendorp Pricing Sensitivity Model, which uses 4 questions to determine acceptable price points. a) too expensive b) so inexpensive you would question the quality c) getting expensive, but would still consider it d) a bargain $$ Amount %Responses Highest acceptable Price Points: Lowest acceptable Part 3. Market Research Jobs-To-Be-Done2 This method can help identify the market opportunity for a customer need, as well as the potential value of the concept. Current Satisfaction How well is this need currently satisfied? Importance How important is this need to you? High Opportunity Gauge market opportunity Satisfaction w/ Tool How well would this tool satisfy this need? Importance How important is this need to you? High Value Gauge value of concept To understand more about the potential market for the concept, include market research questions. For example, you may want to know the demographics of those that respond well to the concept, or whether people would purchase the tool. Below are two methods that can give great market insights. Ryan Kasper ryan.kasper@citrix.com @Dr_RyanKasper