Traditionally, UX researchers have relied heavily on qualitative research methods to gather feedback from users. That means combing through hours of videos of users interacting with the product, making copious notes, and whittling the information down to actionable items for your coworkers.
The process is time-consuming, however, and makes it difficult to track common themes across multiple tests. That’s why UserTesting decided to incorporate survey-style questions into our arsenal of testing techniques for subscription clients. We call these survey-style questions Customer Experience Analytics, and there are three kinds: Multiple choice, Rating Scale, and Written answer.
Learn more from our experts in this on-demand webinar: http://bit.ly/UTmetrics
2. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Customer Experience Analytics are survey-style
questions incorporated into the UserTesting
platform. They make it easy to capture additional
feedback from study participants to round out
qualitative research.
Introduction
3. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
We call these survey-style questions Customer
Experience Analytics, and there are three kinds:
Multiple Choice Rating Scale Written Answer
Introduction
4. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Introduction
Customer Experience
Analytics helps you analyze
responses from large groups
of people quickly and without
much effort.
5. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Why are Customer Experience Analytics helpful?
They save you time.
They improve the quality of your research.
They make it easier to share study results
with others.
Introduction
8. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Need to find out what words or phrases
testers would use to describe your app?
Written-response questions are your best friend.
Save Time
9. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Save Time
Example:
Share the three words that describe the
way you feel about this app:
Happy
Mad
Ah Ha!
Excited!
WOW
Satisfied
Great
Frustrated
10. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Save Time
Want to settle a team debate over which version
of your homepage appeals to more users?
Multiple-choice answers are quick and clear.
11. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Curious if users think a drop-down menu
makes it easier to find something than a list?
Have them try it both ways and rate
the task’s ease or difficulty on a 5-point scale.
Save Time
Very Difficult Very Easy
12. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Responses to these survey-style questions can be viewed
almost instantly and indicate which users struggled most and
least with your product, so you can focus your qualitative analysis
on those users and avoid watching every video start to finish.
Save Time
14. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Improve Quality
Asking survey-style questions during a user test
recording gives your users a chance to explain
WHY they are providing the low ratings or negative
multiple choice responses.
15. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Improve Quality
Answers often concisely highlight certain problem
areas on the site:
“I’m only going to give this a three,
because even though I found the product
I was looking for, it took me three tries to
get the right category.”
16. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Improve Quality
Answers also might highlight confusing questions:
“I’m not sure if you’re asking about the
colors of the icons or the whole site, but…”
18. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Most of the time, our clients are responsible not
only for running user studies, but for passing on the
insights they’ve gained to their team of developers,
designers, and so on.
Share Results
19. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Supplement qualitative findings with charts and
graphs that provide “big picture” information in an
easily-digestible way.
Share Results
In this example, there was a clear preference for the live site.
20. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Share Results
So if you’re having trouble displaying results to your
team (or your boss), a few survey-style responses
can improve your report and impress upon your
team just how actionable an issue is.
22. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Guidelines
Guideline 1: Set specific objectives for each study,
and keep them close by
Write objectives down before you craft a single task,
and keep them visible throughout the test process
to make sure your task writing, analysis, and report
remain focused on the same issues.
24. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Guidelines
Guideline 2: Use analytics questions in moderation
Space your questions out with simple instructions.
25. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Guidelines
Guideline 3: Keep users engaged
Too many rating-scale questions within a single
test causes users to focus on providing the answers
required, and stop talking out loud about decisions.
26. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Guidelines
You can avoid this user fatigue by alternating
question types within the test.
For example, if your objective is to determine how
helpful they consider a specific feature of your app...
27. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Guidelines
Have participants:
1 Use the feature
(Basic task)
2 Describe this feature in 3 words or phrases
(Written-response question)
3 Indicate if the tool is extremely helpful,
somewhat helpful, or not helpful
(Multiple-choice question)
4 Rate how likely they would be to use
the feature if they needed help
(Rating Scale)
28. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Guidelines
Guideline 4: Keep the questions simple
The easiest way to avoid frustrating the user is to
keep the questions simple.
29. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Guidelines
Consider this scenario:
You’re a tester who’s just been instructed to find an
item that you would like to buy from Macys.com and
then add it to the cart.
You easily find an item, but have trouble with the
size and color selector, which makes adding the
item to the cart downright frustrating.
31. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Guidelines
Now you’re in a pickle; you would rate finding the
item as a 5, but adding the item to the cart as a 1.
You can’t proceed with the test until you provide an
answer, and the clock is ticking away.
32. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Guidelines
One way to avoid this is by breaking up the
question into two simpler tasks: first rating the
ease of finding an item, then the ease of adding that
item to the cart. Simple.
33. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Closing Thoughts
So there you have it! Guidelines straight from the
Research Team to help you make the most of User
Testing’s Customer Experience Analytics.
They have proven to be valuable and versatile tools
for us, and now you can confidently add them to
your testing toolkit in future studies!
34. www.UserTesting.com | 1-800-903-9493 | sales@usertesting.com
Introduction
Get even more expert tips from our On-Demand
Webinar, Making the Most of Metrics, here.
Two of our expert researchers will show you:
1. The different types of metrics questions,
and when to use each one.
2. How to write great metrics questions that
will get you insights fast.
3. The top things you can learn from these
types of questions.
Ki Arnould Toni Allen