Slides from the Usability Workshop at Ascend UX Conference. Ran by Eunice Yang and Chris Fick.

1. Usability Testing and Beyond
Chris Fick, PhD | Eunice Yang, PhD
UX Researchers @ PROS

2. Inform
Decide
Understand People & Problems
Prototype a Solution
We design effective product experiences for our users.
You’ve already gone through these steps…

3. Inform Get Feedback
Now we’ll focus on…

4. What can you learn from usability testing?
Is your design easy to
navigate?
What can be
improved?
How does your design
compare?
Are users’ goals easy
to accomplish?

5. How do you test your design?
Task completion rate
Time on task
Pageviews
Click rate
Satisfaction score
https://usabilla.com/blog/ux-travel-sector/

6. How do you test your design?
Task completion rate
Time on task
Pageviews
Click rate
Satisfaction score
https://usabilla.com/blog/ux-travel-sector/

7. How do you test your design?
Task completion rate
Time on task
Pageviews
Click rate
Satisfaction score

8. How do you test your design?
Task completion rate
Time on task
Pageviews
Click rate
Satisfaction score
https://conversionxl.com/blog/visual-engagement-metrics

9. After Scenario Questionnaire (ASQ)
NASA’s task load index is a measure of mental effort (NASA-TLX)
Subjective Mental Effort Questionnaire (SMEQ)
Usability Magnitude Estimation (UME)
Single Ease Question (SEQ)
Standardized User Experience Percentile Rank Questionnaire (SUPR-Q)
Computer System Usability Questionnaire (CSUQ)
Questionnaire For User Interaction Satisfaction (QUIS)
Software Usability Measurement Inventory (SUMI)
Net Promoter Score (NPS)
Usability Metric for User Experience (UMUX)
System Usability Scale (SUS)
How do you test your design?
Task completion rate
Time on task
Pageviews
Click rate
Satisfaction score

10. How do you test your design?
Task completion rate
Time on task
Pageviews
Click rate
Satisfaction score

11. Design the study
Research goal Metrics
& Tasks
Testing Script Recruitment Pilot Go!

12. Practice Usability Testing
Practice moderating, tracking metrics, and taking notes

14. What is your goal with research?
How can we improve Velvet Taco’s
online order rate?

15. What are you going to test?
How long does it take to make an order online?
Can users easily select and customize their order?
What do users think of Velvet taco’s ordering
experience?
Time on task
Task completion rate
Satisfaction score
Comments

16. Moderator’s Form
Introduction
“Thank you for participating. I’m interested in improving users’ experience of the Velvet Taco
website. Today we’re going to explore their website as if we’re potential customers.”
“We’ll go through a scenario for placing an order. Please talk me through what you are thinking as
you go. However, I can’t answer any questions until you are done. Please be honest and remember
that we’re evaluating the website, not you. Do you have any questions before we begin?”
Scenario & Task
“Please read the scenario and task on out loud.”
Participant reads OUT LOUD from “Participant’s Form – Scenario & Task”
“You may begin. Remember to say out loud what you are doing and thinking.” [Start Timer]
Notes
Take notes here on users’ actions, comments, points of frustration.
Stop timer when participant says total price of order
• Task Duration: _________ minutes
• Order total: $_________ ($9.96 is the correct answer)
Follow-up questions
“What are your general impressions of the site?”
“Is there anything that can be improved about the site?”
“Please flip over your form and rate your satisfaction with the website.”
Participant’s Form
Scenario & Task
Imagine that you had to work through lunch and want to pick up food from the Velvet Taco in
Montrose (77006). You want to order ahead using their mobile site.
You will start by going to www.velvettaco.com on your phone.
You want 1 Falafel taco without onions and 1 order of tater-tots. Add these items to your cart.
Stop when you have found the total price of your order and say it out loud. Do not create an
account or enter any personal information.
Questionnaire
Please rate your satisfaction with using Velvet Taco’s website:
Very dissatisfied Dissatisfied Neutral Satisfied Very Satisfied
1 2 3 4 5
Scenario & Task
Imagine that you had to work through lunch and want to pick up food from the Velvet Taco in
Montrose (77006). You want to order ahead using their mobile site.
You will start by going to www.velvettaco.com on your phone.
You want 1 Falafel taco without onions and 1 order of tater-tots. Add these items to your cart.
Stop when you have found the total price of your order and say it out loud. Do not create an
account or enter any personal information.

17. In your Packet📑
Moderator’s Form
Introduction
“Thank you for participating. I’m interested in improving users’ experience of the Velvet Taco
website. Today we’re going to explore their website as if we’re potential customers.”
“We’ll go through a scenario for placing an order. Please talk me through what you are thinking as
you go. However, I can’t answer any questions until you are done. Please be honest and remember
that we’re evaluating the website, not you. Do you have any questions before we begin?”
Scenario & Task
“Please read the scenario and task on out loud.”
Participant reads OUT LOUD from “Participant’s Form – Scenario & Task”
“You may begin. Remember to say out loud what you are doing and thinking.” [Start Timer]
Notes
Take notes here on users’ actions, comments, points of frustration.
Stop timer when participant says total price of order
• Task Duration: _________ minutes
• Order total: $_________ ($9.96 is the correct answer)
Follow-up questions
“What are your general impressions of the site?”
“Is there anything that can be improved about the site?”
“Please flip over your form and rate your satisfaction with the website.”
Participant’s Form
Scenario & Task
Imagine that you had to work through lunch and want to pick up food from the Velvet Taco in
Montrose (77006). You want to order ahead using their mobile site.
You will start by going to www.velvettaco.com on your phone.
You want 1 Falafel taco without onions and 1 order of tater-tots. Add these items to your cart.
Stop when you have found the total price of your order and say it out loud. Do not create an
account or enter any personal information.
Questionnaire
Please rate your satisfaction with using Velvet Taco’s website:
Very dissatisfied Dissatisfied Neutral Satisfied Very Satisfied
1 2 3 4 5
Scenario & Task
Imagine that you had to work through lunch and want to pick up food from the Velvet Taco in
Montrose (77006). You want to order ahead using their mobile site.
You will start by going to www.velvettaco.com on your phone.
You want 1 Falafel taco without onions and 1 order of tater-tots. Add these items to your cart.
Stop when you have found the total price of your order and say it out loud. Do not create an
account or enter any personal information.

18. In your Packet📑
Moderator’s Form
Introduction
“Thank you for participating. I’m interested in improving users’ experience of the Velvet Taco
website. Today we’re going to explore their website as if we’re potential customers.”
“We’ll go through a scenario for placing an order. Please talk me through what you are thinking as
you go. However, I can’t answer any questions until you are done. Please be honest and remember
that we’re evaluating the website, not you. Do you have any questions before we begin?”
Scenario & Task
“Please read the scenario and task on out loud.”
Participant reads OUT LOUD from “Participant’s Form – Scenario & Task”
“You may begin. Remember to say out loud what you are doing and thinking.” [Start Timer]
Notes
Take notes here on users’ actions, comments, points of frustration.
Stop timer when participant says total price of order
• Task Duration: _________ minutes
• Order total: $_________ ($9.96 is the correct answer)
Follow-up questions
“What are your general impressions of the site?”
“Is there anything that can be improved about the site?”
“Please flip over your form and rate your satisfaction with the website.”
Participant’s Form
Scenario & Task
Imagine that you had to work through lunch and want to pick up food from the Velvet Taco in
Montrose (77006). You want to order ahead using their mobile site.
You will start by going to www.velvettaco.com on your phone.
You want 1 Falafel taco without onions and 1 order of tater-tots. Add these items to your cart.
Stop when you have found the total price of your order and say it out loud. Do not create an
account or enter any personal information.
Questionnaire
Please rate your satisfaction with using Velvet Taco’s website:
Very dissatisfied Dissatisfied Neutral Satisfied Very Satisfied
1 2 3 4 5
Scenario & Task
Imagine that you had to work through lunch and want to pick up food from the Velvet Taco in
Montrose (77006). You want to order ahead using their mobile site.
You will start by going to www.velvettaco.com on your phone.
You want 1 Falafel taco without onions and 1 order of tater-tots. Add these items to your cart.
Stop when you have found the total price of your order and say it out loud. Do not create an
account or enter any personal information.

19. In your Packet📑
Scenario & Task
Imagine that you had to work through lunch and want to pick up food from the Velvet Taco in
Montrose (77006). You want to order ahead using their mobile site.
You will start by going to www.velvettaco.com on your phone.
You want 1 Falafel taco without onions and 1 order of tater-tots. Add these items to your cart.
Stop when you have found the total price of your order and say it out loud. Do not create an
account or enter any personal information.

20. In your Packet📑
Moderator’s Form
Introduction
“Thank you for participating. I’m interested in improving users’ experience of the Velvet Taco
website. Today we’re going to explore their website as if we’re potential customers.”
“We’ll go through a scenario for placing an order. Please talk me through what you are thinking as
you go. However, I can’t answer any questions until you are done. Please be honest and remember
that we’re evaluating the website, not you. Do you have any questions before we begin?”
Scenario & Task
“Please read the scenario and task on out loud.”
Participant reads OUT LOUD from “Participant’s Form – Scenario & Task”
“You may begin. Remember to say out loud what you are doing and thinking.” [Start Timer]
Notes
Take notes here on users’ actions, comments, points of frustration.
Stop timer when participant says total price of order
• Task Duration: _________ minutes
• Order total: $_________ ($9.96 is the correct answer)
Follow-up questions
“What are your general impressions of the site?”
“Is there anything that can be improved about the site?”
“Please flip over your form and rate your satisfaction with the website.”
Participant’s Form
Scenario & Task
Imagine that you had to work through lunch and want to pick up food from the Velvet Taco in
Montrose (77006). You want to order ahead using their mobile site.
You will start by going to www.velvettaco.com on your phone.
You want 1 Falafel taco without onions and 1 order of tater-tots. Add these items to your cart.
Stop when you have found the total price of your order and say it out loud. Do not create an
account or enter any personal information.
Questionnaire
Please rate your satisfaction with using Velvet Taco’s website:
Very dissatisfied Dissatisfied Neutral Satisfied Very Satisfied
1 2 3 4 5
Scenario & Task
Imagine that you had to work through lunch and want to pick up food from the Velvet Taco in
Montrose (77006). You want to order ahead using their mobile site.
You will start by going to www.velvettaco.com on your phone.
You want 1 Falafel taco without onions and 1 order of tater-tots. Add these items to your cart.
Stop when you have found the total price of your order and say it out loud. Do not create an
account or enter any personal information.

21. In your Packet📑
Moderator’s Form
Introduction
“Thank you for participating. I’m interested in improving users’ experience of the Velvet Taco
website. Today we’re going to explore their website as if we’re potential customers.”
“We’ll go through a scenario for placing an order. Please talk me through what you are thinking as
you go. However, I can’t answer any questions until you are done. Please be honest and remember
that we’re evaluating the website, not you. Do you have any questions before we begin?”
Scenario & Task
“Please read the scenario and task on out loud.”
Participant reads OUT LOUD from “Participant’s Form – Scenario & Task”
“You may begin. Remember to say out loud what you are doing and thinking.” [Start Timer]
Notes
Take notes here on users’ actions, comments, points of frustration.
Stop timer when participant says total price of order
• Task Duration: _________ minutes
• Order total: $_________ ($9.96 is the correct answer)
Follow-up questions
“What are your general impressions of the site?”
“Is there anything that can be improved about the site?”
“Please flip over your form and rate your satisfaction with the website.”
Participant’s Form
Scenario & Task
Imagine that you had to work through lunch and want to pick up food from the Velvet Taco in
Montrose (77006). You want to order ahead using their mobile site.
You will start by going to www.velvettaco.com on your phone.
You want 1 Falafel taco without onions and 1 order of tater-tots. Add these items to your cart.
Stop when you have found the total price of your order and say it out loud. Do not create an
account or enter any personal information.
Questionnaire
Please rate your satisfaction with using Velvet Taco’s website:
Very dissatisfied Dissatisfied Neutral Satisfied Very Satisfied
1 2 3 4 5
Scenario & Task
Imagine that you had to work through lunch and want to pick up food from the Velvet Taco in
Montrose (77006). You want to order ahead using their mobile site.
You will start by going to www.velvettaco.com on your phone.
You want 1 Falafel taco without onions and 1 order of tater-tots. Add these items to your cart.
Stop when you have found the total price of your order and say it out loud. Do not create an
account or enter any personal information.

22. • Assign moderator and participant
• Run a testing session using the Moderator’s and Participant’s Forms
👥 In pairs
🕑 10 minutes
🛠 Practice Test
🕑

23. Usability Research of Velvet Taco
15 participants tested usability of the mobile site. They were tasked with
adding to their cart. We analyzed where they navigated, what they clicked,
and how long it took to complete the task. Participants then rated their
experience with the site.
5.1
out of 7 in
Ease of Use
4.6
minutes to reach
checkout
8
pageviews before
checkout
“Next time I would probably just call.”

24. Recommendations
Make the “Order online” button more
noticeable
Improve discoverability
Retain information entered on the home
page.
Streamline the workflow
73 % who browsed
the site did not see
the order button
New visitors must
enter their
location twice

25. Test your prototype
Uncover ways to improve your design with usability testing

26. • Identify metrics
• Write introduction
• Craft scenario & task
• Choose follow-up questions
👤 Team
🕑 10 minutes
🛠 Design your study
You want to test the experience and usability of your prototype design for your target user’s main goal.
What will you measure during usability testing? Feel free to choose more than one metric.
Example metrics:
• Task completion
• Time on each step
• Time on task
• Number of clicks
• Number of pages
• Satisfaction score
Brenda
Goal Buy a pair of running shoes
Size Women’s 8D
Color Orange
Laces Cotton
Payment Uses Master Card
Phone 832-902-2045
Email bsmith@pros.com
Address 4 MacGregor Lane, Apt. 3100
Houston, TX
Introduction
Provide participants with an overview of why they are here and what they will do. Create y
introduction or modify the introduction from the Velvet Taco example.
Scenario & Task
What task will tie into your research goal? Determine one task that will ta
What is a good scenario that supports the task?
Follow-up questions, questionnaire
What questions do you want to ask the participant?
🕑

27. • Assign roles
• Test 2 participants, one at a time
• Alternate roles between sessions
👥 Team
🕑 20 minutes
Team roles
1. Moderator
2. Computer
3. Note taker(s)
4. Participant for another team
5. Time keeper (optional)
6. Click counter (optional)
🛠 Conduct your study
🕑

28. Affinity mapping

29. Write take-aways from testing
👥 Individual
🕑 7 minutes
🛠 Analyze
• What worked well in your design?
• What didn’t work well?
• What are must-have improvements?
• What are nice-to-have enhancements?
🕑

30. • Group notes and label themes
• Discuss main findings and
recommendations
👥 Team
🕑 7 minutes
🛠 Discuss
🕑

31. 🎉 Conclusion
• Anyone can do usability testing!
• Practice, practice, practice
• Research, design, repeat
Any feedback is better than no feedback!