Presentation about overcoming the challenges of measuring UX work for clients.

1. Proprietary and Confidential 1
Measuring UX Impact
Proprietary and Confidential 1
April 26th, 2021

2. Proprietary and Confidential 2
Obvious Stuff
01
Proprietary and Confidential 2

3. Obvious Stuff
Proprietary and Confidential 3
Why Measure Impact of UX?
● Prove that problems are being solved
● Validate our methods, designs, and products
● Provide ammunition for future success
● Better leverage in conversations with the client

4. Obvious stuff
Proprietary and Confidential 4
Collaborative Process
1. Define goals
○ Client input required
2. Define KPIs (Key Performance Indicators)
and/or metrics
○ Client input required
3. Design, Measure, Iterate, Repeat

5. Obvious stuff
Proprietary and Confidential 5
Successful UX KPIs & Metrics
- Key to the business
- Easy to measure
- Easy to attribute
- Diagnostic

6. Proprietary and Confidential 6
The hard work
02
Proprietary and Confidential 6

7. Hard work
Proprietary and Confidential 7
Clients need help
Often, clients come to us with vague needs -
● “we need to refresh our website”
● “we need to stay relevant”
● “we need a better UX”
but what does that mean, and how do we make sure
we’re following through with our promise and
understand the commitments we are making?

8. Hard work
Proprietary and Confidential 8
Nerdery
Team
Client
Product
Team

9. Hard work
Proprietary and Confidential 9
Nerdery
Team
Client
Product
Team
Sales
Marketing
Service
IT
Accounting

10. Proprietary and Confidential 10
Hard work
The Client’s Part
● Get to the value
○ Money made
○ Cost saved
○ Manpower saved
● Piggyback on metrics they already measure
○ Conversions
○ Analytics
● Get the benchmarks
○ NPS
○ Satisfaction
○ Analytics

11. Proprietary and Confidential 11
Hard work
How do we do this?
● Get UX higher into the sales process
● Build a relationship with client representative
● Sell them on the benefits good measurements
● Gather “background” through interviews

12. Proprietary and Confidential 12
MTS
03
Proprietary and Confidential 12

13. MTS
Proprietary and Confidential 13
VP of Product
“We need a modern version of
our software on a tablet to
compete better in the market”

14. MTS
Proprietary and Confidential 14
MTS Goals
● Encourage adoption among smaller labs
○ Cheaper implementation of software reduces price
○ Better UX for selling
● Encourage adoption among newer/younger test operators
○ Better UX selling
○ Mobile form factor

15. MTS
Proprietary and Confidential 15
MTS Goals
● Encourage adoption among smaller labs
○ Cheaper implementation of software reduces price
○ Better UX for selling
● Encourage adoption among newer/younger test operators
○ Better UX selling
○ Mobile form factor
● Reduce task time
○ Better UX
● Reduce training time
○ Better UX

16. MTS
Proprietary and Confidential 16
Net Promoter Score
“On a scale of 1-10, how likely are you to recommend this
software to others, with 10 being extremely likely and 1 being not
at all likely?”
- Encourage adoption among newer/younger Test
Operators
- Encourage adoption among smaller labs 1

17. MTS
Proprietary and Confidential 17
Time on Task
We measured the time it took users to complete the task, in this
case it was “configure a procedure.”
In surveys and interviews users self-reported that configuring a
procedure takes between 15-30 minutes.
- Reduce task time 2

18. MTS
Proprietary and Confidential 18
Perceived Ease of Use
After each test test participants are asked
“How difficult was this task to perform on a scale of 1 to 10. With
10 being very difficult and 1 being very easy?”
- Reduce training time
3

19. MTS
Proprietary and Confidential 19
Summary
1. Client had good goals in place: increase adoption, reach
parity with competitors, help sales.
2. UX helped create additional goals to measure success
3. New goals generated unexpected positive results

20. Proprietary and Confidential 20
Further (BCBS MN)
04
Proprietary and Confidential 20

21. Further (BCBS MN)
Proprietary and Confidential 21
VP of Sales & Marketing
“We need to refresh our
member portal and make it more
modern!”

22. Further (BCBS MN)
Proprietary and Confidential 22
Further Goals
● Reduce customer service calls
○ New debit card vendor solves problem
● Increase sales
○ Better UX

23. Further (BCBS MN)
Proprietary and Confidential 23
Further Goals
● Reduce customer service calls
○ New debit card vendor solves problem
○ Better UX
● Increase account activity
○ Better UX
● Increase customer satisfaction
○ Better UX
● Reduce enrollment abandonment
○ Better UX

24. Further (BCBS MN)
Proprietary and Confidential 24
Customer Call Analysis
Customer service calls are all recorded and categorized by topic
making it very easy to identify changes in frequency of certain
calls.
For example less support calls for a specific issue by fixing a
confusing screen or interface.
- Reduce customer service calls
- Customer Satisfaction
- Diagnostic
1

25. KPIs at Further (BCBS MN)
Proprietary and Confidential 25
Net Promoter Score
“On a scale of 1-10, how likely are you to recommend this
software to others, with 10 being extremely likely and 1 being not
at all likely?”
- Customer Satisfaction
2

26. Further (BCBS MN)
Proprietary and Confidential 26
Measure Analytics
Variety of metrics and measurements.
- Increased Activity
- Reduce abandonment
3

27. KPIs at Further (BCBS MN)
Proprietary and Confidential 27
Surveys
Build in optional survey system so users can give feedback
consistently throughout the experience.
- Customer Satisfaction
4

28. Further
Proprietary and Confidential 28
Summary
1. Client did not know what goals to measure and
potentially has bad goals to measure
2. Lots of digging required to identify good metrics
3. Good measurement put in place that didn't previously
exist helped the business tremendously

29. Proprietary and Confidential 29
Thank you
Proprietary and Confidential 29