Presented at Web Unleashed on September 16-17, 2015 in Toronto, Canada More info at www.fitc.ca/webu Measuring UX with Hira Javed OVERVIEW User experience is the sum of a series of interaction. It’s about motivations, expectations, behaviours and attitudes, and that’s hard to quantify. In order to know whether something is working, a signal or an indication of progress is needed, and progress cannot be measured without a clear sense of what success looks like. Metrics are the signals that show whether a UX strategy is working or not. But signals can be ambiguous. This presentation will focus on going beyond basic metrics for evaluating the impact of UX changes, and identifying effective KPIs for measuring the quality of UX. OBJECTIVE To understand the limitations of basic traffic metrics when it comes to measuring UX, and discuss Google’s HEART Framework for a holistic measure of quality UX. TARGET AUDIENCE Anyone involved in strategizing and creating user experiences. FIVE THINGS AUDIENCE MEMBERS WILL LEARN Challenges with measuring UX Limitations of basic traffic metrics Characteristics of effective UX KPIs Google’s HEART Framework for measuring the quality of UX Applying the HEART Framework

1. Identifying and tracking UX
performance indicators
Hira Javed
September 17th, 2015

2. About Me
Usability Specialist
@hirajaved10
linkedin.com/in/hirajaved
UX Instructor

3. The challenge with measuring UX
UX Key Performance Indicators
The HEART Framework and Goals-Signals-
Metrics Process
Overview

4. How do you plan a road trip?

5. “User experience represents the
perception left in someone’s mind
following a series of interactions
between people, devices, and events –
or any combination thereof.”
- Eric Reiss

6. UX = the sum of a series of
interactions

7. What does success look like to you?

8. UX is about…
motivation
attitudes
expectations
behavioural patterns
constraints
That’s hard to quantify.

9. In order to know whether something is
working, you need a signal.
Metrics are the signals that show whether your
UX strategy is working.

10. But…signals can be ambiguous.
Easily collected, yet not always important.
Measurable, yet not always informative.

11. Basic traffic metrics are easy to track and give
a good baseline on how your site is doing, but
they often aren’t very useful for evaluating the
impact of ux changes.

12. Effective UX KPIs are:
Behaviour based
Indicator of performance
Key to the business
Unique to the business
Easy to measure
Diagnostic
Source: http://www.uie.com/articles/power_of_ux_kpi/

13. Behaviour-based: Behavioural information is
a critical indicator of user’s experience.
Performance indicators: A good KPI predicts
an important change in the business,
hopefully with enough of a lead time to
react if necessary.

14. Key to the business: Five business goals KPIs
can be related to:
1) increasing revenues
2) decreasing costs
3) increasing market share
4) increasing revenue from existing
customers and
5) increasing shareholder value

15. Unique to the business: Generic KPIs
produce generic results. KPIs should be
specific to the how the business runs.
Easy to measure: KPIs are measured
frequently, therefore they should be easy to
track.
Diagnostic: KPIs should help identify what
the problem is.

16. The HEART Framework
Source: http://www.gv.com/lib/how-to-choose-the-right-ux-metrics-for-your-product

17. The HEART Framework
Developed by Google’s research team.
Measures the quality of user experience.
Can be applied to a specific feature or the
whole product.

18. Happiness

19. Engagement

20. Adoption

21. Retention

22. Task Success

23. Goals-Signals-Metrics Process
Source: http://www.gv.com/lib/how-to-choose-the-right-ux-metrics-for-your-product

24. Goals-Signals-Metrics Process
Articulate goals as a team.
Identify signals that indicate success of the goal.
Build specific metrics to monitor the signals.

25. Putting it all together !

26. Goals Signals Metrics
Happiness Users share story page
with their social
networks
An increase in traffic
from referral sites
Number of visitors
from referral sites
Engagement Users spend more time
on our website
The amount of
content they
consume
Number of stories
they read per
session
Adoption Users subscribe to CBC
newsletters
Users visit the
subscription page
Number of new
subscribers
Retention Users check CBC news
app frequently during
the day
Users read the story
whenever a
notification is sent
out
Number of
pageviews on a story
linked to a news
alert
Task Success Users can comment on
a story
Users complain
about the
commenting process
Number of
complaints about
commenting process

27. Lessons Learned
• This is a lot harder than it looks!
• Not all elements of the HEART Framework will
apply to you. Need to customize.
• Bridges the gap between product creators
and product users. Creates empathy.
• Helps make direct connection between
product KPI and UX strategy.

28. Thank you.
Questions?