May 25, 1961, address to congress

…this nation should
commit itself to achieving
the goal, before this
decade is out, of ...
Putting a man on the
moon is a big deal.
You’re going to need a plan.
STRATEGY: A long term plan or method for
achieving a particular mission (or goal).
The Apollo mission had eight goals.
3,683 km/h
EARTH

7

4

8
1 2

3

343,304 km

5
6

1 - 1st Burn – Insertion into Earth...
A strategy breaks up
the mission into goals:
MISSION (Strategic Objective)
GOAL

GOAL

GOAL
Tactic

STRATEGY
How?

Tactic
...
Goals should be:

S M A R T
SPECIFIC

MEASURA
BLE

ATTAINABL
E

RELEVANT

TIMERELATED
In order to stay on course.
You need to measure often
and make adjustments.

!
Measurement is the key
to an effective strategy.
Metrics are the standard of measurement
by which efficiency, performance,
progress, or quality of a plan, process, or
prod...
Analytics is the discovery and
communication of meaningful
patterns in data.
Facts and statistics
collected together for
reference or analysis.

Data
Photo: Graham Lavender: http://www.flickr.com/photos/guybrariang/

What is conveyed by a
particular arrangement
or sequenc...
Photo: Graham Lavender: http://www.flickr.com/photos/guybrariang/

Understanding,
based on information
and experience.

Kn...
Photo: http://www.sunilshinde.com/2013/07/greek-odyssey-two-weeks-in-athens-delphi-peloponnese-and-santorini.html

The abi...
SYSTEM

DATA

METRICS

INFORMATION

ANALYSIS

KNOWLEDGE

Knowledge allows you to predict the impact of changes to the syst...
But what
should you
measure?
We need to measure two things:

1
2

Are we achieving
our goals? (Strategic)
How is the system
performing? (Tactical)
Just because your system is
performing, does not mean,
you are meeting your goals.
STRATEGIC
METRICS
(Key Performance Indicators)
Measurements that directly relate to a goal/objective.
GOAL

KPI

METRICS

...
TACTICAL METRICS
(Performance Indicators)

Visits

Completion
Rates

Readability

Ranking

Pages per
visit

Task Time

Rep...
Measure what is

truly important,
not just what is

easy to measure.
Measurement
informs design
The Apollo mission involved a lot of
measurement, and iteration.
1234

8 10

EARTH

MOON

567 9
AS201:
AS202:
AS203:
Apoll...
Measurement is why the Command
Space Module looked like this.
And not like this.
It’s why the
lunar lander,
looked like this.
And not like this.
USER EXPERIENCE:
The positive or negative
experience of using a system
or product.
Image Number: SI 99-15229

Credit: Photo by Mark Avino, National Air and Space Museum, Smithsonian Institution

The User I...
The User Interface of the StarTrek LCARS system
looked good, and seems simple.
But it never had to work in real life.
User Experience consists of the User, the System,
and the Context in which the system is used.

SYSTEM

USER

(product or
...
Aspects of a good web site or application

USER
EXPERIENCE

CREDIBILITY

USABILITY

LOYALTY

• Appropriate

• Intuitive

•...
Elements of User Experience

CONTENT

NAVIGATION

STRUCTU
RE

MESSAGE

User
Experience

FUNCTIO
N

VISUAL
DESIGN

INTERACT...
How do you
measure User
Experience?
QUALITATIVE:
Non-numerical characteristics
“why and how”
QUANTITATIVE:
Numerical characteristics
“where, when and what”
Focus Sessions
Card Sorting
Tree Tests
Web Surveys
Readability Tests
Usability Tests
Accessibility Tests
A/B Tests
Load Tests
$$$

!#!

SALES

COMPLAINTS

???
INQUIRIES

Other Metrics
Web Traffic
When should you
measure?
START
MEASURING
ACTUAL TRAFFIC

DESIGN TO ENABLE
MEASUREMENT
AND MEET GOALS

DETERMINE
GOALS

IMPROVE
DISCOVER

DEFINE

DE...
Surveys, Focus
Sessions, Interviews,
Historical traffic,
Business metrics, etc.

Usability Tests,
A/B Tests, Focus
Session...
When it comes to measuring,
3 things to keep in mind.
Trends > Snapshots
Escape Velocity:

11200. m/s

10 Seconds:
20 Seconds:
30 Seconds:
40 Seconds:
50 Seconds:
60 Seconds:
3 Minutes:

24.7 m/s...
It is better to be

roughly right than
exactly wrong!
- Carveth Read
50% of people like Hot Tea

50% of people like Iced Tea

Based on averages, 100% of people
would like luke warm tea
HiPPO = Highest Paid Person’s Opinion

The more data you have,
the weaker their opinion.
Measurement leads
to big goals
Top 10 Barriers to an effective measurement
strategy
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.

Lack of budget/resources
Lack of stra...
http://xkcd.com/1074/
Why your website needs less design and more measurement. The role of measurement in a UX Strategy.
Upcoming SlideShare
Loading in...5
×

Why your website needs less design and more measurement. The role of measurement in a UX Strategy.

530

Published on

A successful web site must balance the needs of both the business and its users. To do so effectively requires a clear strategy. The core of any good strategy is measurement.

Measurement plays a vital role in any effective strategy, and ultimately in developing a great user experience. But understanding what to measure, and how, is a challenge for many organizations. This presentation and discussion will focus on how measurement is key to any good user experience strategy.

Key Points:

· Why measurement is the core of any good strategy
· How to determine and measure business goals
· The components of user experience and how to measure them

Published in: Business, Technology, Design
2 Comments
3 Likes
Statistics
Notes
No Downloads
Views
Total Views
530
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
2
Likes
3
Embeds 0
No embeds

No notes for slide
  • May 25, 1961, address to congressIn response to the Soviets putting a man in space. (Yuri Gugarin)
  • Getting to the moon is a big mission. They needed a plan.
  • 1 - 1st Burn– Insertion into Earth Orbit2 - 2nd Burn – Trans-lunar Injection3 - Trans-lunar Trajectory (3rd and 4th burns to adjust course not required) - Command Space Module extracts Lunar Module4 - Command Space Module Brakes into Lunar Orbit5 - Lunar Module Separates and Descends to the Moon - Lunar Module Launches from the Moon - Lunar Module Reunites with the Command Space Module6 - Trans-Earth Trajectory7 - Command Module Enters Earth Orbit8 - Command Module Water Recovery in Pacific Ocean
  • Mission (Strategic Objective) = WhatGoal (Objectives) = WhatTactics = WhatStrategy = HowExecution = Doing it.
  • Goals need to be SMART.Specific (clearly defined or identified)Measurable (You must be able to measure them to know if they were successful or not)Attainable (You must be able to actually accomplish them)Relevant (The goal must relate directly to accomplishing your mission)Time-related (The goal must have a deadline)
  • You can’t measure after.
  • Metrics are the measurements of data, that you choose to make.
  • Measurement creates Data.
  • Data, interpreted becomes Information.
  • Information put into action becomes knowledge.
  • Knowledge used to influence future decisions and goals is wisdom.
  • UX is more than just Design.Four key components: Content, Structure, Design and Technology. All four areas need to be measured. - It is almost impossible to measure them independently.
  • Your system is performing, does not (necessarily)mean, you are meeting your goals.
  • Just because your system is performing, does not mean, you are meeting your goals.
  • If what you are measuring, will never help you make a decision, then there is no point measuring it.
  • Good UX is built around a strategy.Strategy is long term. It needs to be broken into steps (Goals)SMART Goals make the impossible, possible.Goals MUST be measurable.
  • ISO 9241-210[1] defines user experience as "a person's perceptions and responses that result from the use or anticipated use of a product, system or service". According to the ISO definition, user experience includes all the users' emotions, beliefs, preferences, perceptions, physical and psychological responses, behaviors and accomplishments that occur before, during and after use. The ISO also list three factors that influence user experience: system, user and the context of use.
  • ISO 9241-210[1] defines user experience as "a person's perceptions and responses that result from the use or anticipated use of a product, system or service". According to the ISO definition, user experience includes all the users' emotions, beliefs, preferences, perceptions, physical and psychological responses, behaviors and accomplishments that occur before, during and after use. The ISO also list three factors that influence user experience: system, user and the context of use.
  • ISO 9241-210[1] defines user experience as "a person's perceptions and responses that result from the use or anticipated use of a product, system or service". According to the ISO definition, user experience includes all the users' emotions, beliefs, preferences, perceptions, physical and psychological responses, behaviors and accomplishments that occur before, during and after use. The ISO also list three factors that influence user experience: system, user and the context of use.
  • The User Experience of your site has a direct impact on your Goals.User Experience affects Credibility, Usability and Loyalty.Which in turn affect Acquisition, Conversion and Engagement.Which are Key Performance Indicators for our Strategic Goals.
  • The User Experience of your site has a direct impact on your Goals.User Experience affects Credibility, Usability and Loyalty.Which in turn affect Acquisition, Conversion and Engagement.Which are Key Performance Indicators for our Strategy Goals.
  • UX is more than just Design.Four key components: Content, Structure, Design and Technology. All four areas need to be measured. - It is almost impossible to measure them independently.
  • UX is more than just Design.Four key components: Content, Structure, Design and Technology. All four areas need to be measured. - It is almost impossible to measure them independently.
  • ISO 9241-210[1] defines user experience as "a person's perceptions and responses that result from the use or anticipated use of a product, system or service". According to the ISO definition, user experience includes all the users' emotions, beliefs, preferences, perceptions, physical and psychological responses, behaviors and accomplishments that occur before, during and after use. The ISO also list three factors that influence user experience: system, user and the context of use.
  • Measuring doesn’t only happen after launch, it must be continuous.The types of measurements will change.
  • 50% of Customers like Iced Tea. 50% like Hot Tea.
  • 50% of Customers like Iced Tea. 50% like Hot Tea.
  • Transcript of "Why your website needs less design and more measurement. The role of measurement in a UX Strategy."

    1. 1. May 25, 1961, address to congress …this nation should commit itself to achieving the goal, before this decade is out, of landing a man on the moon and returning him safely to the Earth.
    2. 2. Putting a man on the moon is a big deal. You’re going to need a plan.
    3. 3. STRATEGY: A long term plan or method for achieving a particular mission (or goal).
    4. 4. The Apollo mission had eight goals. 3,683 km/h EARTH 7 4 8 1 2 3 343,304 km 5 6 1 - 1st Burn – Insertion into Earth Orbit 2 - 2nd Burn – Trans-lunar Injection 3 - Trans-lunar Trajectory (3rd and 4th burns to adjust course not required) - Command Space Module extracts Lunar Module 4 - Command Space Module Brakes into Lunar Orbit 5 - Lunar Module Separates and Descends to the Moon - Lunar Module Launches from the Moon - Lunar Module Reunites with the Command Space Module 6 - Trans-Earth Trajectory 7 - Command Module Enters Earth Orbit 8 - Command Module Water Recovery in Pacific Ocean MOON
    5. 5. A strategy breaks up the mission into goals: MISSION (Strategic Objective) GOAL GOAL GOAL Tactic STRATEGY How? Tactic Tactic Mission, Objective, Goals and Tactics are all “What” EXECUTION Actually doing it. STRATEGY
    6. 6. Goals should be: S M A R T SPECIFIC MEASURA BLE ATTAINABL E RELEVANT TIMERELATED
    7. 7. In order to stay on course. You need to measure often and make adjustments. !
    8. 8. Measurement is the key to an effective strategy.
    9. 9. Metrics are the standard of measurement by which efficiency, performance, progress, or quality of a plan, process, or product can be assessed.
    10. 10. Analytics is the discovery and communication of meaningful patterns in data.
    11. 11. Facts and statistics collected together for reference or analysis. Data
    12. 12. Photo: Graham Lavender: http://www.flickr.com/photos/guybrariang/ What is conveyed by a particular arrangement or sequence of things. Information
    13. 13. Photo: Graham Lavender: http://www.flickr.com/photos/guybrariang/ Understanding, based on information and experience. Knowledge
    14. 14. Photo: http://www.sunilshinde.com/2013/07/greek-odyssey-two-weeks-in-athens-delphi-peloponnese-and-santorini.html The ability to discern or judge what is true, right, or lasting Wisdom
    15. 15. SYSTEM DATA METRICS INFORMATION ANALYSIS KNOWLEDGE Knowledge allows you to predict the impact of changes to the system. Wisdom allows you to know which changes are good for your mission.
    16. 16. But what should you measure?
    17. 17. We need to measure two things: 1 2 Are we achieving our goals? (Strategic) How is the system performing? (Tactical)
    18. 18. Just because your system is performing, does not mean, you are meeting your goals.
    19. 19. STRATEGIC METRICS (Key Performance Indicators) Measurements that directly relate to a goal/objective. GOAL KPI METRICS # of Sales Business Growth Revenue Change Govt. Policy Influence with Decision Makers $ of Sales Policy Briefs Downloaded Petitions Signed
    20. 20. TACTICAL METRICS (Performance Indicators) Visits Completion Rates Readability Ranking Pages per visit Task Time Repeat Visits Keywords Time on Page Directness Social Shares Cost per lead Bounce Rates Disaster Rates Sentiment Conversions and more…
    21. 21. Measure what is truly important, not just what is easy to measure.
    22. 22. Measurement informs design
    23. 23. The Apollo mission involved a lot of measurement, and iteration. 1234 8 10 EARTH MOON 567 9 AS201: AS202: AS203: Apollo 1: (Apollo 2): (Apollo 3): (Apollo 4): Apollo 5: Apollo 6: Apollo 7: Apollo 8: Apollo 9: Unmanned Tests of Rocket System Unmanned Tests of Rocket System Unmanned Tests of Rocket System Disaster, 3 Astronauts killed in Command Space Module (CSM) on launch pad during launch test Test of unmanned CSM Test of unmanned CSM Test of unmanned CSM Unpiloted, first flight of Lunar module (LM) Unpiloted, LM test Manned Earth Orbit Test Flight First Manned Lunar Orbit Test of all components and EVA (Spacewalk) 11121314151617 Apollo 10: Dress Rehearsal – stopped short of landing on the moon Apollo 11: One small step for man. One giant leap for mankind. Apollo 12: Apollo 13: Apollo 14: Apollo 15: Apollo 16: Apollo 17: 2nd Lunar Landing Aborted due to fire during Lunar Orbit 3rd Lunar Landing 4th Lunar Landing with Lunar Roving Vehicle (LRV) 5th Lunar Landing 6th, and last, Lunar Landing
    24. 24. Measurement is why the Command Space Module looked like this.
    25. 25. And not like this.
    26. 26. It’s why the lunar lander, looked like this.
    27. 27. And not like this.
    28. 28. USER EXPERIENCE: The positive or negative experience of using a system or product.
    29. 29. Image Number: SI 99-15229 Credit: Photo by Mark Avino, National Air and Space Museum, Smithsonian Institution The User Interface of the Command Space Module was complicated. It required hundreds of hours of training. But it worked.
    30. 30. The User Interface of the StarTrek LCARS system looked good, and seems simple. But it never had to work in real life.
    31. 31. User Experience consists of the User, the System, and the Context in which the system is used. SYSTEM USER (product or Service) CONTEXT
    32. 32. Aspects of a good web site or application USER EXPERIENCE CREDIBILITY USABILITY LOYALTY • Appropriate • Intuitive • Satisfying • Professiona l • Accessible • Competitive • Efficient • Relevant • Functional ACQUISITION CONVERSION ENGAGEMENT Desired outcomes of a good web site or application
    33. 33. Elements of User Experience CONTENT NAVIGATION STRUCTU RE MESSAGE User Experience FUNCTIO N VISUAL DESIGN INTERACTIO N TECHNOLO GY
    34. 34. How do you measure User Experience?
    35. 35. QUALITATIVE: Non-numerical characteristics “why and how” QUANTITATIVE: Numerical characteristics “where, when and what”
    36. 36. Focus Sessions
    37. 37. Card Sorting
    38. 38. Tree Tests
    39. 39. Web Surveys
    40. 40. Readability Tests
    41. 41. Usability Tests
    42. 42. Accessibility Tests
    43. 43. A/B Tests
    44. 44. Load Tests
    45. 45. $$$ !#! SALES COMPLAINTS ??? INQUIRIES Other Metrics
    46. 46. Web Traffic
    47. 47. When should you measure?
    48. 48. START MEASURING ACTUAL TRAFFIC DESIGN TO ENABLE MEASUREMENT AND MEET GOALS DETERMINE GOALS IMPROVE DISCOVER DEFINE DEFINE HOW YOU WILL MEASURE SUCCESS DESIGN DEVELOP IMPLEMENT TOOLS FOR MEASURING DEPLOY USE KNOWLEDGE GAINED FROM MEASURING TO IMPROVE
    49. 49. Surveys, Focus Sessions, Interviews, Historical traffic, Business metrics, etc. Usability Tests, A/B Tests, Focus Sessions, etc. Load Tests, QC Tests, UAT, more usability tests… IMPROVE DISCOVER DEFINE DESIGN Card Sorting, Treejack tests, Usability Testing Prototypes and wireframes, etc. DEVELOP DEPLOY Usability Testing, Load Testing, Accessibility Testing, Code Testing, etc. Web Traffic, A/B Testing, Usability Testing, Focus Sessions, Surveys, Business Metrics, etc.
    50. 50. When it comes to measuring, 3 things to keep in mind.
    51. 51. Trends > Snapshots
    52. 52. Escape Velocity: 11200. m/s 10 Seconds: 20 Seconds: 30 Seconds: 40 Seconds: 50 Seconds: 60 Seconds: 3 Minutes: 24.7 m/s 56.4 m/s 45.6 m/s 14.5 m/s 20.3 m/s 27.2 m/s 283.2 m/s 2 hrs, 44 Minutes: 10423. m/s Snapshots don’t tell you the whole story
    53. 53. It is better to be roughly right than exactly wrong! - Carveth Read
    54. 54. 50% of people like Hot Tea 50% of people like Iced Tea Based on averages, 100% of people would like luke warm tea
    55. 55. HiPPO = Highest Paid Person’s Opinion The more data you have, the weaker their opinion.
    56. 56. Measurement leads to big goals
    57. 57. Top 10 Barriers to an effective measurement strategy 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Lack of budget/resources Lack of strategy Siloed organization Lack of understanding Too much data Lack of senior management buy-in Difficulty reconciling data IT blockages Lack of trust in analytics Finding staff http://econsultancy.com/uk/reports/online-measurement-and-strategy-report 45% 31% 29% 25% 18% 18% 17% 17% 16% 12%
    58. 58. http://xkcd.com/1074/

    ×