The User Experience Health Check (IA Summit)

User Experience Health Check A measure a day keeps the redesign away A measure a day keeps the redesign away Livia Labate & Austin Govella | IA Summit 2009, Memphis, TN

Presentation Agenda g What is the UX Health Check The origins of the approach When to apply this method How it evolved over the years How to conduct a UX Health Check Who needs to be involved How to report the assessment How to compare over time How to sell the idea

What we’ll really talk about today y y

Where did this come from? The Health Check was created to help product management communicate progress and improvements to executives. • Th There was immense pressure from upper management to show i f tt h progress on a specific product line (new capital investment). • General manager wanted a report she could give our president g p g p every 30 days that showed how we “moved the dial” • By numerically rating our progress on each aspect of the service once a month, we could report whether or not and how much once a month we could report whether or not and how much we “moved the dial”.

User Experience Health Check A method to quantify the qualitative user experience 5

Why do it: Wh d it • Applicable to any (specially the last bullet!) product or service • Measures detailed feature sets or very generalized e o e y ge e a ed notions of service. •IIntroduces a shared d h d language for teams to discuss elements of an di l f experience. 6

How do you do it? 1. Deconstruct the service (list capabilities and chunk logical groups) (list capabilities and chunk logical groups) 2. Choose competitive benchmarks (similar solutions that can be compared) (similar solutions that can be compared) 3. Establish scoring criteria (set t e e e e t a easu g sca e) (set the referential measuring scale) 4. Set target scores 5. Set current score based on existing state Set current score based on existing state 6. Tally up and communicate 7. Rinse and repeat (periodically)! 7 Rinse and repeat (periodically)!

It all starts with a project p j For Example: p • You’re planning a conference (like the 2010 IA Summit!) • You’re building a website (to inform, communicate & publicize) ( i f i & bli i ) • You have a list of things you want to do You have a list of things you want to do 8

Step 1: Deconstruct the Service Break down the parts of the product or service to a level they can be looked at in isolation and still be comparable to examples elsewhere. • By product or services in a(n eco)system By product or services in a(n eco)system • By user’s Mental Model spaces • By generic/universal mental models (consumption and service lifecycles)

Exercise: Generate a list of capabilities This is when we asked the audience to help us by shouting out things that make up the “IA Summit”; the items we should consider when deconstructing the whole experience of attending the event.

Exercise: Chunk capabilities into groups We then had some volunteers come up to the front of the room and cluster the capabilities identified (we wrote then down into post‐it notes) creating chunks of like items to easily roll‐up parts of the UX Health Check.

Step 2: Choose Competitive Benchmarks Identify similar solutions (products and services) that have similar capabilities to the ones identified for comparison. • For each capability answer the question: For each capability answer the question: “Who does this well?” • Select at least one for each capability; you can add Select at least one for each capability; you can add multiple benchmarks for each if relevant. • The benchmark product or services don’t need to be in the same industry or serve the same purpose as the capability you are comparing to.

Exercise: Choose Competitive Benchmarks We briefly went through some of the capabilities and for each one, asked the audience to identify a few relevant benchmarks. For example: “Event registration” yield evite.com, not‐IXDA, etc.

Step 3: Establish the Scoring Criteria Score Rating Considerations < 30 Problem Area Doesn’t meet core user needs or usability standards: It’s broken! 30 > 30 Meets basic user needs; isn’t broken; M t b i d i ’t b k Functional the bare minimum 50 > Parity with Meets user expectations across Benchmarks comparable capabilities 70 > Very Good Better than several benchmarks 80 > Better Than Most Exceeds the main competitors, almost as good as the market leader 90 > Best In Class Market differentiator

Step 4: Set Target Scores Go through each capability and ask the question “How good do we need to be at this to meet our business goals and user expectations? expectations?” • Involve core team of people actively making decisions Involve core team of people actively making decisions about the user experience of the service you are assessing. • Get agreement on the targets through conversation. Everyone is empowered to disagree and defend their perspective; use insights from existing research and i i i h f i i h d competitive analysis to fuel and support arguments.

Exercise: Set target scores The audience contributed what they thought were the right targets for the IA Summit in various capabilities based on the scoring criteria we reviewed. Through discussion we agreed on targets that were originally divergent.

Step 5: Evaluate and Score Current Review current solution and ask “Compared to Compared our targets and where we want to be, how good are we today?” today? • Get agreement on the final score through conversation Get agreement on the final score through conversation. • Everyone is empowered to disagree and defend their perspective • Everyone should be versed in existing information to help decision‐ making (usability evaluations, user feedback, etc)

Exercise: Evaluate and score The audience contributed with their opinion of where certain capabilities were today (during the IA Summit this was being presented). The intermittent Wi‐Fi Internet access did not get a g good score on the spreadsheet!

Step 5: Tally up and communicate Step 5: Tally up and communicate This page intentionally left blank. The point of this slide was to show that the final artifact is not the point of conducting a UX Health Check. What it looks like is not very relevant, but what you communicate and learn through this process that expresses the real value of the work.

Example Communicate the baseline experience

Example Problem areas that need love

Example Gaps where the UX doesn’t meet the vision

Example Chart UX progress over time

Example Map UX evolution to business metrics

A few good reasons to give it a try! g g y • To help answer “How well does this service meet user needs, expectations and motivations?” (i d t ti d ti ti ?” (in terms all tribes can understand). • To provide a snapshot of the experience at a point d h f h in time AND track its evolution over time. • To identify which are the biggest problems and opportunities areas to influence future work prioritization and product direction. prioritization and product direction • To serve as a concrete artifact portraying how your work is directly affecting the service evolution. k i di l ff i h i l i 26

What we covered today y

Ask questions, find out more and let us know if we can help you get started at http://uxhealthcheck.com @austingovella on Twitter http://ThinkingAndMaking.com p g g @livlab on Twitter http://livlab.com/thinkia

http://uxhealthcheck.com	159
http://livlab.com	53
http://iaclients.wgbh.org	10
http://lanyrd.com	10
http://www.slideshare.net	9
http://www.linkedin.com	5
http://capcloud.com	3
http://web.archive.org	2
http://localhost	1

The User Experience Health Check (IA Summit)

by Livia Labate on Mar 25, 2009

Accessibility

Categories

Upload Details

Usage Rights

9 Embeds 252

Statistics

The User Experience Health Check (IA Summit) Presentation Transcript