Usability <> Web Metrics ; Advancing analytics through the lessons of usability ...and improving user experience along the way

Human Factors Interna/onal UX <> AX  Advancing analy0cs through the lessons of usability ...and improving user experience along the way 5  December 2008 David Mahaﬀey Director ‐ Human Factors Interna2onal Kath Straub, PhD Chief Scien2st ‐ Human Factors Interna2onal

Analy0cs is just data.

4

NIH types know a lot about  how to handle data.

Observe Describe Explain Predict

What we’ve learned  doing usability may help analy0cs  move faster 

Analy0cs is also be about  what can happen.

Analy0cs is one part of a bigger picture.

Measurable Business Objec0ves Usability  Web Tes0ng Analy0cs Best Prac0ces Review 29

How did we get here ....?

UX    <>    AX

UX    <>    AX AX is hard

So many reports. So little time ...

So many reports. So little actionable information ... 29

Sounding familiar?

Ins0tu0onaliza0on of Usability • Grassroots/Piecemeal effort • Tactical response • Train wreck • Notice of the champion • Organizational commitment • Recognized as skill • Common Method • Tools & Training 29

How do we go there?

Which comes ﬁrst? The data or the ques2ons.

Define the question(s) Ask: Who cares? Pick the data Collect and track Present just enough Present to drive action

Measurable site goals?

Conversion does not  always mean purchase ...

Analy0cs and usability ‐ similari0es ‐ Planning hEp://ins2tute.nih.gov/index hEp://ins2tute.nih.gov/grants hEp://ins2tute.nih.gov/grants/deadlines   Institute home Health Grants Research Health Pg 1 Deadlines Research Pg 1a Health Pg 2 Grants Pg 2 Research Pg 2 Grants Pg 3 Research Pg 3 Grants Pg 4 Research Pg 4 29

Pain points ... URL naming conven0on hEp://SiteHomepage/default.asp hEp://SiteHomepage/Default.asp hEp://SiteHomepage/default.aspx Institute home Health Grants Research Health Pg 1 Grants Pg 1 Research Pg 1a Health Pg 2 Grants Pg 2 Research Pg 2 Grants Pg 3 Research Pg 3 Grants Pg 4 Research Pg 4 29

Some Key Performance Indicators

Visit cycle Engage & Attract Commit Convert Persuade Unengaged Unconverted Unpersuaded (Bounce) 28

 “Typical” Measures

“Typical” Measures Engage & Attract Commit Convert Persuade captured Visits = # site entries 30

“Typical” Measures Engage & Attract Commit Convert Persuade Bounce 31

“Typical” Measures Engage & Attract Commit Convert Persuade captured converted Tradi0onal Conversion =  Converted /  Captured 32

 Some addi0onal measures to consider ....

Addi0onal Measures ‐ Engaged Visits Engage & Attract Commit Convert Persuade engaged Engaged Visits = # site visits that move N* pages into the site 34

Addi0onal Measures ‐ Engagement Index Engage & Attract Commit Convert Persuade engaged Engagement Index = Engaged visits* / Site entries *Engaged visits are site visits that move [1 to N] clicks into the site  35

Addi0onal Measures ‐ Persuaded Visits Engage & Attract Commit Convert Persuade persuaded Persuaded Visits* = # site visits that enter a conversion funnel *Can be computed for individual funnels or summed over funnels 36

Addi0onal Measures ‐ Persuasion Index Engage & Attract Commit Convert Persuade persuaded Persuasion Index* = Commieed visits  / Visits to PageX *This is a page level measure  37

Addi0onal Measures ‐ Engaged conversions Engage & Attract Commit Convert Persuade engaged converted Engaged Conversions = Converted / Engaged visits 38

Addi0onal Measures ‐ Commieed Conversion Engage & Attract Commit Convert Persuade persuaded converted Persuaded Conversion* = Converted / Conversion commitments *This is a page (sequence) measure  39

Types of conversion Engage & Attract Commit Convert Persuade Persuaded Conversion Engaged Conversion Tradi0onal Conversion 40

What are we measuring? Engage & Attract Commit Convert Persuade Persuasion Usability Conversion Engagement 41

How does this reﬂect site performance? Name awareness & SEO Effectiveness 25000 Content match Good labels Meaningful 20000 Content 15000 Usability 10000 5000 0 Visits Engaged Persuaded Converted

Performance ques0ons & measures Ques2on Measure Evaluates Brand  How many visits to the site? Visits Awareness, SEO Engaged Visits,  Right content,  How many visits move past the ﬁrst page? Engagement Index right labels How many visits move into a “Conversion”  Persuaded Visits,  Persuasion sequence? Persuasion Index What propor2on of engaged visitors  Engaged  Persuasion +  convert? Conversions Usability What propor2on of commiEed visitors  Persuaded  Usability convert? Conversions 1st Impression  content +  What propor2on of visits convert? Conversions Persuasion +  Usability

 Analy0cs that drive usability improvements

Contribu0on Index – Across key tasks 1 +1 +1 +1 +1 17

Contribu0on Index – Across key tasks 1 +1 18

Contribu0on Index – Across key tasks 1 +1 +1 +1 +1 x2 +1 +1 +1 +1 +1 Same page 19

Contribu0on Index – Across tasks =1 +1 +1 +1 = 4 1 x2 Same page 20

Contribu0on Index  Which page / pages contribute most to the  • success of the site?  Iden2fy key “final” pages •  Where people off • • Key informa2on pages (Submission dates; Workshop  desc2p2ons; Ac2ve Mechanisms)  Look at common paths to those pages (possibly  • the culminate in drop off)

Behavioral Segmenta0on 24

Behavioral Segmenta0on Grant seeker • Funded work • Mechanism • Deadlines • Funding Strategy • Reviewers • Portfolio contact 24

Behavioral Segmenta0on  Which clusters of pages do people tend to visit? •  Do the clusters match  • up with personas?  Which addi2onal pages  • should people be seeing  to accomplish the  persona tasks?

Behavioral Segmenta0on

Using Analy0cs to Validate User Research Membership benefits Auto Car maintenance Insurance tools & tips Extended Research car warranty Auto Loan

Using Analy0cs to conduct User Research A | B [|C] Tes0ng Which link makes sense to Rudy? Ruth L. Kirschstein Na0onal Research Service Award NRSA Predoctoral Grant

Performance ques0ons & measures Ques0on Measure Evaluates Which pages are most cri2cal to site  Contribu2on Central content success? Seduc2ve  Are visitors seeing the right clusters of  Behavioral  placement of  things? Segmenta2on associated  content Which labels / images engage best? A|B Testing Labeling

Toward integrated user experience measures

Measurable Business Objec0ves Usability  Web Tes0ng Analy0cs Best Prac0ces Review 29

The web‐2‐ci0zen rela0onship is expanding

Speaker Contact Informa0on David Mahaﬀey david.mahaﬀey@humanfactors.com Kath Straub kath@humanfactors.com For more informa/on about HFI Ed Frease Business Director ‐ Government elf@humanfactors.com 800.242.4480

Usability <> Web Metrics ; Advancing analytics through the lessons of usability ...and improving user experience along the way

0 comments

Notes on slide 1

12 Favorites