Human Factors Interna/onal




UX
<>
AX

Advancing
analy0cs
through
the
lessons
of
usability
...and
improving
user
experie...
Analy0cs
is
just
data.
4
NIH
types
know
a
lot
about

   how
to
handle
data.
Observe
Describe
Explain
Predict
What
we’ve
learned

 doing
usability
may
help
analy0cs

   move
faster

Analy0cs
is
also
be
about

   what
can happen.
Analy0cs
is
one
part
of
a
bigger
picture.
Measurable
             Business
Objec0ves



Usability
                          Web
 Tes0ng                           An...
How
did
we
get
here
....?
UX



<>



AX
UX



<>



AX
 AX
is
hard
So
many
reports.
So
little
time
...
So
many
reports.
So
little
actionable
information
...



                                                        29
Sounding
familiar?
Ins0tu0onaliza0on
of
Usability

  • Grassroots/Piecemeal effort
  • Tactical response
  • Train wreck
  • Notice of the ch...
How
do
we
go
there?
Which
comes
first?
The
data
or
the
ques2ons.
Define the question(s)
Ask: Who cares?
Pick the data
Collect and track
Present just enough
Present to drive action
Measurable
site
goals?
Conversion
does
not

always
mean
purchase
...
Analy0cs
and
usability
‐
similari0es
‐
Planning

hEp://ins2tute.nih.gov/index
hEp://ins2tute.nih.gov/grants
hEp://ins2tute...
Pain
points
...

URL
naming
conven0on
hEp://SiteHomepage/default.asp
hEp://SiteHomepage/Default.asp
hEp://SiteHomepage/def...
Some
Key
Performance
Indicators
Visit
cycle




                    Engage &
     Attract                        Commit   Convert
                    Pers...

“Typical”
Measures
“Typical”
Measures




                  Engage &
       Attract               Commit   Convert
                  Persuade...
“Typical”
Measures




              Engage &
    Attract              Commit   Convert
              Persuade




    Bou...
“Typical”
Measures




                 Engage &
       Attract                Commit      Convert
                 Persua...

Some
addi0onal
measures
to
consider
....
Addi0onal
Measures
‐
Engaged
Visits




                 Engage &
     Attract                     Commit      Convert
   ...
Addi0onal
Measures
‐
Engagement
Index




                            Engage &
       Attract                             ...
Addi0onal
Measures
‐
Persuaded
Visits




                       Engage &
       Attract                              Comm...
Addi0onal
Measures
‐
Persuasion
Index




                        Engage &
       Attract                            Commi...
Addi0onal
Measures
‐
Engaged
conversions




                 Engage &
     Attract                  Commit      Convert
 ...
Addi0onal
Measures
‐
Commieed
Conversion




                       Engage &
       Attract                               ...
Types
of
conversion




                Engage &
    Attract                    Commit          Convert
                Pe...
What
are
we
measuring?




              Engage &
    Attract                   Commit       Convert
              Persuad...
How
does
this
reflect
site
performance?

          Name awareness &
          SEO Effectiveness
  25000
                   ...
Performance
ques0ons
&
measures
Ques2on                                    Measure           Evaluates
                   ...

Analy0cs
that
drive
usability
improvements
Contribu0on
Index
–
Across
key
tasks



                      1   +1   +1   +1   +1




                                  ...
Contribu0on
Index
–
Across
key
tasks



                      1   +1




                                       18
Contribu0on
Index
–
Across
key
tasks



                     1   +1   +1           +1
                                   +...
Contribu0on
Index
–
Across
tasks



                                         =1




                                      ...
Contribu0on
Index

  
Which
page
/
pages
contribute
most
to
the

 •

 success
of
the
site?
     
Iden2fy
key
“final”
pages
...
Behavioral
Segmenta0on




                         24
Behavioral
Segmenta0on




    Grant seeker
    • Funded work
    • Mechanism
    • Deadlines
    • Funding Strategy
    •...
Behavioral
Segmenta0on

     
Which
clusters
of
pages
do
people
tend
to
visit?
 •


     
Do
the
clusters
match

 •

     ...
Behavioral
Segmenta0on
Using
Analy0cs
to
Validate
User
Research


                    Membership
                     benefits




              ...
Using
Analy0cs
to
conduct
User
Research




            A
|
B
[|C]
Tes0ng
 Which
link
makes
sense
to
Rudy?
 Ruth
L.
Kirsch...
Performance
ques0ons
&
measures

 Ques0on                                      Measure         Evaluates

 Which
pages
are...
Toward
integrated
user
experience
measures
Measurable
             Business
Objec0ves



Usability
                          Web
 Tes0ng                           An...
The
web‐2‐ci0zen
rela0onship
is
expanding
Speaker
Contact
Informa0on

David
Mahaffey
david.mahaffey@humanfactors.com


Kath
Straub
kath@humanfactors.com


For more in...
Usability <> Web Metrics ; Advancing analytics through the lessons of usability ...and improving user experience along the...
Usability <> Web Metrics ; Advancing analytics through the lessons of usability ...and improving user experience along the...
Usability <> Web Metrics ; Advancing analytics through the lessons of usability ...and improving user experience along the...
Upcoming SlideShare
Loading in …5
×

Usability <> Web Metrics ; Advancing analytics through the lessons of usability ...and improving user experience along the way

3,874 views

Published on

Traditional Web analytics are designed to capture how easily and how often users “convert” by buying stuff or taking desired actions on Web sites. While Web analytics and other data-oriented measures work well for commercial sites, they often fail to capture the user-friendliness or effectiveness of government and other information-oriented sites.

Within this talk, we will:
* Outline the complementary objectives of usability and Web analytics measures
* Review usability testing methods designed to measure users’ comprehension
* Discuss how the business concept of “conversion” can be applied to sites that provide government information
* Discuss the benefits of integrating the Web analytics and usability data streams
* Describe performance measures that map user experience metrics or measures to NIH Web site business goals
Throughout the talk, they will explore the characteristics of the different data streams. They will also explain how–when intertwined—the data streams may provide even more clearly actionable guidance for Web site improvement.

Published in: Technology, Design
1 Comment
22 Likes
Statistics
Notes
  • شات http://www.aden9.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
3,874
On SlideShare
0
From Embeds
0
Number of Embeds
404
Actions
Shares
0
Downloads
272
Comments
1
Likes
22
Embeds 0
No embeds

No notes for slide

Usability <> Web Metrics ; Advancing analytics through the lessons of usability ...and improving user experience along the way

  1. 1. Human Factors Interna/onal UX
<>
AX
 Advancing
analy0cs
through
the
lessons
of
usability ...and
improving
user
experience
along
the
way 5

December
2008 David
Mahaffey Director
‐
Human
Factors
Interna2onal Kath
Straub,
PhD Chief
Scien2st
‐
Human
Factors
Interna2onal
  2. 2. Analy0cs
is
just
data.
  3. 3. 4
  4. 4. NIH
types
know
a
lot
about
 how
to
handle
data.
  5. 5. Observe Describe Explain Predict
  6. 6. What
we’ve
learned
 doing
usability may
help
analy0cs
 move
faster

  7. 7. Analy0cs
is
also
be
about
 what
can happen.
  8. 8. Analy0cs
is
one
part of
a
bigger
picture.
  9. 9. Measurable Business
Objec0ves Usability
 Web Tes0ng Analy0cs Best
Prac0ces Review 29
  10. 10. How
did
we
get
here
....?
  11. 11. UX



<>



AX
  12. 12. UX



<>



AX AX
is
hard
  13. 13. So
many
reports.
So
little
time
...
  14. 14. So
many
reports.
So
little
actionable
information
... 29
  15. 15. Sounding
familiar?
  16. 16. Ins0tu0onaliza0on
of
Usability • Grassroots/Piecemeal effort • Tactical response • Train wreck • Notice of the champion • Organizational commitment • Recognized as skill • Common Method • Tools & Training 29
  17. 17. How
do
we
go
there?
  18. 18. Which
comes
first? The
data
or
the
ques2ons.
  19. 19. Define the question(s) Ask: Who cares? Pick the data Collect and track Present just enough Present to drive action
  20. 20. Measurable
site
goals?
  21. 21. Conversion
does
not
 always
mean
purchase
...
  22. 22. Analy0cs
and
usability
‐
similari0es
‐
Planning hEp://ins2tute.nih.gov/index hEp://ins2tute.nih.gov/grants hEp://ins2tute.nih.gov/grants/deadlines 
 Institute home Health Grants Research Health Pg 1 Deadlines Research Pg 1a Health Pg 2 Grants Pg 2 Research Pg 2 Grants Pg 3 Research Pg 3 Grants Pg 4 Research Pg 4 29
  23. 23. Pain
points
... URL
naming
conven0on hEp://SiteHomepage/default.asp hEp://SiteHomepage/Default.asp hEp://SiteHomepage/default.aspx Institute home Health Grants Research Health Pg 1 Grants Pg 1 Research Pg 1a Health Pg 2 Grants Pg 2 Research Pg 2 Grants Pg 3 Research Pg 3 Grants Pg 4 Research Pg 4 29
  24. 24. Some
Key
Performance
Indicators
  25. 25. Visit
cycle Engage & Attract Commit Convert Persuade Unengaged Unconverted Unpersuaded (Bounce) 28
  26. 26. 
“Typical”
Measures
  27. 27. “Typical”
Measures Engage & Attract Commit Convert Persuade captured Visits
=
#
site
entries 30
  28. 28. “Typical”
Measures Engage & Attract Commit Convert Persuade Bounce 31
  29. 29. “Typical”
Measures Engage & Attract Commit Convert Persuade captured converted Tradi0onal
Conversion
=

Converted
/

Captured 32
  30. 30. 
Some
addi0onal
measures
to
consider
....
  31. 31. Addi0onal
Measures
‐
Engaged
Visits Engage & Attract Commit Convert Persuade engaged Engaged
Visits
=
#
site
visits
that
move
N*
pages
into
the
site 34
  32. 32. Addi0onal
Measures
‐
Engagement
Index Engage & Attract Commit Convert Persuade engaged Engagement
Index
=
Engaged
visits*
/
Site
entries *Engaged
visits
are
site
visits
that
move
[1
to
N]
clicks
into
the
site
 35
  33. 33. Addi0onal
Measures
‐
Persuaded
Visits Engage & Attract Commit Convert Persuade persuaded Persuaded
Visits*
=
#
site
visits
that
enter
a
conversion
funnel *Can
be
computed
for
individual
funnels
or
summed
over
funnels 36
  34. 34. Addi0onal
Measures
‐
Persuasion
Index Engage & Attract Commit Convert Persuade persuaded Persuasion
Index*
=
Commieed
visits

/
Visits
to
PageX *This
is
a
page
level
measure
 37
  35. 35. Addi0onal
Measures
‐
Engaged
conversions Engage & Attract Commit Convert Persuade engaged converted Engaged
Conversions
=
Converted
/
Engaged
visits 38
  36. 36. Addi0onal
Measures
‐
Commieed
Conversion Engage & Attract Commit Convert Persuade persuaded converted Persuaded
Conversion*
=
Converted
/
Conversion
commitments *This
is
a
page
(sequence)
measure
 39
  37. 37. Types
of
conversion Engage & Attract Commit Convert Persuade Persuaded
Conversion Engaged
Conversion Tradi0onal
Conversion 40
  38. 38. What
are
we
measuring? Engage & Attract Commit Convert Persuade Persuasion Usability Conversion Engagement 41
  39. 39. How
does
this
reflect
site
performance? Name awareness & SEO Effectiveness 25000 Content match Good labels Meaningful 20000 Content 15000 Usability 10000 5000 0 Visits Engaged Persuaded Converted
  40. 40. Performance
ques0ons
&
measures Ques2on Measure Evaluates Brand
 How
many
visits
to
the
site? Visits Awareness,
SEO Engaged
Visits,
 Right
content,
 How
many
visits
move
past
the
first
page? Engagement
Index right
labels How
many
visits
move
into
a
“Conversion”
 Persuaded
Visits,
 Persuasion sequence? Persuasion
Index What
propor2on
of
engaged
visitors
 Engaged
 Persuasion
+
 convert? Conversions Usability What
propor2on
of
commiEed
visitors
 Persuaded
 Usability convert? Conversions 1st
Impression
 content
+
 What
propor2on
of
visits
convert? Conversions Persuasion
+
 Usability
  41. 41. 
Analy0cs
that
drive
usability
improvements
  42. 42. Contribu0on
Index
–
Across
key
tasks 1 +1 +1 +1 +1 17
  43. 43. Contribu0on
Index
–
Across
key
tasks 1 +1 18
  44. 44. Contribu0on
Index
–
Across
key
tasks 1 +1 +1 +1 +1 x2 +1 +1 +1 +1 +1 Same page 19
  45. 45. Contribu0on
Index
–
Across
tasks =1 +1 +1 +1 = 4 1 x2 Same page 20
  46. 46. Contribu0on
Index 
Which
page
/
pages
contribute
most
to
the
 • success
of
the
site? 
Iden2fy
key
“final”
pages • 
Where
people
off • •
Key
informa2on
pages
(Submission
dates;
Workshop
 desc2p2ons;
Ac2ve
Mechanisms) 
Look
at
common
paths
to
those
pages
(possibly
 • the
culminate
in
drop
off)
  47. 47. Behavioral
Segmenta0on 24
  48. 48. Behavioral
Segmenta0on Grant seeker • Funded work • Mechanism • Deadlines • Funding Strategy • Reviewers • Portfolio contact 24
  49. 49. Behavioral
Segmenta0on 
Which
clusters
of
pages
do
people
tend
to
visit? • 
Do
the
clusters
match
 • up
with
personas? 
Which
addi2onal
pages
 • should
people
be
seeing
 to
accomplish
the
 persona
tasks?
  50. 50. Behavioral
Segmenta0on
  51. 51. Using
Analy0cs
to
Validate
User
Research Membership benefits Auto Car maintenance Insurance tools & tips Extended Research car warranty Auto Loan
  52. 52. Using
Analy0cs
to
conduct
User
Research A
|
B
[|C]
Tes0ng Which
link
makes
sense
to
Rudy? Ruth
L.
Kirschstein
Na0onal
Research
Service
Award NRSA Predoctoral
Grant
  53. 53. Performance
ques0ons
&
measures Ques0on Measure Evaluates Which
pages
are
most
cri2cal
to
site
 Contribu2on Central
content success? Seduc2ve
 Are
visitors
seeing
the
right
clusters
of
 Behavioral
 placement
of
 things? Segmenta2on associated
 content Which
labels
/
images
engage
best? A|B Testing Labeling
  54. 54. Toward
integrated
user
experience
measures
  55. 55. Measurable Business
Objec0ves Usability
 Web Tes0ng Analy0cs Best
Prac0ces Review 29
  56. 56. The
web‐2‐ci0zen
rela0onship
is
expanding
  57. 57. Speaker
Contact
Informa0on David
Mahaffey david.mahaffey@humanfactors.com Kath
Straub kath@humanfactors.com For more informa/on about HFI Ed
Frease Business
Director
‐
Government elf@humanfactors.com 800.242.4480

×