Your SlideShare is downloading. ×
0
Marrying Web Analytics
and User Experience
Louis Rosenfeld • 5 August 2009
Delve NYC • Brooklyn
                    1
Web Analytics?

    User Experience?




2
Code “DELVE” for 25% off
 at rosenfeldmedia.com
           3
My recent
struggle



            4
CONTRASTING WEB ANALYTICS AND USER EXPERIENCE
                       5
Who we are
       How we do our work
       What data we use
       How we use that data

CONTRASTING WEB ANALYTICS AND US...
WHO WE ARE
ARE THE STEREOTYPES TRUE?
                            6
VIVE LA DIFFÉRENCE! (FROM MARKO HURST)
                               7
!"#$%&"'()*+),%(-).(%("-&/)0(1/*$%)
        Behavioral                                                   /       Eyetracki...
!"#$%&"'()*+),%(-).(%("-&/)0(1/*$%)
        Behavioral                                                   /       Eyetracki...
HOW WEB ANALYTICS PEOPLE SEE THEIR WORK
(FROM AVINASH KAUSHIK)	
                       9
HOW WEB ANALYTICS PEOPLE SEE THEIR WORK
(FROM AVINASH KAUSHIK)	
                       9
The data that
drives our decisions




              10
The data that
drives our decisions
     Web Analytics                User Experience

       behavioral                   ...
The data that
drives our decisions
     Web Analytics                User Experience

       behavioral                   ...
The data that
drives our decisions
     Web Analytics                User Experience

       behavioral                   ...
The data that
drives our decisions
     Web Analytics                User Experience

       behavioral                   ...
The data that
drives our decisions
     Web Analytics                User Experience

       behavioral                   ...
The data that
drives our decisions
     Web Analytics                User Experience

       behavioral                   ...
Not much use to know
what is happening if you
don’t know why




             11
Not much use to know
what is happening if you
don’t know why

Hard to know why things
are happening if you don’t
know what...
The ways we analyze
our data



           12
The ways we analyze
our data



           12
The ways we analyze
our data



           12
XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad...
XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad...
XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad...
XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad...
Analyzing data
the UX way:
play with the data,
look for patterns, trends,
and outliers
Analyzing data
the UX way:
play with the data,
look for patterns, trends,
and outliers

So what’s being measured?
XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad...
XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad...
Before data analysis:
why are we here?
★ Commerce
★ Lead Generation
★ Content/Media
★ Support/Self-Service

              ...
Before data analysis:
why are we here?
★ Commerce
★ Lead Generation
★ Content/Media
★ Support/Self-Service
Data supports m...
Analyzing data
the WA way:
start with metrics,
benchmark and
measure performance
Analyzing data
the WA way:
start with metrics,
benchmark and
measure performance
But you can’t measure
what you don’t know
WA: Top-down analysis
UX: Bottom-up analysis



           19
what

WA: Top-down analysis
UX: Bottom-up analysis



           19
what

WA: Top-down analysis
UX: Bottom-up analysis

                    why
           19
INTEGRATING WEB ANALYTICS AND USER EXPERIENCE
                       20
Integrating methodologies:
What, then why



            21
Common queries
can drive task analysis




               22
Common queries
can drive task analysis
                      “Can you find a map of
                      the campus?”

   ...
Query data
can augment
personas




              23
Query data
can augment
personas



   “What Steven Searches”
   added to existing persona
   (from Adaptive Path)

       ...
Looking ahead
★ How do we improve other
qualitative methods with data?
★ How do qualitative data
impact quantitative analy...
Methodology takeaways:
★ Qualitative research is
expensive
★ Start with quantitative
research to identify where/when
to us...
Changing how we analyze:
Moving away from
the middle

           26
27
28
What’s in
the middle?




              28
What’s in
the middle?

Your analytics app’s
canned reports

              28
Netflix moved away
from the middle




            29
Netflix moved away
from the middle




            29
Netflix moved away
from the middle




            29
Netflix moved away
from the middle




            29
Netflix moved away
from the middle




            29
Analysis takeaways
★ Canned reports are only a
starting point
★ Move up, move down
★ Be prepared to “roll your own”
★ Dema...
Changing our thinking:
Getting comfortable with
the other

            31
UX people need to get
comfortable with
measuring the
unmeasurable
            32
Can you measure
your content’s
quality?
Systems can help
us objectify the
subjective
                   33
Subjective
                        evaluations...




Can you measure
your content’s
quality?
Systems can help
us objectif...
Subjective
                        evaluations...


                                   ...lead to
Can you measure         ...
UX people need to get
comfortable with numbers
(but just a little)

           34
This is not statistics




               35
This is not statistics
This is not difficult




               35
This is not statistics
This is not difficult
This is very useful




               35
This is not statistics
This is not difficult
This is very useful
(and this is in MS Excel)




              35
WA people need to get
comfortable with stories

            36
WA people need to
understand the value of
intuition and mistakes

            38
C# '&DE#F
<=>>?@A=B

!""#$%&'()*+*(,%+-'.()/-0%'(12*3(4+'(5"6()%'-#7%/ !
!"#$%&'#()*+,-.#/0/1#23#*'#456#3,5#17#1778.#17.19...
C# '&DE#F
<=>>?@A=B

!""#$%&'()*+*(,%+-'.()/-0%'(12*3(4+'(5"6()%'-#7%/ !
!"#$%&'#()*+,-.#/0/1#23#*'#456#3,5#17#1778.#17.19...
C# '&DE#F
<=>>?@A=B

!""#$%&'()*+*(,%+-'.()/-0%'(12*3(4+'(5"6()%'-#7%/ !
!"#$%&'#()*+,-.#/0/1#23#*'#456#3,5#17#1778.#17.19...
Tom Chi:
“Think of your designer as a guide in this
multi-variate optimization process. A good
designer has been all over ...
UX and WA people need
to talk together about
project goals

            41
42
Vanguard and the
quantification of search
                            Target    Oct 3   Oct 10   Oct 16
 Mean distance from...
Changing thinking
takeaways
★ Most things can be quantified
★ Stories and emotions can
make stronger cases than data,
and f...
Challenges: how do we...
★ Bridge cultural gaps?
★ Get different groups to speak
the same language?
★ Design and manage in...
Do we have a choice?
          An individual often uses
          only half their brain

          Effective teams and
   ...
Some day my book
will come...
Search Analytics for Your Site:
Conversations with Your Customers

Louis Rosenfeld & Marko H...
Until then...
Louis Rosenfeld
457 Third Street, #4R
Brooklyn, NY 11215 USA

lou@louisrosenfeld.com
www.louisrosenfeld.com
...
Marrying Web Analytics and User Experience
Marrying Web Analytics and User Experience
Marrying Web Analytics and User Experience
Upcoming SlideShare
Loading in...5
×

Marrying Web Analytics and User Experience

40,122

Published on

Keynote at:
• JBoye Conference; Philadelphia, PA, USA (May 7, 2009)
• IA Konferenz; Hamburg, Deutschland (May 16, 2009)
• Delve NYC; Brooklyn, NY, USA (August 5, 2009)

Published in: Technology, Design
21 Comments
266 Likes
Statistics
Notes
No Downloads
Views
Total Views
40,122
On Slideshare
0
From Embeds
0
Number of Embeds
31
Actions
Shares
0
Downloads
2,338
Comments
21
Likes
266
Embeds 0
No embeds

No notes for slide
  • http://www.nif.or.jp/eng/graph/M7.gif
    http://interactions.acm.org/i/XV/wine.jpg
  • Camille Jordan:
    http://myoops.fgu.edu.tw/twocw/mit/NR/rdonlyres/Mathematics/18-700Fall-2005/4AC2EE51-AA81-45EA-AB73-1935A7F3BAFC/0/chp_jordan2.jpg

    Arthur Rimbaud:
    http://www.stevesilberman.com/celestial/rimbaud/rimbaud.jpg

    Those dreaming eyes: are the looking upon the same thing?
  • Avinash Kaushik: &amp;#x201C;Trinity: A Mindset &amp; Strategic Approach&amp;#x201D; (http://www.kaushik.net/avinash/2006/08/trinity-a-mindset-strategic-approach.html)
  • Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  • Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  • Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  • Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  • Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  • Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  • Bottom line and &amp;#x201C;top line&amp;#x201D;
  • This is how I would do it
  • This is how I&amp;#x2019;d do it
  • This is how I&amp;#x2019;d do it
  • This is how my co-author would do it
  • Start with KPI, then add data
  • Feedback loop
  • Start with KPI, then add data
  • The reports are often as far as we go
    But they&amp;#x2019;re often useless
    &amp;#x2022; No deep, custom analysis (top-down)
    &amp;#x2022; No exploratory data analysis (bottom-up)
  • The reports are often as far as we go
    But they&amp;#x2019;re often useless
    &amp;#x2022; No deep, custom analysis (top-down)
    &amp;#x2022; No exploratory data analysis (bottom-up)
  • &amp;#x201C;The center can not hold!&amp;#x201D;

    You&amp;#x2019;ll notice this isn&amp;#x2019;t a canned report

    This all means putting pressure on commercial analytics apps to change
  • &amp;#x201C;The center can not hold!&amp;#x201D;

    You&amp;#x2019;ll notice this isn&amp;#x2019;t a canned report

    This all means putting pressure on commercial analytics apps to change
  • &amp;#x201C;The center can not hold!&amp;#x201D;

    You&amp;#x2019;ll notice this isn&amp;#x2019;t a canned report

    This all means putting pressure on commercial analytics apps to change
  • &amp;#x201C;The center can not hold!&amp;#x201D;

    You&amp;#x2019;ll notice this isn&amp;#x2019;t a canned report

    This all means putting pressure on commercial analytics apps to change
  • Start with KPI, then add data
  • you can do this, regardless of how you feel about data

    note that it&amp;#x2019;s in Excel
  • you can do this, regardless of how you feel about data

    note that it&amp;#x2019;s in Excel
  • you can do this, regardless of how you feel about data

    note that it&amp;#x2019;s in Excel
  • Yes, data can tell stories

    And sometimes stories make a better case than reports
  • Actually, both sides (Bowman&amp;#x2019;s and Google&amp;#x2019;s) are valid
    But while it won&amp;#x2019;t always be possible to combine WA and UX (in some orgs, one perspective is far dominant--e.g., engineering at Google), you&amp;#x2019;ve got to come halfway

    But... weren&amp;#x2019;t Page and Brin designers of a sort when they started out?
  • Actually, both sides (Bowman&amp;#x2019;s and Google&amp;#x2019;s) are valid
    But while it won&amp;#x2019;t always be possible to combine WA and UX (in some orgs, one perspective is far dominant--e.g., engineering at Google), you&amp;#x2019;ve got to come halfway

    But... weren&amp;#x2019;t Page and Brin designers of a sort when they started out?
  • Transcript of "Marrying Web Analytics and User Experience"

    1. 1. Marrying Web Analytics and User Experience Louis Rosenfeld • 5 August 2009 Delve NYC • Brooklyn 1
    2. 2. Web Analytics? User Experience? 2
    3. 3. Code “DELVE” for 25% off at rosenfeldmedia.com 3
    4. 4. My recent struggle 4
    5. 5. CONTRASTING WEB ANALYTICS AND USER EXPERIENCE 5
    6. 6. Who we are How we do our work What data we use How we use that data CONTRASTING WEB ANALYTICS AND USER EXPERIENCE 5
    7. 7. WHO WE ARE ARE THE STEREOTYPES TRUE? 6
    8. 8. VIVE LA DIFFÉRENCE! (FROM MARKO HURST) 7
    9. 9. !"#$%&"'()*+),%(-).(%("-&/)0(1/*$%) Behavioral / Eyetracking Data Mining/Analysis A/B (Live) Testing Usability Benchmarking (in lab) / Data Source Usability Lab Studies Online User Experience Assessments (“Vividence-like” studies) Ethnographic Field Studies mix Diary/Camera Study Message Board Mining Participatory Design Customer feedback via email Focus Groups Desirability studies Intercept Surveys Attitudinal Phone Interviews Cardsorting Email Surveys mix Qualitative (direct) Approach Quantitative (indirect) Key for Context of Product Use during data collection Natural use of product De-contextualized / not using product © 2008 Christian Rohrer Scripted (often lab-based) use of product Combination / hybrid 20 HOW USER EXPERIENCE PEOPLE SEE THEIR WORK (FROM CHRISTIAN ROHRER) 8
    10. 10. !"#$%&"'()*+),%(-).(%("-&/)0(1/*$%) Behavioral / Eyetracking Data Mining/Analysis A/B (Live) Testing Usability Benchmarking (in lab) / Data Source Usability Lab Studies Online User Experience Assessments (“Vividence-like” studies) Ethnographic Field Studies mix Diary/Camera Study Message Board Mining Participatory Design Customer feedback via email Focus Groups Desirability studies Intercept Surveys Attitudinal Phone Interviews Cardsorting Email Surveys mix Qualitative (direct) Approach Quantitative (indirect) Key for Context of Product Use during data collection Natural use of product De-contextualized / not using product © 2008 Christian Rohrer Scripted (often lab-based) use of product Combination / hybrid 20 HOW USER EXPERIENCE PEOPLE SEE THEIR WORK (FROM CHRISTIAN ROHRER) 8
    11. 11. HOW WEB ANALYTICS PEOPLE SEE THEIR WORK (FROM AVINASH KAUSHIK) 9
    12. 12. HOW WEB ANALYTICS PEOPLE SEE THEIR WORK (FROM AVINASH KAUSHIK) 9
    13. 13. The data that drives our decisions 10
    14. 14. The data that drives our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 10
    15. 15. The data that drives our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 10
    16. 16. The data that drives our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 10
    17. 17. The data that drives our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 10
    18. 18. The data that drives our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 10
    19. 19. The data that drives our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 10
    20. 20. Not much use to know what is happening if you don’t know why 11
    21. 21. Not much use to know what is happening if you don’t know why Hard to know why things are happening if you don’t know what is happening 11
    22. 22. The ways we analyze our data 12
    23. 23. The ways we analyze our data 12
    24. 24. The ways we analyze our data 12
    25. 25. XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17 13
    26. 26. XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17 14
    27. 27. XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17 Q “What were the most common searches?” 14
    28. 28. XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17 Q “What were the most common searches?” 14
    29. 29. Analyzing data the UX way: play with the data, look for patterns, trends, and outliers
    30. 30. Analyzing data the UX way: play with the data, look for patterns, trends, and outliers So what’s being measured?
    31. 31. XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17 16
    32. 32. XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17 Q “Are we converting license plate renewals?” 16
    33. 33. Before data analysis: why are we here? ★ Commerce ★ Lead Generation ★ Content/Media ★ Support/Self-Service 17
    34. 34. Before data analysis: why are we here? ★ Commerce ★ Lead Generation ★ Content/Media ★ Support/Self-Service Data supports metrics 17
    35. 35. Analyzing data the WA way: start with metrics, benchmark and measure performance
    36. 36. Analyzing data the WA way: start with metrics, benchmark and measure performance But you can’t measure what you don’t know
    37. 37. WA: Top-down analysis UX: Bottom-up analysis 19
    38. 38. what WA: Top-down analysis UX: Bottom-up analysis 19
    39. 39. what WA: Top-down analysis UX: Bottom-up analysis why 19
    40. 40. INTEGRATING WEB ANALYTICS AND USER EXPERIENCE 20
    41. 41. Integrating methodologies: What, then why 21
    42. 42. Common queries can drive task analysis 22
    43. 43. Common queries can drive task analysis “Can you find a map of the campus?” “What study abroad options are available to students?” “When is the last home football game of the season?” 22
    44. 44. Query data can augment personas 23
    45. 45. Query data can augment personas “What Steven Searches” added to existing persona (from Adaptive Path) 23
    46. 46. Looking ahead ★ How do we improve other qualitative methods with data? ★ How do qualitative data impact quantitative analyses? 24
    47. 47. Methodology takeaways: ★ Qualitative research is expensive ★ Start with quantitative research to identify where/when to use qualitative methods 25
    48. 48. Changing how we analyze: Moving away from the middle 26
    49. 49. 27
    50. 50. 28
    51. 51. What’s in the middle? 28
    52. 52. What’s in the middle? Your analytics app’s canned reports 28
    53. 53. Netflix moved away from the middle 29
    54. 54. Netflix moved away from the middle 29
    55. 55. Netflix moved away from the middle 29
    56. 56. Netflix moved away from the middle 29
    57. 57. Netflix moved away from the middle 29
    58. 58. Analysis takeaways ★ Canned reports are only a starting point ★ Move up, move down ★ Be prepared to “roll your own” ★ Demand better ad hoc reporting from analytics apps 30
    59. 59. Changing our thinking: Getting comfortable with the other 31
    60. 60. UX people need to get comfortable with measuring the unmeasurable 32
    61. 61. Can you measure your content’s quality? Systems can help us objectify the subjective 33
    62. 62. Subjective evaluations... Can you measure your content’s quality? Systems can help us objectify the subjective 33
    63. 63. Subjective evaluations... ...lead to Can you measure objective decisions your content’s quality? Systems can help us objectify the subjective 33
    64. 64. UX people need to get comfortable with numbers (but just a little) 34
    65. 65. This is not statistics 35
    66. 66. This is not statistics This is not difficult 35
    67. 67. This is not statistics This is not difficult This is very useful 35
    68. 68. This is not statistics This is not difficult This is very useful (and this is in MS Excel) 35
    69. 69. WA people need to get comfortable with stories 36
    70. 70. WA people need to understand the value of intuition and mistakes 38
    71. 71. C# '&DE#F <=>>?@A=B !""#$%&'()*+*(,%+-'.()/-0%'(12*3(4+'(5"6()%'-#7%/ ! !"#$%&'#()*+,-.#/0/1#23#*'#456#3,5#17#1778.#17.19:#;6&%-# !"#$%#&%'(&)%*#+,")#$%%-.#/(01#2($3,4.#5((1+%6"#)('#*%"714%&.#8,"#9(4:7&3%*#)8,)#8%6"#+%,;741# <$%#8%,&#)(#=$7))%&>?#2($3,46"#&%,"(4"#:(&#@07))741#,&%#:,"974,)741#A#,4*#)8%B#"8($#$8B# 5((1+%6"#+("741#7)"#9((+? 2($3,4#C(74%*#5((1+%#)8&%%#B%,&"#,1(#A#)((#+,)%.#8%#4($#",B"?#=8%#9(3',4B6"#%4174%%&"D:7&")# 90+)0&%#$,"#:7&3+B#74#'+,9%.#3%,4741#%;%&B#*%97"7(4#8,*#)(#E%#'&(;%4#)8&(018#%F8,0")7;%#)%")741.# &,)8%&#)8,4#,#&%+7,49%#(4#,#9+%,&#;7"7(4#(:#5((1+%6"#*%"714?#!4*#74#,#E,9-8,4*%*#"+,3#,)#5((1+%#GH# I,&7"",#I,B%&.#)8%#8%,*#(:#J0"%&#%F'%&7%49%.J#8%#4()%"#)8,)#)('#3,4,1%3%4)#74#98,&1%#(:#*%"714# 8,*#4()#E,9-1&(04*#74#)8%#:7%+*K L8%4#,#9(3',4B#7"#:7++%*#$7)8#%4174%%&".#7)#)0&4"#)(#%4174%%&741#)(#"(+;%#'&(E+%3"?#M%*09%#%,98#*%97"7(4#)(#,#"73'+%#+(179# '&(E+%3?#M%3(;%#,++#"0EC%9)7;7)B#,4*#C0")#+((-#,)#)8%#*,),?#/,),#74#B(0&#:,;(&N#O-.#+,0498#7)?#/,),#"8($"#4%1,)7;%#%::%9)"N#2,9-#)(# )8%#*&,$741#E(,&*?#!4*#)8,)#*,),#%;%4)0,++B#E%9(3%"#,#9&0)98#:(&#%;%&B#*%97"7(4.#',&,+BP741#)8%#9(3',4B#,4*#'&%;%4)741#7)# :&(3#3,-741#,4B#*,&741#*%"714#*%97"7(4"? # # # Q%".#7)6"#)&0%#)8,)#,#)%,3#,)#5((1+%#9(0+*46)#*%97*%#E%)$%%4#)$(#E+0%".#"(#)8%B6&%#)%")741#RS#"8,*%"#E%)$%%4#%,98#E+0%#)(#"%%#$8798# (4%#'%&:(&3"#E%))%&?#T#8,*#,#&%9%4)#*%E,)%#(;%&#$8%)8%&#,#E(&*%&#"8(0+*#E%#U.#R#(&#V#'7F%+"#$7*%.#,4*#$,"#,"-%*#)(#'&(;%#3B# 9,"%?#T#9,46)#('%&,)%#74#,4#%4;7&(43%4)#+7-%#)8,)?#T6;%#1&($4#)7&%*#(:#*%E,)741#"098#3747"90+%#*%"714#*%97"7(4"?#=8%&%#,&%#3(&%# %F97)741#*%"714#'&(E+%3"#74#)87"#$(&+*#)(#),9-+%? WF97)741#*%"714#'&(E+%3".#+7-%#)8("%#,)#=$7))%&N#!#"(0&9%#)%++"#0"#)8,)6"#$8%&%#8%6"#1(741.#E0)#2($3,4#8,"46)#9(4:7&3%*#)8,)# B%)?#<X%#'&(37"%"#)(#*7"9+("%#87"#4%$#%3'+(B%&#74#,#:(++($0'#E+(1#'(")?>
    72. 72. C# '&DE#F <=>>?@A=B !""#$%&'()*+*(,%+-'.()/-0%'(12*3(4+'(5"6()%'-#7%/ ! !"#$%&'#()*+,-.#/0/1#23#*'#456#3,5#17#1778.#17.19:#;6&%-# !"#$%#&%'(&)%*#+,")#$%%-.#/(01#2($3,4.#5((1+%6"#)('#*%"714%&.#8,"#9(4:7&3%*#)8,)#8%6"#+%,;741# <$%#8%,&#)(#=$7))%&>?#2($3,46"#&%,"(4"#:(&#@07))741#,&%#:,"974,)741#A#,4*#)8%B#"8($#$8B# 5((1+%6"#+("741#7)"#9((+? 2($3,4#C(74%*#5((1+%#)8&%%#B%,&"#,1(#A#)((#+,)%.#8%#4($#",B"?#=8%#9(3',4B6"#%4174%%&"D:7&")# 90+)0&%#$,"#:7&3+B#74#'+,9%.#3%,4741#%;%&B#*%97"7(4#8,*#)(#E%#'&(;%4#)8&(018#%F8,0")7;%#)%")741.# &,)8%&#)8,4#,#&%+7,49%#(4#,#9+%,&#;7"7(4#(:#5((1+%6"#*%"714?#!4*#74#,#E,9-8,4*%*#"+,3#,)#5((1+%#GH# I,&7"",#I,B%&.#)8%#8%,*#(:#J0"%&#%F'%&7%49%.J#8%#4()%"#)8,)#)('#3,4,1%3%4)#74#98,&1%#(:#*%"714# 8,*#4()#E,9-1&(04*#74#)8%#:7%+*K L8%4#,#9(3',4B#7"#:7++%*#$7)8#%4174%%&".#7)#)0&4"#)(#%4174%%&741#)(#"(+;%#'&(E+%3"?#M%*09%#%,98#*%97"7(4#)(#,#"73'+%#+(179# '&(E+%3?#M%3(;%#,++#"0EC%9)7;7)B#,4*#C0")#+((-#,)#)8%#*,),?#/,),#74#B(0&#:,;(&N#O-.#+,0498#7)?#/,),#"8($"#4%1,)7;%#%::%9)"N#2,9-#)(# )8%#*&,$741#E(,&*?#!4*#)8,)#*,),#%;%4)0,++B#E%9(3%"#,#9&0)98#:(&#%;%&B#*%97"7(4.#',&,+BP741#)8%#9(3',4B#,4*#'&%;%4)741#7)# :&(3#3,-741#,4B#*,&741#*%"714#*%97"7(4"? # # # Q%".#7)6"#)&0%#)8,)#,#)%,3#,)#5((1+%#9(0+*46)#*%97*%#E%)$%%4#)$(#E+0%".#"(#)8%B6&%#)%")741#RS#"8,*%"#E%)$%%4#%,98#E+0%#)(#"%%#$8798# (4%#'%&:(&3"#E%))%&?#T#8,*#,#&%9%4)#*%E,)%#(;%&#$8%)8%&#,#E(&*%&#"8(0+*#E%#U.#R#(&#V#'7F%+"#$7*%.#,4*#$,"#,"-%*#)(#'&(;%#3B# 9,"%?#T#9,46)#('%&,)%#74#,4#%4;7&(43%4)#+7-%#)8,)?#T6;%#1&($4#)7&%*#(:#*%E,)741#"098#3747"90+%#*%"714#*%97"7(4"?#=8%&%#,&%#3(&%# %F97)741#*%"714#'&(E+%3"#74#)87"#$(&+*#)(#),9-+%? WF97)741#*%"714#'&(E+%3".#+7-%#)8("%#,)#=$7))%&N#!#"(0&9%#)%++"#0"#)8,)6"#$8%&%#8%6"#1(741.#E0)#2($3,4#8,"46)#9(4:7&3%*#)8,)# B%)?#<X%#'&(37"%"#)(#*7"9+("%#87"#4%$#%3'+(B%&#74#,#:(++($0'#E+(1#'(")?>
    73. 73. C# '&DE#F <=>>?@A=B !""#$%&'()*+*(,%+-'.()/-0%'(12*3(4+'(5"6()%'-#7%/ ! !"#$%&'#()*+,-.#/0/1#23#*'#456#3,5#17#1778.#17.19:#;6&%-# !"#$%#&%'(&)%*#+,")#$%%-.#/(01#2($3,4.#5((1+%6"#)('#*%"714%&.#8,"#9(4:7&3%*#)8,)#8%6"#+%,;741# <$%#8%,&#)(#=$7))%&>?#2($3,46"#&%,"(4"#:(&#@07))741#,&%#:,"974,)741#A#,4*#)8%B#"8($#$8B# 5((1+%6"#+("741#7)"#9((+? 2($3,4#C(74%*#5((1+%#)8&%%#B%,&"#,1(#A#)((#+,)%.#8%#4($#",B"?#=8%#9(3',4B6"#%4174%%&"D:7&")# 90+)0&%#$,"#:7&3+B#74#'+,9%.#3%,4741#%;%&B#*%97"7(4#8,*#)(#E%#'&(;%4#)8&(018#%F8,0")7;%#)%")741.# &,)8%&#)8,4#,#&%+7,49%#(4#,#9+%,&#;7"7(4#(:#5((1+%6"#*%"714?#!4*#74#,#E,9-8,4*%*#"+,3#,)#5((1+%#GH# I,&7"",#I,B%&.#)8%#8%,*#(:#J0"%&#%F'%&7%49%.J#8%#4()%"#)8,)#)('#3,4,1%3%4)#74#98,&1%#(:#*%"714# 8,*#4()#E,9-1&(04*#74#)8%#:7%+*K L8%4#,#9(3',4B#7"#:7++%*#$7)8#%4174%%&".#7)#)0&4"#)(#%4174%%&741#)(#"(+;%#'&(E+%3"?#M%*09%#%,98#*%97"7(4#)(#,#"73'+%#+(179# '&(E+%3?#M%3(;%#,++#"0EC%9)7;7)B#,4*#C0")#+((-#,)#)8%#*,),?#/,),#74#B(0&#:,;(&N#O-.#+,0498#7)?#/,),#"8($"#4%1,)7;%#%::%9)"N#2,9-#)(# )8%#*&,$741#E(,&*?#!4*#)8,)#*,),#%;%4)0,++B#E%9(3%"#,#9&0)98#:(&#%;%&B#*%97"7(4.#',&,+BP741#)8%#9(3',4B#,4*#'&%;%4)741#7)# :&(3#3,-741#,4B#*,&741#*%"714#*%97"7(4"? # # # Q%".#7)6"#)&0%#)8,)#,#)%,3#,)#5((1+%#9(0+*46)#*%97*%#E%)$%%4#)$(#E+0%".#"(#)8%B6&%#)%")741#RS#"8,*%"#E%)$%%4#%,98#E+0%#)(#"%%#$8798# (4%#'%&:(&3"#E%))%&?#T#8,*#,#&%9%4)#*%E,)%#(;%&#$8%)8%&#,#E(&*%&#"8(0+*#E%#U.#R#(&#V#'7F%+"#$7*%.#,4*#$,"#,"-%*#)(#'&(;%#3B# 9,"%?#T#9,46)#('%&,)%#74#,4#%4;7&(43%4)#+7-%#)8,)?#T6;%#1&($4#)7&%*#(:#*%E,)741#"098#3747"90+%#*%"714#*%97"7(4"?#=8%&%#,&%#3(&%# %F97)741#*%"714#'&(E+%3"#74#)87"#$(&+*#)(#),9-+%? WF97)741#*%"714#'&(E+%3".#+7-%#)8("%#,)#=$7))%&N#!#"(0&9%#)%++"#0"#)8,)6"#$8%&%#8%6"#1(741.#E0)#2($3,4#8,"46)#9(4:7&3%*#)8,)# B%)?#<X%#'&(37"%"#)(#*7"9+("%#87"#4%$#%3'+(B%&#74#,#:(++($0'#E+(1#'(")?>
    74. 74. Tom Chi: “Think of your designer as a guide in this multi-variate optimization process. A good designer has been all over parts of the territory a dozen times on various projects and has studied the design patterns and techniques that help in different problems/situations. Because of this, he or she has intuition on how to approach a problem, just as an experienced software architect has intuition on software design approaches that provide different benefits/drawbacks.” 40
    75. 75. UX and WA people need to talk together about project goals 41
    76. 76. 42
    77. 77. Vanguard and the quantification of search Target Oct 3 Oct 10 Oct 16 Mean distance from 1st 3 13 7 5 Median distance from 1st 2 7 3 1 Count: Below 1st 47% 84% 62% 58% Count: Below 5th 12% 58% 38% 14% Count: Below 10th 7% 38% 10% 7% Precision – Strict 42% 15% 36% 39% Precision – Loose 71% 38% 53% 65% Precision – Permissive 96% 55% 72% 92% Note: quantification, not monetization
    78. 78. Changing thinking takeaways ★ Most things can be quantified ★ Stories and emotions can make stronger cases than data, and for data ★ We need more talking, and more listening 44
    79. 79. Challenges: how do we... ★ Bridge cultural gaps? ★ Get different groups to speak the same language? ★ Design and manage integrated teams? ★ Find better, more open tools? ★ Develop a unified methodology? 45
    80. 80. Do we have a choice? An individual often uses only half their brain Effective teams and organizations use both halves 46
    81. 81. Some day my book will come... Search Analytics for Your Site: Conversations with Your Customers Louis Rosenfeld & Marko Hurst Rosenfeld Media, 2009. rosenfeldmedia.com/books/searchanalytics 48
    82. 82. Until then... Louis Rosenfeld 457 Third Street, #4R Brooklyn, NY 11215 USA lou@louisrosenfeld.com www.louisrosenfeld.com www.rosenfeldmedia.com Twitter: @louisrosenfeld @rosenfeldmedia This presentation @ http://www.slideshare.net/lrosenfeld
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×