SlideShare a Scribd company logo
1 of 85
Marrying Web Analytics
and User Experience
Louis Rosenfeld • 5 August 2009
Delve NYC • Brooklyn
                    1
Web Analytics?

    User Experience?




2
Code “DELVE” for 25% off
 at rosenfeldmedia.com
           3
My recent
struggle



            4
CONTRASTING WEB ANALYTICS AND USER EXPERIENCE
                       5
Who we are
       How we do our work
       What data we use
       How we use that data

CONTRASTING WEB ANALYTICS AND USER EXPERIENCE
                       5
WHO WE ARE
ARE THE STEREOTYPES TRUE?
                            6
VIVE LA DIFFÉRENCE! (FROM MARKO HURST)
                               7
!"#$%&"'()*+),%(-).(%("-&/)0(1/*$%)
        Behavioral                                                   /       Eyetracking                            Data Mining/Analysis
                                                                                                                    A/B (Live) Testing

                                                                     Usability Benchmarking (in lab)



                                                                                                /
             Data Source
                                    Usability Lab Studies                                            Online User Experience Assessments
                                                                                                     (“Vividence-like” studies)


                                    Ethnographic Field Studies
                           mix




                                                                      Diary/Camera Study
                                                                      Message Board Mining
                                    Participatory Design              Customer feedback via email
                                    Focus Groups                              Desirability studies                  Intercept Surveys
        Attitudinal                 Phone Interviews                          Cardsorting                           Email Surveys

                                                                                   mix
                                 Qualitative (direct)                        Approach                       Quantitative (indirect)
                                  Key for Context of Product Use during data collection
                                    Natural use of product                               De-contextualized / not using product
    © 2008 Christian Rohrer         Scripted (often lab-based) use of product            Combination / hybrid
                                                                                                                                     20


HOW USER EXPERIENCE PEOPLE SEE THEIR WORK
(FROM CHRISTIAN ROHRER)
                                                                         8
!"#$%&"'()*+),%(-).(%("-&/)0(1/*$%)
        Behavioral                                                   /       Eyetracking                            Data Mining/Analysis
                                                                                                                    A/B (Live) Testing

                                                                     Usability Benchmarking (in lab)



                                                                                                /
             Data Source
                                    Usability Lab Studies                                            Online User Experience Assessments
                                                                                                     (“Vividence-like” studies)


                                    Ethnographic Field Studies
                           mix




                                                                      Diary/Camera Study
                                                                      Message Board Mining
                                    Participatory Design              Customer feedback via email
                                    Focus Groups                              Desirability studies                  Intercept Surveys
        Attitudinal                 Phone Interviews                          Cardsorting                           Email Surveys

                                                                                   mix
                                 Qualitative (direct)                        Approach                       Quantitative (indirect)
                                  Key for Context of Product Use during data collection
                                    Natural use of product                               De-contextualized / not using product
    © 2008 Christian Rohrer         Scripted (often lab-based) use of product            Combination / hybrid
                                                                                                                                     20


HOW USER EXPERIENCE PEOPLE SEE THEIR WORK
(FROM CHRISTIAN ROHRER)
                                                                         8
HOW WEB ANALYTICS PEOPLE SEE THEIR WORK
(FROM AVINASH KAUSHIK)	
                       9
HOW WEB ANALYTICS PEOPLE SEE THEIR WORK
(FROM AVINASH KAUSHIK)	
                       9
The data that
drives our decisions




              10
The data that
drives our decisions
     Web Analytics                User Experience

       behavioral                     attitudinal

      quantitative                   qualitative

      high fidelity                     artificial

      high volume                    high quality

 This data is about WHAT        This data is about WHY

                           10
The data that
drives our decisions
     Web Analytics                User Experience

       behavioral                     attitudinal

      quantitative                   qualitative

      high fidelity                     artificial

      high volume                    high quality

 This data is about WHAT        This data is about WHY

                           10
The data that
drives our decisions
     Web Analytics                User Experience

       behavioral                     attitudinal

      quantitative                   qualitative

      high fidelity                     artificial

      high volume                    high quality

 This data is about WHAT        This data is about WHY

                           10
The data that
drives our decisions
     Web Analytics                User Experience

       behavioral                     attitudinal

      quantitative                   qualitative

      high fidelity                     artificial

      high volume                    high quality

 This data is about WHAT        This data is about WHY

                           10
The data that
drives our decisions
     Web Analytics                User Experience

       behavioral                     attitudinal

      quantitative                   qualitative

      high fidelity                     artificial

      high volume                    high quality

 This data is about WHAT        This data is about WHY

                           10
The data that
drives our decisions
     Web Analytics                User Experience

       behavioral                     attitudinal

      quantitative                   qualitative

      high fidelity                     artificial

      high volume                    high quality

 This data is about WHAT        This data is about WHY

                           10
Not much use to know
what is happening if you
don’t know why




             11
Not much use to know
what is happening if you
don’t know why

Hard to know why things
are happening if you don’t
know what is happening
             11
The ways we analyze
our data



           12
The ways we analyze
our data



           12
The ways we analyze
our data



           12
XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200
  971 0 0.02
XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ie=UTF-8&client=www&q=license+plate
  &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX
  X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16
XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=regional+transportation+governance
  +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17




                               13
XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200
  971 0 0.02
XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ie=UTF-8&client=www&q=license+plate
  &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX
  X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16
XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=regional+transportation+governance
  +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17




                               14
XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200
  971 0 0.02
XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ie=UTF-8&client=www&q=license+plate
  &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX
  X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16
XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=regional+transportation+governance
  +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17




  Q “What were the most
  common searches?”
                               14
XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200
  971 0 0.02
XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ie=UTF-8&client=www&q=license+plate
  &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX
  X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16
XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=regional+transportation+governance
  +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17




  Q “What were the most
  common searches?”
                               14
Analyzing data
the UX way:
play with the data,
look for patterns, trends,
and outliers
Analyzing data
the UX way:
play with the data,
look for patterns, trends,
and outliers

So what’s being measured?
XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200
  971 0 0.02
XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ie=UTF-8&client=www&q=license+plate
  &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX
  X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16
XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=regional+transportation+governance
  +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17




                               16
XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200
  971 0 0.02
XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ie=UTF-8&client=www&q=license+plate
  &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX
  X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16
XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search?
  access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL
  %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty
  lesheet=www&q=regional+transportation+governance
  +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17




  Q “Are we converting
  license plate renewals?”
                               16
Before data analysis:
why are we here?
★ Commerce
★ Lead Generation
★ Content/Media
★ Support/Self-Service

              17
Before data analysis:
why are we here?
★ Commerce
★ Lead Generation
★ Content/Media
★ Support/Self-Service
Data supports metrics
              17
Analyzing data
the WA way:
start with metrics,
benchmark and
measure performance
Analyzing data
the WA way:
start with metrics,
benchmark and
measure performance
But you can’t measure
what you don’t know
WA: Top-down analysis
UX: Bottom-up analysis



           19
what

WA: Top-down analysis
UX: Bottom-up analysis



           19
what

WA: Top-down analysis
UX: Bottom-up analysis

                    why
           19
INTEGRATING WEB ANALYTICS AND USER EXPERIENCE
                       20
Integrating methodologies:
What, then why



            21
Common queries
can drive task analysis




               22
Common queries
can drive task analysis
                      “Can you find a map of
                      the campus?”

                      “What study abroad
                      options are available to
                      students?”

                      “When is the last home
                      football game of the
                      season?”



               22
Query data
can augment
personas




              23
Query data
can augment
personas



   “What Steven Searches”
   added to existing persona
   (from Adaptive Path)

                               23
Looking ahead
★ How do we improve other
qualitative methods with data?
★ How do qualitative data
impact quantitative analyses?
              24
Methodology takeaways:
★ Qualitative research is
expensive
★ Start with quantitative
research to identify where/when
to use qualitative methods
              25
Changing how we analyze:
Moving away from
the middle

           26
27
28
What’s in
the middle?




              28
What’s in
the middle?

Your analytics app’s
canned reports

              28
Netflix moved away
from the middle




            29
Netflix moved away
from the middle




            29
Netflix moved away
from the middle




            29
Netflix moved away
from the middle




            29
Netflix moved away
from the middle




            29
Analysis takeaways
★ Canned reports are only a
starting point
★ Move up, move down
★ Be prepared to “roll your own”
★ Demand better ad hoc
reporting from analytics apps
              30
Changing our thinking:
Getting comfortable with
the other

            31
UX people need to get
comfortable with
measuring the
unmeasurable
            32
Can you measure
your content’s
quality?
Systems can help
us objectify the
subjective
                   33
Subjective
                        evaluations...




Can you measure
your content’s
quality?
Systems can help
us objectify the
subjective
                   33
Subjective
                        evaluations...


                                   ...lead to
Can you measure                   objective
                                  decisions
your content’s
quality?
Systems can help
us objectify the
subjective
                   33
UX people need to get
comfortable with numbers
(but just a little)

           34
This is not statistics




               35
This is not statistics
This is not difficult




               35
This is not statistics
This is not difficult
This is very useful




               35
This is not statistics
This is not difficult
This is very useful
(and this is in MS Excel)




              35
WA people need to get
comfortable with stories

            36
WA people need to
understand the value of
intuition and mistakes

            38
C# '&DE#F
<=>>?@A=B

!""#$%&'()*+*(,%+-'.()/-0%'(12*3(4+'(5"6()%'-#7%/ !
!"#$%&'#()*+,-.#/0/1#23#*'#456#3,5#17#1778.#17.19:#;6&%-#

                                !"#$%#&%'(&)%*#+,")#$%%-.#/(01#2($3,4.#5((1+%6"#)('#*%"714%&.#8,"#9(4:7&3%*#)8,)#8%6"#+%,;741#
                                <$%#8%,&#)(#=$7))%&>?#2($3,46"#&%,"(4"#:(&#@07))741#,&%#:,"974,)741#A#,4*#)8%B#"8($#$8B#
                                5((1+%6"#+("741#7)"#9((+?


                                2($3,4#C(74%*#5((1+%#)8&%%#B%,&"#,1(#A#)((#+,)%.#8%#4($#",B"?#=8%#9(3',4B6"#%4174%%&"D:7&")#
                                90+)0&%#$,"#:7&3+B#74#'+,9%.#3%,4741#%;%&B#*%97"7(4#8,*#)(#E%#'&(;%4#)8&(018#%F8,0")7;%#)%")741.#
                                &,)8%&#)8,4#,#&%+7,49%#(4#,#9+%,&#;7"7(4#(:#5((1+%6"#*%"714?#!4*#74#,#E,9-8,4*%*#"+,3#,)#5((1+%#GH#
                                I,&7"",#I,B%&.#)8%#8%,*#(:#J0"%&#%F'%&7%49%.J#8%#4()%"#)8,)#)('#3,4,1%3%4)#74#98,&1%#(:#*%"714#
                                8,*#4()#E,9-1&(04*#74#)8%#:7%+*K


     L8%4#,#9(3',4B#7"#:7++%*#$7)8#%4174%%&".#7)#)0&4"#)(#%4174%%&741#)(#"(+;%#'&(E+%3"?#M%*09%#%,98#*%97"7(4#)(#,#"73'+%#+(179#
     '&(E+%3?#M%3(;%#,++#"0EC%9)7;7)B#,4*#C0")#+((-#,)#)8%#*,),?#/,),#74#B(0&#:,;(&N#O-.#+,0498#7)?#/,),#"8($"#4%1,)7;%#%::%9)"N#2,9-#)(#
     )8%#*&,$741#E(,&*?#!4*#)8,)#*,),#%;%4)0,++B#E%9(3%"#,#9&0)98#:(&#%;%&B#*%97"7(4.#',&,+BP741#)8%#9(3',4B#,4*#'&%;%4)741#7)#
     :&(3#3,-741#,4B#*,&741#*%"714#*%97"7(4"? #
     #
     #
     Q%".#7)6"#)&0%#)8,)#,#)%,3#,)#5((1+%#9(0+*46)#*%97*%#E%)$%%4#)$(#E+0%".#"(#)8%B6&%#)%")741#RS#"8,*%"#E%)$%%4#%,98#E+0%#)(#"%%#$8798#
     (4%#'%&:(&3"#E%))%&?#T#8,*#,#&%9%4)#*%E,)%#(;%&#$8%)8%&#,#E(&*%&#"8(0+*#E%#U.#R#(&#V#'7F%+"#$7*%.#,4*#$,"#,"-%*#)(#'&(;%#3B#
     9,"%?#T#9,46)#('%&,)%#74#,4#%4;7&(43%4)#+7-%#)8,)?#T6;%#1&($4#)7&%*#(:#*%E,)741#"098#3747"90+%#*%"714#*%97"7(4"?#=8%&%#,&%#3(&%#
     %F97)741#*%"714#'&(E+%3"#74#)87"#$(&+*#)(#),9-+%?


WF97)741#*%"714#'&(E+%3".#+7-%#)8("%#,)#=$7))%&N#!#"(0&9%#)%++"#0"#)8,)6"#$8%&%#8%6"#1(741.#E0)#2($3,4#8,"46)#9(4:7&3%*#)8,)#
B%)?#<X%#'&(37"%"#)(#*7"9+("%#87"#4%$#%3'+(B%&#74#,#:(++($0'#E+(1#'(")?>
C# '&DE#F
<=>>?@A=B

!""#$%&'()*+*(,%+-'.()/-0%'(12*3(4+'(5"6()%'-#7%/ !
!"#$%&'#()*+,-.#/0/1#23#*'#456#3,5#17#1778.#17.19:#;6&%-#

                                !"#$%#&%'(&)%*#+,")#$%%-.#/(01#2($3,4.#5((1+%6"#)('#*%"714%&.#8,"#9(4:7&3%*#)8,)#8%6"#+%,;741#
                                <$%#8%,&#)(#=$7))%&>?#2($3,46"#&%,"(4"#:(&#@07))741#,&%#:,"974,)741#A#,4*#)8%B#"8($#$8B#
                                5((1+%6"#+("741#7)"#9((+?


                                2($3,4#C(74%*#5((1+%#)8&%%#B%,&"#,1(#A#)((#+,)%.#8%#4($#",B"?#=8%#9(3',4B6"#%4174%%&"D:7&")#
                                90+)0&%#$,"#:7&3+B#74#'+,9%.#3%,4741#%;%&B#*%97"7(4#8,*#)(#E%#'&(;%4#)8&(018#%F8,0")7;%#)%")741.#
                                &,)8%&#)8,4#,#&%+7,49%#(4#,#9+%,&#;7"7(4#(:#5((1+%6"#*%"714?#!4*#74#,#E,9-8,4*%*#"+,3#,)#5((1+%#GH#
                                I,&7"",#I,B%&.#)8%#8%,*#(:#J0"%&#%F'%&7%49%.J#8%#4()%"#)8,)#)('#3,4,1%3%4)#74#98,&1%#(:#*%"714#
                                8,*#4()#E,9-1&(04*#74#)8%#:7%+*K


     L8%4#,#9(3',4B#7"#:7++%*#$7)8#%4174%%&".#7)#)0&4"#)(#%4174%%&741#)(#"(+;%#'&(E+%3"?#M%*09%#%,98#*%97"7(4#)(#,#"73'+%#+(179#
     '&(E+%3?#M%3(;%#,++#"0EC%9)7;7)B#,4*#C0")#+((-#,)#)8%#*,),?#/,),#74#B(0&#:,;(&N#O-.#+,0498#7)?#/,),#"8($"#4%1,)7;%#%::%9)"N#2,9-#)(#
     )8%#*&,$741#E(,&*?#!4*#)8,)#*,),#%;%4)0,++B#E%9(3%"#,#9&0)98#:(&#%;%&B#*%97"7(4.#',&,+BP741#)8%#9(3',4B#,4*#'&%;%4)741#7)#
     :&(3#3,-741#,4B#*,&741#*%"714#*%97"7(4"? #
     #
     #
     Q%".#7)6"#)&0%#)8,)#,#)%,3#,)#5((1+%#9(0+*46)#*%97*%#E%)$%%4#)$(#E+0%".#"(#)8%B6&%#)%")741#RS#"8,*%"#E%)$%%4#%,98#E+0%#)(#"%%#$8798#
     (4%#'%&:(&3"#E%))%&?#T#8,*#,#&%9%4)#*%E,)%#(;%&#$8%)8%&#,#E(&*%&#"8(0+*#E%#U.#R#(&#V#'7F%+"#$7*%.#,4*#$,"#,"-%*#)(#'&(;%#3B#
     9,"%?#T#9,46)#('%&,)%#74#,4#%4;7&(43%4)#+7-%#)8,)?#T6;%#1&($4#)7&%*#(:#*%E,)741#"098#3747"90+%#*%"714#*%97"7(4"?#=8%&%#,&%#3(&%#
     %F97)741#*%"714#'&(E+%3"#74#)87"#$(&+*#)(#),9-+%?


WF97)741#*%"714#'&(E+%3".#+7-%#)8("%#,)#=$7))%&N#!#"(0&9%#)%++"#0"#)8,)6"#$8%&%#8%6"#1(741.#E0)#2($3,4#8,"46)#9(4:7&3%*#)8,)#
B%)?#<X%#'&(37"%"#)(#*7"9+("%#87"#4%$#%3'+(B%&#74#,#:(++($0'#E+(1#'(")?>
C# '&DE#F
<=>>?@A=B

!""#$%&'()*+*(,%+-'.()/-0%'(12*3(4+'(5"6()%'-#7%/ !
!"#$%&'#()*+,-.#/0/1#23#*'#456#3,5#17#1778.#17.19:#;6&%-#

                                !"#$%#&%'(&)%*#+,")#$%%-.#/(01#2($3,4.#5((1+%6"#)('#*%"714%&.#8,"#9(4:7&3%*#)8,)#8%6"#+%,;741#
                                <$%#8%,&#)(#=$7))%&>?#2($3,46"#&%,"(4"#:(&#@07))741#,&%#:,"974,)741#A#,4*#)8%B#"8($#$8B#
                                5((1+%6"#+("741#7)"#9((+?


                                2($3,4#C(74%*#5((1+%#)8&%%#B%,&"#,1(#A#)((#+,)%.#8%#4($#",B"?#=8%#9(3',4B6"#%4174%%&"D:7&")#
                                90+)0&%#$,"#:7&3+B#74#'+,9%.#3%,4741#%;%&B#*%97"7(4#8,*#)(#E%#'&(;%4#)8&(018#%F8,0")7;%#)%")741.#
                                &,)8%&#)8,4#,#&%+7,49%#(4#,#9+%,&#;7"7(4#(:#5((1+%6"#*%"714?#!4*#74#,#E,9-8,4*%*#"+,3#,)#5((1+%#GH#
                                I,&7"",#I,B%&.#)8%#8%,*#(:#J0"%&#%F'%&7%49%.J#8%#4()%"#)8,)#)('#3,4,1%3%4)#74#98,&1%#(:#*%"714#
                                8,*#4()#E,9-1&(04*#74#)8%#:7%+*K


     L8%4#,#9(3',4B#7"#:7++%*#$7)8#%4174%%&".#7)#)0&4"#)(#%4174%%&741#)(#"(+;%#'&(E+%3"?#M%*09%#%,98#*%97"7(4#)(#,#"73'+%#+(179#
     '&(E+%3?#M%3(;%#,++#"0EC%9)7;7)B#,4*#C0")#+((-#,)#)8%#*,),?#/,),#74#B(0&#:,;(&N#O-.#+,0498#7)?#/,),#"8($"#4%1,)7;%#%::%9)"N#2,9-#)(#
     )8%#*&,$741#E(,&*?#!4*#)8,)#*,),#%;%4)0,++B#E%9(3%"#,#9&0)98#:(&#%;%&B#*%97"7(4.#',&,+BP741#)8%#9(3',4B#,4*#'&%;%4)741#7)#
     :&(3#3,-741#,4B#*,&741#*%"714#*%97"7(4"? #
     #
     #
     Q%".#7)6"#)&0%#)8,)#,#)%,3#,)#5((1+%#9(0+*46)#*%97*%#E%)$%%4#)$(#E+0%".#"(#)8%B6&%#)%")741#RS#"8,*%"#E%)$%%4#%,98#E+0%#)(#"%%#$8798#
     (4%#'%&:(&3"#E%))%&?#T#8,*#,#&%9%4)#*%E,)%#(;%&#$8%)8%&#,#E(&*%&#"8(0+*#E%#U.#R#(&#V#'7F%+"#$7*%.#,4*#$,"#,"-%*#)(#'&(;%#3B#
     9,"%?#T#9,46)#('%&,)%#74#,4#%4;7&(43%4)#+7-%#)8,)?#T6;%#1&($4#)7&%*#(:#*%E,)741#"098#3747"90+%#*%"714#*%97"7(4"?#=8%&%#,&%#3(&%#
     %F97)741#*%"714#'&(E+%3"#74#)87"#$(&+*#)(#),9-+%?


WF97)741#*%"714#'&(E+%3".#+7-%#)8("%#,)#=$7))%&N#!#"(0&9%#)%++"#0"#)8,)6"#$8%&%#8%6"#1(741.#E0)#2($3,4#8,"46)#9(4:7&3%*#)8,)#
B%)?#<X%#'&(37"%"#)(#*7"9+("%#87"#4%$#%3'+(B%&#74#,#:(++($0'#E+(1#'(")?>
Tom Chi:
“Think of your designer as a guide in this
multi-variate optimization process. A good
designer has been all over parts of the territory
a dozen times on various projects and has
studied the design patterns and techniques
that help in different problems/situations.
Because of this, he or she has intuition on how
to approach a problem, just as an experienced
software architect has intuition on software
design approaches that provide different
benefits/drawbacks.”
                       40
UX and WA people need
to talk together about
project goals

            41
42
Vanguard and the
quantification of search
                            Target    Oct 3   Oct 10   Oct 16
 Mean distance from 1st         3     13        7        5
 Median distance from 1st       2      7        3        1
 Count: Below 1st             47%    84%      62%      58%

 Count: Below 5th             12%    58%      38%      14%

 Count: Below 10th             7%    38%      10%      7%

 Precision – Strict           42%    15%      36%      39%
 Precision – Loose            71%    38%      53%      65%
 Precision – Permissive       96%    55%      72%      92%




Note: quantification, not monetization
Changing thinking
takeaways
★ Most things can be quantified
★ Stories and emotions can
make stronger cases than data,
and for data
★ We need more talking, and
more listening
              44
Challenges: how do we...
★ Bridge cultural gaps?
★ Get different groups to speak
the same language?
★ Design and manage integrated
teams?
★ Find better, more open tools?
★ Develop a unified methodology?
             45
Do we have a choice?
          An individual often uses
          only half their brain

          Effective teams and
          organizations use both
          halves


            46
Some day my book
will come...
Search Analytics for Your Site:
Conversations with Your Customers

Louis Rosenfeld & Marko Hurst
Rosenfeld Media, 2009.

rosenfeldmedia.com/books/searchanalytics


                            48
Until then...
Louis Rosenfeld
457 Third Street, #4R
Brooklyn, NY 11215 USA

lou@louisrosenfeld.com
www.louisrosenfeld.com
www.rosenfeldmedia.com
Twitter:
	 @louisrosenfeld
	 @rosenfeldmedia

This presentation @ http://www.slideshare.net/lrosenfeld

More Related Content

What's hot

What's hot (7)

Introduction to Information Architecture & Design - SVA Workshop 03/22/14
Introduction to Information Architecture & Design - SVA Workshop 03/22/14Introduction to Information Architecture & Design - SVA Workshop 03/22/14
Introduction to Information Architecture & Design - SVA Workshop 03/22/14
 
Local ESOMAR event: Romania May 2012
Local ESOMAR event: Romania May 2012Local ESOMAR event: Romania May 2012
Local ESOMAR event: Romania May 2012
 
Lean ux deck_aux_120410
Lean ux deck_aux_120410Lean ux deck_aux_120410
Lean ux deck_aux_120410
 
Center for Creative Photography Redesign
Center for Creative Photography RedesignCenter for Creative Photography Redesign
Center for Creative Photography Redesign
 
Ux bootcamp small
Ux bootcamp smallUx bootcamp small
Ux bootcamp small
 
Overlappings and Underpinnings - Content Strategy and Information Architecture
Overlappings and Underpinnings - Content Strategy and Information ArchitectureOverlappings and Underpinnings - Content Strategy and Information Architecture
Overlappings and Underpinnings - Content Strategy and Information Architecture
 
Caleb Brown - Current Résumé
Caleb Brown - Current RésuméCaleb Brown - Current Résumé
Caleb Brown - Current Résumé
 

Viewers also liked

Cart Abandonment Email Marketing
Cart Abandonment Email MarketingCart Abandonment Email Marketing
Cart Abandonment Email Marketing
Silverpop
 

Viewers also liked (20)

Concept Diagram Workshop
Concept Diagram WorkshopConcept Diagram Workshop
Concept Diagram Workshop
 
Google Website Optimizer in Obama Landing Pages
Google Website Optimizer in Obama Landing PagesGoogle Website Optimizer in Obama Landing Pages
Google Website Optimizer in Obama Landing Pages
 
The Science of Analytics
The Science of AnalyticsThe Science of Analytics
The Science of Analytics
 
効果的なAbテストがwebサイトを成長させる~abテスト成功メソッドのご紹介~
効果的なAbテストがwebサイトを成長させる~abテスト成功メソッドのご紹介~効果的なAbテストがwebサイトを成長させる~abテスト成功メソッドのご紹介~
効果的なAbテストがwebサイトを成長させる~abテスト成功メソッドのご紹介~
 
クックパッドのグロースハックについて 20140610 ver1.2(更新版)
クックパッドのグロースハックについて 20140610 ver1.2(更新版)クックパッドのグロースハックについて 20140610 ver1.2(更新版)
クックパッドのグロースハックについて 20140610 ver1.2(更新版)
 
boketeのグロースハック 先生:イセ オサム
boketeのグロースハック 先生:イセ オサムboketeのグロースハック 先生:イセ オサム
boketeのグロースハック 先生:イセ オサム
 
『個客』 視点の行動分析がウェブビジネスを変える!
『個客』 視点の行動分析がウェブビジネスを変える!『個客』 視点の行動分析がウェブビジネスを変える!
『個客』 視点の行動分析がウェブビジネスを変える!
 
Advanced Google Analytics #SearchFest
Advanced Google Analytics #SearchFestAdvanced Google Analytics #SearchFest
Advanced Google Analytics #SearchFest
 
The Anatomy of the Perfect Landing Page
The Anatomy of the Perfect Landing PageThe Anatomy of the Perfect Landing Page
The Anatomy of the Perfect Landing Page
 
アトリビューションのLandscape
アトリビューションのLandscapeアトリビューションのLandscape
アトリビューションのLandscape
 
【eVar7】UX4KPI_web担当者forum_part1.2
【eVar7】UX4KPI_web担当者forum_part1.2【eVar7】UX4KPI_web担当者forum_part1.2
【eVar7】UX4KPI_web担当者forum_part1.2
 
大規模サイトにおけるGoogleアナリティクス導入から成果まで
大規模サイトにおけるGoogleアナリティクス導入から成果まで大規模サイトにおけるGoogleアナリティクス導入から成果まで
大規模サイトにおけるGoogleアナリティクス導入から成果まで
 
営業向け ABテストの統計学 データアーティスト
営業向け ABテストの統計学 データアーティスト営業向け ABテストの統計学 データアーティスト
営業向け ABテストの統計学 データアーティスト
 
【アクセス解析サミット2011】ECナビ ウェブ最適化の取り組み ABテストから会社が動いた
【アクセス解析サミット2011】ECナビ ウェブ最適化の取り組み ABテストから会社が動いた【アクセス解析サミット2011】ECナビ ウェブ最適化の取り組み ABテストから会社が動いた
【アクセス解析サミット2011】ECナビ ウェブ最適化の取り組み ABテストから会社が動いた
 
Cart Abandonment Email Marketing
Cart Abandonment Email MarketingCart Abandonment Email Marketing
Cart Abandonment Email Marketing
 
KAIZEN platformの公開グロースハック!〜 CVRを40%以上向上させた施策 先生:須藤 憲司
KAIZEN platformの公開グロースハック!〜 CVRを40%以上向上させた施策 先生:須藤 憲司KAIZEN platformの公開グロースハック!〜 CVRを40%以上向上させた施策 先生:須藤 憲司
KAIZEN platformの公開グロースハック!〜 CVRを40%以上向上させた施策 先生:須藤 憲司
 
10,000DAUまで 成長させたグロースハック事例
10,000DAUまで 成長させたグロースハック事例10,000DAUまで 成長させたグロースハック事例
10,000DAUまで 成長させたグロースハック事例
 
UIscope流グロースハック~ユーザーテストで「AARRR」をハックする!~
UIscope流グロースハック~ユーザーテストで「AARRR」をハックする!~UIscope流グロースハック~ユーザーテストで「AARRR」をハックする!~
UIscope流グロースハック~ユーザーテストで「AARRR」をハックする!~
 
登録数2倍にしてと言われた時の正しい対処法
登録数2倍にしてと言われた時の正しい対処法登録数2倍にしてと言われた時の正しい対処法
登録数2倍にしてと言われた時の正しい対処法
 
「クックパッドとZaimのグロースハックについて」
「クックパッドとZaimのグロースハックについて」「クックパッドとZaimのグロースハックについて」
「クックパッドとZaimのグロースハックについて」
 

Similar to Marrying Web Analytics and User Experience

PxS’12 - week 4 - qualitative analysis
PxS’12 - week 4 - qualitative analysisPxS’12 - week 4 - qualitative analysis
PxS’12 - week 4 - qualitative analysis
hendrikknoche
 
Reflect network meeting (13/07/2012)
Reflect network meeting (13/07/2012)Reflect network meeting (13/07/2012)
Reflect network meeting (13/07/2012)
Reflect Project
 
SDL 08 Design for Empathy and Change
SDL 08 Design for Empathy and ChangeSDL 08 Design for Empathy and Change
SDL 08 Design for Empathy and Change
Florian Vollmer
 
5 anchovi final
5 anchovi   final5 anchovi   final
5 anchovi final
bbabenko
 
Usability testing for qualitative researchers
Usability testing for qualitative researchersUsability testing for qualitative researchers
Usability testing for qualitative researchers
Kay Corry Aubrey
 
Usability testing for qualitative researchers
Usability testing for qualitative researchersUsability testing for qualitative researchers
Usability testing for qualitative researchers
ResearchShare
 

Similar to Marrying Web Analytics and User Experience (20)

PxS’12 - week 4 - qualitative analysis
PxS’12 - week 4 - qualitative analysisPxS’12 - week 4 - qualitative analysis
PxS’12 - week 4 - qualitative analysis
 
User Research in the Financial Space
User Research in the Financial SpaceUser Research in the Financial Space
User Research in the Financial Space
 
Workshop: 'Self-administered Mobile Survey Workshop' - Dr Michael Bosnjak, Fr...
Workshop: 'Self-administered Mobile Survey Workshop' - Dr Michael Bosnjak, Fr...Workshop: 'Self-administered Mobile Survey Workshop' - Dr Michael Bosnjak, Fr...
Workshop: 'Self-administered Mobile Survey Workshop' - Dr Michael Bosnjak, Fr...
 
Reflect network meeting (13/07/2012)
Reflect network meeting (13/07/2012)Reflect network meeting (13/07/2012)
Reflect network meeting (13/07/2012)
 
Akendi
AkendiAkendi
Akendi
 
Workshop: Managing top tasks #BPCW11
Workshop: Managing top tasks #BPCW11Workshop: Managing top tasks #BPCW11
Workshop: Managing top tasks #BPCW11
 
Collaboration and Sharing
Collaboration and SharingCollaboration and Sharing
Collaboration and Sharing
 
Od2 research framework (1)
Od2 research framework (1)Od2 research framework (1)
Od2 research framework (1)
 
Od1 research framework
Od1 research frameworkOd1 research framework
Od1 research framework
 
Everything you ever wanted to know about UX (*but were afraid to ask)
Everything you ever wanted to know about UX  (*but were afraid to ask)Everything you ever wanted to know about UX  (*but were afraid to ask)
Everything you ever wanted to know about UX (*but were afraid to ask)
 
Davenport Webinar Predictive Analytics
Davenport Webinar Predictive AnalyticsDavenport Webinar Predictive Analytics
Davenport Webinar Predictive Analytics
 
Conversion Conference - What's in YOUR toolkit?
Conversion Conference - What's in YOUR toolkit?Conversion Conference - What's in YOUR toolkit?
Conversion Conference - What's in YOUR toolkit?
 
Prototyping and Scrum
Prototyping and ScrumPrototyping and Scrum
Prototyping and Scrum
 
User-Testing, Testing, 1,2,3
User-Testing, Testing, 1,2,3User-Testing, Testing, 1,2,3
User-Testing, Testing, 1,2,3
 
SDL 08 Design for Empathy and Change
SDL 08 Design for Empathy and ChangeSDL 08 Design for Empathy and Change
SDL 08 Design for Empathy and Change
 
Anchovi NSF final Presentation
Anchovi  NSF final PresentationAnchovi  NSF final Presentation
Anchovi NSF final Presentation
 
5 anchovi final
5 anchovi   final5 anchovi   final
5 anchovi final
 
Usability testing for qualitative researchers
Usability testing for qualitative researchersUsability testing for qualitative researchers
Usability testing for qualitative researchers
 
Usability testing for qualitative researchers
Usability testing for qualitative researchersUsability testing for qualitative researchers
Usability testing for qualitative researchers
 
Information Architecture Summit 2012 Recap
Information Architecture Summit 2012 RecapInformation Architecture Summit 2012 Recap
Information Architecture Summit 2012 Recap
 

More from Louis Rosenfeld

Closing the Findability Gap: 8 better practices from information architecture
Closing the Findability Gap: 8 better practices from information architectureClosing the Findability Gap: 8 better practices from information architecture
Closing the Findability Gap: 8 better practices from information architecture
Louis Rosenfeld
 

More from Louis Rosenfeld (20)

Information Architecture for Truth
Information Architecture for TruthInformation Architecture for Truth
Information Architecture for Truth
 
Falling in and out and in love with Information Architecture
Falling in and out and in love with Information ArchitectureFalling in and out and in love with Information Architecture
Falling in and out and in love with Information Architecture
 
What to do when you don't know what to do
What to do when you don't know what to doWhat to do when you don't know what to do
What to do when you don't know what to do
 
Redesign Must Die (updated Feb 2014)
Redesign Must Die (updated Feb 2014)Redesign Must Die (updated Feb 2014)
Redesign Must Die (updated Feb 2014)
 
8 Information Architecture Better Practices
8 Information Architecture Better Practices8 Information Architecture Better Practices
8 Information Architecture Better Practices
 
Search Analytics for Content Strategists
Search Analytics for Content StrategistsSearch Analytics for Content Strategists
Search Analytics for Content Strategists
 
Closing the Findability Gap: 8 better practices from information architecture
Closing the Findability Gap: 8 better practices from information architectureClosing the Findability Gap: 8 better practices from information architecture
Closing the Findability Gap: 8 better practices from information architecture
 
Site search analytics workshop presentation
Site search analytics workshop presentationSite search analytics workshop presentation
Site search analytics workshop presentation
 
Design to Refine: Developing a tunable information architecture
Design to Refine: Developing a tunable information architectureDesign to Refine: Developing a tunable information architecture
Design to Refine: Developing a tunable information architecture
 
Redesign Must Die
Redesign Must DieRedesign Must Die
Redesign Must Die
 
Adaptable Information Workshop slides
Adaptable Information Workshop slidesAdaptable Information Workshop slides
Adaptable Information Workshop slides
 
Site Search Analytics in a Nutshell
Site Search Analytics in a NutshellSite Search Analytics in a Nutshell
Site Search Analytics in a Nutshell
 
Beyond User Research
Beyond User ResearchBeyond User Research
Beyond User Research
 
Is there such a thing as a good business model for publishing these days?
Is there such a thing as a good business model  for publishing these days?Is there such a thing as a good business model  for publishing these days?
Is there such a thing as a good business model for publishing these days?
 
User Experience + Publishing
User Experience + PublishingUser Experience + Publishing
User Experience + Publishing
 
Improving Findability through Site Search Analytics
Improving Findability through Site Search AnalyticsImproving Findability through Site Search Analytics
Improving Findability through Site Search Analytics
 
Redesign Must Die
Redesign Must DieRedesign Must Die
Redesign Must Die
 
Site Search Analytics
Site Search AnalyticsSite Search Analytics
Site Search Analytics
 
Site Search Analytics: Conversations with your customers
Site Search Analytics:  Conversations with your customersSite Search Analytics:  Conversations with your customers
Site Search Analytics: Conversations with your customers
 
PhillyCHI Site Search Analytics presentation (April 2, 2008)
PhillyCHI Site Search Analytics presentation (April 2, 2008)PhillyCHI Site Search Analytics presentation (April 2, 2008)
PhillyCHI Site Search Analytics presentation (April 2, 2008)
 

Recently uploaded

Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
FIDO Alliance
 

Recently uploaded (20)

AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptx
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 
JohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptxJohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptx
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptx
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptx
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
2024 May Patch Tuesday
2024 May Patch Tuesday2024 May Patch Tuesday
2024 May Patch Tuesday
 
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdfFrisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
Generative AI Use Cases and Applications.pdf
Generative AI Use Cases and Applications.pdfGenerative AI Use Cases and Applications.pdf
Generative AI Use Cases and Applications.pdf
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate Guide
 
الأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهالأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهله
 

Marrying Web Analytics and User Experience

  • 1. Marrying Web Analytics and User Experience Louis Rosenfeld • 5 August 2009 Delve NYC • Brooklyn 1
  • 2. Web Analytics? User Experience? 2
  • 3. Code “DELVE” for 25% off at rosenfeldmedia.com 3
  • 5. CONTRASTING WEB ANALYTICS AND USER EXPERIENCE 5
  • 6. Who we are How we do our work What data we use How we use that data CONTRASTING WEB ANALYTICS AND USER EXPERIENCE 5
  • 7. WHO WE ARE ARE THE STEREOTYPES TRUE? 6
  • 8. VIVE LA DIFFÉRENCE! (FROM MARKO HURST) 7
  • 9. !"#$%&"'()*+),%(-).(%("-&/)0(1/*$%) Behavioral / Eyetracking Data Mining/Analysis A/B (Live) Testing Usability Benchmarking (in lab) / Data Source Usability Lab Studies Online User Experience Assessments (“Vividence-like” studies) Ethnographic Field Studies mix Diary/Camera Study Message Board Mining Participatory Design Customer feedback via email Focus Groups Desirability studies Intercept Surveys Attitudinal Phone Interviews Cardsorting Email Surveys mix Qualitative (direct) Approach Quantitative (indirect) Key for Context of Product Use during data collection Natural use of product De-contextualized / not using product © 2008 Christian Rohrer Scripted (often lab-based) use of product Combination / hybrid 20 HOW USER EXPERIENCE PEOPLE SEE THEIR WORK (FROM CHRISTIAN ROHRER) 8
  • 10. !"#$%&"'()*+),%(-).(%("-&/)0(1/*$%) Behavioral / Eyetracking Data Mining/Analysis A/B (Live) Testing Usability Benchmarking (in lab) / Data Source Usability Lab Studies Online User Experience Assessments (“Vividence-like” studies) Ethnographic Field Studies mix Diary/Camera Study Message Board Mining Participatory Design Customer feedback via email Focus Groups Desirability studies Intercept Surveys Attitudinal Phone Interviews Cardsorting Email Surveys mix Qualitative (direct) Approach Quantitative (indirect) Key for Context of Product Use during data collection Natural use of product De-contextualized / not using product © 2008 Christian Rohrer Scripted (often lab-based) use of product Combination / hybrid 20 HOW USER EXPERIENCE PEOPLE SEE THEIR WORK (FROM CHRISTIAN ROHRER) 8
  • 11. HOW WEB ANALYTICS PEOPLE SEE THEIR WORK (FROM AVINASH KAUSHIK) 9
  • 12. HOW WEB ANALYTICS PEOPLE SEE THEIR WORK (FROM AVINASH KAUSHIK) 9
  • 13. The data that drives our decisions 10
  • 14. The data that drives our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 10
  • 15. The data that drives our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 10
  • 16. The data that drives our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 10
  • 17. The data that drives our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 10
  • 18. The data that drives our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 10
  • 19. The data that drives our decisions Web Analytics User Experience behavioral attitudinal quantitative qualitative high fidelity artificial high volume high quality This data is about WHAT This data is about WHY 10
  • 20. Not much use to know what is happening if you don’t know why 11
  • 21. Not much use to know what is happening if you don’t know why Hard to know why things are happening if you don’t know what is happening 11
  • 22. The ways we analyze our data 12
  • 23. The ways we analyze our data 12
  • 24. The ways we analyze our data 12
  • 25. XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17 13
  • 26. XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17 14
  • 27. XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17 Q “What were the most common searches?” 14
  • 28. XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17 Q “What were the most common searches?” 14
  • 29. Analyzing data the UX way: play with the data, look for patterns, trends, and outliers
  • 30. Analyzing data the UX way: play with the data, look for patterns, trends, and outliers So what’s being measured?
  • 31. XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17 16
  • 32. XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17 Q “Are we converting license plate renewals?” 16
  • 33. Before data analysis: why are we here? ★ Commerce ★ Lead Generation ★ Content/Media ★ Support/Self-Service 17
  • 34. Before data analysis: why are we here? ★ Commerce ★ Lead Generation ★ Content/Media ★ Support/Self-Service Data supports metrics 17
  • 35. Analyzing data the WA way: start with metrics, benchmark and measure performance
  • 36. Analyzing data the WA way: start with metrics, benchmark and measure performance But you can’t measure what you don’t know
  • 37. WA: Top-down analysis UX: Bottom-up analysis 19
  • 38. what WA: Top-down analysis UX: Bottom-up analysis 19
  • 39. what WA: Top-down analysis UX: Bottom-up analysis why 19
  • 40. INTEGRATING WEB ANALYTICS AND USER EXPERIENCE 20
  • 42. Common queries can drive task analysis 22
  • 43. Common queries can drive task analysis “Can you find a map of the campus?” “What study abroad options are available to students?” “When is the last home football game of the season?” 22
  • 45. Query data can augment personas “What Steven Searches” added to existing persona (from Adaptive Path) 23
  • 46. Looking ahead ★ How do we improve other qualitative methods with data? ★ How do qualitative data impact quantitative analyses? 24
  • 47. Methodology takeaways: ★ Qualitative research is expensive ★ Start with quantitative research to identify where/when to use qualitative methods 25
  • 48. Changing how we analyze: Moving away from the middle 26
  • 49. 27
  • 50. 28
  • 52. What’s in the middle? Your analytics app’s canned reports 28
  • 53. Netflix moved away from the middle 29
  • 54. Netflix moved away from the middle 29
  • 55. Netflix moved away from the middle 29
  • 56. Netflix moved away from the middle 29
  • 57. Netflix moved away from the middle 29
  • 58. Analysis takeaways ★ Canned reports are only a starting point ★ Move up, move down ★ Be prepared to “roll your own” ★ Demand better ad hoc reporting from analytics apps 30
  • 59. Changing our thinking: Getting comfortable with the other 31
  • 60. UX people need to get comfortable with measuring the unmeasurable 32
  • 61. Can you measure your content’s quality? Systems can help us objectify the subjective 33
  • 62. Subjective evaluations... Can you measure your content’s quality? Systems can help us objectify the subjective 33
  • 63. Subjective evaluations... ...lead to Can you measure objective decisions your content’s quality? Systems can help us objectify the subjective 33
  • 64. UX people need to get comfortable with numbers (but just a little) 34
  • 65. This is not statistics 35
  • 66. This is not statistics This is not difficult 35
  • 67. This is not statistics This is not difficult This is very useful 35
  • 68. This is not statistics This is not difficult This is very useful (and this is in MS Excel) 35
  • 69. WA people need to get comfortable with stories 36
  • 70.
  • 71.
  • 72. WA people need to understand the value of intuition and mistakes 38
  • 73. C# '&DE#F <=>>?@A=B !""#$%&'()*+*(,%+-'.()/-0%'(12*3(4+'(5"6()%'-#7%/ ! !"#$%&'#()*+,-.#/0/1#23#*'#456#3,5#17#1778.#17.19:#;6&%-# !"#$%#&%'(&)%*#+,")#$%%-.#/(01#2($3,4.#5((1+%6"#)('#*%"714%&.#8,"#9(4:7&3%*#)8,)#8%6"#+%,;741# <$%#8%,&#)(#=$7))%&>?#2($3,46"#&%,"(4"#:(&#@07))741#,&%#:,"974,)741#A#,4*#)8%B#"8($#$8B# 5((1+%6"#+("741#7)"#9((+? 2($3,4#C(74%*#5((1+%#)8&%%#B%,&"#,1(#A#)((#+,)%.#8%#4($#",B"?#=8%#9(3',4B6"#%4174%%&"D:7&")# 90+)0&%#$,"#:7&3+B#74#'+,9%.#3%,4741#%;%&B#*%97"7(4#8,*#)(#E%#'&(;%4#)8&(018#%F8,0")7;%#)%")741.# &,)8%&#)8,4#,#&%+7,49%#(4#,#9+%,&#;7"7(4#(:#5((1+%6"#*%"714?#!4*#74#,#E,9-8,4*%*#"+,3#,)#5((1+%#GH# I,&7"",#I,B%&.#)8%#8%,*#(:#J0"%&#%F'%&7%49%.J#8%#4()%"#)8,)#)('#3,4,1%3%4)#74#98,&1%#(:#*%"714# 8,*#4()#E,9-1&(04*#74#)8%#:7%+*K L8%4#,#9(3',4B#7"#:7++%*#$7)8#%4174%%&".#7)#)0&4"#)(#%4174%%&741#)(#"(+;%#'&(E+%3"?#M%*09%#%,98#*%97"7(4#)(#,#"73'+%#+(179# '&(E+%3?#M%3(;%#,++#"0EC%9)7;7)B#,4*#C0")#+((-#,)#)8%#*,),?#/,),#74#B(0&#:,;(&N#O-.#+,0498#7)?#/,),#"8($"#4%1,)7;%#%::%9)"N#2,9-#)(# )8%#*&,$741#E(,&*?#!4*#)8,)#*,),#%;%4)0,++B#E%9(3%"#,#9&0)98#:(&#%;%&B#*%97"7(4.#',&,+BP741#)8%#9(3',4B#,4*#'&%;%4)741#7)# :&(3#3,-741#,4B#*,&741#*%"714#*%97"7(4"? # # # Q%".#7)6"#)&0%#)8,)#,#)%,3#,)#5((1+%#9(0+*46)#*%97*%#E%)$%%4#)$(#E+0%".#"(#)8%B6&%#)%")741#RS#"8,*%"#E%)$%%4#%,98#E+0%#)(#"%%#$8798# (4%#'%&:(&3"#E%))%&?#T#8,*#,#&%9%4)#*%E,)%#(;%&#$8%)8%&#,#E(&*%&#"8(0+*#E%#U.#R#(&#V#'7F%+"#$7*%.#,4*#$,"#,"-%*#)(#'&(;%#3B# 9,"%?#T#9,46)#('%&,)%#74#,4#%4;7&(43%4)#+7-%#)8,)?#T6;%#1&($4#)7&%*#(:#*%E,)741#"098#3747"90+%#*%"714#*%97"7(4"?#=8%&%#,&%#3(&%# %F97)741#*%"714#'&(E+%3"#74#)87"#$(&+*#)(#),9-+%? WF97)741#*%"714#'&(E+%3".#+7-%#)8("%#,)#=$7))%&N#!#"(0&9%#)%++"#0"#)8,)6"#$8%&%#8%6"#1(741.#E0)#2($3,4#8,"46)#9(4:7&3%*#)8,)# B%)?#<X%#'&(37"%"#)(#*7"9+("%#87"#4%$#%3'+(B%&#74#,#:(++($0'#E+(1#'(")?>
  • 74. C# '&DE#F <=>>?@A=B !""#$%&'()*+*(,%+-'.()/-0%'(12*3(4+'(5"6()%'-#7%/ ! !"#$%&'#()*+,-.#/0/1#23#*'#456#3,5#17#1778.#17.19:#;6&%-# !"#$%#&%'(&)%*#+,")#$%%-.#/(01#2($3,4.#5((1+%6"#)('#*%"714%&.#8,"#9(4:7&3%*#)8,)#8%6"#+%,;741# <$%#8%,&#)(#=$7))%&>?#2($3,46"#&%,"(4"#:(&#@07))741#,&%#:,"974,)741#A#,4*#)8%B#"8($#$8B# 5((1+%6"#+("741#7)"#9((+? 2($3,4#C(74%*#5((1+%#)8&%%#B%,&"#,1(#A#)((#+,)%.#8%#4($#",B"?#=8%#9(3',4B6"#%4174%%&"D:7&")# 90+)0&%#$,"#:7&3+B#74#'+,9%.#3%,4741#%;%&B#*%97"7(4#8,*#)(#E%#'&(;%4#)8&(018#%F8,0")7;%#)%")741.# &,)8%&#)8,4#,#&%+7,49%#(4#,#9+%,&#;7"7(4#(:#5((1+%6"#*%"714?#!4*#74#,#E,9-8,4*%*#"+,3#,)#5((1+%#GH# I,&7"",#I,B%&.#)8%#8%,*#(:#J0"%&#%F'%&7%49%.J#8%#4()%"#)8,)#)('#3,4,1%3%4)#74#98,&1%#(:#*%"714# 8,*#4()#E,9-1&(04*#74#)8%#:7%+*K L8%4#,#9(3',4B#7"#:7++%*#$7)8#%4174%%&".#7)#)0&4"#)(#%4174%%&741#)(#"(+;%#'&(E+%3"?#M%*09%#%,98#*%97"7(4#)(#,#"73'+%#+(179# '&(E+%3?#M%3(;%#,++#"0EC%9)7;7)B#,4*#C0")#+((-#,)#)8%#*,),?#/,),#74#B(0&#:,;(&N#O-.#+,0498#7)?#/,),#"8($"#4%1,)7;%#%::%9)"N#2,9-#)(# )8%#*&,$741#E(,&*?#!4*#)8,)#*,),#%;%4)0,++B#E%9(3%"#,#9&0)98#:(&#%;%&B#*%97"7(4.#',&,+BP741#)8%#9(3',4B#,4*#'&%;%4)741#7)# :&(3#3,-741#,4B#*,&741#*%"714#*%97"7(4"? # # # Q%".#7)6"#)&0%#)8,)#,#)%,3#,)#5((1+%#9(0+*46)#*%97*%#E%)$%%4#)$(#E+0%".#"(#)8%B6&%#)%")741#RS#"8,*%"#E%)$%%4#%,98#E+0%#)(#"%%#$8798# (4%#'%&:(&3"#E%))%&?#T#8,*#,#&%9%4)#*%E,)%#(;%&#$8%)8%&#,#E(&*%&#"8(0+*#E%#U.#R#(&#V#'7F%+"#$7*%.#,4*#$,"#,"-%*#)(#'&(;%#3B# 9,"%?#T#9,46)#('%&,)%#74#,4#%4;7&(43%4)#+7-%#)8,)?#T6;%#1&($4#)7&%*#(:#*%E,)741#"098#3747"90+%#*%"714#*%97"7(4"?#=8%&%#,&%#3(&%# %F97)741#*%"714#'&(E+%3"#74#)87"#$(&+*#)(#),9-+%? WF97)741#*%"714#'&(E+%3".#+7-%#)8("%#,)#=$7))%&N#!#"(0&9%#)%++"#0"#)8,)6"#$8%&%#8%6"#1(741.#E0)#2($3,4#8,"46)#9(4:7&3%*#)8,)# B%)?#<X%#'&(37"%"#)(#*7"9+("%#87"#4%$#%3'+(B%&#74#,#:(++($0'#E+(1#'(")?>
  • 75. C# '&DE#F <=>>?@A=B !""#$%&'()*+*(,%+-'.()/-0%'(12*3(4+'(5"6()%'-#7%/ ! !"#$%&'#()*+,-.#/0/1#23#*'#456#3,5#17#1778.#17.19:#;6&%-# !"#$%#&%'(&)%*#+,")#$%%-.#/(01#2($3,4.#5((1+%6"#)('#*%"714%&.#8,"#9(4:7&3%*#)8,)#8%6"#+%,;741# <$%#8%,&#)(#=$7))%&>?#2($3,46"#&%,"(4"#:(&#@07))741#,&%#:,"974,)741#A#,4*#)8%B#"8($#$8B# 5((1+%6"#+("741#7)"#9((+? 2($3,4#C(74%*#5((1+%#)8&%%#B%,&"#,1(#A#)((#+,)%.#8%#4($#",B"?#=8%#9(3',4B6"#%4174%%&"D:7&")# 90+)0&%#$,"#:7&3+B#74#'+,9%.#3%,4741#%;%&B#*%97"7(4#8,*#)(#E%#'&(;%4#)8&(018#%F8,0")7;%#)%")741.# &,)8%&#)8,4#,#&%+7,49%#(4#,#9+%,&#;7"7(4#(:#5((1+%6"#*%"714?#!4*#74#,#E,9-8,4*%*#"+,3#,)#5((1+%#GH# I,&7"",#I,B%&.#)8%#8%,*#(:#J0"%&#%F'%&7%49%.J#8%#4()%"#)8,)#)('#3,4,1%3%4)#74#98,&1%#(:#*%"714# 8,*#4()#E,9-1&(04*#74#)8%#:7%+*K L8%4#,#9(3',4B#7"#:7++%*#$7)8#%4174%%&".#7)#)0&4"#)(#%4174%%&741#)(#"(+;%#'&(E+%3"?#M%*09%#%,98#*%97"7(4#)(#,#"73'+%#+(179# '&(E+%3?#M%3(;%#,++#"0EC%9)7;7)B#,4*#C0")#+((-#,)#)8%#*,),?#/,),#74#B(0&#:,;(&N#O-.#+,0498#7)?#/,),#"8($"#4%1,)7;%#%::%9)"N#2,9-#)(# )8%#*&,$741#E(,&*?#!4*#)8,)#*,),#%;%4)0,++B#E%9(3%"#,#9&0)98#:(&#%;%&B#*%97"7(4.#',&,+BP741#)8%#9(3',4B#,4*#'&%;%4)741#7)# :&(3#3,-741#,4B#*,&741#*%"714#*%97"7(4"? # # # Q%".#7)6"#)&0%#)8,)#,#)%,3#,)#5((1+%#9(0+*46)#*%97*%#E%)$%%4#)$(#E+0%".#"(#)8%B6&%#)%")741#RS#"8,*%"#E%)$%%4#%,98#E+0%#)(#"%%#$8798# (4%#'%&:(&3"#E%))%&?#T#8,*#,#&%9%4)#*%E,)%#(;%&#$8%)8%&#,#E(&*%&#"8(0+*#E%#U.#R#(&#V#'7F%+"#$7*%.#,4*#$,"#,"-%*#)(#'&(;%#3B# 9,"%?#T#9,46)#('%&,)%#74#,4#%4;7&(43%4)#+7-%#)8,)?#T6;%#1&($4#)7&%*#(:#*%E,)741#"098#3747"90+%#*%"714#*%97"7(4"?#=8%&%#,&%#3(&%# %F97)741#*%"714#'&(E+%3"#74#)87"#$(&+*#)(#),9-+%? WF97)741#*%"714#'&(E+%3".#+7-%#)8("%#,)#=$7))%&N#!#"(0&9%#)%++"#0"#)8,)6"#$8%&%#8%6"#1(741.#E0)#2($3,4#8,"46)#9(4:7&3%*#)8,)# B%)?#<X%#'&(37"%"#)(#*7"9+("%#87"#4%$#%3'+(B%&#74#,#:(++($0'#E+(1#'(")?>
  • 76. Tom Chi: “Think of your designer as a guide in this multi-variate optimization process. A good designer has been all over parts of the territory a dozen times on various projects and has studied the design patterns and techniques that help in different problems/situations. Because of this, he or she has intuition on how to approach a problem, just as an experienced software architect has intuition on software design approaches that provide different benefits/drawbacks.” 40
  • 77. UX and WA people need to talk together about project goals 41
  • 78. 42
  • 79. Vanguard and the quantification of search Target Oct 3 Oct 10 Oct 16 Mean distance from 1st 3 13 7 5 Median distance from 1st 2 7 3 1 Count: Below 1st 47% 84% 62% 58% Count: Below 5th 12% 58% 38% 14% Count: Below 10th 7% 38% 10% 7% Precision – Strict 42% 15% 36% 39% Precision – Loose 71% 38% 53% 65% Precision – Permissive 96% 55% 72% 92% Note: quantification, not monetization
  • 80. Changing thinking takeaways ★ Most things can be quantified ★ Stories and emotions can make stronger cases than data, and for data ★ We need more talking, and more listening 44
  • 81. Challenges: how do we... ★ Bridge cultural gaps? ★ Get different groups to speak the same language? ★ Design and manage integrated teams? ★ Find better, more open tools? ★ Develop a unified methodology? 45
  • 82. Do we have a choice? An individual often uses only half their brain Effective teams and organizations use both halves 46
  • 83.
  • 84. Some day my book will come... Search Analytics for Your Site: Conversations with Your Customers Louis Rosenfeld & Marko Hurst Rosenfeld Media, 2009. rosenfeldmedia.com/books/searchanalytics 48
  • 85. Until then... Louis Rosenfeld 457 Third Street, #4R Brooklyn, NY 11215 USA lou@louisrosenfeld.com www.louisrosenfeld.com www.rosenfeldmedia.com Twitter: @louisrosenfeld @rosenfeldmedia This presentation @ http://www.slideshare.net/lrosenfeld

Editor's Notes

  1. http://www.nif.or.jp/eng/graph/M7.gif http://interactions.acm.org/i/XV/wine.jpg
  2. Camille Jordan: http://myoops.fgu.edu.tw/twocw/mit/NR/rdonlyres/Mathematics/18-700Fall-2005/4AC2EE51-AA81-45EA-AB73-1935A7F3BAFC/0/chp_jordan2.jpg Arthur Rimbaud: http://www.stevesilberman.com/celestial/rimbaud/rimbaud.jpg Those dreaming eyes: are the looking upon the same thing?
  3. Avinash Kaushik: &amp;#x201C;Trinity: A Mindset &amp; Strategic Approach&amp;#x201D; (http://www.kaushik.net/avinash/2006/08/trinity-a-mindset-strategic-approach.html)
  4. Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  5. Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  6. Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  7. Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  8. Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  9. Analysis versus synthesis comes from Lindsay Ellerby article in UXMatters: http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php
  10. Bottom line and &amp;#x201C;top line&amp;#x201D;
  11. This is how I would do it
  12. This is how I&amp;#x2019;d do it
  13. This is how I&amp;#x2019;d do it
  14. This is how my co-author would do it
  15. Start with KPI, then add data
  16. Feedback loop
  17. Start with KPI, then add data
  18. The reports are often as far as we go But they&amp;#x2019;re often useless &amp;#x2022; No deep, custom analysis (top-down) &amp;#x2022; No exploratory data analysis (bottom-up)
  19. The reports are often as far as we go But they&amp;#x2019;re often useless &amp;#x2022; No deep, custom analysis (top-down) &amp;#x2022; No exploratory data analysis (bottom-up)
  20. &amp;#x201C;The center can not hold!&amp;#x201D; You&amp;#x2019;ll notice this isn&amp;#x2019;t a canned report This all means putting pressure on commercial analytics apps to change
  21. &amp;#x201C;The center can not hold!&amp;#x201D; You&amp;#x2019;ll notice this isn&amp;#x2019;t a canned report This all means putting pressure on commercial analytics apps to change
  22. &amp;#x201C;The center can not hold!&amp;#x201D; You&amp;#x2019;ll notice this isn&amp;#x2019;t a canned report This all means putting pressure on commercial analytics apps to change
  23. &amp;#x201C;The center can not hold!&amp;#x201D; You&amp;#x2019;ll notice this isn&amp;#x2019;t a canned report This all means putting pressure on commercial analytics apps to change
  24. Start with KPI, then add data
  25. you can do this, regardless of how you feel about data note that it&amp;#x2019;s in Excel
  26. you can do this, regardless of how you feel about data note that it&amp;#x2019;s in Excel
  27. you can do this, regardless of how you feel about data note that it&amp;#x2019;s in Excel
  28. Yes, data can tell stories And sometimes stories make a better case than reports
  29. Actually, both sides (Bowman&amp;#x2019;s and Google&amp;#x2019;s) are valid But while it won&amp;#x2019;t always be possible to combine WA and UX (in some orgs, one perspective is far dominant--e.g., engineering at Google), you&amp;#x2019;ve got to come halfway But... weren&amp;#x2019;t Page and Brin designers of a sort when they started out?
  30. Actually, both sides (Bowman&amp;#x2019;s and Google&amp;#x2019;s) are valid But while it won&amp;#x2019;t always be possible to combine WA and UX (in some orgs, one perspective is far dominant--e.g., engineering at Google), you&amp;#x2019;ve got to come halfway But... weren&amp;#x2019;t Page and Brin designers of a sort when they started out?