• Like

Not your father's analytics report Mahaffey & Straub, UPA 2011 Atlanta

  • 719 views
Uploaded on

The Home Energy Saver is an action-oriented home energy benchmarking site. The do-it-yourself home energy assessment helps consumers understand and streamline their home energy use through behavioral …

The Home Energy Saver is an action-oriented home energy benchmarking site. The do-it-yourself home energy assessment helps consumers understand and streamline their home energy use through behavioral and structural upgrade recommendations. We present a longitudinal case study describing how we used web-analytics to iteratively benchmark, track and improve the persuasive pull of the site.

More in: Technology , Design , Spiritual
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
719
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
15
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. NOT YOUR FATHERʼS 
WEB ANALYTICS REPORT# David Mahaffey! ! ! Kath Straub!
  • 2. SPECIAL THANKS TO …# Diane Chojnowski! Evan Mills! Social Media Strategist! Staff Scientist
 Usability.org! Berkeley Labs!
  • 3. Practitioner takeaways! •  Define measurable success metrics early! •  Build clear and meaningful KPIs! •  Team with/make use of analytic practitioners! •  Side step the data blizzard! # # Next generation AX# Improve the user journey in real time!
  • 4. Roadmap! •  Practices UX and AX! •  AX tools are difficult! •  Visitor life cycle! •  An example from Home Energy Pros! •  Metrics definitions! •  Persuasion in a Newsletter! •  Persuasion API - Using AX in real time! •  Summary!
  • 5. Practices have complementary processes! Build  KPIs   Collect  Data   Define  Goals   Analyze  Data   Analy:cs  Process   Implement   Test  Alterna:ves  
  • 6. Practices have complementary processes! Build  KPIs   Collect  Data   Define  Goals   Analyze  Data   Analy:cs  Process   Implement   Test  Alterna:ves   Create   Mock-­‐ups     Usability  Tes:ng   Storyboards     Usability  Process   Define  Goals   Analyze  Data   Implement   Revise  and  Retest  
  • 7. Practices have common goals! UX  goals   AX  goals   Understand  describe  why  people  do  what   Show  what  people  actually  do  (sort  of)   they  do    
  • 8. Practices have common goals! UX  goals   AX  goals   Understand  describe  why  people  do  what   Show  what  people  actually  do  (sort  of)   they  do     Con:nuous  improvement   Goal  con:nuous  improvement  requires:     •  Investment  in  the  process   •  A  corporate  champion     Persuade     Explain  and  influence  behavior  toward   Measure  effec:veness  of  site  design   business  goals    
  • 9. Timeline UX and AX! 1980   1990   2000   2010   Internet   WWW   Web  2.0   Events/technology  related  to  UX   What’s  happening  in  prac:ce  of  UX   Events/technology  related  to  AX   What’s  happening  in  prac:ce  of  AX  
  • 10. Timeline UX and AX! 1980   1990   2000   2010   Internet   WWW   Web  2.0   Events/technology  related  to  UX   UX  exist  in  desktop   world  /  un-­‐networked   What’s  happening  in  prac:ce  of  UX   UX  in  so[ware  Cos.  w/   developers   Events/technology  related  to  AX   What’s  happening  in  prac:ce  of  AX  
  • 11. Timeline UX and AX! 1980   1990   2000   2010   Internet   WWW   Web  2.0   Events/technology  related  to  UX   UX  exist  in  desktop   HTML  -­‐  lots  of   world  /  un-­‐networked   sites  no  UX   What’s  happening  in  prac:ce  of  UX   UX  in  so[ware  Cos.  w/   developers   Events/technology  related  to  AX   Log  files   What’s  happening  in  prac:ce  of  AX  
  • 12. Timeline UX and AX! 1980   1990   2000   2010   Internet   WWW   Web  2.0   Events/technology  related  to  UX   UX  exist  in  desktop   HTML  -­‐  lots  of   e-­‐Commerce  need  for   world  /  un-­‐networked   sites  no  UX   UX  heats  up   What’s  happening  in  prac:ce  of  UX   UX  in  various  web   development  orgs   UX  in  so[ware  Cos.  w/   developers   Events/technology  related  to  AX   Log  files   Proto  web   tools   What’s  happening  in  prac:ce  of  AX   AX  owned  by  IT  
  • 13. Timeline UX and AX! 1980   1990   2000   2010   Internet   WWW   Web  2.0   Events/technology  related  to  UX   UX  exist  in  desktop   HTML  -­‐  lots  of   e-­‐Commerce  need  for   world  /  un-­‐networked   sites  no  UX   UX  heats  up   What’s  happening  in  prac:ce  of  UX   UX  in  various  web   development  orgs   UX  in  so[ware  Cos.  w/   UX  begins  rela:onship  w/   developers   marke:ng   Events/technology  related  to  AX   Log  files   Proto  web   Beginning  of  current   tools   web  tools  1.0   What’s  happening  in  prac:ce  of  AX   AX  owned  by  IT   Marke:ng  has  approach/ avoidance  rela:onship  w/  AX    
  • 14. Where do these practices live, who do they serve?! IT   Marke:ng   Both  prac:ces  are  most  associated  with  IT     Both  prac:ces  goals  serve  marke:ng   That  is  where  the  building  and  implementa:on   Audience  of  interest  for  data  is  marke:ng       •  UX  historically  wanted  to  influence     •  Focus  groups  don’t  work  well  for  assessing   •  AX  needed  people  with  these  skills     web  site  design/effec:veness   •  UX  wants  to  know  why  goals  are  not  met     •  AX  focus  on  what  are  people  doing  on  site   (sort  of)   Marke:ng  has  approach/avoidance  rela:onship  with    AX    
  • 15. Where do these practices live, who do they serve?!
  • 16. Roadmap! •  Practices UX and AX! •  AX tools are difficult! •  Visitor life cycle! •  An example from Home Energy Pros! •  Metrics definitions! •  Persuasion in a Newsletter! •  Persuasion API – Using AX in real time! •  Summary!
  • 17. Analytic tools produce data and reports!
  • 18. Analytic tools produce data and reports!
  • 19. Analytic tools produce data and reports!
  • 20. Analytic tools produce data and reports!
  • 21. Analytic tools produce data and reports!
  • 22. Analytic tools produce data and reports!Many  reports.    Li_le  ac:onable  informa:on  ...  
  • 23. Roadmap! •  Practices UX and AX! •  AX tools are difficult! •  Visitor life cycle! •  An example from Home Energy Pros! •  Metrics definitions! •  Persuasion in a Newsletter! •  Persuasion API - Using AX in real time! •  Summary!
  • 24. Visitor life-cycle – Behaviors of interest!1.  Enter   Capture     Entry   Page    
  • 25. Visitor life-cycle – Behaviors of interest!1.  Enter2.  Explore   Capture     Entry   Engaged   Browse  &           Page   Explore     Unengaged  
  • 26. Visitor life-cycle – Behaviors of interest!1.  Enter2.  Explore > Persuasion Opportunity   Capture     Entry   Engaged   Browse  &       Persuasion       Page   Explore   opportunity     Unengaged  
  • 27. Visitor life-cycle – Behaviors of interest!1.  Enter2.  Explore > Persuasion Opportunity3.  Enter Conversion Path   Capture     Entry   Engaged   Browse  &       Persuasion   Conversion  Path   Begin       Page   Explore   opportunity   Conversion         Unengaged   Unpersuaded  
  • 28. Overview of the visitor life-cycle1.  Enter2.  Explore > Persuasion Opportunity3.  Enter Conversion Path > Commit/Convert! Capture     Entry   Engaged   Browse  &       Persuasion   Conversion  Path   Begin   Commit  to   Converted       Page   Explore   opportunity   Conversion   Convert         Unengaged   Unpersuaded   Unconverted  
  • 29. Key transition points in the visitor life-cycle! 1.  Capture 2.  Engagement 3.  Persuasion 4.  Conversion  
  • 30. Key transition points in the visitor life-cycle! 1.  Capture 2.  Engagement 3.  Persuasion 4.  Conversion   Capture     Entry   Engaged   Browse  &       Persuasion   Conversion  Path   Begin   Commit  to   Converted       Page   Explore   opportunity   Conversion   Convert         Unengaged   Unpersuaded   Unconverted  
  • 31. Key transition points in the visitor life-cycle! 1.  Capture 2.  Engagement 3.  Persuasion 4.  Conversion   Capture     Entry   Engaged   Browse  &       Persuasion   Conversion  Path   Begin   Commit  to   Converted       Page   Explore   opportunity   Conversion   Convert         Unengaged   Unpersuaded   Unconverted  
  • 32. Key transition points in the visitor life-cycle! 1.  Capture 2.  Engagement 3.  Persuasion 4.  Conversion   Capture     Entry   Engaged   Browse  &       Persuasion   Conversion  Path   Begin   Commit  to   Converted       Page   Explore   opportunity   Conversion   Convert         Unengaged   Unpersuaded   Unconverted  
  • 33. Key transition points in the visitor life-cycle! 1.  Capture 2.  Engagement 3.  Persuasion 4.  Conversion   Capture     Entry   Engaged   Browse  &       Persuasion   Conversion  Path   Begin   Commit  to   Converted       Page   Explore   opportunity   Conversion   Convert         Unengaged   Unpersuaded   Unconverted  
  • 34. Roadmap! •  Practices UX and AX! •  AX tools are difficult! •  Visitor life cycle! •  An example from Home Energy Pros! •  Metrics definitions! •  Persuasion in a Newsletter! •  Persuasion API - Using AX in real time! •  Summary!
  • 35. Home Energy Saver Suite! For  Consumers   For  Home  Energy  Professionals   Home  Energy   Home  Energy   Home  Energy   Social  Community   Efficiency     Efficiency     Scoring  Tool   Site  for  Home  Energy   Calculator   Calculator   (DOE-­‐ARRA)   Professionals  (NING)  
  • 36. Home Energy Saver Suite! For  Consumers   For  Home  Energy  Professionals   Home  Energy   Home  Energy   Home  Energy   Social  Community   Efficiency     Efficiency     Scoring  Tool   Site  for  Home  Energy   Calculator   Calculator   (DOE-­‐ARRA)   Professionals  (NING)  
  • 37. Home Energy Pros: Capture opportunity
  • 38. Home Energy Pros: Capture opportunity Callouts   Capture     Entry   Page    
  • 39. Home Energy Pros: Engaged visit Callouts  for  more   engagement   opportuni:es   Capture   Engaged     Entry   Browse  &   Page   Explore     Unengaged  
  • 40. Home Energy Pros: Engaged visit Engagement  /   persuasion   opportuni:es     Capture     Entry   Engaged   Browse  &       Persuasion       Page   Explore   opportunity     Unengaged   Unpersuaded  
  • 41. Home Energy Pros: Committed visit Begin  conversion   path  
  • 42. Home Energy Pros: Committed visit Begin  conversion   path   Capture     Entry   Engaged   Browse  &       Persuasion   Conversion  Path   Begin       Page   Explore   opportunity   Conversion     Unengaged   Unpersuaded  
  • 43. Home Energy Pros: Committed visit Con:nue  the   conversion  path   Capture     Entry   Engaged   Browse  &       Persuasion   Conversion  Path   Begin   Commit  to       Page   Explore   opportunity   Conversion   Convert         Unengaged   Unpersuaded   Unconverted  
  • 44. Home Energy Pros: Converted visit Complete   conversion  path   Capture     Entry   Engaged   Browse  &       Persuasion   Conversion  Path   Begin   Commit  to   Converted       Page   Explore   opportunity   Conversion   Convert         Unengaged   Unpersuaded   Unconverted  
  • 45. Roadmap! •  Practices UX and AX! •  AX tools are difficult! •  Visitor life cycle! •  An example from Home Energy Pros! •  Metrics definitions! •  Persuasion in a Newsletter! •  Persuasion API – Using AX in real time! •  Summary!
  • 46. Metrics definitions Metrics are divided into two parts! 1.  Conversion! –  Traditional conversion! –  Engaged conversion! –  Committed conversion! 2.  Indices of engagement! –  Persuasion index! •  Pages are associated with conversion opportunities ! –  Contribution index ! •  The contribution of each page to a conversion!
  • 47. Metric definitions: Conversion types • Tradi:onal  conversion   • Engaged  conversion   • Commi_ed  conversion     Sales   Learn  visitor’s  interests   Networking  Tradi&onal  Conversion   Loyalty   Engaged  Conversion   Commi2ed  Conversion   Site       Visits     Browse Choose a Start   Commit  to   Converted   Enter  Site   & conversion     Conversion   Convert     Explore option Visitor   Unengaged   Unpersuaded   Unconverted  Tradi:onal  Conversion   Engaged  Conversion   Commi_ed  Conversion  =Convα/Vα   =Convα/Vα-­‐VU     =Convw/inpath  /Vw/inpath      
  • 48. Metric definitions: Traditional conversion • Tradi&onal  conversion  =  Total  conversions                                                                                                Total  site  visits  Tradi&onal  Conversion   Site       Browse Choose a Visits   Enter  Site   & conversion Start   Commit  to   Converted       Conversion   Convert     Explore option Visitor   Unengaged   Unpersuaded  Tradi:onal  Conversion  =Convα/Vα  
  • 49. Metric definitions: Engaged conversion • Engaged  conversion  =  Total  conversions                                                                                      Total  site  visits  -­‐  bounce  Tradi&onal  Conversion   Engaged  Conversion   Site       Browse Choose a Visits   Enter  Site   & conversion Start   Commit  to   Converted       Conversion   Convert     Explore option Visitor   Unengaged   Unpersuaded  Tradi:onal  Conversion   Engaged  Conversion  =Convα/Vα   =Convα/Vα-­‐VU    
  • 50. Metric definitions: Committed conversion • Commi3ed  conversion  =  Total  sub-­‐type  conversions                Registra:on                                                      Total  visits  to  that  sub-­‐type s         •                                         Invite    a  priend        v  ideo                                      conversion  page(s)   •                 f                         •  Upload   icture,   •  Post  comment  or  blog   •  Print  or  download  from  site  Tradi&onal  Conversion   Engaged  Conversion   Commi2ed  Conversion   Site       Browse Choose a Visits   Enter  Site   & conversion Start   Commit  to   Converted       Conversion   Convert     Explore option Visitor   Unengaged   Unpersuaded   Unconverted  Tradi:onal  Conversion   Engaged  Conversion   Commi_ed  Conversion  =Convα/Vα   =Convα/Vα-­‐VU     =Convw/inpath  /Vw/inpath      
  • 51. Metric definitions: Committed conversion• Commi3ed  conversion  =  Total  sub-­‐type  conversions                Registra:on                                                      Total  visits  to  that  sub-­‐type s         •                                         Invite    a  priend        v  ideo                                      conversion  page(s)   •                 f                         •  Upload   icture,   •  Post  comment  or  blog   •  Print  or  download  from  site  
  • 52. Conversion metrics for Home Energy Pros 100   Conversion  Rate  (%)   80   60   40   20   3.4  %   0   Tradi:onal  conver:ons   Engaged  conversions   Commi_ed   conversions  blogs  Tradi&onal  Conversion   Site       Browse Choose a Visits   Enter  Site   & conversion Start   Commit  to   Converted       Conversion   Convert     Explore option
  • 53. Conversion metrics for Home Energy Pros 100   Conversion  Rate  (%)   80   60   40   20   9.4  %   3.4  %   0   Tradi:onal  conver:ons   Engaged  conversions   Commi_ed   conversions  blogs  Tradi&onal  Conversion   Engaged  Conversion   Site       Browse Choose a Visits   Enter  Site   & conversion Start   Commit  to   Converted       Conversion   Convert     Explore option
  • 54. Conversion metrics for Home Energy Pros 100   Conversion  Rate  (%)   80   74.3  %   60   40   20   9.4  %   3.4  %   0   Tradi:onal  conver:ons   Engaged  conversions   Commi_ed   conversions  blogs  Tradi&onal  Conversion   Engaged  Conversion   Commi2ed  Conversion   Site       Browse Choose a Visits   Enter  Site   & conversion Start   Commit  to   Converted       Conversion   Convert     Explore option
  • 55. Metrics definitions Metrics are divided into two parts! 1.  Conversion! –  Traditional conversion! –  Engaged conversion! –  Committed conversion! 2.  Indices of engagement! –  Persuasion index! •  Pages are associated with conversion opportunities ! –  Contribution index! •  The contribution of each page to a conversion!
  • 56. Metric definitions: Persuasion index • Persuasion  index  =  Commi_ed  visits  /  Visits  to  a  defined                                                                                                                                                    persuasion  page   Persuasion  Index  Site       Browse Choose aVisits   Enter  Site   & conversion Start   Commit  to   Converted       Conversion   Convert     Explore option Visitor   Unengaged   Unpersuaded   Unconverted  
  • 57. What is a persuasion index? A  Persuasion  Index  is  a  measure  of  the  sites  persuasive  content   •  This  index  is  a  gauge  of  how  effec:vely  a  page  persuades  site   visitors  to  enter  a  conversion  path   •  The  Persuasion  Index  is  a  measure  of  a  site  or  page s   persuasive  content   Persuasion  Index       Browse Start   Commit  to   &     Conversion   Convert   Explore     Unpersuaded   Unconverted   59
  • 58. What is a persuasion index? Persuasion  Index       Browse Start   Commit  to   &     Conversion   Convert   Explore     Unpersuaded   Unconverted   60
  • 59. What is a persuasion index? Persuasion  Index       Browse Start   Commit  to   &     Conversion   Convert   Explore     Unpersuaded   Unconverted  
  • 60. What is a persuasion index? Persuasion  Index       Browse Start   Commit  to   &     Conversion   Convert   Explore     Unpersuaded   Unconverted  
  • 61. Metrics definitions Metrics are divided into two parts! 1.  Conversion! –  Traditional conversion! –  Engaged conversion! –  Committed conversion! 2.  Indices of moving from engagement to conversion! –  Persuasion index! •  Pages are associated with conversion opportunities ! –  Contribution index! •  The contribution of each page to a conversion!
  • 62. What is a contribution index? • Contribu&on  index  =  A  tally  of  the  pages  that  contribute  to   conversion  
  • 63. What is a contribution index? • Contribu&on  index  =  A  tally  of  the  pages  that  contribute  to   conversion   1 +1 +1 +1 +1
  • 64. What is a contribution index? • Contribu&on  index  =  A  tally  of  the  pages  that  contribute  to   conversion   1 +1
  • 65. What is a contribution index? • Contribu&on  index  =  A  tally  of  the  pages  that  contribute  to   conversion   1 + 1 + 1 + 1 + 1 x2 + 1 + 1 + 1 + 1 + 1 Same page
  • 66. What is a contribution index? =1 1 +1 +1 +1 = 4 x2 Same page
  • 67. Roadmap! •  Practices UX and AX! •  AX tools are difficult! •  Visitor life cycle! •  An example from Home Energy Pros! •  Metrics definitions! •  Persuasion in a Newsletter! •  Persuasion API – Using AX in real time! •  Summary!
  • 68. Persuasion example: Using newsletter Capture     Entry   Engaged   Browse  &       Persuasion   Conversion  Path   Begin   Commit  to   Converted       Page   Explore   opportunity   Conversion   Convert         Unengaged   Unpersuaded   Unconverted  
  • 69. Persuasion example: Using newsletter Capture     Entry   Engaged   Browse  &       Persuasion   Conversion  Path   Begin   Commit  to   Converted       Page   Explore   opportunity   Conversion   Convert         Unengaged   Unpersuaded   Unconverted  
  • 70. Page  Views   0   50   100   150   200   250   2-­‐Apr   3-­‐Apr   4-­‐Apr   5-­‐Apr   6-­‐Apr   7-­‐Apr   8-­‐Apr   9-­‐Apr  10-­‐Apr  11-­‐Apr  12-­‐Apr  13-­‐Apr  14-­‐Apr  15-­‐Apr  16-­‐Apr  17-­‐Apr  18-­‐Apr  19-­‐Apr  20-­‐Apr  21-­‐Apr  22-­‐Apr  23-­‐Apr  24-­‐Apr  25-­‐Apr  26-­‐Apr   Engaged  Visits  From  Newsle3er  27-­‐Apr  28-­‐Apr  29-­‐Apr   Engagement: Linking from newsletter30-­‐Apr  1-­‐May   5  Blog  /25  Links   4  Blogs/22  Links   3  Blogs/19  Links   2  Blogs/30  Links   1  Blog  link/2  Links  
  • 71. Engagement: Twitter Tweets   associated  with   the  newsle_er  
  • 72. Engagement: Facebook Facebook  posts   associated  with   the  newsle_er  
  • 73. Roadmap! •  Practices UX and AX! •  AX tools are difficult! •  Visitor life cycle! •  An example from Home Energy Pros! •  Metrics definitions! •  Persuasion in a Newsletter! •  Persuasion API – Using AX in real time! •  Summary!
  • 74. Beyond A/B testing-Using click behaviors to tailor content! Based on work by Cialdini! •  Social proof! •  Scarcity! •  Authority! A  number  of  e-­‐commerce  sites  already  use  Sales  Strategies  to  influence   consumers  buying  decision.  For  example,  they  list  the  “most  popular”   products.     We  pick  it  up  where  they  le[  it.  www.persuasionapi.com  
  • 75. Beyond A/B testing - Using click behaviors to tailor content! Based on work by Cialdini! •  Social proof! •  Scarcity! •  Authority! Based on Cialdini! •  Social proof ! •  Scarcity! •  Authority! Products  can  be  promoted  using  different  Sales  Strategies:  e.g.  Bestsellers   (Social  Proof),  Almost  out  of  stock  (Scarcity),  Recommended  by  (Authority),     etc.  etc.  www.persuasionapi.com  
  • 76. Beyond A/B testing - Using click behaviors to tailor content! Customer  A  sees  most  popular  des:na:on   Customer  B  sees  best  seller  list     most  popular   des:na:on   bestsellers   Persuasion  API  captures  which  strategy  works  best  by  INDIVIDUAL,  and   presents  future  content  using  that  informa:on.  www.persuasionapi.com  
  • 77. Roadmap! •  Practices UX and AX! •  AX tools are difficult! •  Visitor life cycle! •  An example from Home Energy Pros! •  Metrics definitions! •  Persuasion in a Newsletter! •  Persuasion API! •  Summary!
  • 78. Practitioner takeaways! •  Define measurable success metrics early! •  Build clear and meaningful KPIs! –  Engagement! –  Persuasion ! –  Committed conversion! •  Team with/make use of analytic practitioners! •  Side step the data blizzard! •  Use metrics data to ! –  Refine pages within task flows! –  Identify what persuades! # Next generation analytics# Improve the user journey in real time: Tailor content to individualsʼ (click) previous preferences and behaviors!
  • 79. !! ! !Questions?! !!