Web Analytics: A Measurement System of Website

1,189 views

Published on

Prezentacija "Web Analytics: A Measurement System of Website" koju je Dave Rogers
održao na konferenciji iFront 9. juna 2009. godine u Beogradu.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,189
On SlideShare
0
From Embeds
0
Number of Embeds
35
Actions
Shares
0
Downloads
55
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • Web Analytics: A Measurement System of Website

    1. 1. Web Analytics: A Measurement System of Website Performance for Optimization Dave Rogers Former Senior Web Analytics Researcher @ PayPal
    2. 2. Who is Dave Rogers? <ul><li>Almost a decade in research and internet </li></ul><ul><li>1/3 Researcher, 1/3 Analyst, 1/3 Psychologist </li></ul><ul><li>Education: MA Stats/Research; MBA </li></ul><ul><li>Industry experience in academia, agency work, e-commerce, finance and social networking </li></ul><ul><li>Was @ PayPal for over a year </li></ul>Source: ‘mutt’ idea stolen from Angel Morales; LightsOut Marketing answer: a mutt
    3. 3. Discussion <ul><li>Balkans & the Internet </li></ul><ul><li>Web Optimization Ecosystem </li></ul><ul><li>Website Optimization </li></ul><ul><li>Web Analytics Resources </li></ul><ul><li>Q&A </li></ul>
    4. 4. Balkans & the Internet <ul><li>Great time to get started! </li></ul><ul><ul><li>Tools are all figured out (consolidations) </li></ul></ul><ul><ul><li>Many agencies now (available experts!) </li></ul></ul><ul><li>Compete on analytics (hunt vs. get hunted) </li></ul>-4 -2 0 -3 ? ? The Balkans
    5. 5. Web Optimization Ecosystem
    6. 6. Web Site Optimization Ecosystem Source: Web Analytics Demystified (www.webanalyticsdemystified.com) WA Web Analytics VOC Voice of the Consumer TESTING Conversion Optimization CEM Customer Experience Management
    7. 7. Web Analytics <ul><li>The WHAT: </li></ul><ul><li>Visits / Visitors </li></ul><ul><li>Bounce/ Exit rates </li></ul><ul><li>Marketing tasks </li></ul><ul><li>Acquisition </li></ul><ul><ul><li>SEO and SEM </li></ul></ul><ul><ul><li>Affiliate Networks </li></ul></ul><ul><ul><li>Comparison Shopping </li></ul></ul><ul><li>Retention </li></ul><ul><ul><li>Email </li></ul></ul><ul><ul><li>Affiliate Networks </li></ul></ul><ul><ul><li>Comparison Shopping </li></ul></ul>SEM SEO WA VOC TESTING CEM
    8. 8. Voice of the Consumer <ul><li>The WHY: </li></ul><ul><li>Measure customer satisfaction continuously </li></ul><ul><li>Identify what’s driving customer satisfaction and what will have the greatest impact on behavior and financial performance </li></ul>WA VOC TESTING CEM
    9. 9. Customer Experience Management <ul><li>REPLAYS: </li></ul><ul><li>Tealeaf enables you to see your customers' actual online experiences, analyze their motivations and ultimately, gain insight as to why abandonment or other site actions occur </li></ul>WA VOC TESTING CEM
    10. 10. Testing <ul><li>OPTIMIZING: </li></ul>Typical consumer engagement model from Marketing team perspective: Now, who owns the conversion task? Acquisition Marketing Retention Marketing WA VOC TESTING CEM Acquisition Marketing Conversion Optimization Retention Marketing
    11. 11. Website Optimization
    12. 12. <ul><li>What is MV Testing? </li></ul><ul><li>Framework in 8 Steps </li></ul><ul><li>Case Study </li></ul><ul><li>Advanced Testing Techniques </li></ul><ul><li>Ok… now how do I start? </li></ul><ul><li>Bonus: Elements to Start Testing Today! </li></ul>Optimization Discussion
    13. 13. What is MV Testing? … Why? What is average conversion rate?
    14. 14. <ul><li>What is MV Testing? </li></ul><ul><li>Framework in 8 Steps </li></ul><ul><li>Case Study </li></ul><ul><li>Advanced Testing Techniques </li></ul><ul><li>Ok… now how do I start? </li></ul><ul><li>Bonus: Elements to Start Testing Today! </li></ul>
    15. 15. Website Testing Framework <ul><li>Split traffic </li></ul>All Traffic Version A Version B Technology
    16. 16. Website Testing Framework <ul><li>Split traffic </li></ul><ul><li>Display content </li></ul>www.books.com www.books.com Version A (Original, Control) Version B (Test)
    17. 17. Website Testing Framework <ul><li>Split traffic </li></ul><ul><li>Display content </li></ul><ul><li>Log which version </li></ul>
    18. 18. Website Testing Framework <ul><li>Split traffic </li></ul><ul><li>Display content </li></ul><ul><li>Log which version </li></ul><ul><li>Log success event </li></ul>
    19. 19. Website Testing Framework <ul><li>Split traffic </li></ul><ul><li>Display content </li></ul><ul><li>Log which version </li></ul><ul><li>Log success event </li></ul><ul><li>Measure results </li></ul>Conversion Rate = # of success events # of ‘starts’ / version
    20. 20. Website Testing Framework <ul><li>Split traffic </li></ul><ul><li>Display content </li></ul><ul><li>Log which version </li></ul><ul><li>Log success event </li></ul><ul><li>Measure results </li></ul><ul><li>Use stats! </li></ul>
    21. 21. Website Testing Framework <ul><li>Split traffic </li></ul><ul><li>Display content </li></ul><ul><li>Log which version </li></ul><ul><li>Log success event </li></ul><ul><li>Measure results </li></ul><ul><li>Use stats! </li></ul><ul><li>Decide winner </li></ul>
    22. 22. Website Testing Framework <ul><li>Split traffic </li></ul><ul><li>Display content </li></ul><ul><li>Log which version </li></ul><ul><li>Log success event </li></ul><ul><li>Measure results </li></ul><ul><li>Use stats! </li></ul><ul><li>Decide winner </li></ul><ul><li>Push winner live </li></ul>Rinse & Repeat !
    23. 23. <ul><li>What is MV Testing? </li></ul><ul><li>Framework in 8 Steps </li></ul><ul><li>Case Study </li></ul><ul><li>Advanced Testing Techniques </li></ul><ul><li>Ok… now how do I start? </li></ul><ul><li>Bonus: Elements to Start Testing Today! </li></ul>
    24. 24. <ul><li>Test Plan : Improve the efficiency of the product detail page </li></ul><ul><li>Success Metric : Percent of product page views that result in shopping cart “puts” </li></ul><ul><li>The test consisted of 8 elements, or variables, that can take on 18 different values. The resulting search space contains 576 unique combinations of variable values (‘recipes’). </li></ul>‘ Small changes often have huge impact ’
    25. 25. 8 Elements <ul><li>Page Order </li></ul>Fold Line
    26. 26. 8 Elements <ul><li>Page Order </li></ul><ul><li>Title </li></ul>
    27. 27. 8 Elements <ul><li>Page Order </li></ul><ul><li>Title </li></ul><ul><li>Price Presentation </li></ul>
    28. 28. 8 Elements <ul><li>Page Order </li></ul><ul><li>Title </li></ul><ul><li>Price Presentation </li></ul><ul><li>GiftBox Presentation </li></ul>
    29. 29. 8 Elements <ul><li>Page Order </li></ul><ul><li>Title </li></ul><ul><li>Price Presentation </li></ul><ul><li>GiftBox Presentation </li></ul><ul><li>Continue Shopping </li></ul>
    30. 30. 8 Elements <ul><li>Page Order </li></ul><ul><li>Title </li></ul><ul><li>Price Presentation </li></ul><ul><li>GiftBox Presentation </li></ul><ul><li>Continue Shopping </li></ul><ul><li>Button Text </li></ul>
    31. 31. 8 Elements <ul><li>Page Order </li></ul><ul><li>Title </li></ul><ul><li>Price Presentation </li></ul><ul><li>GiftBox Presentation </li></ul><ul><li>Continue Shopping </li></ul><ul><li>Button Text </li></ul><ul><li>Button Shape </li></ul>
    32. 32. 8 Elements <ul><li>Page Order </li></ul><ul><li>Title </li></ul><ul><li>Price Presentation </li></ul><ul><li>GiftBox Presentation </li></ul><ul><li>Continue Shopping </li></ul><ul><li>Button Text </li></ul><ul><li>Button Shape </li></ul><ul><li>Button Color </li></ul>
    33. 33. Results of the Multivariate Test <ul><li>The purpose of the test was to identify which of the 576 unique versions (‘recipes’) of the product-detail page was the best (MV), and to determine how much better it was than the original. </li></ul><ul><li>After the data was collected and analyzed, a new and improved version of the product-detail page was identified and compared head-to-head against the original (essentially an A/B test). </li></ul>$5.25 M Incremental Annual Revenue The new and improved page performed 4.4% better than the original. (with 99% confidence)
    34. 34. <ul><li>What is MV Testing? </li></ul><ul><li>Framework in 8 Steps </li></ul><ul><li>Case Study </li></ul><ul><li>Advanced Testing Techniques </li></ul><ul><li>Ok… now how do I start? </li></ul><ul><li>Bonus: Elements to Start Testing Today! </li></ul>
    35. 35. Testing outside of Web Pages <ul><li>Landing Pages </li></ul><ul><li>Banners </li></ul><ul><ul><li>Person v. product </li></ul></ul><ul><ul><li>Different value propositions </li></ul></ul><ul><li>Emails </li></ul><ul><ul><li>Product v. brand </li></ul></ul><ul><ul><li>Category sale v. sale across categories </li></ul></ul><ul><ul><li>Discount v. free shipping </li></ul></ul><ul><ul><li>Time-of-day / day-of-week </li></ul></ul>Targeted! Triggered!
    36. 36. Behavior Targeting & ‘Personas’ <ul><li>If like this, then show ___. </li></ul><ul><ul><li>Database construction </li></ul></ul><ul><ul><li>Test & develop business rules </li></ul></ul><ul><ul><li>Segmentation & predictive analytics </li></ul></ul><ul><li>Personas </li></ul><ul><ul><li>Represent different user types and their behavior, attitude, ‘world view’ </li></ul></ul><ul><ul><li>User-centered design </li></ul></ul><ul><ul><li>“ Conversion: What About the Other 98 Percent?” </li></ul></ul>Predictive Modeling!
    37. 37. ‘ Manufacturing’ Testing Structure <ul><li>Build a reporting face </li></ul><ul><ul><li>Alert when sample size reached </li></ul></ul><ul><ul><li>Automated decision making </li></ul></ul><ul><ul><li>Output to Dev Team </li></ul></ul><ul><li>Develop the long-term ‘Program’ </li></ul><ul><ul><li>Share learnings throughout org </li></ul></ul><ul><ul><li>Business unit ‘on the hook’ as driver of continually increased conversions </li></ul></ul>
    38. 38. Testing Opportunities Demographics Psych/Social Web Analytics/ Website Behavior Attitudinal Data ? Customer Support Calls & Outcome Transaction Data (R,F,M) Databases Speaking to One Another
    39. 39. <ul><li>What is MV Testing? </li></ul><ul><li>Framework in 8 Steps </li></ul><ul><li>Case Study </li></ul><ul><li>Advanced Testing Techniques </li></ul><ul><li>Ok… now how do I start? </li></ul><ul><li>Bonus: Elements to Start Testing Today! </li></ul>
    40. 40. Build Yourself or Find a Partner? <ul><li>BUILD </li></ul><ul><li>Long-term cost savings </li></ul><ul><li>Back-end advanced testing structure </li></ul><ul><li>Can build custom monitoring and notification system </li></ul><ul><li>Data easily integrated into web analytics </li></ul><ul><li>PARTNER </li></ul><ul><li>Realize continuous optimization earlier </li></ul><ul><li>Some tools include Professional Services </li></ul><ul><li>Can ‘learn’ from the experts </li></ul>
    41. 41. “ Which pages do I test first?” <ul><li>Use your web analytics tool - think ‘funnel’ </li></ul><ul><ul><li>All pages in the check-out process are important </li></ul></ul><ul><ul><li>Where do visitors leave/bounce? </li></ul></ul><ul><ul><li>What are your best selling products/categories? </li></ul></ul><ul><li>Know your audience </li></ul><ul><ul><li>understand needs and create experience for user </li></ul></ul><ul><ul><li>guest or member/account? </li></ul></ul><ul><ul><li>first-time buyer or frequent buyer? </li></ul></ul>Landing Pages? Product Pages? Home Page?
    42. 42. <ul><li>MV Testing is ‘doable’ </li></ul><ul><li>See above- up on the right </li></ul><ul><li>Think: incremental and continuous </li></ul><ul><li>Many times, a small win builds psychological momentum at the organization for continual testing </li></ul>Wrap-Up: Key Considerations ‘ Small changes often have huge impact ’
    43. 43. ‘ Small changes often have huge impact ’ Asked to many web analytics professionals at the 2008 e-Metrics summit:
    44. 44. <ul><li>What is MV Testing? </li></ul><ul><li>Framework in 8 Steps </li></ul><ul><li>Case Study </li></ul><ul><li>Advanced Testing Techniques </li></ul><ul><li>Ok… now how do I start? </li></ul><ul><li>Bonus: Elements to Start Testing Today! </li></ul>
    45. 45. <ul><li>What are different versions of the value proposition? </li></ul><ul><ul><li>Have they been tested before? </li></ul></ul><ul><ul><li>Can they be seasonal? </li></ul></ul><ul><li>Try various formatting techniques </li></ul><ul><ul><li>Font, u , size, bold , color , centered </li></ul></ul><ul><li>Use of exclamation point! or? </li></ul><ul><li># of words: Many v. less </li></ul><ul><li>Consider language </li></ul><ul><ul><li>colloquial; education level </li></ul></ul>Testing Element: Headlines ‘ Small changes often have huge impact ’
    46. 46. <ul><li>Sale </li></ul><ul><ul><li>% off </li></ul></ul><ul><ul><li>$ off </li></ul></ul><ul><ul><li>Buy1Get1 </li></ul></ul><ul><ul><li>Discount for future purchase </li></ul></ul><ul><li>Coupons </li></ul><ul><li>Can pilot test campaigns! </li></ul>Testing Element: Offers … but what kind of shipping? ‘ Small changes often have huge impact ’ Free shipping is king!!!
    47. 47. <ul><li>Value proposition </li></ul><ul><ul><li>emotional v. logical </li></ul></ul><ul><li>Word choice </li></ul><ul><ul><li>Add to Bag v. Add to Cart; Shop Now v. Order Now </li></ul></ul><ul><li>Color </li></ul><ul><li>Shape </li></ul><ul><li>Size </li></ul><ul><li>Placement </li></ul>Testing Element: Call-to-Action ‘ Small changes often have huge impact ’
    48. 48. <ul><li>% image to text to white space </li></ul><ul><li>Use very similar images in the flow </li></ul><ul><li>Size </li></ul><ul><li>Placement </li></ul><ul><li>Contrast to site look & feel </li></ul><ul><li>Personalize! </li></ul><ul><li>Images of people are different </li></ul><ul><ul><li>Which direction are they facing? </li></ul></ul><ul><ul><li>Where are the eyes focusing? </li></ul></ul>Testing Element: Image ‘ Small changes often have huge impact ’
    49. 49. Common Mistake: Clutter <ul><li>Too much text </li></ul><ul><li>Competing for attention </li></ul><ul><li>Overstimulation </li></ul><ul><li>Rest </li></ul><ul><li>Hype-free zone </li></ul><ul><li>‘ Less is more’ </li></ul>‘ Small changes often have huge impact ’
    50. 50. Web Analytics Resources
    51. 51. Web Analytics Resources <ul><li>BLOG ROLL </li></ul><ul><li>kaushik.net/avinash/ </li></ul><ul><li>blog.webanalyticsdemystified.com/weblog/ </li></ul><ul><li>semphonic.blogs.com/ </li></ul><ul><li>grokdotcom.com/ </li></ul><ul><li>blogs.omniture.com/ </li></ul><ul><li>juiceanalytics.com/weblog/ </li></ul><ul><li>webanalysis.blogspot.com/ </li></ul><ul><li>EVENTS & TRAINING </li></ul><ul><li>webanalyticsassociation.org </li></ul><ul><li>eMetrics Summit </li></ul><ul><li>Web Analytics Wednesdays </li></ul><ul><li>Web Analytics Forum Yahoo Group </li></ul><ul><li>UBC Web Analytics </li></ul><ul><li>X Change Conference: Gary Angel </li></ul><ul><li>Predictive Analytics: Eric Siegel </li></ul>WEB ANALYTICS BOOKS OPTIMIZATION BOOKS
    52. 52. Q&A David Rogers [email_address] LinkedIn.com/in/DRAnalytics Twitter.com/DRAnalytics

    ×