www.ivantage.co.uk




                                    Presentation to J P Morgan
                                    ...
www.ivantage.co.uk


     Timings and housekeeping
        Start 09:30
        Break 11:00-11:15
        Lunch 12:30-13...
www.ivantage.co.uk


     Course Overview
           Session   1   –    Goals, conversion points and landing pages
     ...
www.ivantage.co.uk


     Introductions
           First
            Me (Matt Trimmer) & ivantage overview
           I...
www.ivantage.co.uk


     Me
           Founder, Principal Consultant and Managing
            Director of ivantage
     ...
www.ivantage.co.uk


     About ivantage

        Established in 2002
      Independent
      36rd Ranked UK Agency acc...
www.ivantage.co.uk


     Urchin and Google, March 28th 2005

        Urchin on Demand > Google Analytics (November 2005)...
www.ivantage.co.uk


     What we do
     We help grow online businesses by
           Generating web site traffic throu...
www.ivantage.co.uk


     How we do it
       We help grow online businesses by generating, analyzing and
        convert...
www.ivantage.co.uk


     Our vision




       To be the Internet’s leading performance-driven Internet
        marketin...
www.ivantage.co.uk


     Professional services
        Traffic generation
               Search Engine Marketing
      ...
www.ivantage.co.uk


     Training (Public or private on-site)
       Google Analytics
             Google Analytics: In...
www.ivantage.co.uk


     Who we do it for – multi-channel retail




                                                    ...
www.ivantage.co.uk


     Who we do it for – business to business




                                                 14
...
www.ivantage.co.uk


     Who we do it for – Non-Governmental
     Organisations and Governmental Organisations




      ...
www.ivantage.co.uk


     Introductions – your turn!

              You
                Your role
                Your ...
www.ivantage.co.uk


     Google Website Optimiser
           Landing page testing and optimisation tool
           Beta...
www.ivantage.co.uk


     Questions we need to ask ourselves
           What should we test for?
           Which landin...
www.ivantage.co.uk


     What we should test for?
                 The three business objectives
                 Websi...
www.ivantage.co.uk


     The big three business goals?
                Raise revenue
                Lower costs
      ...
www.ivantage.co.uk


     Key Performance Indicators (KPIs)
        A metric
              Miles per Hour (mph)
       ...
www.ivantage.co.uk


     KPI Examples

                         Objective                       Metric               Affe...
www.ivantage.co.uk


     Exercise – Your KPIs and goals
        What are your KPIs for your web
         business?
     ...
www.ivantage.co.uk


     KPIs for your website should include
              Outcome focused events
                “Goa...
www.ivantage.co.uk


     Goals
        Goals should be map back to business objectives
              The commercial goa...
www.ivantage.co.uk


     The importance of goals for you
        Commercially justify your work
        Focus your work...
www.ivantage.co.uk


     The importance of goals for you
        Goals map to business metrics
              Enable you...
www.ivantage.co.uk


     Goals and funnels
        You can use Regular Expressions (RegEx) to gain insight into the funn...
www.ivantage.co.uk


     Goal and funnel behaviour




                                       29
Wednesday, 29 September ...
www.ivantage.co.uk


     Test conversion points not the rate!


                                        Assume some
     ...
www.ivantage.co.uk


     Test pages and conversion page



                                      Test Page




          ...
www.ivantage.co.uk


     What we should test for?
                 Conversions points that lead to
                 Web...
www.ivantage.co.uk


     Exercise
        What are the goals for your website?
            Are you recording them?
    ...
www.ivantage.co.uk


     Which landing pages to test?
           Landing pages by highest bounce rate?
            Land...
www.ivantage.co.uk


     Which landing pages to test?
                 Landing pages that send significant traffic to
   ...
www.ivantage.co.uk


     Exercise
        Can we find highly traffic landing pages?
            Can we map out conversio...
www.ivantage.co.uk


     What to test?
           What’s your gut telling you about your pages?
           What are you...
www.ivantage.co.uk


     What to test?
                 Presentation of offers on
                 Landing pages that s...
www.ivantage.co.uk


     Exercise
      Can you define some alternative presentations for your
       selected landing pa...
www.ivantage.co.uk


     Lunch time




                                     40
Wednesday, 29 September 2010
www.ivantage.co.uk


     How does GWO work?




                                     41
Wednesday, 29 September 2010
www.ivantage.co.uk


     How should we test - two methods
        Simple A/B split testing
            Pros
           ...
www.ivantage.co.uk


     A/B Testing
                                   Testing Pages

     Control script
              ...
www.ivantage.co.uk


     How GWO works
                                    Website




                                A
...
www.ivantage.co.uk


     GWO decides which page to serve




                                        B




              ...
www.ivantage.co.uk


     GA cookies

        UTMa - Unique visitor cookie, lasts for 2 years from set or update on
      ...
www.ivantage.co.uk


     The ___utm.gif request from the ga.js to
     Google’s servers
     __utm.gif?utmwv=6.3&utmn=10...
www.ivantage.co.uk


     What are “Fortune” Cookies?
     •    Text files sent to browsers by websites
         •    1ST o...
www.ivantage.co.uk


     Persistent UTMx (3) cookies
                     Respond with Page A or B
   Browser            ...
www.ivantage.co.uk


     Weaknesses of cookies?
     • Machine dependant
         • As close as we can get to a person
  ...
www.ivantage.co.uk


     A/B split test set-up - exercise
        Visit your website or
            http://www.ivantage...
www.ivantage.co.uk


     A/B split test set-up
        Original page
            Control script at top of page
        ...
www.ivantage.co.uk


     A/B split test set-up - settings
        % traffic to see experimental pages
        Selected ...
www.ivantage.co.uk


     A/B split test set-up - duration
        Start your test
        Test never stop until you sto...
www.ivantage.co.uk


     A/B split test reporting
        2-3 hours delay
            Original and variations




     ...
www.ivantage.co.uk


     A/B split test reporting - key metric
        Estimated conversion rate
            Forecasted...
www.ivantage.co.uk


     A/B split test reporting - other metrics
        Conv/Visitors
            Conversion numbers
...
www.ivantage.co.uk


     A/B split test reporting - inconclusive
        No clear winner
            Pointlessness of t...
www.ivantage.co.uk


     After A/B testing?
        Test again!
        Test next conversion point
        Use multiva...
www.ivantage.co.uk


     Multivariate test set-up - exercise
        Visit your website or
            Clear cookies (F...
www.ivantage.co.uk


     Multivariate Testing

                                    Testing Page
 Control script
         ...
www.ivantage.co.uk


     Multivariate Testing

                                    Testing Page
 Control script
         ...
www.ivantage.co.uk


     How GWO works




                                 Testing Page




                            ...
www.ivantage.co.uk


     How GWO works




                                 Testing Page




                            ...
www.ivantage.co.uk


     Multivariate test set-up
        Original page
            Control script at top of page
     ...
www.ivantage.co.uk


     Multivariate test set-up - settings
        % traffic to see experimental pages?
        Selec...
www.ivantage.co.uk


     Multivariate test reporting
        2-3 hours delay
        Combinations and Page Sections
   ...
www.ivantage.co.uk


     Multivariate test set-up - duration
        Start your test
        Tests never stop until you...
www.ivantage.co.uk


     Multivariate test reporting - page sections
        First to populate with data




           ...
www.ivantage.co.uk


     Multivariate test reporting - key metric
        Relevance rating
            The Page Section...
www.ivantage.co.uk


     Multivariate test reporting - other metrics
        Section Variations detail
            The ...
www.ivantage.co.uk


     Multivariate test reporting - key metric
        Estimated conversion rate
            Forecas...
www.ivantage.co.uk


     Multivariate test reporting - other metrics
        Conv/Visitors (not shown)
            Conv...
www.ivantage.co.uk


     Multivariate test reporting - inconclusive
        No clear winner
            Pointlessness o...
www.ivantage.co.uk


     After Multivariate testing?
        Test again!
        The next conversion point
        Tes...
www.ivantage.co.uk


     Feedback
        http://www.ivantage.co.uk/feedback
        support@ivantage.co.uk
        Wr...
www.ivantage.co.uk


     Getting help
        support@ivantage.co.uk for 60 days




                                   ...
www.ivantage.co.uk




          The End




                                     78


Wednesday, 29 September 2010
Google website optimser 101
Google website optimser 101
Google website optimser 101
Google website optimser 101
Google website optimser 101
Google website optimser 101
Google website optimser 101
Google website optimser 101
Google website optimser 101
Google website optimser 101
Google website optimser 101
Google website optimser 101
Google website optimser 101
Upcoming SlideShare
Loading in...5
×

Google website optimser 101

1,936

Published on

Published in: Technology, Business
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,936
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
347
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Google website optimser 101

  1. 1. www.ivantage.co.uk Presentation to J P Morgan Global Equity Derivatives Group Website Optimiser: An introduction to landing page testing Matt Trimmer Principal Consultant & Managing Director Slides: http://www.slideshare.net/ivantage 1 Wednesday, 29 September 2010
  2. 2. www.ivantage.co.uk Timings and housekeeping  Start 09:30  Break 11:00-11:15  Lunch 12:30-13:30  Break 14:45-15:00  Wrap-up 16:00  Nearest fire exit  Toilets  Water,  Refreshments & snacks 2 Wednesday, 29 September 2010
  3. 3. www.ivantage.co.uk Course Overview  Session 1 – Goals, conversion points and landing pages  Session 2 – What to test and testing types  Session 3 – Conducting and analysing A/B testing  Session 4 – Conducting and analysing Multivariate testing 3 Wednesday, 29 September 2010
  4. 4. www.ivantage.co.uk Introductions  First  Me (Matt Trimmer) & ivantage overview  In a moment  You  Your role  Your department/website/website area  Your objectives for attending today 4 Wednesday, 29 September 2010
  5. 5. www.ivantage.co.uk Me  Founder, Principal Consultant and Managing Director of ivantage  Love the Internet, 23 years IT, technical experience, 14 years Internet  One of 15 Google Accredited Seminar Leaders Globally 5 Wednesday, 29 September 2010
  6. 6. www.ivantage.co.uk About ivantage  Established in 2002  Independent  36rd Ranked UK Agency according to New Media Age rankings  Only agency in UK with these three Google certifications 6 Wednesday, 29 September 2010
  7. 7. www.ivantage.co.uk Urchin and Google, March 28th 2005  Urchin on Demand > Google Analytics (November 2005)  Urchin Software > Urchin Software from Google 7 Wednesday, 29 September 2010
  8. 8. www.ivantage.co.uk What we do  We help grow online businesses by  Generating web site traffic through  Paid Search (PPC)  Natural Search (SEO)  Social media marketing  Display advertising  E-mail marketing  Analyzing web site traffic using  Web Analytics to accurately analyse how visitors find and interact with websites  Converting web site traffic using  Conversion Rate Optimisation to convert visitors into customers or clients 8 Wednesday, 29 September 2010
  9. 9. www.ivantage.co.uk How we do it  We help grow online businesses by generating, analyzing and converting web site traffic:  Using bright, well-trained people who are passionate about what they do  Finding, harnessing and deploying innovative, affordable Internet marketing technology  Planning and executing intelligent campaigns that deliver measurable results  Seeking to continually improve return on client marketing spend  Ensuring high standards of service and client care  Building strong, enduring client relationships  Innovating to add additional value to client relationships  Following ethical business and technical practices  Delivered through  Professional services  Training 9 Wednesday, 29 September 2010
  10. 10. www.ivantage.co.uk Our vision  To be the Internet’s leading performance-driven Internet marketing agency 10 Wednesday, 29 September 2010
  11. 11. www.ivantage.co.uk Professional services  Traffic generation  Search Engine Marketing  Paid search – PPC  PfP Manage  SMA Manage  Natural Search – SEO  SEO Manage  SMO Manage  Display Advertising  PfP Manage  Email Marketing  Email Manage  Traffic analysis  Web analytics implementation  Accelerator services  Web analytics interpretation  Insight services  Web analytics support  Trouble Shooter services  Traffic conversion  Conversion optimisation  Landing Page Optimisation  Post-Click Optimisation  A/B and Multivariate testing  Voice of Customer Analytics 11 Wednesday, 29 September 2010
  12. 12. www.ivantage.co.uk Training (Public or private on-site)  Google Analytics  Google Analytics: Introduction & User Training  Google Analytics: Advanced Technical Implementation  Google Adwords  Google AdWords 101: Introduction & Fundamentals  Google AdWords 201: Optimizing your Campaign  Google AdWords 301 & 302: Advanced Adwords Management and Optimization  Google Website Optimizer  Google Website Optimizer: Introduction to landing page optimisation  Urchin Software from Google  Urchin Software: Introduction & User Training  Urchin Software: Advanced Technical Implementation  Urchin Software: Developing Custom Reports  Search Engine Optimisation (SEO)  SEO: Strategy and on-page fundamentals  SEO: Link building and Social Media Optimisation  Social Media Marketing  SMM: Strategy and SMM & SMO fundamentals  SMM: Social Media Optimisation 12 Wednesday, 29 September 2010
  13. 13. www.ivantage.co.uk Who we do it for – multi-channel retail 13 Wednesday, 29 September 2010
  14. 14. www.ivantage.co.uk Who we do it for – business to business 14 Wednesday, 29 September 2010
  15. 15. www.ivantage.co.uk Who we do it for – Non-Governmental Organisations and Governmental Organisations 15 Wednesday, 29 September 2010
  16. 16. www.ivantage.co.uk Introductions – your turn!  You  Your role  Your department/website/website area  Your objectives for attending today 16 Wednesday, 29 September 2010
  17. 17. www.ivantage.co.uk Google Website Optimiser  Landing page testing and optimisation tool  Beta released in Autumn 2006  Full releases in 2007  Uses Google Analytics technology  Adds utmx, utmxx first party cookies to utma,b,c,z and v  Launched as a multivariate tool  Later released with simplified A/B split test ability 17 Wednesday, 29 September 2010
  18. 18. www.ivantage.co.uk Questions we need to ask ourselves  What should we test for?  Which landing pages should we test?  What should we test?  How should we test?  How do we analyse the test?  How do we make conclusions about the test?  What do we do after the test? 18 Wednesday, 29 September 2010
  19. 19. www.ivantage.co.uk What we should test for?  The three business objectives  Website goals  Conversion points 19 Wednesday, 29 September 2010
  20. 20. www.ivantage.co.uk The big three business goals?  Raise revenue  Lower costs  Increase customer satisfaction  Raises revenue  Lowers costs 20 Wednesday, 29 September 2010
  21. 21. www.ivantage.co.uk Key Performance Indicators (KPIs)  A metric  Miles per Hour (mph)  Defining progress to an objective  To reach the legal motorway speed limit of 70 mph  Enabling you to take corrective action  Accelerate or brake  Speedometer  Web Analytics package 21 Wednesday, 29 September 2010
  22. 22. www.ivantage.co.uk KPI Examples Objective Metric Affect on Metric Increase people visiting Absolute Unique Visitors Increase Increase traffic Visits Increase Increase visits from members Returning Visits Increase Increase sales Transactions Increase Increase leads Goal x (Leads) Increase Increase new visits to the site % New Visits Increase Increase site interaction Average Page Views Increase Decrease short visits Bounce rate Decrease 22 Wednesday, 29 September 2010
  23. 23. www.ivantage.co.uk Exercise – Your KPIs and goals  What are your KPIs for your web business?  Using Google Intelligence to decipher KPIs 23 Wednesday, 29 September 2010
  24. 24. www.ivantage.co.uk KPIs for your website should include  Outcome focused events  “Goals” 24 Wednesday, 29 September 2010
  25. 25. www.ivantage.co.uk Goals  Goals should be map back to business objectives  The commercial goals of your website  Goals should map to visitor tasks  Visitors visit sites to accomplish tasks  Tasks normally have a series of steps  Funnel  Measure conversion through the task  When goals match user tasks  Harmony ensues! 25 Wednesday, 29 September 2010
  26. 26. www.ivantage.co.uk The importance of goals for you  Commercially justify your work  Focus your work  Prioritise your work 26 Wednesday, 29 September 2010
  27. 27. www.ivantage.co.uk The importance of goals for you  Goals map to business metrics  Enable you to build a business case for  Analytics investment  Analysis investment  Action investment  Goals focus on important paths and entrances  Enable you to home-in on problem areas  Reverse Goal Path  Conversion to goals by traffic source  Monetize goals  Enable you to prioritise areas to analyse and optimise  Give goals a goal value  Analyse $ Index  (Goal Value + E-commerce value)/unique page views  Page up-to goal contribution  Focus on high $ Index pages for improvement 27 Wednesday, 29 September 2010
  28. 28. www.ivantage.co.uk Goals and funnels  You can use Regular Expressions (RegEx) to gain insight into the funnel path:  User exhibits behavior:  Page 1 = /index.html  Page 2 = /category-footwear.html  Page 3 = /category-clothing.html  Page 4 = /category-headwear.html  Page 5 = /products/kangol-tropic-player-trilby.html  Page 6= /cart/basketview.html  Page 7= /cart/registration.html  Page 8 = /cart/paymentoptions.html  Goal = /cart/salesorderconfirmation.html  Need to know conversion rate of categories to products to purchase:  Step 1: index.html  Step 2-4: ^/category.*/  Step 5: ^/products.*/  Step 6: ^/cart/basketview.html  Step 7: ^/cart/registration.html  Step 8 : ^/cart/paymentoptions.html  Goal: ^/cart/salesorderconfirmation.html 28 Wednesday, 29 September 2010
  29. 29. www.ivantage.co.uk Goal and funnel behaviour 29 Wednesday, 29 September 2010
  30. 30. www.ivantage.co.uk Test conversion points not the rate! Assume some constants If more products were added to basket? At this point 30 Wednesday, 29 September 2010
  31. 31. www.ivantage.co.uk Test pages and conversion page Test Page Conversion point page 31 Wednesday, 29 September 2010
  32. 32. www.ivantage.co.uk What we should test for?  Conversions points that lead to  Website goals  That further business objectives 32 Wednesday, 29 September 2010
  33. 33. www.ivantage.co.uk Exercise  What are the goals for your website?  Are you recording them?  Use notepad to map the goals out  Can you split the goals into logical conversion points? 33 Wednesday, 29 September 2010
  34. 34. www.ivantage.co.uk Which landing pages to test?  Landing pages by highest bounce rate?  Landing Page Optimisation  Entrance Sources (Top Sources per Page)?  Entrance Keywords ( Top keywords per Page?  All traffic sources  Cross segment by landing page  Navigation summary  Entrance paths  Funnel visualisation  Reverse goal path 34 Wednesday, 29 September 2010
  35. 35. www.ivantage.co.uk Which landing pages to test?  Landing pages that send significant traffic to  Conversions points that lead to  Website goals  That further business objectives 35 Wednesday, 29 September 2010
  36. 36. www.ivantage.co.uk Exercise  Can we find highly traffic landing pages?  Can we map out conversion paths? 36 Wednesday, 29 September 2010
  37. 37. www.ivantage.co.uk What to test?  What’s your gut telling you about your pages?  What are your customers telling you about your pages?  4Q - “Voice of Customer” - site level  KISS Insights - “Voice of Customer” - page Level  Usability testing  Lab  usertesting.com  whatuserdo.com  Heuristic reviews  Site reviewed against a list of expected/excepted critiria 37 Wednesday, 29 September 2010
  38. 38. www.ivantage.co.uk What to test?  Presentation of offers on  Landing pages that send significant traffic to  Conversions points that lead to  Website goals  That further business objectives 38 Wednesday, 29 September 2010
  39. 39. www.ivantage.co.uk Exercise  Can you define some alternative presentations for your selected landing pages?  Can you split these alternative presentations into Page Sections?  Could these sections have variations  Which sections will have biggest impact on conversion point? 39 Wednesday, 29 September 2010
  40. 40. www.ivantage.co.uk Lunch time 40 Wednesday, 29 September 2010
  41. 41. www.ivantage.co.uk How does GWO work? 41 Wednesday, 29 September 2010
  42. 42. www.ivantage.co.uk How should we test - two methods  Simple A/B split testing  Pros  Simple and fast to set-up  Great for low traffic volumes (small number of combinations)  Cons  May miss the fact that some components on two different pages work better together  Complex multivariate testing  Pros  Won’t miss the fact that some components (on your A/B pages) work better together  Highlights best performing Page Sections (areas of the test page)  Highlights best performing Combinations (Variations in Page Sections)  Provides you with a Control that you would keep testing  Cons  May take significant time to produce results (traffic/Combinations) 42 Wednesday, 29 September 2010
  43. 43. www.ivantage.co.uk A/B Testing Testing Pages Control script Conversion Point A Tracking Script Conversion Page Control script Tracking Script B Tracking Script 43 Wednesday, 29 September 2010
  44. 44. www.ivantage.co.uk How GWO works Website A B 44 Wednesday, 29 September 2010
  45. 45. www.ivantage.co.uk GWO decides which page to serve B cookies 45 Wednesday, 29 September 2010
  46. 46. www.ivantage.co.uk GA cookies UTMa - Unique visitor cookie, lasts for 2 years from set or update on every page view UTMb - Visit cookie, lasts for 30 minutes from every page view UTMc - Visit end cookie, not set to expire, does so when browser closes UTMz - Campaign visit tracking cookie, determines traffic sources, lasts for 6 months from set or update on every page view UTMv - Not always set, requires the _setVar() command 46 Wednesday, 29 September 2010
  47. 47. www.ivantage.co.uk The ___utm.gif request from the ga.js to Google’s servers __utm.gif?utmwv=6.3&utmn=101027544&utmsr=1280x854&utmsc=32-bit&utmul=en&utmje=1&utmjv=1.5&utmfl=7.0 r24&utmdt=My%20Website&utmhn=www.site.com&utmp=/index.html&utmac=UA-51244-4&utmcc=__utma %3D117945243.823564440.1131054947.1131080858.1133834608.4%3B+__utmb%3D117945243%3B+__utmc %3D117945243%3B+__utmz%3D117945243.1131054947.1.1.utmccn%3D%28Spring%20Sale%29%7Cutmcsr%3D%28google %29%7Cutmcmd%3D%28ppc%29% %7Cutmtm%3D%28shoes%29%3B Screen Resolution: 1280x854 Page Title: My Site Screen Colors: 32-bit Filename: /index.html Browser Language: English(en) Visits: 4 Java Enabled? Yes (1) Campaigns Clicked: 1 Java Version: 1.5 Campaign: Google(PPC) Spring Sale (shoes) 47 Wednesday, 29 September 2010
  48. 48. www.ivantage.co.uk What are “Fortune” Cookies? • Text files sent to browsers by websites • 1ST or 3rd party • Visit apple.com • apple.com issues cookies (1st party) • If banana.com attempts to issue cookies (3rd party) • Session and persistent • Session • Last for 30 minutes of inactivity • Persistent • Lasts for 2 years 48 Wednesday, 29 September 2010
  49. 49. www.ivantage.co.uk Persistent UTMx (3) cookies Respond with Page A or B Browser web 1st visit server New Visitor B Respond with Page B as last time Browser web Cookie server cookies Subsequent visits Returning Visitor 49 Wednesday, 29 September 2010
  50. 50. www.ivantage.co.uk Weaknesses of cookies? • Machine dependant • As close as we can get to a person • Although modern OSs have multiple user IDs • User 1, User2, User 3 • Can be cleared • Worst scenario? • Can be blocked/not accepted • Most E-commerce sites use 1st party cookies to make basket function • So blocking 1st party cookies degrades your Internet experience • 3rd party cookies are often blocked 50 Wednesday, 29 September 2010
  51. 51. www.ivantage.co.uk A/B split test set-up - exercise  Visit your website or  http://www.ivantageaaa.co.uk/GAWO/index.html  Navigate to Products (A/B)  Clear cookies to see different versions 51 Wednesday, 29 September 2010
  52. 52. www.ivantage.co.uk A/B split test set-up  Original page  Control script at top of page  Sends 1st party cookies to ensure  new visitors enter test (if applicable)  returning visitors see same page  Tracking script at bottom of page  Records page display  Variation pages (A, B, C)  Control and Tracking scripts  Conversion page  Tracking script 52 Wednesday, 29 September 2010
  53. 53. www.ivantage.co.uk A/B split test set-up - settings  % traffic to see experimental pages  Selected % is sent uniformly to variations  Disable losing variations - off or 53 Wednesday, 29 September 2010
  54. 54. www.ivantage.co.uk A/B split test set-up - duration  Start your test  Test never stop until you stop it  Statistical prediction of conversion rate from actual performance  Duration  Calculator  http://www.google.com/support/websiteoptimizer/bin/ answer.py?answer=61688&hl=en_US  Combinations =2 (A,B), 3 (A,B,C)  This is where A/B split testing can be quick 54 Wednesday, 29 September 2010
  55. 55. www.ivantage.co.uk A/B split test reporting  2-3 hours delay  Original and variations 55 Wednesday, 29 September 2010
  56. 56. www.ivantage.co.uk A/B split test reporting - key metric  Estimated conversion rate  Forecasted/predicted conversion rate if rolled out  Based on actual observed conversion rate  % conversion = No. conversion page hits/No. test page hits X 100  Calculated with an 80% confidence rate  Will fall within this range 80% of the time 56 Wednesday, 29 September 2010
  57. 57. www.ivantage.co.uk A/B split test reporting - other metrics  Conv/Visitors  Conversion numbers  Observed improvement  Between original and variation  Chance to beat original  Likelihood of this beating original is rolled out 57 Wednesday, 29 September 2010
  58. 58. www.ivantage.co.uk A/B split test reporting - inconclusive  No clear winner  Pointlessness of testing small changes  Make a big variation change and test again 58 Wednesday, 29 September 2010
  59. 59. www.ivantage.co.uk After A/B testing?  Test again!  Test next conversion point  Use multivariate testing  Bring back discarded (but big) changes 59 Wednesday, 29 September 2010
  60. 60. www.ivantage.co.uk Multivariate test set-up - exercise  Visit your website or  Clear cookies (Firefox)  http://www.ivantageaaa.co.uk/GAWO/index.html  Navigate to Products (MV)  Clear cookies to see different versions 60 Wednesday, 29 September 2010
  61. 61. www.ivantage.co.uk Multivariate Testing Testing Page Control script Conversion Point Conversion Tracking Script Page Section Scripts (8) Tracking Script Variations (in content) 61 Wednesday, 29 September 2010
  62. 62. www.ivantage.co.uk Multivariate Testing Testing Page Control script Conversion Point Conversion Tracking Script Page Section Scripts (8) Tracking Script Combination (Recipe) Variations (in content) 62 Wednesday, 29 September 2010
  63. 63. www.ivantage.co.uk How GWO works Testing Page 63 Wednesday, 29 September 2010
  64. 64. www.ivantage.co.uk How GWO works Testing Page cookies 64 Wednesday, 29 September 2010
  65. 65. www.ivantage.co.uk Multivariate test set-up  Original page  Control script at top of page  Sends 1st party cookies to ensure  new visitors enter test (if applicable)  returning visitors see same page  Tracking script at bottom of page  Records page display  Section scripts (up to 8)  Conversion page  Tracking script 65 Wednesday, 29 September 2010
  66. 66. www.ivantage.co.uk Multivariate test set-up - settings  % traffic to see experimental pages?  Selected % is sent uniformly to variations  Auto-disable losing variations? - off or 66 Wednesday, 29 September 2010
  67. 67. www.ivantage.co.uk Multivariate test reporting  2-3 hours delay  Combinations and Page Sections  Original and combinations 67 Wednesday, 29 September 2010
  68. 68. www.ivantage.co.uk Multivariate test set-up - duration  Start your test  Tests never stop until you stop them  Statistical prediction of conversion rate from actual performance  Duration  Calculator  http://www.google.com/support/websiteoptimizer/bin/ answer.py?answer=61688&hl=en_US  Combinations =Page Sections x Variations  Grows large very quickly » Traffic requirements/time become great » >Combinations > Traffic/time needed to reach conclusion  Assumes a 12% conversion rate improvement 68 Wednesday, 29 September 2010
  69. 69. www.ivantage.co.uk Multivariate test reporting - page sections  First to populate with data 69 Wednesday, 29 September 2010
  70. 70. www.ivantage.co.uk Multivariate test reporting - key metric  Relevance rating  The Page Sections that had the greatest impact on overall results 70 Wednesday, 29 September 2010
  71. 71. www.ivantage.co.uk Multivariate test reporting - other metrics  Section Variations detail  The best Combination does not always comprise of the best Variations 71 Wednesday, 29 September 2010
  72. 72. www.ivantage.co.uk Multivariate test reporting - key metric  Estimated conversion rate  Forecasted/predicted conversion rate if rolled out  Based on actual observed conversion rate  % conversion = No. conversion page hits/No. test page hits X 100  Calculated with an 80% confidence rate  Will fall within this range 80% of the time 72 Wednesday, 29 September 2010
  73. 73. www.ivantage.co.uk Multivariate test reporting - other metrics  Conv/Visitors (not shown)  Conversion numbers  Observed improvement  Between original and variation  Chance to beat Original  Likelihood of this beating original if rolled out  Chance to beat All  Likelihood of this beating all combinations if rolled out 73 Wednesday, 29 September 2010
  74. 74. www.ivantage.co.uk Multivariate test reporting - inconclusive  No clear winner  Pointlessness of testing small changes  Make a big Variation changes and test again 74 Wednesday, 29 September 2010
  75. 75. www.ivantage.co.uk After Multivariate testing?  Test again!  The next conversion point  Test big impact changes  Your Control Combination is replaced by out-performing Combinations  Always try to better the Control Combination 75 Wednesday, 29 September 2010
  76. 76. www.ivantage.co.uk Feedback  http://www.ivantage.co.uk/feedback  support@ivantage.co.uk  Write an honest, considered and fair review to receive for each day attended: 76 Wednesday, 29 September 2010
  77. 77. www.ivantage.co.uk Getting help  support@ivantage.co.uk for 60 days 77 Wednesday, 29 September 2010
  78. 78. www.ivantage.co.uk The End 78 Wednesday, 29 September 2010
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×