Sponsored By:




         What’s Worth A/B Testing?
                                       Presented live September 30, 2010
                                       Featuring Anne Holland
                                       Publisher, WhichTestWon.com
                                       Plus Guest Star: Mike Volpe
                                       VP of Marketing, HubSpot




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Who is HubSpot?
160+ Employees in Cambridge, MA (MIT)
         $33m in VC (3 rounds)
   Marketing Software at $3K-15K/year
Complicated &                           Easy &
    Confusing                           Integrated
                            Lead
 Blogging                  Nurturing
                SEO

                              Social
    Lead       Analytics      Media
Intelligence

                Sales
  Landing                    Email
   Pages        Tools       Marketing
3,200
          Customers


Q1 2007               Q2 2010
Sponsored By:




         What’s Worth A/B Testing?
                                       Presented live September 30, 2010
                                       Featuring Anne Holland
                                       Publisher, WhichTestWon.com
                                       Plus Guest Star: Mike Volpe
                                       VP of Marketing, HubSpot


                                          Admin Note:
                                          Attendees will receive a Special Report PDF and an on-demand mp3 via email.


©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com




   Part I. Why Bother Testing?



©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Why Bother Testing?
           Most sites are designed by:
            Owner’s “gut”
            Pre-formatted templates
            Copying competitors
            “Best practices”

           Not by testing.

©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Better Results From the Same Traffic


                                                               Tested
                       Old Site
                                                                Site



             Typical results for previously untested pages:
             B2B lead generation forms +30-40%
             eCommerce page tests +20-25%
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Testing = Competitive Advantage
              How Many Marketers Use Tests to Optimize Response
                                  Rates?
    100.0%

     90.0%

     80.0%

     70.0%

     60.0%

     50.0%

     40.0%

     30.0%

     20.0%

     10.0%

      0.0%

              Marketers who run A/B tests on             Marketers who run A/B or
                    email campaigns                   Multivariate tests on web pages
      n=505; survey conducted August 2010   ©2010 Online Marketing Connect Aprimo & WhichTestWon com
WhichTestWon.com
How A/B Testing Works




                            Version A                                             Version B



©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Example: A/B Homepage Test




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Results


                                                                                        • 125% lift in clicks
                                                                                          to the download
                                                                                          offer page



©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com



   Part II. Which Web Pages Are
   Most Worth Testing?


©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
First, Test Where the Money Is…

      #1. Lead gen form pages
      #2. Other offer pages
      #3. Landing pages for
        expensive traffic
     $0 x 30% = $0
     $100 x 30% = $130
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Form Pages




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Results

                                                                              • 41.8% lift in lead gen
                                                                                form submissions




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Offer Pages




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Results

                                                                                        • 62.6% lift in paid
                                                                                          subscriptions




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com



   Part III. Which Page Elements
   Are Most Worth Testing?


©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Glance From 4+ Feet Away…
   15” monitor
   1024 x 768 resolution
   Hint: 980 pixels across, 520 down
     •    Buttons & response links
     •    Headline wording
     •    Layout Clarity
     •    Forms
     •    Large images                                                                  Your Page
©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Test Buttons & Response Links




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Results


                                                                       • 32.5% registration
                                                                         gain




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Test Offer Headlines




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Results


                                                                                   • 113% lift in free trial
                                                                                     sign-ups




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Copy (Taxonomy) Test




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Results

                                                                                  • 42% lift in lead gen
                                                                                    form completions on
                                                                                    the next page (for
                                                                                    the word
                                                                                    “Enterprise”)

©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Test Offers on Page Templates




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Results

                                                                               • 72.72% lift in lead gen
                                                                                 form submissions




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Test Key Images




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Results


                                                                               • 37.2% lift in lead gen
                                                                                 form completions




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com



   Part IV. Quick Guidelines:
   Testing in Practice


©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
How Much Does Testing Cost?




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
How Long Should a Test Run For?
     Before-and-After




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Which Traffic Should You Use?




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
How Conclusive Should Results Be?




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com



   Part V. Convincing the CEO to
   Let You Test


©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
When the HIPPO Rules…




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
If You Measure $$ Per Conversion




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
If You Measure Cost Per Conversion




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Lastly, Top 2 Safeguards:

                  1. Politically Correct Tests
                  2. Yank a Dud Test Quickly




©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
How HubSpot Supports Testing

                  1. Easy to Build Landing Pages & Forms
                  2. Easy to Use Reports
                  3. Closed Loop Conversion Rate Data

                                Mike Volpe
                                VP of Marketing, HubSpot


©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Easy to Build Landing Pages

                                                                                        Easy to
                                                                                        edit
                                                                                        No HTML
                                                                                        required



©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Easy to Build Forms

                                                                                        Easy to edit
                                                                                        Drag and drop
                                                                                        No HTML required
                                                                                        Add custom fields


©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Easy to Use Reports

                                                                                   One Click Away from:
                                                                                   • Visits by Source
                                                                                   • Leads by Source
                                                                                   • Customers by Source



©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Easy to Use Reports




   Easily measure landing page conversion rates


©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Closed Loop Conversion Rate Data

                                                                                        Visit to Lead %
                                                                                        and
                                                                                        Lead to
                                                                                        Customer %



©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
WhichTestWon.com
Closed Loop Conversion Rate Data
                                                                                        All Conversion
                                                                                        Rates

                                                                                        Broken Down
                                                                                        Automatically
                                                                                        by Source

©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Complicated &                           Easy &
    Confusing                           Integrated
                            Lead
 Blogging                  Nurturing
                SEO

                              Social
    Lead       Analytics      Media
Intelligence


  Landing
                Sales
                             Email      30 Day Free Trial
   Pages        Tools       Marketing
WhichTestWon.com
  Thank you!

  Anne Holland, Publisher
  WhichTestWon.com
  Email: AnneH@WhichTestWon.com
  Twitter: @WhichTestWon

  Mike Volpe, VP Marketing
  HubSpot
  Email: mvolpe@hubspot.com
  Twitter: @mvolpe
  http://www.HubSpot.com


©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.

What is

  • 1.
    Sponsored By: What’s Worth A/B Testing? Presented live September 30, 2010 Featuring Anne Holland Publisher, WhichTestWon.com Plus Guest Star: Mike Volpe VP of Marketing, HubSpot ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 2.
    Who is HubSpot? 160+Employees in Cambridge, MA (MIT) $33m in VC (3 rounds) Marketing Software at $3K-15K/year
  • 3.
    Complicated & Easy & Confusing Integrated Lead Blogging Nurturing SEO Social Lead Analytics Media Intelligence Sales Landing Email Pages Tools Marketing
  • 4.
    3,200 Customers Q1 2007 Q2 2010
  • 5.
    Sponsored By: What’s Worth A/B Testing? Presented live September 30, 2010 Featuring Anne Holland Publisher, WhichTestWon.com Plus Guest Star: Mike Volpe VP of Marketing, HubSpot Admin Note: Attendees will receive a Special Report PDF and an on-demand mp3 via email. ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 6.
    WhichTestWon.com Part I. Why Bother Testing? ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 7.
    WhichTestWon.com Why Bother Testing? Most sites are designed by: Owner’s “gut” Pre-formatted templates Copying competitors “Best practices” Not by testing. ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 8.
    WhichTestWon.com Better Results Fromthe Same Traffic Tested Old Site Site Typical results for previously untested pages: B2B lead generation forms +30-40% eCommerce page tests +20-25% ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 9.
    WhichTestWon.com Testing = CompetitiveAdvantage How Many Marketers Use Tests to Optimize Response Rates? 100.0% 90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Marketers who run A/B tests on Marketers who run A/B or email campaigns Multivariate tests on web pages n=505; survey conducted August 2010 ©2010 Online Marketing Connect Aprimo & WhichTestWon com
  • 10.
    WhichTestWon.com How A/B TestingWorks Version A Version B ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 11.
    WhichTestWon.com Example: A/B HomepageTest ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 12.
    WhichTestWon.com Results • 125% lift in clicks to the download offer page ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 13.
    WhichTestWon.com Part II. Which Web Pages Are Most Worth Testing? ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 14.
    WhichTestWon.com First, Test Wherethe Money Is… #1. Lead gen form pages #2. Other offer pages #3. Landing pages for expensive traffic $0 x 30% = $0 $100 x 30% = $130 ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 15.
    WhichTestWon.com Form Pages ©2010, WhichTestWon.coma division of Anne Holland Ventures Inc. All rights reserved.
  • 16.
    WhichTestWon.com Results • 41.8% lift in lead gen form submissions ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 17.
    WhichTestWon.com Offer Pages ©2010, WhichTestWon.coma division of Anne Holland Ventures Inc. All rights reserved.
  • 18.
    WhichTestWon.com Results • 62.6% lift in paid subscriptions ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 19.
    WhichTestWon.com Part III. Which Page Elements Are Most Worth Testing? ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 20.
    WhichTestWon.com Glance From 4+Feet Away… 15” monitor 1024 x 768 resolution Hint: 980 pixels across, 520 down • Buttons & response links • Headline wording • Layout Clarity • Forms • Large images Your Page ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 21.
    WhichTestWon.com Test Buttons &Response Links ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 22.
    WhichTestWon.com Results • 32.5% registration gain ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 23.
    WhichTestWon.com Test Offer Headlines ©2010,WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 24.
    WhichTestWon.com Results • 113% lift in free trial sign-ups ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 25.
    WhichTestWon.com Copy (Taxonomy) Test ©2010,WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 26.
    WhichTestWon.com Results • 42% lift in lead gen form completions on the next page (for the word “Enterprise”) ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 27.
    WhichTestWon.com Test Offers onPage Templates ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 28.
    WhichTestWon.com Results • 72.72% lift in lead gen form submissions ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 29.
    WhichTestWon.com Test Key Images ©2010,WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 30.
    WhichTestWon.com Results • 37.2% lift in lead gen form completions ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 31.
    WhichTestWon.com Part IV. Quick Guidelines: Testing in Practice ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 32.
    WhichTestWon.com How Much DoesTesting Cost? ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 33.
    WhichTestWon.com How Long Shoulda Test Run For? Before-and-After ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 34.
    WhichTestWon.com Which Traffic ShouldYou Use? ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 35.
    WhichTestWon.com How Conclusive ShouldResults Be? ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 36.
    WhichTestWon.com Part V. Convincing the CEO to Let You Test ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 37.
    WhichTestWon.com When the HIPPORules… ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 38.
    WhichTestWon.com If You Measure$$ Per Conversion ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 39.
    WhichTestWon.com If You MeasureCost Per Conversion ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 40.
    WhichTestWon.com Lastly, Top 2Safeguards: 1. Politically Correct Tests 2. Yank a Dud Test Quickly ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 41.
    WhichTestWon.com How HubSpot SupportsTesting 1. Easy to Build Landing Pages & Forms 2. Easy to Use Reports 3. Closed Loop Conversion Rate Data Mike Volpe VP of Marketing, HubSpot ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 42.
    WhichTestWon.com Easy to BuildLanding Pages Easy to edit No HTML required ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 43.
    WhichTestWon.com Easy to BuildForms Easy to edit Drag and drop No HTML required Add custom fields ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 44.
    WhichTestWon.com Easy to UseReports One Click Away from: • Visits by Source • Leads by Source • Customers by Source ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 45.
    WhichTestWon.com Easy to UseReports Easily measure landing page conversion rates ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 46.
    WhichTestWon.com Closed Loop ConversionRate Data Visit to Lead % and Lead to Customer % ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 47.
    WhichTestWon.com Closed Loop ConversionRate Data All Conversion Rates Broken Down Automatically by Source ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
  • 48.
    Complicated & Easy & Confusing Integrated Lead Blogging Nurturing SEO Social Lead Analytics Media Intelligence Landing Sales Email 30 Day Free Trial Pages Tools Marketing
  • 49.
    WhichTestWon.com Thankyou! Anne Holland, Publisher WhichTestWon.com Email: AnneH@WhichTestWon.com Twitter: @WhichTestWon Mike Volpe, VP Marketing HubSpot Email: mvolpe@hubspot.com Twitter: @mvolpe http://www.HubSpot.com ©2010, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.