Website	
  Tes)ng	
  Wins	
  

October	
  10,	
  2012	
  -­‐	
  2:00	
  pm	
  ET	
  

                                  Follow	
  @monetate	
  on	
  	
     #websitetes0ngwins	
  
Tips	
  for	
  Webinar	
  A8endees	
  
•	
  Audio	
  problems?	
  Click	
  the	
  “Problem	
  Dialing	
  In”	
  link	
  or	
  switch	
  to	
  
the	
  webinar	
  phone	
  bridge	
  and	
  make	
  sure	
  your	
  phone	
  is	
  muted.	
  
	
  
•	
  Use	
  the	
  “Ques0on”	
  box	
  within	
  GoToWebinar	
  to	
  ask	
  a	
  ques0on	
  or	
  
use	
  the	
  webinar	
  hashtag	
  #websitetes0ngwins	
  
	
  
•	
  A	
  recording	
  of	
  the	
  webinar	
  with	
  the	
  slide	
  deck	
  will	
  be	
  available	
  at	
  
monetate.com/webinars	
  within	
  1	
  or	
  2	
  business	
  days.	
  




                                                                                          #websitetes0ngwins	
  
Today’s	
  Presenters	
  

            Jus0n	
  Rondeau	
  
            Chief	
  Tes0ng	
  Evangelist	
  &	
  Editor	
  
            WhichTestWon	
  
            	
  
            	
  
            	
  
            	
  
            Peter	
  Borden	
  
            Marke0ng	
  Manager	
  
            Monetate	
  




                                             #websitetes0ngwins	
  
Website	
  Tes)ng	
  Wins	
  

Jus)n	
  Rondeau	
  
Chief	
  Tes0ng	
  Evangelist	
  &	
  Editor	
  
WhichTestWon	
  

                                                           #websitetes0ngwins	
  
Industry Analyst & Hands on
Trainer




©2012, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.   #websitetes0ngwins	
  
Case	
  Study	
  #1:	
  Apparel	
  Category	
  Page	
  
Image	
  	
  	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  	
  All	
  rights	
  reserved.	
  	
  	
     #websitetes0ngwins	
  
Winner	
  

                                                                                                                               Results:	
  44%	
  Lia	
  in	
  
                                                                                                                               Purchases	
  &	
  	
  67%	
  Lia	
  
                                                                                                                               in	
  RPV	
  
                                                                                                                               	
  
                                                                                                                               Brand:	
  Jules.fr	
  
                                                                                                                               	
  
                                                                                                                               Lesson:	
  Test	
  Your	
  
                                                                                                                               Category	
  Page	
  
                                                                                                                               Images	
  


©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  	
  All	
  rights	
  reserved.	
  	
  	
          #websitetes0ngwins	
  
Case	
  Study	
  #2:	
  Category	
  Page	
  Display	
  Test	
  
                     Default	
  10	
  Items	
                                                                                            Default	
  50	
  Items	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  	
  All	
  rights	
  reserved.	
  	
  	
  
Winner!	
  
                           Default	
  50	
  Items	
  


                                                                                                                                   Results:	
  35%	
  Lia	
  in	
  
                                                                                                                                   Purchases	
  
                                                                                                                                   	
  
                                                                                                                                   Brand:	
  Palmers	
  
                                                                                                                                   	
  
                                                                                                                                   Lesson:	
  Default	
  
                                                                                                                                   Sehngs	
  Aren’t	
  
                                                                                                                                   Necessarily	
  Op0mal	
  
                                                                                                                                   Sehngs	
  



©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  	
  All	
  rights	
  reserved.	
  	
  	
            #websitetes0ngwins	
  
Case	
  Study	
  #3:	
  Geoloca)on	
  Test	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  	
  All	
  rights	
  reserved.	
  	
  	
     #websitetes0ngwins	
  
Winner	
  



                                                                                                                            Results:	
  2.43%	
  Increase	
  
                                                                                                                            in	
  Sales	
  
                                                                                                                            	
  
                                                                                                                            Brand:	
  KarmaLoop	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  	
  All	
  rights	
  reserved.	
  	
  	
      #websitetes0ngwins	
  
Case	
  Study	
  #4:	
  Cart	
  BuOon	
  Copy	
  	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  All	
  rights	
  reserved.	
  	
     #websitetes0ngwins	
  
Winner	
  



                                                                                                                87.5%	
  Lia	
  in	
  Completed	
  
                                                                                                                Form	
  Submissions	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  All	
  rights	
  reserved.	
  	
     #websitetes0ngwins	
  
Case	
  Study	
  #5:	
  Search	
  BuOon	
  Placement	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  	
  All	
  rights	
  reserved.	
  	
  	
     #websitetes0ngwins	
  
Winner	
  




                                                                               Results:	
  12%	
  Lia	
  in	
  Sales	
  contribu0ng	
  
                                                                               to	
  a	
  2.3	
  Million	
  lia	
  in	
  annual	
  revenue	
  
                                                                               	
  
                                                                               Brand:	
  Mountain	
  Equipment	
  Co-­‐op	
  

©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  	
  All	
  rights	
  reserved.	
  	
  	
     #websitetes0ngwins	
  
Case	
  Study	
  #6:	
  Travel	
  Search	
  Results	
  	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  	
  All	
  rights	
  reserved.	
  	
  	
  
Winner!	
  



                                                                                                               Results:	
  16.36%	
  Lia	
  in	
  Paid	
  
                                                                                                               Bookings	
  
                                                                                                               	
  
                                                                                                               Brand:	
  KRAS	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  	
  All	
  rights	
  reserved.	
  	
  	
     #websitetes0ngwins	
  
Case	
  Study	
  #7:	
  Product	
  Page	
  Test	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  	
  All	
  rights	
  reserved.	
  	
  	
     #websitetes0ngwins	
  
Winner	
   None!	
  Surprised?	
  
Winner?	
  


                                                                                                             Results:	
  No	
  Sta0s0cally	
  
                                                                                                             Significant	
  Difference	
  
                                                                                                             	
  
                                                                                                             Brand:	
  Design3000	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  	
  All	
  rights	
  reserved.	
  	
  	
     #websitetes0ngwins	
  
Our	
  Intui)on	
  is	
  Poor	
  
“Neplix	
  considers	
  90%	
  of	
  what	
  they	
  try	
  to	
  be	
  wrong”	
  -­‐-­‐	
  
Mike	
  Moran	
  



                                                                   “80%	
  of	
  the	
  0me	
  you/we	
  are	
  wrong	
  about	
  what	
  a	
  
                                                                                         customer	
  wants”	
  	
  -­‐-­‐	
  Avinash	
  Kaushik	
  


Microsoa’s	
  sta0s0c	
  from	
  thousands	
  of	
  controlled	
  
experiments:	
  
only	
  10-­‐30%	
  of	
  experiments	
  move	
  the	
  metrics	
  they	
  were	
  
designed	
  to	
  improve	
  


©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  	
  All	
  rights	
  reserved.	
  	
  	
     #websitetes0ngwins	
  
Case	
  Study	
  #8:	
  Security	
  Indicia	
  Test	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  All	
  rights	
  reserved.	
  	
     #websitetes0ngwins	
  
Winner	
  

                                                                                                                           Results:	
  
                                                                                                                           1%	
  Lia	
  in	
  Sales 	
  	
  
                                                                                                                           	
  
                                                                                                                           Big	
  Win	
  Save	
  on	
  
                                                                                                                           Cost	
  for	
  Security	
  
                                                                                                                           Icons	
  
                                                                                                                           	
  



©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  All	
  rights	
  reserved.	
  	
             #websitetes0ngwins	
  
Case	
  Study	
  #9:	
  Copy	
  Test	
  	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  	
  All	
  rights	
  reserved.	
  	
  	
     #websitetes0ngwins	
  
Closer	
  Look	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  All	
  rights	
  reserved.	
  	
     #websitetes0ngwins	
  
Winner	
                                                                                                       Seasonal	
  Test	
  –	
  Focused	
  
                                                                                                               On	
  Sales	
  for	
  the	
  Holidays	
  
                                                                                                               	
  
                                                                                                               Simple	
  Copy	
  –	
  Highligh0ng	
  
                                                                                                               The	
  benefit	
  of	
  the	
  product	
  
                                                                                                               	
  
                                                                                                               Simple	
  to	
  Implement	
  with	
  
                                                                                                               Sizeable	
  Lia	
  
                                                                                                               18.7%	
  Lia	
  in	
  Sales	
  from	
  	
  
                                                                                                               Nov	
  22nd	
  –	
  Dec	
  15th	
  
                                                                                                               	
  
                                                                                                               60.7%	
  Increase	
  	
  from	
  	
  
                                                                                                               Dec	
  16th	
  to	
  Dec	
  23rd	
  
©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  All	
  rights	
  reserved.	
  	
         #websitetes0ngwins	
  
Case	
  Study	
  #10:	
  PPC	
  Landing	
  Page	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  	
  All	
  rights	
  reserved.	
  	
  	
     #websitetes0ngwins	
  
Winner	
  




                                                                              Lia	
  –	
  74%	
  Lia	
  in	
  Completed	
  Car	
  
                                                                              Rentals	
  
                                                                              	
  
                                                                              Reminder	
  –	
  Use	
  Landing	
  Pages	
  
                                                                              &	
  Keep	
  Message	
  Consistent	
  
©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  All	
  rights	
  reserved.	
  	
  
Case	
  Study	
  #11:	
  Product	
  of	
  the	
  Week	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  	
  All	
  rights	
  reserved.	
  	
  	
     #websitetes0ngwins	
  
Winner	
  


                                                                                                           Results:	
  3.6%	
  Lia	
  in	
  RPV	
  
                                                                                                           	
  
                                                                                                           Brand:	
  3Suisses	
  
                                                                                                           	
  
                                                                                                           Lesson:	
  Cross	
  Sells	
  must	
  be	
  
                                                                                                           relevant	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  All	
  rights	
  reserved.	
  	
     #websitetes0ngwins	
  
Case	
  Study	
  #12:	
  Landing	
  Page	
  Cross	
  Sell	
  




©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  All	
  rights	
  reserved.	
  	
     #websitetes0ngwins	
  
Winner	
  

                                                                                                                         Results:	
  Significant	
  Lia	
  
                                                                                                                         in	
  Sales	
  (asked	
  not	
  to	
  
                                                                                                                         disclose)	
  
                                                                                                                         	
  
                                                                                                                         Brand:	
  BestStunGun	
  
                                                                                                                         	
  
                                                                                                                         Lesson:	
  Less	
  expensive	
  
                                                                                                                         cross	
  sells	
  can	
  ease	
  
                                                                                                                         s0cker	
  shock	
  


©2012,	
  WhichTestWon.com	
  a	
  division	
  of	
  Anne	
  Holland	
  Ventures	
  Inc.	
  All	
  rights	
  reserved.	
  	
            #websitetes0ngwins	
  
Thank	
  You	
  

Jus)n	
  Rondeau	
  
Chief	
  Tes0ng	
  Evangelist	
  &	
  Editor	
  
WhichTestWon	
  

                                                      #websitetes0ngwins	
  
Website	
  Tes)ng	
  Wins	
  

Peter	
  Borden	
  
Marke0ng	
  Manager	
  
@Monetate	
  

                                         #websitetes0ngwins	
  
#websitetes0ngwins	
  
#websitetes0ngwins	
  
#websitetes0ngwins	
  
#websitetes0ngwins	
  
#websitetes0ngwins	
  
#websitetes0ngwins	
  
#websitetes0ngwins	
  
#websitetes0ngwins	
  
#websitetes0ngwins	
  
#websitetes0ngwins	
  
#websitetes0ngwins	
  
#websitetes0ngwins	
  
Thank	
  You	
  

Peter	
  Borden	
  
Marke0ng	
  Manager	
  
@Monetate	
  

                            #websitetes0ngwins	
  
Website	
  Tes)ng	
  Wins	
  

Ques)ons	
  &	
  Answers	
  

                                              #websitetes0ngwins	
  

Website Testing Wins!

  • 1.
    Website  Tes)ng  Wins   October  10,  2012  -­‐  2:00  pm  ET   Follow  @monetate  on     #websitetes0ngwins  
  • 2.
    Tips  for  Webinar  A8endees   •  Audio  problems?  Click  the  “Problem  Dialing  In”  link  or  switch  to   the  webinar  phone  bridge  and  make  sure  your  phone  is  muted.     •  Use  the  “Ques0on”  box  within  GoToWebinar  to  ask  a  ques0on  or   use  the  webinar  hashtag  #websitetes0ngwins     •  A  recording  of  the  webinar  with  the  slide  deck  will  be  available  at   monetate.com/webinars  within  1  or  2  business  days.   #websitetes0ngwins  
  • 3.
    Today’s  Presenters   Jus0n  Rondeau   Chief  Tes0ng  Evangelist  &  Editor   WhichTestWon           Peter  Borden   Marke0ng  Manager   Monetate   #websitetes0ngwins  
  • 4.
    Website  Tes)ng  Wins   Jus)n  Rondeau   Chief  Tes0ng  Evangelist  &  Editor   WhichTestWon   #websitetes0ngwins  
  • 5.
    Industry Analyst &Hands on Trainer ©2012, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved. #websitetes0ngwins  
  • 6.
    Case  Study  #1:  Apparel  Category  Page   Image       ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.       #websitetes0ngwins  
  • 7.
    Winner   Results:  44%  Lia  in   Purchases  &    67%  Lia   in  RPV     Brand:  Jules.fr     Lesson:  Test  Your   Category  Page   Images   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.       #websitetes0ngwins  
  • 8.
    Case  Study  #2:  Category  Page  Display  Test   Default  10  Items   Default  50  Items   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      
  • 9.
    Winner!   Default  50  Items   Results:  35%  Lia  in   Purchases     Brand:  Palmers     Lesson:  Default   Sehngs  Aren’t   Necessarily  Op0mal   Sehngs   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.       #websitetes0ngwins  
  • 10.
    Case  Study  #3:  Geoloca)on  Test   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.       #websitetes0ngwins  
  • 11.
    Winner   Results:  2.43%  Increase   in  Sales     Brand:  KarmaLoop   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.       #websitetes0ngwins  
  • 12.
    Case  Study  #4:  Cart  BuOon  Copy     ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.  All  rights  reserved.     #websitetes0ngwins  
  • 13.
    Winner   87.5%  Lia  in  Completed   Form  Submissions   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.  All  rights  reserved.     #websitetes0ngwins  
  • 14.
    Case  Study  #5:  Search  BuOon  Placement   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.       #websitetes0ngwins  
  • 15.
    Winner   Results:  12%  Lia  in  Sales  contribu0ng   to  a  2.3  Million  lia  in  annual  revenue     Brand:  Mountain  Equipment  Co-­‐op   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.       #websitetes0ngwins  
  • 16.
    Case  Study  #6:  Travel  Search  Results     ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      
  • 17.
    Winner!   Results:  16.36%  Lia  in  Paid   Bookings     Brand:  KRAS   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.       #websitetes0ngwins  
  • 18.
    Case  Study  #7:  Product  Page  Test   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.       #websitetes0ngwins  
  • 19.
    Winner   None!  Surprised?   Winner?   Results:  No  Sta0s0cally   Significant  Difference     Brand:  Design3000   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.       #websitetes0ngwins  
  • 20.
    Our  Intui)on  is  Poor   “Neplix  considers  90%  of  what  they  try  to  be  wrong”  -­‐-­‐   Mike  Moran   “80%  of  the  0me  you/we  are  wrong  about  what  a   customer  wants”    -­‐-­‐  Avinash  Kaushik   Microsoa’s  sta0s0c  from  thousands  of  controlled   experiments:   only  10-­‐30%  of  experiments  move  the  metrics  they  were   designed  to  improve   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.       #websitetes0ngwins  
  • 21.
    Case  Study  #8:  Security  Indicia  Test   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.  All  rights  reserved.     #websitetes0ngwins  
  • 22.
    Winner   Results:   1%  Lia  in  Sales       Big  Win  Save  on   Cost  for  Security   Icons     ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.  All  rights  reserved.     #websitetes0ngwins  
  • 23.
    Case  Study  #9:  Copy  Test     ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.       #websitetes0ngwins  
  • 24.
    Closer  Look   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.  All  rights  reserved.     #websitetes0ngwins  
  • 25.
    Winner   Seasonal  Test  –  Focused   On  Sales  for  the  Holidays     Simple  Copy  –  Highligh0ng   The  benefit  of  the  product     Simple  to  Implement  with   Sizeable  Lia   18.7%  Lia  in  Sales  from     Nov  22nd  –  Dec  15th     60.7%  Increase    from     Dec  16th  to  Dec  23rd   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.  All  rights  reserved.     #websitetes0ngwins  
  • 26.
    Case  Study  #10:  PPC  Landing  Page   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.       #websitetes0ngwins  
  • 27.
    Winner   Lia  –  74%  Lia  in  Completed  Car   Rentals     Reminder  –  Use  Landing  Pages   &  Keep  Message  Consistent   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.  All  rights  reserved.    
  • 28.
    Case  Study  #11:  Product  of  the  Week   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.       #websitetes0ngwins  
  • 29.
    Winner   Results:  3.6%  Lia  in  RPV     Brand:  3Suisses     Lesson:  Cross  Sells  must  be   relevant   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.  All  rights  reserved.     #websitetes0ngwins  
  • 30.
    Case  Study  #12:  Landing  Page  Cross  Sell   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.  All  rights  reserved.     #websitetes0ngwins  
  • 31.
    Winner   Results:  Significant  Lia   in  Sales  (asked  not  to   disclose)     Brand:  BestStunGun     Lesson:  Less  expensive   cross  sells  can  ease   s0cker  shock   ©2012,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.  All  rights  reserved.     #websitetes0ngwins  
  • 32.
    Thank  You   Jus)n  Rondeau   Chief  Tes0ng  Evangelist  &  Editor   WhichTestWon   #websitetes0ngwins  
  • 33.
    Website  Tes)ng  Wins   Peter  Borden   Marke0ng  Manager   @Monetate   #websitetes0ngwins  
  • 34.
  • 35.
  • 36.
  • 37.
  • 38.
  • 39.
  • 40.
  • 41.
  • 42.
  • 43.
  • 44.
  • 45.
  • 46.
    Thank  You   Peter  Borden   Marke0ng  Manager   @Monetate   #websitetes0ngwins  
  • 47.
    Website  Tes)ng  Wins   Ques)ons  &  Answers   #websitetes0ngwins