MVT Across Multiple Brands: One approach to successful testing, Webtrends

471 views

Published on

Published in: Technology, Design
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
471
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

MVT Across Multiple Brands: One approach to successful testing, Webtrends

  1. 1. MVT Across Multiple Brands: One approach to successful testing Martin Sheerin Head of Online Product Development, Phones4U Michael Schirrmacher UK Sales Director, Webtrends Webtrends | © 2011 Webtrends, All Rights Reserved.
  2. 2. Why should we Optimize? #1. Improve Customer Experience #2. Improve Conversion & Revenue #3. Deepen Visitor Engagement © 2011 Webtrends, All Rights Reserved. |2
  3. 3. © 2011 Webtrends, All Rights Reserved. |3
  4. 4. Where do I start? Analytics Optimization Identify and quantify opportunities through insight Test, segment and target to improve performance © 2011 Webtrends, All Rights Reserved. |4
  5. 5. What Optimisation Involves? © 2011 Webtrends, All Rights Reserved. |5
  6. 6. Full Factorial….Fractional factorial….? “ok this looks complex” © 2011 Webtrends, All Rights Reserved. |6
  7. 7. So, for now…a managed service is still more popular © 2011 Webtrends, All Rights Reserved. |7
  8. 8. Optimization Maturity Model Operational Reporting “Crawl ” Phase “Walk” Phase “Run” Phase • Personalization Conversions • Web Analytics • A/B/n Testing • More Advanced MVT • Measurement • Basic MVT -1-3 areas • In-session Segmentation • 1 Element Targeting • Multi Element Targeting • 1-Project at a Time • Frequent • No Testing • Ad Network Targeting • Integrated Optimization • External Segment Use • Social & MultiChannel • On-going Optimization Maturity Phases © 2011 Webtrends, All Rights Reserved. |8
  9. 9. Multivariate Test Control £ © 2011 Webtrends, All Rights Reserved. |9
  10. 10. Multivariate Test Control Challenger £ © 2011 Webtrends, All Rights Reserved. | 10
  11. 11. Multivariate Test +£3.2m Lift in special offers sales in UK © 2011 Webtrends, All Rights Reserved. | 11
  12. 12. Martin Sheerin Head of Online Product Development Phones 4U @martinsheerin1 linkedin.com/in/martinsheerin1
  13. 13. Phones 4u • • • • • Multi-channel retailer Over 600 stores Contact centre for sales and service Website: www.phones4u.co.uk Mobile site: m.phones4u.co.uk • Online only brand: – www.dialaphone.co.uk – m.dialaphone.co.uk
  14. 14. Why should you MVT? New idea of creative Build on gut instinct with science Campaign Support Increase Conversion De-risk Projects Avoid mistakes
  15. 15. Why not MVT yourself instead of using a partner? • • • • Time Skills Ensuring that a test is statistically significant Ensures the result is true and not biased to an internal individuals viewpoint • You can do this yourself:
  16. 16. What to look for in an MVT provider? Business understanding Business alignment eCommerce insight Ease of reporting
  17. 17. Our Approach • Test simultaneously on both sites • Use results of test on one site and apply to the other (if its possible) • Be happy to repeat tests annually (or more regularly if you deem necessary)
  18. 18. Our Test Successes
  19. 19. Free Gifts 6.27% Optimal Control Page lift
  20. 20. Customer Behaviour Phones 4u Research Purchase Choose Handset Dialaphone Select Package Allowance (Lifestyle)
  21. 21. Phones 4u Review Tab Control Winner +2.99%
  22. 22. Search Results +16.56%
  23. 23. Simple and Usable Source: Simple and Usable Web, Mobile, and Interaction Design (Voices That Matter) - Giles Colborne (CX Partners) (2010)
  24. 24. Search Results Optimal Control Page +16.56% Experiment Page Lift Control Optimal --+16.56%
  25. 25. Upgrades - Variants
  26. 26. +81.64% Optimal Control Page Lift at next step Experiment Page Lift Control Optimal --+81.64%
  27. 27. white background Level 5 – grey background Level 4 – white background and upgrade promise Level 3 – control grey background and upgrade promise Level 2- Taking this finding we applied it to Phones 4u Optimal
  28. 28. Phones 4u Winning Variant +70.64 % Optimal Control Page Lift at next step Experiment Page Control Optimal Lift --+70.64%
  29. 29. Not satisfied we went with a hunch… Optimal Control Page +0.89% Experiment Page Lift Control Optimal --+0.89%
  30. 30. Moving on with the upgrades test
  31. 31. Control Page Multi-device testing
  32. 32. Applying this idea to Phones 4u
  33. 33. All Traffic (non-iPad) Optimal Control Page Optimal Experiment Page Control Optimal Lift --+4.11%
  34. 34. iPad Version Control Page Optimal Optimal Experiment Page Control Optimal Lift --+4.27%
  35. 35. Summary Apply common sense Only test what you are willing to put live Don’t be satisfied that the result of one test cannot be improved upon Re-test if you think its necessary Sometimes the simplest things give the biggest results Geo targeting Persona driven segmentation Touch commerce (Multi Device) Define what success looks like
  36. 36. Martin Sheerin Head of Online Product Development Phones 4U @martinsheerin1 linkedin.com/in/martinsheerin1

×