1.
Microsoft Case Study: 40% Conversion Rate Increase of SEM Traffic using Multivariate Testing Michael Schultz Director Product Management & Marketing, Microsoft Office Unmanaged Services Frans Keylard Director of Optimization, Widemile

2.
Agenda <ul><li>A/B & Multivariate Testing Overview </li></ul><ul><li>Microsoft Office Live Small Business Case Study </li></ul><ul><ul><li>Business Overview </li></ul></ul><ul><ul><li>Case Study Details </li></ul></ul><ul><li>Top 10 Page Optimization Tips </li></ul>

3.
Conversion Optimization * Source: eMarketer – April 2009 Investment recommendation: 5% to 15% of acquisition budget Testing and segmentation can improve retention performance Automated, data-driven action that increases revenue by optimizing a user’s online experience. $37 Billion annual US spend by 2013. $23 Billion spent in 2008.* Customer Sales Funnel <ul><li>Email marketing </li></ul><ul><li>Loyalty Programs </li></ul><ul><li>Newsletters </li></ul><ul><li>Personalization </li></ul><ul><li>Coupons /Surveys </li></ul>Widemile Focus <ul><li>Key web pages </li></ul><ul><li>Sales funnels </li></ul><ul><li>Landing pages </li></ul><ul><li>Registration pages </li></ul><ul><li>Search marketing </li></ul><ul><li>Display advertising </li></ul><ul><li>Affiliate marketing </li></ul><ul><li>Contextual ads </li></ul><ul><li>Ad targeting </li></ul>Conversion Acquisition Retention

4.
Testing Options – Approach Overview Ideal use: > Validates traffic and conversion rates Limitation : > Simple measurement; no optimization Basic Analytics A/B Split Testing Ideal use: > Best for testing two or more radically different pages <i.e. template testing> Limitation : > No insight into why one page is better, just which won among traffic tested Multivariate Testing (MVT ) Ideal use: > Test different content, simultaneously determining most effective combination > Segmentation can be applied for more granular optimization in follow-on tests Limitation : > Requires sufficient traffic & conversions

5.
MVT Testing Options - Full & Fractional Factorial Fractional Factorial Full Factorial

6.
Requirements for Success Expertise User Interface Proven Architecture Reporting Interface Analyst Creative Web Developer Technology

7.
2009 – Year for Conversion Optimization

8.
Microsoft Office Live Small Business <ul><li>Audience: </li></ul><ul><ul><ul><li>Small businesses, 1-10 employees (real estate agent, local bakery, video store, restaurant) </li></ul></ul></ul><ul><ul><ul><li>Non-profits/Organizations (community groups, soccer clubs, PTSAs) </li></ul></ul></ul><ul><li>Benefits: </li></ul><ul><ul><ul><li>Free website and email </li></ul></ul></ul><ul><ul><ul><li>Free services to view, store and share documents online </li></ul></ul></ul>

9.
Goals Summary <ul><li>Goals: </li></ul><ul><ul><ul><li>Improve sign up conversions </li></ul></ul></ul><ul><ul><ul><li>Better understand audience </li></ul></ul></ul><ul><ul><ul><li>Identify highest impact page factors </li></ul></ul></ul><ul><li>Testing Strategy: </li></ul><ul><ul><ul><li>Leverage proven landing page template </li></ul></ul></ul><ul><ul><ul><li>Execute multivariate test </li></ul></ul></ul><ul><ul><ul><li>Fractional factorial approach to minimize time </li></ul></ul></ul>

10.
Microsoft Case Study <ul><li>Getting started… </li></ul><ul><li>Define page to be tested </li></ul><ul><li>Analyze outside factors </li></ul><ul><ul><li>In-bound traffic </li></ul></ul><ul><ul><li>Traffic volume </li></ul></ul><ul><ul><li>UX - Funnel </li></ul></ul><ul><li>Review page & conversion goals </li></ul>Original web page

11.
Testing Considerations What is the best hero shot? Does the headline grab the customer’s attention? Do these resource links belong here? Does the button maximize conversions? Does this copy help or hurt?

12.
Indentifying Factors to Test 1 2 3 4 5

13.
Defining a Test Matrix No. Factor (Variable) Levels (versions) 1 Hero Shot 4 2 Title 4 3 Description 4 4 Call-To-Action Button 4 5 Resource Links 2

14.
Audience Participation – Best CTA Button Level 2 Level 3 Level 4 Level 1

15.
Efficiency Through Technology 1 of 16 2 of 16 3 of 16 4 of 16 16

16.
And the Winner is… 40% Improvement

17.
Which Factors Made the Biggest Difference Hero Shot Title Description Call-To-Action Resource Link Location High Influence Medium Influence Low Influence

18.
High Influence Factor - Key Learning <ul><li>Factor/level insights </li></ul><ul><li>“ Get Started Free” won </li></ul><ul><li>The free aspect of the service is very important to get people to try it out </li></ul><ul><li>More task focused terminology like “Set up my business!” or “Build My Online Business” lost </li></ul><ul><li>“ Sign Up Free!” which is the typical text found on so many CTA buttons also lost out. </li></ul><ul><li>Future testing around the positioning of the FREE service </li></ul>

19.
Case Study Summary <ul><li>Microsoft Office Live Small Business’ testing and optimizing investment: </li></ul><ul><ul><li>Delivered a 40% lift in conversion </li></ul></ul><ul><ul><li>Reduced customer acquisition cost </li></ul></ul><ul><ul><li>Maximized SEM performance </li></ul></ul><ul><ul><li>Testing and optimizing content is a fast and effective </li></ul></ul><ul><ul><li>way to maximize the performance of your SEM spend. </li></ul></ul>

20.
10 Optimization Tips <ul><li>1. Offer Clarity </li></ul><ul><ul><li>Make sure the offer is clear, relevant, and compelling </li></ul></ul><ul><ul><li>Keep your pages simple, clear, and appealing </li></ul></ul><ul><li>2. Path to Conversion </li></ul><ul><ul><li>Make sure the path to conversion is simple and straightforward </li></ul></ul><ul><ul><li>Keep conversion paths efficient, focused, uncluttered </li></ul></ul><ul><li>3. Performance </li></ul><ul><ul><li>Prospects will not tolerate slow pages or links </li></ul></ul><ul><ul><li>Google penalizes slow page loads </li></ul></ul><ul><li>4. Test Design and User Experience </li></ul><ul><ul><li>Avoid confusing landing page designs and difficult to read text </li></ul></ul><ul><ul><li>Make transitions seamless and keep the consistency high (both messaging and look & feel) </li></ul></ul><ul><li>5. Seasonality </li></ul><ul><ul><li>Seasonally optimize your pages and PPC campaign </li></ul></ul><ul><ul><li>Leverage historical SEM in planning current and future campaigns </li></ul></ul>

21.
10 Optimization Tips <ul><li>6. Target Audience </li></ul><ul><ul><li>Optimize your content based on audience insight </li></ul></ul><ul><ul><li>Capture contact information early to better inform your marketing </li></ul></ul><ul><li>7. Segment Audience </li></ul><ul><ul><li>Optimize for distinct audiences to convert </li></ul></ul><ul><ul><li>Segment your traffic and test to see if unique landing pages would perform better </li></ul></ul><ul><li>8. Offers and Goals </li></ul><ul><ul><li>Create and test offers that have a high perceived value but a low cost to fulfill </li></ul></ul><ul><ul><li>Force your page to have only one clear goal and link ads to landing pages </li></ul></ul><ul><li>9. Consistency </li></ul><ul><ul><li>Build consistency and relevancy between PPC campaigns and where you send your traffic </li></ul></ul><ul><ul><li>Reinforce the call to action/offer on order and lead forms </li></ul></ul><ul><li>10. Competition </li></ul><ul><ul><li>Look at what your competition is doing, but also what they are not doing </li></ul></ul><ul><ul><li>Differentiation can often help, so test new approaches, offers, and incentives </li></ul></ul>

22.
4 Additional Tips for Marketers <ul><li>1. Drink the Kool-Aid and start now! </li></ul><ul><li>2. Think of multivariate testing as a “fixed” marketing expense and allocate budget appropriately </li></ul><ul><li>3. Ensure collaboration between Creative, Optimization Testing and SEM </li></ul><ul><li>4. Make sure Optimization vendor learns the business and understands the audience </li></ul>

23.
Questions? <ul><li>Presenters </li></ul><ul><li>Frans Keylard – [email_address] </li></ul><ul><li>Michael Schultz – [email_address] </li></ul><ul><li>Content Information </li></ul><ul><li>www.widemile.com </li></ul><ul><li>www.officelive.com </li></ul>