Upcoming SlideShare
×

Like this presentation? Why not share!

Multivariate DIY - Daigneault

on Apr 09, 2010

• 1,131 views

If you're not doing multivariate (MVT) optimization tests on your landing pages, you could be leaving hundreds of thousands of dollars on the table....

If you're not doing multivariate (MVT) optimization tests on your landing pages, you could be leaving hundreds of thousands of dollars on the table.

But for those that have checked out MVT, all those tools and consultants can get expensive pretty darn quick. No matter how great the return on investment, sometimes there just isn't any money.

Enter MVT DIY-style! or, multivariate optimization "do it yourself" style. This session will show you how to use Google's free tools so that you can setup your own tests.

Views

Total Views
1,131
Views on SlideShare
1,128
Embed Views
3

Likes
1
21
0

1 Embed3

 http://www.slideshare.net 3

Report content

• Comment goes here.
Are you sure you want to
• M+R Powerpoint Template
• M+R Powerpoint Template
• M+R Powerpoint Template
• M+R Powerpoint Template
• M+R Powerpoint Template

Multivariate DIY - DaigneaultPresentation Transcript

• Multivariate Optimization Do-It-Yourself Edition [150] April 9, 2010 Steve Daigneault
• Goals
• Solid understanding of multivariate optimization and why it’s so great
• Overview of how Google’s free optimization tool works
• Examples of recent tests and results
• Resources and suggested next steps for you to run your own tests
• What is multivariate optimization?
• We all know A/B tests
• Create two versions
• Split traffic
• Watch response rates
• Your landing page submit Trust Mrs. Peacock. She knows how to light candlesticks without killing anyone. Give to the Library
• Things you want to test submit
• Save the Conservatory
• Colonel Mustard Needs You
PICTURE: 2 alternatives
• FIELDS: 1 alternative
• Remove 1 field
Donate now!
• BUTTON: 1 alternative
• Bigger and orange
• How would you A/B test this?
• Set-up different landing pages for:
• Headline: 2 alternatives + 1 control = 3
• Picture: 2 alternatives + 1 control = 3
• Fields: 1 alternative + 1 control = 2
• Button: 1 alternative + 1 control = 2
• 3 x 3 x 2 x 2 = 36 pages
• 36 – 1 control = 35 new pages to build
• Test everything at once
• Input all elements at once
• Software creates many page variations
• Traffic split among page variations
• Takes into account interactions between all the elements
• Fancy math tracks the effect each individual item has on conversions
Multivariate > A/B testing
• HOW DOES OPTIMIZER WORK?
• Actually builds all page variations
• Some software will build fewer pages + use predictive modeling
• Optimizer = free + great for basic tests
• Statistically significant results using the same traffic but using it more wisely
• Optimizes conversions (not average gift*)
• Optimizer then builds page variations, splits traffic and calculates changes in behavior.
Overview
• Choose the page you want to improve.* EX: donation page
• Identify your conversion page.* EX: confirmation page
• Identify page elements you want to test. EX: button and header image *these pages will get tracking code.
7 Steps to Test
• Create variations. EX: new graphic buttons and headers
• Add Optimizer code around the elements you’re testing on your test page.
• Review and launch.
7 Steps to Test
• SHOW ME RESULTS
• Sample results: Page variations
• Estimated conversion rate range
• provides the most immediate insight into overall performance. This column shows you how well each combination is performing relative to your original content.
• Chance to beat original
• the probability that a particular combination will be more successful than your original content. Above 95% (or below 5%) is considered significant. Above 99% (or below 1%) is considered “high confidence”.
• Chance to beat all
• the probability that a combination will be more the successful than all the other combinations running in the experiment. Usually a lower number because it’s competing against many page variations vs. just the original.
• Observed improvement
• displays the percent improvement over the original combination.
• Conversions/impressions
• the raw data of how many conversions and page views a particular combination generated. (You want 200+)
• Sample results: Page elements
• Relevance rating
• shows how much impact a particular section has on your experiment. 0 means it had little to no impact. 5 means it was a major distinguishing factor.
• RECENT TEST RESULTS
• vs.
• vs.
• vs.
• #6: new nav and new button
• #5: new nav and new gift string
• original button
Winner:
• How much traffic do I need?
• How many things can I test?
Traffic Calculator
• How much could improvements help?
• Optimizer Engineer Blog www.gwotricks.com/default.htm
• Technical overview gives you a deeper understanding of the tool