IntroSetup story – MSFT backgroundPaint frustrations/challenges:New line of businessOpinions and guessing3 comp and pick 1 approachQuestions they were askingWho is our customer/market?What content, messaging and layouts will maximize conversions?How do move away from guessing?Maybe you’ve had questions like theseToday I’ll be talking aboutI’m guessing that you’re probably
Helped unlock ancient Egyptian
Correct usage:Revolutionary testing – “big ideas” Major layout changes + redesignsTo answer simple questionsTesting funnels/processIncorrect usage:Evolutionary testing - small changesTest more than 1 change When insight on why something won or what really mattersPros:Easy – only 1 variableFast set-upQuick to executeSimple analysisLower trafficAbility to split trafficCons:Limited to 1 variableLimits innovative thinkingLack of insight what won / lostRestrictive Design heavy relianceBuild-out overhead
We thought it would be beneficial to our audience to get into the proper usage and pros/cons of the different optimization methods.You can think of A/B testing as training wheels and a foundational keystone for general optimization. Everyone should jump in and at a minimum start with A/B. A logical starting point is layout/template testing before getting into testing individual elements.
Advantages:Easy – only tests one change at a timeFast to set up and less QACan be used for quick tests that involve only simple changes to pages (changes made to a copy of the control page)Simple - easy determination of the winner between two. Minimal knowledge of statistics required.Lower traffic requirements - when only 2 experiments are tested.Ability to split traffic disproportionately to experiment pages.DisadvantagesCan only accurately test one change at a timeInhibits innovative thinking when there is only one chance per test to find a major improvementLack of Granularity - Cannot pinpoint what elements caused a page to win or lose.Restrictive – if multiple changes are made in one test it is too easy to accidentally skip beneficial elements – this leaves too many questions unanswered.Testing Scope - Restricted to testing whole pages, and multiple element testing requires highly inefficient sequential A/B Split testing.Limited Design Challenges - requires two or more highly distinct page template designs Build Challenges – multiple complete HTML page build outs from comps take a great deal of time.Testing more than 2 pages might cause more complex statistical knowledge to determine clear winners when confidence intervals overlap.
MVT works best if the operator formulates hypotheses about their audience and tests to see whether these are true. Testing variations of content to best match your audience’s expectations and to drive the desired action.
Difference between customization (My.Yahoo) and Personalization and what’s required to do the latter.Past and future market situation - Discuss the market situation and the legacy bias toward Off Site targeting – ad network behavioral targeting (e.g. Adwords re-targeting) and email re-targeting and personalization. Onsite has been a challenge historically as it required IT resources to track, explore, define and extract segments and changing onsite content was also an IT (CMS) owned/controlled function. With Webtrends Segments and Optimize, marketers can do this on their own.
Engage EMEA 2010 - Site Optimisation Components
Site Optimisation Components<br />Bob Garcia, Webtrends<br />
Website Optimisation Involves…<br />Ongoing discipline striving to improve content relevancein order to maximize conversions.<br />Site Optimisation Tactics:<br />a/b testing<br />multivariate testing<br />analytics<br />segmentation<br />targeting<br />personalisation<br />
What Can be Optimised?<br />Typical areas of focus:<br />LEAD GENERATION<br />SELF-SERVICE<br />ECOMMERCE<br />CONTENT<br />Opportunities:<br />websites<br />mobile<br />facebook<br />carts/funnels<br />segments<br />
Design heavy reliance</li></li></ul><li>MVTMultivariate Testing<br />Overview:<br /><ul><li>Simultaneous testing of multiple variations of multiple elements content to determine most effective combo. </li></ul>When to use: <br /><ul><li>To learn what combinations of messaging, content and position work best. </li></li></ul><li><ul><li>Test multiple elements concurrently
Onsite Targeting Overview<br />Targeting=Segments+Content= The BigR’s<br />
Visualizing Segments<br />New vs. Returning<br />4 KeyProducts<br />8 MajorMarkets<br />16 VisitorPersonas<br />All<br />100% of visitors<br />100% of visitors<br />16 segments = all <br />1 segment = all <br /> Segments : Share one or more preferences or characteristics <br />
What is Personalisation?<br />Process of customizing content to individuals/segments based on their characteristics and/or preferences.<br />FUEL = Visitor Behavioral Data<br />Visitor<br /><ul><li>Email address