Building a Best in Class Testing Program
•Download as PPTX, PDF•
2 likes•1,130 views
Optimizely - Stream:20 webinar of Dec. 4th 2014 In this webinar Alia Formoy, Sr. Digital Marketing Consultant at Stream:20, explains how to structure your tests and build a best in class testing program. She advocates a process of Exploration as opposed to Refinement. Alia explains a 5 step process for effective optimisation: 1. Define Success 2. Identify Bottlenecks 3. Prioritise 4. Test 5. Optimise She will dive into each of these steps and explain it with many examples. For those of you who are struggling with formulating hypotheses, I recommend you to look at the part on Identifying Bottlenecks! For a recording of the full webinar, go to: https://www.youtube.com/watch?v=SO-MI0rxiu0 For more info we recommend that you check out: www.optimizely.com/gettingstarted/ www.stream20.com
Recommended
Recommended
More Related Content
More from Optimizely
More from Optimizely (20)
Recently uploaded
Recently uploaded (20)
Building a Best in Class Testing Program
- 1. Build a Best In Class Testing Program - WEBINAR Alia Formoy
- 2. Who are Stream:20? Digital Sales Planning Digital Organisation Digital Sales Toolkit Digital Sales Delivery Optimising your business structure and skillsets for digital success Ensuring the correct toolkits are implemented for success Providing innovative strategic support and analysis to drive your digital sales function Driving incremental revenue across Desktop/ Mobile/Tablet The Performance-Driven Digital Marketing Consultancy We push up KPIs by rolling out good practice & best practice on the client side London Munich San Francisco
- 3. Who we work with Retail Finance Gaming Telecoms Travel Entertainment Publishing Technology Utilities
- 4. This webinar – what we are going to learn today Testing allows us to improve and optimise all our marketing channels and, importantly, the website. By the end of this webinar, you will be able to: • Build an effective Test Plan • Roll out and refine • Hit your targets for success •Carrying out tests in a planned and strategic way. 1. Testing •Understanding the impact of tests and the results they produce. 2. Analysis •Running further tests to validate and improve results, and incorporating findings into BAU 3. Optimisation
- 5. What testing approach do you use?
- 6. The Testing Process 4. Test • Run the test! Log into Optimizely and click "Create New Experiment." 3. Prioritise • Of the many things you could test, which one makes the most sense to start with? 2. Identify bottlenecks • How do users get from landing on your site to a "success" state? • Where do users drop out or stop moving along the success path? 1. Define success • What metrics do you use to quantify success? • What user actions do you consider "success"?
- 7. 1. Define success: Objectives • First step should be the easiest: What do you want to improve? • What are some of the possible objectives for testing? • Align with overall business objectives Business Goals • Hit Sales target Marketing Goals • Convert more prospects into customers Optimisation Goals • Reduce drop out from payment journey 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success
- 8. 2. Identify bottlenecks 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success
- 9. You should begin your planning with looking at where to test. Use your analytics to uncover the following: Exit Exit Exit Exit 2. Identify bottlenecks: Where do you test? Exit Customermovement Entry to Site Product Page Shopping Basket Your Information Review Order Order Completed 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success • Top landing pages • Top exit pages • Top highest bounce rate pages • Top abandonment points in your funnel • Top most valuable pages to your business • Top converting pages • Lowest converting pages
- 10. 2. Identify bottlenecks: Test Structure • Observation: This is the problem you are having which keeps you from success. • Hypothesis: A way to overcome the problem and get to success. • One of the keys to effective testing is to test a hypothesis – to help us achieve an objective • Using data and your experience / best-practice knowledge to determine what aspects might make a difference in achieving the objectives 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success
- 11. 2. Identify bottlenecks: Shortcuts Observation: Customers are forced to go from: Test idea: Insert an ‘Add to Basket’ button to the products listed. 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success Category Product detail Basket Hypothesis: Allowing customers to skip the middle step would increase conversion to sale
- 12. 2. Identify bottlenecks: Guidance Observation: There is a lack of clarity of which fields are mandatory and what each field means. Hypothesis: Guiding people through the form process will help increase conversion. Test idea: Use ‘tooltip’ image and hover text to provide guidance as you fill out the form, rather than error messages after each submission 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success
- 13. 2. Identify bottlenecks: CTA Test: Insert additional ‘Add to basket’ button at bottom of reviews section. 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success Observation: Clicking on Product reviews pushes you to the bottom of the Product Page where there is no CTA. Hypothesis: Making it easy for those reading reviews to add to basket would increase conversion.
- 14. 2. Identify bottlenecks: High Consideration Test: Push Webchat functionality harder in navigation and as a timed pop up Observation: Analytics show prospects are going to FAQs section and not returning. 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success Hypothesis: Prospects need more interactive help during the quote process.
- 15. 2. Identify bottlenecks: Mobile KPIs Test: Add a link to the subscription page in the header of the page. Observation: These mobile sites are media focused but Time Inc’s KPIs are subscriptions and mobile conversion is low 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success Hypothesis: Pushing a subscription CTA as a focus will drive mobile conversion up
- 16. 2. Identify bottlenecks: Social Trust Observation: Not being aware of product quality and seller reputation is a barrier to conversion. 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success Test: Build trust in the product and seller by using user generated reviews as social proof. Hypothesis: Reinforcing ways to show quality of product and seller will increase conversion.
- 17. 2. Identify bottlenecks: Social Satisfaction Observation: Buying wine online has a barrier of not knowing whether it will be good quality. Test: Influence buying behaviour by using customer satisfaction as social proof. 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success Hypothesis: Highlighting likelihood of previous consumers of buying again would decrease the barrier to conversion.
- 18. 2. Identify bottlenecks: Social Urgency Observation: Conversion from searching for flights to sale can be quite low. Test: Influence buying behaviour with urgency using live data from the website. 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success Hypothesis: Creating a sense of urgency will increase conversion.
- 19. 2. Identify bottlenecks: Device Targeting Hypothesis: Using device specific calls to action may increase conversion. Observation: The conversation to quote rate from the iPad audience was much lower than from Desktop. 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success Test: Use iPad style ‘swipe button’ for the CTA
- 20. 2. Identify bottlenecks: Data Capture 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success Observation: Betfair’s Newsletter sign ups have slowed down on website. Hypothesis: Driving opt ins with an incentive may increase signs ups and conversion. Test: Push sign ups using a hero campaign and bonus code as an incentive to bet.
- 22. 3. Prioritise: Benefits of the Hypothesis Process 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success A good hypothesis is crucial to CRO, otherwise you just test random stuff blindly. The hypotheses you generate from insight and research will result in a list of possible solutions. The solutions will form the basis of your test plan.
- 23. 3. Prioritise: Building the Testing plan Map out testing priorities based on: • Business impact – how hard is it to run this test? • Performance impact – how much could this test improve performance? Commit to what your measure of success will be Commit to when you measure performance Repeat tests to exclude any false results and plan when results will be rolled out and become BAU Work out what assets are needed to run this test 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success Your list of potential solutions based on your hypotheses.
- 24. 4. Test 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success
- 25. 4. Test: Analysis made easy • Optimizely will make it easier for you to know when test results are statistically significant so you can clearly see which variation wins. 4. Test3. Prioritise 2. Identify bottlenecks 1. Define success
- 26. The Optimisation Process Set Goals Gather Insights Form Hypothesis Test Analyse Implement Learnings 1. Define Success 2. Identify Bottlenecks 3. Prioritise 4. Test 5. Optimise
- 27. 5. Optimise 5. Optimise4. Test3. Prioritise 2. Identify bottlenecks 1. Define success
- 28. The benefits of Analysis Original: Sell Build Trust + Sell Desktop Mobile Tablet Sell Sell+ build trust email CR: 2% Even if a test doesn’t give you an immediate win, if it’s planned and analysed right, you’ll learn something that can be fed into the next test round. CR: 6% CR: 7% CR: 3% CR: 5% CR: 2% CR: 3% 5. Optimise 4. Test 3. Prioritise 2. Identify bottleneck s 1. Define success A winning A/B test result is often just the beginning of the test’s value. There’s much more to gain from a smart analysis of the results.
- 29. How the plan builds 5. Optimise 4. Test 3. Prioritise 2. Identify bottleneck s 1. Define success
- 31. What to take away • Exploration is a better test approach than refinement • Align your tests to your business goals • Identifying your barriers is vitally important • Observe, hypothesise and form ideas • Ideas can have many sources • Write a plan and expect it to change • Optimizely makes testing easier • Dig deeper for learnings • There are plenty of insights in failed tests
Editor's Notes
- We have a diverse client base. We know that conversion in retail is fundamentally different to insurance, gaming etc. but we understand the nuances across industries (big or small) and can show the principles remain the same. The problems have been solved before.
- Testing helps prove What are we teaching you today: 1 how to build a plan through testing
- We go in to lots of companies and they think they are doing testing but they are not – they are refining. When you refine, you take one variation and optimise it until it can’t get any better. Problem – is they have unbeatable landing pages but a terrible conversion rate. The solution is exploration. Take a wider view and try out lots of ideas – let the data decide.
- This is an important slide – pay attention, I will refer back to this again as I go along.
- It’s funny, this may all sound obvious but it’s easily forgotten. By being more specific and defining this well, your testing becomes more structured.
- Do your research before you start testing – analytics is a really good place to start, but you can also use info from surveys, focus groups and usability studies. You want a holistic view of what is going on on your website.
- And next up we’ll go into real world examples of formulating a hypothesis.
- Doing something like this is really easy within Optimizely, using the code and image from the Product Page and inserting it into the Test variation without needing dev team resource.
- Now insurance is a high consideration product so this example isn’t as relevant to low price retail products. A complex product will generate more questions.
- Another benefit of Optimizely is being able to test the mobile version of your website.
- Testing and seeing one variation win is just the tip of the iceberg. You have to dig deeper to understand why.