10 Tactics for Building an Optimization Culture

3,257 views
3,069 views

Published on

Slides from a presentation of '10 Tactics for Building an Optimization Culture' webinar, hosted by Brooks Bell and Optimizely.

Full webinar recording with audio can be found here: http://optimizely.wistia.com/medias/xf4yk47rml

https://www.optimizely.com/
http://brooksbell.com/

Published in: Business

10 Tactics for Building an Optimization Culture

  1. 1. 10 Tactics for Building an Optimization Culture OPTIMIZELY & BROOKS BELL 3.4.2014
  2. 2. Welcome! Gregory Ng Chief Marketing Officer Brooks Bell @GregoryNg Kyle Rush Head of Optimization Optimizely @kylerush
  3. 3. 10 Tactics for Developing a Successful Experimentation Culture
  4. 4. What we’ll cover ... 1. Best practices from top testing organizations 2. The people behind testing 3. How to organize your program 4. How to keep the process manageable 5. How to include others and build excitement
  5. 5. Successful experimentation culture must be data-driven.
  6. 6. What does it mean to be data-driven? Progress is compelled by data, rather than by intuition or personal experience. Evidence-based decision making. Using data to inform key decisions.
  7. 7. How data-driven are you, actually? • Do you dictate strategy without referencing data? • Do you rely on the person with the most experience to make the ultimate call? • Do you use words like “think” and “feel” when talking about strategy? • Does your team trust the data?
  8. 8. How do you establish a successful testing culture?
  9. 9. 10 Tactics 1. Clear organizational structure 2. Identify the right talent 3. Establish stakeholders 4. Solicit test ideas from everyone 5. Prioritize tests 6. Share results and successes 7. Build a knowledge base 8. Don’t rush into MVT tests 9. Balance learning and “wins” 10.Go beyond CRO
  10. 10. #1 – Choose the right organization structure • No “one-size-fits-all” • One or multi-person team • Centralized or decentralized? Members of the Obama for America Team
  11. 11. #1 – Choose the right organization structure • Army of One • Ringleader • Fellowship • Kingdom
  12. 12. #2 – Pick the Right Talent • Qualitative and quantitative combination • Curiosity, creativity • People skills • Analytical mindset VP of Marketing E-Commerce Director Program Manager, Retail Optimizely’s customers come from a variety of backgrounds Product Manager
  13. 13. #3 – Establish Stakeholders Early On • Identify the correct stakeholders • Secure an executive sponsor • Develop an approval process for tests • Communicate and share successes
  14. 14. #4 – Solicit Test Ideas from Everyone • Make it easy to submit ideas • Capture interest after sharing results • Follow up for every idea • Create a scoreboard • Attribute experiments to team members Example: Optimizely Scorecard Vote on predicted winning variations
  15. 15. #5 - Know how to Prioritize Tests ESTABLISH A SCORECARD •Relevance to business goals (KPIs) •Potential impact •Potential for learning •Level of effort •Time to significance and available traffic •Likelihood of implementation
  16. 16. #6 – Share Results and Successes • • • • • Make it easy to understand Clarify the primary goal Share 100 percent of results, regardless of outcome Standing meetings Keep score Presenting A/B test findings at company all-hands
  17. 17. #7 – Build a Knowledge Base • Gain broader insights • Lay a foundation for growth • Eliminate single point of failure • Increase accountability • Standardize processes and documentation
  18. 18. #8 – Don’t Rush Into MVT Tests • Ideal for fine-tuning optimization • More complicated but less effective • Fewer learnings
  19. 19. #9 – Balance Learning and “Wins” • • • • Always start with a hypothesis Never settle for a flat result Invitation for deep analysis Segment results, target accordingly
  20. 20. #10 – Go Beyond CRO Optimization doesn’t need to occur in isolation. Almost anyone can test. For instance ...
  21. 21. #10 – Go Beyond CRO • Designers Variation A Variation B Walk designers through A/B tests so they can inform their designs with data
  22. 22. #10 – Go Beyond CRO • Engineers Launch of OFA 2012 New Donation API A/B testing the new platform revealed an architectural bug that was missed in QA
  23. 23. #10 – Go Beyond CRO • Email and Content Marketers A/B Tested Emails and Landing Pages for Subscribers #dataculture
  24. 24. 1. Clear organizational structure 2. Identify the right talent 3. Establish stakeholders 4. Solicit test ideas from everyone In summary ... 5. Prioritize tests 6. Share results and successes 7. Build a knowledge base 8. Don’t rush into MVT tests 9. Balance learning and “wins” 10.Go beyond CRO
  25. 25. Thank you for reading!

×