Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016


Published on

Success and failure with conversion rate optimization largely depends on how well a company is able to embrace a culture of experimentation. Companies that fail with ongoing optimization are generally those that treat it as a side project, so that CRO efforts gain little organizational support. In this presentation, Sean will discuss how leading companies collect unique insights from across the organization, and provide full transparency into the testing process and results. Creating a company wide culture of experimentation requires patience, but the payoff is well worth the effort. Companies with experiment-driven cultures outperform their peers in everything from scaling customer acquisition channels to changing the way all team members focus their efforts towards high value activities.

Published in: Marketing
  • Be the first to comment

Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016

  1. 1. Building a Company-Wide Culture of Experimentation May 18, 2016 Founder & CEO GrowthHackers
  2. 2. Sept 2008: Dropbox Blank Slate • Excellent product coming out of Beta • Team of 7 engineers • Skeptical of marketing • Goal: Create culture of growth experimentation #ConvCon
  3. 3. Seeds of Testing Culture • Built tracking and testing infrastructure • Shared insights (qual and quant) with team • Shared testing wins with team • Encouraged broader participation #ConvCon
  4. 4. Today Dropbox has ½ Billion Users • No marketer from April 2009 – January 2010 • Product/engineers continued to experiment #ConvCon
  5. 5. Growth is Company Wide Effort Product External Channels Promise 1st Experience Ongoing Experience Acquisition Activation Revenue Retention Referral Marketing PR Support Success biz dev CRO or Growth Team #ConvCon
  6. 6. AKA Growth Hacking Rapid experimentation across the full funnel to learn the most effective ways to scale sustainable customer adoption. #ConvCon
  7. 7. Testing Drives Growth • No certainty about what works • Rapid testing across all levers • More testing = More learning Acquisition Referral Activation Revenue Retention Resurrection #ConvCon
  8. 8. Twitter Example #ConvCon
  9. 9. Change Harder Established Company • Day onsite at Microsoft (May 2016) • Goal: Transform to culture of growth experimentation • Founded 1975, 118K employees • Results: TBD #ConvCon
  10. 10. Steps to Experimentation Culture Broad Participation Transparent Learning Effective Process #ConvCon
  11. 11. Process First Broad Participation Transparent Learning Effective Process #ConvCon
  12. 12. Effective Process Idea Backlog Prioritize backlog Launch Tests Analyze Results #ConvCon
  13. 13. Core Team to Manage Process • Start with product manager • Add dedicated people to core as needed to hit testing tempo goals • Core often includes designers, growth engineers, analysts… #ConvCon
  14. 14. Effective Process Idea Backlog Prioritize backlog Launch Tests Analyze Results #ConvCon
  15. 15. Goal: Build Large Backlog of Ideas • Build large backlog of ideas #ConvCon
  16. 16. Prioritize Backlog Unbridled Ideation Prioritize backlog Launch Tests Analyze Results #ConvCon
  17. 17. Select Tests in Weekly Meeting • Use system for prioritization (we use ICE) • Choose tests to launch (no brainstorming) IMPACT CONFIDENCE EASE (1 – 10) (1 – 10) (1 – 10) #ConvCon
  18. 18. Launch Tests Unbridled Ideation Prioritize backlog Launch Tests Capture learning #ConvCon
  19. 19. Launch Tests • Assign a project manager to each test • Use tools and allocate skills needed to implement #ConvCon
  20. 20. Analyze Results Unbridled Ideation Prioritize backlog Launch tests Analyze Results #ConvCon
  21. 21. Analyze Results • Ensure tracking up front for hypothesis • Run sufficient time • Triage analysis by hypothesis first #ConvCon
  22. 22. Example Weekly Meeting Agenda 15 min: KPI review & update focus area 10 min: Review last week’s testing sprint 15 min: Key lessons learned from analyzed tests 15 min: Select tests for this week’s sprint 5 min: Check growth of idea backlog #ConvCon
  23. 23. Requires Transparent Learning Broad Participation Transparent Learning Effective Process #ConvCon
  24. 24. Transparent Learning • Every test leads to learning • Proactively share with idea originator • Share wins with opt in email list • Organize learning for everyone’s easy access #ConvCon
  25. 25. Drive Broad Participation Broad Participation Transparent Learning Effective Process #ConvCon
  26. 26. Company Kickoff • CEO: “everyone seek opportunities for improvement and growth” • Share recent wins and insights • Idea input training • Pizza and beer brainstorm #ConvCon
  27. 27. Collect Ideas Widely External Experts Internal Experts Ideas Tests Results Core Growth Team Drives Process #ConvCon
  28. 28. Standard Experiment Doc for Ideas • Include research, hypothesis, target lever #ConvCon
  29. 29. Provide Feedback on Ideas • Acknowledge all ideas • Praise favorite ideas • Notify if selected • Report back results #ConvCon
  30. 30. Keep it Fun #ConvCon
  31. 31. Idea from Engineer - oEmbed • Data: Embeds = longer sessions • Hypothesis: Same action = more embeds • Result: 41% increase in embedded media Slides Videos #ConvCon
  32. 32. Idea from Analyst: To-do-list • Data: low % of newly registered users commenting in 1st week • Hypothesis: Increase % of first week commenters • Test: Use a To-do-list to guide new users (start with easy activities) #ConvCon
  33. 33. Result: To-do-list • Result: over 700% lift in 1st week commenting #ConvCon
  34. 34. Key Takeaways • Share learning & celebrate wins • Empower full team to generate ideas • Ensure core team has process to manage inputs #ConvCon
  35. 35. Join GH Projects Private Beta at Where Teams Drive Results with High Tempo Testing: 1) Team Ideation 2) Prioritize Testing 3) Capture Learning #ConvCon