Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Opticon 2015-Optimization at Optimizely

1,211 views

Published on

Featuring speakers from Optimizely

Ash Alhashim, Global Director, Sales and Market Development, Optimizely
Amanda Swan, Community Manager, Optimizely
Adam Avramescu, Head of Customer Education, Optimizely

Ever wondered how Optimizely employees use Optimizely? In this session, you’ll learn about our optimization programs, and how we use Optimizely beyond traditional A/B testing. We’ll also uncover how to optimize experiences that don’t have a traditional “conversion” goal. Our Global Director of Sales and Market Development, Ash Alhashim, will walk through an advanced experiment that takes advantage of Optimizely’s most powerful features and how he takes a unique approach to measuring success, perfect for optimizers looking to push their optimization boundaries. Amanda Swan and Adam Avramescu will also share insights from the Optiverse testing program and thoughts on interpreting experiments on properties where “conversion” is unclear.

Published in: Business
  • Be the first to comment

Opticon 2015-Optimization at Optimizely

  1. 1. #opticon2015 Optimization at Optimizely Ash Alhashim Director of Sales & Market Development, Optimizely @ashalhashim ash@optimizely.com Amanda Swan Community Manager, Optimizely @swan606 amanda@optimizely.com Adam Avramescu Head of Customer Education, Optimizely @avramescu adam.avramescu@optimizely.com
  2. 2. We optimize a lot. Today, let’s focus on one slice of the pie.
  3. 3. When experimenting, there can be many sources of complexity When testing, there can be many sources of complexity
  4. 4. A few examples…
  5. 5. #opticon2015 Optimizing Optiverse Amanda Swan Community Manager, Optimizely @swan606 amanda@optimizely.com Adam Avramescu Head of Customer Education, Optimizely @avramescu adam.avramescu@optimizely.com
  6. 6. What is Optiverse? Optimizely’s Community, Knowledge Base, & Academy
  7. 7. What you’ll get out of this portion • Trying to optimize your support site or community? • Testing search or nav structures? • What do you do with low sample size (low conversions)? • How do you optimize without clear conversion funnels?
  8. 8. Effective Communication Channels are Crucial
  9. 9. 4 Steps
  10. 10. Expose themes through qualitative feedback
  11. 11. Test assumptions through conceptual validation
  12. 12. Analyze results for directions & preferences
  13. 13. Loop back to qualitative validation and expand the hypothesis
  14. 14. Optiverse Search
  15. 15. Hypothesis By removing non-essential filters, and making the essential ones more prominent, we can increase filter usage, improving the relevance of results.
 
 Are “content type” filters actually useful and relevant to visitors?
  16. 16. Exposing themes through qualitative feedback
  17. 17. Testing assumptions through conceptual validation
  18. 18. Testing assumptions through conceptual validation
  19. 19. Analyze results for directions & preferences
  20. 20. Analyze results for directions & preferences Support Clicks > Academy & Community Clicks
  21. 21. Loop back to qualitative validation & expand the hypothesis
  22. 22. Loop back to qualitative validation & expand the hypothesis
  23. 23. Loop back to qualitative validation & expand the hypothesis
  24. 24. The Knowledge Base
  25. 25. Hypothesis The existing search experience and relevant sidebar content are less helpful for visitors to find the right support content.
 
 If we create search-first and FAQ-first experiences, will visitors engage with these options more?
  26. 26. Exposing themes through qualitative feedback Search SidebarDrawers
  27. 27. Testing assumptions through conceptual validation Search
  28. 28. Testing assumptions through conceptual validation Sidebar
  29. 29. Testing assumptions through conceptual validation Sidebar Search
  30. 30. Analyze results for directions & preferences
  31. 31. Analyze results for directions & preferences
  32. 32. Analyze results for directions & preferences
  33. 33. Analyze results for directions & preferences
  34. 34. Analyze results for directions & preferences
  35. 35. • Implement prominent search • Search-first vs. search AND sidebar • Different types of FAQ content • What would this look like without the drawers? Loop back to qualitative validation & expand the hypothesis
  36. 36. Summing Up the 4 Steps • Expose themes through qualitative feedback • Test assumptions through conceptual validation • Analyze results for directions & preferences • Loop back to qualitative validation & expand the hypothesis LEARN from every experiment
  37. 37. #opticon2015 Optimizing for Reductions & Operational Efficiencies Ash Alhashim @ashalhashim ash@optimizely.com Director of Sales & Market Development
  38. 38. Background
  39. 39. gmrfmrf@fgjkd.co 6969@aol.com hahahaha@gmail.com poop@poop.com areyou@kidding.me fdfoijdsdjs@fdaf.com fdfoijdsdjs@fdaf.com whatevzzz@gmail.com Background leavemealone@ymail.com
  40. 40. How can we reduce errors without negatively impacting lead gen numbers?
  41. 41. Adding email verification to our lead form will increase the overall lead quality, and allow Sales Development Representatives to spend their time on more revenue generating activity. Hypothesis
  42. 42. All non-signed-in visitors to the homepage that click on “Test it Out” will enter the experiment Audience
  43. 43. Original Variation #1
  44. 44. The Recipe
  45. 45. Ingredients
  46. 46. Ingredients
  47. 47. Body text goes here.
  48. 48. Body text goes here.
  49. 49. Body text goes here.
  50. 50. Ingredients
  51. 51. Body text goes here.
  52. 52. Body text goes here.
  53. 53. The Recipe: Preparation
  54. 54. The Recipe: Preparation • Test launched using conditional activation mode • Stopping Condition: 33,570 visitors • Double blind experiment
  55. 55. Goals
  56. 56. Goals Primary Goal: • Sales Accepted Leads
  57. 57. Goals Primary Goal: • Sales Accepted Leads Secondary Goals: • Lead/Account Creation • Avg. Infer Lead Score • Opportunity Creation • Customer Creation
  58. 58. Results
  59. 59. Body text goes here. Results Statistical
  60. 60. KeyTakeaways • Most experiments result in a statistical tie – plan accordingly • Define success before you run an experiment • Test deep into your funnel, we have the technology • Don’t only optimize for CR lift; test for quality as well
  61. 61. #opticon2015 Ash Alhashim Director of Sales & Market Development, Optimizely @ashalhashim ash@optimizely.com Amanda Swan Community Manager, Optimizely @swan606 amanda@optimizely.com Adam Avramescu Head of Customer Education, Optimizely @avramescu adam.avramescu@optimilzey.com Thank you!

×