Optimizely Customer Showcase

862 views

Published on

JEREMY YOCHES, SENIOR PRODUCT MANAGER, STELLA & DOT
MARK HALVORSON, HEAD OF INTERACTIVE MARKETING, ATLASSIAN
CHARLOTTE WEISKITTEL, SENIOR DIRECTOR, DONORSCHOOSE.ORG

Trying to get more traction for testing at your company? Wondering how to build and scale your team most effectively? Looking for your next big testing win? In this session, we hear directly from testers who have tackled the same challenges.

This session brings together Optimizely users from Atlassian, Stella & Dot, and DonorsChoose.org for a behind--the--scenes look at testing at their organizations. Coming from a variety of industries and company sizes, each speaker presents a unique perspective on how they've built and scaled testing programs successfully.

You'll walk away armed with interesting testing ideas and actionable steps you can take to build a data-driven testing culture at any organization.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
862
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
8
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Optimizely Customer Showcase

  1. 1. Mark  Halvorson   Head  of  Interac2ve   Marke2ng,  Atlassian   @halv0012     Charlo=e   Weiski=el   Sr  Director,  Product   DonorsChoose.org   @noDornaught     Jeremy  Yoches   Sr  Product  Manager,   Stella  &  Dot       Customer  Showcase  
  2. 2. Charlo=e  Weiski=el   DonorsChoose.org  
  3. 3. About  DonorsChoose.org  
  4. 4. A  tes0ng  team  of  one  
  5. 5. Tes0ng  process  
  6. 6. Monthly  dona0ons  
  7. 7. Remove  $500  and  $1000   •  Removing  $500  and   $1000  increased   conversion  by  58%.    
  8. 8. Photo  and  tagline  (mul0variate  test)   •  Female  teacher  performed  best  with  3  of  4  taglines   •  “Join  3,500  people…”  did  slightly  be=er  than  the  original,  but  it  wasn’t  significant.   1.  A  teacher  spends  $40  each  month  on  classroom  essen2als.  (original)   2.  Make  giving  a  monthly  habit.   3.  Join  the  3,500  people  who  support  a  classroom  every  month.   4.  Help  a  classroom  every  month.  
  9. 9. Charge  your  card  now  or  in  a  month?   •  Made  the  change  to  wait   a  month  before  we  had   Op2mizely   •  Removing  men2on  of  the   next  month  increased   conversion  +217%.  
  10. 10. $50,  $25  or  no  incen0ve   •  Conversion  rate   –  $50  incen2ve:  1.8%   –  No  incen2ve:  1.6%   –  $25  incen2ve:  1.3%   •  The  no  incen0ve  group   was  the  least  likely  to   cancel  a`er  two  months   (8%)   •  The  no  incen2ve  group   had  the  highest  average   monthly  signup  amount   of  $26.  
  11. 11. Tips  from  a  lean  non-­‐profit   •  Just  start  tes2ng   •  Clear  the  path  for  the  tester   •  Be  a  broken  record   •  Look  at  the  results  holis2cally,  not  just  A  vs.  B   •  Don’t  overfill  tes2ng  road  map      
  12. 12. Mark  Halvorson   Atlassian  
  13. 13. About  Atlassian   We  help  great  teams  build  be=er  so`ware,  together.   •  Products  include:  JIRA,  Confluence,  Bitbucket,   HipChat…   •  Founded  in  2002   •  Over  35,000  Customers   •  Over  700  employees  globally   •  No  sales  people    
  14. 14. A/B  Tes0ng  at  Atlassian  is  cross  func0onal   Central  Growth  Hacking  Team  to  support:   •  Email     •  Product  Management     •  Customer  PlaDorm     •  Customer  Advocates   •  Internal  Systems         •  Leads   •  Product  Marke2ng   •  Support   •  Bitbucket   •  HipChat  
  15. 15. High  Level  view  of  sta0s0cs.  Blimp  Dashboard  
  16. 16. Examples  
  17. 17. Reduce  Fric0on  
  18. 18. Increase  Visibility  
  19. 19. Test  drive  campaigns  
  20. 20. A   Highlighted  Example  :  Simplify   B  
  21. 21. Lather,  rinse,  repeat   Our  Process   •  Collect  great  ideas   •  Triage  based  on  poten2al  impact   •  Build-­‐out  good  candidates   •  Schedule  to  avoid  conflicts   •  Run  experiments  &  analyze  results.    
  22. 22. Where  do  experiment  ideas  come   from?  
  23. 23. Anywhere.  
  24. 24. Our  Test  Philosophy   •  Ideas  are  everywhere   •  Everything  affec2ng  conversion  is  tested   •  No  traffic  wasted   •  Be  aggressive   •  Regular  cadence   •  Never  run  a  test  that  you  wouldn’t  want  to  win    
  25. 25. Types  of  tests   •  Pebbles   – Change  in  copy  or  CTA  presenta2on   •  Rocks   – New  page  (e.g.  no  Dev  Tools  family  page)   •  Boulders   – Simplified  order  form.    
  26. 26. Experiment  Illumina0  
  27. 27. Priori0zed  Backlog  
  28. 28. JIRA  Ticket  
  29. 29. Corresponding  Test  Page  
  30. 30. Results  Archive  
  31. 31. Tes2ng  @  Stella  &  Dot   Jeremy  Yoches  
  32. 32. About  Stella  &  Dot   •   Stella  &  Dot  helps  women  become  successful   entrepreneurs  with  a  thriving  business  selling   bou2que-­‐style  jewelry  and  bags  through  in-­‐  home   trunk  shows,  online  and  person-­‐to-­‐person.     •  Today  there  are  over  17,000  Independent  Stylists   who  sell  the  accessories  in  the  US,  Canada,  Puerto   Rico,  UK,  France  and  Germany.   •  In  2009,  retail  sales  were  $33  million.  In  2010,   $103  million.  In  2011,  over  $175  million.  In  2012,   sales  hit  $200  million.  And  in  2013  sales  were  $220   million.      
  33. 33. The  Stella  &  Dot  Tes0ng  Team    Jeremy  Yoches  –  Full  0me  product  manager,  and   also  full  0me  conversion  op0mizer   •  Own  the  test  plan/roadmap   •  Develop  and  set  up  tests   •  Analyze,  compile,  share  and  evangelize  results   •  Collaborate  with  teams  as  needed   – Crea2ve  to  create  new  assets   – Dev  help  for  more  complicated  javascript/CSS  
  34. 34. The  Stella  &  Dot  Collabora0on  Model     Share  the  plan,  the  status,  and  the  history   Make  it  easy  for  anyone  in  the  company  to:   •  see  what  tests  are  live  where   •  inves0gate  live  and  past  tests  in  more  detail   •  Propose  new  test  ideas    
  35. 35. Trello  as  collabora0on  hub  
  36. 36. The  Stella  &  Dot  Tes0ng  Success  Story   Many  small  successes  and  important  insights,   A  few  big  wins:   The  Super  Nav  Tests  -­‐   Op0mize  our  “triple  funnel”  by  tes0ng  the  primary   CTAs  in  our  header    
  37. 37. Countless  Varia0ons  -­‐   bueon  combina0ons  and  copy   Original  Super  Nav   Super  Nav  Test  
  38. 38. Super  Nav  Test  Results   3  CTA’s  =  3  compe0ng  goals   Increased  clicks  on  all  3  by  more  than  30%             And  we  are  just  gejng  started…   •  Color,  size,  shape,  etc…  
  39. 39. Mark  Halvorson   Head  of  Interac2ve   Marke2ng,  Atlassian   @halv0012     Charlo=e   Weiski=el   Sr  Director,  Product   DonorsChoose.org   @noDornaught     Jeremy  Yoches   Sr  Product  Manager,   Stella  &  Dot       Q&A  and  thank  you  

×