0
Mark	
  Halvorson	
  
Head	
  of	
  Interac2ve	
  
Marke2ng,	
  Atlassian	
  
@halv0012 	
  	
  
Charlo=e	
  
Weiski=el	
 ...
Charlo=e	
  Weiski=el	
  
DonorsChoose.org	
  
About	
  DonorsChoose.org	
  
A	
  tes0ng	
  team	
  of	
  one	
  
Tes0ng	
  process	
  
Monthly	
  dona0ons	
  
Remove	
  $500	
  and	
  $1000	
  
•  Removing	
  $500	
  and	
  
$1000	
  increased	
  
conversion	
  by	
  58%.	
  
	
  
Photo	
  and	
  tagline	
  (mul0variate	
  test)	
  
•  Female	
  teacher	
  performed	
  best	
  with	
  3	
  of	
  4	
  ...
Charge	
  your	
  card	
  now	
  or	
  in	
  a	
  month?	
  
•  Made	
  the	
  change	
  to	
  wait	
  
a	
  month	
  befo...
$50,	
  $25	
  or	
  no	
  incen0ve	
  
•  Conversion	
  rate	
  
–  $50	
  incen2ve:	
  1.8%	
  
–  No	
  incen2ve:	
  1....
Tips	
  from	
  a	
  lean	
  non-­‐profit	
  
•  Just	
  start	
  tes2ng	
  
•  Clear	
  the	
  path	
  for	
  the	
  teste...
Mark	
  Halvorson	
  
Atlassian	
  
About	
  Atlassian	
  
We	
  help	
  great	
  teams	
  build	
  be=er	
  so`ware,	
  together.	
  
•  Products	
  include:...
A/B	
  Tes0ng	
  at	
  Atlassian	
  is	
  cross	
  func0onal	
  
Central	
  Growth	
  Hacking	
  Team	
  to	
  support:	
 ...
High	
  Level	
  view	
  of	
  sta0s0cs.	
  Blimp	
  Dashboard	
  
Examples	
  
Reduce	
  Fric0on	
  
Increase	
  Visibility	
  
Test	
  drive	
  campaigns	
  
A	
  
Highlighted	
  Example	
  :	
  Simplify	
  
B	
  
Lather,	
  rinse,	
  repeat	
  
Our	
  Process	
  
•  Collect	
  great	
  ideas	
  
•  Triage	
  based	
  on	
  poten2al	
...
Where	
  do	
  experiment	
  ideas	
  come	
  
from?	
  
Anywhere.	
  
Our	
  Test	
  Philosophy	
  
•  Ideas	
  are	
  everywhere	
  
•  Everything	
  affec2ng	
  conversion	
  is	
  tested	
  ...
Types	
  of	
  tests	
  
•  Pebbles	
  
– Change	
  in	
  copy	
  or	
  CTA	
  presenta2on	
  
•  Rocks	
  
– New	
  page	...
Experiment	
  Illumina0	
  
Priori0zed	
  Backlog	
  
JIRA	
  Ticket	
  
Corresponding	
  Test	
  Page	
  
Results	
  Archive	
  
Tes2ng	
  @	
  Stella	
  &	
  Dot	
  
Jeremy	
  Yoches	
  
About	
  Stella	
  &	
  Dot	
  
•  	
  Stella	
  &	
  Dot	
  helps	
  women	
  become	
  successful	
  
entrepreneurs	
  w...
The	
  Stella	
  &	
  Dot	
  Tes0ng	
  Team	
  
	
  Jeremy	
  Yoches	
  –	
  Full	
  0me	
  product	
  manager,	
  and	
  ...
The	
  Stella	
  &	
  Dot	
  Collabora0on	
  Model	
  	
  
Share	
  the	
  plan,	
  the	
  status,	
  and	
  the	
  histor...
Trello	
  as	
  collabora0on	
  hub	
  
The	
  Stella	
  &	
  Dot	
  Tes0ng	
  Success	
  Story	
  
Many	
  small	
  successes	
  and	
  important	
  insights,	
 ...
Countless	
  Varia0ons	
  -­‐	
  
bueon	
  combina0ons	
  and	
  copy	
  
Original	
  Super	
  Nav	
  
Super	
  Nav	
  Tes...
Super	
  Nav	
  Test	
  Results	
  
3	
  CTA’s	
  =	
  3	
  compe0ng	
  goals	
  
Increased	
  clicks	
  on	
  all	
  3	
 ...
Mark	
  Halvorson	
  
Head	
  of	
  Interac2ve	
  
Marke2ng,	
  Atlassian	
  
@halv0012 	
  	
  
Charlo=e	
  
Weiski=el	
 ...
Upcoming SlideShare
Loading in...5
×

Optimizely Customer Showcase

606

Published on

JEREMY YOCHES, SENIOR PRODUCT MANAGER, STELLA & DOT
MARK HALVORSON, HEAD OF INTERACTIVE MARKETING, ATLASSIAN
CHARLOTTE WEISKITTEL, SENIOR DIRECTOR, DONORSCHOOSE.ORG

Trying to get more traction for testing at your company? Wondering how to build and scale your team most effectively? Looking for your next big testing win? In this session, we hear directly from testers who have tackled the same challenges.

This session brings together Optimizely users from Atlassian, Stella & Dot, and DonorsChoose.org for a behind--the--scenes look at testing at their organizations. Coming from a variety of industries and company sizes, each speaker presents a unique perspective on how they've built and scaled testing programs successfully.

You'll walk away armed with interesting testing ideas and actionable steps you can take to build a data-driven testing culture at any organization.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
606
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "Optimizely Customer Showcase"

  1. 1. Mark  Halvorson   Head  of  Interac2ve   Marke2ng,  Atlassian   @halv0012     Charlo=e   Weiski=el   Sr  Director,  Product   DonorsChoose.org   @noDornaught     Jeremy  Yoches   Sr  Product  Manager,   Stella  &  Dot       Customer  Showcase  
  2. 2. Charlo=e  Weiski=el   DonorsChoose.org  
  3. 3. About  DonorsChoose.org  
  4. 4. A  tes0ng  team  of  one  
  5. 5. Tes0ng  process  
  6. 6. Monthly  dona0ons  
  7. 7. Remove  $500  and  $1000   •  Removing  $500  and   $1000  increased   conversion  by  58%.    
  8. 8. Photo  and  tagline  (mul0variate  test)   •  Female  teacher  performed  best  with  3  of  4  taglines   •  “Join  3,500  people…”  did  slightly  be=er  than  the  original,  but  it  wasn’t  significant.   1.  A  teacher  spends  $40  each  month  on  classroom  essen2als.  (original)   2.  Make  giving  a  monthly  habit.   3.  Join  the  3,500  people  who  support  a  classroom  every  month.   4.  Help  a  classroom  every  month.  
  9. 9. Charge  your  card  now  or  in  a  month?   •  Made  the  change  to  wait   a  month  before  we  had   Op2mizely   •  Removing  men2on  of  the   next  month  increased   conversion  +217%.  
  10. 10. $50,  $25  or  no  incen0ve   •  Conversion  rate   –  $50  incen2ve:  1.8%   –  No  incen2ve:  1.6%   –  $25  incen2ve:  1.3%   •  The  no  incen0ve  group   was  the  least  likely  to   cancel  a`er  two  months   (8%)   •  The  no  incen2ve  group   had  the  highest  average   monthly  signup  amount   of  $26.  
  11. 11. Tips  from  a  lean  non-­‐profit   •  Just  start  tes2ng   •  Clear  the  path  for  the  tester   •  Be  a  broken  record   •  Look  at  the  results  holis2cally,  not  just  A  vs.  B   •  Don’t  overfill  tes2ng  road  map      
  12. 12. Mark  Halvorson   Atlassian  
  13. 13. About  Atlassian   We  help  great  teams  build  be=er  so`ware,  together.   •  Products  include:  JIRA,  Confluence,  Bitbucket,   HipChat…   •  Founded  in  2002   •  Over  35,000  Customers   •  Over  700  employees  globally   •  No  sales  people    
  14. 14. A/B  Tes0ng  at  Atlassian  is  cross  func0onal   Central  Growth  Hacking  Team  to  support:   •  Email     •  Product  Management     •  Customer  PlaDorm     •  Customer  Advocates   •  Internal  Systems         •  Leads   •  Product  Marke2ng   •  Support   •  Bitbucket   •  HipChat  
  15. 15. High  Level  view  of  sta0s0cs.  Blimp  Dashboard  
  16. 16. Examples  
  17. 17. Reduce  Fric0on  
  18. 18. Increase  Visibility  
  19. 19. Test  drive  campaigns  
  20. 20. A   Highlighted  Example  :  Simplify   B  
  21. 21. Lather,  rinse,  repeat   Our  Process   •  Collect  great  ideas   •  Triage  based  on  poten2al  impact   •  Build-­‐out  good  candidates   •  Schedule  to  avoid  conflicts   •  Run  experiments  &  analyze  results.    
  22. 22. Where  do  experiment  ideas  come   from?  
  23. 23. Anywhere.  
  24. 24. Our  Test  Philosophy   •  Ideas  are  everywhere   •  Everything  affec2ng  conversion  is  tested   •  No  traffic  wasted   •  Be  aggressive   •  Regular  cadence   •  Never  run  a  test  that  you  wouldn’t  want  to  win    
  25. 25. Types  of  tests   •  Pebbles   – Change  in  copy  or  CTA  presenta2on   •  Rocks   – New  page  (e.g.  no  Dev  Tools  family  page)   •  Boulders   – Simplified  order  form.    
  26. 26. Experiment  Illumina0  
  27. 27. Priori0zed  Backlog  
  28. 28. JIRA  Ticket  
  29. 29. Corresponding  Test  Page  
  30. 30. Results  Archive  
  31. 31. Tes2ng  @  Stella  &  Dot   Jeremy  Yoches  
  32. 32. About  Stella  &  Dot   •   Stella  &  Dot  helps  women  become  successful   entrepreneurs  with  a  thriving  business  selling   bou2que-­‐style  jewelry  and  bags  through  in-­‐  home   trunk  shows,  online  and  person-­‐to-­‐person.     •  Today  there  are  over  17,000  Independent  Stylists   who  sell  the  accessories  in  the  US,  Canada,  Puerto   Rico,  UK,  France  and  Germany.   •  In  2009,  retail  sales  were  $33  million.  In  2010,   $103  million.  In  2011,  over  $175  million.  In  2012,   sales  hit  $200  million.  And  in  2013  sales  were  $220   million.      
  33. 33. The  Stella  &  Dot  Tes0ng  Team    Jeremy  Yoches  –  Full  0me  product  manager,  and   also  full  0me  conversion  op0mizer   •  Own  the  test  plan/roadmap   •  Develop  and  set  up  tests   •  Analyze,  compile,  share  and  evangelize  results   •  Collaborate  with  teams  as  needed   – Crea2ve  to  create  new  assets   – Dev  help  for  more  complicated  javascript/CSS  
  34. 34. The  Stella  &  Dot  Collabora0on  Model     Share  the  plan,  the  status,  and  the  history   Make  it  easy  for  anyone  in  the  company  to:   •  see  what  tests  are  live  where   •  inves0gate  live  and  past  tests  in  more  detail   •  Propose  new  test  ideas    
  35. 35. Trello  as  collabora0on  hub  
  36. 36. The  Stella  &  Dot  Tes0ng  Success  Story   Many  small  successes  and  important  insights,   A  few  big  wins:   The  Super  Nav  Tests  -­‐   Op0mize  our  “triple  funnel”  by  tes0ng  the  primary   CTAs  in  our  header    
  37. 37. Countless  Varia0ons  -­‐   bueon  combina0ons  and  copy   Original  Super  Nav   Super  Nav  Test  
  38. 38. Super  Nav  Test  Results   3  CTA’s  =  3  compe0ng  goals   Increased  clicks  on  all  3  by  more  than  30%             And  we  are  just  gejng  started…   •  Color,  size,  shape,  etc…  
  39. 39. Mark  Halvorson   Head  of  Interac2ve   Marke2ng,  Atlassian   @halv0012     Charlo=e   Weiski=el   Sr  Director,  Product   DonorsChoose.org   @noDornaught     Jeremy  Yoches   Sr  Product  Manager,   Stella  &  Dot       Q&A  and  thank  you  
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×