Congratulations to our
Outperform Award Winners!
Most Dramatic
Business Impact
Most Customer-
Obsessed Company
Culture
Most Transformative
Innovation
Most Inspiring Social
Impact
2
 Welcome to Day 2 of Opticon!
 Sessions will be recorded and available after Opticon
 Share your feedback and rate sessions on the mobile app
Nina Bayatti
Conversion Marketing Manager, ClassPass
Driving Customer Acquisition &
Growth with Experimentation
4
About me
5
Agenda
 ClassPass’ iterative, minimum viable product (MVP)
approach to testing
 Demystifying conversion rate optimization (CRO)
 Applying CRO principles company-wide
 Q&A
6
ClassPass is the
largest studio
fitness aggregator
7
We’re not just testing
new fitness classes.
In 2018, we ran over
150 optimization tests
company-wide.
8
How did we run 150
optimization tests
company-wide?
9
Build minimum viable
product (MVP)
• How can we test this
idea in a low-lift, high-
impact way?
• De-risk resource
investment
• Maintain high test
velocity
Boost test velocity & de-risk resources with an
iterative, MVP approach
10
Build minimum viable
product (MVP)
• How can we test this
idea in a low-lift, high-
impact way?
• De-risk resource
investment
• Maintain high test
velocity
Launch & monitor
• Ask CX team to flag
test-related outreach
• Monitor web behavior
& analytics
Boost test velocity & de-risk resources with an
iterative, MVP approach
11
Build minimum viable
product (MVP)
• How can we test this
idea in a low-lift, high-
impact way?
• De-risk resource
investment
• Maintain high test
velocity
Launch & monitor
• Ask CX team to flag
test-related outreach
• Monitor web behavior
& analytics
Iterate &
re-launch
• Anticipate 2-3
iterations ahead
• Continue to utilize
MVP-first approach
• Utilize initial test data,
customer feedback &
web analytics
Boost test velocity & de-risk resources with an
iterative, MVP approach
12
Build minimum viable
product (MVP)
• How can we test this
idea in a low-lift, high-
impact way?
• De-risk resource
investment
• Maintain high test
velocity
Launch & monitor
• Ask CX team to flag
test-related outreach
• Monitor web behavior
& analytics
Iterate &
re-launch
• Anticipate 2-3
iterations ahead
• Continue to utilize
MVP-first approach
• Utilize initial test data,
customer feedback &
web analytics
Productize
• Share learnings
• Consider impact
when allocating
resourcing for
productization build
Boost test velocity & de-risk resources with an
iterative, MVP approach
13
Demystifying CRO: Beyond the CTA color test
14
Understanding down-
funnel impact of your
CRO initiatives
15
• Bounce rate
• Click through rate
• Pageviews
• Lead captures
• Purchase conversion rate
Your primary KPI is not
telling the full story
16
Reach a confident
conclusion with down-
funnel data
• Quality of leads
• Duration of sales cycle
• Customer acquisition cost
• Operational cost
• Customer retention rate
• Bounce rate
• Click through rate
• Pageviews
• Lead captures
• Purchase conversion rate
Your primary KPI is not
telling the full story
17
Measuring the down-
funnel impact of our
re-engagement
optimization
18
We tested extending our
winback offer
• Winback offer serves as a re-trial period for
inactive members
• Inactive members can experience new
product features & newly added classes
• Tested extending winback offer period from
1 to 2 weeks
19
We tested extending our
winback offer
Hypothesis: If we extend our winback offer from 1 to 2
weeks, then our reactivation rates will increase
because members will have more time to experience
new features and classes. And the lift in reactivations
will offset the costs of the increased duration of the
free offer.
20
Extending our winback offer lifted our
reactivation rate by 25%
Reactivations +25%
ReactivationCVR
B: Extended OfferA: Control
21
Despite the lift in reactivations, our marginal
acquisition was not cost-effective
Control Extended Offer Delta
Reactivations* 100 125 +25% CVR
Cost of Goods (COGs) $15,000 $24,375
+30%
(increased usage)
Retained Members 80 95 -5% CVR
Customer Acquisition Cost $187.50
Marginal CAC $625 > ⅓ Lifetime Value (LTV)
*Reactivation counts are illustrative to better highlight the impact of the purchase CVR, retention CVR and increased cost of
the offer.
22
Identify & iterate down-funnel levers
How can we maintain the 25%
reactivation CVR lift while . . .
• reducing COGs incurred
• maintaining retention rate
so that we can ultimately reach a cost-
effectively increase acquisitions?
Delta
Reactivations* +25% CVR
Cost of Goods (COGs)
+30%
(increased usage)
Retained Members -5% CVR
Customer Acquisition Cost
Marginal CAC $625 > ⅓ LTV
23
Marketing
• Find optimal channels of acquisition
• Personalize experiences across touchpoints
• Optimize web to help educate, inspire & convert
Finance
Apply CRO principles company-wide
24
Marketing
• Find optimal channels of acquisition
• Personalize experiences across touchpoints
• Optimize web to help educate, inspire & convert
Customer Experience
• Find methods of reducing customer confusion
• Determine optimal methods of communication with customers
Apply CRO principles company-wide
25
Marketing
• Find optimal channels of acquisition
• Personalize experiences across touchpoints
• Optimize web to help educate, inspire & convert
Customer Experience
• Find methods of reducing customer confusion
• Determine optimal methods of communication with customers
• Merchandise effectively
• Develop optimal subscription plans
• Understand acquisition curve
Finance
Apply CRO principles company-wide
26
Marketing
• Find optimal channels of acquisition
• Personalize experiences across touchpoints
• Optimize web to help educate, inspire & convert
Customer Experience
• Find methods of reducing customer confusion
• Determine optimal methods of communication with customers
• Merchandise effectively
• Develop optimal subscription plans
• Understand acquisition curve
Product
• De-risk negative impact of new product feature
• Determine acquisition gains
Finance
Apply CRO principles company-wide
Q&A
Thank you!
 Head to the Expo Hall for morning refreshments
 Up next…Ravi Parikh, co-founder of Heap at 11:15am!

Driving Customer Acquisition and Growth with Experimentation

  • 1.
    Congratulations to our OutperformAward Winners! Most Dramatic Business Impact Most Customer- Obsessed Company Culture Most Transformative Innovation Most Inspiring Social Impact
  • 2.
    2  Welcome toDay 2 of Opticon!  Sessions will be recorded and available after Opticon  Share your feedback and rate sessions on the mobile app
  • 3.
    Nina Bayatti Conversion MarketingManager, ClassPass Driving Customer Acquisition & Growth with Experimentation
  • 4.
  • 5.
    5 Agenda  ClassPass’ iterative,minimum viable product (MVP) approach to testing  Demystifying conversion rate optimization (CRO)  Applying CRO principles company-wide  Q&A
  • 6.
    6 ClassPass is the largeststudio fitness aggregator
  • 7.
    7 We’re not justtesting new fitness classes. In 2018, we ran over 150 optimization tests company-wide.
  • 8.
    8 How did werun 150 optimization tests company-wide?
  • 9.
    9 Build minimum viable product(MVP) • How can we test this idea in a low-lift, high- impact way? • De-risk resource investment • Maintain high test velocity Boost test velocity & de-risk resources with an iterative, MVP approach
  • 10.
    10 Build minimum viable product(MVP) • How can we test this idea in a low-lift, high- impact way? • De-risk resource investment • Maintain high test velocity Launch & monitor • Ask CX team to flag test-related outreach • Monitor web behavior & analytics Boost test velocity & de-risk resources with an iterative, MVP approach
  • 11.
    11 Build minimum viable product(MVP) • How can we test this idea in a low-lift, high- impact way? • De-risk resource investment • Maintain high test velocity Launch & monitor • Ask CX team to flag test-related outreach • Monitor web behavior & analytics Iterate & re-launch • Anticipate 2-3 iterations ahead • Continue to utilize MVP-first approach • Utilize initial test data, customer feedback & web analytics Boost test velocity & de-risk resources with an iterative, MVP approach
  • 12.
    12 Build minimum viable product(MVP) • How can we test this idea in a low-lift, high- impact way? • De-risk resource investment • Maintain high test velocity Launch & monitor • Ask CX team to flag test-related outreach • Monitor web behavior & analytics Iterate & re-launch • Anticipate 2-3 iterations ahead • Continue to utilize MVP-first approach • Utilize initial test data, customer feedback & web analytics Productize • Share learnings • Consider impact when allocating resourcing for productization build Boost test velocity & de-risk resources with an iterative, MVP approach
  • 13.
    13 Demystifying CRO: Beyondthe CTA color test
  • 14.
    14 Understanding down- funnel impactof your CRO initiatives
  • 15.
    15 • Bounce rate •Click through rate • Pageviews • Lead captures • Purchase conversion rate Your primary KPI is not telling the full story
  • 16.
    16 Reach a confident conclusionwith down- funnel data • Quality of leads • Duration of sales cycle • Customer acquisition cost • Operational cost • Customer retention rate • Bounce rate • Click through rate • Pageviews • Lead captures • Purchase conversion rate Your primary KPI is not telling the full story
  • 17.
    17 Measuring the down- funnelimpact of our re-engagement optimization
  • 18.
    18 We tested extendingour winback offer • Winback offer serves as a re-trial period for inactive members • Inactive members can experience new product features & newly added classes • Tested extending winback offer period from 1 to 2 weeks
  • 19.
    19 We tested extendingour winback offer Hypothesis: If we extend our winback offer from 1 to 2 weeks, then our reactivation rates will increase because members will have more time to experience new features and classes. And the lift in reactivations will offset the costs of the increased duration of the free offer.
  • 20.
    20 Extending our winbackoffer lifted our reactivation rate by 25% Reactivations +25% ReactivationCVR B: Extended OfferA: Control
  • 21.
    21 Despite the liftin reactivations, our marginal acquisition was not cost-effective Control Extended Offer Delta Reactivations* 100 125 +25% CVR Cost of Goods (COGs) $15,000 $24,375 +30% (increased usage) Retained Members 80 95 -5% CVR Customer Acquisition Cost $187.50 Marginal CAC $625 > ⅓ Lifetime Value (LTV) *Reactivation counts are illustrative to better highlight the impact of the purchase CVR, retention CVR and increased cost of the offer.
  • 22.
    22 Identify & iteratedown-funnel levers How can we maintain the 25% reactivation CVR lift while . . . • reducing COGs incurred • maintaining retention rate so that we can ultimately reach a cost- effectively increase acquisitions? Delta Reactivations* +25% CVR Cost of Goods (COGs) +30% (increased usage) Retained Members -5% CVR Customer Acquisition Cost Marginal CAC $625 > ⅓ LTV
  • 23.
    23 Marketing • Find optimalchannels of acquisition • Personalize experiences across touchpoints • Optimize web to help educate, inspire & convert Finance Apply CRO principles company-wide
  • 24.
    24 Marketing • Find optimalchannels of acquisition • Personalize experiences across touchpoints • Optimize web to help educate, inspire & convert Customer Experience • Find methods of reducing customer confusion • Determine optimal methods of communication with customers Apply CRO principles company-wide
  • 25.
    25 Marketing • Find optimalchannels of acquisition • Personalize experiences across touchpoints • Optimize web to help educate, inspire & convert Customer Experience • Find methods of reducing customer confusion • Determine optimal methods of communication with customers • Merchandise effectively • Develop optimal subscription plans • Understand acquisition curve Finance Apply CRO principles company-wide
  • 26.
    26 Marketing • Find optimalchannels of acquisition • Personalize experiences across touchpoints • Optimize web to help educate, inspire & convert Customer Experience • Find methods of reducing customer confusion • Determine optimal methods of communication with customers • Merchandise effectively • Develop optimal subscription plans • Understand acquisition curve Product • De-risk negative impact of new product feature • Determine acquisition gains Finance Apply CRO principles company-wide
  • 27.
  • 28.
  • 29.
     Head tothe Expo Hall for morning refreshments  Up next…Ravi Parikh, co-founder of Heap at 11:15am!