Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How To Scale Your Product Through Experimentation w/ Milena Court, Product Manager at Tails (Ex BlaBlaCar)

122 views

Published on

A detrimental mistake many startups do is neglecting product experimentation. They focus mainly on experimentation in marketing - optimising ads, copy, creatives, landing pages - because it's relatively easy to do. But this kind of negligence is extremely damaging for your success!

But why don't more startups do this then?

EARLY-STAGE - Specific challenges:
-No budget: “proper” testing tools quite expensive
-Little data to play with
-Not sure what to experiment on / what moves the metrics
massively?
-No time; need to focus on the business

GROWTH STAGE - Specific challenges:
-More budget now, but limited by free tools
-More data about what's happening but not really sure why it's
happening
-So many ideas and opportunities, how to prioritise?

SCALING STAGE - Specific challenges:
-All tools in place, but now constrained by tech time: experiments
need to be “bigger” to have impact + are competing with other
projects
-Harder to experiment with the core product, the company is
branching out on other products
-Experimentation becomes scarier: more users, more to lose,
users get used to things being a certain way (“Why did they
change the UI again?”)

Milena Court, Product Manager in the Growth Squad at Tails.com, joined us at The Family to share her incredible expertise and explain how to overcome the above challenges!

Published in: Business
  • Be the first to comment

  • Be the first to like this

How To Scale Your Product Through Experimentation w/ Milena Court, Product Manager at Tails (Ex BlaBlaCar)

  1. 1. How to scale your product through experimentation Milena Court Product Manager (Growth Squad) @milenacourt
  2. 2. When did you last experiment to learn about your users? Marketing Experimentation Product Experimentation Facebook Ads Split Testing, Google Ads testing, Landing page test etc. ��
  3. 3. My journey experimenting across the startup stages Scale-Up Ridesharing platform Growth Stage Tailored pet food Early Stage Office food delivery
  4. 4. Tails.com We believe every dog deserves to live it’s happiest, healthiest life. That’s why we create a unique recipe for every dog, in the taste they love with the nutrition they need.
  5. 5. Tails.com
  6. 6. What is Product Experimentation? 📈 Measuring the impact of products we’re building on the metrics we care about -> Comparing the existing version of your website (or app) with another version with changes (also known as A/B tests) -> A mindset, evaluating ideas and making decisions based on data and customer behaviour
  7. 7. Why is it important? Define feature Build Forget about it Shorter learning cycles
  8. 8. Top 3 excuses not to experiment with your product I don’t have the right tools to experiment I don’t know which experiments to run I am time-constrained and experimenting is risky This will happen at every stage of your startup, not only to early-stage startups! 🤔 👩‍💻 😱
  9. 9. Experimenting with your product at every stage This will happen at every stage of your startup, not only to early-stage startups! Early-stage Growth-stage Scale-up ● Challenges at each step ● How to overcome these challenges ● Which types of experiments can you run at each stage?
  10. 10. 1. Early-stage startups
  11. 11. Challenges for early-stage startups This will happen at every stage of your startup, not only to early-stage startups! No budget Not much data to play with Winning experi- mentation pressure Testing tools can be expensive and you have a very limited budget You have Google Analytics in place but not much traffic yet The clock is ticking before your next fundraise, what if experiments don’t work? 😫 💪 Start using a free tool Google Optimise, Sixpack etc. Find experiments in customer feedback -> Go through your flow with 10 customers -> Email people who did not convert and ask why Make big bets experiments, leave the small experiments for later (e.g. CTA colors) -> Test your riskiest assumptions to move the metrics
  12. 12. Types of experiments you can run This will happen at every stage of your startup, not only to early-stage startups! Focus mainly on experiments around acquisition (get users to your first main action!) 1) Test different way to describe your product and your USP -> How do we explain how our product work? 2) Test where your main action should be in your funnel -> Should we capture email at the beginning (ex: landing page) or at the end of the funnel (ex: after adding items to the basket)? 3) Test your pricing model -> Should we have a minimum spend and offer free delivery or have no minimum spend but make you pay for delivery?
  13. 13. 2. Growth-stage startups
  14. 14. Challenges for growth-stage startups This will happen at every stage of your startup, not only to early-stage startups! Increasing testing speed Finding more insights Prioritisation Now you have a bit more budget but you’re being limited by free tools More data about what is happening but not really sure why it is happening So many ideas and opportunities for experiments 😫 💪 Set up a more powerful testing tool like Optimizely or Monetate, focus on quality of data + quantity of the experiments Do user research on high exit points in flows for qualitative insights and invest in Product Analytics for quantitative insights Have a good product portfolio strategy: balance safe bets, small rewards with some higher-risk, higher-reward bets (use RICE scoring)
  15. 15. Types of experiments you can run This will happen at every stage of your startup, not only to early-stage startups! Focus mainly on experiments around acquisition but also some experiments around other key metrics like retention 1) Test the order of information in journeys -> When is the right moment to talk about subscription? 2) Test nudges to help users go through a flow -> How can we improve security messaging on checkout pages? 3) Test different designs for a key action -> How can we improve design to show your price more clearly?
  16. 16. 3. Scale-ups
  17. 17. Challenges for scale-ups This will happen at every stage of your startup, not only to early-stage startups! Hard to get results vs. tech time invested Experimenting with new product/market Experimentation becomes scarier Hard to do changes in a mature product that moves the metrics massively Not all your customers are the same across products / markets More users, more to lose, being scared of customer response 😫 💪 Running small tests to justify time to develop a bigger test, we’d rather get the bad news and go back to the drawing board, as early as possible Accept different levels of confidence for tests: you build your testing instinct overtime Add capabilities to test with only a subset of users (e.g. 10%) and scan customer messages for helpful feedback on tests
  18. 18. Types of experiments you can run This will happen at every stage of your startup, not only to early-stage startups! Experiments across the customer journey to move your main metrics 1) Test launching new product line -> How can we test if launching buses is valuable? 2) Test tailored prices vs. same price for everybody -> Should we have dynamic prices based on supply and demand? 3) Test personalised content for specific segments -> How can we help users in remote areas to be more successful?
  19. 19. Invest in product experimentation tools at every stage of your growth from free tools to more advanced tools Use experiments to understand your customers better (create a bank of insights overtime widely shared) Don’t aim for 100% win with experiments, keep iterating and balance your bets ✔ ✔ ✔ If you only remember 3 things :)
  20. 20. “The only strategy that is guaranteed to fail is not taking risks.” — Mark Zuckerberg, Facebook

×