1.
Lean
Experiments
with Lean
Expectations
Lena Sesardic
Product Camp Cascadia 2021
March 6, 2021 10AM PST

2.
What you'll get out of this talk:
How to build less of the wrong products by running lean
experiments.
FOUR types of lean experiments through FIVE real-life
examples.
The importance of lean expectations in staying objective.
When to walk away from an idea.

3.
Lena Sesardic
7 yrs startups/startup-like teams (3.5 yrs PM).
Entrepreneur, Creator & Writer.
Mentor & advise venture teams.
Author of The Making of Product Managers.
I write on lenasesardic.com & for Product
Coalition & The Startup on Medium.

4.
Think in experiments, not products.
What is the quickest way to test an idea?
Riskiest assumptions.
Hypothesis (specific, measurable, disprovable, time-bound).
Expected outcome—stay objective & accountable.

5.
Divorce yourself from your idea
9/10 startups fail.
Even more product features fail.
Most experiments fail.
Failed experiments = learning.

6.
Lean experiments
Few days/weeks to run.
Test a specific hypothesis.
Quantitative/qualitative data as feedback.
Prioritize learning.

7.
Wizard of Oz method
Lean Experiment #1

8.
Humans simulate a functioning product without the
use of technology & without the user's knowledge.
Wizard of Oz method

9.
Real-life Example: Veni
Resource: Veni.ca

10.
Real-life Example: Veni
1. Simple landing page for customers submit orders through.
Resource: Veni.ca

11.
2. Place orders with restaurants on the day manually.
Resource: Veni.ca
Real-life Example: Veni

12.
3. Deliver all meals manually.
Resource: Veni.ca
Real-life Example: Veni

13.
Spent $25.
Batching meal orders is valuable to restaurants.
Collected feedback on the customer experience.
Going forward: Wizard of Oz + landing page analytics
Results:
Resource: Veni.ca
Real-life Example: Veni

14.
Pros & Cons: Wizard of Oz method
No need to build
technology.
Quick to ship.
High-touch service =
learning opportunity.
Easy to pivot.
Requires manual work.
Have to build product
eventually.
May not be polished.
May not be possible for
all product concepts.

15.
Fake Door Demand Test
Lean Experiment #2

16.
Fake Door Demand Test
Fake the existence of a product & measure the
actions users take.

17.
Real-life Example: Kettle & Fire
Resource: How I validated my idea for Kettle & Fire by Justin Mares.

18.
Real-life Example: Kettle & Fire
1. Keyword search & trend analysis.
Resource: How I validated my idea for Kettle & Fire by Justin Mares.

19.
2. Simple landing page with "Order Bone Broth Now!" button &
$50 worth of Bing ads.
Resource: How I validated my idea for Kettle & Fire by Justin Mares.
Real-life Example: Kettle & Fire

20.
3. Redirected to "PayPall $29.99 to xxxxxx@gmail.com."
Resource: How I validated my idea for Kettle & Fire by Justin Mares.
Real-life Example: Kettle & Fire

21.
7814 2384 30.51%
Visitors Conversions Conversion rate
Resource: How I validated my idea for Kettle & Fire by Justin Mares.
Results:
Real-life Example: Kettle & Fire

22.
4. Out of stock follow-up email.
Resource: How I validated my idea for Kettle & Fire by Justin Mares.
Real-life Example: Kettle & Fire

23.
Real-life Example: Customer Camp
Resource: Customer Camp.

24.
1. "GET CERTIFIED" fake door.
Resource: Customer Camp.
Real-life Example: Customer Camp

25.
2. Popup to join waitlist.
Resource: Customer Camp.
Real-life Example: Customer Camp

26.
3. Customer discovery interview.
Resource: Customer Camp.
Real-life Example: Customer Camp

27.
Resource: Customer Camp.
Quantitative data: waitlist sign-ups.
Customer discovery interviews w/ interested parties.
Qualitative data >> deep insights.
Connections with potential future customers.
Results:
Real-life Example: Customer Camp

28.
Pros & Cons: Fake Door Demand Test
Quick to ship landing
page/button.
No need to create or
source product.
Quantitative/qualitative
data for analysis.
Scalable (landing
pages/button).
Paid ads expensive &
require expertise.
False negatives due to
poor UX/ad copy or choice
of ad platform.
Potentially frustrated
customers (use sparingly).

29.
Cold Outreach
Lean Experiment #3

30.
Cold Outreach
Contact people who haven't previously
expressed an interest in the products or
services you're offering.

31.
Real-life Example: Scheduling App X

32.
1. Clickable prototype of what app would look like.
Real-life Example: Scheduling App X

33.
100+ leads for each of 5 niche industries.
2.
Real-life Example: Scheduling App X

34.
5 cold email campaign to the 5 niche industries with clickable
prototype included.
3.
Real-life Example: Scheduling App X

35.
Identified 2 niche industries with highest demand (carpet
cleaning & painting).
50 early customers with signed Letters of Intent (LOIs).
Results:
Real-life Example: Scheduling App X

36.
Pros & Cons: Cold Outreach
Quick to ship.
Quantitative/qualitative
data for analysis.
Scalable (cold emails).
Some lead generation can be
time-consuming & expensive.
False negatives due to poor
email copy or cold calling.
Easy to pivot.
False negatives due to poorly
formed value prop.

37.
Inchworm Medley
Lean Experiment #4

38.
Inchworm Medley
Gauge interest on for a product or service on
social media
+
set up pre-orders to validate willingness to pay.

39.
Real-life Example: Lean SEO Course
Resource: Starter Story

40.
1. Gauged interest with a tweet & blog post.
Real-life Example: Lean SEO Course
Resource: Starter Story

41.
2. Positive qualitative feedback & insight into customer pains.
Real-life Example: Lean SEO Course
Resource: Starter Story

42.
3. Gauge product format with a tweet.
Real-life Example: Lean SEO Course
Resource: Starter Story

43.
4. Pre-orders to validate willingness to pay.
Real-life Example: Lean SEO Course
Resource: Starter Story

44.
Real-life Example: Lean SEO Course
$2,054 in presales in 2 weeks.
$30k in sales within 2 months.
Results:
Resource: Starter Story

45.
Pros & Cons: Inchworm Medley
Multiple checkpoints for
validation.
Builds community.
No need to create or
source product right
away.
Quantitative/qualitative
data for analysis.
More effective with an existing
audience & credibility.
May be difficult to back out of
(social reputation).
Risk of someone stealing
your idea.

46.
Recap of Lean Experiments
#1 Wizard of Oz method #2 Fake door demand test
#3 Cold outreach #4 Inchworm medley

47.
Walking Away From an Idea
Pre-desired outcomes are not met.
We tend to see what we want to see.
Exceptions are rare.

48.
The Value of Lean Experiments
Speed.
Learning.
Hidden ROI.

49.
Thank you!
in/lenasesardic
@LenaSesardic
lenasesardic.com
hello@lenasesardic.com