The document provides advice on common startup mistakes and how to avoid them. It recommends (1) using user personas to understand customers rather than assumptions, (2) learning how to set up tests of hypotheses to validate assumptions, and (3) focusing testing on a small initial customer segment before expanding to validate the business model works before premature scaling. The key is to systematically gather customer feedback, control variables when testing, and iterate based on measurable results rather than biases or assumptions.

1. Top Startup Mistakes
(and how to fix them)

2. A bit about me…
• Originally from Northern California, lived in Silicon Valley
for almost 10 years
• 2007-2012: Silicon Valley Bank – financing early stage
startups all over world alongside top VC firms
• 2012-2014: Received MBA, taught by Steve Blank (father of
Lean Startup), raised venture fund for student-run startups,
and taught self to code
• 2012-2014: Worked for Okta (IPO), New Relic (IPO), Stripe
• 2014-2016: Mixpanel (worked with unicorns on analytics
strategy)
• 2016-2017: PM at Capital One (manager is VP Product for
Alexa)

3. Why is Customer Development
Important?

4. Actual Data
KEY LESSON:
Overwhelming
majority of failure is
due to product failure,
not engineering
failure
Source: https://www.cbinsights.com/blog/startup-
failure-reasons-top/

5. Question
What is (1.01)365?

6. Major Mistakes Companies Make
• Premature scaling
• Not taking the time to understand users
• Biased testing
• No clear plan

7. Premature Scaling
• Do not assume “I know more than the
customer”
– You might, but if you are too smart for them, they
won’t pay you
– Taking the time to understand customers will
inform marketing, sales, design, and engineering
architecture

8. A Silicon Investor once said to me…
• 2 types of questions I get asked by
entrepreneurs:
– “How do I find more paying customers?”
– “My phone won’t stop ringing with people
wanting to buy my product. I can’t get enough
people to close sales. Where do I find them?”

9. Assuming Everyone is Like You
• We replace actual understanding with
assumptions
• 99.9% of the time, we miss critical facts or
don’t get to the level of understanding needed
• You have to do the hard work to deeply
understand your users to build a product they
love (and there is no substitute for hard work)

10. The Astonishingly Funny Story of Mr.
McArthur Wheeler

11. Biased Testing
• Startups are a series of hypotheses
• We don’t want to invest a lot of time building
on unproven hypotheses
• We know as a startup we need to move fast
• To test our hypotheses, we often focus on
data that confirms what we believe and
dismiss data that contradicts what we think

12. No clear plan
• Because we want our company to succeed,
believe in it, and need money, it distorts our
motivations
• We chase easy dollars without digging deeply
into “why?”
• Have to have a disciplined way of assessing,
prioritizing, and reducing risk

13. Let’s talk about solutions

15. Advice for avoiding these mistakes
• Use user personas
• Learn how to set up tests
• Focus, then expand
• Have a clear plan and stick to it
We are going to discuss these, but I want
to put this in a framework first

16. What is Needed?
• A framework for mapping out our
assumptions
• Then, we can assess what our greatest risks
are
• Form hypotheses on these risks
• Systematically test these risks, build user
personas, and validate solutions

17. Lean Canvas

18. Order for Lean Canvas

19. 3 Stages of Startup
• Stage 1: Problem / Solution Fit
– Do I have a problem worth solving?
– “You can’t leave here without me buying that”
– You should be interviewing at least 10 customers per week
• Stage 2: Product / Market Fit
– Have I built something people want?
– “If this product were to go away, I’d be very disappointed”
– You should be interviewing at least 10 customers per week
• Stage 3: Scale
– How do I accelerate growth?
– “How can I grow this business fast enough to keep up with demand?”

20. Process for Each Type of Risk
Product Risk: Getting product right • First make sure you have a problem worth
solving
• Define smallest possible MVP
• Build and validate MVP at small scale
• Verify it at scale
Customer Risk: Building a path to
customer
• Identify who has pain
• Narrow this down to early adopters who
really want product now
• Build scalable inbound channel
Market Risk: Building a viable
business
• Identify competition through existing
alternatives and pick price for solution
• Test pricing by measuring what customers
say
• Then test pricing by what they do (i.e. paying
you)
• Optimize cost structure to make business
model work

21. Which stage are we in?
• What is the problem we are trying to solve?
• What are our assumptions that we need to
validate?

22. Let’s start a business…

23. Bowling Balls Online
• $6 billion industry in US
• Separation between
traditional bowling alleys
(basic services) and
upscale alleys (fancy food,
drinks, dance music)
• Upscale alleys are trying
to appeal to younger
generation

24. User Personas
“Bud”
• 50 years old
• Plays in bowling league with
friends for past 15 years
• At least 3 games per week
• Owns equipment
• Usually only purchases a
Coca-Cola on bowling nights
• Average annual spend: $300
“Ricky”
• 27 years old
• Social activities with friends
including restaurants,
sporting events, gym
• New bowling alley in city
• Go 2x per year
• Buy food, drinks, etc
• Average annual spend: $150
We eventually want these to be 3-4 pages with LOTS of detail

25. What should we do?
• Bud
– Loyal, but not big spender
– Can we get him to spend more with us?
• Ricky
– Infrequent visitor, but spends lot of money
– Can we get him to visit more often?

26. Assumptions audit
• At least 2x per week using Lean Canvas
• What must be true for our business to work?
• What do we not have data to prove?

27. How to Validate
• As we go through Lean Canvas in order, we need to
define what are our assumptions
• Turn these assumptions into falsifiable hypotheses,
then start with the null
• We want to prove our assumptions wrong
• Example:
– For us to sell bowling balls online, people prefer buying
bowling balls at the bowling alley to going online
– Bud prefers his ball which he has had for a long time to
newer technologies; Ricky doesn’t want a ball because he
bowls so infrequently

28. Testing
• Null Hypothesis: For us to sell bowling balls
online, people prefer buying bowling balls at
the alley to going online
• Focus on Bud’s problem:
– Wants to play better, but doesn’t have more time
to devote
– Wants to have a cool new toy among his friends
– Ball will have to last at least 15 years
“Your opinion, while interesting, is
irrelevant”

29. On Interviewing…
• Don’t ask leading questions
– “What are some of your favorite cuisines?”
– NOT “Do you like Indian food?”
• Relevance
– 4 out of 10 (not leading) – statistically relevant
– 100 out of 1000 (leading) – statistically relevant
• Get to the “why?”
– Ask 5 times and don’t assume you know

30. Tests and Experiments
• Maximize for speed and learning -> requires
us to focus on single metric or goal
• Validate qualitatively, verify quantitatively
Understand problem Define solution
Validate qualitativelyVerify quantitatively

31. Build -> Measure -> Iterate

32. Testing with Small Samples
• From Andy Johns: http://firstround.com/review/indispensable-growth-
frameworks-from-my-years-at-facebook-twitter-and-wealthfront

33. Controlling Variables
• Test one variable per time
• Run several tests quickly on different variables
• Always understand the size of your market when evaluating
test results
• Don’t test multiple variables at once!!
• Example (4 different tests with 9 variants; null is 9th
variants):
– Customers: Bud / Ricky
– Channel: Email / Google Ads
– Solution: Bowl Master 1000 / Strike Maker 200

34. Focus, then expand
• In terms of controlling variables, we want to
understand a small segment of users
extremely well
• Validate this group by showing much higher
conversion rates
• Based on testing, we then expand once we
have extraordinary demand from initial
segment

35. Focus, then expand examples
San Francisco riders after events or bad
weather
Swedish music lovers who didn’t want to
pirate music; by invitation-only
Busy, tech-savvy people in SF at the gym
who needed to display their status

36. Online Bowling Ball Business
• Let’s say through our testing, we’ve identified
that we can sell the Bowl Master 1000 through
email to lots of “Buds”
– Test this in single market (Milwaukee)
• There is huge demand for this and we can’t keep
enough inventory in stock
• Now we want to pivot
– Geographic pivot (Chicago – 90 mins away)
– Channel pivot (Google ad sales focused in Milwaukee)
– Customer pivot (go after more “Rickys”)
– Solution pivot (try to sell Strike Maker 200 to “Buds”)

37. Key Lessons for Focus, then Expand
• Use initial segment to work out kinks
– Make sure market is large enough
– Identify marketing messaging that works
– Identify channels that work
– Identify any product issues
– Understand what the retention mechanism is
– Figure out how to create barriers to entry
• Figure out what the challenges will be for scaling
larger
– Start to address these
– Understand what costs are and if scale will fix these

38. Summary
• Because this environment is resource-constrained, you
need to start with a plan and stick to it
• Use Lean Canvas to clarify assumptions
• Create user personas from interview data, not
assumptions
• Structure your tests to fight your own biases
• Start small, and expand outward

39. Answer
37.78