Welcome to
Lean Startup 301
Aubrey Smith
November 16, 2015
Sponsored by:
Agenda
1. Who is in the room?
2. How will this be valuable?
3. What will we cover?
4. How to participate?
5. What should I do next?
Who is in the room?
• Founders
• 1st
Time Entrepreneurs
• Serial Entrepreneurs
• Intrapraneurs
• Designers
• Developers
• Newbies
• Book Readers
How will this be valuable?
This Session Is
• Practical
• Clear
• Direct
• Simple
• Introductory
• Wisdom sharing
• Hands-on
This Session Is Not
• Book review
• Brainstorming
session
• Full of jargon
How will this be valuable?
Addressing the Main Questions
• What is it?
• Why does it work?
• How does it work?
• Why does it matter?
• How can I get started?
• How can I apply this to my company?
What will we cover?
What we will cover today
• Deep dive into MVPs, experiments
• Discussion of Advanced Topics
• Live Case Studies
• Exercise: Design an Experiment
Innovation
Accounting
Governance
Application in
the
Enterprise
Regulated
Markets
How to participate?
Experiment
Exercise
Live Case Studies
• Listen
• Share
• Please, ask questions!
• Need Volunteers
• Group Participation
• Sharing
1
2
What is the Lean Startup?
Key Areas to Discuss
History
Basic Terminology / Definitions
Application / Applicability
Advanced Topics
What is the Lean Startup?
History
• Robert Deming
• Toyota Lean Manufacturing
• Agile Software Development
• Lean Startup
• Steve Blank
• Eric Ries
• Alexander Osterwalder
• Lean Startup Community
What is the Lean Startup?
A method to systematically address
uncertainty through rapid iteration and market
learning.
Core Principles
Entrepreneurs are everywhere
Entrepreneurship is management
Validated Learning
Build-Measure-Learn Loop
Innovation Accounting
What is the Lean Startup?
Common Purpose
We share a vision for the kind of company we
want to create:
A company that continuously creates new
sources of growth.
The Lean Startup is…
Everywhere.
Who’s accountable?
• Jack Stack: The Great Game of Business
Build a culture of ownership
• Key Parallels
Lean Startup capability and accountability
should be the job of everyone at the company
Governance will vary
Customer learning trumps internal feedback
loops and decisions about products/features
Case 1: Lean Startup at GE?
What is FastWorks?
- Organizational change
- Teams to Growth Boards
- 500 projects, 500 coaches
What are GE Beliefs?
- Customers Determine our Success
- Stay Lean to Go Fast
- Learn and Adapt to Win
- Empower and Inspire Each other
- Deliver Results in an Uncertain
World
Key Areas to Discuss
History
Basic Terminology / Definitions
Application / Applicability
Advanced Topics
What is the Lean Startup?
What is The Lean Startup?
Terminology / Definitions
• Entrepreneurs
• Startups
• Uncertainty
• Vision
• Assumptions
• Hypotheses
• Minimum Viable Product
• Validated Learning
• Pivots
Your Vision
What business do you want to create?
And, why?
What will you not pursue?
And, why?
1
2
Now, get the facts...
• Biggest mistake you can make is to not test
the underlying assumptions of your
business plan
• Leap of Faith Assumptions:
If you prove your Leap of Faith
assumptions false, your business plan falls
apart
Isolating critical assumptions
What is the riskiest element of your
business plan?
Impact
Time horizon
OR
Will kill
Won’t kill
Customer Problem
Solution
Business Model
Scale / Growth
Types of Assumptions
• “If then” statements that help design tests
for an assumption
Paypal - If people can easily transact across multiple
devices, they will be more frequent users of the service
• Clarifies your current understanding of what
uncertainty you seek to resolve
Paypal - Specific action? Timing? Value/amount
Draft your Hypothesis
Minimum Viable Product
• Experiment that helps you validate (or
invalidate) hypotheses about the value
or growth potential for a new product
• An MVP helps you answer a specific
question about one or a few of your
critical assumptions
Case 2: Testing @ AirBnb
Test:
• Photography Concierge
MVP
• 20 Photographers in
the field
Critical Hypothesis:
“Professional photographed listings
get 2-3 times more business (and host
don’t turn down free professional
photography.”
(SXSX, Airbnb)
Case 2: Testing @ AirBnb
MVP: Expert Tip
The MVP begins the process of
validated learning…
• This is an experiment, not a product
• No such thing as “an MVP”
• The smallest possible product we can
imagine to achieve maximum learning
about our hypothesis
Build, Measure, Learn Loop
Minimum Viable Product
Translate your critical assumptions into an
experiment:
Validated Learning
1. Isolate critical
assumptions for testing
2. Draft your hypothesis to
be tested
3. Build an experiment
4. Measure the results
5. Collect the data and
learning in a systematic
1. What artifact will
you use?
2. How will you
use/test it?
3. Who will you test
with (segments)
Minimum Viable Product
Translate your critical assumptions into an
experiment:
Validated Learning
1. Isolate critical
assumptions for testing
2. Draft your hypothesis to
be tested
3. Build an experiment
4. Measure the results
5. Collect the data and
learning in a systematic
1. What artifact will
you use?
2. How will you
use/test it?
3. Who will you test
with (segments)
Fears
• Change in direction without a change in
vision
• OR
• Persevere: A team’s decision to test the
next most important hypothesis
Pivot
Innovation Accounting
• The problem with Vanity Metrics
• Quantitative vs. Qualitative Data
• Leading vs. Lagging Indicators
Key Areas to Discuss
History
Basic Terminology / Definitions
Application / Applicability
Advanced Topics
What is the Lean Startup?
Minimum Viable Product
Translate your critical assumptions into an
experiment:
Validated Learning
1. Isolate critical
assumptions for testing
2. Draft your hypothesis to
be tested
3. Build an experiment
4. Measure the results
5. Collect the data and
learning in a systematic
1. Who will you test
with (segments)
2. What artifact will
you use?
3. How will you
use/test it?
Culture of Testing
1. Experiment design is important
2. But, recording and evaluating the learning is
more important
1. Experiment design is important
2. But, recording and evaluating the learning is
more important
• Establish an organization that learns together
• Only step up to the sophistication of the experiment
and innovation accounting when you are ready to do
so as a team
• Team learning is the most important outcome
Culture of Testing
Discipline of Testing
1. Focus the learning
2. Establish the baseline
3. Track MVPs on a dashboard
4. Employ the right cadence of testing
1. Focus the learning
2. Establish the baseline
3. Track MVPs on a dashboard
4. Employ the right cadence of testing
• Only as sophisticated as your organization is ready
to handle
• Remember: A well-designed experiment will inform
what your next experiment will be
• Parallel path testing should only be employed if you
have available resources
Discipline of Testing
Minimum Viable Product
Translate your critical assumptions into an
experiment:
Validated Learning
1. Isolate critical
assumptions for testing
2. Draft your hypothesis to
be tested
3. Build an experiment
4. Measure the results
5. Collect the data and
learning in a systematic
1. Who will you test
with (segments)
2. What artifact will
you use?
3. How will you
use/test it?
• Customers do not always do what they
say they will
• Avoid “Voice of Customer”
• Measure conversion on the path to
purchase (ex. Funnel metrics)
• “Purchase intent” may require a ‘new’ rung
in the funnel
Exchange of Value
Identify “Early Adopters”
• What customers have this problem/need?
• Who uses a ‘workaround’ today?
• Who values this solution for a unique reason?
• How would we classify segments?
• Innovations may cut across traditional segmentation
• Dig for segmentation variables (time, cost)
A. Build – What segments?
A. Build – What artifact?
Begin with a low fidelity artifact
• Paper brochure
• PowerPoint
• Landing page
• Website
• Clickable Mobile App
• 3d CAD Rendering
• Video
A. Build – What test?
Pick a test
• Interviews (Face-to-face, phone, video)
• Digital Campaign
• Google Adwords
• U/X in-person Test
• Email
• Live Demonstration
Test Problem/Solution Fit
• Bring bare bones “artifact”
• Set Context
• Focus on hypotheses to be tested
• Begin with direct statements (“We believe”)
• Allow customers to hint/point you to what pain /
need they do have
• Seek an “exchange of value”
c
The Interview
A. Build
Advanced Concepts
• Concierge MVP (“Man behind the curtain”)
• A/B Split Test
A. Build
Advanced Concepts
• Concierge MVP (“Man behind the curtain”)
• A/B Split Test
• Helps you obtain statistically significant
information about your potential offering
• Allows you to see what customers do with your
product / service / site
• Gives you information to make decisions – such
as, which features to cut / build
Experiment Example
[software]
❶ MVP #1
Display non-working
prototype at a
showroom
❷ MVP #2
Build full-scale driving
prototype to verify
technology
Avg. down-
payment
Pre-order rate
Prototype
• Driving prototype
(MVP2) improved
which indicates
strategy effectiveness
• MVP #2 validated
technical feasibility
assumptions
Learning
↗
↗
• Purchase intent was
validated by innovation
metrics: Pre-order rate
& avg. down payment
Case 3: Lit Motors Test
Exercise – Part I
Please see White Board
Live Case
Ursula Shekufendeh
AppFolio
Live Case: AppFolio
About AppFolio
• Web-based real estate property
management software
• Property managers can market &
manage property portfolio
Testable Hypothesis:
“If we build a tenant screening
service that landlords can
subscribe to, then we can sell a
service to 80% of RentApp users
(and, quickly reach profitability).”
Exercise – Part II
Please see White Board
B. Measure Results
• One metric that matters, OR
• Simple Dashboard / Excel Spreadsheets, AND
• “Exchange of Value”
• Customers say and do different things
• Avoid “Voice of Customer”
• Measure conversion on the path to purchase (ex.
Funnel metrics)
• Create “proxy” in funnel, if needed
Sample Dashboard
❷
✓
✓
✗
→
❸
• Pivot vs. Persevere
• Continuous Deployment
Tips:
• Pivoting is a team sport
• Ground decisions entirely in the learning
• Learning is cumulative
• Good experiments should teach you what
experiment to do next
C. Commit to Learning
Key Areas to Discuss
History
Basic Terminology / Definitions
Application / Applicability
Advanced Topics
What is the Lean Startup?
Innovation Accounting
• Actionable
• Must demonstrate cause and effect
• Removes bias to report / overvalue “numbers that go
up”
• Accessible
• Simple, easy for the team to understand
• Should not require retraining team to read reports
• Auditable
• Credible data reduces the ability to place blame
• The growth operating system requires new
management principles
• No “one size fits all” Lean Startup Process
• Start small to get big
• Small, honest success cases will bring leaders into the
fold
• Sophistication should be aligned with team capability
and resources
Lean Startup in Enterprise
Live Case
Chris Brown
Vodafone Global Enterprise
Questions?
Aubrey Smith
Sparked Advisory
(202) 907-3993
Heather McGough
Co-Founder, Lean Startup Company
(415) 830-2479
Heather@LeanStartup.co
Appendix
• Vision -> Strategy -> Team
• Strategy is leadership-led:
Governance
PortfolioTeam Company
Startup approach opens market opportunities
Important Themes:
• Dialogue is critical
• Layering regulation improves impact
Regulated Markets
Prioritize Assumptions
• Focuses you, your team
• Draws a line in the sand
• Introduces accountability
• Saves time
• Introduces culture of “learning”
Eric’s Point of View

Lean Startup 301

  • 1.
    Welcome to Lean Startup301 Aubrey Smith November 16, 2015 Sponsored by:
  • 2.
    Agenda 1. Who isin the room? 2. How will this be valuable? 3. What will we cover? 4. How to participate? 5. What should I do next?
  • 3.
    Who is inthe room? • Founders • 1st Time Entrepreneurs • Serial Entrepreneurs • Intrapraneurs • Designers • Developers • Newbies • Book Readers
  • 4.
    How will thisbe valuable? This Session Is • Practical • Clear • Direct • Simple • Introductory • Wisdom sharing • Hands-on This Session Is Not • Book review • Brainstorming session • Full of jargon
  • 5.
    How will thisbe valuable? Addressing the Main Questions • What is it? • Why does it work? • How does it work? • Why does it matter? • How can I get started? • How can I apply this to my company?
  • 6.
    What will wecover? What we will cover today • Deep dive into MVPs, experiments • Discussion of Advanced Topics • Live Case Studies • Exercise: Design an Experiment Innovation Accounting Governance Application in the Enterprise Regulated Markets
  • 7.
    How to participate? Experiment Exercise LiveCase Studies • Listen • Share • Please, ask questions! • Need Volunteers • Group Participation • Sharing 1 2
  • 8.
    What is theLean Startup? Key Areas to Discuss History Basic Terminology / Definitions Application / Applicability Advanced Topics
  • 9.
    What is theLean Startup? History • Robert Deming • Toyota Lean Manufacturing • Agile Software Development • Lean Startup • Steve Blank • Eric Ries • Alexander Osterwalder • Lean Startup Community
  • 10.
    What is theLean Startup? A method to systematically address uncertainty through rapid iteration and market learning.
  • 11.
    Core Principles Entrepreneurs areeverywhere Entrepreneurship is management Validated Learning Build-Measure-Learn Loop Innovation Accounting What is the Lean Startup?
  • 12.
    Common Purpose We sharea vision for the kind of company we want to create: A company that continuously creates new sources of growth.
  • 13.
    The Lean Startupis… Everywhere.
  • 14.
    Who’s accountable? • JackStack: The Great Game of Business Build a culture of ownership • Key Parallels Lean Startup capability and accountability should be the job of everyone at the company Governance will vary Customer learning trumps internal feedback loops and decisions about products/features
  • 15.
    Case 1: LeanStartup at GE? What is FastWorks? - Organizational change - Teams to Growth Boards - 500 projects, 500 coaches What are GE Beliefs? - Customers Determine our Success - Stay Lean to Go Fast - Learn and Adapt to Win - Empower and Inspire Each other - Deliver Results in an Uncertain World
  • 16.
    Key Areas toDiscuss History Basic Terminology / Definitions Application / Applicability Advanced Topics What is the Lean Startup?
  • 17.
    What is TheLean Startup? Terminology / Definitions • Entrepreneurs • Startups • Uncertainty • Vision • Assumptions • Hypotheses • Minimum Viable Product • Validated Learning • Pivots
  • 18.
    Your Vision What businessdo you want to create? And, why? What will you not pursue? And, why? 1 2
  • 19.
    Now, get thefacts... • Biggest mistake you can make is to not test the underlying assumptions of your business plan • Leap of Faith Assumptions: If you prove your Leap of Faith assumptions false, your business plan falls apart
  • 20.
    Isolating critical assumptions Whatis the riskiest element of your business plan? Impact Time horizon OR Will kill Won’t kill
  • 21.
  • 22.
    • “If then”statements that help design tests for an assumption Paypal - If people can easily transact across multiple devices, they will be more frequent users of the service • Clarifies your current understanding of what uncertainty you seek to resolve Paypal - Specific action? Timing? Value/amount Draft your Hypothesis
  • 23.
    Minimum Viable Product •Experiment that helps you validate (or invalidate) hypotheses about the value or growth potential for a new product • An MVP helps you answer a specific question about one or a few of your critical assumptions
  • 24.
    Case 2: Testing@ AirBnb Test: • Photography Concierge MVP • 20 Photographers in the field Critical Hypothesis: “Professional photographed listings get 2-3 times more business (and host don’t turn down free professional photography.” (SXSX, Airbnb)
  • 25.
  • 26.
    MVP: Expert Tip TheMVP begins the process of validated learning… • This is an experiment, not a product • No such thing as “an MVP” • The smallest possible product we can imagine to achieve maximum learning about our hypothesis
  • 27.
  • 28.
    Minimum Viable Product Translateyour critical assumptions into an experiment: Validated Learning 1. Isolate critical assumptions for testing 2. Draft your hypothesis to be tested 3. Build an experiment 4. Measure the results 5. Collect the data and learning in a systematic 1. What artifact will you use? 2. How will you use/test it? 3. Who will you test with (segments)
  • 29.
    Minimum Viable Product Translateyour critical assumptions into an experiment: Validated Learning 1. Isolate critical assumptions for testing 2. Draft your hypothesis to be tested 3. Build an experiment 4. Measure the results 5. Collect the data and learning in a systematic 1. What artifact will you use? 2. How will you use/test it? 3. Who will you test with (segments)
  • 30.
    Fears • Change indirection without a change in vision • OR • Persevere: A team’s decision to test the next most important hypothesis Pivot
  • 31.
    Innovation Accounting • Theproblem with Vanity Metrics • Quantitative vs. Qualitative Data • Leading vs. Lagging Indicators
  • 32.
    Key Areas toDiscuss History Basic Terminology / Definitions Application / Applicability Advanced Topics What is the Lean Startup?
  • 33.
    Minimum Viable Product Translateyour critical assumptions into an experiment: Validated Learning 1. Isolate critical assumptions for testing 2. Draft your hypothesis to be tested 3. Build an experiment 4. Measure the results 5. Collect the data and learning in a systematic 1. Who will you test with (segments) 2. What artifact will you use? 3. How will you use/test it?
  • 34.
    Culture of Testing 1.Experiment design is important 2. But, recording and evaluating the learning is more important
  • 35.
    1. Experiment designis important 2. But, recording and evaluating the learning is more important • Establish an organization that learns together • Only step up to the sophistication of the experiment and innovation accounting when you are ready to do so as a team • Team learning is the most important outcome Culture of Testing
  • 36.
    Discipline of Testing 1.Focus the learning 2. Establish the baseline 3. Track MVPs on a dashboard 4. Employ the right cadence of testing
  • 37.
    1. Focus thelearning 2. Establish the baseline 3. Track MVPs on a dashboard 4. Employ the right cadence of testing • Only as sophisticated as your organization is ready to handle • Remember: A well-designed experiment will inform what your next experiment will be • Parallel path testing should only be employed if you have available resources Discipline of Testing
  • 38.
    Minimum Viable Product Translateyour critical assumptions into an experiment: Validated Learning 1. Isolate critical assumptions for testing 2. Draft your hypothesis to be tested 3. Build an experiment 4. Measure the results 5. Collect the data and learning in a systematic 1. Who will you test with (segments) 2. What artifact will you use? 3. How will you use/test it?
  • 39.
    • Customers donot always do what they say they will • Avoid “Voice of Customer” • Measure conversion on the path to purchase (ex. Funnel metrics) • “Purchase intent” may require a ‘new’ rung in the funnel Exchange of Value
  • 40.
    Identify “Early Adopters” •What customers have this problem/need? • Who uses a ‘workaround’ today? • Who values this solution for a unique reason? • How would we classify segments? • Innovations may cut across traditional segmentation • Dig for segmentation variables (time, cost) A. Build – What segments?
  • 41.
    A. Build –What artifact? Begin with a low fidelity artifact • Paper brochure • PowerPoint • Landing page • Website • Clickable Mobile App • 3d CAD Rendering • Video
  • 42.
    A. Build –What test? Pick a test • Interviews (Face-to-face, phone, video) • Digital Campaign • Google Adwords • U/X in-person Test • Email • Live Demonstration
  • 43.
    Test Problem/Solution Fit •Bring bare bones “artifact” • Set Context • Focus on hypotheses to be tested • Begin with direct statements (“We believe”) • Allow customers to hint/point you to what pain / need they do have • Seek an “exchange of value” c The Interview
  • 44.
    A. Build Advanced Concepts •Concierge MVP (“Man behind the curtain”) • A/B Split Test
  • 45.
    A. Build Advanced Concepts •Concierge MVP (“Man behind the curtain”) • A/B Split Test • Helps you obtain statistically significant information about your potential offering • Allows you to see what customers do with your product / service / site • Gives you information to make decisions – such as, which features to cut / build
  • 46.
  • 47.
    ❶ MVP #1 Displaynon-working prototype at a showroom ❷ MVP #2 Build full-scale driving prototype to verify technology Avg. down- payment Pre-order rate Prototype • Driving prototype (MVP2) improved which indicates strategy effectiveness • MVP #2 validated technical feasibility assumptions Learning ↗ ↗ • Purchase intent was validated by innovation metrics: Pre-order rate & avg. down payment Case 3: Lit Motors Test
  • 48.
    Exercise – PartI Please see White Board
  • 49.
  • 50.
    Live Case: AppFolio AboutAppFolio • Web-based real estate property management software • Property managers can market & manage property portfolio Testable Hypothesis: “If we build a tenant screening service that landlords can subscribe to, then we can sell a service to 80% of RentApp users (and, quickly reach profitability).”
  • 51.
    Exercise – PartII Please see White Board
  • 52.
    B. Measure Results •One metric that matters, OR • Simple Dashboard / Excel Spreadsheets, AND • “Exchange of Value” • Customers say and do different things • Avoid “Voice of Customer” • Measure conversion on the path to purchase (ex. Funnel metrics) • Create “proxy” in funnel, if needed
  • 53.
  • 54.
    • Pivot vs.Persevere • Continuous Deployment Tips: • Pivoting is a team sport • Ground decisions entirely in the learning • Learning is cumulative • Good experiments should teach you what experiment to do next C. Commit to Learning
  • 55.
    Key Areas toDiscuss History Basic Terminology / Definitions Application / Applicability Advanced Topics What is the Lean Startup?
  • 56.
    Innovation Accounting • Actionable •Must demonstrate cause and effect • Removes bias to report / overvalue “numbers that go up” • Accessible • Simple, easy for the team to understand • Should not require retraining team to read reports • Auditable • Credible data reduces the ability to place blame
  • 57.
    • The growthoperating system requires new management principles • No “one size fits all” Lean Startup Process • Start small to get big • Small, honest success cases will bring leaders into the fold • Sophistication should be aligned with team capability and resources Lean Startup in Enterprise
  • 58.
  • 59.
    Questions? Aubrey Smith Sparked Advisory (202)907-3993 Heather McGough Co-Founder, Lean Startup Company (415) 830-2479 Heather@LeanStartup.co
  • 60.
  • 61.
    • Vision ->Strategy -> Team • Strategy is leadership-led: Governance PortfolioTeam Company
  • 62.
    Startup approach opensmarket opportunities Important Themes: • Dialogue is critical • Layering regulation improves impact Regulated Markets
  • 63.
    Prioritize Assumptions • Focusesyou, your team • Draws a line in the sand • Introduces accountability • Saves time • Introduces culture of “learning”
  • 64.