The Lean Startup#leanstartup<br />Eric Ries
(@ericries)<br />http://StartupLessonsLearned.com<br />

Entrepreneurship = Awesome<br />It is
the best time in the history of the world to be an entrepreneur<br />Costs are falling in all industries<br />Barriers are being destroyed<br />Disruption and chaos are everywhere<br />

Why build a startup?<br />Only
entrepreneurship combines these three elements<br />Change the world<br />Build an organization of lasting value<br />Make customers’ lives better<br />

Most Startups Fail<br />

Most Startups Fail<br />

Most Startups Fail<br />

Most Startups Fail<br /><ul><li>But it
doesn’t have to be that way.

We can do better.

This talk is about how.</li></li></ul><li>What
is a startup?<br /><ul><li>A startup is a human institution designed to deliver a new product or service under conditions of extreme uncertainty.

Nothing to do with size
of company, sector of the economy, or industry</li></li></ul><li>What is a startup?<br />STARTUP <br />=<br />EXPERIMENT<br />

Entrepreneurship is management<br /><ul><li>Our goal
is to create an institution, not just a product

Traditional management practices fail</li></ul>- general
management” as taught to MBAs<br /><ul><li>Need practices and principles geared to the startup context of extreme uncertainty

Not just for “two guys
in a garage”</li></li></ul><li>The Pivot<br />

I’<br />

The Pivot<br /><ul><li>What do successful
startups have in common?

They started out as digital
cash for PDAs, but evolved into online payments for eBay.

They started building BASIC interpreters,
but evolved into the world's largest operating systems monopoly.

They were shocked to discover
their online games company was actually a photo-sharing site.

Pivot: change directions but stay
grounded in what we’ve learned. </li></li></ul><li>Speed Wins<br />If we can reduce the time between pivots<br />We can increase our odds of success<br />Before we run out of money<br />

A Tale of Two Startups<br
/>

Startup<br />1<br />

Stealth Startup Circa 2001<br />

All about the team<br />

A good plan?<br /><ul><li>Start a
company with a compelling long-term vision.

Raise plenty of capital.

Hire the absolute best and
the brightest.

Hire an experienced management team
with tons of startup experience.

Focus on quality.

Build a world-class technology platform.

Build buzz in the press
and blogosphere.</li></li></ul><li>Achieving Failure<br /><ul><li>Product launch failed </li></ul> - $40MM and five years of pain<br /><ul><li>… but the plan was executed well

Crippled by “shadow beliefs” that
destroyed the effort of all those smart people.</li></li></ul><li>Shadow Belief #1<br /><ul><li>We know what customers want. </li></li></ul><li>Shadow Belief #2<br /><ul><li>We can accurately predict the future. </li></li></ul><li>Shadow Belief #3<br /><ul><li>Advancing the plan is progress.</li></li></ul><li>A good plan?<br /><ul><li>Start a company with a compelling long-term vision.

Raise plenty of capital.

Hire the absolute best and
the brightest.

Hire an experienced management team
with tons of startup experience.

Focus on quality.

Build a world-class technology platform.

Build buzz in the press
and blogosphere.</li></li></ul><li>Startup<br />2<br />

IMVU<br />

IMVU<br />

New plan<br /><ul><li>Shipped in six
months – a horribly buggy beta product

Charged from day one

Shipped multiple times a day
(by 2008, on average 50 times a day)

No PR, no launch

Results 2009: profitable, revenue >
$20MM</li></li></ul><li>Making Progress<br /><ul><li>In a lean transformation, question #1 is – which activities are value-creating and which are waste?

In traditional business, value is
created by delivering products or services to customers

In a startup, the product
and customer are unknowns. We need a new definition of value for startups: validated learning about customers</li></li></ul><li>Traditional Product DevelopmentUnit of Progress: Advance to Next Stage<br />Waterfall<br />Requirements<br />Specifications<br />Design<br />Problem: known<br />Solution:known<br />Implementation<br />Verification<br />Maintenance<br />

Agile Product DevelopmentUnit of Progress:
A line of Working Code<br />“Product Owner” or in-house customer<br />Problem: known<br />Solution:unknown<br />

Lean StartupUnit of Progress: Validated
Learning About Customers ($$$)<br />Customer Development<br />Hypotheses, Experiments,<br />Insights<br />Problem:unknown<br />Data, Feedback,<br />Insights<br />Solution:unknown<br />Agile Development<br />

Minimize TOTAL time through the
loop<br />

There’s much more…<br />Build Faster<br
/>Unit Tests<br />Usability Tests<br />Continuous Integration<br />Incremental Deployment<br />Free & Open-Source<br /> Components<br />Cloud Computing<br />Cluster Immune System<br />Just-in-time Scalability<br />Refactoring<br />Developer Sandbox<br />Minimum Viable Product<br />Learn Faster<br />Split Tests<br />Customer Interviews<br />Customer Development<br />Five Whys Root Cause<br />Analysis<br />Customer Advisory Board<br />Falsifiable Hypotheses<br />Product Owner<br />Accountability<br />Customer Archetypes<br />Cross-functional Teams<br />Semi-autonomous Teams<br />Smoke Tests<br />Measure Faster<br />Funnel Analysis<br />Cohort Analysis<br />Net Promoter Score<br />Search Engine Marketing<br />Real-Time Alerting<br />Predictive Monitoring<br />Measure Faster<br />Split Tests<br />Clear Product Owner<br />Continuous Deployment<br />Usability Tests<br />Real-time Monitoring<br />Customer Liaison<br />

Myth #1<br />Myth<br />Lean means
cheap. Lean startups try to spend as little money as possible.<br />Truth The Lean Startup method is not about cost, it is about speed. <br />

Myth #2<br />Myth<br />The Lean
Startup is only for Web 2.0/internet/consumer software companies.<br />Truth The Lean Startup applies to all companies that face uncertainty about what customers will want. <br />

Myth #3<br />Myth<br />Lean Startups
are small bootstrapped startups.<br />Truth Lean Startups are ambitious and are able<br /> to deploy large amounts of capital. <br />

Myth #4<br />Myth<br />Lean Startups
replace vision with dataor customer feedback.<br />Truth Lean Startups are driven by a compelling vision, and are rigorous about testing each element of this vision<br />

Thanks!<br /><ul><li>Startup Lessons Learned Blog

http://StartupLessonsLearned.com

In print: http://bit.ly/SLLbookbeta

Getting in touch (#leanstartup)

http://twitter.com/ericries

eric@theleanstartup.com

Additional resources

Lean Startup Wiki http://leanstartup.pbworks.com

Lean Startup Circle http://leanstartupcircle.com</li></li></ul><li>How to
build a Lean Startup<br /><ul><li>Let’s talk about some specifics.

Minimum viable product

Five why’s

Rapid split-tests

Continuous Deployment</li></li></ul><li>Minimum Viable Product<br />Learn
Faster<br />Customer Development<br />Five Whys<br />Build Faster<br />Continuous Deployment<br />Small Batches<br />Minimum Viable Product<br />Refactoring<br />Measure Faster<br />Split Testing<br />Actionable Metrics<br />Net Promoter Score<br />SEM <br />

Why do we build products?<br
/><ul><li>Delight customers

Get lots of them signed
up

Make a lot of money

Realize a big vision; change
the world

Learn to predict the future</li></li></ul><li>Possible
Approaches<br /><ul><li>“Maximize chances of success”

build a great product with
enough features that increase the odds that customers will want it

Problem: no feedback until the
end, might be too late to adjust

“Release early, release often”

Get as much feedback as
possible, as soon as possible

Problem: run around in circles,
chasing what customers think they want</li></li></ul><li>Minimum Viable Product<br /><ul><li>The minimum set of features needed to learn from earlyvangelists – visionary early adopters

Avoid building products that nobody
wants

Maximize the learning per dollar
spent

Probably much more minimum than
you think!</li></li></ul><li>Minimum Viable Product<br /><ul><li>Visionary customers can “fill in the gaps” on missing features, if the product solves a real problem

Allows us to achieve a
big vision in small increments without going in circles

Requires a commitment to iteration

MVP is only for BIG
VISION products; unnecessary for minimal products.</li></li></ul><li>Techniques<br /><ul><li>Smoke testing with landing pages, AdWords

SEM on five dollars a
day

In-product split testing

Paper prototypes

Customer discovery/validation

Removing features (“cut and paste”)</li></li></ul><li>Fears<br
/><ul><li>False negative: “customers would have liked the full product, but the MVP sucks, so we abandoned the vision”

Visionary complex: “but customers don’t
know what they want!”

Too busy to learn: “it
would be faster to just build it right, all this measuring distracts from delighting customers”</li></li></ul><li>Five Whys<br />Learn Faster<br />Five Whys Root<br />Cause Analysis<br />Code Faster<br />Continuous Deployment<br />Measure Faster<br />Rapid Split Tests<br />

Five Whys Root Cause Analysis<br
/><ul><li>A technique for continuous improvement of company process.

Ask “why” five times when
something unexpected happens.

Make proportional investments in prevention
at all five levels of the hierarchy.

Behind every supposed technical problem
is usually a human problem. Fix the cause, not just the symptom.</li></li></ul><li>Rapid Split Tests<br />Learn Faster<br />Five Whys Root<br />Cause Analysis<br />Code Faster<br />Continuous Deployment<br />Measure Faster<br />Rapid Split Tests<br />

Split-testing all the time<br /><ul><li>A/B
testing is key to validating your hypotheses

Has to be simple enough
for everyone to use and understand it

Make creating a split-test no
more than one line of code:</li></ul>if( setup_experiment(...) == "control" ) {<br /> // do it the old way<br />} else {<br /> // do it the new way<br />}<br />

The AAA’s of Metrics<br /><ul><li>Actionable

Accessible

Auditable</li></li></ul><li>Measure the Macro<br /><ul><li>Always look
at cohort-based metrics over time

Split-test the small, measure the
large</li></li></ul><li>Continuous Deployment<br />Learn Faster<br />Customer Development<br />Five Whys<br />Build Faster<br />Continuous Deployment<br />Small Batches<br />Minimum Viable Product<br />Refactoring<br />Measure Faster<br />Split Testing<br />Actionable Metrics<br />Net Promoter Score<br />SEM <br />

Continuous Deployment<br /><ul><li>Deploy new software
quickly

At IMVU time from check-in
to production = 20 minutes

Tell a good change from
a bad change (quickly)

Revert a bad change quickly

And “shut down the line”

Work in small batches

At IMVU, a large batch
= 3 days worth of work

Break large projects down into
small batches</li></li></ul><li>Cluster Immune SystemWhat it looks like to ship one piece of code to production:<br /><ul><li>Run tests locally (SimpleTest, Selenium)

Everyone has a complete sandbox

Continuous Integration Server (BuildBot)

All tests must pass or
“shut down the line”

Automatic feedback if the team
is going too fast

Incremental deploy

Monitor cluster and business metrics
in real-time

Reject changes that move metrics
out-of-bounds

Alerting & Predictive monitoring (Nagios)

Monitor all metrics that stakeholders
care about

If any metric goes out-of-bounds,
wake somebody up

Use historical trends to predict
acceptable bounds

When customers see a failure

Fix the problem for customers

Improve your defenses at each
level</li></li></ul><li>Thanks!<br /><ul><li>Startup Lessons Learned Blog

http://StartupLessonsLearned.com

In print: http://bit.ly/SLLbookbeta

Getting in touch (#leanstartup)

http://twitter.com/ericries

eric@theleanstartup.com

Additional resources

Lean Startup Wiki http://leanstartup.pbworks.com

Lean Startup Circle http://leanstartupcircle.com</li></li></ul><li>There’s much
more…<br />Build Faster<br />Unit Tests<br />Usability Tests<br />Continuous Integration<br />Incremental Deployment<br />Free & Open-Source<br /> Components<br />Cloud Computing<br />Cluster Immune System<br />Just-in-time Scalability<br />Refactoring<br />Developer Sandbox<br />Minimum Viable Product<br />Learn Faster<br />Split Tests<br />Customer Interviews<br />Customer Development<br />Five Whys Root Cause<br />Analysis<br />Customer Advisory Board<br />Falsifiable Hypotheses<br />Product Owner<br />Accountability<br />Customer Archetypes<br />Cross-functional Teams<br />Semi-autonomous Teams<br />Smoke Tests<br />Measure Faster<br />Funnel Analysis<br />Cohort Analysis<br />Net Promoter Score<br />Search Engine Marketing<br />Real-Time Alerting<br />Predictive Monitoring<br />Measure Faster<br />Split Tests<br />Clear Product Owner<br />Continuous Deployment<br />Usability Tests<br />Real-time Monitoring<br />Customer Liaison<br />

From vision to MVP<br />The
goal of customer development is to find a market for the product as currently specified<br />Have a strong vision, be prepared to learn whether it makes sense<br />Feedback from customers tells you about them not you<br />No focus groups<br />

Major Sources of Waste<br />Building
something nobody wants<br />Adding features early adopters don’t need<br />AKA adding features that don’t validate/refute current hypothesis<br />Arguing about product priorities<br />Flip-flopping between plans (iterating in a circle)<br />(these are in order of costliness)<br />

Problem team, Solution team<br />Problem
team: working on customer discovery<br />Solution team: working on Minimum Viable Product (MVP)<br />Both teams are cross-functional<br />Commitment from solution team leader to be personally involved with customer discovery<br />Metrics are people, too<br />

Today’s Agenda<br />Develop deep customer
insight: problem presentation, “day in the life”<br />Translate that insight into actionable per-user metrics<br />Build a model of how those metrics lead to massive success<br />Establish a baseline measurement using a minimum viable product<br />… and then move to Customer Validation<br />

Deep Customer Insight (Problem)<br />Get
out of the building and meet real and potential customers<br />Figure out what problems they have<br />Where does your problem rank on the hierarchy of pain?<br />Figure out the how, what, where, and why of customers using a product like yours<br />How do they describe the category, problem, and competitors – learn their language<br />Learn to tell an early adopter from a mainstream customer<br />

Deep Customer Insight (Solution)<br />Learn
how to describe your solution to potential customers<br />Find out if they agree it solves the problem, assuming it works “by magic”<br />Discover barriers to adoption (would they start using a magic product right away?)<br />Offer to pre-order to discover barriers to actual purchase (and to qualify early adopters)<br />

Customer Archetype<br />Succinct description of
insights<br />Designed to be actionable for whole team<br />If more than one, pick one target<br />Rule: always build for the target archetype without humiliating any other archetype<br />Big savings: avoid building features outside archetype description<br />

Actionable Metrics<br />Assume everything you
learned in discovery is true – how would you know?<br />Use customer insight to plot out the specific funnel for your customers: <br />how they find out about your product<br />how they acquire/try/adopt it <br />how they pay for it<br />how they engage over time<br />Establish per-customer metrics for this funnel<br />Most important: How do you know you’re making the product better?<br />

Gross metrics don’t work<br />Why
not focus on gross revenue, profit, or growth rate?<br />Impossible to predict<br />Keeps team working on high-ROI activities, but innovation tends to be low-ROI (at first)<br />Focus on gross numbers tends to erode differentiation (as everyone does the same “obvious” stuff)<br />

Engine of Growth<br />The goal
of actionable metrics is to establish a working and growing business model<br />Need to understand the “ecosystem” of your business at the per-customer level, and make sure it’s value-creating<br />Marginal revenue > marginal cost<br />High volume, low margin<br />Low volume, high margin<br />Need to understand how this ecosystem supports one of three drivers of growth:<br />Paid (CPA < LTV)<br />Viral (Viral coefficient > 1)<br />Sticky (customer retention extremely high)<br />

Build the model<br />Create the
usual spreadsheet model of your business, but<br />Focus on inputs instead of outputs<br />Come up with reasonable assumptions, and make sure that the outputs are sufficient<br />Recognize that the model will probably change, so relationships are more important than specific numbers<br />

Establish a baseline<br />Minimum viable
product: that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort.<br />Each input in the model represents one key hypothesis about the business<br />Use the MVP to measure each input. Eliminate any features that do not pertain to one of the key inputs.<br />Work on the riskiest hypotheses first<br />

Baseline<br />Once you have baseline
numbers for your business, you are ready for customer validation<br />Probably, the numbers will look terrible – that’s OK<br />Figure out what the deltas are between baseline and a good outcome<br />Figure out which numbers are movable and which are fixed<br />

Customer Validation<br />Once you have
a MVP, become more dynamic<br />Shift from one-time activities to continuous flow, measured by validated learning<br />As you learn, you will be able to influence the actual customer behavior in your model<br />

Validated Learning<br />It’s as important
to know why a metric changed as to be able to show change<br />Growth in gross metrics is always ambiguous, too many external factors<br />Key validation techniques:<br />Revenue per customer<br />Cohort analysis<br />Split-testing<br />

BOD accountability <br />Use validated
learning to demonstrate shared sense of progress among:<br />Founders<br />Board of directors<br />Investors/outside stakeholders<br />Each baseline step is progress<br />After baseline, each pivot is progress<br />

Team accountability<br />Charter semi-autonomous cross-functional
teams, starting with just one solution team <br />Select a mutually-agreed goal<br />Team agrees to hit the goal or die trying<br />Team has representatives from all functions <br />Owns product, marketing, deployment decisions<br />At the end of a cycle, team can achieve success by:<br />Hitting the actionable-metric target<br />Demonstrating deep learning about what went wrong<br />Over multiple cycles, must show this learning is improving chances of hitting targets<br />

Pivot<br />When customer validation fails,
it’s time to pivot<br />Most pivots originate in the solution team: they cannot find a way to make the current hypothesis work.<br />Can’t hit actionable targets<br />Don’t improve on those targets over time<br />Each team must bring key data to a pivot meeting:<br />Solution team must have data about what’s not working<br />Problem team must have evidence for a next hypothesis<br />Both teams must have spent time with current customers<br />Pivot: keep most of the business model the same, change one key part at a time<br />

Types of pivots<br />Customer need
pivot: same customer segment, different need/problem<br />Customer segment pivot: same problem, different segment<br />Business architecture pivot: ie from enterprise to consumer<br />Zoom-in feature pivot: remove features to focus on just one key feature<br />Zoom-out feature pivot: add features to become more of a holistic solution<br />Technology pivot: solve same problem but with different technology stack<br />Channel pivot: same problem, same solution, different path to customers<br />Platform pivot: open up an application to third-parties to become a platform (or vice-versa)<br />

Today<br />Develop deep customer insight:
problem presentation, “day in the life”<br />Translate that insight into actionable per-user metrics<br />Build a model of how those metrics lead to massive success<br />Establish a baseline measurement using a minimum viable product<br />