Lean Startup Co., profile picture
Uploaded byLean Startup Co.
7,210 views

Lean Analytics for Intrapreneurs by Allistair Croll

This document provides an introduction to Lean Analytics for intrapreneurs. It begins with two key lessons: companies die when they fail to adopt new business models, and the difference between a rogue agent and special operative is permission. It then discusses Lean Analytics fundamentals like good metrics being understandable, comparative, ratios or rates, and behavior changing. It covers qualitative vs quantitative data, exploratory vs reporting analytics, and examples of leading metrics. The document emphasizes focusing on one key metric per business stage and provides examples of common metrics for different business models and stages.

Embed presentation

Downloaded 150 times
1 / 222
2 / 222
3 / 222
4 / 222
5 / 222
6 / 222
7 / 222
8 / 222
9 / 222
10 / 222
11 / 222
12 / 222
13 / 222
14 / 222
15 / 222
16 / 222
17 / 222
18 / 222
19 / 222
20 / 222
21 / 222
22 / 222
23 / 222
24 / 222
25 / 222
26 / 222
27 / 222
28 / 222
29 / 222
30 / 222
31 / 222
32 / 222
33 / 222
34 / 222
35 / 222
36 / 222
37 / 222
38 / 222
39 / 222
40 / 222
41 / 222
42 / 222
43 / 222
44 / 222
45 / 222
46 / 222
47 / 222
48 / 222
49 / 222
50 / 222
51 / 222
52 / 222
53 / 222
54 / 222
55 / 222
56 / 222
57 / 222
58 / 222
59 / 222
60 / 222
61 / 222
62 / 222
63 / 222
64 / 222
65 / 222
66 / 222
67 / 222
68 / 222
69 / 222
70 / 222
71 / 222
72 / 222
73 / 222
74 / 222
75 / 222
76 / 222
77 / 222
78 / 222
79 / 222
80 / 222
81 / 222
82 / 222
83 / 222
84 / 222
85 / 222
86 / 222
87 / 222
88 / 222
89 / 222
90 / 222
91 / 222
92 / 222
93 / 222
94 / 222
95 / 222
96 / 222
97 / 222
98 / 222
99 / 222
100 / 222
101 / 222
102 / 222
103 / 222
104 / 222
105 / 222
106 / 222
107 / 222
108 / 222
109 / 222
110 / 222
111 / 222
112 / 222
113 / 222
114 / 222
115 / 222
116 / 222
117 / 222
118 / 222
119 / 222
120 / 222
121 / 222
122 / 222
123 / 222
124 / 222
125 / 222
126 / 222
127 / 222
128 / 222
129 / 222
130 / 222
131 / 222
132 / 222
133 / 222
134 / 222
135 / 222
136 / 222
137 / 222
138 / 222
139 / 222
140 / 222
141 / 222
142 / 222
143 / 222
144 / 222
145 / 222
146 / 222
147 / 222
148 / 222
149 / 222
150 / 222
151 / 222
152 / 222
153 / 222
154 / 222
155 / 222
156 / 222
157 / 222
158 / 222
159 / 222
160 / 222
161 / 222
162 / 222
163 / 222
164 / 222
165 / 222
166 / 222
167 / 222
168 / 222
169 / 222
170 / 222
171 / 222
172 / 222
173 / 222
174 / 222
175 / 222
176 / 222
177 / 222
178 / 222
179 / 222
180 / 222
181 / 222
182 / 222
183 / 222
184 / 222
185 / 222
186 / 222
187 / 222
188 / 222
189 / 222
190 / 222
191 / 222
192 / 222
193 / 222
194 / 222
195 / 222
196 / 222
197 / 222
198 / 222
199 / 222
200 / 222
201 / 222
202 / 222
203 / 222
204 / 222
205 / 222
206 / 222
207 / 222
208 / 222
209 / 222
210 / 222
211 / 222
212 / 222
213 / 222
214 / 222
215 / 222
216 / 222
217 / 222
218 / 222
219 / 222
220 / 222
221 / 222
222 / 222
Lean Analytics for Intrapreneurs Lean Startup Conference 2013 @acroll
When you’re a startup your goal is to find a sustainable, repeatable business model. When you’re a big company your goal is to perpetuate one.
Lean Analytics: Use data to build a better business faster.
Intrapreneur: Someone working to produce disruptive change in an organization that has already found a sustainable, repeatable business model.
(Before we get into Lean Analytics, 2 key lessons.) Lesson one: Companies die because they fail to move to new business models.
$1 8” 5.25” 3.5” Clay Christensen, The Innovator’s Dilemma Time eb oo k $1000 N ot op kt De s ic in M $100 ai nf ra om m e pu te r $10 M Cost per MB 14”
Technologies outstrip what the market needs, driven by feedback from the “best” current customer. $1 8” 5.25” $10 end High er stom cu end Low mer usto c $100 $1000 Clay Christensen, The Innovator’s Dilemma Time
The new market has different criteria for success, which are uninteresting to incumbents. $1 $10 $100 Storage capacity Portability $1000 Clay Christensen, The Innovator’s Dilemma Time
Sometimes this has unintended consequences $1 Smaller disc size means less vibration impact, leading to greater density, increasing storage capacity $10 $100 $1000 Clay Christensen, The Innovator’s Dilemma Time
Three kinds of innovation Improve along current metrics... Switch to a new value model Change the business model entirely ...or alter the rate of improvement Sustain/core Innovate/adjacent Disrupt/transformative (optimizing for more of the same) (introduce nearby product, market, or method) (Fundamentally changing the business model)
Lesson two: The difference between a rogue agent and a special operative is permission.
It’s not your job to prove you’re clever. It’s your job to change outcomes.
A quick introduction to Lean Analytics
Kevin Costner is a lousy entrepreneur. Don’t sell what you can make. Make what you can sell.
The core of Lean is iteration.
Unfortunately, we’re all liars.
Everyone’s idea is the best right? No data, no learning. People love this part! (but that’s not always a good thing) This is where things fall apart.
Analytics can help.
Analytics is the measurement of movement towards your business goals.
In a startup, the purpose of analytics is to iterate to product/market fit before the money runs out.
In a big company, analytics replaces opinion with fact.
It helps you innovate on product, market, and method.
Some fundamentals.
A good metric is: Understandable Comparative If you’re busy explaining the data, you won’t be busy acting on it. Comparison is context. A ratio or rate The only way to measure change and roll up the tension between two metrics (MPH) Behavior changing If you’re busy explaining the data, you won’t be busy acting on it.
The simplest rule If a metric won’t change how you behave, it’s a bad metric. h"p://www.flickr.com/photos/circasassy/7858155676/
Metrics help you know yourself. Customers that buy >1x in 90d 1-15% 15-30% >30% Then you are in this mode Your customers will buy from you You are just like 70% Acquisition Once Hybrid 2-2.5 20% per year of retailers Loyalty >2.5 10% per year of retailers of retailers Focus on Low acquisition cost, high checkout Increasing return rates, market share Loyalty, selection, inventory size (Thanks to Kevin Hillstrom for this.)
Qualitative Quantitative Unstructured, anecdotal, revealing, hard to aggregate, often too positive & reassuring. Numbers and stats. Hard facts, less insight, easier to analyze; often sour and disappointing. Warm and fuzzy. Cold and hard.
Exploratory Reporting Speculative. Tries to find unexpected or interesting insights. Source of unfair advantages. Predictable. Keeps you abreast of the normal, day-to-day operations. Can be managed by exception. Cool. Necessary.
Rumsfeld on Analytics we know Are facts which may be wrong and should be checked against data. we don’t know Are questions we can answer by reporting, which we should baseline & automate. we know Are intuition which we should quantify and teach to improve effectiveness, efficiency. we don’t know Are exploration which is where unfair advantage and interesting epiphanies live. know Things we don’t know (Or rather, Avinash Kaushik channeling Rumsfeld)
Slicing and dicing data Active users 5,000 Cohort: Comparison of similar groups along a timeline. 0 Jan (this is the April cohort) Feb Segment: Cross-sectional comparison of all people divided by some attribute (age, gender, etc.) Mar ☀ ☁ Apr May A/B test: Changing one thing (i.e. color) and measuring the result (i.e. revenue.) ☀ ☁ ☀ ☁ Multivariate analysis Changing several things at once to see which correlates with a result.
Which of these two companies is doing better?
  March April May Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50 1 2 3 4 5 January How about this one? February Cohort Is this company growing or stagnating? January $5 $3 $2 $1 $0.5 $6 $4 $2 $1 March $7 $6 $5 April   $8 $7     $9 February May  
Cohort 2 3 4 5 January Look at the same data in cohorts 1 $5 $3 $2 $1 $0.5 February $6 $4 $2 $1   March $7 $6 $5     April $8 $7       May $9         Averages $7 $5 $3 $1 $0.5
The ability to do things in small batches is behind much of the lean movement.
Short cycle time triggers the immune system of big company budgeting.
(this is called foreshadowing.)
Lagging Historical. Shows you how you’re doing; reports the news. Example: sales. Explaining the past. Leading Forward-looking. Number today that predicts tomorrow; reports the news. Example: pipeline. Predicting the future.
Some examples A Facebook user reaching 7 friends within 10 days of signing up (Chamath Palihapitiya) If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang) Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) A LinkedIn user getting to X connections in Y days (Elliot Schmukler) (From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
Which means it’s time to talk about correlation.
10000 1000 100 10 1 Jan Feb Mar Apr May Ice cream consumption Jun Jul Aug Sept Drownings Oct Nov Dec
Correlated Two variables that are related (but may be dependent on something else.) Ice cream & drowning. Causal An independent variable that directly impacts a dependent one. Summertime & drowning.
A leading, causal metric is a superpower. h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
Growth hacking, demystified. Pick a metric to change Find correlation Test causality Optimize the causal factor
http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/ Is social action a leading indicator of donation?
http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/ Is mobile use?
The Lean Analytics framework.
Eric’s three engines of growth Stickiness Virality Price Approach Keep people coming back. Make people invite friends. Spend money to get customers. Math that matters Get customers faster than you lose them. How many they tell, how fast they tell them. Customers are worth more than they cost.
Dave’s Pirate Metrics AARRR Acquisition Activation Retention Revenue Referral How do your users become aware of you? SEO, SEM, widgets, email, PR, campaigns, blogs ... Do drive-by visitors subscribe, use, etc? Features, design, tone, compensation, affirmation ... Does a one-time user become engaged? Notifications, alerts, reminders, emails, updates... Do you make money from user activity? Transactions, clicks, subscriptions, DLC, analytics... Do users promote your product? Email, widgets, campaigns, likes, RTs, affiliates...
The five stages Stage EMPATHY STICKINESS Gate I’ve found a real, poorly-met need that a reachable market faces. I’ve figured out how to solve the problem in a way they will keep using and pay for. VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives. REVENUE The users and features fuel growth organically and artificially. SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem.
Empathy stage: Localmind hacks Twitter Needed to find out if a core assumption—strangers answering questions—was valid. Ran Twitter experiment instead of writing code Asked senders of geolocated Tweets from Times Square random questions; counted response rate Conclusion: high enough to proceed
Stickiness stage: qidiq streamlines invites Survey owner adds recipient to group Survey owner adds recipient to group Survey owner asks question Recipient gets invite Recipient installs mobile app Recipient creates account, profile Recipient can edit profile, etc. Recipient reads survey question Recipient responds to question Recipient sees survey results 70-90% RESPONSE RATE 10-25% RESPONSE RATE Survey owner asks question Recipient reads survey question Recipient responds to question Recipient sees survey results (Later, if needed…) Recipient visits site; no password! Recipient does password recovery One-time link sent to email Recipient creates password Recipient can edit profile, etc.
Six business model archetypes (Yours is probably a blend of these.)
E-commerce SaaS (freemium?) Mobile app (gaming) Two sided marketplace Media User generated content
Customer Acquisition Cost paid direct search wom (Which means eye charts like these.) Viral coefficient Viral rate inherent virality VISITOR Freemium/trial offer Invite Others Upselling rate Upselling Enrollment Capacity Limit User Disengaged User Free user disengagement Paid conversion Engaged User Tiering Paying Customer Support data Reactivation rate Freemium churn Reactivate Trial Over Disengaged Dissatisfied Trial abandonment rate Resolution Cancel Cancel Reactivate Account Cancelled Billing Info Exp. Paid Churn Rate FORMER USERS User Lifetime Value FORMER CUSTOMERS Customer Lifetime Value
Model + Stage = One Metric That Matters. The business you’re in The stage you’re at E-Com SaaS Mobile 2-Sided Media Empathy Stickiness Virality Revenue Scale One Metric That Matters. UCG
Really? Just one?
Yes, one.
In a startup, focus is hard to achieve.
Having only one metric addresses this problem.
www.theeastsiderla.com
Moz cuts down on metrics SaaS-based SEO toolkit in the scale stage. Focused on net adds. Net adds up: Net adds flat: Net adds down: Was a marketing campaign successful? Were customer complaints lowered? Was a product upgrade valuable? Can we acquire more valuable customers? What product features can increase engagement? Can we improve customer support? Are the new customers not the right segment? Did a marketing campaign fail? Did a product upgrade fail somehow? Is customer support falling apart?
Metrics are like squeeze toys. http://www.flickr.com/photos/connortarter/4791605202/
Ecommerce Empathy Stickiness Virality 2-sided market Scale Mobile app User-gen content Media Interviews; qualitative results; quantitative scoring; surveys Loyalty, conversion Inventory, listings CAC, shares, SEM, sharing reactivation (Money from transactions) Revenue SaaS Engagement, Downloads, churn churn, virality Content, spam Traffic, visits, returns Inherent virality, CAC Invites, sharing Content virality, SEM WoM, app ratings, CAC (Money from active users) Transaction, CLV Transactions, commission Upselling, CAC, CLV CLV, ARPDAU Affiliates, white-label Other verticals API, magic #, Spinoffs, mktplace publishers (Money from ad clicks) Ads, donations CPE, affiliate %, eyeballs Analytics, user data Syndication, licenses
Better: bit.ly/BigLeanTable
A company loses a quarter of its customers every year. Is this good or bad?
Not knowing what normal is makes you do stupid things.
Listening to what normal is makes you ignore disruptive things.
(more foreshadowing.)
Baseline: 5-7% growth a week • Are there enough people who really care enough to sustain a 5% growth rate? • Don’t strive for a 5% growth at the expense of really understanding your customers and building a meaningful solution • Once you’re a pre-revenue startup at or near product/market fit, you should have 5% growth of active users each week • Once you’re generating revenues, they should grow at 5% a week “A good growth rate during YC is 5-7% a week,” he says. “If you can hit 10% a week you're doing exceptionally well. If you can only manage 1%, it's a sign you haven't yet figured out what you're doing.” At revenue stage, measure growth in revenue. Before that, measure growth in active users. Paul Graham, Y Combinator
Baseline: 10% visitor engagement/day 30% of users/month use web or mobile app 10% of users/day use web or mobile app 1% of users/day use it concurrently Fred Wilson’s social ratios
Baseline: 2-5% monthly churn • The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s on the job. • Get below a 5% monthly churn rate before you know you’ve got a business that’s ready to grow (Mark MacLeod) and around 2% before you really step on the gas (David Skok) • Last-ditch appeals and reactivation can have a big impact. Facebook’s “don’t leave” reduces attrition by 7%.
Baseline: CLV calculation 25% 5% 2% monthly churn monthly churn monthly churn 100/25=4 100/5=20 100/2=50 The average customer lasts 4 months The average customer lasts 20 months The average customer lasts 50 months
Baseline: CAC under 1/3 of CLV • CLV is wrong. CAC Is probably wrong, too. • Time kills all plans: It’ll take a long time to find out whether your churn and revenue projections are right • Cashflow: You’re basically “loaning” the customer money between acquisition and CLV. • It keeps you honest: Limiting yourself to a CAC of only a third of your CLV will forces you to verify costs sooner. 20 mo. CL $30/month per customer $600 CLV 1/3 spend $200 CAC Now segment those users!
Putting this to work: The Lean Analytics Cycle
Pick a KPI Success! Pivot or give up Draw a new line Try again Did we move the needle? Measure the results Draw a line Find a potential improvement Without data: make a good guess With data: find a commonality Design a test Hypothesis Make changes in production
Do AirBnB hosts get more business if their property is professionally photographed?
Gut instinct (hypothesis) Professional photography helps AirBnB’s business Candidate solution (MVP) 20 field photographers posing as employees Measure the results Compare photographed listings to a control group Make a decision Launch photography as a new feature for all hosts
5,000 shoots per month by February 2012
Hang on a second.
SRSLY? Gut instinct (hypothesis) Professional photography helps AirBnB’s business
Pick a KPI Success! Pivot or give up Draw a new line Try again Did we move the needle? Measure the results Draw a line Find a potential improvement Without data: make a good guess With data: find a commonality Design a test Hypothesis Make changes in production
Without data: make a good guess With data: find a commonality “Gee, those houses that do well look really nice.” “Computer: What do all the highly rented houses have in common?” Maybe it’s the camera. Camera model.
Landing page design A/B testing URL shortening Publisher analytics Cohort analysis General analytics Funnel analytics SaaS analytics User analytics Spying on users Influencer Marketing Gaming analytics User segmentation User interaction Customer satisfaction KPI dashboards
Agenda An introduction to Lean Analytics (30m) The challenges of being big (15m) When you have support (30m) The Lean Analytics framework (30m) A dose of pragmatism (15m) Metrics for innovation portfolios (15m) Break (15m) Some non-tech examples (10m) Tools of the trade (15m) When it’s you against the world (20m) Break (15m)
It’s different when you’re big.
It is way too easy to mix these up. Target market B2B B2C Early stage Big/incumbent Company size/age Less WoM More formal decisions Intrapreneurs Business model vs. company stage Slower cycle time More legacy constraints
If big firms can’t innovate, it’s this guy’s fault.
When product and market are known, companies compete on how they do things.
To get the incremental cost to zero, companies competed on scale. (Literally, an economy of scale)
Scale comes from process, IP, org chart, capitalization. All of these assume the future will be like the past, only more so.
If a startup is an organization designed to search for a sustainable, repeatable business model, then an established company is an organization designed to perpetuate one.
Technology has radically changed the incremental cost of businesses.
Software is eating the world. http://www.flickr.com/photos/ebolasmallpox/3733059220/
An economic order quantity of one. Crafted Massproduced Automated Digital Quantity Few Many Some One Cost High Low Medium Free Lead time Small Large Medium None Self-service Medium None Some Lots Customization High None Some Lots This is why software is eating the world. • Cloud computing • Social media • 3D printing • Per-customer analysis • Mobile tracking • Etc...
Sustainable competitive advantage allows for inertia and power to build up along the lines of an existing business model, which will soon die. Instead, seek transient competitive advantage. Rita Gunther McGrath, The End of Competitive Advantage
Scale is now a liability. Compete on cycle time.
Optimizing the probable means discounting the possible.
This isn’t about a lack of resources.
http://www.flickr.com/photos/maladjusted/5207565912
Blockbuster had a lot going for it.
Plenty of inventory, of course. But that matters less than...
...market intelligence, customers, existing payment approval, and customer history.
The problem was framing: Blockbuster thought it was in the video store management business. Netflix realized it was in the entertainment delivery business.
YOU ARE HERE
OPTIMIZATION OF CURRENT METRICS LOCAL MAXIMUM YOU ARE HERE
INNOVATION WITH NEW RULES YOU ARE HERE GLOBAL MAXIMUM
YOU ARE HERE SHORT-TERM INVESTORS HATE GOING DOWNHILL
First mover advantage happens long before the market emerges. • $1B invested in Nook • $475M operating loss in April 2013 • CEO gone
Constraints slow things down vs.
Everything to lose: Why big companies need innovation.
F500 Life Expectancy (http://csinvesting.org/2012/01/06/fortune-500-extinction/) 75 15 years years 1950 Growth by entering a new business ... 2010 95 99 % fail % fail Corporate Strategy Board Clay Christensen
In other words, if your job is change you have your work cut out for you.
2011 MIT study of 179 large publicly traded firms Companies that use data-driven analytics instead of intuition have 5%-6% higher productivity and profits than competitors. Brynjolfsson, Erik, Lorin Hitt, and Heekyung Kim. "Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance?." Available at SSRN 1819486 (2011).
A dose of pragmatism
Many models for enterprise innovation Core Adjacent Transformative Do the same thing better. Nearby product, market, or method. Start something entirely new. Regional optimizations. Innovation, go-tomarket strategies. Reinvent the business model. • Customer development • Test similar cases • Parallel deployment • Analytics & cycle time • Fail fast • Skunkworks/R&D • Focus on the search • Ignore the current model & margins • Get there faster • Smaller batches • Solution, then testing • Increased accountability
Another way to look at it Core Adjacent Transformative Know the problem (customers tell you it) Know the solution (customers/regulations/ norms dictate it.) Know the problem (market analysis) Don’t know the solution (non-obvious innovation confers competitive advantage.) Don’t know the problem (just an emerging need/ change) Don’t know the solution. Waterfall: Execution matters Agile/scrum: Iteration matters Lean Startup: Discovery matters
The Three Horizons Core Adjacent Horizon 1 improves the Horizon 2 extends the current business operations business into new products, in the next 12 months. markets, or methods in the next 3 years. Those core businesses most readily identified with the company name and those that provide the greatest profits and cash flow. Maximize remaining value. Emerging opportunities, including rising entrepreneurial ventures likely to generate substantial profits in the future but that could require considerable investment. Transformative Horizon 3 changes the industry you’re in and your value network in the next 6 years. Ideas for profitable growth down the road—for instance, small ventures such as research projects, pilot programs, or minority stakes in new businesses. http://www.mckinsey.com/insights/strategy/enduring_ideas_the_three_horizons_of_growth
Lean applies. Startup may not. Core Adjacent Transformative Lean startup Lean methodologies.
Method (new “how”) 3 kinds of innovation Product (new “what”) Market (new “who”)
Another view Core Method (new “how”) Market diversification Distribution innovation Pure startup Adjacent Disruptive Inventive Market (new “who”) Channel expansion Product (new “what”)
A three-maxima model of enterprise innovation Business optimization (five mores) Current state Business model innovation Product, market, method innovation You can convince executives of this because some of it is familiar. This terrifies them because it eats the current business.
Improvement Adjacency Remodeling Do the same, only better. Explore what’s nearby quickly Try out new business models Lean approaches apply, but the metrics vary widely. Sustain/ core Innovate/ adjacent Disrupt/ transformative
More things To more people Sustaining innovation is about more of the same. (says Sergio Zyman) Inventory increase Gifting, wish lists Highly viral offering Low incremental order costs For more money Maximum shopping cart Price skimming/tiering More often Loyal customer base that returns Demand prediction, notification More efficiently Supply chain optimization Per-transaction cost reduction
Software, experimentation, and iterative cycles of learning help you get to the local maximum better and faster. That’s a good thing. But it’s not the only thing.
Adjacent innovation is about changing one part of the model in a way that alters the value network.
Adjacent product to the same market in the same way
Selling the same product to an adjacent market in the same way. Of P&G’s 38 brands, only 19 were sold in Asia as of 2011 Market expansion is seldom selling the same thing to new people. In Asia, P&G needed to Align pricing with novelty (prestige, mass-tige, over-the-counter) Change consumer expectations (moving from dilutes to concentrates) Adjust positioning and ingredients such as white fungus, ginseng, and the parasitic cordyceps
Selling the same product to the same in The biggest innovation logistics of the 20th century. market in a new way. http://www.flickr.com/photos/photohome_uk/1494590209
Selling the same product to the same market in a new way.
(At this point, observant Intrapreneurs should be asking, should P&G be in the house cleaning business? And that would be transformative.)
Transformative innovation is about taking a leap, changing more than one dimension simultaneously in search of a new business model.
If sustaining, incremental innovation produces linear growth, then disruptive, transformative innovation produces exponential growth.
Transformative isolation: Skunkworks
Some non-tech examples.
I lied. Everyone is a tech company.
Cost of experiments: down. http://www.flickr.com/photos/puuikibeach/4789015423 Cost of attention: way up. http://www.flickr.com/photos/elcapitanbsc/3936927326
Let’s pick on restaurants for a while.
Empathy: find the need Before opening, the owner first learns about the diners in her area, their desires, what foods aren’t available, and trends in eating. Key metrics: Popular items; frequent questions; before/after dining patterns. Reference: Emerging need.
Stickiness: confirm the need is met. She develops a menu and tests it out with consumers, making frequent adjustments until tables are full and patrons return regularly. She’s giving things away, asking diners what they think. Variance and uncertain inventory make costs high. Key metrics: Customer loyalty; recommendations; referrals; endorsements; inventory turnover. Reference: Business idea.
Virality: will it spread? She starts loyalty programs to bring frequent diners back, or to encourage people to share with their friends. She engages on Yelp and Foursquare. Key metrics: Customer loyalty; recommendations; referrals; endorsements. Reference: Business positioning
Revenue: prove the business viability With virality kicked off, she works on margins—fewer free meals, tighter controls on costs, more standardization. She focuses on the price of acquiring new customers. Key metrics: Acquisition cost, revenue per cover, capacity, turnover. Reference: Business model.
Scale: prove it’s a market Knowing she can run a profitable business, she funnels revenues into marketing and promotion. She reaches out to food reviewers, travel magazines, and radio stations. She launches a second restaurant, or a franchise. Key metrics: Franchise health; repeatability; problems escalated; variance; franchise revenues. Reference: Business plan.
A line in the sand 30% Labor costs Gross revenue = 24% 20% Too costly? Just right Understaffed?
A leading indicator (Varies by restaurant. McDonalds ≠ Fat Duck.) 50 reservations at 5PM http://www.flickr.com/photos/mysticcountry/3567440970 250 covers that night http://www.flickr.com/photos/avlxyz/4889656453
Restaurant MVP http://www.flickr.com/photos/southbeachcars/6892880699
Is tip amount a leading indicator of longterm revenue?
Why does every table get the same menu?
Is purple ink better? http://tippingresearch.com/uploads/managing_tips.pdf
Stalking customers is pretty easy. http://targetmycustomers.appspot.com http://tippingresearch.com/uploads/managing_tips.pdf
When it’s you vs. the world. (A bagful of tricks from agitators in companies of all sizes.)
The skills you need make you a pariah. Successful innovators share certain attributes. Bad listener: Willfully ignore feedback from your best customers. Cannibal: If successful, destroying existing revenue streams. Job killer: Automation & lower margins are your favorite tools. Security risk: Advocate of transparency, open data, communities. Narcissist: Worry constantly about how you’ll get attention. Slum lord: Sell to those with less money, deviants, and weirdos.
New Know what kind of innovation you’re Market after. Current Based on H. Igor Ansoff’s matrix Startup: New products for new !) markets. News rules, es cc units, business su at organizational re g nd structure. Innovation. (a Market development: Sell existing products to new markets, segments, uses. Export & license. lf a ll a Penetrate: itic ol Increase revenues, fp o market share, product sk ri ed quality, brand as re differentiation. nc I Marketing. Current ut o Product development: Invent new products for your market. R&D, enhancements. Acquisition. Product New
Frame it like a study Product creation is almost accidental. Unlike a VC or startup, when the initiative fails the organization still learns. http://www.flickr.com/photos/creative_tools/8544475139
When in doubt, collect data From tackling the FTA rate to visualizing the criminal justice supply chain.
Use data to create a taste for data Sitting on Billions of rows of transactional data David Boyle ran 1M online surveys Once the value was obvious to management, got license to dig.
Don’t just collect data, chase it.
Understand hidden constraints That pencil story is a myth. Graphite is conductive and explosive. The Minimum Viable Product is Viable for a reason.
Know what has to be built in-house SAP integration Employment law
Think subversively. http://www.flickr.com/photos/bootbearwdc/1243690099/
Everything’s an excuse to experiment
Seriously. Everything.
Run it as a consulting business first. (Just don’t get addicted to it. Your goal is to learn and overcome integration challenges and find the 20% of features that 80% of the market will pay for.)
Convince your boss she asked for this Success! Pick a KPI Pivot or give up Draw a new line Try again Did we move the needle? Measure the results Draw a line in the sand Find a potential improvement Without data: make a good guess With data: find a commonality Design a test Make changes in production Hypothesis
Slaughter a sacred cow: Prove a long-held assumption is wrong and you’ve got people’s attention. Know what you’ll do with it ahead of time.
Take baby steps.
Netflix
Tesla http://www.hdwallpapersinn.com/wp-content/uploads/2012/12/600-tesla.jpg
Twitter’s 140-character limit isn’t arbitrary. It’s constrained by the size of SMS (160 characters) and username (20 characters.) http://i.i.cbsi.com/cnwk.1d/i/tim/2011/11/18/ sms_screen_twitter_activity_stream_270x405.png
Figure out how to translate it back to a simple model that fits the company’s existing value model. If your company dies, this is why.
Intrapreneurs often have to use proxies Stage Startup metrics Intrapreneur metrics Empathy Customers interviewed (needs & solutions), assumptions quantified, TAM, monetization possibility Non-customers interviewed; assumptions quantified, constraints identified, TAM, disruption potential Churn, engagement Support tickets, integration time, call center data, delays Viral coefficient, viral cycle time Net Promoter Score, referrals, case study willingness Revenue Attention, engagement Billable activity; signed LOIs; pilot programs; after-development profitability Scale Automation Contribution, training costs, licensing Stickiness Virality
When you have support. (What companies like P&G, Cognizant, GE, and Motorola do with a formal innovation program.)
Do you really have permission? What resources do you have? Staff, budget, unfettered access to customers? What scope of change can you make? Pricing, product, channel, branding?
Step one: Develop a portfolio approach.
Innovation portfolios at big companies Return Investment Core Adjacent 70% 20% 10% 20% Transformative 10% 70%
1. Organizations’ structures emerge as a way to optimize the current business model. 2. Most innovations will come not from product or market, but from method— business model innovation. 3. Innovation groups must exercise organizational amnesia at the outset.
Step one: Frame your problems.
Maybe not so crazy. http://www.flickr.com/photos/mizrak/4592706544
Step two: Define your gates and filters. These may lead to myopia. They are also your unfair advantages.
Step three: Secure funding, resources, and executive backing.
Step four: Generate new ideas.
Find non-obvious adjacencies POWER GRID WHICH FEEDS A NEEDS LIGHT AN BULB ELECTRICAL GENERATOR HAS A TURBINE LIKE A TURNED AROUND BECOMES A PLANE ENGINE TRAIN ENGINE REQUIRES SPINS & VIBRATES LIKE AN AND LOOKS LIKE A SOFTWARE TO CUT DOWN TREES BETTER MRI MACHINE WIND TURBINE
Build an ecosystem Canada’s largest directory publishing and local marketing services company 1.5M listings from 420K SMB & national customers Revenues >$1.2B 2,500 employees Created third-party listing API Took 8-10 mo (2009-10) to get approval API payoff happened 2y later KPI evolution Yahoo replaced Canadian digital properties search with the YellowAPI Soft: Signups, SDK, downloads Improved SEO, Comscore App usage, deals signed Functional prototype in hours, testing in days, and launching in weeks. Faster time to partnerships Budgets tripled in 2013 API calls generated API-generated revenue
Five common models for transformative innovation Acquisition Collaboration Isolation Buy promising startups Crowdsource, work with universities, suppliers, etc. Create a separate group with different conditions Incubation Internal startup ecosystem; LoB are “investors” Integration The LoB does innovation internally
Step five: Test by doing (experimentation beats projection.)
Focus on the model, not the plan Demand Amt Growth Wk 1 Wk 2 Wk 3 People per day on sidewalk 200 4% 204 Percent that buy a glass 10% 5% 15% 20% 25% Daily customers 20 31 Revenue Price per cup Profit per cup Daily profit 43 225 56 $156 $216 $281 $5 $5 Cost of Goods Cost per cup 216 $5 $5 30.6 41.5 52.8 $1 -2% .98 .96 .94 $4 4.02 4.04 4.06 $80 125 175 228
A business plan is just what happens when you drag the business model to the right.
Designing an experiment Problem, solution, and result hypothesis Test strategy (PoC, survey, interviews, kickstarter, prototype, A/B, etc.) Cohort & segment to be tested Metric or assumption being tested Timebox or total for test Action you’ll take if you pass or fail
Step six: Know what happens afterwards
Qualcomm’s innovation model: What was missing Hypothesis Unclear what happened to founders Needed a middle PoC decision Sustainability, not feasibility http://blogs.berkeley.edu/2013/01/28/ designing-a-corporateentrepreneurship-program-aqualcomm-case-study-part-1-of-2/ Experiment Boot camp Idea generation and selection Existing models Implement Idea advancement POC Biz sustainability Tech feasibility POC Ideas New models Open innovation Strategic value decision Boot camp decision End user/partner desirability Option value Implement decision Actions Exit value Company crowd storm Small team designs & Company extracts value conducts experiments
Step seven: Rinse, repeat.
The Lean Analytics lifecycle of an Intrapreneur Get buy-in Political fallout Find problems; don’t test demand. Skip the business case, do analytics Entitled, aggrieved customers Stickiness Know your real minimum based on expectations, regulations Hidden “must haves”, feature creep Virality Build inherent virality in from the start; attention is the new currency Luddites who don’t understand sharing Revenue Consider the ecosystem, channels, and established agreements Channel conflict, resistance, contracts Hand the baton to others gracefully Hating what happens to your baby Beforehand Empathy Scale
Metrics for innovation portfolios.
Core metrics Business plan. Metrics that matter Assume it will improve. • Return on investment • Total cost of ownership • Improvement in KPI • Total served market Product, market, and method will remain the same Examples: Redesigning packaging; pricing adjustment
Adjacent metrics Business model. Metrics that matter Assume it will fail. • Virality & word of mouth • Early adopter stickiness • Regulation • Total addressable market Your ultimate use case won’t be what you think it is today. Example: Mr. Clean Magic Eraser
Transformative metrics Business idea. Assume it will fail. You hope it will have the consequences you want but aren’t sure how. Example: Headcam recordings of all officers Metrics that matter • People I’ve talked to • Prototype creation speed • Assumptions validated • Problems uncovered • Technical feasibility • Hidden constraints
Key points to clarify in an innovation program Hypothesis Experimentation • Articulate problems • Frame known advantages • Define the right filters • Many idea sources • Confirm funding (money, • Prioritize riskiest • • • people, customer access) Agree on analytical framework Balance market, product, & method adjacencies • • assumptions Time-box assessment stages Test technology, demand, and business feasibility MVP, prototype, pilot, or science as appropriate for type of innovation Implementation • Temporary incubator • Find a home or building • • • one Keep innovators involved Merge metrics with existing business KPIs Synchronize innovation cycles with enterprise cycles (budget, etc.) Portfolio metrics; Gates and KPIs for each stage; mix of core, adjacent, and disruptive innovation.
Hypothesis Goals, constraints, context Sourcing Top-down Bottom-up Sourcing Filtering Biases Strengths Focus Mandate Outside-in ost e l alm as th wil s w this . thi at re a o . e th ange. here a d t ot ch N y dt nee n inl eck a ions I erta P d c rat it. MV of alte e to ak nch m bu Boot camp POC Core Optimization Crosspollinate to current managers Adjacent De-risking Best solution Test/ validate w/current customers Disruptive Reframing R&D M&A Grow as a distinct business Evaluation of the innovation program itself Implementation Socializing Integration Adoption by existing line of business. Independence Creation of a new line of business.
Some tools and tricks
Traction graphs Your business model The stage you’re at ... change often if you’re doing it right. Your one metric So how do you track that over time?
Traction graphs Jan Feb Signups per day This axis changes for each metric Mar Conversion rate Apr Churn rate May Jun Viral coefficient
Traction graphs 0% Jan Signups per day Feb Mar Conversion rate Apr Churn rate May Jun Viral coefficient
Use vanity to get to meaningful metrics Your goal is to produce outcomes If the outcomes require action, and vanity motivates actors, use it But show how the vanity metric is a leading indicator of the real one Web traffic Activation Cart Size Conversion x rate Revenue
The three threes Three assumptions What big bets are you making? • “People will answer questions” • “Organizers are frustrated with how to run conferences” • “We'll make money from parents” • “Amazon is reliable enough for our users.” Three actions to take What are you doing to make these assumptions happen (or identify they’re wrong and change course?) • Product enhancements • Marketing strategies Three experiments to run • Feature tests • Continuous deployment • A/B testing • Customer survey
The three threes Three assumptions Monthly Board, investors, founders Strategy Three actions to take Weekly Executive team Tactics Three experiments to run Daily Employees Execution
The three threes Many people will answer questions Three assumptions Three actions to take Three experiments to run Get more people Change the UI Increase answer % Test timings Test better questions Questions from peers
The problem-solution canvas The Goal is to Learn CURRENT STATUS • List key metrics you’re LAST WEEK’S LESSONS LEARNED AND ACCOMPLISHMENTS) • What did you learn last week? tracking, where they’re at, and • What was accomplished? compare with last few weeks • On track: YES / NO? How are things trending? •
The problem-solution canvas Problem #1 (put name here) HYPOTHESIZED SOLUTIONS • List possible solutions that you’ll start working on next week. Rank them. • Why do you believe each solution will help you solve or complete solve the problem? METRICS / PROOF + GOALS • Metrics you’ll use to measure whether or not the solutions are doing what you hoped (solving the problem) • List proof (qualitative) you’ll use as well • Define goals for the metric Problem #2 (put name here) HYPOTHESIZED SOLUTIONS • List possible solutions that you’ll start working on next week. Rank them. METRICS / PROOF + GOALS • Metrics you’ll use to measure whether or not the solutions are doing what you hoped (solving the problem)
Use proxy data. Just be careful.
“A subjective degree of belief should rationally change to account for evidence.” (AKA Bayes’ Theorem.)
Conclusions
Key points Intrapreneurship is about adjacent or transformative innovation Sustaining innovation focuses on the Five Mores, within the current product, market, method, and business model. Adjacent innovation may come from a new product, market, or method, but the same business model Disruptive innovation has different customers, KPIs, and models The difference between a rogue agent and a special operative is permission Portfolios need framing, sourcing, filters, metrics, and socializing Balancing isolation and integration, R&D and M&A is contentious
“The most important figures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them.” Lloyd S. Nelson
Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
ARCHIMEDES HAD TAKEN BATHS BEFORE.
Once, a leader convinced others in the absence of data.
Now, a leader knows what questions to ask.
Ben Yoskovitz byosko@gmail.com @byosko Alistair Croll acroll@gmail.com @acroll

More Related Content

PDF
Slides from Growthcon 2014 Lean Analytics masterclass
byLean Analytics
 
PDF
Lean Analytics: Using Data to Build a Better Business Faster
byLean Startup Co.
 
PDF
Lean Analytics workshop for Dublin City University, April 2014
byLean Analytics
 
PDF
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
byLean Analytics
 
PDF
Croll lean analytics workshop (3h) - lean ux nyc april 2014
byLean Analytics
 
PPT
Lean analytics: Five lessons beyond the basics
byLean Analytics
 
PDF
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
byLean Analytics
 
PPTX
Marketers flunk the big data test
byTavva G N R S N Prudhvith
 
Slides from Growthcon 2014 Lean Analytics masterclass
byLean Analytics
 
Lean Analytics: Using Data to Build a Better Business Faster
byLean Startup Co.
 
Lean Analytics workshop for Dublin City University, April 2014
byLean Analytics
 
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
byLean Analytics
 
Croll lean analytics workshop (3h) - lean ux nyc april 2014
byLean Analytics
 
Lean analytics: Five lessons beyond the basics
byLean Analytics
 
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
byLean Analytics
 
Marketers flunk the big data test
byTavva G N R S N Prudhvith
 

What's hot

PPTX
Does big data = big insights?
byColin Strong
 
PPTX
How Companies Turn Data Into Business Value
byJamie Hribal
 
PPTX
The ultimate guide to data storytelling | Materclass
byGramener
 
PDF
Understanding Lean Analytics (and how analytics helps businesses win)
byLean Analytics
 
PPTX
Storyfying your Data: How to go from Data to Insights to Stories
byGramener
 
PDF
The Brave 100: The Battle for Supremacy in Small Business Lending
byFrank Rotman
 
PDF
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
byLean Analytics
 
PDF
Harvesting the value from Advanced Analytics
byJaap Vink
 
PDF
Big data can be used at SME's too
byGeorge Antony
 
PPTX
The History | EquipmentFX
byJonathan Usher
 
PDF
Is Your Company Braced Up for handling Big Data
byhimanshu13jun
 
PDF
Be a Great Product Leader (Zynga 2016)
byAdam Nash
 
PDF
CFO's Guide to Business Analytics
byManish Desai
 
PDF
Scott Thomson, Darren Drew. getting data fit
bybetterbigdata
 
PDF
Lean data modeling the art of mitigating risk for changing requirements 201607
byPaul Delgman
 
PDF
Slides from New Media Manitoba Lean Analytics workshop, June 2015
byLean Analytics
 
PPTX
Algorithms and the technology of personalisation final
byColin Strong
 
PDF
Data & Storytelling - What Now?
byGramener
 
PPTX
Reflections On The Value Proposition For Information Quality
byCastlebridge Associates
 
PPTX
Storytelling for analytics | Naveen Gattu | CDAO Apex 2020
byGramener
 
Does big data = big insights?
byColin Strong
 
How Companies Turn Data Into Business Value
byJamie Hribal
 
The ultimate guide to data storytelling | Materclass
byGramener
 
Understanding Lean Analytics (and how analytics helps businesses win)
byLean Analytics
 
Storyfying your Data: How to go from Data to Insights to Stories
byGramener
 
The Brave 100: The Battle for Supremacy in Small Business Lending
byFrank Rotman
 
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
byLean Analytics
 
Harvesting the value from Advanced Analytics
byJaap Vink
 
Big data can be used at SME's too
byGeorge Antony
 
The History | EquipmentFX
byJonathan Usher
 
Is Your Company Braced Up for handling Big Data
byhimanshu13jun
 
Be a Great Product Leader (Zynga 2016)
byAdam Nash
 
CFO's Guide to Business Analytics
byManish Desai
 
Scott Thomson, Darren Drew. getting data fit
bybetterbigdata
 
Lean data modeling the art of mitigating risk for changing requirements 201607
byPaul Delgman
 
Slides from New Media Manitoba Lean Analytics workshop, June 2015
byLean Analytics
 
Algorithms and the technology of personalisation final
byColin Strong
 
Data & Storytelling - What Now?
byGramener
 
Reflections On The Value Proposition For Information Quality
byCastlebridge Associates
 
Storytelling for analytics | Naveen Gattu | CDAO Apex 2020
byGramener
 

Viewers also liked

PDF
Innovation Accounting in Practice
byLean Startup Co.
 
PPT
10,000 Startups
byLean Startup Co.
 
PPTX
Seriously Advanced A/B Testing by Wyatt Jenkins
byLean Startup Co.
 
PPTX
Lean Leadership Lessons by Intuit
byLean Startup Co.
 
PDF
Acquiring Your First Users Out of Thin Air by Kathryn Minshew
byLean Startup Co.
 
PDF
Crossing the Concierge Chasm by Khalid Smith & Nicole Tucker-Smith
byLean Startup Co.
 
PPTX
How a Daily Meeting Can Transform a Distributed Team
byLean Startup Co.
 
PPTX
What Works in Silicon Valley Doesn't Work Everywhere: How to Apply Lean Start...
byLean Startup Co.
 
PPTX
How Validation and Vision Co-exist by Andres Glusman
byLean Startup Co.
 
PDF
Lean Content: Testing Marketing Copy (Instead of Spinning Your Wheels)
byLean Startup Co.
 
PPTX
Bonfire of the Vanity Metrics: Numbers You’re Still Using and Shouldn’t
byLean Startup Co.
 
PDF
Keep Your Executives Close to Your Customers by Usha Viswanathan
byLean Startup Co.
 
PDF
Using Lean Startup Principles to Close the Digital Divide by Navarrow Wright
byLean Startup Co.
 
PPTX
The Medium Is the Message by Patrick Vlaskovits
byLean Startup Co.
 
PDF
Where Is Everyone? Tools You Can Buy and Tools You Can Build for a Connected ...
byLean Startup Co.
 
PDF
I (Heart) Ugly
byLean Startup Co.
 
PPT
Tool Talk: Trello
byLean Startup Co.
 
PDF
Secrets of Rapid Mobile App Development
byLean Startup Co.
 
PPTX
The Purpose of Purpose on a Virtual Team
byLean Startup Co.
 
PPT
Making Decisions By Ignoring Sales Metrics
byLean Startup Co.
 
Innovation Accounting in Practice
byLean Startup Co.
 
10,000 Startups
byLean Startup Co.
 
Seriously Advanced A/B Testing by Wyatt Jenkins
byLean Startup Co.
 
Lean Leadership Lessons by Intuit
byLean Startup Co.
 
Acquiring Your First Users Out of Thin Air by Kathryn Minshew
byLean Startup Co.
 
Crossing the Concierge Chasm by Khalid Smith & Nicole Tucker-Smith
byLean Startup Co.
 
How a Daily Meeting Can Transform a Distributed Team
byLean Startup Co.
 
What Works in Silicon Valley Doesn't Work Everywhere: How to Apply Lean Start...
byLean Startup Co.
 
How Validation and Vision Co-exist by Andres Glusman
byLean Startup Co.
 
Lean Content: Testing Marketing Copy (Instead of Spinning Your Wheels)
byLean Startup Co.
 
Bonfire of the Vanity Metrics: Numbers You’re Still Using and Shouldn’t
byLean Startup Co.
 
Keep Your Executives Close to Your Customers by Usha Viswanathan
byLean Startup Co.
 
Using Lean Startup Principles to Close the Digital Divide by Navarrow Wright
byLean Startup Co.
 
The Medium Is the Message by Patrick Vlaskovits
byLean Startup Co.
 
Where Is Everyone? Tools You Can Buy and Tools You Can Build for a Connected ...
byLean Startup Co.
 
I (Heart) Ugly
byLean Startup Co.
 
Tool Talk: Trello
byLean Startup Co.
 
Secrets of Rapid Mobile App Development
byLean Startup Co.
 
The Purpose of Purpose on a Virtual Team
byLean Startup Co.
 
Making Decisions By Ignoring Sales Metrics
byLean Startup Co.
 

Similar to Lean Analytics for Intrapreneurs by Allistair Croll

PDF
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
byLean Startup Co.
 
PDF
Alistaire croll lean analytics - montreal lean startup circle - september 2018
byLean Startup Circle Montreal
 
PDF
Introduction to Lean Analytics for Lean Startup Circle SF
byLean Analytics
 
PDF
Lean Analytics workshop (from Lean Startup Conf)
byLean Analytics
 
PDF
Lean Analytics @ MicroConf
byLean Analytics
 
PPTX
Innovation accounting and key metrics for startups
byBlaz Kos
 
PDF
Lean Analytics for Startups and Enterprises
byLean Analytics
 
PDF
Lean analytics
byAndrii Gakhov
 
PDF
Digital analytics: Wrap-up (Lecture 12)
byJoni Salminen
 
PDF
online startups metrics
byHatem Kameli
 
PDF
SBC Growth Week - Lean Analytics
byMeasureWorks
 
PDF
Startup analytics
byTopi Järvinen
 
PDF
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
byjeroentjepkema
 
PPTX
Stamats: Analytics Webinar
bycjcunniff
 
PDF
Growth Hacking @ Sup de Pub International Track — part 1
byMaxime Pico
 
PDF
Lean Analytics - TDC2013
byLuca Bastos
 
PPTX
ETE 2013: Going Big with Big Data...one step at a time
byAnita Andrews
 
PDF
Data strategy
byCarbon Five
 
PPT
7 Myths of Lean and How Analytics Can Help
byLean Analytics
 
PPT
Lean analytics 40 minute intro
byPinkesh Shah
 
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
byLean Startup Co.
 
Alistaire croll lean analytics - montreal lean startup circle - september 2018
byLean Startup Circle Montreal
 
Introduction to Lean Analytics for Lean Startup Circle SF
byLean Analytics
 
Lean Analytics workshop (from Lean Startup Conf)
byLean Analytics
 
Lean Analytics @ MicroConf
byLean Analytics
 
Innovation accounting and key metrics for startups
byBlaz Kos
 
Lean Analytics for Startups and Enterprises
byLean Analytics
 
Lean analytics
byAndrii Gakhov
 
Digital analytics: Wrap-up (Lecture 12)
byJoni Salminen
 
online startups metrics
byHatem Kameli
 
SBC Growth Week - Lean Analytics
byMeasureWorks
 
Startup analytics
byTopi Järvinen
 
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
byjeroentjepkema
 
Stamats: Analytics Webinar
bycjcunniff
 
Growth Hacking @ Sup de Pub International Track — part 1
byMaxime Pico
 
Lean Analytics - TDC2013
byLuca Bastos
 
ETE 2013: Going Big with Big Data...one step at a time
byAnita Andrews
 
Data strategy
byCarbon Five
 
7 Myths of Lean and How Analytics Can Help
byLean Analytics
 
Lean analytics 40 minute intro
byPinkesh Shah
 

More from Lean Startup Co.

PPTX
Keynote: Intelligent Growth in Startups: Building More Than a Product to Get ...
byLean Startup Co.
 
PDF
A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...
byLean Startup Co.
 
PDF
A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...
byLean Startup Co.
 
PDF
F3: Employee = Founder: Building a Startup inside a Fortune 500 Company, Max-...
byLean Startup Co.
 
PDF
E3: Amazon’s Approach to Culture Change, Wen Huang, Eric Tachibana
byLean Startup Co.
 
PDF
G3: Reflecting on My Failure to Build a Billion Dollar Company, Sahil Lavingia
byLean Startup Co.
 
PDF
Keynote: Innovation is a Habit, not a Mindset, Diana Kander
byLean Startup Co.
 
PPTX
Keynote: The Subscription Economy: How to Survive the End of Ownership, Tien ...
byLean Startup Co.
 
PPTX
G6: Getting Leaders On Board: Don’t Let Your Innovation Program be Pronounced...
byLean Startup Co.
 
PDF
B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...
byLean Startup Co.
 
PPTX
G1: Brackitz Toy Story: Learning From Mistakes and Mental Models for Success,...
byLean Startup Co.
 
PDF
G5: Big Problems Require Big Solutions: the Story of Earn, Leigh Phillips
byLean Startup Co.
 
PPTX
C3: Optimize, Grow or Catapult: Where to Target Your Organization’s Innovatio...
byLean Startup Co.
 
PDF
B4: The Road to Startup Success is Paved in Pivots, Kate Skavish
byLean Startup Co.
 
PPTX
B5: The 8 Deadly Maladies of Rapid Experimentation, Lukas Oberhuber
byLean Startup Co.
 
PDF
A5: Designing Gamified Experiences to Gain Customer Insights, Richardo Chen &...
byLean Startup Co.
 
PDF
Keynote: From Founder to Investor: Stories From the Trenches of Entrepreneurs...
byLean Startup Co.
 
PPTX
B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...
byLean Startup Co.
 
PPTX
D5: Be the Solution: Use Lean to Solve Community Problems, Katrina Medoff
byLean Startup Co.
 
PPTX
F1: Mastering the Art of Telling Your Story, Jamie Lazzeri
byLean Startup Co.
 
Keynote: Intelligent Growth in Startups: Building More Than a Product to Get ...
byLean Startup Co.
 
A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...
byLean Startup Co.
 
A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...
byLean Startup Co.
 
F3: Employee = Founder: Building a Startup inside a Fortune 500 Company, Max-...
byLean Startup Co.
 
E3: Amazon’s Approach to Culture Change, Wen Huang, Eric Tachibana
byLean Startup Co.
 
G3: Reflecting on My Failure to Build a Billion Dollar Company, Sahil Lavingia
byLean Startup Co.
 
Keynote: Innovation is a Habit, not a Mindset, Diana Kander
byLean Startup Co.
 
Keynote: The Subscription Economy: How to Survive the End of Ownership, Tien ...
byLean Startup Co.
 
G6: Getting Leaders On Board: Don’t Let Your Innovation Program be Pronounced...
byLean Startup Co.
 
B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...
byLean Startup Co.
 
G1: Brackitz Toy Story: Learning From Mistakes and Mental Models for Success,...
byLean Startup Co.
 
G5: Big Problems Require Big Solutions: the Story of Earn, Leigh Phillips
byLean Startup Co.
 
C3: Optimize, Grow or Catapult: Where to Target Your Organization’s Innovatio...
byLean Startup Co.
 
B4: The Road to Startup Success is Paved in Pivots, Kate Skavish
byLean Startup Co.
 
B5: The 8 Deadly Maladies of Rapid Experimentation, Lukas Oberhuber
byLean Startup Co.
 
A5: Designing Gamified Experiences to Gain Customer Insights, Richardo Chen &...
byLean Startup Co.
 
Keynote: From Founder to Investor: Stories From the Trenches of Entrepreneurs...
byLean Startup Co.
 
B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...
byLean Startup Co.
 
D5: Be the Solution: Use Lean to Solve Community Problems, Katrina Medoff
byLean Startup Co.
 
F1: Mastering the Art of Telling Your Story, Jamie Lazzeri
byLean Startup Co.
 

Recently uploaded

PDF
When Paychecks Stop: Are We Financially Ready
byAurumCapital
 
PDF
TikTok Seller Accounts in 2025_ Social Commerce, Trust Signals.pdf
bykeb2bq5cegc6m2xa32ay
 
PDF
How to Buy Instagram accounts for professional brand ... (1).pdf
bykeb2bq5cegc6m2xa32ay
 
PDF
How to Buy Snapchat accounts for professional brand ....pdf
bykeb2bq5cegc6m2xa32ay
 
PDF
How to Buy Instagram accounts for professional brand ....pdf
bykeb2bq5cegc6m2xa32ay
 
PDF
How to Buy Twitter Accounts in 2026 .pdf
bywc0fr9qf43i5qo5kn3
 
PDF
LinkedIn Accounts in 2025_ Social Commerce, Trust Signals.pdf
bykeb2bq5cegc6m2xa32ay
 
PDF
How to Buy Facebook accounts for professional brand ....pdf
bykeb2bq5cegc6m2xa32ay
 
PDF
Step-by-Step Guide to Buying LinkedIn Accounts in 2026.pdf
byasikurrhmanxyzit
 
PDF
How to Buy a USA Facebook Accounts Safely.pdf
byyl62ghphl9tyxn3q8lv5
 
PDF
Top 10 Websites to Buy Facebook Accounts Tone.pdf
byidc1w3adizw383go
 
PDF
J Hamilton - Pensions Playpen 16.12.2025 (1).pdf
byHenry Tapper
 
PPTX
DeFi Passive Income: From definitions to practical examples
byAli Mohammad Saghiri
 
PDF
Institute for Public Relations 2025 Year in Review
byMariumAbdulhussein
 
PDF
Best Platforms to Buy Verified Wise Accounts in the USA.pdf
by#SEO, #usaallhub, #socialmedia, #USAaccounts, #seoservice, #Digitalmarketer
 
PDF
How to Buy USA Facebook Accounts in 2026 (1).pdf
bywc0fr9qf43i5qo5kn3
 
PDF
How to Buy USA Facebook Accounts in 2026 .pdf
bywc0fr9qf43i5qo5kn3
 
PPTX
CMMI Consulting & Implementation Services.pptx
bySunil Yadav
 
PDF
Dubai Multi Commodities Centre (DMCC) – Supplier Code of Conduct Policy
byDubai Multi Commodity Centre
 
PDF
ASD SCO Introduction December 2025 (1).pdf
byHenry Tapper
 
When Paychecks Stop: Are We Financially Ready
byAurumCapital
 
TikTok Seller Accounts in 2025_ Social Commerce, Trust Signals.pdf
bykeb2bq5cegc6m2xa32ay
 
How to Buy Instagram accounts for professional brand ... (1).pdf
bykeb2bq5cegc6m2xa32ay
 
How to Buy Snapchat accounts for professional brand ....pdf
bykeb2bq5cegc6m2xa32ay
 
How to Buy Instagram accounts for professional brand ....pdf
bykeb2bq5cegc6m2xa32ay
 
How to Buy Twitter Accounts in 2026 .pdf
bywc0fr9qf43i5qo5kn3
 
LinkedIn Accounts in 2025_ Social Commerce, Trust Signals.pdf
bykeb2bq5cegc6m2xa32ay
 
How to Buy Facebook accounts for professional brand ....pdf
bykeb2bq5cegc6m2xa32ay
 
Step-by-Step Guide to Buying LinkedIn Accounts in 2026.pdf
byasikurrhmanxyzit
 
How to Buy a USA Facebook Accounts Safely.pdf
byyl62ghphl9tyxn3q8lv5
 
Top 10 Websites to Buy Facebook Accounts Tone.pdf
byidc1w3adizw383go
 
J Hamilton - Pensions Playpen 16.12.2025 (1).pdf
byHenry Tapper
 
DeFi Passive Income: From definitions to practical examples
byAli Mohammad Saghiri
 
Institute for Public Relations 2025 Year in Review
byMariumAbdulhussein
 
Best Platforms to Buy Verified Wise Accounts in the USA.pdf
by#SEO, #usaallhub, #socialmedia, #USAaccounts, #seoservice, #Digitalmarketer
 
How to Buy USA Facebook Accounts in 2026 (1).pdf
bywc0fr9qf43i5qo5kn3
 
How to Buy USA Facebook Accounts in 2026 .pdf
bywc0fr9qf43i5qo5kn3
 
CMMI Consulting & Implementation Services.pptx
bySunil Yadav
 
Dubai Multi Commodities Centre (DMCC) – Supplier Code of Conduct Policy
byDubai Multi Commodity Centre
 
ASD SCO Introduction December 2025 (1).pdf
byHenry Tapper
 

Lean Analytics for Intrapreneurs by Allistair Croll

  • 1.
    Lean Analytics for Intrapreneurs LeanStartup Conference 2013 @acroll
  • 2.
    When you’re astartup your goal is to find a sustainable, repeatable business model. When you’re a big company your goal is to perpetuate one.
  • 3.
    Lean Analytics: Use datato build a better business faster.
  • 4.
    Intrapreneur: Someone working toproduce disruptive change in an organization that has already found a sustainable, repeatable business model.
  • 5.
    (Before we getinto Lean Analytics, 2 key lessons.) Lesson one: Companies die because they fail to move to new business models.
  • 6.
    $1 8” 5.25” 3.5” Clay Christensen, TheInnovator’s Dilemma Time eb oo k $1000 N ot op kt De s ic in M $100 ai nf ra om m e pu te r $10 M Cost per MB 14”
  • 7.
    Technologies outstrip what the market needs,driven by feedback from the “best” current customer. $1 8” 5.25” $10 end High er stom cu end Low mer usto c $100 $1000 Clay Christensen, The Innovator’s Dilemma Time
  • 8.
    The new market has differentcriteria for success, which are uninteresting to incumbents. $1 $10 $100 Storage capacity Portability $1000 Clay Christensen, The Innovator’s Dilemma Time
  • 9.
    Sometimes this has unintended consequences $1 Smaller discsize means less vibration impact, leading to greater density, increasing storage capacity $10 $100 $1000 Clay Christensen, The Innovator’s Dilemma Time
  • 10.
    Three kinds ofinnovation Improve along current metrics... Switch to a new value model Change the business model entirely ...or alter the rate of improvement Sustain/core Innovate/adjacent Disrupt/transformative (optimizing for more of the same) (introduce nearby product, market, or method) (Fundamentally changing the business model)
  • 11.
    Lesson two: The differencebetween a rogue agent and a special operative is permission.
  • 12.
    It’s not yourjob to prove you’re clever. It’s your job to change outcomes.
  • 13.
  • 14.
    Kevin Costner isa lousy entrepreneur. Don’t sell what you can make. Make what you can sell.
  • 15.
    The core ofLean is iteration.
  • 16.
  • 17.
    Everyone’s idea is thebest right? No data, no learning. People love this part! (but that’s not always a good thing) This is where things fall apart.
  • 18.
  • 19.
    Analytics is themeasurement of movement towards your business goals.
  • 20.
    In a startup,the purpose of analytics is to iterate to product/market fit before the money runs out.
  • 21.
    In a bigcompany, analytics replaces opinion with fact.
  • 22.
    It helps youinnovate on product, market, and method.
  • 23.
  • 24.
    A good metricis: Understandable Comparative If you’re busy explaining the data, you won’t be busy acting on it. Comparison is context. A ratio or rate The only way to measure change and roll up the tension between two metrics (MPH) Behavior changing If you’re busy explaining the data, you won’t be busy acting on it.
  • 25.
    The simplest rule If a metricwon’t change how you behave, it’s a bad metric. h"p://www.flickr.com/photos/circasassy/7858155676/
  • 26.
    Metrics help youknow yourself. Customers that buy >1x in 90d 1-15% 15-30% >30% Then you are in this mode Your customers will buy from you You are just like 70% Acquisition Once Hybrid 2-2.5 20% per year of retailers Loyalty >2.5 10% per year of retailers of retailers Focus on Low acquisition cost, high checkout Increasing return rates, market share Loyalty, selection, inventory size (Thanks to Kevin Hillstrom for this.)
  • 27.
    Qualitative Quantitative Unstructured, anecdotal, revealing, hardto aggregate, often too positive & reassuring. Numbers and stats. Hard facts, less insight, easier to analyze; often sour and disappointing. Warm and fuzzy. Cold and hard.
  • 28.
    Exploratory Reporting Speculative. Tries tofind unexpected or interesting insights. Source of unfair advantages. Predictable. Keeps you abreast of the normal, day-to-day operations. Can be managed by exception. Cool. Necessary.
  • 29.
    Rumsfeld on Analytics weknow Are facts which may be wrong and should be checked against data. we don’t know Are questions we can answer by reporting, which we should baseline & automate. we know Are intuition which we should quantify and teach to improve effectiveness, efficiency. we don’t know Are exploration which is where unfair advantage and interesting epiphanies live. know Things we don’t know (Or rather, Avinash Kaushik channeling Rumsfeld)
  • 30.
    Slicing and dicingdata Active users 5,000 Cohort: Comparison of similar groups along a timeline. 0 Jan (this is the April cohort) Feb Segment: Cross-sectional comparison of all people divided by some attribute (age, gender, etc.) Mar ☀ ☁ Apr May A/B test: Changing one thing (i.e. color) and measuring the result (i.e. revenue.) ☀ ☁ ☀ ☁ Multivariate analysis Changing several things at once to see which correlates with a result.
  • 31.
    Which of thesetwo companies is doing better?
  • 32.
      March April May Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50 1 2 3 4 5 January How about this one? February Cohort Isthis company growing or stagnating? January $5 $3 $2 $1 $0.5 $6 $4 $2 $1 March $7 $6 $5 April   $8 $7     $9 February May  
  • 33.
    Cohort 2 3 4 5 January Look at the samedata in cohorts 1 $5 $3 $2 $1 $0.5 February $6 $4 $2 $1   March $7 $6 $5     April $8 $7       May $9         Averages $7 $5 $3 $1 $0.5
  • 34.
    The ability todo things in small batches is behind much of the lean movement.
  • 35.
    Short cycle timetriggers the immune system of big company budgeting.
  • 36.
    (this is calledforeshadowing.)
  • 37.
    Lagging Historical. Shows you howyou’re doing; reports the news. Example: sales. Explaining the past. Leading Forward-looking. Number today that predicts tomorrow; reports the news. Example: pipeline. Predicting the future.
  • 38.
    Some examples A Facebookuser reaching 7 friends within 10 days of signing up (Chamath Palihapitiya) If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang) Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) A LinkedIn user getting to X connections in Y days (Elliot Schmukler) (From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
  • 39.
    Which means it’stime to talk about correlation.
  • 40.
    10000 1000 100 10 1 Jan Feb Mar Apr May Ice creamconsumption Jun Jul Aug Sept Drownings Oct Nov Dec
  • 41.
    Correlated Two variables thatare related (but may be dependent on something else.) Ice cream & drowning. Causal An independent variable that directly impacts a dependent one. Summertime & drowning.
  • 42.
    A leading, causalmetric is a superpower. h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
  • 43.
    Growth hacking, demystified. Picka metric to change Find correlation Test causality Optimize the causal factor
  • 44.
  • 45.
  • 46.
  • 47.
    Eric’s three enginesof growth Stickiness Virality Price Approach Keep people coming back. Make people invite friends. Spend money to get customers. Math that matters Get customers faster than you lose them. How many they tell, how fast they tell them. Customers are worth more than they cost.
  • 48.
    Dave’s Pirate Metrics AARRR Acquisition Activation Retention Revenue Referral Howdo your users become aware of you? SEO, SEM, widgets, email, PR, campaigns, blogs ... Do drive-by visitors subscribe, use, etc? Features, design, tone, compensation, affirmation ... Does a one-time user become engaged? Notifications, alerts, reminders, emails, updates... Do you make money from user activity? Transactions, clicks, subscriptions, DLC, analytics... Do users promote your product? Email, widgets, campaigns, likes, RTs, affiliates...
  • 49.
    The five stages Stage EMPATHY STICKINESS Gate I’vefound a real, poorly-met need that a reachable market faces. I’ve figured out how to solve the problem in a way they will keep using and pay for. VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives. REVENUE The users and features fuel growth organically and artificially. SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem.
  • 50.
    Empathy stage: Localmind hacksTwitter Needed to find out if a core assumption—strangers answering questions—was valid. Ran Twitter experiment instead of writing code Asked senders of geolocated Tweets from Times Square random questions; counted response rate Conclusion: high enough to proceed
  • 51.
    Stickiness stage: qidiq streamlinesinvites Survey owner adds recipient to group Survey owner adds recipient to group Survey owner asks question Recipient gets invite Recipient installs mobile app Recipient creates account, profile Recipient can edit profile, etc. Recipient reads survey question Recipient responds to question Recipient sees survey results 70-90% RESPONSE RATE 10-25% RESPONSE RATE Survey owner asks question Recipient reads survey question Recipient responds to question Recipient sees survey results (Later, if needed…) Recipient visits site; no password! Recipient does password recovery One-time link sent to email Recipient creates password Recipient can edit profile, etc.
  • 52.
    Six business modelarchetypes (Yours is probably a blend of these.)
  • 53.
    E-commerce SaaS (freemium?) Mobile app(gaming) Two sided marketplace Media User generated content
  • 54.
    Customer Acquisition Cost paid direct search wom (Whichmeans eye charts like these.) Viral coefficient Viral rate inherent virality VISITOR Freemium/trial offer Invite Others Upselling rate Upselling Enrollment Capacity Limit User Disengaged User Free user disengagement Paid conversion Engaged User Tiering Paying Customer Support data Reactivation rate Freemium churn Reactivate Trial Over Disengaged Dissatisfied Trial abandonment rate Resolution Cancel Cancel Reactivate Account Cancelled Billing Info Exp. Paid Churn Rate FORMER USERS User Lifetime Value FORMER CUSTOMERS Customer Lifetime Value
  • 55.
    Model + Stage= One Metric That Matters. The business you’re in The stage you’re at E-Com SaaS Mobile 2-Sided Media Empathy Stickiness Virality Revenue Scale One Metric That Matters. UCG
  • 56.
  • 57.
  • 58.
    In a startup,focus is hard to achieve.
  • 59.
    Having only onemetric addresses this problem.
  • 60.
  • 61.
    Moz cuts downon metrics SaaS-based SEO toolkit in the scale stage. Focused on net adds. Net adds up: Net adds flat: Net adds down: Was a marketing campaign successful? Were customer complaints lowered? Was a product upgrade valuable? Can we acquire more valuable customers? What product features can increase engagement? Can we improve customer support? Are the new customers not the right segment? Did a marketing campaign fail? Did a product upgrade fail somehow? Is customer support falling apart?
  • 62.
    Metrics are likesqueeze toys. http://www.flickr.com/photos/connortarter/4791605202/
  • 63.
    Ecommerce Empathy Stickiness Virality 2-sided market Scale Mobile app User-gen content Media Interviews; qualitative results;quantitative scoring; surveys Loyalty, conversion Inventory, listings CAC, shares, SEM, sharing reactivation (Money from transactions) Revenue SaaS Engagement, Downloads, churn churn, virality Content, spam Traffic, visits, returns Inherent virality, CAC Invites, sharing Content virality, SEM WoM, app ratings, CAC (Money from active users) Transaction, CLV Transactions, commission Upselling, CAC, CLV CLV, ARPDAU Affiliates, white-label Other verticals API, magic #, Spinoffs, mktplace publishers (Money from ad clicks) Ads, donations CPE, affiliate %, eyeballs Analytics, user data Syndication, licenses
  • 64.
  • 65.
    A company losesa quarter of its customers every year. Is this good or bad?
  • 66.
    Not knowing whatnormal is makes you do stupid things.
  • 67.
    Listening to whatnormal is makes you ignore disruptive things.
  • 68.
  • 69.
    Baseline: 5-7% growth aweek • Are there enough people who really care enough to sustain a 5% growth rate? • Don’t strive for a 5% growth at the expense of really understanding your customers and building a meaningful solution • Once you’re a pre-revenue startup at or near product/market fit, you should have 5% growth of active users each week • Once you’re generating revenues, they should grow at 5% a week “A good growth rate during YC is 5-7% a week,” he says. “If you can hit 10% a week you're doing exceptionally well. If you can only manage 1%, it's a sign you haven't yet figured out what you're doing.” At revenue stage, measure growth in revenue. Before that, measure growth in active users. Paul Graham, Y Combinator
  • 70.
    Baseline: 10% visitor engagement/day 30% ofusers/month use web or mobile app 10% of users/day use web or mobile app 1% of users/day use it concurrently Fred Wilson’s social ratios
  • 71.
    Baseline: 2-5% monthly churn •The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s on the job. • Get below a 5% monthly churn rate before you know you’ve got a business that’s ready to grow (Mark MacLeod) and around 2% before you really step on the gas (David Skok) • Last-ditch appeals and reactivation can have a big impact. Facebook’s “don’t leave” reduces attrition by 7%.
  • 72.
    Baseline: CLV calculation 25% 5% 2% monthly churn monthlychurn monthly churn 100/25=4 100/5=20 100/2=50 The average customer lasts 4 months The average customer lasts 20 months The average customer lasts 50 months
  • 73.
    Baseline: CAC under 1/3of CLV • CLV is wrong. CAC Is probably wrong, too. • Time kills all plans: It’ll take a long time to find out whether your churn and revenue projections are right • Cashflow: You’re basically “loaning” the customer money between acquisition and CLV. • It keeps you honest: Limiting yourself to a CAC of only a third of your CLV will forces you to verify costs sooner. 20 mo. CL $30/month per customer $600 CLV 1/3 spend $200 CAC Now segment those users!
  • 74.
    Putting this towork: The Lean Analytics Cycle
  • 75.
    Pick a KPI Success! Pivotor give up Draw a new line Try again Did we move the needle? Measure the results Draw a line Find a potential improvement Without data: make a good guess With data: find a commonality Design a test Hypothesis Make changes in production
  • 76.
    Do AirBnB hosts getmore business if their property is professionally photographed?
  • 77.
    Gut instinct (hypothesis) Professionalphotography helps AirBnB’s business Candidate solution (MVP) 20 field photographers posing as employees Measure the results Compare photographed listings to a control group Make a decision Launch photography as a new feature for all hosts
  • 78.
    5,000 shoots permonth by February 2012
  • 79.
    Hang on asecond.
  • 80.
    SRSLY? Gut instinct (hypothesis) Professionalphotography helps AirBnB’s business
  • 81.
    Pick a KPI Success! Pivotor give up Draw a new line Try again Did we move the needle? Measure the results Draw a line Find a potential improvement Without data: make a good guess With data: find a commonality Design a test Hypothesis Make changes in production
  • 82.
    Without data: makea good guess With data: find a commonality “Gee, those houses that do well look really nice.” “Computer: What do all the highly rented houses have in common?” Maybe it’s the camera. Camera model.
  • 83.
    Landing page design A/Btesting URL shortening Publisher analytics Cohort analysis General analytics Funnel analytics SaaS analytics User analytics Spying on users Influencer Marketing Gaming analytics User segmentation User interaction Customer satisfaction KPI dashboards
  • 84.
    Agenda An introduction to LeanAnalytics (30m) The challenges of being big (15m) When you have support (30m) The Lean Analytics framework (30m) A dose of pragmatism (15m) Metrics for innovation portfolios (15m) Break (15m) Some non-tech examples (10m) Tools of the trade (15m) When it’s you against the world (20m) Break (15m)
  • 85.
  • 86.
    It is waytoo easy to mix these up. Target market B2B B2C Early stage Big/incumbent Company size/age Less WoM More formal decisions Intrapreneurs Business model vs. company stage Slower cycle time More legacy constraints
  • 87.
    If big firms can’tinnovate, it’s this guy’s fault.
  • 88.
    When product andmarket are known, companies compete on how they do things.
  • 89.
    To get theincremental cost to zero, companies competed on scale. (Literally, an economy of scale)
  • 90.
    Scale comes fromprocess, IP, org chart, capitalization. All of these assume the future will be like the past, only more so.
  • 91.
    If a startupis an organization designed to search for a sustainable, repeatable business model, then an established company is an organization designed to perpetuate one.
  • 92.
    Technology has radicallychanged the incremental cost of businesses.
  • 93.
    Software is eatingthe world. http://www.flickr.com/photos/ebolasmallpox/3733059220/
  • 94.
    An economic orderquantity of one. Crafted Massproduced Automated Digital Quantity Few Many Some One Cost High Low Medium Free Lead time Small Large Medium None Self-service Medium None Some Lots Customization High None Some Lots This is why software is eating the world. • Cloud computing • Social media • 3D printing • Per-customer analysis • Mobile tracking • Etc...
  • 95.
    Sustainable competitive advantageallows for inertia and power to build up along the lines of an existing business model, which will soon die. Instead, seek transient competitive advantage. Rita Gunther McGrath, The End of Competitive Advantage
  • 96.
    Scale is nowa liability. Compete on cycle time.
  • 97.
    Optimizing the probablemeans discounting the possible.
  • 98.
    This isn’t abouta lack of resources.
  • 99.
  • 100.
    Blockbuster had alot going for it.
  • 101.
    Plenty of inventory,of course. But that matters less than...
  • 102.
  • 103.
    The problem wasframing: Blockbuster thought it was in the video store management business. Netflix realized it was in the entertainment delivery business.
  • 104.
  • 105.
  • 106.
  • 107.
  • 108.
    First mover advantagehappens long before the market emerges. • $1B invested in Nook • $475M operating loss in April 2013 • CEO gone
  • 109.
  • 110.
    Everything to lose: Whybig companies need innovation.
  • 112.
    F500 Life Expectancy (http://csinvesting.org/2012/01/06/fortune-500-extinction/) 75 15 years years 1950 Growth byentering a new business ... 2010 95 99 % fail % fail Corporate Strategy Board Clay Christensen
  • 113.
    In other words,if your job is change you have your work cut out for you.
  • 114.
    2011 MIT studyof 179 large publicly traded firms Companies that use data-driven analytics instead of intuition have 5%-6% higher productivity and profits than competitors. Brynjolfsson, Erik, Lorin Hitt, and Heekyung Kim. "Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance?." Available at SSRN 1819486 (2011).
  • 115.
    A dose ofpragmatism
  • 116.
    Many models forenterprise innovation Core Adjacent Transformative Do the same thing better. Nearby product, market, or method. Start something entirely new. Regional optimizations. Innovation, go-tomarket strategies. Reinvent the business model. • Customer development • Test similar cases • Parallel deployment • Analytics & cycle time • Fail fast • Skunkworks/R&D • Focus on the search • Ignore the current model & margins • Get there faster • Smaller batches • Solution, then testing • Increased accountability
  • 117.
    Another way tolook at it Core Adjacent Transformative Know the problem (customers tell you it) Know the solution (customers/regulations/ norms dictate it.) Know the problem (market analysis) Don’t know the solution (non-obvious innovation confers competitive advantage.) Don’t know the problem (just an emerging need/ change) Don’t know the solution. Waterfall: Execution matters Agile/scrum: Iteration matters Lean Startup: Discovery matters
  • 118.
    The Three Horizons Core Adjacent Horizon1 improves the Horizon 2 extends the current business operations business into new products, in the next 12 months. markets, or methods in the next 3 years. Those core businesses most readily identified with the company name and those that provide the greatest profits and cash flow. Maximize remaining value. Emerging opportunities, including rising entrepreneurial ventures likely to generate substantial profits in the future but that could require considerable investment. Transformative Horizon 3 changes the industry you’re in and your value network in the next 6 years. Ideas for profitable growth down the road—for instance, small ventures such as research projects, pilot programs, or minority stakes in new businesses. http://www.mckinsey.com/insights/strategy/enduring_ideas_the_three_horizons_of_growth
  • 119.
    Lean applies. Startupmay not. Core Adjacent Transformative Lean startup Lean methodologies.
  • 120.
    Method (new “how”) 3 kindsof innovation Product (new “what”) Market (new “who”)
  • 121.
  • 122.
    A three-maxima model ofenterprise innovation Business optimization (five mores) Current state Business model innovation Product, market, method innovation You can convince executives of this because some of it is familiar. This terrifies them because it eats the current business.
  • 123.
    Improvement Adjacency Remodeling Do the same, onlybetter. Explore what’s nearby quickly Try out new business models Lean approaches apply, but the metrics vary widely. Sustain/ core Innovate/ adjacent Disrupt/ transformative
  • 124.
    More things To morepeople Sustaining innovation is about more of the same. (says Sergio Zyman) Inventory increase Gifting, wish lists Highly viral offering Low incremental order costs For more money Maximum shopping cart Price skimming/tiering More often Loyal customer base that returns Demand prediction, notification More efficiently Supply chain optimization Per-transaction cost reduction
  • 125.
    Software, experimentation, and iterativecycles of learning help you get to the local maximum better and faster. That’s a good thing. But it’s not the only thing.
  • 126.
    Adjacent innovation isabout changing one part of the model in a way that alters the value network.
  • 127.
    Adjacent product to thesame market in the same way
  • 128.
    Selling the sameproduct to an adjacent market in the same way. Of P&G’s 38 brands, only 19 were sold in Asia as of 2011 Market expansion is seldom selling the same thing to new people. In Asia, P&G needed to Align pricing with novelty (prestige, mass-tige, over-the-counter) Change consumer expectations (moving from dilutes to concentrates) Adjust positioning and ingredients such as white fungus, ginseng, and the parasitic cordyceps
  • 129.
    Selling the sameproduct to the same in The biggest innovation logistics of the 20th century. market in a new way. http://www.flickr.com/photos/photohome_uk/1494590209
  • 130.
    Selling the sameproduct to the same market in a new way.
  • 131.
    (At this point,observant Intrapreneurs should be asking, should P&G be in the house cleaning business? And that would be transformative.)
  • 132.
    Transformative innovation isabout taking a leap, changing more than one dimension simultaneously in search of a new business model.
  • 133.
    If sustaining, incrementalinnovation produces linear growth, then disruptive, transformative innovation produces exponential growth.
  • 134.
  • 135.
  • 136.
    I lied. Everyoneis a tech company.
  • 137.
    Cost of experiments:down. http://www.flickr.com/photos/puuikibeach/4789015423 Cost of attention: way up. http://www.flickr.com/photos/elcapitanbsc/3936927326
  • 138.
    Let’s pick onrestaurants for a while.
  • 139.
    Empathy: find theneed Before opening, the owner first learns about the diners in her area, their desires, what foods aren’t available, and trends in eating. Key metrics: Popular items; frequent questions; before/after dining patterns. Reference: Emerging need.
  • 140.
    Stickiness: confirm theneed is met. She develops a menu and tests it out with consumers, making frequent adjustments until tables are full and patrons return regularly. She’s giving things away, asking diners what they think. Variance and uncertain inventory make costs high. Key metrics: Customer loyalty; recommendations; referrals; endorsements; inventory turnover. Reference: Business idea.
  • 141.
    Virality: will itspread? She starts loyalty programs to bring frequent diners back, or to encourage people to share with their friends. She engages on Yelp and Foursquare. Key metrics: Customer loyalty; recommendations; referrals; endorsements. Reference: Business positioning
  • 142.
    Revenue: prove thebusiness viability With virality kicked off, she works on margins—fewer free meals, tighter controls on costs, more standardization. She focuses on the price of acquiring new customers. Key metrics: Acquisition cost, revenue per cover, capacity, turnover. Reference: Business model.
  • 143.
    Scale: prove it’sa market Knowing she can run a profitable business, she funnels revenues into marketing and promotion. She reaches out to food reviewers, travel magazines, and radio stations. She launches a second restaurant, or a franchise. Key metrics: Franchise health; repeatability; problems escalated; variance; franchise revenues. Reference: Business plan.
  • 144.
    A line inthe sand 30% Labor costs Gross revenue = 24% 20% Too costly? Just right Understaffed?
  • 145.
    A leading indicator (Variesby restaurant. McDonalds ≠ Fat Duck.) 50 reservations at 5PM http://www.flickr.com/photos/mysticcountry/3567440970 250 covers that night http://www.flickr.com/photos/avlxyz/4889656453
  • 146.
  • 147.
    Is tip amounta leading indicator of longterm revenue?
  • 148.
    Why does everytable get the same menu?
  • 149.
    Is purple inkbetter? http://tippingresearch.com/uploads/managing_tips.pdf
  • 151.
  • 152.
    When it’s youvs. the world. (A bagful of tricks from agitators in companies of all sizes.)
  • 153.
    The skills youneed make you a pariah. Successful innovators share certain attributes. Bad listener: Willfully ignore feedback from your best customers. Cannibal: If successful, destroying existing revenue streams. Job killer: Automation & lower margins are your favorite tools. Security risk: Advocate of transparency, open data, communities. Narcissist: Worry constantly about how you’ll get attention. Slum lord: Sell to those with less money, deviants, and weirdos.
  • 154.
    New Know what kind of innovation you’re Market after. Current Basedon H. Igor Ansoff’s matrix Startup: New products for new !) markets. News rules, es cc units, business su at organizational re g nd structure. Innovation. (a Market development: Sell existing products to new markets, segments, uses. Export & license. lf a ll a Penetrate: itic ol Increase revenues, fp o market share, product sk ri ed quality, brand as re differentiation. nc I Marketing. Current ut o Product development: Invent new products for your market. R&D, enhancements. Acquisition. Product New
  • 155.
    Frame it likea study Product creation is almost accidental. Unlike a VC or startup, when the initiative fails the organization still learns. http://www.flickr.com/photos/creative_tools/8544475139
  • 156.
    When in doubt,collect data From tackling the FTA rate to visualizing the criminal justice supply chain.
  • 157.
    Use data tocreate a taste for data Sitting on Billions of rows of transactional data David Boyle ran 1M online surveys Once the value was obvious to management, got license to dig.
  • 158.
    Don’t just collectdata, chase it.
  • 159.
    Understand hidden constraints That pencilstory is a myth. Graphite is conductive and explosive. The Minimum Viable Product is Viable for a reason.
  • 160.
    Know what hasto be built in-house SAP integration Employment law
  • 161.
  • 162.
  • 163.
  • 164.
    Run it asa consulting business first. (Just don’t get addicted to it. Your goal is to learn and overcome integration challenges and find the 20% of features that 80% of the market will pay for.)
  • 165.
    Convince your bossshe asked for this Success! Pick a KPI Pivot or give up Draw a new line Try again Did we move the needle? Measure the results Draw a line in the sand Find a potential improvement Without data: make a good guess With data: find a commonality Design a test Make changes in production Hypothesis
  • 166.
    Slaughter a sacredcow: Prove a long-held assumption is wrong and you’ve got people’s attention. Know what you’ll do with it ahead of time.
  • 167.
  • 168.
  • 169.
  • 170.
    Twitter’s 140-character limit isn’tarbitrary. It’s constrained by the size of SMS (160 characters) and username (20 characters.) http://i.i.cbsi.com/cnwk.1d/i/tim/2011/11/18/ sms_screen_twitter_activity_stream_270x405.png
  • 171.
    Figure out howto translate it back to a simple model that fits the company’s existing value model. If your company dies, this is why.
  • 172.
    Intrapreneurs often haveto use proxies Stage Startup metrics Intrapreneur metrics Empathy Customers interviewed (needs & solutions), assumptions quantified, TAM, monetization possibility Non-customers interviewed; assumptions quantified, constraints identified, TAM, disruption potential Churn, engagement Support tickets, integration time, call center data, delays Viral coefficient, viral cycle time Net Promoter Score, referrals, case study willingness Revenue Attention, engagement Billable activity; signed LOIs; pilot programs; after-development profitability Scale Automation Contribution, training costs, licensing Stickiness Virality
  • 173.
    When you havesupport. (What companies like P&G, Cognizant, GE, and Motorola do with a formal innovation program.)
  • 174.
    Do you reallyhave permission? What resources do you have? Staff, budget, unfettered access to customers? What scope of change can you make? Pricing, product, channel, branding?
  • 175.
    Step one: Developa portfolio approach.
  • 176.
    Innovation portfolios atbig companies Return Investment Core Adjacent 70% 20% 10% 20% Transformative 10% 70%
  • 177.
    1. Organizations’ structures emergeas a way to optimize the current business model. 2. Most innovations will come not from product or market, but from method— business model innovation. 3. Innovation groups must exercise organizational amnesia at the outset.
  • 178.
    Step one: Frameyour problems.
  • 181.
  • 182.
    Step two: Defineyour gates and filters. These may lead to myopia. They are also your unfair advantages.
  • 183.
    Step three: Securefunding, resources, and executive backing.
  • 184.
  • 185.
    Find non-obvious adjacencies POWERGRID WHICH FEEDS A NEEDS LIGHT AN BULB ELECTRICAL GENERATOR HAS A TURBINE LIKE A TURNED AROUND BECOMES A PLANE ENGINE TRAIN ENGINE REQUIRES SPINS & VIBRATES LIKE AN AND LOOKS LIKE A SOFTWARE TO CUT DOWN TREES BETTER MRI MACHINE WIND TURBINE
  • 186.
    Build an ecosystem Canada’slargest directory publishing and local marketing services company 1.5M listings from 420K SMB & national customers Revenues >$1.2B 2,500 employees Created third-party listing API Took 8-10 mo (2009-10) to get approval API payoff happened 2y later KPI evolution Yahoo replaced Canadian digital properties search with the YellowAPI Soft: Signups, SDK, downloads Improved SEO, Comscore App usage, deals signed Functional prototype in hours, testing in days, and launching in weeks. Faster time to partnerships Budgets tripled in 2013 API calls generated API-generated revenue
  • 187.
    Five common modelsfor transformative innovation Acquisition Collaboration Isolation Buy promising startups Crowdsource, work with universities, suppliers, etc. Create a separate group with different conditions Incubation Internal startup ecosystem; LoB are “investors” Integration The LoB does innovation internally
  • 188.
    Step five: Testby doing (experimentation beats projection.)
  • 189.
    Focus on the model, notthe plan Demand Amt Growth Wk 1 Wk 2 Wk 3 People per day on sidewalk 200 4% 204 Percent that buy a glass 10% 5% 15% 20% 25% Daily customers 20 31 Revenue Price per cup Profit per cup Daily profit 43 225 56 $156 $216 $281 $5 $5 Cost of Goods Cost per cup 216 $5 $5 30.6 41.5 52.8 $1 -2% .98 .96 .94 $4 4.02 4.04 4.06 $80 125 175 228
  • 190.
    A business planis just what happens when you drag the business model to the right.
  • 191.
    Designing an experiment Problem,solution, and result hypothesis Test strategy (PoC, survey, interviews, kickstarter, prototype, A/B, etc.) Cohort & segment to be tested Metric or assumption being tested Timebox or total for test Action you’ll take if you pass or fail
  • 192.
    Step six: Knowwhat happens afterwards
  • 193.
    Qualcomm’s innovation model: Whatwas missing Hypothesis Unclear what happened to founders Needed a middle PoC decision Sustainability, not feasibility http://blogs.berkeley.edu/2013/01/28/ designing-a-corporateentrepreneurship-program-aqualcomm-case-study-part-1-of-2/ Experiment Boot camp Idea generation and selection Existing models Implement Idea advancement POC Biz sustainability Tech feasibility POC Ideas New models Open innovation Strategic value decision Boot camp decision End user/partner desirability Option value Implement decision Actions Exit value Company crowd storm Small team designs & Company extracts value conducts experiments
  • 194.
  • 195.
    The Lean Analyticslifecycle of an Intrapreneur Get buy-in Political fallout Find problems; don’t test demand. Skip the business case, do analytics Entitled, aggrieved customers Stickiness Know your real minimum based on expectations, regulations Hidden “must haves”, feature creep Virality Build inherent virality in from the start; attention is the new currency Luddites who don’t understand sharing Revenue Consider the ecosystem, channels, and established agreements Channel conflict, resistance, contracts Hand the baton to others gracefully Hating what happens to your baby Beforehand Empathy Scale
  • 196.
  • 197.
    Core metrics Business plan. Metricsthat matter Assume it will improve. • Return on investment • Total cost of ownership • Improvement in KPI • Total served market Product, market, and method will remain the same Examples: Redesigning packaging; pricing adjustment
  • 198.
    Adjacent metrics Business model. Metricsthat matter Assume it will fail. • Virality & word of mouth • Early adopter stickiness • Regulation • Total addressable market Your ultimate use case won’t be what you think it is today. Example: Mr. Clean Magic Eraser
  • 199.
    Transformative metrics Business idea. Assumeit will fail. You hope it will have the consequences you want but aren’t sure how. Example: Headcam recordings of all officers Metrics that matter • People I’ve talked to • Prototype creation speed • Assumptions validated • Problems uncovered • Technical feasibility • Hidden constraints
  • 200.
    Key points toclarify in an innovation program Hypothesis Experimentation • Articulate problems • Frame known advantages • Define the right filters • Many idea sources • Confirm funding (money, • Prioritize riskiest • • • people, customer access) Agree on analytical framework Balance market, product, & method adjacencies • • assumptions Time-box assessment stages Test technology, demand, and business feasibility MVP, prototype, pilot, or science as appropriate for type of innovation Implementation • Temporary incubator • Find a home or building • • • one Keep innovators involved Merge metrics with existing business KPIs Synchronize innovation cycles with enterprise cycles (budget, etc.) Portfolio metrics; Gates and KPIs for each stage; mix of core, adjacent, and disruptive innovation.
  • 201.
    Hypothesis Goals, constraints, context Sourcing Top-down Bottom-up Sourcing Filtering Biases Strengths Focus Mandate Outside-in oste l alm as th wil s w this . thi at re a o . e th ange. here a d t ot ch N y dt nee n inl eck a ions I erta P d c rat it. MV of alte e to ak nch m bu Boot camp POC Core Optimization Crosspollinate to current managers Adjacent De-risking Best solution Test/ validate w/current customers Disruptive Reframing R&D M&A Grow as a distinct business Evaluation of the innovation program itself Implementation Socializing Integration Adoption by existing line of business. Independence Creation of a new line of business.
  • 202.
  • 203.
    Traction graphs Your businessmodel The stage you’re at ... change often if you’re doing it right. Your one metric So how do you track that over time?
  • 204.
    Traction graphs Jan Feb Signups per day Thisaxis changes for each metric Mar Conversion rate Apr Churn rate May Jun Viral coefficient
  • 205.
  • 206.
    Use vanity toget to meaningful metrics Your goal is to produce outcomes If the outcomes require action, and vanity motivates actors, use it But show how the vanity metric is a leading indicator of the real one Web traffic Activation Cart Size Conversion x rate Revenue
  • 207.
    The three threes Three assumptions Whatbig bets are you making? • “People will answer questions” • “Organizers are frustrated with how to run conferences” • “We'll make money from parents” • “Amazon is reliable enough for our users.” Three actions to take What are you doing to make these assumptions happen (or identify they’re wrong and change course?) • Product enhancements • Marketing strategies Three experiments to run • Feature tests • Continuous deployment • A/B testing • Customer survey
  • 208.
    The three threes Three assumptions Monthly Board,investors, founders Strategy Three actions to take Weekly Executive team Tactics Three experiments to run Daily Employees Execution
  • 209.
    The three threes Manypeople will answer questions Three assumptions Three actions to take Three experiments to run Get more people Change the UI Increase answer % Test timings Test better questions Questions from peers
  • 210.
    The problem-solution canvas TheGoal is to Learn CURRENT STATUS • List key metrics you’re LAST WEEK’S LESSONS LEARNED AND ACCOMPLISHMENTS) • What did you learn last week? tracking, where they’re at, and • What was accomplished? compare with last few weeks • On track: YES / NO? How are things trending? •
  • 211.
    The problem-solution canvas Problem#1 (put name here) HYPOTHESIZED SOLUTIONS • List possible solutions that you’ll start working on next week. Rank them. • Why do you believe each solution will help you solve or complete solve the problem? METRICS / PROOF + GOALS • Metrics you’ll use to measure whether or not the solutions are doing what you hoped (solving the problem) • List proof (qualitative) you’ll use as well • Define goals for the metric Problem #2 (put name here) HYPOTHESIZED SOLUTIONS • List possible solutions that you’ll start working on next week. Rank them. METRICS / PROOF + GOALS • Metrics you’ll use to measure whether or not the solutions are doing what you hoped (solving the problem)
  • 212.
    Use proxy data.Just be careful.
  • 213.
    “A subjective degreeof belief should rationally change to account for evidence.” (AKA Bayes’ Theorem.)
  • 215.
  • 216.
    Key points Intrapreneurship isabout adjacent or transformative innovation Sustaining innovation focuses on the Five Mores, within the current product, market, method, and business model. Adjacent innovation may come from a new product, market, or method, but the same business model Disruptive innovation has different customers, KPIs, and models The difference between a rogue agent and a special operative is permission Portfolios need framing, sourcing, filters, metrics, and socializing Balancing isolation and integration, R&D and M&A is contentious
  • 217.
    “The most importantfigures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them.” Lloyd S. Nelson
  • 218.
    Pic by Twodollaon Flickr. http://www.flickr.com/photos/twodolla/3168857844
  • 219.
  • 220.
    Once, a leaderconvinced others in the absence of data.
  • 221.
    Now, a leaderknows what questions to ask.
  • 222.