LA
Uploaded byLean Analytics
PDF, PPTX61,562 views

Lean Analytics workshop for Dublin City University, April 2014

The document discusses lean analytics principles focused on measuring startup performance towards achieving product/market fit efficiently through proper use of metrics and analytics. It highlights the significance of understanding customer behavior, applying various analytics techniques such as cohort analysis and A/B testing, and emphasizing growth stages in a startup. The content further illustrates key performance indicators that startups should focus on to drive growth, maintain customer engagement, and improve business metrics.

Embed presentation

Download as PDF, PPTX
1 / 141
2 / 141
3 / 141
4 / 141
5 / 141
6 / 141
7 / 141
8 / 141
9 / 141
10 / 141
11 / 141
12 / 141
13 / 141
14 / 141
15 / 141
16 / 141
Most read
17 / 141
18 / 141
Most read
19 / 141
20 / 141
21 / 141
22 / 141
Most read
23 / 141
24 / 141
25 / 141
26 / 141
27 / 141
28 / 141
29 / 141
30 / 141
31 / 141
32 / 141
33 / 141
34 / 141
35 / 141
36 / 141
37 / 141
38 / 141
39 / 141
40 / 141
41 / 141
42 / 141
43 / 141
44 / 141
45 / 141
46 / 141
47 / 141
48 / 141
49 / 141
50 / 141
51 / 141
52 / 141
53 / 141
54 / 141
55 / 141
56 / 141
57 / 141
58 / 141
59 / 141
60 / 141
61 / 141
62 / 141
63 / 141
64 / 141
65 / 141
66 / 141
67 / 141
68 / 141
69 / 141
70 / 141
71 / 141
72 / 141
73 / 141
74 / 141
75 / 141
76 / 141
77 / 141
78 / 141
79 / 141
80 / 141
81 / 141
82 / 141
83 / 141
84 / 141
85 / 141
86 / 141
87 / 141
88 / 141
89 / 141
90 / 141
91 / 141
92 / 141
93 / 141
94 / 141
95 / 141
96 / 141
97 / 141
98 / 141
99 / 141
100 / 141
101 / 141
102 / 141
103 / 141
104 / 141
105 / 141
106 / 141
107 / 141
108 / 141
109 / 141
110 / 141
111 / 141
112 / 141
113 / 141
114 / 141
115 / 141
116 / 141
117 / 141
118 / 141
119 / 141
120 / 141
121 / 141
122 / 141
123 / 141
124 / 141
125 / 141
126 / 141
127 / 141
128 / 141
129 / 141
130 / 141
131 / 141
132 / 141
133 / 141
134 / 141
135 / 141
136 / 141
137 / 141
138 / 141
139 / 141
140 / 141
141 / 141
Lean Analytics DCU April, 2014 @acroll
Some housekeeping.
Don’t sell what you can make. Make what you can sell. Kevin Costner is a lousy entrepreneur.
The core of Lean is iteration.
Most startups don’t know what they’ll be when they grow up. Hotmail was a database company Flickr was going to be an MMO Twitter was a podcasting company Autodesk made desktop automation Paypal first built for Palmpilots Freshbooks was invoicing for a web design firm Wikipedia was to be written by experts only Mitel was a lawnmower company
Unfortunately, we’re all liars.
Everyone’s idea is the best right? People love this part! (but that’s not always a good thing) This is where things fall apart. No data, no learning.
Analytics can help.
Analytics is the measurement of movement towards your business goals.
In a startup, the purpose of analytics is to iterate to product/market fit before the money runs out.
I have two kids. At least one of them is a girl.
What are the chances the other is a boy?
BB BG GB GG
2 of 3 (66%) are boys. GB GG BG
Some fundamentals.
A good metric is: Understandable If you’re busy explaining the data, you won’t be busy acting on it. Comparative Comparison is context. A ratio or rate The only way to measure change and roll up the tension between two metrics (MPH) Behavior changing If you’re busy explaining the data, you won’t be busy acting on it.
The simplest rule bad metric. If a metric won’t change how you behave, it’s a h"p://www.flickr.com/photos/circasassy/7858155676/
Metrics help you know yourself. Acquisition Hybrid Loyalty 70% of retailers 20% of retailers 10% of retailers You are just like Customers that buy >1x in 90d Once 2-2.5 per year >2.5 per year Your customers will buy from you Then you are in this mode 1-15% 15-30% >30% Low acquisition cost, high checkout Increasing return rates, market share Loyalty, selection, inventory size Focus on (Thanks to Kevin Hillstrom for this.)
Qualitative Unstructured, anecdotal, revealing, hard to aggregate, often too positive & reassuring. Warm and fuzzy. Quantitative Numbers and stats. Hard facts, less insight, easier to analyze; often sour and disappointing. Cold and hard.
Exploratory Speculative. Tries to find unexpected or interesting insights. Source of unfair advantages. Cool. Reporting Predictable. Keeps you abreast of the normal, day-to-day operations. Can be managed by exception. Necessary.
Rumsfeld on Analytics (Or rather, Avinash Kaushik channeling Rumsfeld) Things we know don’t know we know Are facts which may be wrong and should be checked against data. we don’t know Are questions we can answer by reporting, which we should baseline & automate. we know Are intuition which we should quantify and teach to improve effectiveness, efficiency. we don’t know Are exploration which is where unfair advantage and interesting epiphanies live.
MayAprMarFeb Slicing and dicing data Jan 0 5,000 Activeusers Cohort: Comparison of similar groups along a timeline. (this is the April cohort) A/B test: Changing one thing (i.e. color) and measuring the result (i.e. revenue.) Multivariate analysis Changing several things at once to see which correlates with a result. ☀ ☁ ☀ ☁ Segment: Cross-sectional comparison of all people divided by some attribute (age, gender, etc.) ☀ ☁
Which of these two companies is doing better?
  January February March April May Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50 Is this company growing or stagnating? Cohort 1 2 3 4 5 January February March April May $5 $3 $2 $1 $0.5 $6 $4 $2 $1 $7 $6 $5   $8 $7       $9 How about this one?
Cohort 1 2 3 4 5 January February March April May Averages $5 $3 $2 $1 $0.5 $6 $4 $2 $1   $7 $6 $5     $8 $7       $9         $7 $5 $3 $1 $0.5 Look at the same data in cohorts
Lagging Historical. Shows you how you’re doing; reports the news. Example: sales. Explaining the past. Leading Forward-looking. Number today that predicts tomorrow; reports the news. Example: pipeline. Predicting the future.
A Facebook user reaching 7 friends within 10 days of signing up (Chamath Palihapitiya) If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang) Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) A LinkedIn user getting to X connections in Y days (Elliot Schmukler) Some examples (From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
Which means it’s time to talk about correlation.
1 10 100 1000 10000 Ice cream consumption Drownings Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
Correlated Two variables that are related (but may be dependent on something else.) Ice cream & drowning. Causal An independent variable that directly impacts a dependent one. Summertime & drowning.
A leading, causal metric is a superpower. h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
Growth hacking, demystified. Find correlation Test causality Optimize the causal factor Pick a metric to change
Is social action a leading indicator of donation? http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/
Is mobile use? http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/
Why is Nigerian spam so badly written?
Aunshul Rege of Rutgers University, USA in 2009 Experienced scammers expect a “strike rate” of 1 or 2 replies per 1,000 messages emailed; they expect to land 2 or 3 “Mugu” (fools) each week. One scammer boasted “When you get a reply it’s 70% sure you’ll get the money” “By sending an email that repels all but the most gullible,” says [Microsoft Researcher Corman] Herley, “the scammer gets the most promising marks to self-select, and tilts the true to false positive ratio in his favor.” 1000 emails 1-2 responses 1 fool and their money, parted. Bad language (0.1% conversion) Gullible (70% conversion) 1000 emails 100 responses 1 fool and their money, parted. Good language (10% conversion) Not-gullible (.07% conversion) This would be horribly inefficient since humans are involved.
Turns out the word “Nigeria” is the best way to identify promising prospects.
Nigerian spammers really understand their target market. They see past vanity metrics.
The Lean Analytics framework.
Eric’s three engines of growth Virality Make people invite friends. How many they tell, how fast they tell them. Price Spend money to get customers. Customers are worth more than they cost. Stickiness Keep people coming back. Approach Get customers faster than you lose them. Math that matters
Dave’s Pirate Metrics AARRR Acquisition How do your users become aware of you? SEO, SEM, widgets, email, PR, campaigns, blogs ... Activation Do drive-by visitors subscribe, use, etc? Features, design, tone, compensation, affirmation ... Retention Does a one-time user become engaged? Notifications, alerts, reminders, emails, updates... Revenue Do you make money from user activity? Transactions, clicks, subscriptions, DLC, analytics... Referral Do users promote your product? Email, widgets, campaigns, likes, RTs, affiliates...
Stage EMPATHY I’ve found a real, poorly-met need that a reachable market faces. STICKINESS I’ve figured out how to solve the problem in a way they will keep using and pay for. VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives. REVENUE The users and features fuel growth organically and artificially. SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem. Gate Thefivestages
Empathy stage: Localmind hacks Twitter Needed to find out if a core assumption—strangers answering questions—was valid. Ran Twitter experiment instead of writing code Asked senders of geolocated Tweets from Times Square random questions; counted response rate Conclusion: high enough to proceed
Stickiness stage: qidiq streamlines invites Survey owner adds recipient to group Survey owner asks question Recipient reads survey question Recipient responds to question Recipient sees survey results (Later, if needed…) Recipient visits site; no password! Recipient does password recovery One-time link sent to email Recipient creates password Recipient can edit profile, etc. Survey owner adds recipient to group Survey owner asks question Recipient gets invite Recipient reads survey question Recipient responds to question Recipient installs mobile app Recipient creates account, profile Recipient sees survey results Recipient can edit profile, etc. 10-25%RESPONSERATE 70-90%RESPONSERATE
Six business model archetypes (Yours is probably a blend of these.)
E-commerce SaaS (freemium?) Mobile app (gaming) Two sided marketplace Media User generated content
(Which means eye charts like these.) Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Cancel Freemium churn Engaged User Free user disengagement Reactivate Cancel Trial abandonment rate Invite Others Paying Customer Reactivation rate Paid conversion FORMER USERS User Lifetime Value Reactivate FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Support data Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Capacity Limit Upselling rate Upselling Disengaged DissatisfiedTrial Over
Model + Stage = One Metric That Matters. One Metric That Matters. The business you’re in E-Com SaaS Mobile 2-Sided Media UCG Empathy Stickiness Virality Revenue Scale Thestageyou’reat
Really? Just one?
Yes, one.
In a startup, focus is hard to achieve.
Having only one metric addresses this problem.
www.theeastsiderla.com
Moz cuts down on metrics SaaS-based SEO toolkit in the scale stage. Focused on net adds. Was a marketing campaign successful? Were customer complaints lowered? Was a product upgrade valuable? Net adds up: Can we acquire more valuable customers? What product features can increase engagement? Can we improve customer support? Net adds flat: Are the new customers not the right segment? Did a marketing campaign fail? Did a product upgrade fail somehow? Is customer support falling apart? Net adds down:
Metrics are like squeeze toys. http://www.flickr.com/photos/connortarter/4791605202/
Empathy Stickiness Virality Revenue Scale E- commerce SaaS Media Mobile app User-gen content 2-sided market Interviews; qualitative results; quantitative scoring; surveys Loyalty, conversion CAC, shares, reactivation Transaction, CLV Affiliates, white-label Engagement, churn Inherent virality, CAC Upselling, CAC, CLV API, magic #, mktplace Content, spam Invites, sharing Ads, donations Analytics, user data Inventory, listings SEM, sharing Transactions, commission Other verticals (Money from transactions) Downloads, churn, virality WoM, app ratings, CAC CLV, ARPDAU Spinoffs, publishers (Money from active users) Traffic, visits, returns Content virality, SEM CPE, affiliate %, eyeballs Syndication, licenses (Money from ad clicks)
Better: bit.ly/BigLeanTable
What other metrics do you want to know about?
Drawing some lines in the sand.
A company loses a quarter of its customers every year. Is this good or bad?
Not knowing what normal is makes you do stupid things.
Baseline: 5-7% growth a week “A good growth rate during YC is 5-7% a week,” he says. “If you can hit 10% a week you're doing exceptionally well. If you can only manage 1%, it's a sign you haven't yet figured out what you're doing.” At revenue stage, measure growth in revenue. Before that, measure growth in active users. Paul Graham, Y Combinator • Are there enough people who really care enough to sustain a 5% growth rate? • Don’t strive for a 5% growth at the expense of really understanding your customers and building a meaningful solution • Once you’re a pre-revenue startup at or near product/market fit, you should have 5% growth of active users each week • Once you’re generating revenues, they should grow at 5% a week
Baseline: 10% visitor engagement/day Fred Wilson’s social ratios 30% of users/month use web or mobile app 10% of users/day use web or mobile app 1% of users/day use it concurrently
Baseline: 2-5% monthly churn • The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s on the job. • Get below a 5% monthly churn rate before you know you’ve got a business that’s ready to grow (Mark MacLeod) and around 2% before you really step on the gas (David Skok) • Last-ditch appeals and reactivation can have a big impact. Facebook’s “don’t leave” reduces attrition by 7%.
Who is worth more? Today A Lifetime: $200 Roberto Medri, Etsy B Lifetime: $200 Visits
Baseline: Calculating customer lifetime 25% monthly churn 100/25=4 The average customer lasts 4 months 5% monthly churn 100/5=20 The average customer lasts 20 months 2% monthly churn 100/2=50 The average customer lasts 50 months
Baseline: CAC under 1/3 of CLV • CLV is wrong. CAC Is probably wrong, too. • Time kills all plans: It’ll take a long time to find out whether your churn and revenue projections are right • Cashflow: You’re basically “loaning” the customer money between acquisition and CLV. • It keeps you honest: Limiting yourself to a CAC of only a third of your CLV will forces you to verify costs sooner. Lifetime of 20 mo. $30/mo. per customer $600 CLV $200 CAC Now segment those users! 1/3 spend
The Lean Analytics cycle
Draw a new line Pivot or give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data: find a commonality Without data: make a good guess Find a potential improvement Draw a linePick a KPI
Do AirBnB hosts get more business if their property is professionally photographed?
Gut instinct (hypothesis) Professional photography helps AirBnB’s business Candidate solution (MVP) 20 field photographers posing as employees Measure the results Compare photographed listings to a control group Make a decision Launch photography as a new feature for all hosts
5,000 shoots per month by February 2012
Hang on a second.
Gut instinct (hypothesis) Professional photography helps AirBnB’s business SRSLY?
Draw a new line Pivot or give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data: find a commonality Without data: make a good guess Find a potential improvement Draw a linePick a KPI
“Gee, those houses that do well look really nice.” Maybe it’s the camera. “Computer: What do all the highly rented houses have in common?” Camera model. With data: find a commonality Without data: make a good guess
Landing page design A/B testing Cohort analysis General analytics URL shortening Funnel analytics Influencer Marketing Publisher analytics SaaS analytics Gaming analytics User interaction Customer satisfaction KPI dashboardsUser segmentation User analytics Spying on users
Some non-tech examples.
I lied. Everyone is a tech company.
http://www.flickr.com/photos/puuikibeach/4789015423 http://www.flickr.com/photos/elcapitanbsc/3936927326 Cost of experiments: down. Cost of attention: way up.
Let’s pick on restaurants for a while.
A line in the sand Labor costs Gross revenue 30% 24% 20% Too costly? Just right Understaffed? =
A leading indicator http://www.flickr.com/photos/avlxyz/4889656453http://www.flickr.com/photos/mysticcountry/3567440970 50 reservations at 5PM 250 covers that night (Varies by restaurant. McDonalds ≠ Fat Duck.)
http://www.flickr.com/photos/southbeachcars/6892880699 Restaurant MVP
Is tip amount a leading indicator of long- term revenue?
Why does every table get the same menu?
Is purple ink better? http://tippingresearch.com/uploads/managing_tips.pdf
Growth hacking (is a word you should hate but will hear a lot about.)
Growth hacking, demystified. Find correlation Test causality Optimize the causal factor Pick a metric to change
Guerrilla marketing Data- driven learning Subversiveness GROWTH HACKING
A Facebook user reaching 7 friends within 10 days of signing up (Chamath Palihapitiya) If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang) Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) A LinkedIn user getting to X connections in Y days (Elliot Schmukler) (from the 2012 Growth Hacking conference) (These are also great segments to analyze.) The leading indicator
• Growth hacking is simply what marketing should have been doing, but it fell in love with Don Draper and opinions along the way • Optimize a factor you think is correlated with growth The growth hack
AirBnB and Craigslist
What if you’re in a big organization?
The Zero Overhead principle A central theme to this new wave of innovation is the application of core product tenets from the consumer space to the enterprise. In particular, a universal lesson that I keep sharing with all entrepreneurs building for the enterprise is the Zero Overhead Principle: no feature may add training costs to the user. DJ Patil
The B2B stereotype http://www.techdigest.tv/2007/02/im_a_pc_im_a_ma.html Domain expert Disruption expert •Domain expert knows industry and the problem domain. Has a Rolodex; proxy for customers. •Disruption expert knows tech that will produce a change Sees beyond the current model.
The Lean Analytics lifecycle of an Intrapreneur Empathy Consulting to test ideas and bootstrap the business Lock-in, IP control, overfitting Stickiness Standardization and integration; shift from custom to generic Ability to integrate; support Virality Word of mouth, references, case studies Bad vibes; exclusivity Revenue Growing direct sales, professional services, support Pipeline, revenue recognition, comp Scale Channels, analysts, ecosystems, APIs, vertically targeted products Crossing the chasm; Gorillas Stage Do this Fear this
Enterprise-focused startups: Metrics that matter • Ease of customer engagement and feedback • Alpha/beta pipeline • Stickiness and usability • Integration costs • User engagement with the app • Disentanglement after a sale • Support costs • User groups and feedback • Pitch success for channel tools • Barriers to exit
What if you’re in a big organization?
If a startup is an organization designed to search for a sustainable, repeatable business model, then an established company is an organization designed to perpetuate one.
The Lean Analytics lifecycle of an Intrapreneur Empathy Find problems; don’t test demand. Skip the business case, do analytics Entitled, aggrieved customers Stickiness Know your real minimum based on expectations, regulations Hidden “must haves”, feature creep Virality Build inherent virality in from the start; attention is the new currency Luddites who don’t understand sharing Revenue Consider the ecosystem, channels, and established agreements Channel conflict, resistance, contracts Scale Hand the baton to others gracefully Hating what happens to your baby Beforehand Get buy-in Political fallout
Some things that work.
Frame it like a study Product creation is almost accidental. Unlike a VC or startup, when the initiative fails the organization still learns. http://www.flickr.com/photos/creative_tools/8544475139
Transformative isolation: Skunkworks
Use outliers and missed searches to hunt for good ideas & adjacencies (Multi-billion-dollar hygiene product company) 1/8 men have an incontinence issue. 1/3 women do. When search results show a significant number of men searching, this suggests the adjacent (male) market is underserved.
Use data to create a taste for data Sitting on Billions of rows of transactional data David Boyle ran 1M online surveys Once the value was obvious to management, got license to dig.
Focus on the desired behavior, not just the information. http://www.psychologytoday.com/blog/yes/ 200808/changing-minds-and-changing-towels 26% increase in towel re-use with an appeal to social norms; 33% increase when tied to the specific room. Energy Conservation “Nudges” and Environmentalist Ideology: Evidence from a Randomized Residential Electricity Field Experiment - Costa & Kahn 2011 The effectiveness of energy conservation “nudges” depends on an individual’s political ideology ... Conservatives who learn that their consumption is less than their neighbors’ “boomerang” whereas liberals reduce their consumption.
Understand hidden constraints That pencil story is a myth. Graphite is conductive and explosive. The Minimum Viable Product is Viable for a reason.
Know what has to be built in-house SAP integration Employment law
Run it as a consulting business first. (Just don’t get addicted to it. Your goal is to learn and overcome integration challenges and find the 20% of features that 80% of the market will pay for.)
When in doubt, collect data From tackling the FTA rate to visualizing the criminal justice supply chain.
Everything’s an excuse to experiment
Find other ways to collect data; everything is an experiment.
Don’t just collect data, chase it.
Some tools and traps
Traction graphs Your business model The stage you’re at Your one metric ... change often if you’re doing it right. So how do you track that over time?
Traction graphs Jan Feb Mar Apr May Jun Signups per day Conversion rate Churn rate Viral coefficient This axis changes for each metric
Traction graphs Jan Feb Mar Apr May Jun Signups per day Conversion rate Churn rate Viral coefficient 0%
Use vanity to get to meaningful metrics Your goal is to produce outcomes If the outcomes require action, and vanity motivates actors, use it But show how the vanity metric is a leading indicator of the real one x Web traffic Revenue Activation Cart Size Conversion rate
The three threes Three assumptions What big bets are you making? •“People will answer questions” •“Organizers are frustrated with how to run conferences” •“We'll make money from parents” •“Amazon is reliable enough for our users.” Three actions to take What are you doing to make these assumptions happen (or identify they’re wrong and change course?) •Product enhancements •Marketing strategies Three experiments to run •Feature tests •Continuous deployment •A/B testing •Customer survey
The three threes Three assumptions Three actions to take Three experiments to run Monthly Weekly Daily Board, investors, founders Executive team Employees Strategy Tactics Execution
The three threes Three assumptions Three actions to take Three experiments to run Get more people Increase answer % Test better questions Change the UI Test timings Questions from peers Many people will answer questions
The problem-solution canvas CURRENT STATUS • List key metrics you’re tracking, where they’re at, and compare with last few weeks • How are things trending? LAST WEEK’S LESSONS LEARNED AND ACCOMPLISHMENTS) • What did you learn last week? • What was accomplished? • On track: YES / NO? The Goal is to Learn
The problem-solution canvas HYPOTHESIZED SOLUTIONS • List possible solutions that you’ll start working on next week. Rank them. • Why do you believe each solution will help you solve or complete solve the problem? METRICS / PROOF + GOALS Problem #1 (put name here) • Metrics you’ll use to measure whether or not the solutions are doing what you hoped (solving the problem) • List proof (qualitative) you’ll use as well • Define goals for the metric HYPOTHESIZED SOLUTIONS • List possible solutions that you’ll start working on next week. Rank them. METRICS / PROOF + GOALS • Metrics you’ll use to measure whether or not the solutions are doing what you hoped (solving the problem) Problem #2 (put name here)
“The most important figures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them.” Lloyd S. Nelson
Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
ARCHIMEDES HAD TAKEN BATHS BEFORE.
Once, a leader convinced others in the absence of data.
Now, a leader knows what questions to ask.
Alistair Croll acroll@gmail.com @acroll Ben Yoskovitz byosko@gmail.com @byosko
Follow-on: The other business model system diagrams
The mobile app! customer lifecycle! Ratings Reviews Search Leaderboards Purchases Downloads Installs Play Disengagement Reactivation Uninstallation Disengagement Account" creation Virality Downloads," Gross revenue ARPU App sales Activation Churn, CLV In-app" purchases Appstore! Incentivized Legitimate Fraudulent Ratings!
Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014

More Related Content

PDF
Lean Analytics for Startups and Enterprises
byLean Analytics
 
PDF
Lean Analytics @ MicroConf
byLean Analytics
 
PDF
What's Next: Digital Transformation
byOgilvy Consulting
 
PDF
Lean Startup
byJohn Greene
 
PDF
How to develop Startup Pitch Deck
byKashyap Pandya
 
PDF
21 Actionable Growth Hacking Tactics
byJon Yongfook
 
PPTX
Lean startup, customer development, and the business model canvas
bygistinitiative
 
PPTX
Building an MVP
byRolands Krumbergs
 
Lean Analytics for Startups and Enterprises
byLean Analytics
 
Lean Analytics @ MicroConf
byLean Analytics
 
What's Next: Digital Transformation
byOgilvy Consulting
 
Lean Startup
byJohn Greene
 
How to develop Startup Pitch Deck
byKashyap Pandya
 
21 Actionable Growth Hacking Tactics
byJon Yongfook
 
Lean startup, customer development, and the business model canvas
bygistinitiative
 
Building an MVP
byRolands Krumbergs
 

What's hot

PDF
ChatGPT and the Future of Work - Clark Boyd
byClark Boyd
 
PPT
Startup Metrics for Pirates
byDave McClure
 
PDF
Advanced Tactics for Using AI Tools & Big Data Analysis to Improve E-E-A-T
byLily Ray
 
PDF
The essential elements of a digital transformation strategy
byMarcel Santilli
 
PDF
Why, When and How Do I Start a Digital Transformation?
byAcquia
 
PPT
Getting to Product Market Fit - An Overview of Customer Discovery & Validation
byJason Evanish
 
PPTX
Business model canvas
byNikolay Stoyanov
 
PDF
Ultimate Guide to Funnel Optimization
bySean Johnson
 
PDF
Lean Startup + Story Mapping = Awesome Products Faster
byBrad Swanson
 
PDF
Digital Transformation
byEvgeny Tsarkov
 
PDF
Lean Analytics Cycle
byHiten Shah
 
PDF
The Lean Startup
bySean Si
 
PDF
ChatGPT Prompt Engineering
bySupernova Media
 
PDF
The Product Market Fit Cycle (Updated to v. 2.0)
byCarlos Espinal
 
PDF
29 Growth Hacking Quick Wins
byMattan Griffel
 
PPTX
Digital Marketing Plan
byAshley Egan
 
PPTX
Zero to 50m
byDavid Skok
 
PPTX
Marketing Strategy for a new Real estate start up
byMANISH SRIVASTTAVA
 
PPTX
Radical change, the quiet way
bySunny Mervyne Baa
 
PDF
An Intro to Lean Startup
byRyan Hoover
 
ChatGPT and the Future of Work - Clark Boyd
byClark Boyd
 
Startup Metrics for Pirates
byDave McClure
 
Advanced Tactics for Using AI Tools & Big Data Analysis to Improve E-E-A-T
byLily Ray
 
The essential elements of a digital transformation strategy
byMarcel Santilli
 
Why, When and How Do I Start a Digital Transformation?
byAcquia
 
Getting to Product Market Fit - An Overview of Customer Discovery & Validation
byJason Evanish
 
Business model canvas
byNikolay Stoyanov
 
Ultimate Guide to Funnel Optimization
bySean Johnson
 
Lean Startup + Story Mapping = Awesome Products Faster
byBrad Swanson
 
Digital Transformation
byEvgeny Tsarkov
 
Lean Analytics Cycle
byHiten Shah
 
The Lean Startup
bySean Si
 
ChatGPT Prompt Engineering
bySupernova Media
 
The Product Market Fit Cycle (Updated to v. 2.0)
byCarlos Espinal
 
29 Growth Hacking Quick Wins
byMattan Griffel
 
Digital Marketing Plan
byAshley Egan
 
Zero to 50m
byDavid Skok
 
Marketing Strategy for a new Real estate start up
byMANISH SRIVASTTAVA
 
Radical change, the quiet way
bySunny Mervyne Baa
 
An Intro to Lean Startup
byRyan Hoover
 

Similar to Lean Analytics workshop for Dublin City University, April 2014

PDF
Lean Analytics workshop (from Lean Startup Conf)
byLean Analytics
 
PDF
Lean Analytics: Using Data to Build a Better Business Faster
byLean Startup Co.
 
PDF
Alistaire croll lean analytics - montreal lean startup circle - september 2018
byLean Startup Circle Montreal
 
KEY
Lean Startup Metrics & Analytics
byNicola Junior Vitto
 
PDF
Introduction to Lean Analytics for Lean Startup Circle SF
byLean Analytics
 
PDF
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
byLean Startup Co.
 
PDF
Slides from Growthcon 2014 Lean Analytics masterclass
byLean Analytics
 
PDF
Lean Analytics for Intrapreneurs by Allistair Croll
byLean Startup Co.
 
PDF
Boost Your Growth Engine with Analytics
byCemal Buyukgokcesu
 
PDF
Croll lean analytics workshop (3h) - lean ux nyc april 2014
byLean Analytics
 
PDF
online startups metrics
byHatem Kameli
 
PDF
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
byLean Analytics
 
PDF
SBC Growth Week - Lean Analytics
byMeasureWorks
 
PDF
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
byLean Analytics
 
PDF
Startup analytics
byTopi Järvinen
 
PDF
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
byLean Analytics
 
PDF
Slides from New Media Manitoba Lean Analytics workshop, June 2015
byLean Analytics
 
PPT
Lean analytics: Five lessons beyond the basics
byLean Analytics
 
PPT
Lean analytics 40 minute intro
byPinkesh Shah
 
PDF
ESADE Business Analytics Master: Metrics that matter
byJuliana Mendez
 
Lean Analytics workshop (from Lean Startup Conf)
byLean Analytics
 
Lean Analytics: Using Data to Build a Better Business Faster
byLean Startup Co.
 
Alistaire croll lean analytics - montreal lean startup circle - september 2018
byLean Startup Circle Montreal
 
Lean Startup Metrics & Analytics
byNicola Junior Vitto
 
Introduction to Lean Analytics for Lean Startup Circle SF
byLean Analytics
 
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
byLean Startup Co.
 
Slides from Growthcon 2014 Lean Analytics masterclass
byLean Analytics
 
Lean Analytics for Intrapreneurs by Allistair Croll
byLean Startup Co.
 
Boost Your Growth Engine with Analytics
byCemal Buyukgokcesu
 
Croll lean analytics workshop (3h) - lean ux nyc april 2014
byLean Analytics
 
online startups metrics
byHatem Kameli
 
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
byLean Analytics
 
SBC Growth Week - Lean Analytics
byMeasureWorks
 
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
byLean Analytics
 
Startup analytics
byTopi Järvinen
 
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
byLean Analytics
 
Slides from New Media Manitoba Lean Analytics workshop, June 2015
byLean Analytics
 
Lean analytics: Five lessons beyond the basics
byLean Analytics
 
Lean analytics 40 minute intro
byPinkesh Shah
 
ESADE Business Analytics Master: Metrics that matter
byJuliana Mendez
 

More from Lean Analytics

PPTX
Introduction to Lean Analytics webinar (O'Reilly)
byLean Analytics
 
PDF
Understanding Lean Analytics (and how analytics helps businesses win)
byLean Analytics
 
PDF
OnLab Japan introduction to Lean Analytics
byLean Analytics
 
PDF
Lean analytics from Web A Quebec mars 2014
byLean Analytics
 
PDF
Lean Analytics for Nikkei BP
byLean Analytics
 
PPT
7 Myths of Lean and How Analytics Can Help
byLean Analytics
 
PDF
Big data tokyo (extended version)
byLean Analytics
 
PDF
Startup metrics toronto March 19
byLean Analytics
 
PDF
Lean Startup Machine Montreal
byLean Analytics
 
PDF
Making Sense of the Numbers (Lean Analytics)
byLean Analytics
 
PDF
Lean Analytics and Local Government - Alistair Croll - Code for America
byLean Analytics
 
Introduction to Lean Analytics webinar (O'Reilly)
byLean Analytics
 
Understanding Lean Analytics (and how analytics helps businesses win)
byLean Analytics
 
OnLab Japan introduction to Lean Analytics
byLean Analytics
 
Lean analytics from Web A Quebec mars 2014
byLean Analytics
 
Lean Analytics for Nikkei BP
byLean Analytics
 
7 Myths of Lean and How Analytics Can Help
byLean Analytics
 
Big data tokyo (extended version)
byLean Analytics
 
Startup metrics toronto March 19
byLean Analytics
 
Lean Startup Machine Montreal
byLean Analytics
 
Making Sense of the Numbers (Lean Analytics)
byLean Analytics
 
Lean Analytics and Local Government - Alistair Croll - Code for America
byLean Analytics
 

Recently uploaded

PDF
When Is the Right Time to Telegram Accounts_.pdf
by3tb3224qp2qbmz9iivnj
 
PDF
[DSC Europe 25] Laila Kakar - Leveraging AI for Strategic Excellence: Enhanci...
byDataScienceConferenc1
 
PPTX
[DSC Europe 25] Paula Garcia Esteban -Building the Future: The Role of Data S...
byDataScienceConferenc1
 
PPTX
[DSC Europe 25] Marcos Heidemann - Beyond the Hype: Making AI Coding Assistan...
byDataScienceConferenc1
 
PDF
Semantic Technology
byRathachai Chawuthai
 
PDF
[DSC Europe 25] Gordana Milutinovic Dumbelovic - From Insight to Oversight: A...
byDataScienceConferenc1
 
PDF
SIH2025-IDEA-Presentation-Innogenix320250929144422.pdf
bydevichokkakulacse1
 
PPTX
normal labor normal labor normal labor normal labor normal labor normal labor...
bymeksbaba1
 
PDF
Data Infrastructure as Code - Building core data services in a small team
byMichal Gancarski
 
PDF
Serverless Data Infrastructure (examples)
byMichal Gancarski
 
PDF
Security and confidentiality in big data.pdf
byrayessafa21
 
PDF
Community re-Invent re-Cap Slides AWS UserGroup Hamburg 01/2026
byAlexeyVidanov1
 
PDF
The-Authoritarian-Gaze.pdfThe-Authoritarian-Gaze.pdf
by中 央社
 
PPTX
[DSC Europe 25] Dubravko Culibrk - Deep Learning for Mammography.pptx
byDataScienceConferenc1
 
PPTX
Artificial neural network Available for instant download in exchange for a bi...
byvinstonr
 
PPTX
2026 Retreat Presentations (GREBS and WRAPS).pptx
bytobacalebakanmu
 
PPTX
Full_Transformer_hjyhkkiProject_PPT.pptx
byhrishikeshmanu2008
 
PPTX
IKS-Unit 2 Part-1(1).pptxnnnnnnnnnnnnnnn
bykeshavmaggo
 
PPTX
REVIEWER-MddddddddddddATH-10-3rd-quarter.pptx
bydominicdaltoncaling2
 
PPTX
Inventory Managemnet for111111111111111W2
bydrizwan89
 
When Is the Right Time to Telegram Accounts_.pdf
by3tb3224qp2qbmz9iivnj
 
[DSC Europe 25] Laila Kakar - Leveraging AI for Strategic Excellence: Enhanci...
byDataScienceConferenc1
 
[DSC Europe 25] Paula Garcia Esteban -Building the Future: The Role of Data S...
byDataScienceConferenc1
 
[DSC Europe 25] Marcos Heidemann - Beyond the Hype: Making AI Coding Assistan...
byDataScienceConferenc1
 
Semantic Technology
byRathachai Chawuthai
 
[DSC Europe 25] Gordana Milutinovic Dumbelovic - From Insight to Oversight: A...
byDataScienceConferenc1
 
SIH2025-IDEA-Presentation-Innogenix320250929144422.pdf
bydevichokkakulacse1
 
normal labor normal labor normal labor normal labor normal labor normal labor...
bymeksbaba1
 
Data Infrastructure as Code - Building core data services in a small team
byMichal Gancarski
 
Serverless Data Infrastructure (examples)
byMichal Gancarski
 
Security and confidentiality in big data.pdf
byrayessafa21
 
Community re-Invent re-Cap Slides AWS UserGroup Hamburg 01/2026
byAlexeyVidanov1
 
The-Authoritarian-Gaze.pdfThe-Authoritarian-Gaze.pdf
by中 央社
 
[DSC Europe 25] Dubravko Culibrk - Deep Learning for Mammography.pptx
byDataScienceConferenc1
 
Artificial neural network Available for instant download in exchange for a bi...
byvinstonr
 
2026 Retreat Presentations (GREBS and WRAPS).pptx
bytobacalebakanmu
 
Full_Transformer_hjyhkkiProject_PPT.pptx
byhrishikeshmanu2008
 
IKS-Unit 2 Part-1(1).pptxnnnnnnnnnnnnnnn
bykeshavmaggo
 
REVIEWER-MddddddddddddATH-10-3rd-quarter.pptx
bydominicdaltoncaling2
 
Inventory Managemnet for111111111111111W2
bydrizwan89
 

Lean Analytics workshop for Dublin City University, April 2014

  • 1.
  • 2.
  • 3.
    Don’t sell whatyou can make. Make what you can sell. Kevin Costner is a lousy entrepreneur.
  • 4.
    The core ofLean is iteration.
  • 5.
    Most startups don’tknow what they’ll be when they grow up. Hotmail was a database company Flickr was going to be an MMO Twitter was a podcasting company Autodesk made desktop automation Paypal first built for Palmpilots Freshbooks was invoicing for a web design firm Wikipedia was to be written by experts only Mitel was a lawnmower company
  • 6.
  • 7.
    Everyone’s idea is thebest right? People love this part! (but that’s not always a good thing) This is where things fall apart. No data, no learning.
  • 8.
  • 9.
    Analytics is themeasurement of movement towards your business goals.
  • 10.
    In a startup,the purpose of analytics is to iterate to product/market fit before the money runs out.
  • 11.
    I have twokids. At least one of them is a girl.
  • 12.
    What are thechances the other is a boy?
  • 13.
  • 14.
    2 of 3(66%) are boys. GB GG BG
  • 15.
  • 16.
    A good metricis: Understandable If you’re busy explaining the data, you won’t be busy acting on it. Comparative Comparison is context. A ratio or rate The only way to measure change and roll up the tension between two metrics (MPH) Behavior changing If you’re busy explaining the data, you won’t be busy acting on it.
  • 17.
    The simplest rule bad metric. If a metricwon’t change how you behave, it’s a h"p://www.flickr.com/photos/circasassy/7858155676/
  • 18.
    Metrics help youknow yourself. Acquisition Hybrid Loyalty 70% of retailers 20% of retailers 10% of retailers You are just like Customers that buy >1x in 90d Once 2-2.5 per year >2.5 per year Your customers will buy from you Then you are in this mode 1-15% 15-30% >30% Low acquisition cost, high checkout Increasing return rates, market share Loyalty, selection, inventory size Focus on (Thanks to Kevin Hillstrom for this.)
  • 19.
    Qualitative Unstructured, anecdotal, revealing, hardto aggregate, often too positive & reassuring. Warm and fuzzy. Quantitative Numbers and stats. Hard facts, less insight, easier to analyze; often sour and disappointing. Cold and hard.
  • 20.
    Exploratory Speculative. Tries tofind unexpected or interesting insights. Source of unfair advantages. Cool. Reporting Predictable. Keeps you abreast of the normal, day-to-day operations. Can be managed by exception. Necessary.
  • 21.
    Rumsfeld on Analytics (Orrather, Avinash Kaushik channeling Rumsfeld) Things we know don’t know we know Are facts which may be wrong and should be checked against data. we don’t know Are questions we can answer by reporting, which we should baseline & automate. we know Are intuition which we should quantify and teach to improve effectiveness, efficiency. we don’t know Are exploration which is where unfair advantage and interesting epiphanies live.
  • 22.
    MayAprMarFeb Slicing and dicingdata Jan 0 5,000 Activeusers Cohort: Comparison of similar groups along a timeline. (this is the April cohort) A/B test: Changing one thing (i.e. color) and measuring the result (i.e. revenue.) Multivariate analysis Changing several things at once to see which correlates with a result. ☀ ☁ ☀ ☁ Segment: Cross-sectional comparison of all people divided by some attribute (age, gender, etc.) ☀ ☁
  • 23.
    Which of thesetwo companies is doing better?
  • 24.
      January FebruaryMarch April May Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50 Is this company growing or stagnating? Cohort 1 2 3 4 5 January February March April May $5 $3 $2 $1 $0.5 $6 $4 $2 $1 $7 $6 $5   $8 $7       $9 How about this one?
  • 25.
    Cohort 1 23 4 5 January February March April May Averages $5 $3 $2 $1 $0.5 $6 $4 $2 $1   $7 $6 $5     $8 $7       $9         $7 $5 $3 $1 $0.5 Look at the same data in cohorts
  • 26.
    Lagging Historical. Shows you howyou’re doing; reports the news. Example: sales. Explaining the past. Leading Forward-looking. Number today that predicts tomorrow; reports the news. Example: pipeline. Predicting the future.
  • 27.
    A Facebook userreaching 7 friends within 10 days of signing up (Chamath Palihapitiya) If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang) Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) A LinkedIn user getting to X connections in Y days (Elliot Schmukler) Some examples (From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
  • 28.
    Which means it’stime to talk about correlation.
  • 29.
    1 10 100 1000 10000 Ice cream consumptionDrownings Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
  • 30.
    Correlated Two variables thatare related (but may be dependent on something else.) Ice cream & drowning. Causal An independent variable that directly impacts a dependent one. Summertime & drowning.
  • 31.
    A leading, causalmetric is a superpower. h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
  • 32.
  • 33.
    Is social actiona leading indicator of donation? http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/
  • 34.
  • 35.
    Why is Nigerianspam so badly written?
  • 36.
    Aunshul Rege ofRutgers University, USA in 2009 Experienced scammers expect a “strike rate” of 1 or 2 replies per 1,000 messages emailed; they expect to land 2 or 3 “Mugu” (fools) each week. One scammer boasted “When you get a reply it’s 70% sure you’ll get the money” “By sending an email that repels all but the most gullible,” says [Microsoft Researcher Corman] Herley, “the scammer gets the most promising marks to self-select, and tilts the true to false positive ratio in his favor.” 1000 emails 1-2 responses 1 fool and their money, parted. Bad language (0.1% conversion) Gullible (70% conversion) 1000 emails 100 responses 1 fool and their money, parted. Good language (10% conversion) Not-gullible (.07% conversion) This would be horribly inefficient since humans are involved.
  • 37.
    Turns out theword “Nigeria” is the best way to identify promising prospects.
  • 38.
    Nigerian spammers really understandtheir target market. They see past vanity metrics.
  • 39.
  • 40.
    Eric’s three enginesof growth Virality Make people invite friends. How many they tell, how fast they tell them. Price Spend money to get customers. Customers are worth more than they cost. Stickiness Keep people coming back. Approach Get customers faster than you lose them. Math that matters
  • 41.
    Dave’s Pirate Metrics AARRR Acquisition Howdo your users become aware of you? SEO, SEM, widgets, email, PR, campaigns, blogs ... Activation Do drive-by visitors subscribe, use, etc? Features, design, tone, compensation, affirmation ... Retention Does a one-time user become engaged? Notifications, alerts, reminders, emails, updates... Revenue Do you make money from user activity? Transactions, clicks, subscriptions, DLC, analytics... Referral Do users promote your product? Email, widgets, campaigns, likes, RTs, affiliates...
  • 42.
    Stage EMPATHY I’ve found areal, poorly-met need that a reachable market faces. STICKINESS I’ve figured out how to solve the problem in a way they will keep using and pay for. VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives. REVENUE The users and features fuel growth organically and artificially. SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem. Gate Thefivestages
  • 43.
    Empathy stage: Localmind hacksTwitter Needed to find out if a core assumption—strangers answering questions—was valid. Ran Twitter experiment instead of writing code Asked senders of geolocated Tweets from Times Square random questions; counted response rate Conclusion: high enough to proceed
  • 44.
    Stickiness stage: qidiq streamlinesinvites Survey owner adds recipient to group Survey owner asks question Recipient reads survey question Recipient responds to question Recipient sees survey results (Later, if needed…) Recipient visits site; no password! Recipient does password recovery One-time link sent to email Recipient creates password Recipient can edit profile, etc. Survey owner adds recipient to group Survey owner asks question Recipient gets invite Recipient reads survey question Recipient responds to question Recipient installs mobile app Recipient creates account, profile Recipient sees survey results Recipient can edit profile, etc. 10-25%RESPONSERATE 70-90%RESPONSERATE
  • 45.
    Six business modelarchetypes (Yours is probably a blend of these.)
  • 46.
    E-commerce SaaS (freemium?) Mobile app(gaming) Two sided marketplace Media User generated content
  • 47.
    (Which means eye chartslike these.) Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Cancel Freemium churn Engaged User Free user disengagement Reactivate Cancel Trial abandonment rate Invite Others Paying Customer Reactivation rate Paid conversion FORMER USERS User Lifetime Value Reactivate FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Support data Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Capacity Limit Upselling rate Upselling Disengaged DissatisfiedTrial Over
  • 48.
    Model + Stage= One Metric That Matters. One Metric That Matters. The business you’re in E-Com SaaS Mobile 2-Sided Media UCG Empathy Stickiness Virality Revenue Scale Thestageyou’reat
  • 49.
  • 50.
  • 51.
    In a startup,focus is hard to achieve.
  • 52.
    Having only onemetric addresses this problem.
  • 53.
  • 54.
    Moz cuts downon metrics SaaS-based SEO toolkit in the scale stage. Focused on net adds. Was a marketing campaign successful? Were customer complaints lowered? Was a product upgrade valuable? Net adds up: Can we acquire more valuable customers? What product features can increase engagement? Can we improve customer support? Net adds flat: Are the new customers not the right segment? Did a marketing campaign fail? Did a product upgrade fail somehow? Is customer support falling apart? Net adds down:
  • 55.
    Metrics are likesqueeze toys. http://www.flickr.com/photos/connortarter/4791605202/
  • 56.
    Empathy Stickiness Virality Revenue Scale E- commerce SaaS Media Mobile app User-gen content 2-sided market Interviews; qualitativeresults; quantitative scoring; surveys Loyalty, conversion CAC, shares, reactivation Transaction, CLV Affiliates, white-label Engagement, churn Inherent virality, CAC Upselling, CAC, CLV API, magic #, mktplace Content, spam Invites, sharing Ads, donations Analytics, user data Inventory, listings SEM, sharing Transactions, commission Other verticals (Money from transactions) Downloads, churn, virality WoM, app ratings, CAC CLV, ARPDAU Spinoffs, publishers (Money from active users) Traffic, visits, returns Content virality, SEM CPE, affiliate %, eyeballs Syndication, licenses (Money from ad clicks)
  • 57.
  • 58.
    What other metrics doyou want to know about?
  • 59.
    Drawing some linesin the sand.
  • 60.
    A company losesa quarter of its customers every year. Is this good or bad?
  • 61.
    Not knowing whatnormal is makes you do stupid things.
  • 62.
    Baseline: 5-7% growth aweek “A good growth rate during YC is 5-7% a week,” he says. “If you can hit 10% a week you're doing exceptionally well. If you can only manage 1%, it's a sign you haven't yet figured out what you're doing.” At revenue stage, measure growth in revenue. Before that, measure growth in active users. Paul Graham, Y Combinator • Are there enough people who really care enough to sustain a 5% growth rate? • Don’t strive for a 5% growth at the expense of really understanding your customers and building a meaningful solution • Once you’re a pre-revenue startup at or near product/market fit, you should have 5% growth of active users each week • Once you’re generating revenues, they should grow at 5% a week
  • 63.
    Baseline: 10% visitor engagement/day FredWilson’s social ratios 30% of users/month use web or mobile app 10% of users/day use web or mobile app 1% of users/day use it concurrently
  • 64.
    Baseline: 2-5% monthly churn •The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s on the job. • Get below a 5% monthly churn rate before you know you’ve got a business that’s ready to grow (Mark MacLeod) and around 2% before you really step on the gas (David Skok) • Last-ditch appeals and reactivation can have a big impact. Facebook’s “don’t leave” reduces attrition by 7%.
  • 65.
    Who is worthmore? Today A Lifetime: $200 Roberto Medri, Etsy B Lifetime: $200 Visits
  • 66.
    Baseline: Calculating customer lifetime 25% monthlychurn 100/25=4 The average customer lasts 4 months 5% monthly churn 100/5=20 The average customer lasts 20 months 2% monthly churn 100/2=50 The average customer lasts 50 months
  • 67.
    Baseline: CAC under 1/3of CLV • CLV is wrong. CAC Is probably wrong, too. • Time kills all plans: It’ll take a long time to find out whether your churn and revenue projections are right • Cashflow: You’re basically “loaning” the customer money between acquisition and CLV. • It keeps you honest: Limiting yourself to a CAC of only a third of your CLV will forces you to verify costs sooner. Lifetime of 20 mo. $30/mo. per customer $600 CLV $200 CAC Now segment those users! 1/3 spend
  • 68.
  • 69.
    Draw a newline Pivot or give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data: find a commonality Without data: make a good guess Find a potential improvement Draw a linePick a KPI
  • 70.
    Do AirBnB hosts getmore business if their property is professionally photographed?
  • 71.
    Gut instinct (hypothesis) Professionalphotography helps AirBnB’s business Candidate solution (MVP) 20 field photographers posing as employees Measure the results Compare photographed listings to a control group Make a decision Launch photography as a new feature for all hosts
  • 72.
    5,000 shoots permonth by February 2012
  • 73.
    Hang on asecond.
  • 74.
    Gut instinct (hypothesis) Professionalphotography helps AirBnB’s business SRSLY?
  • 75.
    Draw a newline Pivot or give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data: find a commonality Without data: make a good guess Find a potential improvement Draw a linePick a KPI
  • 76.
    “Gee, those houses thatdo well look really nice.” Maybe it’s the camera. “Computer: What do all the highly rented houses have in common?” Camera model. With data: find a commonality Without data: make a good guess
  • 77.
    Landing page designA/B testing Cohort analysis General analytics URL shortening Funnel analytics Influencer Marketing Publisher analytics SaaS analytics Gaming analytics User interaction Customer satisfaction KPI dashboardsUser segmentation User analytics Spying on users
  • 78.
  • 79.
    I lied. Everyoneis a tech company.
  • 80.
  • 81.
    Let’s pick onrestaurants for a while.
  • 82.
    A line inthe sand Labor costs Gross revenue 30% 24% 20% Too costly? Just right Understaffed? =
  • 83.
    A leading indicator http://www.flickr.com/photos/avlxyz/4889656453http://www.flickr.com/photos/mysticcountry/3567440970 50reservations at 5PM 250 covers that night (Varies by restaurant. McDonalds ≠ Fat Duck.)
  • 84.
  • 85.
    Is tip amounta leading indicator of long- term revenue?
  • 86.
    Why does everytable get the same menu?
  • 87.
    Is purple inkbetter? http://tippingresearch.com/uploads/managing_tips.pdf
  • 89.
    Growth hacking (is aword you should hate but will hear a lot about.)
  • 90.
  • 91.
  • 92.
    A Facebook userreaching 7 friends within 10 days of signing up (Chamath Palihapitiya) If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang) Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) A LinkedIn user getting to X connections in Y days (Elliot Schmukler) (from the 2012 Growth Hacking conference) (These are also great segments to analyze.) The leading indicator
  • 93.
    • Growth hackingis simply what marketing should have been doing, but it fell in love with Don Draper and opinions along the way • Optimize a factor you think is correlated with growth The growth hack
  • 94.
  • 98.
    What if you’rein a big organization?
  • 99.
    The Zero Overheadprinciple A central theme to this new wave of innovation is the application of core product tenets from the consumer space to the enterprise. In particular, a universal lesson that I keep sharing with all entrepreneurs building for the enterprise is the Zero Overhead Principle: no feature may add training costs to the user. DJ Patil
  • 100.
    The B2B stereotype http://www.techdigest.tv/2007/02/im_a_pc_im_a_ma.html Domain expert Disruption expert •Domainexpert knows industry and the problem domain. Has a Rolodex; proxy for customers. •Disruption expert knows tech that will produce a change Sees beyond the current model.
  • 101.
    The Lean Analyticslifecycle of an Intrapreneur Empathy Consulting to test ideas and bootstrap the business Lock-in, IP control, overfitting Stickiness Standardization and integration; shift from custom to generic Ability to integrate; support Virality Word of mouth, references, case studies Bad vibes; exclusivity Revenue Growing direct sales, professional services, support Pipeline, revenue recognition, comp Scale Channels, analysts, ecosystems, APIs, vertically targeted products Crossing the chasm; Gorillas Stage Do this Fear this
  • 102.
    Enterprise-focused startups: Metrics thatmatter • Ease of customer engagement and feedback • Alpha/beta pipeline • Stickiness and usability • Integration costs • User engagement with the app • Disentanglement after a sale • Support costs • User groups and feedback • Pitch success for channel tools • Barriers to exit
  • 103.
    What if you’rein a big organization?
  • 104.
    If a startupis an organization designed to search for a sustainable, repeatable business model, then an established company is an organization designed to perpetuate one.
  • 105.
    The Lean Analyticslifecycle of an Intrapreneur Empathy Find problems; don’t test demand. Skip the business case, do analytics Entitled, aggrieved customers Stickiness Know your real minimum based on expectations, regulations Hidden “must haves”, feature creep Virality Build inherent virality in from the start; attention is the new currency Luddites who don’t understand sharing Revenue Consider the ecosystem, channels, and established agreements Channel conflict, resistance, contracts Scale Hand the baton to others gracefully Hating what happens to your baby Beforehand Get buy-in Political fallout
  • 106.
  • 107.
    Frame it likea study Product creation is almost accidental. Unlike a VC or startup, when the initiative fails the organization still learns. http://www.flickr.com/photos/creative_tools/8544475139
  • 108.
  • 109.
    Use outliers andmissed searches to hunt for good ideas & adjacencies (Multi-billion-dollar hygiene product company) 1/8 men have an incontinence issue. 1/3 women do. When search results show a significant number of men searching, this suggests the adjacent (male) market is underserved.
  • 110.
    Use data tocreate a taste for data Sitting on Billions of rows of transactional data David Boyle ran 1M online surveys Once the value was obvious to management, got license to dig.
  • 111.
    Focus on thedesired behavior, not just the information. http://www.psychologytoday.com/blog/yes/ 200808/changing-minds-and-changing-towels 26% increase in towel re-use with an appeal to social norms; 33% increase when tied to the specific room. Energy Conservation “Nudges” and Environmentalist Ideology: Evidence from a Randomized Residential Electricity Field Experiment - Costa & Kahn 2011 The effectiveness of energy conservation “nudges” depends on an individual’s political ideology ... Conservatives who learn that their consumption is less than their neighbors’ “boomerang” whereas liberals reduce their consumption.
  • 112.
    Understand hidden constraints That pencilstory is a myth. Graphite is conductive and explosive. The Minimum Viable Product is Viable for a reason.
  • 113.
    Know what hasto be built in-house SAP integration Employment law
  • 114.
    Run it asa consulting business first. (Just don’t get addicted to it. Your goal is to learn and overcome integration challenges and find the 20% of features that 80% of the market will pay for.)
  • 115.
    When in doubt,collect data From tackling the FTA rate to visualizing the criminal justice supply chain.
  • 116.
  • 117.
    Find other ways tocollect data; everything is an experiment.
  • 118.
    Don’t just collectdata, chase it.
  • 119.
  • 120.
    Traction graphs Your businessmodel The stage you’re at Your one metric ... change often if you’re doing it right. So how do you track that over time?
  • 121.
    Traction graphs Jan FebMar Apr May Jun Signups per day Conversion rate Churn rate Viral coefficient This axis changes for each metric
  • 122.
    Traction graphs Jan FebMar Apr May Jun Signups per day Conversion rate Churn rate Viral coefficient 0%
  • 123.
    Use vanity toget to meaningful metrics Your goal is to produce outcomes If the outcomes require action, and vanity motivates actors, use it But show how the vanity metric is a leading indicator of the real one x Web traffic Revenue Activation Cart Size Conversion rate
  • 124.
    The three threes Three assumptions Whatbig bets are you making? •“People will answer questions” •“Organizers are frustrated with how to run conferences” •“We'll make money from parents” •“Amazon is reliable enough for our users.” Three actions to take What are you doing to make these assumptions happen (or identify they’re wrong and change course?) •Product enhancements •Marketing strategies Three experiments to run •Feature tests •Continuous deployment •A/B testing •Customer survey
  • 125.
    The three threes Three assumptions Threeactions to take Three experiments to run Monthly Weekly Daily Board, investors, founders Executive team Employees Strategy Tactics Execution
  • 126.
    The three threes Three assumptions Threeactions to take Three experiments to run Get more people Increase answer % Test better questions Change the UI Test timings Questions from peers Many people will answer questions
  • 127.
    The problem-solution canvas CURRENTSTATUS • List key metrics you’re tracking, where they’re at, and compare with last few weeks • How are things trending? LAST WEEK’S LESSONS LEARNED AND ACCOMPLISHMENTS) • What did you learn last week? • What was accomplished? • On track: YES / NO? The Goal is to Learn
  • 128.
    The problem-solution canvas HYPOTHESIZEDSOLUTIONS • List possible solutions that you’ll start working on next week. Rank them. • Why do you believe each solution will help you solve or complete solve the problem? METRICS / PROOF + GOALS Problem #1 (put name here) • Metrics you’ll use to measure whether or not the solutions are doing what you hoped (solving the problem) • List proof (qualitative) you’ll use as well • Define goals for the metric HYPOTHESIZED SOLUTIONS • List possible solutions that you’ll start working on next week. Rank them. METRICS / PROOF + GOALS • Metrics you’ll use to measure whether or not the solutions are doing what you hoped (solving the problem) Problem #2 (put name here)
  • 129.
    “The most importantfigures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them.” Lloyd S. Nelson
  • 130.
    Pic by Twodollaon Flickr. http://www.flickr.com/photos/twodolla/3168857844
  • 131.
  • 132.
    Once, a leaderconvinced others in the absence of data.
  • 133.
    Now, a leaderknows what questions to ask.
  • 134.
  • 135.
    Follow-on: The other businessmodel system diagrams
  • 138.
    The mobile app! customerlifecycle! Ratings Reviews Search Leaderboards Purchases Downloads Installs Play Disengagement Reactivation Uninstallation Disengagement Account" creation Virality Downloads," Gross revenue ARPU App sales Activation Churn, CLV In-app" purchases Appstore! Incentivized Legitimate Fraudulent Ratings!