Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference

L
Lean AnalyticsStartup accelerator
••
Lean Analytics Lean Startup Conference December, 2015 @acroll
Some housekeeping.
Kevin Costner is a lousy entrepreneur. Don’t sell what you can make. Make what you can sell.
The core of Lean is iteration.
Most startups don’t know what they’ll be when they grow up. Hotmail was a database company Flickr was going to be an MMO Twitter was a podcasting company Autodesk made desktop automation Paypal first built for Palmpilots Freshbooks was invoicing for a web design firm Wikipedia was to be written by experts only Mitel was a lawnmower company
Waterfall, agile, and lean (Why the old ways don’t work.)
Waterfall approach You know the problem and the solution.
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference
Known set of requirements Known ways to satisfy them Spec Build Test Launch
Agile methodologies Know the problem, find the solution
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference
Known set of requirements Unclear how to satisfy them Problem Build Test Viable? Launch statement Sprints Adjust Unknown set of
Lean approach First, know that you don’t know.
Product/market hypothesis Trial startup Possible problem space Product/ market hypothesis Trial startup Product/ market hypothesis Trial startup Trial startup Product/market hypothesis You are here PIVOT
Why now? First: High rate of change of digital technologies & channels.
Arbitron and radio data
Times a song in “heavy rotation” is played daily 30 15 0 6 26 2007 2012
For modern media this means cycle time shock. Circulation, annually Clicks, instantaneously Letters to the editor, weekly Hashtags, always
Why now? Second: It’s no longer about whether you can build it—it’s about whether anyone will care.
The Attention Economy “What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention, and a need to allocate that attention efficiently among the overabundance of information sources that might consume it.” (Computers, Communications and the Public Interest, pages 40-41, Herbert Simon Martin Greenberger, ed., The Johns Hopkins Press, 1971.)
Lit motors tests the risky part
Unfortunately, we’re all liars.
Everyone’s idea is the best right? People love this part! (but that’s not always a good thing) No data, no learning. This is where things fall apart.
Analytics can help.
Analytics is the measurement of movement towards your business goals.
In a startup, the purpose of analytics is to iterate to product/market fit before the money runs out.
I have two kids. At least one of them is a girl.
What are the chances the other is a boy?
BB BG GB GG
2 of 3 (66%) are boys. GB GG BG
Some fundamentals. Analytics, performance, aggregation, and the right metrics
Fundamental: Web analytics and the long funnel. (This is not a web analytics workshop.)
A simplistic view of web analytics ATTENTION ENGAGEMENT CONVERSION NEW VISITORS GROWTH LOSS BOUNCE RATE CONVERSION RATE x TIME GOAL VALUE ON SITE PAGES PER VISIT NUMBER OF VISITS SEARCHES TWEETS MENTIONS ADS SEEN
Visits Shopping cart Payment options Conversions
Visits Shopping cart Payment options Conversions KPIs Bounce Conversion
Unpaid search Community mentions Visits Shopping cart Payment options Conversions Email campaign Banner ad •Google PageRank •Sessions-to-clicks ratio •Cost of ads (CPM) •Clickthrough rate ? •Open rate •Opt-out rate
Repurposing (spread to other communities) Amplification (virality and message spread) Seed (starting community) Reach (impressions) Visits Shopping cart Payment options Conversions
Viral message spread Reach (impressions) Visits Shopping cart Payment options Conversions Emphasis on getting viral ratio above 1 (Retweeting, Fan, Email forward, Reddit upvote, other loops)
Megablogger proponents Seed (starting community) Reach (impressions) Visits Shopping cart Payment options Conversions Emphasis on convincing highly-followed, highly acted-upon seed group to spread the word.
A call to action Reach (impressions) Visits Shopping cart Payment options Conversions Emphasis on maximizing impression-to-click ratio within the community
Long funnel: Beers for Canada 700,000s 1,642 150 RT 32 7 10 15 x $100 x $20 x $7 Seed ratio: 20 35,000:1 Followers Visitors: 0.23% Conversions: 1.95% 2 Repurposing Amplification: 2.9% Revenues: Average: $39.54 Median: $20 Total: $1,005 2,000
Fundamental: You should care about performance.
Downtime costs Amazon offline ($1M/h) Amazon loses nearly $1M/hour if down (NYT, 2008) 1 hour of network downtime costs $42,000 (Gartner, 2003) Network downtime ($42K/h) 22h outage at eBay cost $2M ($90,909/h) (Internetnews, 1999) eBay offline ($90K/h) Financial company down ($100K/h) 53.2% of finance companies lose over $100,000/hour (nextslm.org) Let’s say $50K/h if you’re serious.
Availability % Downtime/year Loss @$50K/h 90% 36.5 days $43,800,000 95% 18.25 days $21,900,000 98% 7.30 days $8,760,000 99% 3.65 days $4,380,000 99.5% 1.83 days $2,196,000 99.8% 17.52 hours $876,000 99.9% 8.76 hours $438,000 99.95% 4.38 hours $219,000 99.99% 52.6 minutes $43,833 99.999% 5.26 minutes $4,383 99.9999% 31.5 seconds $438 Less than 1h/ year Less than a minute/year
You really don’t want web users to call you. US$12 US$10 US$7 US$5 US$2 US$0 $5.50 $3.00 $0.24 $0.45 Web self-service IVR Email Live phone Low Average High BiT Group White Paper: “Web Self-Service Lowers Call Center Costs and Improves Customer Service” Cost estimates
The login page Function will have a total latency Metric of under 4 seconds Target with a cached browser copy User situation from any US branch office Testing point 95% of the time Percentile weekdays, 8AM ET to 6M PST Time window by synth test at 5m intervals Collection type
10% visitors 7.5% of 5% Percent 2.5% 0% 0-2s 2-4s 4-6s 6-8s 8s + Average page load time across visit that commented on a post
Fundamental: What makes a good metric?
A good metric is: Understandable If you’re busy explaining the data, you won’t be busy acting on it. Comparative Comparison is context. A ratio or rate The only way to measure change and roll up the tension between two metrics (MPH) Behavior changing What will you do differently based on the results you collect?
The simplest rule If a metric won’t change how you behave, it’s a bad metric. h"p://www.flickr.com/photos/circasassy/7858155676/
Metrics help you know yourself. Acquisition Hybrid Loyalty You are just like 70% of retailers 20% of retailers 10% of retailers Customers that buy >1x in 90d Your customers will buy from you Once 2-2.5 per year >2.5 per year Then you are in this mode 1-15% 15-30% >30% Focus on Low acquisition cost, high checkout Increasing return rates, market share Loyalty, selection, inventory size (Thanks to Kevin Hillstrom for this.)
Qualitative Unstructured, anecdotal, revealing, hard to aggregate, often too positive & reassuring. Warm and fuzzy. Quantitative Numbers and stats. Hard facts, less insight, easier to analyze; often sour and disappointing. Cold and hard.
Exploratory Speculative. Tries to find unexpected or interesting insights. Source of unfair advantages. Cool. Reporting Predictable. Keeps you abreast of the normal, day-to-day operations. Can be managed by exception. Necessary.
Rumsfeld on Analytics Things we know don’t know (Or rather, Avinash Kaushik channeling Rumsfeld) we know Are facts which may be wrong and should be checked against data. we don’t know Are questions we can answer by reporting, which we should baseline & automate. we know Are intuition which we should quantify and teach to improve effectiveness, efficiency. we don’t know Are exploration which is where unfair advantage and interesting epiphanies live.
Slicing and dicing data Feb Mar Apr May 5,000 Active users 0 Jan Cohort: Comparison of similar groups along a timeline. (this is the April cohort) A/B test: Changing one thing (i.e. color) and measuring the result (i.e. revenue.) Multivariate analysis Changing several things at once to see which correlates with a result. ☀☁☀☁ Segment: Cross-sectional comparison of all people divided by some attribute (age, gender, etc.) ☀ ☁
Which of these two companies is doing better?
January February March April May Is this company Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50 growing or stagnating? Cohort 1 2 3 4 5 January $5 $3 $2 $1 $0.5 February $6 $4 $2 $1 March $7 $6 $5 April $8 $7 May $9 How about this one?
Cohort 1 2 3 4 5 January $5 $3 $2 $1 $0.5 February $6 $4 $2 $1 March $7 $6 $5 April $8 $7 May $9 Averages $7 $5 $3 $1 $0.5 Look at the same data in cohorts
Lagging Historical. Shows you how you’re doing; reports the news. Example: sales. Explaining the past. Leading Forward-looking. Number today that predicts tomorrow; reports the news. Example: pipeline. Predicting the future.
Some examples A Facebook user reaching 7 friends within 10 days of signing up (Chamath Palihapitiya) If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang) Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) A LinkedIn user getting to X connections in Y days (Elliot Schmukler) (From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
Which means it’s time to talk about correlation.
10000 1000 100 10 1 Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec Ice cream consumption Drownings
Correlated Two variables that are related (but may be dependent on something else.) Ice cream & drowning. Causal An independent variable that directly impacts a dependent one. Summertime & drowning.
A leading, causal metric is a superpower. h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
Why is Nigerian spam so badly written?
Experienced scammers expect a “strike rate” of 1 or 2 replies per 1,000 messages emailed; they expect to land 2 or 3 “Mugu” (fools) each week. One scammer boasted “When you get a reply it’s 70% sure you’ll get the money” “By sending an email that repels all but the most gullible,” says [Microsoft Researcher Corman] Herley, “the scammer gets the most promising marks to self-select, and tilts the true to false positive ratio in his favor.” This would be horribly inefficient since humans are involved. Good language (10% conversion) Not-gullible (.07% conversion) Aunshul Rege of Rutgers University, USA in 2009 1000 emails Bad language (0.1% conversion) 1-2 responses Gullible (70% conversion) 1 fool and their money, parted. 1000 emails 100 responses 1 fool and their money, parted.
Turns out the word “Nigeria” is the best way to identify promising prospects.
Nigerian spammers really understand their target market. They see past vanity metrics.
Fundamental: Be careful rolling things up.
http://upload.wikimedia.org/wikipedia/commons/0/0e/Count-von-count.jpg
0 10 20 30 40 50 60 70 80 90 Age 20 Count 0 Average age = 10
200 0 2 4 6 8 10 12 14 16 18 Page load time (in seconds) # of requests 0 20 Average latency = 5s 95th percentile latency = 19s
KISS
“It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.” http://media.photobucket.com/image/einstein/derekabril/einstein_010.png
“As simple as possible, but no simpler.” *(FYI, this is irony.)
1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s Login Checkout Invite
1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s Login Checkout Invite Average 4s Average 6s Average 9s
1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s Login Checkout Invite Average 95% 4s 8s Average 95% 6s 10s Average 95% 9s 12s
1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s Login Checkout Invite Average 95% Mode 4s 8s 2s Average 95% Mode 6s 10s 5s Average 95% Mode 9s 12s 1s
Login Checkout Invite Aggregate? Average 95% Mode 4s 8s 2s Average 95% Mode 6s 10s 5s Average 95% Mode 9s 12s 1s Average 95% Mode 6s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 5s
1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s Login: <=4s 740 260
1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s Total samples 1000 Below threshold 740 Login: <=4s Percent below target threshold 74% 740 260
1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s Login: <=4s Checkout: <=5s Invite: <=8s Total samples 1000 Below threshold 740 Percent below target threshold 74% Total samples 1000 Below threshold 370 Percent below target threshold 37% Total samples 1000 Below threshold 610 Percent below target threshold 61% 740 260 370 630 610 390
1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s Login: <=4s Checkout: <=5s Invite: <=8s Aggregate? Total samples 1000 Below threshold 740 Percent below target threshold 74% Total samples 1000 Below threshold 370 Percent below target threshold 37% Total samples 1000 Below threshold 610 Percent below target threshold 61% 740 260 370 630 610 390 Total samples 3000 Below threshold 1720 Percent below target threshold 57%
1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s Login: <=4s Checkout: <=5s Invite: <=8s Total samples 1000 Below threshold 740 Percent below target threshold 74% Total samples 400 Below threshold 148 Percent below target threshold 37% Total samples 600 Below threshold 366 Percent below target threshold 61% 740 260 370 630 610 390 Total samples 2000 Below threshold 1254 Percent below target threshold 63% 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s
1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s Weight 1 Weight 5 Weight 2 Login: <=4s Checkout: <=5s Invite: <=8s Total samples 1000 Below threshold 740 Total samples 400 Below threshold 148 Total samples 600 Below threshold 366 740 260 370 630 610 390
1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s Total requests inside target Login page 740/1000 Checkout page 148/400 Invite process 366/600
Total requests inside target Login page 740/1000 Checkout page 148/400 Invite process 366/600 Weight 1 52 Weighted 740/1000 740/2000 732/1200 2212/4200 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s
Total requests inside target Login page 740/1000 Checkout page 148/400 Invite process 366/600 Weight 1 52 Weighted 740/1000 740/2000 732/1200 Total score 2212/4200 53% 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s 1s 2s 3s 4s 5s 6s 7s 8s 9s 10s 11s 12s
The Lean Analytics framework.
Eric’s three engines of growth Virality Make people invite friends. How many they tell, how fast they tell them. Price Spend money to get customers. Customers are worth more than they cost. Stickiness Keep people coming back. Approach Get customers faster than you lose them. Math that matters
Dave’s Pirate Metrics AARRR Acquisition How do your users become aware of you? SEO, SEM, widgets, email, PR, campaigns, blogs ... Activation Do drive-by visitors subscribe, use, etc? Features, design, tone, compensation, affirmation ... Retention Does a one-time user become engaged? Notifications, alerts, reminders, emails, updates... Revenue Do you make money from user activity? Transactions, clicks, subscriptions, DLC, analytics... Referral Do users promote your product? Email, widgets, campaigns, likes, RTs, affiliates...
Gate Stage EMPATHY I’ve found a real, poorly-met need that a reachable market faces. STICKINESS I’ve figured out how to solve the problem in a way they will keep using and pay for. VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives. REVENUE The users and features fuel growth organically and artificially. SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem. The five stages
Empathy stage: Localmind hacks Twitter Needed to find out if a core assumption—strangers answering questions—was valid. Ran Twitter experiment instead of writing code Asked senders of geolocated Tweets from Times Square random questions; counted response rate Conclusion: high enough to proceed
LikeBright’s mechanical turk Used Mechanical Turk, Google Voice to speak w/ 100 single women; paid $2. The interviews lasted typically around 10-15 minutes. Simple interview script with open-ended questions, since he was digging into the problem validation stage of his startup. Founder Nick Soman: “I was amazed at the feedback I got. We were able to speak with one hundred single women that met our criteria in four hours on one evening.” Went back to TechStars and got accepted. LikeBright’s website is now live with a 50% female user base, and recently raised a round of funding. “Since that first foray into interviewing customers, I’ve probably spoken with over a thousand people through Mechanical Turk,”
How to avoid leading the witness Avoid biased wording, preconceptions, or a giveaway appearance. Word your surveys carefully to be neutral. Get them to purchase. Ask them to pay. Demand real introductions. Or ask them “how many of your friends would say Ask “why” several times. Leave lingering, uncomfortable pauses in the conversation and let them fill them. Don’t tip your hand Make the question real Keep digging Look for other clues Have a colleague make notes of when they react, or of their body language.
Stickiness stage: qidiq streamlines invites Survey owner adds recipient to group Survey owner asks question Recipient reads survey question Recipient responds to question Recipient sees survey results (Later, if needed…) Recipient visits site; no password! Recipient does password recovery One-time link sent to email Recipient creates password Recipient can edit profile, etc. Survey owner adds recipient to group Survey owner asks question Recipient gets invite Recipient installs mobile app Recipient creates account, profile Recipient can edit profile, etc. Recipient reads survey question Recipient responds to question Recipient sees survey results 10-25% RESPONSE RATE 70-90% RESPONSE RATE
1200 1000 800 600 400 200 0 January February 1 2 3 4 5 6 7 8 9 Days since last engagement 25000 20000 15000 10000 5000 0 Disengaged (>10 days) Number of users A better approach to engagement This is a good thing.
Virality stage: Timehop focuses on content sharing Focused on percent of daily active users that share their content Aiming for 20-30% of DAU sharing “All that matters now is virality. Everything else—be it press, publicity stunts or something else—is like pushing a rock up a mountain: it will never scale. But being viral will.” - Jonathan Wegener, co-founder
------------------------------------------------------ Get your free private email at http://www.hotmail.com ------------------------------------------------------
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference
v ≠ 1, pt = δp0 (1 – vt+1) / (1 – v) + p0 http://robert.zubek.net/blog/2008/01/30/viral-coefficient-calculation/ Viral coefficient
Or simpler x - > 1 Users Viral coefficient Churn & abandonment
How to calculate it First calculate the invitation rate, which is the number of invites sent divided by the number of users you have. Then calculate the acceptance rate, which is the number of signups or enrollments divided by the number of invites. Then multiply the two together. Your 2,000 customers have sent out 5,000 invitations during their lifetime on your site. Your invitation rate is 2.5. For every ten invitations received, one gets clicked. Your acceptance rate is 0.1. Multiply the two, and you have your viral coefficient: 0.25. Every customer you add will add an addition 25% of a customer.
Revenue stage: Backupify’s Customer Acquisition Payback Initially focused on site visitors Then focused on trials Then switched to signups Today, MRR In early 2010, CAC was $243 and ARPU was only $39 Pivoted to target business users CLV-to-CAC today is 5-6x Now they track Customer Acquisition Payback Target is less than 12 months
Scale stage: Incremental order cost Marginal costs Fixed costs
Six business model archetypes. E-commerce SaaS Mobile Media app User-gen content 2-sided market The business you’re in
(Which means eye charts like these.) Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Freemium churn Cancel Engaged User Free user disengagement Reactivate Trial abandonment Cancel rate Invite Others Upselling rate Upselling Paying Customer Reactivation rate Paid conversion FORMER USERS User Lifetime Value Reactivate Capacity Limit Support data FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Trial Over Disengaged Dissatisfied
Model + Stage = One Metric That Matters. The business you’re in E-Com SaaS Mobile 2-Sided Media UCG One Metric That Matters. Empathy Stickiness Virality Revenue Scale The stage you’re at
Really? Just one?
Yes, one.
In a startup, focus is hard to achieve.
Having only one metric addresses this problem.
www.theeastsiderla.com
Moz cuts down on metrics SaaS-based SEO toolkit in the scale stage. Focused on net adds. Was a marketing campaign successful? Were customer complaints lowered? Was a product upgrade valuable? Net adds up: Can we acquire more valuable customers? What product features can increase engagement? Can we improve customer support? Net adds flat: Are the new customers not the right segment? Did a marketing campaign fail? Did a product upgrade fail somehow? Is customer support falling apart? Net adds down:
Metrics are like squeeze toys. http://www.flickr.com/photos/connortarter/4791605202/
Empathy Stickiness Virality Revenue Scale E-commerce Mobile app User-gen content SaaS Media 2-sided market Interviews; qualitative results; quantitative scoring; surveys Loyalty, conversion CAC, shares, reactivation (Money from transactions) Transaction, CLV Affiliates, white-label Engagement, churn Inherent virality, CAC (Money from active users) Upselling, CAC, CLV API, magic #, mktplace Content, spam Invites, sharing (Money from ad clicks) Ads, donations Analytics, user data Inventory, listings SEM, sharing Transactions, commission Other verticals Downloads, churn, virality WoM, app ratings, CAC CLV, ARPDAU Spinoffs, publishers Traffic, visits, returns Content virality, SEM CPE, affiliate %, eyeballs Syndication, licenses
Better: bit.ly/BigLeanTable
Drawing some lines in the sand.
A company loses a quarter of its customers every year. Is this good or bad?
Not knowing what normal is makes you do stupid things.
Baseline: 5-7% growth a week “A good growth rate during YC is 5-7% a week,” he says. “If you can hit 10% a week you're doing exceptionally well. If you can only manage 1%, it's a sign you haven't yet figured out what you're doing.” At revenue stage, measure growth in revenue. Before that, measure growth in active users. Paul Graham, Y Combinator • Are there enough people who really care enough to sustain a 5% growth rate? • Don’t strive for a 5% growth at the expense of really understanding your customers and building a meaningful solution • Once you’re a pre-revenue startup at or near product/market fit, you should have 5% growth of active users each week • Once you’re generating revenues, they should grow at 5% a week
Baseline: 10% visitor engagement/day 30% of users/month use web or mobile app 10% of users/day use web or mobile app 1%of users/day use it concurrently Fred Wilson’s social ratios
Baseline: 2-5% monthly churn • The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s on the job. • Get below a 5% monthly churn rate before you know you’ve got a business that’s ready to grow (Mark MacLeod) and around 2% before you really step on the gas (David Skok) • Last-ditch appeals and reactivation can have a big impact. Facebook’s “don’t leave” reduces attrition by 7%.
Baseline: Calculating customer lifetime 25% 5% monthly churn monthly churn 100/25=4 100/5=20 The average The average customer lasts customer lasts 4 months 20 months 2% monthly churn 100/2=50 The average customer lasts 50 months
Baseline: CAC under 1/3 of CLV • CLV is wrong. CAC Is probably wrong, too. • Time kills all plans: It’ll take a long time to find out whether your churn and revenue projections are right • Cashflow: You’re basically “loaning” the customer money between acquisition and CLV. • It keeps you honest: Limiting yourself to a CAC of only a third of your CLV will forces you to verify costs sooner. Lifetime of 20 mo. $30/mo. per customer $600 CLV 1/3 spend $200 CAC Now segment those users!
Who is worth more? A Lifetime: B Lifetime: Today $200 $200 Roberto Medri, Etsy Visits
Baseline: 35% of mobile users engaged by day 90 Day one Day 30 Day 60 100% 54% 43% Day 90 35% October, 2012 study of 200,000 apps by Flurry In recent years, third-month engagement has increased from 25% to 35%, but frequency of use has dropped from 6.7 uses a week to 3.7 a week. Smartphone Tablet Uses per week 12.9 times 9.5 times Duration of use 4.1m 8.2m
(Varies widely by product category)
Etsy • Online store for creative types, founded 2005 • $525M Gross Merchandise Sales in 2011, with 19,000,000 members and 800,000 active shops offering 15,000,000 items for sale • 1.4B pageviews per month ~2M iPhone app downloads • Thin revenues: Etsy makes only $0.20 or 3.5% margin • Heavy focus on Customer Lifetime Value (buyer and seller) • Actually residual lifetime value; they take this pretty seriously.
Etsy • The best customers to target are • Recent high-profile customers • Old-time best customers about to churn or just churned • Tiered campaigns • Bronze/silver customers: reinforcement, nudges • Gold customers: premium services • Platinum customers: recognition • What they watch: • Growth of individual product categories • Time to first sale by a user • Average order value • Percentage of visits that convert to a sale • Percentage of return buyers • Distinct sellers within a product category • Time-to-first-sale and average order value by product category Roberto Medri, Etsy
DuProprio/Comfree • Large for-sale-by-owner marketplace • Founded in 1997, 17,000 properties and 5M visits a month • $900 per listing, plus value-added tools & services • Leading goal is to create subscriptions • Launched seller-side logins; then client accounts • Rule of thumb: 1,000 visits equals 1 subscription • Three business objectives: • Convince sellers to list their property on the site; • Convince buyers to register for property match notifications; • Sell the properties. KPI evolution Static traffic Visitor to listing ratio List-to-sold ratio Click-throughs, search results
YPG • Large directory publishing & local marketing w/420K customers, 2,500 employees, and $1,2B/y in revenue • Focus on public API for listings (1.5M geo-coded listings for location apps) • Initially slow to embrace API, but in 2013 have tripled investment • Lets the company find a partner or developer and have a functional prototype in hours, testing in days, and launching in weeks. KPI evolution Soft: Signups, SDK, downloads App usage, deals signed API calls generated API-generated revenue
The Lean Analytics cycle
Pick a KPI Draw a line Draw a new line Pivot or give up Try again Success! Did we move the needle? Measure the results Design a test Make changes in production Find a potential improvement With data: find a commonality Without data: make a good guess Hypothesis
Do AirBnB hosts get more business if their property is professionally photographed?
Gut instinct (hypothesis) Professional photography helps AirBnB’s business Candidate solution (MVP) 20 field photographers posing as employees Measure the results Compare photographed listings to a control group Make a decision Launch photography as a new feature for all hosts
5,000 shoots per month by February 2012
Hang on a second.
SRSLY? Gut instinct (hypothesis) Professional photography helps AirBnB’s business
Pick a KPI Draw a line Draw a new line Pivot or give up Try again Success! Did we move the needle? Measure the results Design a test Make changes in production Find a potential improvement With data: find a commonality Without data: make a good guess Hypothesis
“Gee, those houses that do well look really nice.” Maybe it’s the camera. With data: find a commonality “Computer: What do all the highly rented houses have in common?” Camera model. Without data: make a good guess
Circle of Moms: Not enough engagement • Too few people were actually using the product • Less than 20% of any circles had any activity after their initial creation • A few million monthly uniques from 10M registered users, but no sustained traction • They found moms were far more engaged • Their messages to one another were on average 50% longer • They were 115% more likely to attach a picture to a post they wrote • They were 110% more likely to engage in a threaded (i.e. deep) conversation • Circle owners’ friends were 50% more likely to engage with the circle • They were 75% more likely to click on Facebook notifications • They were 180% more likely to click on Facebook news feed items • They were 60% more likely to accept invitations to the app • Pivoted to the new market, including a name change • By late 2009, 4.5M users and strong engagement • Sold to Sugar, inc. in early 2012
Landing page design A/B testing Cohort analysis General analytics URL shortening Funnel analytics Influencer Marketing Publisher analytics SaaS analytics Gaming analytics User analytics Spying on users User interaction Customer User segmentation satisfaction KPI dashboards
Consider a media company
Valuable, hard Paid revenue sources Word of mouth; public support Why clickbait is on its way out Simple to count Icky, easy What do you want visitors to do? The changing face of engagement Ignore Back away Bounce One-time Lurk Stay silent Hoard Criticise Take Abandon Cancel Upgrade Renew Subscribe Create Endorse Share Respond Interact Explore Stay See Click Downgrade
The tools media can use Editorial decisions Pagerank/reputation Followers/subscribers Topic chosen Format (quiz, story, etc.) Tone (controversy, etc.) Headline, imagery Timing, platform Long-term, sustainable Short-term, transient
Modern media’s new gauntlet Click See Stay Explore Interact Respond Share Endorse Create Subscribe Renew Upgrade Editorial decisions Pagerank/reputation Followers/subscribers Topic chosen Format (quiz, story, etc.) Tone (controversy, etc.) Headline, imagery Timing, platform From here (cats and royalty) To here (an informed electorate and citizen approval)
Some non-tech examples.
I lied. Everyone is a tech company.
Cost of experiments: down. Cost of attention: way up. http://www.flickr.com/photos/puuikibeach/4789015423 http://www.flickr.com/photos/elcapitanbsc/3936927326
Let’s pick on restaurants for a while.
Empathy: find the need Before opening, the owner first learns about the diners in her area, their desires, what foods aren’t available, and trends in eating. Key metrics: Popular items; frequent questions; before/after dining patterns. Reference: Emerging need.
Stickiness: confirm the need is met. She develops a menu and tests it out with consumers, making frequent adjustments until tables are full and patrons return regularly. She’s giving things away, asking diners what they think. Variance and uncertain inventory make costs high. Key metrics: Customer loyalty; recommendations; referrals; endorsements; inventory turnover. Reference: Business idea.
Virality: will it spread? She starts loyalty programs to bring frequent diners back, or to encourage people to share with their friends. She engages on Yelp and Foursquare. Key metrics: Customer loyalty; recommendations; referrals; endorsements. Reference: Business positioning
Revenue: prove the business viability With virality kicked off, she works on margins—fewer free meals, tighter controls on costs, more standardization. She focuses on the price of acquiring new customers. Key metrics: Acquisition cost, revenue per cover, capacity, turnover. Reference: Business model.
Scale: prove it’s a market Knowing she can run a profitable business, she funnels revenues into marketing and promotion. She reaches out to food reviewers, travel magazines, and radio stations. She launches a second restaurant, or a franchise. Key metrics: Franchise health; repeatability; problems escalated; variance; franchise revenues. Reference: Business plan.
A line in the sand Labor costs Gross revenue 30% 24% 20% Too costly? Just right Understaffed? =
A leading indicator 50 reservations at 5PM 250 covers that night (Varies by restaurant. McDonalds ≠ Fat Duck.) http://www.flickr.com/photos/mysticcountry/3567440970 http://www.flickr.com/photos/avlxyz/4889656453
http://www.flickr.com/photos/southbeachcars/6892880699 Restaurant MVP
Is tip amount a leading indicator of long-term revenue?
Why does every table get the same menu?
Is tipping even a good idea? Customers like Servers like tipping tipping because it puts because it means their them in the driver’s talent is rewarded. As a seat. As a diner, you great server, you get control your paid more than your experience, using the peers, because you are power of your tip to a better worker. make sure your server works hard for you. Owners like tipping because it means they don’t have to pay for managers to closely supervise their servers. With customers using tips to enforce good service, owners can be confident that servers will do their best work. Is this true? How would you know? Jay Porter founded the Linkery, San Diego's leading farm-to-table restaurant.
The truth about tips Customers don’t vary tips much according to service. Tipped servers are rewarded for maximizing the number of guests they serve, even though that degrades service quality. Servers learn to profile guests, focusing on stereotypes while giving women, ethnic minorities, the elderly and those from foreign countries bad experiences When a server is punished, the server can keep that information to himself. The message never makes it to a manager, and the problem is never addressed.
The truth about tips Sharp increase in business over the first two months of the new system: Servers’ total pay rose to about $22/hour Most of the cooks started making about $12-14 depending on experience The dishwashers about $10 http://jayporter.com/dispatches/observations-from-a-tipless-restaurant-part-1-overview/
Is purple ink better? http://tippingresearch.com/uploads/managing_tips.pdf
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference
Stalking customers is pretty easy. http://targetmycustomers.appspot.com http://tippingresearch.com/uploads/managing_tips.pdf
Growth hacking (is a word you should hate but will hear a lot about.)
Growth hacking, demystified. Find correlation Test causality Optimize the causal factor Pick a metric to change
http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/ Is social action a leading indicator of donation?
http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/ Is mobile use?
Guerrilla marketing Data-driven learning GROWTH HACKING Subversiveness
The growth hack •Growth hacking is simply what marketing should have been doing, but it fell in love with Don Draper and opinions along the way •Optimize a factor you think is correlated with growth
AirBnB and Craigslist
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference
What if you’re an Intrapreneur?
Business model vs. company stage Early stage Big/incumbent Company size/age B2B Target market B2C More formal decisions Less WoM Slower cycle time More legacy constraints It is way too easy to mix these up. Intrapreneurs
When you’re a startup your goal is to find a sustainable, repeatable business model. When you’re a big company your goal is to perpetuate one.
Intrapreneur: Someone working to produce disruptive change in an organization that has already found a sustainable, repeatable business model.
In a startup, the purpose of analytics is to iterate to product/market fit before the money runs out.
In a big company, analytics replaces opinion with fact.
(Before we get into Lean Analytics, 2 key lessons.) Lesson one: Companies die because they fail to move to new business models.
Cost per MB $1 $10 $100 $1000 Clay Christensen, The Innovator’s Dilemma Time 14” Mainframe 8” Minicomputer 5.25” Desktop 3.5” Notebook
Technologies outstrip what the market needs, driven by feedback from the “best” current customer. $1 $10 $100 $1000 Clay Christensen, The Innovator’s Dilemma 8” 5.25” Time High end customer Low end customer
The new market has different criteria for success, which are uninteresting to incumbents. $1 $10 $100 $1000 Clay Christensen, The Innovator’s Dilemma Time Storage capacity Portability
Sometimes this has unintended consequences $1 $10 $100 $1000 Clay Christensen, The Innovator’s Dilemma Smaller disc size means less vibration impact, leading to greater density, increasing storage capacity Time
Three kinds of innovation Sustain/core (optimizing for more of the same) Innovate/adjacent (introduce nearby product, market, or method) Disrupt/transformative (Fundamentally changing the business model) Improve along current metrics... ...or alter the rate of improvement Switch to a new value model Change the business model entirely
Lesson two: The difference between a rogue agent and a special operative is permission.
It’s different when you’re big.
If big firms can’t innovate, it’s this guy’s fault.
When product and market are known, companies compete on how they do things.
To get the incremental cost to zero, companies competed on scale. (Literally, an economy of scale)
Scale comes from process, IP, org chart, capitalization. All of these assume the future will be like the past, only more so.
If a startup is an organization designed to search for a sustainable, repeatable business model, then an established company is an organization designed to perpetuate one.
Technology has radically changed the incremental cost of businesses.
Software is eating the world. http://www.flickr.com/photos/ebolasmallpox/3733059220/
An economic order quantity of one. Crafted Mass-produced Automated Digital Quantity Few Many Some One Cost High Low Medium Free Lead time Small Large Medium None Self-service Medium None Some Lots Customization High None Some Lots This is why software is eating the world. • Cloud computing • Social media • 3D printing • Per-customer analysis • Mobile tracking • Etc...
Sustainable competitive advantage allows for inertia and power to build up along the lines of an existing business model, which will soon die. Instead, seek transient competitive advantage. Rita Gunther McGrath, The End of Competitive Advantage
Scale is now a liability. Compete on cycle time.
CW&T make a pen
http://www.flickr.com/photos/art_es_anna/288880795/
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference
Optimizing the probable means discounting the possible.
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference
This isn’t about a lack of resources.
http://www.flickr.com/photos/maladjusted/5207565912
Blockbuster had a lot going for it.
Plenty of inventory, of course. But that matters less than...
...market intelligence, customers, existing payment approval, and customer history.
The problem was framing: Blockbuster thought it was in the video store management business. Netflix realized it was in the entertainment delivery business.
YOU ARE HERE
LOCAL MAXIMUM YOU ARE HERE OPTIMIZATION OF CURRENT METRICS
YOU ARE HERE GLOBAL INNOVATION MAXIMUM WITH NEW RULES
YOU ARE HERE SHORT-TERM INVESTORS HATE GOING DOWNHILL
First mover advantage happens long before the market emerges. • $1B invested in Nook • $475M operating loss in April 2013 • CEO gone
Constraints slow things down vs.
Capital cycles don’t fit the short, iterative nature of startup uncertainty 12 month budgeting cycle; annual plan. Future based on past. Agile, scrum, lean iterations. Today’s model. No evidence about the future. Project Project Project Project Project Project Project Project Project Project Project Project (Requires budget insulation)
Everything to lose: Why big companies need innovation.
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference
F500 Life Expectancy (http://csinvesting.org/2012/01/06/fortune-500-extinction/) 75 years 15 years 1950 ... 2010 Growth by entering a new business 95 % fail Corporate Strategy Board 99 % fail Clay Christensen
mikemace.com The slow death of a market leader. “Time to enter the mainstream. Cut prices.” “Let’s cut prices to accelerate our growth.” “We may miss the quarter. Let’s do a price promotion.” “That wasn’t supposed to happen. We’ll have to lay some people off. Revenue over time This is what most managers track. Note that sales keep rising (making you feel safe) until you run off the edge of the cliff. The adoption curve Here’s where you actually are, but you don’t know it because you can’t draw the curve until after the market saturates. Early adopters Late adopters Gross margin percent Declining profit per unit (gross margin) is actually your best signal of trouble.
In other words, if your job is change you have your work cut out for you.
A dose of pragmatism
Many models for enterprise innovation Core Adjacent Transformative Do the same Nearby product, Start something thing better. market, or method. entirely new. Regional optimizations. Innovation, go-to-market strategies. Reinvent the business model. • Get there faster • Smaller batches • Solution, then testing • Increased accountability • Customer development • Test similar cases • Parallel deployment • Analytics & cycle time • Fail fast • Skunkworks/R&D • Focus on the search • Ignore the current model & margins
Another way to look at it Core Adjacent Transformative Know the problem (customers tell you it) Know the solution (customers/regulations/ norms dictate it.) Know the problem (market analysis) Don’t know the solution (non-obvious innovation confers competitive advantage.) Don’t know the problem (just an emerging need/ change) Don’t know the solution. Waterfall: Execution matters Agile/scrum: Iteration matters Lean Startup: Discovery matters
The Three Horizons Core Adjacent Transformative Those core businesses most readily identified with the company name and those that provide the greatest profits and cash flow. Maximize remaining value. Emerging opportunities, including rising entrepreneurial ventures likely to generate substantial profits in the future but that could require considerable investment. Ideas for profitable growth down the road—for instance, small ventures such as research projects, pilot programs, or minority stakes in new businesses. Horizon 1 improves the current business operations in the next 12 months. Horizon 2 extends the business into new products, markets, or methods in the next 3 years. Horizon 3 changes the industry you’re in and your value network in the next 6 years. http://www.mckinsey.com/insights/strategy/enduring_ideas_the_three_horizons_of_growth
Lean applies. Startup may not. Core Adjacent Transformative Lean methodologies. Lean startup
Experiment with product, market, and method.
Method (new “how”) 3 kinds of innovation Product (new “what”) Market (new “who”)
Product (new “what”) Market (new “who”) Method (new “how”) Startup Distribution innovation Market diversification Core More things to more people for more money more often more efficiently. (Zyman) Innovative Change what you sell, or who you sell to, or how you sell it. Transformative Fundamentally change business and value proposition. Rebrand & cannibalize. Channel expansion Disruptive Create what wasn’t possible, based on massive societal or technical change.
Engine as a service
Engine as a service http://www.nasa.gov/images/content/365835main_airplane_noise_qtd2_3024x2016.jpg
“Efficiency is tied to analytics. We’ll still look for new materials, or for the physics of devices, but the analytics ... is what’s really untapped.”
Business optimization (five mores) Current state Product, market, method innovation Business model innovation You can convince executives of this because some of it is familiar. This terrifies them because it eats the current business. A three-maxima model of enterprise innovation
Improvement Adjacency Remodeling Do the same, Explore what’s only better. nearby quickly Try out new business models Lean approaches apply, but the metrics vary widely. Sustain/ core Innovate/ adjacent Disrupt/ transformative
Sustaining Adjacent Disruptive Next year’s car Electric car, same dealer On-demand, app-based car service
Sustaining innovation is about more of the same. (says Sergio Zyman) More things To more people For more money More often Inventory increase Gifting, wish lists Highly viral offering Low incremental order costs Maximum shopping cart Price skimming/tiering Loyal customer base that returns Demand prediction, notification More efficiently Supply chain optimization Per-transaction cost reduction
Blizzard extends the lifespan of WOW Early adopters Rapid growth Market saturation The infamous S-curve (Product lifecycle, Bass diffusion curve, etc.)
Blizzard extends the lifespan of WOW
Blizzard extends the lifespan of WOW Fixing this: sustaining growth with novelty Product & market innovation (“New & improved!”)
Blizzard extends the lifespan of WOW WOW Wrath of the Lich King Burning Crusade Mists of Cataclysm Pandaria
Most of your innovation will be adjacent or sustaining. Question marks! (low market share, high growth rate) May be the next big thing. Consumes investment, but will require money to increase market share. Stars! (high growth rate, high market share) What everyone wants. As market invariably stops growing, should become cash cows. Dogs! (low market share, low growth rate) Barely breaks even, may be a distraction from better opportunities. Sell off or shut down. Pivot to increase growth rate through disruption Cash cows! (high market share, low growth rate) Boring sources of cash, to be milked but not worth additional investment. Growth rate Pivot to increase market share through virality, attention Market share Pivot to redefine problem/ solution through empathy Milk with revenue optimization as growth slows If you don’t like this, go launch a startup.
Software, experimentation, and iterative cycles of learning help you get to the local maximum better and faster. That’s a good thing. But it’s not the only thing.
Adjacent innovation is about changing one part of the model in a way that alters the value network.
Amazon Web Services and the server value network Server computing • Density • GHz • Heat • MIPS Cloud computing • Instances • Objects • Spinup time • Scaleout Capex, financing, TCO, ROI Opex, demand, time to result CIO, enterprise IT CTO, coder, app owner, line of business, startup Value criteria Money Buyer
Adjacent product to the same market in the same way
Selling the same product to an adjacent market in the same way. Of P&G’s 38 brands, only 19 were sold in Asia as of 2011 Market expansion is seldom selling the same thing to new people. In Asia, P&G needed to Align pricing with novelty (prestige, mass-tige, over-the-counter) Change consumer expectations (moving from dilutes to concentrates) Adjust positioning and ingredients such as white fungus, ginseng, and the parasitic cordyceps
Selling the same product to the same market in a new way. The biggest innovation in logistics of the 20th century. http://www.flickr.com/photos/photohome_uk/1494590209
Changing the method of C2C classifieds A blend of who, what, how Classified C2C sales (same “what”) Strictly for Japanese women (targeted “how”) New how (phone is capture, display, payment, transaction) Did 100 interviews w/target users before launch Key insight: Japanese women sell their entire wardrobe twice in their lives 5,000 and 10,000 sales in first month 10% commission fee Average price of items is pretty low, at around 2,000 to 3,000 yen (or $22 to $34) Not an auction: seller decides price Mobile-only model Phone is payment, storefront, and even a way for sellers to build their catalog http://www.sffashtech.com/2012/10/10/a-free-market-fashion-app-exclusively-for-women-japan/
Selling the same product to the same market in a new way.
(At this point, observant Intrapreneurs should be asking, should P&G be in the house cleaning business? And that would be transformative.)
Transformative innovation is about taking a leap, changing more than one dimension simultaneously in search of a new business model.
If sustaining, incremental innovation produces linear growth, then disruptive, transformative innovation produces exponential growth.
Transformative isolation: Skunkworks
Transformative incubation: Metlife Infinity
Transformative incubation: Taser evidence.com Significant market 850K full-time law enforcement officers in the US; 700K state/local; 525K patrol officers 130M incident reports/y. 70M new incidents; 200K involve use of force Only 31% of local police agencies keep computer files on use-of-force incidents Strong product benefits Exonerates the officer 96% of the time. 47% percent increase in charges and summons (2007) Patrol officers spend 15-25% of their time writing incident reports, recorded evidence reduces this by 22%, meaning 50m more on patrol Challenges New business model Pricing unclear SaaS offering Compliance and governance Unions, regulation, chain of evidence Changing the current model (radio is everything)
When it’s you vs. the world. (A bagful of tricks from agitators in companies of all sizes.)
The Lean Analytics lifecycle of an Intrapreneur Beforehand Get buy-in Political fallout Empathy Find problems; don’t test demand. Skip the business case, do analytics Entitled, aggrieved customers Stickiness Know your real minimum based on expectations, regulations Hidden “must haves”, feature creep Virality Build inherent virality in from the start; attention is the new currency Luddites who don’t understand sharing Revenue Consider the ecosystem, channels, and established agreements Channel conflict, resistance, contracts Scale Hand the baton to others gracefully Hating what happens to your baby
The Zero Overhead principle A central theme to this new wave of innovation is the application of core product tenets from the consumer space to the enterprise. In particular, a universal lesson that I keep sharing with all entrepreneurs building for the enterprise is the Zero Overhead Principle: no feature may add training costs to the user. DJ Patil
The job of an intrapreneur is to identify an adjacent market, product, or method that conforms to organizational filters. It is not to improve the current product, market, or method.
Also: a pariah. Successful innovators share certain attributes. Bad listener: Wilfully ignore feedback from your best customers. Cannibal: If successful, destroying existing revenue streams. Job killer: Automation & lower margins are your favorite tools. Security risk: Advocate of transparency, open data, communities. Narcissist: Worry constantly about how you’ll get attention. Slum lord: Sell to those with less money, deviants, and weirdos.
The six habits of highly unrealistic leaders Bad leaders: Filtered information Selective hearing Wishful thinking Fear Emotional overinvestment, Unrealistic expectations from capital markets Good Intrapreneurs: Access to the real information Go where the data takes you Set aside your assumptions Embrace uncertainty Surgical detachment Have high standards with low expectations Confronting Reality (Crown Business), Larry Bossidy and Ram Charan
Know what kind of innovation you’re after. New Current Market development: Sell existing products to new markets, segments, uses. Export & license. Startup: New products for new markets. New rules, business units, organizational structure. Innovation. Current New Market Product Penetrate: Increase revenues, market share, product quality, brand differentiation. Marketing. Product development: Invent new products for your market. R&D, enhancements. Acquisition. Based on H. Igor Ansoff’s matrix Increased risk of political fallout (and great success!)
Use outliers and missed searches to hunt for good ideas & adjacencies 1/8 men have an incontinence issue. 1/3 women do. When search results show a significant number of men searching, this suggests the adjacent (male) market is underserved. (Multi-billion-dollar hygiene product company)
Frame it like a study Product creation is almost accidental. Unlike a VC or startup, when the initiative fails the organization still learns. http://www.flickr.com/photos/creative_tools/8544475139
When in doubt, collect data From tackling the FTA rate to visualizing the criminal justice supply chain.
Use data to create a taste for data Sitting on Billions of rows of transactional data David Boyle ran 1M online surveys Once the value was obvious to management, got license to dig.
Smart Badge 4” e-ink display with name and specialty. Badge scans barcode and gets specs; checks inventory; enters data on a touch screen. Data Exhaust Today: Workers see their own productivity. Coming soon: comparing yourself to 400,000 other employees. Ultimately: Learning what (and who) works well. Tesco connects its workforce
Don’t just collect data, chase it.
Understand hidden constraints That pencil story is a myth. Graphite is conductive and explosive. The Minimum Viable Product is Viable for a reason.
Know what has to be built in-house SAP integration Employment law
http://www.flickr.com/photos/bootbearwdc/1243690099/ Think subversively.
Everything’s an excuse to experiment
Find other ways to collect data; everything is an experiment.
Run it as a consulting business first. (Just don’t get addicted to it. Your goal is to learn and overcome integration challenges and find the 20% of features that 80% of the market will pay for.)
Convince your boss she asked for this Draw a line in the sand Pick a KPI Draw a new line Pivot or give up Try again Success! Did we move the needle? Measure the results Design a test Make changes in production Find a potential improvement With data: find a commonality Without data: make a good guess Hypothesis
Focus on the desired behavior, not just the information. 26% increase in towel re-use with an appeal to social norms; 33% increase when tied to the specific room. http://www.psychologytoday.com/blog/yes/ 200808/changing-minds-and-changing-towels The effectiveness of energy conservation “nudges” depends on an individual’s political ideology ... Conservatives who learn that their consumption is less than their neighbors’ “boomerang” whereas liberals reduce their consumption. Energy Conservation “Nudges” and Environmentalist Ideology: Evidence from a Randomized Residential Electricity Field Experiment - Costa & Kahn 2011
Slaughter a sacred cow: Prove a long-held assumption is wrong and you’ve got people’s attention. Know what you’ll do with it ahead of time.
Take baby steps.
Netflix
Tesla http://www.hdwallpapersinn.com/wp-content/uploads/2012/12/600-tesla.jpg
Twitter’s 140-character limit isn’t arbitrary. It’s constrained by the size http://i.i.cbsi.com/cnwk.1d/i/tim/2011/11/18/ sms_screen_twitter_activity_stream_270x405.png
Figure out how to translate it back to a simple model that fits the company’s existing value model. If your company dies, this is why.
Intrapreneurs often have to use proxies Stage Startup metrics Intrapreneur metrics Empathy Customers interviewed (needs & solutions), assumptions quantified, TAM, monetization possibility Non-customers interviewed; assumptions quantified, constraints identified, TAM, disruption potential Stickiness Churn, engagement Support tickets, integration time, call center data, delays Virality Viral coefficient, viral cycle time Net Promoter Score, referrals, case study willingness Revenue Attention, engagement Billable activity; signed LOIs; pilot programs; after-development profitability Scale Automation Contribution, training costs, licensing
When you have support. (What companies like P&G, Cognizant, GE, and Motorola do with a formal innovation program.)
Do you really have permission? What resources do you have? Staff, budget, unfettered access to customers? What scope of change can you make? Pricing, product, channel, branding?
2011 MIT study of 179 large publicly traded firms Companies that use data-driven analytics instead of intuition have 5%-6% higher productivity and profits than competitors. Brynjolfsson, Erik, Lorin Hitt, and Heekyung Kim. "Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance?." Available at SSRN 1819486 (2011).
The fundamental shift Ask question Define schema Collect data Answer question Refine problem Collect data Ask question Emergent schema Explore data Answer question “Collect first; ask questions later.”
What kind of mandate do you have?
Step one: Develop a portfolio approach.
Innovation portfolios at big companies Core Adjacent Transformative 70% 20% 10% Investment 10% 20% 70% Return
Organizations’ structures emerge as a way to optimize the current business model. Most innovations will come not from product or market, but from method— business model innovation. Innovation groups must exercise organizational amnesia at the outset. 1. 2. 3.
Tomorrow’s company: Running parallel businesses Innovation Sustaining/core Adjacent Transformative/ disruptive Core action Optimizing/ improving Experimenting Searching/ inventing Focus on Known metrics Risk removal Assumption validation Which will live Within current business unit Incubated, then integrated As new/separate entities Problem is Known Known Unknown Solution is Known Unknown Unknown
Step one: Frame your problems.
(See also: Christensen’s Jobs to be done.)
Step two: Define your gates and filters. These may lead to myopia. They are also your unfair advantages.
The 3 stages of the Emerging Business Office Generation makes increasing investments in companies 4 pillars: Capturing, connecting, deciding, and acting on data Market sizing 3 horizons & timeframes Science: 3-5y Development: 1-2y Preparation: <12m “A startup could take a year to talk to as many customers as we Exploration proves out both the tech and business model do in a week.” Challenge the assumptions. “Am I hitting my milestones?” and “Are my assumptions still valid?” Rapid prototyping. Focus on risk and uncertainty: which things don’t we know how to do? Adoption means customer buying in External partner; validate across multiple constituents to ensure it’s a scalable business model. Use customer base as an advantage. C-level conversations almost immediately. 6-8 Generate projects What’s the value proposition? Why are we going to make money? Why Motorola? 4 Explore projects Number of dangling assumptions Rate at which it’s growing/sinking Very deep with small sample size 6-8 Adopt projects Casting a wider net Go-to-market metrics Funnel size and market segments
Step three: Secure funding, resources, and executive backing.
Reinvesting For every $100 they cut, Metlife reinvests $66 in new projects.
Step four: Generate new ideas.
Find non-obvious adjacencies LIGH T ELECTRICAL GENERATOR SOFTWARE TO CUT DOWN TREES BETTER MRI MACHINE POWER GRID PLANE ENGINE REQUIRES TRAIN ENGINE WIND TURBINE NEEDS AN WHICH FEEDS A HAS A TURBINE LIKE A TURNED AROUND BECOMES A SPINS & VIBRATES LIKE AN AND LOOKS LIKE A
Build an ecosystem Canada’s largest directory publishing and local marketing services company 1.5M listings from 420K SMB & national customers Revenues >$1.2B 2,500 employees Created third-party listing API Took 8-10 mo (2009-10) to get approval API payoff happened 2y later Yahoo replaced Canadian digital properties search with the YellowAPI Improved SEO, Comscore Functional prototype in hours, testing in days, and launching in weeks. Faster time to partnerships Budgets tripled in 2013 KPI evolution Soft: Signups, SDK, downloads App usage, deals signed API calls generated API-generated revenue
Three sources of innovation Top-down: Areas where business heads see market trends but white spaces in our offerings. Maybe we can fill this white space. Bottoms-up 160,000 associates worldwide use an app called Spark; if viable we put it in front of the EBA leadership meeting Sparktank meeting—should we put $100-$150K to go and find a first customer. Outside-in M&A easier when there is an EBA structure exists because it specializes in integration with the existing organizaton we bring them into the EBA and help them match Innovation/investment team backs a few people who have a good idea and can use the infrastructure, channel, etc.
Five common models for transformative innovation All employ different models at different times. Acquisition Collaboration Isolation Incubation Integration Buy promising startups Crowdsource, work with universities, suppliers, etc. Create a separate group with different conditions Internal startup ecosystem; LoB are “investors” The LoB does innovation internally
Step five: Test by doing: Experimentation beats projection.
Focus on the model, not the plan Demand People per day on sidewalk Percent that buy a glass Daily customers Revenue Price per cup Cost of Goods Cost per cup Profit per cup Daily profit Amt 200 10% 20 $5 $1 $4 $80 Growth 4% 5% -2% Wk 1 204 15% 31 43 56 $5 $5 $5 .98 Wk 2 216 20% .96 Wk 3 225 25% .94 $156 30.6 $216 41.5 $281 52.8 4.02 4.04 4.06 125 175 228
A business plan is just what happens when you drag the business model to the right.
Designing an experiment Problem, solution, and result hypothesis Test strategy (PoC, survey, interviews, kickstarter, prototype, A/B, etc.) Cohort & segment to be tested Metric or assumption being tested Timebox or total for test Action you’ll take if you pass or fail
Step six: Know what happens afterwards
Qualcomm’s initial innovation model http://blogs.berkeley.edu/2013/01/28/ designing-a-corporate-entrepreneurship- program-a-qualcomm- case-study-part-1-of-2/ Hypothesis Experiment Implement Idea generation and selection Boot camp Idea advancement Ideas Existing models New models Open innovation Tech feasibility Biz feasibility Boot camp decision Implement End user/partner desirability Action s Option value Strategic value Exit value Company crowd storm Small team designs & decision conducts experiments Company extracts value
Qualcomm’s updated model Criteria Fully open to all employees Ideas implemented by existing business/R&D units Efficient way to bubble-up best ideas (and their champions) to the timely attention of top execs Hypothesis Experiment Implement Idea generation and selection Boot camp (3 mo, part time) CEO open call Innovator challenges Idea mgmt. system Discover Network Accelerate Pitch to exec team Self-forming teams Filters Contextual education Mentorship Micro-seed funding Program staff support BU sponsor home Employee team BU Sponsor Program team Value extraction Future option value Strategic value Exit value
Qualcomm’s innovation model: What was missing Hypothesis Experiment Implement POC Idea generation and selection Boot camp Idea advancement Ideas Existing models New models Open innovation Tech feasibility Biz sustain-ability Boot camp decision Implement End user/partner desirability Action s Option value Strategic value Exit value Company crowd storm Small team designs & decision conducts experiments Company extracts value POC decision Unclear what happened to founders Needed a middle PoC decision Sustainability, not feasibility
Step seven: Rinse, repeat.
The Lean Analytics lifecycle of an Intrapreneur Beforehand Get buy-in Political fallout Empathy Find problems; don’t test demand. Skip the business case, do analytics Entitled, aggrieved customers Stickiness Know your real minimum based on expectations, regulations Hidden “must haves”, feature creep Virality Build inherent virality in from the start; attention is the new currency Luddites who don’t understand sharing Revenue Consider the ecosystem, channels, and established agreements Channel conflict, resistance, contracts Scale Hand the baton to others gracefully Hating what happens to your baby
Metrics for innovation portfolios.
Core metrics Metrics that matter • Return on investment • Total cost of ownership • Trouble tickets/issues • Training time • Comparing to others Business plan. Assume it will work. But the market will change by the time you’ve built it. Example: Online parking tickets
Adjacent metrics Metrics that matter • Questions answered • Virality & word of mouth • Early adopter stickiness • Regulation • Total addressable market Business model. Assume it will fail. Your ultimate use case won’t be what you think it is today. Example: Mr. Clean Magic Eraser
Transformative metrics Metrics that matter • People I’ve talked to • Prototype creation speed • Assumptions validated • Problems uncovered • Technical feasibility • Hidden constraints Business idea. Assume it is possible. You hope it will have the consequences you want but aren’t sure how. Example: Headcam recordings of all officers
The Emerging Business Accelerator Three Horizons model Horizon one is traditional services such as app dev (SAP, Oracle) Horizon two are offerings that aren’t quite as big and mainstream, not used by everyone, but have good traction. Smaller revenue contribution (IT infrastructure, vertically focused BPO). Also includes some strategy/tactics Horizon three is about identifying ideas that are worth investing in, allocating investment, and incubating them through our own practices. Projects “graduate” into a lower horizon 20 ventures today, in 1 of 3 dimensions Innovation along 3 dimensions New markets: Either traditional offering in a new place i.e. Latin America, Can’t simply do labor arbitrage. Can also be a new vertical such as government. New technologies: Social, mobile, analytics, cloud, Internet of Things. New delivery models (“products”): Platforms, recurring revenues, building products to enable vertical business processes. Explore-to-graduate criteria Explore phase are looking for a first customer Early startup are looking for early-stage customers. Late-stage startup are trying to show they can deliver for multiple customers; have a business model Growth phase has true P&L. They’re past the cashflow breakeven. We’re saying, “we know enough about price they will pay and how many people will want, and what it costs, so we know margin.” If they can deliver against this we graduate.
Portfolio metrics “Do we need to meet more often?” is a metric.” Number in the pipeline; is it growing? How many are crazy vs. real business ideas? Ideas funded; ideas that were a waste; ideas needing iteration # of meetings, qualitative feedback, pivots Have we convinced someone to sign on for something? Number of proposals issued; pipeline Satisfied delivery, on budget; trouble tickets; delays; escalation; referenceability Profitable independent of costs like development? Ideas Quality Funding Exploration Solution fit Demand Product fit Profitability Cashflow B/E 0% overall profitability, beginning to repay initial investment Graduation Making money overall
Key points to clarify in an innovation program Hypothesis Experimentation Implementation • Articulate problems • Define the right filters • Get ideas from many sources • Confirm funding (money, people, customer access) • Agree on analytical framework • Balance market, product, & method adjacencies • Prioritize riskiest assumptions • Time-box assessment stages • Test technology, demand, and business feasibility • MVP, prototype, pilot, or science as appropriate for type of innovation • Temporary incubator • Find a home or building one • Keep innovators involved • Merge metrics with existing business KPIs • Synchronize innovation cycles with enterprise cycles (budget, etc.) Portfolio metrics; Gates and KPIs for each stage; mix of core, adjacent, and disruptive innovation.
Goals, constraints, context Sourcing Filtering Core Integration Adjacent Disruptive (how to decide?) Adoption by existing line of business. Independence Creation of a new line of business. Cross-pollinate to current managers Evaluation of the innovation program itself Socializin g Test/ validate w/current customers Grow as a distinct business Top-down Bottom-up Outside-in R&D M&A
Some tools and traps
Build a message map. 1. Understand the stages a buyer goes through 2. Create benefits; mitigate objections 3. Target the message to the stage the audience is at
Everyone in the world A. I need a car I should buy B. a car It should be C. a hybrid I should buy D. a Honda Civic
Everyone in the world A. I need a car People who want to drive I should buy B. a car Prospective car buyers It should be C. a hybrid People looking for a hybrid I should buy D. a Honda Civic Honda Civic Hybrid owners
“Isn’t it time you got out of the city?” campaign showing how cars make nature accessible & ridiculing urban hipsters. Ads showing how cars are needed any time (pregnancy, errands, urgent business) and how a car is a “personal assistant.” Urgency (“every time you drive a non-hybrid car you kill the planet a little”) and testimonials from buyers who’ve saved money. Honda branding ads and model-specific promotions. Follow-up satisfaction campaign to encourage buyers to tell their friends Everyone in the world A. I need a car People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” I should buy B. a car Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” It should be C. a hybrid People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” I should buy D. a Honda Civic Honda Civic Hybrid owners
Everyone in the world People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” Honda Civic Hybrid owners Those who don’t need cars • I’m too young to drive • I’m too old to drive • I can walk or take public transit Car users who won’t buy • It’s too expensive for me • I will use a shared car service • It’ll get stolen Those who won’t buy hybrids • Hybrids are gutless • Batteries are toxic & explosive • In the end it costs more than it saves I will buy another brand • I buy domestic • I’ve always driven a VW • Toyotas are reliable • I want something prestigious A. I need a car I should buy B. a car It should be C. a hybrid I should buy D. a Honda Civic
Everyone in the world People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” Honda Civic Hybrid owners Those who don’t need cars • I’m too young to drive • I’m too old to drive • I can walk or take public transit Car users who won’t buy • It’s too expensive for me • I will use a shared car service • It’ll get stolen Those who won’t buy hybrids • Hybrids are gutless • Batteries are toxic & explosive • In the end it costs more than it saves I will buy another brand • I buy domestic • I’ve always driven a VW • Toyotas are reliable • I want something prestigious Sponsor a driving school “Give the gift of driving” campaign for grandparents. PR on dangers of commuting, pedestrian deaths Financing, cashback Sell to carshares; underscore their limitations Theft warranty, tracking services, high-end locks Independent tests, standard metrics (0-60 in X) Lab research, studies ROI calculator; replacement programs Prove Honda hires US workers “Time to leave Germany” ads Spontaneous accel. stories Premium brand (Acura) A. I need a car I should buy B. a car It should be C. a hybrid I should buy D. a Honda Civic
“Isn’t it time you got out of the city?” campaign showing how cars make nature accessible & ridiculing urban hipsters. Ads showing how cars are needed any time (pregnancy, errands, urgent business) and how a car is a “personal assistant.” Urgency (“every time you drive a non-hybrid car you kill the planet a little”) and testimonials from buyers who’ve saved money. Honda branding ads and model-specific promotions. Follow-up satisfaction campaign to encourage buyers to tell their friends Everyone in the world People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” Honda Civic Hybrid owners Those who don’t need cars • I’m too young to drive • I’m too old to drive • I can walk or take public transit Car users who won’t buy • It’s too expensive for me • I will use a shared car service • It’ll get stolen Those who won’t buy hybrids • Hybrids are gutless • Batteries are toxic & explosive • In the end it costs more than it saves I will buy another brand • I buy domestic • I’ve always driven a VW • Toyotas are reliable • I want something prestigious Sponsor a driving school “Give the gift of driving” campaign for grandparents. PR on dangers of commuting, pedestrian deaths Financing, cashback Sell to carshares; underscore their limitations Theft warranty, tracking services, high-end locks Independent tests, standard metrics (0-60 in X) Lab research, studies ROI calculator; replacement programs Prove Honda hires US workers “Time to leave Germany” ads Spontaneous accel. stories Premium brand (Acura) A. I need a car I should buy B. a car It should be C. a hybrid I should buy D. a Honda Civic
The good news: The harsh light of data changes everything.
Big Lots of information, in flight and at rest. Fast Reliable Storage and retrieval in short timeframes. High availability in replication, consistency, and recoverability (Pick any two) Big Data’s iron triangle
Some tools and traps
The three threes Three assumptions What big bets are you making? •“People will answer questions” •“Organizers are frustrated with how to run conferences” •“We'll make money from parents” •“Amazon is reliable enough for our users.” Three actions to take What are you doing to make these assumptions happen (or identify they’re wrong and change course?) •Product enhancements •Marketing strategies Three experiments to run •Feature tests •Continuous deployment •A/B testing •Customer survey
The three threes Three assumptions Three actions to take Three experiments to run Monthly Weekly Daily Board, investors, founders Executive team Employees Strategy Tactics Execution
The three threes Three assumptions Three actions to take Three experiments to run Get more people Increase answer % Test better questions Change the UI Test timings Question s from Many people will answer questions
The problem-solution canvas CURRENT STATUS The Goal is to Learn • List key metrics you’re tracking, where they’re at, and compare with last few weeks • How are things trending? LAST WEEK’S LESSONS LEARNED AND ACCOMPLISHMENTS) • What did you learn last week? • What was accomplished? • On track: YES / NO?
The problem-solution canvas HYPOTHESIZED SOLUTIONS • List possible solutions that you’ll start working on next week. Rank them. • Why do you believe each solution will help you solve or complete solve the problem? METRICS / PROOF + GOALS Problem #1 (put name here) • Metrics you’ll use to measure whether or not the solutions are doing what you hoped (solving the problem) • List proof (qualitative) you’ll use as well • Define goals for the metric HYPOTHESIZED SOLUTIONS • List possible solutions that you’ll start working on next week. Rank them. • Why do you believe each solution will METRICS / PROOF + GOALS • Metrics you’ll use to measure whether or not the solutions are doing what you hoped (solving the problem) Problem #2 (put name here)
“The most important figures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them.” Lloyd S. Nelson
Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
ARCHIMEDES HAD TAKEN BATHS BEFORE.
Once, a leader convinced others in the absence of data.
Now, a leader knows what questions to ask.
Ben Yoskovitz byosko@gmail.com @byosko Alistair Croll acroll@gmail.com @acroll
Workshop: Systems diagrams and NCARB’s business
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference
The mobile app! customer lifecycle! Ratings Reviews Search Leaderboards Purchases App store! App sales Downloads Installs Play Disengagement Reactivation Uninstallation Disengagement Account" creation Virality Downloads," Gross revenue ARPU Activation Churn, CLV In-app" purchases Legitimate Incentivized Fraudulent Ratings!
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference
Traction graphs Your business model The stage you’re at Your one metric ... change often if you’re doing it right. So how do you track that over time?
Traction graphs Jan Feb Mar Apr May Jun Signups Conversion Churn per day rate rate Viral coefficient This axis changes for each metric
Traction graphs Jan Feb Mar Apr May Jun Signups Conversion Churn per day rate rate Viral coefficient 0%
Use vanity to get to meaningful metrics Your goal is to produce outcomes If the outcomes require action, and vanity motivates actors, use it But show how the vanity metric is a leading indicator of the real one Web traffic Activation x Revenue Cart Size Conversion rate
Thinking Backwards: The Solution/Problem approach Mitigation & execution Act & measure Rob Van Haastrecht & Martin Scheepbouwer Identify a clear, known goal Get on same page with relevant facts Agree on goal KPIs Outline possible solutions Proposed solutions List assumptions (causes, actions, costs, risks) Agree on how to test/analyze them Answer/test them (MVP, etc.) See where uncertainty exists Validation & testing Estimate ability to mitigate risks (SWOT) Choose next best action (CxO) Staff team based on goal audacity results
Key points Intrapreneurship is about adjacent or transformative innovation Sustaining innovation focuses on the Five Mores, within the current product, market, method, and business model. Adjacent innovation may come from a new product, market, or method, but the same business model Disruptive innovation has different customers, KPIs, and models The difference between a rogue agent and a special operative is permission Portfolios need sourcing, filters, metrics, and socializing Balancing isolation and integration, R&D and M&A is contentious
Ben Yoskovitz byosko@gmail.com @byosko Alistair Croll acroll@gmail.com @acroll
1 of 361

Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference

••
Download to read offline
Business

A look at the metrics and processes needed to build a better business faster through data, whether you're a startup or a large enterprise.

L
Lean AnalyticsStartup accelerator

Recommended

Lean Analytics @ MicroConf by
Lean Analytics @ MicroConfLean Analytics @ MicroConf
Lean Analytics @ MicroConfLean Analytics
158.4K views•60 slides
Lean Analytics: Using Data to Build a Better Business Faster by
Lean Analytics: Using Data to Build a Better Business FasterLean Analytics: Using Data to Build a Better Business Faster
Lean Analytics: Using Data to Build a Better Business FasterLean Startup Co.
2.4K views•169 slides
Introduction to Lean Analytics for Lean Startup Circle SF by
Introduction to Lean Analytics for Lean Startup Circle SFIntroduction to Lean Analytics for Lean Startup Circle SF
Introduction to Lean Analytics for Lean Startup Circle SFLean Analytics
38.6K views•43 slides
Lean Analytics: A short summary by
Lean Analytics: A short summaryLean Analytics: A short summary
Lean Analytics: A short summaryJan König
23.4K views•35 slides
Introduction to Lean Analytics webinar (O'Reilly) by
Introduction to Lean Analytics webinar (O'Reilly)Introduction to Lean Analytics webinar (O'Reilly)
Introduction to Lean Analytics webinar (O'Reilly)Lean Analytics
4.2K views•46 slides
Metrics 101 by
Metrics 101Metrics 101
Metrics 101Alistair Croll
19.7K views•340 slides

More Related Content

What's hot

Lean Analytics for Intrapreneurs (Lean Startup Conf 2013) by
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics
40.6K views•222 slides
Lean Analytics for Nikkei BP by
Lean Analytics for Nikkei BPLean Analytics for Nikkei BP
Lean Analytics for Nikkei BPLean Analytics
2K views•21 slides
OnLab Japan introduction to Lean Analytics by
OnLab Japan introduction to Lean AnalyticsOnLab Japan introduction to Lean Analytics
OnLab Japan introduction to Lean AnalyticsLean Analytics
14.8K views•113 slides
Slides from Growthcon 2014 Lean Analytics masterclass by
Slides from Growthcon 2014 Lean Analytics masterclassSlides from Growthcon 2014 Lean Analytics masterclass
Slides from Growthcon 2014 Lean Analytics masterclassLean Analytics
6.6K views•143 slides
Lean analytics: Five lessons beyond the basics by
Lean analytics: Five lessons beyond the basicsLean analytics: Five lessons beyond the basics
Lean analytics: Five lessons beyond the basicsLean Analytics
15.1K views•62 slides
Lean Analytics workshop for Dublin City University, April 2014 by
Lean Analytics workshop for Dublin City University, April 2014Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014Lean Analytics
61.3K views•141 slides

What's hot(20)

Lean Analytics for Intrapreneurs (Lean Startup Conf 2013) by Lean Analytics
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics•40.6K views
Lean Analytics for Nikkei BP by Lean Analytics
Lean Analytics for Nikkei BPLean Analytics for Nikkei BP
Lean Analytics for Nikkei BP
Lean Analytics•2K views
OnLab Japan introduction to Lean Analytics by Lean Analytics
OnLab Japan introduction to Lean AnalyticsOnLab Japan introduction to Lean Analytics
OnLab Japan introduction to Lean Analytics
Lean Analytics•14.8K views
Slides from Growthcon 2014 Lean Analytics masterclass by Lean Analytics
Slides from Growthcon 2014 Lean Analytics masterclassSlides from Growthcon 2014 Lean Analytics masterclass
Slides from Growthcon 2014 Lean Analytics masterclass
Lean Analytics•6.6K views
Lean analytics: Five lessons beyond the basics by Lean Analytics
Lean analytics: Five lessons beyond the basicsLean analytics: Five lessons beyond the basics
Lean analytics: Five lessons beyond the basics
Lean Analytics•15.1K views
Lean Analytics workshop for Dublin City University, April 2014 by Lean Analytics
Lean Analytics workshop for Dublin City University, April 2014Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics•61.3K views
Big Data for the CMO by Bruno Aziza
Big Data for the CMOBig Data for the CMO
Big Data for the CMO
Bruno Aziza•10.5K views
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics by Lean Analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analyticsMelbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Lean Analytics•14.6K views
Croll lean analytics workshop (3h) - lean ux nyc april 2014 by Lean Analytics
Croll lean analytics workshop (3h) - lean ux nyc april 2014Croll lean analytics workshop (3h) - lean ux nyc april 2014
Croll lean analytics workshop (3h) - lean ux nyc april 2014
Lean Analytics•8.9K views
Growth Hacking with Cassie Lancellotti-Young by Mattan Griffel
Growth Hacking with Cassie Lancellotti-YoungGrowth Hacking with Cassie Lancellotti-Young
Growth Hacking with Cassie Lancellotti-Young
Mattan Griffel•25.9K views
Lean Analytics - Ben Yoskovitz, Co-author, Lean Analytics & VP Product, Varag... by Traction Conf
Lean Analytics - Ben Yoskovitz, Co-author, Lean Analytics & VP Product, Varag...Lean Analytics - Ben Yoskovitz, Co-author, Lean Analytics & VP Product, Varag...
Lean Analytics - Ben Yoskovitz, Co-author, Lean Analytics & VP Product, Varag...
Traction Conf•1K views
Startup metrics toronto March 19 by Lean Analytics
Startup metrics toronto March 19Startup metrics toronto March 19
Startup metrics toronto March 19
Lean Analytics•6.9K views
Understanding Lean Analytics (and how analytics helps businesses win) by Lean Analytics
Understanding Lean Analytics (and how analytics helps businesses win)Understanding Lean Analytics (and how analytics helps businesses win)
Understanding Lean Analytics (and how analytics helps businesses win)
Lean Analytics•8.9K views
Lean Analytics workshop (from Lean Startup Conf) by Lean Analytics
Lean Analytics workshop (from Lean Startup Conf)Lean Analytics workshop (from Lean Startup Conf)
Lean Analytics workshop (from Lean Startup Conf)
Lean Analytics•79.1K views
Measuring the ROI of content marketing by Jono Alderson
Measuring the ROI of content marketingMeasuring the ROI of content marketing
Measuring the ROI of content marketing
Jono Alderson•3.5K views
Lean Analytics for Startups and Enterprises by Lean Analytics
Lean Analytics for Startups and EnterprisesLean Analytics for Startups and Enterprises
Lean Analytics for Startups and Enterprises
Lean Analytics•41.7K views
Growth Hacking Basics by Morgan Brown
Growth Hacking BasicsGrowth Hacking Basics
Growth Hacking Basics
Morgan Brown•99K views
Startup Growth: Up and to the Right by Intelligent_ly
Startup Growth: Up and to the RightStartup Growth: Up and to the Right
Startup Growth: Up and to the Right
Intelligent_ly•6.2K views
Lean startup metrics by Stuart Eccles
Lean startup metricsLean startup metrics
Lean startup metrics
Stuart Eccles•23.8K views
Managing a Virtual Economy by William Grosso
Managing a Virtual EconomyManaging a Virtual Economy
Managing a Virtual Economy
William Grosso•8.6K views

Viewers also liked

Startup Metrics for Pirates by
Startup Metrics for PiratesStartup Metrics for Pirates
Startup Metrics for PiratesDave McClure
1.2M views•20 slides
Lean Analytics and Local Government - Alistair Croll - Code for America by
Lean Analytics and Local Government - Alistair Croll - Code for AmericaLean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for AmericaLean Analytics
2.5K views•101 slides
Lean Startup Workshop by
Lean Startup WorkshopLean Startup Workshop
Lean Startup WorkshopTathagat Varma
4.1K views•73 slides
Lean startup workshop: practical ways to turn your idea into a successful pro... by
Lean startup workshop: practical ways to turn your idea into a successful pro...Lean startup workshop: practical ways to turn your idea into a successful pro...
Lean startup workshop: practical ways to turn your idea into a successful pro...Made by Many
14.9K views•58 slides
2009 05 01 How To Build A Lean Startup Step By Step by
2009 05 01 How To Build A Lean Startup Step By Step2009 05 01 How To Build A Lean Startup Step By Step
2009 05 01 How To Build A Lean Startup Step By StepEric Ries
31.6K views•26 slides
De rol van Lean Analytics in je omnichannel strategie | Webwinkel Vakdagen by
De rol van Lean Analytics in je omnichannel strategie | Webwinkel VakdagenDe rol van Lean Analytics in je omnichannel strategie | Webwinkel Vakdagen
De rol van Lean Analytics in je omnichannel strategie | Webwinkel VakdagenOnline Boswachters
1.9K views•82 slides

Viewers also liked(19)

Startup Metrics for Pirates by Dave McClure
Startup Metrics for PiratesStartup Metrics for Pirates
Startup Metrics for Pirates
Dave McClure•1.2M views
Lean Analytics and Local Government - Alistair Croll - Code for America by Lean Analytics
Lean Analytics and Local Government - Alistair Croll - Code for AmericaLean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics•2.5K views
Lean Startup Workshop by Tathagat Varma
Lean Startup WorkshopLean Startup Workshop
Lean Startup Workshop
Tathagat Varma•4.1K views
Lean startup workshop: practical ways to turn your idea into a successful pro... by Made by Many
Lean startup workshop: practical ways to turn your idea into a successful pro...Lean startup workshop: practical ways to turn your idea into a successful pro...
Lean startup workshop: practical ways to turn your idea into a successful pro...
Made by Many•14.9K views
2009 05 01 How To Build A Lean Startup Step By Step by Eric Ries
2009 05 01 How To Build A Lean Startup Step By Step2009 05 01 How To Build A Lean Startup Step By Step
2009 05 01 How To Build A Lean Startup Step By Step
Eric Ries•31.6K views
De rol van Lean Analytics in je omnichannel strategie | Webwinkel Vakdagen by Online Boswachters
De rol van Lean Analytics in je omnichannel strategie | Webwinkel VakdagenDe rol van Lean Analytics in je omnichannel strategie | Webwinkel Vakdagen
De rol van Lean Analytics in je omnichannel strategie | Webwinkel Vakdagen
Online Boswachters•1.9K views
Lean Startup and Customer Development: Challenges & Obstacles that Founders N... by Veronica Torras
Lean Startup and Customer Development: Challenges & Obstacles that Founders N...Lean Startup and Customer Development: Challenges & Obstacles that Founders N...
Lean Startup and Customer Development: Challenges & Obstacles that Founders N...
Veronica Torras•3.3K views
How to start a startup 1-10강 by 종익 주
How to start a startup 1-10강How to start a startup 1-10강
How to start a startup 1-10강
종익 주•9.7K views
Lithium Social Strategy Workshop Frankfurt by Sandra Teuber (Reif)
Lithium Social Strategy Workshop FrankfurtLithium Social Strategy Workshop Frankfurt
Lithium Social Strategy Workshop Frankfurt
Sandra Teuber (Reif)•493 views
Lean Startup Thinking by Tathagat Varma
Lean Startup ThinkingLean Startup Thinking
Lean Startup Thinking
Tathagat Varma•7.4K views
Lean Startup Workshop for Startups, Entrepreneurs and Intrapreneurs by Rammohan Reddy
Lean Startup Workshop for Startups, Entrepreneurs and IntrapreneursLean Startup Workshop for Startups, Entrepreneurs and Intrapreneurs
Lean Startup Workshop for Startups, Entrepreneurs and Intrapreneurs
Rammohan Reddy•1.8K views
Organize Lean Startup Machine by Grace Ng
Organize Lean Startup MachineOrganize Lean Startup Machine
Organize Lean Startup Machine
Grace Ng•11.7K views
Smarter Commerce for Insurance - 360 Degree Customer Experience Infographic by Lynn Kesterson-Townes
Smarter Commerce for Insurance - 360 Degree Customer Experience InfographicSmarter Commerce for Insurance - 360 Degree Customer Experience Infographic
Smarter Commerce for Insurance - 360 Degree Customer Experience Infographic
Lynn Kesterson-Townes•1.9K views
UXSG2014 Workshop (Day 1) - Lean Startup (Bryan Long) by ux singapore
UXSG2014 Workshop (Day 1) - Lean Startup (Bryan Long)UXSG2014 Workshop (Day 1) - Lean Startup (Bryan Long)
UXSG2014 Workshop (Day 1) - Lean Startup (Bryan Long)
ux singapore•1.3K views
Making Sense of the Numbers (Lean Analytics) by Lean Analytics
Making Sense of the Numbers (Lean Analytics)Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)
Lean Analytics•2.1K views
Social CRM Definition By Martin Walsh by Martin Walsh
Social CRM Definition By Martin WalshSocial CRM Definition By Martin Walsh
Social CRM Definition By Martin Walsh
Martin Walsh•9.5K views
What the f**k is lean startup and why should I care? by The Happy Startup School
What the f**k is lean startup and why should I care?What the f**k is lean startup and why should I care?
What the f**k is lean startup and why should I care?
The Happy Startup School•10.5K views
The Lean Startup and Customer Development model. In Italy. by Nicola Junior Vitto
The Lean Startup and Customer Development model. In Italy.The Lean Startup and Customer Development model. In Italy.
The Lean Startup and Customer Development model. In Italy.
Nicola Junior Vitto•18.1K views
Business Model Canvas - www.innowatyka.pl by Hubert Dyba
Business Model Canvas - www.innowatyka.plBusiness Model Canvas - www.innowatyka.pl
Business Model Canvas - www.innowatyka.pl
Hubert Dyba•3.3K views

Similar to Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference

Lean Analytics for Intrapreneurs by Allistair Croll by
Lean Analytics for Intrapreneurs by Allistair CrollLean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair CrollLean Startup Co.
7.2K views•222 slides
Alistaire croll lean analytics - montreal lean startup circle - september 2018 by
Alistaire croll lean analytics - montreal lean startup circle - september 2018Alistaire croll lean analytics - montreal lean startup circle - september 2018
Alistaire croll lean analytics - montreal lean startup circle - september 2018Lean Startup Circle Montreal
216 views•57 slides
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking by
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking jeroentjepkema
40.3K views•134 slides
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S... by
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...Kate O'Neill
864 views•74 slides
Smarter Marketing: Using online intelligence to inform customer development by
Smarter Marketing: Using online intelligence to inform customer developmentSmarter Marketing: Using online intelligence to inform customer development
Smarter Marketing: Using online intelligence to inform customer developmentKate O'Neill
794 views•74 slides
Developing a cult of analytics by
Developing a cult of analyticsDeveloping a cult of analytics
Developing a cult of analyticsSteve Jackson
1.5K views•28 slides

Similar to Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conference (20)

Lean Analytics for Intrapreneurs by Allistair Croll by Lean Startup Co.
Lean Analytics for Intrapreneurs by Allistair CrollLean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair Croll
Lean Startup Co.•7.2K views
Alistaire croll lean analytics - montreal lean startup circle - september 2018 by Lean Startup Circle Montreal
Alistaire croll lean analytics - montreal lean startup circle - september 2018Alistaire croll lean analytics - montreal lean startup circle - september 2018
Alistaire croll lean analytics - montreal lean startup circle - september 2018
Lean Startup Circle Montreal•216 views
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking by jeroentjepkema
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
jeroentjepkema•40.3K views
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S... by Kate O'Neill
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
Kate O'Neill•864 views
Smarter Marketing: Using online intelligence to inform customer development by Kate O'Neill
Smarter Marketing: Using online intelligence to inform customer developmentSmarter Marketing: Using online intelligence to inform customer development
Smarter Marketing: Using online intelligence to inform customer development
Kate O'Neill•794 views
Developing a cult of analytics by Steve Jackson
Developing a cult of analyticsDeveloping a cult of analytics
Developing a cult of analytics
Steve Jackson•1.5K views
DIT Digitial Marketing Forum: Analytics by Lar Veale
DIT Digitial Marketing Forum: AnalyticsDIT Digitial Marketing Forum: Analytics
DIT Digitial Marketing Forum: Analytics
Lar Veale•643 views
KD Paine Presentation by Mediabistro
KD Paine PresentationKD Paine Presentation
KD Paine Presentation
Mediabistro•462 views
Fundraising Strategy - useful tools that really work IoF 2011 by Simon Burne
Fundraising Strategy - useful tools that really work IoF 2011Fundraising Strategy - useful tools that really work IoF 2011
Fundraising Strategy - useful tools that really work IoF 2011
Simon Burne•3K views
online startups metrics by Hatem Kameli
online startups metrics online startups metrics
online startups metrics
Hatem Kameli•3.5K views
Brandable newsletter for printers and mailers by Great Reach Communications
Brandable newsletter for printers and mailersBrandable newsletter for printers and mailers
Brandable newsletter for printers and mailers
Great Reach Communications•385 views
Inbound marketing 7 earth-shaking reasons webinar (v8.0) by hiddenpeakint
Inbound marketing 7 earth-shaking reasons webinar (v8.0)Inbound marketing 7 earth-shaking reasons webinar (v8.0)
Inbound marketing 7 earth-shaking reasons webinar (v8.0)
hiddenpeakint•320 views
Start-up Metrics (Smetrics ) by James Alexander
Start-up Metrics (Smetrics )Start-up Metrics (Smetrics )
Start-up Metrics (Smetrics )
James Alexander•1K views
Harvesting the value from Advanced Analytics by Jaap Vink
Harvesting the value from Advanced AnalyticsHarvesting the value from Advanced Analytics
Harvesting the value from Advanced Analytics
Jaap Vink•66 views
SBC Growth Week - Lean Analytics by MeasureWorks
SBC Growth Week - Lean AnalyticsSBC Growth Week - Lean Analytics
SBC Growth Week - Lean Analytics
MeasureWorks•1.5K views
Effective Business Practices 101 (5/8): Power Your Business With Information by Dmitri Tcherbadji
Effective Business Practices 101 (5/8): Power Your Business With InformationEffective Business Practices 101 (5/8): Power Your Business With Information
Effective Business Practices 101 (5/8): Power Your Business With Information
Dmitri Tcherbadji•136 views
Monetization in the trenches 012511 by Chip Cohan
Monetization in the trenches 012511Monetization in the trenches 012511
Monetization in the trenches 012511
Chip Cohan•399 views
Sales training for an IT consulting firm by Allied Consultants
Sales training for an IT consulting firmSales training for an IT consulting firm
Sales training for an IT consulting firm
Allied Consultants•3.4K views
Talent Institute - Frictionless Conversion (workshop) by MeasureWorks
Talent Institute - Frictionless Conversion (workshop)Talent Institute - Frictionless Conversion (workshop)
Talent Institute - Frictionless Conversion (workshop)
MeasureWorks•619 views
2017 06-test withintelligence-conversionelite by Tim Stewart
2017 06-test withintelligence-conversionelite2017 06-test withintelligence-conversionelite
2017 06-test withintelligence-conversionelite
Tim Stewart•1.4K views

Recently uploaded

The Talent Management Navigator Performance Management by
The Talent Management Navigator Performance ManagementThe Talent Management Navigator Performance Management
The Talent Management Navigator Performance ManagementSeta Wicaksana
35 views•36 slides
SWOT Analysis of MBM Group by
SWOT Analysis of MBM GroupSWOT Analysis of MBM Group
SWOT Analysis of MBM GroupAriful Saimon
19 views•4 slides
Better Appeals and Solicitations - Bloomerang.pdf by
Better Appeals and Solicitations - Bloomerang.pdfBetter Appeals and Solicitations - Bloomerang.pdf
Better Appeals and Solicitations - Bloomerang.pdfBloomerang
81 views•51 slides
v s.pptx by
v s.pptxv s.pptx
v s.pptxravikhadalwal
12 views•2 slides
Rangell Auto Detailing by
Rangell Auto Detailing Rangell Auto Detailing
Rangell Auto Detailing rangellautodetailing
18 views•8 slides
CORPORATE COMMUNICATION.pdf by
CORPORATE COMMUNICATION.pdfCORPORATE COMMUNICATION.pdf
CORPORATE COMMUNICATION.pdfAKarthikeyan8
18 views•71 slides

Recently uploaded(20)

The Talent Management Navigator Performance Management by Seta Wicaksana
The Talent Management Navigator Performance ManagementThe Talent Management Navigator Performance Management
The Talent Management Navigator Performance Management
Seta Wicaksana•35 views
SWOT Analysis of MBM Group by Ariful Saimon
SWOT Analysis of MBM GroupSWOT Analysis of MBM Group
SWOT Analysis of MBM Group
Ariful Saimon•19 views
Better Appeals and Solicitations - Bloomerang.pdf by Bloomerang
Better Appeals and Solicitations - Bloomerang.pdfBetter Appeals and Solicitations - Bloomerang.pdf
Better Appeals and Solicitations - Bloomerang.pdf
Bloomerang•81 views
v s.pptx by ravikhadalwal
v s.pptxv s.pptx
v s.pptx
ravikhadalwal•12 views
Rangell Auto Detailing by rangellautodetailing
Rangell Auto Detailing Rangell Auto Detailing
Rangell Auto Detailing
rangellautodetailing•18 views
CORPORATE COMMUNICATION.pdf by AKarthikeyan8
CORPORATE COMMUNICATION.pdfCORPORATE COMMUNICATION.pdf
CORPORATE COMMUNICATION.pdf
AKarthikeyan8•18 views
Basic of Air Ticketing & IATA Geography by Md Shaifullar Rabbi
Basic of Air Ticketing & IATA GeographyBasic of Air Ticketing & IATA Geography
Basic of Air Ticketing & IATA Geography
Md Shaifullar Rabbi •69 views
Navigating the Complexity of Derivatives Valuation 📈 by ValAdvisor
Navigating the Complexity of Derivatives Valuation 📈Navigating the Complexity of Derivatives Valuation 📈
Navigating the Complexity of Derivatives Valuation 📈
ValAdvisor•15 views
Bloomerang Thank Yous Dec 2023.pdf by Bloomerang
Bloomerang Thank Yous Dec 2023.pdfBloomerang Thank Yous Dec 2023.pdf
Bloomerang Thank Yous Dec 2023.pdf
Bloomerang•140 views
Amazing Opportunities: PCD Pharma Franchise in Kerala.pptx by SaphnixMedicure1
Amazing Opportunities: PCD Pharma Franchise in Kerala.pptxAmazing Opportunities: PCD Pharma Franchise in Kerala.pptx
Amazing Opportunities: PCD Pharma Franchise in Kerala.pptx
SaphnixMedicure1•23 views
MechMaf Shipping LLC by MechMaf Shipping LLC
MechMaf Shipping LLCMechMaf Shipping LLC
MechMaf Shipping LLC
MechMaf Shipping LLC•65 views
port23_2023121_resize2.pdf by Sivaphan Wuttingam
port23_2023121_resize2.pdfport23_2023121_resize2.pdf
port23_2023121_resize2.pdf
Sivaphan Wuttingam•32 views
Nevigating Sucess.pdf by TEWMAGAZINE
Nevigating Sucess.pdfNevigating Sucess.pdf
Nevigating Sucess.pdf
TEWMAGAZINE•26 views
Integrating Talent Management Practices by Seta Wicaksana
Integrating Talent Management PracticesIntegrating Talent Management Practices
Integrating Talent Management Practices
Seta Wicaksana•150 views
Netflix Inc. by 125071027
Netflix Inc.Netflix Inc.
Netflix Inc.
125071027•11 views
Engaging Senior Leaders to Accelerate Your Continuous Improvement Program by KaiNexus
Engaging Senior Leaders to Accelerate Your Continuous Improvement ProgramEngaging Senior Leaders to Accelerate Your Continuous Improvement Program
Engaging Senior Leaders to Accelerate Your Continuous Improvement Program
KaiNexus•