Download to read offline
Uploaded byDave Kellogg
Lagging, Leading, and Predictive Indicators
- Leading indicators are metrics that can predict future business performance, while lagging indicators analyze past performance. Leading indicators include things like early stage opportunities and marketing funnel metrics that predict future deals and revenue. - It is important to focus on leading indicators now given potential economic challenges, but metrics can become distorted if over-managed through goals and targets. - Companies should use both operational metrics to manage the business and predictive analytics to understand where to focus resources for future growth. Data modeling can help identify customer attributes that predict faster, bigger deals or higher renewal rates.
More Related Content
Similar to Lagging, Leading, and Predictive Indicators
Lagging, Leading, and Predictive Indicators
- 1. Thoughts on Lagging,Leading, and Predictive Indicators Dave Kellogg Foundry CFO Summit 10/27/22 Revision 1.6 www.Kellblog.com https://twitter.com/Kellblog
- 2. Who is ThisGuy? • Independent consultant, EIR at Balderton Capital, and blogger. • Former operator • CEO: MarkLogic, Host Analytics (Planful) • CMO: Versant, BusinessObjects, Alation (gig) • GM: Salesforce.com • Independent board director* • Alation, Aster Data, CyberGuru, Granular, Nuxeo, Profisee, Scoro, SMA • Advisor* • Examples: Bluecore, GainSight, Tableau, MongoDB, Pigment, Recorded Future • Angel / investor • Examples: Alation, Cube, Cuein, DataGrail, FloQast, Growblocks, Hex, Saurus, Skyflow 2 * List includes both current and former roles
- 3. While the CEOis on the bridge looking forward, many finance teams are on the stern, offering in-depth analyses of the ship’s wake 3
- 4. I ❤ SaaSMetrics • ARR growth • Net dollar retention (NDR, aka NRR) • CAC ratio • Magic number • CAC payback period • Churn rate • LTV/CAC • Rule of 40 score 4 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 5. I ❤ SaaSMetrics, But Which Are They? • ARR growth • Net dollar retention (NDR, aka NRR) • CAC ratio • Magic number • CAC payback period • Churn rate • LTV/CAC • Rule of 40 score 5 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International Forward-looking o backward-looking?
- 6. Forward-Looking Metrics HaveNever Been More Important 6 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 7. Forward-Looking Metrics HaveNever Been More Important 7 2001 Dotcom Bubble Burst 2008 Financial Crisis (Lehman was one of my investors, AMA.) 2022 To-Be-Named Crisis (Except for the other two times) Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 8. Why Are LeadingMetrics So Important Right Now? Potential Inflection Point • Cannot extrapolate recent past • Need to look up and out • Up the funnel and outside the company Fear-Greed Meter Recalibration • Some VCs flip too fast to red • Some founders perma-stuck on green • This is good! We’re counter-cyclical!! 8 Only the Paranoid Survive, Grove Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 9. The real questionisn’t, “what’s happening?” it’s, “what’s happening to us?” 9 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 10. What’s a LeadingIndicator? • Let’s take an example • Annual recurring revenue (ARR) • Leading? • Lagging? • IMHO, it’s both • ARR leads subscription revenue • ARR lags new ARR bookings (and churn ARR) 10 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 11. Let’s Try Again •Stage 1 oppties • Leading? • Lagging? • Again, IMHO it’s both • S1 oppties lead S2 oppties and … closed deals • Insert s2-to-close rate and sales cycle length • S1 oppties lag MQLs and leads 11 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 12. The Marketing FunnelView • Each layer leads the one below • And lags the one above • Please note the potentially considerable irony in telling marketing to focus up-funnel • For 30 years we’ve been moving them down: stop celebrating leads, celebrate MQLs; no, s1 oppties; no, s2 oppties; no, closed deals; no, closed deals that don’t churn, … 12 Visitors Names Responses Leads MQL SAL Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 13. And The TwoFunnels Tie Together SAL (S1) SQL (S2) Solution Fit Demo Vendor of Choice Legal Won Visitors Names Responses Leads MQL SAL Marketing Sales 13 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 14. The Worst Dayof My Marketing Life • In the early CRM days, we watched pipeline coverage • Current-quarter pipeline / current- quarter target • We knew about the 3x rule • Pipeline coverage was a pretty good predictor of bookings • Maybe better than the CRO forecast • We could use pipeline coverage to manage the business 14 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 15. And Then ThisGuy Came Along ● We have determined that reps with 3x pipeline coverage generally hit their number ● Ergo, I will pummel anyone unless they have 3x pipeline coverage ● At all levels in the organization ● And then what happened? 15 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 16. Everyone Had 3xCoverage! • And it was mostly ruined as a predictive metric 16 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 17. “What gets measured,gets managed.” -- Peter Drucker (Well, it’s fairly clear he didn’t says this. See note.) Who else, did precise or roughly? See The Dysfunctional Consequences of Performance Measurements – VF Ridgway (1956) “When a measure becomes a target, it ceases to be a good measure.” -- Charles Goodhart (1975) “When you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind.” Lord Kelvin (1883) So I’ll say my take on it “Managing a metric -- e.g., setting OKRs on it, putting into it standard reports -- ruins it as predictor, as a free indicator of business performance.” – Kellogg (2022) 17 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International Note bene: thanks to Ronny Kohavi for calling out some mistakes in the prior version.
- 18. So When WeSay We Want Leading Indicators ● Is that to predict business performance or to manage it? ● Because you don’t really get to choose both 18 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 19. Demos, Ice Cream,and Drownings ● Deals that get to demo have a 31% chance of closing ○ Therefore, we need more demos ● Wait, do they close at 31% because we did the demo or because we filtered out so many tire kickers along the way? ○ If we reduce the filtering to do more demos won’t that reduce the close rate? ● Drowning deaths and ice cream production are strongly correlated ○ Does that mean that ice cream production causes drownings? 19 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 20. So What DoWe Actually Want? Operational Metrics To Manage and Predict • Manage -- metrics on which we can assign OKRs to managers • Predict – what can best tell us where we’re going to land? • Example: it’s a bad idea to tell marketing to go worry about names because they’re leading indicator Analytics To Determine Where To Go • Focus – on what should we focus time and money? • Example: it’s a good idea to ask the data science team which customers close bigger/faster/higher 20 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 21. Operational Metrics, TheClassics • Funnel volume, conversion rates • Traffic • MQLs • MQL > S1 • S1 > S2 • S2 > S4 • S4 > close • Pipeline progression • Opportunity histogram • Triangulation forecasting 21 Now is the time when you benefit from having done it right all the way along Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 22. Note That It’sActually A Rotten Time To Change How We Do Everything • Compounds extrapolation risk with • Invalidation of historical comparison data • That is, let’s not go change pipeline stage definitions, MQL definition, pipeline scrubbing process, … • (Let’s do that in fair weather, not foul.) 22 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 23. Operational Metrics, ALittle More Creative • Activity as predictor of closing • The non-obvious Gong use-case • Renewal intent as predictor of churn • Better than NPS (loosely coupled) • Post-deployment CSAT • Product usage • Win/close rates segmented by industry and use case • Who buys your product and why is quite possibly changing • Example: selling CI based on onboarding vs. productivity • Relationship score as predictor of close • Build some score that indicates if you’re selling higher and to the business 23 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 24. Analytics to GuideUs • Ceteris paribus, we want deals that close • Bigger – more ARR per unit work • Faster – more velocity is “like adding a month to your year” • Higher – win rate, i.e., probability of close • Ceteris paribus, we want customers who • Renew – need to at least pass CAC payback period • Expand – everybody loves NRR • How do we find them? • This is an analytics / data science problem 24 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 25. Build Data Models(ICP Re-evaluation) • Use an internal team if you have one, find an MSDS intern if you don’t • Techniques: logistic regression, random forests, … • Goal: model that predicts outputs given inputs • Inputs are attributes of a customer (as many as you can get) • Outputs are what “success” means to you – surprisingly elusive • What is success? • Renewed? Expanded? Deployed? Landed big? Landed fast? Is a reference? • Beware two things • Interpretability problem: these can score customers, but typically can’t say “go look for blue eyes” • Extrapolation problem: we are at inflection point -- this might produce Web3 as a great target 25 Dave Kellogg, Creative Commons Attribution, Non Commercial, No Derivatives 4.0 International
- 26. Conclusions • This isyour company • The CFO is on the right • Now is the time you get rewarded for having done metrics right • (If you haven’t, the best time to start is today.) • We want leading indicators to help us with our instrument landing • A funnel view is inherently leading when you “look up” • Beware using managed metrics for prediction • But we also want to figure out which airport to fly to • That’s where model-building (ICP re-evaluation) can help https://www.getsurrey.co.uk/news/surrey-news/cockpit-view-shows-pilot-land-10373724
Editor's Notes
- #14 The marketing funnel isn’t linear, but most people report on something like this I hate demo as a stage but that’s a discussion for another day