METRICS FOR
PRODUCT MANAGERS
Saeed Khan
@saeedwkhan
skhan@transformationlabs.io
February 25, 2020 – TPMA
USING METRICS TO DRIVE REPEATABLE
PRODUCT SUCCESS
Transformation Labs
We help organizations accelerate their
success by enabling highly effective product
teams and team members through coaching,
workshops and advisory services in areas such
as:
• Product Strategy & Roadmapping
• New Product Research & Discovery
• Organizational Assessment & Design
• Positioning & Messaging
• Win/Loss Analysis
• Product Launch
Contact me – skhan@transformationlabs.io
SAEED’S METRICS
23
6
16
45
450K
750M
Years in Product Management
Years working in the US
Products Managed
Product Releases
Kilometers of biz travel
Revenue of products managed
I know why you're here.
I know because I was once
looking for the same
thing.
I was looking for an
answer.
It's the question that
brought you here.
What
are the
Metrics?
METRICS ARE:
• Quantifiable measures
• Track, monitor and assess
• Communicate progress
• Tied to specific objectives
ATTRIBUTES OF METRICS
• Quantitative, Qualitative
ATTRIBUTES OF METRICS
• Quantitative, Qualitative
• Analytic, Operational
ATTRIBUTES OF METRICS
• Quantitative, Qualitative
• Analytic, Operational
• Lagging, Leading
ATTRIBUTES OF METRICS
• Quantitative, Qualitative
• Analytic, Operational
• Lagging, Leading
• Actionable, Vanity
ATTRIBUTES OF METRICS
• Quantitative, Qualitative
• Analytic, Operational
• Lagging, Leading
• Actionable, Vanity
• Direct, Proxy
Which
Metrics
should I
track?
User Metrics
# of Sessions/User
Session Duration
Key Actions/Sessions
# of New Users
New Users/month
User Growth Rate
Customer Service
# of Incoming Support Tickets
Net Promoter Score (NPS)
Ave. First Response Time
Average Time to Close
First Contact Resolution Rate
Customer Satisfaction Rate
Business Metrics
Revenue per day/week/month
Revenue Growth Rate
Customer Lifetime Value (CLV)
Customer Acquisition Cost (CAC)
Ave. Revenue per Account (ARPA)
Monthly Recurring Revenue (MRR)
Annual Contract Value (ARR)
Monthly Churn Rate
Retention Rate
Mobile Metrics
Downloads
Activations
UnInstalls
Daily Active Users
Monthly Active Users
Session Length
Session Interval
Ratings
In-app Purchases
Ecommerce
Average Order Value
Cart Abandonment
Checkout Abandonment
Ave. Shipping Cost
Repeat Customers
Visit/Sale Conversion
Cost/Acquisition
API Metrics
API Calls / Day/Week
Call Failure Rate
Call Response Time
Actions per day/week
Uptime/Downtime
Agile Metrics
Sprint Burndown Rate
Velocity
Release Burnup
Feature Cycle Time
Missed Bugs/Defect Rate
Failed Deployments
Epic Burndown
Defects/Story
Funnel Metrics
Product-Qualified Leads
Qualified Lead Velocity Rate
Organic vs. Paid Traffic ROI
Viral Coefficient
Conversion Rate to Customer
Metrics:
• depend on context
• Must define what, who, when etc.
• imperfect indicators of what we
want to achieve
Which
Analytics
tools should
I use?
PRODUCT ANALYTICS TOOLS
H E A P
KISSmetrics
ANALYTICS PRODUCT STACK
Attribution
Where do your users
come from?
Aggregation
Collect, standardize
and share data
Analysis
Identify Insights
from data
Action
Engage users based
on insights
H E A P
Tools:
• Increase productivity
• Require learning and process
changes
• Are imperfect for many situations
Which
Framework
should I
use?
Pirate
HEART
North Star
Metrics Frameworks
OKRs
Focus
GAME
https://hackernoon.com/metrics-game-framework-5e3dce1be8ac
Frameworks:
• Provide structure and guidance
• Must be applied appropriately for
the given objective
• Are imperfect models
ANALYTICS, METRICS, KPIS
Analytics: analysis of (large) data sets to
identify patterns, trends, & other insights
Metrics: Quantifiable measures to track,
communicate state of business processes
KPIs: important (key) measures that track
progress towards big (strategic) goals.
Data
Information
Knowledge
Insight
47.89
48.59
47.13
47.48
48.07
48.65
48.62
49.39
DATA (USED IN ANALYTICS)
47.89
48.59
47.13
47.48
48.07
48.65
48.62
49.39
DATA (+AGGREGATED)
47.13, 47.48, 48.07, 48.65, 48.62, 49.39
47.13oF, 47.48oF, 48.07oF, 48.65oF, 48.62oF, 49.39oF
INFORMATION (+METADATA)
INFORMATION (+CONTEXT)
1960s, 1970s, 1980s, 1990s, 2000s, 2010s
47.13oF, 47.48oF, 48.07oF, 48.65oF, 48.62oF, 49.39oF
VISUALIZATION – KNOWLEDGE
Ave. Temperature Per Decade in Buffalo NY
https://www.weather.gov/buf/BUFtemp
Ave. Temperature Per Decade in Buffalo NY
https://www.weather.gov/buf/BUFtemp
VISUALIZATION – KNOWLEDGE??
"You see there is
only one
constant.…Causality.
Action, reaction.
Cause and effect.
ACTIONS OUTPUTS METRICS OBJECTIVES
(CAUSE) (EFFECT)
Product Management
lives here
Product Lifecycle Stages
Develop
Launch
Growth
Maturity
Decline
EndofLife
Sales
Time
Product Lifecycle Stages
Product Lifecycle Stages
Buildit
Nailit
Scaleit
Extendit
Milkit
Endit
Sales
Time
Product Lifecycle Objectives
Product Lifecycle Objectives
Objective Description
Build it
Nail it
Scale it
Extend it
Milk it
End it
Build the first version of the product for
specific use case(s) for a target market
Identify & address barriers to wider adoption in
your target market & prepare company for product
growth – more than product/market fit
Scale business (marketing/sales/services/operations
etc.). Focus on expansion & new customer acquisition
Move into new markets, segments, use cases etc.
Growth in new customers & sales into existing base.
Reduce investment but continue to market/sell with
an eye on maximizing profits from customer base
Remove all investment, stop actively marketing and
eventually remove from market
Buildit
Product Lifecycle Objectives
Scaleit
Nailit
Extendit
Endit
Milkit
Product Lifecycle Metrics
A single, holistic model
to track product health
and success across the
product lifecycle.
4 Independent Categories of Focus
Business Organizational
Readiness
Go-To-Market Product
What is important in each area?
Business
Corporate Objectives, Revenue,
Pipeline, Win/Loss, Renewals,
Pricing, etc.
Organization
Internal Training and Enablement
(Sales, Support, Marketing etc.),
Partners, Channels etc.
Go To Market
Positioning, Messaging, Lead
Gen, Marketing Funnel, Sales
Metrics, Collateral etc.
Product
Product Strategy, Capabilities,
Roadmap, Quality, Performance,
Stability, Gaps etc.
At each lifecycle stage,
Product Management focus
and objectives can be
different!
Focus Areas vs. Lifecycle Objectives
Build it Nail it Scale it Extend it Milk it End it
Business
Go-To-Market
Organizational
Readiness
Product
Focus Areas vs. Lifecycle Objectives
Light Focus Medium Focus Heavy Focus
Focus Areas vs. Lifecycle Objectives
Build it Nail it Scale it Extend it Milk it End it
Business
Go-To-Market
Organizational
Readiness
Product
Focus Areas vs. Lifecycle Objectives
Light Focus Medium Focus Heavy Focus
Focus Areas vs. Lifecycle Objectives
Build it Nail it Scale it Extend it Milk it End it
Business
Go-To-Market
Organizational
Readiness
Product
Focus Areas vs. Lifecycle Objectives
Light
Focus
Medium Focus Heavy Focus
Focus Areas vs. Lifecycle Objectives
Build it Nail it Scale it Extend it Milk it End it
Business
Go-To-Market
Organizational
Readiness
Product
Focus Areas vs. Lifecycle Objectives
Light Focus Medium Focus Heavy Focus
Focus Areas vs. Lifecycle Objectives
Build it Nail it Scale it Extend it Milk it End it
Business
Go-To-Market
Organizational
Readiness
Product
Focus Areas vs. Lifecycle Objectives
Light Focus Medium Focus Heavy Focus
Focus Areas vs. Lifecycle Objectives
Build it Nail it Scale it Extend it Milk it End it
Business
Go-To-Market
Organizational
Readiness
Product
Product is in 1 stage at a given time
Light Focus Medium Focus Heavy Focus
Implementing the Model
What questions are important to
answer for each focus area for the
current stage of the product
lifecycle
i.e. the key business metrics 
Sample Questions to Ask
Business
Organizational
Readiness
Go-To-Market
Product
• Are revenues meeting targets?
• What does our current pipeline look
like?
• What does our deal trend look like?
Up/Down?
• What is our NPS? Is it
increasing/decreasing?
• Which pricing levels are performing
to plan?
• Which pricing levels are not?
• Is our acquisition rate growing?
• Is our retention solid?
Sample Questions to Ask
Business
Organizational
Readiness
Go-To-Market
Product
• Are our sales people & sales engineers
trained on the latest product?
• Are Marketing lead gen programs
executing to plan?
• Have our Services teams started engaging
with customers on implementation?
• What is the state of our Partner
enablement program?
• Is Customer Success fully trained on
current version?
• Is Customer Success meeting it’s targets
for new customer implementation
Business Metrics
Organizational Readiness
Product LifeCycle Metrics
A common and holistic view of the
key metrics for your product.
A means to identify, communicate
and proactively resolve issues
Woah…
I know
Metrics.
Rich
Mironov
Saeed Khan
Your Feedback Please
3 Questions
2 Minutes
1 Happy Speaker
http://bit.ly/tpmametrics
Transformation Labs
We help startups and larger organizations
accelerate their success by enabling highly
effective product teams and team members
through coaching, workshops and advisory
services in areas such as:
• Product Strategy & Roadmapping
• New Product Research & Discovery
• Organizational Assessment & Design
• Positioning & Messaging
• Win/Loss Analysis
• Product Launch
Contact me – skhan@transformationlabs.io
METRICS FOR
PRODUCT MANAGERS
Saeed Khan
@saeedwkhan
skhan@transformationlabs.io
February 25, 2020 – TPMA
USING METRICS TO DRIVE REPEATABLE
PRODUCT SUCCESS

Metrics for product managers tpma-feb2020

  • 1.
    METRICS FOR PRODUCT MANAGERS SaeedKhan @saeedwkhan skhan@transformationlabs.io February 25, 2020 – TPMA USING METRICS TO DRIVE REPEATABLE PRODUCT SUCCESS
  • 2.
    Transformation Labs We helporganizations accelerate their success by enabling highly effective product teams and team members through coaching, workshops and advisory services in areas such as: • Product Strategy & Roadmapping • New Product Research & Discovery • Organizational Assessment & Design • Positioning & Messaging • Win/Loss Analysis • Product Launch Contact me – skhan@transformationlabs.io
  • 3.
    SAEED’S METRICS 23 6 16 45 450K 750M Years inProduct Management Years working in the US Products Managed Product Releases Kilometers of biz travel Revenue of products managed
  • 4.
    I know whyyou're here. I know because I was once looking for the same thing. I was looking for an answer. It's the question that brought you here.
  • 5.
  • 6.
    METRICS ARE: • Quantifiablemeasures • Track, monitor and assess • Communicate progress • Tied to specific objectives
  • 7.
    ATTRIBUTES OF METRICS •Quantitative, Qualitative
  • 8.
    ATTRIBUTES OF METRICS •Quantitative, Qualitative • Analytic, Operational
  • 9.
    ATTRIBUTES OF METRICS •Quantitative, Qualitative • Analytic, Operational • Lagging, Leading
  • 10.
    ATTRIBUTES OF METRICS •Quantitative, Qualitative • Analytic, Operational • Lagging, Leading • Actionable, Vanity
  • 11.
    ATTRIBUTES OF METRICS •Quantitative, Qualitative • Analytic, Operational • Lagging, Leading • Actionable, Vanity • Direct, Proxy
  • 12.
  • 14.
    User Metrics # ofSessions/User Session Duration Key Actions/Sessions # of New Users New Users/month User Growth Rate Customer Service # of Incoming Support Tickets Net Promoter Score (NPS) Ave. First Response Time Average Time to Close First Contact Resolution Rate Customer Satisfaction Rate Business Metrics Revenue per day/week/month Revenue Growth Rate Customer Lifetime Value (CLV) Customer Acquisition Cost (CAC) Ave. Revenue per Account (ARPA) Monthly Recurring Revenue (MRR) Annual Contract Value (ARR) Monthly Churn Rate Retention Rate Mobile Metrics Downloads Activations UnInstalls Daily Active Users Monthly Active Users Session Length Session Interval Ratings In-app Purchases Ecommerce Average Order Value Cart Abandonment Checkout Abandonment Ave. Shipping Cost Repeat Customers Visit/Sale Conversion Cost/Acquisition API Metrics API Calls / Day/Week Call Failure Rate Call Response Time Actions per day/week Uptime/Downtime Agile Metrics Sprint Burndown Rate Velocity Release Burnup Feature Cycle Time Missed Bugs/Defect Rate Failed Deployments Epic Burndown Defects/Story Funnel Metrics Product-Qualified Leads Qualified Lead Velocity Rate Organic vs. Paid Traffic ROI Viral Coefficient Conversion Rate to Customer
  • 15.
    Metrics: • depend oncontext • Must define what, who, when etc. • imperfect indicators of what we want to achieve
  • 17.
  • 18.
    PRODUCT ANALYTICS TOOLS HE A P KISSmetrics
  • 19.
    ANALYTICS PRODUCT STACK Attribution Wheredo your users come from? Aggregation Collect, standardize and share data Analysis Identify Insights from data Action Engage users based on insights H E A P
  • 20.
    Tools: • Increase productivity •Require learning and process changes • Are imperfect for many situations
  • 21.
  • 22.
  • 23.
    Frameworks: • Provide structureand guidance • Must be applied appropriately for the given objective • Are imperfect models
  • 24.
    ANALYTICS, METRICS, KPIS Analytics:analysis of (large) data sets to identify patterns, trends, & other insights Metrics: Quantifiable measures to track, communicate state of business processes KPIs: important (key) measures that track progress towards big (strategic) goals.
  • 25.
  • 26.
  • 27.
    47.89 48.59 47.13 47.48 48.07 48.65 48.62 49.39 DATA (+AGGREGATED) 47.13, 47.48,48.07, 48.65, 48.62, 49.39 47.13oF, 47.48oF, 48.07oF, 48.65oF, 48.62oF, 49.39oF INFORMATION (+METADATA) INFORMATION (+CONTEXT) 1960s, 1970s, 1980s, 1990s, 2000s, 2010s 47.13oF, 47.48oF, 48.07oF, 48.65oF, 48.62oF, 49.39oF
  • 28.
    VISUALIZATION – KNOWLEDGE Ave.Temperature Per Decade in Buffalo NY https://www.weather.gov/buf/BUFtemp
  • 29.
    Ave. Temperature PerDecade in Buffalo NY https://www.weather.gov/buf/BUFtemp VISUALIZATION – KNOWLEDGE??
  • 30.
    "You see thereis only one constant.…Causality. Action, reaction. Cause and effect.
  • 31.
    ACTIONS OUTPUTS METRICSOBJECTIVES (CAUSE) (EFFECT)
  • 32.
  • 33.
  • 34.
  • 35.
    Product Lifecycle Objectives ObjectiveDescription Build it Nail it Scale it Extend it Milk it End it Build the first version of the product for specific use case(s) for a target market Identify & address barriers to wider adoption in your target market & prepare company for product growth – more than product/market fit Scale business (marketing/sales/services/operations etc.). Focus on expansion & new customer acquisition Move into new markets, segments, use cases etc. Growth in new customers & sales into existing base. Reduce investment but continue to market/sell with an eye on maximizing profits from customer base Remove all investment, stop actively marketing and eventually remove from market
  • 36.
  • 37.
    Product Lifecycle Metrics Asingle, holistic model to track product health and success across the product lifecycle.
  • 38.
    4 Independent Categoriesof Focus Business Organizational Readiness Go-To-Market Product
  • 39.
    What is importantin each area? Business Corporate Objectives, Revenue, Pipeline, Win/Loss, Renewals, Pricing, etc. Organization Internal Training and Enablement (Sales, Support, Marketing etc.), Partners, Channels etc. Go To Market Positioning, Messaging, Lead Gen, Marketing Funnel, Sales Metrics, Collateral etc. Product Product Strategy, Capabilities, Roadmap, Quality, Performance, Stability, Gaps etc.
  • 40.
    At each lifecyclestage, Product Management focus and objectives can be different!
  • 41.
    Focus Areas vs.Lifecycle Objectives Build it Nail it Scale it Extend it Milk it End it Business Go-To-Market Organizational Readiness Product Focus Areas vs. Lifecycle Objectives Light Focus Medium Focus Heavy Focus
  • 42.
    Focus Areas vs.Lifecycle Objectives Build it Nail it Scale it Extend it Milk it End it Business Go-To-Market Organizational Readiness Product Focus Areas vs. Lifecycle Objectives Light Focus Medium Focus Heavy Focus
  • 43.
    Focus Areas vs.Lifecycle Objectives Build it Nail it Scale it Extend it Milk it End it Business Go-To-Market Organizational Readiness Product Focus Areas vs. Lifecycle Objectives Light Focus Medium Focus Heavy Focus
  • 44.
    Focus Areas vs.Lifecycle Objectives Build it Nail it Scale it Extend it Milk it End it Business Go-To-Market Organizational Readiness Product Focus Areas vs. Lifecycle Objectives Light Focus Medium Focus Heavy Focus
  • 45.
    Focus Areas vs.Lifecycle Objectives Build it Nail it Scale it Extend it Milk it End it Business Go-To-Market Organizational Readiness Product Focus Areas vs. Lifecycle Objectives Light Focus Medium Focus Heavy Focus
  • 46.
    Focus Areas vs.Lifecycle Objectives Build it Nail it Scale it Extend it Milk it End it Business Go-To-Market Organizational Readiness Product Product is in 1 stage at a given time Light Focus Medium Focus Heavy Focus
  • 47.
    Implementing the Model Whatquestions are important to answer for each focus area for the current stage of the product lifecycle i.e. the key business metrics 
  • 48.
    Sample Questions toAsk Business Organizational Readiness Go-To-Market Product • Are revenues meeting targets? • What does our current pipeline look like? • What does our deal trend look like? Up/Down? • What is our NPS? Is it increasing/decreasing? • Which pricing levels are performing to plan? • Which pricing levels are not? • Is our acquisition rate growing? • Is our retention solid?
  • 49.
    Sample Questions toAsk Business Organizational Readiness Go-To-Market Product • Are our sales people & sales engineers trained on the latest product? • Are Marketing lead gen programs executing to plan? • Have our Services teams started engaging with customers on implementation? • What is the state of our Partner enablement program? • Is Customer Success fully trained on current version? • Is Customer Success meeting it’s targets for new customer implementation
  • 50.
  • 51.
  • 52.
    Product LifeCycle Metrics Acommon and holistic view of the key metrics for your product. A means to identify, communicate and proactively resolve issues
  • 53.
  • 54.
    Rich Mironov Saeed Khan Your FeedbackPlease 3 Questions 2 Minutes 1 Happy Speaker http://bit.ly/tpmametrics
  • 55.
    Transformation Labs We helpstartups and larger organizations accelerate their success by enabling highly effective product teams and team members through coaching, workshops and advisory services in areas such as: • Product Strategy & Roadmapping • New Product Research & Discovery • Organizational Assessment & Design • Positioning & Messaging • Win/Loss Analysis • Product Launch Contact me – skhan@transformationlabs.io
  • 56.
    METRICS FOR PRODUCT MANAGERS SaeedKhan @saeedwkhan skhan@transformationlabs.io February 25, 2020 – TPMA USING METRICS TO DRIVE REPEATABLE PRODUCT SUCCESS

Editor's Notes

  • #2 https://www.wallpaperflare.com/city-view-photo-during-night-time-sydney-harbour-bridge-australia-wallpaper-472/1440x900
  • #5 Sorry, a bit of a glitch here. Let me fix that. Any fans of the movie “The Matrix”? So what is the question that brought you here?
  • #6 So what is a metric? Anyone have a definition?
  • #15 So what’s the next question?
  • #16 If you go online, you’ll find lots of articles that tell you exactly what metrics to track. Ignore them.
  • #20 So what’s the next question?
  • #21  Do you use any of these tools? What do you use them for? If not, why not?
  • #22  Do you use any of these tools? What do you use them for? If not, why not?
  • #26 So what’s the next question?
  • #31  Do you use any of these tools? What do you use them for? If not, why not?
  • #32  Do you use any of these tools? What do you use them for? If not, why not?
  • #33  Do you use any of these tools? What do you use them for? If not, why not?
  • #34  Do you use any of these tools? What do you use them for? If not, why not?
  • #62 https://www.wallpaperflare.com/city-view-photo-during-night-time-sydney-harbour-bridge-australia-wallpaper-472/1440x900