Main Takeaways: - How to get realistic on proxy metrics - Launch metrics for every launch or feature

1. www.productschool.com
Measuring What Matters
by Google PM

2. CERTIFICATES
Your Product Management Certificate Path
Product Leadership
Certificate™
Full Stack Product
Management Certificate™
Product Management
Certificate™
20 HOURS40 HOURS40 HOURS

3. Corporate
Training
Level up your team’s Product
Management skills

4. Free Product Management Resources
BOOKS
EVENTS
JOB PORTAL
COMMUNITIES
bit.ly/product_resources
COURSES

6. Measuring what
matters
Metrics for product managers
Shreenath Regunathan

7. Quick what this is (and is not)
How to pick product metrics that are useful / meaningful
NOT
How to measure ads (thought I do that at Google)

8. Warning
Metrics do not absolve you of understanding users
You cannot optimize your way to success
Metrics do not yield ideas, metrics yield decision making frameworks
Metrics do not absolve you of understanding users

9. Quick background
1
Engineering
+
Consulting
@
U
niversity
ofTexas;Charles
River
A
Ssociates
2
Business
roles
in
f50
com
pany
@
neXstep
Beverages
/Coca
Cola
3
G
eneralm
anagem
ent,strategy,
productm
anagem
ent
@
G
oogle
Themes: data influence no prior EXP

10. What do we mean by
measurement
Trackable,
process
Metrics
Measurable /
SMART
Goals
Identify / learn
Insights

11. How do you arrive at useful metrics?
#4
Actioning metrics
#3
What are
meaningful proxy
metric/s
#2
Can we establish a
unambiguous
outcome metric?
#1
What is the
business / user
goal?

12. Understand the real why
Why …
● Grow users
● Grow money
● Grow satisfaction
But
there’s
more...
What is the business
/user goal?

13. Try and identify the north
star metric
Can we establish a
unambiguous outcome
metric?
● Tension between key variables
● Can they even be measured easily?
If the metric selection is uncontroversial, go back and ask more
folks!

14. Define ongoing measurable metrics
● Proxy metrics are measurable easier but are trying to get at
the ‘hard’ metric that you care about
● Examples
○ Marketers: Only spend $ on valuable users who convert
■ Proxy : High ROAS (return on ad spend)
■ Watchout: getting a lot of ready to buy users who might
have been organic conversions is not ideal for a business
○ Google Support team: inbound support requests with high quality
automated answers
■ Proxy: % automated with high satisfaction
■ Watchout: Fixing the product worsens your metrics!
What are meaningful
proxy metric/s

15. Start to make decisions with metrics
● Make key decisions with proxy metrics while checking in on north
star
● Define trade-offs for product strategy and go to market approaches
with these metrics
Actioning metrics
Real world example - measurement product launch
● Coverage of advertisers is a key metric (% of advertisers with measurement)
● But small advertisers cannot get some measurement solutions
● How do you tackle this / decide where to draw the line?
Product: Define and show eligibility - draw a path for advertisers to recognize
Incentive: Ensure org goal to not be 100% coverage, but something meaningful

16. North star & ongoing metrics
Often, North star metrics do not help you make decisions on day to day
decisions since they do not move with your ‘small’ but essential feature
Hence, companies like Google often have a basket of ongoing metrics we
track for every product or feature launch
For advertising product changes, we track a basket of metrics covering
revenue, quality, latency and more together and run staged experiments for
each launch and got phased from 0 to 100% over a few weeks
Actioning metrics

17. Do metrics matter?
Can anyone in your org callout your metrics
Do you think or decide without them often?
How often does this play into cross functional discussions?
In zero-to-one projects, sometimes metrics are not as essential for ongoing
tracking, but helped define investment decision

18. Sample example
Problem: your data team found out that X million Youtube users are paying
for YT Premium but are not consuming content
What do you do?

19. Closing
- Spend time thinking of meaningful outcome metrics
- Get realistic on proxy metrics
- Use launch metrics for every launch or feature
- Have critical reflection on past metrics choices
Don’t be fussed if your outcomes are not in sync - have a learning loop &
revise metric choice
Finally - don’t forget, information is beautiful (also favourite visualization
blog)

20. Thanks!
Reach out to me with feedback or comments
Twitter @Shreenath

21. Reference articles
Very relevant / topical
● Ship outcomes, not just features, with the Product Impact Framework | Inside Intercom
● One Metric to Rule Them All /
● OKR vs KPIs, What is the Difference? - Felipe Castro
● The ultimate guide to defining, writing, and implementing OKRs
Death by metrics
● Don't Let Metrics Undermine Your Business
● The problem with metrics is a big problem for AI · fast.ai
Useful framework - Know Your Customers' “Jobs to Be Done”

22. From the source
“OKRs are not a silver bullet. They cannot substitute for sound judgment, strong leadership,
or a creative workplace culture. But if those fundamentals are in place, OKRs can guide you to
the mountaintop.”
(From John Doerr - Measuring what matters)

23. www.productschool.com
Part-time Product Management Training Courses
and
Corporate Training