Duke University - Masters of Engineering management Product Management club - Session 4 ppt

1. Being a Product Manager
Session 4: Metrics

2. Data is King.
Intuition is important, but...
Metrics!

3. Customer metrics 101
DAU - Daily Active Users
WAU - Weekly Active Users
MAU - Monthly Active Users
Vanity metrics!
Why?

4. Suppose you’re the PM of LinkedIn NewPosts

5. Always give context
for your
DAU/WAU/MAU

6. Churn Rate
% of customers who stopped using your product over a specific time period.
Calculation is simple.
Take the number of customers that you lost last quarter and divide that by the
number of customers that you started with last quarter. The resulting percentage
is your churn rate.
Example: SplitWise had 100 million users on January 1, 2019. It has 97 million
users on April 1, 2020. What is the churn rate?
0.03

7. Customer metrics contd.
● Product usage/adoption (# of sessions, sign-in frequency, sharing, etc.)
● Percent of users who take a specific action that matters
● Feature usage (usage vs. other features)
● Which customer type is using certain features
● Retention or churn rate
● Quality (e.g. average bugs, net promoter score)
Net promoter score measures the customers’ willingness to recommend a product or service to others.
Source: https://www.medallia.com/net-promoter-score/

8. Types of metrics – Business metrics
● Customer lifetime = 1/Churn Rate
○ For example, 30% of our customers are leaving our business every year. What is the customer
lifetime?
● Avg Revenue per User (ARPU) = (Total revenue)/# of users
○ For example, SottaIndustries had a revenue of $250mn with a total customer base of $100mn.
What is the ARPU?
● Lifetime Value (LTV) = Monthly Revenue per user x Customer lifetime
○ For example, PrateekNetworks has a per user monthly revenue of $15 and a churn rate of 0.2.
What is the customer LTV?
● Customer acquisition cost (CAC) = Marketing and sales expense/New customers
○ For example, HopperIndustries had an MSE of $5mn over the last quarter. It acquired 500,000 new
customers over the same time period. What is the CAC?
● Click through rate (CTR) = % of people clicking on a link
● Any many more!

9. For the interview...

10. AARM method
Acquisition: Tracking customer signups for a service. The bar for signing up for a service has gotten lower
and lower, thanks to the popularity of free signup and pay later “freemium” models. The typical acquisition
metric to track is lazy registrations or app downloads.
Activation: Getting users that have completed a lazy registration to register fully. For a social networking
site like Google+, this may include uploading a photo or completing their profile page.
Retention: Getting users to use the service often and behave in a way that helps the user or business. Key
metrics include adding more information to their profile page, checking the news feed frequently or inviting
friends to try the service.
Monetization: Collecting revenue from users. It could include the number of people who are paying for the
service or the average revenue per user (ARPU).

11. Exercise 1
What are the top metrics you would track for an eCommerce website?
Things to consider:
● Examples of top e-commerce websites include:
○ US: Amazon, Walmart, Apple
○ India: Amazon, Flipkart, Snapdeal
○ China: Tmall, JD, Suning
● Consider sales, marketing and website metrics
● Use AARM method: Acquisition, Activation, Retention, Monetization
Source: The Product Manager Interview by Lewis C. Lin

12. Exercise 1 - Model Answer
Acquisition
● Daily sessions
● Cost per acquisition (CPA)
● Cost per click (CPC)
● Cost per impression
(CPM)
● Top search engine terms
leading to website
● Mailing list click through
rate (CTR)
● Mailing list open rate
● Mailing list conversion rate
● Mobile app downloads
Activation
● New registered users
● Mobile app opens
● # of searches on app
● New cart started
● User info given (address,
credit card, etc)
● New customers with
successful purchase
Retention
● Conversion rate
● Recommendation engine
conversion rate
● Shopping cart abandonment
● Shopping cart size
● Visits from activated users
per month
Source: The Product Manager Interview by Lewis C. Lin

13. Exercise 1 - Model Answer
Monetization
● Revenue per customer
● Lapsed customers
● Purchases per year
● Revenue per click
● Cost of sale (ad spend/revenue)
● Customer lifetime value (LTV)
● Cost of shipping
Others
● Average listing position on Google
● COGS
● Shipping time
● Stockouts
● Returns
● Checkout errors
● # of reviews
● Market share
● Net margin
Source: The Product Manager Interview by Lewis C. Lin
Another question? What are the top metrics you would track for a two-sided marketplace? Do it with friends
at home.

14. A popular (and easy) interview question...

15. Root cause analysis
Product XYZ witnessed a 10% drop in DAU. Diagnose the issue.

16. Why do interviewers ask this question?
● Structure
● Fire-fighting
● Exhaustive list of issues
● Ambiguity

17. How to answer?
● Structured
● Insightful questions
● Think from various perspectives
○ Sudden drop vs progressive drop
○ Geography
○ Platform: mobile (iOS, Android), desktop, wearables, etc
○ Demographics: age, gender, etc.
○ Cannibalization: New feature release can hurt existing features
○ PR incident: Facebook political ads
○ Competitors: Released something new?
○ Technical issues

18. Exercise 2
● Live mock
● Need a volunteer...

19. More questions related to metrics:
1. What are the metrics you would like to track for xyz feature?
2. Most important metric for xyz feature?
3. Why is it drop a drop and how can we change it?