There are no sequence on the steps What are the best practices? What models out there? Pros and cons Cohort analysis? Shift over time? Seasonality? School time? Look at the other domains Examples Snapchat features Click speed, guess age, segment, speed of page view, message type Some users are super engaged : do not need to do anything All A/B test? Public features Have an outline: story telling. Why? More deep on 1 or 2 of the bubbles: examples, something more deep than the surface Customize landing page: how much? Push notification in Facebook
As for new or existing users, you could focus on the churn and/or super engaged activities but they have different set of features and different outputs
Explain why you are asking for permission, what does user get Does new services/popular service create more engagement: bitmoji
You know what CEOs want: predicting if user is super engaged in 12 months after the first session?
You do not need to know the answer from the start
Do not commit to one metric
Obama campaign Facebook notification Use user segmentation in A/B test
Have a cool chart with all these dimensions in one place / shows relationships / dependencies
Challenges of Predicting User Engagement
Challenges of Predicting
Data Scientist @ Snapchat
When do you
want to know?
How utilize the
• New users
• Optimizing the registration and onboarding process
• What activities in the first hours/days help users the most to retain/ cause churn?
• How many friends?
• Existing users
• Change in behavior -> what direction they are going
• What is the effect of certain experiences -> To optimize those processes
• Both could be focused on churn and/or super engaged but usually they have
different set of features and outputs
• You know a lot about the users
• Desktop/mobile, time of the day, Android/iOS, PC/Mac, OS version, navigation/messaging
speed, high/low penetration market, permissions
• Need to know user segmentation/personas
• High intent to buy/window shopper, Creators/consumers, adopting fast, level of engagement
More on Who?
• There is always trade off between accuracy and knowing ASAP
• Overcome the false positives by applying a solution that does not have a high
negative effect on the false positives
• Annoying engaged users with push notification?
• Using different solutions depend on user persona/tenure/focus
• Time is critical for churn users
• Short-term / long-term metrics
• Long-term metrics are harder to predict
• What if you have a sporadic purchase behavior?
• Avg purchase once in a quarter: it would be harder to predict
in the next week
• Keep an eye on the seasonality trends
• if a segment of users are coming only on the weekends, then
better to look at the metric over a week
What Time Frame?
• Which one is more important?
• Reducing churn
• Increasing engagement
• It helps you to define the metrics
• It does not necessarily mean different models
• What metric you are looking at?
• Conversion, Time spend, create content, active number of days a week
• Do we have to look at one metric for all the users?
• Users with 10 friends vs. 1 friend in the first week?
• Users in 3 zone:
• Red zone: High probability to churn
• Yellow zone: Low engagement
• Green zone: High engagement
• Have an engagement score
• Daily run to know the engagement score for each user
• What is the segment that has the most change in last x days/months?
• Use engagement to predict other metrics such as customer value
• Optimizing the onboarding process: What activity to suggest users to do
based on the stage they are in (Red / Yellow / Green Zone)
• Optimizing push notification
• A/B test or not A/B test: testing on personas