Main takeaways: - How to build Product Roadmap together with Data Science - How to Prioritize Machine Learning features - Measuring success on Machine Learning models

1. www.productschool.com
Building Scalable ML Products by
TripAdvisor PM & Data Scientist

2. Join 35,000+Product
Managers on
Free Resources
Discover great job
opportunities
Job Portal
prdct.school/PSJobPortalprdct.school/events-slack

3. C O U R S E S
Product
Management
Learn the skills you need to land
a Product Manager job

4. C O U R S E S
Coding
for Managers
Build a website and gain the
technical knowledge to lead
software engineers

5. C O U R S E S
Data Analytics
for Managers
Learn the skills to understand web
analytics, SQL and machine learning
concepts

6. C O U R S E S
Learn how to acquire more users
and convert them into clients
Digital Marketing
for Managers

7. C O U R S E S
UX Design
for Managers
Gain a deeper understanding of your
users and deliver an exceptional end-to-
end experience

8. C O U R S E S
For experienced Product Managers
looking to gain strategic skills needed
for top leadership roles
Product
Leadership

9. C O U R S E S
Corporate
Training
Level up your team’s Product
Management skills

10. T O N I G H T ’ S S P E A K E R S
Mahesh Chayel
&
Tim Conklin

11. Building scalable Machine
Learning products
@Product School, Boston
Mahesh Chayel
https://www.linkedin.com/in/maheshchayel/
Tim Conklin
https://www.linkedin.com/in/tconklin91/

12. “
Three takeaways today:
- How to build product roadmap together with
Data Science
- How to prioritize Machine Learning features
- Measuring success on Machine Learning
models
12

13. Theory and Practise
13

14. 14
Reserve prices are used by

15. Building product
roadmap together
with data science
15

16. When do Product and DS
work together?
Enhanced experience
and functionality
Finding relevant
product or estimates of
different values e.g.
Zillow estimate
Internal functions and
business logic
Improving the way
internal complex
process work e.g.
Reserve price
calculation at Ebay
Expansion to new
verticals and products
Building insights from
data to serve new
segments or product
verticals e.g. Zillow
recommends real estate
developers on what
features buyers care
about
16

17. Start from the basics
Define vision of
your product
Identify
user pain
points
(JTBD)
Explore
Solutions
17
Define problem
to be solved
Understand
user needs
Investigate
current
alternatives

18. Understand key concepts
18
Types of Learning
1. Supervised Learning
a. Regression
b. Classification
2. Unsupervised Learning
a. Clustering
b. Anomaly detection
3. Semi-supervised Learning
4. Reinforced Learning
What matters?
1. Objective function selection
2. Algorithm selection: depending
on problem to solve
3. Feature selection: depending on
availability and usefulness of
data
4. Explainability and
interpretability

19. Prioritizing
machine learning
features together
with data science
19

20. Considerations for both
PM and DS
Real time
requirements
Data and model
dependencies
Data collection
frequency
20
Data collection
methods

21. Prioritization depends on
your test/ launch plan
Prepare for a
soft launch
Plan for
experiment
s
Iterate after
launch
21

22. Example: Reserve Price
22
Progression stage Features required
Optimal reserve price theory Business rules
First basic data science model
Most impactful features + simple
model
Iterate on model A/B testing framework
More complex model versions As needed

23. Measuring success
of ML models
together with data
science
23

24. A/B Testing
The most important piece to
ensuring continuous improvement
24

25. Parameters to check for ML
model success
1. The same business
metrics
a. Acquisition
b. Activation
c. Retention
d. Revenue
e. Referral
2. ML specific metrics
Measuring success of ML
models
Frameworks for running ML
tests
1. Traffic slice based
testing
2. Product specific testing
3. Market specific testing
25

26. Example: Reserve Price
26
Testing Framework When to Use
Traffic Split Testing User behavior, no feedback loop
Market Testing Anticipated feedback loop
Product Testing Focus/small group testing

27. The Obvious reading
list
- “Data Science
for Business:
What You Need
to Know about
Data Mining
and Data-
Analytic
Thinking” by
Foster Provost
- “Storytelling
with Data: A
Data
Visualization
Guide for
Business
Professionals”
by Nussbaumer
Knaflic, Cole
- “Data Smart:
Using Data
Science to
Transform
Information into
Insight” by John
W. Foreman
27

28. www.productschool.com
Part-time Product Management, Coding, Data Analytics, Digital
Marketing, UX Design, Product Leadership courses and
Corporate Training