A brief (25 minute talk) overview of machine learning for product managers and how to start thinking about it in your products

1. Machine Learning for
Product Managers

2. Agenda
• What is Machine Learning?
• Why now?
• Some terms and examples
• What typical cases can benefit
• What to consider

3. What is Machine learning
• Computers learn
without being
explicitly
programmed.
• Tell it what to do –
not how to do it.
• 2-5 years.

4. Why at the top of the hype cycle?
- Finding patterns is hard
- Not enough training data is
available
- High expectations not delivered

5. Types of Input
• Supervised
• Unsupervised
• Reinforcement learning

6. Types of output and use cases
• Regression – Predicting a numerical value (flight cost, Zillow home price)
• Clustering/Recommendation (Netflix, Spotify, Twitter who-to-follow, Customers also bought.)
• Drive exploration
• Understand users better than they understand themselves – Customized products.
• Anomaly detection/Recommendation (Trending, Most liked tweets, Facebook – your fiends liked, unusual
network traffic)
• Drive engagement
• Reduce complexity
• Event driven rather than user initiated.
• Classification – What kind of thing something is (face recognition, fraud detection).
• Back end. Cost reduction.
• User assisted actions (Google email response) and tagging
• Visual search, Audio search
• Dimensionality reduction
• Text synopsis
Frequently done activities or lots of data: If a typical person can do a mental task with less than one second of
thought, we can probably automate it using AI either now or in the near future. (e.g. Security video scanning)

7. Deep learning

9. Reframe as a prediction problem
• Can this be framed as a prediction problem?
• Cost of prediction will fall.
• E.g. Driving
• Breakdown of human activity
• Data
• Prediction
• Judgement
• Action
• Outcomes
https://hbr.org/2016/11/the-simple-economics-of-machine-intelligence

10. Hide the workflow/There’s no accounting for
taste.
• Teach me how to draw a picture
• Is it hard to break down this task into repeatable steps?
• Driving
• Drawing a picture
• Personalization

11. Simple and repetitive
Can now manage complex multimedia data
• Video
• Picture
• Audio
• Sensors – Apple watch, iPhone, etc.

12. Lookup – expanded memory
• QR Code
• What is this bug?
• What’s that song – Shazam
• Inventory
• Translation

13. Do you want fries with that?

14. Two more sensors – recognize and classify
• Visual Search - Computer has eyes .
• Auditory Search – Computer has
ears.
• Data capture and conversion.
• People are more willing to share
vision and audio if computerized

15. Cheap to create content
• Write articles
• Create images from text
• Create photos from sketches (animation)
• Create music
• Create audio with speech patterns

16. Optimize complex behavior
• Have you given up on a problem before?
• Route finding
• Coordinating multiple people, cars, resources

17. Pattern recognition
• Log files
• Security logs
• Root cause analysis

18. Natural Language Processing
• Natural Language Processing - Chatbots

19. How should product managers respond?
• Data
• UX
• Choose/Understand the generated model.
• Leverage existing solutions

20. Role of Product Managers changes
• Own the data. Data as a product.
• Cost of labelling
• Completeness
• Accuracy
• Rare Cases (identifying digits vs. identifying
cancers)
• Unbalanced cost of misclassification
• Parallels to UX.
• Start collecting data. Users are more likely
to provide data if machine processed than
person-processed.
• AI is only as good as the data.

21. User interfaces
- Event driven/Notifications
- When ___ then ____
- Voice, Visual, Audio
- User assist through complexity.
- Haptic response

22. User interface
• Why weekly?
• Should it be an infinite
list?
• Store previous discovers?

23. Considerations – understanding the model
• Understand why and how a model can make wrong predictions
• Explain why something is recommended (better received)
• Linear models
• Decision trees
• Clustering
• How could the product fail catastrophically (pregnancy, racism)
• Loss weighting

24. Building blocks
Leverage existing data (Google image search)
Pre-training

25. Do you really need AI/ML?
• Collect data
• Use heuristics
• Top downloads
• Cheapest
• Most popular
• How accurate is your baseline? If > 75% and not a core feature, don’t
machine learn.
• Generate one insight a week, rather than instant. (email newsletter,
rather than right now)
Netflix prize: One of the teams spent more than
2000 hours of work to deliver 8.43% improvement