Recommended
More Related Content
What's hot
What's hot (20)
Similar to Machine Learning for Product Leaders.
Similar to Machine Learning for Product Leaders. (20)
Recently uploaded
Recently uploaded (20)
Machine Learning for Product Leaders.
- 1. ML Models in Product Management Preyas Joshi 16 years in Product Management Amazon, Ex-CTO of Fundation, entrepreneur, fintech Led multi-disciplinary teams – PM, S/W Engineering, Applied Sci P&L for multi-billion dollar businesses Launched several multi-million dollar businesses Product, Tech Advisor for several startups Patent holder in secure QR code tech Father and husband. Love to read, hike and stay fit.
- 2. ML Models in Product Management Disclaimer All opinions I state today, either in this video or when we chat after this session, are my own and not the views of my current or previous employer. Examples, projects or user stories do NOT depict, nor speculate about projects or experiments with my current or former employers.
- 3. ML in Product Management Preyas Joshi What to expect TYPES OF MODELS USER STORIES ORGANIZATIONAL STRUCTURE WHY ML? WHY NOW? HOW DO YOU IDENTIFY AN ML IDEA? Supervised Unsupervised Reinforced CRM, Lending, Home Services, Metaverse, Retail Organizational structures for ML teams Is ML right for your organization? When ML fails Some ideas and models
- 4. ML in Product Management Preyas Joshi Types of Models Regression Binary Supervised Unsupervised Association Recommender Clustering Detect anomaly Use your own intelligence Train the model Run the model • Analyze data • Find similarities across X vectors • Group data into clusters • Associate known attributes / behavior / events to link each other Predict a value Is this Yes/No, X/Y? Multi class Is this of type X, Y or Z? Extract text Text Analysis Incrementally rewarded intelligence Reinforced Model based Model free Markov (MDP) Q-Learning, SARSA (State-Action- Reward-State-Action
- 5. ML in Product Management Preyas Joshi 5 Types of Problems The global machine learning market is predicted to grow 14X from $8.43 billion (2019) to $119.8B (2027). $119.8B Lending Problem: As a lender, I want to predict the default rate of a business, based on a large number of variables (industry, tenure, owner’s credit score, zip code, revenue, and thousands of other variables, , so I can lend them money at the right rate, while protecting investors. Problem: As a student loan lender, I want to predict the likelihood of the loan being paid based on the student’s college, profession, grades, GPA, and zip code, so I can support more students while protecting investors. Solution: Phase 1 – Train a model to predict the outcome of a loan based on existing results. 100X Iterate and refine model. Regression, binary and multiclass classification, reinforced learning Solution: Phase 1x - A back-tested self-learning credit decisioning model based on prior data and outcomes. Industry User Story / Epic Possible solutions ML model
- 6. ML in Product Management Preyas Joshi 6 Types of Problems The global machine learning market is predicted to grow 14X from $8.43 billion (2019) to $119.8B (2027). $119.8B Retail Problem: Launching a new product in the market is risky. How can a manufacturer produce inventory just in time, based on real-time demand? Solution: Build an A/B test driven eCommerce tech integrated with manufacturing that predicts demand based on order velocity. Supervised: Binary regression, multiclass, clustering, reinforced learning Industry User Story / Epic Possible solutions ML model
- 7. ML in Product Management Preyas Joshi 7 Types of Problems The global machine learning market is predicted to grow 14X from $8.43 billion (2019) to $119.8B (2027). $119.8B Infrastructure Problem: Can I proactively recommend maintenance for a bridge or machine before the problem occurs? Solution: Train a model to assess pictures of existing infrastructure that’s in disrepair to predict the maintenance cycle based on current indicators. Supervised: Binary regression, multiclass. Unsupervised: Association Industry User Story / Epic Possible solutions ML model
- 8. ML in Product Management Preyas Joshi 8 Types of Problems The global machine learning market is predicted to grow 14X from $8.43 billion (2019) to $119.8B (2027). $119.8B Infrastructure Problem: As a Support Engineer, I want to reduce the time I spend searching for troubleshooting tools to solve a technical problem, so I can resolve issues quickly. Solution: A tool recommendation model based on previously used tools to solve similar cases Supervised: multiclass. Unsupervised: Clustering + Association Industry User Story / Epic Possible solutions ML model
- 9. ML in Product Management Preyas Joshi 9 Why ML? 48 percent of CIOs plan do deploy ML driven tech in the next 12 months. Look around. If you’re not currently using ML in your industry, your competitor is. 48% Should I invest in ML? Why now? • Depends on your product lifecycle, competitive space, and opportunities • Data health, readiness, access, data pipelines and infrastructure • Where should you allocate your ML investments? • How do you learn from failure? Culture of open-ended, rapid experimentation. • Do you learn from success - Ethical dilemmas and removal of bias • Organizational attitude to experimentation and high failure tolerance. Think like a VC!
- 10. ML in Product Management Preyas Joshi 10 When can ML projects fail? Root causes • Organization overinvested in ML • Low engagement with business and competitive environment • Low fidelity innovation • Risk aversion • Low experimentation • Works sometimes. But carefully review the underlying use case. Recommendations • Review by internal board of ML advisors • External audits • Incubator mechanism “To a man with a hammer, everything looks like a nail” – Mark Twain
- 11. ML in Product Management Preyas Joshi Organizational Structure ML TEAM PEOPLE, FINANCE PRODUCT, DESIGN TECH, IT BIZ DEV, CUSTOMER MARKETING, SALES SUPPLY CHAIN
- 12. ML in Product Management Preyas Joshi 12 Organizational Structure 48 percent of CIOs plan do deploy ML driven tech in the next 12 months. Look around. If you’re not currently using ML in your industry, your competitor is. 48% Which team does an ML org belong to? • Several options here. • ML team and the hierarchy does not need to have a fixed structure / location • Evolve as the business evolves • Centralize vs break-up into parts • Team / skill specialization • Business lifecycle • Consult (specific project based advise)
- 13. ML in Product Management Preyas Joshi Organizational Structures ML TEAM FINANCE DESIGN TECHNOLOGY BUSINESS DEVELOPMENT MARKETING & SALES SUPPLY CHAIN Service org M L FINANCE DESIGN TECHNOLOGY BUSINESS DEVELOPMENT MARKETING & SALES SUPPLY CHAIN ML M L Respective ML pods FINANCE DESIGN TECHNOLOGY BUSINESS DEVELOPMENT MARKETING & SALES SUPPLY CHAIN ML External consultant
- 14. ML in Product Management Preyas Joshi Organizational Structure ML TEAM FINANCE DESIGN TECHNOLOGY BUSINESS DEVELOPMENT MARKETING & SALES SUPPLY CHAIN Service org • Needs multiple SMEs within the ML org • High program/project management effort • Teams could get underserved because they didn’t identify their ML use case • Deeper ML expertise within ML team helps career progression and deeper learning • Risk of generic solutions due to repetitive reuse of the same solution • Teams need to understand when ML can be used for their use case
- 15. ML in Product Management Preyas Joshi Organizational Structure • Each team identifies their respective use case for ML • Each ML team can develop specialized solutions for their business • Could lead to silos • Could lead to higher attrition if team doesn’t have mentors and a growth path • Allows for the ML team to internally verticalize along Functional, Technical, and Product lines • Each ML team needs to deeply understand business M L FINANCE DESIGN TECHNOLOGY BUSINESS DEVELOPMENT MARKETING & SALES SUPPLY CHAIN ML M L Respective ML pods
- 16. ML in Product Management Preyas Joshi Organizational Structure • Temporary engagement with an existing ML PM leader • Low overhead, low risk. Test and Learn. Invest incrementally. • Hire temporary ML staffing. Double down when the right problems are identified • Each business group proposes / identifies a problem they want to solve FINANCE DESIGN TECHNOLOGY BUSINESS DEVELOPMENT MARKETING & SALES SUPPLY CHAIN ML External consultant
- 17. ML in Product Management Preyas Joshi 17 How do you identify an ML Idea? • Teams present their use case to the ML team • ML team assesses validity of the problem and whether (clean, structured) data exists to test and build outcomes • Risk of identifying several small problems, while big ideas could get lost • Need a mechanism to bubble up big ideas Office Hours Subject Matter Experts External Advisor • ML PM in ML team work with SMEs in various orgs • ML PM assesses validity of problems, and documents detailed user stories and use cases • Presented to the ML experts on the team. Proposals are promoted / rejected based on validity • Develops ML strength within the PM team • Consult an external ML Product Advisor • Review whether your problems, data, infrastructure, and problem space is conducive for ML driven problem solving
- 18. ML in Product Management Preyas Joshi Poll What Next? EVALUATE ML RELEVANCY FOR A PROBLEM DEEP DIVE INTO EACH ML MODEL WRITING REQUIREMENTS FOR ML ENGINEERING TEAMS EXAMPLES OF STARTUPS WHERE PMS ARE USING ML OTHER
- 20. Thank You Linkedin.com/in/preyasjoshi/ Send feedback and connect