Uploaded byPrasad Velamuri
Managing AI Products
This document discusses key considerations for managing AI products. It begins with an overview of intelligent systems and the OODA loop model of decision making. It then covers the different areas of AI including machine learning, deep learning, and supervised vs unsupervised learning. The rest of the document provides guidance on strategic areas for AI product management such as corporate and data strategy, analyzing use cases, building minimal viable products, and influencing other teams to deliver AI capabilities. It emphasizes the importance of data acquisition, network effects, and focusing on practical applications that create business value.
Managing AI Products
- 1. Managing AI Products PrasadVelamuri
- 2. Prasad Velamuri • AIProduct Fellow @ Insight Data Science • Product Manager (+ Co-founder) @Voicy.ai • Product Manager @Cisco • Technology Strategy @Samsung • Product Manager, SW Developer @Juniper
- 3. Agenda • Intelligent Systems •OODA • AI/ ML/ DL • AI Product Management • Strategy • Analyze, Decide, Build ––– • Influence across matrix • Other considerations • Driverless AI • Monetizing AI
- 4.
- 5. Decision Making Insight: Speeding throughthe loop is more important than quality of the decisions Sir John Boyd Distinguished fighter pilot, developed military theories in ‘60s
- 6.
- 7. Economic Value Createdby AI 99% of the EVC by AI today is through Supervised Learning Input Output Picture Is it you or not Loan application Will you repay (%) Ad User Will the user click? Speech Recognition Text Transcript Translation (English) French Image/ Lidar Position
- 8. AI Product Management Whyis it different? •Non-deterministic Product (F1 score) •Atypical Product Testing • Output changes with use •Data science is not engineering • AI models aren’t like databases • Significant time spent on data prep •Still very ‘Researchy’
- 9. Strategic AI ProductManager • Corporate Strategy • PM should drive the Emergent strategy • Few companies adopting Deliberate strategy • Data Strategy • Centralized and secure • Avoid inaccurate, incomplete, out-of-date • Kickstart “Data network effects”
- 10. Data Network Effects Datanetwork effects occur when your product, generally powered by machine learning, becomes smarter as it gets more data from your users. http://mattturck.com/the-power-of-data-network-effects/ Network Effect + Data Network Effect
- 11. Strategic AI ProductManager • Corporate Strategy • PM should drive the Emergent strategy • Few companies adopting Deliberate strategy • Data Strategy • Centralized and secure • Avoid inaccurate, incomplete, out-of-date • Kickstart “Data network effects” • Data acquisition strategies
- 12. Data acquisition strategies •Manual work at least till data network effect kicks in • Crawling the web (e.g., text summarization/ simplification) • Narrow the domain (e.g., vertical chatbots) • Crowdsourcing/ Outsourcing (e.g., Crowdflower, Amazon Mechanical Turk) • Gamification/ Incentivizing user-in-the loop • Data capture SDKs in third party apps • Build “Data trap” (create/sell something valuable to gather data- Tesla?) • Publicly available datasets https://www.kdnuggets.com/2016/06/10-data-acquisition-strategies-startups.html
- 13. Strategic AI ProductManager • Corporate Strategy • PM should drive the Emergent strategy • Few companies adopting Deliberate strategy • Data Strategy • Unified data warehouse and secure • Avoid inaccurate, incomplete, out-of-date • Kickstart “Data network effects” • Data acquisition strategies • Get over “Cold start” problem
- 14. AI Product Management •Analyze – What to build • Decide – How to build • Build – The building Process
- 15. Observe Product Trendsin the AI Market • Develop market insights and macro trends • McKinsey Global Institute (MGI): • Only 12% use cases progressed beyond experimentation stage • Adoption limited outside technology sector • Best-practice is to adopt agile test and learn approach • Free research from MGI, Gartner, CB Insights
- 16. Follow trends inApplied AI research • Your true competitive advantage • Not from expertise in algorithms • Ability to shorten time-to-market of products • Have good handle on latest algorithm advances • Andrej Karpathy’s arxiv-sanity summarizing latest research • Follow influencers
- 17. Cut through hype-focus on practical use cases • Have insights into practical use cases • Identify problem • Perception, Prediction, Personalization
- 18. Identify Problem • Perception •If a typical person can do a mental task with < 1 sec of thought, we can probably automate it using AI now or in the near future (Andrew Ng, HBR, Nov 2016) • Prediction • For any concrete, repeated event that we observe, we can reasonably try to predict the outcome of the next such event (Andrew Ng, NIPS 2016) • Personalization • Serving content desired by users in a personalized manner (Spotify/ Netflix)
- 19. Cut through hype-focus on practical use cases • Have insights into practical use cases • Identify problem • Perception, Prediction, Personalization • The PAC Framework to build use cases • Automate, Classify, Predict • Customers, Product, Operations
- 20. The PAC Framework* CustomersProduct Operations Predict • Which customer will buy • Which user will churn • Sales Forecast • Infrastructure Usage • Employee Attrition Classify • Who might upgrade • Micro segmentation • Customer Input • Bug Classification • Manufacturing Automate • Lead Generation • Call Follow-Up • Bug resolution workflows • Product Training • Operational Workflows • Supply chain * Rob May
- 21. Cut through hype-focus on practical use cases • Have insights into practical use cases • Identify problem • Perception, Prediction, Personalization • The PAC Framework to build use cases • Automate, Classify, Predict • Customers, Product, Operations • AI hierarchy of opportunities • Building on Maslow’s hierarchy of needs
- 22. AI hierarchy ofopportunities* Superpowers for humans Customer Service, Conversation Analytics Retail self-checkout, supply chain optimization, Pricing predictions Security, Predictive Analytics, Autonomous Vehicles, eDiscovery Agricultural monitoring, Disease prevention, Medical Imaging, Smart Home, Geospacial Analytics, Drug Discovery * Ankit Jain (Gradient Ventures) Transcendence Esteem and Education Operational Efficiency Safety Needs Physiological Needs
- 23. Cut through hype-focus on practical use cases • Have insights into practical use cases • Identify problem • Perception, Prediction, Personalization • The PAC Framework to build use cases • Automate, Classify, Predict • Customers, Product, Operations • AI hierarchy of opportunities • Focus on use cases that improve EBIT • RoI, data network effects, data set, drift, tools required
- 24. Customer and Dataobsession • Customer obsession • Going beyond product features & benefits • Understanding meaning for customer’s jobs, their purpose, motivations and the conscious choices they make • Data obsession • Being a champion of digitization while quantifying problems customers care • Build comprehensive datasets needed for quality AI models • Fetching data that reflects user’s jobs, behaviors & interaction patterns.
- 25. Build usable productswith simple AI model • Don’t be over obsessed with complexity of AI models • Accuracy improvements vs user experience improvements • AI MVP pyramid (adapted from Jussi Pasanen’s MVP pyramid)
- 26. • Be familiarwith tools and techniques • Influence AI Engineers, Data Scientists and Data Engineers • API ecosystem that help serve end users • Data ingestion tools (Kafka) • Data processing systems (Spark) • NoSQL DBMS (Cassandra) • Commercial alternatives on AWS & GCP (cost structures) • Avoid reinventing the wheel for commoditized AI techniques Breadth first approach (Data/ Pipeline/ Model)
- 27. • Some crucialapplications involve high liability • Law, medicine and safety • Output requires clear explanation for compliance purposes • Use the approaches to explaining predictions from deep learning • Eliminate Bias* • Articulate organizational values of fairness and equality • Communicate this to all employees (data scientists) • Benchmark training data • Validate algorithms periodically Consider Model Explainability * SAP Design Center
- 28. • Use validatedlearning loops for quick iterations • Conceive use cases and map to capabilities of ML, Deep Learning • Classification (Binary/ Multiclass) • Regression (prediction) • Clustering • Universal approximation of Deep Learning • Tie to a small set of metrics that matter • Challenges of end-to-end AI models optimizing multiple objectives • Agile deep learning Iteratively build use cases with mapped AI models
- 29. Agile Deep Learning Havea sub-loop of “Explore, Experiment and Prioritize”
- 30. • Gracefully handlelow performance scenarios Ensure product fails gracefully
- 31. • Technical languageof AI researchers and data scientists. • Artificial Intelligence, deep learning, machine learning — whatever you’re doing if you don’t understand it — learn it. Because otherwise you’re going to be a dinosaur within 3 years! (Mark Cuban) Understand the fundamentals Monica Rogati
- 32. AI Product Management •Analyze – What to build • Decide – How to build • Build – The building Process
- 33. Influencing across thematrix • Data scientists and AI Engineers • Influencing-Up • SCIPAB model • Key Assertions based on realized benefits from AI Products • Establish credibility • Build Trust
- 34. Other Considerations • DriverlessAI/ Auto ML • Automate laborious tasks- Feature Engineering, Model tuning • Ensembling, Automatic cross-validation, Detect time-series • AI Monetizing 101
- 35. AI Monetization models* •Subscription models • Freemium through monetizing data network effects • Outcome-based • Pay for the outcome (benefit) provided by the product/service • Asset-Sharing • Maximize utilization of product across multiple customers • Revenue-sharing • Sell product at cost, earn a percentage of client’s product sales • Data monetization • Product serves as a vehicle to collect and monetize quality data • Win-win-win models * Heiko Schmidt
- 36. Summary • The currentphase of AI is very promising • Several opportunities to • make elegant products that create tremendous value, • delight customers and significantly transform the business. • AI Product Manager is a catalyst in this transformation
- 37. Resources • This slidedeck available at bit.ly/managingAIproducts • Blog covering salient points in this deck: • blog.insightdatascience.com/moving-towards-managing-ai-products-5268c5e9ecf2 • Follow me: • Twitter @prasadvsd • Linkedin.com/in/pvelamuri
Editor's Notes
- #5 https://www.slideshare.net/eilken1/cactuscon-2017-ooda-loop-in-life-cyber-threat-intelligence
- #6 By Bob?
- #7 https://medium.com/iotforall/the-difference-between-artificial-intelligence-machine-learning-and-deep-learning-3aa67bff5991
- #11 http://mattturck.com/the-power-of-data-network-effects/
- #13 https://www.kdnuggets.com/2016/06/10-data-acquisition-strategies-startups.html
- #28 https://medium.com/sap-design/human-impact-of-biased-ai-in-business-and-how-to-go-beyond-332d51e32e3a