Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Managing AI Products

215 views

Published on

Product Camp Silicon Valley (March 31st, 2018)
Prasad Velamuri

Published in: Technology
  • Be the first to comment

Managing AI Products

  1. 1. Managing AI Products Prasad Velamuri
  2. 2. Prasad Velamuri • AI Product Fellow @ Insight Data Science • Product Manager (+ Co-founder) @Voicy.ai • Product Manager @Cisco • Technology Strategy @Samsung • Product Manager, SW Developer @Juniper
  3. 3. Agenda • Intelligent Systems • OODA • AI/ ML/ DL • AI Product Management • Strategy • Analyze, Decide, Build ––– • Influence across matrix • Other considerations • Driverless AI • Monetizing AI
  4. 4. Observe Orient Decide Act Intelligent Systems LEARN
  5. 5. Decision Making Insight: Speeding through the loop is more important than quality of the decisions Sir John Boyd Distinguished fighter pilot, developed military theories in ‘60s
  6. 6. The field of AI
  7. 7. Economic Value Created by AI 99% of the EVC by AI today is through Supervised Learning Input Output Picture Is it you or not Loan application Will you repay (%) Ad User Will the user click? Speech Recognition Text Transcript Translation (English) French Image/ Lidar Position
  8. 8. AI Product Management Why is it different? •Non-deterministic Product (F1 score) •Atypical Product Testing • Output changes with use •Data science is not engineering • AI models aren’t like databases • Significant time spent on data prep •Still very ‘Researchy’
  9. 9. Strategic AI Product Manager • Corporate Strategy • PM should drive the Emergent strategy • Few companies adopting Deliberate strategy • Data Strategy • Centralized and secure • Avoid inaccurate, incomplete, out-of-date • Kickstart “Data network effects”
  10. 10. Data Network Effects Data network effects occur when your product, generally powered by machine learning, becomes smarter as it gets more data from your users. http://mattturck.com/the-power-of-data-network-effects/ Network Effect + Data Network Effect
  11. 11. Strategic AI Product Manager • Corporate Strategy • PM should drive the Emergent strategy • Few companies adopting Deliberate strategy • Data Strategy • Centralized and secure • Avoid inaccurate, incomplete, out-of-date • Kickstart “Data network effects” • Data acquisition strategies
  12. 12. Data acquisition strategies • Manual work at least till data network effect kicks in • Crawling the web (e.g., text summarization/ simplification) • Narrow the domain (e.g., vertical chatbots) • Crowdsourcing/ Outsourcing (e.g., Crowdflower, Amazon Mechanical Turk) • Gamification/ Incentivizing user-in-the loop • Data capture SDKs in third party apps • Build “Data trap” (create/sell something valuable to gather data- Tesla?) • Publicly available datasets https://www.kdnuggets.com/2016/06/10-data-acquisition-strategies-startups.html
  13. 13. Strategic AI Product Manager • Corporate Strategy • PM should drive the Emergent strategy • Few companies adopting Deliberate strategy • Data Strategy • Unified data warehouse and secure • Avoid inaccurate, incomplete, out-of-date • Kickstart “Data network effects” • Data acquisition strategies • Get over “Cold start” problem
  14. 14. AI Product Management • Analyze – What to build • Decide – How to build • Build – The building Process
  15. 15. Observe Product Trends in the AI Market • Develop market insights and macro trends • McKinsey Global Institute (MGI): • Only 12% use cases progressed beyond experimentation stage • Adoption limited outside technology sector • Best-practice is to adopt agile test and learn approach • Free research from MGI, Gartner, CB Insights
  16. 16. Follow trends in Applied AI research • Your true competitive advantage • Not from expertise in algorithms • Ability to shorten time-to-market of products • Have good handle on latest algorithm advances • Andrej Karpathy’s arxiv-sanity summarizing latest research • Follow influencers
  17. 17. Cut through hype- focus on practical use cases • Have insights into practical use cases • Identify problem • Perception, Prediction, Personalization
  18. 18. Identify Problem • Perception • If a typical person can do a mental task with < 1 sec of thought, we can probably automate it using AI now or in the near future (Andrew Ng, HBR, Nov 2016) • Prediction • For any concrete, repeated event that we observe, we can reasonably try to predict the outcome of the next such event (Andrew Ng, NIPS 2016) • Personalization • Serving content desired by users in a personalized manner (Spotify/ Netflix)
  19. 19. Cut through hype- focus on practical use cases • Have insights into practical use cases • Identify problem • Perception, Prediction, Personalization • The PAC Framework to build use cases • Automate, Classify, Predict • Customers, Product, Operations
  20. 20. The PAC Framework* Customers Product Operations Predict • Which customer will buy • Which user will churn • Sales Forecast • Infrastructure Usage • Employee Attrition Classify • Who might upgrade • Micro segmentation • Customer Input • Bug Classification • Manufacturing Automate • Lead Generation • Call Follow-Up • Bug resolution workflows • Product Training • Operational Workflows • Supply chain * Rob May
  21. 21. Cut through hype- focus on practical use cases • Have insights into practical use cases • Identify problem • Perception, Prediction, Personalization • The PAC Framework to build use cases • Automate, Classify, Predict • Customers, Product, Operations • AI hierarchy of opportunities • Building on Maslow’s hierarchy of needs
  22. 22. AI hierarchy of opportunities* Superpowers for humans Customer Service, Conversation Analytics Retail self-checkout, supply chain optimization, Pricing predictions Security, Predictive Analytics, Autonomous Vehicles, eDiscovery Agricultural monitoring, Disease prevention, Medical Imaging, Smart Home, Geospacial Analytics, Drug Discovery * Ankit Jain (Gradient Ventures) Transcendence Esteem and Education Operational Efficiency Safety Needs Physiological Needs
  23. 23. Cut through hype- focus on practical use cases • Have insights into practical use cases • Identify problem • Perception, Prediction, Personalization • The PAC Framework to build use cases • Automate, Classify, Predict • Customers, Product, Operations • AI hierarchy of opportunities • Focus on use cases that improve EBIT • RoI, data network effects, data set, drift, tools required
  24. 24. Customer and Data obsession • Customer obsession • Going beyond product features & benefits • Understanding meaning for customer’s jobs, their purpose, motivations and the conscious choices they make • Data obsession • Being a champion of digitization while quantifying problems customers care • Build comprehensive datasets needed for quality AI models • Fetching data that reflects user’s jobs, behaviors & interaction patterns.
  25. 25. Build usable products with simple AI model • Don’t be over obsessed with complexity of AI models • Accuracy improvements vs user experience improvements • AI MVP pyramid (adapted from Jussi Pasanen’s MVP pyramid)
  26. 26. • Be familiar with tools and techniques • Influence AI Engineers, Data Scientists and Data Engineers • API ecosystem that help serve end users • Data ingestion tools (Kafka) • Data processing systems (Spark) • NoSQL DBMS (Cassandra) • Commercial alternatives on AWS & GCP (cost structures) • Avoid reinventing the wheel for commoditized AI techniques Breadth first approach (Data/ Pipeline/ Model)
  27. 27. • Some crucial applications involve high liability • Law, medicine and safety • Output requires clear explanation for compliance purposes • Use the approaches to explaining predictions from deep learning • Eliminate Bias* • Articulate organizational values of fairness and equality • Communicate this to all employees (data scientists) • Benchmark training data • Validate algorithms periodically Consider Model Explainability * SAP Design Center
  28. 28. • Use validated learning loops for quick iterations • Conceive use cases and map to capabilities of ML, Deep Learning • Classification (Binary/ Multiclass) • Regression (prediction) • Clustering • Universal approximation of Deep Learning • Tie to a small set of metrics that matter • Challenges of end-to-end AI models optimizing multiple objectives • Agile deep learning Iteratively build use cases with mapped AI models
  29. 29. Agile Deep Learning Have a sub-loop of “Explore, Experiment and Prioritize”
  30. 30. • Gracefully handle low performance scenarios Ensure product fails gracefully
  31. 31. • Technical language of AI researchers and data scientists. • Artificial Intelligence, deep learning, machine learning — whatever you’re doing if you don’t understand it — learn it. Because otherwise you’re going to be a dinosaur within 3 years! (Mark Cuban) Understand the fundamentals Monica Rogati
  32. 32. AI Product Management • Analyze – What to build • Decide – How to build • Build – The building Process
  33. 33. Influencing across the matrix • Data scientists and AI Engineers • Influencing-Up • SCIPAB model • Key Assertions based on realized benefits from AI Products • Establish credibility • Build Trust
  34. 34. Other Considerations • Driverless AI/ Auto ML • Automate laborious tasks- Feature Engineering, Model tuning • Ensembling, Automatic cross-validation, Detect time-series • AI Monetizing 101
  35. 35. AI Monetization models* • Subscription models • Freemium through monetizing data network effects • Outcome-based • Pay for the outcome (benefit) provided by the product/service • Asset-Sharing • Maximize utilization of product across multiple customers • Revenue-sharing • Sell product at cost, earn a percentage of client’s product sales • Data monetization • Product serves as a vehicle to collect and monetize quality data • Win-win-win models * Heiko Schmidt
  36. 36. Summary • The current phase of AI is very promising • Several opportunities to • make elegant products that create tremendous value, • delight customers and significantly transform the business. • AI Product Manager is a catalyst in this transformation
  37. 37. Resources • This slide deck available at bit.ly/managingAIproducts • Blog covering salient points in this deck: • blog.insightdatascience.com/moving-towards-managing-ai-products-5268c5e9ecf2 • Follow me: • Twitter @prasadvsd • Linkedin.com/in/pvelamuri

×