Download as PDF, PPTX
Uploaded byScott W. Ambler
AI For PMs - Leading a Machine Learning (ML) Team
The document discusses the essentials of managing machine learning (ML) initiatives within organizations, emphasizing that these initiatives should not be treated as simple projects but require a disciplined hybrid strategy for success. It covers the AI lifecycle, the importance of high-quality data, ethical considerations, and the necessity of forming dedicated, interdisciplinary teams to overcome challenges. Key takeaways stress the need for realistic expectations, ethical understanding, and a solid preparation phase before initiating AI projects.
More Related Content
Similar to AI For PMs - Leading a Machine Learning (ML) Team
![[DSC DACH 24] Harnessing AI for Sustainable Results: Strategies for Long-Term...](https://cdn.slidesharecdn.com/ss_thumbnails/johannesschauer-240921154533-335db038-thumbnail.jpg?width=640&height=640&fit=bounds)
AI For PMs - Leading a Machine Learning (ML) Team
- 1. Artificial Intelligence for Project Managers: Leadinga Machine Learning Team Scott W. Ambler Agile Data Strategist | Consulting Methodologist Ambysoft Inc. © Ambysoft Inc. 1
- 2. Scott Ambler © AmbysoftInc. 2 • scott@scottambler.com • linkedin.com/in/sambler/ Consulting Methodologist Ambysoft.com Thought Leader AgileData.org Thought Leader AgileModeling.com Co-Creator pmi.org/disciplined-Agile Download these slides at Slideshare.net/ScottWAmbler/
- 3. Machine learning (ML)initiatives aren’t projects It only looks like AI is magic Data is a critical enabler for AI The results of AI can occur at Internet speed at scale 3 Today’s Takeaways © Ambysoft Inc.
- 4. Agenda 4 1.What is(n’t) AI? 2.Thelifecycle of an AI/ML initiative 3.Successfully initiating an AI/ML initiative 4.Ethical considerations with AI 5.Business implications of AI 6.Overcoming the data quality challenge 7.Success and failure factors for AI initiatives © Ambysoft Inc.
- 5. WhatIs(n’t)AI? Artificial intelligence issoftware, not magic © Ambysoft Inc. 5
- 6. Defining Artificial Intelligence(AI) © Ambysoft Inc. 6 Artificial Intelligence (AI) Machine Learning (ML) Deep Learning (DL) Large Language Models (LLMs)/Generative AI Computer systems with ”brain-like” logically structured algorithms called artificial neural networks (ANNs). Computer systems with the ability to learn without being explicitly programmed. Training of ML models often involve significant data curation, sometimes including data labeling. Computer systems able to perform tasks normally requiring human intelligence. Computer systems based on very large deep learning models that are pre-trained on vast amounts of data.
- 7. Artificial Intelligence Isn’tJust GenAI © Ambysoft Inc. 7 Decision trees Random forests Artificial neural networks (ANNs) Support vector machines (SVMs) Ensembles Self-organizing maps Convolutional neural networks (CNNs) Recurrent neural networks (RNNs) And many more…
- 8. Categories of ArtificialIntelligence © Ambysoft Inc. 8 Artificial Narrow Intelligence (ANI) • Specializes in one area and solves one problem. Artificial General Intelligence (AGI) • As smart, or smarter, than a human with a wide range of abilities. Artificial Super Intelligence (ASI) • An intellect that is much smarter than the best human in practically every field. We are here ➔ We are afraid of this ➔ And really afraid of this ➔
- 9. TheMachineLearning Lifecycle You need adisciplined hybrid strategy © Ambysoft Inc. 9
- 10. Four Uncomfortable Observationsand a Claim © Ambysoft Inc. 10 This isn’t a project. Treating this as a series of projects MIGHT work, but it’s most likely just going to inject needless cost, risk, and time into your initiative. A “predictive” approach to an ML initiative has exactly zero chance of success. A “pure agile” approach to an ML initiative has very little chance of success. A “pure lean” approach has a better chance of success, but still isn’t ideal. Claim: A disciplined hybrid strategy will provide your highest chance of success.
- 11. A Very High-LevelView of the ML Lifecycle © Ambysoft Inc. 11 Train the AI (Construction) Operate the AI/ Make Inferences (Production) Learnings Source: Ambysoft.com/essays/machine-learning-lifecycle.html
- 12. Adding Details toConstruction © Ambysoft Inc. 12 Construction Make Inferences Production Learnings Prepare the Data Train the Model Validate the Model Choose a Training Strategy Deploy the AI
- 13. Construction is Iterativeand Incremental © Ambysoft Inc. 13 Construction Make Inferences Production Learnings Deploy the AI Validate the Model Choose a Training Strategy Prepare Data Evolve Reqs. Train Model
- 14. A Machine LearningLifecycle © Ambysoft Inc. 14 Construction Make Inferences Production Learnings Deploy the AI Validate the Model Envision Explore Usage Explore Data Explore Viability Choose a Training Strategy Prepare Data Evolve Reqs. Train Model
- 15. Machine Learning RequiresProduct Teams, Not Project Teams Truly effective ML teams organize as a product team following a continuous delivery strategy. 15 Release 1 R2 R3 Continuous Delivery R 4 R 5 R 6 • The first release is often organized as a project, and typically takes many months. • The next few releases are shorter, often quarterly. • Then you realize that releasing more often, typically monthly, makes sense. It becomes clear that this is no longer a project. • A bi-weekly or even weekly cadence becomes viable given process improvement. • Continuous delivery should be your end goal. Note: The nature of your business may accelerate this timeline
- 16. © Ambysoft Inc. 16 MLLifecycle – MLOps Rendering Explore data, usage, and viability Choose training strategy, Prepare data, Evolve requirements, Train model Make inferences New data & learnings
- 17. ML Lifecycle – Iterative “CRISP-DM” Rendering ©Ambysoft Inc. 17 Prepare Data Train the Model Validate Envision Deploy Operate
- 18. Common Lifecycle Risks ©Ambysoft Inc. 18 Construction Make Inferences Learnings Deploy the AI Envision • Misaligned stakeholders • Inadequate data • Biased data • Difficult to create diverse, cross- disciplined team • Non-viability • Insufficient exploration • “Detached” exploration • Organization isn’t ready • Data technical debt proves worse than originally believed • Creation of a biased model • Creation of an inadequate model • “Never-ending” edge cases ➔ Unpredictable timeline • Non-viability • Biased usage • Non-viability • Unprepared stakeholders • Model drift • Function creep
- 19. AI isn’t magic. Constructionand operation of an AI requires hard work by disciplined teams of knowledgeable people. © Ambysoft Inc. 19
- 20. InitiatinganMLTeam A Hybrid, DisciplinedStrategy © Ambysoft Inc. 20
- 21. Successful Initiation: TheEnvisioning Phase © Ambysoft Inc. 21 Lifecycle Risks • Misaligned stakeholders • Inadequate data • Biased data • Difficult to create diverse, cross- disciplined team • Non-viability • Insufficient exploration • “Detached” exploration • Organization isn’t ready Initial agile modeling focusing on business process/usage Identification of data sources – Gather high-level details Behavioral Risks • AI/ML is seen as a data exercise • Traditional data professionals over model • Leadership is often reticent to invest in staff • Technical people are often optimistic • Leadership is often overly optimistic • Leadership is eager to get going Initial agile modeling of candidate source data Defer until you have the right people who are committed Understand your context- 80% of AI initiatives fail Define your organizational ethics Hire qualified staff Invest in training and coaching Have an explicit viability assessment Decreases risks Increases risks to
- 22. © Ambysoft Inc.22 Does your leadership understand AI? Does your organization have a clear AI strategy? Do you have AI expertise available, including data scientists? Do you have a realistic vision for what you want to achieve with AI? Do you understand your current level of data technical debt? Can you staff teams with dedicated, diverse, and interdisciplinary people? Initial Viability: Is Your Organization Ready for AI?
- 23. BusinessImplications ofAI AI will forceyou to rethink everything © Ambysoft Inc. 23
- 24. © Ambysoft Inc.24 Potential Risks of AI • Rapid, dramatic, and public failure • Reputational risk • Unwarranted trust in AI-generated results • Unrealistic expectations of AI capability • Biased decision making Potential Benefits of AI • Augment the capabilities of your staff • Provide new/improved offerings to your customers • Enhance existing business processes • Replace existing business processes • Increase the reach and scale of your customer offerings • Increased operational efficiency • Increased consistency in decision making • Improved governance Initial Viability: Understand Your Context
- 25. EthicalConsiderations withAI AI magnifies andaccelerates your existing biases © Ambysoft Inc. 25
- 26. Defined, Organizational Ethics •Ethics refers to the moral principles that govern behavior or the conducting of an activity. • Great sources of AI ethics thinking: • Australia’s AI Ethics Principles • Canada’s Artificial Intelligence and Data Act (AIDA) • EU Guidelines on Ethics in Artificial Intelligence • Google’s AI Principles • IBM What is AI Ethics? • Unesco Ethics of Artificial Intelligence • US: The White House’s Blueprint for an AI Bill of Rights • BUT, every organization is different, therefore should determine their own ethics guidelines. © Ambysoft Inc. 26
- 27. Critical Ethical ConsiderationsWith AI 1. Safety • Have you considered the potential of harming people? • Have you considered physical harm, mental harm, financial harm, … ? 2. Human Dignity • Do you understand the overall process that the AI is part of? • Does your model take societal values into account? • Does you model take multi-cultural considerations into account? 3. Fairness • Have you actively considered bias in the development of your model? • Is your model appropriately gender/race/culture/age neutral? • How do you know? 4. Accountability • If the AI makes a decision, who will be held responsible for it? • Can you explain how a decision was made? • Do you know the provenance of the data used to train your model? 5. Trustworthiness • Can your model’s inferences/predictions be challenged? • Are they explainable? • Are they fair? © Ambysoft Inc. 27
- 28. OvercomingtheData Quality(DQ)Challenge It is timeto pay the data piper © Ambysoft Inc. 28
- 29. Potential DQ IssuesFaced by ML Teams © Ambysoft Inc. 29 • Biased data • Insufficient data • Semantic differences across sources • Missing data values • Inconsistent data values • Unclear/unknown sources of record • Ownership limitations • Privacy/security • Data poisoning • Data drift • and many more…
- 30. Data Quality Techniquesfor ML Teams © Ambysoft Inc. 30 Data cleansing Data stewards Data architecture Automated regression testing Data labeling Data masking AI data cleansing (e.g. for MDM) Database modeling Transformation (as in ETL) Reviews (design, architecture) and more…. Data guidance (standards, guidelines, …) Database refactoring Defect logging Data repair Synthetic training data Manual regression testing
- 31. Remember the lawof GIGO: Garbage In, Garbage Out High-quality data is critical for successful AI/ML © Ambysoft Inc. 31
- 32. SuccessandFailure Factors Are you ready? ©Ambysoft Inc. 32
- 33. Are You Readyfor an AI Initiative? © Ambysoft Inc. 33 Do you have realistic expectations? Do you understand the ethical issues surrounding AI? Are your leaders capable? Is your data ready? Can you build diverse, cross-disciplinary teams of dedicated staff? Have you defined your way of working (WoW) for the initiative?
- 34. Machine learning (ML)initiatives aren’t projects It only looks like AI is magic Data is a critical enabler for AI The results of AI can occur at Internet speed at scale 34 Today’s Takeaways © Ambysoft Inc.
- 35. Thank You! © AmbysoftInc. 35 • scott@scottambler.com • linkedin.com/in/sambler/ Consulting Methodologist Ambysoft.com Thought Leader AgileData.org Thought Leader AgileModeling.com Co-Creator pmi.org/disciplined-Agile Download these slides at Slideshare.net/ScottWAmbler/
- 36. Additional Slides © AmbysoftInc. 36
- 37. Would you likethis presentation for your chapter, user group, or organization? Reach out to me at ScottAmbler.com © Ambysoft Inc. 37
- 38. The Agile DataMission To share proven agile and lean strategies for data initiatives. Learn more at AgileData.org © Ambysoft Inc. 38
- 39. The Agile ModelingMission To share proven and effective strategies for modeling/mapping and documentation. Learn more at AgileModeling.com © Ambysoft Inc. 39
- 40. (More) AITerminology It’s confusing atfirst, but you’ll get it © Ambysoft Inc. 40
- 41. Additional AI TermsThat I May Use Today © Ambysoft Inc. 41 • Data labeling. The process of identifying raw data, such as images or text, and adding one or more meaningful labels to it to provide context so that a machine learning (ML) model can learn from it. • Foundational model. An AI model trained on broad data at scale that can then be adapted to a wide range of downstream tasks. Can provide a basis from which to start your own model development. Examples: BloombergGPT and BioGPT. • Semi-supervised learning. A model training approach that uses both labeled and unlabeled data, a hybrid technique between supervised and unsupervised learning. Typically only a small percentage of the data is labeled, as labeling tends to be expensive. Also called weak supervised learning. • Supervised learning. A model training approach where the model is provided examples of inputs, it predicts outputs, and the model is corrected accordingly. • Unsupervised learning. A model is trained without known outputs. The model is given data and the goal is to find interesting patterns in it. Since we don’t know what those patterns will be, there is no obvious way to provide corrections.