Uploaded byAI Black Belt
AI Orange Belt - Session 3
The document discusses several important considerations for companies looking to implement artificial intelligence, including developing an AI transformation playbook, assessing an organization's AI maturity, anticipating costs and timing, deciding whether to build or buy AI solutions, and addressing important legal and ethical issues around explainability, privacy, fairness, and safety. The document provides guidance on how companies can effectively lead their organization into the AI era by establishing the right strategies, processes, and safeguards.
More Related Content
Similar to AI Orange Belt - Session 3
AI Orange Belt - Session 3
- 1. Build AI inyour company management, culture, legal, ethics, governance Orange Belt - Session 3 1
- 2. What we haveseen so far 1. What is AI 2. What can AI do and what it can’t do 3. How to select a project 4. What are the steps necessary for a first successful ML project 2
- 3. AI is nottraditional software A totally different lifecycle 3
- 4.
- 5.
- 6. We saw thislast week 01 03 02 06 04 05 Monitoring & Updates Have the right talents & solutions Maintenance Select the right question Choose the performance metric Decide the level of explainability Identify Use the right architecture Have the talents in place Deploy Find the right data Structure annotate data Clean Data Data Decide on an acceptable error Test on the right scope Evaluate Select the right algorithm Tune the model Model 6
- 7. In practice, everythingis more iterative 7
- 8. Plan for today 0(Base on the case seen in the assignment) What does it feel like to work on a complex ML project ? 1. AI Transformation Playbook 2. AI Maturity of your company 3. Perspectives on costs & roadmap 4. Build vs Buy 5. Legal & ethics considerations 6. Human and AI interactions 8
- 9. 1. AI TransformationPlaybook How to lead your company into the AI era 9 Source : https://landing.ai/ai-transformation-playbook/
- 10. AI Transformation Playbook 1.Execute pilot projects to gain momentum 2. Build an in-house AI team 3. Provide broad AI training 4. Develop an AI strategy 5. Develop internal and external communications 10
- 11. 1. Execute pilotproject to gain momentum • More important for the initial project to succeed rather than be the most valuable • Show traction within 6-12 months • Can be in-house or outsourced 11
- 12. 2. Build anin-house AI team Centralized AI Platform 12
- 13. 3. Provide broadAI training Role What they should learn Nb hours of training Executives and senior business leaders - What AI can do for the company ? - AI Strategy - Resource allocation >= 4 hours Leaders of divisions working on AI projects - Set project direction (both technical and business diligence) - Resource allocation - Monitor progress >= 12 hours AI engineer trainees - Build and ship AI software - Gather data - Execute on specific AI projects >= 100 hours 13
- 14. How to document yourselves AI forEveryone, by Andrew Ng Deep Learning Specialization 3blue1brown Siraj OpenAI ImportAI https://towardsdatascience.c om/ 14
- 15. 4. Develop anAI strategy • Build several difficult AI assets that are broadly aligned with a coherent strategy • Leverage AI to create an advantage specific to your industry sector. • Design strategy aligned with the “Virtuous Cycle of AI” AI plays a role here 15
- 16. 4. Develop anAI strategy • Consider creating a data strategy • Strategic data acquisition • Unified data warehouse • Create network effects and platform advantages • In industries with “winner take all” dynamics, AI can be an accelerator • What about more traditional strategy framework ? • AI can allow a low cost strategy • AI can allow a high value product strategy 16
- 17. 5. Develop internaland external communications • Investor relations • Government relations • Consumer / user education • Talen / recruitment • Internal communications 17
- 18. AI pitfalls toavoid Don’t : Do : - Expect AI to solve everything - Be realistic about what AI can and cannot do, given limitations of technology, data and engineering resources - Hire 2-3 ML engineers and count solely on them to come up with use cases - Pair engineers with business talent and work across cross-functional team to find valuable projects - Expect the AI project to work the first time - Plan for AI development to be an iterative process, with multipe attemps needed to succeed. - Expect traditional planning processes to apply without changes - Work with AI team to establish timeline estimates, milestones, KPIs, etc. - Think you need superstar AI engineers before you can do anything - Keep building the team but get going with the team you already have 18
- 19. Some initial stepsyou can take • Start learning (with this course) • Start brainstorming projets • Hire a few ML/DS people to help • Hire or appoint an AI leader (VP AI, CAIO, …) • Discuss with CEO/Board possibilities of AI Transformation • Will your company be much more valuable and/or more effective if it were good at AI ? 19
- 20. Exercise What is thefirst step in the AI Transformation Playbook for helping your company become good at AI? 20
- 21. Exercise Of the followingoptions, which is the most important trait of your first pilot project? A) Succeed and show traction within 6-10 months B) Drive extremely high value for the business C) Be executed by an in-house team D) None of the above 21
- 22. Exercise Say you arebuilding the DropBox OCR system, and want to accumulate data for your product through having many users. Which of these represents the “Virtuous circle of AI” for this product? 22
- 23. Exercise Why is developingan AI strategy NOT the first step in the AI Transformation Playbook? 23
- 24. 2. AI Maturity Assessthe readiness of your organisation 24
- 25.
- 26. A startup mindsetis always nice 26
- 27. 3-steps roadmap tomaturity Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY DATA PEOPLE LEGALÐICS PRODUCT 27
- 28. Part 1 EXPLORING Part 2 EXPERIMENTING Part3 INTEGRATING STRATEGY No use case No objectives / metrics No budget DATA No infrastructure Data Silos Descriptive analytics PEOPLE No data scientists No education LEGALÐICS No legal compliance No principles or processes PRODUCT Business cases but no development 28
- 29. Part 1 EXPLORING Part 2 EXPERIMENTING Part3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics PEOPLE No data scientists No education LEGALÐICS No legal compliance No principles or processes PRODUCT Business cases but no development 29
- 30.
- 31. Part 1 EXPLORING Part 2 EXPERIMENTING Part3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics descriptive → prescriptive No data consolidation ETL Data Lake Compounding value PEOPLE No data scientists No education LEGALÐICS No legal compliance No principles or processes PRODUCT Business cases but no development 31
- 32.
- 33. Data requirements according tobusiness needs 33
- 34. Define data acquisitionstrategy It’s not a one shot 34
- 35. Data annotation pipeline Togo at scale, you need guidelines, internal and external annotators. Even pre-annotation with machine learning that can be validated DOG CAT 35
- 36. Standardise quality check Findthe relevant metrics Exemple Sound: signal to noise ratio / cross-talk / silence detection 36
- 37. Govern data Different accessrestrictions, stay compliant 37
- 38.
- 39. Part 1 EXPLORING Part 2 EXPERIMENTING Part3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics descriptive → prescriptive No data consolidation ETL Data Lake Compounding AOV PEOPLE No data scientists No education A few data scientists Global acculturation / education Chief Data Officer SWAT Team Use case specialists LEGALÐICS No legal compliance No principles or processes PRODUCT Business cases but no development 39
- 40. Example roles • SoftwareEngineer • Build user interface, web & mobile applications, back end operations, … • Machine Learning Engineer • Input (A) to Output (B) • Machine Learning Researcher • Extend state-of-the-art in ML 40
- 41. Example roles • DataScientist • Examine data and provide insights • Make presentation and communicate to team / executives • Data Engineer • Organize Data • Make sure data is saved in an easily accessible, documented and cost effective way • More and more required as the amount of data managed by companies increases • AI Product Manager • Help decide what to build; what’s feasible and valuable 41
- 42. Getting started witha small team • 1 Software Engineer, or • 1 Machine Learning Engineer / Data Scientist, or • Nobody by yourself 42
- 43.
- 44. Structure a team- decentralised 44
- 45. Structure a team centralised(SWAT) 45
- 46. Structure a team adduse case specialists 46
- 47. Exercise Suppose you arebuilding a trigger word detection system, and want to hire someone to build a system to map from Inputs A (audio clip) to Outputs B (whether the trigger word was said), using existing AI technology. Out of the list below, which of the following hires would be most suitable for writing this software? 47
- 48. Part 1 EXPLORING Part 2 EXPERIMENTING Part3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics descriptive → prescriptive No data consolidation ETL Data Lake Compounding AOV PEOPLE No data scientists No education A few data scientists Global acculturation / education Chief Data Officer SWAT Team Use case specialists LEGALÐICS No legal compliance No principles or processes GDPR Compliant Core explanations & policies Ethical principles Corporate practice Part of incentive programs Core product advantage PRODUCT Business cases but no development 48
- 49. Part 1 EXPLORING Part 2 EXPERIMENTING Part3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics descriptive → prescriptive No data consolidation ETL Data Lake Compounding AOV PEOPLE No data scientists No education A few data scientists Global acculturation / education Chief Data Officer SWAT Team Use case specialists LEGALÐICS No legal compliance No principles or processes GDPR Compliant Core explanations & policies Ethical principles Corporate practice Part of incentive programs Core product advantage PRODUCT Business cases but no development Use of APIs Discrete proof of concepts Pilots AI is core product & core competency 49
- 50. Exercise : createyour own roadmap Part 1 EXPLORING Part 2 EXPERIMENTING Part 3 INTEGRATING STRATEGY No use case No objectives / metrics No budget Use it for optim & prediction Know a few use cases Core product Use case bank Competitive advantage DATA No infrastructure Data Silos Descriptive analytics descriptive → prescriptive No data consolidation ETL Data Lake Compounding AOV PEOPLE No data scientists No education A few data scientists Global acculturation / education Chief Data Officer SWAT Team Use case specialists LEGALÐICS No legal compliance No principles or processes GDPR Compliant Core explanations & policies Ethical principles Corporate practice Part of incentive programs Core product advantage PRODUCT Business cases but no development Use of APIs Discrete proof of concepts Pilots AI is core product & core competency 50
- 51. Exercise Try to identifythe critical 3 next steps for your organization, using the matrix 51
- 52. 3. AI Management Anticipatecosts and timing 52
- 53. prices & timingare not at all set in stone.. 53
- 54. but we canstill assess feasibility 54
- 55. but we canstill assess feasibility 55
- 56. but we canstill assess feasibility 56
- 57. but we canstill assess feasibility 57
- 58. Machine Learning isnot regular Software 58
- 59. Bring back yourexpected performance requirements 59
- 60.
- 61.
- 62. To sum up 1.Anticipate the cost drivers (data, accuracy, problem difficulty) 2. There is a big upfront cost to anticipate as opposed to regular projects 3. The cost scales nonlinearily with the accuracy requirements 4. The time it takes to get that accuracy is nonlinear as well 62
- 63. Exercise What is thebiggest cost of your current idea? What do you think will be the bottleneck? What type of strategy could you devise to anticipate that? 63
- 64. 4. Build vsBuy The eternal dilemma 64
- 65.
- 66. With a consultantyou don’t know, always look to start with a small proof of concept deliverable to prove to yourself that this consultant knows their stuff. Work with the consultant to come up with a project that is a low hanging fruit. Something that they can deliver on quickly without much development effort (e.g. based on existing code they already have, and data you have already collected). If this first step goes well, then you can confidently move to a bigger project scope. 66
- 67.
- 68. Access to resources (cloud,ML libraries, production systems) Access to talent Proven success stories to get you up and running in no time So why would you do everything yourselves ? Don’t try to run faster than the train Collaborate with Startups! 68
- 69. 5. Legal &Ethics What should you be careful about 69
- 70.
- 71. What does aConvolution sees ? VGG16, convolutional layer 1-1, a few of the 64 filters 71
- 72. What does aConvolution sees ? Variation of kernel size Source : 72
- 73. Adversarial Examples The leftimage is predicted with 99.9% confidence as a magpie. 73
- 74. Adversarial Examples Machine Learningclassifiers today are easily fooled ! 74
- 75. Adversarial Examples Machine Learningclassifiers today are easily fooled ! 75
- 76.
- 77. The problem withexplainability https://distill.pub/2019/activation-atlas/ 77
- 78.
- 79. • Maximise customersatisfaction, • Think about human decision making expectations • Complex or direct simple explanations? • Stay pragmatic - “reasonably necessary” Reasonable explanation 79
- 80. Alternative accountability Flagging -Stress test - Auditing 81
- 81.
- 82. 1. AI systemsmust be transparent. 2. An AI must have a “deeply rooted” right to the information it is collecting. 3. Consumers must be able to opt out of the system. 4. The data collected and the purpose of the AI must be limited by design. 5. Data must be deleted upon consumer request. Bernhard DebatinProfessor and Director of the Institute for Applied and Professional Ethics Guidelines for ethical privacy 83
- 83. 1. Clear consentrequests to customer (and easy out) 2. Breach notifications within 72h 3. Right to access all personal data upon request 4. Right to be forgotten in practice 84
- 84. Fairness & Bias Whatis fair anyways? 85
- 85. Risks & Safety Howto protect from potential dangers 86
- 86.
- 87.
- 88.
- 89.
- 90. ‘build a world-classface recognition AI model’ Specific, focus & measurable 91
- 91. ‘build a facerecognition service that can detect male/female genders, with pre- defind specific age groups, and these specific subset of races, and ethnicities in the requirements document (which is grounded in a standard taxonomy from a neutral organization such as the United Nations Race and Ethnicity taxonomy)) with at least 90% accuracy on ‘these’ given specific test datasets’ where ‘these’ test datasets were carefully crafted by the offering management team to have an even distribution of all the genders, age groups, specific races, and ethnicities for which the model is supposed do well’ Specific, focus & measurable 92
- 92.
- 93.
- 94.
- 95.
- 96.
- 97.
- 98. To summarise 1. Specifya robust goal 2. Make sure you respect privacy 3. Make it explainable 4. Make it secure 5. Make it transparent 99
- 99. Exercise Can you identifythe most sensitive data in your organizations? Do you already have a strategy to be compliant? Do you need a big explainability or would you rather get better performance? 100
- 100. 6. Towards betterHCI Human-Computer interactions have to be reconsidered 101
- 101. Human-in-the-loop Design for bettercollaboration 102
- 102. The problem withhype… 103
- 103.
- 104.
- 105.
- 106.
- 107. Human vs Machine •Machines never forget • Crunch numbers and scan fast • Never bored or impatient or tired • People have better emotional nuances + humanity • People have better common sense • People can tackle new tasks easily 108
- 108. 3 types ofcollaboration 109
- 109. AI recommends multipleoptions 110
- 110. AI makes thedecision 111
- 111.
- 112. Keep human psychologyin mind Lots of false positive = irritating Introducing without consultation = resentment Most often good decision = overtrust (automation bias) … 113
- 113. Exercise Try to imaginethe end product of your project, leveraging both AI and humans using the type of collaborations we just talked about What would the user be the most sensitive about ? 114
- 114. Conclusion Create your AIimplementation strategy 115
- 115.