SlideShare a Scribd company logo
HUAWEI TECHNOLOGIES CO., LTD.
www.huawei.com
Managing the AI process:
putting humans (back) in the loop
— Balazs Kegl for Noah's Ark Research Lab, Paris
HUAWEI TECHNOLOGIES CO., LTD. Page 2
 AI research veteran (25 years)
› recently crossing over from academic research to industry
 Leading a team of 15 at Huawei Noah's Ark Lab
in Paris
 Main objective: optimizing engineering systems
› Better
› Cheaper
› More reliable
› Safer
› More energy efficient
Who am I?
HUAWEI TECHNOLOGIES CO., LTD. Page 3
AI: Highly visible recent breakthroughs
HUAWEI TECHNOLOGIES CO., LTD. Page 4
Why these advances
are not already
in engineering systems?
HUAWEI TECHNOLOGIES CO., LTD. Page 5
Engineering systems = ~$10s of trillions per year
HUAWEI TECHNOLOGIES CO., LTD. Page 6
A typical engineering control system
EngineerSystem
𝒂 𝒕
𝒐 𝒕, 𝒓𝒕
Engineer observes
system states and performance indicators,
tunes some parameters time to time
HUAWEI TECHNOLOGIES CO., LTD. Page 7
Automated control, if exists, is based on
deep understanding of the physics
of the system.
HUAWEI TECHNOLOGIES CO., LTD. Page 8
Sometimes it goes wrong
HUAWEI TECHNOLOGIES CO., LTD. Page 9
But mostly it works (it just doesn't learn)
HUAWEI TECHNOLOGIES CO., LTD. Page 10
What is AI (in this context)?
Learn the system behavior
based on historical data
and use it for better control
HUAWEI TECHNOLOGIES CO., LTD. Page 11
Meet the data scientist
 Highly trained scientific minds
› Iterating the scientific method, collecting data, running
a lot of experiments, trial and error
› Work on (predictive) "models"
› Work alone or in small teams
› "Full stack" - little specialization
› Build their sandbox
› Hate operational constraints imposed by
standardized tools
HUAWEI TECHNOLOGIES CO., LTD. Page 12
HUAWEI TECHNOLOGIES CO., LTD. Page 13
Meet the system engineer
 Highly trained operators of systems
› Responsible for safe operation
› Performance is important but secondary (within
limits)
› Little incentive of collecting data and let the data
scientist experiment
HUAWEI TECHNOLOGIES CO., LTD. Page 14
Challenge
The system engineer is the
consumer of AI
(so interfaces should be built between AI and engineer)
And also the
crucial collaborator of the data scientist
HUAWEI TECHNOLOGIES CO., LTD. Page 15
 SE:“I would like you to use AI to control my engineering system.”
 DS: “Ok, can I access your system with an algorithm which takes control of
the system, possibly breaking it sometimes in order to learn?”
 SE: “Over my dead body.”
 DS: “OK, do you have a simulator which I can use to learn a control policy?”
 SE: “We are working on it. But in any case, it will never be good enough to
be trusted.”
 DS: “Can you execute a new control policy, after thorough checking and with
human safeguards, time to time and log the system variables and KPIs?
 SE: “Maybe.”
A typical conversation between data scientists
(DS) and system engineer (SE)
HUAWEI TECHNOLOGIES CO., LTD. Page 16
 Strong top-down mandate
› Don't just create a powerless CDO/CDSO position
 Form "commandos"
› The 70-20-10 rule: 70% of an AI project is change management
 Build trust through iterative pilots
› We need data to start the AI process  we need trust to engage the resources to collect data
 Design and enforce the data science process
› https://towardsdatascience.com/how-to-build-a-data-science-pipeline-f24341848045
 Start building standard operational tools for data science
› Save experience in reusable code
› https://paris-saclay-cds.github.io/ramp-docs/ramp-workflow
Recommendations

More Related Content

Similar to Managing the AI process: putting humans (back) in the loop

What DevOps Isn't
What DevOps Isn'tWhat DevOps Isn't
What DevOps Isn't
Frank Lamantia
 
Testwarez 2009 Use Proper Tool
Testwarez 2009 Use Proper ToolTestwarez 2009 Use Proper Tool
Testwarez 2009 Use Proper Tool
Adam Sandman
 
Agile Chennai 2021 | Achieving High DevOps Maturity through Platform Engineer...
Agile Chennai 2021 | Achieving High DevOps Maturity through Platform Engineer...Agile Chennai 2021 | Achieving High DevOps Maturity through Platform Engineer...
Agile Chennai 2021 | Achieving High DevOps Maturity through Platform Engineer...
AgileNetwork
 
SDLC & DevOps Transformation with Agile
SDLC & DevOps Transformation with AgileSDLC & DevOps Transformation with Agile
SDLC & DevOps Transformation with Agile
Abdel Moneim Emad
 
From Duke of DevOps to Queen of Chaos - Api days 2018
From Duke of DevOps to Queen of Chaos - Api days 2018From Duke of DevOps to Queen of Chaos - Api days 2018
From Duke of DevOps to Queen of Chaos - Api days 2018
Christophe Rochefolle
 
Introduction to DevOps slides.pdf
Introduction to DevOps slides.pdfIntroduction to DevOps slides.pdf
Introduction to DevOps slides.pdf
BoreVishnusai
 
ITLCHN 18 - Automation & DevOps - Automic
ITLCHN 18 -  Automation & DevOps - AutomicITLCHN 18 -  Automation & DevOps - Automic
ITLCHN 18 - Automation & DevOps - Automic
IT Expert Club
 
Embedded software, hardware design and automation software
Embedded software, hardware design and automation softwareEmbedded software, hardware design and automation software
Embedded software, hardware design and automation software
Paul Dao
 
[WSO2Con EU 2018] Enabling Agile Integration Teams
[WSO2Con EU 2018] Enabling Agile Integration Teams[WSO2Con EU 2018] Enabling Agile Integration Teams
[WSO2Con EU 2018] Enabling Agile Integration Teams
WSO2
 
ch1-What are Things?
ch1-What are Things?ch1-What are Things?
ch1-What are Things?
ssuser06ea42
 
bg Meetup München - DevOps Demystified
bg Meetup München - DevOps Demystifiedbg Meetup München - DevOps Demystified
bg Meetup München - DevOps Demystified
borisgloger consulting GmbH
 
Deploying more technology to shift from agility to anti-fragility
Deploying more technology to shift from agility to anti-fragilityDeploying more technology to shift from agility to anti-fragility
Deploying more technology to shift from agility to anti-fragility
Spyros Lambrinidis
 
SETTA'18 Keynote: Intelligent Software Engineering: Synergy between AI and So...
SETTA'18 Keynote: Intelligent Software Engineering: Synergy between AI and So...SETTA'18 Keynote: Intelligent Software Engineering: Synergy between AI and So...
SETTA'18 Keynote: Intelligent Software Engineering: Synergy between AI and So...
Tao Xie
 
Enterprise Architecture in Practice: from Datastore to APIs and Apps
Enterprise Architecture in Practice: from Datastore to APIs and AppsEnterprise Architecture in Practice: from Datastore to APIs and Apps
Enterprise Architecture in Practice: from Datastore to APIs and Apps
WSO2
 
AWS and Dynatrace: Moving your Cloud Strategy to the Next Level
AWS and Dynatrace: Moving your Cloud Strategy to the Next LevelAWS and Dynatrace: Moving your Cloud Strategy to the Next Level
AWS and Dynatrace: Moving your Cloud Strategy to the Next Level
Dynatrace
 
Winning People to DevOps
Winning People to DevOpsWinning People to DevOps
Winning People to DevOps
Matthew Skelton
 
Large scale agile development practices
Large scale agile development practicesLarge scale agile development practices
Large scale agile development practices
Skills Matter
 
Software Development and Quality
Software Development and QualitySoftware Development and Quality
Software Development and Quality
Herwig Habenbacher
 
Robert Mircea & Virgil Chereches: Our Journey To Continuous Delivery at I T.A...
Robert Mircea & Virgil Chereches: Our Journey To Continuous Delivery at I T.A...Robert Mircea & Virgil Chereches: Our Journey To Continuous Delivery at I T.A...
Robert Mircea & Virgil Chereches: Our Journey To Continuous Delivery at I T.A...
Mozaic Works
 
Scaling Enterprise DevOps with CloudBees
Scaling Enterprise DevOps with CloudBeesScaling Enterprise DevOps with CloudBees
Scaling Enterprise DevOps with CloudBees
Deborah Schalm
 

Similar to Managing the AI process: putting humans (back) in the loop (20)

What DevOps Isn't
What DevOps Isn'tWhat DevOps Isn't
What DevOps Isn't
 
Testwarez 2009 Use Proper Tool
Testwarez 2009 Use Proper ToolTestwarez 2009 Use Proper Tool
Testwarez 2009 Use Proper Tool
 
Agile Chennai 2021 | Achieving High DevOps Maturity through Platform Engineer...
Agile Chennai 2021 | Achieving High DevOps Maturity through Platform Engineer...Agile Chennai 2021 | Achieving High DevOps Maturity through Platform Engineer...
Agile Chennai 2021 | Achieving High DevOps Maturity through Platform Engineer...
 
SDLC & DevOps Transformation with Agile
SDLC & DevOps Transformation with AgileSDLC & DevOps Transformation with Agile
SDLC & DevOps Transformation with Agile
 
From Duke of DevOps to Queen of Chaos - Api days 2018
From Duke of DevOps to Queen of Chaos - Api days 2018From Duke of DevOps to Queen of Chaos - Api days 2018
From Duke of DevOps to Queen of Chaos - Api days 2018
 
Introduction to DevOps slides.pdf
Introduction to DevOps slides.pdfIntroduction to DevOps slides.pdf
Introduction to DevOps slides.pdf
 
ITLCHN 18 - Automation & DevOps - Automic
ITLCHN 18 -  Automation & DevOps - AutomicITLCHN 18 -  Automation & DevOps - Automic
ITLCHN 18 - Automation & DevOps - Automic
 
Embedded software, hardware design and automation software
Embedded software, hardware design and automation softwareEmbedded software, hardware design and automation software
Embedded software, hardware design and automation software
 
[WSO2Con EU 2018] Enabling Agile Integration Teams
[WSO2Con EU 2018] Enabling Agile Integration Teams[WSO2Con EU 2018] Enabling Agile Integration Teams
[WSO2Con EU 2018] Enabling Agile Integration Teams
 
ch1-What are Things?
ch1-What are Things?ch1-What are Things?
ch1-What are Things?
 
bg Meetup München - DevOps Demystified
bg Meetup München - DevOps Demystifiedbg Meetup München - DevOps Demystified
bg Meetup München - DevOps Demystified
 
Deploying more technology to shift from agility to anti-fragility
Deploying more technology to shift from agility to anti-fragilityDeploying more technology to shift from agility to anti-fragility
Deploying more technology to shift from agility to anti-fragility
 
SETTA'18 Keynote: Intelligent Software Engineering: Synergy between AI and So...
SETTA'18 Keynote: Intelligent Software Engineering: Synergy between AI and So...SETTA'18 Keynote: Intelligent Software Engineering: Synergy between AI and So...
SETTA'18 Keynote: Intelligent Software Engineering: Synergy between AI and So...
 
Enterprise Architecture in Practice: from Datastore to APIs and Apps
Enterprise Architecture in Practice: from Datastore to APIs and AppsEnterprise Architecture in Practice: from Datastore to APIs and Apps
Enterprise Architecture in Practice: from Datastore to APIs and Apps
 
AWS and Dynatrace: Moving your Cloud Strategy to the Next Level
AWS and Dynatrace: Moving your Cloud Strategy to the Next LevelAWS and Dynatrace: Moving your Cloud Strategy to the Next Level
AWS and Dynatrace: Moving your Cloud Strategy to the Next Level
 
Winning People to DevOps
Winning People to DevOpsWinning People to DevOps
Winning People to DevOps
 
Large scale agile development practices
Large scale agile development practicesLarge scale agile development practices
Large scale agile development practices
 
Software Development and Quality
Software Development and QualitySoftware Development and Quality
Software Development and Quality
 
Robert Mircea & Virgil Chereches: Our Journey To Continuous Delivery at I T.A...
Robert Mircea & Virgil Chereches: Our Journey To Continuous Delivery at I T.A...Robert Mircea & Virgil Chereches: Our Journey To Continuous Delivery at I T.A...
Robert Mircea & Virgil Chereches: Our Journey To Continuous Delivery at I T.A...
 
Scaling Enterprise DevOps with CloudBees
Scaling Enterprise DevOps with CloudBeesScaling Enterprise DevOps with CloudBees
Scaling Enterprise DevOps with CloudBees
 

More from Balázs Kégl

Data-driven hypothesis generation using deep neural nets
Data-driven hypothesis generation using deep neural netsData-driven hypothesis generation using deep neural nets
Data-driven hypothesis generation using deep neural nets
Balázs Kégl
 
Machine learning in scientific workflows
Machine learning in scientific workflowsMachine learning in scientific workflows
Machine learning in scientific workflows
Balázs Kégl
 
A historical introduction to deep learning: hardware, data, and tricks
A historical introduction to deep learning: hardware, data, and tricksA historical introduction to deep learning: hardware, data, and tricks
A historical introduction to deep learning: hardware, data, and tricks
Balázs Kégl
 
Build your own data challenge, or just organize team work
Build your own data challenge, or just organize team workBuild your own data challenge, or just organize team work
Build your own data challenge, or just organize team work
Balázs Kégl
 
RAMP: Collaborative challenge with code submission
RAMP: Collaborative challenge with code submissionRAMP: Collaborative challenge with code submission
RAMP: Collaborative challenge with code submission
Balázs Kégl
 
Deep learning and the systemic challenges of data science initiatives
Deep learning and the systemic challenges of data science initiativesDeep learning and the systemic challenges of data science initiatives
Deep learning and the systemic challenges of data science initiatives
Balázs Kégl
 
What is wrong with data challenges
What is wrong with data challengesWhat is wrong with data challenges
What is wrong with data challenges
Balázs Kégl
 
The systemic challenges in data science initiatives (and some solutions)
The systemic challenges in data science initiatives (and some solutions)The systemic challenges in data science initiatives (and some solutions)
The systemic challenges in data science initiatives (and some solutions)
Balázs Kégl
 
Learning do discover: machine learning in high-energy physics
Learning do discover: machine learning in high-energy physicsLearning do discover: machine learning in high-energy physics
Learning do discover: machine learning in high-energy physics
Balázs Kégl
 
The Paris-Saclay Center for Data Science
The Paris-Saclay Center for Data ScienceThe Paris-Saclay Center for Data Science
The Paris-Saclay Center for Data Science
Balázs Kégl
 

More from Balázs Kégl (10)

Data-driven hypothesis generation using deep neural nets
Data-driven hypothesis generation using deep neural netsData-driven hypothesis generation using deep neural nets
Data-driven hypothesis generation using deep neural nets
 
Machine learning in scientific workflows
Machine learning in scientific workflowsMachine learning in scientific workflows
Machine learning in scientific workflows
 
A historical introduction to deep learning: hardware, data, and tricks
A historical introduction to deep learning: hardware, data, and tricksA historical introduction to deep learning: hardware, data, and tricks
A historical introduction to deep learning: hardware, data, and tricks
 
Build your own data challenge, or just organize team work
Build your own data challenge, or just organize team workBuild your own data challenge, or just organize team work
Build your own data challenge, or just organize team work
 
RAMP: Collaborative challenge with code submission
RAMP: Collaborative challenge with code submissionRAMP: Collaborative challenge with code submission
RAMP: Collaborative challenge with code submission
 
Deep learning and the systemic challenges of data science initiatives
Deep learning and the systemic challenges of data science initiativesDeep learning and the systemic challenges of data science initiatives
Deep learning and the systemic challenges of data science initiatives
 
What is wrong with data challenges
What is wrong with data challengesWhat is wrong with data challenges
What is wrong with data challenges
 
The systemic challenges in data science initiatives (and some solutions)
The systemic challenges in data science initiatives (and some solutions)The systemic challenges in data science initiatives (and some solutions)
The systemic challenges in data science initiatives (and some solutions)
 
Learning do discover: machine learning in high-energy physics
Learning do discover: machine learning in high-energy physicsLearning do discover: machine learning in high-energy physics
Learning do discover: machine learning in high-energy physics
 
The Paris-Saclay Center for Data Science
The Paris-Saclay Center for Data ScienceThe Paris-Saclay Center for Data Science
The Paris-Saclay Center for Data Science
 

Recently uploaded

Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Adtran
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
SOFTTECHHUB
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
Matthew Sinclair
 
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
Zilliz
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
KatiaHIMEUR1
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
SOFTTECHHUB
 
Data structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdfData structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdf
TIPNGVN2
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial ApplicationsLarge Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
Rohit Gautam
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Paige Cruz
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
DianaGray10
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
DianaGray10
 
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionGenerative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Aggregage
 

Recently uploaded (20)

Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
 
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
 
Data structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdfData structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdf
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial ApplicationsLarge Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
 
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionGenerative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
 

Managing the AI process: putting humans (back) in the loop

  • 1. HUAWEI TECHNOLOGIES CO., LTD. www.huawei.com Managing the AI process: putting humans (back) in the loop — Balazs Kegl for Noah's Ark Research Lab, Paris
  • 2. HUAWEI TECHNOLOGIES CO., LTD. Page 2  AI research veteran (25 years) › recently crossing over from academic research to industry  Leading a team of 15 at Huawei Noah's Ark Lab in Paris  Main objective: optimizing engineering systems › Better › Cheaper › More reliable › Safer › More energy efficient Who am I?
  • 3. HUAWEI TECHNOLOGIES CO., LTD. Page 3 AI: Highly visible recent breakthroughs
  • 4. HUAWEI TECHNOLOGIES CO., LTD. Page 4 Why these advances are not already in engineering systems?
  • 5. HUAWEI TECHNOLOGIES CO., LTD. Page 5 Engineering systems = ~$10s of trillions per year
  • 6. HUAWEI TECHNOLOGIES CO., LTD. Page 6 A typical engineering control system EngineerSystem 𝒂 𝒕 𝒐 𝒕, 𝒓𝒕 Engineer observes system states and performance indicators, tunes some parameters time to time
  • 7. HUAWEI TECHNOLOGIES CO., LTD. Page 7 Automated control, if exists, is based on deep understanding of the physics of the system.
  • 8. HUAWEI TECHNOLOGIES CO., LTD. Page 8 Sometimes it goes wrong
  • 9. HUAWEI TECHNOLOGIES CO., LTD. Page 9 But mostly it works (it just doesn't learn)
  • 10. HUAWEI TECHNOLOGIES CO., LTD. Page 10 What is AI (in this context)? Learn the system behavior based on historical data and use it for better control
  • 11. HUAWEI TECHNOLOGIES CO., LTD. Page 11 Meet the data scientist  Highly trained scientific minds › Iterating the scientific method, collecting data, running a lot of experiments, trial and error › Work on (predictive) "models" › Work alone or in small teams › "Full stack" - little specialization › Build their sandbox › Hate operational constraints imposed by standardized tools
  • 12. HUAWEI TECHNOLOGIES CO., LTD. Page 12
  • 13. HUAWEI TECHNOLOGIES CO., LTD. Page 13 Meet the system engineer  Highly trained operators of systems › Responsible for safe operation › Performance is important but secondary (within limits) › Little incentive of collecting data and let the data scientist experiment
  • 14. HUAWEI TECHNOLOGIES CO., LTD. Page 14 Challenge The system engineer is the consumer of AI (so interfaces should be built between AI and engineer) And also the crucial collaborator of the data scientist
  • 15. HUAWEI TECHNOLOGIES CO., LTD. Page 15  SE:“I would like you to use AI to control my engineering system.”  DS: “Ok, can I access your system with an algorithm which takes control of the system, possibly breaking it sometimes in order to learn?”  SE: “Over my dead body.”  DS: “OK, do you have a simulator which I can use to learn a control policy?”  SE: “We are working on it. But in any case, it will never be good enough to be trusted.”  DS: “Can you execute a new control policy, after thorough checking and with human safeguards, time to time and log the system variables and KPIs?  SE: “Maybe.” A typical conversation between data scientists (DS) and system engineer (SE)
  • 16. HUAWEI TECHNOLOGIES CO., LTD. Page 16  Strong top-down mandate › Don't just create a powerless CDO/CDSO position  Form "commandos" › The 70-20-10 rule: 70% of an AI project is change management  Build trust through iterative pilots › We need data to start the AI process  we need trust to engage the resources to collect data  Design and enforce the data science process › https://towardsdatascience.com/how-to-build-a-data-science-pipeline-f24341848045  Start building standard operational tools for data science › Save experience in reusable code › https://paris-saclay-cds.github.io/ramp-docs/ramp-workflow Recommendations