SlideShare a Scribd company logo
© 2021 Thoughtworks | Confidential
ML Governance: A
Practical Guide
Ryan Dawson
Principal Data Consultant
ryan.dawson@thoughtworks.com
Meissane Chami
Lead ML Engineer
meissane.chami@thoughtworks.com
© 2021 Thoughtworks | Confidential
1) Why ML Governance is Confusing
2) An Actionable View of ML Gov
3) Documentation in ML Gov
4) Working out the Details
3
Photo by Bich Tran from Pexels
© 2021 Thoughtworks | Confidential
Why is ML Governance so Confusing?
4
ML GOVERNANCE
Ethics
Responsible
AI
AI for Good
Ethics
Principles
Governance
Frameworks
Model Risk
Management
Graphic Insipred by
https://www.growthbusiness.co.uk/why-governance-must-be-a-priority-
for-startups-2550207/
Bias
Fairness
Transparency
Privacy
Security
MLOps
Reproducibility
Best Practice
Documentation
Audit
Legal
Governance Board
Sign-offs
© 2021 Thoughtworks | Confidential
Governance Requires us
to Stop and Think
● Many teams have little governance
● Tech teams have a delivery focus
● Build now, worry about risks later
● Burden can’t all be on tech
● Requires a collaborative process
5
© 2021 Thoughtworks | Confidential 6
ML
GOVERNANCE
Ethics
Responsible
AI
AI for Good
Ethics
Principles
Governance
Frameworks
Model Risk
Management
Graphic Insipred by
https://www.growthbusiness.co.uk/why-governance-must-be-a-priority-for-startups-
2550207/
Bias
Fairness
Transparency
Privacy
Security
MLOps
Reproducibility
Best Practice
Documentation
Audit
Legal
Governance Board
Sign-offs
© 2021 Thoughtworks | Confidential 7
ML
GOVERNANCE
Ethics
Responsible
AI
AI for Good
Ethics
Principles
Governance
Frameworks
Model Risk
Management
Graphic Insipred by
https://www.growthbusiness.co.uk/why-governance-must-be-a-priority-for-startups-
2550207/
Bias
Fairness
Transparency
Privacy
Security
MLOps
Reproducibility
Best Practice
Documentation
Audit
Legal
Governance Board
Sign-offs
Ethics and
Principles
Tech Practices
and MLOps
Management
and
Frameworks
© 2021 Thoughtworks | Confidential
Ethics is Dominating
the Conversation
● Ethics and Responsible AI are
important
● But only part of the conversation
● Must not neglect the boring stuff
● Boring stuff is core of good
governance
8
© 2021 Thoughtworks | Confidential
Perception of ML Governance
9
Ethics
Governance Board
Bias
Sign-offs
Bureaucracy
Misuse
Privacy
Security
Ethics Principles
MLOps
Explainability
© 2021 Thoughtworks | Confidential
Better View of ML Governance
10
Best Practice
Documentation
Oversight
Board
Decision-making
Peer Review
Ethics
MLOps
Handover
Guidance
© 2021 Thoughtworks | Confidential 11
Slide template based on TearDrop By PresentationGo
Data
Documentation
Data Quality
Data Lineage
Data Labelling
Data Access
Policies
Data Retention
Data Security
Data Architecture
Data Management
Data Integration
ML Risk
Management
ML Best Practice
Responsible AI
ML Governance
Board
Model
Documentation
MLOps
Data Governance ML Governance
© 2021 Thoughtworks | Confidential
The Boring Side of ML
Governance
● How much documentation is appropriate?
● Should you have manual sign-offs?
● If so, when and who should perform them?
● When is an escalation needed?
● What should a governance board do?
● What if you are in a regulated industry?
● How can MLOps help?
● And most importantly, what is the point of
all this governance and how much is too
much?
12
© 2021 Thoughtworks | Confidential
How Much is Too Much?
● Lots of manual checks on code and data will make for slow process
● Team morale may be affected
● Process may not be followed
13
Porridge and bear images public
domain from OpenClipArt
© 2021 Thoughtworks | Confidential
Aside: Bureaucracy = Rule by Desks
14
Desk and Throne public
domain via openclipart
Crown Wissenschaftler-Uni, CC BY-SA
4.0 via Wikimedia Commons
© 2021 Thoughtworks | Confidential
An Actionable View of ML Governance
15
Model card
Model purpose,
design, data
description,
risks
Model
Developer
Model
Validator
Review initiated
when model
ready for
production Model validation
report
Establish clarity,
reproducibility, best
practices
Model owner
approval
Sign-off on clarity,
monitoring plan,
risks
Model
Owner
Changes from
review
© 2021 Thoughtworks | Confidential
Model card
Model purpose,
design, data
description, risks
Model
Developer
Model
Validator
Review initiated
when model ready
for production
Model validation
report
Establish clarity,
reproducibility, best
practices
Model owner
approval
Sign-off on clarity,
monitoring plan, risks
Model
Owner
Changes from
review
© 2021 Thoughtworks | Confidential
Model card
Model purpose,
design, data
description, risks
Model
Developer
Model
Validator
Review initiated
when model ready
for production
Model validation
report
Establish clarity,
reproducibility, best
practices
Model owner
approval
Sign-off on clarity,
monitoring plan, risks
Model
Owner
Changes from
review
© 2021 Thoughtworks | Confidential
Model card
Model purpose,
design, data
description, risks
Model
Developer
Model
Validator
Review initiated
when model ready
for production
Model validation
report
Establish clarity,
reproducibility, best
practices
Model owner
approval
Sign-off on clarity,
monitoring plan, risks
Model
Owner
Changes from
review
© 2021 Thoughtworks | Confidential
Model card
Model purpose,
design, data
description, risks
Model
Developer
Model
Validator
Review initiated
when model ready
for production
Model validation
report
Establish clarity,
reproducibility, best
practices
Model owner
approval
Sign-off on clarity,
monitoring plan, risks
Model
Owner
Changes from
review
© 2021 Thoughtworks | Confidential
Model card
Model
Developer
Model
Validator
Model validation
report
Model owner
approval
Model
Owner
Oversight board
Escalation
possible
Escalation
possible
Escalation
possible
© 2021 Thoughtworks | Confidential
Model card
Model
Developer
Model
Validator
Model validation
report
Model owner
approval
Model
Owner
Oversight board
Escalation
possible
Escalation
possible
Escalation
possible
Oversight board
may also lead a
periodic
review/audit
process
Cycle image public domain by OpenClipArt
© 2021 Thoughtworks | Confidential
Place Decisions with the Right Roles/Guardians
Model
Developer
Model/ Product
Owner
● What does this
model do?
● How does it work?
● What risks does it
have?
● How best to
monitor it?
● Which product/quality
risks are worth taking?
● Which mitigations are
worth the extra time
and effort?
● Sign-off on serious
risks
● Is it ok to use
sensitive/PII data for
this case?
● Where should we be
improving gov/ML as
an org?
Governance
Board
© 2021 Thoughtworks | Confidential
Model Validator is Part of This Too
Model
Developer
Model
Validator
● What does this model do?
● How does it work?
● What risks does it have?
● How best to monitor it?
● Was the development process
robust?
● Has the developer overlooked
anything in best practice or risks?
© 2021 Thoughtworks | Confidential
Documentation in ML Governance
24
24
2
4 2
4
2
4 2
4
© 2021 Thoughtworks | Confidential
Checklists
● Google model cards
● ‘Datasheets for datasets’
● Meta/Facebook reproducibility checklists
● ‘The ML Test Score: A Rubric for ML Production
Readiness and Technical Debt Reduction’
● ‘Towards Yet Another Checklist for New
Datasets’
● ML Cards for D/MLOps Governance by Ian
Hellstrom
Dog Breed Classifier image
from Google
2
5 2
5
© 2021 Thoughtworks | Confidential
Varieties and Purpose of Model Cards
● “Under what conditions does the model perform best
and most consistently? Does it have blind spots? If so,
where?”
○ Dog Breed Classifier: “What kind of photos tend to yield the most
accurate results? Can it handle partially obscured dogs? What about
dogs that are extremely close, extremely far away, or seen from
unusual angles?”
○ Language Translator: “guidance around jargon, slang and dialects,
or measure its tolerance for differences in spelling”
● Card should give an overview of internals and
limitations of model with a view to how it will be used.
● Is an open documentation format. Not a process.
● Focused more on model than data (will return to this)
27
https://modelcards.withgoogle.com/about
© 2021 Thoughtworks | Confidential
Model card
Model purpose,
design, data
description, risks
Model
Developer
Model
Validator
Review initiated
when model ready
for production
Model validation
report
Establish clarity,
reproducibility, best
practices
Model owner
approval
Sign-off on clarity,
monitoring plan, risks
Model
Owner
Changes from
review
© 2021 Thoughtworks | Confidential
Deep-dive Questions on
Process
● Should model validator be from a different
team from model developer?
● Where do Model Cards live?
● Who updates/maintains the cards?
● Does the validator need to fully reproduce
the model and results?
● How much responsibility is on the Model
Developer to explain about the model vs on
the Model Owner to ask Qs?
● Who will be responsible for monitoring in
live?
● How should governance board be formed?
29
© 2021 Thoughtworks | Confidential 30
Model cards will
tend to follow
lead of reference
examples
Detailed
examples are
more burden to
produce and
read
Tricky Questions:
How Much Detail?
● Reference examples will have a big
impact on what documentation
really gets produced
● Examples show developers how
much and what kind of detail is
expected
Photo by Magda Ehlers on Pexels
© 2021 Thoughtworks | Confidential
Test the Process
● Can’t prove workability in a vacuum
● Pick some cases and work it through together
31
Photo from WikiImages on Pixabay
© 2021 Thoughtworks | Confidential
Tradeoffs and Opportunities
32
● Not enough checks and you fail to
surface risks
● Irrelevant/inappropriate checks slow
you down
● Process can encourage best practice -
this is an opportunity
© 2021 Thoughtworks | Confidential
Hidden Risks
33
Known
Risks
Overlooked
Risks
Financial Risk
Legal Risk
Reputation Risk
Quality Risk
Ethical Risk
Delivery Risk
Regulatory Risk
© 2021 Thoughtworks | Confidential
Credit Assessment
34
Known
Risks
Overlooked
Risks
Financial Risks
First Order Gender Bias
Indirect Gender Bias?
(e.g. from occupation)
Actually not overlooked
Similar situation
seemed to have
happened with
AppleCard in 2019
but investigation
found no bias
www.bbc.co.uk/news/business-50432634
www.theverge.com/2021/3/23/22347127/goldman-
sachs-apple-card-no-gender-discrimination
Reputation Risk from
Customer Confusion
© 2021 Thoughtworks | Confidential
Summary
Photo by Romain Dancre from Unsplash
● ML Governance is multi-faceted and can be confusing
● Simple template Process based around model cards and defined roles
● Model Owner role key to Risk Management
● Needs to be shaped to your team/s

More Related Content

What's hot

Digital Transformation & Cloud Profitability
Digital Transformation & Cloud ProfitabilityDigital Transformation & Cloud Profitability
Digital Transformation & Cloud Profitability
Gui Carvalhal
 
Driving your BA Career - From Business Analyst to Business Architect
Driving your BA Career - From Business Analyst to Business ArchitectDriving your BA Career - From Business Analyst to Business Architect
Driving your BA Career - From Business Analyst to Business Architect
Enterprise Architects
 
Cloud Migration: A How-To Guide
Cloud Migration: A How-To GuideCloud Migration: A How-To Guide
Cloud Migration: A How-To Guide
Amazon Web Services
 
TechEvent Cloud Governance
TechEvent Cloud GovernanceTechEvent Cloud Governance
TechEvent Cloud Governance
Trivadis
 
Governments Are Going Digital
Governments Are Going DigitalGovernments Are Going Digital
Governments Are Going Digital
Boston Consulting Group
 
Thabo Ndlela- Leveraging AI for enhanced Customer Service and Experience
Thabo Ndlela- Leveraging AI for enhanced Customer Service and ExperienceThabo Ndlela- Leveraging AI for enhanced Customer Service and Experience
Thabo Ndlela- Leveraging AI for enhanced Customer Service and Experience
itnewsafrica
 
Digital Transformation
Digital TransformationDigital Transformation
Digital Transformation
Vishal Sharma
 
PLM-Seminar at Gardermoen: How the idea of single BoM can fit variant and con...
PLM-Seminar at Gardermoen: How the idea of single BoM can fit variant and con...PLM-Seminar at Gardermoen: How the idea of single BoM can fit variant and con...
PLM-Seminar at Gardermoen: How the idea of single BoM can fit variant and con...
Oleg Shilovitsky
 
Technology Vision 2022: Communications Industry | Accenture
Technology Vision 2022: Communications Industry | AccentureTechnology Vision 2022: Communications Industry | Accenture
Technology Vision 2022: Communications Industry | Accenture
accenture
 
Hold Firm: The State of Cyber Resilience in Banking and Capital Markets
Hold Firm: The State of Cyber Resilience in Banking and Capital MarketsHold Firm: The State of Cyber Resilience in Banking and Capital Markets
Hold Firm: The State of Cyber Resilience in Banking and Capital Markets
accenture
 
Enabling the Utility Business of the Future
Enabling the Utility Business of the FutureEnabling the Utility Business of the Future
Enabling the Utility Business of the Future
accenture
 
Privileged Access Management - Unsticking Your PAM Program - CIS 2015
Privileged Access Management - Unsticking Your PAM Program - CIS 2015Privileged Access Management - Unsticking Your PAM Program - CIS 2015
Privileged Access Management - Unsticking Your PAM Program - CIS 2015
Lance Peterman
 
Growth-Driving Enterprise Innovation Model | Accenture
Growth-Driving Enterprise Innovation Model | AccentureGrowth-Driving Enterprise Innovation Model | Accenture
Growth-Driving Enterprise Innovation Model | Accenture
accenture
 
What is IT Governance?
What is IT Governance?What is IT Governance?
What is IT Governance?
Mansoor Adenwala
 
Solution Architecture and Solution Acquisition
Solution Architecture and Solution AcquisitionSolution Architecture and Solution Acquisition
Solution Architecture and Solution Acquisition
Alan McSweeney
 
Generative AI - The New Reality: How Key Players Are Progressing
Generative AI - The New Reality: How Key Players Are Progressing Generative AI - The New Reality: How Key Players Are Progressing
Generative AI - The New Reality: How Key Players Are Progressing
Vishal Sharma
 
IT4IT - The Full Story for Digital Transformation - Part 2
IT4IT - The Full Story for Digital Transformation - Part 2IT4IT - The Full Story for Digital Transformation - Part 2
IT4IT - The Full Story for Digital Transformation - Part 2
Mohamed Zakarya Abdelgawad
 
Implement cobit in your organization
Implement cobit in your organizationImplement cobit in your organization
Implement cobit in your organization
Cheikh Hamallah DJIBA
 
An Overview of Best Practices for Large Scale Migrations - AWS Transformation...
An Overview of Best Practices for Large Scale Migrations - AWS Transformation...An Overview of Best Practices for Large Scale Migrations - AWS Transformation...
An Overview of Best Practices for Large Scale Migrations - AWS Transformation...
Amazon Web Services
 
Engineering Services: con gli ingegneri per creare valore sostenibile
Engineering Services: con gli ingegneri per creare valore sostenibileEngineering Services: con gli ingegneri per creare valore sostenibile
Engineering Services: con gli ingegneri per creare valore sostenibile
accenture
 

What's hot (20)

Digital Transformation & Cloud Profitability
Digital Transformation & Cloud ProfitabilityDigital Transformation & Cloud Profitability
Digital Transformation & Cloud Profitability
 
Driving your BA Career - From Business Analyst to Business Architect
Driving your BA Career - From Business Analyst to Business ArchitectDriving your BA Career - From Business Analyst to Business Architect
Driving your BA Career - From Business Analyst to Business Architect
 
Cloud Migration: A How-To Guide
Cloud Migration: A How-To GuideCloud Migration: A How-To Guide
Cloud Migration: A How-To Guide
 
TechEvent Cloud Governance
TechEvent Cloud GovernanceTechEvent Cloud Governance
TechEvent Cloud Governance
 
Governments Are Going Digital
Governments Are Going DigitalGovernments Are Going Digital
Governments Are Going Digital
 
Thabo Ndlela- Leveraging AI for enhanced Customer Service and Experience
Thabo Ndlela- Leveraging AI for enhanced Customer Service and ExperienceThabo Ndlela- Leveraging AI for enhanced Customer Service and Experience
Thabo Ndlela- Leveraging AI for enhanced Customer Service and Experience
 
Digital Transformation
Digital TransformationDigital Transformation
Digital Transformation
 
PLM-Seminar at Gardermoen: How the idea of single BoM can fit variant and con...
PLM-Seminar at Gardermoen: How the idea of single BoM can fit variant and con...PLM-Seminar at Gardermoen: How the idea of single BoM can fit variant and con...
PLM-Seminar at Gardermoen: How the idea of single BoM can fit variant and con...
 
Technology Vision 2022: Communications Industry | Accenture
Technology Vision 2022: Communications Industry | AccentureTechnology Vision 2022: Communications Industry | Accenture
Technology Vision 2022: Communications Industry | Accenture
 
Hold Firm: The State of Cyber Resilience in Banking and Capital Markets
Hold Firm: The State of Cyber Resilience in Banking and Capital MarketsHold Firm: The State of Cyber Resilience in Banking and Capital Markets
Hold Firm: The State of Cyber Resilience in Banking and Capital Markets
 
Enabling the Utility Business of the Future
Enabling the Utility Business of the FutureEnabling the Utility Business of the Future
Enabling the Utility Business of the Future
 
Privileged Access Management - Unsticking Your PAM Program - CIS 2015
Privileged Access Management - Unsticking Your PAM Program - CIS 2015Privileged Access Management - Unsticking Your PAM Program - CIS 2015
Privileged Access Management - Unsticking Your PAM Program - CIS 2015
 
Growth-Driving Enterprise Innovation Model | Accenture
Growth-Driving Enterprise Innovation Model | AccentureGrowth-Driving Enterprise Innovation Model | Accenture
Growth-Driving Enterprise Innovation Model | Accenture
 
What is IT Governance?
What is IT Governance?What is IT Governance?
What is IT Governance?
 
Solution Architecture and Solution Acquisition
Solution Architecture and Solution AcquisitionSolution Architecture and Solution Acquisition
Solution Architecture and Solution Acquisition
 
Generative AI - The New Reality: How Key Players Are Progressing
Generative AI - The New Reality: How Key Players Are Progressing Generative AI - The New Reality: How Key Players Are Progressing
Generative AI - The New Reality: How Key Players Are Progressing
 
IT4IT - The Full Story for Digital Transformation - Part 2
IT4IT - The Full Story for Digital Transformation - Part 2IT4IT - The Full Story for Digital Transformation - Part 2
IT4IT - The Full Story for Digital Transformation - Part 2
 
Implement cobit in your organization
Implement cobit in your organizationImplement cobit in your organization
Implement cobit in your organization
 
An Overview of Best Practices for Large Scale Migrations - AWS Transformation...
An Overview of Best Practices for Large Scale Migrations - AWS Transformation...An Overview of Best Practices for Large Scale Migrations - AWS Transformation...
An Overview of Best Practices for Large Scale Migrations - AWS Transformation...
 
Engineering Services: con gli ingegneri per creare valore sostenibile
Engineering Services: con gli ingegneri per creare valore sostenibileEngineering Services: con gli ingegneri per creare valore sostenibile
Engineering Services: con gli ingegneri per creare valore sostenibile
 

Similar to mlops.community meetup - ML Governance_ A Practical Guide.pptx

Agile Mumbai 2022 - Ashwinee Singh | Agile in AI or AI in Agile?
Agile Mumbai 2022 - Ashwinee Singh | Agile in AI or AI in Agile?Agile Mumbai 2022 - Ashwinee Singh | Agile in AI or AI in Agile?
Agile Mumbai 2022 - Ashwinee Singh | Agile in AI or AI in Agile?
AgileNetwork
 
MLOps - Getting Machine Learning Into Production
MLOps - Getting Machine Learning Into ProductionMLOps - Getting Machine Learning Into Production
MLOps - Getting Machine Learning Into Production
Michael Pearce
 
The Good, The Bad, and The Metrics
 The Good, The Bad, and The Metrics The Good, The Bad, and The Metrics
The Good, The Bad, and The Metrics
TeamQualityPro
 
Senior PM with Real World Experience
Senior PM with Real World ExperienceSenior PM with Real World Experience
Senior PM with Real World Experience
Alecia Chrin
 
Evolving Towards a Modernized Platform: Our Success Story
Evolving Towards a Modernized Platform: Our Success StoryEvolving Towards a Modernized Platform: Our Success Story
Evolving Towards a Modernized Platform: Our Success Story
VMware Tanzu
 
Evolving Towards a Modernized Platform: Our Success Story
Evolving Towards a Modernized Platform: Our Success StoryEvolving Towards a Modernized Platform: Our Success Story
Evolving Towards a Modernized Platform: Our Success Story
VMware Tanzu
 
14.1 features
14.1 features14.1 features
14.1 features
Curtis Tsang
 
Why it's time to rethink your approach to Enterprise Architecture
Why it's time to rethink your approach to Enterprise ArchitectureWhy it's time to rethink your approach to Enterprise Architecture
Why it's time to rethink your approach to Enterprise Architecture
LeanIX GmbH
 
Idge dell private cloud2014 qp #1
Idge dell private cloud2014 qp #1Idge dell private cloud2014 qp #1
Idge dell private cloud2014 qp #1
jmariani14
 
Why Data Modeling Is Fundamental
Why Data Modeling Is FundamentalWhy Data Modeling Is Fundamental
Why Data Modeling Is Fundamental
DATAVERSITY
 
The Value of Predictive Analytics and Decision Modeling
The Value of Predictive Analytics and Decision ModelingThe Value of Predictive Analytics and Decision Modeling
The Value of Predictive Analytics and Decision Modeling
Decision Management Solutions
 
Cloud cpr uncc cloud computing conference 2013
Cloud cpr   uncc cloud computing conference 2013Cloud cpr   uncc cloud computing conference 2013
Cloud cpr uncc cloud computing conference 2013
C5_LUCK
 
How to Build a Business Case and Measure the Effects of Leadership Developmen...
How to Build a Business Case and Measure the Effects of Leadership Developmen...How to Build a Business Case and Measure the Effects of Leadership Developmen...
How to Build a Business Case and Measure the Effects of Leadership Developmen...
Human Capital Media
 
PM-1 Overview.ppt
PM-1 Overview.pptPM-1 Overview.ppt
PM-1 Overview.ppt
natisil1
 
Bridging the AI Gap: Building Stakeholder Support
Bridging the AI Gap: Building Stakeholder SupportBridging the AI Gap: Building Stakeholder Support
Bridging the AI Gap: Building Stakeholder Support
Peter Skomoroch
 
SCG Sales Tech Guide
SCG Sales Tech Guide SCG Sales Tech Guide
SCG Sales Tech Guide
Robbie Burns
 
Business Process Design and Re-engineering
Business Process Design and Re-engineeringBusiness Process Design and Re-engineering
Business Process Design and Re-engineering
Mustafa Jarrar
 
Top Trends and Challenges in the Cloud
Top Trends and Challenges in the CloudTop Trends and Challenges in the Cloud
Top Trends and Challenges in the Cloud
Precisely
 
Design decisions in job architectures and competency modeling June 2020
Design decisions in job architectures and competency modeling June 2020Design decisions in job architectures and competency modeling June 2020
Design decisions in job architectures and competency modeling June 2020
Steven Forth
 
IBM Innovate - Uderstanding DevOps
IBM Innovate - Uderstanding DevOpsIBM Innovate - Uderstanding DevOps
IBM Innovate - Uderstanding DevOps
Sanjeev Sharma
 

Similar to mlops.community meetup - ML Governance_ A Practical Guide.pptx (20)

Agile Mumbai 2022 - Ashwinee Singh | Agile in AI or AI in Agile?
Agile Mumbai 2022 - Ashwinee Singh | Agile in AI or AI in Agile?Agile Mumbai 2022 - Ashwinee Singh | Agile in AI or AI in Agile?
Agile Mumbai 2022 - Ashwinee Singh | Agile in AI or AI in Agile?
 
MLOps - Getting Machine Learning Into Production
MLOps - Getting Machine Learning Into ProductionMLOps - Getting Machine Learning Into Production
MLOps - Getting Machine Learning Into Production
 
The Good, The Bad, and The Metrics
 The Good, The Bad, and The Metrics The Good, The Bad, and The Metrics
The Good, The Bad, and The Metrics
 
Senior PM with Real World Experience
Senior PM with Real World ExperienceSenior PM with Real World Experience
Senior PM with Real World Experience
 
Evolving Towards a Modernized Platform: Our Success Story
Evolving Towards a Modernized Platform: Our Success StoryEvolving Towards a Modernized Platform: Our Success Story
Evolving Towards a Modernized Platform: Our Success Story
 
Evolving Towards a Modernized Platform: Our Success Story
Evolving Towards a Modernized Platform: Our Success StoryEvolving Towards a Modernized Platform: Our Success Story
Evolving Towards a Modernized Platform: Our Success Story
 
14.1 features
14.1 features14.1 features
14.1 features
 
Why it's time to rethink your approach to Enterprise Architecture
Why it's time to rethink your approach to Enterprise ArchitectureWhy it's time to rethink your approach to Enterprise Architecture
Why it's time to rethink your approach to Enterprise Architecture
 
Idge dell private cloud2014 qp #1
Idge dell private cloud2014 qp #1Idge dell private cloud2014 qp #1
Idge dell private cloud2014 qp #1
 
Why Data Modeling Is Fundamental
Why Data Modeling Is FundamentalWhy Data Modeling Is Fundamental
Why Data Modeling Is Fundamental
 
The Value of Predictive Analytics and Decision Modeling
The Value of Predictive Analytics and Decision ModelingThe Value of Predictive Analytics and Decision Modeling
The Value of Predictive Analytics and Decision Modeling
 
Cloud cpr uncc cloud computing conference 2013
Cloud cpr   uncc cloud computing conference 2013Cloud cpr   uncc cloud computing conference 2013
Cloud cpr uncc cloud computing conference 2013
 
How to Build a Business Case and Measure the Effects of Leadership Developmen...
How to Build a Business Case and Measure the Effects of Leadership Developmen...How to Build a Business Case and Measure the Effects of Leadership Developmen...
How to Build a Business Case and Measure the Effects of Leadership Developmen...
 
PM-1 Overview.ppt
PM-1 Overview.pptPM-1 Overview.ppt
PM-1 Overview.ppt
 
Bridging the AI Gap: Building Stakeholder Support
Bridging the AI Gap: Building Stakeholder SupportBridging the AI Gap: Building Stakeholder Support
Bridging the AI Gap: Building Stakeholder Support
 
SCG Sales Tech Guide
SCG Sales Tech Guide SCG Sales Tech Guide
SCG Sales Tech Guide
 
Business Process Design and Re-engineering
Business Process Design and Re-engineeringBusiness Process Design and Re-engineering
Business Process Design and Re-engineering
 
Top Trends and Challenges in the Cloud
Top Trends and Challenges in the CloudTop Trends and Challenges in the Cloud
Top Trends and Challenges in the Cloud
 
Design decisions in job architectures and competency modeling June 2020
Design decisions in job architectures and competency modeling June 2020Design decisions in job architectures and competency modeling June 2020
Design decisions in job architectures and competency modeling June 2020
 
IBM Innovate - Uderstanding DevOps
IBM Innovate - Uderstanding DevOpsIBM Innovate - Uderstanding DevOps
IBM Innovate - Uderstanding DevOps
 

More from Ryan Dawson

Conspiracy Theories in the Information Age
Conspiracy Theories in the Information AgeConspiracy Theories in the Information Age
Conspiracy Theories in the Information Age
Ryan Dawson
 
Why is dev ops for machine learning so different - dataxdays
Why is dev ops for machine learning so different  - dataxdaysWhy is dev ops for machine learning so different  - dataxdays
Why is dev ops for machine learning so different - dataxdays
Ryan Dawson
 
Maximising teamwork in delivering software products
Maximising teamwork in delivering software productsMaximising teamwork in delivering software products
Maximising teamwork in delivering software products
Ryan Dawson
 
Maximising teamwork in delivering software products
Maximising teamwork in delivering software products Maximising teamwork in delivering software products
Maximising teamwork in delivering software products
Ryan Dawson
 
Java vs challenger languages
Java vs challenger languagesJava vs challenger languages
Java vs challenger languages
Ryan Dawson
 
Challenges for AI in prod
Challenges for AI in prodChallenges for AI in prod
Challenges for AI in prod
Ryan Dawson
 
From training to explainability via git ops
From training to explainability via git opsFrom training to explainability via git ops
From training to explainability via git ops
Ryan Dawson
 
Why is dev ops for machine learning so different
Why is dev ops for machine learning so differentWhy is dev ops for machine learning so different
Why is dev ops for machine learning so different
Ryan Dawson
 
How open source is funded the enterprise differentiation tightrope (1)
How open source is funded  the enterprise differentiation tightrope (1)How open source is funded  the enterprise differentiation tightrope (1)
How open source is funded the enterprise differentiation tightrope (1)
Ryan Dawson
 
From java monolith to kubernetes microservices - an open source journey with ...
From java monolith to kubernetes microservices - an open source journey with ...From java monolith to kubernetes microservices - an open source journey with ...
From java monolith to kubernetes microservices - an open source journey with ...
Ryan Dawson
 
Whirlwind tour of activiti 7
Whirlwind tour of activiti 7Whirlwind tour of activiti 7
Whirlwind tour of activiti 7
Ryan Dawson
 
Jdk.io cloud native business automation
Jdk.io cloud native business automationJdk.io cloud native business automation
Jdk.io cloud native business automation
Ryan Dawson
 
Identity management and single sign on - how much flexibility
Identity management and single sign on - how much flexibilityIdentity management and single sign on - how much flexibility
Identity management and single sign on - how much flexibility
Ryan Dawson
 
Activiti Cloud Deep Dive
Activiti Cloud Deep DiveActiviti Cloud Deep Dive
Activiti Cloud Deep Dive
Ryan Dawson
 

More from Ryan Dawson (14)

Conspiracy Theories in the Information Age
Conspiracy Theories in the Information AgeConspiracy Theories in the Information Age
Conspiracy Theories in the Information Age
 
Why is dev ops for machine learning so different - dataxdays
Why is dev ops for machine learning so different  - dataxdaysWhy is dev ops for machine learning so different  - dataxdays
Why is dev ops for machine learning so different - dataxdays
 
Maximising teamwork in delivering software products
Maximising teamwork in delivering software productsMaximising teamwork in delivering software products
Maximising teamwork in delivering software products
 
Maximising teamwork in delivering software products
Maximising teamwork in delivering software products Maximising teamwork in delivering software products
Maximising teamwork in delivering software products
 
Java vs challenger languages
Java vs challenger languagesJava vs challenger languages
Java vs challenger languages
 
Challenges for AI in prod
Challenges for AI in prodChallenges for AI in prod
Challenges for AI in prod
 
From training to explainability via git ops
From training to explainability via git opsFrom training to explainability via git ops
From training to explainability via git ops
 
Why is dev ops for machine learning so different
Why is dev ops for machine learning so differentWhy is dev ops for machine learning so different
Why is dev ops for machine learning so different
 
How open source is funded the enterprise differentiation tightrope (1)
How open source is funded  the enterprise differentiation tightrope (1)How open source is funded  the enterprise differentiation tightrope (1)
How open source is funded the enterprise differentiation tightrope (1)
 
From java monolith to kubernetes microservices - an open source journey with ...
From java monolith to kubernetes microservices - an open source journey with ...From java monolith to kubernetes microservices - an open source journey with ...
From java monolith to kubernetes microservices - an open source journey with ...
 
Whirlwind tour of activiti 7
Whirlwind tour of activiti 7Whirlwind tour of activiti 7
Whirlwind tour of activiti 7
 
Jdk.io cloud native business automation
Jdk.io cloud native business automationJdk.io cloud native business automation
Jdk.io cloud native business automation
 
Identity management and single sign on - how much flexibility
Identity management and single sign on - how much flexibilityIdentity management and single sign on - how much flexibility
Identity management and single sign on - how much flexibility
 
Activiti Cloud Deep Dive
Activiti Cloud Deep DiveActiviti Cloud Deep Dive
Activiti Cloud Deep Dive
 

Recently uploaded

Penify - Let AI do the Documentation, you write the Code.
Penify - Let AI do the Documentation, you write the Code.Penify - Let AI do the Documentation, you write the Code.
Penify - Let AI do the Documentation, you write the Code.
KrishnaveniMohan1
 
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptxOperational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
sandeepmenon62
 
Alluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Alluxio Webinar | 10x Faster Trino Queries on Your Data PlatformAlluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Alluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Alluxio, Inc.
 
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
kalichargn70th171
 
Going AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applicationsGoing AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applications
Alina Yurenko
 
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
Luigi Fugaro
 
一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理
dakas1
 
Photoshop Tutorial for Beginners (2024 Edition)
Photoshop Tutorial for Beginners (2024 Edition)Photoshop Tutorial for Beginners (2024 Edition)
Photoshop Tutorial for Beginners (2024 Edition)
alowpalsadig
 
Secure-by-Design Using Hardware and Software Protection for FDA Compliance
Secure-by-Design Using Hardware and Software Protection for FDA ComplianceSecure-by-Design Using Hardware and Software Protection for FDA Compliance
Secure-by-Design Using Hardware and Software Protection for FDA Compliance
ICS
 
42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert
vaishalijagtap12
 
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdfBaha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid
 
Computer Science & Engineering VI Sem- New Syllabus.pdf
Computer Science & Engineering VI Sem- New Syllabus.pdfComputer Science & Engineering VI Sem- New Syllabus.pdf
Computer Science & Engineering VI Sem- New Syllabus.pdf
chandangoswami40933
 
Superpower Your Apache Kafka Applications Development with Complementary Open...
Superpower Your Apache Kafka Applications Development with Complementary Open...Superpower Your Apache Kafka Applications Development with Complementary Open...
Superpower Your Apache Kafka Applications Development with Complementary Open...
Paul Brebner
 
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Paul Brebner
 
Enhanced Screen Flows UI/UX using SLDS with Tom Kitt
Enhanced Screen Flows UI/UX using SLDS with Tom KittEnhanced Screen Flows UI/UX using SLDS with Tom Kitt
Enhanced Screen Flows UI/UX using SLDS with Tom Kitt
Peter Caitens
 
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
widenerjobeyrl638
 
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in NashikUpturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies
 
Modelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - AmsterdamModelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - Amsterdam
Alberto Brandolini
 
Microsoft-Power-Platform-Adoption-Planning.pptx
Microsoft-Power-Platform-Adoption-Planning.pptxMicrosoft-Power-Platform-Adoption-Planning.pptx
Microsoft-Power-Platform-Adoption-Planning.pptx
jrodriguezq3110
 
Beginner's Guide to Observability@Devoxx PL 2024
Beginner's  Guide to Observability@Devoxx PL 2024Beginner's  Guide to Observability@Devoxx PL 2024
Beginner's Guide to Observability@Devoxx PL 2024
michniczscribd
 

Recently uploaded (20)

Penify - Let AI do the Documentation, you write the Code.
Penify - Let AI do the Documentation, you write the Code.Penify - Let AI do the Documentation, you write the Code.
Penify - Let AI do the Documentation, you write the Code.
 
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptxOperational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
 
Alluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Alluxio Webinar | 10x Faster Trino Queries on Your Data PlatformAlluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Alluxio Webinar | 10x Faster Trino Queries on Your Data Platform
 
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
 
Going AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applicationsGoing AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applications
 
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
 
一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理
 
Photoshop Tutorial for Beginners (2024 Edition)
Photoshop Tutorial for Beginners (2024 Edition)Photoshop Tutorial for Beginners (2024 Edition)
Photoshop Tutorial for Beginners (2024 Edition)
 
Secure-by-Design Using Hardware and Software Protection for FDA Compliance
Secure-by-Design Using Hardware and Software Protection for FDA ComplianceSecure-by-Design Using Hardware and Software Protection for FDA Compliance
Secure-by-Design Using Hardware and Software Protection for FDA Compliance
 
42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert
 
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdfBaha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
 
Computer Science & Engineering VI Sem- New Syllabus.pdf
Computer Science & Engineering VI Sem- New Syllabus.pdfComputer Science & Engineering VI Sem- New Syllabus.pdf
Computer Science & Engineering VI Sem- New Syllabus.pdf
 
Superpower Your Apache Kafka Applications Development with Complementary Open...
Superpower Your Apache Kafka Applications Development with Complementary Open...Superpower Your Apache Kafka Applications Development with Complementary Open...
Superpower Your Apache Kafka Applications Development with Complementary Open...
 
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
 
Enhanced Screen Flows UI/UX using SLDS with Tom Kitt
Enhanced Screen Flows UI/UX using SLDS with Tom KittEnhanced Screen Flows UI/UX using SLDS with Tom Kitt
Enhanced Screen Flows UI/UX using SLDS with Tom Kitt
 
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
 
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in NashikUpturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in Nashik
 
Modelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - AmsterdamModelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - Amsterdam
 
Microsoft-Power-Platform-Adoption-Planning.pptx
Microsoft-Power-Platform-Adoption-Planning.pptxMicrosoft-Power-Platform-Adoption-Planning.pptx
Microsoft-Power-Platform-Adoption-Planning.pptx
 
Beginner's Guide to Observability@Devoxx PL 2024
Beginner's  Guide to Observability@Devoxx PL 2024Beginner's  Guide to Observability@Devoxx PL 2024
Beginner's Guide to Observability@Devoxx PL 2024
 

mlops.community meetup - ML Governance_ A Practical Guide.pptx

  • 1. © 2021 Thoughtworks | Confidential ML Governance: A Practical Guide Ryan Dawson Principal Data Consultant ryan.dawson@thoughtworks.com Meissane Chami Lead ML Engineer meissane.chami@thoughtworks.com
  • 2. © 2021 Thoughtworks | Confidential 1) Why ML Governance is Confusing 2) An Actionable View of ML Gov 3) Documentation in ML Gov 4) Working out the Details 3 Photo by Bich Tran from Pexels
  • 3. © 2021 Thoughtworks | Confidential Why is ML Governance so Confusing? 4 ML GOVERNANCE Ethics Responsible AI AI for Good Ethics Principles Governance Frameworks Model Risk Management Graphic Insipred by https://www.growthbusiness.co.uk/why-governance-must-be-a-priority- for-startups-2550207/ Bias Fairness Transparency Privacy Security MLOps Reproducibility Best Practice Documentation Audit Legal Governance Board Sign-offs
  • 4. © 2021 Thoughtworks | Confidential Governance Requires us to Stop and Think ● Many teams have little governance ● Tech teams have a delivery focus ● Build now, worry about risks later ● Burden can’t all be on tech ● Requires a collaborative process 5
  • 5. © 2021 Thoughtworks | Confidential 6 ML GOVERNANCE Ethics Responsible AI AI for Good Ethics Principles Governance Frameworks Model Risk Management Graphic Insipred by https://www.growthbusiness.co.uk/why-governance-must-be-a-priority-for-startups- 2550207/ Bias Fairness Transparency Privacy Security MLOps Reproducibility Best Practice Documentation Audit Legal Governance Board Sign-offs
  • 6. © 2021 Thoughtworks | Confidential 7 ML GOVERNANCE Ethics Responsible AI AI for Good Ethics Principles Governance Frameworks Model Risk Management Graphic Insipred by https://www.growthbusiness.co.uk/why-governance-must-be-a-priority-for-startups- 2550207/ Bias Fairness Transparency Privacy Security MLOps Reproducibility Best Practice Documentation Audit Legal Governance Board Sign-offs Ethics and Principles Tech Practices and MLOps Management and Frameworks
  • 7. © 2021 Thoughtworks | Confidential Ethics is Dominating the Conversation ● Ethics and Responsible AI are important ● But only part of the conversation ● Must not neglect the boring stuff ● Boring stuff is core of good governance 8
  • 8. © 2021 Thoughtworks | Confidential Perception of ML Governance 9 Ethics Governance Board Bias Sign-offs Bureaucracy Misuse Privacy Security Ethics Principles MLOps Explainability
  • 9. © 2021 Thoughtworks | Confidential Better View of ML Governance 10 Best Practice Documentation Oversight Board Decision-making Peer Review Ethics MLOps Handover Guidance
  • 10. © 2021 Thoughtworks | Confidential 11 Slide template based on TearDrop By PresentationGo Data Documentation Data Quality Data Lineage Data Labelling Data Access Policies Data Retention Data Security Data Architecture Data Management Data Integration ML Risk Management ML Best Practice Responsible AI ML Governance Board Model Documentation MLOps Data Governance ML Governance
  • 11. © 2021 Thoughtworks | Confidential The Boring Side of ML Governance ● How much documentation is appropriate? ● Should you have manual sign-offs? ● If so, when and who should perform them? ● When is an escalation needed? ● What should a governance board do? ● What if you are in a regulated industry? ● How can MLOps help? ● And most importantly, what is the point of all this governance and how much is too much? 12
  • 12. © 2021 Thoughtworks | Confidential How Much is Too Much? ● Lots of manual checks on code and data will make for slow process ● Team morale may be affected ● Process may not be followed 13 Porridge and bear images public domain from OpenClipArt
  • 13. © 2021 Thoughtworks | Confidential Aside: Bureaucracy = Rule by Desks 14 Desk and Throne public domain via openclipart Crown Wissenschaftler-Uni, CC BY-SA 4.0 via Wikimedia Commons
  • 14. © 2021 Thoughtworks | Confidential An Actionable View of ML Governance 15 Model card Model purpose, design, data description, risks Model Developer Model Validator Review initiated when model ready for production Model validation report Establish clarity, reproducibility, best practices Model owner approval Sign-off on clarity, monitoring plan, risks Model Owner Changes from review
  • 15. © 2021 Thoughtworks | Confidential Model card Model purpose, design, data description, risks Model Developer Model Validator Review initiated when model ready for production Model validation report Establish clarity, reproducibility, best practices Model owner approval Sign-off on clarity, monitoring plan, risks Model Owner Changes from review
  • 16. © 2021 Thoughtworks | Confidential Model card Model purpose, design, data description, risks Model Developer Model Validator Review initiated when model ready for production Model validation report Establish clarity, reproducibility, best practices Model owner approval Sign-off on clarity, monitoring plan, risks Model Owner Changes from review
  • 17. © 2021 Thoughtworks | Confidential Model card Model purpose, design, data description, risks Model Developer Model Validator Review initiated when model ready for production Model validation report Establish clarity, reproducibility, best practices Model owner approval Sign-off on clarity, monitoring plan, risks Model Owner Changes from review
  • 18. © 2021 Thoughtworks | Confidential Model card Model purpose, design, data description, risks Model Developer Model Validator Review initiated when model ready for production Model validation report Establish clarity, reproducibility, best practices Model owner approval Sign-off on clarity, monitoring plan, risks Model Owner Changes from review
  • 19. © 2021 Thoughtworks | Confidential Model card Model Developer Model Validator Model validation report Model owner approval Model Owner Oversight board Escalation possible Escalation possible Escalation possible
  • 20. © 2021 Thoughtworks | Confidential Model card Model Developer Model Validator Model validation report Model owner approval Model Owner Oversight board Escalation possible Escalation possible Escalation possible Oversight board may also lead a periodic review/audit process Cycle image public domain by OpenClipArt
  • 21. © 2021 Thoughtworks | Confidential Place Decisions with the Right Roles/Guardians Model Developer Model/ Product Owner ● What does this model do? ● How does it work? ● What risks does it have? ● How best to monitor it? ● Which product/quality risks are worth taking? ● Which mitigations are worth the extra time and effort? ● Sign-off on serious risks ● Is it ok to use sensitive/PII data for this case? ● Where should we be improving gov/ML as an org? Governance Board
  • 22. © 2021 Thoughtworks | Confidential Model Validator is Part of This Too Model Developer Model Validator ● What does this model do? ● How does it work? ● What risks does it have? ● How best to monitor it? ● Was the development process robust? ● Has the developer overlooked anything in best practice or risks?
  • 23. © 2021 Thoughtworks | Confidential Documentation in ML Governance 24 24 2 4 2 4 2 4 2 4
  • 24. © 2021 Thoughtworks | Confidential Checklists ● Google model cards ● ‘Datasheets for datasets’ ● Meta/Facebook reproducibility checklists ● ‘The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction’ ● ‘Towards Yet Another Checklist for New Datasets’ ● ML Cards for D/MLOps Governance by Ian Hellstrom Dog Breed Classifier image from Google 2 5 2 5
  • 25. © 2021 Thoughtworks | Confidential Varieties and Purpose of Model Cards ● “Under what conditions does the model perform best and most consistently? Does it have blind spots? If so, where?” ○ Dog Breed Classifier: “What kind of photos tend to yield the most accurate results? Can it handle partially obscured dogs? What about dogs that are extremely close, extremely far away, or seen from unusual angles?” ○ Language Translator: “guidance around jargon, slang and dialects, or measure its tolerance for differences in spelling” ● Card should give an overview of internals and limitations of model with a view to how it will be used. ● Is an open documentation format. Not a process. ● Focused more on model than data (will return to this) 27 https://modelcards.withgoogle.com/about
  • 26. © 2021 Thoughtworks | Confidential Model card Model purpose, design, data description, risks Model Developer Model Validator Review initiated when model ready for production Model validation report Establish clarity, reproducibility, best practices Model owner approval Sign-off on clarity, monitoring plan, risks Model Owner Changes from review
  • 27. © 2021 Thoughtworks | Confidential Deep-dive Questions on Process ● Should model validator be from a different team from model developer? ● Where do Model Cards live? ● Who updates/maintains the cards? ● Does the validator need to fully reproduce the model and results? ● How much responsibility is on the Model Developer to explain about the model vs on the Model Owner to ask Qs? ● Who will be responsible for monitoring in live? ● How should governance board be formed? 29
  • 28. © 2021 Thoughtworks | Confidential 30 Model cards will tend to follow lead of reference examples Detailed examples are more burden to produce and read Tricky Questions: How Much Detail? ● Reference examples will have a big impact on what documentation really gets produced ● Examples show developers how much and what kind of detail is expected Photo by Magda Ehlers on Pexels
  • 29. © 2021 Thoughtworks | Confidential Test the Process ● Can’t prove workability in a vacuum ● Pick some cases and work it through together 31 Photo from WikiImages on Pixabay
  • 30. © 2021 Thoughtworks | Confidential Tradeoffs and Opportunities 32 ● Not enough checks and you fail to surface risks ● Irrelevant/inappropriate checks slow you down ● Process can encourage best practice - this is an opportunity
  • 31. © 2021 Thoughtworks | Confidential Hidden Risks 33 Known Risks Overlooked Risks Financial Risk Legal Risk Reputation Risk Quality Risk Ethical Risk Delivery Risk Regulatory Risk
  • 32. © 2021 Thoughtworks | Confidential Credit Assessment 34 Known Risks Overlooked Risks Financial Risks First Order Gender Bias Indirect Gender Bias? (e.g. from occupation) Actually not overlooked Similar situation seemed to have happened with AppleCard in 2019 but investigation found no bias www.bbc.co.uk/news/business-50432634 www.theverge.com/2021/3/23/22347127/goldman- sachs-apple-card-no-gender-discrimination Reputation Risk from Customer Confusion
  • 33. © 2021 Thoughtworks | Confidential Summary Photo by Romain Dancre from Unsplash ● ML Governance is multi-faceted and can be confusing ● Simple template Process based around model cards and defined roles ● Model Owner role key to Risk Management ● Needs to be shaped to your team/s

Editor's Notes

  1. Intro speakers
  2. Ryan: Lots of discussions of ML Governance never really get into details and can be confusing. We’re going to break through the confusion and tell you how you can really do something about ML Governance at the level of a Data Science team. This is going to involve talking about Documentation. But it’s good documentation - the kind of documentation that can really help you out if you use it wisely. So let’s get started.
  3. First up, why is this topic so confusing? Why do so many people feel like they don’t even know what ML Governance is?
  4. The fact is that many teams right now have very little governance. This is understandable as technologists have a delivery focus which means teams are biased towards building solutions now and worrying about risks later. To get to a better place on governance the burden can’t all be on tech. It has to be a collaborative process. This leaves techies a bit nervous because nobody really knows what is needed and there’s a fear of some bureacrat coming in and telling the team what they can and can’t do. The nervousness is amplified by confusion about what is really needed.
  5. It’s understandable that we get confused about ML Governance. There’s lots of different aspects and it’s easy to get lost in the conversations. There’s hot topics like Responsible AI that gets alot of attention. And this is an MLOps meetup so we obviously love MLOps. But each of these is just a part of ML Governance.
  6. We could roughly cluster different aspects of ML Governance under Ethics and Principles, Tech Practices and MLOps and Management and Frameworks. This helps us get a better picture of ML Governance but it only takes us so far.
  7. We will get into the details very shortly. First just to re-emphasise that this is not a Responsible AI talk. Ethics and Responsible AI are important. But they’re only part of the conversation. We want to talk about the boring side of ML Governance.
  8. Meissane: We also need to think about the relative weight or perceived importance of the topics under ML Governance. When people think about ML Governance they tend to think about something like this with Ethics a big part of it and the documentation and sign-offs stuff falling under bureaucracy that they’d rather not get involved with.
  9. We want to shift this thinking and instead think about documentation and peer review as parts of best practice that should be reinforced by good governance. And sign-offs shouldn’t be about having to beg some bureacrat to tick a box next to your model. It should be about positioning risk trade-off decisions with the right people.
  10. It’s also worth understanding how ML Governance relates to other types of Governance and especially Data Governance. This is important because there’s important overlaps between ML and Data Governance. The main areas of overlap are in documenting datasets. Where the dataset comes from, what it means, how it gets updated and its known limitations. This is needed for both Data and ML Governance and ideally it would fall under Data Governance so that Data Scientists can leverage it. It is not only needed for producing ML models but also for doing: Data Analysis Analytics dashboards generally asking questions of data. Data Labelling likewise has a lot of value for Data Analysis and non-ML applications. Data Lineage is about tracking changes to the data over time: This can be important for ML training pipelines and reproducibility. Once again there are also data analytics use cases where data lineage can be important. Sometimes this can be a requirement of auditors.
  11. Meissane: So let’s talk about how to come up with a ML governance process. Here are the kind of questions we want to talk about to set up ML Governance. READ QUESTIONS
  12. Processes can easily be over engineered. Manual checks slow down the process and can be harder to follow and keep track which can affect the team morale. Simply referring to a it as ‘Best Practice’ isn’t enough to make us trust it. It might just be bureaucracy rebranded. For the team to feel comfortable with the process, it has to be relevant and appropriate.
  13. This is a bit of a side note but the term ‘bureacracy’ literally means ‘rule by desks’. When you are constrained by bureaucracy it does feel like you are at the mercy of something unthinking. However, it’s important to note that what this feeling arises when rules don’t work well for your case. Like you can’t get done what you want to get done because somebody has made a rule without thinking about what you want to do. Processes and rules are not the problem. The problem is when the rules and processes don’t fit with what needs to be done.
  14. Ryan: So now we know what ML Governance is about. How do we make it happen?
  15. Ryan: This slide is going to look super simple. It is not the whole answer to ML Governance. But it is a starting point that we’ll use in this presentation. You can think of this slide as a flexible template for a process that can be adapted for different organisations and teams. The flow hinges around certain key documents that you can see here named as the Model Card, the Model Validation Report and Model owner approval. But the flow is not just about producing good documentation. It’s also about facilitating informed decision-making and positioning decisions with the most appropriate people.
  16. Ryan: The model developer produces a model card which documents the purpose of a model, its design, what data it uses, what risks they can see around it and advice on how the model should and should not be used. This is checked by the model validator.
  17. Ryan: The model validator also checks that the model is reproducible and that the code and documentation is clear. There might be some back and forth here. Think of it like a pull request review process. There might be a separate model validation report from this or it might be a section that gets added to the model card or it might even be a link to a pull request with structured comments.
  18. Ryan: The next step is the model owner. The model owner is also looking for clarity about how the model works and how it should be used and its limitations. But the model owner probably won’t be technical so this needs to be explained at a different level. This might result in some more back and forth on the documentation. Most importantly the model owner needs to know about any risks and trade-offs associated with the model as they will be responsible at a business level for the model within the business product or process in which it is to be used.
  19. Within this process there might also be an escalation route to an oversight board. Not every org will have an oversight board but if you do then they would become involved in cases where a model is identified as high risk, triggering a deeper review with more parties. Factors that could trigger an oversight review: Use of sensitive data or attributes (PII, protected attributes such as gender etc.) Models making decisions with a potential negative impact on an individual or entity Issues arising from ISRM security review Serious concerns about quality of the model and monitoring (e.g. live data not well known and unable to perform desired testing and monitoring)
  20. An oversight board might also lead a periodic review process. Perhaps you do a review every year or at some other frequency. This might be for an external auditor though it’s better to think of this process for non-regulated industries first. We can think of regulated industries separately. You might also do an internal audit to check that documentation is all up to a similar standard. You might also use the information to look for patterns and opportunities within the org. There’s a lot to understand here so let’s try to make it more concrete. We’ll get into the details of model cards and understand what a model validator or model owner would be looking for. But first let’s understand the roles in more detail.
  21. Meissane: In ML governance we want to place right kind of questions and decisions to sit with the appropriate roles. Too often what we’re seeing is that Data Scientists are assumed to have already assessed risks and dealt with them, so that product management and other business managers don’t have to think about them. This is not appropriate as Data Scientists are not empowered to make decisions about what risks are worth taking and are not able to simply make risks go away. Data Scientists are in a position to develop models, to explain what they do and make the risks and trade-offs of models clear. Data Scientists are also in a position to advise on what monitoring will be appropriate for running models in production.
  22. There may be more than one model validator with different intentions. Assumption is that a model validator will be a fellow data scientist. This is necessary in order to check the robustness of the development process. But there may also be some validation from an ML Engineer or Support Engineer or similar to ensure that they know all the background to monitor the model in live. Ideally the Model Developer and an ML Engineer will work together to put together a Deployment and Monitoring plan. That also needs to be part of the extended Model Card as the model owner needs to know about it. They need to know about any deployment risks and what kind of monitoring is achievable as it is part of the overall risk profile.
  23. There has been a lot of discussion about how best to document ML models. We’ve listed the most notable approaches to ML Governance documentation here. Model cards are a checklist that google is trying to popularise. They’re focused on overviews and design trade-offs of models. Fairness and Limitation tradeoffs https://drive.google.com/file/d/1QvwWNfFoweGVjsXF3DXzcrCnz-mx-Lha/preview Datasheets are a kind of checklist for datasets, not for models. So they’re complimentary. So model cards and datasheets both started from a position of reducing misuse, mistakes and bias. Reproducibility checklists started from a different angle. The motivation for reproducibility checklists was more about ensuring the robustness of the results being reported for ML models, especially in research papers. Another angle for checklists is production readiness. ML Test Scores for Production Readiness address deployment and infrastructure and also elements aimed at the ML model such as ensuring that the code is reviewed and in git and that hyperparameters are tuned and that the model chosen is as simple as it can be without loss of performance. With so many different angles to ML documentation, it’s clear that we need to cover a mixture of different concerns in document models. We might choose to do this in one checklist with a range of different sections or we could use a variety of checklists. The ML Cards for D/MLOps Governance link at the bottom of the slide here suggests using separate cards or checklists for different concerns and offers lots of suggestions for questions to include in the checklists.
  24. We should now get into more detail on at least one of these checklist approaches. This will help us picture the idea more clearly. Google’s model cards probably the easiest to explain as google has done a lot of work to try to popularise the idea. Model cards were proposed by google in a research paper. Added toolkit and google vision face and object detection examples. READ SLIDE
  25. Ryan: So that’s model cards. That’s the central piece in the process that we talked about before. Actually you could simply extend the model card concept and treat the three documents from this slide as one big model card. Maybe the model validator just provides feedback that updates the model card. And model owner approval could be recorded on the model card.
  26. Ryan: This can sound easy when you talk about it in a presentation. The difficult thing is making it work for a particular team. There are lots of difficult questions you hit when you try to introduce a process like this in a real team. READ QUESTIONS Answering these questions tends to depend a lot on the context of the team and organisation. You have to talk to people and figure out what everyone will be comfortable with.
  27. Ryan: Making the process work for a team isn’t just about talking to people either. There’s also documentation that shows people what the process is about and that’s super important. There should be reference examples for the documentation - example model cards that show models that make sense for the team. Reference examples will have a big impact on what documentation really gets produced because they show developers what kind of detail is expected.
  28. Talking to people and producing reference documentation is also not enough. You should test out a new process and get feedback and adjust it. I would say adjust it until it is proven but really you can keep adjusting it forever as it can be a living process.
  29. This is just a small piece of general advice about shaping a governance process. You somehow have to decide about how much documentation detail is too much and how many sign-offs are too many. There is no general right answer. Firstly you have to look at your risks and get a sense for what realistically might go wrong and what the implications could be. Then you should work with your team and shape the process together. This ensures everyone feels included and buys in to the process
  30. We’re coming to the end of the presentation now so we want to leave with you with a key thought. ML Governance is about lots of things like best practice and communication and so on but for many organisations the really big thing they need to tackle is risk management. Here’s a useful picture to keep in mind for risk management. We have to be wary of doing our risk assessments in a superficial way. It’s tempting to focus on specific risks or specific types of risk and then not really look for others. The format of the documentation should help practitioners go through risks in a methodical and balanced way. Otherwise you get bitten.
  31. Let’s make this concrete by looking at a famous case of getting bitten by risks in using ML. There are lots of these but one that illustrates the point well is when AppleCard launched in 2019 and its credit assessments were accused of gender bias. Lots of high profile people were critical including Steve Wozniak and David Heinemeier Hanson. The credit assessment service was operated by Goldman Sachs and they were quick to say that they were not using gender as an attribute. So then there was speculation that maybe gender was entering indirectly through other attributes. This could happen as some occupations have big gender bias. In fact an investigation from New York State Department for Financial Services found no gender bias. The problem was actually that people didn’t understand the logic. There were complaints that female spouses were getting lower limits and this was questioned on the basis of shared assets and income. But credit histories are not shared and that was actually part of the algorithm. Where the New York State Department for Financial Services did criticise Goldman Sachs though was on communication and customer response. Goldman had no way to respond to all these complaints and wasn’t able at the time to explain why the credit scores were coming out the way that they were. You can imagine this might have been overlooked or just not prioritised due to the rush to get the AppleCard service live.
  32. Frist two bullets Ryan. Last two Meissane.