1.
AI Governance — Drive Compliance, Efficiency,
and Outcomes from Your AI Lifecycle
—
Scott Buckles
North America Business Unit Executive
Information Architecture

2.
When you interact with IBM, this serves as your authorization to Dataversity or its vendor to provide your contact information to IBM in order for IBM to follow up on your interaction. IBM’s use of your
contact information is governed by the IBM privacy policy. IBM Watson / 12.15.20 / © 2020 IBM Corporation

3.
Use Cases Driving a Data Governance Strategy
Governance for InsightsGovernance for Compliance
Discover, classify and
manage information in
ways that meet the
obligations enforced
by both regulatory and
corporate mandates
Provide safe access to
trusted, high quality
data while facilitating
effective collaboration
among team members
to become a data
driven organization
When you interact with IBM, this serves as your authorization to Dataversity or its vendor to provide your contact information to IBM in order for IBM to follow up on your interaction. IBM’s use of your
contact information is governed by the IBM privacy policy. IBM Watson / 12.15.20 / © 2020 IBM Corporation

4.
The Data Governance Journey
Siloed
Efforts
Reactive
Proactive
Business
Ready
Data Quality is not a focus at point of creation. Continuous
Improvement in your Information Supply Chain does not
exist.
Departmental
Data
Improvements
Enterprise level information governance funded and
sustained as a part of “How You Do Business.”
Your data is Business Ready for all consumers now and as
tomorrow’s requirements emerge.
Limited metrics
not directly tied
to governance
Range of
disconnected,
discipline-
specific tools
Data Stewards,
Policies & Rules
Business Focused
Defined, formally
reviewed
governance metrics
Enterprise-based
integration &
governance tools
with LOB access
When you interact with IBM, this serves as your authorization to Dataversity or its vendor to provide your contact information to IBM in order for IBM to follow up on your interaction. IBM’s use of your
contact information is governed by the IBM privacy policy. IBM Watson / 12.15.20 / © 2020 IBM Corporation

5.
When you interact with IBM, this serves as your authorization to Dataversity or its vendor to provide your contact information to IBM in order for IBM to follow up on your interaction. IBM’s use of your
contact information is governed by the IBM privacy policy. IBM Watson / 12.15.20 / © 2020 IBM Corporation

6.
$2.9 trillion
6.2 billion hours
When you interact with IBM, this serves as your authorization to Dataversity or its vendor to provide your contact information to IBM in order for IBM to follow up on your interaction. IBM’s use of your
contact information is governed by the IBM privacy policy. IBM Watson / 12.15.20 / © 2020 IBM Corporation

7.
4
The AI Ladder
A prescriptive approach to accelerating
the journey to AI
Infuse
Operationalize AI throughout
the business
Analyze
Build and scale AI
with trust
and transparency
Collect
Make data simple
and accessible
Organize
Create a business-ready
analytics foundation
Modernize
Make your data ready
for an AI and hybrid
cloud world
When you interact with IBM, this serves as your authorization to Dataversity or its vendor to provide your contact information to IBM in order for IBM to follow up on your interaction. IBM’s use of
your contact information is governed by the IBM privacy policy. IBM Watson / 12.15.20 / © 2020 IBM Corporation

8.
Use Cases Driving a Data Governance Strategy
Governance for AI
AI Governance is the program,
best practices, and controls to
ensure AI capabilities perform
appropriately,
ethically,
morally, &
legally
to mitigate market and social
risk while benefiting business
objectives.
When you interact with IBM, this serves as your authorization to Dataversity or its vendor to provide your contact information to IBM in order for IBM to follow up on your interaction. IBM’s use of your
contact information is governed by the IBM privacy policy. IBM Watson / 12.15.20 / © 2020 IBM Corporation

9.
9
What is AI governance?
AI strategy
Strategic
imperatives
Use cases
Competencies
Technologies
Explainable AI
Fairness
Traceability
Understandability
Auditability
AI governance
Model management
Digital ethics
Compliance
Monitoring
Quality
Source: Gartner
When you interact with IBM, this serves as your authorization to Dataversity or its vendor to provide your contact information to IBM in order for IBM to follow up on your interaction. IBM’s use of your
contact information is governed by the IBM privacy policy. IBM Watson / 12.15.20 / © 2020 IBM Corporation

10.
10
Why AI governance?
By 2022…
65%of enterprises will task
CIOs to transform & modernize
governance policies to confront
risks by AI, ML, Data Privacy &
Ethics
Compliance
Trust
Efficiency
Align AI strategy with regulations & legal
requirements
Maintain Cust Sat & Brand Value by ensuring
trustworthy & transparent AI
Improve speed to market & reduce costs by
standardizing/optimizing AI development &
deployment
When you interact with IBM, this serves as your authorization to Dataversity or its vendor to provide your contact information to IBM in order for IBM to follow up on your interaction. IBM’s use of your
contact information is governed by the IBM privacy policy. IBM Watson / 12.15.20 / © 2020 IBM Corporation Source: IDC FutureScape: Worldwide CIO Agenda 2019 Predictions, idc.com, October 2018

11.
Enterprises
must consider
regulatory
compliance as
they scale AI
throughout
their business
11
CCPA
GDPR
When you interact with IBM, this serves as your authorization to Dataversity or its vendor to provide your contact information to IBM in order for IBM to follow up on your interaction. IBM’s use of your
contact information is governed by the IBM privacy policy. IBM Watson / 12.15.20 / © 2020 IBM Corporation

12.
Enterprises must consider data privacy
consisting of many pieces
Proposal
for Algorithmic
Accountability
Act
• Expands consumer
privacy rights to more
closely align with the
EU’s GDPR
• Regulates AI systems
across industries in the
United States to reduce
bias and discrimination
• Requires all public
agencies to conduct an
impact analysis for AI
models
• Requires model risk
management for all
models in financial
services
Source: If applicable, describe source origin
12
Canadian
National AI
Strategy
CRPA SR 11-7
When you interact with IBM, this serves as your authorization to Dataversity or its vendor to provide your contact information to IBM in order for IBM to follow up on your interaction. IBM’s use of your
contact information is governed by the IBM privacy policy. IBM Watson / 12.15.20 / © 2020 IBM Corporation

13.
Confronting and ending
support for biased facial
recognition
Gender-biased
Apple credit card approval
process
Enterprises must consider brand
as they scale AI throughout their
business
13
Gender-biased recruitment
software
Unethical usage
of personal data
When you interact with IBM, this serves as your authorization to Dataversity or its vendor to provide your contact information to IBM in order for IBM to follow up on your interaction. IBM’s use of your
contact information is governed by the IBM privacy policy. IBM Watson / 12.15.20 / © 2020 IBM Corporation

14.
Use your
model
AI
Ready Data
14
Trust your
model
Know your
model
Use your data
Trust your
data
Know your
data
Business
Ready Data
When you interact with IBM, this serves as your authorization to Dataversity or its vendor to provide your contact information to IBM in order for IBM to follow up on your interaction. IBM’s use of your
contact information is governed by the IBM privacy policy. IBM Watson / 12.15.20 / © 2020 IBM Corporation

15.
Roles in the AI governance lifecycle
15
App
implementation
Approval
Validation
Black box testing
Model development
White box testing
Origination
Business user
ML engineer
SW developer
Business
approver
Risk reviewer
Model validator
Data scientist
Business owner
Governed catalogue
Model facts
(metrics, intent, etc.)
Lineage
Governed features
Business terms
Policies
Monitor/Check
Business KPIs analysis
Performance
Compliance
Change in external
assumptions
Chief Risk
Office
Data and
model
governance
Each persona
contributes model facts
and can receive
aggregate facts (fact
sheets) from the
repository
Each persona uses
metrics and KPIs
to validate, approve,
or improve a model
(or an AI powered
app) before and in
production
– Data scientist and CDO interact
on data sources through
metadata repository
– Policy enforcement occurs at
many places (build time, validation,
production monitoring)
– CRO defines the tests and
criteria for validating models
– Validators implement and execute
tests based on CRO guidance
– Risk professionals review outcome of
risk management tests
– Business approver uses CRO guidelines
to implement first
line of defense
– Business users and auditors
will use framework to fulfill
audit requirements
Chief Data
Office
Data and
model
governance
Deployment
Production
monitoring
Continuesimprovement
When you interact with IBM, this serves as your authorization to Dataversity or its vendor to provide your contact information to IBM in order for IBM to follow up on your interaction. IBM’s use of your
contact information is governed by the IBM privacy policy. IBM Watson / 12.15.20 / © 2020 IBM Corporation

16.
16When you interact with IBM, this serves as your authorization to Dataversity or its vendor to provide your contact information to IBM in order for IBM to follow up on your interaction. IBM’s use of your
contact information is governed by the IBM privacy policy. IBM Watson / 12.15.20 / © 2020 IBM Corporation

17.
Enterprises must consider
regulatory compliance as
they scale AI throughout
their business
Canada
2017—National AI Strategy
launched
2020—All public agencies
must do an impact analysis
for AI models
European Union
2019—Guidelines
for AI development
Partnerships on AI
Partnership between tech
companies to study best
practices and impact of AI
AI Now Institute
NYU research center focused
on social implications of AI
USA
SR 11–7 requires
model risk
management for all models
in financial services
2019—Proposal
for Algorithmic
Accountability Act
Mexico
2018—General principles
for AI development in
the government
Finland
2018—Report on
policy recommendations
for reskilling workers
17When you interact with IBM, this serves as your authorization to Dataversity or its vendor to provide your contact information to IBM in order for IBM to follow up on your interaction. IBM’s use of your
contact information is governed by the IBM privacy policy. IBM Watson / 12.15.20 / © 2020 IBM Corporation

18.
Documentation of model
inputs and behavior requires
manual work; amplified by
changes in data and model
versions
Challenges when
implementing AI systems for
production scenarios
18
Companies have multiple
tools and platforms that do
not easily share metadata
about models
Current practices and tools
not optimized for
AI (for example, bias as
a factor in data quality
analysis)
When you interact with IBM, this serves as your authorization to Dataversity or its vendor to provide your contact information to IBM in order for IBM to follow up on your interaction. IBM’s use of your
contact information is governed by the IBM privacy policy. IBM Watson / 12.15.20 / © 2020 IBM Corporation