Explain your algorithmic decisions for gdpr

Pierre FEILLET @ IBM
Laurent Grateau @ IBM
Explain your algorithmic decisions for
GDPR

© Copyright IBM Corporation, 2018
Who we are
• Members of the IBM Decision Lab in Paris
• Architects and Developers in Operational Decision Management
• Delivering decision automation products for ww financial
companies and other industries
• Making AI symbolic systems for eligibility, pricing, fraud detection
& more
grateau@fr.ibm.comfeillet@fr.ibm.com
2

Disclaimer
• Notice: Clients are responsible for ensuring their own compliance with various laws and regulations, including the European
Union General Data Protection Regulation. Clients are solely responsible for obtaining advice of competent legal counsel as to
the identification and interpretation of any relevant laws and regulations that may affect the clients’ business and any actions
the clients may need to take to comply with such laws and regulations. The products, services, and other capabilities
described herein are not suitable for all client situations and may have restricted availability. IBM does not provide legal,
accounting or auditing advice or represent or warrant that its services or products will ensure that clients are in compliance
with any law or regulation.
• IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.
• Information regarding potential future products is intended to outline our general product direction and it should not be relied on in
making a purchasing decision.
• The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material,
code or functionality. Information about potential future products may not be incorporated into any contract.
• The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual
throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the
amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed.
Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
3
None of the statements contained herein constitutes legal advice – it is process advice only.

Agenda
• A GDPR Overview
• Algorithm transparency is coming
• Case studies
• Best practices
• eXplainable Artificial Intelligence beyond GDPR
4

Agenda
• A GDPR Overview
• Algorithm transparency is coming
• Case studies
• Best practices
• eXplainable Artificial Intelligence beyond GDPR
5

Global Data Privacy Regulation
Compliance Data
Protection
Personal
Data
Simply….
6
6

May 25, 2018
The GDPR – What is it?
Up to 4%or €20M
Potential penalty for non-compliance
Per Incident!
Global
Impact
5 Key General Data Protection Regulation Obligations
Rights of EU
Data Subjects
Security of
Personal Data
Consent Accountability of
Compliance
Data Protection by
Design and by Default
The EU General Data Protection Regulation (GDPR) comes into effect on 25 May
2018 and presents an important change in data privacy. The legislation aims to
increase control to EU data subjects over their Personal Data and simplify the
regulatory environment for international business..
7 7

Personal Data?
Sensitive Personal Data:
data consisting of racial or ethnic origin, political opinions, religious or philosophical
beliefs, or trade union membership, genetic data, biometric data, data concerning
health or data concerning a natural person's sex life or sexual orientation. The
commission or alleged commission by them of any offence; or any proceedings for
any offence committed or alleged to have been committed by them, the disposal of
such proceedings or the sentence of any court in such proceedings.
Personal Data:
an identifier such as a name, an identification number, location data, online
identifier or to one or more factors specific to the physical, physiological, genetic,
mental, economic, cultural or social identity of that person.
“any information relating to an identified or identifiable natural person” (Art. 2(a))
Direct identifier – E.g. name, passport number, phone number
Indirect identifier – E.g. The CEO of company X in April 2018
8 8

What are the Rights and Impacts of the GDPR?
• The right to be
informed
• The right of access
• The right to
rectification
• The right to erasure
• The right to
restrict processing
• The right to data
portability
• The right to object
• Rights in relation to
automated decision
making and profiling.
Enhanced, harmonized and
extended rights for individuals
located in the EU • Controllers and Processors liable for non-
compliance
• Controllers legally bound to validate Processor’s
compliance
• Data Protection Officer obligatory
• Stringent data security and breach management
• Processors jointly responsible for cross-border
data transfers
• All relationships, transactions, and data flows
must be transparent, documented, and
auditable
Organizational Impact
9

The GDPR has global ramifications
• Data Privacy relates to the rights of individuals to control
how their public and non-public Personal Data is collected
and used
• Data Security relates to the protection of Personal Data
against loss, unauthorized access, destruction, use,
modification, disclosure, etc.
• GDPR & EU Term is “Data Protection” = Security,
Encryption, Access Controls.
• Data Governance defines who can take what actions with
what Personal Data, when, under which circumstances, and
using what methods
The GDPR applies to Controller(s), and Processor(s) located in the EEA. It also applies to
Controllers and Processors located outside the EEA if they are processing Personal Data in
relation to the offering of goods and services to individuals located in the EEA, or for purposes
of monitoring their behaviour.
Key definitions:
Data
Privacy
Data
Security
Data
Governance
10

Key terms for understanding the GDPR
An identified
or identifiable
living natural
person
Controller
Determines the
purpose and
means of
processing of
Personal Data
Processor
Processes
Personal Data
on behalf of
the Controller
Processing
Any operation performed on Personal Data
(includes storage, access), anywhere in the world
Personal Data
Any information relating
to a Data Subject
Data Subject
11

Processor vs. Controller
As a Processor example: Online Banking Service
• The bank (IBM’s Client) collects data from its customers via an online banking app and processes this data to provide
services to its customer: Bank is the Controller
• Data is stored, managed and processed using the IBM Cloud and/or 3rd party vendors: IBM is the Processor
Data Subject
ContractOnline Banking
App (Bank)
IBM Cloud Sub Processor
Agreements
3rd Party
Vendor
Bank (IBM Client)
PI/SPI Data
Contract
! ! !
• Controller: The organization determining the purposes and means of the processing of Personal Data
• Processor: An organization that processes Personal Data on behalf of a Controller
• Data Subject: The individual whose data is being processed
12

Processor vs. Controller
As a Controller example: An internal HR employee database system
• Company collects data directly from its employees and processes it for employment related purposes: this company is the
Controller
Data Subject
Notice/
Agreement
Internal HR System Company’s IT Processor Agreement 3rd Party
Vendor
PI/SPI Data
! !!
• Controller: The organization determining the purposes and means of the processing of Personal Data
• Processor: An organization that processes Personal Data on behalf of a Controller
• Data Subject: The individual whose data is being processed
13

GDPR 2018 – Logic Transparency Beyond Data Protection
• Article 15 of the regulation specifically
mentions the right for individuals to obtain
meaningful information about the logic
involved in certain automatic decisions
concerning them, as well as the
significance and the envisaged
consequences of such processing for that
individual.
• Article 22 establishes the right of
individuals to not be subject to an
automated decision-making process where
those decisions have “a legal effect” or “a
similar, significant effect” on the individual.
• It is likely to profoundly impact AI-driven
business models. Automated decision-
making is often defined as the ability to
make decisions by technological means
without human involvement.
Control and clarity of the
decision making process is of
great value
14

GDPR Article 9 – For Non Discriminatory decision
Paragraph 1 shall not apply if several cases, first being when there is an Explicit
consent except where Union or Member State law provide that the prohibition
referred to in paragraph 1 may not be lifted by the data subject.
15

GDPR 2018 – Viewing inside the black box
• These regulations have profound implications
for data-driven organizations.
• For those organizations that provide credit
(e.g. mortgages, personal loans, credit cards)
this is not totally new; there are already
regulations in place to prevent discrimination
and enforce clarity in data usage.
• Making an automated decision needs to be
within a structured framework so that HOW
the decision is made can be understood.
• The decision logic needs to be sufficiently
interpretable to allow an explanation of WHY a
decision was made about an individual.
• The issue is more complex where “black box”
systems are deployed typically machine
learning algorithms.
• Another issue with black box systems is that
they may inadvertently become discriminatory.
For example, if a group of postcodes is used
as a factor in an automated decision-making
algorithm, this may divide groups along ethnic
lines, whether intentionally or not.
• While algorithm is neutral in its theory human
selected dataset, or way to cleanse it and
apply algorithm may bring a bias in the
decision making.
• A transparent “white box” approach would
include a review process to ensure that this
type of issue does not occur.
• Symbolic AI approaches like rules propose a
white box decision logic.
16

GDPR 2018 – A Decision Automation Checklist
What does it mean
Inventory Know your automated decision making systems, already in production or in dev.
Traceability Data Which data has been used to make an automated decision for each individual.
Algorithm Decision models are open and interpretable.
Same requirements apply whatever you leverage an analytical model, a prescriptive
model, or any custom implementation
DevOps The process of deploying models into production is crystal clear, so that which model
version is in use at any one time is well understood – this implies clear version
control on analytical/prescriptive models and a careful testing process before
deployment.
Decisions A history of decisions and how they have been made is available through an audit
trail. Ease to retrieve a decision made for an individual with its data and logic.
Consistency Decisions Decisions are consistent across channels (for example web, email and SMS all
provide the same offer).
17

GDPR 2018 – Do’s and Don’ts
You receive an explanation/information request for a given decision
Do Don’t
Inventory You know if the decision for the individual was
automated or human
What decision? I have no clue if this decision was
taken by a machine or manually by a human.
Traceability Data From a decision store you retrieved the decisions
and the copy of data inputs and outputs
Well we should have a trace in a server log
somewhere in the production cluster.
Algorithm Prefer open and readable Decision models.
Same requirements apply whatever you leverage a
analytical model, a prescriptive model, or any
custom implementation
Well my neural network does a great job but is hard to
explain why it rejected the applicant’s request
DevOps The process of deploying models into production is
clear, so that which model is in use at any one time
is well understood – this implies clear version
control on analytical models and a careful testing
process before deployment.
I remember that my Data Scientist made several
versions for the analytical model but cannot be sure of
the version used for this decision.
Decisions A history of decisions and how they have been
made is available to give an audit trail. I easily
access to a decision made for an individual.
I forget past decisions to better focus on the next
onesJ
Consistency Decisions Decisions are consistent across channels (for
example web, email and SMS all provide the same
offer).
Provide divergent decisions depending on your
channel.
Mean a specific compliance for each channel
1

GDPR 2018 – Decision Automation Tips
• Determinism in the decision
automation
• All data explicitly declared in
an integration contract
• Decision Replay can be done
with data and decision model
as used for the original
decision
• Consider OpenTracing to
capture data and algo usage
19

Live Demo
20
An example of Loan approval automation with Data and algorithm traceability
Loan
Validation
Decision
Service
Data
Warehouse
Algorithm
Source code
Version
Repository
REST
Response
Transaction ID
MobileWeb
Web Billing
Backe
ndQuoting
Decision Trace with
Transaction ID
REST
Request
Transaction ID
Service version
Algorithm version
Input/Output parameters
Decision Logic details

Decision Transparency with Business Rules
• Try Decision Composer on line:
• https://decision-composer.mybluemix.net/
21
• Author and execute rules
in Docker:
• https://hub.docker.com/r/ib
mcom/odm/
• Kubernetes and Helm charts
on Github
• https://github.com/ODMDev

A Decision Explanation Thesis
• A thesis conducted by IBM,
Ecole Centrale and UPMC
• Based on Halpern theory
• Support decision explanation
by building a causal model
• Causal links emitted step by
step during the reasoning
• Enable kind of Fishikawa
graph applied to decisions
Applicant ‘s
age is
under 18
Loan
application
is rejected
Applicant ‘s
debt to income
ratio is greater
than 0.3
22
cause
cause
https://tel.archives-ouvertes.fr/tel-01726252/document

Explain Artificial Intelligence - XAI
• Machine Learning and Explanation
• The black box problem, from Decision Tree to Deep Neural
Network
• Research
• XAI at DARPA; See IJCAI, ICML proceedings
• Lime - producing model-agnostic explanations, explaining
the results of any ML system by looking only at its inputs
and outputs.
• Symbolic AI based on Human knowledge like Rule Engines
23

Take Away
• Coming 25th of May with global impact on Data
Privacy and decision automation
• GDPR will change durably organizations –> Privacy
by Design
• Compliance will be key to keep and increase
confidence with clients and partners
• Ethics outside of regulatory compliance remain key
for your business
• Transparency and Explanation of AI will be
determining acceptance factors for society
Beyond
GDPR and
toward XAI
24
GDPR

Links
• GDPR
• https://ec.europa.eu/info/law/law-topic/data-protection_en
• ibm.com/gdpr
• #gdpr
• #ibmgdpr
• XAI Research
• DARPA XAI
• https://arxiv.org/pdf/1708.08296.pdf
• http://home.earthlink.net/~dwaha/research/meetings/ijcai17-
xai/
• https://arxiv.org/abs/1602.04938
25

Merci
Thank you
GDPR
Beyond
GDPR and
toward XAI
26

Explain your algorithmic decisions for gdpr

More Related Content

What's hot

Similar to Explain your algorithmic decisions for gdpr

Recently uploaded

Explain your algorithmic decisions for gdpr