Should we regulate Artificial Intelligence? What are the challenges to face bias in data and algorithms? What is trustworthy AI? AI HLEG (European Commission) and AIGO (OECD) feedback experiences and recommendations. Example in precision medicine: AI/ML for medical devices

1. Faut-il réguler l’intelligence
artificielle ?
www.mediantechnologies.com
Chief Scienceand InnovationOfficer– MedianTechnologies
Nozha BOUJEMAA
IBM Cloud Academy

2. Data & Algorithms
• Data are everywhere in personal and professional environment
• Algorithms making sense and value from these data are pervasive in more and more
digital services.
• Algorithmic-based decisions are embedded from the processing of personal data to
sensitive data in critical industrial systems such : health-care, personalized medicine,
autonomous cars, precise agriculture, conversational agents or public services
• Big Data Technologies, agnostic to applications, are enablers for AI capabilities in real-life
services
« 2 sides of the same coin »
www.mediantechnologies.com- Nozha Boujemaa2

3. Data & Algorithms
• Rising benefits from Big Data and AI technologies have wide impact on our economy and
social organization ;
• Transparency and trust of such Algorithmic Systems(data & algorithms) becoming
competitivenessfactors for Data-driven economy ;
• Data analytics is changing from description of past to predictive and prescriptive analytics
for decision support ;
• Importance of remedying the information asymmetry between the producer of the
digital service and its consumer, be it citizen or professional – B2C or B2B => civil rights,
competition, sovereignty.
« 2 sides of the same coin »
www.mediantechnologies.com- Nozha Boujemaa3

4. Algorithmic systems in every day life
• Some dominant platforms on the market play a role of "prescriber”
by directing a large share of user traffic:
• Ranking mechanisms (search engine),
• Recommendation mechanisms and contentselection
Product or service recommendation: is it most appropriate for the consumer
(personalization) or the most appropriate to the seller (given the stock)?
• Opacity of the use made of sensitive data and how they are processed,
• What about the consent? Is it always respected?
• Credit scoring, recruitment, how fair is this?
• Predictivejustice?
• “Free” Business models ?
⇒New discrimination between those who know how algorithms work ad who do not
www.mediantechnologies.com- Nozha Boujemaa4

5. • Decision explanation and tractability: Trust and Transparency of computer-
aided decision-making process (decision responsibility):what are the
different criteria/data/settings that have led to the specific decision in order
to understand the global path for the reasoning?
• “How Can I trust Machine Learning prediction?” it happens to build the
model of the object context rather the object itself
• Robustness to bias/diversion/corruption
Transparent and Accountable Data Management and Analytics
Nozha Boujemaa - 5
www.mediantechnologies.com- Nozha Boujemaa5

6. Explanation:
Ribeiro et al. 2016, LIME: Why should I trust you?
Explaining the predictions of any classifier
Safe AI: Robustness and Explanation
Robustness:
Goodfellow, Shlens and Szegedy 2015,“Explaining and
Harnessing Adversarial Examples”
Nozha Boujemaa - 6
www.mediantechnologies.com- Nozha Boujemaa6

7. Algorithmic Systems Bias
Mastering Big Data Technologies: Bias problems could impact data technologies
accuracy and people’s lives
Challenges 1: Data Inputs to an Algorithm
– Poorly selected data
– Incomplete, incorrect, or outdated data
– Data sets that lack disproportionately represent certain populations
– Malicious attack
Challenges 2: The Design of Algorithmic Systemsand Machine Learning
– Poorly designed matching systems
– Unintentional perpetuation and promotion of historical biases
– Decision-making systems that assume correlation implies causation
www.mediantechnologies.com- Nozha Boujemaa7

8. Challenges
• It is a mistake to assume they are objective simply because they are
data-driven. Algorithms are encapsulatedopinions through decision
parameters and learning data
• Implementing the “Transparent-by-Design”: fairness/equity, loyalty,
neutrality => “Value-by-Design”
• Mastering the accuracy and robustness of Big Data & AI techniques:
bias, diversion/corruption, reproducibility, source of unintentional
discrimination
Nozha Boujemaa - 8
www.mediantechnologies.com- Nozha Boujemaa8

9. Challenges : Trustworthy AI
 Responsible: Compliance with Regulation/Policy and Social Values/Ethics
 Robust and safe: against bias, corruption, noise, reproducibility, repetability etc
 Auditability and Responsible-by-Design tools and algorithms for socio-economic
empowerment
 AI is part of the solution and not only the law! Algorithmic tools to monitor the
behavior of AI technologies(traceability, interpretability etc)
 Algorithmic tools to empower regulation bodies for law execution efficiency
 Governance of Data is key, ML algorithms are shared in open-source but NOT Data
 Available Data ≠ Exploitable
Transparency Tools vs GDPR vs Cloud Act (Clarifying Lawful Overseas Use of Data Act) ?
www.mediantechnologies.com- Nozha Boujemaa9

10. Challenges / Efforts
 Complex concepts, Dependent on cultural context, law context, etc.
Transparency, Accountability, Loyalty, Fairness, Equity, Intelligibility, Explainability,
Traceability, Auditability, Proof and Certification, Performance, Ethics, Responsibility
 Pedagogy and explanation, awareness rising, uses-cases, (all public! Including scientists)
Ethical ≠ Responsible, Transparent ≠ Make available the source code
International collaboration is key (AI HLG- EC, OECD, UNESCO etc)
 Interdisciplinary co-conception of solutions, How responsible is a ML algorithm?
 Interdisciplinary training for Data Scientists:law, sociology and economy, Careful
software reuse => mastering information leaks (SRE)
Nozha Boujemaa - 10
www.mediantechnologies.com- Nozha Boujemaa10

11. International Efforts – AI HLEG EC
https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai
Artificial Intelligence - High Level Expert Group of the European Commission (AI HLEG Chair: Pekka Ala-
Pietilä, 2 Vice-Chairs: Nozha Boujemaa D1 & Barry O’Sullivan D2)
Requirements:
1.Human agency and oversight (fundamental rights)
2.Technical robustness and safety
3.Privacy and data governance
4.Transparency (Including traceability)
5.Diversity, non-discrimination and fairness
6.Societal and environmental wellbeing (Including sustainability and democracy
7.Accountability
=> Living assessment list through key economic sectors
www.mediantechnologies.com- Nozha Boujemaa11

12. International Efforts – AI HLEG EC
https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai
Artificial Intelligence - High Level Expert Group of the European Commission (AI HLEG Chair: Pekka Ala-
Pietilä, 2 Vice-Chairs: Nozha Boujemaa & Barry O’Sullivan)
www.mediantechnologies.com- Nozha Boujemaa12
Realising Trustworthy AI throughout the system’s entire life cycle

13. Living documents throught Assessment List (sectorial pilots):
1.Human agency and oversight (fundamental rights)
2.Technical robustness and safety :
1.Resilience to attack and security:
2.Fallback plan and general safety:
3.Accuracy
4.Reliabilityand reproducibility:
3.Privacy and data governance
1.Respect for privacy and data Protection:
2.Qualityand integrityof data:
3.Access to data:
4.Transparency (Including traceability)
1.Traceability:
2.Explainability
3.Communication
5.Accountability through Auditability
6. Societal and environmental well-being
www.mediantechnologies.com- Nozha Boujemaa13

14. International Efforts – AIGO
https://www.oecd.org/going-digital/ai/principles/
Artificial IntelligenceExpert Group at the OECD
Principles released May 23 2019, Book June 11 2019
1.AI should benefit people and the planet by driving inclusive growth, sustainable development and well-being.
2.AI systems should be designed in a way that respects the rule of law, human rights, democratic values and
diversity, and they should include appropriate safeguards – for example, enabling human intervention where
necessary – to ensure a fair and just society.
3.There should be transparency and responsible disclosure around AI systems to ensure that people
understand when they are engaging with them and can challenge outcomes.
4.AI systems must function in a robust, secure and safe way throughout their lifetimes, and potential risks
should be continually assessed and managed.
5.Organizations and individuals developing, deploying or operating AI systems should be held accountable for
their proper functioning in line with the above principles.
www.mediantechnologies.com- Nozha Boujemaa14

15. FDA consultation
AI/Machine Learning
Software As Medical
Device
Nozha Boujemaa - 15
www.mediantechnologies.com- Nozha Boujemaa15
Responsible AI in HealthCare
Purpose: Patient safety and security
=> Master side effects: potential errors and
conditionsof correctalgorithmicoutcome

16. The traditional paradigm of medical device regulation was not
designed for adaptive AI/ML technologies
 In the current framework, FDA would require a new
premarket submission when the AI/ML software
modification significantly affects:
o device performance
o safetyand effectiveness.
o device’s intendeduse
o major change to the software algorithm.
 The new proposed framework addresses the critical
question of regulating:
o What is the AI/ML software modification?
o How does it affect Product Lifecycle RegulatoryApproach?
o How are Premarket Assurance ofSafety and Effectiveness assessed?
16
Responsible AI in HealthCare
www.mediantechnologies.com- Nozha Boujemaa

17. Take away messages: TrustworthyAI => Proof of Trust
Should we regulate more AI?
⇒ Commitment to Traceability foster Self-Regulation
Do we need explainability? Which explainability?
⇒Enable Technical Accountability & Auditability
⇒Insure Robustness
– Data selection & life cycle monitoring,
– Algorithmic repeatability, reproducibility, interpretability
– Risk assessment and management
www.mediantechnologies.com- Nozha Boujemaa17

18. www.mediantechnologies.com- Nozha Boujemaa18

19. Thank you!
Our Core Values
Leadinginnovationwith purpose
Combine the spirit of innovationwith our passion and convictionto
help cure cancer and other debilitatingdiseases.
Committing toqualityin all we do
Be dedicated to qualityin everythingwe do. Qualitybegins with us
and we are committed to it.
Supporting our customersin achieving theirgoals
Listen to the needs of our customers and help make their goals our
goals through our innovation,imagingexpertise,superior services
and qualitysolutions.
Putting the patientfirst
There is a person at the other end of the images we analyze who is
countingon us to do everythingwe can to help make them healthier.
www.mediantechnologies.com19