Transparency in AI is important for fairness, privacy, reliability and trust. While transparency can help open "black boxes" and understand model decisions, there are also technical and audience limits. Developers may not understand all parts of complex algorithms, and transparency alone does not guarantee critical evaluation. Total transparency also raises ethical questions about privacy and how much should be revealed.

1. Responsible Data Science
- against Black
Boxes

2. Definition of transparency
Definition of black boxes in AI
Roots of transparency
Why do we need transparency in AI?
Transparency & opening the black boxes
Transparency & its limits
Hyper Transparency & morality - open questions
Summary

3. What is
transparency

4. ⇒ “ the quality of being done in an open way without secrets.”
~~ Cambridge University Dictionary
Definition of
transparency
⇒ “General definitions of transparency refers to it as ‘lifting the veil of secrecy’
or ‘the ability to look clearly through the windows of an institution’. […]
Everything is out in the open and can be scrutinized. Transparency is
contrasted with opaque policy measures, where it is hard to discover who
takes the decisions, what they are, and who gains and who loses.”
~~(Albert Meijer, 2009)

5. ⇒ “ a system or process that uses
information to produce a particular set of
results, but that works in a way that is
secret or difficult to understand”
~~ Cambridge University Dictionary
Definition of
black boxes in AI

6. “So, if I’m not mistaken, most, if not all of these deep
learning approaches, or even more generally
machine learning approaches are, essentially black
boxes, in which you can’t really inspect how the
algorithm is accomplishing what it is accomplishing.”
~~ (towardsdatascience, web)
“No one really knows how the most advanced
algorithms do what they do. That could be a
problem.”
~~ (technologyreview, web)
Definition of
black boxes in AI

7. “ Transparency is about the fear of secrets and the feeling that seeing
something may lead to control over it. For the liberal democrats openness
creates security. “
~~ (Mike Ananny, Kate Crawford,2016)
Roots of
transparency

8. “institution to reduce government and business corruption of developing countries.”In the
early 90s, the World Bank’s with its so-called politically neutral position was failing to address
corruption in its loan-giving to the nations. Peter
Eigen, a German manager at the World Bank and his team created Transparency International,
an institution to reduce government and business corruption of developing countries. The first
project of this institution was to advocate against corruption and for transparency through its
Latin American chapters working with the Organization of American States (OAS).”
~~( Carolyn Ball, 2009)
Exemple : Transparency International
Roots of
transparency

9. “rights of statutory access to government records.” is regarded as the world’s first law
supporting the freedom of the press and freedom of information the Freedom of the Press Act
abolished the censorship of all printed publications. This granted citizens access to official
documents to encourage the free exchange of ideas.”
~~ (Britannica, web)
Example extra: Freedom of press act 1766 - Swedish
legislation
Roots of
transparency

10. Transparency at the beginning was interpreted as a mean to battle corruption
by the scholars.
~~( Carolyn Ball, 2009)
Roots of
transparency

11. Have you read George Orwell’s novel “1984” ? Did you watch Black Mirror episode “Nosedive” ?
So imagine big brother and social ranking , this is what China is up to with its Social Credit
System in 2020.
The concept of ranking the citizen in china is not new, this goes back to the rule of Mao. The
implementation of this system failed at this era, but thanks to/ because of AI this is now
possible.
Beyond the privacy/moral issues, imagine all the situations where this program will run your
life and decide whether or not you’re a good citizen, deserve access to a credit, give you a rank
for an organ transplantation, ect.
~~( wired.co, web)
Example : Social Credit System
Why do we need
transparency in AI ?

12. Why do we
need
Transparency

13. - Fairness, un-biasness : No discrimination, knowing why the decision was
made
- Privacy: Protection of the privacy of data and the users
- Reliability : Robustness, well generalization against errors & attacks
- Causality : true relations between the associations and not just spurious
correlation
- Trust
~~(Doshi-Velez, F. & Kim, B., 2017)
Why do we need
transparency in AI ?

14. Eirini Malliaraki :
“ We also need to remind ourselves that algorithms don’t exercise their power
over us. People do.”
~~(Toward ethical, transparent and fair AI/ML, web)
Transparency &
opening the black
boxes

15. The CNIL (Commission nationale de l'informatique et des libertés), France’s Data Protection
Agency imposed a fine of 50 millions Euros against Google for violating “the obligations of
transparency and information” rules imposed by the EU’s General Data Protection Regulation
(GDPR). It cames as a surprise even to Google with their 29 pages of EU privacy policy.
~~(Cpomagazine, web)
Example : Google & RGPD
Why the CNIL was unsatisfied ?
“… the information provided by GOOGLE is not easily accessible to users.”
Transparency &
opening the black
boxes

16. What does it
mean to open
a black box

17. - Understand models. (the right of explanation, council of Europe; 2017)
- Learn from models and prevent them from making our mistakes.
- Responsible Data Science.
We seek interpretability from transparency
Interpretability is the understandability...
Transparency &
opening the black
boxes

18. Is transparency
sufficient
to open
black boxes

19. Technical limits:
- The cyberspace is infinite, the cybertime and the human cognitive
space are limited.
- The algorithms are dynamic.
- The algorithms are complex.
~~(Jakko Kemper & Daan Kolkman,2018)
Transparency &
its limits

20. “ The 2050 calculator is an energy and emission open sourced model. The developers of the
2050 Calculator, however, noted that very few people bothered to look into the
documentation. More importantly, they felt that by open-sourcing the model, people were
less inclined to contest its outcomes.”
~~(Jakko Kemper & Daan Kolkman,2018)
Example 1 : The 2050 Calculator
Transparency &
its limits

21. “Pensim2 is one of a number of dynamic microsimulation models in existence around the
world. The principal purpose of this model is to estimate the future distribution of pensioner
incomes.”
1. the Pensim2 model’s reviews was conducted by the United States Congressional Budget
Office, an organization that also uses a dynamic microsimulation model and agreed
about it.
2. In an interview developers said they felt more familiar with some parts of that
algorithmic model and less familiar with other parts. Moreover, some parts of the
algorithmic model may even be completely unknown to them.
~~(Jakko Kemper & Daan Kolkman,2018)
Example 2 : The pensim2
Transparency &
its limits

22. Can you deduce the
limits of
transparency from
these previous
examples

23. Audience limits:
- Experts may not have a real external objective view.
- Developers cannot understand all the part of the models.
- Transparency may lead to a less critical attitude.
~~(Jakko Kemper & Daan Kolkman,2018)
Transparency &
its limits

25. Is total transparency ethical?
Do you want a total transparent life ?
What are the limits that we mustn't cross with transparency ?
⇒ As we’re living in an “algorithmic life” this leads us to question:
should we open all the black boxes ?
Hyper Transparency &
morality - open questions

27. - (Albert Meijer, 2009): Albert Meijer Understanding modern transparency, june 2009
- (Mike Ananny, Kate Crawford,2016) : Ananny, Mike & Crawford, Kate. (2016). Seeing without knowing: Limitations
of the transparency ideal and its application to algorithmic accountability
- ( Carolyn Ball, 2009) : What Is Transparency? Carolyn Ball September 2009
- (Doshi-Velez, F., & Kim, B., 2017) : Doshi-Velez, F., & Kim, B. (2017). Towards A Rigorous Science of Interpretable
Machine Learning, (Ml)
- (Jakko Kemper & Daan Kolkman,2018): Transparent to whom? No algorithmic accountability without a critical
audience, Information, Communication & Society, DOI: 10.1080/1369118X.2018.147796
- (Britannica, web) : https://www.britannica.com/topic/Freedom-of-the-Press-Act-of-1766
- (towardsdatascience, web) https://towardsdatascience.com/the-black-box-metaphor-in-machine-learning-
4e57a3a1d2b0
- (Christoph m, ML Book): https://christophm.github.io/interpretable-ml-book/storytime.html#storytime
- (Cpomagazine, web): https://www.cpomagazine.com/data-protection/gdpr-and-the-trouble-with-transparency/
- (Toward ethical, transparent and fair AI/ML, web) https://medium.com/@eirinimalliaraki/toward-ethical-
transparent-and-fair-ai-ml-a-critical-reading-list-d950e70a70ea
- ( wired.co, web) https://www.wired.co.uk/article/chinese-government-social-credit-score-privacy-invasion
- (technologyreview, web) https://www.technologyreview.com/s/604087/the-dark-secret-at-the-heart-of-ai/
Sources