Slides of the keynote at the Agile Testing Days Conference, Potsdam, November 16th, 2021.

1. My slides are available for you at:
Coming to terms with
intelligence in machines
Prof. Dr. Dagmar Monett
Computer Science Dept., Berlin School of Economics and Law
dagmar.monett-diaz@hwr-berlin.de
https://www.slideshare.net/dmonett/monett-2021-atd
Keynote, Nov. 16, 2021

2. 2
@dmonett
IG: @ThePartyEye

3. 3
@dmonett
IG: @ThePartyEye

4. 4
@dmonett
Main statement:
There are no truly
intelligent artifacts
on the horizon yet.

5. 5
@dmonett
“The long-term dream of AI is to
build machines that have the
that people
have—to build machines that are
self-aware, conscious and
autonomous in the same way
that people like you and me are.”
Wooldridge, M. (2020). The Road to Conscious Machines: The Story of AI. UK: Pelican Random House.

6. 6
@dmonett
“Intelligence measures an agent’s ability
to achieve goals in a wide range of
environments.” (Legg & Hutter, 2007)
Fogel, D. B. (2006). Defining Artificial Intelligence. In Evolutionary Computation: Toward a New Philosophy of Machine Intelligence.
Third Edition, pp. 1-32. The Institute of Electrical and Electronics Engineers, Inc., IEEE Press.
Legg, S. and Hutter, M. (2007). Universal Intelligence: A Definition of Machine Intelligence. Minds and Machines, 17(4):391-444,
Springer.
McCarthy, J. (2007). What is Artificial Intelligence? Computer Science Department, School of Engineering, Stanford University.
Wang, P. (2008). What Do You Mean by "AI"? In P. Wang, B. Goertzel, and S. Franklin (eds.), Artificial General Intelligence 2008,
Proceedings of the First AGI Conference, Frontiers in Artificial Intelligence and Applications, 171:362-373. IOS Press Amsterdam,
The Netherlands.
“[Artificial Intelligence is] the science and
engineering of making intelligent
machines. ... It is related to the similar task
of using computers to understand human
intelligence.” (McCarthy, 2007)
“Intelligence is] the capability of a system
to adapt its behavior to meet its goals in a
range of environments.” (Fogel, 2006)
Four examples of AI definitions
“The essence of intelligence is the
principle of adapting to the environment
while working with insufficient
knowledge and resources.” (Wang, 2008)

7. 7
@dmonett
Misleading news and hype around AI

8. 8
@dmonett
“
, the one
“available through
the newspapers,
books and films.”
Collins, H. (2018). Artifictional Intelligence: Against Humanity’s Surrender to Computers. Cambridge, UK: Polity Press.

9. 9
@dmonett
“If you rely on movies and science
fiction (and even some popular non-
fiction) for your view of AI, you will
be afraid of AI becoming conscious,
turning malevolent, and trying to
enslave or kill us all. But given
,
this is not what most people in the AI
community worry about.”
Mitchell, M. (2019). Artificial Intelligence: A Guide for Thinking Humans. UK: Pelican Random House.

10. 10
@dmonett
“Neither deep learning
nor other forms of
second-wave AI, nor any
proposals yet advanced
for third-wave,
.”
Smith, B. C. (2019). The Promise of Artificial Intelligence: Reckoning and Judgment. Cambridge, MA: The MIT Press.

11. 11
@dmonett
“The myth is not that true AI is
possible. As to that, the future of AI
is a scientific unknown.
–that we have
already embarked on the path that
will lead to human-level AI, and then
superintelligence. We have not.”
Larson, E. J. (2021). The Myth of Artificial Intelligence: Why computers can’t think the way we do. Cambridge, MA: Berlknap,
Harvard University Press.

12. 12
@dmonett
“The reality of AI for
the foreseeable
future
to the
grand dream.”
Wooldridge, M. (2020). The Road to Conscious Machines: The Story of AI. UK: Pelican Random House.

13. 13
@dmonett
Key learning 1:
There is
on
defining intelligence.

14. 14
@dmonett
No consensus
Journal editors (1921). Intelligence and Its Measurement: A Symposium. Journal of Educational Psychology, Vol 12(3), 123-147.
Detterman, D. K. (1986). Qualitative Integration: The Last Word? In R. J. Sternberg and D. K. Detterman (eds.), What is intelligence?
Contemporary Viewpoints on its Nature and Definition, pp. 163-166. Norwood, NJ: Ablex.
Chollet, F. (2019). The Measure of Intelligence. arXiv:1911.01547 [cs.AI].
“There is very great disagreement concerning the concept of
intelligence.” (Journal editors 1921)
“[A] substantial disagreement on a single definition still abounds.”
(Detterman 1986)
“It is a testimony to the immaturity of our field that the question of
what we mean when we talk about intelligence still doesn’t have a
satisfying answer.” (Chollet 2019)

15. 15
@dmonett
“The lack of specificity allows journalists, entrepreneurs, and
marketing departments to say virtually anything they want.”
(Lipton, 2018)
“[T]he public knowledge and understanding on AI [...] is
suffering from a lack of transparency as to capabilities and
thus impacts of AI.” (Nemitz, 2018)
“[A] lack of clarity in terms of definitions and objectives seems
to have plagued the [AI] field right back to its origins in the
1950s. This makes tracing [its] evolution . . . a difficult task.”
(AI in the UK, 2018, p. 156)
No consensus and its consequences
http://approximatelycorrect.com/2018/06/05/ai-ml-ai-swirling-nomenclature-slurried-thought/
http://dx.doi.org/10.1098/rsta.2018.0089
https://publications.parliament.uk/pa/ld201719/ldselect/ldai/100/100.pdf

16. 16
@dmonett
Key learning 2:
Both human and
machine intelligence
are concepts
.

17. 17
@dmonett
14
Educational Psychology
USA, Europe
Prediction of behavior
14
Only definitions of human
intelligence
Participants
Primary
affiliation
Countries
Focus
Definitions
Type of
definitions
Symposium
1921
Defining (A)I: A comparison

18. 18
@dmonett
5 (14) educational psychologists define intelligence
[ is …]
… the power of good responses from the point of view
of truth or facts; (Thorndike, 1921)
… the ability to carry on abstract thinking; (Terman , 1921)
… having learned or ability to learn to adjust oneself to
the environment; (Colvin , 1921)
… the capacity for knowledge; (Henmon, 1921)
… the capacity to acquire capacity. (Woodrow, 1921)
As referred to in Lanz, P. (2000). The Concept of Intelligence in Psychology and Philosophy. In Cruse, H., Dean, J., and Ritter, H. (eds.)
Prerational Intelligence: Adaptive Behavior and Intelligent Systems Without Symbols and Logic, Vol. 1, 19-30, Springer.

19. 19
@dmonett
14
Educational Psychology
USA, Europe
Prediction of behavior
14
Only definitions of human
intelligence
25
Diverse Psychologies
(educational, cognitive,
behavioral, social, cross-
cultural, etc.)
USA, Europe
Understanding of behavior
25
Mostly definitions of human
intelligence
Participants
Primary
affiliation
Countries
Focus
Definitions
Type of
definitions
Symposium Symposium
1921 1986
Defining (A)I: A comparison

20. 20
@dmonett
16 (25) leading psychologists define intelligence
[ is] an elusive concept (Estes, 1986); an illusory unified capacity
(Horn, 1986); a cognitive proficiency (Glaser, 1986); a polymorphous set of qualities
elusive to define, explain, and measure (Brown, 1986); a pluralistic (Anastasi, 1986),
context-dependent concept (Anastasi, 1986; Sternberg, 1986); a medley of important
events, a mixture of different things (Horn, 1986); a finite set of independent
abilities operating as a complex system (Detterman, 1986); the sum total of all
cognitive processes (Das, 1986); a collective term for demonstrated, mental
individual differences (Hunt, 1986); mental self-government (Sternberg, 1986); a
judgement or attribution that people do, and not a quality residing in the
individual (Goodnow, 1986); a hypothetical (Zigler, 1986), culture-bound, ethnocentric,
and excessively narrow (Berry, 1986), societal construct, a concept in the mind of
a society at large (Carroll, 1986).
A summary of some of the definitions that are included in Sternberg, R. J. and Detterman, D. K. (1986). What is intelligence?
Contemporary Viewpoints on its Nature and Definition. Norwood, NJ: Ablex.

21. 21
@dmonett
AGISI survey
14
Educational Psychology
USA, Europe
Prediction of behavior
14
Only definitions of human
intelligence
25
Diverse Psychologies
(educational, cognitive,
behavioral, social, cross-
cultural, etc.)
USA, Europe
Understanding of behavior
25
Mostly definitions of human
intelligence
567 (academia: 79.7%)
Computer Science,
Engineering, Biology,
Neurosciences, Philosophy,
Cognitive Science, etc.
57+ countries
Computation of behavior
343 (+ 4128 opinions)
Explicit distinction human vs.
machine intelligence
Participants
Primary
affiliation
Countries
Focus
Definitions
Type of
definitions
Symposium Symposium
1921 1986 2019
Defining (A)I: A comparison

22. 22
@dmonett
24.7.2017—25.7.2019 57+ 184+
Academia (N=452, 79.7%)
Industry (N=116, 20.5%)
Researchers (N=435, 76.7%)
Educators (N=197, 34.7%)
Developers, Engineers (N=90, 15.9%)
567 responses
4,128
opinions
343 new,
suggested
definitions
9x2 definitions of (human/machine)
intelligence to agree upon
AGISI survey Defining (machine) Intelligence
(Partial results) Monett, D. and Lewis, C. W. P. (2018). Getting clarity by defining Artificial Intelligence—A Survey. In Müller, Vincent C.
(Ed.), Philosophy and Theory of Artificial Intelligence 2017. SAPERE 44 (pp. 212-214). Berlin: Springer.
See http://agisi.org/Survey_intelligence.html

23. 23
@dmonett
9x2 (+) definitions of (human/machine)
intelligence to agree upon
See http://agisi.org/Defs_intelligence.html

24. 24
@dmonett
A widely accepted definition of intelligence
Gottfredson, L. S. (1997a). Mainstream science on intelligence: An editorial with 52 signatories,
history, and bibliography. Intelligence, 24: 13-23.
As cited in Haier, R. J. (2017). The Neuroscience of Intelligence. New York: Cambridge University
Press.
Most accepted definition, AGISI survey

25. 25
@dmonett
1 target article on defining AI (Wang, 2019)
20 commentaries from leading AI experts
1 extended answer from target author
Wang, P. (2019). On Defining Artificial Intelligence. Journal of Artificial General
Intelligence, 10(2), 1–37.
Monett, D., Lewis, C. W. P., & Thórisson, K. R. (eds.) (2020). Special Issue “On Defining
Artificial Intelligence.” Journal of Artificial General Intelligence, 11(2), 1–100.
But still no general consensus in the AI community!

26. 26
@dmonett
“ .
Rather, artificial intelligence is both
embodied and material, made from
natural resources, fuel, human labor,
infrastructures, logistics, histories, and
classifications. AI systems are not
autonomous, rational, or able to discern
anything without extensive,
computationally intensive training with
large datasets or predefined rules and
rewards.”
Crawford, K. (2021). Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. Yale University Press.

27. 27
@dmonett
Key learning 3:
between
AI and Software
Engineering
.

28. 28
@dmonett
AI?
It is software!

29. 29
@dmonett
Monett, D., & Lemke, C. (2021). AI-ware: Bridging AI and Software Engineering for responsible and sustainable intelligent artefacts.
In van Giffen, B., Koehler, J., Brenner, W., & Albayrak, C. A., Managing Artificial Intelligence (pp. 66-73), Workshop Paper Series,
INFORMATIK 2021, Institute of Information Management, University of St. Gallen. A workshop co-located with INFORMATIK 2021,
the 51st Annual Conference of the German Informatics Society (GI), September 29th - October 1st, 2021, in Berlin and online.
Soft-
ware
AI-
ware

30. 30
@dmonett
(Monett & Lemke, 2021)
AI-
ware
The need to be aware of what AI is and is not,
its subfields, as well as a critical appraisal of
where does applying AI make sense at all.
The conjunction of AI and Software
Engineering, mainly –but not only– the former
learning from the variety of well-established
techniques and good practices from the latter,
at the same time extending them.

31. 31
@dmonett
research agenda & Responsible AI
(Monett & Lemke, 2021)
Societal and
ethical
perspective
Algorithmic
perspective
Data-oriented
perspective
Framework-
based
perspective
Economics
perspective
Interdisciplinary
perspective

32. 32
@dmonett
Ex.: Requirements engineering as a process
Adapted from
(Schenkel, 2014)

33. 33
@dmonett
Ex.: Requirements engineering as a process
Adapted from
(Schenkel, 2014)
Responsible AI:
in all stages!
RAI
RAI
RAI
RAI
RAI
RAI
RAI
RAI

34. 34
@dmonett
Ex.: Requirements development in detail
Adapted from
(Wiegers & Beatty,
2013)

35. 35
@dmonett
Ex.: Requirements development in detail
Adapted from
(Wiegers & Beatty,
2013)
Responsible AI:
in all stages!
RAI
RAI
RAI
RAI

36. 36
@dmonett
Agile development? No problem!
© Ignacio Palomo Duarte
https://flic.kr/p/8QjDJy

37. 37
@dmonett
Agile development? No problem!
Responsible AI:
in all stages!
RAI
RAI
RAI
RAI
RAI
RAI
RAI
RAI RAI
RAI
RAI
RAI
RAI
RAI
RAI
RAI
RAI
RAI
RAI
RAI
RAI
© Ignacio Palomo Duarte
https://flic.kr/p/8QjDJy

38. 38
@dmonett
How?

39. 39
@dmonett
Responsible AI, Regulating AI

40. 40
@dmonett
https://ec.europa.eu/futurium/en/ai-alliance-consultation/guidelines/2
“[A] non-exhaustive
[with
] to operationalise
Trustworthy AI. It particularly
applies to AI systems that
directly interact with users, and is
primarily addressed to
developers and deployers of AI
systems.”

41. 41
@dmonett
“Did you assess how your
system behaves in
unexpected situations
and environments?”
Cartoon created at www.projectcartoon.com
(Trustworthy AI assessment list, 2019)

42. 42
@dmonett
“Did you consider the
potential impact or safety
risk to the environment
or to animals?”
Cartoon created at www.projectcartoon.com
(Trustworthy AI assessment list, 2019)

43. 43
@dmonett
“In case the AI system
features a chat bot or
conversational system,
are the human end users
made aware of the fact
that they are interacting
with a non-human
agent?”
Cartoon created at www.projectcartoon.com
(Trustworthy AI assessment list, 2019)

44. 44
@dmonett
“Did you consider ways to
develop the AI system or
train the model without
or with minimal use of
potentially sensitive or
personal data?”
Cartoon created at www.projectcartoon.com
(Trustworthy AI assessment list, 2019)

45. 45
@dmonett
“Did you consider
diversity and
representativeness of
users in the data?
Did you test for specific
populations or
problematic use cases?”
Cartoon created at www.projectcartoon.com
(Trustworthy AI assessment list, 2019)

46. 46
@dmonett
“Did you put in place
mechanisms that
facilitate the system’s
auditability by internal
and/or independent
actors?”
Cartoon created at www.projectcartoon.com
(Trustworthy AI assessment list, 2019)

47. Put it
on your backlog!
That’s how RAI gets done.
@dmonett

48. 48
@dmonett
It is software!

49. 49
@dmonett
Responsible AI needs
responsible humans.

50. 50
@dmonett
Main statement:
There are no truly
intelligent artifacts
on the horizon yet.

51. 51
@dmonett
We are
the intelligent
ones.

52. Contact
/monettdiaz
@dmonett
Prof. Dr.
Dagmar Monett
52
http://monettdiaz.com
dagmar.monett-diaz@hwr-berlin.de
Prof. Dr. Computer Science
(Artificial Intelligence, Software Engineering)
Computer Science Dept.
Co-Director M.Sc. Digital Transformation
Berlin School of Economics and Law (HWR Berlin)
Faculty of Cooperative Studies
Template: ATD and www.allppt.com