Mohamed Oubbati Embodied Intelligence

Embodied Intelligence 
when BRAIN, BODY, and ENVIRONMENT interact 
Mohamed Oubba? 
March, 2012 
1 Dr. Oubba?, March 2012

Embodied Intelligence 
when BRAIN, BODY, and ENVIRONMENT interact 

Intelligence 

Can we design intelligence? 

I mean human‐like intelligence, not 
just the “appearance” of intelligence 

YES  

YES  
BUT 
1. we need to know how. 
2. we need tools to implement this know‐how. 

YES  
BUT 
1. we need to know how. 
2. we need tools to implement this know‐how. 
This talk is about 
these two points 

What is intelligence? 
In 1921, the Journal of Educa?onal Psychology 
asked 14 experts for deﬁni?ons 

Result: 14 different defini?ons! 
In 1921, the Journal of Educa?onal Psychology 
asked 14 experts for defini?ons 

Intelligence is diﬃcult to deﬁne  
and understand 

Intelligence is difficult to define  
and understand 
Maybe one should try to build ar?ficial 
intelligent systems in order to understand 
intelligence 

Ar?ﬁcial Intelligence 

•  “ …the study of how to make computers do things at 
which, at the moment, people are beber” 
A.I. research is  
(Rich & Knight, 1991) 
•  “…the art of crea?ng machines that perform func?ons 
that require intelligence when performed by people“  
(Kurzweil, 1990) 
•  The modern deﬁni?on of A.I. is “the study and design 
of intelligent agents” 

The current AI has produced success in the narrow:  
well‐deﬁned problems, with veriﬁable results, in 
predictable environments, using well‐understood 
techniques. 

In 1997, Deep Blue beat Kasparov. 
Some AI programs are extremely successful at 
what they are specialized. 

‐  natural language understanding 
‐  general vision 
‐  understanding a movie 
‐  wri?ng 
Although many AI systems are beyond human‐level 
competence at some individual tasks, AI has failed 
(miserably) in many domains like 
Failure of the current AI 

To overcome the current AI problems we 
must return back to the original goal of AI: 
„create systems that are humanly intelligent“  
Failure of AI 

To overcome the current AI problems we 
must return back to the original goal of AI: 
„create systems that are humanly intelligent“  
Failure of AI 
aims to create a computer‐based intelligence at least as 
smart as a human. 
(since 2001) 

• Conventional AI (also called Narrow AI, Applied AI, Weak AI 
or Classic AI) 
(also called Strong AI , Human‐level AI, Real AI,…) 
Conventional AI vs. AGI   
Intelligence is the ability to achieve pre‐deﬁned goals, 
under constraints. 
Intelligence is the ability to expand and develop new 
abili?es. 

aims to create machines that act as if they were 
intelligent, i.e. the appearance of intelligence. 
aims to create, approach, and exceed human intelligence. 
22 Dr. Oubba?, March 2012 
• Conventional AI (also called Narrow AI, Applied AI, Weak AI 
or Classic AI) 
(also called Strong AI , Human‐level AI, Real AI,…)

Conventional A.I.
PROBLEM 
SPACE 
[Adapted from Dr. Goertzel] Conventional AI vs. AGI   

most people work on conventional AI 

most people work on conventional AI 
...maybe because working on AGI is risky: 
1.  Maybe it is  “impossible”. 
2.  It is diﬃcult to get funding for risky 
„ambi?ous“ research projects that aim to go 
towards human‐level intelligence. 
3.  Achieving human‐level intelligence means the 
possibility to achieve Superintelligence that 
could be out of control. 

I consider the following ques?ons 

YES 

YES 
we have a solid proof that it is possible — namely 
ourselves.  

How can a robot learn like a child? 

How can a robot learn like a child? 
 by trying to reconstruct the processes of 
development of the child's mind. 

Child Development 

Child development is a 
process every child goes 
through. 
It includes 
physical development, personal development,  
social development, and cogni?ve development. 
child development 

Child development is a 
process every child goes 
through. 
It includes 
physical development, personal development,  
social development, and cogni?ve development. 

From birth, babies begin to ac?vely gather and 
process informa?on from the surrounding world, 
and use the data to develop percep?on and 
thinking skills.  
Cogni?ve Development 

years 
World 0 
World 1 
World 2 
World 3 
The process of understanding the world changes as a 
func?on of age & experience 

years 
World 0 
World 1 
World 2 
World 3 
The process of understanding the world changes as a 
func?on of age & experience 
How does this happen? 

Jean Piaget (1896‐1980)  
(Psychologist) 
Lev Vygotsky (1896‐1934) 
(Psychologist) 
How children’s cogni?on develop? 
Two typical sources of inspira?on 

(Psychologist) 
Piaget was interested in how a child 
adapts to its environment.  
(he described this ability as intelligence) 

(Psychologist) 
Piaget was interested in how a child 
adapts to its environment.  
(he described this ability as intelligence) 
Adapta?on 
Assimila?on  Accomoda?on 

Assimila?on 
Adding new 
experiences to the 
exis?ng ones, and 
reinforce them. 
Accommoda?on 
Adjus?ng exis?ng 
experiences to deal 
with new situa?ons. 
Piaget’s assump?ons 

Development takes place in dis?nct stages of cogni?ve development. 

Sensorimotor (~Birth to 2 yrs)
achieving simple goals in 
a sensorimotor space 

Preopera?onal (~2 to 6 yrs)
Represen?ng the world, and go beyond  
the connec?on of sensorimotor 
informa?on. 

informa?on. 
Concrete Opera?onal (~ 7-12 yrs) reasoning logically about events. 

informa?on. 
Concrete Opera?onal (~ 7-12 yrs) reasoning logically about events. 
Formal Opera?onal 
(adolescence
to adult)
reasoning in a more abstract, 
idealis?c, and logical way. 

Piaget did not emphasize on the role 
of social interac?on. 

(Psychologist) 
Most of a child’s 
cogni?ve skills evolve 
from social interac?ons 

(Psychologist) 
Most of a child’s 
cogni?ve skills evolve 
from social interac?ons 
Development occurs within the  
„zone of proximal development“ 

range of tasks that are 
too complex to be 
mastered alone but can 
be accomplished with 
guidance. 
Vygotsky’s assump?on 
 Zone of Proximal Development 

Child can  Child cannot 
Zone of 
Proximal 
Development 
Vygotsky’s assump?on 
 Zone of Proximal Development 
Learning / Development 
54 
child development

56 
Social engagement 
Individual capabili?es  
   Zone of Proximal Development 
Sensorimotor 
Preopera?onal 
Concrete  
Opera?onal 
Formal  
Opera?onal 
Dr. Oubba?, March 2012

…so how can we create a system 
that exhibits autonomous 
development? 

development? 
We need „informal“ deﬁni?ons for intelligence, 
anyway….. 

development? 
I will call them signs of intelligence 
We need „informal“ deﬁni?ons for intelligence, 
anyway….. 

•  Learn (understand) from experience. 
•  Use reasoning to solve problems. 
•  Adapt quickly and successfully to new situa?ons. 
•  Use the acquired knowledge to manipulate the 
environment. 
[R. J. Sternberg, editor. Handbook of Intelligence. Cambridge University Press, 2000.]
[U. Neisser, et. al., Intelligence: Knowns and unknowns. American Psychologist, 51(2):77–101, 1996.]
Signs of Intelligence  
(possibly their are many other) 
How to create autonomous development? 

•  Learn (understand) from experience. 
•  Use reasoning to solve problems. 
•  Adapt quickly and successfully to new situa?ons. 
•  Use the acquired knowledge to manipulate the 
environment. 
(possibly their are many other) 

The most common idea of these signs can be 
summarized as 
intelligence is a property of a system that is 
interac?ng successfully with a problem, or 
an environment. 

To create autonomous development we have to 
design  1) a developmental learning process, 2) 
that interacts with real environments. 
control 
architecture  
That can learn and 
develop 
Environment 
That supports this 
development

Brain 
Physical 
body 
Interac?ng with the environment requires 
a physical body 
environment 

How important is the body to 
development? 

The body facilitates 
learning from the 
physical environment 
iCub‐robot (EU‐projects) 
The role of the body 
Explora?on, 
Experimenta?on,... 

Robot Kismet (MIT) 
Imita?on, Learning 
from demonstra?on, 
etc. 
The body facilitates 
learning from social 
environment 
The role of the body How to create autonomous development? 

But the actual „classic“ robo?cists 
(conven?onal AI) never disputed the 
importance of the physical body to their 
research. 

Some people believe that 
just connec?ng a computer 
to sensors and placing it on 
wheels is suﬃcient to create 
the „body“ which is required 
for intelligence. 

Intelligence requires a body which has to be able to 
embody cogni?ve processes, that aﬀects and be 
aﬀected by the environment. 

Intelligence requires a body which has to be able to 
embody cogni?ve processes, that aﬀects and be 
aﬀected by the environment. 
This is the no?on of Embodiment 

Embodiment vs. Body 
The fundamental diﬀerence lies in the degree 
of interac?on between the body and the  
brain (control program). 

Let’s consider an embodied agent... 

assume that this agent is interac?ng 
with a complex environment 

Conventional AI seeks an exact representa?on of the 
external world in order to achieve the desired task. 

The Body here 
is used just as 
an observer... 

However, the external world is too complex, and 
requires not just con?nual observa?on, but also 
reasoning and adapta?on. 
an observer... 

an observer... 
Furthermore, Knowledge (about the environment) 
is not everything.  

Only by learning from interac?on the agent will  
be able to understand the environment, and use 
this knowlege to achieve its goals. 

How to design a developmental 
learning process ? 

Developmental Architecture How to create autonomous development? 

Even simple creatures discover new strategies 
and techniques to get survival resources, or to 
ﬁght predators. 

Back to our agent… 
Dr. Oubba?, March 2012  83

What drives the agent to learn? 

Task 
…by giving it a task? 

Not only 
Developmental Architecture 
Task

Not only 
...but how to program an agent without knowing the 
task!? 
Task

Mo?va?on? 
Task

Mo?va?on?  This might work... 
Task

Mo?va?on is what gets us to act. 
Psychology 

There are two main categories of mo?va?on 
Intrinsic mo?va?on 
doing something just 
because it is enjoyable.  
(eg. curiosity)  
doing something because 
it leads to external 
rewards. (e.g. hunger)  
Extrinsic mo?va?on 

sa?sfac?on 
+ 
‐  consump?on 
(Inspired by the PSI theory proposed by the psychologist Dietrich Dörner) 
A mo?ve as a container 
The ﬁlling state of 
the container 
represents the 
degree of 
sa?sfac?on. 

Gathering resources 
+ 
‐  energy consump?on 
Extrinsic mo?va?on 
Hunger 

Exploring Novelty 
+ 
‐  Predictability of 
the enviroment 
Intrinsic mo?va?on 
Curiosity 

Hunger 
Curiosity 
Body Mind 

Curiosity 
Hunger 
Mo?ves will drive the agent to learn as much as possible 
about the environment. 

Curiosity 
Hunger 
The needs of the body and those of the mind may 
compete in certain circumstances. 

Curiosity 
Hunger 
Associa?ng importance to one mo?ve over the other is 
constantly re‐evaluated by experience. 

Curiosity 
Hunger 
Learning how to balance between mo?ves will provide the agent the 
ability to solve a faced problem in several ways. 
Thus, more ﬂexibility (authority) in handling situa?ons. 

how to ar?culate preferences 
among mo?ves?  

how to ar?culate preferences 
among mo?ves?  
We consider this as a mul?objec?ve 
problem. 

We propose a hierarchical adap?ve‐cri?c 
design (ACD) that allows an embodied agent  
to adapt its behavior in presence of several 
mo?ves. 
How to solve mul?objec?ve problems? 

The proposed architecture is mainly based on reservoir compu?ng.  
Reservoir 
Only the output 
connec?ons are 
to be trained 

Cri?c 1 
Cri?c 2 
Cri?c N 
A higher level reservoir may serve as a cri?c to 
learn separate value func?ons for each mo?ve 
simulatneously.  
High‐level ACD 

Cri?c 1 
Cri?c 2 
Cri?c N 
1.  This would have an advantage to learn from separate state 
spaces speed up learning. 

Cri?c 1 
Cri?c 2 
Cri?c N 
2.  The ability to learn from heterogenous data with the same 
weights (i.e. memory is in the recurrent ac?va?on,not only in 
the synap?c weights) the reservoir would be able to cope with 
the conﬂic?ng demands imposed by the mo?ves.  

Low‐level ACDs 
At a low level several  
reservoirs may have the role to 
learn op?mal behavior from 
the con?nuous and uncertain 
state space to sa?sfy the 
selected mo?ves. 
γ 

Low‐level ACDs 
γ 
Thus, our agent runs several 
ACDs in parallel that learn 
from diﬀerent state spaces. 

Hierarchical ACD with 
reservoir compu?ng to 
solve mul?objec?ve 
problems 

M. Oubba?, J. Uhlemann, and G. Palm (2012), Adap7ve learning in con7nuous 
environment using actor‐cri7c design and echo‐state networks. FROM ANIMALS TO 
ANIMATS (SAB12).  
ACD with reservoir compu?ng: preliminary results 

Currently, we are working on this 
developmental learning system  

We imagine how the agent explores the environment in order to 
sa?sfy ist inborn drives. We imagine how it con?nually wants to 
master the environment, and how it doeas ist best to successfully 
face challenges. We imagine how this agent asks for social 
interac?on to learn from humans, what it cannot learn 
independently... This is what we want to realize. 

Once the ﬁrst AGI is built, it is diﬃcult to predict 
what further degrees of intelligence can be built. 

If these goals are not oriented to our needs, it 
would lead to unpredictable, and probably 
dangerous, consequences for our survival. 

Danke 

Mohamed Oubbati Embodied Intelligence

More Related Content

More from Mohamed Oubbati

Recently uploaded

Mohamed Oubbati Embodied Intelligence