2. The problem
It is difficult to build autonomous systems
through a top-down approach:
• the behavior might be too complex for the
designer to control
• the environment is noisy and not perfect
• the world is unpredictable
2
Tuesday, October 11, 11
3. Evolutionary robotics is a branch of robotics
that uses evolutionary methodologies
to develop controllers for autonomous robots.
Nolfi, Floreano [2004] - MIT Press
3
Tuesday, October 11, 11
4. The objective
We wanted to analyze the possibility
of applying adaptive processes
to embodied & situated agents
considering
evolutionary, individual and social learning.
4
Tuesday, October 11, 11
5. E&S agents
• Embodied: the agent can exploit the
characteristics of the robot (shape,
sensors, actuators etc.).
• Situated: the solution can exploit the
possible interactions that the environments
offers.
5
Tuesday, October 11, 11
6. The methodology
E-puck Robot Simulation
Problem: categorize 10 objects (Good, Poisonous)
6
Tuesday, October 11, 11
8. 1st goal
Implement an algorithm for individual learning.
The algorithm should start
with one set of candidate parameters
and it would modify them by trial & error.
Decision: start from Simulated Annealing *
* "Optimization by Simulated Annealing", Kirkpatrick, S.; Gelatt, C. D.; Vecchi, M. P. (1983) - Science
8
Tuesday, October 11, 11
9. Simulated Annealing
Temperature:
It probabilistically accepts
mutations that decrease
the fitness.
The probability decreases
with time.
It allows the algorithm to
jump out of local minima.
9
Tuesday, October 11, 11
10. Stochasticity in E&S
Evaluation depends on
the (random) initial conditions:
10
Tuesday, October 11, 11
11. The intuition
Temperature Stochasticity
0.9 0.9
0.675 0.675
0.45 0.45
0.225 0.225
0 0
100 200 300 400 500 10 20 30 40 50
Probability of accepting negative Probability of accepting negative
mutations decreases with the mutations decreases with the
increase of time increase of #evaluations
11
Tuesday, October 11, 11
12. Contributions
Substitute external stochasticity with internal:
• Remove Temperature
• Start with few evaluations and increase with time
Results
• Simplifies the algorithm
• Better performance (~10% improvement)
• Lighter algorithm (~50% less evaluations for us)
12
Tuesday, October 11, 11
13. 2nd goal
Implement an algorithm for social learning.
The algorithm should take advantage
of the interaction with an expert agent
to acquire an adaptive solution
that is improved and/or in less time.
Decision: apply individual learning to imitation.
13
Tuesday, October 11, 11
14. Why?
Social learning should avoid reinventing the wheel.
In principle, when guided, learning is faster & safer.
It should be the basis for cultural evolution.
14
Tuesday, October 11, 11
15. How?
There are simpler forms of social learning:
• social facilitation
• contagious behavior
• stimulus enhancement
15
Tuesday, October 11, 11
16. How (technically)?
Fitness function: student should learn to give
outputs similar to the agent’s, given the same input.
16
Tuesday, October 11, 11
17. How (technically)?
Pure imitation brings to under-fitting individuals.
We introduced a hybrid approach.
f it = f itsoc · (1 ↵) + f itind · ↵
c
↵= N
17
Tuesday, October 11, 11
18. Contributions
• Modeled social learning with simple form of imitation
• Modeled hybrid social-individual learning approach
Results
• Performance on the problem is not improved
• Adaptive behavior is acquired faster
• More agents acquire an adaptive behavior
18
Tuesday, October 11, 11
19. Intuitive interpretation
parameters space solutions space
Social learning as a method
for promising initial parameters selection.
Social learning as a method
for jumping out of local maxima.
19
Tuesday, October 11, 11
