Adaptation                          in embodied & situated agents                                      Author: Claudio Mar...
The problem                It is difficult to build autonomous systems                through a top-down approach:         ...
Evolutionary robotics is a branch of robotics                       that uses evolutionary methodologies                  ...
The objective                            We wanted to analyze the possibility                                of applying a...
E&S agents                     • Embodied: the agent can exploit the                          characteristics of the robot...
The methodology                          E-puck Robot       Simulation            Problem: categorize 10 objects (Good, Po...
The evolutionary process                                     7Tuesday, October 11, 11
1st goal                   Implement an algorithm for individual learning.                                  The algorithm ...
Simulated Annealing                                       Temperature:                                       It probabilis...
Stochasticity in E&S                                Evaluation depends on                            the (random) initial ...
The intuition                            Temperature                                  Stochasticity             0.9       ...
Contributions             Substitute external stochasticity with internal:             •       Remove Temperature         ...
2nd goal                          Implement an algorithm for social learning.                            The algorithm sho...
Why?                 Social learning should avoid reinventing the wheel.                 In principle, when guided, learni...
How?               There are simpler forms of social learning:                • social facilitation                • conta...
How (technically)?           Fitness function: student should learn to give           outputs similar to the agent’s, give...
How (technically)?               Pure imitation brings to under-fitting individuals.                      We introduced a h...
Contributions             •       Modeled social learning with simple form of imitation             •       Modeled hybrid...
Intuitive interpretation          parameters space                         solutions space                                ...
Questions?                              20Tuesday, October 11, 11
Upcoming SlideShare
Loading in …5
×

Adaptation in Embodied & Situated Agents

3,541 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
3,541
On SlideShare
0
From Embeds
0
Number of Embeds
2,848
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Adaptation in Embodied & Situated Agents

  1. 1. Adaptation in embodied & situated agents Author: Claudio Martella Collaborators: Dott. Stefano Nolfi (ISTC - CNR) Prof. N.A. Borghese (AIS Lab - UniMi) October, 2011 1Tuesday, October 11, 11
  2. 2. The problem It is difficult to build autonomous systems through a top-down approach: • the behavior might be too complex for the designer to control • the environment is noisy and not perfect • the world is unpredictable 2Tuesday, October 11, 11
  3. 3. Evolutionary robotics is a branch of robotics that uses evolutionary methodologies to develop controllers for autonomous robots. Nolfi, Floreano [2004] - MIT Press 3Tuesday, October 11, 11
  4. 4. The objective We wanted to analyze the possibility of applying adaptive processes to embodied & situated agents considering evolutionary, individual and social learning. 4Tuesday, October 11, 11
  5. 5. E&S agents • Embodied: the agent can exploit the characteristics of the robot (shape, sensors, actuators etc.). • Situated: the solution can exploit the possible interactions that the environments offers. 5Tuesday, October 11, 11
  6. 6. The methodology E-puck Robot Simulation Problem: categorize 10 objects (Good, Poisonous) 6Tuesday, October 11, 11
  7. 7. The evolutionary process 7Tuesday, October 11, 11
  8. 8. 1st goal Implement an algorithm for individual learning. The algorithm should start with one set of candidate parameters and it would modify them by trial & error. Decision: start from Simulated Annealing * * "Optimization by Simulated Annealing", Kirkpatrick, S.; Gelatt, C. D.; Vecchi, M. P. (1983) - Science 8Tuesday, October 11, 11
  9. 9. Simulated Annealing Temperature: It probabilistically accepts mutations that decrease the fitness. The probability decreases with time. It allows the algorithm to jump out of local minima. 9Tuesday, October 11, 11
  10. 10. Stochasticity in E&S Evaluation depends on the (random) initial conditions: 10Tuesday, October 11, 11
  11. 11. The intuition Temperature Stochasticity 0.9 0.9 0.675 0.675 0.45 0.45 0.225 0.225 0 0 100 200 300 400 500 10 20 30 40 50 Probability of accepting negative Probability of accepting negative mutations decreases with the mutations decreases with the increase of time increase of #evaluations 11Tuesday, October 11, 11
  12. 12. Contributions Substitute external stochasticity with internal: • Remove Temperature • Start with few evaluations and increase with time Results • Simplifies the algorithm • Better performance (~10% improvement) • Lighter algorithm (~50% less evaluations for us) 12Tuesday, October 11, 11
  13. 13. 2nd goal Implement an algorithm for social learning. The algorithm should take advantage of the interaction with an expert agent to acquire an adaptive solution that is improved and/or in less time. Decision: apply individual learning to imitation. 13Tuesday, October 11, 11
  14. 14. Why? Social learning should avoid reinventing the wheel. In principle, when guided, learning is faster & safer. It should be the basis for cultural evolution. 14Tuesday, October 11, 11
  15. 15. How? There are simpler forms of social learning: • social facilitation • contagious behavior • stimulus enhancement 15Tuesday, October 11, 11
  16. 16. How (technically)? Fitness function: student should learn to give outputs similar to the agent’s, given the same input. 16Tuesday, October 11, 11
  17. 17. How (technically)? Pure imitation brings to under-fitting individuals. We introduced a hybrid approach. f it = f itsoc · (1 ↵) + f itind · ↵ c ↵= N 17Tuesday, October 11, 11
  18. 18. Contributions • Modeled social learning with simple form of imitation • Modeled hybrid social-individual learning approach Results • Performance on the problem is not improved • Adaptive behavior is acquired faster • More agents acquire an adaptive behavior 18Tuesday, October 11, 11
  19. 19. Intuitive interpretation parameters space solutions space Social learning as a method for promising initial parameters selection. Social learning as a method for jumping out of local maxima. 19Tuesday, October 11, 11
  20. 20. Questions? 20Tuesday, October 11, 11

×