Adaptation in Embodied & Situated Agents

Adaptation in embodied & situated agents Author: Claudio Martella Collaborators: Dott. Stefano Nolﬁ (ISTC - CNR) Prof. N.A. Borghese (AIS Lab - UniMi) October, 2011 1Tuesday, October 11, 11

The problem It is difﬁcult to build autonomous systems through a top-down approach: • the behavior might be too complex for the designer to control • the environment is noisy and not perfect • the world is unpredictable 2Tuesday, October 11, 11

Evolutionary robotics is a branch of robotics that uses evolutionary methodologies to develop controllers for autonomous robots. Nolﬁ, Floreano [2004] - MIT Press 3Tuesday, October 11, 11

The objective We wanted to analyze the possibility of applying adaptive processes to embodied & situated agents considering evolutionary, individual and social learning. 4Tuesday, October 11, 11

E&S agents • Embodied: the agent can exploit the characteristics of the robot (shape, sensors, actuators etc.). • Situated: the solution can exploit the possible interactions that the environments offers. 5Tuesday, October 11, 11

The methodology E-puck Robot Simulation Problem: categorize 10 objects (Good, Poisonous) 6Tuesday, October 11, 11

The evolutionary process 7Tuesday, October 11, 11

1st goal Implement an algorithm for individual learning. The algorithm should start with one set of candidate parameters and it would modify them by trial & error. Decision: start from Simulated Annealing * * "Optimization by Simulated Annealing", Kirkpatrick, S.; Gelatt, C. D.; Vecchi, M. P. (1983) - Science 8Tuesday, October 11, 11

Simulated Annealing Temperature: It probabilistically accepts mutations that decrease the ﬁtness. The probability decreases with time. It allows the algorithm to jump out of local minima. 9Tuesday, October 11, 11

Stochasticity in E&S Evaluation depends on the (random) initial conditions: 10Tuesday, October 11, 11

The intuition Temperature Stochasticity 0.9 0.9 0.675 0.675 0.45 0.45 0.225 0.225 0 0 100 200 300 400 500 10 20 30 40 50 Probability of accepting negative Probability of accepting negative mutations decreases with the mutations decreases with the increase of time increase of #evaluations 11Tuesday, October 11, 11

Contributions Substitute external stochasticity with internal: • Remove Temperature • Start with few evaluations and increase with time Results • Simpliﬁes the algorithm • Better performance (~10% improvement) • Lighter algorithm (~50% less evaluations for us) 12Tuesday, October 11, 11

2nd goal Implement an algorithm for social learning. The algorithm should take advantage of the interaction with an expert agent to acquire an adaptive solution that is improved and/or in less time. Decision: apply individual learning to imitation. 13Tuesday, October 11, 11

Why? Social learning should avoid reinventing the wheel. In principle, when guided, learning is faster & safer. It should be the basis for cultural evolution. 14Tuesday, October 11, 11

How? There are simpler forms of social learning: • social facilitation • contagious behavior • stimulus enhancement 15Tuesday, October 11, 11

How (technically)? Fitness function: student should learn to give outputs similar to the agent’s, given the same input. 16Tuesday, October 11, 11

How (technically)? Pure imitation brings to under-ﬁtting individuals. We introduced a hybrid approach. f it = f itsoc · (1 ↵) + f itind · ↵ c ↵= N 17Tuesday, October 11, 11

Contributions • Modeled social learning with simple form of imitation • Modeled hybrid social-individual learning approach Results • Performance on the problem is not improved • Adaptive behavior is acquired faster • More agents acquire an adaptive behavior 18Tuesday, October 11, 11

Intuitive interpretation parameters space solutions space Social learning as a method for promising initial parameters selection. Social learning as a method for jumping out of local maxima. 19Tuesday, October 11, 11

Questions? 20Tuesday, October 11, 11

http://blog.acaro.org	1399
http://blog.acaro.org.	1
http://translate.googleusercontent.com	1
http://abtasty.com	1

Adaptation in Embodied & Situated Agents

by Claudio Martella

Accessibility

Categories

Upload Details

Usage Rights

4 Embeds 1,402

Statistics

Adaptation in Embodied & Situated Agents Presentation Transcript