2. Course summary What are multi-robot systems and when to use them Reactive algorithms for multi-robot swarms Deliberative algorithms for multi-robot teams Gradient-based modeling and control Probabilistic modeling and control Optimization Learning (today)
3. Upcoming Fall break November 30-December 11: project presentations Teach your peers about a specific aspect of multi-robot systems Recall background and theory from the class Present your project and results Final reports due December 18
4. Presentations Particlefilters Gradient-based approaches Large-scale distributed systems Reactive Swarms Swarm Intelligence Multi-Robot Teams You are giving the lecture this day! Coordinate among yourselves to present common material! We want to recall what we have seen in the course and learn something!
5. Today Learning in multi-robot systems Genetic algorithms and Particle Swarm Optimization Advantages of GA and PSO in distributed systems
6. Encode controllers parametrically (e.g.,Braitenberg parameters) into strings (chromosomes) Evaluate with robot test runs Cross-over and mutatechromosomes Repeat until end criterion met Genetic Algorithms phenotype http://en.wikipedia.org/wiki/Genetic_algorithm
7. Particle Swarm Optimization Controller parameters span search space Instances of controllers are particles in search space Particles fly through search space Direction Velocity Inertia Attraction to positions with best results, both for the individual particle and for neighborhoods Optimization algorithm Evaluate controllers Update particles Continue Fitness Parameter 2 Parameter 1 Current speed Next best solution Neighbors’s best solution Current position Own best solution Swarm Intelligence (The Morgan Kaufmann Series in Artificial Intelligence) by Russell C. Eberhart, Yuhui Shi, and James Kennedy, 2001.
8. Single robot learning: Example Modular Robots Gait generated by a Central Pattern Generator (CPG) Find parameters for CPGthat maximize forward motion YvanBourquin, Self-Organization of Locomotion in Modular Robots, M.Sc. Thesis, University of Sussex & EPFL
9. Gait optimization results YvanBourquin, Self-Organization of Locomotion in Modular Robots, M.Sc. Thesis, University of Sussex & EPFL
10. Parallel Learning withMulti-Agent Optimization Standard technique with multi-agent optimization: evaluate in serial at each iteration Very slow evolution In multi-robot systems, can perform parallel evaluations J. Pugh and A. Martinoli. Multi-Robot Learning with Particle Swarm Optimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 441 - 448, 2006.
11. Parallel Learning withMulti-Agent Optimization Standard technique with multi-agent optimization: evaluate in serial at each iteration Very slow evolution In multi-robot systems, can perform parallel evaluations J. Pugh and A. Martinoli. Multi-Robot Learning with Particle Swarm Optimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 441 - 448, 2006.
12. Parallel Learning withMulti-Agent Optimization Standard technique with multi-agent optimization: evaluate in serial at each iteration Very slow evolution In multi-robot systems, can perform parallel evaluations J. Pugh and A. Martinoli. Multi-Robot Learning with Particle Swarm Optimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 441 - 448, 2006.
13. Parallel Learning withMulti-Agent Optimization Standard technique with multi-agent optimization: evaluate in serial at each iteration Very slow evolution In multi-robot systems, can perform parallel evaluations J. Pugh and A. Martinoli. Multi-Robot Learning with Particle Swarm Optimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 441 - 448, 2006.
14.
15.
16. Results averaged over 100 trialsAverage Performance Throughout Evolution Performance of Best Evolved Controllers J. Pugh and A. Martinoli. Multi-Robot Learning with Particle Swarm Optimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 441 - 448, 2006. “Best-effort comparison”
22. Relation between embodiment, learning algorithm parameters, and fitness? Topology Range J. Pugh and A. Martinoli. Multi-Robot Learning with Particle Swarm Optimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 441 - 448, 2006.
23. Varying CommunicationRange - Results Average swarm performance during evolution J. Pugh and A. Martinoli. Multi-Robot Learning with Particle Swarm Optimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 441 - 448, 2006.
24. Example 3: Aggregation – Group learning, group fitness Same neural network setup as in obstacle avoidance Additional capability – sense relative positions of other nearby robots Additional inputs to neural network – center of mass (x,y) of detected robots Fitness function: where robRP(i) is number of robots in range of robot i J. Pugh and A. Martinoli. Multi-Robot Learning with Particle Swarm Optimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 441 - 448, 2006.
26. Search Scenario Use team of e-pucks (mobile robot with 7 cm diameter) Robots must find (within 10 cm) targets in 4 mx 4 marena Robots can sense “intensity”, e.g. loudness of an audio source Once found, targets are instantly moved to new location Search continues indefinitely, though controller parameters may be changed
27. Example 4: Bacteria-InspiredSearch Algorithm E. coli chemotaxis: Move Check gradient If positive, keep direction If negative, tumble Replicate approach for searching robots Add collaboration – instead of tumble, go towards nearby robot with strongest detection J. Pugh and A. Martinoli. Distributed Adaptation in Multi-Robot Search using Particle Swarm Optimization. In Proceedings of the 10th International Conference on the Simulation of Adaptive Behavior, Lecture Notes in Computer Science, pages 393-402, 2008.
32. Group Performance: number of targets found (within 10 cm) in evaluation span Individual Fitness: average detected power intensity – used for controller evaluation Detected intensity for robot iof all targets j: Distance detections inaccurate due to background noise Optimal parameter set affected by number of targets, power of targets Search Experiments
35. Results averaged over 250 trialsAverage Individual Fitness Average Group Performance J. Pugh and A. Martinoli. Distributed Adaptation in Multi-Robot Search using Particle Swarm Optimization. In Proceedings of the 10th International Conference on the Simulation of Adaptive Behavior, Lecture Notes in Computer Science, pages 393-402, 2008.
36. Simulation vs. Real robots Number of performance evaluations usually very high -> infeasible on real robots Simulation is an abstraction Might not model noise accurately Might not model inter-robot variations accurately Solution: multi-level modeling 90% of the evaluations using model/simulation 10% using real hardware
37. Summary Population-based optimization algorithms are well suited for learning in multi-robot systems (parallelization) Main difficulty: selection of an appropriate fitness function GA and PSO are heuristics, are not guaranteed to perform, and are highly susceptive to parameter choice and algorithmic variations This is in contrast to analytical optimization
