November 16, Learning

739 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
739
On SlideShare
0
From Embeds
0
Number of Embeds
29
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

November 16, Learning

  1. 1. Multi-Robot Systems<br />CSCI 7000-006<br />Monday, November 16, 2009<br />NikolausCorrell<br />
  2. 2. Course summary<br />What are multi-robot systems and when to use them<br />Reactive algorithms for multi-robot swarms<br />Deliberative algorithms for multi-robot teams<br />Gradient-based modeling and control<br />Probabilistic modeling and control<br />Optimization<br />Learning (today)<br />
  3. 3. Upcoming<br />Fall break<br />November 30-December 11: project presentations<br />Teach your peers about a specific aspect of multi-robot systems<br />Recall background and theory from the class<br />Present your project and results<br />Final reports due December 18<br />
  4. 4. Presentations<br />Particlefilters<br />Gradient-based approaches<br />Large-scale distributed systems<br />Reactive Swarms<br />Swarm Intelligence<br />Multi-Robot Teams<br />You are giving the lecture this day! Coordinate among yourselves to present common material! We want to recall what we have seen in the course and learn something!<br />
  5. 5. Today<br />Learning in multi-robot systems<br />Genetic algorithms and Particle Swarm Optimization<br />Advantages of GA and PSO in distributed systems<br />
  6. 6. Encode controllers parametrically (e.g.,Braitenberg parameters) into strings (chromosomes)<br />Evaluate with robot test runs<br />Cross-over and mutatechromosomes<br />Repeat until end criterion met<br />Genetic Algorithms<br />phenotype<br />http://en.wikipedia.org/wiki/Genetic_algorithm<br />
  7. 7. Particle Swarm Optimization<br />Controller parameters span search space<br />Instances of controllers are particles in search space<br />Particles fly through search space<br />Direction<br />Velocity<br />Inertia<br />Attraction to positions with best results, both for the individual particle and for neighborhoods<br />Optimization algorithm<br />Evaluate controllers<br />Update particles<br />Continue<br />Fitness<br />Parameter 2<br />Parameter 1<br />Current speed<br />Next best solution<br />Neighbors’s best solution<br />Current position<br />Own best solution<br />Swarm Intelligence (The Morgan Kaufmann Series in Artificial Intelligence) by Russell C. Eberhart, Yuhui Shi, and James Kennedy, 2001.<br />
  8. 8. Single robot learning: Example Modular Robots<br />Gait generated by a Central Pattern Generator (CPG)<br />Find parameters for CPGthat maximize forward motion<br />YvanBourquin, Self-Organization of Locomotion in Modular Robots, M.Sc. Thesis, University of Sussex & EPFL<br />
  9. 9. Gait optimization results<br />YvanBourquin, Self-Organization of Locomotion in Modular Robots, M.Sc. Thesis, University of Sussex & EPFL<br />
  10. 10. Parallel Learning withMulti-Agent Optimization<br />Standard technique with multi-agent optimization: evaluate in serial at each iteration<br />Very slow evolution<br />In multi-robot systems, can perform parallel evaluations<br />J. Pugh and A. Martinoli. Multi-Robot Learning with Particle Swarm Optimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 441 - 448, 2006.<br />
  11. 11. Parallel Learning withMulti-Agent Optimization<br />Standard technique with multi-agent optimization: evaluate in serial at each iteration<br />Very slow evolution<br />In multi-robot systems, can perform parallel evaluations<br />J. Pugh and A. Martinoli. Multi-Robot Learning with Particle Swarm Optimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 441 - 448, 2006.<br />
  12. 12. Parallel Learning withMulti-Agent Optimization<br />Standard technique with multi-agent optimization: evaluate in serial at each iteration<br />Very slow evolution<br />In multi-robot systems, can perform parallel evaluations<br />J. Pugh and A. Martinoli. Multi-Robot Learning with Particle Swarm Optimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 441 - 448, 2006.<br />
  13. 13. Parallel Learning withMulti-Agent Optimization<br />Standard technique with multi-agent optimization: evaluate in serial at each iteration<br />Very slow evolution<br />In multi-robot systems, can perform parallel evaluations<br />J. Pugh and A. Martinoli. Multi-Robot Learning with Particle Swarm Optimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 441 - 448, 2006.<br />
  14. 14. Example 2: Obstacle Avoidance – Group learning, individual fitness<br />Artificial Neural Network Control<br />Fitness function* rewards speed, straight movement, and avoiding obstacles:<br /><ul><li>V = average wheel speed, Δv = difference between wheel speeds, i = value of most active proximity sensor</li></ul>*Floreano, D. and Mondada, F. (1996) Evolution of Homing Navigation in a Real Mobile Robot. IEEE Transactions on Systems, Man, and Cybernetics--Part B: Cybernetics, 26(3), 396-407. <br />
  15. 15. Parallel Learning Results<br /><ul><li> 20 individuals/particles for GA/PSO divided among 20 robots, evolved for 100 iterations
  16. 16. Results averaged over 100 trials</li></ul>Average Performance<br />Throughout Evolution<br />Performance of Best<br />Evolved Controllers<br />J. Pugh and A. Martinoli. Multi-Robot Learning with Particle Swarm Optimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 441 - 448, 2006.<br />“Best-effort comparison”<br />
  17. 17. Communication-Based Neighborhoods<br />Ring Topology - Standard<br />
  18. 18. Communication-Based Neighborhoods<br />2-Closest – Model 1<br />
  19. 19. Communication-Based Neighborhoods<br />Radius r (40 cm) – Model 2<br />
  20. 20. Communication-Based Neighborhoods<br />Performance of best controllers after evolution<br /><ul><li>Both GA and PSO are sensitive to algorithmic parameters
  21. 21. Difficult to compare and to design without analytical foundations
  22. 22. Relation between embodiment, learning algorithm parameters, and fitness? </li></ul>Topology<br />Range<br />J. Pugh and A. Martinoli. Multi-Robot Learning with Particle Swarm Optimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 441 - 448, 2006.<br />
  23. 23. Varying CommunicationRange - Results<br />Average swarm performance during evolution<br />J. Pugh and A. Martinoli. Multi-Robot Learning with Particle Swarm Optimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 441 - 448, 2006.<br />
  24. 24. Example 3: Aggregation – Group learning, group fitness<br />Same neural network setup as in obstacle avoidance<br />Additional capability – sense relative positions of other nearby robots<br />Additional inputs to neural network – center of mass (x,y) of detected robots<br />Fitness function:<br />where robRP(i) is number of robots in range of robot i<br />J. Pugh and A. Martinoli. Multi-Robot Learning with Particle Swarm Optimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 441 - 448, 2006.<br />
  25. 25. Group Learning andCredit Assignment - Results<br />Performance of best controllers after evolution<br />
  26. 26. Search Scenario<br />Use team of e-pucks (mobile robot with 7 cm diameter) <br />Robots must find (within 10 cm) targets in 4 mx 4 marena<br />Robots can sense “intensity”, e.g. loudness of an audio source<br />Once found, targets are instantly moved to new location<br />Search continues indefinitely, though controller parameters may be changed<br />
  27. 27. Example 4: Bacteria-InspiredSearch Algorithm<br />E. coli chemotaxis:<br />Move<br />Check gradient<br />If positive, keep direction<br />If negative, tumble<br />Replicate approach for searching robots<br />Add collaboration – instead of tumble, go towards nearby robot with strongest detection<br />J. Pugh and A. Martinoli. Distributed Adaptation in Multi-Robot Search using Particle Swarm Optimization. In Proceedings of the 10th International Conference on the Simulation of Adaptive Behavior, Lecture Notes in Computer Science, pages 393-402, 2008.<br />
  28. 28. Improved search performance through parameter optimization<br />Adaptable parameters:<br />STEP_SIZE<br />Bacteria-InspiredSearch Algorithm<br />
  29. 29. Improved search performance through parameter optimization<br />Adaptable parameters:<br />STEP_SIZE<br />RL_RANGE<br />Bacteria-InspiredSearch Algorithm<br />
  30. 30. Improved search performance through parameter optimization<br />Adaptable parameters:<br />STEP_SIZE<br />RL_RANGE<br />CW_LIMIT<br />Bacteria-InspiredSearch Algorithm<br />
  31. 31. Improved search performance through parameter optimization<br />Adaptable parameters:<br />STEP_SIZE<br />RL_RANGE<br />CW_LIMIT<br />CCW_LIMIT<br />Bacteria-InspiredSearch Algorithm<br />
  32. 32. Group Performance: number of targets found (within 10 cm) in evaluation span<br />Individual Fitness: average detected power intensity – used for controller evaluation<br />Detected intensity for robot iof all targets j:<br />Distance detections inaccurate due to background noise<br />Optimal parameter set affected by number of targets, power of targets<br />Search Experiments<br />
  33. 33. Search Adaptation Results<br /><ul><li> 50 robots, 3 targets with power 10, 120 second evaluations
  34. 34. Compare small, medium, and large PSO neighborhoods
  35. 35. Results averaged over 250 trials</li></ul>Average Individual Fitness<br />Average Group Performance<br />J. Pugh and A. Martinoli. Distributed Adaptation in Multi-Robot Search using Particle Swarm Optimization. In Proceedings of the 10th International Conference on the Simulation of Adaptive Behavior, Lecture Notes in Computer Science, pages 393-402, 2008.<br />
  36. 36. Simulation vs. Real robots<br />Number of performance evaluations usually very high -&gt; infeasible on real robots<br />Simulation is an abstraction<br />Might not model noise accurately<br />Might not model inter-robot variations accurately<br />Solution: multi-level modeling<br />90% of the evaluations using model/simulation<br />10% using real hardware<br />
  37. 37. Summary<br />Population-based optimization algorithms are well suited for learning in multi-robot systems (parallelization)<br />Main difficulty: selection of an appropriate fitness function<br />GA and PSO are heuristics, are not guaranteed to perform, and are highly susceptive to parameter choice and algorithmic variations<br />This is in contrast to analytical optimization<br />

×