Academic Course: 10 On-line adaptation, learning, evolution
Upcoming SlideShare
Loading in...5
×
 

Academic Course: 10 On-line adaptation, learning, evolution

on

  • 339 views

By Gusz Eiben & Mark Hoogendoorn

By Gusz Eiben & Mark Hoogendoorn

Statistics

Views

Total Views
339
Views on SlideShare
228
Embed Views
111

Actions

Likes
0
Downloads
1
Comments
0

2 Embeds 111

http://www.aware-project.eu 110
http://translate.googleusercontent.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Academic Course: 10 On-line adaptation, learning, evolution Academic Course: 10 On-line adaptation, learning, evolution Presentation Transcript

  • Designed by Gusz Eiben & Mark Hoogendoorn On-line adaptation, learning, evolution
  • Designed by Gusz Eiben & Mark Hoogendoorn Outline • Population-based Adaptive Systems • Types of adaptation: evolution, individual (lifetime) learning, social learning • Machine learning • Reinforcement learning • Off-line vs. on-line adaptation
  • Designed by Gusz Eiben & Mark Hoogendoorn Population-based Adaptive Systems PAS have two essential features •They consist of a group of basic units that can perform actions, e.g., computation, communication, interaction, etc. •The ability to adapt at – individual level (modify agent ) and/or – group level (add/remove agent).
  • Designed by Gusz Eiben & Mark Hoogendoorn Types of adaptation • Evolutionary learning (EL): Changes at population level (assumed non-Lamarckian) • Lifetime learning (LL): Changes at agent level – Individual learning (IL): adaptation autonomously through a purely internal procedure – Social learning (SL): adaptation through interaction /communication
  • Designed by Gusz Eiben & Mark Hoogendoorn Taxonomy of adaptation Adaptation Evolutionary Learning Lifetime Learning Individual Learning Social Learning
  • Designed by Gusz Eiben & Mark Hoogendoorn Taxonomy of adaptation 2 Adaptation Evolutionary Learning Lifetime Learning Individual Learning Social Learning Learning Evolution
  • Designed by Gusz Eiben & Mark Hoogendoorn Adaptation ≠ operation • Operation: controller is being used – Sensory inputs  outputs (motor, comm. device) – Robot behavior changes, not the controller • Adaptation: controller is being changed – Present controller  new controller – Uses utility/reward/fitness info – It may require • One single robot – learning • More robots – evolution, social learning • Adaptation + operation = generate + test • Off-line (initial controller design, before start) vs. on-line (after start)
  • Designed by Gusz Eiben & Mark Hoogendoorn Genotype Developmental Engine(decoder) Genetic operators: mutation & xover Learning operators Robot behavior State of the environment Phenotype = controller Reward Fitness Selection operators
  • Designed by Gusz Eiben & Mark Hoogendoorn Genotype Developmental Engine(decoder) Genetic operators: mutation & xover Learning operators Robot behavior State of the environment Phenotype = controller Reward Fitness Selection operators
  • Designed by Gusz Eiben & Mark Hoogendoorn Genotype Developmental Engine(decoder) Genetic operators: mutation & xover Learning operators Robot behavior State of the environment Reward Fitness Selection operators Phenotype controllershape
  • Designed by Gusz Eiben & Mark Hoogendoorn Phenotype Genotype Developmental Engine(decoder) Genetic operators: mutation & xover Learning operators Robot behavior State of the environment Reward Fitness Selection operators controllershape
  • Designed by Gusz Eiben & Mark Hoogendoorn Evolutionary loop Genotype DevelopmentalEngine Genetic operators: mutation & xover Learning operator(s) Robot behavior Changes in environment Controller = phenotype Reward Fitness Selection operator(s)
  • Designed by Gusz Eiben & Mark Hoogendoorn Learning loop Genotype DevelopmentalEngine Genetic operators: mutation & xover Learning operator(s) Robot behavior Changes in environment Controller = phenotype Reward Fitness Selection operator(s)
  • Designed by Gusz Eiben & Mark Hoogendoorn ENVIRONMENTAGENT Reward r(t) State s(t) Action a(t)
  • Designed by Gusz Eiben & Mark Hoogendoorn Reinforcement learning Agent in situation/state st chooses action at World changes to situation/state st+1 Agent perceives situation st+1 and gets reward rt+1 Telling the agent what to do is its POLICY πt(s, a) = P r{at = a|st = s} Given the situation at time t is s, the policy gives the probability the agent’s action will be a. For example: πt(s, goforward) = 0.5, πt(s, gobackward) = 0.5. Reinforcement learning ⇒ Get/find/learn the policy
  • Designed by Gusz Eiben & Mark Hoogendoorn Further reading • Evert Haasdijk and A.E. Eiben and Alan F.T. Winfield, Individual Social and Evolutionary Adaptation in Collective Systems , Serge Kernbach (eds.) , Handbook of Collective Robotics , Pan Stanford , 2011