Evolution as a Tool for Understanding and Designing Collaborative Systems


Published on

Keynote talk by Wilfried Elmenreich at PRO-VE 2011:

Self-organizing phenomena can be found in many social systems, either forcing collaboration or destroying it. Typically, these properties have not been designed by a central ruler but evolved over time. While it is straightforward to find examples in many social systems, finding the appropriate interaction rules to design such systems from scratch is difficult due to the unpredictable or counterintuitive nature of such emergent and complex systems. Therefore, we propose evolutionary models to examine and extrapolate the effect of particular collaboration rules. Evolution, in this context, does not replace the work of analyzing complex social systems, but complements existing techniques of simulation, modeling, and game theory in order to lead for a new understanding of interrelations in collaborative systems.

Published in: Education, Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Evolution as a Tool for Understanding and Designing Collaborative Systems

  1. 1. Evolution as a Tool for Understanding and Designing Collaborative Systems Wilfried Elmenreich Mobile Systems Group/Lakeside Labs Institute for Networked and Embedded Systems Alpen-Adria Universität Klagenfurt
  2. 2. Overview <ul><li>Cooperation theory </li></ul><ul><li>The evolutionary approach </li></ul><ul><li>Selection models </li></ul><ul><li>Agent behavior rules </li></ul><ul><li>A tool for evolving SOS </li></ul><ul><li>Application examples </li></ul><ul><li>Challenges </li></ul>
  3. 3. Cooperation Theory <ul><li>Cooperation theory addresses the common tension between </li></ul><ul><ul><li>what is good for the individual actor in the short run, and </li></ul></ul><ul><ul><li>what is good for the group in the long run </li></ul></ul><ul><li>Typically employs game theory as the basis for analysis </li></ul><ul><li>Robert Axelrod published Evolution of Cooperation in 1984 </li></ul><ul><ul><li>Still we are not done with answering this question today... </li></ul></ul>
  4. 4. Application Examples <ul><li>A soccer team </li></ul><ul><ul><li>Discipline, selfishness, assistance/cooperation… </li></ul></ul><ul><li>Employees/company </li></ul><ul><ul><li>„ Teamwork“ issue </li></ul></ul><ul><li>Energy consumers and producers in the „Smart Grid“ </li></ul>Source: Wikimedia
  5. 5. Prisoner‘s Dilemma <ul><li>Characterization of 1-move 2-person Prisoner’s Dilemma: </li></ul><ul><ul><ul><li>R: the reward for mutual cooperation </li></ul></ul></ul><ul><ul><ul><li>P: the punishment for mutual defection </li></ul></ul></ul><ul><ul><ul><li>T: payoff for unilateral defection </li></ul></ul></ul><ul><ul><ul><li>S: the sucker’s payoff </li></ul></ul></ul><ul><li>The Prisoner’s Dilemma is defined by T > R > P > S </li></ul><ul><li>Mutual cooperation should be in favor over coordinated alternation of unilateral cooperation: R > (T+S) / 2 </li></ul>R,R T,S S,T P,P
  6. 6. Prisoner‘s Dilemma Strategies <ul><li>Always defect: </li></ul><ul><ul><li>Best for individual actor in the short run </li></ul></ul><ul><ul><li>Bad for overall group </li></ul></ul><ul><li>Always cooperate: </li></ul><ul><ul><li>Good for group </li></ul></ul><ul><ul><li>Rewards antisocial behavior </li></ul></ul><ul><li>A sophisticated strategy </li></ul><ul><ul><li>Tit-for-Tat </li></ul></ul><ul><ul><li>Forgiveness concept </li></ul></ul><ul><ul><li>Retaliation concept </li></ul></ul>
  7. 7. Public Goods Game <ul><li>A social dilemma </li></ul><ul><li>Different story, same properties as Prisoneer‘s Dilemma </li></ul><ul><li>Typical much more than 2 players </li></ul><ul><li>Altruistic behavior best for group </li></ul><ul><li>Best to defect for an individual (Freerider) </li></ul><ul><ul><li>Tragedy of the commons </li></ul></ul>
  8. 8. Some questions <ul><li>What is the most rational strategy? </li></ul><ul><li>What are the boundary conditions for a strategy to be optimal? </li></ul><ul><li>Why do real systems converge to equilibria with non-rational behavior? </li></ul><ul><li>Put your research topic here! </li></ul>
  9. 9. The Evolutionary Approach <ul><li>Research the evolution of rules via Darwinian process </li></ul><ul><li>Model the social game (e.g. Public Goods Game) as the problem </li></ul><ul><li>Evolve the behavioral rules of the agents </li></ul>
  10. 10. Individual selection based on individual fitness <ul><li>Select the soccer players that score the most goals </li></ul><ul><li>A: altruistic agent </li></ul><ul><li>S: selfish agent </li></ul>A A A A A S S S S S S S S
  11. 11. Individual selection based on group fitness <ul><li>Select the best performing agents from the best performing groups </li></ul>A S A S S S A S A S S S S S S
  12. 12. Group selection based on group fitness <ul><li>Select the best performing agents from the best performing groups </li></ul>A S A S S S A S A A S A A S A
  13. 13. Kin selection <ul><li>Hamilton’s rule of inclusive fitness: if r b > c then cooperate r relatedness b benefit c cost </li></ul><ul><li>r = probability that two individuals have homologous alleles identical by descent </li></ul><ul><li>J.B.S. Haldane: “I would lay down my life for two brothers or eight cousins.” </li></ul>
  14. 14. The ongoing discussion: Group vs. Kin selection <ul><li>Social insects make a strong point for kin selection </li></ul><ul><ul><li>Cooperation without being able to reproduce </li></ul></ul><ul><ul><li>But does not cover all evolved cooperations </li></ul></ul><ul><li>Spatial models make a strong point for group selection </li></ul><ul><ul><li>Group selection is especially in favor for areas with natural barriers </li></ul></ul><ul><ul><li>Analysis requires now a connection (network) model </li></ul></ul>
  15. 15. Searching for the Rules <ul><li>Simulation of target system as playground </li></ul><ul><li>Evolvable model of local behavior (e.g., fuzzy rules, ANN) </li></ul><ul><li>Define goal via fitness function (e.g., maximize throughput in a network) </li></ul><ul><li>Run evolutionary algorithm to derive local rules that fulfill the given goal </li></ul>System model Goals (fitness function) Simulation of game Explore solutions Evaluate & Iterate Analyze results
  16. 16. Evolutionary Algorithm
  17. 17. Agent behavior to be evolved <ul><li>Controls the agents of the SOS </li></ul><ul><li>Processes inputs (from sensors) and produces output (to actuators) </li></ul><ul><li>Must be evolvable </li></ul><ul><ul><li>Mutation </li></ul></ul><ul><ul><li>Recombination </li></ul></ul><ul><li>We cannot easily do this with an algorithm represented in C code… </li></ul>Agent Control System „Agent‘s Brain“
  18. 18. Artificial Neural Networks <ul><li>Each neuron sums up the weighted outputs of the other connected neurons </li></ul><ul><li>The output of the neuron is the result of an activation function (e.g. step, sigmoid function) applied to this sum </li></ul><ul><li>Neural networks are distinguished by their connection structure </li></ul><ul><ul><li>Feed forward connections (layered) </li></ul></ul><ul><ul><li>Recursive (Ouput neurons feed back to input layer) </li></ul></ul><ul><ul><li>Fully meshed </li></ul></ul>
  19. 19. Evolving Neural Networks 3.2 -1.2 3.2 3.2 -0.1 -4.2 0.2 0.0 3.5 -1.2 3.2 3.2 -0.1 -4.2 0.2 0.0 Mutation 0.0 -1.2 1.2 3.2 -0.1 1.2 1.2 0.0 3.5 2.2 3.2 3.2 -0.1 -4.2 0.2 0.5 Recombination 3.2 -1.2 3.2 3.2 -0.1 -4.2 0.2 0.0
  20. 20. A Framework for Evolutionary Design <ul><li>FREVO (Framework for Evolutionary Design) </li></ul><ul><li>Modular Java tool allowing fast simulation and evolution </li></ul><ul><li>„ Frevo“ means also a hot, „boiling“ dance around here </li></ul>
  21. 21. Framework for Evolutionary Design <ul><li>FREVO defines flexible components for </li></ul><ul><ul><li>Controller representation </li></ul></ul><ul><ul><li>Problem specification </li></ul></ul><ul><ul><li>Optimizer </li></ul></ul>
  22. 22. Giving FREVO a Problem <ul><li>Basically, we need a simulation of the problem </li></ul><ul><li>Interface for input/output connections to the agents </li></ul><ul><ul><li>E.g. for the public goods game: </li></ul></ul><ul><ul><li>Your input last round </li></ul></ul><ul><ul><li>Your revenue </li></ul></ul><ul><li>Feedback from a simulation run -> fitness value </li></ul>
  23. 23. Example: Evolving Team Play in Robot Soccer <ul><li>Simulated robot soccer teams </li></ul><ul><li>Goal evolve behavior for individual players </li></ul><ul><li>Team selection strategy </li></ul><ul><ul><li>Not the top scorer but the winning teams are selected </li></ul></ul><ul><ul><li>This selection forces cooperation within team </li></ul></ul>
  24. 24. Modeling of individual soccer players
  25. 25. Agent I/O by example of an ANN
  26. 26. Composite fitness function
  27. 27. Results (movie) <ul><li>http://www.youtube.com/watch?v=cP035M_w82s </li></ul>
  28. 28. Evolution of cooperative behavior
  29. 29. Evolution of cooperative behavior <ul><li>Public goods game model </li></ul><ul><li>6 players randomly selected from the pool </li></ul><ul><li>Neural network model for each player </li></ul><ul><li>Player decides how much money to contribute to a pot </li></ul><ul><li>Pot function: money is tripled and evenly distributed among all </li></ul><ul><li>Players have knowledge about their own and other‘s decisions from previous round </li></ul>
  30. 30. Freerider behavior evolves
  31. 31. A synergy effect <ul><li>Make the pot function overproportional to the submitted money </li></ul><ul><li>E.g. by a square function: pot = c x 2 </li></ul><ul><li>This significantly increases the motivation for cooperation </li></ul><ul><li>Does this immediately lead to cooperation? </li></ul>
  32. 32. Public Goods Game with Synergy Model <ul><li>Still no cooperation? </li></ul>
  33. 33. Cooperation evolves in the long term <ul><li>It needs some random fluctuations to set the coopertive strategy loose </li></ul><ul><li>Quickly converges to (partial) cooperative behavior </li></ul>
  34. 34. Not the end of the story… <ul><li>Spatial model could enable group-selection-like evolution of altruism </li></ul><ul><ul><li>But what spatial model? </li></ul></ul><ul><ul><ul><li>2D von Neumann neighborhood </li></ul></ul></ul><ul><ul><ul><li>2D Moore neighborhood </li></ul></ul></ul><ul><ul><ul><li>Hexagon neighborhood </li></ul></ul></ul><ul><ul><ul><li>Hypercubes </li></ul></ul></ul><ul><ul><ul><li>Network models </li></ul></ul></ul><ul><li>Extending the model with team selection for agents could enable kin selection feature </li></ul><ul><ul><li>At least as many questions </li></ul></ul>
  35. 35. Challenges <ul><li>Evolutionary modeling gives new insight to the problem </li></ul><ul><ul><li>Answers the question: Why is a particular non-optimal behavior so prevalent in a population? </li></ul></ul><ul><li>Evolutionary modeling also raises new questions </li></ul><ul><ul><li>How should the agent‘s brain be structured? </li></ul></ul><ul><ul><li>How should the selection function be modeled? </li></ul></ul><ul><ul><li>What is an appropriate spatial model? </li></ul></ul>
  36. 36. Visit us <ul><li>(open source): www.frevotool.tk </li></ul><ul><li>Project MESON (Design Methods for Self-Organizing Systems): meson.lakeside-labs.com </li></ul><ul><li>Lakeside Labs Cluster www.lakeside-labs.com </li></ul>