Evolution as a Tool for Understanding and Designing Collaborative Systems

Evolution as a Tool for Understanding and Designing Collaborative Systems Wilfried Elmenreich Mobile Systems Group/Lakeside Labs Institute for Networked and Embedded Systems Alpen-Adria Universität Klagenfurt

Overview
Cooperation theory
The evolutionary approach
Selection models
Agent behavior rules
A tool for evolving SOS
Application examples
Challenges

Cooperation Theory
Cooperation theory addresses the common tension between
what is good for the individual actor in the short run, and
what is good for the group in the long run
Typically employs game theory as the basis for analysis
Robert Axelrod published Evolution of Cooperation in 1984
Still we are not done with answering this question today...

Application Examples
A soccer team
Discipline, selfishness, assistance/cooperation…
Employees/company
„ Teamwork“ issue
Energy consumers and producers in the „Smart Grid“
Source: Wikimedia

Prisoner‘s Dilemma
Characterization of 1-move 2-person Prisoner’s Dilemma:
R: the reward for mutual cooperation
P: the punishment for mutual defection
T: payoff for unilateral defection
S: the sucker’s payoff
The Prisoner’s Dilemma is defined by T > R > P > S
Mutual cooperation should be in favor over coordinated alternation of unilateral cooperation: R > (T+S) / 2
R,R T,S S,T P,P

Prisoner‘s Dilemma Strategies
Always defect:
Best for individual actor in the short run
Bad for overall group
Always cooperate:
Good for group
Rewards antisocial behavior
A sophisticated strategy
Tit-for-Tat
Forgiveness concept
Retaliation concept

Public Goods Game
A social dilemma
Different story, same properties as Prisoneer‘s Dilemma
Typical much more than 2 players
Altruistic behavior best for group
Best to defect for an individual (Freerider)
Tragedy of the commons

Some questions
What is the most rational strategy?
What are the boundary conditions for a strategy to be optimal?
Why do real systems converge to equilibria with non-rational behavior?
Put your research topic here!

The Evolutionary Approach
Research the evolution of rules via Darwinian process
Model the social game (e.g. Public Goods Game) as the problem
Evolve the behavioral rules of the agents

Individual selection based on individual fitness
Select the soccer players that score the most goals
A: altruistic agent
S: selfish agent
A A A A A S S S S S S S S

Individual selection based on group fitness
Select the best performing agents from the best performing groups
A S A S S S A S A S S S S S S

Group selection based on group fitness
Select the best performing agents from the best performing groups
A S A S S S A S A A S A A S A

Kin selection
Hamilton’s rule of inclusive fitness: if r b > c then cooperate r relatedness b benefit c cost
r = probability that two individuals have homologous alleles identical by descent
J.B.S. Haldane: “I would lay down my life for two brothers or eight cousins.”

The ongoing discussion: Group vs. Kin selection
Social insects make a strong point for kin selection
Cooperation without being able to reproduce
But does not cover all evolved cooperations
Spatial models make a strong point for group selection
Group selection is especially in favor for areas with natural barriers
Analysis requires now a connection (network) model

Searching for the Rules
Simulation of target system as playground
Evolvable model of local behavior (e.g., fuzzy rules, ANN)
Define goal via fitness function (e.g., maximize throughput in a network)
Run evolutionary algorithm to derive local rules that fulfill the given goal
System model Goals (fitness function) Simulation of game Explore solutions Evaluate & Iterate Analyze results

Evolutionary Algorithm

Agent behavior to be evolved
Controls the agents of the SOS
Processes inputs (from sensors) and produces output (to actuators)
Must be evolvable
Mutation
Recombination
We cannot easily do this with an algorithm represented in C code…
Agent Control System „Agent‘s Brain“

Artificial Neural Networks
Each neuron sums up the weighted outputs of the other connected neurons
The output of the neuron is the result of an activation function (e.g. step, sigmoid function) applied to this sum
Neural networks are distinguished by their connection structure
Feed forward connections (layered)
Recursive (Ouput neurons feed back to input layer)
Fully meshed

Evolving Neural Networks 3.2 -1.2 3.2 3.2 -0.1 -4.2 0.2 0.0 3.5 -1.2 3.2 3.2 -0.1 -4.2 0.2 0.0 Mutation 0.0 -1.2 1.2 3.2 -0.1 1.2 1.2 0.0 3.5 2.2 3.2 3.2 -0.1 -4.2 0.2 0.5 Recombination 3.2 -1.2 3.2 3.2 -0.1 -4.2 0.2 0.0

A Framework for Evolutionary Design
FREVO (Framework for Evolutionary Design)
Modular Java tool allowing fast simulation and evolution
„ Frevo“ means also a hot, „boiling“ dance around here

Framework for Evolutionary Design
FREVO defines flexible components for
Controller representation
Problem specification
Optimizer

Giving FREVO a Problem
Basically, we need a simulation of the problem
Interface for input/output connections to the agents
E.g. for the public goods game:
Your input last round
Your revenue
Feedback from a simulation run -> fitness value

Example: Evolving Team Play in Robot Soccer
Simulated robot soccer teams
Goal evolve behavior for individual players
Team selection strategy
Not the top scorer but the winning teams are selected
This selection forces cooperation within team

Modeling of individual soccer players

Agent I/O by example of an ANN

Composite fitness function

Results (movie)
http://www.youtube.com/watch?v=cP035M_w82s

Evolution of cooperative behavior

Evolution of cooperative behavior
Public goods game model
6 players randomly selected from the pool
Neural network model for each player
Player decides how much money to contribute to a pot
Pot function: money is tripled and evenly distributed among all
Players have knowledge about their own and other‘s decisions from previous round

Freerider behavior evolves

A synergy effect
Make the pot function overproportional to the submitted money
E.g. by a square function: pot = c x 2
This significantly increases the motivation for cooperation
Does this immediately lead to cooperation?

Public Goods Game with Synergy Model
Still no cooperation?

Cooperation evolves in the long term
It needs some random fluctuations to set the coopertive strategy loose
Quickly converges to (partial) cooperative behavior

Not the end of the story…
Spatial model could enable group-selection-like evolution of altruism
But what spatial model?
2D von Neumann neighborhood
2D Moore neighborhood
Hexagon neighborhood
Hypercubes
Network models
Extending the model with team selection for agents could enable kin selection feature
At least as many questions

Challenges
Evolutionary modeling gives new insight to the problem
Answers the question: Why is a particular non-optimal behavior so prevalent in a population?
Evolutionary modeling also raises new questions
How should the agent‘s brain be structured?
How should the selection function be modeled?
What is an appropriate spatial model?

Visit us
(open source): www.frevotool.tk
Project MESON (Design Methods for Self-Organizing Systems): meson.lakeside-labs.com
Lakeside Labs Cluster www.lakeside-labs.com

http://demesos.blogspot.com	709
http://demesos.blogspot.com.au	35
http://demesos.blogspot.de	21
http://demesos.blogspot.co.uk	14
http://demesos.blogspot.in	7
http://demesos.blogspot.ca	6
http://demesos.blogspot.com.ar	5
http://demesos.blogspot.se	4
http://demesos.blogspot.com.br	4
http://demesos.blogspot.co.nz	3
http://demesos.blogspot.com.es	3
http://demesos.blogspot.it	2
http://www.demesos.blogspot.com	1

Evolution as a Tool for Understanding and Designing Collaborative Systems

by Wilfried Elmenreich on Oct 16, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

13 Embeds 814

Statistics

Evolution as a Tool for Understanding and Designing Collaborative Systems — Presentation Transcript