MAS course - Lect 9


Published on

Published in: Business, Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

MAS course - Lect 9

  1. 1. LECTURE 9: Cooperation in MAS (III): voting, coalition formation Artificial Intelligence II – Multi-Agent Systems Introduction to Multi-Agent Systems URV, Winter-Spring 2010
  2. 2. Outline of the lecture Other kinds of cooperation in MAS Voting mechanisms Properties of social choice rules Plurality / Binary / Borda / Condorcet Coalition formation Task allocation via coalition formation
  3. 3. Cooperation hierarchy [last lectures] MAS Independent Cooperative Self-interested Benevolent Discrete Emergent With Without communication - communication - Explicit Implicit Reactive systems Deliberative Negotiators Partial Global Auctions Planning Coalition Voting Contract Net formation
  4. 4. Distributed Decision Making - Voting Mechanism which chooses the outcome of a negotiation based on the inputs (votes) given by all agents to a set of competing options Truthful voters vote for the candidate they think is best Voters are not always truthful E.g. “useful vote” effect in politics: more than two candidates, and you are quite certain that your preferred candidate doesn’t have any chance
  5. 5. Basic elements Aim of the negotiation: rank feasible social outcomes based on the individual ranking of those outcomes by each agent A - set of n agents O - set of m feasible outcomes Each agent i has a preference relation <i : O x O, asymmetric and transitive
  6. 6. Social choice rule Input: the agents preference relations (<1, …, <n) Output: elements of O sorted according to the input - gives the social preference relation <* of the agent group In other words – creates ordering for the group of alternatives, so that the most (socially) preferred alternative is chosen
  7. 7. Desirable properties of the social choice rule (I) A social preference ordering <* should exist for all possible inputs <* should be defined for every pair of alternatives (o, o')∈O <* should be asymmetric and transitive over O The outcomes should be Pareto efficient: If ∀i ∈A, o <i o' then o <* o‘ Do not misorder the options if all agents agree
  8. 8. Desirable properties of the social choice rule (II) The scheme should be independent of irrelevant alternatives Taking out/Adding an irrelevant alternative should not affect the winner of the vote No agent i should be a dictator in the sense that o <i o' implies o <* o' regardless of the preferences of the other agents
  9. 9. Arrow's impossibility theorem-1951 No social choice rule satisfies all of the six conditions E.g. in many cases an irrelevant alternative affects the outcome of the voting protocol Some of the constraints must be relaxed We may not require <* to be always defined We may not require that <* is asymmetric and transitive Economy Nobel Prize 1972
  10. 10. Voting mechanisms Some voting mechanisms: Plurality protocol (highest number of votes wins) Binary protocol (series of votes of 2 options each) Borda protocol (sum of all the preferences of the agents) Condorcet protocol (pairwise comparison of options, given full preference ordering of each agent) All the protocols are problematic in one sense or another
  11. 11. Plurality protocol Each agent can give 1 vote to 1 of the alternatives The alternative with the highest number of votes wins
  12. 12. Problems of plurality voting (I) Useful vote Quite common effect of political polls 45 % option A, 40% option B, 15% option C The most socially preferred option is A => it should be the winner if all agents vote truthfully The agents that prefer option C know that they have no possibility to win, and most of them (80%) prefer option B to option A => 80% vote for B and 20% for A A: 48% B: 52%, and B finally wins
  13. 13. Problems of plurality voting (II) Huge effect of irrelevant alternative 2 basic options, A and B, around 50% each Another –very minoritary option- C appears, attracting 1% of the voters of A A: 49% B:50% C:1% and B wins Loss of information about the preferences of each voter Each agent can only give 1 vote, even if it considers 2-3-4 “good” alternatives
  14. 14. Problems of plurality voting (III) Strange effects 42% A C B D 26% B C D A 17% D C B A 15% C D B A A wins with 42% of the votes 58% of the voters preferred other options 58% consider A the worst option 100% consider C the 1st-2nd best option
  15. 15. Advantages of plurality voting Most simple voting mechanism Very efficient from the computational point of view Equality principle, as it preserves the idea of 1 agent = 1 vote
  16. 16. Binary protocol All the options are ordered and then evaluated in pairs (options 1 and 2, the winner with option 3, the winner with option 4, etc.) The option that wins the last evaluation is the overall winner win(o5, win (o4, win (o3, win(o2,o1))))
  17. 17. The ordering problem x > z > y (35%) y > x > z (33%) z > y > x (32%) Note that y is preferred to x (65-35), x is preferred to z (68-32), and z is preferred to y (67-33) win(x, win(y,z))=x win(y,win(x,z))=y win(z, win(x,y))=z The order of the pairings affects the outcome ! The voter organiser may influence the result The last options have more chances of winning
  18. 18. Another problematic example 35% of agents have preferences cfd fbfa 33% of agents have preferences afcfd fb 32% of agents have preferences bfafcfd Win(a,c)=a Win(a,b)=b Win(b,d)=d => d Wins • d was the worst alternative for 32% • d was not the best alternative for anyone • Everybody prefers c to d (!)
  19. 19. Summary of problems of binary voting Decisive role of the ordering of the alternatives An alternative x may win even if there is another alternative x’ which is preferred to x by all agents Alternatives may be misordered Temporal cost of the voting process (sequence of pairwise eliminative votes)
  20. 20. Borda protocol-1770 For each voter, we assign |O| points for the highest preference, |O|-1 points for the second, and so on Example: one agent considers the order ABCD A:4 points, B:3 points, C:2 points, D:1 point The points are added across the voters and the alternative with the highest count becomes the social choice [similar to Eurovision song contest]
  21. 21. Borda paradox If the worst alternative –d- is removed a>b>c>d a>b>c b>c>d>a b>c>a c>d>a>b c>a>b a>b>c>d a>b>c b>c>d>a b>c>a c>d>a>b c>a>b a>b>c>d a>b>c a=15, b=14, c=13 a=18, b=19, c=20, d=13
  22. 22. Inverted-order paradox 1. x>c>b>a 2. a>x>c>b 3. b>a>x>c Borda protocol with 4 alternatives 4. x>c>b>a x=22, a=17, b=16, c=15 5. a>x>c>b 6. b>a>x>c Order: xabc 7. x>c>b>a If we remove x: 1. c>b>a c=15, b=14, a=13 2. a>c>b 3. b>a>c The second one is c, not a 4. c>b>a 5. a>c>b 6. b>a>c 7. c>b>a
  23. 23. Problems of the Borda protocol Most computationally expensive The Eurovision Song Contest is endless … Eliminating one irrelevant alternative may totally change the outcome of the protocol Winner => Last Second worst => Winner Total order changes if options are removed one by one
  24. 24. Condorcet protocol-s.XVIII Each voter ranks the candidates in order of preference Each candidate is compared to each other If a candidate wins all the comparisons, it is the winner of the election In the event of a tie, use another resolution method (e.g. Borda count)
  25. 25. Example (I) Election of the capital city of Tennessee Everybody prefers to have the capital as close as possible
  26. 26. Example (II) Nashville wins
  27. 27. Problem of Condorcet method Possibility of circular ambiguities No alternative wins all other alternatives There are many ways to resolve them Keep the candidate that wins more matches (Copeland) Take into account the relative strengths of defeats (Minimax, Ranked Pairs, Schulze, …) Can look at the winning votes or at the winning margin
  28. 28. Example of influence of protocol (I) 45 ABCD, 25 CDBA, 20 DBCA, 10 DCBA Plurality protocol: A-45 B-0 C-25 D-30 A wins quite clearly, B has no votes However, note that 55% say that A is the worst option (!)
  29. 29. Example of influence of protocol (II) 45 ABCD, 25 CDBA, 20 DBCA, 10 DCBA Pairwise comparisons A loses with B,C and D B wins C, C wins D, D wins B Binary protocol A can never win (!) Order ABCD: D wins Order BDAC: C wins Order CDBA: B wins
  30. 30. Example of influence of protocol (III) 45 ABCD, 25 CDBA, 20 DBCA, 10 DCBA Condorcet protocol: no option beats all the others, and B/C/D tied with 2 victories Borda protocol A: 235 B: 265 C:260 D:240 B wins (although all the options get similar values) B is not the first option for anyone !!! B is a good consensual option (2nd for 65%, 3rd for 35%), whereas A was very problematic (45% very positive support, but 55% very negative support) Recall that B had 0 votes in the plurality protocol (!)
  31. 31. Insincere (Strategic) Voters If a self-interested agent can benefit from insincerely declaring his preferences, he will do so E.g. Suppose your choice will likely come in second place. If you rank the first choice of the rest of the group (insincerely) very low, you may lower that choice enough so yours is first However, knowledge of the agent’s true preferences is rarely available
  32. 32. Coalition formation
  33. 33. What is a Coalition? Coalitions are (temporary) collections of individuals working together for the purpose of achieving a task Coalition formation is the process whereby an agent decides to cooperate with other agents, because The task cannot be performed by a single agent The task can be performed more efficiently by several agents working together
  34. 34. Interesting aspects of coalitions Agents usually bring different, complementary capabilities to the coalition – they are not clones ! When the task is completed, the payoff is distributed, the coalition is disbanded and agents continue to pursue their own agendas
  35. 35. Issues in coalition formation Given a set of tasks and a set of agents, which coalitions should an agent attempt to form? What mechanism can an agent use for coalition formation? What guarantees regarding efficiency and quality can the mechanism provide? One a coalition has formed, how should its members go about the distribution of work/payoff?
  36. 36. Solution types Mechanisms for benevolent agents are usually much simpler than those for self- interested agents A centralised design of coalitions is usually much simpler to execute than a distributed one Coalition formation may be external or internal
  37. 37. External coalition formation By imposition: an external agency makes decisions Agents advertise skills (capabilities) and prices (cost) Requestor defines properties of coalition to the external agency External process computes optimal coalition
  38. 38. Internal coalition formation By self-organisation: coalitions are established by group interactions Multi-lateral negotiation Identification of tasks to be solved Can be static (given by the user at the beginning) or dynamic (generated by the own agents at run-time) Negotiation of outcomes (self-interested agents)
  39. 39. Coalition formation activities Coalition structure generation Optimizing coalition value Pooling the resources of the agents to maximize the coalition value Payoff distribution Deciding how to distribute the payoff between coalition members (equally, outputs, role)
  40. 40. Coalition structure generation Partition into exhaustive [disjoint] coalitions Given 3 agents {a1, a2, a3}, there are seven possible coalitions {a1}, {a2}, {a3}, {a1, a2}, {a1, a3}, {a2, a3}, {a1, a2, a3} and five coalition structures {a1, a2, a3}, {{a1}, {a2, a3}}, {{a2}, {a1, a3}}, {{a3}, {a1, a2}}, {{a1}, {a2}, {a3}} It may not be necessary or appropriate to generate all coalition structures in advance, depending on the application domain
  41. 41. Task allocation via coalition formation There is a set of tasks that have to be distributed among a group of agents Each task has a certain set of requirements, and each agent has a certain set of capabilities The aim is to find the optimal distribution of tasks to subgroups of agents
  42. 42. Definitions (I) A set of n agents: N={A1,A2,...,An} Each agent Ai has a positive vector of capabilities Bi=<bi1,...,bir> Each capability is a property that quantifies the agent’s ability in some aspect Capabilities may be expendable (e.g. amount of memory) or non-expendable (e.g. ability to perform an action) There is an evaluation function for each capability, that transforms it into monetary units
  43. 43. Definitions (II) Set of m independent tasks T={t1, …, tm} There is a vector of capabilities needed to perform each task tl, Bl={bl1, …, blr} The benefit gained from performing each task depends on the capabilities required for its performance To simplify the problem, it can be a linear function on the amount of resources
  44. 44. Simplifying conditions on coalitions A coalition is a group of agents that decide to cooperate to achieve a common task A coalition can work on a single task at a time Each agent only belongs to one coalition A coalition C has a vector of capabilities Bc (sum of the capabilities of the agents in the coalition) A coalition C can perform a task t iff for all 1<=i<=r bti <= bCi
  45. 45. Coalitional cost For each coalition C and specific task t, it is possible to calculate the coalitional value V, that measures the joint utility that the members of C can reach if they cooperate to satisfy t. This value depends on the capabilities contributed by the team members and the number of coalition members The coalitional cost c is the reciprocal of the coalitional value The aim is maximising the coalitional value (i.e. minimising the cost)
  46. 46. Task allocation process Heuristic: prefer small-sized coalitions, to reduce communication and coordination costs Maximum coalitional size allowed: k Initial coalitional state: n single agents Step by step, formation of coalitions 1 new coalition in each iteration When an agent joins a coalition, it quits the coalition formation process
  47. 47. Algorithm of task allocation Preliminary stage: all possible coalitions are distributed among all agents Collective, not centralised process Iterative stage: Coalitional values (for each pair –coalition,task-) are (re)calculated One coalition C is formed Agents in coalition C quit the coalition formation process
  48. 48. Stage 1: initial calculations of agent Ai (I) Li = empty_set This list will contain, at the end of stage 1, the coalitions whose value agent i will have to calculate Pi = all permutations up to k agents containing agent i Those are all the possible coalitions in which agent i could participate
  49. 49. Stage 1: initial calculations of agent Ai (II) For each permutation in Pi do Contact agent j, which is in the permutation (i.e. j is in some possible coalition(s) with i) In the first contact, request capabilities Bj Commit to calculate the value of some of the coalitions including i and j = Sij Pi = Pi – Sij Li = Li + Sij End_for [at the same time] For each agent k that contacts i, Pi = Pi – Ski
  50. 50. Example (I) 4 agents: A1, A2, A3, A4 Maximum number of coalition members: 3 (k) Possible coalitions for A1 (P1): {A1}, {A1, A2}, {A1, A3}, {A1, A4}, {A1, A2, A3}, {A1, A2, A4}, {A1, A3, A4} Coalitions with A2= {A1, A2}, {A1, A2, A3}, {A1, A2, A4}
  51. 51. Example (II) A1 contacts A2, requests B2 and commits to calculate S12 = { {A1, A2}, {A1, A2, A3} } The coalitions in S12 are added to L1 and deleted from P1 Agent 2 deletes S12 from P2 [ Later, delete from P1 the coalitions commited by agent 2, S21 ] Similar process with agents A3 and A4
  52. 52. End of stage 1 All the agents have (collectively) distributed all possible coalitions Each agent i has a list Li of coalitions whose value it has to (repeatedly) calculate
  53. 53. Stage 2a: calculate coalitional values (I) Licr = Li (in the first iteration) For each coalition C in Licr do Ec = empty set Bc = sum(Bk) for all agents k in C [potential capability of coalition C] For each pending task tj, do Check if coalition C can do task tj (Bc >= Bj) If it can, calculate the net benefit of tj for C, ej ej = sum of market value of capabilities needed in tj – sum of the capabilities costs – internal coordination costs Ec = Ec + {(tj, ej)} End for
  54. 54. Stage 2a: calculate coalitional values (II) Ec contains the benefits that coalition C can gain from each of the tasks it is capable of performing (tcbest, Vc) = max-value(Ec) Coalitional value of C and its related task cc = 1 / Vc Coalitional cost of C End for At the end of this step, each agent has calculated the best coalitional value/cost for each coalition, and it knows the most profitable task for each coalition
  55. 55. Example (IIIa) Following the previous example, assume that L1 = { {A1, A2}, {A1, A3}, {A1, A2, A3}, {A1, A3, A4}} A1 calculates the best value (and cost) of each coalition in L1 V12, V13, V123, V134 c12, c13, c123, c134
  56. 56. Example (IIIb) How is V134 calculated? Assume there are 3 possible tasks The coalition {1,3,4} can make task 1, with a net benefit 50 The coalition {1,3,4} can’t make task 2 The coalition {1,3,4} can make task 3, with a net benefit 35 Thus, the coalitional value of the coalition {1,3,4} (V134) is 50, and its coalitional cost c134 is 0.02 If this coalition is formed in this iteration, it will be assigned task 1
  57. 57. Stage 2b: form one coalition (I) The weight wi of a coalition Ci is defined as wi = ci / |Ci| Ci = coalition of Li with minimum weight Best coalition for agent i Each agent announces publicly the weight of its best coalition wlow = minimum of the announced weights Best coalition for all agents [Clow, tlowbest]
  58. 58. Stage 2b: form one coalition (II) If I am an agent in Clow, join the other agents in Clow to perform task tlowbest Delete the members of Clow from the list of candidates to future coalitions Li = Li – coalitions with agents in Clow T = T – tlowbest Licr = coalitions of Li whose value has to be recalculated Those coalitions whose previous best value was making tlowbest
  59. 59. Example (IV) Following the previous example, A1 calculates the weights of its coalitions e.g. w12= c12 / 2 ; w134= c134 / 3 A1 sends its best weight (e.g. w12), and receives the best weigths calculated by agents A2, A3 and A4 After that, all agents know who has found the best coalition e.g. w12 is the minimum => the best coalition is {A1,A2}
  60. 60. Example (V) Agents A1 and A2 join together to perform the task t12best (A1 knows which task it is) These agents quit the coalition formation process In this example, in the next iteration, agents A3 and A4 would calculate their possibilities of coalition, with respect to the pending tasks, and would decide whether to join or not
  61. 61. Ideas for the practical exercise (I) Voting Different firemen could vote which fire to fight if they want to attack always together to join forces to stop fires as soon as possible The vote of each firetruck could depen on its actual position, the position of the fires, its actual state (idle, in route to a fire, attacking a fire), its charge of fuel, etc.
  62. 62. Ideas for the practical exercise (II) Coalition formation (I) The fires on the city could be the pending tasks. The firetrucks could decide how to join in subgroups (coalitions) to fight each of the fires Same ideas for policemen/roads, or ambulances/buildings with wounded people
  63. 63. Ideas for the practical exercise (III) Coalition formation (II) Another idea could be to make a coalition of some firetrucks, police cars and ambulances for each of the buildings on fire. The working groups of the course are encouraged to use different coordination mechanisms for the different elements of the problem (e.g. to use a Contract Net protocol to assign ambulances and a voting mechanism to assign firetrucks).
  64. 64. Readings for this week Article “Methods for task allocation via agent coalition formation”. Shehory, Kraus. Chapters 12 and 13 of the book by M.Wooldridge “An introduction to MultiAgent Systems” (2nd edition). Chapter 9 of the book by M.Fasli “Agent technology for e-commerce”.