Published on

Coordination on MAS (PGP,Contract Net)

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. LECTURE 7: Cooperation in MAS (I) Artificial Intelligence II – Multi-Agent Systems Introduction to Multi-Agent Systems URV, Winter–Spring 2010
  2. 2. Outline of the lecture Coordination Kinds of cooperation in MAS Emergent cooperation Cooperation with explicit communication Deliberative Partial Global Planning Negotiation Contract Net Auctions [specific lecture]
  3. 3. What is Coordination? “Coordination is the process of managing interdependencies between activities ” - Malone and Crowston, 1991 Coordination problems occur when: • An agent has a choice in its actions within some task, and the choice affects its and other agents’ performance • The order in which actions are carried out affects performance • The time at which actions are carried out affects performance
  4. 4. Subproblem Interdependencies (I) Subproblems are the same/overlapping, but different agents have either alternative methods or data that can be used to generate a solution Subproblems are part of a larger problem in which a solution to the larger problem requires that certain constraints exist among the solutions to its subproblems
  5. 5. Subproblem Interdependencies (II) It is not possible to decompose the problem into a set of subproblems such that there is a perfect fit between the location of information, expertise, processing, and communication capabilities in the agent network and the computational needs for effectively solving each subproblem Contention for resources among the subproblems
  6. 6. Implications of subproblem interdependencies It may be impossible to completely solve one subproblem without first partially solving another subproblem Solving or partially solving one subproblem may simply make it easier to solve another subproblem Knowing the solution to one subproblem may obviate the need to solve another How an agent orders which subgoals to do and when to communicate the (partial) results of solving a subgoal can significantly affect global performance
  7. 7. Another vision of coordination Deciding for each agent in the context of other agent activities What activities it should do, when it should do them and how it should do them (planning, scheduling) What it should communicate, when it should communicate and to whom (cooperation) Domain information and Control information Communication and scheduling of activities are intimately connected This is a highly complex computational problem, especially if optimal solutions are required
  8. 8. Many approaches to coordination Type of information available about static and dynamic behaviour of agents Cost of acquiring current state of other agents Cost of finding out the abilities of other agents Importance of optimal solution Cost of computing coordination decisions Implications of generating non-optimal coordination Real-time requirements How long you have to make a decision There is no one best approach to Coordination Complex Multi-attributed Optimization Problem
  9. 9. Cooperation hierarchy (Franklin) MAS Independent Cooperative Self-interested Benevolent Discrete Emergent With Without communication - communication - Explicit Implicit Deliberative Negotiators
  10. 10. Benevolent Agents If we “own” the whole system, we can design agents to help each other whenever asked In this case, we can assume agents are benevolent: our best interest is their best interest Problem-solving in benevolent systems is cooperative distributed problem solving Benevolence simplifies the system design task enormously! [ Example: practical exercise]
  11. 11. Self-Interested Agents If agents represent individuals or organizations, then we cannot make the benevolence assumption E.g. buyers/sellers in e-commerce Agents will be assumed to act to further their own interests, possibly at the expense of others Potential for conflict It may complicate the design task enormously!
  12. 12. Cooperation hierarchy (Franklin) MAS Independent Cooperative Self-interested Benevolent Discrete Emergent With Without communication - communication - Explicit Implicit Deliberative Negotiators
  13. 13. Discrete MAS Independent agents Each agent pursues its own agenda The agendas of the agents bear no relation to one another For example, one agent can filter e-mail while another one gathers information from the Web No cooperation
  14. 14. MAS with emergent behaviour Agents can cooperate with no intention of doing so The system can exhibit high-level, complex, intelligent, coordinated behaviour without any designed coordination mechanisms, just as a side effect of the interactions among agents Example [recall reactive architectures] Puck gathering robots – Beckers
  15. 15. Puck gathering robots World with a set of pucks Set of autonomous, independent robots, that can pick up pucks and drop them A simple rule-based individual behaviour of each robot, without any communication or coordination with the other robots, can lead to a complex global system behaviour (putting all the pucks together in a pile)
  16. 16. Behavioural rules (I) Rule 1 – Pick up pucks If (there is not a puck in the gripper) & (there is a puck ahead) then take the puck in the gripper Rule 2 – Drop pucks together If (there is one puck in the gripper) & (there is a puck ahead) then drop the puck, go backward for one second and turn at a random angle
  17. 17. Behavioural rules (II) Rule 3 – Exploration If there are no pucks ahead then go forward Rule 4 – Avoid obstacles If there is an obstacle (wall or other robot) ahead then avoid the obstacle (turn at a random angle and go forward)
  18. 18. Important Factors Agents Don't need to know about each other, don’t communicate with each other Don't have special roles Loosing a few doesn't matter Environment Acts as a communication mechanism Is affected by the actions of all individuals Result: surprisingly coherent group (coordinated) action
  19. 19. Subsumption as Coordination Robotics as a Multi-Agent System Brooks Subsumption Architecture: Layers of controllers Each layer creates a competence Higher layers subsume lower layers Can be seen, at a high level of abstraction, as coordination of autonomous entities Each layer takes its own decisions
  20. 20. Intelligence as emergent behaviour Basic element of human intelligence: neural activity Neurons: very small element with very little computational power The interaction between an enormous amount of neurons leads to human-level complex thought patterns and intelligence !
  21. 21. Cooperation hierarchy (Franklin) MAS Independent Cooperative Self-interested Benevolent Discrete Emergent With Without communication - communication - Explicit Implicit Deliberative Negotiators
  22. 22. Cooperative agents The agendas of the agents include cooperating with other agents in the system in some way Explicitly: intentional sending and receiving of communicative signals (e.g. via a common blackboard or via messages) Implicitly: without explicit messages (e.g. observing and reacting to the behaviour of the other agents of the system)
  23. 23. Cooperation hierarchy (Franklin) MAS Independent Cooperative Self-interested Benevolent Discrete Emergent With Without communication - communication - Explicit Implicit Deliberative Negotiators
  24. 24. Deliberative agents Agents with inference and planning capabilities Some kind of explicit distributed planning mechanism, based on information exchange, addressed to solve collectively a given problem
  25. 25. Partial Global Planning (PGP) – Durfee, Lesser Distributed planning technique Integrates planning and execution Not the usual Planning-Scheduling-Action cycle Dynamic domains with unpredictable, unreliable information The tasks are inherently distributed; each agent performs its own task Initially applied in the Distributed Vehicle Monitoring (DVM) problem, then extended to be domain independent
  26. 26. Distributed Vehicle Monitoring
  27. 27. Goal in the DVM problem The agents are not aware of the global state of the system; however, there is a common goal: converge on a consistent map of vehicle movements by integrating the partial tracks formed by different agents into a single complete map or into a consistent set of local maps distributed among agents Coordination by means of partial plans exchange
  28. 28. Difficulties in DVM (I) The data sensed in an area cannot be exhaustively processed in a timely manner Huge volume of incoming data Many noisy data generated by sensors/environment, which should not be processed Correlations between data sensed in nearby locations provide constraints on whether/how that data should be processed
  29. 29. Difficulties in DVM (II) Sensor overlap implies possible processing duplication The same data should not be processed by different agents Sensing demands in an area vary heavily dynamically The allocation of work should be done in a very dynamic way, depending on the workload of each agent at each moment
  30. 30. Partial Global Planning phases 1. Create local plans 2. Exchange local plans 3. Generate Partial Global Plans 4. Optimize Partial Global Plans
  31. 31. PGP steps (I) 1- Create the local plan of each agent Each agent represents its own expected activity (to solve its assigned tasks) with a local (tentative) plan, at two levels: higher level – most important steps (actions) to be followed to solve the problem, abstract plan lower level –it specifies primitive operations to achieve the next step in the abstract plan; as the plan is executed, new details are added incrementally
  32. 32. Schematic view of a local plan Length1 Length2 A4 Local Plan (Actions) A1 A2 A3 A5 O1 O2 O3 Next operations (short term details)
  33. 33. Local plan characteristics Local plans may involve alternative actions depending on the results of previous actions and changes in the environment [conditional plans] See actions A4 and A5 in the previous slide They have to be dynamically modifiable in an easy way
  34. 34. Plan components Name Creation time Set of objectives to achieve List of planned actions for data processing Major steps in the plan Set of primitive problem-solving operations (short-term details) Predictions of how long each action will take and the expected outcome of each action Rating (plan importance)
  35. 35. Basic components of an action Preconditions for the action Results of the action Set of data to be processed by the action Set of procedures to be applied to the data Estimated start time of the action Estimated end time of the action Estimated degree of confidence in the result
  36. 36. Ordering actions within a plan Prefer actions that concurrently achieve multiple goals Prefer actions expected to require less resources (especially time) Prefer actions that will strongly verify or refute that some goals are worth pursuing E.g. analysis that confirms that a signal follows a vehicle trace detected by a neighbour agent
  37. 37. PGP steps (II) 2- Exchange plans The agents exchange information about their local plans with other agents (usually high-level information) Goals Long-term strategy Plan rating
  38. 38. Distribution of local plans Each agent must have knowledge about the MAS organisational structure, so that it can infer the role of each agent in the problem solving process and decide which information to send to which agents It would be very expensive and inefficient to send all the local plans to all the agents in the system !!! E.g. a police car probably doesn’t need to exchange its plans with all the fire trucks and ambulances
  39. 39. Nodes meta-level organisation Information that each node must know to infer the organisational structure Nodes it has authority over Nodes that have authority over it Nodes with equal authority
  40. 40. Coordination possibilities Centralised coordination A node has authority over all other nodes Hierarchical coordination Each node has 1 “boss” and some “subordinate” nodes Lateral coordination All nodes have the same authority
  41. 41. PGP steps (III) 3- Generate Partial Global Plans (PGPs) Each agent models the collective activity of the system, by combining the received local partial plans into a Partial Global Plan Check dependencies between the received information and its own local plan Identify when the goals of one or more agents can be considered subgoals of a single global goal: partial global goal Identify opportunities to improve coordination E.g. two agents having to solve the same subproblem
  42. 42. Components of a PGP (I) Partial global goal: final aim of the global plan Plan activity map: plan actions to be executed concurrently by itself and the other agents, including costs and expected results of actions Initially, it will contain the union of all the actions of all local plans
  43. 43. Criteria for rating the actions in the Plan Activity Map The action extends a partial result E.g. vehicle tracking hypothesis The action produces a partial result that might help some other agents in forming partial results How long the action is expected to take
  44. 44. Components of a PGP (II) From the modified Plan Activity Map, the agent builds a Solution Construction Graph: how the agents should interact, including specifications about what partial results to exchange when to exchange them who to exchange them with It must take into account the estimated time of the actions, the results they will provide, etc.
  45. 45. PGP steps (IV) 4- Optimize Partial Global Plans Each agent has a Planner Module, especialised in analyzing the received information to detect if there are several agents working on the same goal. This information is put in the Plan Activity Map, along with the expected future behaviour and expected results of the other agents
  46. 46. Optimizing Plans Local plan + Plan Activity Map => New modified Local Plan Optimized using the updated knowledge of the system Solution Construction Graph Concrete details about when to send particular results to specific agents in the future
  47. 47. Possible optimizations of Local Plan Task reordering Change the order of the actions in the plan Task reallocation Move some actions to nodes without assigned work Weight of authority Change local plan according to the decisions of the nodes that have more authority This information is obtained by analyzing the local plans of nodes with higher authority
  48. 48. Benefits of PGP Systems with highly dynamic behaviour All plans can be adapted to dynamic changes in the environment => flexibility However, if an agent changes its local plan, it has to inform other agents (e.g. those that were waiting for a partial result) Efficiency If different agents work on the same/similar subproblems, they will notice that fact in their local plans and reassign their tasks appropriately
  49. 49. Negotiation techniques Cooperation mechanisms with explicit information exchange in which there is some sort of competition between the agents Auctions [specific lecture] English / Dutch / Vickrey / FPSB Multi-attribute auctions Combinatorial auctions Contract Net Others …
  50. 50. Task Sharing and Result Sharing Two main modes of cooperative problem solving: task sharing: activities are distributed among the agents of the system result sharing: information (partial or final results) is distributed
  51. 51. The Contract Net A well known task-sharing protocol for task allocation is the Contract Net: 1. Recognition 2. Announcement 3. Bidding 4. Awarding 5. Expediting [More general and detailed presentation than in the lecture about communication protocols]
  52. 52. 1-Recognition In this stage, an agent recognizes it has a problem it wants help with An agent has a goal, and either… realizes it cannot achieve the goal in isolation — does not have capability realizes it would prefer not to achieve the goal in isolation (typically because of solution quality, deadline, use of resources, etc.)
  53. 53. 2-Announcement In this stage, the agent with the task sends out an announcement of the task which includes a specification of the task to be achieved Specification must encode: Description of the task itself Any constraints (e.g. deadlines, quality constraints) Meta-task information (e.g. preference on attributes) The announcement is then broadcast
  54. 54. 3-Bidding Agents that receive the announcement decide for themselves whether they wish to bid for the task Factors: agent must decide whether it is capable of expediting task agent must evaluate the cost of making the task and the benefits it can get from making it If an agent chooses to bid, then it submits a tender, detailing the conditions on which it can execute the task
  55. 55. 4-Awarding The agent that sent the task announcement must choose between bids & decide who to “award the contract” to The result of this process is communicated to the agents that submitted a bid
  56. 56. 5-Expediting The successful contractor then expedites the task That may involve generating further manager-contractor relationships: sub- contracting
  57. 57. Contract Net The collection of nodes is the “contract net” Each node on the network can, at different times or for different tasks, be a manager or a contractor When a node gets a composite task (or for any reason can’t solve its present task), it breaks it into subtasks (if possible) and announces them (acting as a manager), receives bids from potential contractors, and then awards the job
  58. 58. Issues for Implementing Contract Net How to… … specify tasks? … specify quality of service? … select between competing offers? … differentiate between offers based on multiple criteria?
  59. 59. Node Issues Task Announcement Task Announcement Manager
  60. 60. Idle Node Listening to Task Announcements Manager Potential Contractor Manager Manager
  61. 61. Node Submitting a Bid Bid Manager Potential Contractor
  62. 62. Manager listening to bids Bids Potential Manager Contractor Potential Contractor
  63. 63. Manager Making an Award Award Manager Contractor
  64. 64. Contract Established Contract Manager Contractor
  65. 65. FIPA-Contract Net protocol
  66. 66. Contract Net node modules Local database Knowledge base, info. on the state of negotiations and the state of the solution of tasks Interface module Sends/receives messages, deals with the communication with the other nodes Task processor Executes the tasks assigned to the node Contract processor Studies new offered tasks, submits bids, formalizes contracts
  67. 67. Domain-Specific Evaluation Task announcement message prompts potential contractors to use domain specific task evaluation procedures E.g. identifying important attributes There is deliberation going on, not just selection — perhaps no tasks are suitable at present Manager considers the submitted bids using a domain specific bid evaluation procedure
  68. 68. Efficiency Modifications Focused addressing — when general broadcast isn’t required Agents could automatically learn which are the most appropriate nodes for common tasks Directed contracts — when manager already knows which node is appropriate For instance when a very similar task has already been done in the past The nodes can make proactive offers to potential managers of the kind of tasks they are able to execute
  69. 69. FIPA- Iterated Contract Net protocol
  70. 70. Features of Contract Net Protocol Two-way dynamic transfer of information Mutual selection Bidders select from among task announcements Managers select from among bids Local evaluation Preserving autonomony and private information of agents
  71. 71. Suitable Applications Hierarchy of Tasks Subtasks are large enough (and it’s worthwhile to spend effort to distribute them wisely) Primary concerns are distributed control, achieving reliability, avoiding bottlenecks
  72. 72. Limitations Other stages of distributed problem solving are non-trivial: Problem Decomposition Solution Synthesis Computational overhead Messages Time – deliberation, analyze offer/bid, wait for decisions
  73. 73. Ideas for the practical exercise (I) Emergent cooperation Different firemen could be made to fight together a certain fire, without explicit coordination mechanisms E.g. each fire truck goes to the nearest fire Reactive agents, rule-based behaviour of each fireman Different police cars could be working on the same road blocking
  74. 74. Ideas for the practical exercise (II) Explicit cooperation Exchange of messages between firemen/policemen/ambulances (or higher-level coordinators) to distribute fires, form teams, coordinate joint movements, etc. Some kind of global planning
  75. 75. Ideas for the practical exercise (III) Negotiation Different firemen could “negotiate” who fights a certain fire Negotiation via auctions or via Contract Net It could even be a multi-attribute auction, depending on their distance to the fire, whether they are already fighting other fires or on their way to other fires, ...
  76. 76. Readings for this week Chapter 8 of the book An introduction to MultiAgent Systems (M. Wooldridge), 2nd ed. Chapter 4 of the book Agentes Software y Sistemas Multi-Agente (A. Mas) Paper on cooperation hierarchy (Doran et al.) Paper on Partial Global Planning (Durfee, Lesser) Paper on Contract Net (Smith)