For Master and Doctorate students

1. Interactions in Multi Agent Systems
Dr. Sara Manzoni
Complex Systems and Artificial Intelligence research center
Department of Computer Science, Systems and
Communication
University of Milano-Bicocca
4th
Summer School AACIMP-2009
Achievements and Applications of
Contemporary Informatics, Mathematics and
Physics
Lecture 2 – 12.08.2009

2. Multi Agent System (MAS)
“A modeling and computational approach
considering that simple or complex
activities can be the fruits of interaction
between autonomous and independent
entities (i.e. agents) which operate within
communities (i.e. organized structures) in
accordance with modes of cooperation (=
collaboration + coordination + conflict
resolution) in order to fulfill given goals”

3. How to describe a phenomenon (solve
a problem) as the result of collective
behavior
• Modeling the problem as a structured set of
entities (i.e. organization) able to
– Act in an environment
– Interact: communicate and cooperate in order to fulfill
(common) tasks
– Perceive (locally) the environment and adapt their
behavior according to perceptions
– Possess their own resources, skills, tendencies and
objectives (explicit or implicit)
– Behave (e.g. plan actions) tending towards the
satisfaction of objectives, taking into account available
resources, according to their skills, and depending on
their perceptions

4. Design of a MAS
What should be modeled?
• Agents
• Organization
• Interactions
• Environment

5. Design of a MAS (1)
Agents
• Agent architecture (Internal structure)
and agent behavior (Agent model)
– actions that can be undertaken
– environment perception
– adaptation mechanism
– goal fulfillment mechanism
• Tools: operative modeling, formalization
and specification languages, knowledge
representation languages
– E.g. production rules, Petri nets

6. Design of a MAS
(2) Organization
Leaving aside the dynamic dimension, an
organization can be defined and analyzed
– Functionally (roles, tasks, capacities)
– Structurally (divisions, interconnections,
relationships)
Fixed,
predefined
structure (e.g.
Hierarchy)
Variable according to
predefined
mechanisms (e.g.
auction protocols)
Variable,
structure
emerging from
system behaviour

7. Design of a MAS (3)
Interactions (1/3)
• An interaction occur when two or more agents
are brought into a dynamic relationship through
a set of reciprocal actions
• Interactions develop out of a series of actions
whose consequences in turn have an influence
on the future behavior of agents
• During interactions, agents are in contact with
each other
– Directly
– Through another agent
– Through the environment

8. Interactions assume ...
• Presence of agents capable of interacting
and/or communicating
• Situations which can serve as meeting
point of agents
• Dynamic elements allowing local and
temporary relationships between agents
• “slack” in relationships between agents
enabling them to detach themselves from
it (agent autonomy)

9. Interactions and organizations
• Interactions are an element necessary for
the setting up of social organization
• Groups are
– the result of interactions
– the preferred locations in which interactions
occur
• Interaction is the crucial element in
organizations  Source and Product of
the permanence of the organization

10. Interaction situation
• A concept introduced to describe activities of agents in
order to identify different types of interactions by linking
interactions to the elements of which they are composed
• Defines abstract interaction categories independent of
their concrete realizations, by distinguishing them
according to
– Main invariables that we find everywhere
– Differences between situations
An assembly of behaviors resulting from the grouping of
agents which have to act in order to attain their
objectives, with attention being paid to the more or less
resources which are available to them and their
individual skills

11. Example – Building of a house
• Type of interaction  Cooperation situation
requiring coordination of actions
• Interaction situation in which the assembly of
behaviors of the agents (i.e. workforce, architect,
owner, project manager, ...) is characterized by
their own objectives (the same house looked at
from the viewpoints of different agents) and their
skills (know-how of the architect and of different
skilled workers) with attention being paid to the
available resources (raw materials, financing,
tooling, building site)

13. Collective robotics
Bio-inspired opt algorithms

14. A classification of
Interaction situations
• According to compatibility of goals
– Agents cooperate when their goals are compatible 
positive interaction situations
– Agents compete when their goals are incompatible 
negative interaction situations
• According to agent ability to available resources
– Conflict arises when resources are insufficient 
negative interaction situations
• According to agent ability to fulfill tasks
– Collaboration arises when agents have insufficient
ability to solve complex problems  positive
interaction situations

15. Compatibility of goals in reactive
agents
• Negative interaction: the survival
behavior of the one entail the death of
the other
• Positive interaction: the behavior of the
one is not negatively affected by that of
the other
– Cooperation: the behavior of the one is
reinforced by the behavior of the other
• Indifference: the behavior of the one is
not affected at all (neither positively nor
negatively) by the behavior of the other

16. Symbiosis and prey-predator
• Symbiosis between organisms A and B (e.g. A
nourishes B and B defends A from predators):
reactive cooperation
– Heterogeneous organisms cooperate since each
organism is reinforced by the presence and behavior of
other one
• Prey-predator model: antagonistic cooperation
– Predators cooperate (e.g. group formation) to hunt the
prey
– Antagonistic relationship between predators and their
preys – the survival of preys entails the failure of
predators

17. Resources
• All the environmental
and material
elements that can be
used by agents to
carry out their
actions
• Conflicts arise when
two or more agents
need the same
resources at the same
time and in the same
place
Resources
wanted by A
Resources
wanted by B
Conflict zone for
accessing resources
When?
Where?

18. Solving conflict situations
with coordination
• Synchronization (from distributed
systems research)
– of movements
– of access to resources
• Coordination by planning (from
AI): Multi-agent planning
– Centralized planning for multiple
agents – one planner
– Centralized coordination for
partial planning – one coordinator
– Distributed planning
• Reactive coordination
– Coordination by situated actions
(potential fields or marking the
environment)
• Coordination by regulation: rules
– to anticipate and eliminate a-
priori conflict situations
– to manage conflict resolution

19. Coordination in forest ecosystem
Competition on available
resources, needed for
survival and reproduction
Each portion of the territory
- can be inhabited by a tree
- contains a given amount of resources
needed by plants to sprout, grow, survive,
and reproduce themselves
C = {R, P, M, T, S}, where:
R = {R1,…,Rm} – amount of resources
M = {M1,…,Mm} – maximum amount of each
resource
P = {P1,…, Pm} – amount of each resource
produced by the cell at each update step
T – plant state (if any)
S = {s1,...,sn} – number of seeds of each
species present in the cell
Different plant species can inhabit
the same area and compete for
the same resources

20. Interaction through resources
• The presence of a plant limits the
sunlight diffusion to neighbours and
seeds’ growth
• Different species have different needs
in terms of resources
• Resources are produced and
consumed by plants
• Resource distribution on the territory

21. Agents skills and tasks
• Tasks
– can be carried out by a single alone (no interaction
required)
– can be carried out alone but the accomplishment
is facilitated by the support of other agents
– need several agents to be accomplished
• In cases of interaction, the resulting system
posses new properties that can be described
as new emerging functionalities
– the produced object is more than the simple sum
of the skills of each of the agents
– interactions between agents enhance the result

22. Types of interaction (1)
Goals Resources Skills Type
Compatible Sufficient Sufficient Independence
Compatible Insufficient Sufficient Obtrusion
Compatible Insufficient Insufficient Coordinate
Collaboration
Incompatible Sufficient Sufficient Individual Competition
Incompatible Sufficient Insufficient Collective Competition
Incompatible Insufficient Sufficient Individual Conflict on
resources
Incompatible Insufficient Insufficient Collective Conflict on
resources
J. Ferber, “Multi-Agent Systems: an introduction to distributed artificial intelligence”, 1999

23. Types of interaction (2)
• Independence (G, R, S): simple juxtaposition of actions
carried out by agent independently without effective
interaction
• Simple collaboration (G, R, s): simple addition of skills,
without requiring coordination of actions (e.g. When
knowledge is shared among agents)
• Obstruction (G, r, S): agents get in touch in
accomplishing their tasks, but they do not need one
another
• Coordinated collaboration (G, r, s): agents have to
coordinate their actions to have synergic advantages of
pooled skills (e.g. industrial activities, network control,
design and manufacturing of product) – most complex
coordination

24. Types of interaction (3)
• Pure individual competition (g, R, S): resources are not
limited and the competition is not related to them (e.g.
running racing)
• Pure collective competition (g, R, s): agents have to group
into coalitions or associations to be able to achieve their
goals. Two phase process: individuals ally into groups +
groups are set one against another (e.g. sailing
competition)
• Individual conflict over resources (g, r, S): the object of
conflict is the insufficient resource (e.g. Territory,
financial position, animals defending their territory,
humans willing to obtain a better job)
• Collective conflicts over resources (g, r, s): all forms of
collective conflicts in which the objective is to obtain
possession of territory or a resource (e.g. Wars, monopoly
of a good) – collective competition + individual conflict on
resources

25. INTERACTION MODELS IN
MULTI-AGENT SYSTEMS
• Agent internal architecture can be separated by the
(interaction) model that defines the way agents communicate
• This approach allows the modelling, design and
implementation of heterogeneous entities, sharing an
environment in which they can interact
• Many different interaction models have been defined and
implemented
• Often inspired by other disciplines (e.g., social science,
linguistics, biology)

26. INTERACTION MODELS IN MAS:
A TAXONOMY
Agent
interaction
Direct
interaction
Indirect
interaction
With a-priori
acquaintance
Agent discovery
through middle agents
Middle agents &
acquaintance models
Guided/mediated
by artifacts
Spatially founded
interaction

27. Direct interaction models
• Agents are able to directly exchange
information
• Information exchange
– Communication/conversation rules (“protocol”) 
Agent Communication Language (ACL)
– Message structure (shared ontology)  Content
Language
• Information exchange is indiscriminate
– Once an agent knows another one, it will be able
to communicate with it
– No external, contextual factors are considered

28. Direct interaction model example: KQML
• Knowledge Query and Manipulation Language (KQML) and
Knowledge Interchange Format (KIF) are results of the
ARPA Knowledge Sharing Effort
– KQML is an ACL, a high level interaction language
– KIF is a content language, defining syntax of contents
• KQML defines performatives (basic messages to compose
conversations among agents)
• KIF allows to represent information and knowledge about
agents, beliefs, desires, intentions, perceptions plans and
thus their environment
• Agents must share an ontology, in terms a common
vocabulary and agreed upon meanings to describe a
domain subject

29. KQML Message (speech act)
A KQML speech act is described by a list of
attribute/value pairs e.g. :content,
:language, :from, :in-reply-to.
(tell :sender bookShopAgent123
:receiver ksAgent
:in-reply-to id7.34.96.45391
:ontology books
:language Prolog
:content “price(ISBN3429459,24.95)”)
performative
parameter
value

30. A KQML Dialogue
Agents A and B “talking” about the prices of
books bk1 and bk2:
A to B: (ask-if (> (price bk1) (price bk2)))
B to A: (reply true)
B to A: (inform (= (price bk1) 25.50))
B to A: (inform (= (price bk2) 19.99))
For convenience message format above is simplified and
attribute/value pairs for :ontology etc. are omitted.

31. KQML performatives

32. Some requirements
• Agents need to know their communication partners
– Common approach is to have specific facilitators that are
known by every agent and allow them to get acquainted
– Problems: how many of those ‘middle agents’
(robustness) ? How to keep the aligned ?
• A semantic must be defined to obtain/enforce
meaningful conversations
– Agent considered as a logical reasoner with beliefs, desires
and intentions
– Pre and post conditions defined in terms of a of logic
formalization
– Actualization of postconditions triggers preconditions of
other performatives
– What about autonomy ?

33. Other tools for communication
semantics
• The specification of conversations can be
done through several formal models
– Finite State Machines based
– Petri nets based
• The former approach has been widely
used to model, analyze and demonstrate
properties of network protocols
• These appraches also limit agents’
autonomy

34. Direct interaction models: pros
• Similarity to existing protocols for distributed
systems
– Point-to-point message passing
– Easy implementation on top of existing middleware
platforms
• Simple integration with deliberative agents approach
– Agents exchange facts conforming to some kind of
formally defined ontology
• Formal semantics of ACLs can be easily specified
– Communication semantics is related to agents’ beliefs,
decisions, intentions

35. Direct interaction models: cons
• Information exchange occurs according to specific rules
– Network protocol like issues (conversation rules, message
formats)
 Semantical issues
• communication semantics related to agent internals (beliefs,
decisions, intentions)
• normative semantics limits agents’ autonomy
• Exchanged information must conform to an ontology
that is somehow shared by the agents
 Ontology issue
• Agents need to be aware of the presence of a
communication partner
 Discovery issue
• Direct interaction models do not provide abstractions to
represent elements of agents context

36. Direct interaction models:
some enhancements
• Discovery issue and agent context
– Middle agents as specific agents collecting
and providing acquaintance information to
entities of the system
– Not a single middle agent, but a network of
them, organized in order to provide
robustness and structure
– Not just mere agent name service, but
information on provided services

37. Agent
interaction
Direct
interaction
Indirect
interaction
With a-priori
acquaintance
Agent discovery
through middle agents
Middle agents &
acquaintance models
Guided/mediated
by artifacts
Spatially founded
interaction
INTERACTION MODELS IN MAS:
A TAXONOMY

38. Indirect interaction models
• Agents interact through an intermediate entity
• This medium supplies specific interaction
mechanisms and access rules
• These rules and mechanisms define agent local
context and perception
• Time and space uncoupling
• Name uncoupling

39. Agent
interaction
Direct
interaction
Indirect
interaction
With a-priori
acquaintance
Agent discovery
through middle agents
Middle agents &
acquaintance models
Guided/mediated
by artifacts
Spatially founded
interaction
INTERACTION MODELS IN MAS:
A TAXONOMY

40. Artifact-mediated interaction
• Agents access a shared artifact that
– they can observe
– they can modify
• Such artifact is a communication channel
characterized by an intrinsically broadcast
transmission
• Specific laws regulating access to this medium
• It represents a part of agents’ environment

41. Blackboard systems
“Metaphorically we can think of a set of workers,
all looking at the same blackboard: each is able to
read everything that is on it, and to judge when he
has something worthwhile to add to it.”
(A. Newell, 1962)
Blackboard
W1 W2 Wn
Concurrent access control

42. Linda: a specific blackboard based system
• Tuple space: a sort of blackboard in which tuples
(record-like data structures) can be inserted,
inspected and extracted by agents
• Operations
– out(t) puts a new tuple in the Tuple Space, after
evaluating all fields; the caller agent continues
immediately
– in(t) looks for a tuple in the Tuple Space; if not found the
agent suspends; when found, reads and deletes it
– rd(t) looks for a tuple in the Tuple Space; if not found the
agent suspends; when found, reads it
– inp(t) looks for a tuple in the Tuple Space; if found,
deletes it and returns TRUE; if not found, returns FALSE
– rdp(t) looks for a tuple in the Tuple Space; if found, copies
it and returns TRUE; if not found, returns FALSE

43. Matching rules in Linda
• Example:
out("string", 10.1, 24, "another string")
real f; int i;
rd("string", ?f, ?i, "another string")  succeeds
in("string", ?f, ?i, "another string")  succeeds
rd("string", ?f, ?i, "another string")  does NOT
succeed
• Example:
out(1,2)
rd(?i,?i)  does not succeed (whatever is the type of i)

44. From Linda, to mobility and beyond
• Distributed tuple spaces: these systems
allow to have a conceptually shared tuple
space that is spread in a distributed
environment
• More than just distribution
– Programmable, reactive tuple spaces: adding a
behaviour to tuple spaces
– Including organizational abstractions (roles,
policies) to enhance access rules
• References: M. Mamei, F. Zambonelli

45. Artifact-mediated interaction
models: pros and cons
• Advantages
– The artifact represents an abstraction of agents’
environment, and the burden of interaction is
moved from the agents to their environment
– Interaction is mediated, and can thus be
controlled (enforcement/enactment of
organizational rules)
• Issues
– Complex implementation (in distributed
environments)
– How to integrate different artifacts and contexts ?

46. Agent
interaction
Direct
interaction
Indirect
interaction
With a-priori
acquaintance
Agent discovery
through middle agents
Middle agents &
acquaintance models
Guided/mediated
by artifacts
Spatially founded
interaction
INTERACTION MODELS IN MAS:
A TAXONOMY

47. Spatially founded interaction
• Artifact mediated interaction are a first step in
agents’ environment modelling
• Such artifacts represent very focused parts of the
environment, and cannot consider the parts of
agents’ context that does not pertain the specific
artifact
– They represent a single specific context of interaction
• Other approaches bring the environment
metaphor to a deeper level, providing spatially
founded interaction mechanisms
• Spatial features of the environment are explicitily
considered by interaction mechanisms

48. Ancestors of Spatial Interaction: CAs
• A Cellular Automata (CA) is a set of homogeneous cells,
evolving in discrete time steps
• Cells form a regular n-dimensional lattice
– Homogeneous neighborhood (e.g. Von Neumann, Moore)
• Cells characterized by
– A state, belonging to a finite set representing possible cell states
– A transition rule, describing cell state dynamics
• Cell  sort of reactive agent
– Which cannot move in the environment
– Can only interact with neighbouring cells according to precisely
defined rules
von Neumann
Neighbourhood
Moore
Neighbourhood
Extended
Moore Neighbourhood

49. Swarm (and the likes) agent
environment
• Swarm and many derived projects
provide specific environments in which
agents may be placed and interact
• Regular lattices supporting diffusion of
signals that are
– Emitted by entities
– Spread in the spatial structure
– Affecting other entities
– Evaporating over time
• Diffusion strictly related to specific
environmental structures

50. Spatial
structure
Agents and
behaviours
At-a-distance
interaction
A coordination model for
self-organizing agents
[S. Bandini, S. Manzoni, C. Simone, Dealing with Space in Multi-Agent System: a model for Situated
MAS, in Proc. of AAMAS 2002, ACM Press, New York, 2002]
SCA (MMASS) –
Formal and computational
framework where to
describe, represent and
simulate complex systems
according to a situated
MAS approach

51. Coordination as result of interactions
Field-based interaction model
- Indirect interaction model between
agents
- Intrinsically multicast
- Agent interactions occur when
agent states are “compatible”

52. Interaction through Fields
• Fields are generated by agents to interact at-a-distance and
asynchronously
• f = <Wf, Diffusionf, Comparef, Composef>
– Wf: set of field values
– Diffusionf: P X Wf X P Wf X…XWf
field distribution function
– Composef: Wf …XWf Wf
field composition function
– Comparef: Wf X Wf  {True, False}
field comparison function

53. Agents Perception
T  < ∑T, PerceptionT, ActionT>
Set of states that agents of type T can assume
Set of allowed actions for agents of type T
PerceptionT: ∑T [N X Wf1] …[N X Wf|F|]
•PerceptionT(s) = (cT(s), tT(s))
•cT(s): coefficient applied to field values
•tT(s): sensibility threshold to fields
•An agent perceives a field fi when
CompareT(ci
T(s)…wfi,ti
T(s)) is True

54. Field based interaction: emission & perception
• Fields are signals
emitted by agents and
diffused in the
environment
• Their intensity is
possibly modulated in
their diffusion
• Other agents may
perceive these signals
according to their
perceptive capability,
state and the signal
value they receive
• Effect of perception
defined by agent
behavioural
specification
CompareT(f×c,t) = false
CompareT(f×c,t) = true
CompareT(f×c,t) = false
emit(f)

55. Agent Coordination Language:
primitives
action: emit(s,f,p)
condit: state(s)
effect: present(f, p)
action: trigger(s,fi,s’)
condit: state(s), perceive(fi)
effect: state(s’)

56. Subway station scenario
• Various crowd behaviors can take
place
• Passengers' behaviors difficult to
predict
• Crowding dynamics emerges
– Social interactions between
passengers  social rules
– Interactions between single
passengers and the environment
(signs, doors, constraints)
action: transport(p,fi,q)
condit: position(p), empty(q), near(p,q), perceive(fi)
effect: position(q), empty(p)

57. Coordinated movement in space