Models for a Multi-Agent System Based on Wasp-Like Behaviour for Distributed Patients Repartition

Models for a Multi-Agent System Based on Wasp-Like Behaviour
for Distributed Patients Repartition
DANA SIMIAN, FLORIN STOICA
University Lucian Blaga of Sibiu
Department of Computer Science
Str. Dr. Ion Ratiu 5-7, 550012, Sibiu
ROMANIA
dana.simian@ulbsibiu.ro, florin.stoica@ulbsibiu.ro
CORINA SIMIAN
University Lucian Blaga of Sibiu
Faculty of Sciences
Str. Dr. Ion Ratiu 5-7, 550012, Sibiu
ROMANIA
corinafirst@yahoo.com
Abstract: In this paper are presented two models for an adaptive multi-agent system for dynamic repartition of
the patients within a sanatoria network. The agents use wasp task allocation behaviour, combined with a model of
wasp dominance hierarchy formation. An adaptive method allows patients to enter in the system for the first time.
The rules for the update of response thresholds assure that an optimal repartition from the point of view of type
and gravity of the diseases of a patient is made. Our approach depends on many system’s parameters, which define
the repartition policies.
Key–Words:Wasp-behaviour, Multiagent system
1 Introduction
Agent-based approaches to manufacturing scheduling
and control have gained increasing attention in recent
years. Coordination of multiple agents interacting in
dynamic environments is an important part of many
practical problems. Systems composed of several in-
teracting autonomous agents have a big potential to be
used in complex real-world problems solving.
There are many examples of effective, adaptive be-
haviour in natural multi-agent systems ([1], [2], [3],
[8]),[10]) and computational analogies of these sys-
tems have served as inspiration for multi-agent op-
timization and control algorithms in a variety of do-
mains and contexts. Metaheuristics inspired from na-
ture represent a powerful and robust approach to solve
NP-difficult problems. Self-organization, direct and
indirect interactions between individuals make possi-
ble the identification of intelligent solutions to com-
plex problems. The bio-inspired Ant Colony Opti-
mization (ACO) model ([9], [10], [11], [12], [13],
[16]) simulates real ant behaviour to find the mini-
mum length path between the ant nest and the food
source. An ant algorithm is essentially a system based
on agents that simulate the natural behaviour of ants
including mechanisms of cooperation and adaptation.
In [20], Theraulaz et al. present a model for the self-
organization that takes place within a colony of wasps.
Interactions between members of the colony and the
local environment result in dynamic distribution of
tasks such as foraging and brood care. In addition,
a hierarchical social order among the wasps of the
colony is formed through interactions among indivi-
dual wasps of the colony. This emergent social order
is a succession of wasps from the most dominant to
the least dominant. Theraulaz et al. model these two
aspects of wasp behaviour as distinct behaviours with-
out making any explicit connection between the two.
Cicirelo V.A. and Smith S.F have used these two in-
dependent models of wasp behaviour to develop coor-
dination mechanisms for a multi-agent factory control
system where each machine is modeled as a wasp nest
comprised of wasp-like agents which take part in rout-
ing and scheduling activities ([6], [7]).
Multi-agent systems promote a simplicity of de-
sign. A multi-agent approach to developing complex
systems involves many agents capable of interacting
with each other to achieve objectives. Such an ap-
proach includes the ability to solve large and complex
problems, interconnection of multiple existing legacy
systems and the capability to handle domains in which
the expertise is distributed ([14], [15], [20]).
Many real problems are analogues with dis-
tributed coordination of resources and with manufac-
turing control tasks in a factory ([17], [18]). Models
inspired from nature and multi-agent systems can be
used for solving large complex problems, particularly
those with a dynamic character.
The aim of this paper is to define agent-based
models for the repartition of the patients in many sana-
toria taking into account the type and the gravity of
the disease and the previous results obtained for the
patient in a given sanatorium. This is a real problem
9th WSEAS International Conference on EVOLUTIONARY COMPUTING (EC’08), Sofia, Bulgaria, May 2-4, 2008
ISBN: 978-960-6766-58-9 82 ISSN: 1790-5109

which health system in Romania is confront with. The
models we propose are based on the adaptive wasp
colonies behaviour. The agents use wasp task alloca-
tion behaviour, combined with a model of wasp dom-
inance hierarchy formation. The system we want to
develop has many common characteristics with a dis-
tributed manufacturing system.
The remainder of this paper is organized as fol-
lows. In section 2 we present the wasp behaviour
model. In the section 3 are presented the models for
our multi-agent system and in section 4 are presented
conclusions and future work.
2 Wasp behaviour model
Theraulaz et al. present a model for self-organization
within a colony of wasps ([20]). In a colony of wasps,
individual wasp interacts with its local environment
in the form of a stimulus-response mechanism, which
governs distributed task allocation. An individual
wasp has a response threshold for each zone of the
nest. Based on a wasp’s threshold for a given zone
and the amount of stimulus from brood located in this
zone, a wasp may or may not become engaged in the
task of foraging for this zone. A lowest response
threshold for a given zone amounts to a higher like-
lihood of engaging in activity given a stimulus. In [3]
is discussed a model in which these thresholds remain
fixed over time. Later, in [19] is considered that a
threshold for a given task decreases during time pe-
riods when that task is performed and increases oth-
erwise. In [5], Cicirello and Smith, present a system
which incorporates aspects of the wasp model which
have been ignored by others authors. They consider
three ways in which the response thresholds are up-
dated. The first two ways are analogous to that of
real wasp model. The third is included to encour-
age a wasp associated with an idle machine to take
whatever jobs rather than remaining idle. The model
of wasp behaviour also describes the nature of wasp-
to-wasp interaction that takes place within the nest.
When two individuals of the colony encounter each
other, they may with some probability interact in a
dominance contest. The wasp with the higher social
rank will have a higher probability of dominating in
the interaction. Wasps within the colony self-organize
themselves into a dominance hierarchy. In [5] is in-
corporated this aspect of the behaviour model, that is
when two or more of the wasp-like agents bid for a
given job, the winner is chosen through a tournament
of dominance contests.
3 Main results. Wasp agents
In this section we present our approach to the
problem of allocating dynamically the places from
many sanatoria to patients in the system such that the
results of the treatment be maximize and to allow as
many new patients to enter in the system. The model
is made using many variables and parameters such
that its flexibility is increased. We next define the
problem’s terms.
A list of the type of diseases must be avail-
able, by example: neurological, cardiac. Let suppose
that in this codification we have n type of diseases:
t1, . . . , tn. We will use a scale for the diseases of
the patient. In our model we use a scale from 0 to
a, a ∈ Z+ but effective value of the superior limit of
the scale is not important. We use the scale to obtain
an ordering of the type of diseases of patients on their
gravity. A patient P can be registered in the system
with mP diseases. An arbitrary disease, i, has asso-
ciate a type, tdi and a number of gravity points gi. Let
be:
dP,k =
mP
i = 1
tdi=k
gi (1)
the amount of the points obtained for the diseases of
type k, k ∈ {1, . . . , n} of the patient P.
Every sanatorium has a score for various types of
diseases.
sj,k ∈ {−a, . . . , a}, a ∈ Z+ (2)
is the score of the health unit j, for the diseases of type
k, with j = 1, . . . , u and u the number of health units.
A negative score means a contraindication; a positive
score means an indication for this disease. As big a
score is as indicated is this sanatorium for this type of
disease. The number of free places in the sanatorium
is also known in every moment.
Every health unit has a list of the patients who
stayed at list once in this unit and a score concern-
ing the effects of this stage on the patient behaviour.
For every patient, P, it is known the number, nP,l
of times he stayed in the sanatorium l, l = 1, . . . , u
and the score for every stay, spP,l,i ∈ {−a, . . . , a},
i = 1, . . . nP,l, a > 0. Using this information, an
average score is calculate:
mP,l =



nP,l
i=1
spP,l,i/nP,l if nP,l > 0
0 if nP,l = 0
(3)
We can formulate the problem in many ways, each
way leading to a different architecture of our system.
ISBN: 978-960-6766-58-9 83 ISSN: 1790-5109

We can associate patients with the factory machines
and sanatoria with the factory commands or we can
associate sanatoria with factory machines and patients
with the factory commands. We will use P-wasps for
the first case and S- wasps for the second.
3.1 P- wasp
We associate to each patient from our system a wasp
agent, named P-wasp, which bid for a sanatorium de-
pending on the patient profile of diseases. Each P-
wasp has a set of response thresholds as like in the
wasp behaviour model. The response thresholds are
associated to every sanatorium
WP = {wP,j} (4)
where wP,j is the response threshold of wasp associ-
ated to patient P for the sanatorium j. Every sana-
torium from the system, has associated, for each P-
wasp, such a response thresholds. The threshold value
wP,j may vary in an interval [wmin, wmax].
Sanatoria in the system that have free places,
broadcast to all of the P- wasps a stimulus SP,j, which
is proportional to the length of time the sanatorium
has been waiting for assignment of a free place to a
patient. The P- wasp will bid for a place in the sana-
torium j if and only if
max
k∈{1,...,n}
(dP,k·sj,k) > 0 and min
k∈{1,...,n}
(dP,k·sj,k) ≥ 0
(5)
These conditions assure that the P-wasp will not
choose a contraindicated sanatorium for any type of
disease of patient P.
A P- wasp will chose a place in the sanatorium j with
the probability
P(P, j) =
Sγ
P,j
Sγ
P,j + wγ
P,j
(6)
The exponent γ > 1 is a system parameter. In [19] is
used such a rule for task allocation with γ = 2. Us-
ing (6), as lower the response thresholds is, as bigger
the probability of binding a place in the sanatorium is.
But, using this rule, a wasp can bid for a place if a
hight enough stimulus is emitted.
Each P- wasp, at all times, knows all the informa-
tion about its patient. This knowledge is necessary in
order to adjust the response thresholds for the various
sanatoria. This update occurs at each time step. The
difference from the model presented in other papers
([5], [6], [7], [19], [20] ) is that these response thresh-
olds are automatically updated taking into account the
type and gravity of the disease of a patient and the
specific of the sanatorium.
wP,j = wP,j − δ ·
n
k=1
dP,k · sj,k (7)
If the sanatorium is contraindicated for a type of dis-
ease, than sj,k < 0 and the response threshold is in-
creased. As greater the number of the diseases type of
a patient can be treated in a sanatorium as lower the
response threshold for this sanatorium is.
Therefore, P- wasp stochastically decides whether or
not to bid for a place in a sanatorium, according to the
type and gravity of diseases of a patient, to the specific
of the sanatorium, the length of time the free place in a
sanatorium has been waiting and the response thresh-
old. The response thresholds are reinforced to encour-
age the P- routing wasp to bid on a place in a sanato-
rium which maximize the effects of the treatment for
all types of the diseases that a patient have.
If two ore more P- wasps respond positively to
the same stimulus, that is bid for the same sanato-
rium, these learning routing wasps enter in a domi-
nance contest only if their number is greater than the
number of free places in this sanatorium. We intro-
duced a method for deciding which P- wasps from a
group of competing wasps get the places in a sanato-
rium. We make an hybrid model, using a fitness func-
tion. For every P-wasp which want to bid a place in
the sanatorium l, we compute the value of function
fP = 1 + α ·
n
k=1
dP,k · sj,k + β · mP,l, (8)
with mP,l given in (5).
The descending order of the value of this function for
the competitors P-wasps, will give the order in which
these wasps will take the places in sanatorium l. The
parameters α and β defines the policies of repartition.
If β = 0 we do not consider the previous results ob-
tained for the patient P in the sanatorium l.
Another way to solve the problem of competition
between two P-wasps is the hierarchical model from
a nest. We calculate for every P-wasp the value
FP = 1 + α ·
1
n
k=1 dP,k · sj,k
+ β ·
1
mP,l
(9)
A small value for β will encourage new patients to
enter in the system. Let p and q be the P- wasps in a
dominance contest. P-wasp p will get the place with
probability
Pc(p, q) = P(Wasp p win |Fp, Fq) =
F2
q
F2
p + F2
q
(10)
ISBN: 978-960-6766-58-9 84 ISSN: 1790-5109

In this way, P - wasps associated to patients with
the same type and gravity of diseases and with same
results obtained before in this sanatorium, will have
equal probabilities of getting the place. For the same
type of diseases, the patient with bigger gravity of the
disease has a higher probability of taking the place.
For the same gravity of the disease, the patient which
obtained in this sanatorium better treatment results be-
fore, has a higher probability of taking the place.
3.2 S- wasp
In this model we consider that each sanatorium has as-
sociated an agent wasp, named S-wasp. Every patient
enter in the system with only one type of disease for
which he wants to obtain a treatment in a sanatorium.
A list of all the types of diseases of every patient is
also available. Each S- wasp has a response threshold
for every patient in the system. We denote by wj,i the
response threshold of the sanatorium j for the patient
i. The response thresholds are situated in an interval
[wmin, wmax] and are updated at each time step, to
encourage a S-wasp to bid a patient with the type of
disease which has the higher score for this sanatorium.
A S-wasp associated to the sanatorium j, will choose
a patient only if
max
k∈{1,...,n}
(dP,k · sj,k) > 0 and noj > 0, (11)
where noj represents the number of free places in
sanatorium j. This number is updated daily. If the
type of disease, ti, of the patient i has the property
that
sj,ti = max
k=1,...,n
{sj,k} (12)
then
wj,i = wj,i − δ1 (13)
else
wj,i = wj,i + δ2 (14)
If the sanatorium j has empty queue then
wj,i = wj,i − δτ
3 , (15)
where τ is the length of time the sanatorium has been
idle and is an exponent.
The δ1, δ2 and δ3 are positive system constants.
Patients in the system that have not been assigned
yet to a sanatorium and that are awaiting assignment,
broadcast to all of the learning routing wasp a stim-
ulus Sj,i, which is proportional to the length of time
the patient has been waiting for assignment to a sana-
torium.
S-wasp stochastically decides whether or not choose
a patient according to the main type of disease of the
patient, the length of time the patient has been waiting
and the response threshold. The equation (15) encour-
ages a wasp associated to an idle sanatorium to take a
patient with a type of disease which not realize a max-
imum score for this sanatorium, rather then remaining
idle.
The S wasp j will choose the patient i with probabil-
ity
P(j, i) =
Sγ
j,i
Sγ
j,i + wγ
j,i
(16)
The exponent γ is a system parameter and is chosen
as γ > 1.
If two ore more S-wasps respond positively to a same
stimulus, that is they want to choose a same patient, i,
than they enter in a dominance contest. We deﬁne for
the S- wasp j, the force Fj,i, as:
Fj,i = 1 + α ·
1
n
k=1 di,k · sj,k
+ β ·
1
mi,j
, (17)
with mi,j given in (3).
Let j and l be the S wasps in a dominance contest for
taking the patient i. S- wasp j will get the patient with
probability
Pc(j, l) = P(Wasp j win |Fj,i, Fl,i) =
F2
l,i
F2
j,i + F2
l,i
(18)
In this way, S-wasps associated with sanatorium with
equivalent scores and equivalent previous results in
treatment of the patient i, will have equal probabili-
ties of getting the patient.
4 Conclusion
In this paper are presented two models for an adaptive
multi-agent system for repartition of the patients in
many sanatoria. The models are based on the adaptive
wasp colonies behaviour. The models we introduced
allows the repartition of the patients taking into ac-
count one ore many diseases of the patient, the gravity
of these diseases and the specialization of the sanato-
ria in treatment of these types of diseases. The mod-
els also allow the consideration of the previous results
in the same sanatorium. Some initial conditions are
used in order to avoid the repartition of a patient to
a sanatorium which can be contraindicated for a type
of disease of the patient. The system for repartition of
the patients has many common characteristics with the
distributed manufacturing system. The update rules
for the response thresholds, used in our models are
different from these ones used for distributed manu-
facturing systems in [5], [6], [7]. The functions used
in competition contest of the wasp agent allow the
ISBN: 978-960-6766-58-9 85 ISSN: 1790-5109

maximization of the results of the treatments, by tak-
ing into account the previous results obtained for a pa-
tient in a sanatorium. Our approach depends on many
system parameters. For the implementation these pa-
rameters were tuned by hand.
The next direction of our studies is to compare
the results obtained for the two models introduced for
various profiles of the patients and for different sets of
system parameters.
Acknowledgment: This work was completed with
the support of the research grant of the Roma-
nian Ministry of Education and Research, CNCSIS
33/2007.
References:
[1] E. Bonabeau, M. Dorigo, G. Theraulaz, Swarm
intelligence: From natural to artificial sys-
tems.Santa Fe Institute Studies of Complexity,
Oxford University Press, 1999.
[2] E. Bonabeau, M. Dorigo, G. Theraulaz, Inspi-
ration for optimization from social insect be-
haviour, Nature , vol. 406, 2000, pp. 39–42.
[3] E. Bonabeau, G. Theraulaz, J.I. Demeubourg,
Fixed response thresholds and the regulation of
division of labor in insect societies, Bull. Math.
Biol., vol. 60, 1998, pp. 753–807.
[4] S. Bussemann , Agent - oriented programming
of manufacturing control tasks, Proceeding of
the 3rd International Conference on Multi-Agent
Systems, ICMAS, 1998, pp. 57–63.
[5] V. A. Cicirelo, S.F. Smith, Wasp-like Agents for
Distributed Factory coordination, Autonomous
Agents and Multi-Agent Systems, Vol. 8, No.
3,2004, pp. 237–267.
[6] V. A. Cicirelo, S.F. Smith, Distributed Coordi-
nation of Resources via Wasp-like Agents, First
NASA GSFC/JPL Workshop on Radical Agent
Concepts (WRAC), January, 2002.
[7] V. A. Cicirello, S. F. Smith, Wasp nests for self-
configurable factories, Agents 2001, Proceed-
ings of the Fifth International Conference on Au-
tonomous Agents, ACM Press, May-June 2001.
[8] V. A. Cicirelo, S.F. Smith, Insect Societies and
Manufacturing, The IJCAI-01 Workshop on Ar-
tificial Intelligence and Manufacturing: New AI
Paradigms for Manufacturing, August, 2001.
[9] J. L. Deneubourg, S. Aron, S. Goss, J.M. Pas-
teels, The Self-Organizing Exploratory Pattern
of the Argentine Ant, Journal of Insect Be-
haviour, 3,1990, pp. 159–168.
[10] M. Dorigo, G. Di Caro, L.M. Gambardella,
Ant algorithms for discrete optimization, Artifi-
cial Life, 5, 1999, pp. 137–172.
[11] M. Dorigo, C. Blum, Ant Colony Optimiza-
tion Theory: A Survey,Theoretical Computer
Science, 344. 2-3, 2005, pp. 243–278.
[12] M. Dorigo, Optimization, Learning and Natu-
ral Algorithms, Ph.D. Thesis, Politecnico di Mi-
lano, Italy, 1992.
[13] M. Dorigo, G. Di Caro,The Ant Colony Opti-
mization Meta-Heuristic, D. Crone, M. Dorigo
and F. Glover (editors), New Ideas in Optimiza-
tion, McGraw-Hill, 1999, pp. 11–32.
[14] V. Parunak , A. Baker, S. Clark, The AARIA
agent architecture. From manufacturind require-
ments to agent-based system design, Proceed-
ings of the ICAA’98 Workshop on Agent based
Manufacturing, 1998.
[15] N. Policella, S.F. Smith, A. Cesta, A. Oddi,
Steps Toward Computing Robust Schedules,
Proceedings CP-03 Workshop on Online Con-
straint Solving: Handling Change and Uncer-
tainty, Cork, Ireland, September, 2003.
[16] B. Ribeiro, R. F. Albrecht, A. Dobnikar,
D. W. Pearson, N. C. Steele, Wasp swarm
optimization of logistic systems, Adaptive and
Natural Computing Algorithms Springer Vi-
enna,2005, pp. 264–267.
[17] D. Simian, C. Simian, I. Moisil, I. Pah, Com-
puter mediated communication and collabora-
tion in a virtual learning environment based on
a multi-agent system with wasp-like behavior,
Lecture Notes in Computer Science, Springer
Berlin Heildelberg, LSCC 2007, 2007, pp. 606–
614.
[18] I. Moisil, I. Pah, D. Simian, C. Simian, Ant
Colony Models for a Virtual Educational Envi-
ronment based on a Multi-Agent System, Lec-
ture Notes in Computer Science, Springer Berlin
Heildelberg, LSCC 2007,2007, pp. 565–573
[19] G. Theraulaz , E. Bonabeau, J. I. Demeubourg,
Response threshold reinforcement and divizion
of labour in insects societies, Proc. R Spob Lon-
don B.,vol. 265, no. 1393,1998, pp. 327-‘335.
[20] G. Theraulaz , E. Bonabeau, J. Gervet , J. I. De-
meubourg , Task differention in policies wasp-
colonies.A model for self-organizing groups of
robots, From animals to Animats: Proceedings
of the First International Conference on Simula-
tion of Adaptiv behaviour, 1991, pp. 346–355.
ISBN: 978-960-6766-58-9 86 ISSN: 1790-5109

Models for a Multi-Agent System Based on Wasp-Like Behaviour for Distributed Patients Repartition

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (17)

Similar to Models for a Multi-Agent System Based on Wasp-Like Behaviour for Distributed Patients Repartition

Similar to Models for a Multi-Agent System Based on Wasp-Like Behaviour for Distributed Patients Repartition (20)

More from infopapers

More from infopapers (20)

Recently uploaded

Recently uploaded (20)

Models for a Multi-Agent System Based on Wasp-Like Behaviour for Distributed Patients Repartition