Doctoral consortium presentation at IJCAI-21 (Montreal - Canada) 19 August 2021

1. Multi-agent approach to resource allocation in
autonomous vehicle fleets
Alaa Daoud
19th -26th August, 2021
IJCAI-DC
at IJCAI 2021

2. Outlines
1. Context and motivation
2. Contribution
3. Evaluation
4. Conclusion
1

3. Context and motivation

4. On-Demand Transport
Figure 1: Dial A Ride Problem (DARP)
2

5. Existing approaches
Centralized dispatching
• Requests are centralized in a portal
• Linear/ Mixed integer program models
⇒ NP-Hard problem, lack of scalablity
• Continuous access to the portal
⇒ expensive with a critical bottleneck
Decentralized allocation
• Decentralized autonomous decisions
⇒ need for conflict detection and avoidance
protocols
• peer-to-peer (P2P) communication
⇒ need for scalable communication model to
ensure best information sharing
3

6. Objectives
Research questions
1. How can we define a uniform representation of the different solution
methods?
2. How can we define a uniform representation of the vehicle
communication model?
3. How can we assess the feasibility and quality of solution methods?
4. Can we combine efficiency, responsiveness, and conflict avoidance in
one dynamic-setting solution method?
4

7. Objectives (cont.)
• Autonomy: Each vehicle is an autonomous agent (solve sub-problems)
• Dynamic: Global solution is a dynamic aggregation of local solutions
• Constrained communication: scalable communication model is required
• Global infrastructure: ⇒ complete graph
• Scalable message passing management: ⇒ incomplete connected graph
• Peer-to-Peer with connection range: ⇒ disconnected graph
• Genericity: Agent behavior abstraction
⇒ adaptive to different solution approaches
5

8. Contribution

9. AV-OLRA model
A generic model to ODT’s dynamic resource allocation problem
Extends the Online Localized Resource Allocation
(OLRA) [Zargayouna et al., 2016] by considering Autonomous
Vehicle (AV) fleets with communication constraints
R, V, G, T
• R: a dynamic set of requests
• V: a fleet of m vehicles
• G: a graph defining the road network
• T : the problem’s time horizon
6

10. Vehicle communication
Communication range and direct connectivity
Vehicles communicate within limited communication range
d_ctd : V × V × T → {0, 1}
defines if two vehicles are connected directly to each other
d_ctd(i, j, t) =



1, if distance loct
i , loct
j
≤ r : r = min(rngi , rngj )
0, otherwise
7

11. Vehicle communication (cont.)
Transitive connectivity
To maximize their connectivity, two vehicles can be connected transitively
ctd : V × V × T → {0, 1}
generalizes the d_ctd with the transitive connectivity.
ctd(i, j, t) =



1, if d_ctd(i, j, t) or ∃k : ctd(i, k, t)ctd(k, j, t)
0, otherwise
8

12. Vehicle communication (cont.)
Connected sets
A connected set is a set of entities that are connected directly or by
transitivity.
CS : V × T → 2V
CS(i, t) = {j ∈ V|ctd(i, j, t)}
The connected sets are dynamic entities; they are created, split, merged at
run-time based on the vehicles’ movement.
A vehicle v may communicate at time t only with the members of its
connected set by directed or broadcast messages.
9

13. Vehicle communication (cont.)
10

14. Autonomous Vehicle (AV) agents
Generic vehicle behavior
Communicating
Acting
Planning
information sharing
coordination
update schedule
update beliefs
11

15. Autonomous Vehicle (AV) agents (cont.)
Communicating sub-behavior
• join(c): agent joins a connected set c as a result of being in the
communication range of one of its members,
• leave(c): agent leaves its connected set c as a result of being
disconnected from all its members,
• send(m, a): agent sends a message m to another agent a in condition
they are in the same connected set,
• receive(m): agent receives a message m from another agent in its
connected set (once received and read, the message is stored in the
agent’s belief base),
• broadcast(m) similar to send(m, a) but here the agent doesn’t specify
the receiving agent, instead it broadcasts the message to the whole
connected set members.
12

16. Autonomous Vehicle (AV) agents (cont.)
Acting Sub-behavior
Marauding Moving
Dropping
off
Picking
up
request
to serve
no passenger
at origin
client
served
arrived
at destination
passenger
at origin
ready to
drive client
13

17. Autonomous Vehicle (AV) agents (cont.)
Abstract planning sub-behavior
Updating
schedule
Choosing
option
Coordinating
options
available
out of options
disagreement
option
selected
agreement
14

18. AV-OLRA Solutions
A solution for AV-OLRA is defined for each connected set as an aggregation
of the allocations of all vehicles in this set, avoiding all conflicts that could
happen. Solution methods depend mainly on the adopted coordination
mechanism (CM):
CM := hDA, AC, AMi
• DA: level of decision autonomy ⇒ centralized (C) / decentralized (D)
• AC: agents’ cooperativeness level ⇒ sharing (S) / no-sharing (N)
• AM: the allocation mechanism ⇒ GREEDY / MILP / DCOP / AUCTIONS
15

19. ORNInA
Online Rescheduling with Neighborhood exchange based on Insertion
heuristic and Auctions
• Combinatorial auctions to allocate resources ⇒ feasible global solution
• Demand exchange strategies⇒ optimize global solution
16

20. ORNInA (cont.)
Auction criteria (insertion heuristic)
Bidd
v (Tstart , cost)
d3(F → G)
Current path:
Potential path:
17

21. ORNInA (cont.)
Improvement candidates
Each vehicle looks for requests that are scheduled by others and could be
inserted in its schedule with lower cost.
Figure 2: V1 finds new candidate for improvement d(D → C)
18

22. ORNInA (cont.)
Pull Auction
A vehicle V1 may select one potential improvement candidate (request d) a
time (1-opt) and inter an auction with d’s serving vehicle V2
pull_cost = V1’s marginal cost to insert d
pull_gain = V2’s marginal cost to abandon d
if(pull_gain pull_cost) : V1andV2 update their schedules
Figure 3: V1’s potential improved schedule by inserting d(D → C)
19

23. Evaluation

24. Experimental environment
Simulation framework
20

25. Experimental environment (cont.)
Urban network: unique urban infrastructure map for all our experiments
• between (45.4325,4.3782) and (45.437800,4.387877)
• 1400 edges have been extracted from Open Street Map
• post-processed to produce a graph of 71 edges
• 40 (uniformly distributed) demand emission sources
Communication: Dedicated Short-Range Communication (DSRC)
realistic communication range of 250 meters.
Execution: Java-based multi-agent system
1000-cycle long scenarios
octa-core Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz
32GB DDR4 RAM.
FRODO library [Léauté et al., 2009] for DCOP algorithms
21

26. Results
QoS evolution
4 6 8 10 12 14 16 18 20
number of vehicles
0.2
0.3
0.4
0.5
0.6
0.7
0.8
#
served
requests
/
#
requests Selfish
Dispatching
Auctions
MGM-2
DSA-A (p=0.5)
22

27. Results (cont.)
QoB evolution
4 6 8 10 12 14 16 18 20
number of vehicles
30
40
50
60
70
80
profit Selfish
Dispatching
Auctions
MGM-2
DSA-A (p=0.5)
23

28. Results (cont.)
Communication cost and statistics
max avg msg per comm. reschedule
Coordination msg size msg size agent load rate
Selfish 140 88 6 2.21 MB 2.0
Dispatching 3500 168 21 11.2 MB 3.0
Auction 140 112 53 37.7 MB 1.5
MGM-2 210 25 5040 297.6 MB 12.0
DSA 236 20 5015 75.1 MB 13.0
24

29. Conclusion

30. Summary
Our contribution
• A multi-agent model of ODT system
• A generic model for solution methods
• Market based solution method
• Implementation of variety of coordination mechanisms
• Preliminary comparison of their performance and robustness
On-going and future work
• Assessment with real world data-sets (e.g. NYC-TLC)
→ systematic evaluation on real world scenario
• Exploring the direction of ML prediction methods
→ deterministic demands
• Exploring the direction of explainability
→ providing transparent recommendations for solution methods and
the suitable settings for the different problem instances
25

31. Thanks
Thank you!
26

32. References
Agatz, N. A., Erera, A. L., Savelsbergh, M. W., and Wang, X. (2011).
Dynamic ride-sharing: A simulation study in metro atlanta.
Transportation Research Part B: Methodological, 45(9):1450 – 1464.
Campbell, A. M. and Savelsbergh, M. (2004).
Efficient insertion heuristics for vehicle routing and scheduling
problems.
Transportation science, 38(3):369–378.
Cramton, P., Shoham, Y., and Steinberg, R. (2007).
An overview of combinatorial auctions.
ACM SIGecom Exchanges, 7(1):3–14.
27

33. References (cont.)
Danassis, P., Filos-Ratsikas, A., and Faltings, B. (2019).
Anytime Heuristic for Weighted Matching Through
Altruism-Inspired Behavior.
In Proceedings of the Twenty-Eighth International Joint Conference on
Artificial Intelligence, pages 215–222, Macao, China.
Dey, K. C., Rayamajhi, A., Chowdhury, M., Bhavsar, P., and Martin, J.
(2016).
Vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I)
communication in a heterogeneous wireless network –
Performance evaluation.
Transportation Research Part C: Emerging Technologies, 68:168–184.
Egan, M. and Jakob, M. (2015).
Market Mechanism Design for Profitable On-Demand Transport
Services.
arXiv:1501.01582 [cs].
28

34. References (cont.)
El Falou, M., Itmi, M., El Falou, S., and Cardon, A. (2014).
On demand transport system’s approach as a multi-agent planning
problem.
In 2014 International Conference on Advanced Logistics and Transport
(ICALT), pages 53–58, Tunis, Tunisia. IEEE, IEEE.
Glaschenko, A., Ivaschenko, A., Rzevski, G., and Skobelev, P. (2009).
Multi-Agent Real Time Scheduling System for Taxi Companies.
AAMAS, page 8.
Grau, J. M. S. and Romeu, M. A. E. (2015).
Agent Based Modelling for Simulating Taxi Services.
Procedia Computer Science, 52:902–907.
Grau, J. M. S., Romeu, M. A. E., Mitsakis, E., and Stamos, I. (2013).
Agent based modeling for simulation of taxi services.
Journal of Traffic and Logistics Engineering, 1(2):159–163.
29

35. References (cont.)
Léauté, T., Ottens, B., and Szymanek, R. (2009).
FRODO 2.0: An open-source framework for distributed constraint
optimization.
In Proceedings of the IJCAI’09 Distributed Constraint Reasoning
Workshop (DCR’09), pages 160–164, Pasadena, California, USA.
https://frodo-ai.tech.
Picard, G., Balbo, F., and Boissier, O. (2018).
Approches multiagents pour l’allocation de courses à une flotte de
taxis autonomes.
In RIA2018, number 2 in Revue d’intelligence artificielle, pages 223–247.
Shen, W. and Lopes, C. (2015).
Managing Autonomous Mobility on Demand Systems for Better
Passenger Experience.
In PRIMA 2015: Principles and Practice of Multi-Agent Systems, volume
9387, pages 20–35, Cham. Springer International Publishing.
30

36. References (cont.)
Solomon, M. M. (1987).
Algorithms for the Vehicle Routing and Scheduling Problems with
Time Window Constraints.
Operations Research, 35(2):254–265.
van Lon, R. R., Holvoet, T., Vanden Berghe, G., Wenseleers, T., and
Branke, J. (2012).
Evolutionary synthesis of multi-agent systems for dynamic
dial-a-ride problems.
In Proceedings of the fourteenth international conference on Genetic and
evolutionary computation conference companion - GECCO Companion
’12, page 331, Philadelphia, Pennsylvania, USA. ACM Press.
Zargayouna, M., Balbo, F., and Ndiaye, K. (2016).
Generic model for resource allocation in transportation. application
to urban parking management.
Transportation Research Part C: Emerging Technologies, 71:538 – 554.
31