This document describes research on developing a system for defining and applying rules to regulate unmanned aerial vehicle (UAV) traffic in urban environments. It proposes an urban airspace model to simulate UAV traffic, a language to express rules, and using neuroevolution to generate UAV controllers. Experimental evaluation of the model showed that manually defined rules lowered traffic efficiency but increased safety, as intended, validating the model. The goal is to automatically generate more efficient and safer UAV controllers through the use of rules and neuroevolution.

1. Design and Experimental Evaluation of a System
for the Definition and Application of Rules
for UAVs Traffic in the Urban Environment
Student: Giuseppe Lombardi
Supervisor: Eric Medvet
Co-supervisor: Alberto Bartoli
Department of Engineering and Architecture
University of Trieste
Italy
4/10/2017, Trieste
http://machinelearning.inginf.units.it

2. UAVs
Cities now
Lombardi Giuseppe UAVs Traffic Rules 2 / 27

3. UAVs
Smart cities in the the future
Lombardi Giuseppe UAVs Traffic Rules 2 / 27

4. UAVs
Smart cities in the the future
There will be different UAVs for:
urban security
traffic surveilance
goods delivery
etc. . .
Lombardi Giuseppe UAVs Traffic Rules 2 / 27

5. UAVs
Smart cities in the the future
There will be different UAVs for:
urban security
traffic surveilance
goods delivery
etc. . .
UAVs are becoming increasingly popular!
Lombardi Giuseppe UAVs Traffic Rules 2 / 27

6. UAVs popularity
UAVs popularity by data:
Estimated number of UAVs shipped in 2017: 3 million
Lombardi Giuseppe UAVs Traffic Rules 3 / 27

7. UAVs popularity
UAVs popularity by data:
Estimated number of UAVs shipped in 2017: 3 million
Projected number of UAVs in the US by 2020: 7 million
Lombardi Giuseppe UAVs Traffic Rules 3 / 27

8. UAVs popularity
UAVs popularity by data:
Estimated number of UAVs shipped in 2017: 3 million
Projected number of UAVs in the US by 2020: 7 million
Projected size of the UAVs industry in 2020: $ 3.3 billion
Lombardi Giuseppe UAVs Traffic Rules 3 / 27

9. UAVs popularity
UAVs popularity by data:
Estimated number of UAVs shipped in 2017: 3 million
Projected number of UAVs in the US by 2020: 7 million
Projected size of the UAVs industry in 2020: $ 3.3 billion
In an urban environment there will be a massive UAVs traffic.
How to regulate it in terms of efficency and safety?
Lombardi Giuseppe UAVs Traffic Rules 3 / 27

10. Rules
Now:
there are no official rule systems
different communities still discussing
Lombardi Giuseppe UAVs Traffic Rules 4 / 27

11. Rules
Now:
there are no official rule systems
different communities still discussing
The rules should be online and on-board computable by the UAVs
(controller).
Lombardi Giuseppe UAVs Traffic Rules 4 / 27

12. Rules
Now:
there are no official rule systems
different communities still discussing
The rules should be online and on-board computable by the UAVs
(controller).
Key questions:
which language to express those rules?
how to write them?
Lombardi Giuseppe UAVs Traffic Rules 4 / 27

13. Rules and controller
Given a set of rules:
does it impact on traffic efficency and safety?
does it impact on the controller?
Lombardi Giuseppe UAVs Traffic Rules 5 / 27

14. Rules and controller
Given a set of rules:
does it impact on traffic efficency and safety?
does it impact on the controller?
can we automatically construct a controller more efficient and safer?
Lombardi Giuseppe UAVs Traffic Rules 5 / 27

15. Rules and controller
Given a set of rules:
does it impact on traffic efficency and safety?
does it impact on the controller?
can we automatically construct a controller more efficient and safer?
Proposed solutions:
1 an urban airspace model to simulate the UAVs traffic
2 a language to express realistic rules
3 evolutionary approach to generate UAVs controller
Lombardi Giuseppe UAVs Traffic Rules 5 / 27

16. The urban airspace model
Table of Contents
1 The urban airspace model
2 Language for rules
3 Experimental evaluation
4 Neuroevolution
Lombardi Giuseppe UAVs Traffic Rules 6 / 27

17. The urban airspace model
The urban airspace model
Goals (and trade-offs):
detailed enough to include concepts amenable to be regulated
simple enough to allow for simulation
Describes:
the physical world
the UAVs controller’s behavior
Lombardi Giuseppe UAVs Traffic Rules 7 / 27

18. The urban airspace model
The urban airspace model
Discrete space:
cells
buildings
UAVs moving in it
UAVs:
position
target position
status (in {alive, collided})
controller
Stochastic collisions
With discrete time
Lombardi Giuseppe UAVs Traffic Rules 8 / 27

19. The urban airspace model
Collisions
At each time step, an UAV may
stochastically be involved in a collision:
into a building
in a cell around a building
another UAV in the same cell
another UAV in an adjacent cell
Lombardi Giuseppe UAVs Traffic Rules 9 / 27

20. The urban airspace model
UAV controller
Algorithm executed by an UAV at each time step.
Lombardi Giuseppe UAVs Traffic Rules 10 / 27

21. The urban airspace model
UAV controller
Algorithm executed by an UAV at each time step.
Input:
1 unlimited knowledge of the non-moving objects (buildings map)
2 a limited knowledge of moving object aroud it (other UAVs)
3 the target cell
4 unlimited knowledge of the effects of the rules (denied regions)
Lombardi Giuseppe UAVs Traffic Rules 10 / 27

22. The urban airspace model
UAV controller
Algorithm executed by an UAV at each time step.
Input:
1 unlimited knowledge of the non-moving objects (buildings map)
2 a limited knowledge of moving object aroud it (other UAVs)
3 the target cell
4 unlimited knowledge of the effects of the rules (denied regions)
Output:
1 a movement in order to update the UAV position
Lombardi Giuseppe UAVs Traffic Rules 10 / 27

23. The urban airspace model
UAV controller
Considered two different controllers:
A* controller (A*)
1 given a start position
2 explores all possible paths
3 to the target cell
4 avoiding obstacles
5 chose the cheapest
Square-Arc controller (SA)
1 reach a safe altitude moving up
2 reach the target x-coordinate
moving on x axis
3 reach the target y-coordinate
moving on y axis
4 reach the ground moving down
Lombardi Giuseppe UAVs Traffic Rules 11 / 27

24. Language for rules
Table of Contents
1 The urban airspace model
2 Language for rules
3 Experimental evaluation
4 Neuroevolution
Lombardi Giuseppe UAVs Traffic Rules 12 / 27

25. Language for rules
Rules
Rule:
each rule is a pair R = (B, c)
B is a non-empty set of boxes
c is a condition
Lombardi Giuseppe UAVs Traffic Rules 13 / 27

26. Language for rules
Rules
Rule:
each rule is a pair R = (B, c)
B is a non-empty set of boxes
c is a condition
At a given time, for a given rule, an UAV evaluates if it meets the rule
condition c.
Lombardi Giuseppe UAVs Traffic Rules 13 / 27

27. Language for rules
Rules
Rule:
each rule is a pair R = (B, c)
B is a non-empty set of boxes
c is a condition
At a given time, for a given rule, an UAV evaluates if it meets the rule
condition c.
=⇒ If so, it denies the cells of the space refered by the set of boxes B.
Lombardi Giuseppe UAVs Traffic Rules 13 / 27

28. Language for rules
Rules
Box:
position of the center of the box
side length
shape of the box
center inclusion or not
Lombardi Giuseppe UAVs Traffic Rules 14 / 27

29. Language for rules
Rules
Box:
position of the center of the box
side length
shape of the box
center inclusion or not
Box examples
Lombardi Giuseppe UAVs Traffic Rules 14 / 27

30. Language for rules
Rules
Box:
position of the center of the box
side length
shape of the box
center inclusion or not
A condition works on:
the UAV position
its target cell
the closest other UAV
close buildings
Box examples
Lombardi Giuseppe UAVs Traffic Rules 14 / 27

31. Language for rules
Language for the rules
As a context-free grammar:
R ::=({ boxes }, conditions )
boxes ::= boxes , box | box
box ::=( x-coord , y-coord , z-coord , digit ,
shape , center-inclusion )
digit ::=0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
shape ::=full | +x | −x | +y | −y | +z | −z
center-inclusion ::=incl | excl
x-coord ::=x | xf | xU,closest | digit digit
y-coord ::=y | yf | yU,closest | digit digit
z-coord ::=z | zf | zU,closest | digit digit
conditions ::= conditions ∧ condition | condition | true
condition ::= basic-condition | ¬ basic-condition
basic-condition ::= coord-condition | dist-coondition
coord-condition ::= x-coord op x-coord | x-coord op digit digit |
y-coord op y-coord | y-coord op digit digit |
z-coord op z-coord | z-coord op digit digit
op ::= = | <
dist-condition ::= dist ≤ digit
dist ::=dU,closest | d+x
B,closest | d−x
B,closest | d
+y
B,closest | d
−y
B,closest | d+z
B,closest | d−z
B,closest
Lombardi Giuseppe UAVs Traffic Rules 15 / 27

32. Language for rules
Example rules
“flight at a min altitude of 4”
“keep a min distance of 2 from other UAVs”
“keep a min distance of 2 from buildings”
Lombardi Giuseppe UAVs Traffic Rules 16 / 27

33. Language for rules
Example rules
“flight at a min altitude of 4”
B ={(x, y, z, 1, −z, excl), (x, y, z, 1, −x , excl),
(x, y, z, 1, +x , excl), (x, y, z, 1, −y , excl),
(x, y, z, 1, +y , excl)}
R1 =(B, z < 4 ∧ ¬x = xf )
R2 =(B, z < 4 ∧ ¬y = yf )
“keep a min distance of 2 from other UAVs”
R =({xU,closest, yU,closest, zU,closest, 1, full, incl}, true)
“keep a min distance of 2 from buildings”
R−x =({(x, y, z, 1, −x , excl)}, d−x
B,closest ≤ 2)
R+x =({(x, y, z, 1, +x , excl)}, d+x
B,closest ≤ 2)
R−y =({(x, y, z, 1, −y , excl)}, d
−y
B,closest ≤ 2)
R+y =({(x, y, z, 1, +y , excl)}, d
+y
B,closest ≤ 2)
R−z =({(x, y, z, 1, −z, excl)}, d−z
B,closest ≤ 2)
Lombardi Giuseppe UAVs Traffic Rules 16 / 27

34. Experimental evaluation
Table of Contents
1 The urban airspace model
2 Language for rules
3 Experimental evaluation
4 Neuroevolution
Lombardi Giuseppe UAVs Traffic Rules 17 / 27

35. Experimental evaluation
Goals
UAV urban airspace traffic model:
is it sound?
find values for parameters
Lombardi Giuseppe UAVs Traffic Rules 18 / 27

36. Experimental evaluation
Goals
UAV urban airspace traffic model:
is it sound?
find values for parameters
Handwritten rules:
do this set of rules effectively impact on the UAV traffic?
Lombardi Giuseppe UAVs Traffic Rules 18 / 27

37. Experimental evaluation
Efficiency and accidentality
For each computed simulation, global indexes representing:
Traffic efficiency
Traffic accidentality (safety)
Lombardi Giuseppe UAVs Traffic Rules 19 / 27

38. Experimental evaluation
Efficiency and accidentality
For each computed simulation, global indexes representing:
Traffic efficiency → Average Ground Speed (AGS)
Traffic accidentality (safety) → Average Accidentality (AA)
Lombardi Giuseppe UAVs Traffic Rules 19 / 27

39. Experimental evaluation
Simulations
Experiments for the model validation made with:
4 building sets with different ratio of space occupied by buildings
up to tens of UAVs
3 different controller choice policies
1 always A*
2 always SA
3 A* or SA with equal probability
Values averaged on 30 simulations.
Lombardi Giuseppe UAVs Traffic Rules 20 / 27

40. Experimental evaluation
Traffic model validation
0 20 40 60 80
0
0.2
0.4
0.6
Low efficiency
High efficiency
nUAVs (injected traffic)
AGS(Efficiency)
0 20 40 60 80
0
0.5
1
1.5
Low safety
High safety
nUAVs (injected traffic)
AA(Accidentality)
Lombardi Giuseppe UAVs Traffic Rules 21 / 27

41. Experimental evaluation
Traffic model validation
0 20 40 60 80
0
0.2
0.4
0.6
nUAVs (injected traffic)
AGS(Efficiency)
0 20 40 60 80
0
0.5
1
1.5
nUAVs (injected traffic)
AA(Accidentality)No rules
Efficiency Safety
low nUAVs high high
high nUAVs low low
Lombardi Giuseppe UAVs Traffic Rules 21 / 27

42. Experimental evaluation
Traffic model validation
0 20 40 60 80
0
0.2
0.4
0.6
nUAVs (injected traffic)
AGS(Efficiency)
0 20 40 60 80
0
0.5
1
1.5
nUAVs (injected traffic)
AA(Accidentality)No rules Hand-written rules
Efficiency Safety Efficiency Safety
low nUAVs high high < >
high nUAVs low low
Rules → lower efficiency, higher safety: sound!
Lombardi Giuseppe UAVs Traffic Rules 21 / 27

43. Experimental evaluation
Traffic model validation
0 20 40 60 80
0
0.5
1
1.5
nUAVs (injected traffic)
AA(Accidentality)
No Rules
0 20 40 60 80
0
0.5
1
1.5
nUAVs (injected traffic)
Handwritten Rules
A* SA Mix
Controller choice policy tends to become irrilevant with rules!
Lombardi Giuseppe UAVs Traffic Rules 22 / 27

44. Experimental evaluation
Traffic model validation
0 20 40 60 80
0
0.2
0.4
0.6
nUAVs (injected traffic)
AGS(Efficiency)
No Rules
0 20 40 60 80
0
0.2
0.4
0.6
nUAVs (injected traffic)
Handwritten Rules
A* SA Mix
Controller choice policy tends to become irrilevant with rules!
Lombardi Giuseppe UAVs Traffic Rules 22 / 27

45. Neuroevolution
Table of Contents
1 The urban airspace model
2 Language for rules
3 Experimental evaluation
4 Neuroevolution
Lombardi Giuseppe UAVs Traffic Rules 23 / 27

46. Neuroevolution
Neural Network Controller (NNC)
Input: same four inputs as the algorithm controllers.
Output:
mx ∈ {−1x , 0x , +1x }
my ∈ {−1y , 0y , +1y }
mz ∈ {−1z, 0z, +1z}
(e.g. mx = +1x → x = x + 1)
Lombardi Giuseppe UAVs Traffic Rules 24 / 27

47. Neuroevolution
Neural Network Controller (NNC)
Input: same four inputs as the algorithm controllers.
Output:
mx ∈ {−1x , 0x , +1x }
my ∈ {−1y , 0y , +1y }
mz ∈ {−1z, 0z, +1z}
(e.g. mx = +1x → x = x + 1)
Lombardi Giuseppe UAVs Traffic Rules 24 / 27

48. Neuroevolution
Neuroevolution
Neuroevolution:
used state-of-the-art → NEAT = NeuroEvolution of Augmenting
Topologies (EA)
evolves networks topology and weights
individuals = neural networks
Lombardi Giuseppe UAVs Traffic Rules 25 / 27

49. Neuroevolution
Neuroevolution
Neuroevolution:
used state-of-the-art → NEAT = NeuroEvolution of Augmenting
Topologies (EA)
evolves networks topology and weights
individuals = neural networks
fitness → minimize Average Final Distance (AFD)
f (NN) = AFD =
1
nactual
nactual
j=1
dj
f
captures a controller’s ability to move approaching the target
Lombardi Giuseppe UAVs Traffic Rules 25 / 27

50. Neuroevolution
Results
Trains with and without rules, 3 runs each with 300 individuals in 50
generations
0 20 40
0
1
2 Bad fitness
Good fitness
Generation
AFD(Fitness)
No rules
0 20 40
0
1
2 Bad fitness
Good fitness
Generation
Handwritten Rules
Lombardi Giuseppe UAVs Traffic Rules 26 / 27

51. Neuroevolution
Results
Trains with and without rules, 3 runs each with 300 individuals in 50
generations
0 20 40
0
1
2
Generation
AFD(Fitness)
No rules
0 20 40
0
1
2
Generation
Handwritten Rules
NNC, nmax = 5, with buildings
A*, nmax = 5, with buildings
Lombardi Giuseppe UAVs Traffic Rules 26 / 27

52. Neuroevolution
Future work
Results just presented need to be experimented further more.
Then:
More fine-grained model
Rules synthesis based on Grammatical Evolution
Co-Evolution of the rules and the controller
Lombardi Giuseppe UAVs Traffic Rules 27 / 27

53. Neuroevolution
Thanks!
Lombardi Giuseppe UAVs Traffic Rules 27 / 27