This document describes research on developing optimization and decision support tools for air traffic management. The research aims to build models of area supervisor decisions and algorithms to generate advisory schedules. A case study focuses on building a decision model and tool for area configuration selection that optimizes objectives like matching configurations to traffic levels while minimizing disruptions. The tool is evaluated using historical air traffic data and simulations, and is shown to better meet objectives compared to actual operations. Overall the research contributes improved models and algorithms for generating advisory schedules to support human air traffic managers.

1. Optimization and Analytics
for Air Traffic Management
Enabling Decision-Support Tools
Michael Bloem
Stanford University NASA Ames Research Center

2. Outline
• Air traffic management background
• Research topics
• Research objective and approach
• Case study:
decision-support tool for area supervisors
• Summary of contributions
2

3. Air Traffic Management is Important
3

4. Air Traffic Management is Important
civil aviation
was responsible for
5.4% of US GDP
in 2012
[FAA’s 2014 "The Economic Impact of Civil Aviation on the US Economy"]
3

5. Air Traffic Management is Important
civil aviation
was responsible for
5.4% of US GDP
in 2012
837 million passengers
carried by airlines
operating in US airspace
in 2012
[FAA’s 2014 "The Economic Impact of Civil Aviation on the US Economy"]
3

6. Air Traffic Management is Complex
4

7. Air Traffic Management is Complex
∼50,000 flights/day
5,000+ flights simultaneously
4

8. Air Traffic Management is
Accomplished Largely by
Human Decision Makers
5

9. Air Traffic Management is
Accomplished Largely by
Human Decision Makers
5

10. Air Traffic Management is
Accomplished Largely by
Human Decision Makers
5

11. Air Traffic Management is
Accomplished Largely by
Human Decision Makers
5

12. Air Traffic Management is
Accomplished Largely by
Human Decision Makers
5

13. Research Topics: ATM Decisions
6

14. Research Topics: ATM Decisions
1 Area supervisors:
area configuration selection
– Motivation: prescriptive decision model
6

15. Research Topics: ATM Decisions
1 Area supervisors:
area configuration selection
– Motivation: prescriptive decision model
2 Operations managers:
assignment of flights to slots
– Motivation: insight into airline delay costs
6

16. Research Topics: ATM Decisions
1 Area supervisors:
area configuration selection
– Motivation: prescriptive decision model
2 Operations managers:
assignment of flights to slots
– Motivation: insight into airline delay costs
3 Flow managers:
Ground Delay Program implementation
– Motivation: predictive capability and insights
6

17. Research Objective
decision-
support
tool
7

18. Research Objective
solution
algorithm
decision-
support
tool
decision
model
constraints
objective
7

19. Research Approach
expert input
& feedback
solution
algorithm
decision-
support
tool
decision
model
constraints
objective
8

20. Research Approach
expert input
& feedback
operational
decision
data
solution
algorithm
decision-
support
tool
decision
model
constraints
objective
8

21. Research Approach
fast-time
simulations
expert input
& feedback
operational
decision
data
solution
algorithm
decision-
support
tool
decision
model
constraints
objective
8

22. Research Approach
fast-time
simulations
human-in-
the-loop
simulations
expert input
& feedback
operational
decision
data
solution
algorithm
decision-
support
tool
decision
model
constraints
objective
8

23. Outline
• Air traffic management background
• Research topics
• Research objective and approach
• Case study:
decision-support tool for area supervisors
• Summary of contributions
9

24. En-Route Air Traffic Control Centers
10

25. En-Route Air Traffic Control Centers
10

26. Cleveland Center
11

27. Cleveland Center Area 4
46
45
12

28. Cleveland Center Area 4
47
48
49
4546
36,000
31,000
24,000Longitude
Latitude
Altitude
13

29. Sample Area of Specialization Resources
Sectors
47
48
49
4546 4546
47 49
48
14

30. Sample Area of Specialization Resources
Sectors
Controllers
available to fill
operating
positions
47
48
49
4546 4546
47 49
48
09:00–09:30 :
09:30–11:00 :
14

31. Sample Area of Specialization Resources
Sectors
Controllers
available to fill
operating
positions
Workstations
47
48
49
4546 4546
47 49
48
09:00–09:30 :
09:30–11:00 :
48
14

32. Modeling Decisions and Developing Algorithms
fast-time
simulations
human-in-
the-loop
simulations
expert input
& feedback
operational
decision
data
solution
algorithm
decision-
support
tool
decision
model
constraints
objective
15

33. Modeling Decisions and Developing Algorithms
fast-time
simulations
15

34. Sample Sectors, Configuration, and Traffic
sample sectors:
16

35. Sample Sectors, Configuration, and Traffic
sample sectors: at time step k:
Ck
16

36. Sample Sectors, Configuration, and Traffic
Tk
sample sectors: at time step k:
Ck
16

37. Decision Model
Configuration Schedule Advisory Problem (CSA)
minimize
K
k=1
gk(Ck−1, Tk−1, Ck, Tk)
subject to Ck ∈ Ck, k = 0, 1, 2, . . . , K
17

38. Decision Model
Configuration Schedule Advisory Problem (CSA)
minimize
K
k=1
gk(Ck−1, Tk−1, Ck, Tk)
subject to Ck ∈ Ck, k = 0, 1, 2, . . . , K
gk: single-time step advisory cost
17

39. Decision Model
Configuration Schedule Advisory Problem (CSA)
minimize
K
k=1
gk(Ck−1, Tk−1, Ck, Tk)
subject to Ck ∈ Ck, k = 0, 1, 2, . . . , K
gk: single-time step advisory cost
Ck: set of valid configurations at k
17

40. Sample Scenario
0
time
step1 2 3 4 5
18

41. Sample Scenario
0
time
step1 2 3 4 5
18

42. Sample Scenario
0
time
step1 2 3 4 5
18

43. Sample Scenario
0
time
step1 2 3 4 5
18

44. Sample Scenario
0
time
step1 2 3 4 5
18

45. Sample Scenario
0
time
step1 2 3 4 5
18

46. Configuration Constraints
0
time
step1 2 3 4 5
19

47. Configuration Constraints
0
time
step1 2 3 4 5
19

48. Configuration Constraints
0
time
step1 2 3 4 5
C2
19

49. Possible Algorithm Advisory
0
time
step1 2 3 4 5
20

50. Possible Algorithm Advisories
0
time
step1 2 3 4 5
21

51. Possible Algorithm Advisories
0
time
step1 2 3 4 5
25 = 32 possibilities
21

52. Advisory Cost: Static Cost
gS
k
(Ck, Tk)
20% 40% 60% 80% 100% 120%
0
2
4
6
8
10
Open Sector Load
[aircraft count/Monitor Alert Parameter]
Two−operating position
static costgS
k
(Ck, Tk)
22

53. Advisory Cost: Static Cost
gS
k
(Ck, Tk)
20% 40% 60% 80% 100% 120%
0
2
4
6
8
10
Open Sector Load
[aircraft count/Monitor Alert Parameter]
Two−operating position
static cost
One−operating position
static cost
gS
k
(Ck, Tk)
22

54. Advisory Cost: Reconfiguration Cost
gR
k
(Ck−1, Tk−1, Ck, Tk)
(1)
(1)
(2)
(1)
(2)
(2)
(1)
Ck−1, Tk−1 Ck, Tk
23

55. Advisory Cost: Reconfiguration Cost
gR
k
(Ck−1, Tk−1, Ck, Tk)
sector & flights moving
(1)
(1)
(2)
(1)
(2)
(2)
(1)
Ck−1, Tk−1 Ck, Tk
between workstations
23

56. Advisory Cost: Reconfiguration Cost
gR
k
(Ck−1, Tk−1, Ck, Tk)
open sector gaining
sector & flights moving
(1)
(1)
(2)
(1)
(2)
(2)
(1)
Ck−1, Tk−1 Ck, Tk
between workstations
2nd operating position
23

57. Advisory Cost: Reconfiguration Cost
gR
k
(Ck−1, Tk−1, Ck, Tk)
open sector gaining
open sector losingsector & flights moving
(1)
(1)
(2)
(1)
(2)
(2)
(1)
Ck−1, Tk−1 Ck, Tk
between workstations 2nd operating position
2nd operating position
23

58. Advisory Cost: Reconfiguration Cost
gR
k
(Ck−1, Tk−1, Ck, Tk)
open sector gaining
open sector losingsector & flights moving
(1)
(1)
(2)
(1)
(2)
(2)
(1)
Ck−1, Tk−1 Ck, Tk
between workstations 2nd operating position
2nd operating position
single-time step advisory cost:
gk(Ck−1, Tk−1, Ck, Tk) = gS
k
(Ck, Tk)+βR
gR
k
(Ck−1, Tk−1, Ck, Tk)
23

59. Advisory Cost: Reconfiguration Cost
gR
k
(Ck−1, Tk−1, Ck, Tk)
open sector gaining
open sector losingsector & flights moving
(1)
(1)
(2)
(1)
(2)
(2)
(1)
Ck−1, Tk−1 Ck, Tk
between workstations 2nd operating position
2nd operating position
single-time step advisory cost:
gk(Ck−1, Tk−1, Ck, Tk) = gS
k
(Ck, Tk)+βR
gR
k
(Ck−1, Tk−1, Ck, Tk)
operational decision data & fast-time simulations
leveraged to select βR
23

60. Minimum-Cost Advisory
0
time
step1 2 3 4 5
0
00 3
0 1
0 0
8
5
0
24

61. Minimum-Cost Advisory
0
time
step1 2 3 4 5
0
00 3
0 1
0 0
8
5
0
gS
k
(C1, T1)
24

62. Minimum-Cost Advisory
0
time
step1 2 3 4 5
0
00 3
0 1
0 0
8
5
0
gR
k
(C3, T3, C4, T4)
gS
k
(C1, T1)
24

63. Decision Model and Solution Algorithm
Configuration Schedule Advisory Problem (CSA)
minimize
K
k=1
gS
k
(Ck, Tk) + βR
gR
k
(Ck−1, Tk−1, Ck, Tk)
subject to Ck ∈ Ck, k = 0, 1, 2, . . . , K
25

64. Decision Model and Solution Algorithm
Configuration Schedule Advisory Problem (CSA)
minimize g(C, T)
subject to Ck ∈ Ck, k = 0, 1, 2, . . . , K
25

65. Decision Model and Solution Algorithm
Configuration Schedule Advisory Problem (CSA)
minimize g(C, T)
subject to Ck ∈ Ck, k = 0, 1, 2, . . . , K
Shortest path problem → use A∗ algorithm
25

66. Decision Model Relative to Previous Research:
Objectives and Constraints
Advisory
Characteristic
match
configurations to
traffic
few disruptive
reconfigurations
number of open
sectors
number of
operating
positions
26

67. Decision Model Relative to Previous Research:
Objectives and Constraints
Advisory
Characteristic
Cano et al.
(2007)
MB et al.
(2008–2009)
Tien et al.
(2010–2012)
match
configurations to
traffic
constraint
and
optimize
constraint constraint
few disruptive
reconfigurations
constraint
not
considered
constraint
number of open
sectors
minimize minimize
not
considered
number of
operating
positions
not
modeled
not modeled minimize
26

68. Decision Model Relative to Previous Research:
Objectives and Constraints
Advisory
Characteristic
Cano et al.
(2007)
MB et al.
(2008–2009)
Tien et al.
(2010–2012)
match
configurations to
traffic
constraint
and
optimize
constraint constraint
few disruptive
reconfigurations
constraint
not
considered
constraint
number of open
sectors
minimize minimize
not
considered
number of
operating
positions
not
modeled
not modeled minimize
26

69. Decision Model Relative to Previous Research:
Objectives and Constraints
Advisory
Characteristic
Cano et al.
(2007)
MB et al.
(2008–2009)
Tien et al.
(2010–2012)
Decision
Model
match
configurations to
traffic
constraint
and
optimize
constraint constraint optimize
few disruptive
reconfigurations
constraint
not
considered
constraint optimize
number of open
sectors
minimize minimize
not
considered
constraint
number of
operating
positions
not
modeled
not modeled minimize constraint
26

70. Decision Model Relative to Previous Research:
Objectives and Constraints
Advisory
Characteristic
Cano et al.
(2007)
MB et al.
(2008–2009)
Tien et al.
(2010–2012)
Decision
Model
match
configurations to
traffic
constraint
and
optimize
constraint constraint optimize
few disruptive
reconfigurations
constraint
not
considered
constraint optimize
number of open
sectors
minimize minimize
not
considered
constraint
number of
operating
positions
not
modeled
not modeled minimize constraint
26

71. Decision Model vs. Operational Decisions:
Problem Instances for Fast-Time Simulations
47
48
49
4546 4546
47 49
48
• traffic and configurations from
231 days in 2011 and 2012
• 6 am to midnight local time
• rolling horizon: implement first
hour of two-hour advisories
• five-minute time steps
27

72. Decision Model vs. Operational Decisions:
Few Disruptive Reconfigurations
3 6 9 12 15
0
500
1000
1500
2000
2500
Number of
Open Sector
Instances
Duration [hours]
operational
model
28

73. Decision Model vs. Operational Decisions:
Match Configurations to Traffic
20% 40% 60% 80% 100% 120%
0
2
4
6
8
10
Open Sector Load
[aircraft count/Monitor Alert Parameter]
Low High
Just Right
gS
k
(Ck, Tk)
29

74. Decision Model vs. Operational Decisions:
Match Configurations to Traffic
Low Just Right High0
20
40
60
80
100
Percent of
Open Sector-
Minutes
operational
model
30

75. Modeling Decisions and Developing Algorithms
fast-time
simulations
31

76. Modeling Decisions and Developing Algorithms
fast-time
simulations
31

77. Experts’ Feedback and Suggestion
Limitations of decision model
32

78. Experts’ Feedback and Suggestion
Limitations of decision model
1 difficult to quantify safe and efficient operations
32

79. Experts’ Feedback and Suggestion
Limitations of decision model
1 difficult to quantify safe and efficient operations
2 does not model individual human controllers
32

80. Experts’ Feedback and Suggestion
Limitations of decision model
1 difficult to quantify safe and efficient operations
2 does not model individual human controllers
Suggestions for solution algorithm
32

81. Experts’ Feedback and Suggestion
Limitations of decision model
1 difficult to quantify safe and efficient operations
2 does not model individual human controllers
Suggestions for solution algorithm
• return ∼ 3 good advisories
32

82. Experts’ Feedback and Suggestion
Limitations of decision model
1 difficult to quantify safe and efficient operations
2 does not model individual human controllers
Suggestions for solution algorithm
• return ∼ 3 good advisories
• one advisory: optimal for decision model
32

83. Experts’ Feedback and Suggestion
Limitations of decision model
1 difficult to quantify safe and efficient operations
2 does not model individual human controllers
Suggestions for solution algorithm
• return ∼ 3 good advisories
• one advisory: optimal for decision model
• other advisories: not too sub-optimal
32

84. Experts’ Feedback and Suggestion
Limitations of decision model
1 difficult to quantify safe and efficient operations
2 does not model individual human controllers
Suggestions for solution algorithm
• return ∼ 3 good advisories
• one advisory: optimal for decision model
• other advisories: not too sub-optimal
• distinct advisories
32

85. Advisory Difference Metric
Φ(C, C′
)
Φ(C, C′
) =
K
k=1
ϕ(Ck, C′
k
)
33

86. Advisory Difference Metric
Φ(C, C′
)
Φ(C, C′
) =
K
k=1
ϕ(Ck, C′
k
)
Configuration difference metric
ϕ(Ck, C′
k
) =
1 if Ck and C′
k
combine sectors differently
0 else
33

87. Multiple-Advisories Problem Statement
M ϵ-Optimal d-Distinct Configuration Schedule
Advisories Problem (M-ϵ-d-CSAs)
minimize
M
m=1
g(Cm
, T)
subject to |CM
| = M
Cm
k
∈ Ck k = 0, 1, 2, . . . , K, m = 1, 2, . . . , M
C1
∈ C⋆
CSA
( = optimal advisories for CSA)
g(Cm
, T) − g(C1
, T)
g(C1, T)
≤ ϵ m = 2, 3, . . . , M
Φ(Cm
, Cm′
) ≥ d ∀m, m′
where m = m′
34

88. Multiple-Advisories Problem Statement
M ϵ-Optimal d-Distinct Configuration Schedule
Advisories Problem (M-ϵ-d-CSAs)
minimize
M
m=1
g(Cm
, T)
subject to |CM
| = M
Cm
k
∈ Ck k = 0, 1, 2, . . . , K, m = 1, 2, . . . , M
C1
∈ C⋆
CSA
( = optimal advisories for CSA)
g(Cm
, T) − g(C1
, T)
g(C1, T)
≤ ϵ m = 2, 3, . . . , M
Φ(Cm
, Cm′
) ≥ d ∀m, m′
where m = m′
M-ϵ-d-CSAs is NP-complete
34

89. Algorithms for M-ϵ-d-CSAs
35

90. Algorithms for M-ϵ-d-CSAs
1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)
• novel algorithm based on A∗
35

91. Sequential Distinct A∗
(SDA∗
)
Finding advisory C1:
C, T C1shortest path
algorithm
36

92. Sequential Distinct A∗
(SDA∗
)
Finding advisory Cm:
C, T Cm
{C1, C2, . . . , Cm−1}
constrained
shortest path
algorithm
36

93. Sequential Distinct A∗
(SDA∗
)
Finding advisory Cm:
C, T Cm
{C1, C2, . . . , Cm−1}
constrained
shortest path
algorithm
constrained shortest path problem is NP-complete
36

94. Constrained Shortest Path Algorithm:
Forward Distinct A∗
(FDA∗
)
For some λ ≥ 0, find ˜C2 that minimizes Lagrangian
L(C2
, λ) = g(C2
, T)
advisory
cost
+λ d − Φ(C1
, C2
)
similarity
cost
37

95. Constrained Shortest Path Algorithm:
Forward Distinct A∗
(FDA∗
)
For some λ ≥ 0, find ˜C2 that minimizes Lagrangian
L(C2
, λ) = g(C2
, T)
advisory
cost
+λ d − Φ(C1
, C2
)
similarity
cost
Another shortest path problem → use A∗
37

96. FDA∗
and Duality
Proposition
If M = 2, ϵ = ∞, and |C⋆
CSA
| = 1,
then FDA∗ implements the dual objective:
h(λ) = minimize
C2∈C
L(C2
, λ)
38

97. FDA∗
and Duality
Proposition
If M = 2, ϵ = ∞, and |C⋆
CSA
| = 1,
then FDA∗ implements the dual objective:
h(λ) = minimize
C2∈C
L(C2
, λ)
Corollary: If M = 2, ϵ = ∞, |C⋆
CSA
| = 1,
λ = λ⋆
, and strong duality holds,
then ˜C2 satisfies a necessary condition for C2⋆
(˜C2 = second advisory returned by FDA∗)
38

98. Sequential Distinct A∗
(SDA∗
)
Finding advisory Cm:
C, T Cm
{C1, C2, . . . , Cm−1}
constrained
shortest path
algorithm
constrained shortest path problem is NP-complete
39

99. Sequential Distinct A∗
(SDA∗
)
Finding advisory Cm:
C, T Cm
{C1, C2, . . . , Cm−1}
FDA∗
λ
39

100. Algorithms for M-ϵ-d-CSAs
1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)
• novel algorithm based on A∗
40

101. Algorithms for M-ϵ-d-CSAs
1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)
• novel algorithm based on A∗
2 Forward Backward Value Iteration
with Sequential Advisory Search (FBVISAS)
• novel algorithm based on value iteration
40

102. Forward Backward Value Iteration
with Sequential Advisory Search (FBVISAS)
0
time
step1 2 3 4 5
41

103. Forward Backward Value Iteration
with Sequential Advisory Search (FBVISAS)
0
time
step1 2 3 4 5
41

104. Forward Backward Value Iteration
with Sequential Advisory Search (FBVISAS)
0
time
step1 2 3 4 5
41

105. Forward Backward Value Iteration
with Sequential Advisory Search (FBVISAS)
0
time
step1 2 3 4 5
41

106. Theoretical Guarantees for FBVISAS:
the Good News
42

107. Theoretical Guarantees for FBVISAS:
the Good News
Simple M-ϵ-d-CSAs instance: M = 2, ϵ = ∞, |C⋆
CSA
| = 1,
C1 = C2 = · · · = CK, and C0 ∈ C1
42

108. Theoretical Guarantees for FBVISAS:
the Good News
Simple M-ϵ-d-CSAs instance: M = 2, ϵ = ∞, |C⋆
CSA
| = 1,
C1 = C2 = · · · = CK, and C0 ∈ C1
Reconfiguration cost-dominated M-ϵ-d-CSAs instance:
reconfiguring never decreases the advisory cost
42

109. Theoretical Guarantees for FBVISAS:
the Good News
Simple M-ϵ-d-CSAs instance: M = 2, ϵ = ∞, |C⋆
CSA
| = 1,
C1 = C2 = · · · = CK, and C0 ∈ C1
Reconfiguration cost-dominated M-ϵ-d-CSAs instance:
reconfiguring never decreases the advisory cost
Proposition
If one exists, FBVISAS finds an optimal C2⋆
for
simple reconfiguration cost-dominated instances
42

110. Theoretical Guarantees for FBVISAS:
the Bad News
Static cost-dominated M-ϵ-d-CSAs instance: βR = 0
43

111. Theoretical Guarantees for FBVISAS:
the Bad News
Static cost-dominated M-ϵ-d-CSAs instance: βR = 0
Proposition
If d > 1, FBVISAS does not return any C2 for
simple static cost-dominated instances
43

112. Algorithms for M-ϵ-d-CSAs
1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)
• novel algorithm based on A∗
2 Forward Backward Value Iteration
with Sequential Advisory Search (FBVISAS)
• novel algorithm based on value iteration
44

113. Algorithms for M-ϵ-d-CSAs
1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)
• novel algorithm based on A∗
2 Forward Backward Value Iteration
with Sequential Advisory Search (FBVISAS)
• novel algorithm based on value iteration
3 Lowest-Cost Paths (LCP)
• finds optimal solution to relaxation
• many efficient algorithms
44

114. Algorithms for M-ϵ-d-CSAs
1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)
• novel algorithm based on A∗
2 Forward Backward Value Iteration
with Sequential Advisory Search (FBVISAS)
• novel algorithm based on value iteration
3 Lowest-Cost Paths (LCP)
• finds optimal solution to relaxation
• many efficient algorithms
4 Value Iteration Fraction Optimal
with Exhaustive Advisory Search
• finds optimal solution
• not computationally efficient
44

115. Modeling Decisions and Developing Algorithms
fast-time
simulations
45

116. Modeling Decisions and Developing Algorithms
fast-time
simulations
45

117. Evaluation of Algorithms on Small Instances
47
48
49
4546 4546
47 49
48
• traffic and configurations from
two days in December 2011
• nine two-hour instances per day
• 18 total instances
• five-minute time steps
• C requires same number of open
sectors as were used operationally
• request two advisories (M = 2)
• ϵ = 0.2 constraint on cost of C2
• d requires 30 minutes of different
airspace configurations
46

118. Small Instances: Properties of C2
SDA∗
-SC FBVISAS LCP
0
20
40
60
80
100
Percent of
Instances
optimal C2⋆
47

119. Small Instances: Properties of C2
SDA∗
-SC FBVISAS LCP
0
20
40
60
80
100
Percent of
Instances
optimal C2⋆
feasible C2
47

120. Evaluation of Algorithms on Realistic Instances
47
48
49
4546 4546
47 49
48
• traffic and configurations from
231 days in 2011 and 2012
• 6 am to midnight local time
• rolling horizon: implement first
hour of two-hour advisories
• 4158 total instances
• five-minute time steps
• C only requires same initial
configuration as was used
operationally
• request three advisories (M = 3)
• ϵ = 0.5 constraint on cost
• d requires 30 minutes of different
airspace configurations
48

121. Realistic Instances: Advisory Costs
0.7 0.8 0.9 1.1 1.2 1.3
0
20
40
60
80
100
0.99 1.01
Percent of
Instances
Cost Ratio
g(C2
,T)+g(C3
,T) for SDA∗
-SC
g(C2,T)+g(C3,T) for FBVISAS
SDA∗-SC
worse
SDA∗-SC
better
mean = 1.02
49

122. Realistic Instances: Computation Times
(on a Workstation)
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Computation
Times
[seconds]
SDA∗-SC FBVISAS
50

123. Modeling Decisions and Developing Algorithms
fast-time
simulations
51

124. Modeling Decisions and Developing Algorithms
51

125. Operational Airspace Sectorization
Integrated System (OASIS)
52

126. OASIS Screenshot: Multiple Advisories
53

127. Human-in-the-Loop Simulations Set Up
• eight retired FAA
personnel
• four simulated
scenarios
54

128. Human-in-the-Loop Simulations Set Up
• eight retired FAA
personnel
• four simulated
scenarios
Three experimental conditions in which users
1 generate configuration schedule
2 select from among algorithm-generated advisories
3 select from among algorithm-generated advisories
and then modify
54

129. Human-in-the-Loop Simulations Results
[Lee et al., 2013]
55

130. Human-in-the-Loop Simulations Results
[Lee et al., 2013]
Advisories enable safe and efficient operations
• average acceptability of selected
algorithm-generated advisories: > 4 out of 5
• user modifications were minor and led to
no significant improvement in acceptability
55

131. Human-in-the-Loop Simulations Results
[Lee et al., 2013]
Advisories enable safe and efficient operations
• average acceptability of selected
algorithm-generated advisories: > 4 out of 5
• user modifications were minor and led to
no significant improvement in acceptability
Providing multiple advisories added value
• in more than 60% of instances, users selected
second or third advisory
• when asked how many advisories they wanted the
tool to provide, users requested an average of 2.8
55

132. Human-in-the-Loop Simulations Results
[Lee et al., 2013]
Advisories enable safe and efficient operations
• average acceptability of selected
algorithm-generated advisories: > 4 out of 5
• user modifications were minor and led to
no significant improvement in acceptability
Providing multiple advisories added value
• in more than 60% of instances, users selected
second or third advisory
• when asked how many advisories they wanted the
tool to provide, users requested an average of 2.8
Computation times were rated as highly acceptable
55

133. Modeling Decisions and Developing Algorithms
56

134. Outline
• Air traffic management background
• Research topics
• Research objective and approach
• Case study:
decision-support tool for area supervisors
• Summary of contributions
57

135. Summary of Contributions
58

136. Summary of Contributions
1 Area supervisors:
area configuration selection
– developed prescriptive decision model
– designed efficient algorithms to find
low-cost and distinct paths
– implemented algorithm in decision-support tool
58

137. Summary of Contributions
1 Area supervisors:
area configuration selection
– developed prescriptive decision model
– designed efficient algorithms to find
low-cost and distinct paths
– implemented algorithm in decision-support tool
2 Operations managers:
assignment of flights to slots
– designed maximum-likelihood-based algorithm that
– generated insights from operational decision data
58

138. Summary of Contributions
1 Area supervisors:
area configuration selection
– developed prescriptive decision model
– designed efficient algorithms to find
low-cost and distinct paths
– implemented algorithm in decision-support tool
2 Operations managers:
assignment of flights to slots
– designed maximum-likelihood-based algorithm that
– generated insights from operational decision data
3 Flow managers:
Ground Delay Program implementation
– deployed supervised learning and inverse
reinforcement learning models that
– produced predictive capability and insights
58

139. Relevant peer-reviewed conference papers
1 M. Bloem and P. Gupta, "Configuring Airspace Sectors with Approximate Dynamic
Programming," ICAS, September 2010.
2 M. Bloem and H. Huang, "Evaluating Delay Cost Functions with Airline Actions in
Airspace Flow Programs," USA/Europe ATM R&D Seminar, June 2011.
3 M. Bloem and N. Bambos, "Coordinated Tactical Air Traffic and Airspace Management,"
IEEE CDC, December 2011.
4 M. Bloem, M. Drew, C. F. Lai, and K. Bilimoria, "Advisory Algorithm for Scheduling Open
Sectors, Operating Positions, and Workstations," AIAA ATIO, September 2012.
5 M. Bloem, H. Huang, and N. Bambos, "Approximating the Likelihood of Historical Airline
Actions to Evaluate Airline Delay Cost Functions," IEEE CDC, December 2012.
6 M. Bloem and N. Bambos, "An Approach for Finding Multiple Area of Specialization
Configuration Advisories," AIAA ATIO, August 2013.
7 M. Bloem and N. Bambos, "Ground Delay Program Analytics with Behavioral Cloning
and Inverse Reinforcement Learning," AIAA ATIO, June 2014.
8 M. Bloem and N. Bambos, "Infinite Time Horizon Maximum Causal Entropy Inverse
Reinforcement Learning," IEEE CDC, December 2014.
Relevant journal articles
1 M. Bloem, P. Gupta, and P. Kopardekar, "Algorithms for Combining Airspace Sectors," Air
Traffic Control Quarterly, Vol. 17, No. 4, 2009.
2 M. Bloem, M. Drew, C. F. Lai, and K. D. Bilimoria, "Advisory Algorithm for Scheduling Open
Sectors, Operating Positions, and Workstations," AIAA Journal of Guidance, Control, and
Dynamics, Vol. 37, No. 4, July–August 2014.
3 M. Bloem and N. Bambos, "Air Traffic Control Configuration Advisories from
Near-Optimal Distinct Paths," AIAA Journal of Aerospace Information Systems, Vol. 11,
No. 11, November 2014.
4 M. Bloem and N. Bambos, "Ground Delay Program Analytics with Behavioral Cloning
and Inverse Reinforcement Learning," AIAA Journal of Aerospace Information Systems,
accepted on 21 December 2014, available online on 03 March 2015.
5 M. Bloem and N. Bambos, "Stochastic Models of Ground Delay Program
Implementation for Prediction, Simulation, and Insight," Journal of Aerospace
Operations, invited submission to special issue, expected publication in late 2015.
59

140. Acknowledgments
• Stanford
• NASA
• Family & friends
60

141. Questions?

142. Backup Slides
62

143. Why Improve Area Configuration Planning?
1 more efficient flights
2 fewer disruptions to area supervisors and
traffic managers
3 better controller staff management
4 prepare for increased flexibility in future operations
→ more complex planning
63

144. Modeling Decisions and Developing Algorithms
fast-time
simulations
64

145. Operational Decision Data Set and
Fast-Time Simulation Setup
47
48
49
4546 4546
47 49
48
• traffic and configurations from
230 days in 2011 and 2012
• 6 am to midnight local time
• rolling horizon: implement first
hour of two-hour advisories
• five-minute time steps
• only airspace configurations
• constraint: only use typical
configurations
65

146. Static Cost versus Reconfiguration Cost
700 800 900 1000 1100
0
50
100
150
200
Total
Reconfiguration
Cost
Total Static Cost
operational
βR = 0.5
βR
= 15
ratio(βR) =
total reconfiguration cost(βR)
total static cost(βR)
error(βR) = ratio(operational) − ratio(βR)
66

147. Selecting βR
0 1.75 5.0 10 15
5
10
15
20
25
Total Error
βR
set βR = 1.75
67

148. Uncertainty in Traffic Predictions:
Why Not Incorporated?
• Experts understand prediction errors
• Avoid inconsistency with other deterministic tools
68

149. Uncertainty in Traffic Predictions:
Uncertain Future Aircraft Counts
0 5 10 15 20
0
0.05
0.1
0.15
0.2
0.25
Aircraft Count
Probability prediction
distribution
69

150. Uncertainty in Traffic Predictions:
Impact on Advisory Costs
Advisories generated using predicted traffic with realistic errors
are typically 2.5%–12.5% costlier
than advisories generated with perfectly-predicted traffic
70

151. Uncertainty in Traffic Predictions:
Approximate Dynamic Programming Approach
• Finite time horizon Markov decision process (MDP)
– state: sector aircraft counts and predictions thereof
– action: current configuration (not a schedule)
– state transitions: based on prediction errors in
operational tool
– objective: similar to decision model (CSA)
• Rollouts algorithm performs 15% better than a
heuristic and within 2% of optimal
• Rollouts algorithm computes solutions fast enough
for real-time implementation
M. Bloem and P. Gupta, "Configuring Airspace Sectors with
Approximate Dynamic Programming," ICAS, September 2010.
71

152. Uncertainty in Traffic Predictions:
Other Approaches
• improve predictions with a statistical model
• discount contribution to cost by predicted aircraft
• stochastic shortest path problem
• risk-averse advisory
• larger MDP formulation
• robust set of advisories
72

153. SDA∗
with Shortcuts (SDA∗
-SC)
Objectives:
1 computation time ≤ that of A∗
for each advisory
→ use data from reverse A∗ for C1
to find "shortcuts"
2 no re-tuning of λ per instance or advisory
→ normalize advisory cost and similarity cost terms
73

154. Forward Backward Value Iteration
with Sequential Advisory Search (FBVISAS)
• Use value iteration to find minimum-cost advisory
through each Ck ∈ Ck, k = 1, . . . , K
• Sort these advisories from low to high cost
• Initialize set of advisories to return: CM ← ∅
• Loop:
– select next advisory
– if advisory is sufficiently different from other
advisories in CM, add it to CM
– if |CM| = M, stop
74

155. Multiple Advisories Problem is NP-Complete
For an arbitrary instance of the independent set
problem on G(V, E), construct M-ϵ-d-CSAs instance:
• M = required number of independent nodes
• gk() = 0
• K = 1 and C1 = V
• d = 1
• configuration difference metric:
ϕ(Ck, C′
k
) =
1 if Ck and C′
k
independent in G(V, E)
0 else
∃ solution to the independent set instance
⇔ ∃ solution to the M-ϵ-d-CSAs instance
75

156. Computational Complexity
of Multiple Solutions Problem Algorithms
Algorithm Complexity
VIFOEAS O(n3K)a
FBVISAS O(n2
K + nK log(nK + 1))
SDA∗-SC O(Mn2K log(nK + 1))
Eppstein lowest-cost paths O(n2
K + nK log(nK + 1) + M)
Suurballe lowest-cost node-disjoint paths O(Mn2K log(nK + 1))b
a This does not include the complexity of searching through
|Cϵ|
M
advisory
subsets.
b This assumes that Dijkstra’s algorithm is used as a subroutine for finding
shortest paths.
76

157. Airline Delay Cost Model Evaluation
decision-
support
tool
77

158. Airline Delay Cost Model Evaluation
Motivation: how costly is a flight delay?
78

159. Airline Delay Cost Model Evaluation
Motivation: how costly is a flight delay?
Operational decision data:
default assignment
ABC1
10:00
ABC2
10:30
Slot #60
11:00
Slot #90
12:00
78

160. Airline Delay Cost Model Evaluation
Motivation: how costly is a flight delay?
Operational decision data:
airline-selected assignment
ABC1
10:00
ABC2
10:30
Slot #60
11:00
Slot #90
12:00
78

161. Airline Delay Cost Model Evaluation
Motivation: how costly is a flight delay?
Operational decision data:
airline-selected assignment
ABC1
10:00
ABC2
10:30
Slot #60
11:00
Slot #90
12:00
Contribution: novel algorithm enables determination of
delay cost model and cost noise parameters that
maximize an approximation of likelihood of data
78

162. Airline Delay Cost Model Evaluation
• developed novel algorithm for finding cost noise
parameters that maximize an approximation of the
likelihood of minimum-cost matching problem
instance solutions, given a candidate cost model
79

163. Airline Delay Cost Model Evaluation
• developed novel algorithm for finding cost noise
parameters that maximize an approximation of the
likelihood of minimum-cost matching problem
instance solutions, given a candidate cost model
• evaluated more than ten proposed airline delay
cost models using hundreds of slot swap decisions
79

164. Airline Delay Cost Model Evaluation
• developed novel algorithm for finding cost noise
parameters that maximize an approximation of the
likelihood of minimum-cost matching problem
instance solutions, given a candidate cost model
• evaluated more than ten proposed airline delay
cost models using hundreds of slot swap decisions
• determined delay cost model that maximizes
approximate likelihood of data for each airline
79

165. Airline Delay Cost Model Evaluation
• developed novel algorithm for finding cost noise
parameters that maximize an approximation of the
likelihood of minimum-cost matching problem
instance solutions, given a candidate cost model
• evaluated more than ten proposed airline delay
cost models using hundreds of slot swap decisions
• determined delay cost model that maximizes
approximate likelihood of data for each airline
• estimated cost noise parameters for each delay
cost model for each airline
79

166. Ground Delay Program (GDP)
Implementation Modeling
decision-
support
tool
80

167. Ground Delay Program (GDP)
Implementation Modeling
Operational decision data:
arrival rate control arrivals
start end per
airport time time hour
SFO 9:00 am noon 30
81

168. Ground Delay Program (GDP)
Implementation Modeling
Operational decision data:
arrival rate control arrivals
start end per
airport time time hour
SFO 9:00 am noon 30
Motivation: predict & understand GDP implementation
81

169. Ground Delay Program (GDP)
Implementation Modeling
Operational decision data:
arrival rate control arrivals
start end per
airport time time hour
SFO 9:00 am noon 30
Motivation: predict & understand GDP implementation
Contribution: produced predictive models and insights
by developing behavioral cloning and inverse
reinforcement learning models of GDP implementation
81

170. Ground Delay Program (GDP)
Implementation Modeling
• evaluated behavioral cloning and inverse
reinforcement learning models of GDP
implementation at EWR and SFO using hundreds of
days of operational decision data
82

171. Ground Delay Program (GDP)
Implementation Modeling
• evaluated behavioral cloning and inverse
reinforcement learning models of GDP
implementation at EWR and SFO using hundreds of
days of operational decision data
• demonstrated that a behavioral cloning model can
produce superior predictions of GDP
implementation
82

172. Ground Delay Program (GDP)
Implementation Modeling
• evaluated behavioral cloning and inverse
reinforcement learning models of GDP
implementation at EWR and SFO using hundreds of
days of operational decision data
• demonstrated that a behavioral cloning model can
produce superior predictions of GDP
implementation
• determined that neither class of model provides
much evidence that conditions beyond those in the
next two hours impact GDP implementation
82

173. Ground Delay Program (GDP)
Implementation Modeling
• evaluated behavioral cloning and inverse
reinforcement learning models of GDP
implementation at EWR and SFO using hundreds of
days of operational decision data
• demonstrated that a behavioral cloning model can
produce superior predictions of GDP
implementation
• determined that neither class of model provides
much evidence that conditions beyond those in the
next two hours impact GDP implementation
• estimated a reward function that provides insight
into metrics guiding GDP implementation
82