Performance Management of Functional Airspace Blocks
Paula R MARK
ISIATM 2013
Istanbul
Introduction
• Why do we need performance management for
the Functional Airspace Block Approach
(FABA)?
• ​How may we evaluate FAB performance? ​
• ​Is there any future scope?
The performance of any organization is impacted
upon by a variety of socioeconomic latent factors
of hybridization. (Ménard, 2003).
The performance management of air traffic control
systems of the FABA are not exempt from this
universal economic consequence.
• To illustrate:
Imagine that you have a business and due to an international regulation
you have to merge with a similar business in another country.
Can you identify the benefits of this hybridization?
What might make this merger difficult?
We call the obstacles the latent factors.
The FAB initiative is a countermeasure to
diminish inefficiencies in the ATM system
It is an integrated airspace concept that is
designed to group states providing air
navigation services (ANSPs) into
functional blocks or teams according to
uniform requirements so as to optimise
efficiency
What is the FAB initiative?
The merging of airspaces into functional blocks
or teams are also composite or hybrid airspaces
that should facilitate improvements in
performance.
Performance Management (PM) is
about the evaluation of the merging
process
One way of managing the
performance of these airspace
blocks is to use CBA (Cost Benefit
Analysis) which is based on the
assumption of generating revenue
The FAB Approach (FABA) is a consolidation of
organizational strategy and an alignment of
resources.
The PM of FABs involves monitoring the
performance of blocks of airspaces using
organizational, technical and economic metrics
These are the metrics that describe how each ANSP operate and
how they align their resources in the merging or hybridization
process of aviation blocks
But how do we know which metrics of performance are the best?
Also how can we quantify latency in an environment where the
data is inconsistent?
Economic hybridization while highly beneficial also yields
increased latency as a byproduct in merging systems
Latent factors of hybridization
What happens when organizations merge?
- Organizational Dysfunction becomes magnified after a
hybrid process
This deficiency impacts negatively upon ANSP performance
which in turn affects the FAB scenario
According to the economic principle of marginal substitution
- there will always be a tradeoff between the cost and the
quality of ATM services
- the principle is extended to optimizing efficiency which is a
composite good
Each consumer/ANSP will substitute differently to keep their
maximised utility constant
Optimal efficiency will exist in a state of flux
- Given the conflicting views of stakeholders, it is not easy
for ANSPs to reconcile the aims of regulatory agencies,
interested parties, airline companies and ATCOs under the
umbrella of ATM hybridization
FABs are like icebergs, a beautiful
economic concept but -
The presence of Latent factors
makes FAB performance difficult to
measure
How do we manage the
performance of these FABs?
Managing FAB performance
We can look at the Efforts that each ANSP made in subscribing to
the FABA.
But this consolidation process is fairly nascent.
Instead, we will evaluate the
parameters of performance, the
KPAs in a multilevel setting via a
gllamm analysis
The word gllamm stands for generalised
latent linear and mixed modelling in advanced
statistics.
This model is useful with inconsistent data
from ambiguous environments which is typical
of air traffic control.
From a statistical viewpoint, the underlying
principle of gllamm analysis is based on the
maximum likelihood of an outcome given the
limitations of a group of test variables.
In the domain of social sciences, the gllamm
technique is used either as a preliminary
process to test the sutitability of parameters in
advanced regression analyses.
Or, it is used in multi level analyses to explore,
predict and validate relationships between the
test variables and the outcome in each group
Size of sample and number of test variables will
affectthe time for the analysis.
Limitations of the gllamm technique
To manage FAB performance, we assume that FAB
efficiency is a function of the efficiencies of a group of
ANSPs which in turn is a function of specific
performance metrics that are organizational, technical
and economic.
FAB performance = f {OTE parameters, latent effects}
We also take into considereation the latent effects of
hybridization.
The data is derived from 4 datasets that measured
diverse indications of performance for 31 ANSPs
from the period 2006-2013 that are divided into 9
airspace blocks
The statististical technique: stepwise gllamm
Number of FABS: 9
Number of ANSPs; 28
Data provided from EUROCONTROL dashboard
and Performance Review Unit
Software used: STATA
The gllamm technique tests the theory that FAB performance
is a function of ANSP performance in each state.
In turn, ANSP performance is a result of a combination of
technical, economic and organizational parameters.
The gllamm technique also takes into account the byproduct of
latent or random effects at the hybridized level.
We are making the assertion that the efficiency or performance of a FAB is
affected by the productivity of air navigation services (ANS) per state.
If common strategic objectives are met that yield increases in safety
performance, the state's inefficiency in providing ANS decreases and in
turn, the FAB efficiency increases.
Evaluating the metrics (KPAs) of the FABA
FAB efficiency
of group j with i
ANSPs
Vector of the
KPAs foe the ith
ANSP of the jth
FAB
The latent
effects that
impact upon the
performance of
the jth FAB
Key Performance Areas Variable Ability to predict FAB
performance at 90% statistical
significance
FAB performance FAB Efficiency Dependent variable
(FABE)
Dependent variable
Safety Performance and
Management
Effectiveness of Safety Management
(ESM)
Significant predictor
Economic efficiency Costs of gatetodelays per composite
flight hour ( COD )
Military airspace Thriftiness ( MTS)
Significant predictor
Not a significant predictor
Operational efficiency Volume of Controlled Airspace
(VCA)
Number of Operational units:
ACCs, TWRs and AFIS (NOU)
Size of staff (SOS)
Scope of services (OSS)
Significant predictor
Not a very significant predictor
Not a significant predictor
Not a significant predictor
Technical efficiency Aggregated complexity score (ACS) Significant predictor
Environmental efficiency Traffic Variability Indicator ( TVI) Not a very significant predictor
Innovation Innovation Strategy (IST) Not a significant predictor
The gllamm confirms that these Performance metrics or KPA’s:
Cost of delays
Safety management
Volume of airspace
Number of operational units and
Level of complexity
are significant predictors of FAB performance
Predictors of FAB
performance at the
state level
p value z value Effect upon FAB
performance at 90%
statistical significance
Safety management .005 2.79 Decrease
Cost of delays .000 3.54 Decrease
Volume of controlled
airspace
.000 -3.68 Decrease
Number of operational
units
.024 2.26 ncrease
Traffic variability indicator .0481 -0.71 Increase
Complexity .069 1.82 Increase
Organizational efficiency .000 456.01 Increase
Variance at FAB level
Log likelihood Number of
level 1 units
Number of level 2 units
Adjusted R^2
.02
123.597
28
9
.63
What scope is there for further
study?
-repeat the gllamm for another 3-5
years
-change the variables
-carry out benchmarking studies
Do you recall?
Why do we need PM for FABA?
How may we evaluate FAB
performance?
What scope is there for further
study?
References
[1] Dr. Aubrey C. Daniels, Performance Management: Changing Behavior That
Drives Organizational Effectiveness], 4th ed., Performance Management
Publications, 1981, 1984, 1989, 2006. ISBN 0-937100-08-0
[2] Sophia Rabe-Hesketh and Anders Skrondal, Multilevel and Longitudinal
Modeling Using Stata, 2nd ed., ISBN-13: 978-1-59718-008-5
[3] Geeta Kumari, Neha Kaleramna and K.M.Pandey, Study on Performance
Management System of Private Companies: a Case Study of Endurance,
International Journal of Innovation, Management and Technology, Vol. 1, No. 5,
December 2010 ISSN: 2010-0248
[4] PMeZine the Performance Management Magazine, November 13th, 2000 «
What is Performance Management », an interview with Dr. Aubry Daniels
[5] Prof. Claude Ménard, 2004, « The economics of hybrid organizations »,
The Journal of Institutional and Theoretical Economics, JITE (160) 1-32,Mohr
Siebeck – ISSN 0932-4569
[6] Lynn Godkin and Seth Allcorn, 2009, "Institutional narcissism, arrogant
organization disorder and interruptions in organizational learning", The Learning
Organization, Vol. 16 (1) 40-57
[7] Prof. Andy Neely, Chris Adams and Mike Kennerley, the
Performance Prism: the Scorecard for Measuring and Managing
Business Success, PearsonEducation 2002 ISBN: 0273653342
[8] Kim Andersson, 2012 Multifunctional Wetlands and Stakeholder
Engagement: Lessonsfrom Sweden, Working Paper Stockholm
Environmental Institute
[9] Rabe-Hesketh and Skrondal, 2008 Multilevel and Longitudinal
Modelling using STATA, 2nd edition STATA Press USA ISBN:
9781597180405
[10] J. B. Osterstock, J.C. MacDonald, M.M. Bogesssand M.S.
Brown, 2010,“Technical note: Analysis of ordinal outcomes from
carcass data in beef cattle research”, J. Anim. Sci., 88:3384-3389
[11] Strategy, Organizational Effectivenessand Performance
Management: from Basics to Best practices by Soren Dressler, 2004
Universal Publishers ISBN: 1-58112-532-1
Performance Management of Functional Airspace Blocks

Performance Management of Functional Airspace Blocks

  • 1.
    Performance Management ofFunctional Airspace Blocks Paula R MARK ISIATM 2013 Istanbul
  • 2.
    Introduction • Why dowe need performance management for the Functional Airspace Block Approach (FABA)? • ​How may we evaluate FAB performance? ​ • ​Is there any future scope?
  • 3.
    The performance ofany organization is impacted upon by a variety of socioeconomic latent factors of hybridization. (Ménard, 2003). The performance management of air traffic control systems of the FABA are not exempt from this universal economic consequence.
  • 4.
    • To illustrate: Imaginethat you have a business and due to an international regulation you have to merge with a similar business in another country. Can you identify the benefits of this hybridization? What might make this merger difficult? We call the obstacles the latent factors.
  • 5.
    The FAB initiativeis a countermeasure to diminish inefficiencies in the ATM system It is an integrated airspace concept that is designed to group states providing air navigation services (ANSPs) into functional blocks or teams according to uniform requirements so as to optimise efficiency What is the FAB initiative?
  • 6.
    The merging ofairspaces into functional blocks or teams are also composite or hybrid airspaces that should facilitate improvements in performance.
  • 7.
    Performance Management (PM)is about the evaluation of the merging process
  • 8.
    One way ofmanaging the performance of these airspace blocks is to use CBA (Cost Benefit Analysis) which is based on the assumption of generating revenue
  • 9.
    The FAB Approach(FABA) is a consolidation of organizational strategy and an alignment of resources. The PM of FABs involves monitoring the performance of blocks of airspaces using organizational, technical and economic metrics
  • 10.
    These are themetrics that describe how each ANSP operate and how they align their resources in the merging or hybridization process of aviation blocks But how do we know which metrics of performance are the best?
  • 11.
    Also how canwe quantify latency in an environment where the data is inconsistent? Economic hybridization while highly beneficial also yields increased latency as a byproduct in merging systems
  • 12.
    Latent factors ofhybridization What happens when organizations merge? - Organizational Dysfunction becomes magnified after a hybrid process This deficiency impacts negatively upon ANSP performance which in turn affects the FAB scenario
  • 13.
    According to theeconomic principle of marginal substitution - there will always be a tradeoff between the cost and the quality of ATM services - the principle is extended to optimizing efficiency which is a composite good
  • 14.
    Each consumer/ANSP willsubstitute differently to keep their maximised utility constant Optimal efficiency will exist in a state of flux
  • 15.
    - Given theconflicting views of stakeholders, it is not easy for ANSPs to reconcile the aims of regulatory agencies, interested parties, airline companies and ATCOs under the umbrella of ATM hybridization
  • 16.
    FABs are likeicebergs, a beautiful economic concept but - The presence of Latent factors makes FAB performance difficult to measure
  • 17.
    How do wemanage the performance of these FABs? Managing FAB performance
  • 18.
    We can lookat the Efforts that each ANSP made in subscribing to the FABA. But this consolidation process is fairly nascent.
  • 19.
    Instead, we willevaluate the parameters of performance, the KPAs in a multilevel setting via a gllamm analysis
  • 20.
    The word gllammstands for generalised latent linear and mixed modelling in advanced statistics. This model is useful with inconsistent data from ambiguous environments which is typical of air traffic control.
  • 21.
    From a statisticalviewpoint, the underlying principle of gllamm analysis is based on the maximum likelihood of an outcome given the limitations of a group of test variables.
  • 22.
    In the domainof social sciences, the gllamm technique is used either as a preliminary process to test the sutitability of parameters in advanced regression analyses.
  • 23.
    Or, it isused in multi level analyses to explore, predict and validate relationships between the test variables and the outcome in each group
  • 24.
    Size of sampleand number of test variables will affectthe time for the analysis. Limitations of the gllamm technique
  • 25.
    To manage FABperformance, we assume that FAB efficiency is a function of the efficiencies of a group of ANSPs which in turn is a function of specific performance metrics that are organizational, technical and economic. FAB performance = f {OTE parameters, latent effects} We also take into considereation the latent effects of hybridization.
  • 26.
    The data isderived from 4 datasets that measured diverse indications of performance for 31 ANSPs from the period 2006-2013 that are divided into 9 airspace blocks The statististical technique: stepwise gllamm Number of FABS: 9 Number of ANSPs; 28 Data provided from EUROCONTROL dashboard and Performance Review Unit Software used: STATA
  • 27.
    The gllamm techniquetests the theory that FAB performance is a function of ANSP performance in each state. In turn, ANSP performance is a result of a combination of technical, economic and organizational parameters. The gllamm technique also takes into account the byproduct of latent or random effects at the hybridized level.
  • 28.
    We are makingthe assertion that the efficiency or performance of a FAB is affected by the productivity of air navigation services (ANS) per state. If common strategic objectives are met that yield increases in safety performance, the state's inefficiency in providing ANS decreases and in turn, the FAB efficiency increases.
  • 29.
    Evaluating the metrics(KPAs) of the FABA FAB efficiency of group j with i ANSPs Vector of the KPAs foe the ith ANSP of the jth FAB The latent effects that impact upon the performance of the jth FAB
  • 30.
    Key Performance AreasVariable Ability to predict FAB performance at 90% statistical significance FAB performance FAB Efficiency Dependent variable (FABE) Dependent variable Safety Performance and Management Effectiveness of Safety Management (ESM) Significant predictor Economic efficiency Costs of gatetodelays per composite flight hour ( COD ) Military airspace Thriftiness ( MTS) Significant predictor Not a significant predictor Operational efficiency Volume of Controlled Airspace (VCA) Number of Operational units: ACCs, TWRs and AFIS (NOU) Size of staff (SOS) Scope of services (OSS) Significant predictor Not a very significant predictor Not a significant predictor Not a significant predictor Technical efficiency Aggregated complexity score (ACS) Significant predictor Environmental efficiency Traffic Variability Indicator ( TVI) Not a very significant predictor Innovation Innovation Strategy (IST) Not a significant predictor
  • 31.
    The gllamm confirmsthat these Performance metrics or KPA’s: Cost of delays Safety management Volume of airspace Number of operational units and Level of complexity are significant predictors of FAB performance
  • 32.
    Predictors of FAB performanceat the state level p value z value Effect upon FAB performance at 90% statistical significance Safety management .005 2.79 Decrease Cost of delays .000 3.54 Decrease Volume of controlled airspace .000 -3.68 Decrease Number of operational units .024 2.26 ncrease Traffic variability indicator .0481 -0.71 Increase Complexity .069 1.82 Increase Organizational efficiency .000 456.01 Increase Variance at FAB level Log likelihood Number of level 1 units Number of level 2 units Adjusted R^2 .02 123.597 28 9 .63
  • 33.
    What scope isthere for further study? -repeat the gllamm for another 3-5 years -change the variables -carry out benchmarking studies
  • 34.
    Do you recall? Whydo we need PM for FABA? How may we evaluate FAB performance? What scope is there for further study?
  • 37.
    References [1] Dr. AubreyC. Daniels, Performance Management: Changing Behavior That Drives Organizational Effectiveness], 4th ed., Performance Management Publications, 1981, 1984, 1989, 2006. ISBN 0-937100-08-0 [2] Sophia Rabe-Hesketh and Anders Skrondal, Multilevel and Longitudinal Modeling Using Stata, 2nd ed., ISBN-13: 978-1-59718-008-5 [3] Geeta Kumari, Neha Kaleramna and K.M.Pandey, Study on Performance Management System of Private Companies: a Case Study of Endurance, International Journal of Innovation, Management and Technology, Vol. 1, No. 5, December 2010 ISSN: 2010-0248 [4] PMeZine the Performance Management Magazine, November 13th, 2000 « What is Performance Management », an interview with Dr. Aubry Daniels [5] Prof. Claude Ménard, 2004, « The economics of hybrid organizations », The Journal of Institutional and Theoretical Economics, JITE (160) 1-32,Mohr Siebeck – ISSN 0932-4569 [6] Lynn Godkin and Seth Allcorn, 2009, "Institutional narcissism, arrogant organization disorder and interruptions in organizational learning", The Learning Organization, Vol. 16 (1) 40-57
  • 38.
    [7] Prof. AndyNeely, Chris Adams and Mike Kennerley, the Performance Prism: the Scorecard for Measuring and Managing Business Success, PearsonEducation 2002 ISBN: 0273653342 [8] Kim Andersson, 2012 Multifunctional Wetlands and Stakeholder Engagement: Lessonsfrom Sweden, Working Paper Stockholm Environmental Institute [9] Rabe-Hesketh and Skrondal, 2008 Multilevel and Longitudinal Modelling using STATA, 2nd edition STATA Press USA ISBN: 9781597180405 [10] J. B. Osterstock, J.C. MacDonald, M.M. Bogesssand M.S. Brown, 2010,“Technical note: Analysis of ordinal outcomes from carcass data in beef cattle research”, J. Anim. Sci., 88:3384-3389 [11] Strategy, Organizational Effectivenessand Performance Management: from Basics to Best practices by Soren Dressler, 2004 Universal Publishers ISBN: 1-58112-532-1