Thi is the presentation at the 15 EWEPA 2017 Conference held in London (Senate House) 12-15 June 2017

1. Explaining differences in efficiency:
the case of local government literature
a Department of Economics, Statistics and Finance “Giovanni Anania”, University of Calabria
I-87036, Arcavacata di Rende (CS), Italy
b Department of Computer, Control and Management Engineering “Antonio Ruberti”
Sapienza University, I- 00185, Rome, Italy
c Accenture S.p.A. Capital Market - Financial Services IGEM, 20154 Milan, Italy
Aiello Francesco
Department of Economics, Statistics and Finance “Giovanni Anania”
University of Calabria, I-87036 Rende (CS), Italy
francesco.aiello@uncial.it
Bonanno Graziella
Department of Computer, Control and Management Engineering “Antonio Ruberti”
Sapienza University, I- 00185, Rome, Italy
bonanno@diag.uniroma1.it
Luigi Capristo
Accenture S.p.A. Capital Market - Financial Services IGEM Milan, Italy
luigi.capristo@gmail.it
15TH EUROPEAN WORKSHOP ON EFFICIENCY AND PRODUCTIVITY ANALYSIS (EWEPA) 2017 Organised
by the Centre for Productivity and Performance (CPP) and the School of Business and Economics of
Loughborough University, Senate House, London, 12-15 June 2017.

2. The paper is a follow-up of some recent research
• Aiello F., Bonanno G., (2017) “On the sources of
heterogeneity in banking efficiency literature” Journal of
Economic Survey, DOI: 10.1111/joes.12193
• Aiello F., Bonanno G., (2017) “Multilevel empirics for small
banks in local markets», Papers in Regional Science,
DOI: 10.1111/pirs.12285
• Bonanno G, De Giovanni D., Domma F. (2017) «The wrong
skewness problem: a re-specification of stochastic frontiers”,
Journal of Productivity Analysis, DOI:10.1007/s11123-017-
0492-8
• Aiello F., Bonanno G., (2017), “Efficiency in local government.
Italian municipalities can do better”, (first draft in Oct. 2017)
17/06/2017Francesco Aiello - EWEPA 2017 - London Pagina 2

3. Outline
• Why an MRA on Local Government Efficiency? Motivations
• Related literature
• Dataset creation
• The MRA in a nutshell
• Fitted models and results
• Conclusions
17/06/2017Francesco Aiello - EWEPA 2017 - London Pagina 3

4. 17/06/2017 Pagina 4
Motivations (1/2)
• The institutional architecture of many countries has changed
rapidly since the 1990s due to extensive deregulation aimed
at optimizing the use of public resources in offering
services of general interest at local level
• The institutional reforms accelerate over the last 15 years,
thereby increasing the interest on economists and public
administration to evaluate the efficiency level and the key-
factors influencing the performance of the public sector (Lovell
2002)
• Importantly, the institutional framework on how municipalities
work differ country-by-country and, therefore, it is reasonable
to assume that the heterogeneity in national norms
translates into heterogeneity in municipality efficiency
Francesco Aiello - EWEPA 2017 - London

5. 17/06/2017 Pagina 5
Motivations (2/2)
• Theory provides clear insights to define a unit-decision as efficient
or not, but results are extremely different on empirical grounds
• There are several and different approaches to estimate efficiency
with no consensus on the superiority of one method over the
others (Coelli and Perelman 2000)
Examples of choices to be made in empirics:
• Parametric vs non-parametric
• Stochastic vs deterministic
• FDH or DEA
• Number of inputs and outputs to be considered in the frontiers
• Functional form to be assigned to the frontier
• Distribution better fitting vi and/or ui (Normal, LogDagum, Gamma)
• Econometrics used in estimating the frontiers
• All this choices affect results, thereby causing
heterogeneity
Francesco Aiello - EWEPA 2017 - London

6. Related Literature
• The sensitivity of results to model specifications has been
addressed in several individual studies which compare the
results that different methods (i.e. parametric vs.
nonparametric) yield from a fixed sample of municipalities
(Athanassopoulos and Triantis 2016; De Borger and
Kerstens 1996; Geys and Moesen 2009; Worthington
2000)
• The reviews provided by da Cruz and Marques (2014)
Narbò Perpitna and De Witte (2017), Worthington and
Dollery (2000) are excellent and offer valuable arguments
in terms of why results differ
• However, no study has yet quantified the impact of
methodological choices on the variability of efficiency
scores in local government.
17/06/2017 Pagina 6Francesco Aiello - EWEPA 2017 - London

7. Pagina 7
Meta-Analysis Regression
• MA evaluates the relationship between the
dependent variable (that is the main result of the
analyzed studies) and a lot of features of every
paper. Here, the dependent variable is the
efficiency score of original papers
• Phrased differently, by modeling all the relevant
differences across studies on a given subject, MA
permits to understand the role of each varying
factor in determining the heterogeneity of
outcomes. In brief, it deals with the difficulty to
compare results of empirical works
17/06/2017Francesco Aiello - EWEPA 2017 - London

8. 17/06/2017 Pagina 8
Meta Regression in Economics
• The use of MA is growing in economics and regards
a very wide spectrum of subjects
• 626 MA papers in Economics from 1980 to 2010,
with an exponential growth in 2000s’. Many of them
appeared in AER, JPE, RESTAT and, in JES
• Agricultural economics is the area of research with
the highest proportion of MA papers, followed by
industrial economics, labour economics and
consumers economics.
Francesco Aiello - EWEPA 2017 - London

9. Pagina 9
Efficiency and MA
• Few MA papers dealt with the issue of
efficiency. Some examples are
• Three are on agriculture. Bravo-Ureta et al. (2007)
Thiam et al. (2001), Kolawole (2009)
• Brons et al. (2005) focus on urban transport
• Iršová and Havránek (2010) focus just on US
banks and consider 32 papers published over
1977-1997
• Aiello and Bonanno (2017) review 120 efficiency
studies – with 1661 observations – on banking
published over the period 2000–2014
17/06/2017Francesco Aiello - EWEPA 2017 - London

10. 17/06/2017 Pagina 10
Papers selected in our MRA on Local Goverment
• The search yields a sample of 54 papers
published from 1993 to 2016
• Provided that many studies report multiple
estimates of efficiency, the dataset under
analysis comprises a total of 360 observations
Francesco Aiello - EWEPA 2017 - London

11. Dataset assembling process
17/06/2017Francesco Aiello - EWEPA 2017 - London Pagina 11

12. 17/06/2017 Pagina 12
Efficiency in Local Government
Does Heterogeneity exist?
0 .2 .4 .6 .8 1
PARAM
NON PARAM
(a) by method
0 .2 .4 .6 .8 1
SFA
FDH
DEA
(b) by method II
0 .2 .4 .6 .8 1
TE
CE
(c) by efficiency type
0 .2 .4 .6 .8 1
PANEL
CROSS SECTION
(d) by data type
0 .2 .4 .6 .8 1
REGIONAL
NATIONAL
(e) by geographical focus
0 .2 .4 .6 .8 1
EUROPE
NON EUROPE
(f) by country
Francesco Aiello - EWEPA 2017 - London

13. 17/06/2017 Pagina 13
Efficiency in Local Government
Does Heterogeneity exist?
Heterogeneity in Inputs and Outputs
N.INPUTS
N. OUTPUTS
1 2 3 4 5 6 7 8 9
1 25 3 0 6 44 24 10 1 0
2 1 6 44 4 1 2 0 0 4
3 0 55 10 7 3 2 0 0 0
4 0 0 0 6 10 0 10 2 11
5 0 0 0 0 0 0 63 0 0
6 0 2 0 0 0 2 0 0 0
0.71
0.78
0.59
0.73
0.66
Francesco Aiello - EWEPA 2017 - London

14. 17/06/2017 Pagina 14
Estimated models
• The disturbance e = ε/S is corrected for heteroscedasticity; all variables in
the full model is weighted through the variance indicator S
• ei ~ N(0 , σ2
i) is the disturbance and ui ~ N(0 , τ2) is the primary-study fixed-
effect.
• The parameter τ2 is the between-study variance, which must be estimated
from the data as in Harbord and Higgins (2008).
• To provide some robustness of the results to clustering, we adopt a two-step
procedure as in Gallet and Doucouliagos (2014) and adopted by Aiello and
Bonanno (2017). An REML regression is run in the first step, while in the
second step we run a WLS regression in which the weights also include the
value of τ2 retrieved from the first step. This ensures that the REML
estimates will be robust to clustering at the study level.
Random Effect framework
iiiii euXSE  j
*
j
*
10
*
β
Francesco Aiello - EWEPA 2017 - London

15. 17/06/2017 Pagina 15
Estimated models: Variables (1/2)
• PARAM: dummy equal to 1 for the parametric group of
studies and 0 for the others (All the sample)
• FDH is 1 when efficiency scores are derived from
primary studies using FDH (the controlling group
comprises the point observations from papers using
DEA) (Nonparametric sample)
• VRS is 1 if the primary study uses VRS (controlling
group=CRS studies) (Nonparametric sample)
• TE is 1 if the primary estimation refers to technical
efficiency (controlling group=cost frontiers)
• Panel is 1 if original works used panel data, 0 cross-section
Francesco Aiello - EWEPA 2017 - London

16. 17/06/2017 Pagina 16
Estimated models: Variables (2/2)
• Dimension: given by the sum of the number of inputs and
outputs of the frontier
• Sample Size: the number of observations used in primary
papers when estimating the efficiency score
• DREG is 1 for efficiency observations related to specific
sample of municipalities belonging to one or specific
regions of a country (DREG=1). Controlling group=
observations from national local government
• Europe is 1 if the primary study used data from an
European country (controlling group=efficiency scores
from papers focusing on the RoW)
• Time Effect: Year of publication (or Year of Estimation)
Francesco Aiello - EWEPA 2017 - London

17. RESULTS (All the sample)
Table 2 Meta-regression analysis of Local Governments efficiency scores (All sample)
Variables Model 1
Constant 0.9284***
1/S -0.000011**
DREG -0.1464***
EUROPE -0.1544**
Observations 308
17/06/2017Francesco Aiello - EWEPA 2017 - London Pagina 17

18. RESULTS (All the sample)
Table 2 Meta-regression analysis of Local Governments efficiency scores (All sample)
Variables Model 1 Model 2
Constant 0.9284*** 20.7125***
1/S -0.000011** -0.000006
DREG -0.1464*** -0.1158
EUROPE -0.1544** -0.0945*
TE 0.0759
PANEL 0.1667***
PARAM 0.1500
Year of publication -0.0100***
lDIM 0.0399
lSIZE -0.0006
Observations 308 294
17/06/2017Francesco Aiello - EWEPA 2017 - London Pagina 18

19. RESULTS (All the sample)
Table 2 Meta-regression analysis of Local Governments efficiency scores (All sample)
Variables Model 1 Model 2 Model 3
Constant 0.9284*** 20.7125*** 18.8669***
1/S -0.000011** -0.000006 0.000011
DREG -0.1464*** -0.1158 -0.1685***
EUROPE -0.1544** -0.0945* -0.1748***
TE 0.0759 0.1799**
PANEL 0.1667*** 0.1103**
PARAM 0.1500 0.1646*
Year of publication -0.0100*** -0.0092***
lDIM 0.0399 0.0705*
lSIZE -0.0006 0.0355**
lSIZE*MANY -0.0548***
Observations 308 294 294
17/06/2017Francesco Aiello - EWEPA 2017 - London Pagina 19

20. 17/06/2017Francesco Aiello - EWEPA 2017 - London Pagina 20
Variables Model 4
Constant 5.0604
1/S 0.000002
DREG -0.2466 ***
EUROPE -0.2340 ***
TE 0.2152 **
PANEL 0.1099 **
FDH 0.2308 ***
Year of publication -0.0022
lDIM 0.0756 **
lSIZE 0.0022
lSIZE*MANY -0.0533 ***
VRS
Observations 267
RESULTS (Nonparametric sample)

21. 17/06/2017Francesco Aiello - EWEPA 2017 - London Pagina 21
Variables Model 4 Model 5
Constant 5.0604 1.4620
1/S 0.000002 -0.000014
DREG -0.2466 *** -0.2032 **
EUROPE -0.2340 *** -0.1608 ***
TE 0.2152 ** 0.0949
PANEL 0.1099 ** 0.1802 ***
FDH 0.2308 *** 0.2918 ***
Year of publication -0.0022 -0.0003
lDIM 0.0756 ** 0.0376
lSIZE 0.0022 -0.0308 *
lSIZE*MANY -0.0533 ***
VRS 0.0657 **
Observations 267 267
RESULTS (Nonparametric sample)

22. 17/06/2017 Pagina 22
Variables Model 4 Model 5 Model 6
Constant 5.0604 1.4620 3.6452
1/S 0.000002 -0.000014 0.000003
DREG -0.2466 *** -0.2032 ** -0.2527 ***
EUROPE -0.2340 *** -0.1608 *** -0.2369 ***
TE 0.2152 ** 0.0949 0.2003 *
PANEL 0.1099 ** 0.1802 *** 0.1217 ***
FDH 0.2308 *** 0.2918 *** 0.2454 ***
Year of publication -0.0022 -0.0003 -0.0015
lDIM 0.0756 ** 0.0376 0.0685 **
lSIZE 0.0022 -0.0308 * 0.0050
lSIZE*MANY -0.0533 *** -0.0520 ***
VRS 0.0657 ** 0.0544 **
Observations 267 267 267
RESULTS (Nonparametric sample)
Francesco Aiello - EWEPA 2017 - London

23. 17/06/2017 Pagina 23
Conclusions
• Parametric methods yield higher levels of efficiency than
nonparametric studies
• Relaxing the convexity hypothesis matters: FHD papers yield, on
average, higher efficiency scores than DEA studies
• Studies assuming VRS yield higher efficiency levels than papers
based on CRS
• Efficiency in paper using panel data is higher than paper based on
cross sectional data
• Efficiency increases with the number of inputs and outputs (the
marginal effect decreases as the dimension increases)
• The heterogeneity in results is significantly dependent on the
sample size used in primary papers
• When focusing on a given region the results are, on average, lower
than when analysing the national system of local government
Francesco Aiello - EWEPA 2017 - London

24. 17/06/2017 Pagina 24
Insights for future work
• While results are robust to different samples of observations, the study
has some limitations depending on data quality. Many primary papers
do not report any detail regarding their empirical setting
• A lesson that we have learnt is that it is a good practice for primary
papers to provide full explanations, not only so that readers are
informed concerning each single study, but also because it would help
the understanding of some key issues in the efficiency literature
• For instance, it would be valuable for academics to know if
heterogeneity in local government efficiency might be explained by
orientation in technology (input- vs output-oriented models).
Similarly, the data available for our MRA do not allow us to determine
whether efficiency differs according to the municipality size analysed
in the primary papers (i.e. small vs large municipality)
• Researchers might address these issues in future work by performing
a new MRA. However, this is feasible only if primary papers provide
more detailed information than those used in this meta-study
Francesco Aiello - EWEPA 2017 - London

25. The paper is downloadable from
• WP series at DESF, UNICAL
• REPEC
• MPRA Archive
• ResearchGate
Comments are welcome