A presentation by Olivier Delobbe (Banque Centrale du Luxembourg) at the Workshop on Public Finances in Riga on 21 June 2016

1. 1
Discussion of “Anatomy of sovereign
distress: The role of financial sector
fragility”
Olivier Delobbe
Public Finance Workshop
Riga, 21 June 2016

2. Disclaimer
This presentation should not be reported
as representing the views of the BCL or the
Eurosystem.
The views expressed are those of the
author and may not be shared by other
research staff or policymakers in the BCL
or the Eurosystem.
2

3. 3
 Early Warning System (EWS) for sovereign distress heavily
debated since the onset of the global financial crisis.
 Published papers on fiscal stresses mainly focus on 2 methods:
 Univariate non parametric signalling approach
(Baldacci et al (2011), Hernandez de Cos et al (2014), Berti et al (2012).
 Multivariate parametric approach
(Manase, Roubini and Schimmelpfenning (2013).
 The contribution of this paper is two-fold:
 New method: random forest ensemble method /tree based method
(Breiman, 2001).
 Financial stability variables as predictors.
Summary - 1

4. 4
 Fiscal stress is defined as follows:
 Default on debt obligation towards creditors:
- Debt service not paid by due date or within a specified grace
period
- Restructuring with losses to some of the creditors
 Deterioration in market access (pressure on ST and LT yields)
 Sovereign defaults are collected from 2 databases (Data base on
Sovereign Default 2015 and Laeven & Valencia 2013).
 Basic data for calculating the deterioration in market access
extracted from IFS-IMF database.
 Financial sector indicators from World Bank-GFDD 2013 and
macro-economic indicators from IMF-WEO 2015.
 In total, 115 variables are collected mostly focusing on macro and
financial indicators.
Summary -2

5. 5
EWS for sovereign distress: random forest (RF) ensemble method
 Random forest ensemble method predicting the binary outcome
(occurrence of sovereign distress event clusters) by estimating a
large number of decision trees and combining the prediction of
individual trees into an ensemble forecast.
 Classification And Regression Trees (CART) used to model the
conditional distribution of a response variable given explanatory
variables.
Maximization of homogeneity of the resulting response
(dissagregation of the data set until one of the stopping
rules is satisfied).
 Example of conclusions drawn from CART: likehood of sovereign
distress increases with large negative output gaps, large negative
structural balances or large private sector debt issuances.
Summary - 3

6. 6
 Drawback of CART: over-fitting, highly dependent on data
set , low level of robustness move to Random
Forest ensemble method (RF).
 Build forest made up of multiple CART trees (each CART
tree estimated on a different subsample of observations and
variables) and then combine the predictions from individual
trees into an overall prediction (predictions of the individual
trees averaged out across the bootstrap samples).
 Ensemble prediction = outcome that receive most votes
from individual trees.
Summary – 4

7. 7
RF output:
 Importance given to a variable depending on its appearance in many trees and at the top
split nodes of the trees
 Correspondance between predictors and outcome (partial dependance plot)
 Treatment of class imbalance since distress events represent a small proportion of the
overall cases (Synthetic Minority Over-sampling Technique - SMOTE)
Results: Estimated RF performs well (perfect prediction of fiscal distress measured if ROC =1)
Top predictors of sovereign distress in Advanced Economies:
macro indicators (gross national savings level, unemployment rate,…)
financial indicators (growth of domestic credit, banking sector efficiency,...)
Summary - 5

8. 8
 New advanced technique (RF) on EWS for sovereign distress but still
work in progress with a lot of “?” in the text.
 The paper establishes strong links between private sector issues and
public sector issues (fiscal stress) – but public and private issues are
not always directly and intuitively linked.
 Overall, somehow pessimistic about RF approach.
A lot of focus on the technique/method, less on the economics, the
intuition and the results (the usefulness of the analysis gets lost in the
“forest”). Conclusions could be more detailed (are there policy
messages?). What are the best performing indicators? Are all of these
115 indicators useful?
Comments – 1: General issues

9. 9
 Advanced, complicated and not fully transparent technique (RF) – how
do the results compare to simpler techniques, e.g. non-parametric
signalling approach, Logit/Probit, CART…
 How can one infer from the results that some specific indicators are
particularly important in explaining distress in AE – difficult to see it in
the paper, no interpretation of the results.
 Presentation of results: prediction of fiscal stress (ROC), sensitivity
(true positive predictions) and specificity (true negative predictions).
Quid of false positive and false negative predictions?
Comments – 2: General issues

10. 10
 Definition of fiscal stress – how are the averages computed (given that
there have been large swings in the bond yields): recursively or over
the full sample? Possible to add other indicators of fiscal stress?
(increase in the public debt in % of GDP?)
 Is there a problem with the definition of fiscal stress? Illustrate with an
example (take a country) and show with a chart how the fiscal stress is
triggered. Difficult for the reader to figure it out when going through the
paper!
 Figure 2: Belgium has a high public debt, but it does not feature in the
list. Why?. Belgium was involved in Dexia/ Fortis, but does not feature
in the list of countries.
 Italy has surprisingly few stress periods since 2005, is it really the
case?
 Greece had no fiscal stress from 2000 to 2010, although it breached
the 3% rule each year. Is it credible?
Comments – 3: Definition of fiscal stress

11. 11
 A large database was compiled because lots of data are needed for
this approach. Not always clear whether sample involves data from
1998-2015 or 1997-2015.
 Exclusion of distress observations occuring within 2 years from a
prior distress event. Why 2 years? Robustness checks mentioned
in the paper, unfortunately not available.
 Table 2, for each indicator used in the paper, provide the expected
sign.
 Table 5: explain variable importance and partial dependence plot –
what do the values and lines mean? Are the results coming from a
“black box” or is it possible to explain certain elements?
 Where are the thresholds (generated by the algorythm) for the RF
approach? More explanation concerning the way thresholds are set
would be useful. Is the threshold generated by CART for the output
gap (- 3.2%) appropriate?
Comments – 4: Results

12. 12
 Selection of key indicators.
 Add some competitiveness indicators.
 Practical cases for individual countries: when would fiscal crises in
GR, PT, IE,ES have been anticipated?
 Possible to anticipate crises ex ante? How does the model work in
real time? Using the model for out of sample forecasts?
 If the link between the financial sector and the sovereign is cut
(SSM, SRM), is this approach still useful?
Conclusion: interesting paper on a interesting topic but it needs more
developments/methodological clarity and possible policy
recommendations.
Comments – 5: Suggestions for future work?