The global growth slowdown (3.9 % to -2.1% ) affected advanced and developing countriesAdvanced economies: 2.6% to -3.3%Developing Bank clients: 6 % to 1 % Europe and Latin America: 7 % to – 2 %
A contribution of the analysis, from the perspective of evaluation in uncertainty, was the scenario analysis it undertook, in terms of projecting the poverty impact of the crisis, based on a series of data on GDP growth effects, trade and capital flow effects across countries, and assumptions on demand in different segments of the world economy (Figure 5).
Data for selective comparisons of the Bank relative to other IFIs Comparisons with lending patterns of other IFIs was undertaken at multiple levels – the response to changes in GDP alone, to composite overall stress measures, and compared to specific measures of stress. Interviews with staff and evaluators at the European Bank for Reconstruction and Development (EBRD), International Monetary Fund (IMF), Inter-American Development Bank (IDB), Asian Development Bank (ADB), AfDB, European Investment Bank (EIB), and the European Union (EU)Evaluative evidence provided in recent internal evaluations of some of these agencies was drawn upon (ADB 2011; IMF 2011; EBRD 2010).
EBRD’s evaluation of its crisis response traced the character of response in three phases of time; the pre-crisis period (2006 to August 2007), the period of rising instability in the run-up to the crisis (September 2007 to the third quarter of 2008), and the period of the crisis itself (from the fourth quarter of 2008 to the end of 2009). The typology of the three phases was built up based on alternative data series that traced the course of the crisis, as illustrated in Figure 11 below, where use is made of two series of data on rising interest rates – the European Central Bank deposit facility rates, and the ‘FED funds rate’, rates announced by the European and US central banks that indicate their perceptions of market trends.
The EMBI and the TED spread are both measures of the rise in perceptions of the risk in lending, measured against a risk free benchmark. Both are denominated in basis points. Rising indices / spreads indicates that liquidity is being withdrawn and that lenders believe the risk of default is increasing.
Global Crisis Response Evaluations at the World Bank Group
Evaluation Approaches, Methods and Data in a Rapidly Changing Context: Global Crisis Response Evaluations at the World Bank Group Independent Evaluation Group World Bank / IFC / MIGA European Evaluation Society Helsinki, October 2012 Anjali Kumar, Andaleeb Alam and Ali Khadr IEG World Bank Group1
The Global Economic Crisis and the Challenge to WBG Rapid movement of economic events during the global crisis of 2008-2009 Some stock markets lost 50% of value in days Global growth slowdown (3.9 % to -2.1% ) Unprecedented scale, contagion Estimated 50-64 million more poor people Motivating strong response from WBG 117 countries received Bank loans during 2009- 10; tripling of IBRD lending to $100m 17 received crisis support during 1993–2003 Call for ‘real time’ evaluation of response2
Crisis Evaluation: Nature of the ChallengeDaily data show the decline in value of the UK Complexity of CrisisFTSE index from 5,600 to 3,800 in weeks Response Evaluations Events are sudden, unexpected and fast-evolving in character Evaluative lessons are needed in a compressed time frame Conventional evaluation techniques that use information based on a sequential logical framework are of limited use3 Uncertainty prevails
Crisis Evaluation: Nature of the Challenge Scope of the present paper Questions asked in crisis evaluation Data usable to answer such questions Use of high frequency data Crisis evaluations at other IFIs / MDBs Are such economic events becoming more frequent? ‘Black swan’ events may increase because they are probabilistic outliers Increased need for corresponding4 evaluative techniques
Crisis Evaluation –Nature of Evaluative Questions Evaluative Questions – complex evaluations under uncertainty Skills demonstrated in dealing with complexity and change Speed of response; reflecting information on events as they occur Identification of risks involved Results framework, learning: Quality at entry5
Crisis Evaluation: IEG’s Phased ResponseMonthly data on emerging markets showed IEG prepared a series of ‘real time’declines in industrial production and credit evaluations on WBG Crisisgrowth not revealed in annual averages Response 12% 1.5 Review of WBG Response to Past Crises 10% (2009) 1.0 Industrial Production (sd units) Private Credit Growth (q-o-q) 17 Country Case studies – no current crisis data 8% 0.5 Phase I Evaluation of WBG Crisis 6% Response (2010) 4% 0.0 Real time evaluation focused on volume, speed, and early results – selective use of data to 2% -0.5 benchmark Bank actions 0% -1.0 Phase II Evaluation of WBG Crisis Response (2011) Jan-08 Jan-10 Jan-07 Jan-09 Jul-07 Jul-08 Jul-09 Jul-10 Oct-07 Oct-08 Oct-09 Oct-10 Apr-07 Apr-08 Apr-10 Apr-09 Used high frequency data to analyze patterns of Private Credit Growth (q-o-q) Industrial Production (sd units) Bank support relative to crisis incidence6
1. Retrospective Evaluation - WB Response toPast Crises – 17 case studies Evaluative questions based on preceding principles Focus on: Scale and modality of the lending response Impact on the Bank’s own balance sheet Partnerships with other multilateral agencies Content of the WB crisis-response operations Macroeconomic trade and financial and fiscal Use of Data The crisis retrospective did not look at data on the current crisis, It did trace data on previous crises- –extent of World Bank’s response relative to its baseline lending, –and relative to other international financial institutions;7 –time taken to return to normal lending patterns.
2. WBG Response to the Global Crisis: Phase I Evaluation Private Capital Flows during the crisis: Evaluative questions : January 2007–December 2010 Design aspects –readiness, relevance, poverty focus, Implementation aspects – Speed, internal organization, instruments, monitoring and evaluation Early Outcomes and Prospects- Meeting Objectives, additionality, debt sustainability Data Use: Annual economic data used to trace effects of crisis on Bank borrower / client countries8
2. WBG Response to the Global Crisis Phase I Evaluation Alternative Scenarios – Poverty in Phase I Example: Developing Countries in 2015 and 2020 Data Use in Evaluation under Uncertainty: Scenario analysis Ten year projections of poverty impact of the crisis Based on a series of recent data on GDP growth, trade, capital flows and demand assumptions9
WBG Response to the Global Crisis -Phase II Scope of the Evaluation Utilization of high frequency data across a range of economic variables To track multiple dimensions of stress These multiple measures of stress were only partially correlated. Analysed relative to other IFIs and MDBs Multidimensionality of crisis Exchange rate and foreign exchange reserve stress Financial Stress, including market, credit and banking system Social indicators – unemployment, private consumption; and Fiscal deficit and public debt as a percentage of GDP)10
WB Response to the Global Crisis Phase II Evaluation Methods Use of High Frequency data to measure multiple dimensions of stress that were not necessarily correlated11
Evaluating the Lending Response relative to stress Incremental Lending Relative to Levels of Simple and Composite stress measures Crisis: World Bank Simple - changes in GDP; Composite - principal factor analysis 700% Peak to trough, period average 600% Illustrative bands ranking countries according to stress in diagram 500% Are compared with the distribution of % Increase in Lending 400% incremental lending. 300% Underlying these data are regressions That measure crisis intensity and 200% incremental Bank response on a 100% continuous basis Caveats and limitations of the analysis Country demand, country performance, 0% other IFIs Growth Decline Controls can however be introduced for some of these factors12
Volume and Distribution of Support - Other IFIs / MDBs Incremental Lending Relative to Levels of Increased WB lending patterns were also Crisis: Comparisons with Other Donors compared with other MDBs 12 1.2 Correlations of each institution’s incremental support to crisis intensity Increase in Other Major Donor Lending in client countries 10 1 Increase in World Bank Lending 8 0.8 Based on data obtained from relevant IFIs as % of GDP as % of GDP Comparisons undertaken at multiple 6 0.6 4 0.4 levels Response to changes in GDP alone, to 2 0.2 composite overall stress measures, and compared to specific measures of stress 0 0 Do not analyse other MDBs’ overall Other Major Donors (incl. IMF) Other Major Donors (excl. IMF/EIB/EU) response Comparisons limited to countries that World Bank13 were common borrowers
Crisis Evaluation and High Frequency Data - EBRD EBRD’s crisis response evaluation established a three phase typology Based on two high frequency data series on interest rates European Central Bank deposit facility rate ‘FED funds rate14
Crisis Evaluation and High Frequency Data - EBRD EBRD’s evaluation then traces crisis response in each phase Eg: evolution of EBRD loan pricing in response to market signals15
IMF – Evaluation of the Fund in the Run Up to the Crisis The IMF evaluated the quality of its surveillance in the run up to the crisis Evolution of crisis traced also using high frequency data: EMBI Global Spread (Left) TED spread (right)16
Conclusions and Suggestions Rapidly changing situations pose special challenges for evaluators Different questions That can use high speed data for their response In order to give real time feedback Financial and some macroeconomic data exist In multiple dimensions and high frequencies However more systematic monthly data on GDP, fiscal deficits etc are still to be produced More difficult are data on social dimensions Eg integrated global databases on employment or consumption Greater global cooperation on such data would be a benefit To policy makers and to evaluators17