A discussion of how we are creating an index of financial supervisory transparency using Bayesian IRT methods.

1. Measuring International Financial Supervisory
Transparency
Christopher Gandrud, Mark Copelovitch, and Mark
Hallerberg
December 4, 2014
FRT Index 1 / 27

2. Why

3. nancial supervisory transparency?
Financial supervisory transparency has been lauded as promoting:
I

4. nancial system stability,
I democratic legitimacy for supervisors.
FRT Index Motivation 2 / 27

5. Why

6. nancial supervisory transparency?
Financial supervisory transparency has been lauded as promoting:
I

7. nancial system stability,
I democratic legitimacy for supervisors.
FRT Index Motivation 2 / 27

8. Promotion
Supervisory transparency has been promoted by
international/supra-national institutions including:
I IMF,
I Basel Committee,
I European Union.
FRT Index Motivation 3 / 27

9. But. . .
We lack reliable, cross-country, and cross-time indicators of

10. nancial supervisory transparency to test these assertions.
FRT Index Motivation 4 / 27

11. Objective
Our objectives are to:
I Develop a reliable and valid indicator of supervisory
transparency across countries and time.
I Largely complete.
I Use the indicator to examine:
I why countries become more/less transparent,
I how, if at all, supervisory transparency aects economic
outcomes.
FRT Index Motivation 5 / 27

12. Objective
Our objectives are to:
I Develop a reliable and valid indicator of supervisory
transparency across countries and time.
I Largely complete.
I Use the indicator to examine:
I why countries become more/less transparent,
I how, if at all, supervisory transparency aects economic
outcomes.
FRT Index Motivation 5 / 27

13. Objective
Our objectives are to:
I Develop a reliable and valid indicator of supervisory
transparency across countries and time.
I Largely complete.
I Use the indicator to examine:
I why countries become more/less transparent,
I how, if at all, supervisory transparency aects economic
outcomes.
FRT Index Motivation 5 / 27

14. Objective
Our objectives are to:
I Develop a reliable and valid indicator of supervisory
transparency across countries and time.
I Largely complete.
I Use the indicator to examine:
I why countries become more/less transparent,
I how, if at all, supervisory transparency aects economic
outcomes.
FRT Index Motivation 5 / 27

15. Objective
Our objectives are to:
I Develop a reliable and valid indicator of supervisory
transparency across countries and time.
I Largely complete.
I Use the indicator to examine:
I why countries become more/less transparent,
I how, if at all, supervisory transparency aects economic
outcomes.
FRT Index Motivation 5 / 27

16. Methodological Contribution
(At least) two important methodological contributions:
I Develop a Hierarchical Bayesian Item Response Theory-based
unique indicator of countries' willingness to credibly reveal
basic facts about their

17. nancial systems to international
actors.
I Show that missing

18. nancial system data is often
endogenous to

19. nancial system diculties and policymaker's
aspirations.
FRT Index Motivation 6 / 27

20. Methodological Contribution
(At least) two important methodological contributions:
I Develop a Hierarchical Bayesian Item Response Theory-based
unique indicator of countries' willingness to credibly reveal
basic facts about their

21. nancial systems to international
actors.
I Show that missing

22. nancial system data is often
endogenous to

23. nancial system diculties and policymaker's
aspirations.
FRT Index Motivation 6 / 27

24. Predecessors
Previous supervisory transparency indices generally use surveys of
supervisors and then sum responses.
I Lierdorp et al. (2013)
I Arnone, Darbar, and Gambini (2007)
I Seelig and Novoa (2009)
I Masciandaro, Quintyn, and Taylor (2008)
FRT Index Creating the FRT Index 7 / 27

25. Issues with previous methods
There are a number of issues with previous methods:
I Ironically, many of the surveys are not transparent.
I Survey methods are laborious.
I Surveys rely on temporally ephemeral information.
I So, survey methods provide only brief windows, not time
series.
I Summing responses assumes that each item should be
weighted equally.
I High non-response rate (Liedorp et al. had a response rate
of 57%). This information is often ignored.
I No estimation of uncertainty.
FRT Index Creating the FRT Index 8 / 27

26. Issues with previous methods
There are a number of issues with previous methods:
I Ironically, many of the surveys are not transparent.
I Survey methods are laborious.
I Surveys rely on temporally ephemeral information.
I So, survey methods provide only brief windows, not time
series.
I Summing responses assumes that each item should be
weighted equally.
I High non-response rate (Liedorp et al. had a response rate
of 57%). This information is often ignored.
I No estimation of uncertainty.
FRT Index Creating the FRT Index 8 / 27

27. Issues with previous methods
There are a number of issues with previous methods:
I Ironically, many of the surveys are not transparent.
I Survey methods are laborious.
I Surveys rely on temporally ephemeral information.
I So, survey methods provide only brief windows, not time
series.
I Summing responses assumes that each item should be
weighted equally.
I High non-response rate (Liedorp et al. had a response rate
of 57%). This information is often ignored.
I No estimation of uncertainty.
FRT Index Creating the FRT Index 8 / 27

28. Issues with previous methods
There are a number of issues with previous methods:
I Ironically, many of the surveys are not transparent.
I Survey methods are laborious.
I Surveys rely on temporally ephemeral information.
I So, survey methods provide only brief windows, not time
series.
I Summing responses assumes that each item should be
weighted equally.
I High non-response rate (Liedorp et al. had a response rate
of 57%). This information is often ignored.
I No estimation of uncertainty.
FRT Index Creating the FRT Index 8 / 27

29. Issues with previous methods
There are a number of issues with previous methods:
I Ironically, many of the surveys are not transparent.
I Survey methods are laborious.
I Surveys rely on temporally ephemeral information.
I So, survey methods provide only brief windows, not time
series.
I Summing responses assumes that each item should be
weighted equally.
I High non-response rate (Liedorp et al. had a response rate
of 57%). This information is often ignored.
I No estimation of uncertainty.
FRT Index Creating the FRT Index 8 / 27

30. Issues with previous methods
There are a number of issues with previous methods:
I Ironically, many of the surveys are not transparent.
I Survey methods are laborious.
I Surveys rely on temporally ephemeral information.
I So, survey methods provide only brief windows, not time
series.
I Summing responses assumes that each item should be
weighted equally.
I High non-response rate (Liedorp et al. had a response rate
of 57%). This information is often ignored.
I No estimation of uncertainty.
FRT Index Creating the FRT Index 8 / 27

31. Issues with previous methods
There are a number of issues with previous methods:
I Ironically, many of the surveys are not transparent.
I Survey methods are laborious.
I Surveys rely on temporally ephemeral information.
I So, survey methods provide only brief windows, not time
series.
I Summing responses assumes that each item should be
weighted equally.
I High non-response rate (Liedorp et al. had a response rate
of 57%). This information is often ignored.
I No estimation of uncertainty.
FRT Index Creating the FRT Index 8 / 27

32. Our Approach
Missing data as data! (see Hollyer et al. 2014)
Treat

33. nancial regulatory transparency (FRT) as an unobserved
latent variable.
Our FRT Index summarizes countries' likelihood of reporting
yearly data to indices included in the World Bank's Global
Financial Development Database (GFDD).
FRT Index Creating the FRT Index 9 / 27

34. Observations and items
60 high income countries, 22 years (1990-2011), 14 items.
FRT Index Creating the FRT Index 10 / 27

35. The model
yk;c;t =
1 if item k reported in country c; year t
0 if item k not reported in country c; year t
Estimate (based on Stan Development Team 2014, 49-50):
Pr(yk;c;t = 1jc;t) = logit[exp(
k) (c;t

36. k + )]
where:
I c;t is the estimated propensity for country c at year t to
report item k. This can be thought of as the transparency
score.
I
k is the discrimination parameter for item k
I

37. k is the diculty parameter for item k
I is the mean transparency
FRT Index Creating the FRT Index 11 / 27

38. The model
yk;c;t =
1 if item k reported in country c; year t
0 if item k not reported in country c; year t
Estimate (based on Stan Development Team 2014, 49-50):
Pr(yk;c;t = 1jc;t) = logit[exp(
k) (c;t

39. k + )]
where:
I c;t is the estimated propensity for country c at year t to
report item k. This can be thought of as the transparency
score.
I
k is the discrimination parameter for item k
I

40. k is the diculty parameter for item k
I is the mean transparency
FRT Index Creating the FRT Index 11 / 27

41. The model
yk;c;t =
1 if item k reported in country c; year t
0 if item k not reported in country c; year t
Estimate (based on Stan Development Team 2014, 49-50):
Pr(yk;c;t = 1jc;t) = logit[exp(
k) (c;t

42. k + )]
where:
I c;t is the estimated propensity for country c at year t to
report item k. This can be thought of as the transparency
score.
I
k is the discrimination parameter for item k
I

43. k is the diculty parameter for item k
I is the mean transparency
FRT Index Creating the FRT Index 11 / 27

44. The model
yk;c;t =
1 if item k reported in country c; year t
0 if item k not reported in country c; year t
Estimate (based on Stan Development Team 2014, 49-50):
Pr(yk;c;t = 1jc;t) = logit[exp(
k) (c;t

45. k + )]
where:
I c;t is the estimated propensity for country c at year t to
report item k. This can be thought of as the transparency
score.
I
k is the discrimination parameter for item k
I

46. k is the diculty parameter for item k
I is the mean transparency
FRT Index Creating the FRT Index 11 / 27

47. Priors (1)
c;1990 N(0; 1)
Rescentered by c;19901990
SD;1990
Priors for t 1
c;t N(c;t1; c)8t 1
with half-Cauchy prior (see Gelman, 2007; Polson and Scott, 2012)
c Cauchy(0; 0:25)
FRT Index Creating the FRT Index 12 / 27

48. Priors (2)
Similarly:
Cauchy(0; 0:25)

49. N(0;

50. )
N(0;
)
(1)
where

51. Cauchy(0; 0:25)
Cauchy(0; 0:25)
(2)
FRT Index Creating the FRT Index 13 / 27

52. Estimation
We estimated the model using Stan/No-U-Turn Sampler
(recommended for highly correlated data).
FRT Index Creating the FRT Index 14 / 27

53. Accessing source and data
The source code is available at:
https://github.com/FGCH/FRTIndex
The (beta version) of the FRT Index set can be downloaded into
R with:
frt index - repmis::source data(`http://bit.ly/1rZ49jB')
FRT Index Creating the FRT Index 15 / 27

54. What are we actually measuring?
The willingness of a country to report minimally credible
information about its

55. nancial system to international
institutions and investors.
FRT Index Creating the FRT Index 16 / 27

56. FRT Index Overview (1990)
Canada
Australia
United States
Japan
Netherlands
Hungary
Poland
Norway
Trinidad and Tobago
Finland
Denmark
France
Spain
Belgium
Israel
Iceland
Korea, Rep.
Bahamas, The
Portugal
Italy
Cyprus
Ireland
Greece
Barbados
Switzerland
New Zealand
Austria
Malta
Equatorial Guinea
Luxembourg
Macao SAR, China
Singapore
Bahrain
Saudi Arabia
Sweden
United Kingdom
Aruba
Hong Kong SAR, China
United Arab Emirates
Germany
Oman
Qatar
Estonia
Kuwait
Slovak Republic
Slovenia
San Marino
Croatia
Czech Republic
Brunei Darussalam
New Caledonia
Curacao
French Polynesia
Isle of Man
Liechtenstein
Andorra
Bermuda
Faeroe Islands
Monaco
Cayman Islands
1990
−3 −1 0 1 3
FRT Index Creating the FRT Index 17 / 27

57. FRT Index Overview (2011)
Croatia
United States
Netherlands
Australia
Belgium
Germany
Slovenia
Greece
Spain
Korea, Rep.
Portugal
Switzerland
Iceland
Finland
Italy
France
Singapore
Austria
Ireland
Brunei Darussalam
Bahamas, The
Oman
Cyprus
Malta
Qatar
Luxembourg
Equatorial Guinea
Japan
United Arab Emirates
Kuwait
Hong Kong SAR, China
Slovak Republic
Saudi Arabia
Macao SAR, China
Estonia
Israel
Trinidad and Tobago
Denmark
Hungary
Czech Republic
Poland
Sweden
United Kingdom
Aruba
Bahrain
New Zealand
Canada
Barbados
Norway
San Marino
Monaco
Faeroe Islands
Liechtenstein
Cayman Islands
Isle of Man
Curacao
French Polynesia
Andorra
New Caledonia
Bermuda
2011
−20 −10 −3 −1 0 1 3 5
FRT Index Creating the FRT Index 18 / 27

58. Stable Countries
0.20
0.15
0.10
0.05
Saudi Arabia
1990 1995 2000 2005 2010
0.10
0.05
0.00
United Kingdom
1990 1995 2000 2005 2010
2.5
2.0
1.5
1.0
United States
1990 1995 2000 2005 2010
FRT Index Creating the FRT Index 19 / 27

59. Improving Countries
1
0
−1
−2
Brunei Darussalam
1990 1995 2000 2005 2010
3
2
1
0
−1
Croatia
1990 1995 2000 2005 2010
2.0
1.5
1.0
0.5
0.0
Germany
1990 1995 2000 2005 2010
2.0
1.5
1.0
0.5
Greece
1990 1995 2000 2005 2010
2.0
1.5
1.0
0.5
Iceland
1990 1995 2000 2005 2010
2.5
2.0
1.5
1.0
0.5
0.0
Korea, Rep.
1990 1995 2000 2005 2010
0.8
0.6
0.4
0.2
0.0
−0.2
Oman
1990 1995 2000 2005 2010
1.5
1.0
0.5
Portugal
1990 1995 2000 2005 2010
0.75
0.50
0.25
0.00
Qatar
1990 1995 2000 2005 2010
2
1
0
Slovenia
1990 1995 2000 2005 2010
2.0
1.5
1.0
0.5
Spain
1990 1995 2000 2005 2010
0.3
0.2
0.1
0.0
−0.1
United Arab Emirates
1990 1995 2000 2005 2010
FRT Index Creating the FRT Index 20 / 27

60. Declining Countries
12
8
4
0
Canada
1990 1995 2000 2005 2010
2
1
0
−1
Czech Republic
1990 1995 2000 2005 2010
2.0
1.5
1.0
0.5
0.0
Denmark
1990 1995 2000 2005 2010
3
2
1
0
Hungary
1990 1995 2000 2005 2010
2.0
1.5
1.0
0.5
0.0
Israel
1990 1995 2000 2005 2010
8
6
4
2
0
Japan
1990 1995 2000 2005 2010
4
3
2
1
0
−1
Norway
1990 1995 2000 2005 2010
3
2
1
0
Poland
1990 1995 2000 2005 2010
FRT Index Creating the FRT Index 21 / 27

61. Comparison to frequency measure
2
1
0
−1
−2
−3
−3 −2 −1 0 1 2
Median FRT Index (standardized)
Proportion Reported (standardized)
FRT Index Comparison to other methods 22 / 27

62. Discrimination parameter
How well reporting an item predicts reporting other items.
l
l
l
l
l
l
l
l
l
l
l
l
l
l
Bank deposits to GDP (%)
Financial system deposits to GDP (%)
Private credit by deposit money banks to GDP (%)
Liquid liabilities to GDP (%)
Domestic credit to private sector (% of GDP)
Credit to government and state owned enterprises to GDP (%)
Liquid liabilities in millions USD (2000 constant)
Bank credit to bank deposits (%)
Central bank assets to GDP (%)
Deposit money bank assets to deposit money bank assets and central bank assets (%)
Insurance company assets to GDP (%)
Mutual fund assets to GDP (%)
Bank lending−deposit spread
Nonbank financial institutions' assets to GDP (%)
0.0 2.5 5.0 7.5 10.0
Coefficient
FRT Index Comparison to other methods 23 / 27

63. Diculty parameter
On average how well reported is the item.
Higher scores indicate lower reporting, i.e. more `dicult' to
report.
l
l
l
l
l
l
l
l
l
l
l
l
l
l
Nonbank financial institutions' assets to GDP (%)
Mutual fund assets to GDP (%)
Insurance company assets to GDP (%)
Central bank assets to GDP (%)
Deposit money bank assets to deposit money bank assets and central bank assets (%)
Bank deposits to GDP (%)
Financial system deposits to GDP (%)
Liquid liabilities to GDP (%)
Private credit by deposit money banks to GDP (%)
Liquid liabilities in millions USD (2000 constant)
Domestic credit to private sector (% of GDP)
Credit to government and state owned enterprises to GDP (%)
Bank credit to bank deposits (%)
Bank lending−deposit spread
−1 0 1 2 3
Coefficient
FRT Index Comparison to other methods 24 / 27

64. Comparison to survey/frequency measures
Comparision to Liedorp et al. (2013)
l l
l
l l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
2
1
0
−1
−2
0.00 0.25 0.50 0.75
FRT (median stnd)
Political (Liedorp et al. stnd)
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l l
l
3
2
1
0
−1
−2
0.00 0.25 0.50 0.75
FRT (median stnd)
Economic (Liedorp et al. stnd)
l
l
l
l
l
l
l l
l
l l
l
l
l
l
l
l l l
3
2
1
0
−1
0.00 0.25 0.50 0.75
FRT (median stnd)
Procedural (Liedorp et al. stnd)
l l
l
l l
l
l
l
l
l l
l
l l
l
l l
l
l
3
2
1
0
−1
−2
0.00 0.25 0.50 0.75
FRT (median stnd)
Policy (Liedorp et al. stnd)
l l
l l
l
l
l l
l l l
l
l l
l
l
l
l
l
2
1
0
−1
−2
0.00 0.25 0.50 0.75
FRT (median stnd)
Operational (Liedorp et al. stnd)
l
l
l
l
l
l
l
l
l
l
l l
l
l
l
l
l
l
3
2
1
0
l −1
0.00 0.25 0.50 0.75
FRT (median stnd)
Total (Liedorp et al. stnd)
FRT Index Comparison to other methods 25 / 27

65. Annoying issues. . .
1. There is a possibility that missing-ness is sometimes caused by
World Bank data handling errors rather than countries'
willingness to report.
For example, Bank Deposits to GDP (%) is not reported for the
UK. However, a mirror of the GFDD (FRED) does have the data.
http://research.stlouisfed.org/fred2/series/
DDOI02GBA156NWDB
2. Missing data may be entered by later governments/
supervisors.
FRT Index Comparison to other methods 26 / 27

66. To-Do
I Further re

67. ne the Index.
I Understand why countries increase/decrease their reporting.
I Examine how reporting is associated with economic outcomes:
I Investment
ows
I Financial stability
FRT Index Conclusion 27 / 27

68. To-Do
I Further re

69. ne the Index.
I Understand why countries increase/decrease their reporting.
I Examine how reporting is associated with economic outcomes:
I Investment
ows
I Financial stability
FRT Index Conclusion 27 / 27

70. To-Do
I Further re

71. ne the Index.
I Understand why countries increase/decrease their reporting.
I Examine how reporting is associated with economic outcomes:
I Investment
ows
I Financial stability
FRT Index Conclusion 27 / 27