Presentation by Christiane Arndt, Programme Co-ordinator Measuring Regulatory Performance, and Antonia Custance Baker, Policy Analyst, OECD, at the 6th Expert Meeting on Measuring Regulatory Performance: Evaluating Stakeholder Engagement in Regulatory Policy, Breakout Session 2, The Hague, 16-18 June 2014. Further information is available at http://www.oecd.org/gov/regulatory-policy/

1. Indicators and good practice
database
Christiane Arndt, Programme Co-ordinator Measuring
Regulatory Performance, OECD
Antonia Custance Baker
Breakout session 2
6th Expert Meeting on Measuring Regulatory Performance
17 June 2014

2. • Are all questions on stakeholder
engagement clear? Do any questions leave
room for interpretation?
Questions for discussion

3. • Why do we need indicators on
consultation?
• How can the REG indicators be
complemented with data/indicators from
other sources?
• How would you use a good practice
database to complement the indicators?
How could it look like?
Questions for discussion

4. • What are the pros and cons of different
aggregation methods?
Questions for discussion

5. Tables with all
answers of each
country
Indicators of
Regulatory
Management
Systems
Worldwide
Governance
Indicator (WGI) on
Regulatory Quality
Product Market
Regulation Indicator
Degree of aggregation

6. • Simple weights (e.g. OECD Product Market
Regulation Indicators)
• Expert weights (e.g. Indicators of Regulatory
Management Systems 2008)
• Statistical weighting methods based on variation
and correlation of data such as PCA, unobserved
component model (e.g. Worldwide Governance
Indicators)
• Users can set weights themselves online (e.g.
Better Life Index)
Method of aggregation
See also: OECD, European Commission (2008), Handbook on Constructing
Composite Indicators

7. OECD Product Market Regulation
Indicators

8. Worldwide Governance Indicators

9. Note: This graph summarises information about the existence of key elements of formal consultation processes for primary laws in OECD member countries. It
does not gauge whether these processes have been effective.
Questions: Weights:
a) Is public consultation with parties affected by regulations a routine part of developing draft new primary laws? No=0, In some cases=0.5, Always=1
When is it conducted? a(i) at the inception of the legal proposal? No=0, Yes=0.75
a(ii) during the drafting of a law? No=0, Yes=0.5
If the answer is “always” or “in some cases” to a) Is consultation mandatory? No=0, Yes=0.5
b(v) Are there consultation guidelines? No=0, Yes=0.5
If so, are they mandatory? No=0, Yes=0.5
b(vii) What forms of public consultation are routinely used:
- Broad circulation of proposals for comment?
- Public notice and calling for comment?
- Public meeting?
- Simply posting proposals on the Internet?
- Advisory group?
- Preparatory public commission/committee?
If ticked, weight= 0.25, 0.5, 0.25, 0.25, 0.25, 0.25
b(viii) Can any member of the public choose to participate in the consultation? No=0, Yes=0.5
c(i) What is the minimum period for allowing consultation comments inside government? 1 week=0.125, 2 weeks=0.25, 4 weeks or more=0.5
c(ii) What is the minimum period for allowing consultation comments by the public, including citizens and business and civil
society organisations?
2 weeks=0.125, 3 weeks=0.2, 4 weeks=0.25, 6
weeks=0.4, 8 weeks= 0.5, 12 weeks or more= 0.75
d(i) Are the views of participants in the consultation process made public? No=0, Yes=0.5
d(ii) Are regulators required to respond in writing to the authors of consultation comments? No=0, Yes= 0.25
d(iii) Are the views expressed in the consultation process included in the regulatory impact analysis? No=0, Yes=0.5
d(iv) Is there a process to monitor the quality of the consultation process? (e.g. surveys or other methods, please specify in
comments)
No=0, Yes=0.5
d(v) Is guidance available on how to conduct effective consultation? No=0, Yes=0.5
Source: Question 9 / OECD Regulatory Management Systems’ Indicators Survey 2008. www.oecd.org/regreform/indicators
Formal consultation processes: Primary laws (2008)
0
1
2
3
4
5
6
7
8
9
ITA
FRA
JPN
PRT
LUX
USA
BEL
CZE
ESP
AUT
SVK
GRE
NLD
TUR
ICE
DEU
HUN
DNK
MEX
NOR
IRL
POL
FIN
AUS
CAN
NZL
KOR
CHE
SWE
EU
UK
5th percentile 95th percentile Index
9.5

10. Formal consultation processes: Subordinate legislation (2008)
0
1
2
3
4
5
6
7
8
9 TUR
BEL
GRE
ITA
ICE
LUX
PRT
AUT
SVK
NLD
FRA
DEU
DNK
HUN
CZE
ESP
CHE
IRL
POL
NOR
USA
FIN
JPN
AUS
MEX
NZL
CAN
KOR
SWE
EU
UK
5th percentile 95th percentile Index
9.5
Note: This graph summarises information about the existence of key elements of formal consultation processes for subordinate regulations in OECD member
countries. It does not gauge whether these processes have been effective.
Questions: Weights:
a) Is public consultation with parties affected by regulations a routine part of developing new draft subordinate regulations? No=0, In some cases=0.5, Always=1
When is it conducted? a(i) at the inception of the legal proposal? No=0, Yes=0.75
a(ii) during the drafting of a regulatory impact statement (RIS)? No=0, Yes=0.5
If the answer is “always” or “in some cases” to a) Is consultation mandatory? No=0, Yes=0.5
b(v) Are there consultation guidelines? No=0, Yes=0.5
If so, are they mandatory? No=0, Yes=0.5
b(vii) What forms of public consultation are routinely used:
- Broad circulation of proposals for comment?
- Public notice and calling for comment?
- Public meeting?
- Simply posting proposals on the Internet?
- Advisory group?
- Preparatory public commission/committee?
If ticked, weight= 0.25, 0.5, 0.25, 0.25, 0.25, 0.25
b(viii) Can any member of the public choose to participate in the consultation? No=0, Yes=0.5
c(i) What is the minimum period for allowing consultation comments inside government? 1 week=0.125, 2 weeks=0.25, 4 weeks or more=0.5
c(ii) What is the minimum period for allowing consultation comments by the public, including citizens, business and civil
society organisations?
2 weeks=0.125, 3 weeks=0.2, 4 weeks=0.25, 6
weeks=0.4, 8 weeks= 0.5, 12 weeks or more= 0.75
d(i) Are the views of participants in the consultation process made public? No=0, Yes=0.5
d(ii) Are regulators required to respond in writing to the authors of consultation comments? No=0, Yes= 0.25
d(iii) Are the views expressed in the consultation process included in the regulatory impact analysis? No=0, Yes=0.5
d(iv) Is there a process to monitor the quality of the consultation process? (e.g. surveys or other methods, please specify in
comments)
No=0, Yes=0.5
d(v) Is guidance available on how to conduct effective consultation? No=0, Yes=0.5
Source: Question 9 / OECD Regulatory Management Systems’ Indicators Survey 2008. www.oecd.org/regreform/indicators

11. • OECD, Better Life Index
• OECD, Product Market Regulation Indicators
• Worldbank, Worldwide Governance Indicators
• OECD, Services Trade Restrictiveness Index –
Regulatory Database
• OECD, Services Trade Restrictiveness Index –
Policy Simulator
• OECD, Observatory of Public Sector Innovation
Links to indicators and databases

12. • What are the pros and cons of different
aggregation methods?
• How would you use a good practice
database to complement the indicators?
How could it look like?
Questions for discussion

13. • Stockholm workshop
• Drafting a list of practices based on findings from Stockholm workshop
• Invitation to all RPC delegates and delegates from TUAC and BIAC to join the
Steering Group to be actively involved in the project
• Consultation on list of practices with RPC delegates
• Final list of practices
Phase 1
Discussions about assessing the
implementation of the
Recommendation
June 2013 – March 2014
• Drafting of questionnaire based on list of practices
• Discussion of draft questionnaire with Steering Group at one-day meeting
• Revision of questionnaire based on comments from Steering Group members
and comments from experts across the OECD
• Revised questionnaire circulated for piloting
• Final clarification of questions, answer options and definitions based on pilots
and meetings in The Hague
• Final questionnaire
Phase 2
Development of questionnaire
February 2014 – June 2014
• Questionnaire sent to countries
• Preliminary analysis and identification of data issues
• Discussion of survey results at RPC meeting in November with delegates,
TUAC and BIAC
• Steering group meeting on construction of indicators
• Adjustment and analysis of data
Phase 3
Data collection, cleaning and
analysis
July 2014 – March 2015
• Draft Regulatory Policy Outlook and final data and indicators presented at RPC
• Publication of Regulatory Policy Outlook and final database
• Development of best practice database in selected areas
Phase 4
Publication and communication of
results
April 2015 – end 2015
Timeline