Presentation by Gary Banks, Chair of the Regulatory Policy Committee, at the 9th Conference on Measuring Regulatory Performance - Closing the Regulatory Cycle: Effective ex post Evaluation for Improved Policy Outcomes which took place in Lisbon on 20-21 June 2017. Further information is available at www.oecd.org/gov/regulatory-policy/measuring-regulatory-performance.htm.

1. Effective ex-post evaluation:
purpose and challenges
Gary Banks
Presentation to OECD’s 9th Expert Meeting on Measuring
Regulatory Performance
June 2017
Lisbon, Portugal

2. OECD Council Recommendation #5
“Conduct systematic reviews of regulation
against policy goals, including costs and
benefits, to ensure that regulations
-- remain up to date, cost effective and
consistent, and
-- deliver intended policy objectives”
2

3. Ex post evaluation: why needed?
• All regulations are ‘experiments’
─ Many will not have been done well
─ Others will have passed their ‘use by date’
• The ‘stock’ of regulation is much greater than
the flow and gains are potentially large
• Evaluations can build trust and help sustain
political support
• Ex post reviews can also provide learnings for
future regulatory actions
3

4. 4
There are multiple costs of regulation
Benefits foregone if
regulation is ineffective
 other perverse effects
 other ‘non -market’
distortions
Fees and charges a
Economic distortions
 dead weight losses
 lower investment
 lower innovation
Substantive compliance
costs
 investments in systems
training
 higher cost of investment
Administrative costs to
business
 paper work time
 reporting time
Administration cost to
regulators
Benefitsneededtojustifycosts
Compliance costs ‘Distortion’ costs
Costs to government Costs to business b
Costs to community
Coststocommunity

5. 5
Ex post reviews complete the ‘regulatory cycle’
Stage II–Establishment
 Design to include
embedded reviews
 Development of regulator
management strategies
Stage IV- Review
 Programmed reviews
o Sunsetting
o Embedded
o PIRS
 Ad hoc reviews
o In-depth
o Specific benchmarking
o Public stocktakes
Stage I- Decision
 RIS triage
o Identify need for
embedded reviews
o Set sunset flags
 Stock-flow linkage rules
Stage III - Administration
 Regulator management
strategies
 Monitoring review
requirements
Lessons from regulators
on what works
Lessons from
expost evaluation

6. What is an effective system?
• Comprehensive coverage over time
− including related instruments
• Targeted and proportionate
− where most potential for net gains
• Timeliness of reviews
• Effective consultation mechanisms
• Independence where appropriate
• Institutionalised oversight
− Monitoring performance of the overall system
6

7. 7
Approaches to ex post evaluation
Programmed reviews Ad hoc reviews Ongoing ‘management’
•Sunsetting
•Embedded in statute
•Post implementation
reviews
– process failure
– catch-all
•Public stocktakes
•‘Principles-based’
reviews
•Benchmarking
•‘In-depth’ reviews
•Stock-flow linkages
– Budgets
– ‘In-Out’ / ‘Offsets’
•Red tape reduction
targets

8. Key questions to be answered …
• Appropriate? (Still a valid rationale?)
• Effective? (Achieved the intended outcome?)
• Efficient? (Unnecessary costs or unintended
impacts?)
• A better alternative?
8

9. Fewer countries have formal requirements
Source: 2015 Regulatory Policy Outlook
9
28
24
28
ConsultationRIA Ex-post evaluation of
existing regulation
21
24
20
3433
29
35

10. How do OECD countries measure up?
10
10
Source: OECD, Regulatory Policy Outlook 2015
http://www.oecd.org/gov/regulatory-policy/measuring-regulatory-performance.htm

11. Ex post reviews in Australia: a ‘mixed bag’?
According to a Productivity Commission review,
improvements were needed in the following areas:
• embedding evaluations in regulatory design
• proportionality/depth of reviews
• prioritisation and monitoring
• timeliness (PC 2011)
11
11

12. Some obstacles to better practice
• ‘Technical’
−(Highly) skilled people are scarce
−Data are often poor
• ‘Political’
−Greater focus on the new than the old
−Perceived risks from a poor ‘score’
−Beneficiaries may oppose scrutiny
−Administrative ‘inertia’ is ever present!
12

13. Ways forward? Insights from Australia
• Building ex post evaluation into a regulatory
regime from the outset has advantages
• ‘Automatic’ triggers can be useful
• Attention to timing can make a big difference
• Skill development is fundamental
• Need avoid administrative ‘overload’
• Creating oversight arrangements helps
13

14. Effective ex-post evaluation:
purpose and challenges
Gary Banks
Presentation to OECD’s 9th Expert Meeting on Measuring
Regulatory Performance
June 2017
Lisbon, Portugal