Presentation by Kevin Williams, OECD Secretariat, at the 11th annual meeting of the OECD Senior Budget Officials Performance and Results network, OECD, 26-27 November 2015

1. EVALUATING THE
PERFORMANCE OF
OECD COMMITTEES
11th ANNUAL MEETING OF THE OECD SENIOR
BUDGET OFFICIALS PERFORMANCE AND
RESULTS NETWORK
27 NOVEMBER 2015
Kevin Williams – Head of In-depth Evaluation
(SGE-EVIA)

2. Evaluation in the OECD
2
Integrated Management Cycle
Programme Implementation Reporting (PIR)
Medium-term Orientations exercise (MTO)
In-depth Evaluation of Committees (IDE)
Committee activities and work programme development and
implementation (including peer reviews, evaluation
guidance, etc.)
Members and Partner countries
Organisational
focus
Substantive
focus

3. Integrated Management Cycle
Biennium 2013-14 Biennium 2015-16 Biennium 2017-18
PIR
MTO
PIR PIR
MTO MTO
IDE
Prospective
Restrospective

4. The objective of IDE is to provide a mechanism through
which Council can assess whether OECD Committees are…
– conducting processes…
– delivering outputs…
– achieving impacts…
4
IDE’s overarching objective
… in line with Members’ policy
expectations and with the OECD’s
comparative advantage

5. 5
Evaluation criteria for IDE
• the extent to which the Committee is
• producing products of the requisite quality for the
resources allocated (technical efficiency)
• how well it is functioning (process efficiency)
Efficiency
• the extent to which a Committee’s mandate
and work programme objectives are aligned
with Members’ policy needs and
concerns
Relevance
• the extent to which policy impacts resulting from
the Committee’s products are occurring and
whether they correspond with areas of highest
policy needs and concerns
Effectiveness

6. 6
IDE Cycles and timeline
1st IDE Cycle
2005 2012
2nd IDE Cycle
2017

7. Example of an OECD Committee
Public
Governance
Committee (PGC)
Working Party
of Senior
Budget
Officials
(SBO)
Network on
Public
Employment
and
Management
(PEM)
Network on
Public Sector
Integrity
Network on E-
Government
High Level
Risk Forum
(HLRF)
Working Party
of Leading
Practitioners
on Public
Procurement
(LPP)
Network of
Senior
Officials from
Centres of
Government
(COG)
Network on Financial
Management
Network of Parliamentary
Budget Officials and
Independent Fiscal Institutions
Network on Performance and
Results

8. 8
Committee policy cycle
Policy needs
Biennial Work
Programme
Processes Outputs
Policy impacts
Mandate
Policy objectives
Resources
Committeeorientation
Committee functioning
Use
Awareness
Dissemination
andtakeup
Policy needs and impacts

9. • Committee mandates and work programmes
aim to address Members’ policy needs, e.g.
– to deliver better, more cost-effective and user-
centric public services
– to further enhance corruption resistance in risk
areas at the political-administrative and public-
private sector interface
– to ensure greater inclusiveness, and increased
economic and social resilience through risk
management
– …
Examples of policy objectives

10. • OECD knowledge products and instruments:
– Statistics, indicators, databases, related development
work (e.g. methodological frameworks)
– Benchmarking and comparative reviews
– Reports and analyses (incl. peer reviews of Member
and non-Member economies)
– Conferences, workshops, forums
– Guidelines and recommendations
– Formal agreements
10
Committee outputs

11. • Knowledge, information, data, guidance,
recommendations, etc. embodied in a Committee output:
– substantively represent or form the basis of government
policy
– are considered as the standard for policy setting
– have been raised in Parliament, been the subject of
Ministerial and/or official announcements
– have been proposed to be enacted as legislation, enacted as
legislation or the subject of international agreement
– have been raised in major public forums as being
authoritative for policy direction
– …
Examples of policy impact

12. 12
IDE outputs and outcomes
Conclusions
and
ratings
Recommendations
and Good Practices
• Reinforced
transparency
and accountability to
Council
• More informed
Council decisions on
mandate
appropriateness and
renewal
Analyses
and
findings
• Mandate and work programme
development
• Meeting preparation and
conducting
• Work programme oversight
• Vertical coordination
• Horizontal working
• Engagement with Partner
countries
• Involvement of other
international organisations and
stakeholder bodies
• Communication and
dissemination
Monitoring
of
implementation
• Design and
implementation of
relevant actions
• Improved
committee
performance

13. • Planning, timing and purpose
• Support from senior management
• Involvement of stakeholders
• Dissemination of results
• Monitoring follow-up of recommendations
• Evaluation quality
• Resource availability
Factors affecting the likelihood of evaluation
use*
*Bastiaan de Laat and Kevin Williams from Enhancing Evaluation Use: Insights from
Internal Evaluation Units, Marlène Läubli Loud and John Mayne (eds), Sage, 2014.

14.  Accountability to governing bodies
and awareness raising amongst
stakeholders more broadly
 Improving the design and
implementation of ‘interventions’
 Resource (re)allocation within
‘interventions’
 Supporting organisational learning
 Setting strategic or policy priorities
 Resource (re)allocation between
‘interventions’
Different evaluation uses

15. • To what extent does your experience of the
evaluation use tally with the previous
slide?
• Why is the use of evaluation results less
prevalent in priority setting and resource
(re)allocation contexts?
Possible discussion questions