Developing a More Systematic Approach to Evaluation Across Policy Instruments

•

0 likes•162 views

Presentation by Knut Klepsvik, Ministry of Finance, Norway, at the 9th Conference on Measuring Regulatory Performance - Closing the regulatory cycle: effective ex post evaluation for improved policy outcomes which took place in Lisbon on 20-21 June 2017. Further information is available at www.oecd.org/gov/regulatory-policy/measuring-regulatory-performance.htm.

Breakout session 3
DEVELOPING A MORE SYSTEMATIC
APPROACH TO EVALUATION ACROSS
POLICY INSTRUMENTS
Knut Klepsvik
Deputy Director General, Budget Department, Ministry of
Finance, Norway

Breakout Session 3
DEVELOPING A MORE SYSTEMATIC APPROACH TO EVALUATION ACROSS POLICY
INSTRUMENTS
Discussion points
• Facilitate joined-up approaches to evaluation?
• Which institutions are best placed to prepare evaluations?
• Methodological approaches to underpin the evaluation of the
cumulative impacts?
Kick-off presentations
• Bruno Damásio, New University of Lisbon
• Stephan Naundorf, Federal Chansellery, Germany

Breakout Session 3
DEVELOPING A MORE SYSTEMATIC APPROACH TO EVALUATION ACROSS POLICY
INSTRUMENTS
» Key takeaway 1: Different understandings of the term
evaluation
» Key takeaway 2: Some implications of different approaches of
evaluation
» Key takeaway 3: Several institutional frameworks noticed
» Key takeaway 4: Specific examples of evaluations mentioned

Breakout Session 3
DEVELOPING A MORE SYSTEMATIC APPROACH TO EVALUATION ACROSS POLICY
INSTRUMENTS
Examples
» Netherland draws up a programme to improve a sector, and
measure progress against that programme
» Austria has a system of scrutinizing and publishing whether
budgetary goals have been met or not
» Very comprehensive evaluations in Australia (by the
Productivity Commission)
» MoF in Switzerland registered along the evaluation process a
source of saving resources also on the government side
» Norway has common requirements for evaluation of
government measures and an internet portal for all evaluations

Presentation by Caroline Wigerstad, Better Regulation, Agency for Economic and Regional Growth, Sweden, at the 9th Conference on Measuring Regulatory Performance - Closing the Regulatory Cycle: Effective ex post Evaluation for Improved Policy Outcomes which took place in Lisbon on 20-21 June 2017. Further information is available at www.oecd.org/gov/regulatory-policy/measuring-regulatory-performance.htm.

Engaging stakeholders effectively for a targeted approach in evaluation

OECD Governance

Processes and institutions for effective ex post evaluation

OECD Governance

This document summarizes a breakout session at an OECD conference on measuring regulatory performance. The session discussed developing more systematic approaches to joint ex-post evaluation across policy instruments. While ex-ante evaluation is undertaken more systematically than ex-post, ex-post approaches vary and oversight is often weak. The session aimed to promote synergies between regulatory and expenditure evaluation, such as common frameworks and transparency norms. Questions focused on practical examples of joint evaluations, their value, and enabling better collaboration between institutions.

Engaging Stakeholders Effectively for a Targeted Approach in Evaluation

OECD Governance

Evaluation in Budgeting and Public Expenditure

OECD Governance

Policy impact evaluation: some notes and the challenge of joint effects

OECD Governance

Presentation by Bruno Damásio, ISEG - Universidade de Lisboa and NOVA IMS - Universidade NOVA de Lisboa, Portugal, at the 9th Conference on Measuring Regulatory Performance - Closing the Regulatory Cycle: Effective ex post Evaluation for Improved Policy Outcomes which took place in Lisbon on 20-21 June 2017. Further information is available at www.oecd.org/gov/regulatory-policy/measuring-regulatory-performance.htm.

Effective ex post Evaluation: Purpose and Challenges

OECD Governance

Institutionalising ex post Evaluation to Feed Back into the Regulatory Process

OECD Governance

Presentation by Manuel Cabugueira, Legal Centre of the Presidency of the Council of Ministers (CEJUR), Portugal, at the 9th Conference on Measuring Regulatory Performance - Closing the Regulatory Cycle: Effective ex post Evaluation for Improved Policy Outcomes which took place in Lisbon on 20-21 June 2017. Further information is available at www.oecd.org/gov/regulatory-policy/measuring-regulatory-performance.htm.

Processes and Institutions for Effective ex post Evaluation

OECD Governance

Administrative burdens created by the national transposition of EU legislation

OECD Governance

Presentation by Professor Licínio Lopes Martins, Center for Studies in Public Law and Regulation, Faculty of Law of the University of Coimbra, Portugal, at the 9th Conference on Measuring Regulatory Performance - Closing the Regulatory Cycle: Effective ex post Evaluation for Improved Policy Outcomes which took place in Lisbon on 20-21 June 2017. Further information is available at www.oecd.org/gov/regulatory-policy/measuring-regulatory-performance.htm.

Evaluating the performance of OECD Committees -- Kevin Williams, OECD Secreta...

OECD Governance

Publilc sector productivity - Patrick Dunleavy, United Kingdom

OECD Governance

Policy evaluation in the Netherlands -- Birgit ten Cate - Netherlands

OECD Governance

This document summarizes the evaluation system and policy review process in the Netherlands. It discusses the main evaluation instruments, including policy reviews conducted by line departments and spending reviews led by the Ministry of Finance. It notes several issues with the current approach, including a lack of data, quality concerns, asking the wrong questions at the wrong time, and lack of impact on actual policy changes. Finally, it considers some alternatives to improve policy reviews, such as starting the review process earlier, increasing cooperation with researchers to enhance data quality, and establishing more external influence to ensure a critical perspective.

Evaluation of regulations and beyond -- Christiane Arndt, OECD Secretariat

OECD Governance

Molly Irwin - Evidence informed policy making - 27 June 2017

OECD Governance

Presentation by Molly Irwin, Chief Evaluation Officer, U.S. Department of Labor, at the event on Governing better through evidence-informed policy making, 26-27 June 2017. The event was organised by the OECD Directorate for Public Governance in cooperation with the European Commission’s Joint Research Centre (JRC), the Campbell Collaboration and the International Network for Government Science Advice (INGSA). For further information please see http://www.oecd.org/gov/evidence-informed-policy-making.htm

Public sector productivity - Peter van de Ven, OECD

OECD Governance

Administrative Modernisation

OECD Governance

Presentation by Maria Manuel Leitão Marques, Minister of the Presidency and Administrative Modernization, Portugal, at the 9th Conference on Measuring Regulatory Performance - Closing the Regulatory Cycle: Effective ex post Evaluation for Improved Policy Outcomes which took place in Lisbon on 20-21 June 2017. Further information is available at www.oecd.org/gov/regulatory-policy/measuring-regulatory-performance.htm.

Evaluating OECD performance on performance & evaluation: successes, challenge...

OECD Governance

Building Capacity for Evidence-Informed Policy-Making Lessons from Country Ex...

OECD Governance

Evaluation and the budget - the role of SAIs -- Leonardo Albernaz, Brazil

OECD Governance

OECD Principles on Budgetary Governance - Anne Keller, OECD

OECD Governance

Understanding progress to date in Ireland's reform of its budgetary architect...

OECD Governance

Developing a More Systematic Approach to Evaluation Across Policy Instruments

OECD Governance

This document discusses the challenges Germany faces in developing a more systematic approach to evaluating policy instruments. It notes that several actors in Germany are responsible for conducting ex post evaluations of government actions, including the Federal Parliament, Court of Auditors, Constitutional Court, and Government. However, it identifies seven key challenges: 1) clearly defining what should be evaluated and who is affected; 2) focusing analyses on the right areas; 3) using the right evaluation instruments; 4) ensuring data is meaningful and accessible; 5) effectively communicating evaluation results; 6) understanding cumulative effects of policies; and 7) avoiding getting stuck in repetitive evaluation cycles. The document argues that overcoming these challenges can lead to more rewarding outcomes.

Evaluating stakeholder engagement: Practices in OECD countries, Laura Seiffer...

OECD Governance

Evaluating stakeholder engagement in regulatory policy, Christiane Arndt

OECD Governance

2014.11.28 - NAEC Group Meeting_Lamia Kamal-Chaoui

OECD_NAEC

This document summarizes the work of the OECD's Inclusive Growth Initiative. It established a multidimensional framework to measure inclusive growth that accounts for income as well as non-income factors like employment and health. Between 2012-2014, the initiative delivered reports applying this framework and identifying policy areas that can boost both growth and inclusiveness. Going forward, it will refine the methodology, integrate inclusive growth analysis across policy sectors, and conduct national and regional case studies to strengthen the evidence base and mainstream inclusive growth policymaking.

Introduction to ClimateXChange 14.01.22

Edinburgh Centre for Carbon Innovation

ClimateXChange was created by the Scottish Government to address the mismatch between climate change research funding and policy teams' access to research outputs. It aims to deliver objective, independent, and authoritative evidence to support Scottish Government policy development and implementation regarding climate change and transitioning to a low carbon economy. ClimateXChange provides a call down service for timely policy support, facilitates conversations between researchers and policymakers, and coordinates near-term, policy-relevant research projects in mitigation and adaptation. Examples of its work include reports on social factors for successful community renewable projects and emissions abatement from peatland restoration.

What's hot

Implementing the One-for-One Rule: Background, Mechanics, Results and Lessons

OECD Governance

Regulatory Budgets and Stock-Flow Linkage Rules

OECD Governance

Evaluation methods for improved policy outcome

OECD Governance

Processes and Institutions for Effective ex post Evaluation

OECD Governance

Administrative burdens created by the national transposition of EU legislation

OECD Governance

Evaluating the performance of OECD Committees -- Kevin Williams, OECD Secreta...

OECD Governance

Publilc sector productivity - Patrick Dunleavy, United Kingdom

OECD Governance

Policy evaluation in the Netherlands -- Birgit ten Cate - Netherlands

OECD Governance

Evaluation of regulations and beyond -- Christiane Arndt, OECD Secretariat

OECD Governance

Molly Irwin - Evidence informed policy making - 27 June 2017

OECD Governance

Public sector productivity - Peter van de Ven, OECD

OECD Governance

Administrative Modernisation

OECD Governance

Evaluating OECD performance on performance & evaluation: successes, challenge...

OECD Governance

Building Capacity for Evidence-Informed Policy-Making Lessons from Country Ex...

OECD Governance

Evaluation and the budget - the role of SAIs -- Leonardo Albernaz, Brazil

OECD Governance

OECD Principles on Budgetary Governance - Anne Keller, OECD

OECD Governance

Understanding progress to date in Ireland's reform of its budgetary architect...

OECD Governance

Developing a More Systematic Approach to Evaluation Across Policy Instruments

OECD Governance

Evaluating stakeholder engagement: Practices in OECD countries, Laura Seiffer...

OECD Governance

Evaluating stakeholder engagement in regulatory policy, Christiane Arndt

OECD Governance

What's hot (20)

Implementing the One-for-One Rule: Background, Mechanics, Results and Lessons

Regulatory Budgets and Stock-Flow Linkage Rules

Evaluation methods for improved policy outcome

Processes and Institutions for Effective ex post Evaluation

Administrative burdens created by the national transposition of EU legislation

Evaluating the performance of OECD Committees -- Kevin Williams, OECD Secreta...

Publilc sector productivity - Patrick Dunleavy, United Kingdom

Policy evaluation in the Netherlands -- Birgit ten Cate - Netherlands

Evaluation of regulations and beyond -- Christiane Arndt, OECD Secretariat

Molly Irwin - Evidence informed policy making - 27 June 2017

Public sector productivity - Peter van de Ven, OECD

Administrative Modernisation

Evaluating OECD performance on performance & evaluation: successes, challenge...

Building Capacity for Evidence-Informed Policy-Making Lessons from Country Ex...

Evaluation and the budget - the role of SAIs -- Leonardo Albernaz, Brazil

OECD Principles on Budgetary Governance - Anne Keller, OECD

Understanding progress to date in Ireland's reform of its budgetary architect...

Developing a More Systematic Approach to Evaluation Across Policy Instruments

Evaluating stakeholder engagement: Practices in OECD countries, Laura Seiffer...

Evaluating stakeholder engagement in regulatory policy, Christiane Arndt

Similar to Developing a More Systematic Approach to Evaluation Across Policy Instruments

2014.11.28 - NAEC Group Meeting_Lamia Kamal-Chaoui

OECD_NAEC

Introduction to ClimateXChange 14.01.22

Edinburgh Centre for Carbon Innovation

Mobilising Evidence for Good Governance - OECD Presentation by Stéphane Jacob...

OECD Governance

Designing Policy Experimentation

Danish Design Centre

BRAINPOoL Final Conference: Integrated & Innovative Policy Making

nefwellbeing

This document outlines a proposed organizational change program to help shift policymaking beyond a sole focus on GDP. It argues that political leadership is needed to open economic policymaking to new "Beyond GDP" ideas that consider multiple dimensions of societal progress. The program would include changes to people and organizations, like new training and recruitment, as well as changes to the policymaking process, such as new analytical tools, budgeting approaches, and engagement with outside ideas. The next steps proposed are to create communities of practice to share best practices, identify needed changes in specific contexts, build support networks, communicate efforts, and take initial actions.

TCI 2016 Clusters strengthening macro-regional collaboration in the Baltic Se...

TCI Network

Capstone Report - BCG Final

Danielle Scism

This document provides an overview of the evolution of deliverology as a public sector management approach since its origins in the UK in 2001. It details the proliferation of delivery units globally that employ target-based performance management to improve critical public services. The report aims to map how deliverology has evolved, examine evidence of its effectiveness, and identify conditions facilitating adoption. It finds that while delivery units have adapted to different contexts, there is no conclusive evidence of their effectiveness but common sense and governance indicators imply value. Ongoing political support, specific priorities/targets, a performance culture, and supportive institutions best facilitate adoption.

Draft Low Emission Strategy - collaborative development. Andrew Taylor

STEP_scotland

MOVE Congress 2015: Innovative Fundraising

International Sport and Culture Association (ISCA)

Increase income from corporate, foundation and governmental donors: a training session from MOVE Congress 2015, opened with a workshop introducing to new fundraising trends and concepts, lead by Marvin Radford, ISCA Head of External Relations and Fundraising. Corporate fundraising by Kai Troll, ISCA Development Director: http://www.slideshare.net/NowWeMOVE/move-congress-2015-corporate-fundraising-innovations Erasmus+ updates by Michal Rynkowski, Directorate General for Education and Culture, European Commission: http://www.slideshare.net/NowWeMOVE/move-congress-2015-erasmus-sport-programme-updates More about MOVE Congress: http://www.movecongress.com/

Spending Reviews - Wojciech ZIELINSKI, OECD

OECD Governance

Evidencing the Benefits of Cluster Programmes by Madeline Smith, James Wilso...

TCI Network

The document discusses challenges around evaluating cluster programs and proposes a framework to assess their effects. It reviews literature finding clusters positively impact firm innovation, productivity, and regional growth. Case studies of cluster programs found effects on collaboration, economic performance, and regional competitiveness. Participants discussed populating the framework with indicators and applying it to a UK cluster initiative case. Key questions for cluster evaluation include what level and types of effects the policy aims to achieve. The working group seeks to advance understanding and practice of cluster evaluation.

General session modernization_2016

Светла Иванова

This document discusses the modernization of European official statistics and proposes the development of a toolbox to help national statistical systems modernize. It describes workshops that have been held on modernization and outlines principles from Paris21 for developing national strategies for statistical development. The document proposes that a toolbox could contain existing and new tools like guidelines, manuals, and applications to help countries design national strategies for modernizing their statistical systems. It provides examples of the types of tools that could be included and how the toolbox would be managed and updated. The goal of the toolbox is to provide resources to help statistical organizations implement modernization strategies and monitor their progress.

Progress of the Baltic SCOPE Monitoring and evaluation framework topic *

Pan Baltic Scope / Baltic SCOPE

The document discusses the development of an M&E (monitoring and evaluation) framework for evaluating transboundary collaboration in marine spatial planning. It provides an overview of the framework, which is based on a literature review and project interviews/observations. The framework focuses on cross-border MSP collaboration and provides guidance on evaluating such processes. It introduces a theory-based evaluation approach using a logic model to understand how MSP plans and policies produce change through activities, actions, outputs and outcomes. The document is being updated based on additional analysis and comments are requested by the end of August to further improve the framework.

Introduction to EVBRES

Caroline Blaine

EVBRES is a 4-year EU-funded research network that aims to encourage clinical researchers to use an evidence-based research (EBR) approach. The network was established to address issues found in meta-research indicating that clinical researchers do not systematically consider or place new study results in the context of existing evidence. Not doing so can lead to redundant and unnecessary studies, waste of resources, and patient exposure to suboptimal treatments. The EVBRES network, consisting of over 100 participants from 35 European countries, establishes four working groups to identify stakeholder implications, develop EBR training and resources, support systematic review efforts, and conduct meta-research on EBR implementation. The overall goal is to promote systematicity and transparency across

GPNOct2017-Budget-Transparency

Holly Richards

Budget transparency brings many benefits for citizens and for society, but putting it into practice can sometimes appear as a daunting task. This session will highlight the Budget Transparency Toolkit - developed by the OECD with the participation of the Global Initiative for Fiscal Transparency (GIFT) Network - and the practical steps that it provides for supporting openness, integrity and accountability in public financial management. Furthermore, it will shine a spotlight on the specific role that parliaments have in ensuring budget transparency, highlighting emerging good practices. It will be followed by a discussion on measuring budget transparency in parliaments.

Introduction to EVBRES

Caroline Blaine

EVBRES is a 4-year EU-funded research network that aims to encourage the use of evidence-based research approaches in clinical research. It was established in 2018 to promote collaboration and capacity building. The network's goal is to implement systematicity and transparency in all phases of research to ensure studies are valuable and avoid redundant research. It has over 160 participants from 37 countries working in four working groups on topics like identifying stakeholder implications and developing an evidence-based research course. The first workshop meeting was held in Bergen, Norway in February 2019.

Workshop proceedings of "Identifying contextualized indicators to measure SDGs"

4th Wheel Social Impact

Keeping social impact management at the centre, 4th Wheel Social Impact is committed to strengthening social programs in India by improving the way they are designed, implemented, monitored and evaluated. The organization believes the integration of data, technology and partnerships will enable the achievement of the Sustainable Development Goals (SDGs). The workshop focussed on Theory of Change, Indicator Development, SDG linkages of indicators. This workshop was supported by Swedish Institute.

URBACT Summer University 2013 - Labs - Attractive cities for young people - S...

URBACT

Targeted Topics Forum Introduction and Facilitation Notes | Blane Harvey

NAP Global Network

2nd International Conference on Inclusive Innovation and Innovative Managemen...

APGICO- Associação Portuguesa de Inovação e Criatividade

This document summarizes a study on using creative problem solving (CPS) to improve efficiency at a non-profit organization called the Parish of St. Christopher. Over 8 months, researchers facilitated CPS sessions to help identify issues like a lack of coordination between sectors. Action plans were developed to address problems through initiatives like defining common procedures, improving communication, and engaging employees. Follow-up sessions found accomplishments like developing an institutional video and manual. The CPS method helped create projects without external facilitators and led to changes like more participation and a customer-focused attitude.

Similar to Developing a More Systematic Approach to Evaluation Across Policy Instruments (20)

2014.11.28 - NAEC Group Meeting_Lamia Kamal-Chaoui

Introduction to ClimateXChange 14.01.22

Mobilising Evidence for Good Governance - OECD Presentation by Stéphane Jacob...

Designing Policy Experimentation

BRAINPOoL Final Conference: Integrated & Innovative Policy Making

TCI 2016 Clusters strengthening macro-regional collaboration in the Baltic Se...

Capstone Report - BCG Final

Draft Low Emission Strategy - collaborative development. Andrew Taylor

MOVE Congress 2015: Innovative Fundraising

Spending Reviews - Wojciech ZIELINSKI, OECD

Evidencing the Benefits of Cluster Programmes by Madeline Smith, James Wilso...

General session modernization_2016

Progress of the Baltic SCOPE Monitoring and evaluation framework topic *

Introduction to EVBRES

GPNOct2017-Budget-Transparency

Introduction to EVBRES

Workshop proceedings of "Identifying contextualized indicators to measure SDGs"

URBACT Summer University 2013 - Labs - Attractive cities for young people - S...

Targeted Topics Forum Introduction and Facilitation Notes | Blane Harvey

2nd International Conference on Inclusive Innovation and Innovative Managemen...

More from OECD Governance

Public Integrity Indicators Slides

OECD Governance

Summary of the OECD expert meeting: Construction Risk Management in Infrastru...

OECD Governance

Using AI led assurance to deliver projects on time and on budget - D. Amratia...

OECD Governance

ECI in Sweden - A. Kadefors, KTH Royal Institute of Technology, Stockholm (SE)

OECD Governance

This document discusses different construction project delivery and payment models. It begins by outlining common delivery models like design-bid-build and design-build. It then explains different payment methods that can be used like fixed price, unit prices, and cost-reimbursable. The document also discusses pricing strategies and how they relate to risk transfer between parties. It provides details on collaborative models like early contractor involvement and discusses selecting the optimal contract based on a client's project risks, desired influence, and market conditions.

Building Client Capability to Deliver Megaprojects - J. Denicol, professor at...

OECD Governance

Procurement strategy in major infrastructure: The AS-IS and STEPS - D. Makovš...

OECD Governance

Procurement of major infrastructure projects 2017-22 - B. Hasselgren, Senior ...

OECD Governance

ECI Dutch Experience - A. Chao, Partner, Bird&Bird & J. de Koning, Head of Co...

OECD Governance

ECI in Sweden - A. Kadefors, KTH Royal Institute of Technology, Stockholm

OECD Governance

EPEC's perception of market developments - E. Farquharson, Principal Adviser,...

OECD Governance

Geographical scope of the lines in Design and Build - B.Dupuis, Executive Dir...

OECD Governance

Executive Agency of the Dutch Ministry of Infrastructure and Water Management...

OECD Governance

Presentation of OECD Government at a Glance 2023

OECD Governance

Paris, 30 June, 2023 Presentation by Elsa Pilichowski, Director for Public Governance, OECD. The 2023 edition of Government at a Glance provides a comprehensive overview of public governance and public administration practices in OECD Member and partner countries. It includes indicators on trust in public institutions and satisfaction with public services, as well as evidence on good governance practices in areas such as the policy cycle, budgeting, procurement, infrastructure planning and delivery, regulatory governance, digital government and open government data. Finally, it provides information on what resources public institutions use and how they are managed, including public finances, public employment, and human resources management. Government at a Glance allows for cross-country comparisons and helps identify trends, best practices, and areas for improvement in the public sector. See: https://www.oecd.org/publication/government-at-a-glance/2023/

The Protection and Promotion of Civic Space: Strengthening Alignment with Int...

OECD Governance

Digital Government Review Türkiye - Assessment and Recommendations

OECD Governance

Buliding-Financial-Resilience-to climate-Impacts.pdf

OECD Governance

OECD Publication "Building Financial Resilience to Climate Impacts. A Framework for Governments to manage the risks of Losses and Damages. Governments are facing significant climate-related risks from the expected increase in frequency and intensity of cyclones, floods, fires, and other climate-related extreme events. The report Building Financial Resilience to Climate Impacts: A Framework for Governments to Manage the Risks of Losses and Damages provides a strategic framework to help governments, particularly those in emerging market and developing economies, strengthen their capacity to manage the financial implications of climate-related risks. Published in December 2022.

Session-7-OECD-Strengthening-climate.pptx

OECD Governance

Session-6-Building-Financial-Resilience.pptx

OECD Governance

Session-5-Assurance-on-sustainability-Peter-Welch.pptx

OECD Governance

Session-4-Sovereign-green-bonds-Fatos-Koc.pptx

OECD Governance

This document summarizes developments in sovereign green bond markets. It discusses approaches to incorporating environmental, social, and governance (ESG) factors into public debt management. Sovereign green bond issuance has grown significantly in both advanced and emerging economies since 2016. Green bonds make up the largest share of the labeled bond market. Major benefits of sovereign green bonds include their positive impact on creditworthiness and alignment with ESG policies. However, issuers also face challenges such as additional costs and complexity of the issuance process. Common leading practices emphasize transparency, collaboration, and commitment to reporting.

More from OECD Governance (20)