This document discusses measuring the impact of Rural Development Programmes (RDPs) at the EU level. It outlines the basics of assessing RDP impacts, including using the Common Monitoring and Evaluation Framework (CMEF) and identifying intervention logic and additional indicators. It also discusses the challenges of evaluation, like determining causality and separating programme effects from other influences. Finally, it examines the specific issues around measuring the seven fields of impacts from RDPs, including economic growth, employment, biodiversity, and climate change. Methodological challenges are presented for quantifying each impact area.

1. Measuring the Impact of the RDP Issues being addressed at an EU level with regards to measuring the impact of the Rural Development programmes B. Schuh 1 www.nrn.ie

2. Content Assessment of impacts in the RDPs – the basics Process of assessing RDP impacts Evaluation architecture/ challenges to overcome The seven fields of impacts (econ. growth, employment creation, productivity, biodiversity, HNV, water quality, climate change) www.nrn.ie 2

3. Assessment of impacts in the RDPs – the basics: CMEF – the „bible“ Intervention logic Additional programme specific indicators Evaluation questions www.nrn.ie 3 reference for impact supporting and counterproductive trends

4. Process of assessing RDP impacts: Gauging the evidence of change Identifying the drivers of change Understanding change and concluding on future interventions www.nrn.ie 4

5. Evaluation architecture/ challenges to overcome: Factors determinig evaluation architecture: Methodological challenges to overcome: Dealing with uncertainties Reducing complexity through a consistent approach Constraints in utilization of the evaluation results – the evaluation and policy cycle The counterfactual assessment of impacts  quasi-experimental design, non-experimental design; DiD method Taking into account and cross-relating impacts at micro and macro level  e.g. econometric modeling, CGE models, system dynamics modeling Netting out the programme effects by reducing deadweight, leverage, displacement, substitution and multiplier effects data collection and processing  qualitative & quantitative data, FADN & Co Bridging the gap between measuring impact indicators and providing answers on programme impacts  qualitative methods as add-ons – interviews, CS www.nrn.ie 5

7. Limitations concerning measuring labour productivity  Competitive Performance, Revealed Comparative Advantage, Growth Competitiveness Indicator, Domestic Resource Costwww.nrn.ie 6

8. The environmental impact indicators – special issues: Data availability Systemic borders Environmental impacts do evolve also from those measures, which do not deem that such impacts will occur Conception of the environment within the evaluation (ecosystem functionsvs. ecosystem services) Difficult to depict the full range of rather vast, complex fields of environmental phenomena like “climate change” or “Biodiversity loss” “evaluation” vs. assessment – aggregation methods not easily applicable Cumulative impacts – crossing effects between environmental impacts www.nrn.ie 7

9. The environmental impact indicators: Biodiversity Measured by Farmland Bird Index (FBI) – bottom-up aggregation of micro level observations Crucial issue of regional/ national specifics, bottom-up assessment, additional information (‘control’ other influences), more than birds High Nature Value farming/ forestry Measured through: Land cover characteristics, especially farmland with a high proportion of semi-natural vegetation and in some cases a diversity of land cover types. Farming practices, especially a low use of inputs (including live­stock density) and specific practices such as shepherding, late hay-cutting, orchard grazing and arable fallowing. Strong dependence on baseline data  comparison of baseline conditions, main challenge indicator as „work in progress“ www.nrn.ie 8

10. The environmental impact indicators 2: Waterquality Measured as changes in gross nutrient balance (GNB), should be interpreted as a potential risk indicator for water quality only; assessment by bottom up approach – aggregation as methodological challenge (modelling – e.g. RAUMIS) Many uncertainties remain  different land cover, land use and farming types & atmospheric N fixation and deposition - measuring water quality in agricultural catchment, net nitrogen balance as additional indicators Climate change Measured as net greenhouse gas emissions reduction and production of renewable energy Limitations Additionality/net effects, Displacement of energy & production, Boundary issues, issue of the temporal attribution of longer-term impacts to the policy period of the spending www.nrn.ie 9