Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Evaluation of business start-up policies by Jonathan Potter, Senior Economist, OECD LEED Programme


Published on

Presentation from the capacity building seminar “Financing business start-up by under-represented groups”, 27-29 June 2012, Trento – Italy; organised by the Local Economic and Employment Development (LEED) Programme and its Trento Centre at the OECD in collaboration with the Directorate-General Employment, Social Affairs and Inclusion of the European Commission. See

Evaluation of business start-up policies by Jonathan Potter, Senior Economist, OECD LEED Programme

  1. 1. OECD-European Commission Capacity-building seminar ‘Financing Business Start-up by Disadvantaged and Under-Represented Groups’, Trento, 27-29 June 2012 Evaluation of Business Start-up PoliciesJonathan PotterSenior EconomistOECD Centre for SMEs, Entrepreneurship & Local Development (CFE)
  2. 2. Presentation structure1. Principles of evaluation2. Evaluation approaches – logical framework – indicators – qualitative and quantitative evaluation3. Conclusions
  3. 3. 1.1 What is evaluation?Papaconstantinou and Polt (1997):“Evaluation refers to a process that seeks todetermine as systematically and objectively aspossible the relevance, efficiency and effectivenessof an activity in terms of its objectives”Purpose is to provide decision makers with thebest information to answer crucial questions . . .
  4. 4. 1.2 Evaluation questions• Is the activity relevant?• Is it achieving its objectives?• Is it cost effective and proportional to what it seeks to achieve?• Should the activity be continued or terminated?• If continued, how can it be improved?
  5. 5. 1.3 Assessment criteriaCriterion ExplanationAppropriateness (relevance) Extent to which the programme objectives are relevant to the needs of the economy, and address a market failure or social inequityEffectiveness Extent to which the activity is achieving the programme objectives, net of what would have happened in the absence of the initiativeEfficiency Extent to which costs are minimised in achieving the programme’s objectivesSustainability To what extent can benefits be expected to last after the intervention is completedProcess management Extent to which the programme is well managed
  6. 6. 1.4 The counterfactualOutputs, e.g. Assistance number of optionbusinesses started by Do nothing the target option group Additionality Non- additionality Baseline Project Time What would happen without the programme Our interest is in the difference between the assisted output and the non-assisted output
  7. 7. 1.5 Evaluation in the policy cycle Definition of rationale for policy Appraisal of Feedback policy options Evaluation of Implementation results of programmes (benefits – and projects costs) Monitoring of progress
  8. 8. 2.1 The logical framework – concept• Analytical planning tool to clarify causal relations in a policy intervention• Operational chain of activities, immediate results and subsequent impacts• Can be used at programme level and at project level• Can be used for strategy design and for monitoring and evaluation of projects
  9. 9. 2.2 Logic framework – microfinance and business support project Resources Activities Outputs Outcomes Impacts /inputs E.g. public E.g. numbers E.g. number of E.g. reduction E.g. reduction investment in of loans new firms in number of in guarantees of provided; created by participants unemployment non- hours of unemployed who are in the target performing mentoring people; unemployed; group; loans; fund provided; new increased increase in increased administration contacts with skills; number of entrepreneur- cost; public banks and improved starts; increase ship rate cost of business business in survival of business support proposals; new starts; support agencies relationships increased with banks lending to target group
  10. 10. 2.3 EQUAL Business Creation Theme Logic Model
  11. 11. 2.4 IndicatorsType ExamplesInputs Funds allocated Total budget Funds allocated as % of budgetActivities Number of people contacted Number of people having received services Number of proposals for the creation of new businesses Number of advisors % of beneficiaries belonging to target groupsOutputs/outcomes Attitudes to entrepreneurship Business start rate Survival rate of enterprises Reduction in unemployment rateOther Satisfaction of beneficiaries with the programme Satisfaction of managers with the programme
  12. 12. 2.5 Methods for evaluationQualitative approach• Based on stakeholder and beneficiary interviews• Deep questioning on processes and changes in decisions and behaviour• Strength is that it explores processes and engages participants in policy learningQuantitative approach• Compares performance of assisted and matched firms and individuals• Accounts for selection bias• Strength is that is gives robust impact estimates 12
  13. 13. 3. Conclusions• Evaluation is critical to policy success but is not yet properly integrated in policy process• At this stage of the EU policy process the emphasis is on ex ante evaluation, implying: – development of logical frameworks – Identification of indicators to measure, targets to reach and creation of a monitoring programme• Once activities have been implemented, will need to consider the nature of evaluations to commission; may need some data collection now