Junior Project Management course                                               13-15 April 2011                           ...
Selection of the Experts – Who? Who are the Evaluators? Balance between  •   Academic and industrial expertise  •   Geogra...
The Evaluation Process                                                                Proposal  Administrative check by th...
On-site vs Remote Evaluation                            On-site                      RemoteConfidentiality             App...
Briefing for the Evaluators Introduction to the Work Programme objectives Introduction to the Guide for Applicants and for...
Individual Evaluation  The evaluators work independently   •   No communication is allowed   •   Computer is provided if n...
Individual Evaluation  2-4 proposals per day  Max 2 hours per proposal  First impressions      Title      Summary      Obj...
Evaluation Criteria  „Within scope” assessment           Detailed reading  First Reading = First impression    Read specif...
Evaluation Criteria - CP                                                   From the Guide for Applicants:                 ...
Individual Evaluation ReportIn FP7: a 4-5 page document completed by each expert    I. Evaluation summary    II. Recommend...
Marking System                                       Scores 0-5 points awarded for each main                              ...
Individual Evaluation - Results Total score, overall comments  •   for the whole project (strengths and weaknesses, overal...
The Consensus Meeting Experts meet together to come to a consensus Minutes of the meeting are recorded by the „Rapporteur”...
The Consensus Meeting  Roundtable discussion on each criteria:   •   The higher/lower marks of an expert have to be explai...
If There Is No Consensus: 1.   New evaluator(s) can be appointed:         Individual evaluation 2.   Extended Consensus Me...
Goals of the Panel Meeting  To assess and compare the consensus reports of the different sub-panels:     Overall quality u...
Final Decision, Negotiation  Commission prepares the following lists:   •   Ranking list of the eligible proposals (over t...
Thank you for your attention!                               info@europamedia.org                          © Europa Media  ...
Upcoming SlideShare
Loading in …5
×

Evaluation of project proposals in FP7

3,074 views
2,848 views

Published on

Published in: Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,074
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
65
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Evaluation of project proposals in FP7

  1. 1. Junior Project Management course 13-15 April 2011 Budapest, HungaryEvaluation of Project Proposals Ms. Gabriella Lovasz Assistant Managing Director Europa Media
  2. 2. Selection of the Experts – Who? Who are the Evaluators? Balance between • Academic and industrial expertise • Geographical representation of Europe • Gender Expertise in the following fields • Research in the given scientific area • Industrial/commercial applications • International project design and implementation • Administrative, policy-oriented or other knowledge as required by the Call Register as an ”expert”: https://cordis.europa. eu/emmfp7/
  3. 3. The Evaluation Process Proposal Administrative check by the EC Evaluation by the experts: Eligibility Briefing for the experts Pre-check by Commission Individual evaluation Consensus evaluation Final ranking Individual Individual Individual evaluation evaluation evaluation Contract negotiations start Consensus meeting Scores, ESR Tresholds Ethical review if needed Panel review with Hearing (optional – IP/NoE) Applicants informed of results ESR Commission Ranking
  4. 4. On-site vs Remote Evaluation On-site RemoteConfidentiality Applies Applies Controlled No controlPayment 450 EUR/day 450 EUR/dayTime available for Limited Open (but maximised byindividual evaluation payment)Access to extra resources No access Unlimited(Internet)Stay in Brussels Fully Only for the consensus and panel meetingBenefit Controlled confidentiality Background check can be Better focus done – better understanding
  5. 5. Briefing for the Evaluators Introduction to the Work Programme objectives Introduction to the Guide for Applicants and formal criteria Overview on the general evaluation principles, and the details of the evaluation process Overview on the scores and the definitions of „Excellent” and „Poor”, etc. Introduction to any special EC priorities/issues Clarify the possible “conflict of interest” Discussions Evaluation principles: Excellence Transparency Confidentiality Fairness & Impartiality Ethical and Security considerations Efficiency and Speed
  6. 6. Individual Evaluation The evaluators work independently • No communication is allowed • Computer is provided if needed (personal laptops are not allowed) • The internet might be used (depending on the programme) Each proposal is evaluated by 3-5 evaluators Each evaluator fills out the „Individual Assessment Report” (IAR) forms
  7. 7. Individual Evaluation 2-4 proposals per day Max 2 hours per proposal First impressions Title Summary Objectives Partnership Consistency & formatting Length First 15-30 minutes of the evaluation are crucial
  8. 8. Evaluation Criteria „Within scope” assessment Detailed reading First Reading = First impression Read specific chapter for the = ABSTRACT, OBJECTIVES, specific criterion (find it) Partner list Make rough notes Decide score Complete IAR Technical issues Management Finances/Resources Impact
  9. 9. Evaluation Criteria - CP From the Guide for Applicants: Evaluation criteria applicable to Coordination and support actions (Supporting)S/T QUALITY IMPLEMENTATION IMPACT“Scientific and/or technological “Quality and efficiency of the “Potential impact through the excellence implementation and the development, dissemination(relevant to the topics addressed by management” and use of project results” the call)”• Soundness of concept, and quality • Appropriateness of the • Contribution, at the European of objectives management structure and [and/or international] level, to the• Progress beyond the state-of-the-art procedures expected impacts listed in the work• Quality and effectiveness of the • Quality and relevant experience of programme under the relevant support action mechanisms, and the individual participants topic/activity associated work plan • Quality of the consortium as a • Appropriateness of measures for• Appropriate comparative perspective whole (including dissemination and/or exploitation of in relation to the proposed research complementarities, balance) project results, and management of • Appropriateness of the allocation intellectual property. and justification of the resources to be committed (staff, equipment, …)
  10. 10. Individual Evaluation ReportIn FP7: a 4-5 page document completed by each expert I. Evaluation summary II. Recommendation Criteria: S/T QUALITY “Scientific and/or technological excellence (relevant to the topics addressed by the call)” IMPLEMENTATION “Quality and efficiency of the implementation and the management” IMPACT “Potential impact through development, dissemination and use of project results” Criterion Threshold Max Scientific quality 3 5 Implementation 3 5 Impact 3 5 Total 10 15
  11. 11. Marking System Scores 0-5 points awarded for each main 0 Fails to address the criterion or cannot be and sub-criteria (0-Fail/Poor, 5- judged Excellent) 1 – 1,5 Poor. Half points can be given! There are serious weaknesses Detailed explanation required 2 – 2,5 Fair. • should be given by the Broadly addresses the criterion, but …. evaluators, not only simple scores Thresholds for each main criterion 3 – 3,5 Good. Addresses well, but improvements are Threshold for the overall mark necessary This can be more than the sum of 4 – 4,5 Very good. the thresholds of the sub-criteria! Certain improvements are still possible 5 Excellent. Any shortcomings are minor
  12. 12. Individual Evaluation - Results Total score, overall comments • for the whole project (strengths and weaknesses, overall recommendations) Qualification • recommended or not recommended for funding – each threshold must be reached! The amount of the requested funding is also evaluated: realistic or too much, should be reduced, adequate to the work plan, etc. When all experts have finished, the proposal is ready for Consensus evaluation
  13. 13. The Consensus Meeting Experts meet together to come to a consensus Minutes of the meeting are recorded by the „Rapporteur” The Commission representative is the „Moderator” Experts have a few minutes to read and understand each others comments Preliminary discussions followed by detailed assessment of all criteria
  14. 14. The Consensus Meeting Roundtable discussion on each criteria: • The higher/lower marks of an expert have to be explained and justified • Discussion. Arguments - counterarguments Consensus must be obtained for • Scores for each main criterion (the consensus score is NOT the mathematical average of the individual evaluators score) • Text of explanation and justification The scores of all criteria + the corresponding text agreed upon: IT IS A CONSENSUS Consensus Report – drafted by the “Rapporteur”
  15. 15. If There Is No Consensus: 1. New evaluator(s) can be appointed: Individual evaluation 2. Extended Consensus Meeting with the involvement of the new evaluators 3. If there is no consensus afterwards: Decision by majority of votes
  16. 16. Goals of the Panel Meeting To assess and compare the consensus reports of the different sub-panels: Overall quality under the different topics Number of proposals evaluated Special attention to Proposals that scored very high but failed on one criteria Proposals with equal score near the funding thresholds Final ranking according to scores: Established for each topic Established for the programme as a whole Proposals with equal scores will be ranked: Objectives, Relevance, Impact Horizontal issues
  17. 17. Final Decision, Negotiation Commission prepares the following lists: • Ranking list of the eligible proposals (over the thresholds) • List of those proposals for which the negotiation process can start – taking into account the available total fund for the given panel • Reserve list • in case of withdrawal – if the negotiation is not successful from any side! • List of rejected proposals
  18. 18. Thank you for your attention! info@europamedia.org © Europa Media It is not allowed to use or distribute the content and design of the presentation without prior agreement.

×