Interactive Powerpoint_How to Master effective communication
6_how_to_write_an_ier_0.pdf
1. HOW TO WRITE AN INDIVIDUAL
EVALUATION REPORT (IER)
MARKUS MÜLLER
EXPERT BRIEFING 2020
2. • Open immediately your allocated proposals and screen for consortium
members and persons involved (part A and part B.4)
• Check if any COI (cf. definitions in your contract)
• If no COI please accept immediately
• If you have (a doubt about) a COI please declare COI via SEP explaining
the reason
• REA will take a decision (clearance or un- / new assignment)
Before starting – check for COI
3. • Check the call / (sub-)topic and the ToA of your proposals (p.1/2 of part A)
• Read carefully the (sub-)topic description of the Work Programme :
• Title
• Specific Challenge : topic background information (context, policy landscape, etc.)
• Scope: topic description and requirements
• Expected Impact: the proposal should demonstrate how to achieve them
• Other information: ToA, indicative budget, TRL...
• Check the web-briefing material for further topic information
What is expected from the applicants?
4. • PART A: abstract, administrative data, budget table, ethics issues table
• PART B.1-3 - MOST RELEVANT PART FOR IER:
• 1. Excellence -> input for C1 "Excellence"
• 2. Impact -> input for C2 "Impact"
• 3. Implementation -> input for C3 "Implementation"
• Part B.4-6
• 4. Consortium Members -> input for Operational Capacity
• 5. Ethics -> input for Ethics Screening
• 6. Security -> input for Security Screening
How is the proposal structured?
5. • Three evaluation criteria :
• − Excellence (relevant to the description of the call or topic)
• − Impact
• − Quality and Efficiency of the Implementation
• The criteria are adapted to each ToA, as specified in the WP
How to evaluate?
6. Evaluation Criteria
To the extent that the proposed work corresponds to the topic
description in the work programme:
• Clarity and pertinence of the objectives;
• Soundness of the concept, and credibility of the proposed
methodology;
• ToA specific sub-criteria (e.g. IA/RIA: beyond SoA, innovation
potential,...)
Excellence
• The extent to which the outputs of the project would contribute
to each of the expected impacts mentioned in the work
programme under the relevant topic;
• ToA specific sub-criteria (e.g. IA/RIA: exploitation, dissemination
of results,...)
Impact
• Quality and effectiveness of the work plan, ...
• Appropriateness of the management structures and procedures...
• Complementarity of the participants ...
• Appropriateness of the allocation of tasks, ...
Implement
.
7. • Read proposal and assess against the evaluation criteria
• Without discussing it with anybody else
• Factual assessment – no personal interpretation
• Check if the proposal is relevant to the (sub-)topic
• Enter your comments (both positive and negative) on each criterion/sub-
criterion in the SEP tool
• Explain shortcomings (e.g. giving concrete examples)
• No recommendations (e.g. on work plan, resources)
• Score all evaluation criteria in consistency with comments
• Save – and if finalised – submit (latest by 30/09)
How to create the IER?
8. • The proposal fails to address the criterion or cannot be assessed due to missing
or incomplete information.
• Poor. The criterion is inadequately addressed, or there are serious inherent
weaknesses.
• Fair. The proposal broadly addresses the criterion, but there are significant
weaknesses.
• Good. The proposal addresses the criterion well, but a number of shortcomings
are present.
• Very Good. The proposal addresses the criterion very well, but a small number of
shortcomings are present.
• Excellent. The proposal successfully addresses all relevant aspects of the criterion.
Any shortcomings are minor.
How to set the scores?
0
1
2
3
4
5
9. • Scoring from 0 to 5 with 0.5 steps
• Threshold for individual criteria = 3
Proposals with significant weaknesses have to be scored below threshold
• Overall threshold (sum of ind. scores) = 10
• Ranking based on the total score
• Exception for IAs : weighting factor of 1.5 on IMPACT
Proposal Scoring and Thresholds
10. • A proposal partially relevant to the topic -> to be reflected in EXCELLENCE
• Approach generic and not adapted to the topic -> to be reflected in
EXCELLENCE
• Insufficient exploitation plans -> to be reflected in IMPACT
• Consortium lacking expertise in core areas -> to be reflected in
IMPLEMENTATION
• Quality / role of practitioners insufficient (if required) -> to be reflected in
IMPLEMENTATION
• No double penalty: if the concept is not appropriate but the workplan is
consistent with it -> Penalisation only in EXCELLENCE
Always use the right criterion!
11. • Do not summarize the proposal, evaluate it
• Assess what is written in the proposal, no assumptions
• Double check all factual statements
• First comments, then scores
• Informative comments (reason, example) on negative and positive aspects
• Right length of comments : 1-2 sentences per sub-criterion, ~200 [1200]
words [characters] per criterion
• Use attributes to classify shortcomings (e.g. minor, moderate or significant)
• No double penalty
Final IER & CR Writing Tips…
12. • Secure Societies Workprogramme 2018-2020
• General Annexes (e.g. ToA, Evaluation Criteria, TRL)
• Expert Standard Briefing
• Standard Proposal Template RIA/IA
• Standard Proposal Template CSA
• Standard Proposal Template PCP
• Need Help? REA-SECURITY-EXPERTS@ec.europa.eu
Further Reading...
14. • The objectives of ACRONYM (addressing sub-topic N of topic M) are very clear and
explained in good detail. They are pertinent to the chosen topic (and sub-topic), and are
supported by measurable success criteria.
• Overall, the concept is sound and elaborated with very good precision. However, less
detail has been provided for the use of X and Y. In addition, the inclusion of technology L in
the context of the proposal has not been justified. This is a shortcoming.
• The methodology is very credible. The path to delivering the expected results is defined
accurately, and very good validation and evaluation procedures have been defined.
• An excellent analysis of the state of the art has been provided. There is high potential for the
work to go beyond the state of the art, in particular in the domains of X, Y. On the other side,
the possible advances in the field of Z are less well explained, which is a minor
shortcoming.
• Interdisciplinary approaches (and gender dimension) have been considered very well,
including the fields of A, B, and D, which is a strong aspect of the proposal. There is good
use of stakeholder knowledge, in particular, in the area of C.
Dummy Evaluation Report – Crit. 1 (EXC)
SCORE: 4
15. • The proposal does not give sufficient evidence that, if successful, it will contribute to all
expected impacts of the work programme. In particular, the expected impacts of A, B and C
have not been sufficiently addressed. This is a significant weakness [-> below threshold !].
• Nevertheless, the output of the proposal may contribute to create new market opportunities and
enhance innovation capacity in the sector A.
• Effective individual exploitation and commercialisation plans have been developed, however
the overall exploitation plan is not sufficiently elaborated.
• An effective dissemination plan has been presented with clearly defined target audiences and
channels. Activities include X, Y, Z and are inked to key performance parameters, which is a
very positive aspect.
• A communication plan has been outlined, however appropriate target audiences and relevant
channels have not been sufficiently identified, which is a shortcoming.
• IPR and knowledge management have only been briefly addressed.
Dummy Evaluation Report – Crit. 2 (IMP)
SCORE: 2.5
16. • A coherent and effective work plan has been elaborated with well defined dependencies
between tasks and work packages. The tasks have been mapped very well to the objectives
and appropriate milestones have been set. The resources have been allocated
appropriately.
• The described management structures are generic and are not adapted to a project of this
size. In particular, they lack A and B and mechanisms for C and D are not appropriately
presented. This is a significant shortcoming.
• Risk management has been addressed well in general. A range of relevant risks has been
identified and analysed, and appropriate mitigation measures have been described.
However, specific risks related to the development of X,Y and Z, critical for the success of the
proposal, have not been sufficiently addressed. Innovation management has been
addressed appropriately.
• The participants show good complementarity and the consortium has the expertise to carry
out the work proposed. Tasks have been allocated to partners in line with their expertise. All
participants have a valid role and adequate resources.
Dummy Evaluation Report – Crit. 3 (IMPL)
SCORE: 3
18. Evaluation criteria (CSA)
Excellence
(to the extent that the proposal addresses
the WP topic)
Impact
Quality and efficiency of the
implementation
• Clarity and pertinence of the
objectives;
• Soundness of the concept, and
credibility of the proposed
methodology;
• Quality of the proposed
coordination and/or support
measures.
• The extent to which the outputs of the project
would contribute to each of the expected
impacts mentioned in the work programme
under the relevant topic;
• Quality of the proposed measures to:
• Exploit and disseminate the project
results (including management of IPR), and
to manage research data where relevant.
• Communicate the project activities to
different target audiences
• Quality and effectiveness of the work
plan, including extent to which the
resources assigned to work packages
are in line with their objectives and
deliverables;
• Appropriateness of the management
structures and procedures, including
risk and innovation management;
• Complementarity of the participants and
extent to which the consortium as whole
brings together the necessary expertise;
• Appropriateness of the allocation of
tasks, ensuring that all participants have
a valid role and adequate resources in
the project to fulfil that role.
19. Evaluation criteria (PCP)
Excellence
(to the extent that the proposal addresses
the WP topic)
Impact
Quality and efficiency of the
implementation
• Clarity and pertinence of the
objectives;
• Soundness of the concept, and
credibility of the proposed
methodology;
• Progress beyond the state of the art
in terms of the degree of innovation
needed to satisfy the procurement
need.
• The extent to which the outputs of the project
would contribute to each of the expected
impacts mentioned in the work programme
under the relevant topic;
• Strengthening the competitiveness and
growth of companies by developing
innovations meeting the needs of European
and global procurement markets
• Quality of the proposed measures to :
• Exploit and disseminate the project results
(including management of IPR) and to
manage research data where relevant.
• Communicate the project activities to
different target audiences
• More forward-looking concerted procurement
approaches that reduce fragmentation of
demand for innovative solutions
• Quality and effectiveness of the work
plan, including extent to which the
resources assigned to work packages
are in line with their objectives and
deliverables;
• Appropriateness of the management
structures and procedures, including
risk and innovation management;
• Complementarity of the participants and
extent to which the consortium as whole
brings together the necessary expertise;
• Appropriateness of the allocation of
tasks, ensuring that all participants have
a valid role and adequate resources in
the project to fulfil that role.
