This document discusses evaluation criteria for assessing technology transfer projects at early stages. It presents criteria in areas like the innovation's originality, scientific relevance, market opportunities, intellectual property, and the project team. Evaluation may involve multiple stages from initial assessment to incubation. The approach aims to provide transparency and fairness in evaluating projects' commercialization potential and allocating resources.

1. Evaluation Criteria
FITT
(Fostering Interregional Exchange in ICT Technology Transfer)
www.FITT-for-Innovation.eu

2. Criteria for evaluation of transfer projects
 The described practice is designed to assist in the preliminary
assessment of research-grounded technology projects for their
commercialization potential in the realm of technology transfer.
 The process of assessing research projects is necessitated by the high
failure rate, and resulting high cost, of technologies either prior to
reaching the market or once in the market.
 The covered Evaluation Criteria are intended to provide guidance for
assessing an idea, a technology or a research project, at an early-stage
of technology transfer (thus prior to product development).
2 | 02.2010 Evaluation criteria

3. The evaluation process
 Project evaluation may take place at various stages
• Early-stage (proof-of-concept “maturation” towards technology transfer)
• Pre-incubation  Incubation
 Our focus is Early Stage Project Evaluation, which may appear
• In a continuous manner (or at regular intervals)
• Based on a CFP (Call For Proposal, typically once per year)
 Such early stage evaluation covers :
• Evaluation criteria
• A process for the application of these criteria, including the
structure/organization of the evaluation committee
 The current practice focuses on recommended Evaluation Criteria
3 | 02.2010 Evaluation criteria

4. An illustration of the evaluation process
Incubation entry
Preincubation
Early-stage eval eval
entry eval
n
n Incubatio
Pre-incubatio
Research Development Proof-of-concept Market
Licensing
Process :
Evaluation criteria
Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee
• Description of project to be evaluated
(document)
Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee Aaaaaaaaa
Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee
Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee
Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee
Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee Aaaaaaaaa
Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee
Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee Aaaaaaaaa
Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee
Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee
• Evaluation criteria
Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee
Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee
Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee
Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd EeeeeBbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee
Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee Aaaaaaaaa
Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee
Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee
Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee
Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee
• Jury (evaluation committee)
Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee
Aaaaaaaaa Bbbbbbbbbb Bbbbbbbbbb Cccccc Ddddd Eeeee
4 | 02.2010 Evaluation criteria

5. Coverage/definition of evaluation criteria
 Evaluation criteria should cover three main aspects of a project
• Technical aspects
• Market assessment
• Team considerations
 Evaluation criteria should be defined and published in advance in order to
allow the evaluated teams to adapt to the process
• Evaluation criteria will be used to establish the overall process, evaluation
documents and the selection committee
 Evaluation criteria may be used by the evaluation committee to
• Allocate funds/resources to selected projects
• Provide consultancy to the project team (for example, to coach the team
on aspects considered as “weak”)
5 | 02.2010 Evaluation criteria

6. Evaluation criteria
 Possible evaluation criteria
• Lots of possible evaluation methods/criteria are
mentioned in the literature
• Several possible groups of criteria :
Originality of the innovation Profile of the inventor Positive ROI/NPV calculations
Scientific return/opportunities for the laboratory Business opportunity Venture value
Project feasibility Market opportunities/threats Regulatory constraints
Potential users IP (protection issues, prior art) Business model
Scientific relevance of the project Lab support Financial return
Team aspects Realism of the announced plan Social & economical impact
Risk management Potential applications Production issues
6 | 02.2010 Evaluation criteria

7. Focus on first-stage evaluation criteria
 Most important criteria for first-stage evaluation
Positive ROI/NPV calculations
Originality of the innovation Profile of the inventor
Venture value
Scientific return/opportunities Business opportunity
for the laboratory
Regulatory constraints
Project feasibility Market opportunities/threats
Business model
Potential users IP (protection issues, prior art)
Financial return
Scientific relevance of the Lab support
project
Social & economical impact
Team aspects Realism of the announced plan
Production issues
Risk management Potential applications
Deemed premature for 1st stage evaluation
7 | 02.2010 Evaluation criteria

8. The DIGITEO example - Global positioning
 The OMTE checklist is used for maturation projects
8 | 02.2010 Evaluation criteria

9. Timing of the annual call for proposal
→ March : launch of call for proposal/deadline for
submissions
Long → April : preselection of 10 projects
selection
→ May: coaching by Digiteo’s marketing team
process
→ June/July : final application, oral présentation,
deliberation, final decision
→ September
Digiteo’s CFP (OMTE)
9 | 02.2010 Evaluation criteria

10. From proposal to selection
 ~ 10 proposals
 Preselection classification performed by Digiteo’s scientific
committee and marketing staff
 Coaching : work on the three components technology/marketing/IP 
submit presentation for the final selection
 Selection process :
• External experts (technology transfer specialists from : industry
cluster, incubator, Paris region, OSEO innovation fund, chamber
of commerce, etc.)
• Digiteo’s technology transfer committee
• Formal selection announced by Digiteo’s steering committee
 5 projects selected (budget constraints)
10 | 02.2010 Evaluation criteria

11. Selection steps
4. Final decision DIGITEO’s Steering Committee
Technology
Scientific Transfer
Committee Expert Panel
Committee
1. Scientific relevance 2. TT potential 3. Recommandations
Technical differentiation Value creation
11 | 02.2010 Evaluation criteria

12. Digiteo’s evaluation checklist
12 | 02.2010 Evaluation criteria

13. DIGITEO – Method/criteria
 Evaluation method :
• Evaluation of the applications according to the 12 criteria
• Individual evaluators may apply assessment scores from 1 to 3 (3 being
the highest)
 Evaluation criteria used for the OMTE call for projects
 « Product/technology » aspects
 Originality/uniqueness and scientific relevance, project feasibility and opportunities
created for the laboratory.
 « Market » aspects
 Ongoing research contracts and IP related to the project, first applications and
users considered.
 « Team » aspects
 Support of the laboratories in the process, project manager identified to manage
the project, realism of the planning proposed and evaluation of the risks by the
applicants.
13 | 02.2010 Evaluation criteria

14. DIGITEO – « Product » criteria
 1. Originality of the innovation
 Originality/uniqueness in comparison with state-of-the-art ?
 Definition of the future « product » ?
 Positioning compared to competitors ?
 2. Scientific relevance of the project
 Compatibility with the research themes covered by Digiteo ?
 Scientific excellence in the field?
 Degree of scientific maturation ( is the technology close to a « product ») ?
 3. Project Feasibility
 Technical feasibility of the project?
 Feasibility of the planning, with regard to a transfer?
 Description of the transfer model envisaged (transfer to an industrial partner / creation of start-
up) ?
 4. Scientific opportunities created for the laboratory
 Consequences of the development on the scientific activities of the lab ?
 Future impact of the project on the lab’s strategy ?
 Impact on the external communications of the lab?
14 | 02.2010 Evaluation criteria

15. DIGITEO – « Market » criteria
 5. Ongoing research contracts
 Ongoing contracts with industrial partners?
 Other contracts/ scientific activities ?
 Since when? For how long?
 6. Intellectual property (patents, know-how)
 Background knowledge of the teams involved ?
 Protection envisaged (foreground) for the new knowledge and software derivating from it;
 Is an IP analysis requested by the teams (analysis of the prior art, patent landscape and
« freedom to operate ») ?
 7. First potential applications
 Types/examples of applications ?
 Value Proposition (solution to which problem) ?
 Applications realised by which kind of company (software company, service provider) ?
 8. First potential users
 Existing and potential actors/ partners to target for the transfer?
 Example of end-user for the integrated solution ?
 Draft definition of the targeted market (size, segmentation, competitors) ?
15 | 02.2010 Evaluation criteria

16. DIGITEO – « Team » criteria
 9. Support of the laboratories
 Support of the laboratories involved ?
 Balance between the teams involved (complementarity, synergy) ?
 Common commitment to a real transfer ?
 10. Project manager in charge
 Profile of the project manager and implication in the project ?
 Capability of managing all aspects of the project, keeping with the transfer objective?
 Motivation to handle the 3 aspects : technical, IP, marketing ?
 11. Realism of the planning
Realism of the planning with regards to the 3 aspects:
 Technical
 IP
 Marketing
 12. Evaluation/ consideration of the risks
Identification and management of the risks :
 Technical
 IP
 Marketing
16 | 02.2010 Evaluation criteria

17. DIGITEO - Assessment
 Useful tool to be used as a checklist throughout the evaluation process
 The final selection has to include the assessment of the presentation made
in front of the jury. Grade given by the jury is based for 50% on written
application and 50% on the oral presentation.
 The jury should include a majority of external experts
 Final selection : classification/ranking of the presented projects (top 5 
selected)
 Some « Digiteo specifics » not to be considered for a generic checklist
17 | 02.2010 Evaluation criteria

18. Pros & Cons
PROs CONs
• This practice attempts to formalize • Only a selected number of
methods that are already in use
criteria are highlighted
(most of the time on an ad hoc
basis) • Some criteria may need to
be further developed
• The methodology and associated
tools (call for proposal, criteria,
etc.) are readily available and can
be adapted to each individual case
18 | 02.2010 Evaluation criteria

19. Why ?
• Methodology developed by Digiteo in order to manage the incoming flow of
technology transfer proposals
• Need for a consistent set of criteria for all steps of evaluation process,
communicated transparently to all involved partners : project teams, internal
Digiteo evaluators, “technology transfer coaches” and external experts
• Without this methodology, involved parties would get the impression that
projects might be evaluated/selected based on obscure reasons. This would
leave the doors open for debate, accusations for “unfair competition” and
backstage lobbying
19 | 02.2010 Evaluation criteria

20. Why/impact ?
 Impact : Why is it a good practice?
• The Digiteo community judges this approach transparent, fair and clearly
communicated
• We may recommend this approach based on our own experience
20 | 02.2010 Evaluation criteria

21. Outcome
 What happened after the implementation :
• The approach turned out as expected
 Final selection (with external experts) is based on relative ranking among the
presented projects
 The scoring system is only used for individual evaluation purposes
• However, you also have to manage those projects that were not finally
selected
 Debrief the teams that were not selected
 Clearly communicate the reasons for not being selected
 Focus on things to be improved (and how to improve them)
 Encourage them to apply again with an enhanced proposal
21 | 02.2010 Evaluation criteria

22. Outcome – plans for the future
 Plans for the future?
• The approach should be further developed/detailed :
 Definition of terms
 Explanation on how to apply each of the listed criteria (with some
examples)
22 | 02.2010 Evaluation criteria

23. Suggested Readings
 Link to code book
• Technology development/maturation
• Proof of concept
• (Opportunity) assessment
 Link to related websites
OMTE call for proposal and projects selected during previous editions:
http://www.digiteo.fr/Digiteo_OMTE
23 | 02.2010 Evaluation criteria