New scientific publications means new scientific evaluation. Whose the penalty?

1. Evaluation of Science 2.0
New generation of publications means new generation
of evaluation of publications
Daniele Tosi

2. Framework
• Universities and research centers need some
form of evaluation of researchers
• Number and impact factor of publications
represent the common way to evaluate the
quality of researchers
• But how to extend the evaluations to 2.0
publications?
• Redefine the criteria for publications and review

3. The actual situation

4. Evaluation
• Number of publications
• Impact factor
• Citation index
Score ~ # Publ. x IF x CI

5. Reviewing
• Double blind review
• Random reviewers,
picked up manually
• On/off review, score
transparent for readers

6. Problem 1
➡ How to compare different researchers?
VS
Publications + Patents + Prototypes Publications

7. Actual situation
• Standard publications dominate the evaluation
of performances for researchers
• Unfair situation: theoretical and experiments-
only researchers achieve better score
• Technology transfer does not return enough
publications, leading to an inferior score
➡ Need to evangelize a new scoring system for
scientific publications

8. Problem 2
➡ How to integrate Science 2.0 publications?
VS
Publications Publications
Science 2.0

9. Actual situation
• Open Science and Science 2.0 publications
are not accounted
• General-purpose publications and reports are
not included
• Penalty for scientists who switch to Science
2.0, advantage for standard publications
➡ Need to evangelize the inclusion of Science
2.0 publications into researchers’ records

10. <
Publications Publications
Science 2.0
<
Publications + Patents + Prototypes Publications

11. 1 =
Publications Publications
Science 2.0
=
Publications + Patents + Prototypes Publications

12. Step 1
• Struggle for introducing a new system for
evaluating researchers
• To set guidelines for researcher and let
them open to Science 2.0 without penalty
• To be accepted as a standard from research
institutions

13. 2 >
Publications Publications
Science 2.0
>
Publications + Patents + Prototypes Publications

14. Step 2
• Publications 2.0 are designed to achieve a
wider read, more extended reviews and a
network connection to interested people
• They should therefore receive a higher score
• A researcher is willing to switch from
traditional to 2.0 publications

15. Mission
To set 2.0 publication as the new publication standard

16. Objectives
• Generate awareness of 2.0 publications
among researchers
• Provide an evaluation tool that ranks 2.0
publications and authors
• Provide a legal and official process to
embed 2.0 publications in research projects

17. Strategy
✓Generate awareness of 2.0 publications
among researchers
• Viral growth from SocNet users up to
conservative scientists
• Identify decision makers for publications and
evangelize them
• Advertise equality between 1.0 and 2.0
publications

18. Strategy
✓Provide an evaluation tool that ranks 2.0
publications and authors
• Evaluate and account the increase number of
document reads
• Recruit early-adopter scientists to test and
arrange a new evaluation system
• Ensure no penalty for 2.0 users

19. Strategy
✓Provide a legal and official process to
embed 2.0 publications in research projects
• Identify decision makers among official research
projects commissioner and evaluators
• Broadcast 2.0 publications as a recognized output
of research projects
• Embed scientific SocNets with shortcuts for
research project management

20. Framework
• This draft is NOT a real market analysis
• Comments and corrections are welcome

21. Thank you
Stay tuned for the next episode