Presentation on quality criteria for responsible research and innovation (RRI) initiatives and the release of a catalogue of good RRI practices by Athena Institute at the Lisbon RRI Hubs meeting on 23 April 2015

1. WP1
Towards a catalogue of good RRI practices
Lisbon, 2015

2. | Working definition (D1.1)
Science
Education
Ethics
Gender
Equality
Governance
Public
Engagement
Open
Access

3. |
Developing the set of quality criteria and indicators
• stakeholder workshops across Europe in Fall 2014
practical activities, emerging insights
• review of policy and academic literature
theoretical ideas, old and new insights
• promising practice sheets collected by HUBs
reality check

4. | List of quality criteria (D1.3)
1. Diversity and Inclusion
Criteria
Specification PA Outc.
Indicators/sub-
criteria
Questions that invite thinking about indicators and criteria 2ac
3f
Engaging a variety of
stakeholder groups
Wide range
Is there a wide range of stakeholders involved, such that
there is a diversity of values and a diversity of types of
knowledge/expertise (i.e., experiential knowledge,
scientific knowledge) represented and/or generated?
(Rowe and Frewer, 2000)
2ac
Relevant voices
Is there diversity in the stakeholders engaged such that all
relevant voices are heard – silent as well as loud (i.e.,
stakeholder groups that might not feel immediately
empowered to let their view know and stakeholder groups
that do)?
2ac
Demographic
diversity
Is there diversity within the stakeholder groups involved in
terms of gender, ethnicity, class, age and other
demographics?
2ac
Sufficient amount
Are sufficiently many perspectives and participants
included, such that eventual outcomes are robust?
(ScienceWise, 2013)
2ac
Early involvement
Are relevant stakeholders involved from early stages of the
R&I trajectory onwards?
2c
Criteria formulated per
process requirement Subcriteria and questions
to think about it
Relation to policy agendas
and outcomes

5. |
Diversity
and
Inclusion
Engaging a
variety of
stakeholder
groups
Variety of
means of
stakeholder
engagement
Engagement of
publics
Attention for
appropriate
R&I models
Institutional
diversity

6. |
Engaging a
variety of
stakeholder
groups
Wide
range
Demogra
phic
diversity
Sufficient
amount
Relevant
voices
Is there a wide variety of
stakeholders involved, such that
there is a diversity of values and a
diversity of types of
knowledge/expertise?
Is there diversity in the
stakeholders engaged such that all
relevant voices are heard – silent
as well as loud?
Is there diversity within the
stakeholder groups involved in
terms of ender, ethnicity, class,
age and other demographics?
Are sufficiently many perspectives
and participants included, such that
eventual outcomes are robust
(ScienceWise, 2013)?

7. |
Openness
and Trans-
parency
Honest and
clear (re-)
presentation
Open and clear
communication
about the
deliberation
and decision-
making
Open and clear
communication
about results
Appropriate
means and
content of
communication
and education
per actor
Openness to
critical scrutiny
from all
stakeholders

8. |
Anticipatio
n and
Reflection
Analysis of
background,
situation and
context
Envisioning of
plausible
futures
Variety of
impacts
Facilitate
deliberation on
values, needs,
interests,
choices and
problem
definitions
Addressing
roles in RI
trajectories

9. |
Responsive
ness and
Adaptive
Change
Structure for
seeking and
incorporating
feedback
Flexible
process
management
Development
and
implementation
of evaluation
strategies
Flexible
attitudes to
revise views
and actions
Changing
responsibilities
Application of
results

10. |D1.3: This is not a checklist
• The prime importance of context
• Always keep the outcomes in mind
1. Evaluative framework to think about RRI initiatives
• A stimulus and inspiration to (re)shape research and innovation
2. Self-assessment tool(s) for those active in R&I/ RRI
• Quality criteria provide specific assistance to different stakeholders
3. Selection of promising practices for D1.4
• Loose fit: how do practices incorporate criteria and indicators?

11. |
D1.4: Methodology

12. |
7
26
8
9
1
Tool
Project
Program
Organization
Other
Included practices:
51 in total

13. |
Some thoughts about practice classification
• Diversity of RRI practices
o Informing policy, practice, society
• Inventory of stakeholder views
o Considerable overlap between stakeholders
o Lack of stakeholder collaboration

14. |
‘Learning for
doing’
‘Learning for
governance’
‘Learning for
learning’
RRI as a societal learning process
Classifying promising practices
Researchers
Business/Industry
CSOs
Policy makers
CSOs
researchers
Educators
Researchers

15. |
Example: ‘Learning for learning’
Name Leading organization Country
Geographical
scale Type of practice
Xplore Health Barcelona Science Park European European Project
Center for the Promotion of Science
Ministry of Education, Science and
Technological Development Croatia National Organization
Fundación Ibercivis Fundación Ibercivis Spain National Organization
Sparkling Science
Federal Ministry of Science, Research and
Economy Austria National Program
Science.lu Fonds National de la Recherche Luxemburg Luxemburg National Tool
Homoresponsabilis in the Globalized world
TIME Foundation – eco projects/ Groupe One
Belgium European European Tool
RVP - Educational Program Framework Ministry of Education Czech Republic National Tool
Agora Scienza Centro interuniversitario The University of Torino Italy National Organization
Ustanova Hia eksperimentov Slovenia National Organization
Communicating Science Project Slovenian Research Agency (ARRS) Slovenia National Project
Scientific Support Committee of the Walloon Rural Network
Tr@me scrl / Walloon Rural Network Support
Unit Belgium Regional Project
Sociale innovatiefabriek
Vlaams netwerk van verenigingen waar armen
het woord nemen Belgium Regional Organization

16. |
‘Learning for
doing’
‘Learning for
governance’
‘Learning for
learning’
RRI as a societal learning process
Classifying promising practices
HAO2
PULSE
Marlisco
Challenge-driven Innovation
Voices
Forschungswende
Frame Reflection Lab
Explore Health
Nanopinion

17. |
HAO2
Small British company that
aims to promote the societal
inclusion of individuals who
are often excluded from
employment, especially
people with disabilities
(such as autism)
R&A
A&R
O&T
Diversity
&
Inclusion

18. |
Challenge-driven Innovation Program by Swedish
innovation agency
VINNOVA. It funds research
and innovation aimed at
tackling societal challenges
that involves all relevant
stakeholders. Its three-
stage funding scheme is
very implementation -
oriented and has built in
mechanisms for promoting
responsiveness and
adaptive change.
D&I
A&R
O&T
Respon-
siveness
&
Adaptive
change

19. |
Frame reflection lab
Amsterdam-developed playful
tool that helps reflect on
values and assumptions
underlying people’s views on
science and technology. It
combines enable AV stories
with stepwise reflection
exercises.
D&I A&R
O&T
Anticipation
& Reflection

20. |
Remarkable patterns
1. Practices do not (have to) incorporate all RRI processes + outcomes
2. Most projects further away from R&I process. Selection bias or fact?
3. Many RRI projects receive funding for a limited time (EC); difficult to
maintain the project or the product afterwards (continuity, commitment)
4. There is more awareness/ consultation than participation
5. Inclusive deliberation in place, but what about policy impact and change
6. It's either Responsible Research or Responsible Innovation