The document discusses a conference on open peer review, innovative dissemination, and alternative metrics. It summarizes the objectives to explore the current landscape of altmetrics indicators, generate a taxonomy of scientific knowledge dissemination channels and altmetrics indicators, and suggest indicators to assess research impact and quality. It provides results of the conference including a bibliometric analysis of altmetrics research and a final taxonomy of linkages between dissemination channels based on expert interviews. It discusses challenges like normalization issues in altmetrics and outlines an outlook for making altmetrics more entrenched in the current value system of science.

1. [Open Peer
Review/Innovative
Dissemination/Alternati
ve Metrics] in OpenUP
OpenUp conference–Sept. 5,
2018, Brussels

2. 2018-09-12 2
Objectives
• Explore the current landscape of altmetrics and open
metrics, including a systematic overview of current
indicators
• generate a validated taxonomy of channels of scientific
knowledge dissemination and altmetrics indicators
• … and suggest indicators enabling assessing impact
and quality of the underlying research.

3. 3
State-of-play: Altmetrics on the rise

4. 4
State-of-play

5. 2018-09-12 5
State-of-play
• Altmetrics is extremely heterogeneous
• Scrutinization attempts are (still) “en vogue”
• There are some crucial challenges to solve
• Normalization issues - in an open universe (vs. closed universe
of bibliographic databases)
• Coverage (Differences between data providers)
• Conceptual scrutinization and construct validity (Impact Flavor
debate, taxonomic attempts)

6. 2018-09-12 6
Results
• Bibliometric analysis of altmetrics-related research.
• Implementation of tracking infrastructure for altmetrics-related Twitter activities (Corpus of
more than 54k Twitter messages).
• Desk research to identify current trends and state of the art in altmetrics, condensing a
total of 416 publications.
• Desk research on research impact current situation, issues, and opportunities >SWOT
analysis based on the state of the art of altmetrics
• In-depth analysis of survey data to inform taxonomy development to assess the overall role
and relevance of altmetrics (Inform, Disseminate, Appreciate, Impact)
• Final taxonomy of linkages between altmetrics and innovative dissemination, conducting
more than 20 expert interviews

7. 7
Information behaviour, academic reward system and
society at large

8. 8
D5.4: Taxonomical tree of Dissemination
Channels based on OpenUp survey

9. 10
D5.4: Example of taxonomy table
Dimension: Doing Science and
Science made public
Service 1
Blogging
Service 2
twitter
Service 3
Faceboo
k
Service 4
RG
Attention (expert interviews, desk
research)
Medium High High Medium
General scholarly appreciation
(Survey data)
Medium Low Very low Medium
Scholarly relevance (literature
review, expert interviews)
Yes No No Yes
Field specific appreciation (expert
interviews)
Yes No No Yes
Metrics (desk research) Citations
Downloads
Views
Mentions
Followers
Mentions
Views
Likes
Activities
within the
network
Provider coverage (desk research) Altmetric,
Plum
Altmetric,
Plum, PloS,
Impactstory
Altmetric
., Plum
None
Technical Accessibility (API) Partly Yes (but
restricted)
Yes (but
restricte
d)
No
Intergenerational differences in
assessment (survey)
High Medium Low NA

10. 2018-09-12 11
OpenUP Hub
• Overview of main content in OpenUP hub for this
area:
• Altmetrics providers (Platforms, which collect data from
different sources & produce Altmetrics)
• Altmetrics features (SWOT analysis, Data sources & the
Taxonomy of Altmetrics)
• Training videos: Current issues in Altmetrics
• …blog posts on Altmetrics

11. 2018-09-12 12
Outlook
• Altmetrics is a booming, but heterogeneous field
• Potential for assessing novel forms of dissemination and accounting for
dynamics within science
• But: so far not entrenched within the current value system of science
What’s likely to be relevant in the future?
• Content-based perspective (What do we count? Who engages? With what?
How? Why?)
• Reflexive perspective (How do altmetrics feed back into science?)
• Understanding gaming effects (Offerbots, (Re-)tweet farms, “social media
suppliers”)

12. 13
Name
Email

13. Backup
1) "Which of these channels listed do you use most
frequently to disseminate your own research to reach your
target groups?" (Disseminate)
2) "How frequently do you use these sources to inform your
professional work as a researcher?" (Inform)
3) "Please rate how being represented in these dissemination
activities is generally appreciated within your field of
research" (Appreciate)
4) "What potential do the following dissemination channels
have to lead to a wider societal impact?" (Society)
For each question, a battery of 19 dissemination channels
was assessed on a 5-item end-point-scaled Likert scale.