An introduction to altmetrics, the complementary metrics of research impact. The presentation covers some of the challenges with more traditional measures, and the potential of and challenges with altmetrics. The presentation gives a brief overview of the background to a new research project about measuring the societal impact of open science.

1. Background image: http://commons.wikimedia.org/wiki/File:Water_drop_impact_on_a_water-surface_-_(5).jpg
Altmetrics
Measuring the impact of scientific activities
Kim Holmberg
Research Unit for the Sociology of Education
University of Turku
(e) kim.j.holmberg@utu.fi
(w3) http://kimholmberg.fi
@kholmber
Oulu,
8.5.2015

2. Bibliometrics
the study of quantitative aspects of
the production, dissemination, and
use of recorded information
Tague-Sufcliffe, 1992

3. 0
1000
2000
3000
4000
5000
6000
7000
8000
9000
201420132012201120102009200820072006200520042003200220012000199919981997199619951994
The increase of publications about bibliometrics as indexed by Web of Science
between 1994 and 2014 (data retrieved from Web of Science on 9 February, 2015).

5. Goodhart’s Law:
When a measure becomes
the target, it ceases to be
a good measure
Goodhart, 1975

6. Outreach and Impact Leader, - Ref:1455401
“ We are seeking to recruit … to support, develop
and promote outreach and impact activities …
Our aim is to maximize the widest possible
impact of the research programme in whichever
way is most appropriate …
The salary range will be £41,430 - £48,873 per
annum...”
Taking impact seriously

7. Problems with
citation-based indicators

8. Problem 1:
Time

9. 0 2 4 6 8 10 12 14 16 18 20
Impact factor window
Immediacy index window
Cited half-life
50% of
citations
50% of
citations
Time after publication (Years)
Citations

10. 0 2 4 6 8 10 12 14 16 18 20
Time after publication (Years)
Citations
Time of research assessment
Option 1: Journal impact factor, Julkaisufoorumi, …
Potential
future
impact

11. 0 2 4 6 8 10 12 14 16 18 20
Time after publication (Years)
Citations
Time of research assessment
Option 2: Citations (after n years)
Past
success and
impact

12. 0 2 4 6 8 10 12 14 16 18 20
Time after publication (Years)
Citations How can we measure
impact in realtime?

13. Problem 2:
Journals

15. 1
vs.

16. Fenner, M. & Lin, J. (2013). Altmetrics in Evolution: Defining & Redefining the Ontology of Article-Level Metrics.
http://dx.doi.org/10.3789/isqv25no2.2013.04
2

28. Problem 3:
Impact

29. Science
IMPACT ON:

30. Science
Society
Environment
Culture
Economy
Education
IMPACT ON:

31. SOLUTION?

32. Altmetrics
the study and use of scholarly
impact measures based on activity
in online tools and environments
Priem, J. (2014). Altmetrics. In Cronin, B. & Sugimoto, C.R.
(Eds.). Beyond Bibliometrics: Harnessing Multidimensional
Indicators of Scholarly Impact. MIT Press, 2014.

36. Created by
researchers
Created by
the public
Altmetrics

37. Created by
researchers
Indicating
future
scientific
impact?
Altmetrics

38. Indicating
other types of
impact, such
as societal
impact?
Created by
the public
Altmetrics

39. Researchers
The public

40. Rank Title Altmetric score News Blogs Tweets
1 Experimental evidence of massive-scale emotional
contagion through social networks
5044 301 130 3801
2 Variation in Melanism and Female Preference in Proximate
but Ecologically Distinct Environments
4823 4 11 6669
3 Artificial sweeteners induce glucose intolerance by
altering the gut microbiota
3500 127 34 3303
4 Stimulus-triggered fate conversion of somatic cells into
pluripotency
2986 159 92 3295
5 Dogs are sensitive to small variations of the Earth's
magnetic field
2956 54 33 2668
6 Christmas 2013: Research: The survival time of chocolates
on hospital wards: covert observational study
2734 3 4 3084
7 Epidemiological modeling of online social network
dynamics
2392 156 19 1649
8 Searching the Internet for evidence of time travelers 2246 34 22 2262
9 Conscious Brain-to-Brain Communication in Humans Using
Non-Invasive Technologies
2160 82 32 1660
10 Were James Bond's drinks shaken because of alcohol
induced tremor?
2146 127 14 1559
Top 10 articles according to the Altmetric Score (Source: http://www.altmetric.com/top100/2014/)

41. 0
50
100
150
200
250
300
350
0
1000
2000
3000
4000
5000
6000
7000
8000
1 2 3 4 5 6 7 8 9 10
Altmetric score
Tweets
News
Blogs
The volume of tweets, blog posts, and news stories for the top ten research articles
from 2014 as measured with altmetrics, and their Altmetric Score (number of news
stories and blog posts are depicted on the secondary axis to the right in the figure)

42. Researchers
The public
Lowimpact
Highimpact

43. Altmetrics
Provide a more nuanced
image of the total impact
that research has made?

44. Citations Altmetrics
Altmetrics are not alternatives to citations

45. Citations Tweets
Older publications Newer publications
The effects of time on the volume of citations and tweets
(Haustein, Peters, Sugimoto, Thelwall, & Larivière, 2014)
The effects of time

46. Heterogeneity of altmetrics

47. doiDigital Object Identifier
Most altmetrics identified from:

48. Bollen J, Van de Sompel H, Hagberg A, Bettencourt L, Chute R, Rodriguez MA, et al. (2009) Clickstream
Data Yields High-Resolution Maps of Science. PLoS ONE 4(3): e4803. doi:10.1371/journal.pone.0004803
Disciplinary differences

49. Bots?
Haustein, S., Bowman, T.D., Holmberg, K., Tsou, A., Sugimoto, C.R. & Lariviére, V. (in press). Tweets as impact indicators: Examining the implications of
automated “bot” accounts on Twitter. To appear in JASIST. arXiv version available at: http://arxiv.org/ftp/arxiv/papers/1410/1410.4139.pdf.
Gaming altmetrics?

50. Stories behind the altmetrics?

51. Altmetrics and
open science?

52. Measuringthe Societal Impact of Open Science
AvoimenTieteenYhteiskunnallisenVaikuttavuudenMittarit
Project is financed by the Ministry of Education and Culture in Finland (2015-2016)

53. Goals of the research project:
1) Investigate altmetrics (at country-level, i.e.
Finland) and create a prototype to
demonstrate them
2) Develop methods to map societal impact of
research (and to analyze the stories behind
the altmetrics)

54. Shameless self
promotion:
Book
forthcoming
later in 2015

55. Kim Holmberg
kim.j.holmberg@utu.fi
http://kimholmberg.fi
@kholmber
Kiitos!
Slides available at
http://www.slideshare.net/kholmber