This document discusses problems with current metrics for measuring research quality and impact, such as journal impact factors. It argues that research is moving too slowly and producing poor quality results. Alternative metrics called "altmetrics" are proposed to provide better data on what research is actually being read and used. The document introduces Mendeley as a company that aggregates research data in the cloud and collects rich signals about papers, people, and topics to provide analytics about research impact and help make science more collaborative and transparent.

1. Social metrics for research:
Quantity and Quality
William Gunn
Head of Academic Outreach
william.gunn@mendeley.com
@mrgunn

2. A Big Problem
“The state of knowledge
of the human race is
sitting in the scientists’
computers, and is
currently not shared […]
We need to get it
unlocked so we can
tackle those huge
problems.”

3. Research is too slow

4. Research is poor quality
- Amgen: 47 of 53 “landmark” oncology publications could
not be reproduced.
- Bayer: 43 of 67 oncology & cardiovascular projects were
based on contradictory results
- Dr. John Ioannidis: 432 publications purporting sex
differences in hypertension, multiple sclerosis, or lung
cancer. Only one data set was reproducible.

5. https://secure.flickr.com/photos/fireflythegreat/2845637227/

6. Journal Impact Factor
Number of citations
Citeable items = Impact Factor

7. Problems with Impact Factor
It's inaccurate

8. Problems with Impact Factor

9. Problems with Impact Factor
“During discussions with
Thomson Scientific over which
article types the company deems
as “citable,” it became clear that
the process of determining a
journal's impact factor is
unscientific and arbitrary.”
http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0030291

10. The higher the impact factor,
the more likely the research is to
be retracted, partly due to
intense competition.
http://bjoern.brembs.net/news766.html.11
Highly Tweeted articles are 11x
more likely to be highly cited.
(Eysenbach 2011)
http://www.jmir.org/2011/4/e123/
What matters is who is
reading your work!

11. Why are altmetrics
important?
Get better data on what's working
Get it faster
Provide qualitative descriptions, not just
quantitative measures

12. Mendeley makes science more collaborative and
transparent:
...and aggregates research
data in the cloud
Mendeley extracts
research data…
Collecting rich signals
from domain experts.

13. 200 million documents uploaded
2.0 million users
Cambridge
Stanford University
MIT
Imperial College London
University of Oxford
Harvard University
University of Michigan
University College London
University of California at Berkeley
Columbia University

14. Rich user profile data

15. Papers, people, topics

22. Google Analytics for research