http://www.niso.org/news/events/2012/nisowebinars/alternative_metrics/          NISO Webinar:Beyond Publish or Perish: Alt...
Technical Lead ALM Project                                                 Martin Fennerhttp://www.flickr.com/photos/wally...
Journal-based metrics correlate poorlywith numbers for individual article                                         3
PLOS ONE is not a second-tier journalCombined Scopus citation counts for all PLOSBiology and PLOS Medicine articles publis...
The megajournal has changed how we use thejournal as a filter                                         5
Citations are only a small fraction of how a       paper is reused and discussed                                   100%   ...
The PLOS Article-Level Metrics projectstarted in 2009 and tracks usage, citationsand social web activity for all PLOS arti...
8
9
Why use article-level metrics?                                 10
Should article-level metrics be used to evaluate the impact of research?• Can numbers reflect the impact of research, acro...
What do we measure?                      12
Gaming and Spam                  13
Metrics as a discovery tool                              14
Data-driven stories about the post-publication reception of a paper                                      15
http://blogs.bbsrc.ac.uk/index.php/2012/09/summertime-reprise/http://dx.doi.org/10.1371/journal.pcbi.1000204              ...
17
Business intelligence for ajournal, university of funder                                18
http://bit.ly/alm36                  19
20
Tweets about Big Food CollectionData collected August 19, 2012.                                     21
Identify Open Access articles published by PLOS,funded by the European Commission SeventhFramework Programme (FP7)Match ar...
Author locations from the ENGAGE projectFirst authors (blue) and last authors (green).European Network for Genetic and Gen...
Scopus         24
Mendeley           25
Thank you            Wordle of altmetrics manifesto by AJ Cann            http://www.flickr.com/photos/ajc1/6795008004/   ...
Altmetrics and Revolutions:New media, new metrics, and a new scholarly communications system.                            N...
In the beginning:  ...was the letter.   • slow   • costly   • duplication   • best solution given available     technology
The First RevolutionIn 1665, Oldenburg published ThePhilisophical Transactions of theRoyal Society (now OA :).This applied...
The First Revolutionpromoted homogeneity ofoutputs.The standardized article was born ofthe need for industrial-scale repli...
The Second Revolutionwill promote diversity ofoutputs.Since publication is nearly free, itbecomes trivial to capture the m...
conversationstoriesanalysisdata
Looking beneath the foliage:    Web promises new tools for conversation.     reference managers                    blogs  ...
Examples: Mendeley2 million userlibraries200 milliondocumentsuploaded, 75Munique (MEDLINEhas 18 million...)
Examples: Twitter    In one month, over 58k citations from    Twitter to scholarly articles (citwaitions?)   It is like ha...
Examples: Twitter
Bring on themetrics.
A network of ideas: bibliometrics.In 1961, Garfield creates theScience Citation Index. • replaces expert judges with   cro...
But only part of the picture1. Only one type of person: academics.2. Only one kind of resource: scholarly articles.3. Only...
What about all the other uses?    Reading, annotating, bookmarking, saving,    discussing, sharing, etc?"...there are     ...
What about all the other uses?We start to confuse“the kind of use we cantrack” with useand “citation impact”with impact.
Bibliometrics mined impacton the first scholarly Web.  altmetrics minesimpact on the next              one.
Why altmetrics?• We can more fairly evaluate researchers, publications,  and institutions if we look at the whole impact p...
Journals need an upgrade.• First journals went hand in hand with the Scientific Revolution.  Applied the most advanced tec...
We don’t use the Web.• Berners-Lee created the Web as a scholarly communication tool.• Today the Web has revolutionized ev...
But what if we did?The Decoupled Journal article: a case study.
The second revolution has started.Once we have alt-citation data, it’s too useful toignore; alternative filters and even c...
A wise man, that Chekov.
Questions?Jason Priem @jasonpriem, http://jasonpriem.org                        This is a living document; if a particular...
NISO Webinar: Beyond Publish or Perish: Alternative Metrics for Scholarship
NISO Webinar: Beyond Publish or Perish: Alternative Metrics for Scholarship
NISO Webinar: Beyond Publish or Perish: Alternative Metrics for Scholarship
NISO Webinar: Beyond Publish or Perish: Alternative Metrics for Scholarship
NISO Webinar: Beyond Publish or Perish: Alternative Metrics for Scholarship
NISO Webinar: Beyond Publish or Perish: Alternative Metrics for Scholarship
NISO Webinar: Beyond Publish or Perish: Alternative Metrics for Scholarship
NISO Webinar: Beyond Publish or Perish: Alternative Metrics for Scholarship
NISO Webinar: Beyond Publish or Perish: Alternative Metrics for Scholarship
NISO Webinar: Beyond Publish or Perish: Alternative Metrics for Scholarship
NISO Webinar: Beyond Publish or Perish: Alternative Metrics for Scholarship
NISO Webinar: Beyond Publish or Perish: Alternative Metrics for Scholarship
NISO Webinar: Beyond Publish or Perish: Alternative Metrics for Scholarship
Upcoming SlideShare
Loading in …5
×

NISO Webinar: Beyond Publish or Perish: Alternative Metrics for Scholarship

9,734 views

Published on

Increasingly, many aspects of scholarly communication—particularly publication, research data, and peer review—undergo scrutiny by researchers and scholars. Many of these practitioners are engaging in a variety of ways with Alternative Metrics (#altmetrics in the Twitterverse). Alternative Metrics take many forms but often focus on efforts to move beyond proprietary bibliometrics and traditional forms of peer referencing in assessing the quality and scholarly impact of published work. Join NISO for a webinar that will present several emerging aspects of Alternative Metrics.

Published in: Education

NISO Webinar: Beyond Publish or Perish: Alternative Metrics for Scholarship

  1. http://www.niso.org/news/events/2012/nisowebinars/alternative_metrics/ NISO Webinar:Beyond Publish or Perish: Alternative Metrics for Scholarship November 14, 2012Speakers: Martin Fenner, Jason Priem, Aalam Wassef
  2. Technical Lead ALM Project Martin Fennerhttp://www.flickr.com/photos/wallyg/156079118/
  3. Journal-based metrics correlate poorlywith numbers for individual article 3
  4. PLOS ONE is not a second-tier journalCombined Scopus citation counts for all PLOSBiology and PLOS Medicine articles published in2009, as well as top 200 PLOS ONE articles in 2009 Citation counts collected November 8, 2012 4
  5. The megajournal has changed how we use thejournal as a filter 5
  6. Citations are only a small fraction of how a paper is reused and discussed 100% 22.2% Article-Level Metrics from November 8, 2012 for 63,771 PLOS Papers 0.3% 6
  7. The PLOS Article-Level Metrics projectstarted in 2009 and tracks usage, citationsand social web activity for all PLOS articlesUsage Citations Social WebPLOS Journals CrossRef Mendeley(HTML, PDF, XML) Scopus CiteULikePubMed Central Web of Science Facebook(abstract, fulltext, PubMed Central TwitterPDF, figures) Research Blogging PLOS Comments Wikipedia 7
  8. 8
  9. 9
  10. Why use article-level metrics? 10
  11. Should article-level metrics be used to evaluate the impact of research?• Can numbers reflect the impact of research, across disciplines and over time?• Do the currently available metrics really measure impact or something else?• Does the use of metrics for evaluation create undesired incentives?• How easily can metrics be changed by self-promotion and gaming? 11
  12. What do we measure? 12
  13. Gaming and Spam 13
  14. Metrics as a discovery tool 14
  15. Data-driven stories about the post-publication reception of a paper 15
  16. http://blogs.bbsrc.ac.uk/index.php/2012/09/summertime-reprise/http://dx.doi.org/10.1371/journal.pcbi.1000204 16
  17. 17
  18. Business intelligence for ajournal, university of funder 18
  19. http://bit.ly/alm36 19
  20. 20
  21. Tweets about Big Food CollectionData collected August 19, 2012. 21
  22. Identify Open Access articles published by PLOS,funded by the European Commission SeventhFramework Programme (FP7)Match articles with grant numbersShow the impact of these publications PLOS Search API CORDIS 2562 Articles 1166 Articles 22
  23. Author locations from the ENGAGE projectFirst authors (blue) and last authors (green).European Network for Genetic and Genomic Epidemiologyhttp://www.openaire.eu/en/component/openaire/project_info/default/530?grant=201413 23
  24. Scopus 24
  25. Mendeley 25
  26. Thank you Wordle of altmetrics manifesto by AJ Cann http://www.flickr.com/photos/ajc1/6795008004/ 26
  27. Altmetrics and Revolutions:New media, new metrics, and a new scholarly communications system. Nov 14, NISO
  28. In the beginning: ...was the letter. • slow • costly • duplication • best solution given available technology
  29. The First RevolutionIn 1665, Oldenburg published ThePhilisophical Transactions of theRoyal Society (now OA :).This applied the best availabletechnology (the printing press) tovastly improve dissemination.
  30. The First Revolutionpromoted homogeneity ofoutputs.The standardized article was born ofthe need for industrial-scale replicationand interchangeability.
  31. The Second Revolutionwill promote diversity ofoutputs.Since publication is nearly free, itbecomes trivial to capture the missingpieces of the scholarly record.
  32. conversationstoriesanalysisdata
  33. Looking beneath the foliage: Web promises new tools for conversation. reference managers blogs social bookmarking social networks
  34. Examples: Mendeley2 million userlibraries200 milliondocumentsuploaded, 75Munique (MEDLINEhas 18 million...)
  35. Examples: Twitter In one month, over 58k citations from Twitter to scholarly articles (citwaitions?) It is like having a jury preselect what will probably interest you…. Occasionally there will be something that people will link to, and it will change what I think, or what I’m doing, or what I’m interested in. -study participant
  36. Examples: Twitter
  37. Bring on themetrics.
  38. A network of ideas: bibliometrics.In 1961, Garfield creates theScience Citation Index. • replaces expert judges with crowdsourced judgements • based on existing patterns of use: mining, not asking. • And thats awesome!
  39. But only part of the picture1. Only one type of person: academics.2. Only one kind of resource: scholarly articles.3. Only one kind of use: using to support ascholarly article.
  40. What about all the other uses? Reading, annotating, bookmarking, saving, discussing, sharing, etc?"...there are Heart of scholarly Spotting emergingundoubtedly highly communication research fronts willuseful journals that is "visits, personal require trackingare not cited contacts, and "formal andfrequently." letters." (Bernal, informal(Garfield, 1972) 1944) communication" (Kuhn, 1977)
  41. What about all the other uses?We start to confuse“the kind of use we cantrack” with useand “citation impact”with impact.
  42. Bibliometrics mined impacton the first scholarly Web. altmetrics minesimpact on the next one.
  43. Why altmetrics?• We can more fairly evaluate researchers, publications, and institutions if we look at the whole impact picture.• We can assess impact faster• Etc, etc.• Much more important: quantifying impact means we can teach machines what’s important. This changes the game. And the game needs to be changed.
  44. Journals need an upgrade.• First journals went hand in hand with the Scientific Revolution. Applied the most advanced technology available to the problem of spreading scholarship.• Today’s journals are still the best scholarly communication system possible using 17th-century technology.• They’ve got some problems: o Slow o Restrictive format: function follows form o Closed o Inconsistent quality control o Hard to innovate
  45. We don’t use the Web.• Berners-Lee created the Web as a scholarly communication tool.• Today the Web has revolutionized everything but scholarly communication.• Online journals are essentially paper journals, delivered by faster horses. In the early days, CERN maintained a list of all the world’s Web servers. Haha, CERN ur so krazy.• But today, we can fit every single meaningful outlet for scholarship (25k journals + some conferences) on one list, too.
  46. But what if we did?The Decoupled Journal article: a case study.
  47. The second revolution has started.Once we have alt-citation data, it’s too useful toignore; alternative filters and even certification pathsbased on this data will open.As Peter Vinkler says, citation graph data is likeChekhov’s gun: once on stage, it has to be fired.
  48. A wise man, that Chekov.
  49. Questions?Jason Priem @jasonpriem, http://jasonpriem.org This is a living document; if a particular version is important to you, make sure you link to that revision.

×