Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Niso Article Level Metrics Presentation For Online 2


Published on

Published in: Technology, Business
  • I like this service ⇒ ⇐ from Academic Writers. I don't have enough time write it by myself.
    Are you sure you want to  Yes  No
    Your message goes here

Niso Article Level Metrics Presentation For Online 2

  1. 1. Committed to making the world’s scientific and medical literature a public resource Article-Level Metrics (at PLoS & beyond) Peter Binfield, Managing Editor of PLoS ONE,
  2. 2. <ul><li>Article-Level Metrics (at PLoS & beyond) </li></ul><ul><li>These are the slides and audio from a webinar organised by NISO on May 13 th , 2009. The topic of the webinar was “ New Applications of Usage Data”. </li></ul><ul><li>This segment (the second of three) was given by Peter Binfield, Managing Editor of PLoS ONE </li></ul><ul><li>More information (as well as details of the other presentations) can be found at the NISO site at: </li></ul>
  3. 3. <ul><li>Presentation Outline </li></ul><ul><li>Brief overview of PLoS </li></ul><ul><li>What is the problem? </li></ul><ul><li>Current assessment methods </li></ul><ul><li>Possible assessment methods </li></ul><ul><li>Article-Level Metrics at PLoS and beyond </li></ul><ul><li>(also, see the NISO resource page for relevant links) </li></ul>
  4. 4. <ul><li>The Public Library of Science (PLoS) </li></ul><ul><li>Largest Not for Profit, Open Access Publisher </li></ul><ul><li>Based in San Francisco and Cambridge, UK </li></ul><ul><li>Publisher of 7 journals </li></ul><ul><ul><li>PLoS Biology, PLoS Medicine, PLoS Pathogens, PLoS Computational Biology, PLoS NTDs, PLoS Genetics, PLoS ONE </li></ul></ul><ul><li>Largest journal is PLoS ONE </li></ul><ul><ul><li>Which is almost doubling in volume each year </li></ul></ul>
  5. 5. <ul><li>What is the Problem? </li></ul><ul><li>As a consumer of research output: </li></ul><ul><li>How do you measure the ‘worth’ or the ‘impact’ of research output </li></ul><ul><ul><li>typically, but not exclusively, a journal article </li></ul></ul><ul><li>and how do you filter what you read? </li></ul><ul><ul><li>You can’t read everything, so how do you filter to find the ‘best’ material for your interests? </li></ul></ul><ul><li>Aside: ‘impact’ is a loosely used term </li></ul><ul><ul><li>We could use words like ‘utility’ ; ‘value’ ; ‘relevance’ ; ‘interest (to me)’ ; ‘Return on Investment’ etc. </li></ul></ul><ul><ul><li>For this presentation I will just use the word ‘impact’ </li></ul></ul>
  6. 6. <ul><li>What is the Problem? </li></ul><ul><li>Again, as a consumer of research output: </li></ul><ul><li>At what level of granularity do you want to measure ‘impact’ and why are you measuring it? </li></ul><ul><ul><li>Journals - to know which titles to subscribe to? </li></ul></ul><ul><ul><li>Research groups / institutions - to know who to fund? </li></ul></ul><ul><ul><li>Individual researchers - to know who to promote? </li></ul></ul><ul><ul><li>Individual articles - to know what to read? </li></ul></ul><ul><li>and how do you compare ‘impact’ across different journals / disciplines / sub-disciplines / publishers? </li></ul>
  7. 7. <ul><li>What is the Problem? </li></ul><ul><li>Given these inherent complexities, how is ‘impact’ currently measured? </li></ul><ul><ul><li>Citation based measures (mostly) </li></ul></ul><ul><ul><li>Usage based measures (occasionally) </li></ul></ul><ul><ul><li>Anecdotal / personal evaluations (often) </li></ul></ul>
  8. 8. <ul><li>Current Assessment Methods </li></ul><ul><li>(quick overview) </li></ul><ul><li>Mostly journal-based measures </li></ul><ul><ul><li>the Impact Factor </li></ul></ul><ul><ul><li>COUNTER usage reports </li></ul></ul><ul><li>Some article-level or ‘person’-level measures </li></ul><ul><ul><li>the h-index </li></ul></ul><ul><ul><li>Article level citation information </li></ul></ul><ul><ul><li>Article level usage data </li></ul></ul>
  9. 9. <ul><li>Current Assessment Methods </li></ul><ul><li>Article level usage data </li></ul><ul><ul><li>What does it mean to have ‘a download’? Can a download be equated to ‘impact’? If you downloaded it, did you read it? </li></ul></ul><ul><ul><li>Easy to game? </li></ul></ul><ul><ul><li>What is a high vs. low number in any given field, or over any given time frame? </li></ul></ul><ul><ul><li>An apparent fear by data owners (publishers) that publicizing this data could be dangerous </li></ul></ul><ul><ul><li>Very little provision of this data – only the ‘Journal of Vision’ and the ‘Frontiers’ series currently provide hard data. Some other journals provide summaries (e.g. ‘top downloaded articles’ from HighWire titles) but no actual numbers are provided </li></ul></ul>
  10. 10. Journal of Vision Screenshot
  11. 11. Frontiers in Neuroscience Screenshot
  12. 12. NEJM Screenshot
  13. 13. <ul><li>Possible assessment methods </li></ul><ul><li>There are many possible ways to assess the ‘impact’ of any piece of research (or any researcher): </li></ul><ul><ul><li>Citations ; Usage ; Media Coverage ; Blog Coverage ; Discussion Thread activity ; Expert Ratings ; Social Bookmarking activity ; whether or not public policy was affected; where the research was done ; who is citing it ; who is reading it etc etc </li></ul></ul><ul><li>Current technology, and Web 2.0, is now making it possible to automate many of these metrics </li></ul>
  14. 14. <ul><li>Possible assessment methods </li></ul><ul><li>If these types of measures could be collated and evaluated correctly then they could be more powerful (and useful) than citations or usage metrics alone </li></ul><ul><li>This is what PLoS is attempting to do with our Article-Level Metrics program - to implement new approaches to the evaluation and filtering of journal articles. </li></ul>
  15. 15. <ul><li>Article-Level Metrics (at PLoS) </li></ul><ul><li>Important to note: Article-Level Metrics at PLoS are not just about citations and usage. The concept refers to a whole range of additional measures which might give a window on ‘impact’ </li></ul><ul><li>We are providing metrics at the article-level, for every article, in every one of our titles. </li></ul><ul><li>We are the first publisher to provide this range of data, but we hope that others will follow </li></ul>
  16. 16. <ul><li>Article-Level Metrics (at PLoS) </li></ul><ul><li>Rollout Plan: </li></ul><ul><li>We are starting by acquiring and presenting the relevant data. </li></ul><ul><li>We have a preference for data that is not ‘owned’ by third parties, and we are seeking ‘open’ APIs onto the data (so that anyone else can verify or replicate these measures themselves). </li></ul><ul><li>‘ Phase 2’ will provide data analysis, filtering and navigation tools using this data. </li></ul>
  17. 17. <ul><li>Article-Level Metrics (at PLoS) </li></ul><ul><li>In March 2009, we launched with: </li></ul><ul><li>Number of Citations </li></ul><ul><ul><li>PubMedCentral and Scopus </li></ul></ul><ul><li>Amount of Blog coverage </li></ul><ul><ul><li>Postgenomic, Nature Blogs and Bloglines </li></ul></ul><ul><li>Number of Social Bookmarks </li></ul><ul><ul><li>CiteULike and Connotea </li></ul></ul><ul><li>Commenting, notation and ‘star ratings’ </li></ul><ul><li>Each measure is provided as a simple number, but with a link to a landing page at the third party </li></ul>
  18. 18. Current View (March ‘09)
  19. 19. Current View (March ‘09)
  20. 20. Citeulike Landing Page
  21. 21. Current View (March ‘09)
  22. 22. Postgenomic Landing Page
  23. 23. <ul><li>Article-Level Metrics (at PLoS) </li></ul><ul><li>Future Expansion of Current Provision </li></ul><ul><ul><li>Add ‘behavioural’ or ‘reference management’ tools (e.g. Mendeley) </li></ul></ul><ul><ul><li>Expand the number of sources for each data type (e.g. more blog aggregators, more citation sources) </li></ul></ul><ul><ul><li>Add expert ratings (e.g. F1000 or similar) </li></ul></ul><ul><ul><li>Add news coverage (hopefully) </li></ul></ul><ul><ul><li>Add usage data </li></ul></ul>
  24. 24. <ul><li>Article-Level Metrics (at PLoS) </li></ul><ul><li>Adding Usage Data </li></ul><ul><li>In mid-2009, we will add usage data to every PLoS article, showing: </li></ul><ul><ul><li>HTML Page Views </li></ul></ul><ul><ul><li>PDF Downloads </li></ul></ul><ul><ul><li>XML Downloads </li></ul></ul><ul><li>To be displayed numerically and graphically, including historical data, and in the context of other articles within the journal. </li></ul>
  25. 25. <ul><li>Article-Level Metrics (at PLoS) </li></ul><ul><li>Future Developments </li></ul><ul><li>Provide an open data set </li></ul><ul><li>COUNTER certification of the usage data </li></ul><ul><li>Inclusion of usage data from third parties (e.g. PubMed Central) </li></ul><ul><li>De-duplication of the data sources </li></ul><ul><li>Provision of tools to act on all these metrics for: </li></ul><ul><ul><li>Discovery, Filtering and Comparison purposes </li></ul></ul><ul><li>Adherence to any standards that may evolve </li></ul>
  26. 26. <ul><li>Article-Level Metrics (beyond PLoS) </li></ul><ul><li>Article-Level Metrics could be very significant for the academic community on a number of levels. </li></ul><ul><li>We do not want this to be a PLoS-only initiative. By demonstrating it, we hope other publishers will follow </li></ul><ul><li>We hope that acquisition and presentation standards will evolve or be created </li></ul><ul><li>We hope that by creating an open data set of standardized (cross journal) results, academia will then develop new and powerful ways to combine and evaluate this data </li></ul><ul><li>There is likely to be a range of ways to evaluate impact, for a range of different fields and different purposes, but the important thing will be consistent, reliable and comparable data sets </li></ul>
  27. 27. Article-Level Metrics We could be at the start of a major new development in academic publishing.
  28. 28. Article-Level Metrics We could be at the start of a major new development in academic publishing. To misquote from Jerry Macquire:
  29. 29. Article-Level Metrics We could be at the start of a major new development in academic publishing. To misquote from Jerry Macquire: “ Show Me The Metrics!”
  30. 30. Article-Level Metrics We could be at the start of a major new development in academic publishing. To misquote from Jerry Macquire: “ Show Me The Metrics!” Peter Binfield Managing Editor, PLoS ONE [email_address]