Published on

presented at "Assessing Contribution, Assessing Usage: Metrics in a New Context" NFAIS Virtual Conference, March 28, 2014 -

Published in: Education, Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. The NISO Altmetrics Initiative NFAIS Virtual Seminar: Assessing Contribution, Assessing Usage: Metrics in a New Context March 28, 2014 Nettie Lagace (@abugseye) NISO Associate Director for Programs
  2. 2. What’s NISO? • Non-profit industry trade association accredited by ANSI with 150+ members • Mission of developing and maintaining standards related to information, documentation, discovery and distribution of published materials and media • Represent US interests to ISO TC46 (Information and Documentation) and also serve as Secretariat for ISO TC46/SC 9 (Identification and Description) • Responsible for standards like ISSN, DOI, Dublin Core metadata, DAISY digital talking books, OpenURL, SIP, NCIP, MARC records and ISBN (indirectly) • Volunteer driven organization: 400+ spread out across the world
  3. 3. Premise of “Standards” • Consensus standards created by a community with various stakeholders • Trust • Leading to broader acceptance • Standards as plumbing • Standards facilitate trade, commerce and innovation • Standards reduce costs • Standards support better communication and interoperability across systems
  4. 4. Why worth funding? • Scholarly assessment is critical to the overall process – Which projects get funded – Who gets promoted and tenure – Which publications are prominent • Assessment has been based on citation since the 60s • Today’s scholars multiple types of interactions with scholarly content are not reflected – Is “non-traditional” scholarly output important too?
  5. 5. Why worth funding? • In order to move out of “pilot” and “proof-of- concept” phases … • Altmetrics must coalesce around commonly understood definitions, calculations and data sharing practices • Altmetrics must be able to be audited • Organizations who want to apply metrics will need to understand them and ensure consistent application and meaning across the community
  6. 6. What can be measured & how do we measure it? Image: Flickr user karindalziel
  7. 7. At what granularity do we measure?
  8. 8. Which measurements are valuable? "Watch your valuables!" by melancon is licensed under CC BY 2.0
  9. 9. Consistency in what is counted
  10. 10. Authorship Disambiguation & Contributor Roles Element Identification
  11. 11. • How do we ensure the validity of those measurements? • How to (or should we) deal with other issues such as gaming? • What infrastructure do we need for alternative metrics? • Basic definitions (so we are all taking about the same thing) • Open exchange of component data • Research on the use and integration of new scholarship forms "Q is for Question Mark" by b4b2 is licensed under CC BY 2.0
  12. 12. 2 Phases • Phase 1: Hold meetings of stakeholders to define a high-level list of issues – October 2013, San Francisco – December 2013, Washington, DC – January 2014, Philadelphia – Public Webinars – White paper output, public presentations, public feedback • Phase 2: Create Working Group within NISO structure, to create recommended practice(s) and/or standard(s) – Education/training efforts to ensure implementation • Final report to Sloan due November 2015
  13. 13. Steering Committee • Euan Adie, Altmetric • Amy Brand, Harvard University • Mike Buschman, Plum Analytics • Todd Carpenter, NISO • Martin Fenner, Public Library of Science (PLoS) (Chair) • Michael Habib, Reed Elsevier • Gregg Gordon, Social Science Research Network (SSRN) • William Gunn, Mendeley • Nettie Lagace, NISO • Jamie Liu, American Chemical Society (ACS) • Heather Piwowar, ImpactStory • John Sack, HighWire Press • Peter Shepherd, Project Counter • Christine Stohn, Ex Libris • Greg Tananbaum, SPARC (Scholarly Publishing & Academic Resources Coalition)
  14. 14. 1 2 3 San Francisco – PLOS ALM Washington DC – CNI Philadelphia – ALA Midwinter Community Meetings Oct 2013-Jan 2014
  15. 15. Meetings’ General Format • Held in conjuction with another (industry/community) meeting • Morning: lightning talks, post-it brainstorming • Afternoon: discussion groups – X – Y – Z – Report back/react • Livestreamed (video recordings now available)
  16. 16. Meeting Lightning Talks • Expectations of researchers • Exploring disciplinary differences in the use of social media in scholarly communication • Altmetrics as part of the services of a large university library system • Deriving altmetrics from annotation activity • Altmetrics for Institutional Repositories: Are the metadata ready? • Snowball Metrics: Global Standards for Institutional Benchmarking • International Standard Name Identifier •, Plum Analytics, Mendeley reader survey • Twitter Inconsistency “Lightning" by snowpeak is licensed under CC BY 2.0
  17. 17. SF Meeting – General outputs • The importance of best practices for media coverage of science (using DOIs, etc.) • More Altmetrics research is needed and could be promoted through this group • Providing a standard set of research outputs that we can use to compare different services • The importance of use cases for specific stakeholder groups in driving the discussion forward
  18. 18. Discussions San Francisco Washington DC Philadelphia Business & Use Cases Business & Use Cases Use Cases (3X) Quality & Data science Qualitative vs. Quantitative Data Integrity Definitions Definitions/Defining Impact Definitions Development & Infrastructure Identifying Stakeholders and their Values Standards Future Proofing
  19. 19.
  20. 20. Thank you! Questions? Nettie Lagace @abugseye