The Infrastructure for Alternative Metrics

Like this? Share it with your network

Share

The Infrastructure for Alternative Metrics

  • 933 views
Uploaded on

The Infrastructure for Alternative Metrics: What do we need to compare digital apples to digital apples? ...

The Infrastructure for Alternative Metrics: What do we need to compare digital apples to digital apples?
Todd Carpenter, Executive Director, NISO
Presented at AAAS Annual Meeting: February 16, 2014
Chicago, IL

More in: Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
933
On Slideshare
568
From Embeds
365
Number of Embeds
16

Actions

Shares
Downloads
4
Comments
0
Likes
0

Embeds 365

http://www.infodocket.com 168
http://www.niso.org 115
http://feedly.com 37
http://www.scoop.it 16
http://www.newsblur.com 10
http://reader.aol.com 4
http://127.0.0.1 3
http://www.feedspot.com 3
http://digg.com 2
http://newsblur.com 1
http://www.inoreader.com 1
http://wordpress.com 1
http://reader.delooper.net 1
http://www.hanrss.com 1
https://www.commafeed.com 1
http://translate.googleusercontent.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. The Infrastructure for Alternative Metrics What do we need to compare digital apples to digital apples? ! Todd Carpenter, Executive Director, NISO AAAS Annual Meeting Chicago, IL | February 16, 2014
  • 2. We hate discussing infrastructure
  • 3. Photo: Minneapolis College
 of Art and Design Library
  • 4. About • Non-profit industry trade association accredited by ANSI with 190+ members • Mission of developing and maintaining technical standards related to information, documentation, discovery and distribution of published materials and media • Volunteer driven organization: 400+ spread out across the world • Responsible in various wasy for standards like ISSN, DOI, Dublin Core metadata, DAISY digital talking books, OpenURL, MARC records, and ISBN
  • 5. Photo by dumbledad on Flickr http://www.flickr.com/photos/dumbledad/298650884/!
  • 6. But what is the “infrastructure” for assessment?
  • 7. We stand on the shoulders of giants...
  • 8. The Mighty Citation
  • 9. Impact factor
  • 10. Thomson-Reuters 
 IMPACT FACTOR
  • 11. Two Stanford Students
  • 12. Two Stanford Students
  • 13. They wrote a paper
  • 14. They started a company in someone’s garage Photo: AP
  • 15. A simple conclusion can be drawn Citations are awesome, aren’t they? A perfect solution to all the world’s problems, no?
  • 16. Back to Sir Isaac Newton “What Des-Cartes did was a good step.You have added much several ways, & especially in taking ye colours of thin plates into philosophical consideration. If I have seen further it is by standing on ye shoulders of Giants."
  • 17. Caveat emptor Citations aren’t always what we think they are
  • 18. Another problem with citations
  • 19. So where to from here?
  • 20. Define altmetrics, please
  • 21. Social Media Metrics Image: Danny Brown, The State of Social Media Marketing 2012
  • 22. Usage Based Metrics
  • 23. Click Map Analysis
  • 24. Network Analysis Clickstream data yields highresolution maps of science Johan Bollen, Herbert Van de Sompel, et al. PLoS One, February 2009
  • 25. Network Analysis Clickstream data yields highresolution maps of science Johan Bollen, Herbert Van de Sompel, et al. PLoS One, February 2009
  • 26. Behavioral Metrics
  • 27. Sentiment Analysis
  • 28. Non-traditional content Assessing impact of 
 new forms of communication isn’t as simple as one might think. Image: Domenico, Caron, Davis, et al.
  • 29. Non-traditional content Assessing impact of 
 new forms of communication isn’t as simple as one might think. Image: Domenico, Caron, Davis, et al.
  • 30. We’ve got what, now how?
  • 31. What can be measured? Image: Flickr user karindalziel
  • 32. How do we 
 measure things?
  • 33. At what granularity do we measure things?
  • 34. Consistency in what is counted Source:  Scott  Chamberlain,  Consuming  Article-­‐Level  Metrics: Observations  And  Lessons  From  Comparing  Aggregator  Provider  Data,  Information  Standards  Quarterly,  Summer  2013,  Vol  25,  Issue  2.
  • 35. Which measurements are valuable?
  • 36. How do we ensure the validity of those measurements?
  • 37. How to (or even if we should) deal with other issues such as gaming?
  • 38. What infrastructure do we need for alternative metrics?
  • 39. Basic Definitions ! (So we are all talking about the same thing)
  • 40. Comparison across providers Source:  Scott  Chamberlain,  Consuming  Article-­‐Level  Metrics: Observations  And  Lessons  From  Comparing  Aggregator  Provider  Data,
  Information  Standards  Quarterly,  Summer  2013,  Vol  25,  Issue  2.
  • 41. Authorship ----Disambiguation & Contributor Roles
  • 42. Element Identification
  • 43. Open exchange of component data
  • 44. = TRUST
  • 45. Where does NISO fit in?
  • 46. Alterna(ve  Assessment   Ini(a(ve   ! Phase  1  Mee(ngs October  9,  2013    -­‐  San  Francisco,  CA December  11,  2013  -­‐  Washington,  DC January  23-­‐24  -­‐  Philadelphia,  PA ! Phase  1  report  expected  in  May  2014
  • 47. Just  a  few  ideas  from  in-­‐person  mee(ngs   Disambigua)on  problems   Create  a  central  repository  for  data   Define  provenance  for  metrics   Foster  culture  of  DOI  use  in  mass  media   Establish  defini)ons  for  metric  types   Define  data  extract/gathering  methods   Standardize  units  of  measure   Define  levels  of  transparency/privacy   !
  • 48. Alterna(ve  Assessment  Ini(a(ve   ! Phase  2 Presenta(ons  of  report  (June  2014) Priori(za(on  Effort  (June  -­‐  Aug,  2014) Project  approval  (Sept  2014) Working  group  forma(on  (Oct  2014) Consensus  Development  (Nov  2014  -­‐  Dec  2015) Trial  Use  Period  (Dec  15  -­‐  Mar  16) Publica(on  of  final  recommenda(ons  (Jun  16)
  • 49. May 15, 2013 74
  • 50. Thank you! Todd Carpenter, Executive Director tcarpenter@niso.org @TAC_NISO ! National Information Standards Organization (NISO) 3600 Clipper Mill Road, Suite 302 Baltimore, MD 21211 USA +1 (301) 654-2512 www.niso.org