• Share
  • Email
  • Embed
  • Like
  • Private Content
The Infrastructure for Alternative Metrics
 

The Infrastructure for Alternative Metrics

on

  • 757 views

The Infrastructure for Alternative Metrics: What do we need to compare digital apples to digital apples? ...

The Infrastructure for Alternative Metrics: What do we need to compare digital apples to digital apples?
Todd Carpenter, Executive Director, NISO
Presented at AAAS Annual Meeting: February 16, 2014
Chicago, IL

Statistics

Views

Total Views
757
Views on SlideShare
439
Embed Views
318

Actions

Likes
0
Downloads
4
Comments
0

15 Embeds 318

http://www.infodocket.com 165
http://www.niso.org 72
http://feedly.com 37
http://www.scoop.it 16
http://www.newsblur.com 10
http://reader.aol.com 4
http://www.feedspot.com 3
http://127.0.0.1 3
http://digg.com 2
http://newsblur.com 1
http://www.inoreader.com 1
http://wordpress.com 1
http://reader.delooper.net 1
http://www.hanrss.com 1
https://www.commafeed.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    The Infrastructure for Alternative Metrics The Infrastructure for Alternative Metrics Presentation Transcript

    • The Infrastructure for Alternative Metrics What do we need to compare digital apples to digital apples? ! Todd Carpenter, Executive Director, NISO AAAS Annual Meeting Chicago, IL | February 16, 2014
    • We hate discussing infrastructure
    • Photo: Minneapolis College
 of Art and Design Library
    • About • Non-profit industry trade association accredited by ANSI with 190+ members • Mission of developing and maintaining technical standards related to information, documentation, discovery and distribution of published materials and media • Volunteer driven organization: 400+ spread out across the world • Responsible in various wasy for standards like ISSN, DOI, Dublin Core metadata, DAISY digital talking books, OpenURL, MARC records, and ISBN
    • Photo by dumbledad on Flickr http://www.flickr.com/photos/dumbledad/298650884/!
    • But what is the “infrastructure” for assessment?
    • We stand on the shoulders of giants...
    • The Mighty Citation
    • Impact factor
    • Thomson-Reuters 
 IMPACT FACTOR
    • Two Stanford Students
    • Two Stanford Students
    • They wrote a paper
    • They started a company in someone’s garage Photo: AP
    • A simple conclusion can be drawn Citations are awesome, aren’t they? A perfect solution to all the world’s problems, no?
    • Back to Sir Isaac Newton “What Des-Cartes did was a good step.You have added much several ways, & especially in taking ye colours of thin plates into philosophical consideration. If I have seen further it is by standing on ye shoulders of Giants."
    • Caveat emptor Citations aren’t always what we think they are
    • Another problem with citations
    • So where to from here?
    • Define altmetrics, please
    • Social Media Metrics Image: Danny Brown, The State of Social Media Marketing 2012
    • Usage Based Metrics
    • Click Map Analysis
    • Network Analysis Clickstream data yields highresolution maps of science Johan Bollen, Herbert Van de Sompel, et al. PLoS One, February 2009
    • Network Analysis Clickstream data yields highresolution maps of science Johan Bollen, Herbert Van de Sompel, et al. PLoS One, February 2009
    • Behavioral Metrics
    • Sentiment Analysis
    • Non-traditional content Assessing impact of 
 new forms of communication isn’t as simple as one might think. Image: Domenico, Caron, Davis, et al.
    • Non-traditional content Assessing impact of 
 new forms of communication isn’t as simple as one might think. Image: Domenico, Caron, Davis, et al.
    • We’ve got what, now how?
    • What can be measured? Image: Flickr user karindalziel
    • How do we 
 measure things?
    • At what granularity do we measure things?
    • Consistency in what is counted Source:  Scott  Chamberlain,  Consuming  Article-­‐Level  Metrics: Observations  And  Lessons  From  Comparing  Aggregator  Provider  Data,  Information  Standards  Quarterly,  Summer  2013,  Vol  25,  Issue  2.
    • Which measurements are valuable?
    • How do we ensure the validity of those measurements?
    • How to (or even if we should) deal with other issues such as gaming?
    • What infrastructure do we need for alternative metrics?
    • Basic Definitions ! (So we are all talking about the same thing)
    • Comparison across providers Source:  Scott  Chamberlain,  Consuming  Article-­‐Level  Metrics: Observations  And  Lessons  From  Comparing  Aggregator  Provider  Data,
  Information  Standards  Quarterly,  Summer  2013,  Vol  25,  Issue  2.
    • Authorship ----Disambiguation & Contributor Roles
    • Element Identification
    • Open exchange of component data
    • = TRUST
    • Where does NISO fit in?
    • Alterna(ve  Assessment   Ini(a(ve   ! Phase  1  Mee(ngs October  9,  2013    -­‐  San  Francisco,  CA December  11,  2013  -­‐  Washington,  DC January  23-­‐24  -­‐  Philadelphia,  PA ! Phase  1  report  expected  in  May  2014
    • Just  a  few  ideas  from  in-­‐person  mee(ngs   Disambigua)on  problems   Create  a  central  repository  for  data   Define  provenance  for  metrics   Foster  culture  of  DOI  use  in  mass  media   Establish  defini)ons  for  metric  types   Define  data  extract/gathering  methods   Standardize  units  of  measure   Define  levels  of  transparency/privacy   !
    • Alterna(ve  Assessment  Ini(a(ve   ! Phase  2 Presenta(ons  of  report  (June  2014) Priori(za(on  Effort  (June  -­‐  Aug,  2014) Project  approval  (Sept  2014) Working  group  forma(on  (Oct  2014) Consensus  Development  (Nov  2014  -­‐  Dec  2015) Trial  Use  Period  (Dec  15  -­‐  Mar  16) Publica(on  of  final  recommenda(ons  (Jun  16)
    • May 15, 2013 74
    • Thank you! Todd Carpenter, Executive Director tcarpenter@niso.org @TAC_NISO ! National Information Standards Organization (NISO) 3600 Clipper Mill Road, Suite 302 Baltimore, MD 21211 USA +1 (301) 654-2512 www.niso.org