Do We Need Standards
for
Alternative Metrics?
What is NISO hoping to
accomplish?
Todd Carpenter, Executive Director, NISO	...
About
•

Non-profit industry trade association accredited
by ANSI with 150+ members	


•

Mission of developing and maintai...
Phase 1
Discussion
Forums

NISO Alternative Assessment 

Metrics Initiative
!

Brainstorming Meetings 	

October 9, 2013 /...
Define altmetrics,
please
Usage Based Metrics
Clickstream Analysis
Social Media Metrics

Image: Danny Brown, The State of Social Media Marketing 2012
Clickstream network analysis
Clickstream data yields highresolution maps of science
Johan Bollen, Herbert Van de Sompel, e...
Behavioral Metrics
Multivariate analysis
Sentiment Analysis
Citation of non-traditional content
Citations to new
forms of
communication
aren’t as simple
as one might
think.	

!

Defin...
We’ve got what,
now how?
What can be measured?

Image: Flickr user karindalziel
How do we measure
things?
At what granularity do
we measure things?
Which measurements
are valuable?
How do we ensure the
validity of those
measurements?
How to
(or even if we should)
deal with other issues
such as gaming?
Issues we face with
alternative metrics
Basic Definitions
!

(So we are all talking
about the same thing)
Comparison across providers

Source:	
  Scott	
  Chamberlain,	
  Consuming	
  Article-­‐Level	
  Metrics:
Observations	
  ...
Consistency in what is counted

Source:	
  Scott	
  Chamberlain,	
  Consuming	
  Article-­‐Level	
  Metrics:
Observations	...
Authorship
----Disambiguation
&
Contributor Roles
Open exchange of
component data
TRUST
What is NISO trying to do?
Alterna(ve	
  Assessment	
  
Ini(a(ve	
  
!

Phase	
  1	
  Mee(ngs
October	
  9,	
  2013	
  	
  -­‐	
  San	
  Francisco,	
...
Alterna(ve	
  Assessment	
  Ini(a(ve	
  
!

Phase	
  2
Presenta(ons	
  of	
  report	
  (June	
  2014)
Priori(za(on	
  Effor...
Criticism
Why now?
Isn’t it too soon?
Can NISO to do this?
How soon is now?
These things take time
It will be hard to change
Capital “S” Standards
vs
standards
(i.e., best practice)
“All I see now is
blonde, brunette,
redhead”
ISQ Altmetrics Issue
Now Available Open Access!

www.niso.org/publications/isq/
One Very Last Thing…

Image from Will Lion: http://www.flickr.com/photos/will-lion/2646213692/
Thank you!
Todd Carpenter, Executive Director	

tcarpenter@niso.org	


National Information Standards Organization (NISO)	...
Carpenter Update on NISO Altmetrics Initiative at CNI Fall meeting in Washington, DC
Carpenter Update on NISO Altmetrics Initiative at CNI Fall meeting in Washington, DC
Upcoming SlideShare
Loading in...5
×

Carpenter Update on NISO Altmetrics Initiative at CNI Fall meeting in Washington, DC

907

Published on

Todd Carpenter's Update on the NISO Alternative assessment metrics Initiative at CNI Fall meeting in Washington, DC on December 9, 2013.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
907
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
7
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Carpenter Update on NISO Altmetrics Initiative at CNI Fall meeting in Washington, DC

  1. 1. Do We Need Standards for Alternative Metrics? What is NISO hoping to accomplish? Todd Carpenter, Executive Director, NISO NISO Alternative Assessment Initiative CNI Fall Project Briefing Washington, DC | December 9, 2013
  2. 2. About • Non-profit industry trade association accredited by ANSI with 150+ members • Mission of developing and maintaining technical standards related to information, documentation, discovery and distribution of published materials and media • Volunteer driven organization: 400+ spread out across the world • Responsible in various wasy for standards like ISSN, DOI, Dublin Core metadata, DAISY digital talking books, OpenURL, MARC records, and ISBN
  3. 3. Phase 1 Discussion Forums NISO Alternative Assessment 
 Metrics Initiative ! Brainstorming Meetings October 9, 2013 // December 11, 2013 // January 23, 2014 Washington, DC | San Francisco, CA | Philadelphia, PA
  4. 4. Define altmetrics, please
  5. 5. Usage Based Metrics
  6. 6. Clickstream Analysis
  7. 7. Social Media Metrics Image: Danny Brown, The State of Social Media Marketing 2012
  8. 8. Clickstream network analysis Clickstream data yields highresolution maps of science Johan Bollen, Herbert Van de Sompel, et al. PLoS One, February 2009
  9. 9. Behavioral Metrics
  10. 10. Multivariate analysis
  11. 11. Sentiment Analysis
  12. 12. Citation of non-traditional content Citations to new forms of communication aren’t as simple as one might think. ! Define a use of this? Image: Domenico, Caron, Davis, et al.
  13. 13. We’ve got what, now how?
  14. 14. What can be measured? Image: Flickr user karindalziel
  15. 15. How do we measure things?
  16. 16. At what granularity do we measure things?
  17. 17. Which measurements are valuable?
  18. 18. How do we ensure the validity of those measurements?
  19. 19. How to (or even if we should) deal with other issues such as gaming?
  20. 20. Issues we face with alternative metrics
  21. 21. Basic Definitions ! (So we are all talking about the same thing)
  22. 22. Comparison across providers Source:  Scott  Chamberlain,  Consuming  Article-­‐Level  Metrics: Observations  And  Lessons  From  Comparing  Aggregator  Provider  Data,   Information  Standards  Quarterly,  Summer  2013,  Vol  25,  Issue  2.
  23. 23. Consistency in what is counted Source:  Scott  Chamberlain,  Consuming  Article-­‐Level  Metrics: Observations  And  Lessons  From  Comparing  Aggregator  Provider  Data,   Information  Standards  Quarterly,  Summer  2013,  Vol  25,  Issue  2.
  24. 24. Authorship ----Disambiguation & Contributor Roles
  25. 25. Open exchange of component data
  26. 26. TRUST
  27. 27. What is NISO trying to do?
  28. 28. Alterna(ve  Assessment   Ini(a(ve   ! Phase  1  Mee(ngs October  9,  2013    -­‐  San  Francisco,  CA December  11,  2013  -­‐  Washington,  DC January  23-­‐24  -­‐  Philadelphia,  PA ! Phase  1  report  expected  in  May  2014
  29. 29. Alterna(ve  Assessment  Ini(a(ve   ! Phase  2 Presenta(ons  of  report  (June  2014) Priori(za(on  Effort  (June  -­‐  Aug,  2014) Project  approval  (Sept  2014) Working  group  forma(on  (Oct  2014) Consensus  Development  (Nov  2014  -­‐  Dec  2015) Trial  Use  Period  (Dec  15  -­‐  Mar  16) Publica(on  of  final  recommenda(ons  (Jun  16)
  30. 30. Criticism
  31. 31. Why now? Isn’t it too soon? Can NISO to do this?
  32. 32. How soon is now?
  33. 33. These things take time
  34. 34. It will be hard to change
  35. 35. Capital “S” Standards vs standards (i.e., best practice)
  36. 36. “All I see now is blonde, brunette, redhead”
  37. 37. ISQ Altmetrics Issue Now Available Open Access! www.niso.org/publications/isq/
  38. 38. One Very Last Thing… Image from Will Lion: http://www.flickr.com/photos/will-lion/2646213692/
  39. 39. Thank you! Todd Carpenter, Executive Director tcarpenter@niso.org National Information Standards Organization (NISO) 3600 Clipper Mill Road, Suite 302 Baltimore, MD 21211 USA +1 (301) 654-2512 www.niso.org
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×