Todd Carpenter's Update on the NISO Alternative assessment metrics Initiative at CNI Fall meeting in Washington, DC on December 9, 2013.

1. Do We Need Standards
for
Alternative Metrics?
What is NISO hoping to
accomplish?
Todd Carpenter, Executive Director, NISO
NISO Alternative Assessment Initiative
CNI Fall Project Briefing
Washington, DC | December 9, 2013

2. About
•
Non-profit industry trade association accredited
by ANSI with 150+ members
•
Mission of developing and maintaining technical
standards related to information, documentation,
discovery and distribution of published materials
and media
•
Volunteer driven organization: 400+ spread out
across the world
•
Responsible in various wasy for standards like
ISSN, DOI, Dublin Core metadata, DAISY digital
talking books, OpenURL, MARC records, and ISBN

3. Phase 1
Discussion
Forums
NISO Alternative Assessment
Metrics Initiative
!
Brainstorming Meetings
October 9, 2013 // December 11, 2013 //
January 23, 2014
Washington, DC | San Francisco, CA | Philadelphia, PA

4. Define altmetrics,
please

5. Usage Based Metrics

6. Clickstream Analysis

7. Social Media Metrics
Image: Danny Brown, The State of Social Media Marketing 2012

8. Clickstream network analysis
Clickstream data yields highresolution maps of science
Johan Bollen, Herbert Van de Sompel, et al.
PLoS One, February 2009

9. Behavioral Metrics

10. Multivariate analysis

11. Sentiment Analysis

12. Citation of non-traditional content
Citations to new
forms of
communication
aren’t as simple
as one might
think.
!
Define a use of
this?
Image: Domenico, Caron, Davis, et al.

13. We’ve got what,
now how?

14. What can be measured?
Image: Flickr user karindalziel

15. How do we measure
things?

16. At what granularity do
we measure things?

17. Which measurements
are valuable?

18. How do we ensure the
validity of those
measurements?

19. How to
(or even if we should)
deal with other issues
such as gaming?

20. Issues we face with
alternative metrics

21. Basic Definitions
!
(So we are all talking
about the same thing)

22. Comparison across providers
Source:
Scott
Chamberlain,
Consuming
Article-­‐Level
Metrics:
Observations
And
Lessons
From
Comparing
Aggregator
Provider
Data,
Information
Standards
Quarterly,
Summer
2013,
Vol
25,
Issue
2.

23. Consistency in what is counted
Source:
Scott
Chamberlain,
Consuming
Article-­‐Level
Metrics:
Observations
And
Lessons
From
Comparing
Aggregator
Provider
Data,
Information
Standards
Quarterly,
Summer
2013,
Vol
25,
Issue
2.

24. Authorship
----Disambiguation
&
Contributor Roles

25. Open exchange of
component data

26. TRUST

27. What is NISO trying to do?

30. Alterna(ve
Assessment
Ini(a(ve
!
Phase
1
Mee(ngs
October
9,
2013
-­‐
San
Francisco,
CA
December
11,
2013
-­‐
Washington,
DC
January
23-­‐24
-­‐
Philadelphia,
PA
!
Phase
1
report
expected
in
May
2014

31. Alterna(ve
Assessment
Ini(a(ve
!
Phase
2
Presenta(ons
of
report
(June
2014)
Priori(za(on
Effort
(June
-­‐
Aug,
2014)
Project
approval
(Sept
2014)
Working
group
forma(on
(Oct
2014)
Consensus
Development
(Nov
2014
-­‐
Dec
2015)
Trial
Use
Period
(Dec
15
-­‐
Mar
16)
Publica(on
of
final
recommenda(ons
(Jun
16)

32. Criticism

33. Why now?
Isn’t it too soon?
Can NISO to do this?

34. How soon is now?

35. These things take time

36. It will be hard to change

37. Capital “S” Standards
vs
standards
(i.e., best practice)

38. “All I see now is
blonde, brunette,
redhead”

39. ISQ Altmetrics Issue
Now Available Open Access!
www.niso.org/publications/isq/

40. One Very Last Thing…
Image from Will Lion: http://www.flickr.com/photos/will-lion/2646213692/

41. Thank you!
Todd Carpenter, Executive Director
tcarpenter@niso.org
National Information Standards Organization (NISO)
3600 Clipper Mill Road, Suite 302
Baltimore, MD 21211 USA
+1 (301) 654-2512
www.niso.org