• Like

Alternative assessment metrics initiateve meeting 1 opening

  • 340 views
Uploaded on

Todd Carpenter's presentation at the opening of the Alternative Assessment discussion forum in San Francisco, CA on 10/09/13

Todd Carpenter's presentation at the opening of the Alternative Assessment discussion forum in San Francisco, CA on 10/09/13

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
340
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
3
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Do We Need Standards for Alternative Metrics? What are we hoping to accomplish? Todd Carpenter, Executive Director, NISO NISO Alternative Assessment Initiative Discussion Forum #1 San Francisco, CA | October 9, 2013 Friday, October 11, 13
  • 2. • Non-profit industry trade association accredited by ANSI with 150+ members • Mission of developing and maintaining technical standards related to information, documentation, discovery and distribution of published materials and media • Volunteer driven organization: 400+ spread out across the world • Responsible in various wasy for standards like ISSN, DOI, Dublin Core metadata, DAISY digital talking books, OpenURL, MARC records, and ISBN About Friday, October 11, 13
  • 3. We stand on the shoulders of giants... Friday, October 11, 13
  • 4. The Mighty Citation Friday, October 11, 13
  • 5. Impact factor Friday, October 11, 13
  • 6. Thomson-Reuters IMPACT FACTOR Friday, October 11, 13
  • 7. Two Stanford Students Friday, October 11, 13
  • 8. They wrote a paper Friday, October 11, 13
  • 9. They started a company in someone’s garage Photo:AP Friday, October 11, 13
  • 10. Friday, October 11, 13
  • 11. Citations are awesome, aren’t they? A perfect solution to all the world’s problems, no? A simple conclusion can be drawn Friday, October 11, 13
  • 12. “What Des-Cartes did was a good step.You have added much several ways, & especially in taking ye colours of thin plates into philosophical consideration. If I have seen further it is by standing on ye shoulders of Giants." Back to Sir Isaac Newton Friday, October 11, 13
  • 13. Citations aren’t always what we think they are Caveat emptor Friday, October 11, 13
  • 14. Another problem with citations Friday, October 11, 13
  • 15. So where to from here? Friday, October 11, 13
  • 16. Define altmetrics, please Friday, October 11, 13
  • 17. Usage Based Metrics Friday, October 11, 13
  • 18. Clickstream Analysis Friday, October 11, 13
  • 19. Social Media Metrics Image: Danny Brown,The State of Social Media Marketing 2012 Friday, October 11, 13
  • 20. Clickstream data yields high- resolution maps of science Johan Bollen, HerbertVan de Sompel, et al. PLoS One, February 2009 Clickstream network analysis Friday, October 11, 13
  • 21. Clickstream data yields high- resolution maps of science Johan Bollen, HerbertVan de Sompel, et al. PLoS One, February 2009 Clickstream network analysis Friday, October 11, 13
  • 22. Behavioral Metrics Friday, October 11, 13
  • 23. Multivariate analysis Friday, October 11, 13
  • 24. Sentiment Analysis Friday, October 11, 13
  • 25. Image: Domenico, Caron, Davis, et al. Citation of non-traditional content Citations to new forms of communication aren’t as simple as one might think. Define a use of this? Friday, October 11, 13
  • 26. We’ve got what, now how? Friday, October 11, 13
  • 27. Image: Flickr user karindalziel What can be measured? Friday, October 11, 13
  • 28. How do we measure things? Friday, October 11, 13
  • 29. At what granularity do we measure things? Friday, October 11, 13
  • 30. Which measurements are valuable? Friday, October 11, 13
  • 31. How do we ensure the validity of those measurements? Friday, October 11, 13
  • 32. How to (or even if we should) deal with other issues such as gaming? Friday, October 11, 13
  • 33. Issues we face with alternative metrics Friday, October 11, 13
  • 34. Basic Definitions (So we are all talking about the same thing) Friday, October 11, 13
  • 35. Comparison across providers Source:  Scott  Chamberlain,  Consuming  Article-­‐Level  Metrics: Observations  And  Lessons  From  Comparing  Aggregator  Provider  Data,   Information  Standards  Quarterly,  Summer  2013,  Vol  25,  Issue  2. Friday, October 11, 13
  • 36. Consistency in what is counted Source:  Scott  Chamberlain,  Consuming  Article-­‐Level  Metrics: Observations  And  Lessons  From  Comparing  Aggregator  Provider  Data,   Information  Standards  Quarterly,  Summer  2013,  Vol  25,  Issue  2. Friday, October 11, 13
  • 37. Authorship ----- Disambiguation & Contributor Roles Friday, October 11, 13
  • 38. Open exchange of component data Friday, October 11, 13
  • 39. TRUST Friday, October 11, 13
  • 40. What is NISO trying to do? Friday, October 11, 13
  • 41. Friday, October 11, 13
  • 42. Friday, October 11, 13
  • 43. Alterna(ve  Assessment   Ini(a(ve   Phase  1  Mee(ngs October  9,  2013    -­‐  San  Francisco,  CA December  11,  2013  -­‐  Washington,  DC January  23-­‐24  -­‐  Philadelphia,  PA Phase  1  report  expected  in  May  2014 Friday, October 11, 13
  • 44. Alterna(ve  Assessment  Ini(a(ve   Phase  2 Presenta(ons  of  report  (June  2014) Priori(za(on  Effort  (June  -­‐  Aug,  2014) Project  approval  (Sept  2014) Working  group  forma(on  (Oct  2014) Consensus  Development  (Nov  2014  -­‐  Dec  2015) Trial  Use  Period  (Dec  15  -­‐  Mar  16) Publica(on  of  final  recommenda(ons  (Jun  16) Friday, October 11, 13
  • 45. Criticism Friday, October 11, 13
  • 46. Why now? Isn’t it too soon? Can NISO to do this? Friday, October 11, 13
  • 47. How soon is now? Friday, October 11, 13
  • 48. These things take time Friday, October 11, 13
  • 49. It will be hard to change Friday, October 11, 13
  • 50. Friday, October 11, 13
  • 51. “All I see now is blonde, brunette, redhead” Friday, October 11, 13
  • 52. One Very Last Thing… Image from Will Lion: http://www.flickr.com/photos/will-lion/2646213692/ Friday, October 11, 13
  • 53. Thank you! Todd Carpenter, Executive Director tcarpenter@niso.org National Information Standards Organization (NISO) 3600 Clipper Mill Road, Suite 302 Baltimore, MD 21211 USA +1 (301) 654-2512 www.niso.org Friday, October 11, 13