Alternative assessment metrics initiateve meeting 1 opening

Like this? Share it with your network

Share

Alternative assessment metrics initiateve meeting 1 opening

  • 617 views
Uploaded on

Todd Carpenter's presentation at the opening of the Alternative Assessment discussion forum in San Francisco, CA on 10/09/13

Todd Carpenter's presentation at the opening of the Alternative Assessment discussion forum in San Francisco, CA on 10/09/13

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
617
On Slideshare
610
From Embeds
7
Number of Embeds
1

Actions

Shares
Downloads
3
Comments
0
Likes
1

Embeds 7

https://twitter.com 7

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Do We Need Standards for Alternative Metrics? What are we hoping to accomplish? Todd Carpenter, Executive Director, NISO NISO Alternative Assessment Initiative Discussion Forum #1 San Francisco, CA | October 9, 2013 Friday, October 11, 13
  • 2. • Non-profit industry trade association accredited by ANSI with 150+ members • Mission of developing and maintaining technical standards related to information, documentation, discovery and distribution of published materials and media • Volunteer driven organization: 400+ spread out across the world • Responsible in various wasy for standards like ISSN, DOI, Dublin Core metadata, DAISY digital talking books, OpenURL, MARC records, and ISBN About Friday, October 11, 13
  • 3. We stand on the shoulders of giants... Friday, October 11, 13
  • 4. The Mighty Citation Friday, October 11, 13
  • 5. Impact factor Friday, October 11, 13
  • 6. Thomson-Reuters IMPACT FACTOR Friday, October 11, 13
  • 7. Two Stanford Students Friday, October 11, 13
  • 8. They wrote a paper Friday, October 11, 13
  • 9. They started a company in someone’s garage Photo:AP Friday, October 11, 13
  • 10. Friday, October 11, 13
  • 11. Citations are awesome, aren’t they? A perfect solution to all the world’s problems, no? A simple conclusion can be drawn Friday, October 11, 13
  • 12. “What Des-Cartes did was a good step.You have added much several ways, & especially in taking ye colours of thin plates into philosophical consideration. If I have seen further it is by standing on ye shoulders of Giants." Back to Sir Isaac Newton Friday, October 11, 13
  • 13. Citations aren’t always what we think they are Caveat emptor Friday, October 11, 13
  • 14. Another problem with citations Friday, October 11, 13
  • 15. So where to from here? Friday, October 11, 13
  • 16. Define altmetrics, please Friday, October 11, 13
  • 17. Usage Based Metrics Friday, October 11, 13
  • 18. Clickstream Analysis Friday, October 11, 13
  • 19. Social Media Metrics Image: Danny Brown,The State of Social Media Marketing 2012 Friday, October 11, 13
  • 20. Clickstream data yields high- resolution maps of science Johan Bollen, HerbertVan de Sompel, et al. PLoS One, February 2009 Clickstream network analysis Friday, October 11, 13
  • 21. Clickstream data yields high- resolution maps of science Johan Bollen, HerbertVan de Sompel, et al. PLoS One, February 2009 Clickstream network analysis Friday, October 11, 13
  • 22. Behavioral Metrics Friday, October 11, 13
  • 23. Multivariate analysis Friday, October 11, 13
  • 24. Sentiment Analysis Friday, October 11, 13
  • 25. Image: Domenico, Caron, Davis, et al. Citation of non-traditional content Citations to new forms of communication aren’t as simple as one might think. Define a use of this? Friday, October 11, 13
  • 26. We’ve got what, now how? Friday, October 11, 13
  • 27. Image: Flickr user karindalziel What can be measured? Friday, October 11, 13
  • 28. How do we measure things? Friday, October 11, 13
  • 29. At what granularity do we measure things? Friday, October 11, 13
  • 30. Which measurements are valuable? Friday, October 11, 13
  • 31. How do we ensure the validity of those measurements? Friday, October 11, 13
  • 32. How to (or even if we should) deal with other issues such as gaming? Friday, October 11, 13
  • 33. Issues we face with alternative metrics Friday, October 11, 13
  • 34. Basic Definitions (So we are all talking about the same thing) Friday, October 11, 13
  • 35. Comparison across providers Source:  Scott  Chamberlain,  Consuming  Article-­‐Level  Metrics: Observations  And  Lessons  From  Comparing  Aggregator  Provider  Data,   Information  Standards  Quarterly,  Summer  2013,  Vol  25,  Issue  2. Friday, October 11, 13
  • 36. Consistency in what is counted Source:  Scott  Chamberlain,  Consuming  Article-­‐Level  Metrics: Observations  And  Lessons  From  Comparing  Aggregator  Provider  Data,   Information  Standards  Quarterly,  Summer  2013,  Vol  25,  Issue  2. Friday, October 11, 13
  • 37. Authorship ----- Disambiguation & Contributor Roles Friday, October 11, 13
  • 38. Open exchange of component data Friday, October 11, 13
  • 39. TRUST Friday, October 11, 13
  • 40. What is NISO trying to do? Friday, October 11, 13
  • 41. Friday, October 11, 13
  • 42. Friday, October 11, 13
  • 43. Alterna(ve  Assessment   Ini(a(ve   Phase  1  Mee(ngs October  9,  2013    -­‐  San  Francisco,  CA December  11,  2013  -­‐  Washington,  DC January  23-­‐24  -­‐  Philadelphia,  PA Phase  1  report  expected  in  May  2014 Friday, October 11, 13
  • 44. Alterna(ve  Assessment  Ini(a(ve   Phase  2 Presenta(ons  of  report  (June  2014) Priori(za(on  Effort  (June  -­‐  Aug,  2014) Project  approval  (Sept  2014) Working  group  forma(on  (Oct  2014) Consensus  Development  (Nov  2014  -­‐  Dec  2015) Trial  Use  Period  (Dec  15  -­‐  Mar  16) Publica(on  of  final  recommenda(ons  (Jun  16) Friday, October 11, 13
  • 45. Criticism Friday, October 11, 13
  • 46. Why now? Isn’t it too soon? Can NISO to do this? Friday, October 11, 13
  • 47. How soon is now? Friday, October 11, 13
  • 48. These things take time Friday, October 11, 13
  • 49. It will be hard to change Friday, October 11, 13
  • 50. Friday, October 11, 13
  • 51. “All I see now is blonde, brunette, redhead” Friday, October 11, 13
  • 52. One Very Last Thing… Image from Will Lion: http://www.flickr.com/photos/will-lion/2646213692/ Friday, October 11, 13
  • 53. Thank you! Todd Carpenter, Executive Director tcarpenter@niso.org National Information Standards Organization (NISO) 3600 Clipper Mill Road, Suite 302 Baltimore, MD 21211 USA +1 (301) 654-2512 www.niso.org Friday, October 11, 13