Alternative assessment metrics initiateve meeting 1 opening
 

Alternative assessment metrics initiateve meeting 1 opening

on

  • 529 views

Todd Carpenter's presentation at the opening of the Alternative Assessment discussion forum in San Francisco, CA on 10/09/13

Todd Carpenter's presentation at the opening of the Alternative Assessment discussion forum in San Francisco, CA on 10/09/13

Statistics

Views

Total Views
529
Views on SlideShare
524
Embed Views
5

Actions

Likes
1
Downloads
3
Comments
0

1 Embed 5

https://twitter.com 5

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Alternative assessment metrics initiateve meeting 1 opening Alternative assessment metrics initiateve meeting 1 opening Presentation Transcript

    • Do We Need Standards for Alternative Metrics? What are we hoping to accomplish? Todd Carpenter, Executive Director, NISO NISO Alternative Assessment Initiative Discussion Forum #1 San Francisco, CA | October 9, 2013 Friday, October 11, 13
    • • Non-profit industry trade association accredited by ANSI with 150+ members • Mission of developing and maintaining technical standards related to information, documentation, discovery and distribution of published materials and media • Volunteer driven organization: 400+ spread out across the world • Responsible in various wasy for standards like ISSN, DOI, Dublin Core metadata, DAISY digital talking books, OpenURL, MARC records, and ISBN About Friday, October 11, 13
    • We stand on the shoulders of giants... Friday, October 11, 13
    • The Mighty Citation Friday, October 11, 13
    • Impact factor Friday, October 11, 13
    • Thomson-Reuters IMPACT FACTOR Friday, October 11, 13
    • Two Stanford Students Friday, October 11, 13
    • They wrote a paper Friday, October 11, 13
    • They started a company in someone’s garage Photo:AP Friday, October 11, 13
    • Friday, October 11, 13
    • Citations are awesome, aren’t they? A perfect solution to all the world’s problems, no? A simple conclusion can be drawn Friday, October 11, 13
    • “What Des-Cartes did was a good step.You have added much several ways, & especially in taking ye colours of thin plates into philosophical consideration. If I have seen further it is by standing on ye shoulders of Giants." Back to Sir Isaac Newton Friday, October 11, 13
    • Citations aren’t always what we think they are Caveat emptor Friday, October 11, 13
    • Another problem with citations Friday, October 11, 13
    • So where to from here? Friday, October 11, 13
    • Define altmetrics, please Friday, October 11, 13
    • Usage Based Metrics Friday, October 11, 13
    • Clickstream Analysis Friday, October 11, 13
    • Social Media Metrics Image: Danny Brown,The State of Social Media Marketing 2012 Friday, October 11, 13
    • Clickstream data yields high- resolution maps of science Johan Bollen, HerbertVan de Sompel, et al. PLoS One, February 2009 Clickstream network analysis Friday, October 11, 13
    • Clickstream data yields high- resolution maps of science Johan Bollen, HerbertVan de Sompel, et al. PLoS One, February 2009 Clickstream network analysis Friday, October 11, 13
    • Behavioral Metrics Friday, October 11, 13
    • Multivariate analysis Friday, October 11, 13
    • Sentiment Analysis Friday, October 11, 13
    • Image: Domenico, Caron, Davis, et al. Citation of non-traditional content Citations to new forms of communication aren’t as simple as one might think. Define a use of this? Friday, October 11, 13
    • We’ve got what, now how? Friday, October 11, 13
    • Image: Flickr user karindalziel What can be measured? Friday, October 11, 13
    • How do we measure things? Friday, October 11, 13
    • At what granularity do we measure things? Friday, October 11, 13
    • Which measurements are valuable? Friday, October 11, 13
    • How do we ensure the validity of those measurements? Friday, October 11, 13
    • How to (or even if we should) deal with other issues such as gaming? Friday, October 11, 13
    • Issues we face with alternative metrics Friday, October 11, 13
    • Basic Definitions (So we are all talking about the same thing) Friday, October 11, 13
    • Comparison across providers Source:  Scott  Chamberlain,  Consuming  Article-­‐Level  Metrics: Observations  And  Lessons  From  Comparing  Aggregator  Provider  Data,   Information  Standards  Quarterly,  Summer  2013,  Vol  25,  Issue  2. Friday, October 11, 13
    • Consistency in what is counted Source:  Scott  Chamberlain,  Consuming  Article-­‐Level  Metrics: Observations  And  Lessons  From  Comparing  Aggregator  Provider  Data,   Information  Standards  Quarterly,  Summer  2013,  Vol  25,  Issue  2. Friday, October 11, 13
    • Authorship ----- Disambiguation & Contributor Roles Friday, October 11, 13
    • Open exchange of component data Friday, October 11, 13
    • TRUST Friday, October 11, 13
    • What is NISO trying to do? Friday, October 11, 13
    • Friday, October 11, 13
    • Friday, October 11, 13
    • Alterna(ve  Assessment   Ini(a(ve   Phase  1  Mee(ngs October  9,  2013    -­‐  San  Francisco,  CA December  11,  2013  -­‐  Washington,  DC January  23-­‐24  -­‐  Philadelphia,  PA Phase  1  report  expected  in  May  2014 Friday, October 11, 13
    • Alterna(ve  Assessment  Ini(a(ve   Phase  2 Presenta(ons  of  report  (June  2014) Priori(za(on  Effort  (June  -­‐  Aug,  2014) Project  approval  (Sept  2014) Working  group  forma(on  (Oct  2014) Consensus  Development  (Nov  2014  -­‐  Dec  2015) Trial  Use  Period  (Dec  15  -­‐  Mar  16) Publica(on  of  final  recommenda(ons  (Jun  16) Friday, October 11, 13
    • Criticism Friday, October 11, 13
    • Why now? Isn’t it too soon? Can NISO to do this? Friday, October 11, 13
    • How soon is now? Friday, October 11, 13
    • These things take time Friday, October 11, 13
    • It will be hard to change Friday, October 11, 13
    • Friday, October 11, 13
    • “All I see now is blonde, brunette, redhead” Friday, October 11, 13
    • One Very Last Thing… Image from Will Lion: http://www.flickr.com/photos/will-lion/2646213692/ Friday, October 11, 13
    • Thank you! Todd Carpenter, Executive Director tcarpenter@niso.org National Information Standards Organization (NISO) 3600 Clipper Mill Road, Suite 302 Baltimore, MD 21211 USA +1 (301) 654-2512 www.niso.org Friday, October 11, 13