Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Measuring Impact: Towards a data citation metric

697 views

Published on

How the ViBRANT and eMonocot projects are building tools, including a modified implementation of Bourne and Fink's 'Scholar Factor', the Biodiversity Data Journal, and Scratchpad's user metrics and statistics modules.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Measuring Impact: Towards a data citation metric

  1. 1. Measuring Impact Ed Baker & The Scratchpads Team
  2. 2. ViBRANT and eMonocot
  3. 3. ViBRANT and eMonocot • Projects that focus on data collection, curation and reuse
  4. 4. ViBRANT and eMonocot • Projects that focus on data collection, curation and reuse • Would like metrics to know how well we are doing
  5. 5. ViBRANT and eMonocot
  6. 6. ViBRANT and eMonocot Projects based around communities
  7. 7. ViBRANT and eMonocot Projects based around communities Provide tools for communities
  8. 8. ViBRANT and eMonocot Projects based around communities Provide tools for communities • Quantify user contribution to multi-user projects
  9. 9. ViBRANT and eMonocot Projects based around communities Provide tools for communities • Quantify user contribution to multi-user projects • Quantify how content generated by the community is used/reused
  10. 10. Citizen Science
  11. 11. Citizen Science • COMBER
  12. 12. Citizen Science • COMBER • anymals+plants
  13. 13. Citizen Science • COMBER • anymals+plants • Citizen Science profile
  14. 14. Citizen Science • COMBER • anymals+plants • Citizen Science profile Metrics allow for more competitive activities and for users to quantify their involvement
  15. 15. Who needs impact metrics?
  16. 16. Who needs impact metrics? • The authors of data
  17. 17. Who needs impact metrics? • The authors of data • People who employ data authors
  18. 18. Who needs impact metrics? • The authors of data • People who employ data authors • People who reuse the data
  19. 19. Who needs impact metrics? • The authors of data • People who employ data authors • People who reuse the data Am I useful?
  20. 20. Who needs impact metrics? • The authors of data Am I useful? • People who employ data authors • People who reuse the data Do we have our priorities right?
  21. 21. Who needs impact metrics? • The authors of data Am I useful? • People who employ data authors • People who reuse the data Do we have our priorities right?
  22. 22. Who needs impact metrics? • The authors of data Am I useful? • People who employ data authors • People who reuse the data Hidden gem? Do we have our priorities right?
  23. 23. Who needs impact metrics? • The authors of data Am I useful? • People who employ data authors • People who reuse the data Hidden gem? Avoid like the plague? Do we have our priorities right?
  24. 24. Our premise
  25. 25. We want people to share data…
  26. 26. …people want to please the “powers that be” …
  27. 27. … the powers that be like statistics.
  28. 28. So we made some
  29. 29. But the people weren’t happy.
  30. 30. But the people weren’t happy. Of all the things I do you only care about papers?
  31. 31. But the people weren’t happy. The only way you value my papers is by citation in other papers? Of all the things I do you only care about papers?
  32. 32. But the people weren’t happy. The only way you value my papers is by citation in other papers? Of all the things I do you only care about papers? The impact of my career is measured by one number?
  33. 33. So some people made their own
  34. 34. But there are still problems
  35. 35. But there are still problems This STILL only deals with my publications!
  36. 36. But there are still problems This STILL only deals with my publications! How do I get credit for data?
  37. 37. But there are still problems This STILL only deals with my publications! How do I get credit for data? I wrote some useful biodiversity software – what about that?
  38. 38. Are alt metrics useful?
  39. 39. Are alt metrics useful?
  40. 40. Are alt metrics useful? These are fairly scholarly
  41. 41. Are alt metrics useful? These measure broader public impact These are fairly scholarly
  42. 42. Are alt metrics useful? These measure broader public impact These are fairly scholarly • A good start
  43. 43. Are alt metrics useful? These measure broader public impact • A good start • Much broader than just scholarly citations These are fairly scholarly
  44. 44. Are alt metrics useful? These measure broader public impact • A good start • Much broader than just scholarly citations • Actually ignores traditional citations These are fairly scholarly
  45. 45. Are alt metrics useful? These measure broader public impact • A good start • Much broader than just scholarly citations • Actually ignores traditional citations These are fairly scholarly • Still only applied to papers
  46. 46. What are we doing?
  47. 47. What are we doing?
  48. 48. What are we doing?
  49. 49. What are we doing?
  50. 50. Biodiversity Data Journal Data papers (descriptions of data)
  51. 51. Biodiversity Data Journal Data papers (descriptions of data) • Allows data to become citable
  52. 52. Biodiversity Data Journal Data papers (descriptions of data) • Allows data to become citable • Allows data to participate in standard credit/metric systems
  53. 53. Biodiversity Data Journal Data papers (descriptions of data) • Allows data to become citable • Allows data to participate in standard credit/metric systems • … allows data to participate in alt metrics
  54. 54. Scratchpads Statistics Module What content exists on a Scratchpad site?
  55. 55. Scratchpads Statistics Module What content exists on a Scratchpad site? • Filter by user or taxonomic term
  56. 56. Scratchpads Statistics Module What content exists on a Scratchpad site? • Filter by user or taxonomic term • How much content is there?
  57. 57. Scratchpads Statistics Module What content exists on a Scratchpad site? • Filter by user or taxonomic term • How much content is there? • What kind of content is it?
  58. 58. Scratchpads Statistics Module What content exists on a Scratchpad site? • Filter by user or taxonomic term • How much content is there? • What kind of content is it? • How many registered users?
  59. 59. Scratchpads Statistics Module What content exists on a Scratchpad site? • Filter by user or taxonomic term • How much content is there? • What kind of content is it? • How many registered users? • How often is the content viewed?
  60. 60. Scratchpads Statistics Module Available now: http://antkey.org/stats [scratchpad.url]/stats
  61. 61. Scratchpads Metrics Module Puts the Scratchpad content in a broader context
  62. 62. Scratchpads Metrics Module Puts the Scratchpad content in a broader context • Users and content
  63. 63. Scratchpads Metrics Module Puts the Scratchpad content in a broader context • Users and content • Opt-in (not enabled by default)
  64. 64. Scratchpads Metrics Module Puts the Scratchpad content in a broader context • Users and content • Opt-in (not enabled by default) • Modular (pick and choose what to include)
  65. 65. Scratchpads Metrics Module Puts the Scratchpad content in a broader context • Users and content • Opt-in (not enabled by default) • Modular (pick and choose what to include) • Available by end of the year
  66. 66. Scratchpads User Metrics A partial implementation of the Scholar Factor proposed by Bourne and Fink
  67. 67. Scratchpads User Metrics A partial (modified) implementation of the Scholar Factor proposed by Bourne and Fink Citations
  68. 68. Scratchpads User Metrics A partial (modified) implementation of the Scholar Factor proposed by Bourne and Fink Citations Scholar Factor
  69. 69. Scratchpads User Metrics A partial (modified) implementation of the Scholar Factor proposed by Bourne and Fink Citations Software / Data Scholar Factor
  70. 70. Scratchpads User Metrics A partial (modified) implementation of the Scholar Factor proposed by Bourne and Fink Citations Software / Data Scholar Factor Grant / manuscript reviews
  71. 71. Scratchpads User Metrics A partial (modified) implementation of the Scholar Factor proposed by Bourne and Fink Citations Software / Data Scholar Factor Grant / manuscript reviews
  72. 72. Scratchpads User Metrics Scholar Factor
  73. 73. Scratchpads User Metrics Scratchpads User Metrics Module Scholar Factor
  74. 74. Scratchpads User Metrics Google Scholar (h-index) Scratchpads User Metrics Module Scholar Factor
  75. 75. Scratchpads User Metrics Google Scholar (h-index) Scratchpad Scratchpads User Metrics Module Scholar Factor
  76. 76. Scratchpads User Metrics Google Scholar (h-index) Scratchpad GitHub Scratchpads User Metrics Module Scholar Factor
  77. 77. Scratchpads User Metrics Google Scholar (h-index) Scratchpad GitHub Scratchpads User Metrics Module Scholar Factor ?
  78. 78. Scratchpads User Metrics Modular Google Scholar (h-index) Scratchpad GitHub Scratchpads User Metrics Module Scholar Factor ?
  79. 79. Scratchpads User Metrics Modular Google Scholar (h-index) Scratchpad GitHub Scratchpads User Metrics Module Scholar Factor ?
  80. 80. Scratchpads User Metrics Modular Google Scholar (h-index) Scratchpad GitHub Scratchpads User Metrics Module Scholar Factor ?
  81. 81. Same modular structure for content
  82. 82. Same modular structure for content • Number of web links to page
  83. 83. Same modular structure for content • Number of web links to page • Scholarly links (Google Scholar)
  84. 84. Same modular structure for content • Number of web links to page • Scholarly links (Google Scholar) • Social links
  85. 85. Summary We’re not there yet.
  86. 86. Summary We’re not there yet, but we have …
  87. 87. Summary We’re not there yet, but we have… • Modular framework to add services as they become available
  88. 88. Summary We’re not there yet, but we have… • Modular framework to add services as they become available • multi-dimensional – not just papers and citations
  89. 89. Summary We’re not there yet, but we have… • Modular framework to add services as they become available • multi-dimensional – not just papers and citations • Not a one size fits all approach
  90. 90. Summary We’re not there yet, but we have… • Modular framework to add services as they become available • multi-dimensional – not just papers and citations • Not a one size fits all approach

×