Download to read offline
![Scratchpads Statistics Module
Available now:
http://antkey.org/stats
[scratchpad.url]/stats](https://image.slidesharecdn.com/measuring-impact-131031053031-phpapp01/85/Measuring-Impact-Towards-a-data-citation-metric-60-320.jpg)
Uploaded byEdward Baker
Measuring Impact: Towards a data citation metric
The document discusses the need for impact metrics in data collection and sharing projects, particularly in community-driven initiatives like citizen science. It highlights the challenge of quantifying user contributions and the importance of recognizing various forms of impact beyond traditional academic citations. The Scratchpads metrics module is introduced as a tool to provide insights into content usage and user engagement, while emphasizing a modular and multi-dimensional approach to metrics.
More Related Content
![The Right Metrics for Generation Open [Open Access Week 2014]](https://cdn.slidesharecdn.com/ss_thumbnails/bpr2cyg7tkadlxkbr9lo-signature-49a229ccc192ecfdd279541f6016379be3d277ac873aafea168a8257eb161f85-poli-141023162319-conversion-gate01-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Measuring Impact: Towards a data citation metric
Measuring Impact: Towards a data citation metric
- 1. Measuring Impact Ed Baker& The Scratchpads Team
- 2.
- 3. ViBRANT and eMonocot •Projects that focus on data collection, curation and reuse
- 4. ViBRANT and eMonocot •Projects that focus on data collection, curation and reuse • Would like metrics to know how well we are doing
- 5.
- 6. ViBRANT and eMonocot Projectsbased around communities
- 7. ViBRANT and eMonocot Projectsbased around communities Provide tools for communities
- 8. ViBRANT and eMonocot Projectsbased around communities Provide tools for communities • Quantify user contribution to multi-user projects
- 9. ViBRANT and eMonocot Projectsbased around communities Provide tools for communities • Quantify user contribution to multi-user projects • Quantify how content generated by the community is used/reused
- 10.
- 11.
- 12. Citizen Science • COMBER •anymals+plants
- 13. Citizen Science • COMBER •anymals+plants • Citizen Science profile
- 14. Citizen Science • COMBER •anymals+plants • Citizen Science profile Metrics allow for more competitive activities and for users to quantify their involvement
- 15. Who needs impactmetrics?
- 16. Who needs impactmetrics? • The authors of data
- 17. Who needs impactmetrics? • The authors of data • People who employ data authors
- 18. Who needs impactmetrics? • The authors of data • People who employ data authors • People who reuse the data
- 19. Who needs impactmetrics? • The authors of data • People who employ data authors • People who reuse the data Am I useful?
- 20. Who needs impactmetrics? • The authors of data Am I useful? • People who employ data authors • People who reuse the data Do we have our priorities right?
- 21. Who needs impactmetrics? • The authors of data Am I useful? • People who employ data authors • People who reuse the data Do we have our priorities right?
- 22. Who needs impactmetrics? • The authors of data Am I useful? • People who employ data authors • People who reuse the data Hidden gem? Do we have our priorities right?
- 23. Who needs impactmetrics? • The authors of data Am I useful? • People who employ data authors • People who reuse the data Hidden gem? Avoid like the plague? Do we have our priorities right?
- 24.
- 25. We want peopleto share data…
- 26. …people want toplease the “powers that be” …
- 27. … the powersthat be like statistics.
- 28. So we madesome
- 29. But the peopleweren’t happy.
- 30. But the peopleweren’t happy. Of all the things I do you only care about papers?
- 31. But the peopleweren’t happy. The only way you value my papers is by citation in other papers? Of all the things I do you only care about papers?
- 32. But the peopleweren’t happy. The only way you value my papers is by citation in other papers? Of all the things I do you only care about papers? The impact of my career is measured by one number?
- 33. So some peoplemade their own
- 34. But there arestill problems
- 35. But there arestill problems This STILL only deals with my publications!
- 36. But there arestill problems This STILL only deals with my publications! How do I get credit for data?
- 37. But there arestill problems This STILL only deals with my publications! How do I get credit for data? I wrote some useful biodiversity software – what about that?
- 38. Are alt metricsuseful?
- 39. Are alt metricsuseful?
- 40. Are alt metricsuseful? These are fairly scholarly
- 41. Are alt metricsuseful? These measure broader public impact These are fairly scholarly
- 42. Are alt metricsuseful? These measure broader public impact These are fairly scholarly • A good start
- 43. Are alt metricsuseful? These measure broader public impact • A good start • Much broader than just scholarly citations These are fairly scholarly
- 44. Are alt metricsuseful? These measure broader public impact • A good start • Much broader than just scholarly citations • Actually ignores traditional citations These are fairly scholarly
- 45. Are alt metricsuseful? These measure broader public impact • A good start • Much broader than just scholarly citations • Actually ignores traditional citations These are fairly scholarly • Still only applied to papers
- 46. What are wedoing?
- 47. What are wedoing?
- 48. What are wedoing?
- 49. What are wedoing?
- 50. Biodiversity Data Journal Datapapers (descriptions of data)
- 51. Biodiversity Data Journal Datapapers (descriptions of data) • Allows data to become citable
- 52. Biodiversity Data Journal Datapapers (descriptions of data) • Allows data to become citable • Allows data to participate in standard credit/metric systems
- 53. Biodiversity Data Journal Datapapers (descriptions of data) • Allows data to become citable • Allows data to participate in standard credit/metric systems • … allows data to participate in alt metrics
- 54. Scratchpads Statistics Module Whatcontent exists on a Scratchpad site?
- 55. Scratchpads Statistics Module Whatcontent exists on a Scratchpad site? • Filter by user or taxonomic term
- 56. Scratchpads Statistics Module Whatcontent exists on a Scratchpad site? • Filter by user or taxonomic term • How much content is there?
- 57. Scratchpads Statistics Module Whatcontent exists on a Scratchpad site? • Filter by user or taxonomic term • How much content is there? • What kind of content is it?
- 58. Scratchpads Statistics Module Whatcontent exists on a Scratchpad site? • Filter by user or taxonomic term • How much content is there? • What kind of content is it? • How many registered users?
- 59. Scratchpads Statistics Module Whatcontent exists on a Scratchpad site? • Filter by user or taxonomic term • How much content is there? • What kind of content is it? • How many registered users? • How often is the content viewed?
- 60. Scratchpads Statistics Module Availablenow: http://antkey.org/stats [scratchpad.url]/stats
- 61. Scratchpads Metrics Module Putsthe Scratchpad content in a broader context
- 62. Scratchpads Metrics Module Putsthe Scratchpad content in a broader context • Users and content
- 63. Scratchpads Metrics Module Putsthe Scratchpad content in a broader context • Users and content • Opt-in (not enabled by default)
- 64. Scratchpads Metrics Module Putsthe Scratchpad content in a broader context • Users and content • Opt-in (not enabled by default) • Modular (pick and choose what to include)
- 65. Scratchpads Metrics Module Putsthe Scratchpad content in a broader context • Users and content • Opt-in (not enabled by default) • Modular (pick and choose what to include) • Available by end of the year
- 66. Scratchpads User Metrics Apartial implementation of the Scholar Factor proposed by Bourne and Fink
- 67. Scratchpads User Metrics Apartial (modified) implementation of the Scholar Factor proposed by Bourne and Fink Citations
- 68. Scratchpads User Metrics Apartial (modified) implementation of the Scholar Factor proposed by Bourne and Fink Citations Scholar Factor
- 69. Scratchpads User Metrics Apartial (modified) implementation of the Scholar Factor proposed by Bourne and Fink Citations Software / Data Scholar Factor
- 70. Scratchpads User Metrics Apartial (modified) implementation of the Scholar Factor proposed by Bourne and Fink Citations Software / Data Scholar Factor Grant / manuscript reviews
- 71. Scratchpads User Metrics Apartial (modified) implementation of the Scholar Factor proposed by Bourne and Fink Citations Software / Data Scholar Factor Grant / manuscript reviews
- 72.
- 73. Scratchpads User Metrics ScratchpadsUser Metrics Module Scholar Factor
- 74. Scratchpads User Metrics GoogleScholar (h-index) Scratchpads User Metrics Module Scholar Factor
- 75. Scratchpads User Metrics GoogleScholar (h-index) Scratchpad Scratchpads User Metrics Module Scholar Factor
- 76. Scratchpads User Metrics GoogleScholar (h-index) Scratchpad GitHub Scratchpads User Metrics Module Scholar Factor
- 77. Scratchpads User Metrics GoogleScholar (h-index) Scratchpad GitHub Scratchpads User Metrics Module Scholar Factor ?
- 78. Scratchpads User Metrics Modular GoogleScholar (h-index) Scratchpad GitHub Scratchpads User Metrics Module Scholar Factor ?
- 79. Scratchpads User Metrics Modular GoogleScholar (h-index) Scratchpad GitHub Scratchpads User Metrics Module Scholar Factor ?
- 80. Scratchpads User Metrics Modular GoogleScholar (h-index) Scratchpad GitHub Scratchpads User Metrics Module Scholar Factor ?
- 81. Same modular structurefor content
- 82. Same modular structurefor content • Number of web links to page
- 83. Same modular structurefor content • Number of web links to page • Scholarly links (Google Scholar)
- 84. Same modular structurefor content • Number of web links to page • Scholarly links (Google Scholar) • Social links
- 85.
- 86. Summary We’re not thereyet, but we have …
- 87. Summary We’re not thereyet, but we have… • Modular framework to add services as they become available
- 88. Summary We’re not thereyet, but we have… • Modular framework to add services as they become available • multi-dimensional – not just papers and citations
- 89. Summary We’re not thereyet, but we have… • Modular framework to add services as they become available • multi-dimensional – not just papers and citations • Not a one size fits all approach
- 90. Summary We’re not thereyet, but we have… • Modular framework to add services as they become available • multi-dimensional – not just papers and citations • Not a one size fits all approach
Editor's Notes
- #3 This work has been done under the ViBRANT and eMonocot projects. Much of this relates to Scratchpads which are in a simple terms an online content management system for biodiversity data (if you really want to know more you may want to leave and go next door as there is a summary of the project going on there).
- #4 These projects focus on data collection and curation, and making it available for reuse. Both projects use Scratchpads for a large part of this process.
- #5 In all large projects we like to know how well we are doing. But this is a pretty selfish use of statistics and metrics.
- #7 Communities are the core of these projects. We build tools for communities.
- #8 so we also need to consider how we build metrics tools for communities.
- #9 This might be allowing people to see how much they have contributed to a project, or conversely letting people using the project see who has contributed
- #10 We’d like a way to show people how the content they have generated is being used: who links to it, has it been shared socially, has it been traditionally cited, have people reused the data for novel purposes.
- #11 As part of ViBRANT there are a few citizen science projects so we also need to consider how we build metrics tools for communities
- #13 Collecting observation data via mobile phone
- #14 We’re working on a citizen science platform for Scratchpads
- #15 Metrics provide an added layer of incentive for citizen scientists – they could form the basis of a gamification reward system and allow people to quanify their involvement.
- #19 People who reuse their data. All of these different people need metrics to answer specific questions.
- #29 Perhaps the stereotypical metric – the h-index
- #33 We’ve found a way of creating a number out of one aspect of our output, and many people use it as proxy for the rest of what scientists do.
- #34 There has been a move towards alternative metrics, this example uses the altmetric
- #35 But there are still some problems with this method
- #41 Some parts of the metric, like Mendeley, can be considered to be fairly scholarly
- #49 Scratchpads metrics module
- #50 Scratchpads statistics module
- #51 It’s possible to publish data from a Scratchpad in the recently launched Biodiversity Data Journal, so an opportunity for some metrics activity.
- #52 Through CrossRef DOI
- #55 The Scratchpads statistics module provides information about the content on a given Scratchpad.
- #56 Filtering by user or taxonomic term means it’s easy to find out how much people have contributed, and how content is available for a given taxon.
- #58 If you’re looking for references or images you know how rich a resource this site might be.
- #59 Might give an indication of how active a community is
- #60 Do other people find the content useful?
- #64 This isn’t perfect, in fact it’s pretty experimental. So it will be on an opt-in basis.
- #65 Not every community wants the same set of criteria.
- #68 Traditional citations still play a part.
- #70 You also get credit for software you have written and datasets that you have created
- #71 Also get credit for grant and manuscript reviews. This is perhaps the trickiest part – but we are working with Pensoft to include this for reviews of papers in their journals.
- #72 Some of these things take more effort than others, so they are weighted.
- #73 So we have seen the theoretical model, now for the implementation we have created.
- #74 The user metrics module deals with the aggregation, weighting and display of metrics data.
- #75 A number of helper modules are responsible for getting the required data from external sources.
- #78 It’s easy to write a module that can supply data – maybe as little as 20 lines of code.
- #79 Modular design allow for a mix-and-match set of functionality, that also has some degree of future proofing. As a maybe slightly extreme example…
- #80 … we could add Facebook….
- #81 .. And remove legacy systems
- #83 One of the most basic web metrics available
- #84 We have done studies on links to Scratchpad content in articles on Google Scholar – and there has been a noticeable increase in recent years.
- #85 Also take some ideas from the altmetric community.