• Like

Snowball Metrics: University-owned Benchmarking to Reveal Strengths within All Activities - Dr. Lisa Colledge, Snowball Metrics Program Director, Elsevier

  • 1,041 views
Uploaded on

June 18, 2014 …

June 18, 2014
NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends
Snowball Metrics: University-owned Benchmarking to Reveal Strengths within All Activities
- Dr. Lisa Colledge, Snowball Metrics Program Director, Elsevier

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,041
On Slideshare
0
From Embeds
0
Number of Embeds
5

Actions

Shares
Downloads
5
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Presented by Dr Lisa Colledge Snowball Metrics Programme Director, Elsevier 18 June 2014 www.snowballmetrics.com Snowball Metrics: University-owned Benchmarking to Reveal Strengths within All Activities 1
  • 2. The origins of Snowball Metrics Competitive award won by Imperial College London and Elsevier to investigate the state of research management in England 2 Report available at http://www.snowballmetrics.com/wp-content/uploads/research-information- management1.pdf
  • 3. The origins of Snowball Metrics Competitive award won by Imperial College London and Elsevier to investigate the state of research management in England Clear trends were voiced: • “Unless you have [data] you cannot make informed decisions; you would be acting based on opinions and hearsay.” 3 Report available at http://www.snowballmetrics.com/wp-content/uploads/research-information- management1.pdf
  • 4. The origins of Snowball Metrics Competitive award won by Imperial College London and Elsevier to investigate the state of research management in England Clear trends were voiced: • “Unless you have [data] you cannot make informed decisions; you would be acting based on opinions and hearsay.” • “[There is little] thought leadership and knowledge development around best practice.” 4 Report available at http://www.snowballmetrics.com/wp-content/uploads/research-information- management1.pdf
  • 5. The origins of Snowball Metrics Competitive award won by Imperial College London and Elsevier to investigate the state of research management in England Clear trends were voiced: • “Unless you have [data] you cannot make informed decisions; you would be acting based on opinions and hearsay.” • “[There is little] thought leadership and knowledge development around best practice.” 5 Report available at http://www.snowballmetrics.com/wp-content/uploads/research-information- management1.pdf • “The principle drivers for our systems are often external… but they shouldn’t be… a research strategy should… be developed… to respond to our strengths and the external environment, our systems should be defined to run our business.”
  • 6. University recommendations 6 Universities and funders should work more collaboratively, and develop stronger relationships with suppliers “Universities should work together more to make their collective voice heard by external agencies.” “The lack of a long-term vision makes it hard to… co-operate within a university let alone across the sector.” “Suppliers do not know what research offices do on a daily basis.” “How educated are we at asking suppliers the right questions?” 1
  • 7. University recommendations 7 Universities and funders should work more collaboratively, and develop stronger relationships with suppliers “Universities should work together more to make their collective voice heard by external agencies.” “The lack of a long-term vision makes it hard to… co-operate within a university let alone across the sector.” “Suppliers do not know what research offices do on a daily basis.” “How educated are we at asking suppliers the right questions?” “Someone needs to take ownership of the process: it is impossible to please all of the people all of the time so somebody needs to be strong enough to stand behind decisions and follow through.” “It would be great if the top five [universities] could collaborate.” 1 2
  • 8. Snowball Metrics are… • Endorsed by a group of distinguished UK universities to support their strategic decision making • Tried and tested methodologies that are available free-of-charge to the higher education sector • Absolutely clear, unambiguous definitions enable apples-to-apples comparisons so universities can benchmark themselves against their peers to judge the excellence of their performance 8
  • 9. Snowball Metrics are… • Endorsed by a group of distinguished UK universities to support their strategic decision making • Tried and tested methodologies that are available free-of-charge to the higher education sector • Absolutely clear, unambiguous definitions enable apples-to-apples comparisons so universities can benchmark themselves against their peers to judge the excellence of their performance Snowball Metrics are unique because: • Universities drive this bottom up • Academia – industry collaboration 9
  • 10. Snowball Metrics approach 10 Vision: Snowball Metrics enable benchmarking by driving quality and efficiency across higher education’s research and enterprise activities, regardless of system and supplier • Bottom-up initiative: universities define and endorse metrics to generate a strategic dashboard. The community is their guardian • Draw on all data: university, commercial and public • Ensure that the metrics are system- and tool-agnostic • Build on existing definitions and standards where possible and sensible
  • 11. The output of Snowball Metrics 11 www.snowballmetrics.com/metrics “Recipes” – agreed and tested metric methodologies – are the output of Snowball Metrics From Statement of Intent: • Agreed and tested methodologies… are and will continue to be shared free-of-charge • None of the project partners will at any stage apply any charges for the methodologies • Any organization can use these methodologies for their own purposes, public service or commercial Statement of Intent available at http://www.snowballmetrics.com/wp-content/uploads/Snowball-Metrics- Letter-of-Intent.pdf
  • 12. Founding project partners 12 Snowball Metrics Project Partners
  • 13. Main roles and responsibilities • Everyone covers their own costs • Universities – Agree the metrics to be endorsed as Snowball Metrics – Determine methodologies to generate the metrics in a commonly understood manner to enable benchmarking, regardless of systems • Elsevier – Ensures that the methodologies are feasible – Distribute the outputs using global communications networks – Day-to-day project management of the global program • Outside the remit of the Snowball Metrics program – Nature and quality of data sources used to generate Snowball Metrics – Provision of tools to enable generation and use Snowball Metrics 13
  • 14. Snowball Metrics Vision Snowball Metrics drive quality and efficiency across higher education’s research and enterprise activities, regardless of system and supplier Key aims • Agree, test and share metrics recipes in all areas of the “research and enterprise landscape” • Drive adoption of Snowball Metrics as global standards for institutional benchmarking – By implementation in tools, systems and reports – By engaging with university groups in key research-intensive nations 14
  • 15. The Snowball Metrics Landscape 15
  • 16. Denominators 16
  • 17. Recipes available 17
  • 18. Altmetrics Snowball Metrics • needs to be able to aggregate counts beyond article level • Needs to be able to share the entire recipe
  • 19. Altmetrics Scholarly Activity Number of times that an institution’s output has been posted in online tools that are typically used by academic scholars e.g. Mendeley, CiteULike, Academia.edu, ResearchGate Scholarly Commentary Number of times that an institution’s output has been commented on in online tools that are typically used by academic scholars e.g. science blogs, video posts, peer reviews, post-publication comments, Wikipedia Mass Media Number of times that an institution’s output has been referred to by press clippings and news websites Social Activity Number of times that an institution’s output has stimulated social media posts e.g. Facebook, Twitter, Reddit, LinkedIn Snowball Metrics • needs to be able to aggregate counts beyond article level • Needs to be able to share the entire recipe
  • 20. Altmetrics Scholarly Activity Number of times that an institution’s output has been posted in online tools that are typically used by academic scholars e.g. Mendeley, CiteULike, Academia.edu, ResearchGate Scholarly Commentary Number of times that an institution’s output has been commented on in online tools that are typically used by academic scholars e.g. science blogs, video posts, peer reviews, post-publication comments, Wikipedia Mass Media Number of times that an institution’s output has been referred to by press clippings and news websites Social Activity Number of times that an institution’s output has stimulated social media posts e.g. Facebook, Twitter, Reddit, LinkedIn Snowball Metrics • needs to be able to aggregate counts beyond article level • Needs to be able to share the entire recipe Scholarly Activity Scholarly Activity per FTE Scholarly Activity per publication
  • 21. Globalizing Snowball Metrics 21 US • University of Michigan • University of Minnesota • Northwestern University • University of Illinois at Urbana- Champaign • Arizona State University • MD Anderson Cancer Center • Kansas State University Australia / New Zealand • University of Queensland • University of Western Australia • University of Auckland • University of Wollongong • University of Tasmania • Massey University • The University of Canberra • Charles Darwin University Interest and support from: • Japan RU11 metrics group • Association of Pacific Rim Universities (APRU) • European Commission for H2020 • Fundação para a Ciência e a Tecnologia (FCT) in Portugal
  • 22. Supporting Snowball Metrics • Feedback about how useful it would be to you to have comparative information based on the Snowball Metrics • Feedback on the clarity of the recipes in this book • Call for influential bodies in the social sciences and arts and humanities to collaborate with the initiative to agree on practical metrics that are particularly useful for these areas • Ask funders and suppliers to adopt Snowball Metrics. Change requires momentum: please help us provide it • snowballmetrics@elsevier.com 22
  • 23. THANK YOU FOR YOUR TIME AND ATTENTION! l.colledge@elsevier.com snowballmetrics@elsevier.com