Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Reputation Matters
Towards an author economy
Melinda Kenneway
Director Director and co-founder
TBI Communications Kudos
Me...
Topics
1. Once upon a time …
2. What is reputation?
3. How is it measured?
- publications and other digital assets
- resea...
Once upon a time …
Every day, a miracle …
Distance selling …
Wake up call: NAR goes OA
Seeing the bigger picture
Libraries
Publishers
Universities
Government
ResearchersFunders
Intermediaries
Learned societies
Why submit? Two views…
what is reputation?
A definition
reputation
r pj te ( )n/ɛ ʊˈ ɪʃ ə
noun: reputation; plural noun: reputations
1. the beliefs or opinions that ...
Publication
performance
Institutional
performance
Researcher
performance
Publication
performance
Institutional
performance
Researcher
performance
Publication performance
• Rise of article level metrics
article-level-metrics.plos.org
Publication performance
• Rise of article level metrics
• Introduction of altmetrics
www.altmetric.com
www.impactstory.org
Publication performance
• Rise of article level metrics
• Introduction of altmetrics
• New units of publishing: data/image...
www.datadryad.org
www.figshare.com
Publication performance
• Rise of article level metrics
• Introduction of altmetrics
• New units of publishing: data/image...
www.rubruq.com
www.peerageofscience.org
Publication performance
• Rise of article level metrics
• Introduction of altmetrics
• New units of publishing: data/image...
www.plumanalytics.com
Publication performance
• Rise of article level metrics
• Introduction of altmetrics
• New units of publishing: data/image...
am.ascb.org/dora
Publication
performance
Institutional
performance
Researcher
performance
Researcher performance
• Publication output
• Publication impact
• Funding
• Other income (e.g. patents)
• Affiliations (i...
www.google.com/scholar
www.klout.com
www.researchgate.com
www.peerindex.com
The problem with single score systems…
www.researchcorecard.com
Publication
performance
Institutional
performance
Researcher
performance
Institutional performance
UK Research Excellence Framework
Outputs 65%:
“originality, significance and rigor”
Impact sub-p...
Research assessmentSector UK US Germany China Japan
Government RCUK
assessment
- pathways
to impact
STAR Metrics ESF
Guide...
“13 carefully
calibrated
performance
indicators”
“subjective
judgment of
senior,
published
academics”
http://www.snowballmetrics.com
https://becker.wustl.edu/impact-assessment/model
www.researchfish.com
convergence
what this means
Metrics are here to stay
Metrics are here to stay
“If you measure something
people change their behaviour.”
3 predictions
1. Emergence of scoring
and reward systems for all
kinds of academic activity
www.publons.com
2. Complex algorithms for
assessment of publication
and author impact
“what happened next”
www.frontiersin.org
3. Emergence of systems
and tools to influence
performance metrics
www.growkudos.com
Thank you
Melinda Kenneway
Director Director and co-founder
TBI Communications Kudos
Melinda.Kenneway@tbicommunications.co...
Qs 5 group c melinda kenneway
Upcoming SlideShare
Loading in …5
×

Qs 5 group c melinda kenneway

1,384 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Qs 5 group c melinda kenneway

  1. 1. Reputation Matters Towards an author economy Melinda Kenneway Director Director and co-founder TBI Communications Kudos Melinda.Kenneway@tbicommunications.com melinda@growkudos.com
  2. 2. Topics 1. Once upon a time … 2. What is reputation? 3. How is it measured? - publications and other digital assets - researchers - institutions 4. What this means 5. 3 predictions
  3. 3. Once upon a time …
  4. 4. Every day, a miracle …
  5. 5. Distance selling …
  6. 6. Wake up call: NAR goes OA
  7. 7. Seeing the bigger picture Libraries Publishers Universities Government ResearchersFunders Intermediaries Learned societies
  8. 8. Why submit? Two views…
  9. 9. what is reputation?
  10. 10. A definition reputation r pj te ( )n/ɛ ʊˈ ɪʃ ə noun: reputation; plural noun: reputations 1. the beliefs or opinions that are generally held about someone or something.
  11. 11. Publication performance Institutional performance Researcher performance
  12. 12. Publication performance Institutional performance Researcher performance
  13. 13. Publication performance • Rise of article level metrics
  14. 14. article-level-metrics.plos.org
  15. 15. Publication performance • Rise of article level metrics • Introduction of altmetrics
  16. 16. www.altmetric.com
  17. 17. www.impactstory.org
  18. 18. Publication performance • Rise of article level metrics • Introduction of altmetrics • New units of publishing: data/images/blogs
  19. 19. www.datadryad.org
  20. 20. www.figshare.com
  21. 21. Publication performance • Rise of article level metrics • Introduction of altmetrics • New units of publishing: data/images/blogs • Pre-publication evaluation
  22. 22. www.rubruq.com
  23. 23. www.peerageofscience.org
  24. 24. Publication performance • Rise of article level metrics • Introduction of altmetrics • New units of publishing: data/images/blogs • Pre-publication evaluation • Tools for institutional assessment
  25. 25. www.plumanalytics.com
  26. 26. Publication performance • Rise of article level metrics • Introduction of altmetrics • New units of publishing: data/images • Tools for institutional assessment • Anti-impact factor: DORA
  27. 27. am.ascb.org/dora
  28. 28. Publication performance Institutional performance Researcher performance
  29. 29. Researcher performance • Publication output • Publication impact • Funding • Other income (e.g. patents) • Affiliations (institutional reputation) • ‘Esteem factors’ • Membership of societies/ed boards etc • Conference activity • Awards and prizes
  30. 30. www.google.com/scholar
  31. 31. www.klout.com
  32. 32. www.researchgate.com
  33. 33. www.peerindex.com
  34. 34. The problem with single score systems…
  35. 35. www.researchcorecard.com
  36. 36. Publication performance Institutional performance Researcher performance
  37. 37. Institutional performance UK Research Excellence Framework Outputs 65%: “originality, significance and rigor” Impact sub-profile 20%: “unit’s reach and significance” Environment sub-profile 15%: “research environment vitality and sustainability”
  38. 38. Research assessmentSector UK US Germany China Japan Government RCUK assessment - pathways to impact STAR Metrics ESF Guidelines NNSFC – expert review panels CSTP (focus on peer review) Higher Education Research Excellence Framework – 20% now based on ‘impact’ (case studies + pilot Impactfinder Peer review, citation analysis – no formal framework Research Rating (introduced early 2013) Publication metrics (e.g. impact factor) NIAD-UE (evolving assessment framework – no assessment prior to 2008) Private Non- Profit Funder- specific Funder-specific (focused on peer review) Funder- specific (informed peer review) “No routine evaluation conducted” Research Center for Science Systems (peer review)
  39. 39. “13 carefully calibrated performance indicators”
  40. 40. “subjective judgment of senior, published academics”
  41. 41. http://www.snowballmetrics.com
  42. 42. https://becker.wustl.edu/impact-assessment/model
  43. 43. www.researchfish.com
  44. 44. convergence
  45. 45. what this means
  46. 46. Metrics are here to stay
  47. 47. Metrics are here to stay “If you measure something people change their behaviour.”
  48. 48. 3 predictions
  49. 49. 1. Emergence of scoring and reward systems for all kinds of academic activity
  50. 50. www.publons.com
  51. 51. 2. Complex algorithms for assessment of publication and author impact “what happened next”
  52. 52. www.frontiersin.org
  53. 53. 3. Emergence of systems and tools to influence performance metrics
  54. 54. www.growkudos.com
  55. 55. Thank you Melinda Kenneway Director Director and co-founder TBI Communications Kudos Melinda.Kenneway@tbicommunications.com melinda@growkudos.com

×