Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Research Assessment toSupport Research Impact              Cameron Neylon University of Cape Town - 10 August 2012
You are free to:           Copy, share, adapt, or re-mix;           Photograph, film, or broadcast;           Blog, live-bl...
There is a gaping (ethical, moral)  hole in our assessment systemshttp://www.flickr.com/photos/amanderson/2421035280 CC-BY
But if we throw it all out...?Something missing...http://www.flickr.com/photos/limaoscarjuliet/3305886294 CC-BY
key pointshttp://www.flickr.com/photos/y_i/2360600227 CC-BY
http://www.flickr.com/photos/andrec/2893549851 CC-BY
I don’t have answers...                     ...only questionshttp://www.flickr.com/photos/drachmann/327122302 CC-BY-SA
What do you value?
What is your mission?
Values   Mission
Mission   Assessment
Mission   Assessment
Mission   Assessment
http://www.flickr.com/photos/28481088@N00/2050334184 CC-BY
Can we measure better?http://www.flickr.com/photos/sterlic/4299631538 CC-BY
An assertion.
We want to seescholarship used.
In the right places...http://www.flickr.com/photos/velkr0/3472576304 CC-BY
In the right places...http://www.flickr.com/photos/argonne/3465398655 CC-BY
In the right places...http://www.flickr.com/photos/ol1/6048544977 CC-BY-SA
...at the right time.http://www.flickr.com/photos/robbie73/3387189144 CC-BY-SA
We want scholarship tobe re-used and re-usable
Impact = Re-use
Application = Re-use
Commercialised = Re-use
Education = Re-use
Engagement = Re-use
...but also...
Citation = Re-use
http://www.flickr.com/photos/flavinsky/3410980137 CC-BY
I can’t tell you which                                      ones matter to you...http://www.flickr.com/photos/andrec/289354...
...but we do have   new ways to measure these...
Articles with impact...http://www.flickr.com/photos/babasteve/3140688308 CC-BY
...but (relatively)few citations (yet)
PLOS papers authored by University of Cape Town staff   Usage data from the PLoS journal website (combined HTML views and ...
PLOS papers authored by University of Cape Town staffUsage data from the PLoS journal website (combined HTML views and PDF...
PLOS papers authored by University of Cape Town staffUsage data from the PLoS journal website (combined HTML views and PDF...
PLOS papers authored by University of Cape Town staffUsage data from the PLoS journal website (combined HTML views and PDF...
5000   0       0   20
PLOS papers authored by University of Cape Town staffUsage data from the PLoS journal website (combined HTML views and PDF...
PLOS papers authored by University of Cape Town staffUsage data from the PLoS journal website (combined HTML views and PDF...
PLOS papers authored by University of Cape Town staffUsage data from the PLoS journal website (combined HTML views and PDF...
Lots of data...http://total-impact.org/collection/yhb7tr
...but here’s a story
altmetric.com
What’s the missing audience for theseSouth African papers?
...here’s one...
Not very high views
PLOS papers authored by University of Cape Town staffUsage data from the PLoS journal website (combined HTML views and PDF...
...but highly relevant interactions
...driven by this person...
...who I can talk to.
See http://bit.ly/NJOyUn for the whole conversation
This. Is Impact.
It’s not about where     you publish...
...but who you reach
p://www.flickr.com/photos/cappellmeister/39508741 CC-BY
Think at network scale...
You’re used to thinkingabout reaching scholars...
...but at the scale of the network...
...there are many  ways to make  connections...
...and not all of them with your traditional      audience..
You don’t even knowwho most of them are..
...so how do youmaximise reach?
http://flickr.com/photos/virtualsugar/316200555/ CC-BY
key pointshttp://www.flickr.com/photos/y_i/2360600227 CC-BY
http://www.flickr.com/photos/andrec/2893549851 CC-BY
Values   Mission
http://www.flickr.com/photos/28481088@N00/2050334184 CC-BY
http://www.flickr.com/photos/sterlic/4299631538 CC-BY
Mission   Assessment
Assessment   Impact
p://www.flickr.com/photos/cappellmeister/39508741 CC-BY
Think like the network.
We can’t build a system on yesterday’s truths.
We could maintain asystem on today’s....
...for a whilehttp://www.flickr.com/photos/22746515@N02/4375854224 CC-BY
You build for the future             on tomorrow’s realities...http://www.flickr.com/photos/yakobusan/2436481628 CC-BY
Innovators don’t                          follow markets...http://www.flickr.com/photos/petrick/2291498814 CC-BY
...they build them.http://www.flickr.com/photos/98528214@N00/312210315/ CC-BY-SA
Networks are the future.http://www.flickr.com/photos/pagedooley/3143417443 CC-BY
Not rankings...http://www.flickr.com/photos/73112697@N03/6597508893 CC-BY
http://www.flickr.com/photos/mgifford/3558463424 CC-BY-SA
@cameronneylonhttp://cameronneylon.net
Research Assessment to Support Research Impact
Research Assessment to Support Research Impact
Research Assessment to Support Research Impact
Research Assessment to Support Research Impact
Research Assessment to Support Research Impact
Research Assessment to Support Research Impact
Research Assessment to Support Research Impact
Research Assessment to Support Research Impact
Research Assessment to Support Research Impact
Research Assessment to Support Research Impact
Research Assessment to Support Research Impact
Upcoming SlideShare
Loading in …5
×

Research Assessment to Support Research Impact

4,328 views

Published on

A talk given at the SCAP Symposium on August 12 2012. The talk focuses on the question of how new tools for tracking the use and re-use of research provide new mechanisms by which we can demonstrate and investigate research impact with a particular focus on those papers published from the University of Cape Town with PLOS.

Published in: Technology, Travel, Business
  • Be the first to comment

Research Assessment to Support Research Impact

  1. 1. Research Assessment toSupport Research Impact Cameron Neylon University of Cape Town - 10 August 2012
  2. 2. You are free to: Copy, share, adapt, or re-mix; Photograph, film, or broadcast; Blog, live-blog, or post video of;This presentation. Provided that: You attribute the work to its author and respect the rights and licenses associated with its components.
  3. 3. There is a gaping (ethical, moral) hole in our assessment systemshttp://www.flickr.com/photos/amanderson/2421035280 CC-BY
  4. 4. But if we throw it all out...?Something missing...http://www.flickr.com/photos/limaoscarjuliet/3305886294 CC-BY
  5. 5. key pointshttp://www.flickr.com/photos/y_i/2360600227 CC-BY
  6. 6. http://www.flickr.com/photos/andrec/2893549851 CC-BY
  7. 7. I don’t have answers... ...only questionshttp://www.flickr.com/photos/drachmann/327122302 CC-BY-SA
  8. 8. What do you value?
  9. 9. What is your mission?
  10. 10. Values Mission
  11. 11. Mission Assessment
  12. 12. Mission Assessment
  13. 13. Mission Assessment
  14. 14. http://www.flickr.com/photos/28481088@N00/2050334184 CC-BY
  15. 15. Can we measure better?http://www.flickr.com/photos/sterlic/4299631538 CC-BY
  16. 16. An assertion.
  17. 17. We want to seescholarship used.
  18. 18. In the right places...http://www.flickr.com/photos/velkr0/3472576304 CC-BY
  19. 19. In the right places...http://www.flickr.com/photos/argonne/3465398655 CC-BY
  20. 20. In the right places...http://www.flickr.com/photos/ol1/6048544977 CC-BY-SA
  21. 21. ...at the right time.http://www.flickr.com/photos/robbie73/3387189144 CC-BY-SA
  22. 22. We want scholarship tobe re-used and re-usable
  23. 23. Impact = Re-use
  24. 24. Application = Re-use
  25. 25. Commercialised = Re-use
  26. 26. Education = Re-use
  27. 27. Engagement = Re-use
  28. 28. ...but also...
  29. 29. Citation = Re-use
  30. 30. http://www.flickr.com/photos/flavinsky/3410980137 CC-BY
  31. 31. I can’t tell you which ones matter to you...http://www.flickr.com/photos/andrec/2893549851 CC-BY
  32. 32. ...but we do have new ways to measure these...
  33. 33. Articles with impact...http://www.flickr.com/photos/babasteve/3140688308 CC-BY
  34. 34. ...but (relatively)few citations (yet)
  35. 35. PLOS papers authored by University of Cape Town staff Usage data from the PLoS journal website (combined HTML views and PDF downloads). Circle size correlates with CrossRef citation counts. Total Views 14000 12000 10000 8000 6000 4000 2000 0 0 20 40 60 80 MonthsALM data visualisations courtesy of Martin Fenner
  36. 36. PLOS papers authored by University of Cape Town staffUsage data from the PLoS journal website (combined HTML views and PDF downloads). Circle size correlates with CrossRefcitation counts.Total Views 14000 Guidance for Evidence-Informed Policies about Health Systems: Rationale for and Challenges of Guidance Development 12000 Building the Field of Health Policy and Systems Research: 10000 Framing the Questions, PLOS Medicine 8000 6000 4000 2000 0 0 20 40 60 80 Months
  37. 37. PLOS papers authored by University of Cape Town staffUsage data from the PLoS journal website (combined HTML views and PDF downloads). Circle size correlates with CrossRefcitation counts.Total Views 14000 12000 10000 8000 6000 4000 2000 0 0 20 40 60 80 Months
  38. 38. PLOS papers authored by University of Cape Town staffUsage data from the PLoS journal website (combined HTML views and PDF downloads). Circle size correlates with CrossRefcitation counts. Red circles represent papers with "HIV" in title.Total Views 15000 “HIV” in title 10000 5000 0 0 20 40 60 80 Months
  39. 39. 5000 0 0 20
  40. 40. PLOS papers authored by University of Cape Town staffUsage data from the PLoS journal website (combined HTML views and PDF downloads). Circle size correlates with CrossRefcitation counts.Total Views 14000 12000 10000 8000 6000 4000 2000 0 0 20 40 60 80 Months
  41. 41. PLOS papers authored by University of Cape Town staffUsage data from the PLoS journal website (combined HTML views and PDF downloads). Circle size correlates with CrossRefcitation counts.Total Views 50000 40000 30000 20000 10000 0 0 20 40 60 80 Months
  42. 42. PLOS papers authored by University of Cape Town staffUsage data from the PLoS journal website (combined HTML views and PDF downloads). Circle size correlates with CrossRefcitation counts. Red circles represent papers with "Health" in title.Total Views Toward a Global View of Alcohol, Tobacco, Cannabis, and Cocaine 50000 Use: Findings from the WHO World Mental Health Surveys 40000 Patient Adherence to Tuberculosis Treatment: 30000 A Systematic Review of Qualitative Research 20000 10000 0 0 20 40 60 80 Months
  43. 43. Lots of data...http://total-impact.org/collection/yhb7tr
  44. 44. ...but here’s a story
  45. 45. altmetric.com
  46. 46. What’s the missing audience for theseSouth African papers?
  47. 47. ...here’s one...
  48. 48. Not very high views
  49. 49. PLOS papers authored by University of Cape Town staffUsage data from the PLoS journal website (combined HTML views and PDF downloads). Circle size correlates with CrossRefcitation counts. Red circles represent papers with "HIV" in title.Total Views 15000 10000 Not an obvious outlier... 5000 0 0 20 40 60 80 Months
  50. 50. ...but highly relevant interactions
  51. 51. ...driven by this person...
  52. 52. ...who I can talk to.
  53. 53. See http://bit.ly/NJOyUn for the whole conversation
  54. 54. This. Is Impact.
  55. 55. It’s not about where you publish...
  56. 56. ...but who you reach
  57. 57. p://www.flickr.com/photos/cappellmeister/39508741 CC-BY
  58. 58. Think at network scale...
  59. 59. You’re used to thinkingabout reaching scholars...
  60. 60. ...but at the scale of the network...
  61. 61. ...there are many ways to make connections...
  62. 62. ...and not all of them with your traditional audience..
  63. 63. You don’t even knowwho most of them are..
  64. 64. ...so how do youmaximise reach?
  65. 65. http://flickr.com/photos/virtualsugar/316200555/ CC-BY
  66. 66. key pointshttp://www.flickr.com/photos/y_i/2360600227 CC-BY
  67. 67. http://www.flickr.com/photos/andrec/2893549851 CC-BY
  68. 68. Values Mission
  69. 69. http://www.flickr.com/photos/28481088@N00/2050334184 CC-BY
  70. 70. http://www.flickr.com/photos/sterlic/4299631538 CC-BY
  71. 71. Mission Assessment
  72. 72. Assessment Impact
  73. 73. p://www.flickr.com/photos/cappellmeister/39508741 CC-BY
  74. 74. Think like the network.
  75. 75. We can’t build a system on yesterday’s truths.
  76. 76. We could maintain asystem on today’s....
  77. 77. ...for a whilehttp://www.flickr.com/photos/22746515@N02/4375854224 CC-BY
  78. 78. You build for the future on tomorrow’s realities...http://www.flickr.com/photos/yakobusan/2436481628 CC-BY
  79. 79. Innovators don’t follow markets...http://www.flickr.com/photos/petrick/2291498814 CC-BY
  80. 80. ...they build them.http://www.flickr.com/photos/98528214@N00/312210315/ CC-BY-SA
  81. 81. Networks are the future.http://www.flickr.com/photos/pagedooley/3143417443 CC-BY
  82. 82. Not rankings...http://www.flickr.com/photos/73112697@N03/6597508893 CC-BY
  83. 83. http://www.flickr.com/photos/mgifford/3558463424 CC-BY-SA
  84. 84. @cameronneylonhttp://cameronneylon.net

×