Research evaluation: is it our business?  Librarians in the brave new world of research evaluation Andria McGrath Senior I...
What is research evaluation? <ul><li>External evaluation – in the UK the RAE  now REF 3 major elements of the RAE:  </li><...
The players in institutions <ul><li>Research Office </li></ul><ul><li>PVC Research </li></ul><ul><li>Research group/dept h...
Evolving interest <ul><li>REF events at King’s 2008 -2010 </li></ul><ul><li>1 st  event – Beyond the RAE – representatives...
Events Nov 08 and June 09 <ul><li>Focus on bibliometrics – pilots </li></ul><ul><li>Pre-event survey before Nov event: </l...
Data collection systems <ul><li>Some – only concerned with collecting RAE pubs </li></ul><ul><li>Others – aimed at collect...
REF focus changed <ul><li>Bibliometrics element toned down </li></ul><ul><li>Impact (4 th  King’s REF event in June 2010) ...
CRIS  Current Research Information Systems Already common in mainland Europe
CERIF <ul><li>Common European Research Information Format </li></ul><ul><li>A standard – uses XML </li></ul><ul><li>Object...
R4R project - Ready for REF <ul><li>JISC project – King’s and Southampton </li></ul><ul><li>CERIF4REF XML schema created <...
Librarians and CRIS systems <ul><li>St Andrews experience </li></ul><ul><li>If research office led – get in on the ground ...
Which librarians? <ul><li>Senior managers </li></ul><ul><li>Subject/liaison librarians </li></ul><ul><li>Research support ...
Strategic directions for libraries <ul><li>RLUK strategic themes </li></ul><ul><li>1 st  – redefining the research library...
Some new areas for librarians associated with research evaluation <ul><li>CRIS systems & CERIF </li></ul><ul><li>New ways ...
New opportunities <ul><li>Showing our value </li></ul><ul><ul><li>To administrators / research office </li></ul></ul><ul><...
Bibliometrics – the new buzz word <ul><li>Awareness growing </li></ul><ul><li>By librarians </li></ul><ul><li>By research ...
Librarians – natural synergy <ul><li>Citation databases </li></ul><ul><li>Citation reports </li></ul><ul><li>Training offe...
By research leaders <ul><li>A few initiates who understand the issues leading the way </li></ul><ul><li>Percentiles tables...
Academics <ul><li>Some beginning to take an interest </li></ul><ul><li>Discovering: </li></ul><ul><ul><li>ResearcherID </l...
Holy grail <ul><li>Research information system </li></ul><ul><li>Dashboard for research managers – eg for grant informatio...
Is it our business?
Upcoming SlideShare
Loading in …5
×

Research Evaluation: Is It Our Business Keynote

1,016 views
958 views

Published on

Research evaluation: is it our business?: Librarians in the brave new world of research evaluation by Andria McGrath, Senior Information Specialist, Research Support, King’s College London. Presentation at the Research Evaluation: Is It Our Business? The Role of Librarians in the Brave New World of Research Evaluation 29 June 2011, University of Birmingham, Edgbaston Campus.

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,016
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Research Evaluation: Is It Our Business Keynote

  1. 1. Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research Support, King’s College London
  2. 2. What is research evaluation? <ul><li>External evaluation – in the UK the RAE now REF 3 major elements of the RAE: </li></ul><ul><ul><li>publication outputs, </li></ul></ul><ul><ul><li>grant income awarded, </li></ul></ul><ul><ul><li>research students </li></ul></ul><ul><li>Vital importance – link to income </li></ul><ul><li>Benchmarking – Rankings </li></ul><ul><li>Internal evaluation </li></ul>
  3. 3. The players in institutions <ul><li>Research Office </li></ul><ul><li>PVC Research </li></ul><ul><li>Research group/dept heads </li></ul><ul><li>Administrators </li></ul><ul><li>Librarians – publications expertise </li></ul><ul><li>Institutional repository managers </li></ul>
  4. 4. Evolving interest <ul><li>REF events at King’s 2008 -2010 </li></ul><ul><li>1 st event – Beyond the RAE – representatives from 75 universities – 1/3 librarians </li></ul><ul><li>Survey – </li></ul><ul><ul><li>biggest challenges – verifying publication lists </li></ul></ul><ul><ul><li>Biggest concerns for REF – bibliometrics – data sources, subject differences, skilling up to cope </li></ul></ul><ul><li>http://www.kcl.ac.uk/iss/support/ref </li></ul>
  5. 5. Events Nov 08 and June 09 <ul><li>Focus on bibliometrics – pilots </li></ul><ul><li>Pre-event survey before Nov event: </li></ul><ul><ul><li>Only about half of institutions had a centralized publications data collection system </li></ul></ul><ul><ul><li>One or two mentioned Symplectic </li></ul></ul><ul><ul><li>One mentioned something called Pure </li></ul></ul>
  6. 6. Data collection systems <ul><li>Some – only concerned with collecting RAE pubs </li></ul><ul><li>Others – aimed at collecting all publication info </li></ul><ul><li>Wider uses – profile pages / internal evaluation </li></ul><ul><li>Some creating internal ‘home grown’ systems </li></ul><ul><ul><li>Separate systems for 3 RAE data elements </li></ul></ul><ul><ul><li>Integrated system, taking feeds from other systems (eg Research Gateway at King’s) </li></ul></ul>
  7. 7. REF focus changed <ul><li>Bibliometrics element toned down </li></ul><ul><li>Impact (4 th King’s REF event in June 2010) </li></ul><ul><li>Not publication impact – practical impacts of research </li></ul><ul><li>Research leaders had seen the light on </li></ul><ul><ul><li>data collection systems </li></ul></ul><ul><ul><li>bibliometrics </li></ul></ul>
  8. 8. CRIS Current Research Information Systems Already common in mainland Europe
  9. 9. CERIF <ul><li>Common European Research Information Format </li></ul><ul><li>A standard – uses XML </li></ul><ul><li>Objects or entities with attributes eg project, person, organizational unit; </li></ul><ul><li>Relationships – 'linking relations’ </li></ul><ul><li>Rich semantics – roles and time </li></ul>
  10. 10. R4R project - Ready for REF <ul><li>JISC project – King’s and Southampton </li></ul><ul><li>CERIF4REF XML schema created </li></ul><ul><li>Plugins for Eprints, Dspace and Fedora </li></ul><ul><li>Transfer data to REF collection system </li></ul><ul><li>Research admins, IR managers, systems staff and librarians involved </li></ul><ul><li>http://r4r.cerch.kcl.ac.uk/ </li></ul>
  11. 11. Librarians and CRIS systems <ul><li>St Andrews experience </li></ul><ul><li>If research office led – get in on the ground floor </li></ul><ul><li>Librarians have plenty of expertise to offer </li></ul>
  12. 12. Which librarians? <ul><li>Senior managers </li></ul><ul><li>Subject/liaison librarians </li></ul><ul><li>Research support librarians </li></ul><ul><li>Instutional repository managers </li></ul><ul><li>Information resources depts. – subs </li></ul><ul><li>Enquiries staff </li></ul>
  13. 13. Strategic directions for libraries <ul><li>RLUK strategic themes </li></ul><ul><li>1 st – redefining the research library model </li></ul><ul><li>New services / new roles for staff </li></ul><ul><li>Value of libraries for research and researchers (RIN/RLUK report) </li></ul><ul><li>RLUK – Reskilling for Research (forthcoming) </li></ul><ul><li>Research support librarians </li></ul>
  14. 14. Some new areas for librarians associated with research evaluation <ul><li>CRIS systems & CERIF </li></ul><ul><li>New ways of interacting with bibliographic / citation databases </li></ul><ul><li>APIs – for publication or citation data </li></ul><ul><li>Discovering other institutional systems eg grants DBs / student systems that feed into CRISs </li></ul><ul><li>Bibliometrics principles and new products </li></ul>
  15. 15. New opportunities <ul><li>Showing our value </li></ul><ul><ul><li>To administrators / research office </li></ul></ul><ul><ul><li>To academics and senior research leaders </li></ul></ul><ul><ul><li>To Graduate School </li></ul></ul><ul><li>Opportunities to advocate Open access </li></ul>
  16. 16. Bibliometrics – the new buzz word <ul><li>Awareness growing </li></ul><ul><li>By librarians </li></ul><ul><li>By research leaders in institutions </li></ul><ul><li>By academics </li></ul>
  17. 17. Librarians – natural synergy <ul><li>Citation databases </li></ul><ul><li>Citation reports </li></ul><ul><li>Training offered – eg by Thomson Reuters </li></ul><ul><li>Training points up limitations of basic measures </li></ul><ul><li>Normalizing of measures crucial but difficult without specialised products </li></ul>
  18. 18. By research leaders <ul><li>A few initiates who understand the issues leading the way </li></ul><ul><li>Percentiles tables in Essential Science Indicators used for evaluations </li></ul><ul><li>H-index being used by some </li></ul><ul><li>At least one institution has employed a bibliometrician </li></ul>
  19. 19. Academics <ul><li>Some beginning to take an interest </li></ul><ul><li>Discovering: </li></ul><ul><ul><li>ResearcherID </li></ul></ul><ul><ul><li>Citation reports in citation databases </li></ul></ul><ul><ul><li>Google Scholar </li></ul></ul><ul><li>Graduates – training opportunity – “Making an impact” </li></ul>
  20. 20. Holy grail <ul><li>Research information system </li></ul><ul><li>Dashboard for research managers – eg for grant information </li></ul><ul><li>Captures data with minimum effort </li></ul><ul><li>Integrated with bibliometrics </li></ul><ul><li>For internal & external purposes – evaluation and promotional </li></ul>
  21. 21. Is it our business?

×