Research Evaluation: Is It Our Business Keynote

Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research Support, King’s College London

What is research evaluation?
External evaluation – in the UK the RAE now REF 3 major elements of the RAE:
publication outputs,
grant income awarded,
research students
Vital importance – link to income
Benchmarking – Rankings
Internal evaluation

The players in institutions
Research Office
PVC Research
Research group/dept heads
Administrators
Librarians – publications expertise
Institutional repository managers

Evolving interest
REF events at King’s 2008 -2010
1 st event – Beyond the RAE – representatives from 75 universities – 1/3 librarians
Survey –
biggest challenges – verifying publication lists
Biggest concerns for REF – bibliometrics – data sources, subject differences, skilling up to cope
http://www.kcl.ac.uk/iss/support/ref

Events Nov 08 and June 09
Focus on bibliometrics – pilots
Pre-event survey before Nov event:
Only about half of institutions had a centralized publications data collection system
One or two mentioned Symplectic
One mentioned something called Pure

Data collection systems
Some – only concerned with collecting RAE pubs
Others – aimed at collecting all publication info
Wider uses – profile pages / internal evaluation
Some creating internal ‘home grown’ systems
Separate systems for 3 RAE data elements
Integrated system, taking feeds from other systems (eg Research Gateway at King’s)

REF focus changed
Bibliometrics element toned down
Impact (4 th King’s REF event in June 2010)
Not publication impact – practical impacts of research
Research leaders had seen the light on
data collection systems
bibliometrics

CRIS Current Research Information Systems Already common in mainland Europe

CERIF
Common European Research Information Format
A standard – uses XML
Objects or entities with attributes eg project, person, organizational unit;
Relationships – 'linking relations’
Rich semantics – roles and time

R4R project - Ready for REF
JISC project – King’s and Southampton
CERIF4REF XML schema created
Plugins for Eprints, Dspace and Fedora
Transfer data to REF collection system
Research admins, IR managers, systems staff and librarians involved
http://r4r.cerch.kcl.ac.uk/

Librarians and CRIS systems
St Andrews experience
If research office led – get in on the ground floor
Librarians have plenty of expertise to offer

Which librarians?
Senior managers
Subject/liaison librarians
Research support librarians
Instutional repository managers
Information resources depts. – subs
Enquiries staff

Strategic directions for libraries
RLUK strategic themes
1 st – redefining the research library model
New services / new roles for staff
Value of libraries for research and researchers (RIN/RLUK report)
RLUK – Reskilling for Research (forthcoming)
Research support librarians

Some new areas for librarians associated with research evaluation
CRIS systems & CERIF
New ways of interacting with bibliographic / citation databases
APIs – for publication or citation data
Discovering other institutional systems eg grants DBs / student systems that feed into CRISs
Bibliometrics principles and new products

New opportunities
Showing our value
To administrators / research office
To academics and senior research leaders
To Graduate School
Opportunities to advocate Open access

Bibliometrics – the new buzz word
Awareness growing
By librarians
By research leaders in institutions
By academics

Librarians – natural synergy
Citation databases
Citation reports
Training offered – eg by Thomson Reuters
Training points up limitations of basic measures
Normalizing of measures crucial but difficult without specialised products

By research leaders
A few initiates who understand the issues leading the way
Percentiles tables in Essential Science Indicators used for evaluations
H-index being used by some
At least one institution has employed a bibliometrician

Academics
Some beginning to take an interest
Discovering:
ResearcherID
Citation reports in citation databases
Google Scholar
Graduates – training opportunity – “Making an impact”

Holy grail
Research information system
Dashboard for research managers – eg for grant information
Captures data with minimum effort
Integrated with bibliometrics
For internal & external purposes – evaluation and promotional

Is it our business?

Research Evaluation: Is It Our Business Keynote

by JIBS User Group

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 1

Statistics