talk at Bibliometrics in Practice event at Manchester, 27th June 2016 https://www.eventbrite.com/e/bibliometrics-in-practice-tickets-25268284118

1. Bibliometrics in Practice
evaluating REF
Alan Dix
Talis and University of Birmingham
http://alandix.com/ref2014/

2. Who – Alan Dix
first love mathematics
all of it! … including numerical models and statistics
academic for 35 years
human computer interaction + all sorts
Talis senior researcher
T&L software for HE (reading lists, etc.)
REF 2014 panel member
SP11 Computer Science and Informatics

3. What – REF public domain data
(virtually) complete list of outputs:
– excluding a few confidential ones
– for each: name, doi, ACM topic area, Scopus
citations (where used)
Google scholar citations for each
– gathered after REF (not used in assessment)
UoA profiles
sub-area profiles
N.B. highlighted data SP11 only

4. Why?
sub-area profiles – large differences
applied and human-oriented areas low 4*
various possible reasons:
weak UK research
long tail
bias in process
how to distingish?

5. How – bibliometrics
citation analysis – Scopus & Google Scholar
N.B. metrics to validate evaluation
≠ metrics for evaluation
large numbers
500–1000 per sub area
low 100s per institution

6. Results – scary
apparent emergent bias – 5-10 fold!
sub-areas:
theory vs. applied
instututions:
Russell Group vs pre vs post 1992
+ gender impact