Pride and prejudices:Problems with nationaland international leaguetablesDr Paul Greatrix, Registrar,University of Notting...
League Tables•    Background•    Who wants to know?•    Who are the providers of this valuable     information?•    Mad, b...
BackgroundThey sell papers...•    ...and, some would suggest, create   3     perverse incentives
Who wants to know?•  The Government         •  Parents, teachers,•  The State                 advisors                    ...
Who’s responsible for providing thisvaluable data?•  The Times             •  Shanghai Jiao Tong                          ...
Invaluable information•  For all stakeholders•  Intelligent decision-making•  Better than prejudice•  Reflecting the reali...
League tables are a bad thing...“The silly season that marks the publication ofUniversity league tables is nonsensical and...
But we’ll use them anyway!“Having said that, I’m notashamed to report that we came avery creditable 79th overall, withmy o...
Dangerous...•  Criteria used do not reflect quality of education•  Historical data•  Variation over time•  Scores are inst...
... extremely dangerous...•  Apples and elephants and   paperclips•  Perverse incentives•  Hugely political•  Open to mani...
The Times 20111     Oxford       11    York2     Cambridge    12    Bath3     LSE          13    Bristol4     Imperial    ...
Sunday Times 20111     Cambridge     11    Loughborough2     Oxford        12    Newcastle3     Durham        13    Sheffi...
The Guardian 20111     Cambridge      11=   Exeter2     Oxford         11=   Sussex3     St Andrews     13    SOAS4     LS...
Complete University Guide 2011    1                   Cambridge    11    Bristol    2                   Oxford       12   ...
International value-added•  HE is now a global business•  Global branding assisted by competitive   ranking•  Internationa...
Just as dangerous...•  Again, the criteria used do not reflect   quality of education•  Archaic and irrelevant data•  Majo...
Times Higher Education World Rankings 20114     Oxford               48   Manchester6     Cambridge            56   King’s...
QS World Rankings 2011 1  Cambridge (1)     64   LSE (80) 5 Oxford (6)         67   Birmingham (59) 6 Imperial (7)       7...
Shanghai Jiao Tong ARWU 2011 5   Cambridge (5)    53 Edinburgh (54) 10 Oxford (10)       68 King’s (63) 20 UCL (21)       ...
Another approach:Ecole des Mines de Paris Rankings1    Harvard                    11 U of Pennsylvania2    Tokyo Universit...
Another Approach: Webometrics1    Massachusetts Institute of Technology2    Stanford University3    Harvard University4   ...
Webometrics Europe1    University of Cambridge, World Rank: 282    Swiss Federal Institute of Technology WR: 403    Univer...
International approaches 1Shanghai Jiao Tong: Ranking of World Universities   –  6 indicators covering: quality of educati...
International approaches 2                             24
Other tables•  ALOHA, PISA, OECD•  China – number of billionaires•  Just research performance: ranking of   scientific pap...
.Guess theLeagueTable!
Which league table?1)  Loughborough University2)  University of Sheffield3)  University of East Anglia4)  University of Ca...
Which league table?1)  Leeds Metropolitan University 26.722)  University of Liverpool 26.633)  University of Wales Institu...
Which league table?1  University of Glamorgan 10.92  University of Wales Institute, Cardiff 10.63  University of Brighton ...
Which league table?1 University of Nottingham, UK2 Northeastern University, US3 University of Connecticut, US4 University ...
Which league table?1    Greenwich2    Sheffield Hallam3    Kingston4    Westminster5    East London6    Central Lancashire...
Which league table?1    Webb Institute                    70.92    Carleton College                  61.33    Princeton Un...
Which league table?1. Oxford              392. Cambridge           213. Manchester          84=. Imperial College   54=. O...
Which league table?1)  Nottingham Trent University2)  University of Gloucestershire3)  University of Worcester4)  Universi...
An indicator too far(not covered in the UK tables - yet)•    Alumni giving•    Academic staff pay•    Research income•    ...
What is to be done•    Publish prolifically and get top RAE grades•    Be Highly Cited•    Get articles in Nature and Scie...
Conclusions•  They aren’t going to go away•  The international dimension will become increasingly   significant•  Methodol...
FinallyFor a copy of the presentation and further league tablecommentary and observations, visit:http://registrarism.wordp...
Upcoming SlideShare
Loading in...5
×

Aua league tables 2012

2,585

Published on

Pride and Prejudices: Problems with National and International League Tables. Presentation from AUA conference 2012

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,585
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Aua league tables 2012"

  1. 1. Pride and prejudices:Problems with nationaland international leaguetablesDr Paul Greatrix, Registrar,University of Nottingham 1
  2. 2. League Tables•  Background•  Who wants to know?•  Who are the providers of this valuable information?•  Mad, bad and dangerous•  National and international dimensions•  Measuring the unmeasurable? 2
  3. 3. BackgroundThey sell papers...•  ...and, some would suggest, create 3 perverse incentives
  4. 4. Who wants to know?•  The Government •  Parents, teachers,•  The State advisors •  Employers – national and•  The Funding Councils international•  Potential Students •  Overseas sponsors•  Alumni •  Jo Public•  Journalists •  The Universities •  Dangerous obsessives 4
  5. 5. Who’s responsible for providing thisvaluable data?•  The Times •  Shanghai Jiao Tong University•  Sunday Times •  QS•  The Guardian •  US News and World Report•  Complete University Guide •  HEFCE •  Others...•  Financial Times •  and, of course,•  THE Government 5
  6. 6. Invaluable information•  For all stakeholders•  Intelligent decision-making•  Better than prejudice•  Reflecting the realities of the market place•  We have a right to know•  Employers have the right to know 6•  We aren’t stupid
  7. 7. League tables are a bad thing...“The silly season that marks the publication ofUniversity league tables is nonsensical and illogical.As any New Scientist knows, letters into numbers;quality into quantity won’t go. League tables aresimplistic, divisive and undermine the qualitativenature of a University’s work” 7
  8. 8. But we’ll use them anyway!“Having said that, I’m notashamed to report that we came avery creditable 79th overall, withmy own department rating aparticularly good score forresearch- and as I remarked to the Dean,you can’t get much better thanthat.” 8
  9. 9. Dangerous...•  Criteria used do not reflect quality of education•  Historical data•  Variation over time•  Scores are institutional averages – mask strengths•  Distorting effect of weightings and scalings and data manipulation•  Many of criteria used are inter-related (ie not independent) 9
  10. 10. ... extremely dangerous...•  Apples and elephants and paperclips•  Perverse incentives•  Hugely political•  Open to manipulation•  Delivered by journalists•  Spurious precision•  Serious consequences for universities, departments, staff and students 10
  11. 11. The Times 20111 Oxford 11 York2 Cambridge 12 Bath3 LSE 13 Bristol4 Imperial 14 Sussex5 UCL 15 Edinburgh6 St Andrews 16 Nottingham6 Durham 17= Leicester7 UCL 17= Sheffield8 Warwick 19 Southampton9 Lancaster 20 Loughborough10 Exeter 11
  12. 12. Sunday Times 20111 Cambridge 11 Loughborough2 Oxford 12 Newcastle3 Durham 13 Sheffield4 LSE 14 Imperial5 Bath 15= Nottingham6 St Andrews 15= York7 UCL 17 King’s8 Warwick 18 Lancaster9 Exeter 19 Sussex10 Bristol 20 Glasgow 12
  13. 13. The Guardian 20111 Cambridge 11= Exeter2 Oxford 11= Sussex3 St Andrews 13 SOAS4 LSE 14 Bath5 UCL 15 York6 Warwick 16 Edinburgh7 Lancaster 17 Leicester8 Durham 18 UEA9 Loughborough 19= Nottingham10 Imperial 19= Surrey 13
  14. 14. Complete University Guide 2011 1 Cambridge 11 Bristol 2 Oxford 12 York 3 Imperial 13 Edinburgh 4 LSE 14 Southampton 5 Durham 15 Exeter 6 St Andrews 16 King’s 7 UCL 17 Nottingham 8 Warwick 18 SOAS 9 Lancaster 19= Loughborough 10 Bath 19= SussexWednesday, 4 April 12 14
  15. 15. International value-added•  HE is now a global business•  Global branding assisted by competitive ranking•  International benchmarking increasingly important, especially in research•  Student recruitment is increasingly international...•  ...mobile students are increasingly choosy•  It’s all good healthy fun 15
  16. 16. Just as dangerous...•  Again, the criteria used do not reflect quality of education•  Archaic and irrelevant data•  Major biases to large, English- speaking, research- and science- intensive universities•  Institutional scores are extraordinarily broad brush•  Distorting effect of weightings, scalings and data manipulation•  Far from comprehensive surveys of peers and employers 16
  17. 17. Times Higher Education World Rankings 20114 Oxford 48 Manchester6 Cambridge 56 King’s8 Imperial College 66 Bristol17 University College 83 Durham London 85 St Andrews36 Edinburgh 99 Sussex47 LSE 17
  18. 18. QS World Rankings 2011 1  Cambridge (1) 64 LSE (80) 5 Oxford (6) 67 Birmingham (59) 6 Imperial (7) 72 Sheffield (69) 7 UCL (4) 74 Nottingham (73) 20 Edinburgh (22) 75 Southampton (81) 27 King’s (21) 93 Leeds (85) 29 Manchester (30) 95 Durham (92) 30 Bristol (27) 96 York (88) 50 Warwick (53) 97 St Andrews (95) 59 Glasgow (77) 18
  19. 19. Shanghai Jiao Tong ARWU 2011 5 Cambridge (5) 53 Edinburgh (54) 10 Oxford (10) 68 King’s (63) 20 UCL (21) 70 Bristol (66) 24 Imperial (26) 85 Nottingham (84) 38 Manchester (44) 97 Sheffield (88) 19
  20. 20. Another approach:Ecole des Mines de Paris Rankings1  Harvard 11 U of Pennsylvania2  Tokyo University 12 Columbia3  Keio University 13= Stanford 13= Tohoku U4  HEC (France) 13= Nottingham5= Kyoto 16 MIT5= Oxford 17 Institute for Study of7  Ecole Polytechnique Politics, Paris8  Waseda 18 U St Gallen9  ENA (France) 19 U Sao Paolo10  Seoul National University 19 Northwestern U 20
  21. 21. Another Approach: Webometrics1 Massachusetts Institute of Technology2 Stanford University3 Harvard University4 University of California Berkeley5 Cornell University6 University of Michigan7 California Institute of Technology8 University of Minnesota9 University of Illinois Urbana Champaign10 University of Texas Austin 21
  22. 22. Webometrics Europe1 University of Cambridge, World Rank: 282 Swiss Federal Institute of Technology WR: 403 University of Oxford WR: 424 University of Helsinki WR: 585 University of Oslo WR: 606 University of Edinburgh WR: 657 University College London WR: 688 Utrecht University WR: 719 University of Manchester WR: 8210 Royal Institute of Technology (Sweden) WR: 84 22
  23. 23. International approaches 1Shanghai Jiao Tong: Ranking of World Universities –  6 indicators covering: quality of education; quality of faculty; research output; performance relative to size –  Includes: Nobel and Fields winners among alumni and staff; highly cited researchers; articles in Nature and Science; articles in citation indices. 23
  24. 24. International approaches 2 24
  25. 25. Other tables•  ALOHA, PISA, OECD•  China – number of billionaires•  Just research performance: ranking of scientific papers•  Most conservative, best parties, most interns•  European initiative – U-Multirank•  Islamic universities 25
  26. 26. .Guess theLeagueTable!
  27. 27. Which league table?1)  Loughborough University2)  University of Sheffield3)  University of East Anglia4)  University of Cambridge5)  University of Dundee6)  University of Oxford7)  University of Glasgow8)  University of Leeds9)  Aberystwyth University10) University of Southampton
  28. 28. Which league table?1)  Leeds Metropolitan University 26.722)  University of Liverpool 26.633)  University of Wales Institute, Cardiff 26.144)  Bath Spa University 24.955)  University of Glamorgan 22.266)  University of Bath 21.977)  University of Brighton 21.988)  Buckinghamshire New University 21.599)  Newcastle University 21.210) De Montfort University 21.0
  29. 29. Which league table?1  University of Glamorgan 10.92  University of Wales Institute, Cardiff 10.63  University of Brighton 9.54  Leeds Metropolitan University 9.45  Aberystwyth University 9.16  Manchester Metropolitan University 8.47  University of Dundee 8.28  University of Liverpool 8.19  University of Strathclyde 7.710  University College London 7.6 29
  30. 30. Which league table?1 University of Nottingham, UK2 Northeastern University, US3 University of Connecticut, US4 University College Cork, Ireland5 Linkoping University, Sweden6 University of California, Berkeley, US7 University of California, Los Angeles, US8 Washington University In St. Louis, US9 University of California Merced, US10 University of Bath, UK 30
  31. 31. Which league table?1 Greenwich2 Sheffield Hallam3 Kingston4 Westminster5 East London6 Central Lancashire7 Leeds Met8 Wolverhampton9 Coventry10 Middlesex 31
  32. 32. Which league table?1 Webb Institute 70.92 Carleton College 61.33 Princeton University 60.34 Middlebury 60.15 Amherst 59.56 Williams College 57.67 Centre College 56.78 Indiana Institute of Technology 55.19 Davidson College 54.910 Thomas Aquinas College 52.5 32
  33. 33. Which league table?1. Oxford 392. Cambridge 213. Manchester 84=. Imperial College 54=. Open University 56=. Durham 46=. Sussex 48=. St. Andrews 38=. Birkbeck 3 33
  34. 34. Which league table?1)  Nottingham Trent University2)  University of Gloucestershire3)  University of Worcester4)  University of Plymouth5)  Bournemouth University6)  University of Greenwich7)  Bath Spa University8)  University of East Anglia9)  University of Hertfordshire10) Leeds Metropolitan
  35. 35. An indicator too far(not covered in the UK tables - yet)•  Alumni giving•  Academic staff pay•  Research income•  Citations•  Brand impactBut international tables will increasingly influencemethodologies of UK tables 35
  36. 36. What is to be done•  Publish prolifically and get top RAE grades•  Be Highly Cited•  Get articles in Nature and Science•  Win a Nobel Prize or a Fields Medal•  Make sure your students all get top jobs with big multinationals (and then win Nobels)•  Plug the institution relentlessly and cultivate peers, Headmasters/ mistresses and employers•  Improve NSS results (and SSRs) - every little helps•  Spend more on Library and IT and everything else to do with teaching and learning•  Recruit more international students and staff (all with Nobel potential) 36
  37. 37. Conclusions•  They aren’t going to go away•  The international dimension will become increasingly significant•  Methodologies - for both national and international tables - are all dubious, at best•  They can and will be used by many different groups – but can be dangerous in the wrong hands•  Handle with great care! 37
  38. 38. FinallyFor a copy of the presentation and further league tablecommentary and observations, visit:http://registrarism.wordpress.com/Also on Twitter: @registrarism 38
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×