A Comparison of Global University Rankings<br />Erhan Erkut, PhD<br />President, Ozyegin University<br />December 2010<br ...
Why am I interested?<br />A quant person<br />Rankings, measurement, multiple criteria<br />Using bibliometrics for quite ...
Rankings matter<br />2005, U of Malaya #89 in THES<br />Foreign student classification error + lower reputation rankings.....
Rankings can be manipulated<br />QS: Alexandria #147 <br />#4 in citations—behind only Caltech, M.I.T. and Princeton, and ...
Criticizing rankings is a popular sport among academics.<br />All multi-criteria multi-stakeholder rankings can be critici...
Global university ranking systems<br />ARWU (Jiao Tong)<br />Times Higher Education<br />Quacquarelli Symonds<br />Webomet...
And…<br />UK - Research Assessment Exercise<br />US News & World Report<br />Financial Times -- MBA rankings<br />Gourman ...
Top 10 in the 2009 rankings<br />9<br />
Top 10 (2009)<br />Harvard #1<br />Stanford, MIT, Johns Hopkins, Washington and Michigan: 4/6<br />UCLA and Berkeley: 3/6<...
An empirical comparison of global ranking systems<br />Proof of concept<br />Assumption<br />There is a “true” ranking<br ...
Example<br />12<br />
Example<br />U2<br />M1<br />M2<br />M3<br />U1<br />13<br />
Example<br />U2<br />M1<br />M2<br />M3<br />U1<br />14<br />
Example<br />U2<br />M1<br />M2<br />M3<br />U1<br />15<br />
A defensible way of combining rankings<br />Nonlinear optimization problem with as many variables as the number of univers...
With actual data…<br />
Distance (similarity) Matrix<br />18<br />
“Optimally” aggregated rankings<br />
Turkish Univ? (2009)<br />ARWU 2009<br />İstanbul 424<br />THE-QS 2009<br />Bilkent 360<br />İTÜ, İstanbul Ü, Koç 401-500<...
Factors impacting research output of Turkish universities negatively<br />Researcher<br />Time<br />Financial support<br /...
Ranking Turkish Universities<br />Paper and citation counts<br />Research Grants<br />Research Prizes (Medals)<br />Grad S...
H-index ranking of Turkish U founded before 2000<br />TOP<br />BOTTOM<br />23<br />
The best predictor of H-index is …<br />24<br />
1.11 * (N)0.52<br />25<br />
Overperformers (% Error)<br />26<br />
Underperformers (%Error)<br />27<br />
Problems with rankings<br />Inputs vs. Outputs<br />Need to measure outputs<br />Measurement problems<br />Citation-based ...
Problems with rankings<br />A soccer team against a water polo team<br />Emphasis on<br />Technical<br />Medicine<br />Soc...
30<br />
Upcoming SlideShare
Loading in …5
×

ERHAN ERKUT 2010

1,136
-1

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,136
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

ERHAN ERKUT 2010

  1. 1. A Comparison of Global University Rankings<br />Erhan Erkut, PhD<br />President, Ozyegin University<br />December 2010<br />1<br />
  2. 2. Why am I interested?<br />A quant person<br />Rankings, measurement, multiple criteria<br />Using bibliometrics for quite some time<br />Hiring, promotions, grants, partnerships, …<br />Frustrated with the lack of transparency & accountability in the Turkish U system.<br />The public has a right to know<br />2<br />
  3. 3. Rankings matter<br />2005, U of Malaya #89 in THES<br />Foreign student classification error + lower reputation rankings...2006  #169<br />“Shocking Global Slide”<br />“Crisis in Malaysia's Public Universities”<br />Vice Chancellor (Rector) was fired!<br />The Great Brain Race<br />Ben Wildavsky, 2010<br />3<br />
  4. 4. Rankings can be manipulated<br />QS: Alexandria #147 <br />#4 in citations—behind only Caltech, M.I.T. and Princeton, and ahead of both Harvard and Stanford<br />Mohamed El Naschie of Alexandria U published 320 of his own articles in a scientific journal of which he was also the editor.<br />New York TimesNov. 14, 2010<br />4<br />
  5. 5. Criticizing rankings is a popular sport among academics.<br />All multi-criteria multi-stakeholder rankings can be criticized.<br />Thank you for all the work you do.<br />Please continue doing it.<br />Keep improving it.<br />5<br />
  6. 6. Global university ranking systems<br />ARWU (Jiao Tong)<br />Times Higher Education<br />Quacquarelli Symonds<br />Webometrics <br />HEEACT<br />Leiden<br />SCImago<br />6<br />
  7. 7.
  8. 8. And…<br />UK - Research Assessment Exercise<br />US News & World Report<br />Financial Times -- MBA rankings<br />Gourman Report – Law Schools<br />…<br />EU<br />Attempt to measure teaching and contribution to society<br />€1.1M<br />2009 – multidimensional ranking system<br />2010 – engineering and business faculties of 150 univ. <br />First results: 2011 May<br />8<br />
  9. 9. Top 10 in the 2009 rankings<br />9<br />
  10. 10. Top 10 (2009)<br />Harvard #1<br />Stanford, MIT, Johns Hopkins, Washington and Michigan: 4/6<br />UCLA and Berkeley: 3/6<br />Common features<br />Rich country<br />Old university<br />Large university<br />Emphasis on grad. programs<br />10<br />
  11. 11. An empirical comparison of global ranking systems<br />Proof of concept<br />Assumption<br />There is a “true” ranking<br />Each method is an approximation<br />Method<br />Take the top 10 from each ranking<br />Form the union (20 univ.)<br />Find the true ranks such that <br />The sum of Euclidean distances between the rankings and the true rank vector is minimized<br />11<br />
  12. 12. Example<br />12<br />
  13. 13. Example<br />U2<br />M1<br />M2<br />M3<br />U1<br />13<br />
  14. 14. Example<br />U2<br />M1<br />M2<br />M3<br />U1<br />14<br />
  15. 15. Example<br />U2<br />M1<br />M2<br />M3<br />U1<br />15<br />
  16. 16. A defensible way of combining rankings<br />Nonlinear optimization problem with as many variables as the number of universities <br />Can use any distance metric (not limited to Euclidean)<br />16<br />
  17. 17. With actual data…<br />
  18. 18. Distance (similarity) Matrix<br />18<br />
  19. 19. “Optimally” aggregated rankings<br />
  20. 20. Turkish Univ? (2009)<br />ARWU 2009<br />İstanbul 424<br />THE-QS 2009<br />Bilkent 360<br />İTÜ, İstanbul Ü, Koç 401-500<br />Webometrics<br />METU 499<br />HEEACT (top 500)<br />Leiden (top 250)<br />SCImago <br />Hacettepe 304, Ankara 363, Gazi 420, METU465, Ege 491, İstanbul 497 <br />20<br />
  21. 21. Factors impacting research output of Turkish universities negatively<br />Researcher<br />Time<br />Financial support<br />Infrastructure<br />Human resources<br />Career management<br />Not today’s topic. More on this in the paper.<br />21<br />
  22. 22. Ranking Turkish Universities<br />Paper and citation counts<br />Research Grants<br />Research Prizes (Medals)<br />Grad School Admission Exam Results<br />URAP<br />H-index<br />22<br />
  23. 23. H-index ranking of Turkish U founded before 2000<br />TOP<br />BOTTOM<br />23<br />
  24. 24. The best predictor of H-index is …<br />24<br />
  25. 25. 1.11 * (N)0.52<br />25<br />
  26. 26. Overperformers (% Error)<br />26<br />
  27. 27. Underperformers (%Error)<br />27<br />
  28. 28. Problems with rankings<br />Inputs vs. Outputs<br />Need to measure outputs<br />Measurement problems<br />Citation-based errors<br />Impact adjustments<br />Area normalization<br />Impact on the society<br />PhD production<br />Undergraduate employment<br />Start-ups, patents<br />Contribution to the economy<br />28<br />
  29. 29. Problems with rankings<br />A soccer team against a water polo team<br />Emphasis on<br />Technical<br />Medicine<br />Social sciences<br />Fine arts<br />We need area-specific rankings<br />Business, Engineering, Law, Physics, …<br />29<br />
  30. 30. 30<br />

×