Presentation in Doha March 2017

1. International Rankings
and the Assessment of
Teaching: A Regional
Perspective
Richard Holmes
University Ranking Watch

2. Contents
• International university rankings
• Assessment of teaching in rankings
• Arab universities and international
rankings
• Arab universities and teaching
indicators
• The way forward

3. Teaching indicators in global
rankings
• Current global rankings emphasise
research and say very little or nothing
about teaching
• URAP, NTU, Leiden, US News, 100%
• Shanghai ARWU at least 90% (10% award
winning alumni)

4. continued
• QS 60% - 80% (Staff Student Ratio
includes research only staff)
• THE -- effectively 84.75% (teaching
survey 90% + correlation with research
• RUR – 76%

5. continued
• Webometrics – 83% internet data
• IC4U – entirely internet data
• U-Multirank includes several teaching
related dimensions -- student staff
ratio, graduating on time, graduation
rate, overall learning experience, etc

6. Assessment of teaching in
national university rankings
• National rankings put more emphasis on
teaching
US News
• Input – quality of students (high school
rank, tests) staff student ratio, class
size
• Output – completion, value added

7. continued
Guardian
• Input – Staff Student Ratio, spending
• Output -- student satisfaction, graduate
success
• Times Higher Education
• Simulation of Teaching Excellence
Framework includes student satisfaction,
completion and employment

8. continued
Shanghai Best Chinese Universities
• National college entrance exam scores
• Graduate employment rate

9. continued
Perspektywy
• Employer preferences
• Assessment by academic staff

10. Teaching indicators in
international rankings -- input
• Staff student ratio
• (THE, QS, RUR, U-Multirank)
• Income (THE)
• Academic staff with doctorates (U-
Multirank, QS Arab region)

11. Teaching indicators in international
rankings -- outcomes
• Nobel and Fields awards (Shanghai)
• CEOs and prizes (CWUR Jeddah)
• Employer review (QS)
• Survey of postgraduate teaching by
researchers (THE, RUR)
• Survey of teaching by students (U
Multirank)
• THE and QS employability surveys

12. Arab universities in
international rankings
Webometrics lists 26000 universities
• 989 in the Arab world
• 270 have an “excellence” rank above
5778
Challenge for Arab university is
• to assess universities about 300 with
any significant research capacity
• and about 700 with no research profile

13. Arab universities in global
rankings
Shanghai rankings – Saudi Arabia 4, Egypt
1 in top 500 (King Abdulaziz University
and King Saud University in top 200)

14. continued
Times Higher Education world rankings--
26 from 10 countries
(8 from Egypt) -- highest is King
Abdulaziz University is 201-250
QS -- 30 (7 from Saudi Arabia) -- King
Fahd University of Petroleum and
Minerals 187th

15. continued
Round University Rankings – 12 (4 from
Egypt) – Qatar University 360th
U-Multirank – 23 (5 from Egypt)

16. continued
• Current international rankings include
only a fraction of Arab universities
• Best is QS with 30
• Assessment of teaching is very limited

17. Performance of Arab universities
on teaching indicators
• THE teaching group – 26 Arab universities
listed
Top 5
• KFUPM, Saudi Arabia
• American University Beirut, Lebanon
• King Abdulaziz University, Saudi Arabia
• Khalifah University, UAE
• American University Cairo, Egypt

18. RUR teaching indicators
• 12 Arab universities
Top five
• American University Beirut, Lebanon
• American University Cairo, Egypt
• Qatar University
• KFUPM, Saudi Arabia
• Kufa University, Iraq

19. QS world rankings – faculty
student ratio
• 18 Arab universities with scores
Top five
• KFUPM, Saudi Arabia
• King Saud University, Saudi Arabia
• KAUST, Saudi Arabia
• Khalifa University, UAE
• Universite Sainte Joseph, Lebanon

20. QS world rankings – employer
survey
• 12 Arab universities
Top Five
• American University in Beirut, Lebanon
• American University in Cairo, Egypt
• Cairo University, Egypt
• American University in Sharjah, UAE
• American University in Dubai, UAE

21. ARWU alumni
1 Arab university -- Cairo

22. CWUR –quality of education
(prizes)
• 3 Arab universities (377 total)
• American University Beirut
• Alexandria University
• Cairo University

23. CWUR – alumni employment
• 6 Arab universities
• KFUPM, Saudi Arabia
• King Saud University
• American University Beirut
• Cairo University
• United Arab Emirates University
• Alexandria University

24. QS arab region – faculty student
ratio
Top seven with score of 100
• Jubail Industrial College, Saudi Arabia
• Gulf University, Bahrain
• Dar Al Hekma College for Women, Saudi Arabia
• Yanbu Industrial College, Saudi Arabia
• Al Nahrain University, Iraq
• Hawler Medical University, Iraq
• Université Antonine, Beirut, Lebanon

25. QS Arab region -- employer
reputation
50 universities
Top 5 [same as world rankings]
• American University Beirut, Lebanon
• American University Cairo, Egypt
• Cairo University, Egypt
• American University Sharjah, UAE
• American University Dubai, UAE

26. US News Arab region rankings
employer survey
124 universities
Employer reputation top seven
• KFUPM
• American University Sharjah
• American University Cairo
• King Abdulaziz University (5)
• King Abdullah University of Science and Technology
(5)
• Ain Shams University (5)

27. To summarise
• At present none of the international rankings
is satisfactory
• Only a fraction of Arab universities are
represented – best global ranking is QS WUR
with scores for 30 and regional US News Arab
region with 124 universities
• Some indicator scores suggest problems with
collecting data and validity

28. continued
Some trends
• Saudi and Gulf universities do well for
resources
• Egypt and UAE for employer surveys, or maybe
it’s the “American” part

29. continued
• It is now possible to assess research capability
although there are obvious problems with
indicators, data and weighting
• Assessing teaching is more of a problem –
accurate data is often not available, only a
few places are covered, definition of teaching
• And third mission, even more so

30. continued
• Looking at existing rankings, universities
should forget about overall rankings and focus
on specific indicators
• These should not be bundled with other
indicators (THE and RUR)

31. continued
• Preferably not submitted by institutions but
derived from third party sources not created
for ranking purposes
• Assess reasonable number of universities
• Should not produce counter-intuitive results

32. continued
• At the moment the best available data would
appear to be the US News employer survey
• 124 universities, third party data, reasonable
results

33. Alternative indicators ?
LinkedIn “interested in attending” (60)
Top 5
• Helwan University
• American University in Cairo
• Arab Academy of Science Technology and
Maritime Transport
• Cairo University
• Ain Shams University

34. What is associated with positive
teaching ?
• Looking at the UK, some of these
things are associated with student
outcomes such as completion,
satisfaction, employment, “value
added” -- correlation with “career”
• Exam scores (AET) .820
• REF .667
• Staff student ratio .647
• Spending .569

35. continued
• V-C salary .428
• Average salary .277
• Teaching qualification -.378

36. To conclude
Suggestions
• Arab universities and governments work on
harmonization and standardization of
secondary school exams and admissions tests
• Establish a regional body to collect
comparable data on income, staff, faculty,
salaries, qualifications
• Encourage international assessments to assess
university learning outcomes

37. continued
UNTIL THEN
• Decide what sort of university you want to be
• De-emphasise overall ranking scores and focus
on valid and relevant indicators
• That are transparent, independent, reliable
and sensible
• In general use rankings, don’t let them use you