Presentation for XХXII International Conference Problems of Decision Making under Uncertainties (PDMU-2018), August 27-31, 2018, Prague, Czech Republic DOI: 10.13140/RG.2.2.27143.44966

1. Decision-making on assessment of
higher education institutions under
uncertainty
Vladimir Bakhrushin, Prof., D.Sc.
Zaporizhia National Technical University,
Institute of Educational Analytics, MESU,
HERE team
PDMU-2018
XXXII International Conference
August 27-31, 2018
Prague, Czech Republic

2. Some problems and applications of
assessment
Budget funding of HEI on education and
research activity
University ratings
Staff and departments ratings
Accreditation of HEI
Status of HEI (Research, National)

3. Budget funding of higher education
Today the main mechanism of budget funding is the state order based on
forecasting of needs in specialists of different professions and norms of
expenses for training.
Until 2016, the distribution of the state order was carried out by the Ministry
of Education and Science based on the distribution of previous years and its
own preferences, which often were subjective.
Since 2016, the algorithm of a broad competition is used. It is based on the
Gail-Shepley algorithm and uses the priorities of entrants and their
competitive scores calculated on the basis of external independent evaluation.

4. The main problems of the existing model
Inconsistency of the predictions to reality under conditions of
rapid changes in society, economy and technology;
Inconsistency of orientation on profession under conditions of
massification of higher education;
Mismatch between the financial norms and needs;
Increasing of disparities in the distribution of budget financing
between regions and fields of knowledge

5. Budget code
State budget expenditures for higher education are distributed between
higher education institutions based on the formula, which should take
into account, in particular, the following parameters:
number of students according to the levels of education and
specialties and the ratio of the cost of training;
the level of results of external independent evaluation of entrants;
indicators of the quality of educational and scientific activity of higher
educational institutions

6. Apportionment formula
FS – stability funding (80-85% of previous year
total funding)
FD – performance-based funding in support of
development (10-15%)
FR – reserve (5%)
FRFDFSF ++=

7. Stability funding
Kcont – correction on changing of the contingent
of students, which takes into account their
distribution by levels of education and specialties
Kinf – correction on inflation
( ) ( ) ?*K*KtFStFS
FSFS
infcontii
i
i
∗−=
= ∑
1

8. Development funding
KDi – summary score of i-th HEI for performance-based
indicators;
h – minimum funding per student;
Ni – number of students at i-th university.
∑
=
−−=
i
i
i
i
KD
KD
FDFD
FRFSFFD
( )iiii N*h;FDFSmaxF +=

9. Optimization problem
The apportionment formula in such cases actually sets the objective function in the
multidimensional optimization problem under uncertainty. Some features:
there are no clearly defined criteria of optimality;
the influence of indicators on the criterion is not well known;
the data used for the assessment may contain statistical and other errors;
admissibility of using common indicators and assessment methods for different
fields of knowledge and different sciences has not been proved.

10. Uncertainty of objectives
More efficient use of budget funds - what is the
efficiency?
Reduction of HEI quantity – for what?
Improving the quality of higher education – but what is
the quality?
Increasing the competitiveness of HEI and HE system as
a whole - Why will it grow?
There is no reliable information about the effect of the
selected parameters on the objectives

11. Efficiency
Result / Cost
Those who say about efficiency in education often forget
about costs
Result: Gross Enrollment Ratio, Gross Domestic Product,
Human Development Index, …?
What are the optimal values for GER and some other possible
results?
Do GER, GDP, HDI, … depend only on budget funding for HE?

12. Initial Government Funding
per Student, $
Built using the data from: http://data.uis.unesco.org

13. Budget funding (% GDP)
Built using the data from: http://data.uis.unesco.org

14. Budget funding
(% of government spending)
Built accor using the data from: http://data.uis.unesco.org

15. Conclusion 1
Ukraine needs to substantially increase spending
per student, but does not have the necessary
funds.
Therefore, today the main tasks are:
to increase the efficiency of the use of available
funds;
to create favorable conditions for attracting
extrabudgetary funds.

16. Gross Enrollment Ratio
Built using the data from: http://data.uis.unesco.org

17. Educational migration
Built using the data from: http://data.uis.unesco.org

18. GDP per Capita, PPP (cur. int. $)
Built using the data from: http://data.worldbank.org

19. Human Development Index
Built using the data from: http://hdr.undp.org/en/data#

20. Conclusion 2
In the context of global massification of higher education and
global competition, Ukraine can not significantly reduce the
overall involvement of young people in higher education.
Therefore, it is necessary to distribute budgetary financing in such
a way as to maximize the growth of indicators of economic and
social development of Ukraine. But not always the state support
should be in the form of budgetary funding, sometimes it is
enough not to interfere

21. Reduction of HEIs quantity
Are more large HEI more qualitative?
Are more large HEI more effective?
Are more large HEI more competitive?
Is the optimal size the same for different
profiles – classic, polytechnic, pedagogical,
medical, arts, …?

22. University Size vs ARWU position
Built using the data from: http://www.shanghairanking.com/ARWU2017.html

23. HEI size for different profiles in Switzerland
Federal Statistical Office
https://www.bfs.admin.ch/bfs/en/home/statistics/education-science/pupils-
students.gnpdetail.2018-0165.html

24. Quality of Education
Standards and Guidelines for Quality
Assurance in the European Higher Education
Area (ESG-2015)
Standards ISO-9000-2015
Ukrainian Laws of education and higher
education
Various university ratings

25. Forbs vs ARWU for USA
Harvard University: 1 / 1
Stanford University: 2 / 2
Princeton University, MIT, Caltech: 4-6 / 4-6
…
Brown University: 9 / 100 – 150
Dartmouth College 12 / 201 - 300
Pomona College, Claremont McKenna College,
Williams College: 10, 11, 13 / -Built using the data from: https://www.forbes.com/top-colleges/list/#tab:rank;
http://www.shanghairanking.com/ARWU2017.html

26. Competitiveness
Is effect of selected parameters the same for
different institutions?
There are different possibilities to attract the
extrabudgetary funds in different fields of
education and different regions
There is responsibility of the government for
providing the needs in specialists of certain
professions

27. Uncertainty of criteria
How many criteria must be?
What exactly criteria can be taken?
What indicators can be taken?
What normalization procedures should be
taken for the indicators?
What should be the structure of the integral
criterion?

28. Requirements on criteria
Validity
Transparence
Intelligibility for non-specialists
Availability of reliable data
The possibility of independent verification of
data

29. Indicators
Cost indicators
The quality of the contingent of students
Results of scientific activity
Overall quality indicators
Employment of graduates

30. Cost indicators
Number of students studying for budget funds
(is non constant during financial year)
Normative of budget funding per student for
field of knowledge, levels and forms of
education (how to calculate, how to take into
account the differences between HEI)

31. Quality of the contingent
All students or only those who study for budget funds
Entrants or enrolled
All together or with consideration of fields of study
Independent external evaluation scores or competitive score
Indicators – mean, median, discrimination coefficient, the
share of those whose score is not less than the given value, …

32. Distribution of competitive scores
Built using the data from: https://vstup.edbo.gov.ua/offers/

33. Distributions of IEE scores
http://testportal.gov.ua/wp-content/uploads/2017/08/ZVIT_ZNO_2017_Tom_2.pdf

34. Results of scientific activity
The share of academic staffs who have 5 or
more publications indexed in Scopus or Web
of Science Core Collection
Share of funds received from scientific and
research activity

35. Overall quality indicators
Ranking of institution in one of the ratings
ARWU, QS, THE, taking into account position
in the rating (by the highest rank in case of
ranking in several ratings)

36. Employment of graduates
The average or median taxes paid by
graduates of certain years
Due to inaccessibility of the necessary data, such
indicator can be introduced only in 2-3 years

37. Some other indicators
Several other indicators are yet discussed now
Some indicators were rejected because they are not valid in
real conditions:
share of doctors and candidates of science;
number of scientific publications and patents;
number of foreign students;
share of postgraduate students who received a previous
higher education degree in another HEI

38. Thanks
The report uses the ideas and comments of
participants of the MESU working groups
expressed during discussions.
The author is especially grateful to V. Kovtunets,
Yu.Rashkevitch, O. Sharov, Ye. Stadny
Thanks for Your attention