QUALITY OF EDUCATION IN THE UNIVERSITIES OF UKRAINE: STUDENT ASSESSMENT

QUALITY OF EDUCATION
IN THE UNIVERSITIES
OF UKRAINE:
STUDENT ASSESSMENT
NON-GOVERNMENTAL ORGANIZATION ELIBUKR

IN COOPERATION WITH
THE NATIONAL UNIVERSITY OF KYIV-MOHYLA ACADEMY
AND
NATIONAL AGENCY FOR HIGHER EDUCATION QUALITY
ASSURANCE (UKRAINE)

WITH SUPPORT OF
THE EMBASSY OF BULGARIA IN KYIV, UKRAINE
AS PART OF
THE PROJECT IMPLEMENTATION
«UKRAINIAN QUALITY ASSURANCE E. SERVICE FOR EDUCATION»:
2019-2020

KYIV – 2020

ABOUT US
Non-governmental organization
ELIBUKR
was founded in 2009 aiming to promote the integration of
higher education and science of Ukraine in the world
academic community. ELIBUKR projects intend to develop
scholarly communication, Open Science, and Open Access
to scholarly information, academic integrity, quality of
education.
Read more: http://elibukr.org
02

02
The National University of
Kyiv-Mohyla Academy
(founded in 1615, rebuilt in 1991) is a classic research university, the
initiator of most innovations in the field of higher education in Ukraine.
Read more: http://www.ukma.edu.ua
National Agency for Higher Education
Quality Assurance
(founded in 2015) – is an independent, permanent collegial body that
implements the state policy in the field of quality assurance in higher
education. Considers itself as a catalyst for change in higher education
in Ukraine to form a culture of its quality.
Read more: https://naqa.gov.ua/
Quality of education in the universities of Ukraine: Student assessment
Kyiv - 2020 03
PROJECT PARTNERS

IMPORTANCE OF THE PROJECT
Students are stakeholders and real
participants in the quality assurance
system of educational activities in
universities. Their opinion is important for
assessing the quality of teaching, as well
as for the development of training courses
and educational programs.
One of the main tools for assessing the
quality of Higher Education Institutions is
regular student surveys conducting.
Unfortunately, such surveys, particularly in
Ukraine, have not yet become a daily
practice for most Higher Education
Institutions. Also, algorithms for the
systematic application of their results have
not yet been created.
The project aims to change
the situation for the better by
developing a technical and
methodological solution for
conducting surveys of
students (also, if necessary,
surveys of any other
stakeholders, including
graduates and employers)
and creating an
Independent Nationwide
Research Portal for Quality
Assurance in Higher
Education.
Ensuring the quality of higher education is
a requirement of modernity, an indisputable
priority for the academic community and
public education policy in countries of The
European Higher Education Area (EHEA)
and other developed countries.
Education and science affect all structural
elements of any society, are a factor in its
modernization, social, technological, and
innovative development. They are also
included in the parameters of national
security. Without exaggeration, the future
of the country depends on the quality of
higher education.
How to assess the
quality of education in Ukrainian
Higher Education Institutions?
· To make sure that the
university develops and responds
to the challenges of time,
to the needs of the labor market?
1
2
Compare?
04

National University of Kyiv-Mohyla Academy (NaUKMA) – is the
oldest and the most time-honored university in Ukraine (founded
in 1615), today – a classic research university, the initiator of
most innovations in the field of higher education in Ukraine since
1991.
WHY WAS THE PROJECT LAUNCHED IN
THE NATIONAL UNIVERSITY OF
KYIV-MOHYLA ACADEMY?
The quality of education has always been
and remains a key priority for NaUKMA.
Insistence on high standards, zero
tolerance towards any academic
dishonesty, deficiency of corruption,
critical thinking, and an atmosphere of
true academic freedom are the hallmarks
of the internal culture of this university.
Regular student surveys on
the quality of courses and
teaching have been among
the components of the
internal quality assurance
system of education at
NaUKMA since the 1990s.
Kyiv - 2020 05

Upon words of the former NaUKMA President (2007-2014), the
former Minister of Education of Ukraine (2014-2016), the Head of
the National Agency for Higher Education Quality Assurance
(since 2017), Professor Serhiy Kvit:
“Initially, we had the task of resuming
regular student surveys before each
session regarding the quality of courses
and teaching.
Back in the 1990s, such surveys were
conducted on paper in NaUKMA and were
freely distributed throughout the university
in consolidated files.
In those days students could simply fill out
questionnaires with a few questions - and
immediately pass them over to volunteers.
Now the situation has changed on a global
scale, as the transition to online surveys in
all universities around the world has led to
a sharp decline in student participation.
Now they need to find the time and
inspiration to personally register and fill out
lots of questionnaires for all the courses.
first, in order not to perceive calls for
surveys as spam, students attention
should be drawn to the teachers, who
should devote their time to filling in
surveys during their courses.
Second, students should feel that their
participation in surveys affects the
situation at the university.
Answering my question about the best way
to involve students in such surveys, one of
the leaders of The European Students'
Union (ESU), Lea Meister at a symposium
in Brussels in June 2017, indicated that:
06

The National University of Kyiv-Mohyla
Academy decided to follow this path -
giving students time and opportunity to
register and answer the questionnaire on
the last pair before the session. Now,
according to the "Regulations on the
educational process of NaUKMA", the
online survey has become a part of the
training itself. It is considered that the
student has completed the course when
he/she had attended classes, assessed
the course, and passed the exam or test.
It is important to emphasize that there is no
coercion. Students recognize the
importance of regular surveys, understand
that in this way they can affect the
academic quality of the University.
However, paradoxically, they do not have
enough time. They also ask questions
about anonymity. NaUKMA, for its part,
emphasizes that such surveys should only
be anonymous. It should become part of
the academic culture of the Kyiv-Mohyla
Academy to clarify the real circumstances,
take into account students' perspectives
(through their answers to open questions),
disseminate positive teaching experience,
promote the academic quality of the
University.”
Source:
Serhiy Kvit. Quality Assurance of a Modern University
(2017): https://bit.ly/2RLIMbt
Kyiv - 2020 07

The National University of Kyiv-Mohyla Academy (Kyiv)
V.N. Karazin Kharkiv National University (Kharkiv)
Uzhhorod National University (Uzhhorod)
Vasyl Stus Donetsk National University (Vinnytsia)
Donetsk National Technical University (Pokrovsk, Donetsk
Region)
Petro Mohyla Black Sea National University (Mykolaiv)
Since NaUKMA has many years of
experience in organizing, conducting, and
summarizing the data of student surveys
on the quality of teaching and learning
(including in a modern online format), a
model was built that can be
recommended to other universities and
scaled throughout the country.
The pilot project of the Independent
Nationwide Research Portal for Quality
Assurance in Higher Education is
currently being tested at 6 universities in
Ukraine (including 2 displaced from the
zone of temporary occupation):
08
Independent Nationwide Research Portal for
Quality Assurance in Higher Education

RESEARCH
The conducted analysis
demonstrated the extremely significant
and central place of systematic student
surveys on the quality of teaching and
learning in the internal system of quality
assurance of education.
An analysis of international experience in
the quality assurance system of higher
education through the study of
regulations and projects of:
• The European University Association
(EUA) concerning the development of the
culture of quality in European
universities,
• The European Association for Quality
Assurance in Higher Education (ENQA)
(formerly the European Network for
Quality Assurance in Higher Education),
• Standards and Guidelines for Quality
Assurance in the European Higher
Education Area (ESG),
• Project QUAERE-562013-EPP-1-2015-
1-PL-EPPKA2-CBHE-SP «Quality
Assurance System in Ukraine:
Development on the Base of ENQA
Standards and Guidelines»: (2016
survey
conducted in 217 universities in Ukraine),
• as well as relevant experience of specific
universities: The University of Toronto
(Canada), The University of Glasgow
(Scotland, UK), Maastricht University (The
Netherlands), Stanford University (USA),
Princeton University (USA), Rutgers
University (USA), et al.
The subject of the study was only one key
component of the system - student
surveys on the quality of teaching, while
other relevant components were analyzed
(usually indicated by respondents: quality
assurance processes in teaching and
learning; evaluation; procedures for
approval and monitoring of curricula;
qualification of professors and faculty;
educational resources and student
support; information systems, etc.
Kyiv - 2020 09

25,200
In Ukraine, student surveys are conducted
in various forms in many Higher Education
Institutions, at least as evidenced by the
2016 survey: "Regular student surveys on
the quality of educational activities are
conducted in 176 Higher Education
Institutions in Ukraine" (see
https://bitly.su/erCYVSCY). However, only
20% of these surveys relate to the actual
quality of educational programs.
As a rule, Higher Education Institutions do
not have regulated norms for considering
the results of student surveys and taking
appropriate measures based on the results
of these surveys. A short number of
universities provided students who took
part in the surveys with information about
the results and the course of actions taken
based on these results.
This trend does not allow students to form
an awareness that they are the real
participants in the quality assurance
system of educational activities.
75%
About the quality of work of scientific
and pedagogical workers
*data of the 2016 survey: "Regular student
surveys on the quality of educational activities
are conducted in 176 Higher Education
Institutions in Ukraine"
Does your higher education
institution conduct regular student
surveys? *
20%
About the quality of
educational programs
5%
Surveys are not conducted
Such surveys are an important component of
student-centered learning in the world's leading universities. They allow
better consideration of students' needs and requests, as well they become a
tool for ensuring academic integrity and combating corruption.
10

GENERAL TRENDS IN
CONDUCTING SURVEYS
Only consistent problem solving of
these issues allows us to obtain results
that will be reliable, valid, consistent,
and representative, and therefore will
allow concluding on the quality of
teaching disciplines, to assess existing
trends.
Several main issues commonly relate to
organizing such surveys:
• Ensuring the confidentiality and
anonymity of respondents (students),
• Prevention of results manipulation - both
at the level of deformation of the numbers
or in the act of completion of
questionnaires, as well as at the level of
interpretation and further results usage,
• Ensuring a high level of student
participation ("Response Rate") and, e.g.
the percentage of responses to the total
number of students who had taken the
discipline).
• Involvement and interest of teachers and
administration of the Higher Education
Institutions in the constructive use of the
results to improve the quality of teaching
and educational programs.
Kyiv - 2020 11

NaUKMA EXPERIENCE
The single portal, developed
following the Project,
guarantees the anonymity and
confidentiality of each
respondent through
Authorization, Access Control,
and differentiation of Access
Levels.
Regular student surveys have been
conducted at the National University of Kyiv-
Mohyla Academy since the mid-1990s and
starting from 2016 they have completely
switched to the online format. These surveys
must be conducted regularly (three times per
year, before each session).
Under the scope of the project, a detailed
analysis of the previous experience of
surveys in NaUKMA was conducted,
especially concerning the mass involvement
of students, the timing of its conduct (this
must be done before obtaining the results of
the exam or test) or the relevance of the
results only to students registered for a
course, monitoring results during longer
(than one semester) period, correlation of
results to overall success, analysis of use
and consideration of results for
acceptance of academic or administrative
events, etc. The methodology of the
Survey (preparation of research tools,
collection, processing, and interpretation
of results, etc.) was analyzed in detail.
Based on the study of the NaUKMA
experience and the relevant research, a
methodological and technical solution
(creation of the Independent Nationwide
Research Portal for Quality Assurance in
Higher Education) is proposed, which
allows Ukrainian universities to introduce
systematic and consistent student surveys
on the quality of teaching.
12

THE LEVEL OF STUDENT
PARTICIPATION
With the transition from paper to online
surveys, the level of student participation
(Response Rate) decreases. Many
solutions have been proposed to solve
the problem, including initiation of the
obligatory participation in the discipline
and face-to-face interviews using tablets
(which turned out to be very expensive).
It emerged that the most effective way to
increase the level of student participation
is to work with the faculty and ask (or
directly formulate a requirement) for time
to fill in an electronic questionnaire during
the last seminar or lecture.
Therefore, information work, including
advertising campaigns, SMM campaigns
and personalized emailing on the
importance of the survey should be
conducted not only with students but also
with the faculty, among whom,
unfortunately, sometimes might be an
opposition to the idea of such surveys.
Increasing the level of student
participation (response rate) and,
accordingly, the uttermost use of the
survey results in the educational process
is possible only through the formation
and dissemination of a culture of such
surveys, understanding that they are not
a temporary whim, but are an integral
part of learning and educational process
at university on the national level and
broader for Academia.
This entails, in fact, about forming a
(corporate/institutional) culture of
conducting such systematic surveys and
ensuring a high level of trust at all stages.
Of course, when it comes to trust, the
issue of avoiding manipulation and
ensuring the anonymity and
confidentiality of respondents becomes
fundamental. Information about students'
answers, their distributions, and survey
results, to which the faculty and
administration have access, is always an
aggregated, generalized information, by
which it is not possible to identify an
individual student.
Student surveys are
an integral part of
university life.
Kyiv - 2020 13

It should be noted that with the initiation
of authorization, the level of student
participation in NaUKMA initially
decreased significantly (almost twice).
This can be partly explained by the
peculiarity of the registration process in
the system, the unwillingness to spend
extra time on registration or perform any
additional actions in general, the
vanishing effect of novelty. Among the
main reasons for low activity was also
the lack of feedback (how the student
surveys affected the review of courses of
a department, the work of a particular
teacher).
In the following stages of the Survey, the
level of participation (Response Rate)
increased slightly (to 21% and then to
43%), considering the average for the
university in all specialties and years of
study. However, it should be noted that
the number of questionnaires in most
disciplines is less than 10, only a few
disciplines from all over the university
had a student participation rate of more
than 60%. On the positive side, we note
that the level of participation among 1-2-
year students is higher than among
senior students, especially in master's
programs.
The constant need to inform students
more about the survey appears relevant,
as well as on the peculiarities of its
conduct and results processing,
guarantees of anonymity in the analysis
of responses, and in general, the role
and importance of survey results on the
educational process improvement.
Over the four years of
online student surveys
on the quality of
teaching and learning at
NaUKMA, the
percentage of trust in e-
services has grown:
from 10-12% of the total
number of students in
the early stages to over
53% in the last year.
We emphasize the importance of the
corporate or institutional culture of surveys
and providing feedback and constructive
criticism in general, both at the level of
students and the level of the faculty and
the administration of the Higher Education
Institutions.
It is important to position student surveys
as not a temporary phenomenon, but an
integral part of learning and, more broadly,
university life, as well as one, albeit very
important, component of the education
quality assurance process.
14

ON THE PECULIARITIES OF THE SURVEY
RESULTS USAGE IN THE SYSTEM OF
EDUCATION QUALITY ASSURANCE
not exceptions, but indicate systematic
successful practices or victories or
problematic moments or shortcomings. At
the same time, when discussing the
results and their use, it should be
emphasized that it is primarily a question
of quality assurance of teaching
disciplines, not teachers.
The same teacher might receive different
"Grades" for the different disciplines,
especially if students of different
specialties or faculties take these courses.
Therefore, "High / Low Grades" of
individual courses for a year or two cannot
be considered as a sufficient basis for
output about teachers, but, of course, is a
signal to the department about the need
to monitor the situation.
It should also be acknowledged that
students of different programs and
different years may have a different
middle or baselines - that is, evaluate all
disciplines critically.
Accordingly, "Assessments" as such are
not sufficient to conclude - their dynamics,
trends, "Open Answers" and relevant
trends must also be considered.
Inquiries about the quality of teaching
cannot be expected to be immediate,
although some expectations can be met
by both students and the faculty,
especially those who are actively involved
in the educational process.
Accordingly, it makes sense to run an
information campaign with the emphasis
that answering the questionnaire now will
help improve the course for students in
the future, and that the survey cannot be
used as an element of "reconciliation" nor
a magic tool for instant improvement.
The results of surveys can be used
meaningfully and productively only by
recording a certain consistency, reliability,
durability, and repeatability of the results;
making sure that the data obtained are
Kyiv - 2020 15

All this indicates the need to develop
policies for dissemination of survey
results, ensuring maximum privacy of all
stakeholders, along with the fullest
possible use of results in the educational
process.
It is also important to emphasize the
guarantees of anonymity and
confidentiality of students who took part in
the surveys, even in situations where they
do not deny the disclosure of their names.
It is essential to form an understanding
that students and teachers are not
antagonists or opponents, but colleagues
and allies who are united by a common
goal and common cause.
Constructive and positive
attitude to the result of all
participants in the educational
process - ensuring and
improving the quality of
teaching and learning - is the
key to success.
It should be noted that there is merit in
supplementing the student surveys on the
quality of disciplines with surveys of the
faculty on their professional activities in
the Higher Education Institutions and
surveys of teachers and students on the
work of administrative departments (for
students - for example, deans).
16

The list of Closed-Ended Statements:
• During the lecture, the teacher explained
the material well.
• The course interested me and
contributed to my intellectual development.
• Overall, I am satisfied with the quality of
the course.
• I am equally satisfied with the lectures
and seminars.
• I am satisfied with the way the teacher
helped me, provided enough comments
and "feedback".
• The teacher encouraged the active
participation of students.
• The thematic material of the course was
well structured.
• The course was sufficiently supplied with
literature, information, materials, and
technical resources.
• The grading system was clear and
consistent with the content of the
discipline.
• Compared to other disciplines, this
was…
• Practical classes and/or seminars helped
me to better understand the content of the
discipline.
• The teacher's assessment was fair.
THE STRUCTURE OF THE QUESTIONNAIRE
ON THE QUALITY OF COURSES IN NaUKMA
Initially, the Survey at NaUKMA was
conducted based on the Questionnaire
with 12 Closed-Ended Statements, with
which students (respondents) had to
agree or disagree (on a scale from 5 to
1).
The volume of the Questionnaire
consisting of 12-14 questions is on the
verge of optimal, and its increase even
by 3-4 questions with a high probability
will reduce the level of answers/fillings.
The Questionnaire also
contained 2 Open-Ended
Questions:
• What did you like most about
studying the discipline?
• What did you not like the
most and what could be
changed?
Kyiv - 2020 17

The question of assessing those
disciplines that are taught by several
teachers (in the most common version -
lectures and seminars are given by
different people) remains as a problem
point of the research. At the stage of the
Questionnaire and the Survey
preparation, it was decided that for such
courses students should fill in the
Questionnaire once, indicating in the
appropriate line the names of both (all)
teachers. In terms of content, the
statements of the questionnaire can be
divided into those related to lectures and
those that are more about seminars. This
approach based on the assumption that
the need to fill in the same questionnaire
for each teacher would reduce the level of
responses in general.
A pilot survey showed that in most cases,
students filled in the Questionnaire either
by indicating the names of two teachers,
as expected, or exclusively by the name
of the seminar teacher. In this case, in
response to the first Open-Ended
Statement, it was often stated that the
lectures were given by another teacher
and the takeaway from these lectures. In
a few isolated cases, students did not
bother to fill in the Questionnaire twice for
each teacher, but all of them recorded a
drastic gap in the assessment of the level
of teaching lectures and seminars (in
most cases - lectures are rated very well,
while seminars - poorly).
However, the need to fill in two
questionnaires (or even three) for the
same subject does not increase the level
of student involvement (response rate)
and the number of answers.
Moreover, it raises the question of how
much students can differentiate
impressions from lectures and seminars
and, accordingly, different teachers from
each other.
The survey is usually conducted during
the last few weeks of the course before
the session and in general, at this time
students have more or less holistic and
integrated impressions of the discipline.
18

EVALUATION SYSTEM
Particular attention in the Survey (the
Questionnaire) was paid to the System of
Evaluation and Fairness of Evaluation.
The analysis of the results, in particular
with the involvement of answers to Open-
Ended Statements, showed that students
disagree on the assessment system and
its adequacy and transparency,
compliance with the content of the
discipline ("The grading system was clear
and consistent with the content of the
discipline") and impressions of the
fairness of the grades (“The teacher's
assessment was fair”).
Whereas the issue of teacher evaluation
and attitude is discussed by students in
Open-Ended Questions, it is very
sufficient that Closed-Ended Questions
can record this.
For example, very often students stated
that the assessment system is adequate
and understandable, but the actual grades
are not fair.
Of course, having an Open-Ended
Questions helps to specify the problem,
for example, there was a clarification
about the availability of the faculty
members, conflict of interests, and so on.
The Questionnaire "Testing" in real
conditions, namely during the several
survey staging points at NaUKMA,
showed that the Statement "I am equally
satisfied with the lectures and seminars"
was ambiguously interpreted by the
respondents and was accordingly
removed from the questionnaire.
Thus, to some extent, the problem of
interviewing in co-teaching disciplines
requires further testing of tools. It is worth
mentioning that different solutions may be
suitable for different Higher Education
Institutions or even different faculties and
specialties.
Kyiv - 2020 19

It should be noted that it is important to
conduct the Survey before the session,
before the final exam, because the actual
grade obtained in the discipline should
not affect the answers of the
respondents.
After the first two stages of the Survey,
the Questionnaire has been edited.
As already mentioned, one question was
removed, in exchange for the question of
readiness to recommend this course to
others, the expected assessment of the
discipline (answer in the A-E range in
complying with European Credit Transfer
and Accumulation System (ECTS) scale)
and the subjective assessment of the
time required to prepare for classes
(Interval Assessment ):
• "I would advise others to enroll in this
course"
• "What grade do you expect to get in this
discipline?"
• "How many hours a week did you spend
preparing for this discipline?"
The use of the Questionnaire in the
following stages of the survey showed
that the Closed-Ended Statements work
well, do not duplicate each other. It
should also be mentioned that in general
the list of the Questionnaire statements
and questions as well as the applied
methodology of both the Questionnaire
and the Survey correspond to and are
consistent with those used in Western
Universities to assure the quality of
teaching disciplines.
Theoretically, one or two (1-2) more
questions can be added to the
Questionnaire. Also, the Questionnaire
can be adjusted to the needs of different
Higher Education Institutions. At the level
of the Independent Nationwide Research
Portal and with the participation of the
National Agency for Higher Education
Quality Assurance (Ukraine), it makes
sense to identify the key issues that will
be in all questionnaires (Digital Core) and
the variable part, which may differ.
Although the level of participation
(Response Rate) in the Survey at
NaUKMA does not allow, in general, to
talk about certain correlations or their
absence, it is possible to mention certain
trends at least for certain disciplines: e.g.,
the level of satisfaction with the course
and willingness to recommend it to other
students. Meanwhile, there is no such
correspondence between the complexity
of the course and / or the expected
assessment and the satisfaction and / or
willingness to recommend it.
However, it should be emphasized again
that these are only preliminary
conclusions or a hypothesis, while higher
student participation (Response Rate) is
generally needed for more reliable and
representative results.
20

FEATURES OF INTERPRETATION
OF THE QUESTIONNAIRE QUALITATIVE
PART (OPEN-ENDED QUESTIONS)
With great credibility, Ukrainian
universities, including NaUKMA, will have
to come a long way to ensure consistently
high levels of student participation
(Response Rate) in the Survey (80%) and
to build an appropriate corporate culture
of participation and trust.
Accordingly, the problem of an insufficient
number of answers to conduct the
quantitative analysis of Closed-Ended
Statements of the Questionnaire is highly
relevant.
At the same time, the inability of the
results quantitative analysis does not
exclude the discussion and interpretation
of qualitative data, namely the answers to
two Open-Ended Questions.
The essential points to remember are the
limitations imposed by the peculiarities of
qualitative methodology - namely, that not
so much the prevalence is being recorded
as the range of opinions and the main
content categories without specifying their
level of prevalence.
Also, the limitation is the inability to draw
unambiguous conclusions about any
course or teacher.
Moreover, with low participation rates
(response rate), the level of open answers
is quite high - it can even exceed 50% (ie
- more than half of the completed
questionnaires have answers to Open-
Ended Questions).
Usually, this figure is up to 25% (up to a
quarter of questionnaires). This is partly
due to the phenomenon of self-selection -
the first and most active students to fill in
the questionnaires are those students
who have something to say about the
disciplines and their teaching - both good
and bad.
Therefore, these students are always
inclined to fill in the questionnaires.
Furthermore, some of them even put their
names (in Open-Ended Questions). This
proves once again the importance of
surveys as a tool/feedback/opportunity to
express and convey students' views to the
faculty, departments, and administration.
Kyiv - 2020 21

From the analysis of NaUKMA's
experience, most of the questionnaires
contain meaningful answers to both
questions, indicating both the positive
aspects of the course and what students
would like to change. A separate category
consists of questionnaires, where the
answer to one of the questions rather
continued the main positive aspect, in
most cases, the assessment of the
course. Together, all this indicates a
rather positive and constructive attitude of
students.
Moreover, if there were several Open-
Ended Questions in a questionnaire about
the course, then, as a rule, students
pointed out similar strengths and
weaknesses, systematically mentioned
the same issues. That is, a certain
repeatable accuracy of answers was
recorded.
At the same time, the presence of several
negative comments does not deny the
high assessment of the course in general
(as well as vice versa, but the opposite
situation is less common).
Naturally, this does not preclude the
presence of single "emissions", both
positive and negative, which may be
evidence of a special subjective attitude.
All this only emphasizes that to use the
obtained results, e.g. by departments, it is
necessary to understand that their
additional "Verification" (Triangulation)
The attitude of students
is
overwhelmingly
constructive, focused on
learning, and provides
continuous
improvement.
from other sources is necessary, which
will show systematicity and durability
concerning these positive or negative
aspects of teaching. Moreover, the
specifics of the survey generally require
repeated results over a long time and
preclude the "Immediacy" of conclusions
based on only one or two surveys.
In addition to their impressions of the
courses and teaching, students have
mentioned about their success or
problems encountered while taking the
course, expressed admiration or
disagreement, wrote reflections on the
specifics of the curriculum and
organization of training, reflected on their
own experience and experience of
colleagues. The general impression from
reading the answers to Open-Ended
Questions convinces that the attitude of
students is mostly constructive and
focused on learning and continuous
improvement. It seems that this attitude of
students should be appreciated and
nurtured, through feedback.
22

• Structure of the course - first of all, the
presence of Thematic Work Plan /
Syllabus and its observance, logical and
systematic presentation of material,
consistency of lectures, seminars and
practicality of classes, correspondence of
the name and annotation of the discipline
to its actual content and educational goals
(focus on achieving or developing), clarity
of the content and essence of educational
tasks, in particular, written works,
availability of methodical instructions and
requirements to them. This is exactly what
students mention in the first place when
pointing out the shortcomings of the
course.
• The teacher's attitude to students and
the evaluation system - its intelligibility
and transparency and its fairness is
pointed out, also indicating bias or
inappropriate attitude towards students.
As a rule, the answer to two Closed-
Ended Statements of the questionnaire
about the evaluation system is revealed in
depth. In general, mainly the systematic
and consistent remarks (repeated in time
and on different stages) and within a
separate sample group (within one stage)
about the shortcomings and strengths are
ANALYSIS & INTERPRETATION
The detailed analysis of answers to Open-Ended Questions
allows us to group them by main topics or issues. Among them:
the key points for the University to pay
attention to improve the quality of
education.
• The level of interest of the teacher.
Disadvantages that do not depend on the
faculty and the department, namely
comments on the place and time of
classes, logistics (including laboratory
equipment and the convenience of course
websites), the number of students in
groups. It is worth noting that such
comments cannot be rejected or ignored.
Whenever possible, the wishes of
students should be taken into account,
where the situation can be improved by
the joint efforts of the faculty, university
administration, teachers, and the
department.
• Therefore, answers to Open-Ended
Questions can and should be used in the
process of the quality assurance in higher
education, in case they are carefully and
professionally interpreted, even when the
level of student participation (Response
Rate) in the survey is 30-40% and
insufficient for quantitative (statistical)
analysis of answers to Closed-Ended
Statements of the questionnaire.
Kyiv - 2020 23

24
FUNCTIONALITY OF AN INDEPENDENT
NATIONWIDE RESEARCH PORTAL FOR
QUALITY ASSURANCE IN HIGHER
EDUCATION
With the help of the portal, educational
system users and managers will conduct
the focused research and surveys of a
wide range of the system beneficiaries.
According to the survey results, the portal
will form an appropriate analysis of the
external quality assurance of education
system. Also, the portal functionality will
inform the National Agency for Higher
Education Quality Assurance and the
Ministry of Education and Sciences of
Ukraine on operational issues in
education, track trends and tendencies.
It will also ensure that decision-making
criteria are in place in the field of higher
education and will facilitate the
establishment of accessible and
understandable reporting, supported by
disregarded data.
The research results will inform about the
need to conduct operational inspections of
quality assurance systems and
mechanisms for working with the received
recommendations.

Kyiv - 2020 25
on direct curricula,
on the quality of education in the
relevant programs,
and on their university overall.
In the further steps of the portal
development, special attention will be paid
to the development of a module for
monitoring the effectiveness of quality
assurance procedures for educational and
professional training, as well as
automation of preparation and publication
of statistical information for reports on
quality of higher education in Ukraine.
At another point, the beneficiaries of the
system will have 24/7 access to a direct
impact on the quality of education and
their own experience in the higher
education system of Ukraine.
They will have the opportunity to leave
feedback:
They will also be able to respond to more
strategic decisions in the education quality
assurance system.
At the university level, heads of
institutions will have access to the
appropriate level of analytics, which will
inform them about possible strategic and
operational decisions to ensure the quality
of education in their universities.
Access to the portal will be restricted to
users of the education system only. They
will be properly authenticated in the
system.
This will protect against the provision of
inaccurate data, as well as ensure the
confidentiality of data, which will eliminate
any dishonest impact.
In the future, it will be possible to add
login through other providers, such as
Google, if the Higher Education
Institutions is using the corporate
environment (the provider must support
the OpenID Connect protocol).
It is also possible to establish cooperation
with id.gov.ua to use other means of
identification: EDS, etc.).

26
THE MAIN STEPS
OF THE PORTAL MANAGEMENT
Step 1
Login to the system under the roles
provided in the system is carried out by
following the hyperlink to the
authentication page.
At the MVP stage, the login is
implemented through the Office 365
authentication service. We chose this
method of Authentication and
Authorization for several reasons. In
2014, Microsoft developed a program for
Ukrainian higher education institutions,
according to which students and teachers
began to use the Office 365 platform on
preferential terms. Using a corporate
email address to log in allows
administrators in Higher Education
Institutions accurately identify their user,
and higher education seekers can
securely access the system.
The service development roadmap
provides the use of Google+ and BankID
for authorization (depending on the
dynamics of this authentication method
implementation in Ukraine).
After filling in the required data (password,
login, etc.), the system provides
information about the user (name,
surname, e-mail, etc.) and uses this
information to authorize the user in our
system.

Step 2
Kyiv - 2020 27
the choice of disciplines for the survey
on the quality of teaching;
the page on which the survey results
are posted;
the feedback form;
settings for Higher Education
Institutions (for administrators).
The user gets to the main page, where,
depending on his role in the system, the
user has access to the functionality of the
portal:
At the top of the page the user has access
to a menu with information about this user
and buttons to go to the main menu and
settings page.
At the bottom of the page, the user has
access to a block with information about
the owners and organization.
The user will also have access to the
button, clicking on which will provide this
user with help on how to use the service.

28
Step 3
Presentation of the discipline choice and
questionnaire.
The feedback page provides the user with
a list of available disciplines and writing a
review.
By clicking on the button next to the
selected disciplines, the user goes to
the questionnaire filling page.
If the user has already filled out the
questionnaire for a specific discipline, the
fill button will be replaced by an edit button
that allows user to edit the data that was
entered in the questionnaire for a specific
discipline.

An employee of the university's quality
assurance department has an additional
option "View results" in the main menu,
clicking on which the user goes to the
university selection page to get the results
of a survey on the discipline of a particular
university that interests him. Portal
guarantees results anonymity.
Kyiv - 2020 29
Step 4
Step 5
The discipline results page displays
statistics in the form of graphs and
charts, and it is possible to download
the report form.

30
Step 6
If the user fills in the questionnaire for the
first time, the questionnaire is presented in
the form of various fields with blank or
unselected data.
If the user has already filled in the
questionnaire, the text or variants of the
completed questions will already be given.
After all questions are answered, at the
bottom of the page there will be a button
"Send Feedback", which groups all the
answers and sends them to the server and
the button "Cancel", which completely
deletes all the data entered this time and
goes to the feedback page.
Step 7
If the user selects "Leave Feedback",
user gets to a new page. The page
displays an input form in which the
user writes his question, requests or
feedback on the use of the system.
If the user is not registered, this user
must indicate his contact details in the
feedback form. To send a "System
Feedback", the user checks the
entered data and presses the
confirmation button.
The functionality of the Independent
Nationwide Research Portal for
Quality Assurance in Higher
Education allows a wide range of
users to influence the quality of
education in Ukraine.

Further steps in the development
of the portal's functionality include
the following stages (Product Roadmap):
• development of an online system for monitoring the
effectiveness of quality assurance procedures for
educational and professional training;
• development of an online system for accreditation of
education and training programs;
• creation and maintenance of an IT register of educational
and training programs.
Kyiv - 2020 31
PRODUCT ROADMAP

QUALITY OF EDUCATION
IN THE UNIVERSITIES OF UKRAINE:
STUDENT ASSESSMENT. AS PART OF THE PROJECT
IMPLEMENTATION «UKRAINIAN QUALITY ASSURANCE E. SERVICE
FOR EDUCATION»: 2019-2020 / KVIT S. , YAROSHENKO T. ,
KOSTYUK O. , OSYPCHUK H. , YAROSHENKO O. , NGO ELIBUKR. –
KYIV, 2020. - 28 P.
PHOTO CREDENTIALS:
NATIONAL UNIVERSITY OF KYIV-MOHYLA ACADEMY,
OLEKSANDRA LYSTOPAD (NAUKMA SCIENTIFIC LIBRARY),
OLEKSANDRA YAROSHENKO (ELIBUKR),
ARTEM SAKH (NAUKMA),
IMAGES WITH OPEN LICENSE - PEXELS (PEXELS.COM)

QUALITY OF EDUCATION IN THE UNIVERSITIES OF UKRAINE: STUDENT ASSESSMENT

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to QUALITY OF EDUCATION IN THE UNIVERSITIES OF UKRAINE: STUDENT ASSESSMENT

Similar to QUALITY OF EDUCATION IN THE UNIVERSITIES OF UKRAINE: STUDENT ASSESSMENT (20)

Recently uploaded

Recently uploaded (20)

QUALITY OF EDUCATION IN THE UNIVERSITIES OF UKRAINE: STUDENT ASSESSMENT