ReportAssessment of public universities and their faculties                       (2005)           (c) ARRA, Bratislava 20...
The report includes results of the project „Quality assessment                     of research and development at the univ...
The authors of this report would like to thank the members of the Boardof Trustees of ARRA, the Board of Advisors of ARRA,...
Introduction        When officials started to assess research in higher education in Great Britain in1980 (the Research Se...
six-year cycles. This means that in our fast changing world the information used     in a given evaluation may not necessa...
1.     ARRA – independent evaluation of               universities      The Academic Ranking and Rating Agency (ARRA) is t...
•    the division of faculties into six groups according to the so-called                      Frascati Manual (details gi...
medium term. To this end, universities need to ensure balanced quality in their     educational and research activities. W...
2.     The history and role of ranking and                  experience of it in other countries          The history of ra...
accommodation, the costs associated with study, the attractiveness of the place     of study, the options for sports, cult...
of higher education, it seems natural that we should ask which of them is the      best in their group.         This issue...
components. Another reason for the use of faculties as the basic unit for     ranking is the fact that there is practicall...
3.    Classification of faculties      ARRAs approach in assessing Slovak universities is the same as is usedelsewhere in ...
Table1: Classification of faculties (or universities) into subject areas according to ARRA10University                    ...
University                                Group     Faculty                                          AbbreviationSlovenská...
University                                 Group     Faculty                                          AbbreviationUniverzi...
University                                   Group   Faculty                                         Abbreviation       Un...
4.        Quality criteria                ARRA selected a number of criteria from those initially proposed anddiscussed wi...
in this report but they are not included in the final score for the universities. The     reason why several of the approv...
5.     Basic characteristics of the indicators and                their classification into groups     The indicators that...
supervise PhD students (and a small number of holders of scientific ranks IIa       and I, about whom there is no informat...
SV2         The number of students studying full and part time divided by the number ofteachers. Points are awarded in the...
SV10      Number of students taking part in study abroad through the Erasmus andSocrates programs and using SAIA grants, p...
6.     Method for the assessment of faculties and              universities       The indicators listed in Chapter 5 have ...
Method for the assessment of faculties and universities      An example for illustration purposes may be the indicator VV1...
Table 5: Assessment of a university by individual areas and overall      Faculty                                          ...
groups, relatively homogenous groups within the faculty corresponding     approximately to fields of studies. Such an anal...
7.      Commentary on individual indicators16      Science and research"Publications and citations" Group17VV1 Number of s...
Table 6: Number of papers for the years 1995 – 2004 per one creative worker (situation in 2004)       Group               ...
chapters in them are produced mainly from the results of the scientific papers of     their author or authors. These scien...
0,0                               0,5                                            1,0                                      ...
Faculty group – TECH   ŠpecTechn TUAD          0,00        ŠpecInž ŽU         0,00       Archit STUBA        0,02         ...
Faculty group – HUM    VýtvarUm AU        0,00        VŠVU BL        0,00      Umení TUKE 0,00        Teol.TVU       0,00 ...
Faculty group – SOC   Zdravotnícka PU     0,00         Športu PU     0,00   Sredoeur.Št.UKF     0,00       SocEkon UK     ...
VV2            Number of citations per CW according to the WoS database              Citations represent another indicator...
be extremely high. We believe, however, that the removal of self-citations      would not have a significant effect on the...
Table 8: Average number of citations per paper for Slovak public universities, 1995 – 2004         University             ...
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Assessment HEI 2005
Upcoming SlideShare
Loading in...5
×

Assessment HEI 2005

1,040

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,040
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Assessment HEI 2005"

  1. 1. ReportAssessment of public universities and their faculties (2005) (c) ARRA, Bratislava 2005 1
  2. 2. The report includes results of the project „Quality assessment of research and development at the universities and institutes of Slovak Academy of Sciences in Bratislava Region“, financed by European Social Fund.This report originated also with financial aid of the following organizations: www.pss.sk www.luba.sk www.orange.sk www.worldbank.sk www.yhman.sk www.hp.skMedial partner of ARRA is:www.trend.sk
  3. 3. The authors of this report would like to thank the members of the Boardof Trustees of ARRA, the Board of Advisors of ARRA, in particular prof. Brunovský,prof. Šucha, prof. Štich (the chairperson of the Board of Advisors), doc. Ferák, prof.Šlachta and others, and also Ing. M. Beblavý, the members of the Board of Trustees(in particular the chairperson Ing. J. Kollár) for their critical but always relevantcomments, and also for frequent discussions, analyses and reviews of draft materialin the production of the report. One of the factors that enabled ARRA to produce thefollowing analysis was the fact that the SR Ministry of Education publishes a greatdeal of material and information about higher education. We would like to thank doc.RNDr. Peter Mederly, CSc. for his valuable discussion of this material. Conversationswith Don Thornhill and Lewis Purser, experts that the World Bank arranged forARRA, made a significant contribution to developing the criteria and the methodologyused in the report, and they also deserve our thanks. We would like to point out,however, that the opinions presented in the following report are not necessarilyidentical with those of the persons named here. The ARRA Agency was able to carry out its activities thanks to contributionsfrom its sponsors, in particular Prvá stavebná sporiteľňa, Orange Slovensko, ĽudováBanka, the World Bank, HP and others. The assessment of the work of the Bratislavafaculties of universities was supported by a grant from the European Social Fund. 2
  4. 4. Introduction When officials started to assess research in higher education in Great Britain in1980 (the Research Selectivity Exercise [RSE], later the Research AssessmentExercise [RAE]), their aim was not only to increase universities accountability to thegeneral public for their activities but also to map their scientific activity and compareit with that of foreign institutions. It is noteworthy that although this process thatwas initially greeted with a degree of mistrust and resistance (like every reform) bythe academic community in particular, not only did it bring about an increase in thenumber of relevant scientific publications and citations1, but in 2000, when the mostrecent assessment took place, 55% of higher education workplaces were ranked inthe highest category of assessment. One of the factors in the success of the Britishsystem was that the results of assessment were and are directly linked to thefinancing of higher education2. This experience shows that the assessment of qualityis an instrument for achieving goals and not a goal in and of itself. In addition, inother countries as well as Great Britain there is an independent system for theevaluation of universities, which determines their ranking according to predeterminedcriteria. In Slovakia there is as yet no functioning system of this sort. There has beenno institution or organisation (either state or non-state) that would systematicallyaddress the evaluation of research and educational performance in higher educationaccording to clear criteria set in advance that the general public can understand,drawing on information that is in the public domain or which can be easily obtained(e.g. using Act no. 211/2000 Coll. on free access to information). This is not a role carried out even by the Accreditation Commission, thegovernments advisory body. The Accreditation Commission has different tasks andobjectives3. At present, although the SR Ministry of Education is trying to introduceRAE-like procedures in the complex system for accrediting universities, this is takinga relatively long time to be put into practice and make an impact. This system mayalso seem a little ponderous to the general public, as it is planned to carry it out in1 The Times Higher Education Supplement (THES), 1 July 2005, p. 4.2 Adams J., Science 296, May 2002, p. 805.3 Section 81 and following of Act no. 131/2002 Coll. on Higher Education and changes and additions to certainacts. 3
  5. 5. six-year cycles. This means that in our fast changing world the information used in a given evaluation may not necessarily reflect the current situation in thehigher education market. In Slovakia as in other countries, there is increasing pressure to enable thegeneral public (including prospective university students, their parents andemployers) and even the government and parliament to gain fast and easy access toa comprehensible overview of selected indicators that characterise or describe thequality of universities and/or their faculties. This pressure has increased sharply sincethe government tried to introduce financial cooperation by students in theireducation (tuition fees) as part of their reform package. This pressure also led to thecreation of the civil association the Academic Ranking and Rating Agency (ARRA;www.arra.sk) at the end of 2004, although the idea of it has been forming for anumber of years. It is not surprise that it was founded by former student leadersfrom the 1996 – 2002 period, who gained their own experience of study not only inSlovak higher education, but also abroad. This allowed them to compare the qualityof tuition at home and abroad and to see that Slovak higher education is really inneed of change. It needs it above all to enable its institutions to compete in an openEuropean market against universities, which are already actively recruiting ourstudents (e.g. branches of foreign universities but also Czech and Austrianuniversities). 4
  6. 6. 1. ARRA – independent evaluation of universities The Academic Ranking and Rating Agency (ARRA) is the first independentSlovak institution to decide to provide the public with regular reports on the qualityof individual universities in Slovakia and information comparing them with eachother. ARRA provides this information by fulfilling its objectives and abiding byquality evaluation procedures agreed in advance. The objectives of the ARRA civil association include: • providing the public with information on the quality of individual universities in Slovakia, • introducing a method of assessing the quality of education provided by universities in Slovakia, • creating an independent evaluation of the quality of education provided in individual study programs and fields of study in universities in Slovakia, • regularly ranking universities, the affiliated faculties and sections by the quality of the education the provide and the quality of their research and development ("ranking"), • assigning universities a rating based on the level of quality of their individual activities, • attempting to stimulate competition between individual universities and their faculties. The role of the agency is not to replace the duties and tasks of the SRgovernment Accreditation Commission. The procedure that ARRA has used in assessing public universities (whentalking about Slovak universities from now on, only public institutions will be referredto) in Slovakia in 2004, was based on the following steps: • the selection of indicators for the quality of education and research in individual universities and the assignment of a certain number of points to each faculty for the performance in this or that indicator (indicators are arranged into groups and each group of indicators gained a certain number of points), 5
  7. 7. • the division of faculties into six groups according to the so-called Frascati Manual (details given below) in order to compare only faculties that have the same orientation and similar working conditions, • assigning points scores to faculties (the ranking of faculties in individual groups according to the Frascati Manual is based on average points score in individual groups of indicators), • the calculation of the points score for universities (universities are ranked according to the average points gained by their faculties). The criteria (or indictors) by which ARRA produced its ranking focus on theintensity of "performance" and not on overall performance. For example, one of thecriteria is the total number of publications by the given faculty listed in the Web ofKnowledge database produced by the company Thomson Scientific Co. divided bythe number of creative workers in the faculty (teachers and researchers). If thenumber of creative workers did not divide the overall number of publications, the sizeof the faculty would be the main influence and not the intensity of its work. The selection of criteria was also influenced by the fact that ARRA used onlyinformation in the public domain and did not, at least in its 2004 evaluations, requestinformation from individual faculties. ARRA views the assessment of the performance of universities and facultiesfor 2004 as a pilot project. ARRAs results for 2004 will be analyzed in detail and thisanalysis will be used to prepare the assessment of public universities in future years. The ranking produced for the assessed institutions is drawn fromincontrovertible official data and domestic and foreign sources in the public domain. Since 1 May 2004, Slovak universities are part of the EU higher education areaand this has brought with it not only rights but also obligations, including theobligation to maintain and increase the quality of all their activities. Slovakia fullyaccepts the much-cited Bologna Process, which aims, amongst other things, toincrease the competitiveness of European higher education in comparison with otherparts of the world and has even incorporated it into its legislative system4. Qualityhigher education is the key to building a knowledge economy in Slovakia and willbring about the necessary structural changes in the countrys economy over the4 Act no. 131/2002 Coll. on Higher Education and changes and additions to certain acts. 6
  8. 8. medium term. To this end, universities need to ensure balanced quality in their educational and research activities. We are convinced that full use should bemade of any instruments that can contribute to the achievement of the objectives forthe increased quality of Slovak universities so that Slovak universities keep up withthe national and international competition and attract students from both Slovakiaand abroad. 7
  9. 9. 2. The history and role of ranking and experience of it in other countries The history of ranking is not a very long one. The first institutions to beranked were a selection of American colleges in the magazine U.S. News & WorldReport in 1983. The phenomenon grew rapidly however, and not only in the Anglo-Saxon countries (e.g. in Great Britain in the Times Higher Education Supplement, theTimes Good University Guide, Guardian, Financial Times etc., in Canada McLeans),but also in other parts of the world (in Germany Stern, Spiegel and DAAD, in JapanAsiaweek [until 2001], Asahi Shimbun, in Poland Wprost, Perspektywy, Polityka, inSwitzerland Swisup!, in China the Higher Education Institute of Shanghai Jiao TongUniversity among others). There is quite a good overview of the sources on rankingon the Internet5, where one can also find very specific evaluations e.g. for lawfaculties or institutions providing education in economics or MBA degrees (Master ofBusiness Administration). In fact, even in Slovakia we have noticed attempts bysome media to produce such an evaluation (SME, Hospodárske Noviny, TREND,.týždeň, Národná obroda, Pravda, TA3, Slovenský rozhlas, STV...), although thesehave been incomplete and focused on only a part of the problem. The basic and common characteristic of all foreign approaches is that theyattempt to provide information to the general public, in particular to prospectivestudents, in order to help them select a suitable school or faculty and/or area ofstudy or study program. Ranking is not in principle an activity carried out for its ownsake to put universities into "some sort of" order (although such evaluations are alsopublished) but ranking is or tries to be an aid for people who are choosing a place tostudy and need orientation in the education marketplace. No ranking system canautomatically identify the best faculty or university for an applicant but theinformation provided can be of great assistance in their decision-making. Researchhas shown that at present that the position of a school in this or that ladder is asecondary factor in their choice of where to study. The main factor is whether thegiven faculty or university will enable them to study the area of study or studyprogram that they are interested in. It is only after that – if there are a number ofproviders – that they start to consider other criteria such as the availability of5 http://ed.sjtu.edu.cn/rank/2004/Resources.htm 8
  10. 10. accommodation, the costs associated with study, the attractiveness of the place of study, the options for sports, cultural and other activities and also thestanding of the faculty or the university among other institutions. In addition toproviding information to the public and prospective students, assessments of qualityaim to increase competition among universities through their output so that – as inthe worlds advanced economies – prospective students are guided not only by thegeographical proximity of a university but also the quality of the education that itprovides. We expect that the importance of quality factors will increase hand in handwith the complexity and the technical and intellectual demands of the Slovakeconomy. The second common characteristic of assessments is that although they usedifferent numbers of criteria, there is nearly always a relatively small number ofgroups of criteria that reflect the institutions performance in research and educationand related parameters, perspectives on the institution from within (students and theacademic community) and also from outside (e.g. employers, or even the schoolsown graduates). Another very important characteristic of ranking is that every such assessmentmust always be looked at in terms of the criteria that it uses. In other words, everyranking corresponds only to the criteria that are chosen and used. In addition everycompiler (but also every reader) may consider (and then also makes) certainindicators significantly more important than others (e.g. research performance isgiven a higher value than, say, the number of students per teacher) and maytherefore give them a different weighting, which can in turn make a fundamentaldifference in the ranking of the faculties and universities in "his or her" resultingtable. For this reason ranking provides an image of the university and its statusalongside other universities entirely in terms of the selected indicators. For thisreason also, everyone involved in creating ranking tries to choose a range of criteriaand indicators that will be relevant to the broadest possible group of recipients. A number of universities insist that this form of comparison has limitedsignificance because every university is unique and special in some way, or has itsown specific characteristics. We can agree with the claims of uniqueness to someextent. However, if there are, for example, 4 faculties of law, 5 faculties ofphilosophy or 3 faculties of medicine, providing the same degree for the same form 9
  11. 11. of higher education, it seems natural that we should ask which of them is the best in their group. This issue has given rise to other common arguments against ranking, that itis not possible to compare the incomparable, such as universities with 13 differentfaculties and a highly focused institution which is not even divided into faculties. Thisobjection is acceptable. There is, however, a reasonable solution to this problem.Since 1963, the OECD countries (the Organization for Economic Cooperation andDevelopment) have categorized systems of subjects in science and technologyaccording to the Frascati Manual6, whose current version dating from 20027 dividessuch subjects into 6 groups:a) natural sciences,b) engineering and technology,c) medical sciences,d) agricultural sciences,e) social sciences,f) humanities.From 1 June 2005 this division will be included in Slovak law8, i.e. also in the Slovakresearch community.9 After the introduction of such a division into the ranking, it is clear thattheological faculties will not be compared with medical faculties or technicallyoriented faculties with social science faculties. It will be possible to compare facultieswith the same scientific orientation side by side. Prospective students will thus beable to determine which is the best faculty providing education in their area ofinterest. The quality of a university is also determined by the quality of its faculties(apart from the University of Veterinary Medicine and the Academy of Fine Arts,which have no faculties and are therefore assessed as a whole). It is therefore quitenatural that the final ranking of universities depends on the quality of its6 The authorship of this manual is quite often attributed incorrectly. Because the first meeting of OECD experts inthis matter took place in 1963 in the Italian village of Frascati, the work that was created there was called theManual from Frascati or The Frascati Manual.7 Frascati Manual, 6th Edition, OECD 2002, Paris, p. 67.8 Act no. 172/2005 Coll. on the organisation of state support for research and development and additions to Actno. 575/2001 Coll. on the organisation of government activities and the organisation of the central stateadministration as amended.9 A more detailed breakdown is given in the International Standard Classification of Education (ISCED) 1997,UNESCO, November 1997. 10
  12. 12. components. Another reason for the use of faculties as the basic unit for ranking is the fact that there is practically no university covering all fields ofstudy in Slovakia. When assessing such universities it can also happen that the finalresult will be a certain weighted average of the quality of individual components ofthe university and that the weighting of the excellent faculties may not be enough tobalance the weaker ones, or vice versa, the quality of the excellent faculties maycover for the weaker ones. Precisely for the reason the ranking of universities iscarried out at the end, to make clear their strengths and weaknesses. It is up to theuniversity itself whether to take measures to increase the quality of its components,so that the better faculties do not suffer because of the weaker ones. That such asystem is able to function successfully is shown by the example of Great Britain andits RAE system. In the final analysis, it is up to prospective students to decide, just asemployers must decide when employing graduates. And it is always better ifdecisions are taken on the basis of accessible and verified data rather thantraditional, often inaccurate and incomplete information or rather feelings associatedwith to this or that institution. An essential public requirement is also an answer to the question as to whatuse universities make of their public funding and how effectively they manage them.For example, for 2003, the National Council of the Slovak Republic approvedsubsidies for public funds to public universities amounting to SKK 8.3 billion, for 2004SKK 9.4 billion and an extra billion for 2005 This means that in three yearsuniversities spent at least SKK 28.1 billion from public funds. The proposed statebudget for 2006 allocates SKK 11.6 billion to public higher education. It is reasonableto ask what quality students, graduates and society as a whole gets from this money,i.e. whether and to what extent these funds really improve this segment of theresearch and education process for which they were originally intended. 11
  13. 13. 3. Classification of faculties ARRAs approach in assessing Slovak universities is the same as is usedelsewhere in the world and is based on a number of pillars. The first is quantitativeinformation in the public domain, which is generally accepted as a reliable indicatorof academic quality. The second is an independent viewpoint and results. The third isa group (cluster) approach to the assessment of faculties and universities. Thefaculties of Slovak universities were divided into groups based on their field of studyusing the definitions given in the Frascati Manual as follows: • natural sciences (NAT) consisting of mathematics and computer sciences, physical sciences, chemical sciences, biological sciences and Earth and related environmental sciences • engineering and technology (TECH) consisting of civil engineering, electrical engineering, mechanical engineering and other engineering and technological sciences, • medical sciences (MED) including basic medicine and dentistry, clinical medicine and pharmaceutical sciences (nursing and healthcare are included in the social sciences), • agricultural sciences (AGRO) consisting of agriculture, forestry, fisheries, veterinary medicine and allied subjects, • social sciences (SOC) including psychology, economics, educational sciences, law, political science, nursing, healthcare, other social sciences. • humanities (HUM) are history, languages and literature and other humanities. Some higher education faculties are hard to categorize into a particularsubject area due to the diversity of their components (e.g. departments), which carryout activities lying in multiple subject areas. An example is the faculties ofphilosophy, whose activities cover the areas of both the humanities and the socialsciences. They were classified according to the subject area into which the majorityof activities fall. If the dean of a particular faculty contacts us to request that thefaculty that he or she manages should be classified in a different subject area, ARRAconsiders the request. 12
  14. 14. Table1: Classification of faculties (or universities) into subject areas according to ARRA10University Group Faculty AbbreviationAkadémia umení (Academy of Arts) AUBanská Bystrica HUM Fakulta dramatických umení DramUm AUwww.aku.sk (Faculty of Dramatic Arts) HUM Fakulta muzických umení MuzUm AU (Faculty of Performing Arts) HUM Fakulta výtvarných umení VýtvarUm AU (Faculty of Fine Arts and Design)Ekonomická univerzita (University of Economics) EU BABratislava SOC Fakulta hospodárskej informatiky HospInfo EU BAwww.euba.sk (Faculty of Economic Informatics) SOC Fakulta medzinárodných vzťahov MedzVzťah EU BA (Faculty of International Relations) SOC Fakulta podnikového manažmentu PodnMan EU BA (Faculty of Business Management) SOC Národohospodárska fakulta NárHosp EU BA (Faculty of National Economy) SOC Obchodná fakulta Obchod EU BA (Faculty of Commerce) SOC Podnikovohospodárska fakulta PodnHosp EU BA (Faculty of Business Economics)Katolícka univerzita (Catholic University) KURužomberok HUM Filozofická fakulta Fil KUwww.ku.sk (Faculty of Philosophy) SOC Pedagogická fakulta Pedag KU (Pedagogical Faculty) HUM Teologická fakulta Teol KU (Faculty of Theology)Prešovská univerzita (University of Prešov) PUPrešov HUM Fakulta humanitných a prírodných vied HumPrír PUwww.unipo.sk (Faculty of the Humanities and Natural Sciences) SOC Fakulta manažmentu Manag PU (Faculty of Management) SOC Fakulta športu TV PU (Faculty of Sports) SOC Fakulta zdravotníctva Zdravotnícka PU (Faculty of Health Care) HUM Filozofická fakulta Fil PU (Faculty of Arts) HUM Gréckokatolícka bohoslovecká Greckokat PU (Greek Catholic Theological Faculty) SOC Pedagogická fakulta Pedag PU (Faculty of Education) HUM Pravoslávna bohoslovecká fakulta Pravosl PU (Orthodox Theological Faculty)Slovenská poľnohospodárska univerzita (Slovak University of Agriculture) SPUNitra AGRO Fakulta agrobiol. potr.zdr. Agro SPUwww.uniag.sk (Faculty of Agrobiology and Food Resources) AGRO Fakulta biotechnológie a potravinárstva BiotPotr SPU (Faculty of Biotechnology and Food Sciences) SOC Fakulta ekonomiky a manažmentu EkonomMan. SPU (Faculty of Economics and Management) SOC Fakulta európskych štúdií Eur.ŠT. SPU (Faculty of European Studies and Regional Development) AGRO Fakulta záhradníctva a krajinného inžinierstva Záhrad SPU (Faculty of Horticulture and Landscape Engineering) TECH Mechanizačná fakulta Mech SPU (Faculty of Agricultural Engineering) 13
  15. 15. University Group Faculty AbbreviationSlovenská technická univerzita (Slovak University of Technology) STUBABratislava TECH Fakulta architektúry Archit STUBAwww.stuba.sk (Faculty of Architecture) TECH Fakulta elektrotechniky a informatiky Elektr STUBA (Faculty of Electrical Engineering and Information Technology) TECH Fakulta chemickej a potravinárskej technológie ChemTechn STUBA (Faculty of Chemical and Food Technology) NAT Fakulta informatiky a informačných technológií Infor.aInf.Tech. STUBA (Faculty of Informatics and Information Technologies) TECH Materiálovotechnologická fakulta MatTechn STUBA (Faculty of Material Sciences and Technology) TECH Stavebná fakulta Stav STUBA (Faculty of Civil Engineering) TECH Strojnícka fakulta Stroj STUBA (Faculty of Mechanical Engineering)Technická univerzita Košice (Technical University of Košice) TUKEKošice SOC Ekonomická fakulta Ekonom TUKEwww.tuke.sk (Faculty of Economics) TECH Fakulta BERG Ban TUKE (Faculty of Mining, Ecology, Process Control and Geotechnology) TECH Fakulta elektrotechniky a informatiky Elektr TUKE (Faculty of Electrical Engineering and Informatics) HUM Fakulta umení Umení TUKE (Faculty of Arts) TECH Fakulta výrobných technológií VýrTech TUKE (Faculty Manufacturing Technologies) TECH Hutnícka fakulta Hutn TUKE (Faculty of Metallurgy) TECH Stavebná fakulta Stav TUKE (Faculty of Civil Engineering) TECH Strojnícka fakulta Stroj TUKE (Faculty of Mechanical Engineering)Technická univerzita Zvolen (Technical University in Zvolen) TUZVZvolen AGRO Drevárska fakulta Drev TUZVwww.tuzvo.sk (Faculty of Wood Sciences and Technology) NAT Fakulta ekológie a environmentalistiky Ekolenv TUZ (Faculty of Ecology and Environmental Sciences) TECH Fakulta environmentálnej a výrobnej techniky EnvirTech TUZV (Faculty of Environmental and Manufacturing Technology) AGRO Lesnícka fakulta Les TUZV (Faculty of Forestry)Trenčianska univerzita A. Dubčeka (Alexander Dubček University in Trenčín) TUADTrenčín TECH Fakulta mechatroniky MechTron TUADwww.tnuni.sk (Faculty of Mechatronics) TECH Fakulta priemyselných technológií PriemTechn TUAD (Faculty of Industrial Technologies) SOC Fakulta sociálno-ekonomických vzťahov SocEkon TUAD (Faculty of Social and Economic Relations) TECH Fakulta špeciálnej techniky ŠpecTechn TUAD (Faculty of Special Technology)Trnavská univerzita (University of Trnava) TVUTrnava SOC Fakulta zdravotníctva a sociálnej práce ZdravSoc TVUwww.truni.sk (Faculty of Health Care and Social Work) HUM Filozofická fakulta Fil TVU (Faculty of Arts) SOC Pedagogická fakulta Pedag TVU (Faculty of Education) SOC Právnicka fakulta Práv TVU (Faculty of Law) HUM Teologická fakulta Teol TVU (Faculty of Theology) 14
  16. 16. University Group Faculty AbbreviationUniverzita Komenského (Comenius University) UKBratislava HUM Evanjelická bohoslovecká fakulta Evanj UKwww.uniba.sk (Evangelical Theological Faculty) SOC Fakulta managementu Manag UK (Faculty of Management) NAT Fakulta matematiky, fyziky a informatiky FMFI UK (Faculty of Mathematics, Physics and Informatics) SOC Fakulta sociálnych a ekonomických vied SocEkon UK (Faculty of Social and Economic Sciences) SOC Fakulta telesnej výchovy a športu TV UK (Faculty of Physical Education and Sports) MED Farmaceutická fakulta Farm UK (Faculty of Pharmacy) HUM Filozofická fakulta Fil UK (Faculty of Arts) MED Jesseniova lekárska fakulta JessenLek UK (Jessenius Faculty of Medicine) MED Lekárska fakulta Lek UK (Faculty of Medicine) SOC Pedagogická fakulta Pedag UK (Faculty of Education) SOC Právnicka fakulta Práv UK (Faculty of Law) NAT Prírodovedecká fakulta Prír UK (Faculty of Natural Sciences) HUM Rímskokatolícka cyrilomet. bohoslovecká fakulta RímsKat UK (Roman Catholic Theological Faculty of Cyril and Methodius)Univerzita Konštantína Filozofa (Constantine the Philosopher University) UKFNitra NAT Fakulta prírodných vied Prír UKFwww.ukf.sk (Faculty of Natural Sciences) SOC Fakulta sociálnych vied Soc UKF (Faculty of Social Sciences and Health) SOC Fakulta stredoeurópskych štúdií Stredoeur.Št. UKF (Faculty of Central European Studies) HUM Filozofická fakulta Fil UKF (Faculty of Arts) SOC Pedagogická fakulta Pedag UKF (Faculty of Education)Univerzita Mateja Bela (Matej Bel University) UMBBanská Bystrica SOC Ekonomická fakulta Ekonom UMBwww.umb.sk (Faculty of Economics) SOC Fakulta financií Finan UMB (Faculty of Finances) HUM Fakulta humanitných vied Hum UMB (Faculty of the Humanities) SOC Fakulta politických vied a medzinár. vzťahov Polit UMB (Faculty of Political Sciences and International Relations) NAT Fakulta prírodných vied Prír UMB (Faculty of Natural Sciences) HUM Filologická fakulta Filolo UMB (Faculty of Philology) SOC Pedagogická fakulta Pedag UMB (Faculty of Education) SOC Právnicka fakulta Práv UMB (Faculty of Law)Univerzita Pavla Jozefa Šafárika (Pavol Jozef Šafárik University) UPJŠKošice SOC Fakulta verejnej správy VerSpr UPJŠwww.upjs.sk (Faculty of Public Administration) MED Lekárska fakulta Lek UPJŠ (Faculty of Medicine) SOC Právnicka fakulta Práv UPJŠ (Faculty of Law) NAT Prírodovedecká fakulta Prír UPJŠ (Faculty of Natural Sciences) 15
  17. 17. University Group Faculty Abbreviation Univerzita sv. Cyrila a Metoda (University of St. Cyril and Methodius) UCM Trnava SOC Fakulta masmed.kom. MasMed UCM www.ucm.sk (Faculty of Mass Media Communication) NAT Fakulta prírodných vied Prír UCM (Faculty of Natural Sciences) HUM Filozofická fakulta Fil UCM (Faculty of Arts) Univerzita veterinárskeho lekárstva AGRO Košice UVL (University of Veterinary Medicine) www.uvm.sk Vysoká škola múzických umení (Academy of Music and Dramatic Arts) VŠMU Bratislava HUM Divadelná fakulta Divadelná VŠMU www.vsmu.sk (Faculty of Theatre Arts) HUM Filmová a televízna fakulta FilmTel VŠMU (Faculty of Film and Television) HUM Hudobná a tanečná fakulta HudTan VŠMU (Faculty of Music and Dance) Vysoká škola výtvarných umení HUM Bratislava VŠVU (Academy of Fine Arts and Design) www.vsvu.sk Žilinská univerzita (University of Žilina) ŽU Žilina TECH Elektrotechnická fakulta Elektr ŽU www.utc.sk (Faculty of Electrical Engineering) SOC Fakulta PEDAS Pedas ŽU (Faculty of Operation and Economics of Transport and Communications) NAT Fakulta prírodných vied Prír ŽU (Faculty of Natural Sciences) SOC Fakulta riadenia a informatiky Riadenia ŽU (Faculty of Management and Informatics) TECH Fakulta špeciálneho inžinierstva ŠpecInž ŽU (Faculty of Special Engineering) TECH Stavebná fakulta Stav ŽU (Faculty of Civil Engineering) TECH Strojnícka fakulta Stroj ŽU (Faculty of Mechanical Engineering)x1010 The stated list of public universities was composed according to Annex 1 of Act no. 131/2002 Coll. on highereducation as amended, and the names of all universities were given alphabetically. The list of faculties wascomposed according to a report of the Ministry of Education to the SR National Council in July 2005. Someinformation was taken from material from the Centre for Equivalence of Diplomas in Bratislava. The ARRA listincludes all public universities apart from the J. Selye University in Komarno, which has not completed theeducation cycle for even the first level of university education. The list of faculties does not include seminaries ofthe catholic faculty. In 2005 private universities operated in Slovakia in addition to the public universities (numbers inparentheses: full time students, part time students called external ones):- the Bratislava College of Law, BVŠP (167, 354),- St. Elizabeths University of Health and Social Work, VŠZaSP (0, 1181),- College of Economics and Management of Public Administration, VŠEM (41, 463),- College of Management in Trenčin, VŠM (620, 131). Private universities are not, however, currently included in ARRAs assessments. The main reason is thatnearly all private universities (apart from the College of Management in Trenčin) are relatively new and thereforein 2004 did not have any graduates. Data on them are also less accessible and incomplete. 16
  18. 18. 4. Quality criteria ARRA selected a number of criteria from those initially proposed anddiscussed with domestic and foreign experts, as shown in Table 2:Table 2: Criteria for assessing the quality of universities Criteria for assessing the quality of universities Category Num. Label Title Institution’s 1 ** RP1 Graduates’ quality (employers’ view) A reputation 2 RP2 Peer review 3 * VV1 Number of scientific publications in WoK database in 1995 – 2004 per number of creative workers (CW) 4 * VV2 Number of citations per CW according to the WoK database 5 * VV3 Number of scientific publications from VV1 with more than 5 citations per CW 6 * VV4 Number of PhD students in full time study (year 2004) in proportion to the number of professors and associate professors 7 * VV5 Average annual number of PhD graduates in 2002 - 2004 in proportion to the number of associate professors and professors Science and 8 * VV6 Number of PhD students in full time study in proportion to the total number of B research students studying full time in 2004 9 VV6a PhD studies’ success rate 10 * VV7 Combined grant funding from the VEGA and KEGA agencies per one creative worker in 2004 11 * VV8 Funding from AR (applied research), ISTC (international scientific and technical cooperation) and ASST (Agency for the Support of Science and Technology) grants per one creative worker 12 * VV9 Overall grant funding per one creative worker (2004) 13 VV10 Grant funding from the European Social Fund 14 SK1 Students with Internet access at the faculty 15 SK2 Students with Internet access at the dormitory 16 SK3 Number of students accommodated in a dormitory in proportion to overall number of dormitory applications Students’ 17 SK4 Number of students with scholarships granted by a university C comfort 18 SK5 Possibilities for sport and cultural life at the university 19 SK6 Number of subjects taught with literature accessible via Internet 20 SK7 Number of students attending the students survey on quality of education 21 SK8 How have the results of a students survey on quality of education manifested in quality and organisation of educational process? 22 * SV1 Number of students divided by the number of teachers 23 * SV2 Number of students divided by the number of professors and associate professors 24 * SV3 The ratio of teachers with a PhD to the total number of teachers 25 * SV4 The number of professors and associate professors divided by the number of all teachers 26 * SV5 Average age of professors holding a functional position (active professor) Study and D 27 ** SV6 Admissions proceedings: ratio of actual number of applications received to the education planned number 28 * SV7 Admissions proceedings: number of registered students in proportion to the number of offers 29 ** SV8 Number of students with foreign state citizenship divided by the total number of students studying full time 30 ** SV9 Number of university graduates unemployed for over 6 months 31 ** SV10 Students taking part in study abroad 32 * F1 Regular expenditure per one student E Financing 33 * F2 Success of the universitys business activities 34 * F3 The proportion of grant funding to the universitys budget for its main activities From the given list, data was available only for those criteria that are markedwith one or with two stars. The criteria RP1, SV6, SV8, SV9 and SV10 are mentioned 17
  19. 19. in this report but they are not included in the final score for the universities. The reason why several of the approved criteria were not used is that the report isthe result of a pilot project and not all the information was accessible at the time ofassessment. In the case of some indicators the collection of data was still notcomplete or collection would require more time and be expensive. These include:- the results of student surveys,- the results of surveys among teachers in individual faculties,- information on the financing of individual faculties. In some cases the data for all schools are extremely low and therefore – toprevent the assessment from being too dependent on fluctuation in small numbers –they were not included in the scoring, although they are mentioned in the report(they are marked with two stars) Some data were available only for universities as a whole and not for individualfaculties (in Table 2 they are highlighted in turquoise). These are included in the finalassessment of the universities but not, however, of their faculties. Numbers for professors and associate professors refer to the number of chairsoccupied in 2004. 18
  20. 20. 5. Basic characteristics of the indicators and their classification into groups The indicators that ARRA uses are classified in a number of materially compactgroups. For purposes of clarity, this section lists the basic characteristics of theindividual indicators and classifies them into groups. The names of the groupsprovide sufficient explanation of why a given indicator belongs in its group. A moredetailed analysis of the indicators is given in Part 8, in which specific examples will beused to show how they affect the assessment.11"Science and Research" Category"Publications and citations" GroupVV1Number of scientific articles12 in proportion to the number of creative workers (CW),i.e. teachers and researchers and artistic employees with higher education, inperiodicals registered in the Thomson database for the years 1995 – 2004.13VV2Number of citations in proportion to the number of CW. Only citations of papersincluded in VV1 will be counted. Citations of papers published in or before 1994 willtherefore not be included. The decisive factor is whether the given work appeared inthe database as at 31 December 2004.VV3The number of papers published in the years 1995 – 2004, for which more than 5citations had been registered by 31 December 2004, divided by the number of CW inthe faculty."PhD Studies" GroupVV4The ratio of the number of PhD students in full time study to the number ofprofessors and associate professors. As a rule, associate professors and professors11 Unless stated otherwise, information given relates to 2004.12 The terms publication and citation hereafter refer to publications from the stated database and citation inpapers included in it.13 http://www.thomson.com/scientific/scientific.jsp, Thanks to the SR Ministry of Education, all universities inSlovakia have access to this database as do the universitys teachers and research and artistic employees, theAccreditation Commission, the SR Ministry of Education and the Slovak Academy of Sciences 19
  21. 21. supervise PhD students (and a small number of holders of scientific ranks IIa and I, about whom there is no information in the public domain) and this is oneof the duties of their employment under the Higher Education Act no. 131/2002 Coll.VV5The annual average number of PhD graduates in the period 2002 -2004 divided bythe number of professors and associate professors in 2004. Since the number ofgraduates fluctuates from year to year, it has been necessary to expand the timescale for the collection of these data. As a baseline we took the minimum length offull time PhD study under applicable legislation, i.e. 3 years.VV6The proportion of the number of PhD students in full time study to the total numberof students studying full time in 2004."Grant Success" GroupVV7Overall grant funding from the VEGA and KEGA agencies per creative worker.VV8Funding from AR (applied research) ISTC (international scientific and technicalcooperation) and ASST (agency for the support of science and technology) grantsper creative worker.VV9Total grant funding for each creative worker."Study and Education" Category14"Students and Teachers" GroupSV1The number of students in full and part time study divided by the number ofteachers. Points are awarded so that the lower the number of students per teacherthe higher the number of points.14 Where students are referred to in this category of indicators, it refers to students in bachelors and mastersprograms studying both full and part time. The only exception is indicator SV10, which includes – whereinformation is available – full time PhD students. Where it is not explicitly stated (e.g. financing), students are notrecalculated according to the SR Ministry of Education formula. 20
  22. 22. SV2 The number of students studying full and part time divided by the number ofteachers. Points are awarded in the same way as for indicator SV1.SV3The proportion of teachers with PhD to the total number of teachers. It is assumedthat every professor and associate professor in a functional position has a PhD.SV4The ratio of teachers with a PhD to the total number of teachers.SV5The average age of active professors. Points are awarded using the value of theformula 100/(average age of professors). The term active professor (associateprofessor) refers to any person who occupies the functional position of a professor orassociate professor15."Applications for study" GroupSV6Admissions proceedings: the number of applications submitted to the plannednumber to be admitted, i.e. the number of applications to study divided by thenumber of study places offered by the faculty.SV7Admissions proceedings: number of registered students to the number of applicantsaccepted, i.e. the number of applicants registered divided by the number of placesoffered for study.SV8Number of students with foreign state citizenship divided by the total number ofstudents studying full time."University Level Criteria" GroupSV9Number of graduates unemployed for longer than 6 months per 100 universitygraduates in 2004.15 Act no. 131/2002 Coll. on higher education and changes and additions to certain acts. 21
  23. 23. SV10 Number of students taking part in study abroad through the Erasmus andSocrates programs and using SAIA grants, per 100 students of the university."Financing" CategoryF1Running costs expenditure from state funding for one student of the university.F2The proportion of the result of the universitys business activities to the overall costsof its main activities. This indicator reflects the proportion of its research andeducation expenditure that the institution is able to cover from its own businessactivities.F3The proportion of funds from public grants (VEGA, KEGA, AP MŠ, APVT a MVTS) tothe overall costs of its main activities. This indicator shows what part of itsexpenditure the institution is able to gain through the active efforts of its employeesin free competition for public funds made available to support research anddevelopment."Reputation" CategoryRP1Quality of graduates (from the perspective of employers). The weekly magazineTREND cooperated with ARRA to carry out a survey of businesses belonging to theAssociation for the Development of Human Resources (a questionnaire sent toaround 110 firms operation in Slovakia). The conclusions from the information gainedare not included in the final score for the faculties and universities and will bepublished separately. 22
  24. 24. 6. Method for the assessment of faculties and universities The indicators listed in Chapter 5 have been selected to ensure that a higherscore corresponds to higher performance. For example, one of the indicators is the number of students to one professoror associate professor. The higher this number is, the less likely it is that professorsand associate professors can give individual attention to students. From theperspective of individual approach, which ARRA considers to be most beneficial forstudents, it is better if this indicator has a lower value. If, however, we consider itsinverted value (the number of associate professors and professors for a hundredstudents), a higher level of performance in the faculty corresponds to a higher valueof this indicator. The number of students includes both full and part time students.The degree awarded to a part time student is the same as the degree awarded to afull time student and the quality of education provided to both should be the same,so teachers should pay the same amount of attention to full and part time students. In most cases the number in the table is the same as the indicator. There areonly three exceptions:• SV1 shows the number of teachers per hundred full and part time students,• SV2 shows the number of professors and associate professors per hundred full and part time students.• SV5 shows the number 100/(average age of active professors). The faculty that scored the highest in a particular indicator is assigned 100points. The other faculties receive points by linear interpolation, with zero pointsbeing assigned to a zero value. Each faculty of each university is assessed in the group to which it wasassigned in Chapter 4. The overall ranking of faculties in each group depends on theaverage points scores in the groups of indicators. The overall ranking of universities is based on the average points scores oftheir faculties. Details are given in later chapters. 23
  25. 25. Method for the assessment of faculties and universities An example for illustration purposes may be the indicator VV1 (number ofpublications per creative worker) and the faculty group AGRO (which covers arelatively small number of faculties).Table 3: Values of the VV1 indicator for faculties in the AGRO group Faculty Creative workers Published Ratio Points papersUniversity of Veterinary Medicine 183 1077 5,89 100Faculty of Forestry TUZV 83,8 187 2,23 38Faculty of Biotechnology and Food Sciences SPU 54,7 90 1,65 28Faculty of Agrobiology and Food Resources SPU 159,2 151 0,95 16Faculty of Wood Sciences and Technology TUZV 123,9 62 0,50 9Faculty of Horticult. and Landscape Engineering SPU 69,9 7 0,10 2 The first column contains the name of the faculty (for other faculties see thecollected results), the second column contains the number of creative workers, thethird column gives number of publications in the WoK database for 1995 – 2004. Thenext column has the number of publications for each creative worker followed by thenumber of points corresponding to this number in this group (calculated as the valuein the fourth column divided by the highest value in the fourth column multiplied bya hundred and rounded to a whole number). The next step is to combine the valuesfor indicators VV1, VV2 and VV3 into summary values for the "Publications andCitations" indicator group (see Table 4).Table 4: Assignment of scores for the "Publications and Citations" indicator group for faculties in the AGROgroup Faculty VV1 VV2 VV3 PointsUniversity of Veterinary Medicine 100 100 100 100Faculty of Forestry TUZV 26 38 36 33Faculty of Biotechnology and Food Sciences SPU 31 28 19 26Faculty of Agrobiology and Food Resources SPU 32 16 11 20Faculty of Wood Sciences and Technology TUZV 19 9 4 11Faculty of Horticult. and Landscape Engineering SPU 5 2 4 3The first three columns show the number of points that each of the faculties gainedin the "Publications and Citations" indicator group. The last column shows thenumber of points calculated as the average of all the indicators belonging to thegroup. The values in this column are transferred to the final assessment of thefaculties in the group and the overall assessment of the universities. The method used to prepare the output value for universities is shown inTable 5. The Technical University in Zvolen is given as an example. 24
  26. 26. Table 5: Assessment of a university by individual areas and overall Faculty Teachers and Applications Publications PhD Grants Average Universitys Students for study and citations studies score Faculty of Forestry 82 87 33 62 73 67 Faculty of Wood Sciences and Technology 77 90 11 74 43 59 Faculty of Environmen. and Manufactur. Technology 70 71 1 51 84 55 Faculty of Ecology and Environmental Sciences 65 80 9 58 30 49 Technical University in Zvolen 57,6 The evaluations that each faculty received in the indicator groups are given inindividual columns, with the names of the groups being given in the first row. Forexample, the number 33 in the row for the Faculty of Forestry and the column"Publications and Citations" is taken from Table 4. The results for all indicator groupsare then summed and divided by the number of indicators (total/5). The resultingscore for the whole university is the average score of its faculties. The overallevaluation of the universities also includes a sixth group of indicators (Financing, F1– F3), in which data are available only for the university as a whole. In Table 5, the Faculties of Forestry and Wood Sciences and Technology havepoints from the AGRO Group (Table 4), the Faculty of Ecology and EnvironmentalSciences from the NAT group and the Faculty of Environmental and ManufacturingTechnology from the TECH group. A few more notes need to be given on this scheme for the assessment of facultiesand universities.- The most significant information is given in Table 3. This informs us the number of publications from the faculty for the 1995 – 2004 period. This number can be compared with foreign institutions or institutes of SAV (the Slovak Academy of Sciences) working in the same area. ARRA will carry out such a comparison in the near future.- The benchmark for faculties success in a given indicator is the most successful Slovak faculty in the relevant group, not some comparison point from abroad, so it is necessary to compare the relevant data in the tables and not the point score.- It is necessary to bear in mind that the gradual aggregation of information (the creation of averages of averages as can be most clearly seen in the data for the university in Table 5) the information value of the ranking changes, becoming relative.- In the ideal scenario, the analysis would go even deeper and instead of faculties as a whole we would assess, and compare with each other and with foreign 25
  27. 27. groups, relatively homogenous groups within the faculty corresponding approximately to fields of studies. Such an analysis cannot currently beperformed using data in the public domain. It is however ARRAs ambition for thefuture to assess also individual study programs in similar/related areas. 26
  28. 28. 7. Commentary on individual indicators16 Science and research"Publications and citations" Group17VV1 Number of scientific publications and number of creative workers The scientific performance of individual faculties can be assessed in variousways Around the world, the most widely recognized approach is to use the numberof publications, citations, major awards, ability to obtain research funding in opencompetition. The VV1 criterion is based on the number of publications calculated forone creative worker (CW) in the time period 1995 to 2004, i.e. 10 years. The termcreative worker refers to all teachers and research and artistic employees of thefaculty with higher education. A publication is any work that is recorded in the WoSdatabase18 for the ten-year period 1995 – 2004. A work is included in the ARRA analysis if at least one of its authors papers ina faculty of a public university in SR19. The current ARRA analysis does not take intoconsideration the number of authors of the given work (publication). A survey of thedatabase shows that the average number of papers published by authors from publicuniversities in SR increased gradually in the 1995 – 2004 period – in 1995 there were999 papers and in 2004 the number had increased to 1273. In 2004 public universities in SR had a total of 9 810.5 teachers and 1 269research and artistic workers in with higher education, representing a total of11 079.5 creative workers. This means that for each worker 0.12 papers werepublished in 2004. The results in the tables and graphs published in this ARRA reportshow that the average number of papers per creative worker depend to a largeextent on the groups of faculties given in the Frascati Manual.16 To decode a abbreviation of a faculty in the graphs please have a look at the Table 1 in Chapter 3.17 The selection of data from a ten-year period disadvantages faculties that have existed for a shorter time. Itshould therefore be assumed that their ranking in this group of indicators might improve in future. A ten-yearperiod for the assessment of scientific performance is the generally used standard.18 The Thomson Web of Knowledge (WoK) includes the following databases: Web of Science (WoS), ScienceCitation Index Expanded (SCI-EXPANDED), Social Sciences Citation Index (SSCI), Arts & Humanities CitationIndex (A&HCI) (available for the years 1985 to 2005) and Essential Science Indicators (ESI). The WoK database(including WoS and ESI) is available to all employees of all universities, the Slovak Academy of Sciences, theAccreditation Commission and the SR Ministry of Education.19 Where the name of the author could not be linked to a faculty from the database, the link was made throughthe list of university (faculty) employees on the web page. 27
  29. 29. Table 6: Number of papers for the years 1995 – 2004 per one creative worker (situation in 2004) Group Papers published in 1995 – 2004 per one creative worker TECH 0,0 – 7,8 NAT 0,1 – 5,8 AGRO 0,2 – 3,2 MED 1,1 – 3,2 SOC 0,0 – 2,5 HUM 0,0 – 0,6 A number of faculty employees, especially in the groups SOC and HUM oftenpublish their work in periodicals that are not included in the WoK database. Thismeans that the approach has a negative impact on the arts faculties in particular.Nevertheless the ARRA analysis includes only papers from the WoK database. Thereasons for this are as follows: - the WoK database is in the public domain, which is in accordance with ARRAs data collection plan, - the inclusion of a magazine in the WoK database provides a certain guarantee of quality, since inclusion is based on an assessment of the level of the published articles, the history of the periodical and the like, - the WoK database is a source of bibliographic data that is recognized by the general academic community - the WoK database is available from any location in the world and the data it contains can be compared internationally, - in each group of faculties there are faculties that have publications in the WoK database, which shows that it is possible to publish in periodicals that are in this database; the fears of discrimination in some faculties and fields of study are reduced by the fact that they are compared only within their own group and with faculties that have the same or a similar orientation, - assessments of universities and social sciences in other countries20 are based on similar principles of records of publications in the WoK. Other important scientific papers that WoS does not record are patents andmonographs (or articles in monographs). We believe that these types of publicationare included in the survey to a certain extent, even if indirectly. Monographs or20 E.g. THES, October 21, 2005, p. 10. 28
  30. 30. chapters in them are produced mainly from the results of the scientific papers of their author or authors. These scientific results are as a rule also published inarticles in recognized periodicals. A quite natural question to ask is whether the numbers stated in individual graphsand tables for indicator VV1 are low, average or high. Comparison with otherassessments of a similar type in other countries led to the conclusion that they areon the low side. A serious answer to this question depends on a comparison withthe faculties of other – foreign – universities or comparison with the results of non-university scientific institutions (e.g. the institutes of the Slovak Academy ofSciences). As one of the first, ARRA will make a comparison with the individualinstitutes of SAV in the near future. (Red lines in the graphs indicate average values). Number of publications per one creative worker in 2004 VV1 Faculty group – MED 3,5 3,21 3,0 2,5 2,0 1,5 1,42 1,30 1,11 1,0 0,5 0,0 Farm UK JessenLek UK Lek UK Lek UPJŠ 29
  31. 31. 0,0 0,5 1,0 1,5 2,0 2,5 3,0 3,5 0 1 2 3 4 5 6 7 VeterLek. UVL 3,15 Prír UPJŠ 5,80 FMFI UK 4,40 Agro SPU 1,01 Faculty group – AGRO Prír UK 3,97 Faculty group – NAT BiotPotr. SPU 0,97 Ekolenv TUZ 0,96 Prír UMB 0,70 Les TUZV 0,82 Prír UKF 0,63 Drev TUZV Prír UCM 0,61 0,33 Infor a InfTech 0,18 STUBA Záhrad SPU 0,16 Prír ŽU 0,1530
  32. 32. Faculty group – TECH ŠpecTechn TUAD 0,00 ŠpecInž ŽU 0,00 Archit STUBA 0,02 Stav ŽU 0,08 Stroj TUKE 0,08 Stroj ŽU 0,11 Mech SPU 0,12 VýrTech TUKE 0,12 EnvirTech. TUZV 0,14 MatTechn STUBA 0,20 MechTron TUAD 0,21 Stav TUKE 0,23 Elektr ŽU 0,31 Stroj STUBA 0,38 Stav STUBA 0,58 Ban TUKE 0,61 Elektr TUKE 0,63 PriemTech TUAD 1,62 Elektr STUBA 2,03 Hutn TUKE 2,25 ChemTechn. STUBA 7,80 0 1 2 3 4 5 6 7 8 9 31
  33. 33. Faculty group – HUM VýtvarUm AU 0,00 VŠVU BL 0,00 Umení TUKE 0,00 Teol.TVU 0,00 Teol KU 0,00 Pravosl.PU 0,00 MuzUm AU 0,00 HudTan VŠMU 0,00 Greckokat.PU 0,00 Filolo UMB 0,00 FilmTel VŠMU 0,00 DramUm AU 0,00 Divadelná VŠMU 0,00 Fil UKF 0,03 Fil UCM 0,04 RímsKat UK 0,04 Fil KU 0,07 Hum UMB 0,10 HumPrír PU 0,25 Fil TVU 0,27 Evanj UK 0,38 Fil UK 0,48 Fil PU 0,62 0,0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 32
  34. 34. Faculty group – SOC Zdravotnícka PU 0,00 Športu PU 0,00 Sredoeur.Št.UKF 0,00 SocEkon UK 0,00 SocEkon TUAD 0,00 Soc UKF 0,00 Práv UMB 0,00 Práv TVU 0,00 Pedas ŽU 0,00 Pedag UMB 0,00 MasMed UCM 0,00 Manažment PU 0,00 Pedag UKF 0,02 Polit UMB 0,02 Pedag PU 0,04 PodnHosp. EU BA 0,05 Práv UPJŠ 0,06 Pedag KU 0,06 Práv UK 0,07 EkonomManSPU 0,07 Pedag UK 0,08 Ekonom UMB 0,09 Riadenia ŽU 0,09 Finan UMB 0,16 Pedag TVU 0,16 Manag UK 0,19 TV UK 0,21 VerSpr UPJŠ 0,22 EurŠt SPU 0,24 PodnMan EU BA 0,33 MedzVzťah EU BA 0,33 HospInfo EU BA 0,42 NárHosp. EU BA 0,45 Ekonom TUKE 0,50 Obchod EU BA 0,62 ZdravSoc TVU 2,46 0,0 0,5 1,0 1,5 2,0 2,5 3,0 33
  35. 35. VV2 Number of citations per CW according to the WoS database Citations represent another indicator of scientific performance that iswidely accepted around the world, in particular of the domestic and internationalresponse to the published results of the scientific work of individuals and workgroups. In discussions of this indicator there have been a number ofmisunderstandings, which is why we give more space to its explanation. In the WoSdatabase, every work is linked to information about the number of times and inwhich papers it has been cited, and the citations of each authors work can be found.Citations for all papers come only from the periodicals for which WoS collectsinformation The papers whose citations are analyzed here are precisely those papersthat were included in indicator VV1, i.e. papers published in the years 1995 – 2004and recorded in the WoS database. For example, if an author in a particular facultypublished a work in 1994 (i.e. before the monitored period), which has a number ofcitations, neither this work nor its citations are included in the current ARRA analysis.If the work was published in 2002, the work appears in the VV1 indicator and itscitations will be counted in the VV2 indicator. This leads to a certain reduction in thenumber of citations but this is not considered to be a fundamental problem. Thesame conditions were applied to each university in the study. This reduction has alsohelped to capture the current situation, which is more important for todaysprospective students than the rich scientific history of the university since itsfoundation. Citations for papers published in the years 1995 – 2004 are accumulating in2005 and will continue to accumulate in future years. The analysis of citations wascarried out as at 31 December 2004, and therefore the set of citations includes onlythose from before this date. A classic problem in analyzing citations is excluding self-citations, i.e. citationsof papers where at least one of the authors of the citing work is one of the authorsof the cited work. When calculating the number of citations of the work of a givenauthor, especially in career progression, self-citations are not taken intoconsideration. The ARRA analysis however includes self-citations. The available andsuitable software for the analysis of publications and citations does not contain filtersthat could exclude self-citations. The time requirements for "manual" filtering would 34
  36. 36. be extremely high. We believe, however, that the removal of self-citations would not have a significant effect on the results presented here. Just for information, employees of Slovak universities published a total of11 163 papers that are recorded in WoK in the period 1995 – 2004. 43 545 citationsof these papers were recorded. Of these 11 163 papers, however, 3 382 papers i.e.30%, did not receive a single citation. This fact shows that even the papers ownauthor did not cite it during this period. As mentioned above, the number of citations was determined for the period1995 – 2004 for papers that were published in the same period. In the HUM group,only 6 faculties recorded any result, with the other 17 (of which 8 are faculties of art)having no record. The situation is the same in the SOC group – 19 faculties out of atotal of 36 did not record any data. The VV2 indicator combined with the VV1 indicator makes possible certaincomparisons with the highest international level. If we divide result VV2 by resultVV1, we get the average number of citations for one published paper. Some of themore successful faculties of public universities in SR are shown in Table 7.Table 7: Average number of citations per paper for certain faculties of public universities, 1995 – 2004Faculty Average number of citations per one paperFaculty of Mathematics, Physics and Informatics, UK 8,30Faculty of Natural Sciences, UK 4,24Faculty of Chemical and Food Technology, STUBA 4,19Faculty of Medicine, UPJŠ 4,11Faculty of Pharmacy, UK 3,99Jessenius Faculty of Medicine, UK 3,98Faculty of Natural Sciences, UPJŠ 3,77Faculty of Medicine, UK 3,60University of Veterinary Medicine 1,87 It is also possible to give a comparison of this indicator for all Slovak publicuniversities. 35
  37. 37. Table 8: Average number of citations per paper for Slovak public universities, 1995 – 2004 University Average number of citations per paper (1995 – 2004) Comenius University, Bratislava 5,2 Slovak University of Technology, Bratislava 3,9 Pavol Jozef Šafárik University, Košice 3,9 University of Trnava, Trnava 3,7 Matej Bel University, Banská Bystrica 2,2 Alexander Dubček University in Trenčín, Trenčín 2,1 Constantine the Philosopher University, Nitra 2,1 University of Veterinary Medicine, Košice 1,9 Technical University of Košice, Košice 1,7 Technical University in Zvolen, Zvolen 1,5 University of St. Cyril and Methodius, Trnava 1,1 Slovak University of Agriculture, Nitra 1,0 University of Žilina, Žilina 0,9 University of Prešov, Prešov 0,3 University of Economics, Bratislava 0,3 Catholic University, Ružomberok 0,2 Academy of Arts, Banská Bystrica 0,0 Academy of Music and Dramatic Arts, Bratislava 0,0 Academy of Fine Arts and Design, Bratislava 0,0 21 A comparison with other countries can be made fairly simply because THESuses the same methodology. First those of the best universities:Table 9: Average number of citations per paper for some of the most successful universities in the world, 1995 –2004University Average number of citations per paper published in 1995 – 2004Harvard University 20,6Princeton University 17,7Stanford University 17,3University of California, Berkeley 16,0ETH Zurich 14,0Cambridge University 12,9Oxford University 12,2 Middle of the table in the stated magazine:Table 10: Average number of citations per paper for some other universities from around the world, 11095 –2004University Average number of citations per paper published in 1995 – 2004University of British Columbia 10,1Copenhagen University 8,7Osaka University 7,3La Sapienza University Roma 7,821 The Times Higher Education Supplement (THES), 7 October 2005, p. 9. 36

×