Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Jan Sadlak: Measuring Work and Performance of the University

810 views

Published on

Published in: Education
  • Be the first to comment

  • Be the first to like this

Jan Sadlak: Measuring Work and Performance of the University

  1. 1. 11 Measuring Work and Performance of the University: ReasonsMeasuring Work and Performance of the University: Reasons and Practices [in the context of sustainability and otherand Practices [in the context of sustainability and other challenges]challenges] Dr Jan SadlakDr Jan Sadlak,, President, IREG Observatory on Academic Ranking andPresident, IREG Observatory on Academic Ranking and ExcellenceExcellence Panel Session:Panel Session: Metrics: Measuring Sustainability Outcomes for Global ImpactMetrics: Measuring Sustainability Outcomes for Global Impact 2014 ISCN Conference,2014 ISCN Conference, Harvard University/MIT, Cambridge, MA, 3 June, 2014Harvard University/MIT, Cambridge, MA, 3 June, 2014
  2. 2. 22 Reasons for use of "metrics" in higher educationReasons for use of "metrics" in higher education  Higher education has moved from a guild-like status onHigher education has moved from a guild-like status on the periphery of society to more central place as thethe periphery of society to more central place as the wealth and good functioning of nations now depends onwealth and good functioning of nations now depends on the performance of higher education as never before.the performance of higher education as never before.  This change is reflected by such trends as;This change is reflected by such trends as; massificationmassification,, diversificationdiversification [institutional and study[institutional and study programs], pressure onprograms], pressure on public fundingpublic funding,, competitioncompetition atat national and international levels.national and international levels.  Higher education is under increasing pressure toHigher education is under increasing pressure to demonstrate its inner working and performance.demonstrate its inner working and performance. In otherIn other words, it is a call for “words, it is a call for “evidenced-based accountabilityevidenced-based accountability”” while striving forwhile striving for ““excellenceexcellence”” has to be demonstrated inhas to be demonstrated in a manner going beyond self-declaration.a manner going beyond self-declaration.
  3. 3. 33 Higher education andHigher education and ““sustainable developmentsustainable development””  Sustainable development is recognized as a definingSustainable development is recognized as a defining concept for responding to Earth’s problems withconcept for responding to Earth’s problems with imperative of economic and social development as wellimperative of economic and social development as well as being a subject to good governance.as being a subject to good governance.  Coping with those challenges requires an unprecedentedCoping with those challenges requires an unprecedented integration of insights across various academicintegration of insights across various academic disciplinesdisciplines. Only HE institutions bring together this. Only HE institutions bring together this range of knowledge.range of knowledge.  HE institutions see their own functioning from theHE institutions see their own functioning from the perspective of “perspective of “sustainabilitysustainability”” especially in the contextespecially in the context of its relationship with society [of its relationship with society [““Third MissionThird Mission””].].
  4. 4. 44 Forms, strength and weakness of principal metrics ofForms, strength and weakness of principal metrics of quality and excellencequality and excellence The number and type of instruments for analysis of “quality” and “excellence”The number and type of instruments for analysis of “quality” and “excellence” is quite restrained. It includes:is quite restrained. It includes: • Accreditation;Accreditation; • Peer-review;Peer-review; • Benchmarking;Benchmarking; • Awards;Awards; • Ranking [league tables].Ranking [league tables]. All the above instruments rely onAll the above instruments rely on quantitative measurementsquantitative measurements. This is. This is specially the case of “rankings” which collects information from:specially the case of “rankings” which collects information from: • Public domain data and informationPublic domain data and information;; • Data and information collected by institutions of higher education;Data and information collected by institutions of higher education; • [International data[International data bases -bases - ISI Web of Knowledge, Scopus,ISI Web of Knowledge, Scopus, andand Google’s ScholarGoogle’s Scholar; and; and • Surveys.Surveys.
  5. 5. 55 University rankingsUniversity rankings  rankings give evidence of performance related torankings give evidence of performance related to researchresearch performanceperformance andand approximation of teaching qualityapproximation of teaching quality;;  provide information on the standing of higher educationprovide information on the standing of higher education systems of higher education, institutions, academicsystems of higher education, institutions, academic discipline and study programmes;discipline and study programmes;  foster clarification of the mission among higher educationfoster clarification of the mission among higher education institutions and influence pattern and forms of internationalinstitutions and influence pattern and forms of international collaboration;collaboration;  provide benchmarking against highly-performing systems,provide benchmarking against highly-performing systems, institutions and study programmes;institutions and study programmes;  provide additional rationale for allocation of funds;provide additional rationale for allocation of funds;  represent a key differentiator between legitimaterepresent a key differentiator between legitimate institutions and programmes and degree mills;institutions and programmes and degree mills;  reveal certain taboos about inner workings of academicsreveal certain taboos about inner workings of academics and institutions.and institutions.
  6. 6. 66 University rankings and sustainability [Third Mission]University rankings and sustainability [Third Mission] Majority of rankings are covering research and teaching activities. However,Majority of rankings are covering research and teaching activities. However, measuring work and performance of HE institutions in relation tomeasuring work and performance of HE institutions in relation to “sustainability” is gaining growing attention. In this regard a pioneering“sustainability” is gaining growing attention. In this regard a pioneering role has been carried out by such international initiatives as:role has been carried out by such international initiatives as:  UI Green Metric World University RankingUI Green Metric World University Ranking, which since 2010 is, which since 2010 is produced by the University of Indonesia [in collaboration with the Higherproduced by the University of Indonesia [in collaboration with the Higher Education Evaluation & Accreditation Council of Taiwan and EducationalEducation Evaluation & Accreditation Council of Taiwan and Educational Policy Canada];Policy Canada];   E3M: European Indicators and Ranking Methodology for UniversityE3M: European Indicators and Ranking Methodology for University Third MissionThird Mission, has been a three years research project (2008-2011), has been a three years research project (2008-2011) which was carried out by consortium of 8 universities and higher educationwhich was carried out by consortium of 8 universities and higher education institutions [coordinated by Technical University of Valencia, Spain] withinstitutions [coordinated by Technical University of Valencia, Spain] with financial support of the Europe Commission;financial support of the Europe Commission;  U-MultirankU-Multirank is an independent ranking [implemented by consortium] withis an independent ranking [implemented by consortium] with financial support in its initial years from the European Union. Its firstfinancial support in its initial years from the European Union. Its first edition was presented on May 13, 2014.edition was presented on May 13, 2014.
  7. 7. 77 Use of rankings for enhancement of performance,Use of rankings for enhancement of performance, excellence and innovation [1]excellence and innovation [1]  even the critics admit thateven the critics admit that rankings provide thought-provoking datarankings provide thought-provoking data for the whole range of activities including those directly and indirectlyfor the whole range of activities including those directly and indirectly relevant to sustainability;relevant to sustainability;  aaassessment can produce tensionsassessment can produce tensions butbut metrics provide a transparentmetrics provide a transparent and reliable lens that allows us to filter out emotionalism and ideology. Notand reliable lens that allows us to filter out emotionalism and ideology. Not to mention verification of validity of research findings;to mention verification of validity of research findings;  complexity of “sustainability” agenda should not discouragecomplexity of “sustainability” agenda should not discourage searching ofsearching of its metricsits metrics.. ““No matter how difficult your measurement problem seems, the first stepNo matter how difficult your measurement problem seems, the first step in measurement is to change and assumption you may have been makingin measurement is to change and assumption you may have been making for a very long time. Instead of assuming that it is a fundamentallyfor a very long time. Instead of assuming that it is a fundamentally immeasurable intangible, assume that it is measurable. Then the onlyimmeasurable intangible, assume that it is measurable. Then the only question becomes whether you are clever enough to figure out how”.question becomes whether you are clever enough to figure out how”. [[SourceSource: Douglas Hubbard. “Everything is Measurable” downloaded from: Douglas Hubbard. “Everything is Measurable” downloaded from http://www.cio.com/article/print/112101 on 16 April 2010http://www.cio.com/article/print/112101 on 16 April 2010]]
  8. 8. 88 Use of rankings for enhancement of performance,Use of rankings for enhancement of performance, excellence and innovation [2]excellence and innovation [2]  rankings are convenient tool forrankings are convenient tool for marketingmarketing,, policypolicy debatesdebates, and, and setting up the benchmarks forsetting up the benchmarks for excellenceexcellence. Metrics provided by rankings can facilitate. Metrics provided by rankings can facilitate taking adequate account of the complex relation betweentaking adequate account of the complex relation between potential and display of pursuing excellence in higherpotential and display of pursuing excellence in higher education.education.  rankings are data driven therefore HE institutions arerankings are data driven therefore HE institutions are confronted with such issues as data interpretation whichconfronted with such issues as data interpretation which can be constructive or paralyzing when outcomes ofcan be constructive or paralyzing when outcomes of rankings are being interpreted with excessive zeal.rankings are being interpreted with excessive zeal.  we are moving from the “rankings are here to stay andwe are moving from the “rankings are here to stay and there is nothing you can do about it” tothere is nothing you can do about it” to a more nuanceda more nuanced and sophisticated dialogueand sophisticated dialogue about assessment,about assessment, evaluation and improvement of learning and instruction.evaluation and improvement of learning and instruction.
  9. 9. 99 Academic excellence and sustainability agendaAcademic excellence and sustainability agenda Academic excellence is an intersection of a number of factors types ofAcademic excellence is an intersection of a number of factors types of institution, mission, history, location, funding and institutionalinstitution, mission, history, location, funding and institutional leadershipleadership. It is therefore difficult to formulate “one fits all” set of metrics. It is therefore difficult to formulate “one fits all” set of metrics for enhancement of performance, collaboration and innovation, whichfor enhancement of performance, collaboration and innovation, which represent a framework for sustainability agenda.represent a framework for sustainability agenda. Principles of ExcellencePrinciples of Excellence [formulated for “teaching and learning”] but its[formulated for “teaching and learning”] but its basic features can be expanded to other works of the university, includingbasic features can be expanded to other works of the university, including those under agenda of “sustainability”:those under agenda of “sustainability”:  Aim High – and Make Excellence Inclusive;Aim High – and Make Excellence Inclusive;  Give Students a Compass;Give Students a Compass;  Teach the Arts of Inquiry and Innovation;Teach the Arts of Inquiry and Innovation;  Engage the Big Questions;Engage the Big Questions;  Connect Knowledge with Choices and Action;Connect Knowledge with Choices and Action;  Foster Civic, Intercultural, and Ethical Learning;Foster Civic, Intercultural, and Ethical Learning;  Assess Students’ Ability to Apply Learning to Complex Problems.Assess Students’ Ability to Apply Learning to Complex Problems. [[SourceSource: Association of American Colleges and Universities (AACU), 2007].: Association of American Colleges and Universities (AACU), 2007].
  10. 10. 1010 Concluding remarksConcluding remarks It needs to be repeatedly said that ranking can only be aIt needs to be repeatedly said that ranking can only be a proxy reflection of the complex work of higher educationproxy reflection of the complex work of higher education enterprise. Therefore:enterprise. Therefore:  Do not subject the mission of the university to “tyranny ofDo not subject the mission of the university to “tyranny of seeking status” it is therefore wrong policy andseeking status” it is therefore wrong policy and management to take any personal or institutional solelymanagement to take any personal or institutional solely based on rankings;based on rankings;  HE institutions need to come to terms with new landscapeHE institutions need to come to terms with new landscape of HE in which competition is more evident than before butof HE in which competition is more evident than before but if they are to compete they should be inspired but do notif they are to compete they should be inspired but do not be mesmerized by ranking;be mesmerized by ranking;  Academic rankings are an entrenched phenomenon aroundAcademic rankings are an entrenched phenomenon around the world and it is recognized as one of widely-usedthe world and it is recognized as one of widely-used methods of “quality assessment”.methods of “quality assessment”.

×