Cecilia Tsui 2008 Higher Education Development: Quality, Relevance and Impact
Upcoming SlideShare
Loading in...5

Cecilia Tsui 2008 Higher Education Development: Quality, Relevance and Impact






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Cecilia Tsui 2008 Higher Education Development: Quality, Relevance and Impact Cecilia Tsui 2008 Higher Education Development: Quality, Relevance and Impact Document Transcript

    • Report - OECD, IMHE General Conference 2008 Outcomes of Higher Education – Quality, Relevance and Impact 8-10 September 2008 - Paris, France Cecilia C B S Tsui, PhD candidate, University College, The University of London, United Kingdom Please note I submitted this report to the OECD, IMHE in October 2008. I am translating this report into a paper. Please do not quote. 1 Introduction The OECD, IMHE Conference 2008, entitled “Outcomes of Higher Education: Quality, Relevance and Impact”, was well attended with participants from over 65 countries. In addition to the plenary speeches and panels, there were sessions elaborating on the following: Parallel Group Sessions Sub-themes Parallel Group Sessions on OECD Projects  Institutional measures to assess and  Thematic review of tertiary education policy improve quality and policy determinants of investment in  Reputation and ranking – the impact of tertiary education institutional strategy and behaviour of  University futures and open educational international ranking tables resources  Assessing learning and employment outcomes  Facilities for higher education /  Balancing the needs and expectations of higher education and regions society with the autonomy of institutions  Supporting quality teaching and the mobility  Value for money and efficiency in higher of researchers education  Quality and relevance: some policy responses  Institutional diversity and transparency The conference was a great success: as reflected in the number of participants from various countries, and the impressive quality of papers / presentations. The conference theme was timely, with documentation clearly setting out the objectives. The plenary speeches projected proactive approaches stimulating participants to reflect critically on key issues, as well as the emerging combination of outcome measures, including global rankings, and institutional classifications, which are constituent technologies of a global knowledge-status system. The speakers contributed a wealth of information on diverse initiatives. In sum, the conference achieved the aims to facilitate information and experiences sharing, to identify best practices, to develop recommendations, as well as to facilitate international co-operation and networking within the higher education community. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 1
    • Furthermore, the conference provided an intellectually and a culturally rich platform for me to reflect on related issues in higher education. I also treasure the chance of conversing with experts, and policy-makers. Some of them have driven the development of quality assurance in different places, such as, Australia, Europe, the United States and Hong Kong. I particularly appreciate listening to presentations demonstrating analytical reasoning with socio-economic and theoretical backgrounds by leading scholars. I find the conference presentations highly relevant to my PhD study, titled “Quality Assurance in Higher Education: A Hong Kong Perspective”, focusing on conceptual issues of quality, accountability and managing change. This paper reports my observation of the quality assurance movement in relation to the conference, in capacity as a research student in higher education. While I have gained my experience primarily from Hong Kong and England, I will also draw on development of other countries, where appropriate. This paper commences with the changing socio-economic contexts, leading to a sketch of the evolution of the quality assurance movement. It is followed by a highlight of key messages I perceived at the conference. This paper concludes with a discussion of main issues, as well as achievements and challenges of the quality assurance movement in higher education. 2 Changing Contexts… Changing Needs, Expectations and Roles of Universities In giving his opening address, Professor Berdahl alerted that the environment in which all of our universities operate is remarkably different from what it was two or three decades ago. There have been significant global economic and social changes. 2.1 Globalization - Global Knowledge Economy and Higher Education In today’s global knowledge economies, HEIs are mediums for a wide range of cross-border relationships and continuous global flows of people, information, knowledge, technologies, products and financial capital (Marginson and Wende van der 2007a). As objects and, at the same time, agents (Scott 1998) of globalization, universities will become both less and more important: (a) less important – for universities will no longer be on their own or out in front in future society; as there is a proliferation of “knowledge institutions” (which may have little in common with traditional universities even if they adopt “university” brands); and (b) more important – for the society will be increasingly structured in terms of knowledgeability, and the production of ‘technical’ will become even more important as an engine of economic development (Scott 2000). In face of socio-economic challenges, Professor Berdahl emphasized: Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 2
    • … The process of globalization has increased demand for greater accountability and comparability of educational outcomes. Accountability and transparency are essential ingredients of a globalized economy. Outcome measures or assurances of the comparability of degrees are a natural consequence of globalization. In today’s globalized world coupled with knowledge- and conceptual-economy, universities are no longer the only contributors to the production of knowledge (Bleiklie and Powell 2005, Scott 2004). I believe new approaches to accountability, and outcome measures will boost higher education to develop new capacity, to reinvent, re-enchant itself, to compromise its own integrity in order to allow a new configuration of “knowledge institutions” to develop (Scott 2000). At the same time, HEIs will go through waves of learning curves, and will be challenged by new forms of “knowledge institutions”, such as consultants and think-tanks (Scott 2004). 2.2 Transformation of Technology, K-economy and Open Source Ecology Professor Berdahl pointed out that the transformation of technology had facilitated the development of distance learning, as well as virtual universities and virtual classrooms. This is, in part, the promise of the for-profit institutions. He added that if learning outcomes of such institutions can be shown to be comparable to traditional universities, such efficiencies are thought to be able to relieve governments and parents bearing the high costs of education. In his speech, Professor Marginson alerted the importance of the ‘global knowledge economy’ or ‘k-economy’ transformation ushered in by the Internet. He explained different interpretations of ‘k-economy’ and conceptual issues relating to knowledge as a global public good. Professor Marginson delineated that the production and dissemination of knowledge goods, the creation of communicative networks and the emergence of markets are all convergent processes. In the knowledge economy there are two heterogenous sources of dynamism: (a) economic commerce and (b) free cultural creations. Freely circulating knowledge goods blend by osmosis into the communicative open source ecology, which is their medium of production, and multiply. The benefits of being in the network grow exponentially, because of the ever-expanding number of connections. The cost of each new unit addition to the network is constant. Total cost grows in linear pattern and the benefit/cost ratio continually increases. The rate of expansion of the network increases over time until all potential nodes are included. Hence, the open source ecology is dynamic - characterized by its scale, fertility, flexibility and disorder. I consider that the technological advancement and the Internet have brought changes to processes of teaching and learning, such as (a) change in the learning behaviour and competencies of students, (b) possible shift of the role of teaching staff in higher education towards an advisor, feedback provider and controller, (c) rapid spread and turnover of knowledge and so on (Teichler 2001). Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 3
    • In addition, the rapid development of information and computing technologies has facilitated the open source flows of information and ideas. As delineated in Professor Marginson’s presentation, the open source ecology has great implications on economic and educational systems. 2.3 Massification … Concerns for Standards… Demands for Outcome Measures The development of mass higher education after the Second World War exhibited a rapid growth in the number of students seeking a university-type education (Gibbons 1998, Trow 2000). Since then, the history of higher education in the United States and Europe has been a history of the expansion of access and its consequences (Trow 2000) (Note 1). As for Asia and the Pacific, there has been rapid growth in student enrolments and expansion in higher education provision since 1998 (UNESCO 2003). Nowadays, there are many developed countries near to achieving ‘universal’ higher education (Brennan 2007, Guri-Rosenblit et al 2007); following Trow’s (1973, 1974) celebrated distinction among elite, mass and universal (Note 1) forms of higher education. The massification of higher education has created substantial pressures and changes in higher education. There is tremendous demographic transformation of tertiary education. Professor Berdahl pointed out that the number of students enrolled in higher education worldwide has doubled since 1990, reaching an estimated 135 million today (Richard Freeman, Harvard and NBER cited in Berdahl 2008) [conference presentation]. In some countries, such as, Australia, Denmark, Finland, Iceland, Italy, the Netherlands, New Zealand, Norway, and Poland, the tertiary education enrollment rate is more than 40% (OECD, Education at a Glance cited in Berdahl 2008) [conference presentation]. With the increasing demand for tertiary education, Professor Berdahl concerned that the numbers merely seeking credential rather than an education have increased. As a result, there have been calls for more rigorous standards on HEIs to ensure that the learning of students meets minimum requirements. Professor Berdahl pointed out that with the process of ‘massification’ and the recognition of the economic utility of higher education; it is increasingly viewed as a commodity. The rapid growth of for-profit institutions, with their emphasis on preparation for employment, has spurred the demands for outcome measures, especially for those outcomes, like job placement, which can be easily measured. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 4
    • 2.4 Quality Needs Diversity Amidst the changing contexts and anxiety for falling standards and reputation in today’s higher education sector, Professor Brink presented alternative, but impressive, views on widening participation (increasing the ratio of participation of societal groups under-represented in higher education) and equality issues; based on Michael Young’s (1973) concept of meritocracy. On the basis of his experience in South Africa, Australia, and the United Kingdom, he discussed some common fears and negative opinions, including (a) ‘standards will drop’, (b) ‘our reputation will suffer’, (c) ‘it’s not our problem’, (d) ‘it’s unfair’, and ‘it’s a waste of time’. To argue that diversity is a necessary ingredient of quality, Professor Brink illustrated cases of successful widening participation programmes, such as ‘Flexible on Access, Firm on Success’, which turned weak starters into strong finishers with extra support and guidance. He alerted that quality needs diversity. Based on Professor Brink’s viewpoints, I suggest that we need to embrace the changing phenomena of massification and diversification as part of social development for knowledge-based society. As Bruno Carapinha, a student representative, responded that higher education must build upon two pillars of equal relevance: (a) the pursuit of quality of education and (b) equity of our societies. Indeed, by the advent of mass education beginning in North America in the late 1960s and in full swing in Europe by the beginning of the 1990s, the notion of quality has evolved into ‘fitness for purpose’, allowing HEIs to demonstrate their achievement of objectives according to purposes and missions (Lenn 2004). Quality is a matter of interpretation; as Professor Brink stated “… different interpretations are possible, because you put words to numbers, meanings to words, and social parameters to meanings”. In a sense, performance is relative to context, as the concept of ‘quality’ is relative. On the other hand, Teichler (2008) analyzes that expansion of student enrollment is desirable, and expansion is interwined with diversification of HEIs. The majority of actors and experts predict a growing diversification (Teichler 2003) - through the establishment of new providers and the creation of various consortia and partnerships between universities for research and / or teaching purposes (Guri-Rosenblit 2007). Teichler (2008) suggests that increasing diversity of HEIs is beneficial, in terms of quantity, quality, relevance and efficiency of higher education. I consider Teichler’s (2008) analysis relates to Trow’s (1974) views on the functions of higher education in its three forms as being: Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 5
    •  Elite: shaping the mind and character of the ruling class; preparation for elite roles  Mass: transmission of skills; preparation for a broader range of technical and economic elite roles  Universal: adaptation of the ‘whole population’ to rapid social and technological change I observe that, nowadays, to meet changing socio-economic challenges, these three forms – elite, mass and universal – of higher education co-exist in different stratification of societies. The massified and diversified higher education systems also call for multi-dimensional quality assurance approaches (see Appendix A) (Brennan 2007:39), as advocated by stakeholders. In sum, I support Teichler (2001) that it is the co-existence of interesting interpretations and confused claims of key actors that drive the higher education sector to find new ways of mastering the changing conditions. 2.5 Tertiary Education and Society – Relevance and Impact Education relates to every part of a person’s life, and to every aspect of our society. We are in a changing society, with lots of unannounced changes (Clark 1996). Hence, the education sector and the society need to have closer collaboration to map out education, and research for future development. Regarding social relevance and impact issues, at the conference, there were three institutional cases demonstrating how HEIs collaborate with communities for (a) curriculum development, (b) learning enhancement and (c) research with entrepreneurial spirits. These cases demonstrated that nowadays HEIs integrate with the society at large, but not just ivory towers. I summarize these three cases as follows: Knowledge as a public property: the societal relevance of scientific research At the beginning of his presentation, Professor Bouter raised four questions – (a) Why did you start? (b) What did you do? (c) What answer did you get? and (d) What does it mean anyway? His presentation then focused on the last question – What does it mean anyway? - so as to alert the academic community that scientific research should have social-cultural value and economic value. He cited two examples, in areas of health and life sciences, illustrating how scientific research makes societal relevance and impacts at the VU University of Amsterdam, the Netherlands. While there have been lots of efforts to identify indicators of societal relevance, we must note that the nature of research and time frame vary enormously. Sometimes the societal impact of a study is readily apparent, but it often takes many years to make it felt. Professor Bouter finally invited the academic community to take up the challenge of developing concrete performance indicators, in a creative and constructive manner. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 6
    • I consider Professor Bouter’s proposed performance indicators would help researchers to better formulate research questions / frameworks, as well as to articulate and apply research results to the society with close community connectivity. Nevertheless, how well Professor Bouter’s proposal would be received by the academia is another matter of interest. Applying knowledge for the public good: societal relevance of university social service programmes for sustainable community development Dr Landeros illustrated how social service is integrated into the substantive functions of the university. The integration can be an effective strategy to address societal relevance and attend to issues of inequality and sustainability, especially in developing countries. Universities in Latin America face the challenge of being a direct agent of change and a force of social integration. Through community engagement, universities can better fulfill their mission and justifiably compete on par with first world institutions in their relevance to society. So, teaching, research and service should work as complementary parts of a whole, which mutually enhance and fortify each other. In her presentation, Dr Landeros also shared the tradition of university-based community social service in Mexico, and illustrated the case of Universidad La Salle, Mexico City, Community Service in Residence Programme aiming to poverty reduction. This presentation demonstrated that social service could be a linchpin among research, teaching and extension to achieve a greater positive impact on society. Balancing the needs and expectations of society with the autonomy of higher education institutions Professor Purcell raised a few key questions about needs, expectations, roles, and autonomy of universities in our contemporary society. She then addressed the following questions:  What are the HE needs and expectations of society? The role of a university is to further the knowledge of the community and to encourage and develop scholarship and learning. Based on contexts of the United Kingdom, and the University of Plymouth, Professor Purcell delineated that universities should (a) contribute to growing the knowledge economy, (b) develop and maintain a broad, advanced knowledge base, (C) provide quality education, training along the ideals of lifelong learning, (d) create centers of excellence in research, consultancy and advice, (e) embed international, multi-cultural education in the curriculum, (f) promote equal opportunities and access to higher education. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 7
    •  What does a university bring to a region? Universities impact greatly on their community - not only contributing to the regional skills base, local employment and educational opportunities, but also in the provision of cultural and sporting facilities. Professor Purcell cited how the University of Plymouth and universities in the United Kingdom contribute to the economy in the region, and in the country respectively. In addition, she pointed out that universities bring an array of opportunities to the community, such as (a) education – lifelong learning, (b) employment and providers of skilled graduates, (c) cultural and sporting facilities, (d) business support – personal, professional and business development, (e) supporting the public sector and local services, (f) enterprise, innovation and knowledge transfer opportunities, (g) significant contribution to the regional economy.  What is meant by the autonomy of universities? Professor Purcell reiterated the important role of a university in our society, and cited the following quotation to illustrate the importance of autonomy. In order to compete effectively at home and abroad, universities must be able to control their policies and resources. Without such autonomy for universities we cannot play our full part in realizing the Government’s and the country’s objectives (Rick Trainor, Universities UK Annual Meeting 2007). Professor Purcell further stressed that a university should have autonomy in a number of areas, for example (a) staff and students – admissions, progress and discipline, (b) curriculum and teaching – methods, examinations, content, text books, (c) academic quality – academic standards, quality audits, accreditation working with QAA, (d) research and publication – postgraduate teaching, priorities, freedom to publish, (e) governance – councils, academic boards, student associations, (f) administration and finance-funding of institutions, operating grants, capital, (g) global competition and so on.  How can we balance society’s expectations and a university’s autonomy? Is there a conflict? Universities must ensure society and students are getting the best deal without compromising academic freedom and standards. Universities do not exist in isolation but as part of a wider community – often with global reach and impact. If HEIs are truly to meet the needs of stakeholders in business, the professions, the public sector and the community at large – we need to become more demand-led and better attuned to the priorities of our community. We can influence the development of society through our academic values, quality standards and stakeholder engagement. To conclude the presentation, Professor Purcell’s advice was: Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 8
    •  To ensure there is a balance between society’s expectations and individual institution autonomy:  Higher Education Institutions need to be transparent and accountable, particularly in the areas of academic quality and income, building sustainable partnerships both in the UK and overseas.  Organizations need to regularly assess risk and diversify their funding streams to ensure financial security, running the institution in a professional and business-like approach.  Universities need to work with stakeholders to gain market intelligence and have robust governance structures and procedures.  We can neither afford to be ‘an ivory tower’ nor do we want to be one! As illustrated in presentations of this conference, the higher education community has been trying to figure out how to assess learning outcomes in this new environment. In the past, quality, impact and relevance pointed to student access issues, to academic governance issues and to relevance for society. In recent years, they have changed their meanings and have become familiar in different contexts, that of levels of professional quality, international culture and global competition. Professor Noorda pointed out that quality, relevance, and impact are key criteria for measuring university performance. The three criteria are powerful drivers of strategic developments in higher education. However, with the changing environment, HEIs must embrace and promote transparent and robust diversity. For example, responding to staff / student mobility, existing systems (arrangements) need to make room for more joint programmes and degrees, as well as the development of inter-national textbooks, curricula and a multi-national teaching faculty. In addition, Professor Berdahl remarked that HEIs have been struggling to deal with the impact that demographic, consumer-oriented, technological, political and global imperatives have brought to institutions. I agree with Professor Berdahl that a discussion of the quality, relevance, and impact of higher education outcomes must begin with these changing realities. We must ensure that HEIs are relevant to and have an impact on society’s needs of the twenty-first century. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 9
    • 3 An Overview of Evolution of Quality Assurance Mechanisms … Meeting Changing Socio-economic Needs The story of quality assurance in higher education is both old and new. Responding to changing societal development and needs, quality assurance mechanisms have evolved, and have become more sophisticated emphasizing on evidence-based learning outcomes. At the conference, Dr Ewell gave a brief review of learning assessment in the United States. Professor Amaral and colleagues delineated the developments in Europe and the United States. Observing presentations of this conference, I see that there are changing conceptions of quality assurance resulting from the expansion of higher education and social changes. I, therefore, present an overview of the evolution of quality assurance since the 1980s, hoping it helps analyzing the contemporary context in this paper. 3.1 Quality is Always a Concern Quality has always been a concern in higher education. van Vught (1993) states that the debate on the most appropriate form of quality assurance dates back to academic medieval Europe. Neave (2004a) (at a personal interview in 2004) boldly claims: … quality has always been an issue… except the vocabulary, and the new procedures, which we have to be sensitive because the world is moving so fast … what we are making sure that the certificates issued by universities are up to a minimal standard… that function has always been there… it has formed in different modes… in different institutional and political rotations… The higher education community has had different mechanisms in place to uphold academic standards of courses and degrees (Harman 2000). He illustrates that for countries and places with strong connections with Britain (e.g. Hong Kong, Singapore); there are well-developed systems of external examiners. Some other countries (e.g. United States, the Philippines) have had long experience with accreditation mechanisms (Harman 2000). Following the Second World War, the tertiary education sector has been expanding, and responding to diverse social changes. 3.2 Since 1980s – Public Concerns for Explicit Quality Assurance Mechanisms Since the early 1980s, there have been explicit concerns for quality and change in higher education (Dill et al 1996, Dill 2007, Green 1994, Mayhew et al 1990, OECD 2008a, van Vught and Westerheijden 1994). Systematic and explicit mechanisms of quality assurance started from a majority of states in the United States (Dill 2007). Dr Ewell, at the conference, made a remark on the changing phenomenon: Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 10
    • The beginning of assessment as an explicit topic of policy discussion in the U.S. is usually marked at 1985, the year that the first national conference on this topic was held… Despite… voluntary efforts, state mandates constituted the most important stimulus for assessment in its early years… more than 90% of institutions reported that they were in some way engaged with “assessment” by 1990. With occasional exceptions, the federal government was not much concerned about the quality of student learning outcomes until about three years ago. This changed markedly in the summer of 2005 when Secretary of Education Margaret Spellings – stimulated by evidence of eroding U.S. competitiveness in tertiary degree production when compared to leading OECD countries – convened the Secretary’s Commission on the Future of Higher Education. Following the United States, new national quality assurance policies were introduced to the France in 1984, the United Kingdom in 1985 and the Netherlands in 1985 (van Vught and Westerheijden 1993, Vroeijenstijn 2007). According to Dill (2007), the developments [in the United States] were diffused to other countries in Europe, Asia and eventually around the globe (Dill 2007). At the conference, Professor Amaral and colleagues pointed out: In Europe the development of quality assurance activities has started much later than in the U.S. The emergence of the “Evaluative State” (Neave 1998:7) was observed in the late 1980s, with increasing public relevance given to quality… The development of quality assurance in Europe has been fast. Schwarz and Westerheijden (2004) report that in the early 1990s less than 50% of the European countries had initiated quality assessment activities at supra-institutional level, while in 2003 all countries except Greece entered into some form of supra-institutional assessment. The European quality assurance systems share important procedural elements – internal self-evaluation, visit by an external expert review panel, external evaluation and public reporting (Thune 2002). However, there are important differences in political discourses (Neave 1998, 2004b) that range from a mainly European and political discourse, with universities assumed as a public service (e.g. France and Sweden) to a mainly economic discourse, market-based and inspired in the U.S. (e.g. UK and the Netherlands) with the role of the state seen as excessive (Neave 2004b). There are also differences in the ownership of the system and in the consequences of quality assessment – with or without direct influence on funding. Regarding development in Asia and the Pacific, Harman (1996) delineates that in most cases the initiative has come from governments and government agencies. However, in some cases, such as Korea, New Zealand, Taiwan and the Philippines, the HEIs themselves have played an important role in moves to establish new quality assurance mechanisms. The following highlights the chronology of early development in a few countries in Asia and the Pacific Region: Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 11
    •  In Australia, quality assurance emerged on the political agenda in the early 1990s. The Committee for Quality Assurance in Higher Education (CQAHE) was established in 1993, and the Committee was discontinued after its 1995 review. By the end of the 1990s there was renewed interest in establishing formal quality assurance mechanisms. The Australian Universities Quality Agency (AUQA) was set up in 2000 (Meek 2005:27-30).  China introduced national quality evaluation mechanisms in 1985. By 1993, eight disciplines encompassing 55 degrees had been reviewed (Sensicle 1993 cited in Harman 1996).  Korea introduced a comprehensive system of university accreditation in 1991 (Harman 1996).  In Taiwan, since 1991 efforts have been under way to improve the system of specialized accreditation of programmes and institutions operated for some years by the Ministry of Education (Su 1993 cited in Harman 1996). As a gauge for the growth of quality assurance in higher education globally, the International Network of Quality Assurance Agencies in Higher Education was founded in 1991, in Hong Kong, with 20 member countries (Lenn 2004). The growth of [quality assurance] is mirrored in East Asia and the Pacific region – 12 of the agencies were founded since 1991, and 11 of those since 1994 (Lenn 2004). Despite differences in ideology, size and stage of development of tertiary education sectors (Amaral et al 2008 [conference presentation], Scott 2005), governments have decided that traditional academic controls are inadequate to contemporary challenges (see Appendix B for rationale for the rapid growth of quality assurance in higher education). In sum, since the early 1980s, there have been growing interests in establishing policy mechanisms to ensure quality and accountability in tertiary education (El-Khawas 1998). 3.3 Quality Assurance Mechanisms – Old Wine in New Bottles Based on principles of (a) accountability, (b) compliance with standards and (c) quality improvement, there have been various scopes of responsibility, approaches and methods of quality assurance at different levels to meet respective national and international needs. Comparing with traditional methods, I observe that the quality assurance movement has induced a few explicit and systematic procedures - aiming to strengthen core values, incorporate new forms of management, and project new roles of higher education. I summarize my observation below:  Strengthening emphasis on teaching / learning Governments create mechanisms to investigate teaching / learning conditions or encourage HEIs to set up internal mechanisms for assuring that established standards are met (Martin and Stella 2007). This has strengthened that learning and teaching are key missions of HEIs. However, Scott (2004:13) alerts: “… we are moving towards a ‘new economy’ in which segregated domains labeled research or scholarship (…learning and teaching) are becoming redundant...”. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 12
    • I suggest we should take heed of the fact that the boundary between ‘teaching’ and ‘research’ is getting fuzzier.  Demonstrating regular and systematic documentation and reporting to stakeholders There are more formal and deliberate mechanisms of review and reporting, as well as new ways of demonstrating to a wider group of stakeholders the standing and relevance of degrees. The more systematic monitoring of performance helps to provide evidence to substantiate claims about quality, and ensuring course relevance to community needs (Harman 2000). I believe this development is built on the Total Quality Management spirit of keeping records systematically, and reporting regularly with reflective learning, and client-focused culture.  Emphasis on Outcomes The outcome-based approach has been practised in the United States for about twenty-five years (Ewell n.d.). In recent years, there is a global trend of adopting this approach to ensure academic standards. It is becoming more concerned with outcomes than with inputs and throughputs (Martin and Stella 2007). I consider this is in parallel with the ideals of New Public Management stressing the need for hands-on management of organizational units, and performance need to be evaluated in terms of clear, measurable output indicators. It is also due to the rise of global knowledge economies, calling for new modes of knowledge production (Scott 2004) and mobility of knowledge workers – hence, evidenced-based learning outcomes.  From National to International Arenas - Diplomatic Relations In addition, the recent initiatives of quality assurance, such as, development of international quality assurance and worldwide quality, are forming part of the education acumen of the diplomatic relations between great nations (Neave 2004a). Moreover, with increasing globalization and internationalization of higher education, the outcome-based approach quality assurance will help facilitating mobility of staff and students. My view is that from public management perspective, these changes for explicit, regular measures are good. Nevertheless, there are cultural conflicts against the traditional academic freedom. But, with quality assurance development evolving in the last two decades, the professional academic values, and culture should have changed in one way or another. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 13
    • 3.4 Quality Assurance Movement – Journey of Change The quality assurance movement is a journey of change. As from the early 1980s, there have been four to five phases of quality assurance schemes (see Appendix C) focusing on different problems and with different roles of quality assurance (Westerheijden 2007:11). As a matter of fact, quality assurance mechanisms are designed with a built-in facility for learning and change; while maintaining the delicate balance between the functions of improvement and accountability, even if all other conditions (e.g. external aspects) remain equal. The changing nature of quality assurance mechanisms also avoids bureaucratization and window dressing so as to ensure improvement and accountability (Jeliazkova & Westerheijden 2001). On the other hand, Brennan (2007:39) summarizes five types of quality assurance methods (see Appendix A), reflecting a continuing balance of power within and outside higher education in the United Kingdom. In my view, the movement of quality assurance has facilitated the development of higher education, which was, at one time, in crisis (OECD 1987, World Bank 1994); following rapid expansion of higher education after the Second World War, and against the backdrop of global economic and social changes. Nowadays, higher education is a huge and diversified sector; operating in changing contexts with new forms of management and educational modes so as to meet stakeholders’ needs, which carry consumer-oriented, political and global imperatives. I observe that quality assurance is a policy instrument, as an umbrella, embracing a wide range of interests and issues, such as, policy, governance, teaching, learning, assessment, student support and performance management. The wide spectrum of issues is also due to the fact that changing one component in a system will inevitably affect the others. The quality assurance movement has been a change agent, under the ideals of New Public Management (Note 2). Despite tensions, I believe quality assurance will continue to play significant functions to ensure academic standards, accountability, cost-effectiveness, and to promote transparency, improvement, and change. To sustain efforts and to build capacity, quality culture in HEIs, as well as to provide better information to stakeholders, I consider it timely to discuss outcomes, relevance, and impact issues, in contemporary sense. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 14
    • 4 Key Messages This section delineates the key messages that I perceived at the conference. There are three messages; namely: (a) public demand for outcomes – learning and research, (b) rankings, league tables and classifications, and (c) the growing accountability agenda. 4.1 Public Demand for Outcomes – Learning and Research At his opening address, Professor Berdahl remarked that responding to a call from the OECD Education Ministers in June 2006, the discussion of ‘student learning outcomes’ has been the major topic in quality assurance. In April 2007, the experts concluded that such assessment was a ‘reachable’ goal and that ‘reliable information on learning outcomes would only rise in importance’ in the future. Professor Berdahl stated: Let us begin with a discussion of “learning outcomes” which has gained such traction over the last decade. The question of “learning outcomes” has its origins from outside the academy in the demands for greater efficiency and accountability that arose in the 1980s, largely from the governments funding higher education. We must note that the public demand for greater transparency in the learning outcomes for students has its parallel in the public’s demand for outcomes for its investment in research. Professor Berdahl also analyzed the university-industry partnership, and its consequences to socio-economic development, as well as research in various disciplines in HEIs. In the wake of the above external, deeply-held, and widespread concerns, there are various studies to figure out how to assess learning outcomes, or how much students are actually learning in HEIs, in this new environment. Among the various assessments, I would like to highlight the following three, as Professor Berdahl spelled out:  Collegiate Learning Assessment (CLA) in the United States CLA is test designed to assess how students’ critical thinking, analytical reasoning, problem solving and written communication competencies have developed during the course of their college years. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 15
    •  Roadmap for the OECD Assessment of Higher Education Learning Outcomes (AHELO) Feasibility Study (launched in July 2008) AHELO seeks to examine achievement in both generic skills and disciplinary skills, with a value-added and contextual component of both.  Qualifications Framework It is a framework of competencies, which students at each level of degree attainment must demonstrate in some manner. There are three levels; namely: Transnational level – the ‘Framework for Qualifications of the European Higher Education Area’ provides the broadest, most generic levels of attainment, with a progression from more concrete, occupation-oriented competencies at the lowest level, to more abstract, independent and complex competencies at higher levels. National level – the development of ‘National Qualification Frameworks’ (consistent with the European-wide qualifications framework) are specific to the institutional structure and needs of each of the participating nations. The purpose is not to create identical degrees in each country, or identical paths for attaining degrees, but to create analogous degrees that have common competencies that can be understood everywhere, creating a ‘zone of trust’. Institutional level – ‘Institutional Qualification Frameworks’ are specific to the curriculum of each institution and, beyond that, in what is known as the ‘Tuning Project’ qualifications frameworks specific to each discipline or field, with benchmarks for subject specific knowledge and generic skills and competencies to be attained at each level of study. In general, there are two views of assessment:  Assessment for institutional and disciplinary improvement  Assessment for public information, policy formation and resource allocation Furthermore, Professor Marginson alerted that comparative measures of student learning in higher education are an obvious gap in the apparatus of policy instruments and a frontier issue for the sector. In his words: Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 16
    • … If established on a credible and comprehensive basis, and communicated well, such measures could be expected to become important to prospective domestic and foreign students, employers and investors in higher education, not to mention governments focused on national capacity and the competitiveness of national k-economies… … AHELO is likely both to emphasize the claims of those institutions that position themselves as doyens of synergistic Humboltian leadership in research and teaching simultaneously; and also to encourage some bifurcation between the two missions, as alternative routes to institutional primacy. In the longer term such developments are potentially transformative of the sector and its relations with society and economy, in the national, meta-national regional and global dimensions of action. He then commented on the present assessment of learning outcomes in the OECD countries, and made a few remarks about assessment for consideration as follows:  Unit of measurement – institutional vs national basis  Breath of inclusion of nations – AHELO assessment of student learning outcomes would be viable if most but not all countries be included  Capacity of measurement – measurement should encompass the diversity of learning objectives, modes and contents and in different languages, as well as cultural variations in academic disciplines and the manner in which the learned professions are practised  Components of measurement – mix of generic and discipline-based skills and knowledge deemed appropriate for comparative purpose  Form of assessment – summative or value-added assessments  Employability – employability vs proxy measures, e.g. graduate preparedness Measuring outcomes of education is not an easy task, but essential for students. As reflected in some conference presentations, such as Campbell (2008), and Chalmers (2008), studies are underway to identify and implement teaching / learning quality indicators in terms of input, process, output and outcome. I consider that in today’s knowledge-based society with massified and diversified higher education sector, the above-mentioned assessments are proactive measures. HEIs need to provide students with clear indications of what their paths should look like, what levels of knowledge and skills will qualify them for degree awards, and what their degrees mean. These road signs will be highly beneficial for future growth and development of students. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 17
    • 4.2 Rankings, League Tables and Classifications Rankings – High on Agendas of Stakeholders “… In recent years, as many countries have introduced tuition fees, and as tuition prices have escalated in the United States and elsewhere, rankings have been the focus of increased scrutiny…”, remarked by Alisa Cunningham at the conference. In her presentation, Professor Hazelkorn claimed that rankings started out as an innocuous consumer product (Note 3) for undergraduate domestic students, and it has become both a manifestation and driver of the global battle for excellence. In some countries, such as, Argentina, Nigeria, and India, the ranking exercise is undertaken as part of the accreditation process (Salmi and Saroyan 2007). On the other hand, concerning about the academia, Professor Marginson remarked that the new system for assigning ordered values to knowledge (‘k-status’) includes: (a) league tables and university rankings in research, (b) publication metrics and citation metrics, (c) journal hierarchies, and (d) other comparative measures such as learning outcomes. Professor Hazelkorn delineated findings of an OECD comparative study conducted in these few years. Rankings and league tables are high on agendas of various stakeholders - politicians, HE leaders, staff, students and parents. In her words: … Politicians regularly refer to them as a measure of their nation’s economic strengths and aspirations, universities use them to help set or define targets mapping their performance against the various metrics, while academics use rankings to bolster their own professional reputation and status… According to Professor Hazelkorn, mounting evidence indicates that rankings are perceived as playing critical roles as follows:  Enabling and facilitating universities to maintain and build reputation  High-achieving students, and international / postgraduate students use rankings to ‘short list’ institutions  External stakeholders use rankings to influence decisions about funding, sponsorship and recruitment / employment Rankings-consciousness is rising rapidly because benefits are perceived to flow from high ranking. Conversely, ‘fear of falling’ and the negative publicity associated with it can be as great for highly ranked or ambitious universities as non-appearance can be for others. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 18
    • I agree with Professor Hazelkorn that rankings are encouraging and influencing a number of matters, including:  Modernization and rationalization of institutions  Professionalization of services  Marketization of higher education  Research mission and fields of investigation  Curriculum and disciplines  Faculty recruitment and new career / contractual arrangements  Student choice and employment opportunities As illustrated in conference presentations, rankings and similar benchmarking assessments do influence institutional behaviour and performance. At the same time, we should beware of shortcomings of rankings. Furthermore, Professor Marginson pronounced, rankings, metrics, classifications, learning outcomes are common terms for HEIs nowadays. I particularly appreciate Professor Marginson’s warning about the culture of comparison, and his call for clean comparisons - which are (a) inclusive on a global scale, (b) permit a broad range of missions and institutional cultures, (c) sustain self-determination, (d) support genuine creativity, (e) foster universal improvement, as well as (f) competition for measured performance open to merit and innovation. The most important thing is that global rankings as the benchmark only makes sense if the indicators are appropriate – otherwise, governments and institutions risk transforming their system as institutions to conform to metrics designed by others for unrelated purposes. Rankings – Pros and Cons Rankings and league tables have been around for a long time, but there were dramatic growth in the past few decades (Altbach 2006). In the era of massification, rankings are also inevitable. It is natural that those who finance higher education and the public want to know which institutions are the best. Governments and funding authorities have to decide how best to invest resources. Mass higher education requires differentiation since institutions serve diverse purposes and students attend HEIs for many reasons. Rankings can help to define differentiated academic systems if they can be devised to capture a variety of metrics, and thus help stakeholders to make well-informed decisions (Altbach 2006). Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 19
    • At the conference, Alisa Cunningham remarked that we have to beware of problems and weaknesses, such as, inconsistent results, data accuracy, and methodological issues. She then illustrated a few examples:  Rankings may skew student, institutional or governmental decision-making  Changes in methodology and weighting may result in inconsistent results  Single number, overall rankings do not reflect nuances within institutions  Available data may not be ideal to achieve purposes Harvey (2008) criticizes that the link of rankings to quality is naïve. The construction of indices by which institutions / departments are ranked is arbitrary, inconsistent and based on convenience measures. The operationalisation of the concept of quality is cursory at best, and nearly all focus on just one notion of quality - excellence. He further remarks that critiques of rankings come from an array of sources, and are based on methodological, pragmatic, moral and philosophical concerns; including: (a) selection of indicators, (b) weighting of indicators, (c) reliability, (d) statistical insignificance, (e) institutional, political and cultural environment, (f) focus of attention, and (g) competition (Harvey 2008). On the other hand, Altbach (2006) comments the problems of rankings as follows:  Many rankings resemble ‘popularity contests’ – asking groups in the academic community their opinions about peer institutions  Rankings count factors such as (a) external funding, (b) numbers of articles and books written by faculty members, (c) library resources, (d) proportion of faculty members with advanced degrees, and (e) quality of students (measured by scores on admissions or other tests). These numbers are assumed to be a proxy for quality, which they are to a significant extent. However, the number of articles published does not necessarily relate to the quality or impact of the articles.  Rankings generally do not include teaching quality. There are no widely accepted methods for measuring teaching quality, and assessing the impact of education on students is so far an unexplored area.  Rankings do not accommodate universities have different missions and goals. Rankings tend to emphasize the norms of the top research universities. The assumption is that ‘one size fits all’ and that the norms of the research universities are the gold standard. As illustrated, ranking is a tool, and an outcome from stakeholders / market perspective. There are pros and cons of rankings (see Appendix D for a summarized list). Its recent rise also reflects a widespread phenomenon in an increasingly competitive world of higher education. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 20
    • There is strong realization that some form of national and international comparators is useful, inevitable and here to stay. Higher education has to learn to live with those commercialized rankings. There are also huge policy implications, and a role for educating public opinion. Various international initiatives, such as the OECD, the EU, are responding to these challenges (Hazelkorn 2007). Amidst existing rankings, there are various research studies being conducted to prepare for the growing mobility of students / staff and to provide more channels for information. I consider that the CHE ranking study and the classification study, reported in the following sub-sections, are proactive approaches to counterbalance the global competitive scenarios. Searching for Best Practice - The CHE Ranking The landscape of rankings of HEIs has become diverse at international and national levels. Dr Westerheijden and colleagues remarked: Increasing public interest in university rankings is reflected also in the amount of academic literature that has been devoted to the issue in recent years. University rankings have been examined from methodological, technical and conceptual aspects. While the literature points to many serious problems in university rankings, there seems to be a general consensus that “rankings are here to stay” (Merisotis 2002) and that energy should go into improving rankings rather than fighting them (e.g. Marginson and Wende 2007b; Dill and Soo 2005; van Dyke 2005). Moreover, well-designed rankings can provide students with valuable information and encourage accountability in universities. Recent research consistently draws attention on potential dangers of poorly constructed rankings and suggests some principles that would make a ranking sound and beneficial. They also shared findings of their pilot project – the CHE Ranking at the conference. The aim is to pilot the CHE Ranking further, beyond the German-language area, to HEIs in the Netherlands and in the Flemish community of Belgium. Dr Westerheijden and colleagues pointed out various issues in the design of university rankings. I sum up the main points below:  Critical issues in the design of university rankings  Aggregate vs multi-dimensional rankings  University vs discipline rankings  Ranking vs clustering  Choice of indicators and measurement  The effect of rankings on HE systems  International rankings Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 21
    • As shown in various conference presentations, the international experience with university rankings provides many lessons about the implications of different ranking systems. I support views of Dr Westerheijden and colleagues that while rankings have been heavily criticized from conceptual and methodological standpoints, and for their potentially dysfunctional effects, the criticisms should be considered as a constructive input in the process of improving the quality and effectiveness of rankings. Institutional Diversity and Transparency Classifying European Institutions of Higher Education Professor van Vught’s presentation addressed the importance of diversity in European higher education, and proposed developing different ranking instruments. Bartelse and van Vught (2007) argue that a better understanding of the various types of HEIs, their missions and provisions, will support the European aim of (a) increasing student mobility, (b) inter-institutional and university-industry cooperation, (c) the recognition of degrees and hence (d) the international competitiveness. In addition, Professor van Vught emphasized classifications help to enhance the benefits of rankings and minimize their shortcomings: If we wish to maintain and even increase the diversity of higher education systems, we will have to develop different ranking instruments in which different forms of institutional performance can be compared. We should design multiple ranking instruments that enable us to make inter-institutional comparisons per category or type of institution. In order to create higher levels of diversity in higher education systems, we therefore need to develop typologies of higher education (or classifications) the diversity of institutional missions or profiles should be made transparent, offering the different stakeholders a better understanding of the specific ambitions and performances of the various types of higher education institutions. van Vught (2008:21) At the conference, Professor van Vught pointed out the need to create more transparency about institutional diversity. He explored the concept of diversity, and discussed the history of European higher education from the perspective of this concept. Then, he took stock of the outcomes regarding diversity in the Bologna process, and remarked that the institutional diversity of European higher education largely remains ‘hidden’. Professor van Vught also delineated the classification instrument as a tool to create transparency about institutional diversity. He stated that the classification instrument should be highly useful for institutional strategies, inter-institutional partnerships, benchmarking and networking. I observe that this classification instrument should provide a reliable and transparent source of information, and profiles of HEIs, from educational and professional perspectives, rather than the present market-dominated ones. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 22
    • Convergence and / or Diversity? The Argument of Transparency Professor van Damme stated that higher education systems are engaged in processes of convergence driven by an array of factors, including (a) globalization, (b) the integrated system of scientific research, (c) the transnational mobility of students, teaching staff, researchers and institutional leaders, (d) the technology-driven expansion of new teaching and learning modes, and (e) the international labour market of graduates. He then illustrated the importance of regional processes of integration, and cited the Bologna Process in Europe as an example. In addition, he stressed the importance of transparency, and shared a number of models of convergence, such as the Tuning Project, Qualification System, AHELO and Global Rankings. Professor van Damme then raised two questions: (a) Does convergence on systems level mean that institutions become more and more similar to one another? (b) Is system convergence a favourable condition for institutional diversification? to further illustrate that transparency becomes crucial. As a result, new instruments of enhancing system transparency are needed to complement national regulation, based on systematically collected and comparable data. He concluded that evidence-based transparency creates a more favourable environment for both the development of integrated higher education systems as for the safeguarding of institutional autonomy than ‘wild’ competition based on unproven institutional reputations. 4.3 The Growing Accountability Agenda There has been growing accountability agenda, as illustrated in a few conference presentations. In his opening address, Professor Berdahl started with a brief discussion of ‘learning outcomes’, and ‘rankings’. He then elaborated that in fulfilling the test of accountability, there are four fundamental principles:  The essential element in the learning process is the role of the faculty  Peer review is important in the assessment of learning outcomes and expectations  The crux of public accountability is to assure the degrees awarded attain commonly understood levels of attainment  Accountability is not just to satisfy public authorities and choice in a comparative market model, but also to inform students of the expectations that the pursuit of a particular degree will impose upon them. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 23
    • Dr Salmi pointed out two significant examples of increasing calls for accountability, and raised the question whether this is a favourable development for higher education: … In Europe, an important part of the ongoing Bologna process consists in designing a qualifications framework that will provide common performance criteria in the form of learning outcomes and competencies for each degree awarded. In the US, one of the key recommendations made by the Commission on the Future of Higher Education set up by the Secretary of Education Spellings in 2004 was to call for measures of student learning to assess the actual added value achieved by tertiary education institutions. He further alerted that the growing accountability agenda is also reflected in the multiplicity of stakeholders, the multiplicity of themes under scrutiny, and the multiplicity of instruments and channels of accountability. I agree with Dr Salmi that these three dimensions make for a situation of unprecedented complexity. As a result, in a number of countries, there are complaints from universities about performance indicators overload, stressing that too much energy and time is spent on mining and reporting the data monitored by the government. Moreover, the rankings published by the press have given extra burden and frustration to the academia. There is a risk of pushing HEIs to ‘doctor’ their statistics to improve their standing in the rankings. Another concern is that institutions may pay more attention to factors, such as SAT scores and donations from alumni that receive prominence in the rankings at the expense of improving teaching and learning, which should be more important from an education viewpoint. I agree with Dr Salmi that the universal push for accountability has made the role of university leaders much more demanding. Despite the above-mentioned negative impacts, there are positive benefits of accountability. Dr Salmi revealed that employers, students and HEIs all benefit from increased information about existing programmes and labour market outcomes. For example, in countries where surveys of student engagement are conducted regularly, e.g. Australia, Canada, United Kingdom, United States, higher school graduates are better equipped to choose which HEIs they would like to attend. Also, accountability can be useful when HEIs use their reporting obligations as a management tool to monitor ability to meet strategic targets. To alleviate the pressures, Dr Salmi put forward two principles of good accountability, which I perceive as positive and proactive: Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 24
    • (a) Accountability works better when it is experienced in a constructive way. Hence, accountability should be more about making strategic choices to improve results; rather than justifying a poor performance. (b) The most effective accountability mechanisms are those that are mutually agreed or are voluntarily embraced by HEIs. This ensures a greater sense of responsibility with respect to the feedback process and fuller ownership of the agreed instruments. Apart from Dr Salmi’s recommendations, Professor El-Khawas, at the conference, suggested to define the role of academics in accountability. She pointed out that the policy debate on accountability had given too little attention to systems of assessing learning outcomes (too much focused on inputs). To improve the situation, academics must be involved in (a) developing assessments, (b) testing and refining them, and (c) responding with changes in programmes or delivery. I consider this is a very good suggestion because it provides top-down, and bottom-up approaches in policy design and implementation. It also helps academic staff to reflect on their long held values, as well as assumptions, and to adopt a student outcomes approach to accountability. In sum, I support that academic staff must be accountable to those whom we serve – our students and our societies; as Professor Berdahl put forward in his speech. On the other hand, academic staff must be accountable to the academia itself. Hence, academic freedom in research and teaching is also important so as to achieve a balanced approach to accountability in higher education. I believe that stakeholders and the academia should work hand-in-hand to draw on the positive side of accountability. 5 Discussions and Conclusions As illustrated in sections two and three of this paper, quality assurance mechanisms have been developed within the political, socio-economic and higher education policy contexts in which the HEIs situate. Despite various tensions (relating to accountability, improvement, changing HEI structure, culture and roles of academic staff), this paper delineates that the quality assurance movement has been playing significant roles to transform higher education to meet challenges of today’s knowledge society. It is generally agreed that quality assurance mechanisms serve as policy instruments to achieve a variety of matters. This section discusses main issues of quality assurance, including (a) maintaining the essence of education, (b) enhancing human capital, (c) serving as a change agent, as well as (d) achievements and challenges ahead. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 25
    • 5.1 Quality Assurance: Quality vs Diversity – Maintaining the Essence of Education, Preventing Diploma Disease In face of expanding higher education systems all over the world coupled with globalization and rise of new providers, I observe that quality assurance mechanisms play significant roles to ensure academic standards; as well as to facilitate the development of higher education, which was in crisis (OECD 1987, World Bank 1994) associated with diverse socio-economic factors and decreasing government funding support per student unit after the Second World War. On reflection, I consider that quality assurance mechanisms serve as a gatekeeper to prevent problems of Diploma Disease (Dore 1976) projected by Ronald Dore (1976:8). Dore’s (1976) major claim is that the Diploma Disease is a serious problem – with an expanded education system (a) pupils will become more concerned about being certified, or ‘credentialling’; but not with the mastery of subject matters, or being educated; and (b) educational institutions, at all levels, will be at risk of becoming mere certificate-issuing factories. These concerns do happen in contemporary society. But, I consider that the quality assurance mechanisms have, in some ways, given certain degrees of protection to safeguard academic standards. In fact, since the expansion of higher education during the 1960s-1990s, there have been critiques about falling academic standards, and poor attitudes towards learning and a number of other problems. Harvey (2008) maintains that massification has not broadened access. There has been an overall deterioration in higher education. In recent months, Alderman (2008) warns that academic standards are in decline - students who would once have been failed their degrees pass, and students who would once have been awarded respectable lower seconds are now awarded upper seconds and even firsts. Williams (Chief Executive of QAA, UK) (2008) responds to comments about falling academic standards. He argues that standards are robust, and that the QAA procedures regularly and repeatedly demonstrate the strength of academic quality and standards throughout a very diverse system. Furthermore, degree classification grew up when the higher education system was small and homogeneous. These days, the significance of classification for the future lives and careers of hundreds of thousands of graduates each year is huge. Also, Williams (2008) alerts that in such a complex world as higher education, the old is replaced by the new in every-decreasing timeframes. New challenges need new solutions, and HEIs are facing up to this with an impressive willingness. Quality and standards are easy words to say, but they represent very complex ideas. I consider debates about academic standards result from different interpretations on notions of quality. Let’s contemplate - which notion of quality, excellence, or fitness for purpose, one should hold in today’s massified, or even universalized higher education systems? Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 26
    • Regarding falling academic standards, Harvey (2008) delineates that when exploring impact on ‘quality of degree’ is whether this refers to some abstract ‘gold standard’ notion of what an ideal-type degree should be, or whether it relates to the student learning experience. There are those who would argue that, despite quality assurance, a degree is not what it used to be. Indeed, when a tiny proportion of (privileged) people attended university, the degree was exclusive and quite different from today. Whether the exclusivity made it any better (other than more marketable) is a moot point. The suggestion by advocates of a golden age is that when it was exclusive, the quality of an awarded degree was ‘higher’, implying that students did more in order to successfully achieve an award. These comments are highly contentious. Harvey (2008:4) quotes the Vice-Chancellor of the University of Surrey as follows: I believe it can be argued that the quality of the corresponding top 13% (that is 13% of 43% [of the cohort going into higher education in England]) of graduates in the HE sector is at least as strong as it was for the full cohort in the 1970s. Many senior managers and government leaders experienced a more elite education system and may not fully appreciate the balance between quality and diversity we see today. The vast majority of the additional 30% who gain a HE experience will benefit, raising the overall educational attainment of the nation’s workforce significantly (Snowden 2008). We should note that since massification of higher education during 1960s-1990s, the notion of quality has evolved into ‘fitness for purpose’. At the conference, Professor Brink alerted that quality is a matter of interpretation, and quality needs diversity. It is clear that debates about quality and standards are influenced by perception and experiences of individuals. I agree with Snowden (2008) that we cannot compare the present massified and diversified systems with the elite one. I appreciate Teichler’s views (2001) that it is the co-existence of interpretations and confused claims of key actors that drive the higher education sector to find new ways of mastering the changing conditions. Also, I support Williams (2008) that quality and standards are easy words to say, but they represent very complex ideas. I suggest we need to ask ‘what if’ questions. If there were no quality assurance mechanisms in massified education systems, what would happen? 5.2 Outcome-Based Quality Assurance: Building Academic Culture of Evidence – Enhancing Human Capital in Knowledge Society Nowadays, there has been increasing demand and attention for higher education, which is both a wealth generating body, and a wealth distributing body (Neave 2004a). Investment of human capital, as a key growth input (Fuente 2003), is at the heart of strategies in many countries to promote economic prosperity, fuller employment and social cohesion (OECD 1998). Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 27
    • Human capital is considered as one of the four essential ‘pillars’ – being (a) innovation, (b) technology, (c) human capital, and (d) enterprise dynamics - in societal development (Asgeirsdottir 2005) (See Appendix E). Furthermore, OECD (2007) reiterates that education is a major key factor in forming human capital. Owing to the centrality of the knowledge economy to 21st century development, higher education has assumed unprecedented and significant roles in educating people for the new economy and in creating new knowledge (Altbach 1998). Hence, the higher education sector pays extensive and visible attention to ‘student learning outcomes’. Ewell (n.d.) delineates that the conversation about outcomes and assessment in higher education began in earnest about twenty-five years ago in the United States, about fifteen years ago in Australasia, and somewhat more recently in Europe with the emergence of the Bologna process. As well, Hong Kong is presently undergoing outcome-based audit with government funded HEIs. The rise of outcome-based approach has led to two important trends. First, evidence of the attainment of important learning outcomes – increasingly aligned to global or cross-national standards – is becoming a central element of national accreditation or quality assurance mechanisms. Second, equally important, an institution’s capacity to harness evidence about student learning outcomes that go beyond individual classrooms is critical to self-improvement and curricular alignment. HEIs worldwide are reaching the conclusion that this kind of intentionality about student learning is increasingly necessary in today’s world (Ewell n.d.). I believe the development of assessments, such as, the Collegiate Learning Assessment (CLA) and the Roadmap for the OECD Assessment of Higher Education Learning Outcomes (AHELO) Feasibility Study, and Qualification Frameworks presented at the conference will facilitate learners to take charge of their portfolios, and enable HEIs to prepare for increasing collaboration with stakeholders. I observe that outcome-based quality assurance will take the higher education sector to another step forward. It will facilitate unlocking the black box of the teaching-learning-assessment processes. Also, as Dr Ewell suggested at the conference that this outcome-based approach should enable HEIs to adopt a proactive stance in order to harness external mandates for internal improvement, and to communicate actively with stakeholders. According to Ewell (n.d.), outcome-based approaches provide a way to establish aligned standards of achievement consistent with an increasingly globalized economy and civil society. At the same time, through their incorporation in authentic academic cultures of evidence in individual institutional settings, they provide a way of continuously improvement to the process of teaching and learning. My view is that this approach will boost the education sector to work closely with stakeholders in a proactive manner. Again, it will raise the par for HEIs. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 28
    • In particular, this outcome-based approach quality assurance serves as an agent to meet changing needs of learners as well as workplace in knowledge society, where traditional bureaucracies in industrial era are collapsing, and there are now new ways of organizing work. As educators, we need to ask the fundamental question – are we preparing youngsters for societal changes? (Cheng 2004) 5.3 Quality Assurance - A Change Agent Quality assurance is a change agent: responding to social changes and under the influence of New Public Management (NPM). The quality assurance movement is a result of direct governmental initiative (e.g. France and the UK), or indirect government pressure (e.g. the Netherlands); aiming to change the steering of and, or renew the higher education sector (Kogan et al 2000, Neave 1988, Stensaker 2002, Westerheijden, Brennan and Maassen 1994). As Professor Berdahl’s opening address emphasized: …But the movement to assess and measure the outcomes of higher education stems, I believe, from much more fundamental changes in the societies of the developed world There are studies showing that quality assurance changes institutional policies and practices in both positive and negative ways (Brennan and Shah 2000, Gosling 2004, Stensaker 2002). On the positive side, studies show that there is raising profile of teaching and learning in HEIs (Brennan and Shah 2000), greater centralization (Stensaker 1996, 1999a, 1999b cited in Stensaker 2002) in procedures and in organizational decision-making, as well as more autonomous role for the institutional management, HEIs are more open and transparent - with quantifiable measures, more information than ever before are published about higher education and its outcomes (Stensaker 2002). On the other hand, studies show that there are negative changes, such as HEIs becoming more ‘bureaucratic’. As well, there are studies reporting on tensions among quality, culture and change (Strydom et al 2004). In my opinion, quality assurance has facilitated higher education to meet changing societal needs, assuring academic standards and fulfilling accountability and enhancement. Bearing in mind that (a) HEIs originate from the oldest type of organization since medieval ages, and (b) with massification and globalization of higher education after the Second World War, the purpose, scope, scale, level, direction, and frequency of change have been huge and diverse; not to mention organizational issues, such as, changing student profiles, culture and resistance to change. I believe the quality efforts, research findings, and experiences gained in the last quarter century should form a good foundation for further studies and adaptations for organizational learning. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 29
    • Moreover, an array of factors, including (a) the missions and purposes of higher education, (b) the expectation to contribute to social goals, (c) the recent emphasis on economic utility, (d) the established professional culture of HEIs, and (e) the tensions resulted from contextual changes (Gibbons 2002:7) make changing HEIs profound, complex and challenging. I consider these are reasons why the quality assurance movement itself is a journey of change – with quality assurance mechanisms changing to meet new demands and to stimulate organizational learning. I estimate that the outcome-based quality assurance approach will further raise the par. 5.4 Outcomes of Higher Education: Quality, Relevance and Impact - Achievements and Challenges Ahead Achievements Since 1980s, the quality assurance movement has been evolving and contributing to the development of higher education. Professor Berdahl alerted the importance of ‘learning outcomes’ at his opening speech, and remarked: The question of “learning outcomes” has its origins from outside the academy in the demands for greater efficiency and accountability that arose in the 1980s, largely from the governments funding higher education. From within academia, it began with a subtle, but significant shift from the emphasis on improving the quality of faculty teaching to an emphasis on improving the quality of student learning. Comparing the above remark by Professor Berdahl with other conference presentations - elaborating on various themes, such as, (a) applying the Balance Scorecard to raise institutional profile, (b) researching quality indicators through ‘input-process-output-outcome’ model, (c) researching new forms of rankings and classifications to counter-balance the market dominant scenarios and so on - I see that significant progress with quality assurance mechanisms has been made. Since the 1980s there have been tensions and controversies on various issues, including: language, concepts, power, and change, but quality assurance mechanisms have evolved, and have become more sophisticated emphasizing on evidence-based learning outcomes. Referring to presentations of this conference and the body of literature on the subject of ‘quality’, I agree with other observers that both national and institutional quality assurance have played and will continue to play important roles in (a) maintaining academic standards, (b) achieving improvements specially in teaching and learning, and (c) giving employers and the wider public increased confidence in higher education awards, as well as the knowledge and skills gained by students (Harman 2000). Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 30
    • As delineated in section 3.4 of this paper, there are four to five phases of quality assurance mechanisms focusing on different problems and with different roles of quality assurance (Westerheijden 2007:11). The question of whether quality assurance has had any impacts remains to be answered (Harvey 2008). Yet, Stensaker (2007:60-61 cited in Harvey 2008) identifies four areas of impact; namely, (a) power, (b) professionalism, (c) permeability and (d) public relations. I perceive that quality assurance, through hard and soft managerialism, has brought the higher education sector forward from elite to mass, and to universal systems meeting challenges in today’s global knowledge economies. Challenges Ahead Higher education has been facing strong external and internal forces since the 1980s. With trends of outcome-based quality assurance, proliferation of commercialized rankings, and growing accountability agenda, I consider that there will be great challenges ahead. As Dr Salmi, at the conference, remarked, higher education is facing a situation of unprecedented complexity. The following section delineates a few major issues.  Concepts – Excellence vs Fitness for Purpose As shown in section 5.1, the debates about quality and academic standards result from different interpretations. Many observers and stakeholders had their higher education experience in an elite system. Their comments and views are based on the notion of ‘excellence’, and, in many cases conflict with the current mass, or universal contexts (with different types and sectors of higher education), where the notion of ‘fitness for purpose’ is more appropriate. I would suggest that publicity work should be done to educate stakeholders in this aspect.  Learning Outcomes From perspectives of employers and students, measuring outcomes seems imperative in our knowledge societies with high staff and student mobility. However, assessing the learning- teaching-assessment processes means turning tacit knowledge into explicit, measurable formats. It is not an easy task. It may take some time to research performance indicators, and to train up academic staff to make good use of new processes / procedures so as to achieve the desired outcomes. Governments and HEIs have to put in great efforts and resources for staff development.  Rankings The current situation is market dominated. Harvey (2008:13) warns that rankings provide a real threat to quality processes. In his words: Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 31
    • … The simplistic processes and idiosyncratic construction of league tables appear to have more popular appeal and even credibility amongst a range of stakeholders, including political decision makers, than the meticulous hard work of quality agencies. … the fundamental problem is that they do not appear to ‘reward’ teaching. In particular, contrary to some recent developments in quality assurance that focus on enhancing the student experience of learning, rankings place a potential brake on the development of critical transformed learners. Developing a critical education is not a way to move up league tables. Many stakeholders, including quality agencies, researchers and academic staff hold strong views against rankings. The OECD (2008b cited in Harvey 2008) also points out that the bias in the information base of existing rankings towards research outcomes could detract from efforts to improve educational performance. As shown in section 4 of this paper, I am glad the CHE ranking and Classification studies are under-way so as to counter-balance the current market-dominated situation.  Diversity… Marketization… Commodification… Diploma Disease…? Higher education is massified, marketized, diversified and commodified. There are different types of higher education at different levels, including: transnational higher education, on-line education and virtual universities and academic-business collaboration and so on. The diversified systems pose challenges for quality assurance agencies. There is a growing phenomenon of commodification of higher education. The implications are that HEIs are drawn into the market, producing and selling knowledge as a commodity. There is a risk of putting consumers at the center of institutional strategy. Worst of all, there are increasing degree mills selling low-quality or fake qualifications. Whenever I see advertisements of degree mills, I ask two questions – Why higher education is commodified? Are there any organizations to stop the danger of falling into diploma disease?  Sustaining Quality – Establishing Institutional and Professional Cultures amidst Changes Change is the only constant. Higher education has been subjecting to increased accountability and scrutiny since the 1980s. There have been waves of changes in various aspects. These system level and institutional changes have had fundamental implications for staff workloads, as well as institutional and professional cultures. To sustain quality efforts in the past two decades or so, HEIs have to pay attention to build quality culture, and to shape HEIs as an ideal workplace for talented people to join the force, as some participants pointed out at the end of the conference. I consider that the enhancement of institutional and professional cultures is essential. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 32
    •  Changing Academic Values Under the influence of New Public Management, HEIs have changed in different ways. The relationships between universities and stakeholders have changed as well. Henkel (1997) reflects that institutional leaders have to respond to the insinuation of new values - performance, efficiency, and competition in the emerging markets. Many academics, if not all, have to struggle holding on to the traditional values and conceptions of professional practice that belonged to an elite system. I foresee that the rise of outcome-based quality assurance and the growing accountability agenda will drive HEIs and academic staff to go through another wave of changes. Hence, governments and HEIs need to provide support to facilitate change management, and to establish academic values in contemporary sense. All in all, the roles of HEIs are immense, complex and vital in today’s globalized economy. There are many opportunities and challenges emerging, with political, economic and social implications. As a result, governments, supranational organizations, HEIs and stakeholders have been re-defining various issues in higher education from time to time. In sum, with (a) the rise of knowledge institutions, (b) new modes of knowledge production, (c) diversification of higher education, and (d) public demand for outcomes in learning and research, the management of higher education calls for a mastery of the art and science of applying the right dose of managerialism and academic autonomy. Cecilia C B S Tsui 30 October 2008 Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 33
    • Notes: 1 Expansion of Higher Education  According to Martin Trow (1973) The Transition from Elite to Mass to Universal Access, the fundamental character of a nation’s higher education system changes – its purposes, academic standards, curricula, internal governance etc., as the portion of a society’s young people attending increase from below 15% (elite), to below 50% (mass), to greater than 50% (universal access).  Neave (2004a) points out that UK became a mass higher education system in the course of the late 1980s. 2 New public management  Shattock, M. (2008) “The change from private to public governance of British higher education: its consequences for higher education policy making 1980-2006”, Higher Education Quarterly, 162 (3) 181-203  According to Pollitt and Boukaert (2000), New Public Management (NPM) is a “bundle of disparate elements”. There are three main themes:  Modernization – bringing in faster and more flexible ways of budgeting, managing and accounting for the delivery of services.  Marketization – introducing market type mechanisms, separating purchasers from producers, encouraging user responsiveness, turning clients into customers, making public sector organizations compete with one another.  Minimization – outsourcing or “hollowing out”, pushing decision-making downwards to smaller units, under the semblance of giving them greater autonomy.  Marginson and Wende (2007a:9) states” “The new public management tends towards universality in the United Kingdom, Australia and New Zealand, in much of Eastern Europe and Asia, and in parts of the developing world where reforms in higher education are often generated in World Bank loans-financed programmes. In developed nations and the relatively robust policy systems of emerging nations such as China, Singapore and Malaysia, the reforms are often motivated by desires for global competitiveness but generated from within the nation. The new public management has been applied less completely in Western Europe and North America. But it has influence everywhere...” 3 League Tables as Policy Instruments  Salmi and Saroyan (2007:1) states “In 1963, at the University of California at Berkeley, faculty and the administration objected when the radical student newspaper on campus, Cal Reporter, took the initiative to publish student evaluations of courses and professors. Despite this initial resistance, student evaluations have become part of many universities’ internal accountability mechanisms”. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 34
    • Appendix A The Evolution of Quality Assurance Methods in UK Higher Education Brennan (2007: 39) Type 1 Subject focus – knowledge and curricula ‘Academic’ Professional authority Quality values vary across institution Type 2 Institutional focus – policies and procedures ‘Managerial’ Managerial authority Quality values in variant across institution Type 3 People focus – skills and competencies ‘Pedagogic’ Staff developers / educationalist influence Quality values in variant across institution Type 4 Output focus – graduate standards / learning outcomes ‘Employment Focus’ Employment / professional authority Quality values both variant and in variant across institution Type 5 Student focus – student satisfaction ‘Consumerist’ Authority of the marketplace Quality values vary across institution Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 35
    • Appendix B Rationale for Rapid Growth of Quality Assurance in Higher Education The heighten interests for quality assurance is attributed to an array of pressures, influences and interrelated trends (Altbach et al 1999, Brennan and Shah 2000, Craft 1992, El-Khawas 1998, Harman 1994, 2000, Kristoffersen and Woodhouse 2005, Martin and Stella 2007, van Damme 2002, Westerheijden et al 1994, World Bank 1994); including:  Massification of higher education with increased numbers of students and demographic changes – leading to a change in the nature of higher education, and a feeling that it is necessary to check explicitly those institutional QA procedures are keeping pace with the change. More money is being spent on higher education – leading to an increased desire to ensure that the money is being well-spent  Diversification and privatization of higher education – leading to the need for stringent external checks on them  Deteriorating infrastructure and lack of resources for non-salary expenditures  Decline of academic standards, and concerns on the quality of teaching / research activities  Growing demands from employers and the professions  Increased pressures for increased public accountability – with increased public funding for higher education, and increased national needs for educated manpower, governments want to hold HEIs explicitly accountable for the nature of the graduates  Changing social contexts with fiscal constraints calling for performance and cost-effectiveness  In some countries, decreasing micro-management by governments of HEIs in return for the introduction of an external QA process  Globalization and internationalization of tertiary education, and trade agreements – leading to transnational mobility of students, and a need to have national QA process that is visible to and credible in other countries  Mobility of professionals requiring greater standardization among qualifications  GATs and borderless markets for tertiary education  Changing public management ideologies, i.e. the New Public Management concept  International market for quality assurance services  Gradual development of a more competitive education market where quality becomes an asset and labeling device  Rise of New Public Management and related concepts, i.e. new managerialism and reinventing government (Osborne and Gaebler 1992)  Decline of trust in public institutions and in professionals (Amaral et al 2008) Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 36
    • Appendix C Phase Model of Quality Assurance Schemes (Westerheijden 2007: 11) Role of Internal Evaluation / Nature of Phase Problems Quality Assurance Information Base External Review 1 Serious doubts Identifying substandard Descriptive reports; Summative; accreditation, about educational educational programmes performance indicators checking standards, report to standards the state 2 Doubts about the (i) Public accountability Descriptive / strategic reports Ranking of institutions; efficiency of the (ii) Creating quality selling of (‘self’) covering: identifying good practices; higher education awareness in (i) performance, and report(s) to the state and system and / or institutions (ii) procedures institutions institutions 3 Doubts about Stimulate self-regulation Self-evaluation reports about Audit report to the innovation capacity of institutions’ (i) procedures, and institution and to the state capacity and public accountability (ii) performance quality assurance capacity of institutions 4 Need to stimulate Split between Split between Split between sustainable quality (i) improvement based (i) self-evaluation reports (i) audit report to the culture in on self-regulation, about processes and institution, and institutions and strategies based on (ii) verifying data to be (ii) public accountability SWOT and incorporated in public benchmarking, and databanks (ii) self-reporting about performance indicators New Decreasing Market regulation, i.e. Performance indicators about Publication of comparative Challenges transparency of informing clients ‘products’ performance indicators; higher education (students, employers) (knowledge and skills of standardized testing of systems graduates) graduates? Note: SWOT, strengths, weaknesses, opportunities, threats. Reproduced with slight adaptations from Jeliazkova, M. and Westerheijden, D. F. (2002) “Systematic adaptation to a changing environment: towards a next generation of quality assurance models”, Higher Education, 44, 433-448 Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 37
    • Appendix D Rankings and League Tables – Pros and Cons Pros Cons  Providing easily interpretable information  Bias towards science, biomedical, and  Information transparency for stakeholders technology disciplines, English-language (consumer information) publications, and traditional research outputs  Allocation of funds and formats  Stimulating competitions  Bias towards well-established, well-endowed  Differentiating different types of institutions HEIs and programmes  Unfair to new HEIs with ‘third mission’  Contributing to national quality evaluations activities  Contributing to debates about the definition of  Costly: inflate academic arms race “quality” in higher education  Propelling a growing gap between elite and mass HE, with greater institutional stratification  Over-reliance on research data and peer review  ‘report card’ approach – single number, overall rankings do not reflect true institutional characteristics  Generally do not include teaching quality  May skew student, institutional, government decision-making  Changes in methodology and weighting may result in inconsistent results  Available data may not be ideal to achieve purpose Source: Altbach (2006), Harvey (2008), Hazelkorn (2007) Conference Presentations: Cunningham (2008) Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 38
    • Appendix E Four Essential Pillars in Societal Development Asgeirsdottir (2005) Knowledge Economy Highly- Globalisation R&D Internet MNEs skilled s e n ie is io ogan pr at olum ital er ic s ov ew hn H nt m n N ec E a In T ap yn C D Economic Fundamentals “Economic Fundamentals” (Asgeirsdottir 2005) include the following:  Stable macro-economic policies that allow long-term planning, such as well functioning products and capital markets  Well-functioning labour, product and capital markets  Efficient training policies that help secure that the low-educated are equipped with the right skills, thus avoiding “knowledge divide”  Competition policies, which drives down the costs of technologies  Liberalisation of telecommunication policies  Openness to trade and foreign direct investments to let in “new ideas” Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 39
    • References Note: Conference presentations are not listed below. Papers are available at: http://www.oecd.org/edu/imhegeneralconference2008/papers Alderman, G. (2008). A grotesque bidding game is undermining university standards. In TIMESONLINE, The Times. 18 June 2008. http://www.timesonline.co.uk/tol/comment/columnists/guest_contributors/article4158426.ece?print =yes&randn… Altbach, P. G. (1998). Comparative Higher Education: Knowledge, the University and Development. Greenwich, CT: Ablex. Altbach, P. G. and Davis, T. M. (1999). Global challenge and national response: notes for an international dialogue on higher education. In P. G. Altbach and P. M. Paterson (eds.) Higher Education in the 21st Century: Global Challenge and National Response. IIE Research Report No. 29. Altbach, P. G. (2006). The dilemmas of ranking. In International Higher Education, 42, Winter 2006. Amaral, A. (2006). Preface. In D. F. Westerheijden, B. Stensaker and M. J. Rosa (eds.) Quality Assurance in Higher Education – Trends in Regulation, Translation and Transformation. Dordrecht: Springer. Amaral, A., Rosa, M. J., Tavares, D. A. (2008). Supra-national accreditation, trust and institutional autonomy. A paper presented at the OECD, IMHE General Conference 2008 – Outcomes of Higher Education, Quality Relevance and Impact, 8-10 September 2008, Paris, France. Asgeirsdottir, B. (2005). The knowledge economy and the role of higher education. A speech presented at the Conference on Mission, Money, Management, organized by the China National Institute for Educational Research, OECD Institutional Management in Higher Education and Beijing Geely University, China. Bartelse, J. and van Vught, F. (2007). Institutional profiles: towards a typology of higher education institutions in Europe. In IAU Horizons, 13( 2-3): 9-11. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 40
    • Bleiklie, I. and Powell, W. M. (2005). Universities and the production of knowledge – introduction. In Higher Education, 49: 1-8. Brennan, J. and Shah, T. (2000). Managing Quality in Higher Education: An International Perspective on Institutional Assessment and Change. OECD, SRHE and Open University Press. Brennan, J. (2007). Equity, quality and employability – lessons from the United Kingdom. In B. M. Kehm (ed.) Looking Back to Look Forward – Analyses of Higher Education After the Turn of the Millennium. Werkstattberichte 67. International Centre for Higher Education Research Kassel, University of Kassel, Kassel, Germany. Campbell, S. (2008). Assessment reform as a stimulus for quality improvement in university learning and teaching: an Australian case study. A paper presented at the IMHE General Conference 2008 – Outcomes of Higher Education: Quality Relevance and Impact, 8-10 September, 2008, Paris, France. Cheng, K. M. (2004). Questioning education – learning and society in a post-industrial era. Professorial Inaugural Lecture at the Opening of the CCEAM Conference, Commonwealth Council for Educational Administration and Management, 20 October, 2004, The University of Hong Kong, Hong Kong. Clark, B. (1996). Diversification of higher education: viability and change. In L. Meek, L. Goedegebuure, O. Kivinen and R. Rinne (eds.) The Mockers and Mocked: Comparative Perspective on Differentiation, Convergence and Diversity in Higher Education. Surrey: International Association of Universities and Elsevier Science Ltd. Craft, A. (1992). (ed.) Quality Assurance in Higher Education – Proceedings of an International Conference. London: The Falmer Press. Dill, D. D. (1992). Academic administration. In B. R. Clark and G. Neave (eds.) The Encyclopedia of Higher Education, pp. 1318-1329. Oxford: Pergamon Press. Dill, D. D., Massy, W. F., Williams, P. R., and Cook, C. M. (1996). Accreditation and academic quality assurance: can we get there from here? Change, 28(5): 16-24. Dill, D. D., and M. Soo (2005). Academic quality, league tables, and public policy: a cross-national analysis of university ranking systems. In Higher Education 49(4), pp.495-534. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 41
    • Dill, D. D. (2007). Quality assurance in higher education: practices and issues. In B. McGaw, E. Baker and P. P. Peterson (Editors-in-Chief). The 3rd International Encyclopedia of Education. Elsevier Publications. http://www.unc.edu/ppaq/docs/Encyclopedia_Final.pdf. Dore, R. (1976). The Diploma Disease – Education, Qualification and Development. London: George Allen and Unwin Limited. Ewell, P. (n.d.). Building Academic Cultures of Evidence: A Perspective on Learning Outcomes in Higher Education. http://www.ied.edu.hk/ob1/files/peter ewell_full presentation.pdf. El-Khawas, E. (1998). Quality assurance in higher education: recent progress; challenges ahead. A paper presented at the UNESCO World Conference on Higher Education, 5-9 October 1998, Paris, France. Fuente, A. de la. (2003). Human Capital and National Competitiveness: The State of the Evidence. http://ec.europa.eu/employment_social/news/2003/sep/fuente_en.pdf. Gibbons, M. (1998). Higher Education Relevance in the 21st Century. Washington, DC: World Bank. Gibbons, M. (2002). Globalisation and the future of higher education (1). A presentation at the Conference of Globalisation: What Issues are at Stake for Universities, 20 September 2002, Universite Laval, Quebec, Canada. http://www.bi.ulaval.cal/globalisation_universities/pages/actes/GibbonsMichael.pdf. Gosling, D. (2004). The impact of a national policy to enhance teaching quality and status, England, the United Kingdom. In Quality Assurance in Education. 12(3): 136-149. Green, D. (1994). (ed.) What is Quality in Higher Education? Buckingham: SRHE and Open University Press. Green, D. and Harvey, L. (1993). Quality assurance in Western Europe: trends, practices and issues. In Proceedings of the 5th International Conference on Assessing Quality in Higher Education, July 1993, pp177-196, Bonn. France. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 42
    • Guri-Rosenblit, S., Sebkova, H. and Teichler, U. (2007). Massification and diversity of higher education systems: interplay of complex dimensions. A paper presented at the 2nd Regional Research Seminar for ENA – Main Transformations, Challenges and Emerging Patterns in Higher Education Systems, 5-6 March, 2007, Paris, France. Harman, G. (1994). Regional perspectives: Asia and the Pacific. In International Developments in Assessing Quality in Higher Education – Selected Papers from an International Conference, Montreal 1993. London: The Falmer Press. Harman, G. (1996). Quality Assurance for Higher Education: Developing and Managing Quality Assurance for Higher Education Systems and Institutions in Asia and the Pacific. Bangkok: UNESCO Asia-Pacific Centre of Educational Innovation for Development (ACEID). Harman, G. (2000). Conference summary. In G. Harman (ed.). Quality Assurance in Higher Education - Proceedings of the International Conference on Quality Assurance in Higher Education: Standards, Mechanisms and Mutual Recognition, 8-10 November, 2000, Bangkok: Ministry of University Affairs of Thailand and UNESCO Principal Regional Office for Asia and the Pacific. Harvey, L. (2008). Assaying improvement. Keynote address at the EAIR 30th Annual Forum – Polishing the Silver: Are We Really Improving Higher Education? Denmark: Copenhagen Business School. http://www.eair.nl/Newhome/template/forum/powerpoint/harvey.pdf. Hazelkorn, E. (2007). OECD: consumer concept becomes a policy instrument. In University World News. http://www.universityworldnews.com/article.php?story=200711081… Henkel, M. (1997). Academic values and the university as corporate enterprise. In Higher Education Quarterly, 51(2), 134-143. Jeliazkova, M. and Westerheijden, D. F. (2001). A next generation of quality assurance models – on phases, levels and circles in policy development. A paper presented at the CHER 14th Annual Conference – Higher Education and Its Clients: Institutional Responses to Changes in Demand and in Environment, 2-4 September, 2001, Dijon. Jeliazkova, M. and Westerheijden, D. F. (2002) Systematic adaptation to a changing environment: towards a next generation of quality assurance models. In Higher Education, 44, pp.433-448. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 43
    • Kogan, M., Bauer, M., Bleilie, I. and Henkel, M. (2000). Transforming Higher Education – A Comparative Study. London: Jessica Kingsley. Kristoffersen, D. and Woodhouse, D. (2005). An Overview of World Issues in Quality Assurance. http://www.apqn.org/files/virtual_library/articles/world_issues_in_qa.pdf. Lenn, M. P. (2004). Quality Assurance and Accreditation in Higher Education in East Asia and the Pacific. Paper No. 2004-6. Washington, DC: World Bank. Marginson, S. and Wende, M. van der (2007a). Globalization and Higher Education. Education Working Paper No. 8. Paris: OECD. Marginson, S. and Wende, M. van der (2007b). To rank or to be rank: the impact of global rankings in higher education. In Journal of Studies in International Education, 11(3-4), pp.306-329. Martin, M., and Stella, A. (2007). External Quality Assurance in Higher Education: Making Choices. Paris: UNESCO: International Institute for Educational Planning. Mayhew, L. B., Ford, P. J., and Hubbard, D. L. (1990). The Quest for Quality – The Challenges for Undergraduate Education in the 1990s. San Francisco, CA: Jossey-Bass Publishers. Meek, L. (2005). Quality assurance in Australian higher education. A paper presented at the 3rd Regional Research Seminar for Asia and the Pacific - Best Practices in Relation to Knowledge Production, Promoting Research and Gender Equity and H.E. Quality. 25-26 April, 2005. Seoul, the Republic of Korea. Merisotis, J. P. (2002) On the ranking of higher education institutions. In Higher Education in Europe, xxvii(4), pp.361-363. Neave, G. (1988). On the cultivation of quality, efficiency and enterprise: an overview of recent trends in higher education in Western Europe. In European Journal of Education, 23, pp.7-23. Neave, G. (1998). The Evaluative State Reconsidered. In European Journal of Education, 33 (3), pp. 265-284. Neave, G. (2004a). Personal Interview conducted in December 2004 at Neave’s office. Centre of Higher Education Policy Studies (CHEPS), University of Twente, the Netherlands. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 44
    • Neave, G. (2004b). The temple and its guardians: an excursion into the rhetoric of evaluating higher education. In The Journal of Finance and Management in Colleges and Universities, 1, pp.211-227. OECD (1987). Universities Under Scrutiny. Paris: OECD. OECD (1997a). Thematic Review of the First Years of Tertiary Education. Paris: Directorate for Education, Employment, Labour and Social Affairs, OECD. OECD (1997b). Internationalsation of Higher Education. Paris: Centre for Educational Research and Innovation, OECD. OECD (1998). Human Capital Investment – An Introduction Comparison. Paris: Centre for Educational Research and Innovation, OECD. OECD (2007). Human Capital – How What You Know Shapes Your Life. Paris: OECD. OECD (2008a). Tertiary Education for the Knowledge Society – OECD Thematic Review of Tertiary Education: Synthesis Report – Overview. Paris: OECD. OECD (2008b). Directorate for Education 2008. Informal OECD Ministerial Meeting on Evaluating the Outcomes of Higher Education, Tokyo, 11-12 January 2008, chaired by Kisaburo Tokai, Minister for Education, Culture, Sports, Science and Technology, Japan. http://www.oecd.org/document/45/0,3343,en_2649_33723_39903213_1_1_1_1.00.html. Osborne, D. and T. Gaebler (2002). Re-inventing Government: How the Entrepreneurial Spirit is Transforming the Government. Reading MA : Addison-Wesley. Pollitt, C. and Boukaert, G. (2000). Public Management Reform: A Comparative Analysis. Oxford: Oxford University Press. Salmi, J. and Saroyan, A. (2007). League tables as policy instruments. In International Higher Education, No. 47. http://www.bc.edu/bc_org/avp/soe/cihe/newsletter/Number47/p2_Sa… Schwarz, S. and Westerheijden, D. (2004). Accreditation and Evaluation in the European Higher Education Area. Dordrecht: Kluwer Academic Press. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 45
    • Scott, P. (1998). Massification, internationalization and globalization. In P. Scott (eds.) The Globalization of Higher Education. Buckingham: Open University Press. Scott, P. (2000). Globalization and higher education: challenges for the 21st century. In Journal of Studies in International Education, spring 2000. Scott, P. (2004). Knowledge work in a knowledge society: rethinking the links between university teaching and research. A paper presented at the Higher Education Academy Learning and Teaching Conference 2004: Delivering Excellence, 29 June – 1 July, 2004, the University of Hertfordshire, United Kingdom. Scott, P. (2005). Mass higher education – ten years on. In Perspectives, 9(3), July 2005. Sensicle, A. (1993). One country, two (education) systems: a perspective from Hong Kong. A paper presented at the First Biennial Conference and General Conference of the International Network of Quality Assurance Agencies in Higher Education, Montreal, Canada. Shattock, M. (2008). The change from private to public governance of British higher education: its consequences for higher education policy-making 1980-2006. In Higher Education Quarterly, 62(3), pp.181-203. Snowden, C. (2008) We cannot rely on industry to develop our graduates. In Guardian, 24 June 2008. http://www.guardian.co.uk/education/2008/jun/24/highereducation.uk Stensaker, B. (1996). Organizational Development and Management – The Use and Effects of Evaluations in Universities and Colleges. Report 8/96. Oslo: NIFU. Stensaker, B. (1999a). User surveys in external assessments: problems and prospects. In Quality in Higher Education, 5(3), pp.255-264. Stensaker, B. (1999b). External quality auditing in Sweden: are departments affected? In Higher Education Quarterly, 53(4), pp.353-368. Stensaker, B. (2002). Trance, transparency and transformation – the impact of external quality monitoring on higher education. Keynote presentation at the 7th Quality in Higher Education International Seminar, 29-31 October, 2002. Melbourne, Australia. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 46
    • Stensaker, B. (2007). Impact of quality processes. In L. Bollaert, S. Brus, B. Curvale, L. Harvey, E. Helle, H. T. Jensen, J. Komljenovic, A. Orphanides, A. Sursock (eds.) Embedding Quality Culture in Higher Education: A Selection of Papers From the 1st European Forum for Quality Assurance. pp.59-62. Brussels: European University Association. http://www.eua.be/fileadmin/user_upload/files/Publications/EUA_QA_Forum_publication.pdf. Strydom, J. F., Zulu, N. and Murray, L. (2004). Quality, culture and change. In Quality in Higher Education, 10(3), pp. 207-217. Su, J. L. (1993). Specialized accreditation in Taiwan: issues and perspectives. A paper presented at the First Biennial Conference and General Conference of the International Network of Quality Assurance Agencies in Higher Education, Montreal. Teichler, U. (2001). Mass higher education and the need for new responses. In Tertiary Education and Management, 7: 3-7. Teichler, U. (2003). The future of higher education and the future of higher education research. In Tertiary Education and Management, 9: 171-185. Teichler, U. (2008). Diversification? trends and explanations of the shape and size of higher education. In Higher Education, 56: 349-379. Thune, C. (2002). Future relationships and dilemmas of quality assurance and accreditation. A paper presented at the 24th EAIR Annual Forum, Prague. Trow, M. (1973). Problems in the Transition from Elite to Mass Higher Education. Berkeley, CA: Carnegie Commission on Higher Education. Trow, M. (1974). Problems in the transition from elite to mass higher education, In Policies for Higher Education. Paris: OECD. Trow, M. (1994a). Academic Reviews and the Culture of Excellence. Stockholm: The Council for Studies of Higher Education. Trow, M. (1994b). Managerialism and the Academic Profession: Quality and Control, London: QSC/Open University. Trow, M (2000). From Mass Higher Education to Universal Access: The American Advantage. http://repositories.dclib.org/cgi/viewcontent.cgi?article=1051&context=cshe. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 47
    • van Damme, D. (2002). Quality assurance in an international environment: national and international interests and tensions. Background paper for the CHEA International Seminar III, 24 January, 2002, San Francisco, USA. van Damme, D. (2003). The worldwide quality register – the state of the debate and questions for further discussion, introductory note for the IAUP workshop, 24 March 2003, Brussels, Belgium. van Dyke, N. (2005). Twenty years of university report cards. In Higher Education in Europe, 30(2), pp.103-125. van Vught, F. A. (1992). Higher education quality assurance in Europe: the next step. In Proceedings of the 4th International Conference on Assessing Quality in Higher Education, July 1992, pp117-143, Enschede, Netherlands. van Vught, F. A. (1993). Towards a general model of quality assessment in higher education. A paper presented to the First Biennial Conference of the International Network of Quality Assurance Agencies in Higher Education, 24-28 May, 2003, Montreal, Canada. van Vught, F. A. and Westerheijden, D. F. (1993). Quality Management and Quality Assurance in European Higher Education: Methods and Mechanisms. Luxembourg: Office for Official Publications of the Commission of the European Communities. van Vught, F. A. and Westerheijden, D. F. (1994). Towards a General Model of Quality Assessment in Higher Education. In Higher Education, 28, pp. 355-371. van Vught, F. A. (2008). Mission diversity and reputation in higher education. In Higher Education Policy, 21(2), pp.151-174. Vroeijenstijn, A. I. (2007). External quality assessment in Dutch universities: past and future. In B. M. Kehm (ed.) Looking Back to Look Forward – Analyses of Higher Education After the Turn of the Millennium. Werkstattberichte 67. International Centre for Higher Education Research Kassel, University of Kassel, Kassel, Germany. Westerheijden, D. F., Brennan, J., and Massen, P.A.M. (1994). Changing contexts of quality assessment: recent trends in West European higher education. In D. F. Westerheijden, J. Brennan, and P.A.M. Massen (eds.) Changing Contexts of Quality Assessment: Recent Trends in West European Higher Education. Utrecht: Uitgeverij Lemma B.V. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 48
    • Westerheijden, D.F. (2006). Changing concepts of quality in the assessment of study programmes, teaching and learning. A paper presented at Conference - Quality Assessment in Institutions of Higher Education in Europe: Problems, Practices and Solutions of the Academia Europaea, 23-25 March 2006, Pavia, Italy. Westerheijden, D. F. (2007). The Changing concepts of quality in the assessment of study programmes, teaching and learning www.portlandpress.com/pp/books/online/QAHEE/001/0005/0010005.pdf. Williams, P. (2008). Quality: easy to say, harder to put into practice – Peter Williams on why, despite recent accusations, the QAA believes that standards in UK higher education are robust. In The Guardian, 1 July 2008. http://www.guardian.co.uk/education/2008/jul/01/highereducation.uk/… World Bank (1994). Higher Education: The Lessons of Experience. Development in Practice Series. Washington, DC: World Bank. Young, M. D. (1973). The Rise of the Meritocracy: 1870-2033 – An Essay on Education and Equality. Harmondsworth: Penguin. Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 49