Presentacio Cbuc 08.04.14 V.Def


Published on

Presentation by Nuria Balague (Universitat Autonoma de Catalunya) about Quality Programs.
April 14th, 2008, Barcelona

Published in: Economy & Finance, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Presentacio Cbuc 08.04.14 V.Def

  1. 1. EVALUATION AND QUALITY IN SPANISH ACADEMIC LIBRARIES Núria Balagué Deputy Librarian Barcelona, 14 April 2008
  2. 2. <ul><li>Evaluation and Quality in libraries </li></ul><ul><li>The three levels of quality assurance </li></ul><ul><ul><li>European level </li></ul></ul><ul><ul><li>National level </li></ul></ul><ul><ul><li>Institutional level </li></ul></ul><ul><li>Summing-up ideas </li></ul>Outline:
  3. 3. 1.Evaluation and quality in libraries
  4. 4. user needs and expectations The quality gap Library information services value chain library resources and planning library results
  5. 5. = Measurement + Improvement <ul><li>Evaluation and quality are complementary terms </li></ul><ul><li>Why to evaluate if we do not intend to improve detected weaknesses ? </li></ul><ul><li>How do we know that we are improving in the right direction if we do not submit to periodic evaluations to confirm it and to help us to define new lines of improvement? </li></ul>
  6. 6. Quality Approaches in libraries <ul><li>1960-1990 Statistics, indicators and standards </li></ul><ul><li>1989-2000 Evaluation and TQM </li></ul><ul><li>1990 – EFQM, ISO 9001, LibQual + </li></ul><ul><li>Inputs -> Outputs -> Outcomes </li></ul><ul><li>Resources -> Services -> Users -> </li></ul><ul><li>Continuous improvement </li></ul>
  7. 7. <ul><li>what customers perceive as a result of the comparison of products or services with others, and through reflecting on their own expectations. </li></ul><ul><li>is dynamic; it is a social construct, always relative and continuously evolving: what satisfies the customer today, will fail to do so tomorrow. </li></ul><ul><li>educational sector has become a consumer item governed by the laws of the market and by the impulses and trends influencing citizens’ perception </li></ul>About Quality
  8. 8. <ul><li>Libraries are in transition. Defining and measuring what they do, how much they do, and how well they do it is difficult, because what they do is constantly changing. </li></ul><ul><li>(Denise Troll Covey, 2002) </li></ul>
  9. 9. <ul><li>set of structured processes that serve to plan, direct, communicate, control and improve the quality within an organization. </li></ul><ul><li>process approach helps to gather, organize and distribute suitable information to suitable people, in optimum conditions of costs and time, with the purpose of being able to make the most suitable decisions . </li></ul>A quality-assurance system, a good investment <ul><li>develop aspects such as quality policies and quality objectives, establish the functions and responsibilities of staff members, define products and services, identify working processes and the necessary resources to carry them out, and fix mechanisms of supervision, control, prevention, correction and continuous improvement </li></ul>
  10. 10. <ul><li>Developing and implementing a quality management system consists of several steps including the following: </li></ul><ul><li>a) determining the needs and expectations of customers and other interested parties. </li></ul><ul><li>b) establishing the quality policy and quality objectives of the organization. </li></ul><ul><li>c) determining the processes and responsibilities necessary to attain the quality objectives. </li></ul>Quality management systems approach(1)
  11. 11. Quality management systems approach(2) <ul><li>d) determining and providing the resources necessary to attain the quality objectives. </li></ul><ul><li>e) establishing methods to measure the effectiveness and efficiency of each process. </li></ul><ul><li>f) applying these measures to determine the effectiveness and efficiency of each process. </li></ul><ul><li>g) determining means of preventing nonconformities and eliminating their causes. </li></ul><ul><li>h) establishing and applying a process for continual improvement of the quality management system. </li></ul>
  12. 12. 2. The three levels of quality assurance 1. European level
  13. 13. <ul><li>Bologna Declaration (June 1999)  European Higher Education Area ( EHEA ). </li></ul><ul><li>Bergen Conference (May 2005)  Standards and Guidelines for Quality Assurance in the European Higher Education Area </li></ul><ul><li>proposed by the European Network for Quality Assurance in Higher Education (ENQA). </li></ul>EUROPE
  14. 14. Guidelines for Quality Assurance in the EHEA, Section 1.5 <ul><li>“ In addition to their teachers, students rely on a range of resources to assist their learning. These vary from physical resources such as libraries or computing facilities to human support in the form of tutors, counsellors, and other advisers. Learning resources and other support mechanisms should be readily accessible to students, designed with their needs in mind and responsive to feedback from those who use the services provided. Institutions should routinely monitor, review and improve the effectiveness of the support services available to their students.” </li></ul>
  15. 15. 2. The three levels of quality assurance 2. National level
  16. 16. <ul><li>STRATEGIC LINE No. 3: </li></ul><ul><li>REBIUN AND QUALITY </li></ul><ul><li>To enhance, promote and share resources and expertise that favor potential development of the people who work in University libraries and the quality improvement of management and services </li></ul><ul><li>Strategic goals </li></ul><ul><li>1st Strategic Objective: </li></ul><ul><li>Evaluation and Best Practices </li></ul><ul><li>To promote, disseminate, lead and share evaluation processes for the improvement and assurance of quality of library services, the effectiveness and efficiency of the allocated resources and accountability to the institution. </li></ul><ul><li>2nd Strategic Objective: </li></ul><ul><li>Leadership, management and skills </li></ul><ul><li>To continue joint actions to improve and increase the professionalism of University libraries workers to cope with new challenges </li></ul>
  17. 17. 1994--
  18. 19. Spanish National Plans for Evaluation of the Quality of Universities 1st Plan (1996-2000) • To promote the institutional evaluation. • To compile a homogeneous methodology and common criteria. • To provide society with relevant and objective information on the level of quality of Spanish universities. 2nd. Quality Plan (2001-2006) continued strategic lines of its predecessor. It also sought to promote the participation of the Autonomous Communities by creating autonomic agencies for the quality assessment of higher education. 72 universities 1.450.000 students • To provide educational authorities with objective information for making the decisions under their power.
  19. 20. <ul><li>[Previously 3 libraries were evaluated between 1996-98 without specific guide for libraries] </li></ul><ul><li>Some 50 Spanish university libraries were subjected to a formal evaluation process (about 75% of all Spanish university libraries). </li></ul><ul><li>4 different evaluation guides ( high coincidences in the key aspects) </li></ul><ul><ul><li>AQU-CBUC, then assumed by the Council of Universities (based on evaluation UK) </li></ul></ul><ul><ul><li>ACUA (based on EFQM) </li></ul></ul><ul><ul><li>ANECA (based on EFQM) </li></ul></ul><ul><ul><li>AQU-CBUC (new guide, 2006) </li></ul></ul><ul><li>Same main components of the overall process: </li></ul><ul><ul><li>a self-evaluation </li></ul></ul><ul><ul><li>an external evaluation </li></ul></ul><ul><ul><li>a final report </li></ul></ul>Spanish University Libraries Evaluations 1998-2006
  20. 21. Spanish National Agency for Quality Assessment and Accreditation <ul><li>Its activities include: </li></ul><ul><li>Improving university teaching, research and management activities. </li></ul><ul><li>Fostering the performance of Higher Education Institutions by monitoring objective and transparent processes. </li></ul><ul><li>Providing public administrations with appropriate information for decision-making within the scope of their authority. </li></ul><ul><li>Providing society with information about the achievement of universities’ aims. </li></ul>
  21. 22. Certification of the Quality of University Library Services 10% 3.2. Criteria related to active participation of library service in innovation and cooperation projects. 20% 3.1. Criteria related to the development of improvement plans. 3. ELEMENTS RELATED TO IMPROVEMENTS DERIVED FROM THE EVALUATION PROCESS 10% 2.4. Personnel Policy: education, training and promotion. 10% 2.3. Management, organization and processes. 5% 2.2. Level of relation between the library service and its users: library / teaching, library/ research. 10% 2.1. Level of integration of the library service in the institution 2. ELEMENTS RELATED TO THE PROCESSES 10% 1.3. User satisfaction level. 15% 1.2. Results: circulation, interlibrary loans, user education use of electronic resources, etc. 10% 1.1. Resources: financial, bibliographical, physical space, etc. <ul><li>ELEMENTS BASED ON DATA </li></ul>Ponderation EVALUATION CRITERIA
  22. 23. <ul><ul><li>Objectives </li></ul></ul><ul><li>Promote evaluation </li></ul><ul><li>Improve library services </li></ul><ul><li>Public recognition </li></ul>Certification of the Quality of University Library Services INTERRUPTED PROGRAMME 2,640,339 € 25 Total 889,644 € 6 9 2006 877,695 € 12 16 2005 873,000 € 7 16 2004 Grants total amount Certified libraries Library candidates Year
  23. 24. Evaluation of Catalan university libraries <ul><li>1st. evaluation of Catalan university libraries (1999-2001). Main aims: </li></ul><ul><li>to evaluate library processes and the organizations supporting them. </li></ul><ul><li>to assess the added value of libraries in terms of how they strengthen student learning and contribute to the betterment of research. </li></ul><ul><li>a cross evaluation of Library Services was performed and a final report was prepared. </li></ul><ul><li>the Library Services Evaluation Guide (1999) was based on the methodology proposed in “The effective academic library” (UK, 1995). </li></ul><ul><li>2nd. evaluation of Catalan university libraries (2006-2007). </li></ul><ul><li>new guide: Evaluation Guide of Library Services and their contribution to the quality of learning and research (2006). </li></ul><ul><li>a final report of the cross-evaluation will be published in 2008. </li></ul><ul><li>autonomous region of Spain, 7,000,000 inhabitants, own language (Catalan). </li></ul><ul><li>8 public and 5 private universities. </li></ul><ul><li>200,000 students, 15,000 lecturers and 7,000 administrative staff </li></ul>
  24. 25. Evaluation criteria <ul><li>The library within the university </li></ul><ul><ul><li>1. 1. The regulatory framework 1.2. Strategic planning 1.3. Towards the transformation of LS 1.4. LS's relations with CBUC. </li></ul></ul><ul><li>Core programmes </li></ul><ul><ul><li>Related to : 2.1. Teaching and autonomous learning 2.2. Research 2.3. Digital library and the use of IT 2.4. Cooperation and innovation. </li></ul></ul><ul><li>Organization, management and processes </li></ul><ul><ul><li>3.1. Organization 3.2. Processes 3.3. Management of participation and communication. </li></ul></ul><ul><li>Quality assurance </li></ul><ul><ul><li>4.1. Structure and organization 4.2. Monitoring mechanisms (improvement plans) 4.3. Mechanisms to know user satisfaction. </li></ul></ul><ul><li>Resources </li></ul><ul><ul><li>5.1. Staff 5.2. Buildings and equipments 5.3. Collections 5.4. Budget. </li></ul></ul><ul><li>Results </li></ul><ul><li>Related to: 6.1. Provision of services and volume of activity 6.2. Satisfaction of users and the staff 6.3. Efficiency in the provision of service </li></ul>
  25. 26. Evaluation of Catalan university libraries <ul><li>First evaluation </li></ul><ul><li>(1999-2001) </li></ul><ul><li>UB </li></ul><ul><li>UAB </li></ul><ul><li>UPC </li></ul><ul><li>UPF </li></ul><ul><li>UdG </li></ul><ul><li>UdL </li></ul><ul><li>URV </li></ul><ul><li>Second Evaluation </li></ul><ul><li>(2006-2007) </li></ul><ul><li>UB </li></ul><ul><li>UAB </li></ul><ul><li>UPC </li></ul><ul><li>UPF </li></ul><ul><li>UdG </li></ul><ul><li>UdL </li></ul><ul><li>URV </li></ul><ul><li>UAO </li></ul><ul><li>UVic </li></ul><ul><li>UOC </li></ul><ul><li>different guide </li></ul><ul><li>more evaluation experience </li></ul><ul><li>(almost) similar problems with </li></ul><ul><li>data gathering </li></ul><ul><li>more libraries </li></ul><ul><li>more cooperation </li></ul><ul><li>more IT </li></ul><ul><li>more user -focused </li></ul>
  26. 27. Evaluation of Catalan university libraries <ul><li>No negative reactions </li></ul><ul><li>Improvements are evident </li></ul><ul><li>Training in quality improvement </li></ul><ul><li>We know better our weaknesses </li></ul><ul><li>Lack of community involvement </li></ul><ul><li>Some improvements are in hands of the university (not libraries) </li></ul><ul><li>Little support for improvement </li></ul>External assessment is useful in providing advice, new ideas, and some help from outside the organization, and in the validation of the views of internal reviewers. It works on relevant information (evidences not on intuitions).
  27. 28. 2. The three levels of quality assurance 3. Initiatives at an Institutional level
  28. 29. Best Practices (UAM) Quality teams, TQM (UCarlos III, UAB,…)
  29. 30. Strategic plan s (UPC, UNED, La Rioja...)
  30. 31. Balanced scorecard (UPF,...)
  31. 32. Service charts (UAB, UPF, UPC, …)
  32. 33. User surveys (UPC, …)
  33. 34. LibQual+ (UB, Granada,… )
  34. 35. EFQM (Cádiz, …)
  35. 36. Acquisitions Database Non Conforming Database Non Documentary Vendors Database Reading Rooms Database ILL Database VTLS Corrective and Preventive Actions Vendors Continuous Evaluation UAB Libraries ISO 9001 Quality System Users Surveys, Suggestions and Claims SERVICE LEVEL AGREEMENT Internal & External Audits Annual Review of the Quality System Quality Indicators tra cea bi li ty continuous improvement
  36. 37. UAB Libraries quality journey                                   ANECA Quality Certification                                   AQU evaluation                                   ISO 9000 quality system certification                                   Quality groups                                   European projects                                     06 05 04 03 02 01 00 99 98 97 96 95 94 93 92 91 90  
  37. 38. 3. Summing-up ideas
  38. 39. Multicertification?
  39. 40. Quality: Benefits and Opportunities (1) <ul><li>The adoption of a quality management system is a strategic decision of each organization </li></ul><ul><li>It is not a purpose itself, but a means for the development of a continuous improvement dynamics within the library. </li></ul><ul><li>We may use: </li></ul><ul><ul><li>a model (as EFQM) </li></ul></ul><ul><ul><li>a standard (as ISO 9001) </li></ul></ul><ul><ul><li>the criteria of some quality prize or quality agency </li></ul></ul><ul><ul><li>using only the common sense, unrelated to any “official” reference. </li></ul></ul><ul><li>D ifferent approaches are not incompatible or exclussive, and can be simultaneous </li></ul><ul><li>Not a restrictive positioning but an integrative approach in a quality management environment opened to different techniques and models </li></ul>
  40. 41. Quality: Benefits and Opportunities (2) <ul><li>Today’s organizations are being over audited, which indirectly means that large amount of money and time are (and will be) invested on evaluation. </li></ul><ul><li>If libraries establish a solid foundation for a formal quality-management system, it will be much easier &quot;to adapt the output format&quot; to the specific requirements of any given evaluation that they will have to undergo in order to support university strategies. </li></ul>
  41. 42. <ul><li>To confront the challenges and the demands of the European Higher Education Area (EHEA). </li></ul><ul><li>To consolidate the culture of quality. </li></ul><ul><li>To promote appropriate ethics and values within the organization. </li></ul><ul><li>To find out if there is an overload of bureaucracy. </li></ul><ul><li>To find out gaps in the quality management system. </li></ul><ul><li>A mean to constantly analyze the processes performances. </li></ul><ul><li>No “no-go” areas. </li></ul><ul><li>Detection of incipient problems. </li></ul><ul><li>Possibility of immediate implantation of preventive actions. </li></ul>Evaluation: Benefits and Opportunities (1)
  42. 43. Evaluation: Benefits and Opportunities (2) <ul><li>Evaluation as a routine exercise (positive sense) </li></ul><ul><li>Work team reinforcement </li></ul><ul><li>Encourage openness </li></ul><ul><li>Self-reflection on individual tasks </li></ul><ul><li>Self-consciousness of the importance of one’s own work </li></ul><ul><li>It develops a problem-solving culture </li></ul><ul><li>To provide information, advice, challenge and support for the decision-making </li></ul>
  43. 44. But... <ul><li>It is impossible to evaluate everything and overcontrol is as bad as undercontrol. </li></ul><ul><li>Creating a culture of assessment is not simple, it means changing the mind set and daily activity of librarians and support staff. </li></ul><ul><li>Internet and e-technologies impact the evaluation process, in that evaluations need to become cognizant of the new structures and paradigms of the e-library. </li></ul><ul><ul><ul><li>“ The methods for assessing new resource delivery and the skills required to conduct such assessments evolve at a slower rate” (D.Troll). </li></ul></ul></ul><ul><li>The objective and criteria of quality evaluations should be gradually improved. </li></ul>