Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Educational lifecycle process assessment


Published on

How to assess an educational lifecycle process with ISO/IEC 19796-1 and ISO/IEC 15504-2

Published in: Education
  • Be the first to comment

Educational lifecycle process assessment

  1. 1. Educational lifecycle process assessment 4 April 2011 Based on ISO/IEC 19796-1, with ISO/IEC 15504-2
  2. 2. <ul><li>This work is published under the Creative Commons licence &quot;BY-NC-ND 3.0” </li></ul><ul><li>This work is free to share under the creative commons licence: </li></ul><ul><ul><li>&quot;Attribution – Noncommercial – No Derivate 3.0&quot; </li></ul></ul><ul><li>You can copy, distribute and transmit the work under the following conditions: </li></ul><ul><ul><li>1. Attribution </li></ul></ul><ul><ul><li>2. Noncommercial </li></ul></ul><ul><ul><li>3. No derivative works </li></ul></ul><ul><li>The full licence (legal code) can be read online here: </li></ul><ul><ul><li><> </li></ul></ul>4 April 2011
  3. 3. Presentation <ul><li>Situation in educational organization concerning quality management </li></ul><ul><li>Why an educational lifecycle process assessment? </li></ul><ul><li>ELPA </li></ul><ul><ul><li>What is this? </li></ul></ul><ul><ul><li>Which standards are used? </li></ul></ul><ul><ul><li>How does it work? </li></ul></ul>4 April 2011
  4. 4. Educational lifecycle process <ul><li>Educational organizations: activities are similar from one organization to another </li></ul>4 April 2011
  5. 5. Quality management for educational organizations <ul><li>ISO/IEC 19796-1: general framework to describe and develop quality management and quality assurance for educational organizations </li></ul><ul><li>ISO 900x </li></ul><ul><li>National standards (AFNOR, DIN, BSI, etc.) </li></ul><ul><li>National certifications ( Qfor -BE, OPQF - FR, eduQua - CH, etc.) </li></ul>4 April 2011
  6. 6. Why an educational lifecycle process assessment ? <ul><li>Educational organizations generally follow the same process but… </li></ul><ul><ul><li>Does the process meets its purposes? </li></ul></ul><ul><ul><li>What about the expected results? </li></ul></ul><ul><ul><li>What about the capability/maturity of the process? </li></ul></ul><ul><ul><li>What are its strenghts and its weaknesses? </li></ul></ul><ul><ul><li>How can we compare these educational organizations? </li></ul></ul><ul><ul><li>Are they able to offer quality in the design of their courses and to offer an adequate training solution? </li></ul></ul>4 April 2011
  7. 7. Educational lifecycle process assessment <ul><li>Object: </li></ul><ul><ul><li>A methodology that assesses the performance and capability of educational lifecycle process, whatever the adopted quality approach, by applying </li></ul></ul><ul><ul><ul><li>the generic assessment framework of ISO/IEC 15504-2 </li></ul></ul></ul><ul><ul><ul><li>to ISO/IEC 19796-1, the generic educational lifecycle process. </li></ul></ul></ul><ul><li>Objectives: </li></ul><ul><ul><li>To determine to what extent the education lifecycle process is implemented in an organization </li></ul></ul><ul><ul><li>To measure its capability level </li></ul></ul>4 April 2011
  8. 8. Educational lifecycle process assessment <ul><li>A framework… </li></ul><ul><ul><li>To assess education lifecycle process in a standard, objective, and repeatable way. </li></ul></ul><ul><li>Based on </li></ul><ul><ul><li>ISO/IEC 19796-1 and ISO/IEC 15504-2 </li></ul></ul><ul><ul><li>Inspired from the TIPA methodology </li></ul></ul><ul><ul><li>To assess IT Service Management processes in IT organizations, </li></ul></ul><ul><ul><li> </li></ul></ul>4 April 2011
  9. 9. <ul><ul><li>ISO/IEC 15504-2: Information technology - Process assessment - Part 2: Performing an assessment, 2003 </li></ul></ul><ul><ul><ul><li>ISO/IEC SC7 : Software and systems engineering </li></ul></ul></ul><ul><ul><li>ISO/IEC 19796-1: Information technology – Learning, education, and training – Quality management, assurance and metrics. Part 1 General approach, 2005 </li></ul></ul><ul><ul><ul><li>ISO/IEC SC36: I nformation technology for learning, education and training </li></ul></ul></ul>Use of ISO/IEC standards 4 April 2011
  10. 10. ISO/IEC 15504-2 <ul><li>Mostly known for software process improvement, but applicable to any process in any field of activity </li></ul><ul><li>Gives a common reference framework for process assessment </li></ul><ul><li>Sets out the minimum requirements </li></ul><ul><ul><li>For doing an assessment that ensure consistency and repeatability of the ratings </li></ul></ul><ul><li>Gives a generic mean to measure : Process performance and Process capability </li></ul>4 April 2011 ISO/IEC JTC1 SC7
  11. 11. Common reference framework of ISO/IEC 15504-2 for process assessment 4 April 2011 <ul><li>Measurement f ramework </li></ul><ul><li>Capability levels </li></ul><ul><li>Process attributes </li></ul><ul><li>Rating scale </li></ul><ul><li>Process Reference Model </li></ul><ul><li>Domain and scope </li></ul><ul><li>Process purpose </li></ul><ul><li>Process outcomes </li></ul><ul><li>Process Assessment Model </li></ul><ul><li>Process performance indicators </li></ul><ul><li>Process capability indicators </li></ul>Assessment process
  12. 12. Measurement framework <ul><li>ISO/IEC 15504 proposes different steps that a process has to follow to become effective and perfectly mastered. </li></ul>4 April 2011 <ul><li>Measurement f ramework </li></ul><ul><li>Capability levels </li></ul><ul><li>Process attributes </li></ul><ul><li>Rating scale </li></ul>
  13. 13. Measurement framework: Capability levels 4 April 2011 0: Incomplete 1: Performed Informal 2: Managed Monitored and planned 3: Established Well defined 4: Predictable Quantitatively managed 5: Optimizing Continuous improvement Person’s view Organization’s view The process is not implemented and fails to achieve its purpose The process is implemented and achieves its purposes The process is managed and results are specified, controlled and maintained A standard process is defined and used throughout the org. The process is executed consistently within defined limits The process is continuously improved to meet relevant business goals
  14. 14. Measurement framework: Process attributes <ul><li>e.g. Level 3: Established process - A standard process is defined and used throughout the organization </li></ul><ul><ul><li>PA 3.1 Process definition attribute </li></ul></ul><ul><ul><ul><li>Check if the process is established: first check if a standard process is defined in the organization </li></ul></ul></ul><ul><ul><li>PA 3.2 Process deployment attribute </li></ul></ul><ul><ul><ul><li>Check if the standard is actually implemented and used in all the organization </li></ul></ul></ul>4 April 2011
  15. 15. ISO/IEC 19796-1 <ul><li>Is a standard to assist educational organisations to develop their own quality system </li></ul><ul><li>Is a reference framework for the description of quality approaches </li></ul><ul><li>Is a Process Reference Model following the ISO/IEC 15504 terminology </li></ul><ul><ul><li>“ A model comprising definitions of processes in a lifecycle described in terms of process purpose and outcomes, together with an architecture describing the relationships between the processes” </li></ul></ul><ul><ul><li>A high-level description of the processes </li></ul></ul>4 April 2011 PRM ISO/IEC 19796-1
  16. 16. ISO/IEC 19796-1 <ul><li>Provides a PRM that covers the whole educational lifecycle : </li></ul><ul><ul><ul><li>Needs Analysis </li></ul></ul></ul><ul><ul><ul><li>Framework Analysis </li></ul></ul></ul><ul><ul><ul><li>Conception/Design </li></ul></ul></ul><ul><ul><ul><li>Development/Production </li></ul></ul></ul><ul><ul><ul><li>Implementation </li></ul></ul></ul><ul><ul><ul><li>Learning Process </li></ul></ul></ul><ul><ul><ul><li>Evaluation/Optimization </li></ul></ul></ul><ul><li>Provides a generic description model for each of the 7 process categories that contains 38 sub-processes </li></ul>4 April 2011
  17. 17. ISO/IEC 19796-1, a compliant ISO/IEC 15504-2 PRM (?) 4 April 2011 A description scheme for quality management, an example… (Source: ISO/IEC 19796-1, 2005) ISO/IEC 19796-1 : description of the processes in terms of process purpose and outcomes.
  18. 18. Process Assessment Model <ul><li>The PAM forms the basis for the collection of evidence and rating process capability . It is related to one or more PRM </li></ul><ul><li>In our case, the PAM will be built with: </li></ul><ul><ul><li>Datas from Luxembourgish education professionals </li></ul></ul><ul><ul><li>ISO /IEC 15504-2 Performing an assessment </li></ul></ul><ul><ul><li>ISO/IEC 15504-5: An exemplar process assessment model </li></ul></ul><ul><ul><li>ISO /IEC 19796-1: Quality management, assurance and metrics - General approach </li></ul></ul>4 April 2011 <ul><li>PAM </li></ul><ul><li>Indicators : activities, resources, results / usual activities specific to a given process, inputs, outputs </li></ul>
  19. 19. How does it work? 4 April 2011 Measurement f ramework ISO/IEC 15504-2: Performing an assessment PRM ISO/IEC 19796-1 <ul><li>PAM </li></ul><ul><li>Educational process indicators </li></ul>Assessment process ISO/IEC 15504-2 ISO/IEC 15504-3: Guidance on performing an assessment
  20. 20. Process assessment 4 April 2011 Capability Dimension Process Reference Model (PRM) Process Dimension Level 0: Incomplete Level 4: Predictable (2 attributes) Level 3: Established (2 attributes) Level 1: Performed (1 attribute) Level 2: Managed (2 attributes) Level 5: optimizing (2 attributes) ISO/IEC 15504-2 ISO/IEC 19796-1 Needs Analysis Framework Analysis Conception/Design Development/Production Implementation Learning process Evaluation/Optimization Processes and sub-processes
  21. 21. Assessment indicators 4 April 2011 Maturity Dimension Process Assessment Process Dimension Level 0: Incomplete Level 4: Predictable Level 3: Established Level 1: Performed Level 2: Managed Level 5: optimizing Needs Analysis Framework Analysis Conception/Design Development/Production Implementation Learning process Evaluation/Optimization For each of the 9 attributes Process capability assessment (Level 1 to 5) based on Process Attribute Indicators (PAI) : activities, resources and results Level 1 Additional indicators for process performance assessment based on performance indicators: usual activities, intputs and ouputs Amplification for : « Process performance » Process Attribute of Level 1
  22. 22. Process attributes and rating scale 4 April 2011 N P L F 1% 15% 16% 50% 51% 85% 86% 100% Not achieved Achieved Level 1 Performed Level 2 Managed Level 3 Established Level 4 Predictable Level 5 Optimizing Process Performance Performance Management Work Product Management Process Definition Process Deployment Process Measurement Process Control Process optimization Process innovation Needs analysis   F L L P N     Framework analysis F F F F F   L   p Rating scale Fully Largely Partially Not F   L   P N
  23. 23. Assessment of educational lifecycle processes <ul><li>Assessment of process implementation </li></ul><ul><ul><li>Is the process performed, is it in place in the organization? </li></ul></ul><ul><ul><li>Does the process meet its purpose? </li></ul></ul><ul><ul><li>Does it produce its expected results? </li></ul></ul><ul><li>Assessment of process capability </li></ul><ul><ul><li>What is the maturity level of each process? </li></ul></ul><ul><ul><li>What is working well? What activities are well executed? </li></ul></ul><ul><ul><li>What should be improved? What activities are missing? </li></ul></ul>4 April 2011
  24. 24. A New Work Item for ISO/IEC JTC1 SC36 <ul><li>ISO/IEC JTC1 SC36 - State College Plenary Meeting (Septembre 2010): proposes a Part 6 Conformity Assessment Model for ISO/IEC 19796-1 </li></ul><ul><li>Part 6 of ISO/IEC 19796 will propose: </li></ul><ul><ul><li>An assessment model for ISO/IEC 19796-1 as a framework to assess the adoption and the application of ISO/IEC 19796-1 </li></ul></ul><ul><ul><li>A Guideline for certification purposes and assessment of quality systems based on Part 1 </li></ul></ul>4 April 2011