Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

NTLT 2013 - Paul Maramos - How we are measured impacts upon how we teach

234 views

Published on

How we are measured impacts upon how we teach

Published in: Education, Business
  • Be the first to comment

  • Be the first to like this

NTLT 2013 - Paul Maramos - How we are measured impacts upon how we teach

  1. 1. How we are measured impacts upon how we teach Paul Marambos Southern Institute of Technology paul.marambos@sit.ac.nz
  2. 2. Research Topic Will EPIs contribute to the achievement of key objectives for the tertiary sector?
  3. 3. Why measure?  Nearly 700,000 New Zealanders participated in formal tertiary education in 2009, including industry training.  The Government spends around $4 billion a year in operating expenditure
  4. 4. Background  Widespread increase in measurement within organisations.  Management accounting – drive to link research to practice  Case study approach
  5. 5. Content  Key tertiary objectives for New Zealand  Education Performance Indicators (EPIs)  Research undertaken  Results  Suggested changes to the EPIs
  6. 6. Key tertiary objectives for New Zealand 1. provide New Zealanders of all backgrounds with opportunities to gain world-class skills and knowledge 2. raise the skills and knowledge of the current and future workforce to meet labour market demand and social needs 3. produce high-quality research to build New Zealand's knowledge base, respond to the needs of the economy and address environmental and social changes 4. enable Maori to enjoy education success as Maori.
  7. 7. Government expectations of Universities The Government expects universities to:  enable a wide range of students to successfully complete degree and post- graduate qualifications  undertake internationally recognised original research  create and share new knowledge that contributes to New Zealand’s economic and social development and environmental management.
  8. 8. Government expectations of Polytechnics The Government expects polytechnics to:  enable a wide range of students to complete industry-relevant certificate, diploma and applied degree qualifications  enable local access to appropriate tertiary education  support students with low literacy, language, and numeracy skills to improve these skills and progress to higher levels of learning  work with industry to ensure that vocational learning meets industry needs.
  9. 9. Education Performance Indicators (EPIs): 1. Qualification Completion Rate 2. Course Completion Rate 3. Retention 4. Progression from levels 1/2/3 to levels 4/5/6
  10. 10. Show EPI’s and league tables for Polytechnics and Universities
  11. 11. Linkages between objectives and measures ….. “The Government …...... expect to see better course and qualification completion and progression rates for students as a result of higher quality teaching and learning, and more effective and culturally responsive pastoral care.”
  12. 12. Guideline from management accounting research …. Care needs to be taken both to observe the measures actually in use, and also areas where measures are absent. It is a truism that what is measured tends to drive out what is not measured, and so omissions may be as influential as measures in use. (Ferreira & Otley, 2006)
  13. 13. Guideline from management accounting research ….
  14. 14. Was guideline from management accounting research followed…. There is insufficient explanation provided as to how the TEC’s EPIs were derived from its strategic objectives, eg by utilising strategy maps in order to link objectives to measures.
  15. 15. Guideline from management accounting research …. it is as much the process of developing and using the CPM system, as it is the resultant performance measures that yield motivational benefits. To drive motivation the CPM system should be developed and used in a way that enhances the employees’ participation, psychological empowerment, and goal commitment (Bourne, Franco-Santos, & Lucianetti, 2012).
  16. 16. Was guideline from management accounting research followed…. There is little evidence to suggest that a consultative process took place with key stakeholders during their development.
  17. 17. Guideline from management accounting research …. The evidence suggests that the information included in CPM systems is likely to increase the understanding of individuals regarding what is expected from them at work (Bourne, Franco-Santos, & Lucianetti, 2012).
  18. 18. Was guideline from management accounting research followed…. How educators have interpreted and responded to the implementation of the EPI’s is a potential area for further research
  19. 19. Guideline from education research …. The lack of a demonstrated causal logic between the measures used in many of the university league tables for first-level degree programmes and student learning outcomes …… encourages the gaming behaviour and strategic mis-investments reported in the US (Dill & Soo, 2005).
  20. 20. Was guideline from education research followed…. An area that needs further research is looking at whether using EPIs to create league tables for universities and polytechnics has encouraged “gaming behaviour” amongst tertiary organisations in New Zealand.
  21. 21. Guideline from education research …. The use of the results of performance measurement in league tables assumes that all those being measured start from the same point; however there are usually more sources of difference between same-service providers than there are similarities (Adcroft & Willis, 2005).
  22. 22. EPIs: Qualification completion One completion: University 1 One completion: University 2 Was guideline from education research followed….
  23. 23. Incongruent behaviour TEOs who select only the most academically able students, as shown by their secondary school achievements, effectively deny educational opportunities to those whose socio-economic demographic meant that they may never have had a chance to obtain similar secondary school achievements. (McClelland, 2012)
  24. 24. Recommendations 1. Outline how vision, key success factors and strategic objectives for the tertiary sector are linked. 2. Outline how the TEC’s EPIs were derived from its strategic objectives 3. Consider whether alternative or additional EPIs could be used in order to ensure that strategic objectives are met.. 4. If EPIs are revised make use of a consultative process that ensures all key stakeholders are involved.
  25. 25. Recommendations (cont.) 5. Consider what levers universities and polytechnics can use with regard to improving the scores of their EPIs 6. Determine what expectations the EPIs communicate to tertiary organisations and their teaching staff, and whether these expectations are in line with the strategic plan for the tertiary sector. 7. Measure individual programmes, this will provide more meaningful information that enables effective benchmarking. 8. Consider providing information on employment rates of graduates for individual programmes.

×