The document discusses how tertiary education in New Zealand is measured using Education Performance Indicators (EPIs), and whether these EPIs effectively contribute to the key objectives of the tertiary sector. It analyzes guidelines from management accounting and education research on developing effective performance measures and links them to the EPIs. While the EPIs may drive completion rates, the document recommends refining them to better reflect strategic objectives, ensure stakeholder input, and reduce potential gaming behavior among tertiary institutions. Measuring programs individually rather than institutions could provide more meaningful benchmarking.

1. How we are
measured
impacts upon
how we teach
Paul Marambos
Southern Institute of
Technology
paul.marambos@sit.ac.nz

2. Research Topic
Will EPIs contribute to
the achievement of
key objectives for
the tertiary sector?

3. Why measure?
 Nearly 700,000 New Zealanders
participated in formal tertiary education
in 2009, including industry training.
 The Government spends around $4 billion
a year in operating expenditure

4. Background
 Widespread increase in measurement
within organisations.
 Management accounting – drive to link
research to practice
 Case study approach

5. Content
 Key tertiary objectives for New Zealand
 Education Performance Indicators (EPIs)
 Research undertaken
 Results
 Suggested changes to the EPIs

6. Key tertiary objectives for New
Zealand
1. provide New Zealanders of all backgrounds with
opportunities to gain world-class skills and
knowledge
2. raise the skills and knowledge of the current and
future workforce to meet labour market demand
and social needs
3. produce high-quality research to build New
Zealand's knowledge base, respond to the needs
of the economy and address environmental and
social changes
4. enable Maori to enjoy education success as Maori.

7. Government expectations of
Universities
The Government expects universities to:
 enable a wide range of students to
successfully complete degree and post-
graduate qualifications
 undertake internationally recognised original
research
 create and share new knowledge that
contributes to New Zealand’s economic and
social development and environmental
management.

8. Government expectations of
Polytechnics
The Government expects polytechnics to:
 enable a wide range of students to complete
industry-relevant certificate, diploma and applied
degree qualifications
 enable local access to appropriate tertiary
education
 support students with low literacy, language, and
numeracy skills to improve these skills and
progress to higher levels of learning
 work with industry to ensure that vocational
learning meets industry needs.

9. Education Performance
Indicators (EPIs):
1. Qualification Completion
Rate
2. Course Completion Rate
3. Retention
4. Progression from levels
1/2/3 to levels 4/5/6

10. Show EPI’s and
league tables for
Polytechnics and
Universities

11. Linkages between objectives
and measures …..
“The Government …...... expect to
see better course and qualification
completion and progression rates for
students as a result of higher quality
teaching and learning, and more
effective and culturally responsive
pastoral care.”

12. Guideline from management
accounting research ….
Care needs to be taken both to
observe the measures actually in
use, and also areas where
measures are absent. It is a
truism that what is measured
tends to drive out what is not
measured, and so omissions may
be as influential as measures in
use. (Ferreira & Otley, 2006)

13. Guideline from
management
accounting
research ….

14. Was guideline from management
accounting research followed….
There is insufficient explanation
provided as to how the TEC’s
EPIs were derived from its
strategic objectives, eg by
utilising strategy maps in order to
link objectives to measures.

15. Guideline from management
accounting research ….
it is as much the process of developing and
using the CPM system, as it is the resultant
performance measures that yield
motivational benefits. To drive motivation
the CPM system should be developed and
used in a way that enhances the
employees’ participation, psychological
empowerment, and goal commitment
(Bourne, Franco-Santos, & Lucianetti, 2012).

16. Was guideline from management
accounting research followed….
There is little evidence to suggest
that a consultative process took
place with key stakeholders
during their development.

17. Guideline from management
accounting research ….
The evidence suggests that the information
included in CPM systems is likely to increase
the understanding of individuals regarding
what is expected from them at work
(Bourne, Franco-Santos, & Lucianetti, 2012).

18. Was guideline from management
accounting research followed….
How educators have interpreted
and responded to the
implementation of the EPI’s is a
potential area for further
research

19. Guideline from education
research ….
The lack of a demonstrated causal logic
between the measures used in many of the
university league tables for first-level degree
programmes and student learning
outcomes …… encourages the gaming
behaviour and strategic mis-investments
reported in the US (Dill & Soo, 2005).

20. Was guideline from education
research followed….
An area that needs further
research is looking at whether
using EPIs to create league
tables for universities and
polytechnics has encouraged
“gaming behaviour” amongst
tertiary organisations in New
Zealand.

21. Guideline from education
research ….
The use of the results of performance
measurement in league tables assumes that
all those being measured start from the
same point; however there are usually more
sources of difference between same-service
providers than there are similarities (Adcroft
& Willis, 2005).

22. EPIs: Qualification completion
One completion:
University 1 One completion:
University 2
Was guideline from education
research followed….

24. Incongruent behaviour
TEOs who select only the most
academically able students, as shown by
their secondary school achievements,
effectively deny educational opportunities
to those whose socio-economic
demographic meant that they may never
have had a chance to obtain similar
secondary school achievements.
(McClelland, 2012)

25. Recommendations
1. Outline how vision, key success factors
and strategic objectives for the tertiary
sector are linked.
2. Outline how the TEC’s EPIs were derived
from its strategic objectives
3. Consider whether alternative or
additional EPIs could be used in order
to ensure that strategic objectives are
met..
4. If EPIs are revised make use of a
consultative process that ensures all key
stakeholders are involved.

26. Recommendations (cont.)
5. Consider what levers universities and
polytechnics can use with regard to improving
the scores of their EPIs
6. Determine what expectations the EPIs
communicate to tertiary organisations and
their teaching staff, and whether these
expectations are in line with the strategic plan
for the tertiary sector.
7. Measure individual programmes, this will
provide more meaningful information that
enables effective benchmarking.
8. Consider providing information on
employment rates of graduates for individual
programmes.