Download PowerPoint Document

Module Objectives
By the end of this module, participants should:
Understand the role of KPI in a quality assurance system.
Understand the attributes of well designed KPI.
Recognise different types of KPI and the consequences of using different types.
Develop KPI for given goals.
Be able to constructively critique a KPI against its intended purpose.

Session Outline
A model for designing good KPI:
KPI in the planning hierarchy
Objectives, indicators, measures, targets and results
Conceptually understanding KPI:
Why use KPI?
ADRI and KPI
Lag and Lead indicators
Tools for critically analysing the design of KPI:
SMARTER KPI
Tips for good KPI
Workshops

1. A model for designing good KPI

KPI in the Planning Hierarchy A one-to-one relationship (e.g. one measure has one target) A one-to-many relationship (e.g. one goal may have many strategies) Mission Goal Objective Indicator Measure Target Result Strategy

KPI in the Planning Hierarchy KPI may be thought of as essential elements of the overall planning and monitoring system. They are derived from the College’s Mission and help track performance towards accomplishing the Mission. A KPI is not a single component, but an integrated collection of components . Mission Goal Objective Indicator Measure Target Result Strategy Indicator Measure Target Result

Designing Good KPIs
Objectives: what are we trying to achieve ?
May be more than one indicator for each objective
Each objective will have strategies on how to achieve them
Indicators: what are you going to measure?
used to assess the present state of progress and to suggest an appropriate course of action.
Measures: how are you going to measure it?
can be qualitative or quantitative data related to inputs, processes or outputs.
Targets: what is the desired result ?
can be minima targets, stretch targets or a combination.
Results: what have you actually achieved ?

Objective
High Quality Teaching
Indicator (one of several)
Student satisfaction with the teaching they experience
Measure
mean student response, per class, to the question, “overall, how satisfied are you with this teacher?”
on a 1 to 5 Likert-type scale
Targets
At least 3.5 on a 1 to 5 scale (where 5 is the best): OR
Best in class compared with benchmark partners
Constructing Good KPIs

2. Conceptually understanding KPI

Why use KPIs?
Key Performance Indicators help us to know whether we are succeeding in our mission.
They are an important management tool for tracking progress against strategic goals.
Working with KPIs encourages system thinking.
in most systems 85% of problems can usually be attributed to the system and 15% to the individual (Juran)
KPIs direct and prioritise behavior towards achievement of the vision, goals and objectives.

APPROACH RESULTS DEPLOYMENT IMPROVEMENT

ADRI and KPIs
Each phase of the ADRI process can involve KPIs.
Examples of ‘Approach’ KPIs include:
Staff training program participation; Student recruitment by target
Examples of ‘Deployment’ KPIs include:
Internet uptime; Teaching space usage; Student class attendance
Examples of ‘Results’ KPIs include:
Student pass rates; Program accreditation
Examples of ‘Improvement’ KPIs include:
Number of review recommendations resolved on time; Policy updates

Balanced Scorecard KPIs
Kaplan & Norton designed a ‘Balanced Scorecard’, with ‘lag’ and ‘lead’ KPIs. Useful for managing large numbers of KPIs in a complex organisation.
‘ Lag’ indicators tell how well a system performed. Mostly used in ADRI ‘Results’ and ‘Deployment’ phases. Examples include:
Student employment rates; research publications; number of prizes
‘ Lead’ indicators help predict system performance. Mostly used in ADRI ‘Approach’ and ‘Improvement’ phases. Examples include:
Staff training; Investment in advertising; Student-computer ratios

Lag, Lead & the Planning Hierarchy Put another way, ‘lag’ KPI focus on the extent to which outcomes-based objectives have been achieved. ‘ Lead’ KPI focus on the extent to which strategies for achieving the objectives are being successfully implemented. Therefore we can structurally separate lag and lead KPI in the planning hierarchy. Mission Goal Objective Indicator Measure Target Result Strategy

Lag, Lead & the Planning Hierarchy There may be separate KPI for objectives and their associated strategies. Provided that objectives are outcomes-based, and strategies support objectives, then: Strategy Indicator Measure Target Result Lag KPI set Lead KPI set Mission Goal Objective Indicator Measure Target Result

3. Tools for critically analysing KPI

S
M
A
R
T
E
R
pecific easurable ligned ealistic imely thical ecorded

Other Tips for Good KPIs
Have them!
Align them to standards.
Test them for validity and reliability.
Discuss and review them. Are they really key ? Don’t overestimate their importance!
Differentiate between lag and lead indicators.
Benchmark them.
Do something with them. KPI should lead to change.
Keep them simple – but not too simple!
Balance focus on positive and negative news (i.e. opportunities for improvement).

Humanise the Process
Address staff concerns and ideas about performance management.
Address concerns about the uncertainty of the data.
Regardless of the topic, keep performance review processes fair and transparent.
Place performance review within a context of learning.
Enable a two way dialogue about performance.
Remember to highlight good news!

Workshop Example Indicators for Learning and Teaching
Example
For each of these three objectives, strategies are provided.
KPIs are then constructed using the model: indicator -> measure -> target
KPIs are then tested using SMARTER
2. The program will be viable. 1. Students will learn to solve problems. 3. The program will be relevant to industry and employers. Objectives

Workshop Example #1 Indicators for Learning and Teaching SPECIFIC? Yes. The indicator is tightly focused on the single issue of courses design. MEASURABLE? Yes. The number of courses that comply can easily be counted. ALIGNED? Yes. The measure relates directly to the indicator; the target relates directly to the measure. REALISTIC? Yes. The KPI has been made realistic by separating new courses, which can implement the strategy directly, from existing courses, which will need to be changed. TIMELY? Yes. The measure is each semester, allowing for current data and trend data. The target is time-specific. ETHICAL? There are no obvious ethical concerns with collecting and using this information. RECORDED? The target implies that trend data will need to be kept and analysed until at least 2010. TIMELY? Yes. The measure is each semester, allowing for current data and trend data. The target is time-specific. REALISTIC? Yes. The target does not appear unachievable. SPECIFIC? Yes. The indicator is tightly focused on the single issue of assessing student’s problem solving abilities. ALIGNED? Yes. The measure relates directly to the indicator; the target relates directly to the measure. MEASURABLE? Yes. Student pass rates are easily measured. RECORDED? The target implies that trend data will need to be kept and analysed until at least 2008. ETHICAL? There are no obvious ethical concerns with collecting and using this aggregated information.  These are SMARTER KPIs Target: 80% by 2007 Measure: student pass rate/semester Design and implement problem based assessment of student learning Indicator: courses include learning outcomes that relate to problem solving Design and deliver problem based curriculum Students learn to solve problems Target: 100% of new courses; 100% of all courses by 2010 Indicator: courses include PBL assessment exercises Measure: number of courses each semester with problem solving learning outcomes included in institutional course guides KPIs Strategy Objective

Workshop Example #2 Indicators for Learning and Teaching TIMELY? No. There is no time definition provided for this KPI REALISTIC? No. Any national or international benchmark will show that the target of 100% pass rates is not realistic. SPECIFIC? No. ‘Student satisfaction’ is very broad. Satisfaction with exactly what? ALIGNED? No. Student satisfaction and student pass rates are not the same thing. MEASURABLE? Yes. Student pass rates are easily measured. RECORDED? There is no indication on how this KPI will be recorded, reported and stored. ETHICAL? Maybe not. The target of 100% may encourage staff to mark too softly. TIMELY? No. There is no time definition provided for this KPI REALISTIC? No. If the measures and targets cannot be measured and do not align, then how can they be realistic? SPECIFIC? No. ‘Student satisfaction’ is very broad. Satisfaction with exactly what? ALIGNED? No. Staff qualifications is not necessarily related to whether the program is excellent. MEASURABLE? No. What is the means for measuring that the program is ‘excellent’? RECORDED? There is no indication on how this KPI will be recorded, reported and stored. ETHICAL? No. This KPI implies that staff can be held accountable for ‘student satisfaction’ and the ‘excellence’ of the program and by virtue of their qualifications. The case for this is not justified here.  These are not SMARTER KPIs Let’s improve them… Measure: (a) Student pass rates (b) Excellent program Target: (a) 100% (b) All staff qualified Indicator: Student satisfaction. Attract the best students by direct marketing of programs to school leavers. The program will be viable. KPIs Strategy Objective

Workshop Example #2 Indicators for Learning and Teaching Measure: (a) Percentage of enrolled students who listed the program as their 1 st preference (via Admission Centre) (b) Proportion of 1 st preference students in national top 10% by academic rank Target: (a) More than 50% (b) At least 20% and up to 40% (assuming four competitor programs in Oman) Indicator: Student program preferences by academic performance. Attract the best students by direct marketing of programs to school leavers. The program will be viable. KPIs Strategy Goal

Workshop Example #3 Indicators for Learning and Teaching Indicator: Acceptance of program by industry Measure: (a) Professional accreditation of program by 2009 (b) Positive feedback from employers and industry Indicator: Graduate employability rate Establish Industry Advisory Boards (IABs) for each program. Invite key employers to be members. The program will be relevant to industry and employers. Measure: % graduates in full time employment in the industry Target: above national average Target: (a) Professional accreditation p assed (b) Positive IAB Program Review KPIs Strategy Goal

Teaching and Learning (done)
Student Outcomes
Educational Support Resources
Research and Research Training
Community Service / Engagement
Organisational Leadership
Staffing
Institutional Infrastructure
Information and Communication
Workshop Instructions Select your topic
Activity
Select an area (from the list) that you are currently working on.
What are two or three main objectives for that area?
What strategies could you have in place to achieve each objective?
Construct appropriate KPIs, including Indicators -> Measures -> Targets.
Check them. Are your KPI SMARTER?
Are they Lag or Lead KPI?
When you have finished (and ONLY then!), try clicking each area for further examples of good KPI.
When you are ready you may EXIT

Workshop Activities Student Outcomes (a) Indicator: Student progress rates Progress Scheme identifies ‘at risk’ students, by Department, and invites them to participate in student learning support services. Students will achieve high rates of success in their learning Measure: Percentage of students progressing to the next level of study without having to repeat any units; differentiated by those who participated in the Progress Scheme and those who didn’t. Target: Improvement trends over three years for both cohorts, but steeper for Progress Scheme students. KPI Strategy Objective

Workshop Activities Student Outcomes (b) Return to list Target: At least 90% Measure: Percentage of graduates in preferred employment within 12 months of graduating Academic Advisors are available for career planning Students will be highly satisfied with their career choices Indicator: Graduate recruitment by employment preference KPI Strategy Objective

Workshop Activities Educational Support Resources Return to list Target: Average satisfaction is above 4 on a 1-5 likert type scale Measure: Academic staff satisfaction results on customer feedback surveys Indicator: Staff and student satisfaction with the help desk The library has developed a ‘one-stop-shop’ help desk for increased support to academic staff and students The library supports learning outcomes Actively communicate with College departments to identify IT requirements and problems with IT services IT services support learning outcomes Measure: Library User Survey asks staff & students to rate their satisfaction with and relevance of helpdesk features. Gap measure is developed. Indicator: Client (Dept) satisfaction Target: The features with highest relevance also have equally high satisfaction KPI Strategy Objective

Workshop Activities Research and Research Training (a) Indicator: External grant applications Staff can access institutional seed grants money to encourage external applications for research funding. We will improve the level of research income via external grants
Measure:
Proportion of FTE Teaching+ Research staff submitting an external grant application (Lead)
Proportion of applications successful (Lag)
Target:
50% of all FTE staff submitting an external grant by 2010.
20% success rate
KPI Strategy Objective

Workshop Activities Research and Research Training (b) Return to list Indicator: S tudents successfully completing postgraduate research degrees in a timely fashion System of bi-annual progress reporting for HDR students that includes follow up and evaluation processes Measure: Proportion of HDR students graduating each year against total number of HDR students scheduled to complete each year.
Target:
at least 30 by 2010
Over 75% scheduled students graduate
We will improve the postgraduate research experience KPI Strategy Objective

Workshop Activities Community Service / Engagement (a) Click here to return to list Return to list Indicator: Participation on community boards / committees Staff are encouraged to nominate for community level boards and committees We will provide leadership to our local community Measure: Proportion of FTE staff (academic + non academic) who hold positions on boards and committees) Target: At least 50% KPI Strategy Objective

Workshop Activities Community Service / Engagement (b) Return to list
Target:
At least 30% in 2007; with an increasing trend of 3% every year for next three years.
At least 50%; and more than any other local College.
Measure:
Proportion of local region new enrolments against College’s total new enrolments
Proportion of local region new enrolments against total local region school leavers
Schools liaison program is conducted with schools in the College’s catchment area. We will increase the number of new enrolments from students in the local region Indicator: Number of commencing students from our region KPI Strategy Objective

Workshop Activities Organizational Leadership Return to list Target: Mean satisfaction is a t least 4 on a 1-5 Likert-type scale. Measure: Mean staff responses to questions about climate. Indicator: Board and Committee Plans Every Board and Committee is required to develop a Plan that is aligned to the College Strategic Plan and includes performance indicators Strategic planning is practiced across all boards and committees The leadership team meets monthly to identify, discuss and respond to any issues raised by staff about the organisational climate, and to review progress on previous responses. We will create and maintain a positive organisational climate Measure: Proportion of Boards and Committees with a current Plan that complies with the College’s template Indicator: Staff satisfaction with their organisational climate. Target: 100% KPI Strategy Objective

Workshop Activities Staffing (a) Return to list Indicator: Staff turnover due to poor recruitment practices Staff are recruited into positions that are related to goals in the College Strategic Plan. Exit interviews are conducted with staff who leave the organisation. People are placed in meaningful jobs
Measure:
expenditure related to staff turnover (eg. Recruitment expenses into recurrent jobs, retrenchment pay)
Number of staff who resign each year
Target:
No more than 0.5% total annual budget
Reduce by 3% per year

Workshop Activities Staffing (b) Return to list Target: At least 60% Measure: Annual percentage of all staff who participated in professional development Professional development is a high priority within the organisation. All staff have access to a professional development program. Staff skills and competencies are developed on the job Indicator: Staff development activity KPI Strategy Objective

Workshop Activities Institutional Infrastructure Return to list Target: Usage at least 60% aggregated across all teaching spaces Measure: The total amount of people using the room over the total capacity of the teaching space. Indicator: Expenditure against budget. All budget centre managers are trained in the College’s financial management processes. Sound financial management is practiced Space planning is incorporated into the Colleges Academic Plan Effective space management supports learning outcomes Measure: Monthly variance of expenditure against budget; reported by account code. Indicator: Occupancy Adjusted for Usage Target: Expenditure no more than 5% over or 10% under budget. KPIs Strategy Objective

Workshop Activities Information and Communication Return to list Target: At least once per month Measure: Number of times QA issues are addressed in minutes Indicator: Level of media awareness of the institution Media and Marketing Strategy clearly targets media outlets Effective marketing maintains the high profile of the institution Heads of Depts hold regular staff meetings where QA issues are discussed Messages about quality assurance from management are consistently deployed to staff
Measure:
Number of ‘hits’ in targeted media outlets (eg newspapers, radio)
Ratio of positive to negative ‘hits’
Indicator: Evidence of discussions about QA policies and strategies in staff meeting minutes
Target:
increasing over time
Greater than 60%

KPI Designing Key Performance Indicators Congratulations on completing this training module.

Download PowerPoint Document

by Jackie72 on Jun 02, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 1

Statistics

Download PowerPoint Document — Presentation Transcript