This presentation gives a fundamental understanding about Kirkpatrick's four levels of evaluation model. It also includes a brief of the fifth level of evaluation by Philip that forms the Kirkpatrick-Philip Model.
Each level is different. Level 1 evaluates Reaction ; Level 2 measures Learning; Level 3 refers to Behavior. Level 4 looks at Results.
At this level, it is important to: collect reactions to instructor, programme, and the learning environment communicate to participants that their feedback is valued Rationale for this level: Reaction can make or break a training programme. Positive reactions are important for trainers who run both in-house and public programmes, e.g., what participants say is often reflected to the managers, who make decisions about the future of training programmes.
Level 1 evaluates Reaction, hence, it is important that people give a positive impression when relating their experience to others who might decide whether to experience the same
Kirkpatrick identified 3 ways of learning that can be accomplished as a result of attending training programmes: Change attitudes - programmes dealing with topics like diversity in the workforce aim primarily at changing attitudes. Improve knowledge - understand new concepts, principles, and/or techniques being taught Increase skills - technical programmes/OJT aim at developing/improving skills. Programmes on topics like leadership, motivation, and communication can aim at all three objectives Rationale for level 2 Learning has taken place when one or more of the following occurs: * Attitudes are changed * Knowledge is increased * Skills is improved
Level 2 measures Learning.
Rationale for level 3: Two reasons for evaluation of behaviour: 1) Benefit of the change 2) Identify if change of behaviour did not occur
Level 3 refers to Behavior, such as “ What behaviour/impact has the training achieved? ” Impacts can be monetary, efficiency, morale, teamwork, etc.
The final results can include increased production, improve quality, decreased costs, reduced frequency, and/or severity of accidents, increased sales, reduced turnover, and higher profits. These are the important reasons for having training programmes. Therefore, state these terms in the final objectives of the training program. These terms can be on long-term basis.
Level 4 looks at Results
No shortcut for the Kirkpatrick’s 4 levels of evaluation. Evaluation bypassing levels 1 & 2 in order to measure changes in behaviour (level 3) is a serious mistake. For e.g., suppose no change in behaviour is discovered. Assumed conclusion is that the programme was ineffective and should be discontinued. This conclusion may not be accurate. Reactions may have been accomplished, but the level 3 or 4 conditions may not have been present. In order for change to occur, four conditions are necessary: 1. The person must have a desire to change 2. The person must know what to do and how to do it 3. The person must work in the right climate 4. The person must be rewarded for changing The training programme can accomplish the first two requirements by creating a positive attitude toward the desired change and by teaching the necessary knowledge and skills. The third condition, right climate, refers to the participant ’s immediate supervisor. Without the right climate, participants may not attempt to use what they have learned. Ways to create a positive job climate: * involve bosses in the development of the programme, for e.g., asking bosses to help determine the needs of subordinates. Such involvement helps to ensure that a programme teaches practical concepts, principles, and techniques. *present the training programme to the bosses/supervisors before the participants are trained.
The fifth level “ROI” was a recent addition to the model by Phillip. It refers to evaluating the return on investment; but this was not in Kirkpatrick’s original model. Putting into perspective ROI in education, what does it mean? How do we know if the training investment pay off? Think about that.
This is an hour long video about ROI in education at the 2010 Social Enterprise Conference, where Professor Jonah Rockoff leads a discussion of how capital is being raised for education reform, and where is it being used to develop innovative, scalable, and sustainable solutions in education. At this point, as ROI is not the focus for this Kirkpatrick’s 4 levels of evaluation presentation, I do encourage you to watch it at your own time. This video is definitely informative and will expand your knowledge about the understanding for the returns on investment in education. See the relevance of what you are instilling in the training evaluation model and you will see the returns on investment soon to come.
+ KIRKPATRICK’S LEVELS OF EVALUATION Cindy Yew Curriculum Project Manager Knowledge Universe, CIRC-AIC Asian International College, Singapore 31 January 2012
Dr Donald +Kirkpatrick’sideas onmeasuringtrainingeffectivenesswere originallypublished in1959. Four level model. Simple. Flexible. Comprehensive. Iae or : im. ro mg suc g . m e pc
The four levels of evaluation… + kirkpatrick’s model Level 4: Level 3: Level 2: Level 1 : Image source: uncw.edu
Level 1: Reaction +measures how those who participate in the programme react to the learning experience “a measure of customer satisfaction” - Dr Don Kirkpatrick Image source: wordpress.com
Level 1: Reaction + Level 1 evaluates reaction e.g., Questionnaire Content Methods Media Trainer style Facilities Course materials • Did the participants like the training? • Did they consider the training relevant? • Did they find the trainer engaging?Iae or : ma rgnlotomg suc es inad yu m e u a .c
Level 1: Reaction +How to collect reactions? 6. Measure reactions against the standard & follow-up where necessary
Level 2: Learning +the extent to which participants change attitudes,improve knowledge, and/or increase skillas a result of attending the programme “in order to evaluate learning, the specific objectives must be determined” - Dr Don Kirkpatrick Image source: cartoonstock.com
Level 2: Learning + Level 2 measures learning e.g., Interviews Surveys Tests (pre- and post-) Observations Combinations • Did the participants learn what was intended to be taught? • Did the participants experience what was intended for them to experience?Iae or : ma rgnlotomg suc es inad yu m e u a .c
Level 3: Behaviour + the extent to which change in behaviour has occurredbecause the participant attended the training programme. One primary question for training at this level is if there is a transfer of knowledge to application at daily activities? Iae or : b oso g . o mg suc o k olcm e .g e
Level 3: Behaviour Level 3 refers to behaviour + e.g., Survey the people who observes the participant in Monetary action back on the job Efficiency Morale Teamwork etc… • Did the participants put their learning into effect when back on the job? • Were the relevant skills and knowledge used? • Was there a measurable change in the activity and performance of the participants when back in their roles?Iae or : ma rgnlotomg suc es inad yu m e u a .c
defined as the final results that Level 4: Results +occurred because the participantsattended the programme. Iae or : b g o.cm mg suc l s to e op
Level 4: Results Level 4 looks at results + e.g., Key Performance Indicators Volume Values Percentages Timescales Return on Investment Other quantifiable aspects of organisational performance, e.g. no. of attrition, wastage, quality rating, achievement of quality and standards, etc. • Did the KPI improve? • To what extent has it improved?Iae or : ma rgnlotomg suc es inad yu m e u a .c
kirkpatrick-philip’s model + ROI = evaluating the return on investment
+ Credits Sources: Evaluating Training Programs: The Four Levels (3rd Edition) - http://bit.ly/uothqS Kirkpatricks Four-Level Model of Training Evaluation - http://bit.ly/vUuY6c Kirkpatrickpartners - www.kirkpatrickpartners.com/OurPhilosophy/tabid/66/Default.aspx Kirkpatrick’s Four Level of Evaluation - http://youtu.be/oBpI0B5PtaA ROI in Education: Increasing Innovation and Choice - http://youtu.be/qRm9htSI3FM