Training Evaluation


Published on

Published in: Business, Technology
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Training Evaluation

  1. 1. Training Evaluation Karishma Dhage PGDM – C Roll No - 131
  2. 2. Training Evaluation “Any attempt to obtain information (feedback) on the effects of training program and to assess the value of training in the light of that information for improving further training” – By Hamblin (1970)
  3. 3. Need for Evaluation ? • Cost Benefit Returns from Training Investment (Major stakeholders like Training Department, Senior Manager, Sponsors, Clients, Trainees) • Enable Improvements in the assessment of Training Needs (check suitability & feasibility of the objectives set for training) • Self-correcting feedback (evaluation will help decide area of improvement) • Feedback on the performance of the Trainees (aims at improving knowledge & attitude of the trainee)
  4. 4. Trading Cycle Identification of Training Needs Develop Training Objectives Measure Training Results Design Training Curriculum Design/Select Training Methods Conduct Training Programme Design Training Methodology
  5. 5. Models of Evaluation • Hamblin Model of Evaluation • Kirkpatrick’s Design of Evaluation • Warr’s Framework of Evaluation • Virmani & Premil’a Model of Evaluation • Peter Bramely’s Model of Evaluation
  6. 6. Hamblin Model of Evaluation • We can evaluate at any of the 4 levels namely: Reaction, Learning, Job Behavior & Functioning • Objectives of the training should be set at each of the 4 levels • According to Hamblin, to be effective, any scheme of evaluation has to be tailor made to fit the needs of the users of training • These approaches are possible for external training as well as in company training
  7. 7. Approaches to Evaluating External & In–Company Training In Company Training Ultimate Value Approach Trainee Centered Approach Cost Benefit Criteria Organization Development criteria Job Behavior Criteria External Training Client Centered Approach Trainee Centered A Learning Criteria Trainee Centered Approach Reach criteria Transfer Criteria Trainee Centered A
  8. 8. Kirkpatricks Design of Evaluation
  9. 9. Warr’s Framework of Evaluation • Context Evaluation (C) : Obtaining & using information about the current operational context, that is, about individual differences & organisational deficiencies • Input Evaluation (I) : determining & using facts & opinion about the available human & material training resources in order to choose between alternative training methods • Process Evaluation (P) : monitoring the training as it is in progress. This involves continuous examination of administrative arrangements & feedback from trainees • Outcome Evaluation (O) : Measuring the consequences of training
  10. 10. Warr’s Framework of Evaluation contd 3 levels of outcome evaluation has been distinguished 1. Immediate Outcome : the changes in trainees knowledge, skills & attitudes which can be identified immediately after the completion of training 2. Intermediate Outcome : the changes in trainee’s actual work behavior which result from training assessment, involves monitoring performance on the job 3. Long Term Outcome : the changes in the functioning of part or all of the organization which have resulted from changes in the work behavior initiated through training
  11. 11. Vermani & Premila’s Model of Evaluation 3 STAGES 1. Pre-Training Evaluation (Period before training during which trainees have expectations from the course) 2. Context & Input Evaluation ( Teaching & Learning stage) 3. Post-Training Evaluation (the stage after training, where trainee’s are supposed to integrate training with their job performance
  12. 12. Peter Bramely’s Model of Evaluation Evaluation Before Designing Learning Event • If the intention is to improve performance by changing behavior, some evaluation should be carried out before a learning activity is designed & run • Knowledge & theory may be necessary but, if behavioral change is expected, some practice of the type of behavior expected will be required
  13. 13. Peter Bramely’s Model of Evaluation contd Evaluation During the Event • Evaluation during a learning activity can be a useful source of information with which we can improve the quality of the event • A focus on the utility of the learning & action planning for its application in the workplace will reinforce the link between objectives of the event & organizational requirements
  14. 14. Peter Bramely’s Model of Evaluation contd Evaluation After the Event • Most difficult stage in the art of evaluation • After the event, analysis would measure changes in effectiveness at the 1. Organizational Level 2. Team Level 3. Individual Level • Change in Behavior (to assess whether change has taken place) • Change in Learning
  15. 15. David Reay’s Approach to Evaluation STAGE 1 – THE TRAIL PHASE • Early Tentative phase of development. 1. Development Stage – involves an informal trail of a sample set of materials, exercises or activities 2. Pilot Testing - more refined operation & tells the evaluation whether the training being evaluated is suitable for the target population
  16. 16. David Reay’s Approach to Evaluation contd STAGE 2 – THE ONGOING PHASE 1. Validation is used to measure the effectiveness of a whole training design 1. Formative evaluation is the monitoring of the effectiveness of different aspects of training on a continuous basis with a view to modification
  17. 17. David Reay’s Approach to Evaluation contd STAGE 3 – THE FINAL PHASE • Seeks answers to the following questions 1. Were the training needs properly identified 2. Were the learning objectives relevant 3. Were the performance standards identified 4. Were the right priorities established 5. Did the training meet the objectives set for it 6. How did the learners feel about the training 7. Were people prepared for training 8. Was the training pitched at the right level 9. Was the trainee able to transfer the training to do the job 10. Did the training result in some benefit to the organization
  18. 18. Pre Training Evaluation • • • • • • IDENTIFICATION OF TRAINING NEEDS Trainer carries out Training needs Survey Uses semi structured interviews Constructs & administers diagnostic questionnaires EVALUATE PERFORMANCE STANDARDS Directly linked to training needs Performance standards identified as KRA’s focus on Productivity, profit, Achieving deadlines, Machine Down time, coping with external pressure etc Method of evaluation would include detailed description of each behavior & method of measuring quality of behavior
  19. 19. Pre Training Evaluation contd. EVALUATE TRAINING OBJECTIVES • To assess whether there is a goal congruence between the trainee’s needs & training objectives EVALUATE TRAINEE’S PROFILE • Trainee’s level of knowledge, attitude & skill prior & post training • Not all but some measures of pre training knowledge, skill & attitudes is desirable to compare them with post training performance
  20. 20. Pre Training Evaluation contd. • • • • INPUT EVALUATION Committee approach, brainstorming session would help in situation where feedback on the subject is already available Helps to establish the validity of the training design EVALUATION DURING THE TRAINING Both evaluation & subsequent modification need careful handling Some methods are examined below
  21. 21. Pre Training Evaluation contd. 1. Observation- trainer’s perceptiveness & observation skills cannot be discounted 2. Behavior Analysis- relevant for imbibing certain skills & attitudes such as team orientation & attitudes towards learners 3. Course Audits- can be conducted at the end of the day or midway through the course 4. Session Assessment- done by using semantic differential scale, 3 point or 5 point & trainee is supposed to only tick mark
  22. 22. Post Training Evaluation REACTION EVALUATION • Useful for restructuring training programmes, so long as it is not confused with an evaluation of learning/job behavior • Learning’s divided into 3 sections 1. Knowledge learning 2. Skill learning 3. Attitude learning
  23. 23. Evaluating training staff performance • • • • EVALUATING THE INDIVIDUAL TRAINER When responsibilities are grouped under a common accountability, measurement of performance against accountability becomes the first means of evaluation Impact of trainers activities on solving human performance provides means of evaluation Elements of the reputation of the trainer includes comments by line manager about the trainer or the results of training External reputation is a measure of trainer performance
  24. 24. Evaluating training staff performance 1. 2. 3. 4. EVALUATING THE TOTAL STAFF Accomplishment of departmental objectives Economic accomplishment of the department. Training should not cost an organization anything. Results of its activities should be worth more than the cost of staffing & running it How effectively the department uses its resources is a third basis of evaluation. One method of measuring this is to have a system for establishing priorities of departmental activities The department’s reputation & the budget request trend can be evaluated
  25. 25. Evaluation of the effectiveness of Training •Modify training to meet learner’s needs by altering exercises, changing training methods & reorganizing the sequence of topics •Document any changes you make & explain these changes to the instructional designers •Use the end of course evaluation to generate ideas for making the training better •Keep in mind that most forms of training evaluations are subjective. Hence, it will be important for you to look at the overall picture & watch trends rather than get bogged down with feedback from one source
  26. 26. Evaluation Reporting Strategy •It helps to justify the existence of the training department, showing the unique value added by the training function to the organization •Second reason is to make a “go” or “no go” decision regarding continuation of a training programme in the future •To gather information to make programme improvements •To meet the unique needs of different audiences
  27. 27. Format of the Evaluation Report If in written form it should be organized as follows 1. Cover Page: this provides the title of the report, the date, the company name, and your contact information 2. Table of Contents: this provides title of all section & sub section, with page numbers 3. Executive Summary: this is a one to two page summary of the report that is aimed at busy senior managers who want to skim a short summary & get all the key facts 4. Evaluation Report: provides specific & detailed information about the training & the learner’s performance. It contains a brief background description of the training, purpose of the evaluation, key question answered by the study, any assumptions/limitations on the study, methods used to carry out the evaluation & the results 5. Conclusion & Recommendation: draws conclusion from evaluation & offers recommendation for improving training
  28. 28. Contents of the Evaluation Report Be sure to include in any report such information as: •The needs that the training was intended to meet •The training objectives for the course •Results of end of course reaction surveys, if they were administered •Results of tests taken by learner’s in aggregate or individual form, if tests were administered •Results of on the job follow up studies to assess transfer of training to the job, if they have been conducted •Impact of training outcome study results, if they have been conducted
  29. 29. Evaluating attitude change •Most difficult evaluating process of all in the review of training •Problem lies in the ability to quantify changes before & after training •It is essential to have evidence of changes in approach, style & adaptability as a result of the learning process •Changes in the affective area of behavior are concerned with experiences of individual persons learning to act in their roles in the concern •A caring company that trains its staff notices the outward & visible signs & is not afraid to comment on nonquantifiable aspects of performance such as motivation, attitudes to work, team building & morale as aresult of its commitment to training
  30. 30. Training Evaluation Challenges •During observation information can be difficult to interpret •During surveys & questionnaires they are more impersonal for participants, & usually not good for getting quotes in participant’s own words
  31. 31. Bibliography • Enriching Human Capital through T & D by P L Rao • • www.