KMS2014DESIGN & MANAGEMENT OF TRAINING          PROGRAMME   Evaluation of Training Programmes
Objectives:At the end of this unit, students will be ableto:• Explain why evaluation is important.• Identify and choose ou...
Introduction• Training effectiveness  – the benefits that the company and the trainees    receive from training• Training ...
Reasons for EvaluatingTraining• Companies are investing millions of dollars  in training programs to help gain a  competit...
Training evaluation provides thedata needed to demonstrate thattraining does provide benefits to          the company.
Types of evaluation• Formative evaluation• Summative evaluation
Formative Evaluation• The evaluation of training that takes place  during program design and development• Helps to ensure ...
Pilot Testing• The process of previewing the training  program with potential trainees and  managers or with other custome...
Summative Evaluation• Evaluation conducted to determine  the extent to which trainees have  changed as a result of partici...
Why Should A Training Program BeEvaluated?• To identify the program’s strengths  and weaknesses• To assess whether content...
Why Should A Training Program BeEvaluated?-ctd.• To gather data to assist in marketing  training programs• To determine th...
The Evaluation Process              Conduct a Needs Analysis        Develop Measurable Learning Outcomes           and Ana...
Training Outcomes: Kirkpatrick’s Four-Level Framework of Evaluation Criteria Level Criteria     Focus   1    Reactions   T...
Outcomes Used in EvaluatingTraining Programs      Cognitive      Skill-Based      Outcomes       Outcomes                 ...
• Cognitive outcomes  – determine the degree to which trainees are    familiar with the principles, facts, techniques,    ...
• Affective outcomes  – include attitudes and motivation  – reaction outcomes – trainees’ perceptions    of the program in...
• Return on Investment (ROI)  – comparing the training’s monetary    benefits with the cost of the training    • direct co...
Determining Whether OutcomesAre Good Good training outcomes need to be: • Relevant • Reliable • Discriminative • Practical
Good Outcomes:Relevance• Criteria relevance – extent to which  training programs are related to  learned capabilities emph...
Criterion deficiency, relevance,and contamination:                                         Outcomes                       ...
Good Outcomes           (continued)• Reliability – degree to which outcomes  can be measured consistently over time• Discr...
Training EvaluationPractices
Training Program Objectives and TheirImplications for Evaluation:Objective                       Learning                 ...
Evaluation Designs: Threats toValidity• Threats to validity refer to a factor that  will lead one to question either:  – t...
Threats to Validity    Threats to Internal             Threats to         Validity             External Validity• Company ...
Methods to Control for Threats toValidity            Pre- and Posttests            Use of Comparison                 Group...
Methods to Control for Threats toValidity-ctd.• Pretests and Posttests – to improve the  internal validity of the study re...
Types of EvaluationDesigns•Posttest – only      •Time Series•Pretest / Posttest   •Solomon Four–                      Grou...
Factors That Influence the Typeof Evaluation DesignFactor                 How Factor Influences Type of Evaluation DesignC...
Importance of Training CostInformation - ROI• To understand total expenditures for  training, including direct and indirec...
To calculate return on investment(ROI), follow these steps:1. Identify outcome(s) (e.g., quality,   accidents)2. Place a v...
To Calculate Return onInvestment (ROI), follow thesesteps: ctd.5. Determine training costs (direct costs +   indirect cost...
Determining Costs for a Cost-Benefit Analysis:    Direct Costs      Indirect Costs                                   Compe...
Measuring Human Capital andTraining Activity• Expenditure per employee• Learning hours received per employee• Expenditure ...
What did I learn from       here?
Upcoming SlideShare
Loading in...5
×

Unit 8 updated

919

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
919
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
61
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • 9
  • 9 The following suggestions specify the types of changes in jobs that are most likely to lead to improvements in each of the five core dimensions. (1) Combine tasks - managers should put existing fractionalized tasks back together to form a new, larger module of work. This increases skill variety and task identify. (2) Create natural work units - managers should design tasks that form an identifiable and meaningful whole. This increases employee “ownership” of the work and encourages employees to view their work as meaningful and important rather than as irrelevant and boring. (3) Establish client relationships - the client is the user of the product or service that the employee works on. Whenever possible, managers should establish direct relationships between workers and their clients. This increases skill variety, autonomy, and feedback for the employee. (4) Expand jobs vertically - vertical expansion means giving employees responsibilities and controls that were formerly reserved for management. It partially closes the gap between the “doing” and “controlling” aspects of the job, and it increases employee autonomy. (5) Open feedback channels - by increasing feedback, employees not only learn how well they are performing their jobs but also whether their performances are improving, deteriorating, or remaining at a constant level. Ideally, employees should receive performance feedback directly as they do their jobs rather than from management on an occasional basis.
  • Unit 8 updated

    1. 1. KMS2014DESIGN & MANAGEMENT OF TRAINING PROGRAMME Evaluation of Training Programmes
    2. 2. Objectives:At the end of this unit, students will be ableto:• Explain why evaluation is important.• Identify and choose outcomes to evaluate a training program.• Discuss the process used to plan and implement a good training evaluation.• Discuss the strengths and weaknesses of different evaluation designs.• Explain the process of conducting a cost- benefit analysis for a training program.
    3. 3. Introduction• Training effectiveness – the benefits that the company and the trainees receive from training• Training outcomes or criteria – measures that the trainer and the company use to evaluate training programs• Training evaluation – the process of collecting the outcomes needed to determine if training is effective• Evaluation design – from whom, what, when, and how information needed for determining the effectiveness of the training program will be collected
    4. 4. Reasons for EvaluatingTraining• Companies are investing millions of dollars in training programs to help gain a competitive advantage• Training investment is increasing because learning creates knowledge – This differentiates between those companies and employees who are successful and those who are not• Because companies have made large dollar investments in training and education and view training as a strategy to be successful, they expect the outcomes or benefits related to training to be measurable
    5. 5. Training evaluation provides thedata needed to demonstrate thattraining does provide benefits to the company.
    6. 6. Types of evaluation• Formative evaluation• Summative evaluation
    7. 7. Formative Evaluation• The evaluation of training that takes place during program design and development• Helps to ensure that: – the training program is well organized and runs smoothly – trainees learn and are satisfied with the program• Provides information about how to make the program better
    8. 8. Pilot Testing• The process of previewing the training program with potential trainees and managers or with other customers• It can be used: – as a “dress rehearsal” to show the program to managers, trainees, and customers – for formative evaluation
    9. 9. Summative Evaluation• Evaluation conducted to determine the extent to which trainees have changed as a result of participating in the training program• May also measure the return on investment (ROI) that the company receives from the training program
    10. 10. Why Should A Training Program BeEvaluated?• To identify the program’s strengths and weaknesses• To assess whether content, organization, and administration of the program contribute to learning and the use of training content on the job• To identify which trainees benefited most or least from the program
    11. 11. Why Should A Training Program BeEvaluated?-ctd.• To gather data to assist in marketing training programs• To determine the financial benefits and costs of the programs• To compare the costs and benefits of training versus non-training investments• To compare the costs and benefits of different training programs to choose the best program
    12. 12. The Evaluation Process Conduct a Needs Analysis Develop Measurable Learning Outcomes and Analyze Transfer of Training Develop Outcome Measures Choose an Evaluation Strategy Plan and Execute the Evaluation
    13. 13. Training Outcomes: Kirkpatrick’s Four-Level Framework of Evaluation Criteria Level Criteria Focus 1 Reactions Trainee satisfaction 2 Learning Acquisition of knowledge, skills, attitudes, behavior 3 Behavior Improvement of behavior on the job 4 Results Business results achieved by trainees
    14. 14. Outcomes Used in EvaluatingTraining Programs Cognitive Skill-Based Outcomes Outcomes Return on Affective Results Investment Outcomes
    15. 15. • Cognitive outcomes – determine the degree to which trainees are familiar with the principles, facts, techniques, procedures, or processes emphasized in the training program – measure what knowledge trainees learned in the program• Skill-based outcomes – assess the level of technical or motor skills – include acquisition or learning of skills and use of skills on the job
    16. 16. • Affective outcomes – include attitudes and motivation – reaction outcomes – trainees’ perceptions of the program including the facilities, trainers, and content• Results – determine the training program’s payoff for the company
    17. 17. • Return on Investment (ROI) – comparing the training’s monetary benefits with the cost of the training • direct costs • indirect costs • benefits
    18. 18. Determining Whether OutcomesAre Good Good training outcomes need to be: • Relevant • Reliable • Discriminative • Practical
    19. 19. Good Outcomes:Relevance• Criteria relevance – extent to which training programs are related to learned capabilities emphasized in the training program• Criterion contamination – extent that training outcomes measure inappropriate capabilities or are affected by extraneous conditions• Criterion deficiency – failure to measure training outcomes that were emphasized in the training objectives
    20. 20. Criterion deficiency, relevance,and contamination: Outcomes Outcomes Identified by Outcomes Needs Related to Measured in Assessment and Training Evaluation Included in Objectives Training Objectives Contamination Relevance Deficiency
    21. 21. Good Outcomes (continued)• Reliability – degree to which outcomes can be measured consistently over time• Discrimination – degree to which trainee’s performances on the outcome actually reflect true differences in performance• Practicality – the ease with which the outcomes measures can be collected
    22. 22. Training EvaluationPractices
    23. 23. Training Program Objectives and TheirImplications for Evaluation:Objective Learning TransferOutcomes Reactions: Did trainees like the program? Skill- Ratings by peers or managers Did the environment help Based: based on observation of behavior learning? Was material meaningful? Cognitive: Pencil-and-paper tests Affective: Trainees’ motivation or job attitudes Skill-Based: Performance on a work sample Results: Did company benefit through sales, quality, productivity, reduced accidents, and complaints? Performance on work equipment
    24. 24. Evaluation Designs: Threats toValidity• Threats to validity refer to a factor that will lead one to question either: – the believability of the study results (internal validity), or validity) – the extent to which the evaluation results are generalizable to other groups of trainees and situations (external validity)
    25. 25. Threats to Validity Threats to Internal Threats to Validity External Validity• Company •Reaction to pretest• Persons •Reaction to• Outcome Measures evaluation •Interaction of selection and training •Interaction of methods
    26. 26. Methods to Control for Threats toValidity Pre- and Posttests Use of Comparison Groups Random Assignment
    27. 27. Methods to Control for Threats toValidity-ctd.• Pretests and Posttests – to improve the internal validity of the study results: – pretraining measure – posttraining measure• Use of comparison groups – Hawthorne effect
    28. 28. Types of EvaluationDesigns•Posttest – only •Time Series•Pretest / Posttest •Solomon Four– Group•Pretest / Posttestwith ComparisonGroup
    29. 29. Factors That Influence the Typeof Evaluation DesignFactor How Factor Influences Type of Evaluation DesignChange potential Can program be modified?Importance Does ineffective training affect customer service, product development, or relationships between employees?Scale How many trainees are involved?Purpose of training Is training conducted for learning, results, or both?Organization culture Is demonstrating results part of company norms and expectations?Expertise Can a complex study be analyzed?Cost Is evaluation too expensive?Time frame When do we need the information?
    30. 30. Importance of Training CostInformation - ROI• To understand total expenditures for training, including direct and indirect costs• To compare costs of alternative training programs• To evaluate the proportion of money spent on training development, administration, and evaluation as well as to compare monies spent on training for different groups of employees• To control costs
    31. 31. To calculate return on investment(ROI), follow these steps:1. Identify outcome(s) (e.g., quality, accidents)2. Place a value on the outcome(s)3. Determine the change in performance after eliminating other potential influences on training results.4. Obtain an annual amount of benefits (operational results) from training by comparing results after training to results before training (in dollars)
    32. 32. To Calculate Return onInvestment (ROI), follow thesesteps: ctd.5. Determine training costs (direct costs + indirect costs + development costs + overhead costs + compensation for trainees)6. Calculate the total savings by subtracting the training costs from benefits (operational results)7. Calculate the ROI by dividing benefits (operational results) by costs  The ROI gives you an estimate of the dollar return expected from each dollar invested in training.
    33. 33. Determining Costs for a Cost-Benefit Analysis: Direct Costs Indirect Costs Compensation Development Overhead for Costs Costs Trainees
    34. 34. Measuring Human Capital andTraining Activity• Expenditure per employee• Learning hours received per employee• Expenditure as a percentage of revenue• Cost per learning hour received• Percentage of expenditures for external services• Learning hours received per training and development staff member• Average percentage of learning activities outsourced• Average percentage of learning content by content area (e.g. basic skills, customer service, executive development)• Average percentage of learning hours provided via different delivery methods (instructor led, technology based)
    35. 35. What did I learn from here?
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×