Recommended
More Related Content
Similar to HRD849 ROI
Similar to HRD849 ROI (20)
Recently uploaded
Recently uploaded (20)
HRD849 ROI
- 1. Return on Investment: AReturn on Investment: A Performance ManagementPerformance Management ApplicationApplication HRD849HRD849 Evaluation for Training and DevelopmentEvaluation for Training and Development Spring 2009Spring 2009 George ShiraGeorge Shira
- 2. ObjectivesObjectives Introduction to ROIIntroduction to ROI Evolution of ROIEvolution of ROI From Kirkpatrick to PhillipsFrom Kirkpatrick to Phillips ROI PrinciplesROI Principles As developed by PhillipsAs developed by Phillips An ROI ExampleAn ROI Example Performance Management TrainingPerformance Management Training ConclusionConclusion ReferencesReferences
- 3. Introduction to ROIIntroduction to ROI Simple equation with many applicationsSimple equation with many applications ROI = Benefits – InvestmentROI = Benefits – Investment Developed by Jack Phillips in 1970s and 1980sDeveloped by Jack Phillips in 1970s and 1980s Widespread use in 1990sWidespread use in 1990s Great tool to evaluate program impactsGreat tool to evaluate program impacts
- 4. Introduction to ROIIntroduction to ROI Benefits:Benefits: Objective measuresObjective measures Provides clear monetary impactProvides clear monetary impact Easily understood by upper managementEasily understood by upper management Same as financial ROISame as financial ROI Results can be used to improve or discontinueResults can be used to improve or discontinue programsprograms
- 5. Evolution of ROIEvolution of ROI Kirkpatrick’s evaluation model (effectiveness)Kirkpatrick’s evaluation model (effectiveness) Level 1 – ReactionsLevel 1 – Reactions Level 2 – LearningLevel 2 – Learning Level 3 – TransferLevel 3 – Transfer Level 4 – ResultsLevel 4 – Results Phillips adds a 5Phillips adds a 5thth level (benefits)level (benefits) Level 5 – Return on InvestmentLevel 5 – Return on Investment
- 6. ROI PrinciplesROI Principles Developed by Phillips to ensure a credible,Developed by Phillips to ensure a credible, conservative approach to evaluationconservative approach to evaluation 12 Principles divided into 4 subcategories12 Principles divided into 4 subcategories Evaluation PlanningEvaluation Planning Data Collection and AnalysisData Collection and Analysis Investment GuidelinesInvestment Guidelines CommunicationCommunication
- 7. An ROI ExampleAn ROI Example Phillips recommends a 5 step process toPhillips recommends a 5 step process to segment the calculation into manageable pieces.segment the calculation into manageable pieces. Step 1: Focus on a single unitStep 1: Focus on a single unit Step 2: Determine a value for each unitStep 2: Determine a value for each unit Step 3: Calculate the change in performanceStep 3: Calculate the change in performance Step 4: Obtain an annual amountStep 4: Obtain an annual amount Step 5: Determine the annual valueStep 5: Determine the annual value
- 8. ConclusionConclusion ROI (%) = Benefits – CostsROI (%) = Benefits – Costs Some consider calculations intimidatingSome consider calculations intimidating No common languageNo common language Small issues when compared to benefitsSmall issues when compared to benefits CostsCosts x 100x 100
- 9. ReferencesReferences Kirkpatrick, D. (1996). Great ideas revisited.Kirkpatrick, D. (1996). Great ideas revisited. Training & Development, 50Training & Development, 50(1), 54.(1), 54. Phillips, J. J. (1996a). How much is the training worth? (cover story).Phillips, J. J. (1996a). How much is the training worth? (cover story). Training &Training & Development, 50Development, 50(4), 20.(4), 20. Phillips, Jack. (1996b) Measuring roi: the fifth level of evaluation.Phillips, Jack. (1996b) Measuring roi: the fifth level of evaluation. Technical & SkillsTechnical & Skills Training, 4,Training, 4, 10-13.10-13. Phillips, J. J. (1996c). ROI: The search for best practices.Phillips, J. J. (1996c). ROI: The search for best practices. Training & Development, 50Training & Development, 50(2), 42.(2), 42. Phillips, J. J. (1996d). Was it the training?Phillips, J. J. (1996d). Was it the training? Training and Development, 50Training and Development, 50(3), 28-32.(3), 28-32. Phillips, J. J. (2007). Measuring roi fact, fad, or fantasy?Phillips, J. J. (2007). Measuring roi fact, fad, or fantasy? T+D, 61T+D, 61(4), 42-46.(4), 42-46. Phillips, J. J., & Phillips, P. P. (2008). Distinguishing ROI myths from reality.Phillips, J. J., & Phillips, P. P. (2008). Distinguishing ROI myths from reality. PerformancePerformance Improvement, 47Improvement, 47(6), 12-17.(6), 12-17.
- 10. ReferencesReferences Kirkpatrick, D. (1996). Great ideas revisited.Kirkpatrick, D. (1996). Great ideas revisited. Training & Development, 50Training & Development, 50(1), 54.(1), 54. Phillips, J. J. (1996a). How much is the training worth? (cover story).Phillips, J. J. (1996a). How much is the training worth? (cover story). Training &Training & Development, 50Development, 50(4), 20.(4), 20. Phillips, Jack. (1996b) Measuring roi: the fifth level of evaluation.Phillips, Jack. (1996b) Measuring roi: the fifth level of evaluation. Technical & SkillsTechnical & Skills Training, 4,Training, 4, 10-13.10-13. Phillips, J. J. (1996c). ROI: The search for best practices.Phillips, J. J. (1996c). ROI: The search for best practices. Training & Development, 50Training & Development, 50(2), 42.(2), 42. Phillips, J. J. (1996d). Was it the training?Phillips, J. J. (1996d). Was it the training? Training and Development, 50Training and Development, 50(3), 28-32.(3), 28-32. Phillips, J. J. (2007). Measuring roi fact, fad, or fantasy?Phillips, J. J. (2007). Measuring roi fact, fad, or fantasy? T+D, 61T+D, 61(4), 42-46.(4), 42-46. Phillips, J. J., & Phillips, P. P. (2008). Distinguishing ROI myths from reality.Phillips, J. J., & Phillips, P. P. (2008). Distinguishing ROI myths from reality. PerformancePerformance Improvement, 47Improvement, 47(6), 12-17.(6), 12-17.
Editor's Notes
- Hello, this is George Shira. Welcome to my presentation on Return on Investment or ROI. I chose this topic because I work in an organization that has been under extreme cost pressures in recent years. Budgets for associate development and training are commonly reduced, so I wanted to learn more about the financial side of program evaluation. I think that ROI is a very useful technique to understand the true costs and benefits of development programs and communicate them clearly throughout the organization. I choose to complete a research paper style project for this class that digs into the origins of the technique and give guidelines on how it should be applied. I also wanted to develop some kind of handout or job aid that you could use to help you remember the key points in an ROI process.
- Here are some of the objectives that we will cover in this presentation, We’ll start with a brief introduction and move on to the evolution of ROI. Then we’ll talk about the principles of ROI as described by Phillips and the steps that should be followed when calculating ROI.
- Very simple equation- ROI=Benefits-Investment. It reminds me of the equation F=ma. The concept is not complicated or cumbersome, but there are thousands of applications. Engineers spend entire careers understanding and using all of the applications of F=ma. Phillips began developing the process as a grad student in the 50s and 60s. It was described in a series of training publications in the 70s, and it gained a lot of popularity in the 1990s. Currently it has become a common method of evaluation and it is often used to evaluate program impacts.
- Let’s talk about some of the benefits of ROI. Objective measures- ROI only works if impacts of training and costs can be quantified. Sometimes the most difficult step determining how to measure performance changes objectively. Shows a monetary impact. The impact can be positive or negative. ROI is the language of management. It uses the same terminology as many other financial indicators, so it is easily understood. It enables HRD professionals to speak with data, both performance data and financial data. Another benefit is that it can be used to tweak existing programs to maximize their financial impact or to determine which programs are no longer beneficial. A program with negative ROI shows that there are issues present that should be addressed or that the program should be discontinued.
- ROI is an add-on to Kirkpatrick’s evaluation model, so let’s talk a little about that first. Level 1 (Reaction) simply gauges whether or not participants liked or enjoyed the training. Participant feedback forms completed at the end of a training session are a common example of Level 1 data. Level 2 (Learning) measures whether or not any additional knowledge, skills or attitudes were altered due to the training. A post-program knowledge assessment is an example of a Level 2 tool. Level 3 (Behavior) is more difficult to evaluate than the previous two levels because it is a measure of the changes in a participant’s actions as a direct result of the training. Level 3 data is often gained by peer or supervisor input, along with participant data. Level 4 (Results) measures the actual business results that are changed due to the training. Objective measures such as sales quantity, profitability, productivity, inventory levels and product quality are examples of Level 4 data. The Kirkpatrick model is focused on evaluating training effectiveness, without considering the financial benefits (or losses). Phillips adds a “Fifth Level” of evaluation that quantifies the costs and benefits of training.
- To determine ROI, the benefits collected at Level 4 of Kirkpatrick’s model are converted to monetary values and then compared to the total cost of training. In order to be effective, the benefits of training must be expressed in a clear, concise manner. Phillips provides us with 12 principles, divided into four categories, to guide ROI processes. Evaluation Planning 1) Collect data at lower levels for a high level evaluation 2) The previous level of evaluation is not required to be comprehensive when planning a high level evaluation Data Collection and Analysis 3) Use only the most credible sources 4) Use only conservative calculations 5) Isolate the effects of a project 6) Assume that no improvement occurred if no data is available 7) Do not neglect estimation errors 8) Avoid the use of extreme data points and unsupported claims Investment Guidelines 9) Only consider the first year of benefits when calculating ROI for short term solutions 10) Fully load all investment costs 11) Intangible measures are purposefully not converted to monetary values Finally, Communication 12) Communicate the results of ROI to key stakeholders
- Okay, if we understand the principles that form the underlying philosophy, it should be easy to calculate ROI. Unfortunately, it can be tough to start. If we consider Performance Management training, where supervisors learn effective goal setting and management strategies. How do we determine the benefits of a program like that? Fortunately, Phillips provides these steps that should be used to get started when calculating ROI. Step 1: Focus on a single unit Step 2: Determine a value for each unit Step 3: Calculate the change in performance Step 4: Obtain an annual amount Step 5: Determine the annual value The overriding theme is to “objectify” performance characteristics and assign them dollar values. I actually walk through the ROI calculation for a Performance Management training program using in my paper, but I won’t go through that here. You can download the paper on the Clemson HRD site.
- If ROI is so great, then why doesn’t everyone use it? Some are intimidated by the data gathering and calculations. These are not core competencies for many HR groups. Currently there are no existing standards for estimating costs and benefits so company to company comparisons may not be valid. Overall these are small issues when compared to the benefits using this technique.
- The works shown are this slide are the tip of the iceberg when comes to research on ROI. I focused on source work compiled by Phillips and Kirkpatrick for this project, but there are tons of great examples out there that you can use for comparison. So, remember to follow the five steps to get started and be sure to account for all twelve principles when completing ROI calculations. You can check out my paper at the HRD group site or on the wiki page if you want to see a more thorough analysis of the techniques and the five steps applied to a Performance Management training example. There is also a handout that shows the 12 principles, 5 steps and 5 levels of evaluation.