Successfully reported this slideshow.

ROI For HR Practice Toolkit

1,595 views

Published on

Published in: Technology, Education
  • Be the first to comment

ROI For HR Practice Toolkit

  1. 1. ROI of Human Capital Interventions<br />A Practice Toolkit<br />
  2. 2.
  3. 3. Step 1 <br />Presenting Problem to Real Problem<br />
  4. 4. Step 1. Presenting Problem to Real Problem<br />The first task of the practitioner is almost always analysis<br />Starts with a “presenting” problem – not a “hunch” but a problem … as it presents itself<br />
  5. 5. Step 1. Presenting Problem to Real Problem<br />Example…<br />Let's take an example – higher staff turnover for Graduate Engineers (let say compared to an industry standard)<br />The first question should not be, what skills and knowledge are required. It should be which metric(s) are affected negatively.<br />
  6. 6. Step 1. Presenting Problem to Real Problem<br />Example…<br />
  7. 7. Step 1. Presenting Problem to Real Problem<br />1.1 Calculating benchmark employee cost<br />
  8. 8. Step 1. Presenting Problem to Real Problem<br />1.2 Cost of 'covering' a vacant position (Calculated costs of other employees 'filling in' while the position is vacant)<br />
  9. 9. Step 1. Presenting Problem to Real Problem<br />1.3 Cost to fill a vacant position<br />
  10. 10. Step 1. Presenting Problem to Real Problem<br />1.4 On boarding & Orientation cost <br />
  11. 11. Step 1. Presenting Problem to Real Problem<br />1.5 Cost of productivity ramp-up (During the first 3 months, an average new employee performs at 50% productivity of a tenured top performing employ<br />
  12. 12. Step 1. Presenting Problem to Real Problem<br />1.6 Unit cost for filling a vacancy <br />R 74964<br />Two important learning points:<br />Know your metrics<br />Negotiate, and even request help from your client to determine the right metrics. They are closer to the business and at the same time you obtain by in for the intervention.<br />
  13. 13. Step 1. Presenting Problem to Real Problem<br />1.7 Total cost of replacing graduate engineers<br />R 74964 x 6 = R 449784<br />Let's assume that we had to replace five graduate engineers the past six months <br />
  14. 14. Step 1. Presenting Problem to Real Problem<br />Now we can move on to analysis<br />We know now which metrics are affected<br />Consultant speciality<br />
  15. 15. Step 1. Presenting Problem to Real Problem<br />The basic tasks in analysis<br />
  16. 16. Step 1. Presenting Problem to Real Problem<br />Two types of data<br />Quantitative<br />”The truth is in a number” – however does not say much about context<br />Data<br />” More about context, how people experience in their own words”. <br />Qualitative<br />
  17. 17. Data gathering<br />Before <br />intervention<br />Before <br />intervention<br />Step 1. Presenting Problem to Real Problem<br />Questionnaires<br />Surveys<br />Tests<br />Interviews<br />Focus groups<br />Observation<br />Performance records<br />Knowledge and skills testing<br />Program follow up<br />Project assignments<br />After <br />intervention<br />After <br />intervention<br />
  18. 18. Step 1. Presenting Problem to Real Problem<br />Data gathering<br />Before <br />intervention<br />Target population/participants;<br />Supervisors of target population/participants;<br />Subordinates of target population/participants;<br />Peer Group;<br />HRD Staff;<br />Documentation:<br /><ul><li>Organizational Performance Records:
  19. 19. Disciplinary records;
  20. 20. Safety records;
  21. 21. Scorecards</li></ul>After <br />intervention<br />
  22. 22. Step 1. Presenting Problem to Real Problem<br />Contextual factors to consider when selecting data collection methods<br />Before <br />intervention<br />After <br />intervention<br />Time for participants to respond;<br />Costs;<br />Amount of disruption of normal activities;<br />What is the level of certainty that you will achieve with the result;<br />Practicality;<br />Culture/Philosophy – example the 10% syndrome.<br />
  23. 23. Step 1. Presenting Problem to Real Problem<br />Requirements for effective data collection instruments and techniques<br />Before <br />intervention<br />Construct validity ( methods, techniques and test instruments should be evaluated in terms of its theoretical grounding);<br />Predictive validity (methods, techniques and test instruments should be evaluated regularly in terms of its ability to predict what it purport to predict);<br />Face validity – for line management very important!<br />After <br />intervention<br />
  24. 24. Step 1. Presenting Problem to Real Problem<br />Requirements for effective data collection instruments and techniques<br />Before <br />intervention<br />After <br />intervention<br />Reliability:<br />Produce consistent results over time, that differentiates effectively, and are independent of the assessor/data collector, or context.<br />
  25. 25. Step 1. Presenting Problem to Real Problem<br />Data consolidation<br />
  26. 26. Step 1. Presenting Problem to Real Problem<br />Data analysis<br />
  27. 27. Note the following !<br />Because this toolkit does not aim to transfer knowledge about analysis as such the previous slides provides a brief and high level overview, for the sake of a clearity… for the sake of the toolkit we assume that the analysis provided clarity about the problem….<br />
  28. 28. Step 1. Presenting Problem to Real Problem<br />Brief Exposition of Problem<br />Our analysis shows that the HR and line managers involved in selection decisions do interviews in a haphazard and nonscientific fashion<br />Solution<br />Following discussions with consultants, HR experts and line all agreed we need to implement competency based interviewing<br />Lets assume this based on the core example<br />
  29. 29. Step 1. Presenting Problem to Real Problem<br />The training department is instructed to develop an appropriate program and put all line and support staff through training – they however indicate that a change in results is required quickly and that ROI for the training need to be justified soon after implementation…<br />Lets assume this based on the core example<br />
  30. 30. Presenting Problem to Real Problem<br />So we have addressed this even before we asked any questions about training – and now we can move on to the next step<br />Starts with a “presenting” problem – not a “hunch” but a problem … as it presents itself<br />
  31. 31. Step 2a <br />Planning the Intervention (Starting With the End In Mind)<br />
  32. 32. Step 2a: Planning the Intervention (With ROI in Mind)<br />When we plan an intervention – considering that we want to measure ROI- we really reverse engineer (or start with the end in mind) – We use the Kirkpatrick Model to achieve this<br />
  33. 33. Step 2a: Planning the Intervention (With ROI in Mind)<br />Back to our case study to show how we plan “backwards” – remember these objectives and measures for clarification, based on our core example/case study (re graduate engineers)<br />
  34. 34. Step 2a: Planning the Intervention (With ROI in Mind)<br />Back to our case study to show how we plan “backwards” – remember these objectives and measures a merely examples<br />Remember this is a snapshot – there is a whole learning map and course development process underlying these objectives<br />
  35. 35. Step 2a: Planning the Intervention (With ROI in Mind)<br />Back to our case study to show how we plan “backwards” – remember these objectives and measures a merely examples<br />
  36. 36. Step 2a: Planning the Intervention (With ROI in Mind)<br />What have we achieved by planning the intervention this way?<br /> ROI ?<br />Results<br />A path – trail linking the business need with the intervention<br />Results<br />Results<br />Results<br />
  37. 37. Step 2a: Planning the Intervention (With ROI in Mind)<br />We can also go some steps further and plan where we will get the information to evaluate the intervention and how…<br />
  38. 38. Step 2b <br />Planning the Intervention (Planning Beyond Training – With ROI In Mind)<br />
  39. 39. Step 2b: Planning the Intervention (With ROI in Mind)<br />Usually once we have done the planning and developed the training intervention we are ready to implement…<br />
  40. 40. We need to determine what technique(s) we will use to show that the intervention actually had an impact – but our dilemma is this…<br />Step 2b: Planning the Intervention (With ROI in Mind)<br />External Factors<br />Management<br />Attention<br />OR<br /> SOMETHING <br />ELSE?<br />Incentives<br />AND YES THERE IS<br />IMPROVEMENT<br />AFTER<br />PROGRAM<br />Systems/Procedures<br />Changes<br />Here is our training<br />program<br />IS IT THE<br />TRAINING <br />PROGRAM?<br />
  41. 41. Step 2b: Planning the Intervention (With ROI in Mind)<br />We have to isolate the effects of our intervention!<br />Isolating the results/effect of our intervention is the most difficult and also most important element of building the case for value (ROI);<br />Because of this, value at the higher levels is seldom determined;<br />However, the credibility not only of your study to show value, but also the training function is based on this element.<br />
  42. 42. Step 2b: Planning the Intervention (With ROI in Mind)<br />We have to isolate the effects of our intervention!<br />The complexity of isolating the effect, is a function of the variables that are in play at any moment in time in an organization;<br />The analyst should however always try to prove a causal relationship to an acceptable degree of accuracy;<br />Acknowledging the forces that influence individual, group and business performance is the first step in this process.<br />
  43. 43. Step 2b: Planning the Intervention (With ROI in Mind)<br />We have to isolate the effects of our intervention!<br />Remember that we are trying to isolate the effect of out intervention to an acceptable degree of accuracy – this makes very important to involve and agree with the client:<br />What metrics should change as result of the intervention (see step 1)<br />What isolation technique will be used (because they differ in terms of accuracy)<br />Let’s consider the isolation techniques…<br />
  44. 44. Step 2b: Planning the Intervention (With ROI in Mind)<br />Let's look at the techniques!<br />Trend line analysis of performance data;<br />Control groups;<br />Participant’s estimation of impact;<br />Supervisor’s estimation of impact;<br />Management’s estimation of impact;<br />Use of previous studies<br />Subordinate’s report of other factors;<br />Estimating the impact of other factors;<br />Use of customer input.<br />With these techniques we try to isolate the effects of our intervention<br />
  45. 45. Step 2b: Planning the Intervention (With ROI in Mind)<br />Let's look at the techniques!<br />Trend line analysis of performance data;<br />Control groups;<br />Participant’s estimation of impact;<br />Supervisor’s estimation of impact;<br />Management’s estimation of impact;<br />Use of previous studies<br />Subordinate’s report of other factors;<br />Estimating the impact of other factors;<br />Use of customer input.<br />With these techniques we try to isolate the effects of other interventions and then declare the rest as due to our effort<br />
  46. 46. Step 2b: Planning the Intervention (With ROI in Mind)<br />Trend Line Analysis<br />Draw a trend-line using previous performance data as a base;<br />When intervention is conducted, actual performance is plotted and compared to the trend line;<br />Improvement in actual performance over what the trend line predicted can then be reasonably attributed to the intervention.<br />
  47. 47. Advantages<br />Disadvantages<br />Show quick results<br />Easy to calculate<br />Easy to understand<br />Inexpensive<br />Not difficult to organize (basically just monitor changes in metrics)<br />A good first order analysis<br />Uses only one factor to isolate the effect of the intervention (time- before and after implementation)<br />Thus less accurate and client could be sceptical to accept only an analysis of the trend line.<br />Trend Line Analysis- Advantages and Disadvantages<br />Step 2b: Planning the Intervention (With ROI in Mind)<br />
  48. 48. Example- Not from our central case study…<br />Step 2b: Planning the Intervention (With ROI in Mind)<br />Effect of the intervention<br />Average<br />Pre Program 55<br />COMPLAINTS<br />Average<br />Post Program 35<br />Sexual<br />Harassment<br />Prevention Program<br />O N D J F M A M J J A S O N D J F M A M J J A S O<br />TIME<br />
  49. 49. Step 2b: Planning the Intervention (With ROI in Mind)<br />Control Groups Technique<br />Similar to classical research the control group method, involves an experimental group and a control group;<br />The so called experimental group will be the target of our intervention;<br />The control group would be a group that is paired with the experimental group, but would not undergo the intervention;<br />Important things to remember with this method are:<br />Paring need to be thought about carefully so as not to dilute the impact of the intervention;<br />Although not as rigid as classical research key factors that could dilute the impact or assessment of the impact of the intervention need to be controlled.<br />
  50. 50. Advantages<br />Disadvantages<br />More accurate than most techniques<br />If the pairing and control of variables are agreed with the client the outcome has high level of credibility<br />With this control groups, given the level of accuracy it is possible to benchmark the ROI of certain interventions given a predetermined set of variables<br />More complicated<br />Require more planning, negotiation, control and monitoring<br />As result of above many practitioners are unwilling to use this technique<br />Requires absolute agreement with the client environment that variables will be kept constant for the duration of the ROI evaluation.<br />Control Groups- Advantages and Disadvantages<br />Step 2b: Planning the Intervention (With ROI in Mind)<br />
  51. 51. Step 2b: Planning the Intervention (With ROI in Mind)<br />Schematic Explanation of control groups design<br />1. Pairing<br />2. Control Variables<br />Control <br />Group<br />M2<br />M1<br />Experimental<br />Group<br />Intervention<br />M1<br />M2<br />*M = measurement<br />
  52. 52. Looking at pairing and control of variables<br />Step 2b: Planning the Intervention (With ROI in Mind)<br />PAIRING<br />Note:<br />Too small then meaningful pairing is not possible – too big then you are not going to be able to control the variables during and after the implementation of your intervention;<br />Remember we go for reasonable accuracy therefore you pair the major elements and also control the major variables!!<br />The above makes it very important to agree upfront with your client environment what how pairing will be done and which variables will be controlled.<br />Select your <br />experimental and control group <br />carefully<br />VARIABLES<br />
  53. 53. Step 2b: Planning the Intervention (With ROI in Mind)<br />So what do you typically pair?<br />Size<br />Nature of work<br />Leadership stype<br />Performance level<br />General demography<br />Pair for (Primarily)<br />Negotiate creafully<br />
  54. 54. Step 2b: Planning the Intervention (With ROI in Mind)<br />So what do you typically try to control?<br />Size<br />Nature of work<br />Leadership stype<br />Performance level<br />General demography<br />structure<br />Function <br />Recognition and reward<br />Turn over<br />Systems and procedures<br />Changes in:<br />Changes in:<br />+<br />Negotiate creafully<br />Negotiate creafully<br />
  55. 55. Step 2b: Planning the Intervention (With ROI in Mind)<br />Remember your decision about pairing and control of variables….<br />= (f) (your knowledge and judgement) + agreement with the client environment + practicability<br />
  56. 56. Step 2b: Planning the Intervention (With ROI in Mind)<br />Estimation of impact by third parties ( this technique can include an estimation of the perceived impact of the intervention, by participants, participants supervisors, participants managers)<br />This technique provides participants with a questionnaire, and based on the improvement ask them to respond to the following:<br />What percent of this improvement can be attributed to the application of skills/techniques/knowledge gained in the training program?<br />What is the basis for this estimation?<br />What confidence do you have in this estimate, expressed as a percentage?<br />What other factors contributed to this improvement in performance?<br />What other individuals or groups could estimate this percentage or determine the amount?<br />
  57. 57. Step 2b: Planning the Intervention (With ROI in Mind)<br />Estimation of impact by third parties – how do you increase the confidence level for this technique? <br />Individuals who do not respond to the questionnaire or provide non- usable data on the questionnaire are assumed to report no improvement/impact; <br />Extreme data and unrealistic claims are omitted from the analysis;<br />Only annualized values are used, it is assumed that there are no benefits from the program after the first year of implementation; <br />The confidence level, expressed as a percent, is multiplied by the improvement value to reduce the amount of the improvement by the potential error;<br />If value is expressed it is also factored by the confidence value.<br />
  58. 58. Advantages<br />Disadvantages<br />Easy to implement;<br />Based on the assumption that participants are capable of determining or estimating impact;<br />Although an estimate, this value will usually have considerable credibility with management because participants are at the centre of the change or improvement;<br />High face validity.<br />Less accurate than control groups<br />Based on perceptions – could be regarded as Return on Expectation rather than ROI<br />Could become victim of “group think”<br />Estimation Of Impact By Third Parties - Advantages And Disadvantages<br />Step 2b: Planning the Intervention (With ROI in Mind)<br />
  59. 59. Example from an actual coaching intervention- Not from our central case study…<br />Step 2b: Planning the Intervention (With ROI in Mind)<br />Participants and management agreed which the respective groups will rate<br />All indicators were pre agreed with the client<br />
  60. 60. So now we can complete our plan<br />Step 2b: Planning the Intervention (With ROI in Mind)<br />
  61. 61. Note – because the techniques related to isolating the impact of other factors are not used often it is only referred to in this toolkit…<br />Step 2b: Planning the Intervention (With ROI in Mind)<br />Trend line analysis of performance data;<br />Control groups;<br />Participant’s estimation of impact;<br />Supervisor’s estimation of impact;<br />Management’s estimation of impact;<br />Use of previous studies<br />Subordinate’s report of other factors;<br />Estimating the impact of other factors;<br />Use of customer input.<br />With these techniques we try to isolate the effects of other interventions and then declare the rest as due to our effort<br />
  62. 62. Step 3<br />Implementation and Data Gathering (With ROI in Mind)<br />
  63. 63. Implementation and Data Gathering (With ROI in Mind)<br />Referring back to our original example (turnover in graduate engineers) – let us assume the following:<br />We opted for a control group evaluation method<br />
  64. 64. Implementation and Data Gathering (With ROI in Mind)<br />Referring back to our original example (turnover in graduate engineers) – let us assume the following:<br />Six months after the intervention the need for replacement of graduates were down by 50% in the experimental <br />group<br />
  65. 65. Implementation and Data Gathering (With ROI in Mind)<br />Referring back to our original example (turnover in graduate engineers) – let us assume the following:<br />In the control group the need for replacement of graduate engineers also down by 10%...possibly because of anticipation of changes in HR approach<br />
  66. 66. Implementation and Data Gathering (With ROI in Mind)<br />Referring back to our original example (turnover in graduate engineers) – let us assume the following:<br />In the control group the need for replacement of graduate engineers also down by 10%...possibly because of anticipation of changes in HR approach<br />
  67. 67. So because to the method we used we can assign the change in te metric to our intervention, mainly because we controlled major variables. We do this as follows…<br />
  68. 68. Step 4<br />Converting the Change As Result of the Intervention to Value and Calculating ROI<br />
  69. 69. Step 4.a Converting the Change As Result of the Intervention to Value<br />Referring back to our original example (turnover in graduate engineers)…this is how it is dome…<br />
  70. 70. Because this is such a vast area to know here is some additional input re calculating the value of change in a metric..<br />Step 4.a Converting the Change As Result of the Intervention to Value<br />Output<br />Increases<br />Time<br />Savings<br />Primary Measurements<br />of Improvement<br />Cost<br />Savings<br />Quality<br />Improvement<br />
  71. 71. Step 4.a Converting the Change As Result of the Intervention to Value<br />Always try to link to a metric – alternatively use the method of Estimation Of Impact By Third Parties – See slide 55 and 56 (this method is also referred to as Return on Expectation) <br />
  72. 72. Step 4.a Converting the Change As Result of the Intervention to Value<br />See whether one can link tangible values to these?<br />
  73. 73. Step 4.a Converting the Change As Result of the Intervention to Value<br />And these….<br />
  74. 74. Or determining value on a case by case basis using historical data<br />Actual cost <br />from records<br />Legal fee;<br />Settlement<br /> cost<br />Cost <br />R 285,000<br />Total of <br />35 complaints<br />Estimated<br /> additional <br />from staff<br />Staff time;<br />Management <br />time<br />Step 4.a Converting the Change As Result of the Intervention to Value<br />R 285,000/35 = Cost per complaint<br />Cost of Sexual Harassment Complaint<br />
  75. 75. Step 4.a Converting the Change As Result of the Intervention to Value<br />In our case example we did determine the value of the change and can therefore now move on to calculate cost benefit and therefore also ROI<br />
  76. 76. Step 4.b Calculating ROI<br />First thing to do now is to determine the cost of the intervention – again back to our main example …<br />
  77. 77. Step 4.b Calculating ROI<br />First thing to do now is to determine the cost of the intervention – again back to our main example …<br />Net Program Benefits (R224892- R150,000)<br />Program Costs (R150,000)<br />Benefit/Cost Ratio =<br />Ratio = 0.499928<br />
  78. 78. Step 4.b Calculating ROI<br />Net Program Benefits (R224892- R150,000) <br />Program Costs (R150,000)<br />First thing to do now is to determine the cost of the intervention – again back to our main example …<br />X 100<br />1<br />Benefits ROI =<br />Benefit ROI = 50% thus for every R1 I spent on this program you got your money back through saving as well as an additional 50 cents.<br />

×