Your SlideShare is downloading. ×
ROI For HR Practice Toolkit
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

ROI For HR Practice Toolkit

1,006
views

Published on

Published in: Technology, Education

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,006
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
55
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. ROI of Human Capital Interventions
    A Practice Toolkit
  • 2.
  • 3. Step 1
    Presenting Problem to Real Problem
  • 4. Step 1. Presenting Problem to Real Problem
    The first task of the practitioner is almost always analysis
    Starts with a “presenting” problem – not a “hunch” but a problem … as it presents itself
  • 5. Step 1. Presenting Problem to Real Problem
    Example…
    Let's take an example – higher staff turnover for Graduate Engineers (let say compared to an industry standard)
    The first question should not be, what skills and knowledge are required. It should be which metric(s) are affected negatively.
  • 6. Step 1. Presenting Problem to Real Problem
    Example…
  • 7. Step 1. Presenting Problem to Real Problem
    1.1 Calculating benchmark employee cost
  • 8. Step 1. Presenting Problem to Real Problem
    1.2 Cost of 'covering' a vacant position (Calculated costs of other employees 'filling in' while the position is vacant)
  • 9. Step 1. Presenting Problem to Real Problem
    1.3 Cost to fill a vacant position
  • 10. Step 1. Presenting Problem to Real Problem
    1.4 On boarding & Orientation cost
  • 11. Step 1. Presenting Problem to Real Problem
    1.5 Cost of productivity ramp-up (During the first 3 months, an average new employee performs at 50% productivity of a tenured top performing employ
  • 12. Step 1. Presenting Problem to Real Problem
    1.6 Unit cost for filling a vacancy
    R 74964
    Two important learning points:
    Know your metrics
    Negotiate, and even request help from your client to determine the right metrics. They are closer to the business and at the same time you obtain by in for the intervention.
  • 13. Step 1. Presenting Problem to Real Problem
    1.7 Total cost of replacing graduate engineers
    R 74964 x 6 = R 449784
    Let's assume that we had to replace five graduate engineers the past six months
  • 14. Step 1. Presenting Problem to Real Problem
    Now we can move on to analysis
    We know now which metrics are affected
    Consultant speciality
  • 15. Step 1. Presenting Problem to Real Problem
    The basic tasks in analysis
  • 16. Step 1. Presenting Problem to Real Problem
    Two types of data
    Quantitative
    ”The truth is in a number” – however does not say much about context
    Data
    ” More about context, how people experience in their own words”.
    Qualitative
  • 17. Data gathering
    Before
    intervention
    Before
    intervention
    Step 1. Presenting Problem to Real Problem
    Questionnaires
    Surveys
    Tests
    Interviews
    Focus groups
    Observation
    Performance records
    Knowledge and skills testing
    Program follow up
    Project assignments
    After
    intervention
    After
    intervention
  • 18. Step 1. Presenting Problem to Real Problem
    Data gathering
    Before
    intervention
    Target population/participants;
    Supervisors of target population/participants;
    Subordinates of target population/participants;
    Peer Group;
    HRD Staff;
    Documentation:
    • Organizational Performance Records:
    • 19. Disciplinary records;
    • 20. Safety records;
    • 21. Scorecards
    After
    intervention
  • 22. Step 1. Presenting Problem to Real Problem
    Contextual factors to consider when selecting data collection methods
    Before
    intervention
    After
    intervention
    Time for participants to respond;
    Costs;
    Amount of disruption of normal activities;
    What is the level of certainty that you will achieve with the result;
    Practicality;
    Culture/Philosophy – example the 10% syndrome.
  • 23. Step 1. Presenting Problem to Real Problem
    Requirements for effective data collection instruments and techniques
    Before
    intervention
    Construct validity ( methods, techniques and test instruments should be evaluated in terms of its theoretical grounding);
    Predictive validity (methods, techniques and test instruments should be evaluated regularly in terms of its ability to predict what it purport to predict);
    Face validity – for line management very important!
    After
    intervention
  • 24. Step 1. Presenting Problem to Real Problem
    Requirements for effective data collection instruments and techniques
    Before
    intervention
    After
    intervention
    Reliability:
    Produce consistent results over time, that differentiates effectively, and are independent of the assessor/data collector, or context.
  • 25. Step 1. Presenting Problem to Real Problem
    Data consolidation
  • 26. Step 1. Presenting Problem to Real Problem
    Data analysis
  • 27. Note the following !
    Because this toolkit does not aim to transfer knowledge about analysis as such the previous slides provides a brief and high level overview, for the sake of a clearity… for the sake of the toolkit we assume that the analysis provided clarity about the problem….
  • 28. Step 1. Presenting Problem to Real Problem
    Brief Exposition of Problem
    Our analysis shows that the HR and line managers involved in selection decisions do interviews in a haphazard and nonscientific fashion
    Solution
    Following discussions with consultants, HR experts and line all agreed we need to implement competency based interviewing
    Lets assume this based on the core example
  • 29. Step 1. Presenting Problem to Real Problem
    The training department is instructed to develop an appropriate program and put all line and support staff through training – they however indicate that a change in results is required quickly and that ROI for the training need to be justified soon after implementation…
    Lets assume this based on the core example
  • 30. Presenting Problem to Real Problem
    So we have addressed this even before we asked any questions about training – and now we can move on to the next step
    Starts with a “presenting” problem – not a “hunch” but a problem … as it presents itself
  • 31. Step 2a
    Planning the Intervention (Starting With the End In Mind)
  • 32. Step 2a: Planning the Intervention (With ROI in Mind)
    When we plan an intervention – considering that we want to measure ROI- we really reverse engineer (or start with the end in mind) – We use the Kirkpatrick Model to achieve this
  • 33. Step 2a: Planning the Intervention (With ROI in Mind)
    Back to our case study to show how we plan “backwards” – remember these objectives and measures for clarification, based on our core example/case study (re graduate engineers)
  • 34. Step 2a: Planning the Intervention (With ROI in Mind)
    Back to our case study to show how we plan “backwards” – remember these objectives and measures a merely examples
    Remember this is a snapshot – there is a whole learning map and course development process underlying these objectives
  • 35. Step 2a: Planning the Intervention (With ROI in Mind)
    Back to our case study to show how we plan “backwards” – remember these objectives and measures a merely examples
  • 36. Step 2a: Planning the Intervention (With ROI in Mind)
    What have we achieved by planning the intervention this way?
    ROI ?
    Results
    A path – trail linking the business need with the intervention
    Results
    Results
    Results
  • 37. Step 2a: Planning the Intervention (With ROI in Mind)
    We can also go some steps further and plan where we will get the information to evaluate the intervention and how…
  • 38. Step 2b
    Planning the Intervention (Planning Beyond Training – With ROI In Mind)
  • 39. Step 2b: Planning the Intervention (With ROI in Mind)
    Usually once we have done the planning and developed the training intervention we are ready to implement…
  • 40. We need to determine what technique(s) we will use to show that the intervention actually had an impact – but our dilemma is this…
    Step 2b: Planning the Intervention (With ROI in Mind)
    External Factors
    Management
    Attention
    OR
    SOMETHING
    ELSE?
    Incentives
    AND YES THERE IS
    IMPROVEMENT
    AFTER
    PROGRAM
    Systems/Procedures
    Changes
    Here is our training
    program
    IS IT THE
    TRAINING
    PROGRAM?
  • 41. Step 2b: Planning the Intervention (With ROI in Mind)
    We have to isolate the effects of our intervention!
    Isolating the results/effect of our intervention is the most difficult and also most important element of building the case for value (ROI);
    Because of this, value at the higher levels is seldom determined;
    However, the credibility not only of your study to show value, but also the training function is based on this element.
  • 42. Step 2b: Planning the Intervention (With ROI in Mind)
    We have to isolate the effects of our intervention!
    The complexity of isolating the effect, is a function of the variables that are in play at any moment in time in an organization;
    The analyst should however always try to prove a causal relationship to an acceptable degree of accuracy;
    Acknowledging the forces that influence individual, group and business performance is the first step in this process.
  • 43. Step 2b: Planning the Intervention (With ROI in Mind)
    We have to isolate the effects of our intervention!
    Remember that we are trying to isolate the effect of out intervention to an acceptable degree of accuracy – this makes very important to involve and agree with the client:
    What metrics should change as result of the intervention (see step 1)
    What isolation technique will be used (because they differ in terms of accuracy)
    Let’s consider the isolation techniques…
  • 44. Step 2b: Planning the Intervention (With ROI in Mind)
    Let's look at the techniques!
    Trend line analysis of performance data;
    Control groups;
    Participant’s estimation of impact;
    Supervisor’s estimation of impact;
    Management’s estimation of impact;
    Use of previous studies
    Subordinate’s report of other factors;
    Estimating the impact of other factors;
    Use of customer input.
    With these techniques we try to isolate the effects of our intervention
  • 45. Step 2b: Planning the Intervention (With ROI in Mind)
    Let's look at the techniques!
    Trend line analysis of performance data;
    Control groups;
    Participant’s estimation of impact;
    Supervisor’s estimation of impact;
    Management’s estimation of impact;
    Use of previous studies
    Subordinate’s report of other factors;
    Estimating the impact of other factors;
    Use of customer input.
    With these techniques we try to isolate the effects of other interventions and then declare the rest as due to our effort
  • 46. Step 2b: Planning the Intervention (With ROI in Mind)
    Trend Line Analysis
    Draw a trend-line using previous performance data as a base;
    When intervention is conducted, actual performance is plotted and compared to the trend line;
    Improvement in actual performance over what the trend line predicted can then be reasonably attributed to the intervention.
  • 47. Advantages
    Disadvantages
    Show quick results
    Easy to calculate
    Easy to understand
    Inexpensive
    Not difficult to organize (basically just monitor changes in metrics)
    A good first order analysis
    Uses only one factor to isolate the effect of the intervention (time- before and after implementation)
    Thus less accurate and client could be sceptical to accept only an analysis of the trend line.
    Trend Line Analysis- Advantages and Disadvantages
    Step 2b: Planning the Intervention (With ROI in Mind)
  • 48. Example- Not from our central case study…
    Step 2b: Planning the Intervention (With ROI in Mind)
    Effect of the intervention
    Average
    Pre Program 55
    COMPLAINTS
    Average
    Post Program 35
    Sexual
    Harassment
    Prevention Program
    O N D J F M A M J J A S O N D J F M A M J J A S O
    TIME
  • 49. Step 2b: Planning the Intervention (With ROI in Mind)
    Control Groups Technique
    Similar to classical research the control group method, involves an experimental group and a control group;
    The so called experimental group will be the target of our intervention;
    The control group would be a group that is paired with the experimental group, but would not undergo the intervention;
    Important things to remember with this method are:
    Paring need to be thought about carefully so as not to dilute the impact of the intervention;
    Although not as rigid as classical research key factors that could dilute the impact or assessment of the impact of the intervention need to be controlled.
  • 50. Advantages
    Disadvantages
    More accurate than most techniques
    If the pairing and control of variables are agreed with the client the outcome has high level of credibility
    With this control groups, given the level of accuracy it is possible to benchmark the ROI of certain interventions given a predetermined set of variables
    More complicated
    Require more planning, negotiation, control and monitoring
    As result of above many practitioners are unwilling to use this technique
    Requires absolute agreement with the client environment that variables will be kept constant for the duration of the ROI evaluation.
    Control Groups- Advantages and Disadvantages
    Step 2b: Planning the Intervention (With ROI in Mind)
  • 51. Step 2b: Planning the Intervention (With ROI in Mind)
    Schematic Explanation of control groups design
    1. Pairing
    2. Control Variables
    Control
    Group
    M2
    M1
    Experimental
    Group
    Intervention
    M1
    M2
    *M = measurement
  • 52. Looking at pairing and control of variables
    Step 2b: Planning the Intervention (With ROI in Mind)
    PAIRING
    Note:
    Too small then meaningful pairing is not possible – too big then you are not going to be able to control the variables during and after the implementation of your intervention;
    Remember we go for reasonable accuracy therefore you pair the major elements and also control the major variables!!
    The above makes it very important to agree upfront with your client environment what how pairing will be done and which variables will be controlled.
    Select your
    experimental and control group
    carefully
    VARIABLES
  • 53. Step 2b: Planning the Intervention (With ROI in Mind)
    So what do you typically pair?
    Size
    Nature of work
    Leadership stype
    Performance level
    General demography
    Pair for (Primarily)
    Negotiate creafully
  • 54. Step 2b: Planning the Intervention (With ROI in Mind)
    So what do you typically try to control?
    Size
    Nature of work
    Leadership stype
    Performance level
    General demography
    structure
    Function
    Recognition and reward
    Turn over
    Systems and procedures
    Changes in:
    Changes in:
    +
    Negotiate creafully
    Negotiate creafully
  • 55. Step 2b: Planning the Intervention (With ROI in Mind)
    Remember your decision about pairing and control of variables….
    = (f) (your knowledge and judgement) + agreement with the client environment + practicability
  • 56. Step 2b: Planning the Intervention (With ROI in Mind)
    Estimation of impact by third parties ( this technique can include an estimation of the perceived impact of the intervention, by participants, participants supervisors, participants managers)
    This technique provides participants with a questionnaire, and based on the improvement ask them to respond to the following:
    What percent of this improvement can be attributed to the application of skills/techniques/knowledge gained in the training program?
    What is the basis for this estimation?
    What confidence do you have in this estimate, expressed as a percentage?
    What other factors contributed to this improvement in performance?
    What other individuals or groups could estimate this percentage or determine the amount?
  • 57. Step 2b: Planning the Intervention (With ROI in Mind)
    Estimation of impact by third parties – how do you increase the confidence level for this technique?
    Individuals who do not respond to the questionnaire or provide non- usable data on the questionnaire are assumed to report no improvement/impact;
    Extreme data and unrealistic claims are omitted from the analysis;
    Only annualized values are used, it is assumed that there are no benefits from the program after the first year of implementation;
    The confidence level, expressed as a percent, is multiplied by the improvement value to reduce the amount of the improvement by the potential error;
    If value is expressed it is also factored by the confidence value.
  • 58. Advantages
    Disadvantages
    Easy to implement;
    Based on the assumption that participants are capable of determining or estimating impact;
    Although an estimate, this value will usually have considerable credibility with management because participants are at the centre of the change or improvement;
    High face validity.
    Less accurate than control groups
    Based on perceptions – could be regarded as Return on Expectation rather than ROI
    Could become victim of “group think”
    Estimation Of Impact By Third Parties - Advantages And Disadvantages
    Step 2b: Planning the Intervention (With ROI in Mind)
  • 59. Example from an actual coaching intervention- Not from our central case study…
    Step 2b: Planning the Intervention (With ROI in Mind)
    Participants and management agreed which the respective groups will rate
    All indicators were pre agreed with the client
  • 60. So now we can complete our plan
    Step 2b: Planning the Intervention (With ROI in Mind)
  • 61. Note – because the techniques related to isolating the impact of other factors are not used often it is only referred to in this toolkit…
    Step 2b: Planning the Intervention (With ROI in Mind)
    Trend line analysis of performance data;
    Control groups;
    Participant’s estimation of impact;
    Supervisor’s estimation of impact;
    Management’s estimation of impact;
    Use of previous studies
    Subordinate’s report of other factors;
    Estimating the impact of other factors;
    Use of customer input.
    With these techniques we try to isolate the effects of other interventions and then declare the rest as due to our effort
  • 62. Step 3
    Implementation and Data Gathering (With ROI in Mind)
  • 63. Implementation and Data Gathering (With ROI in Mind)
    Referring back to our original example (turnover in graduate engineers) – let us assume the following:
    We opted for a control group evaluation method
  • 64. Implementation and Data Gathering (With ROI in Mind)
    Referring back to our original example (turnover in graduate engineers) – let us assume the following:
    Six months after the intervention the need for replacement of graduates were down by 50% in the experimental
    group
  • 65. Implementation and Data Gathering (With ROI in Mind)
    Referring back to our original example (turnover in graduate engineers) – let us assume the following:
    In the control group the need for replacement of graduate engineers also down by 10%...possibly because of anticipation of changes in HR approach
  • 66. Implementation and Data Gathering (With ROI in Mind)
    Referring back to our original example (turnover in graduate engineers) – let us assume the following:
    In the control group the need for replacement of graduate engineers also down by 10%...possibly because of anticipation of changes in HR approach
  • 67. So because to the method we used we can assign the change in te metric to our intervention, mainly because we controlled major variables. We do this as follows…
  • 68. Step 4
    Converting the Change As Result of the Intervention to Value and Calculating ROI
  • 69. Step 4.a Converting the Change As Result of the Intervention to Value
    Referring back to our original example (turnover in graduate engineers)…this is how it is dome…
  • 70. Because this is such a vast area to know here is some additional input re calculating the value of change in a metric..
    Step 4.a Converting the Change As Result of the Intervention to Value
    Output
    Increases
    Time
    Savings
    Primary Measurements
    of Improvement
    Cost
    Savings
    Quality
    Improvement
  • 71. Step 4.a Converting the Change As Result of the Intervention to Value
    Always try to link to a metric – alternatively use the method of Estimation Of Impact By Third Parties – See slide 55 and 56 (this method is also referred to as Return on Expectation)
  • 72. Step 4.a Converting the Change As Result of the Intervention to Value
    See whether one can link tangible values to these?
  • 73. Step 4.a Converting the Change As Result of the Intervention to Value
    And these….
  • 74. Or determining value on a case by case basis using historical data
    Actual cost
    from records
    Legal fee;
    Settlement
    cost
    Cost
    R 285,000
    Total of
    35 complaints
    Estimated
    additional
    from staff
    Staff time;
    Management
    time
    Step 4.a Converting the Change As Result of the Intervention to Value
    R 285,000/35 = Cost per complaint
    Cost of Sexual Harassment Complaint
  • 75. Step 4.a Converting the Change As Result of the Intervention to Value
    In our case example we did determine the value of the change and can therefore now move on to calculate cost benefit and therefore also ROI
  • 76. Step 4.b Calculating ROI
    First thing to do now is to determine the cost of the intervention – again back to our main example …
  • 77. Step 4.b Calculating ROI
    First thing to do now is to determine the cost of the intervention – again back to our main example …
    Net Program Benefits (R224892- R150,000)
    Program Costs (R150,000)
    Benefit/Cost Ratio =
    Ratio = 0.499928
  • 78. Step 4.b Calculating ROI
    Net Program Benefits (R224892- R150,000)
    Program Costs (R150,000)
    First thing to do now is to determine the cost of the intervention – again back to our main example …
    X 100
    1
    Benefits ROI =
    Benefit ROI = 50% thus for every R1 I spent on this program you got your money back through saving as well as an additional 50 cents.