Measuring ROI in Training

in L&D/ HRD
welcome to the 2 day intense and practical workshop on training evaluation

at the end of
the program
within a
year

visual representations of
achievements, learning, skills, competencies
=

No business need for the program Program linked to specific business
needs
No effort to prepare the work
environment to support program
Environment prepared to support
program
ResultsActivity

No business need for the program Program linked to specific business
needs
No effort to prepare the work
environment to support program
Environment prepared to support
program
Reporting on program is input
focused
Reporting on program is output
focused

Reaction Objectives
1) Find it important to your success on the job
2) Find the case studies relevant and useful
3) Find the learning process fun

Learning Objectives
1) Identify the different evaluation levels
2) Align programs to business needs
3) Identify at least 3 ways to collect data
4) Describe at least 2 ways to isolate the effects of the program
5) Calculate ROI given benefits & cost

Application Objectives
1) Develop an evaluation plan for a specific program

Impact Objectives
1) Complete a ROI study in your organization

Is training evaluation
an ‘art’ or ‘science’
?

both!
the data calculation is a
‘science’
Influencing people to give
you the data is an ‘art’

Kirkpatrick Model Phillips ROI Model
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1Developed
Inputs / IndicatorsLevel 0
Level 5

• Number of projects
• Number of participants
• Hours
• Request
• Attendance
• Costs (fully loaded)
• Efficiencies
Level 0 - Inputs and Indicators

ReactionLevel 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Level 5

Reaction , Satisfaction & Planned action

Reaction
Reaction, Satisfaction & Planned
Action
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Level 5
Learning Learning

Reaction
Action
Learning Learning
Behaviour Application
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Level 5

Week 1
Week 2 Week 3
Week 4 Week 5 Week 6 Week 7
Week 8
Pre
Assessment
Classroom & Online Training
ILT (3hours, 5 alternative days)
Online (10 hours, 10 days)
Post Training Support - Email Coach
(2 emails a week & online exercises) Post
Assessment
intermediate
Deutsche Bank training flow

Reaction
Action
Learning Learning
Results Business Impact
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Level 5

Customer satisfaction
Sales
Units manufactured
Students graduated
increase

Costs
Quality complaints
Time to project completion
Ineffective programs
decrease

Reaction
Action
Learning Learning
Isolating the effects of the program
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Level 5

Reaction
Action
Learning Learning
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
ROI (ROE)Level 5

All programs are ‘not evaluated’ at all levels…

Criteria for L4 and L5
( refer job aid )

________________
________________
________________
________________
________________
________________
0. Input
1. Reaction
2. Learning
3. Application
4. Impact
5. ROI
________________
________________
________________
________________
________________
________________
________________
________________
________________
________________
________________
________________
Levels Current Target Suggested
100%
90 -100%
40 - 60%
30%
10 - 20%
5 - 10%
*2007 Survey of Users, N = 235

Level Current status Evaluation Targets
0 Less then 25% of the programs 100%
1 100% of the workshops conducted. 100%
2 Done for more than 60-70% 90%
3 Less than 10% of the training 50%
4 Not done for any training 30%
5 Not done for any training 15%

What is your
organizations criteria
?

Benefits/ Cost Ratio =
Program Benefits
Program Cost
ROI =
Program Benefits – Program cost
Program Costs
X 100

Program Benefits from Participants (1st year) = 12,30,000
Costs per Program (25 participants) = 6,88,000
12, 30, 000
6,88, 000
= 1.79
ROI =
12,30,000 – 6,88,000
6,88, 000
= 79%X 100

only if the previous level take place, the following can take place

L1
Reaction
L2
Learning
L3
Application
L4
Impact
L5
ROI
but just because the previous level took place, it does not mean that the
following will take place

Reaction
Action
Learning Learning
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
10 step by step of collecting, analyzing, reporting, using systematic &
methodic
Return on InvestmentLevel 5

Develop
Objectives of
Solution (s)
Develop
Evaluation
Plans and
Baseline Data
Collect
Data During
Solution
Implementation
Collect
Data After
Solution
Implementation
Level 2: Learning Level 4: Business
Impact
Level 1: Reaction,
Satisfaction, and
Planned Actions
Level 3:
Application/
Implementation
Evaluation Planning Data Collection

Isolate the
Effects
Convert Data
to Monetary
Value
Calculate the
Return on
Investment
Tabulate Costs
of Solution
Identify
Intangible
Measures
Generate
Impact
Study
Level 5:
ROI
Intangible Measures
Data Analysis Reporting

Reaction
Action
Learning Learning
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
methodic
12 guiding principles to be conservative in approach

Reaction
Action
Learning Learning
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
methodic
12 guiding principles to be conservative in approach
5 books 40 books, 3000 case studies

Companies invest in
learning measurement
outperformed others by
17%
!
Source: McBassi & Company, December 2009

Develop
Objectives of
Solution(s)
Develop
Evaluation
Plans and
Baseline Data
Evaluation Planning

One of the biggest problem
is the strong program
alignment to the business
goals
!
Source: Bersin, The Training Measurement Book

primary (activity)
heuristic
coherent (results)
Needs Analysis Maturity

Business Alignment Model
Learning Needs
Needs
Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Learning Objectives
Application ObjectivesPerformance Needs
Impact ObjectivesBusiness Needs
Payoff Needs ROI Objectives 5
4
3
2
5
4
3
2
1 1Preference Needs
Program Evaluation

Learning Needs
Needs
Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Learning Objectives
4
3
2
5
4
3
2
1 1Preference Needs
Program Evaluation
start here end here

program
sponsor
business managers
participants

Learning Needs
Needs
Reaction
Learning
Application
Impact
ROI
Performance Needs
Business Needs
Payoff Needs 5
4
3
2
5
4
3
2
1 1Preference Needs
Program Evaluation
start here end here
Reaction Objectives
Learning Objectives
Impact Objectives
ROI Objectives

Linking Needs, Objectives and Evaluation
Learning Needs
Needs
Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Learning Objectives
4
3
2
5
4
3
2
1 1Preference Needs
Program Evaluation

Aligning needs to business
Learning Needs
Preference Needs
Performance Needs
Business Needs
Payoff Needs Is the problem worth solving?5
4
3
2
1
Needs

Pay off need is the financial impact of
the problem

Learning Needs
Preference Needs
Performance Needs
What business measures affect the problem? (lag measure)Business Needs
4
3
2
1
Needs

Business Impact Metrics
• Decrease in time to billability
• Decrease in manager/ lead time
spent
• Decrease in time to competency

Learning Needs
Preference Needs
What should happen on the job to impact the business measure? (lead)Performance Needs
4
3
Needs

audience behaviour condition degree

audience behaviour condition degree
Supervisors should counsel their team members when there is an
unplanned absenteeism at least 9 out of 10 times

If you do not have the
confidence of line managers,
no amount of analysis will help
you case.
Source: Mckinsey Oct, 2010

Learning Needs
Preference Needs
What skills/ knowledge must be developed to meet the performance need?
What should change in the work environment to impact the business measure? (lead)Performance Needs
4
3
2
Needs

Learning Needs
Preference Needs How should the solutions be structured for participants satisfaction?
What skills/ knowledge must be developed to meet the performance need?
4
3
2
1
Needs

instructor led game based experiential
simulation webinars others

Learning Needs
Preference Needs How should the solutions be structured for participants satisfaction?
What skills/ knowledge must be developed to meet the work performance need?
4
3
2
1
Needs

Aligning objectives to needs
Learning Objectives
Reaction Objectives How should the participants react?
What will the participants learn at the end of the program?
How should the audience behave under given condition to an acceptable degree (abcd)?Application Objectives
How much of the increase/ decrease should take place in your business measure?Impact Objectives
ROI Objectives What is the expected ROI?
Program

Aligning evaluation to objectives
Learning
Reaction How do I collect the participants reactions to the program?
How will I test if the participants have acquired the skills / knowledge?
How will I track if my participants are applying the learning on the job? (lead measure)Application
Where do I get the increase/ decrease/ cost avoidance data? (lag measure)Impact
ROI What is the program ROI?5
4
3
2
1
Evaluation

Learning Needs
Preference Needs
Performance Needs
Business Needs
Payoff Needs
5
4
3
2
1
Absenteeism is causing 15,00,000
monthly
Unplanned absenteeism is 9%
Discussion between team
members and supervisor are not
occurring
Counseling and discussion skills
Counseling skills must be practical and should
happen through ILT & Coaching

A well defined problem is half
solved

Develop
Objectives of
Solution (s)
Develop
Evaluation
Plans and
Baseline Data
Collect
Data During
Solution
Implementation
Collect
Data After
Solution
Implementation

Objectives are the success
measure for each of the needs

ROI Objectives
Learning Needs
Preference Needs
Performance Needs
Business Needs
Payoff Needs
5
4
3
2
1
occurring
Counseling skills must be practical and
should happen immediately
Reaction Objectives
Learning Objectives
Impact Objectives
monthly

Learning Objectives
Reaction Objectives How should the participants react?
Impact Objectives
ROI Objectives
Program

ROI Objectives
Learning Needs
Preference Needs
Performance Needs
Business Needs
Payoff Needs
5
4
3
2
1
occurring
4/5 on the practicality of the
program and relevance to job
Reaction Objectives
Learning Objectives
Impact Objectives
monthly

Learning Objectives
Reaction Objectives
What will the participants learn at the end of the program?
Impact Objectives
ROI Objectives
Program

ROI Objectives
Learning Needs
Preference Needs
Performance Needs
Business Needs
Payoff Needs
5
4
3
2
1
occurring
Participants will learn 3 key
counseling skills
Reaction Objectives
Learning Objectives
Impact Objectives
monthly

Learning Objectives
Reaction Objectives
How should the audience behave under given condition to an acceptable degree (abcd)?Application Objectives
Impact Objectives
ROI Objectives
Program

ROI Objectives
Learning Needs
Preference Needs
Performance Needs
Business Needs
Payoff Needs
5
4
3
2
1
occurring
Counseling skills must be
practical and should happen
immediately
counseling skills
Counseling discussion conducted in
95% situations when an unexpected
absence occurs
Reaction Objectives
Learning Objectives
Impact Objectives
monthly

Learning Objectives
Reaction Objectives
How much of the increase/ decrease should take place in your business measure?Impact Objectives
ROI Objectives
Program

ROI Objectives
Learning Needs
Preference Needs
Performance Needs
Business Needs
Payoff Needs
5
4
3
2
1
occurring
immediately
counseling skills
absence occurs
Reduce absenteeism to 5%
Reaction Objectives
Learning Objectives
Impact Objectives
monthly

Learning Objectives
Reaction Objectives
Impact Objectives
ROI Objectives What is the expected ROI?
Program

ROI Objectives
Learning Needs
Preference Needs
Performance Needs
Business Needs
Payoff Needs
5
4
3
2
1
occurring
immediately
counseling skills
absence occurs
Reduce absenteeism to 5%
ROI of 20%
Reaction Objectives
Learning Objectives
Impact Objectives
monthly

Business Measure - Hard Data
Time
Output
Costs
Quality
Page 3.05

Work habits
Climate/ Satisfaction
Initiative/ Innovative
Customer Service
Development
Image
Page 3.07
Business Measure - Soft Data

with the subsequently acquired data
which serves as a basis for
Initial collection of data
COMPARISON

Sales Skills - Tata Sky
Learning Needs
Preference Needs Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Learning Objectives
Payoff Needs ROI Objectives
End HereStart Here
5
4
3
2
5
4
3
2
1 1
Increase Sales
Increase EPRS
Increase walkin conversion
Profiling of the customer,
Standardization in customer
demo, & transaction
Greeting
Asking right questions to the prospects
to understand the wants and needs,
Match the needs with features &
benefits
ILT and role plays 4/5 on the practicality of the
program and relevance to
job
Participants will learn key
interaction & influencing skills
Use the skills consistently, which is 100% of the
times
+ve ROI
Track business dashboards
every month
Calculate ROI, BCR
Increase walk-in conversion ratio
Increase the subscription recharge by 10%
Increase Sales by 10%
Action plan activity sheet
Facilitator assessment through role
plays
Reaction and planned action through a
questionnaire at the end of the program

Learning Needs
Preference Needs
Performance Needs
Business Needs
Payoff Needs
5
4
3
2
1
Increase Sales
Increase EPRS
demo, & transaction
Greeting
understand the wants and needs
Influencing
ILT and role plays
4/5 on the practicality and
relevance
interaction & influencing
skills
Use 100% of the skills
consistently with every customer
interaction
+ve ROI
Increase Sales by
Increase the subscription recharge by

V Model – Tata Sky
Learning Needs
Preference Needs Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Learning Objectives
Payoff Needs ROI Objectives
End HereStart Here
5
4
3
2
5
4
3
2
1 1
Increase Sales
Increase EPRS
demo, & transaction
Greeting
Asking right questions to the prospects
to understand the wants and needs,
Match the needs with features &
benefits
ILT and role plays 4/5 on the practicality of the
program and relevance to
job
interaction & influencing skills
Use the skills consistently, which is 100% of the
times
+ve ROI
Track business dashboards
every month
Calculate ROI, BCR
Increase the subscription recharge by 10%
Increase Sales by 10%
Action plan activity sheet
Facilitator assessment through role
plays
Reaction and planned action through a
questionnaire at the end of the program

Each level includes:
• Data Sources
• Data Collection
• Key Questions
• Key Issues
Collect Data After
Solution
Implementation
Isolate
the Effects
of the Solution
Convert Data To
Monetary
Value
Calculate
The Return On
Investment
Tabulate
Costs of
Solution
Identify
Intangibles
Significant Influences
• Policy Statement
• Procedures and Guidelines
• Staff Skills
• Management Support
• Technical Support
• Organizational Culture
Phillips ROI Methodology TM
Develop
Report and
Communicate
Results
Level 4 Level 3 Level 3 and 2 Level 1
Level 3
Level 4
Level 5
Level 5
Results-Based Approach
Intangible
Benefits
Specify Skill/
Knowledge
Deficiencies of
Affected
Population
Training
Required
?
Level 2
Problem/
Opportunity
Present or
Anticipated
Identify Job
Performance
Needs, Gaps,
and Why
Identify
Solutions
Identify
Preferences
Develop
Objectives/
Evaluation
Strategy
Identify Transfer
Strategy Options
and L-2 & L-3
Support For All
Stakeholders
Identify
Business,
Needs,
Gaps, and
Stakeholder
s
Design
Solution and
Stakeholder
Components
Consider
Resources/
Logistics
Delivery
Develop
Content/
Materials
Conduct/
Implement
Solution
Level 1
Level 2
Implement
Pre-activity
Implement
Transfer
Strategy
- Solution
- Pre-work Activity/Transfer Strategy
Page 3.9

Develop
Objectives of
Solution (s)
Develop
Evaluation
Plans and
Baseline Data
Collect
Data During
Solution
Implementation
Collect
Data After
Solution
Implementation
Step 2

1. Data collection Plan
2. ROI analysis plan
Evaluation Planning ( 2 docs)

Data Items
(Usually
Level 4
Method for
Isolation
Methods of
Converting
Data to
Monetary
Values
Cost Categories Intangible
Benefits
Communication
Targets for Final
Report
Other
Influences/
Issues
During
Application
Monthly
conversion
ratio
Participant
estimate
(ROMs &
ROI)
Standard
Value.
(1)Facilitation Fees
(2)Program Materials
(3)Market survey Fees
(4)Facility costs
(5)Participants
salaries
ROMs
ROIs
Exec. V P
Retail
operations
Any
change in
market
factor
ROI Analysis Plan– Tata Sky

Develop
Objectives of
Solution (s)
Collect
Data During
Solution
Implementation
Collect
Data After
Solution
Implementation
Level 2: Learning
Level 1: Reaction,
Satisfaction, and
Planned Actions

• Change
• Your control
• Important to your organization

1. Relevance to work
2. Important to success
3. Would recommend to others
4. Amount of new information
5. Method of delivery
6. Learning Environment
7. Planned Action
Level 1 Target Areas

Number of Ratings
Rating Course 1 Course 2
5 (excellent)
4 (good)
3 (fair)
2 (poor)
1 (dismal)
Average Satisfaction 3.7 3.9

Number of Ratings
Rating Course 1 Course 2
5 (excellent) 5 20
4 (good) 18 4
3 (fair) 15 6
2 (poor) 0 5
1 (dismal) 0 3
Average Satisfaction 3.7 3.9

1. Skills
2. Knowledge
3. Capability
4. Awareness

1. Test
2. Simulation
3. Business games
4. In-basket
5. Case Studies
6. Role playing
7. Assessment center
8. Self, team, facilitator assessment
Techniques for Measuring
Learning

Develop
Objectives of
Solution (s)
Develop
Evaluation
Plans and
Baseline Data
Collect
Data During
Solution
Implementation
Collect
Data After
Solution
Implementation
Level 4:
Business Impact
Level 3:
Application/
Implementation

1. Skills - Demonstration of the
new skills on the job
2. Knowledge
3. Capacity – Demonstrate ability
on the job
4. Awareness – Demonstrate the
required behaviors on the job

Java
Development
Java
Maintenance
DOTNET
Development
DOTNET
Maintenanc
e
Testing
Development
Testing
Maintenance Average
Leads recognizing
percentage of the content
covered in Orchard
Learning to be directly
relevant to the projects
85% 73% 98% 65% 74.50% 82% 80%
Campus Minds applying
percentage of the relevant
content in their respective
projects and have
attributed this to Orchard
Learning
88% 97% 90% 82% 66% 51% 79%
Appendix 3 – Project relevance

• Manager does not support the learning
• Process does not support learning
• No opportunity to use the skills
• No time to implement the skills
• Didn’t learn anything that could be applied to work
• Resources are not available to implement the skills
• Dint see a need to implement
• No body cares for the change

Increase / Decrease / Cost
avoidance in:
1. Hard data
2. Soft Data

L4 - Time to Billability
May 2013 : 136 days
May 2014: 57 days
No. of days saved per CM is 78.8 days

L4 - Time to Billability
Total no. of days saved by May 2014 Campus Mind (78) to
become billable is 6146 days

A. Survey
B. Test
C. Questionnaire
D. Interview
E. Focus Group
F. Observation
G. Performance Records
Data Collection Instruments

S.No Instrument
Level
1
Level
2
Level
3
Level
4
1 Survey
2 Test
3 Questionnaire
4 Interview
5 Focus Group
6 Observation
7 Performance reports
8 Action Planning

20% analysis
80% data capture
80%
analysis
20% data capture
Measurement as a project Measurement as a process

Isolate the
Effects
Convert Data
to Monetary
Value
Calculate the
Return on
Investment
Tabulate Costs
of Solution
Identify
Intangible
Measures
Generate
Impact
Study
Level 5:
ROI
Intangible Measures
Step
5

Isolate the effects from other factors

1. Trend line analysis of performance data
2. Use of a control group arrangement
3. Estimate of the program’s impact (percent)
4. Use of experts/previous studies

20%
10%
J F M A M J J A S O N D J
REJECT
RATES
18.5% Pre Program Six-Month Average
Projected Average —
Using Pre Data as a
Base 14.5%
7% Post Program
Six-Month
Average
CPI Program Conducted
MONTHS

Established trend should continue on the same path
No other variables and influences should entered the
process after the training

Sales & Customer Service Training
Retail Chain (130 stores)

no program
Control Group
Experimental Group
L4 baseline
program
same results
increase by 12%
3 months
difference in sales

• Participants estimate
• Supervisor estimate
• Customer Input
• Experts Inputs

Wisdom of Crowds
Francis Galton

Ox Weighted – 1,198 pounds
Crowd guessed– 1,197 pounds

Get the most credible
source under one roof

1. Describe the task and explain how it will be used
2. Identify and list all potential factors
3. Estimate the contribution for each factors
4. Obtain the confidence level for error correction
5. Calculate the attributable data

Dear Colleague
MindTree and GreenBooks, an expert training evaluation company,
have conducted a study to measure the results of the training that the
Minds of May 2014 underwent in Orchard Program.
The study showed the following result:
Number of days saved per mind to become billable – 63 days
(average comparison data of May 2013 and 2014)
Upon interacting with the heads of Recruitment, Delivery and CDM,
we have identified that the following factors could have potentially
contributed to the save in number of days for the Mind's billability:

Project Demand
• Delivery head indicated that the project demand of 2014 was higher than 2013
Quality of recruitment process
• College candidates of relevant disciplines (Computer Science, IT) only were
considered for recruitment. Unlike the previous batch, students from other
disciplines (Civil, Automation, Mechanical, Biotech etc) were not considered in this
batch
• Panel Briefing emphasized quality of candidates over quantity
• Selection criteria was stringent and adhered to
Orchard Learning
• 3 months of training led by MindTree's Project Leads
• Course was designed with project and case application embedded
• Course content was created based on inputs from project managers
• Assessments process was rigorous and included case application. Only those Minds
who cleared were allowed into projects
Project assimilation and on the job learning
• Knowledge assimilated as a result of working on the project and learning from
peers, leads etc.
Others
• Leads were assigned KPI to absorb more Minds (c1) into their team
• There was a focus drive to increase billability of the Minds (c1)
• Management attention to reduce billability time

Business Impact: 6146 days
Project Demand: 20.25%
Quality of recruitment process: 19.25%
Orchard Learning: 18.25%
Project assimilation and on the job learning: 18.71%
Others: 4%

Sales %
Attributable to
training
Confidence Level
ERPS %
Attributable to
training
Confidence
Level
Total Average for
the Group
42% 86% 42% 88%
Corrected Group average for sales 36% of the increase in Sales of Boxes
Corrected Group average for
Subscription Recharge(EPRS)
37% of Increase in Monthly Recharge( EPRS)
Tata Sky used Supervisors Estimates

Case Study
National Bank
Page 5.10

Isolate the
Effects
Convert Data
to Monetary
Value
Calculate the
Return on
Investment
Tabulate Costs
of Solution
Identify
Intangible
Measures
Generate
Impact
Study
Level 5:
ROI
Intangible Measures
Step 6

• Converting output to contribution - standard value (profit/savings)
• Converting the cost of quality - standard value
• Converting employee’s time (using compensation)
• Using historical costs/savings
• Using internal and external experts
• Using data from external databases/studies
Methods of converting data to monetary
value

Service/Production Workers - Hourly (e.g. Courier) 40 - 70%
Skilled Hourly (e.g. Machinist) 75 - 100%
Clerical/Administrative (e.g. Scheduler) 50 - 80%
Professional (e.g. Sales Representative, Nurse,
Accountant)
75 - 125%
Technical (e.g. Computer Technician) 100 - 150%
Engineers (e.g. Chemical Engineer) 200 - 300%
Specialists (e.g. Computer Software Designer) 200 - 400%
Supervisors/Team Leaders (e.g. Section Supervisor) 100 - 150%
Middle Managers (e.g. Department Manager) 125 - 200%
Turnover Summary

Exit cost of previous employee Lost productivity
Recruiting cost Quality problems
Employee cost Customer dissatisfaction
Orientation cost Loss of expertise/know
Training cost Supervisor’s time for turnover
Wages and salaries while training Temporary replacemen
Parameters of a Turnover

Converting Data to Monetary Values:
Step 1 Define the unit of measure
(Increase in sales, decrease in absenteeism, time saving)
Step 2 Determine the value of each unit
One unit of sales, one cost of an absentee, one hour of time saving)
Step 3 Calculate the change in performance
Change during the reporting period
Step 4 Determine the annual amount of change
Step 5 Calculate the total value of improvement
The value of one unit of improvement multiplied by the amount of annual change
during the reporting period

Evaluation of a Team-Building Program in a Manufacturing Plant
Unit of measure = one less grievance
Using internal experts on the labour relations staff, the cost of an average
grievance was estimated to be 18,000, considering time and direct cost
Six months after the program, total grievances per month had declined by 10.
Supervisors isolated the effects of the program, determining that 7 of the 10
grievance reductions were because of the program
Using the adjusted value of 7 per month yields an annual improvement of 84
Annual value = 84 x 18,000
= 15, 12,000

Winning Every Walk-in – Tata Sky
One new customer,
Subscription recharge for a month
Average revenue earned is Rs. 300 for a set top box(Finance dept. worked out).
Increased subscription recharge per month= Rs. 21,27,878 (after deduction 70%
cost)
Average increment in sales boxes is 1016. Out of which 36% was attributed to
training.
(1016 x 36)/ 100 = 365
The percentage increase in subscription recharge attributable to training is 37%.
(2127878 x 37)/ 100 = 7,87,315
Annual value of increased sales in the set top box : 300 x 365 x 12 = 13,14,000
Annual value of increased in subscription recharge : 7,87,315 x 12 = 94,47,780

Mindtree
Time to billability reduced in days
Each day of billability is Rs. 9997
No. off days days saved is 6146. Out of which 18% was attributed to training.
1107 days were attributed to training
Step 4 Determine the (annual) amount of change
1107 days in a year
Total benefit : 1107 * 9997 = 1,10,66,679

TURNOVER COSTS SUMMARY
Entry Level - Hourly, Non Skilled (e.g. Fast Food Worker) 30 - 50%
Service/Production Workers - Hourly (e.g. Courier) 40 - 70%
Skilled Hourly (e.g. Machinist) 75 - 100%
Clerical/Administrative (e.g. Scheduler) 50 - 80%
Professional (e.g. Sales Representative, Nurse, Accountant) 75 - 125%
Technical (e.g. Computer Technician) 100 - 150%
Engineers (e.g. Chemical Engineer) 200 - 300%
Specialists (e.g. Computer Software Designer) 200 - 400%
Supervisors/Team Leaders (e.g. Section Supervisor) 100 - 150%
Middle Managers (e.g. Department Manager) 125 - 200%

To Convert or Not to Convert
No
Is there
a
standard
value?
Yes
Is there a
Method to
get there?
Move to
Intangible
Benefits
Can we
get there
with
minimum
resources
?
No
Yes
No No
Can we
convince
executive of
credibility in
2 minutes?
Convert
data and
add to
numerator
Yes Yes
Add to
numerator
Move to
Intangible
Benefits
Move to
Intangible
Benefits

Isolate the
Effects
Convert Data
to Monetary
Value
Calculate the
Return on
Investment
Tabulate Costs
of Solution
Identify
Intangible
Measures
Generate
Impact
Study
Level 5:
ROI
Intangible Measures
Step
7

Fully load the cost
• Needs Analysis cost
• Development cost
• Delivery cost
• Evaluation cost
• Others

S. No The Cost category
Value( In
Indian
Rupees)
1. Facilitation fees for the Master trainer 2,00,000
2. The cost for the content 12,00,000
3. The cost for the Market survey 25,000
4.
Participants Salaries(As per managerial grades):
M1-2 Participants( @ Rs.10000 per day) 40,000
M2 – 2 Participants( @ 7000 per day) 28,000
M5 – 22 Participants ( @ 1400 per day) 61,600
5.
Cost of internal venue for the six batches done for the Company staff
(ROIs and the ROMs). This included rent of venue and food.
1,50,000
6.
Cost of the 25 training batches for the Experience Zone Employees (@
Rs.25000 per batch).This included rent of venue and food.
6,25,000
7.
Cost of 6 refresher batches – required to tackle the employee churn (@
Rs.25000 per batch).
1,50,000
Total Costs for the training program 25,51,600
Fully loaded Cost – Tata Sky

Isolate the
Effects
Convert Data
to Monetary
Value
Calculate the
Return on
Investment
Identify
Intangible
Measures
Generate
Impact
Study
Level 5:
ROI
Intangible Measures
Step
8

Program Benefits
Program Cost
ROI =
Program Benefits – Program
Costs
Program Costs
X 100
Calculate BCR & ROI

Program Benefits = 1,07,58,691
Program Costs = 25,51,600
1,07,58,691
2551600
= 4.21
ROI =
1,07,58,691 – 25,51,600
25,51,600
= 321%X 100

Program Benefits = 1,10,66,679
Program Costs = 78,00,000
1,10,66,679
78,00,000
= 1.41
ROI =
1,10,66,679 – 78,00,000
78,00,000
= 41%X 100

Isolate the
Effects
Convert Data
to Monetary
Value
Calculate the
Return on
Investment
Tabulate Costs
of Solution
Identify
Intangible
Measures
Generate
Impact
Study
Level 5:
ROI
Step

Common Intangibles
 Brand Awareness
 Job satisfaction
 Organizational
Commitment
 Work Climate
 Employee Engagement
 Employee Stress
 Employee Innovation
 Employee Networking
 Employee Collaboration
 Employee partnering
 Customer Satisfaction
/Dissatisfaction
 Reputation
 Image
 Customer Complaints
 Customer Response Time
 Customer Loyalty
 Teamwork
 Cooperation
 Conflict
 Decisiveness
 Communication
 Accountability
 Alliances
 Awards
 Capability
 Capacity
 Creativity
 Clarity
 Caring
 Corporate Social
Responsibility
 Human Life
 Intellectual Capital
 Leadership
 Poverty
 Sustainability
 Work/Life Balance

• The confidence of the Experience zone employee increased due to the training. This
reflected in all the activities of the store. The morale of the experience zone employee as
well that of the owner increased. The effect was noticeable but since we could not
measure the difference in confidence pre and post training, this was reported as an
intangible benefit.
• The better customer management at the stores reduced the incoming call load at the
contact center. As we could not use any method to isolate the effect we decided to keep it
as an intangible method.
• Better handling of customers during the time of sale also ensured that they recharged for
the first month more regularly. This reduced the casual churn of the customers due to not
recharging their accounts in time. Here again since we could not find a way to isolate this
particular effect of training we decided to list this as an intangible benefit.
• The ROIs got skilled on the job coaching. This will help in better channel management.
Intangibles – Tata Sky

Isolate the
Effects
Convert Data
to Monetary
Value
Calculate the
Return on
Investment
Tabulate Costs
of Solution
Identify
Intangible
Measures
Generate
Impact
Study
Level 5:
ROI
Intangible Measures
Step 10

• General information
• Methodology for impact study
• Data analysis
• Costs
• Results
• Barriers and enablers
• Summary of findings
• Conclusions and recommendations
• Exhibits
Complete Report

2016, GreenBooks Learning Solutions
get started
Shyam Sunder
ss@greenbooks.co.in
+91 – 7766 100 400
www.greenbookslearning.com

Measuring ROI in Training

More Related Content

What's hot

Viewers also liked

Similar to Measuring ROI in Training

Recently uploaded

Measuring ROI in Training