Measuring ROI in Training

1,620
-1

Published on

Published in: Education, Business, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,620
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
177
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Measuring ROI in Training

  1. 1. Measuring ROI in Training
  2. 2. Reaction Objectives1)  Perceive it to be relevant to your work2)  Find it important to your success on the job3)  Find the case studies relevant and useful
  3. 3. Learning Objectives1)  Identify the different evaluation levels2)  Align programs to business needs3)  Identify latest 3 ways to collect data4)  Describe at least 2 ways to isolate the effects of the program5)  Calculate ROI given benefits & cost
  4. 4. Application Objectives1)  Develop an evaluation plan for a specific program
  5. 5. Impact Objectives1)  Improve the effectiveness of the programs2)  Redesign ineffective programs3)  Show ROI on your programsSix months after completing the workshop, you should:
  6. 6. Case Study[ Reliance Insurance Organization ]Page 1.4
  7. 7. Discussion Questions1)  Is this situation typical?2)  What are the basic issues in the case?3)  How could this situation be avoided in the future?4)  What should Marge do? Be specific?5)  What should Frank be expecting in the future?
  8. 8. ActivityVs Results
  9. 9. ActivityVs ResultsNo business need for the program Program linked to specific businessneedsNo effort to prepare the workenvironment to support programEnvironment prepared to supportprogramReporting on program is inputfocusedReporting on program is outputfocusedObjectives are defined at lowerlevelsObjectives are defined at higherlevels
  10. 10. Training Evaluationis
  11. 11. ?ARTScience
  12. 12. evolution of evaluation…
  13. 13. 1954
  14. 14. Peter Drucker
  15. 15. If you can’t measure itYou can’t manage it
  16. 16. 1959
  17. 17. DonaldKirkpatrick
  18. 18. 1. Reaction2. Learning3. Behavior4. Results
  19. 19. 1970
  20. 20. Jack Phillips
  21. 21. 5. ROI
  22. 22. Kirkpatrick Model Phillips ROI ModelReactionLevel 1Level 2Level 3Level 4AccountabilityProcess ModelStandardsPublicationsUsageDr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1DevelopedInputs / IndicatorsLevel 0Level 5
  23. 23. Reaction, Satisfaction
  24. 24. Reaction, Satisfaction & Planned action
  25. 25. Kirkpatrick Model Phillips ROI ModelReaction Reaction, Satisfaction & Planned ActionLevel 1Level 2Level 3Level 4AccountabilityProcess ModelStandardsPublicationsUsageDr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1DevelopedInputs / IndicatorsLevel 0Level 5Learning Learning
  26. 26. Kirkpatrick Model Phillips ROI ModelReaction Reaction, Satisfaction & Planned ActionLearning LearningBehaviour ApplicationLevel 1Level 2Level 3Level 4AccountabilityProcess ModelStandardsPublicationsUsageDr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1DevelopedInputs / IndicatorsLevel 0Level 5
  27. 27. LearningApplication
  28. 28. If the participants know the L3 behaviors are being trackedthey will be more likely to apply
  29. 29. Kirkpatrick Model Phillips ROI ModelReaction Reaction, Satisfaction & Planned ActionLearning LearningBehaviour ApplicationResults Business ImpactLevel 1Level 2Level 3Level 4AccountabilityProcess ModelStandardsPublicationsUsageDr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1DevelopedInputs / IndicatorsLevel 0Level 5
  30. 30. Customer satisfactionSalesUnits manufacturedStudents graduatedincrease
  31. 31. CostsQuality complaintsTime to project completionIneffective programsdecrease
  32. 32. Cost avoidance
  33. 33. Kirkpatrick Model Phillips ROI ModelReaction Reaction, Satisfaction & Planned ActionLearning LearningBehaviour ApplicationResults Business ImpactIsolating the effects of the programLevel 1Level 2Level 3Level 4AccountabilityProcess ModelStandardsPublicationsUsageDr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1DevelopedInputs / IndicatorsLevel 0Level 5
  34. 34. Incentives Systems/ProceduresTrainingManagementAttention
  35. 35. Incentives Systems/ProceduresTrainingManagementAttention
  36. 36. Kirkpatrick Model Phillips ROI ModelReaction Reaction, Satisfaction & Planned ActionLearning LearningBehaviour ApplicationResults Business ImpactIsolating the effects of the programLevel 1Level 2Level 3Level 4AccountabilityProcess ModelStandardsPublicationsUsageDr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1DevelopedInputs / IndicatorsLevel 0ROE / ROILevel 5
  37. 37. Return on Expectations(ROE)
  38. 38. Show Me the Money
  39. 39. Return on Investment(ROI)
  40. 40. Defining BCR & ROIBenefits/ Cost Ratio =Program BenefitsProgram CostROI =Net Program BenefitsProgram CostsX 100
  41. 41. Program Benefits from Participants (1st year) = ` 12,30,000Costs per Program (25 participants) = ` 6,88,000Benefits/ Cost Ratio =12, 30, 0006,88, 000= 1.79ROI =12,30,000 – 6,88,0006,88, 000= 79%X 100
  42. 42. Cause & Effect for 5 Levels
  43. 43. Lost the Evidence
  44. 44. Kirkpatrick Model Phillips ROI ModelReaction Reaction, Satisfaction & Planned ActionLearning LearningBehaviour ApplicationResults Business ImpactIsolating the effects of the programLevel 1Level 2Level 3Level 4AccountabilityProcess ModelStandardsPublicationsUsageDr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1Developed10 step by step of collecting, analyzing, reporting, using systematic & methodic90–100%70–80%20–30%10–20%01–05%Inputs / IndicatorsLevel 0 100%Return on InvestmentLevel 5
  45. 45. DevelopObjectives ofSolution (s)DevelopEvaluationPlans andBaseline DataCollectData DuringSolutionImplementationCollectData AfterSolutionImplementationLevel 2: Learning Level 4: Business ImpactLevel 1: Reaction,Satisfaction, andPlanned ActionsLevel 3:Application/ImplementationEvaluation Planning Data CollectionTHE PROCESS MODEL
  46. 46. Isolate theEffectsConvert Datato MonetaryValueCalculate theReturn onInvestmentTabulate Costsof SolutionIdentifyIntangibleMeasuresGenerateImpactStudyLevel 5:ROIIntangible MeasuresData Analysis Reporting
  47. 47. Kirkpatrick Model Phillips ROI ModelReaction Reaction, Satisfaction & Planned ActionLearning LearningBehaviour ApplicationResults Business ImpactIsolating the effects of the programLevel 1Level 2Level 3Level 4AccountabilityProcess ModelStandardsPublicationsUsageDr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1Developed10 step by step of collecting, analyzing, reporting, using systematic & methodic12 guiding principles to be conservative in approach90–100%70–80%20–30%10–20%01–05%Inputs / IndicatorsLevel 0 100%Return on InvestmentLevel 5
  48. 48. 12 Guiding PrinciplesJob Aid
  49. 49. Kirkpatrick Model Phillips ROI ModelReaction Reaction, Satisfaction & Planned ActionLearning LearningBehaviour ApplicationResults Business ImpactIsolating the effects of the programLevel 1Level 2Level 3Level 4AccountabilityProcess ModelStandardsPublicationsUsageDr. Donald Kirkpatrick Dr. Jack PhillipsLevel 1Developed10 step by step of collecting, analyzing, reporting, using systematic & methodic12 guiding principles to be conservative in approach5 books 40 books,3000 case studiesSlowly phasing out 3000 organizations in 50 countries90–100%70–80%20–30%10–20%01–05%Inputs / IndicatorsLevel 0 100%Return on InvestmentLevel 5
  50. 50. Characteristics of Evaluation LevelsChain of Value of Customer Frequency Difficulty of Impact Information Focus of Use Assessment Satisfaction Lowest Consumer Frequent Easy Learning Application Impact ROI Highest Client Infrequent Difficult
  51. 51. All program are ‘not evaluated’ at alllevels…
  52. 52. Criteria for L4 and L5
  53. 53. Level Measurement category Current statusEvaluationTargets0 Inputs and Indicators: 100%1 Reaction and planned action: 100%2To measure what the participants havelearned in the program-90%3 Application and Implementation: 50%4 Impact and consequences: 30%5 Return on investment: 10%
  54. 54. Level Measurement category Current statusEvaluationTargets0 Inputs and Indicators: Less then 25% of the programs 100%1 Reaction and planned action: 100% of the workshops conducted. 100%2To measure what the participants havelearned in the program-Done for more than 60-70% 90%3 Application and Implementation: Less than 10% of the training 50%4 Impact and consequences: Not done for any training 30%5 Return on investment: Not done for any training 15%Tata Sky
  55. 55. What is your organization’scurrent status?
  56. 56. Evaluation PlanningCategory 1
  57. 57. DevelopObjectives ofSolution (s)DevelopEvaluationPlans andBaseline DataEvaluation Planning
  58. 58. Hard DataOutputTimeCostsQualityPage 3.5
  59. 59. Soft DataWork habitsClimate/ SatisfactionInitiative/ InnovativeCustomer ServiceDevelopmentImagePage 3.7
  60. 60. Baseline Data
  61. 61. Initial collection of data which serves as a basis forCOMPARISON with the subsequently acquired data.
  62. 62. 125 Kgs 86 Kgs
  63. 63. Existing Expected
  64. 64. Evaluation Planning WorksheetPage 3.2 (pre-work)
  65. 65. AnalysisDesignDevelopmentImplementationEvaluation}ADDIE
  66. 66. ProjectBusiness Alignment and Forecasting The ROI Process ModelBusiness Alignment ModelLearning NeedsPreference NeedsMeasurement andEvaluationReactionLearningApplicationImpactROIReaction ObjectivesLearning ObjectivesApplication ObjectivesPerformance NeedsImpact ObjectivesBusiness NeedsPayoff Needs ROI ObjectivesEnd HereStart Here543254321 1ROI NeedsAnalysis
  67. 67. Do an ROI based needs
  68. 68. ProjectLearning NeedsPreference Needs ReactionLearningApplicationImpactROIReaction ObjectivesLearning ObjectivesApplication ObjectivesPerformance NeedsImpact ObjectivesBusiness NeedsPayoff Needs ROI ObjectivesEnd HereStart Here543254321 1
  69. 69. 2 things
  70. 70. 1. Different Hats
  71. 71. Business HeadTrainerProgram Evaluator
  72. 72. 2. Cause & Effect
  73. 73. (Training Need)
  74. 74. primaryheuristiccoherentStripping Uncertainty in Needs Analysis
  75. 75. Learning NeedsPreference NeedsPerformance NeedsBusiness NeedsPayoff Needs54321Absenteeism is causing 15,00,000monthlyUnplanned absenteeism is 9%Discussion between team membersand supervisor are not occurringCounseling and discussion skillsCounseling skills must be practicaland should happen immediately
  76. 76. DevelopObjectives ofSolution (s)DevelopEvaluationPlans andBaseline DataCollectData DuringSolutionImplementationCollectData AfterSolutionImplementationEvaluation Planning Data Collection
  77. 77. ROI ObjectivesLearning NeedsPreference NeedsPerformance NeedsBusiness NeedsPayoff Needs54321Discussion between team membersand supervisor are not occurringCounseling and discussion skillsCounseling skills must be practicaland should happen immediately4/5 on the practicality of theprogram and relevance to jobParticipants will learn 3 keycounseling skillsCounseling discussion conducted in95% situations when an unexpectedabsence occursReduce absenteeism to 5%ROI of 20%Reaction ObjectivesLearning ObjectivesApplication ObjectivesImpact ObjectivesAbsenteeism is causing 15,00,000monthlyUnplanned absenteeism is 9%
  78. 78.    The best way to Predict a future is to …create it.“”
  79. 79. DevelopObjectives ofSolution (s)DevelopEvaluationPlans andBaseline DataCollectData DuringSolutionImplementationCollectData AfterSolutionImplementationEvaluation Planning Data Collection
  80. 80. V ModelLearning NeedsPreference Needs ReactionLearningApplicationImpactROIReaction ObjectivesLearning ObjectivesApplication ObjectivesPerformance NeedsImpact ObjectivesBusiness NeedsPayoff Needs ROI ObjectivesEnd HereStart Here543254321 1Absenteeism is causing`15,00,000 monthlyUnplanned absenteeism is 9%and increasing.Discussion between teammember and supervisor arenot occurringDeficiency in counseling anddiscussion skillsCounseling skills must bepractical and should happenimmediately4/5 on the practicality ofthe program and relevanceto jobParticipants will learn 3 keycounseling skillsCounseling discussion conducted in 95% situationswhen an unexpected absence occursReduce absenteeism to 5%ROI of 20%Reaction Questionnaire atthe end of the programPractice session during program andfacilitator assessmentFollow up questionnaire toparticipants to check the frequencyof discussionsMonitor absenteeismrecords for 6 monthsCalculate ROI, BCR
  81. 81. V Model –Tata SkyLearning NeedsPreference Needs ReactionLearningApplicationImpactROIReaction ObjectivesLearning ObjectivesApplication ObjectivesPerformance NeedsImpact ObjectivesBusiness NeedsPayoff Needs ROI ObjectivesEnd HereStart Here543254321 1Increase SalesIncrease EPRSIncrease walkin conversionProfiling of the customer,Standardization in customer demo,& transactionGreetingAsking right questions to the prospects tounderstand the wants and needs,Match the needs with features & benefitsILT and role plays 4/5 on the practicality of theprogram and relevance to jobParticipants will learn key interaction& influencing skillsUse the skills consistently, which is 100% of thetimes+ve ROITrack business dashboardsevery monthCalculate ROI, BCRIncrease walk-in conversion ratioIncrease the subscription recharge by 10%Increase Sales by 10%Action plan activity sheetFacilitator assessment through roleplaysReaction and planned action through aquestionnaire at the end of the program
  82. 82. Learning NeedsPreference NeedsPerformance NeedsBusiness NeedsPayoff Needs54321Increase SalesIncrease EPRSProfiling of the customer,Standardization in customerdemo, & transactionGreetingunderstand the wants and needsInfluencingILT and role plays4/5 on the practicality andrelevanceParticipants will learn keyinteraction & influencing skillsUse the skills consistently, which is100% of the times+ve ROIIncrease Sales byIncrease the subscription recharge by
  83. 83. V Model –Tata SkyLearning NeedsPreference Needs ReactionLearningApplicationImpactROIReaction ObjectivesLearning ObjectivesApplication ObjectivesPerformance NeedsImpact ObjectivesBusiness NeedsPayoff Needs ROI ObjectivesEnd HereStart Here543254321 1Increase SalesIncrease EPRSProfiling of the customer,Standardization in customer demo,& transactionGreetingAsking right questions to the prospects tounderstand the wants and needs,Match the needs with features & benefitsILT and role plays 4/5 on the practicality of theprogram and relevance to jobParticipants will learn key interaction& influencing skillsUse the skills consistently, which is 100% of thetimes+ve ROITrack business dashboardsevery monthCalculate ROI, BCRIncrease the subscription recharge by 10%Increase Sales by 10%Action plan activity sheetFacilitator assessment through roleplaysReaction and planned action through aquestionnaire at the end of the program
  84. 84. V Model – Publishing FirmLearning NeedsPreference Needs ReactionLearningApplicationImpactROIReaction ObjectivesLearning ObjectivesApplication ObjectivesPerformance NeedsImpact ObjectivesBusiness NeedsPayoff Needs ROI ObjectivesEnd HereStart Here543254321 1Potential project loss of`1,25,00,000 a year-Improve quality of emails-Decrease supervisors monitoringtimeExcellent email communicationskills for a global audienceStructure, tone, style and grammarand global cultureBlended – ILT, LMS and coaching Overall score of 90%Participants will learn email skillsUse appropriate email writing skills 100% of thetimes while writing emailsError rates to zeroDecrease managers monitoring time to nilROI of 50%Online reaction QuestionnaireOnline pre and post assessmentEmail scorecardPerformance records &InterviewCalculate ROI, CBR
  85. 85. Each level includes:•  Data Sources•  Data Collection•  Key Questions•  Key IssuesCollect Data AfterSolutionImplementationIsolatethe Effectsof the SolutionConvert Data ToMonetaryValueCalculateThe Return OnInvestmentTabulateCosts ofSolutionIdentifyIntangiblesSignificant Influences•  Policy Statement•  Procedures and Guidelines•  Staff Skills•  Management Support•  Technical Support•  Organizational CulturePhillips ROI Methodology TMDevelopReport andCommunicateResultsLevel 4 Level 3 Level 3 and 2 Level 1Level 3Level 4Level 5Level 5Results-Based ApproachIntangibleBenefitsSpecify Skill/KnowledgeDeficiencies ofAffected PopulationTrainingRequired?Level 2Problem/OpportunityPresent orAnticipatedIdentify JobPerformanceNeeds, Gaps, andWhyIdentifySolutionsIdentifyPreferencesDevelopObjectives/EvaluationStrategyIdentify TransferStrategy Options andL-2 & L-3 SupportFor All StakeholdersIdentifyBusiness,Needs, Gaps,andStakeholdersDesignSolution andStakeholderComponentsConsiderResources/LogisticsDeliveryDevelopContent/MaterialsConduct/ImplementSolutionLevel 1Level 2ImplementPre-activityImplementTransferStrategy- Solution- Pre-work Activity/Transfer StrategyPage 3.9
  86. 86. Page 3.12( More Examples )
  87. 87. Exercise 3.16
  88. 88. DevelopObjectives ofSolution (s)DevelopEvaluationPlans andBaseline DataCollectData DuringSolutionImplementationCollectData AfterSolutionImplementationEvaluation Planning Data CollectionStep 2
  89. 89. 1.  Data collection Plan2.  ROI analysis planEvaluation Planning ( 2 docs)
  90. 90. Page 3.24( sample )
  91. 91. Level Objectives Measures/DataDataCollectionmethodData Source Timings Responsibility1 Positive reactionAn overall averagerating of 4/5QuestionnaireParticipants of theprogramAt the end ofeach workshopFacilitator2Will be able to demonstrate allthe 3 skills during the role playFacilitators ratingObservationof skill practiceby facilitatorFacilitatorDuring theworkshopFacilitator3(1)Will be able to assess all theEZs for adherence to thetaught processes(2)Will be able to bring all theemployees to a consistent levelof display of learned skills.Completion of ActionplanAction plansheetRetail OperationIncharges(ROIs)Retail OperationManagers(ROMs)30 – 45 daysafter theworkshopRegional ManagersRetail Operations4(1)The walk in conversion ratiowill increase(2)The sales will increase(3)The customer retention willincreaseMonthly data from theRetail OperationVertical of theExperience ZonesBusinessperformancemonitoringDashboard,Retail Operations45 – 90 daysafter theworkshopRegional ManagersRetail Operations5 ROI No Objective was set for this level in the beginning. The expectation was to have a positive ROI.Data Collection Plan– Tata Sky
  92. 92. Level Objectives Measures/DataDataCollectionmethodData Source Timings Responsibility1Reaction to the program,content and facilitator.An overall averagerating of 4.5/5OnlinequestionnaireParticipants of theprogramAt the end ofeach workshopFacilitator2 Assess the email writing skills  Score above 90% inthe post assessmentOnline test ParticipantAt the end ofeach workshopFacilitator3Demonstrate appropriatestructure, tone, style andgrammar in emails100% score on thescore cardScore card ParticipantWeeklymonitoring ofemails for 3monthsFacilitator4Improve quality of emailsDecrease managers monitoringtimeDecrease email errorsto zeroDecrease manager’smonitoring time to nilPerformancerecords,InterviewManagers3 months aftertrainingFacilitator5 ROI 50% ROI was set.Data Collection Plan– Publishing
  93. 93. Data Items(UsuallyLevel 4Method forIsolationMethods ofConvertingData toMonetaryValuesCost Categories IntangibleBenefitsCommunicationTargets for FinalReportOtherInfluences/IssuesDuringApplicationMonthlyconversionratioParticipantestimate(ROMs &ROI)StandardValue.(1)Facilitation Fees(2)Program Materials(3)Market survey Fees(4)Facility costs(5)Participants salariesROMsROIsExec.V PRetailoperationsAny changein marketfactorROI Analysis Plan– Tata Sky
  94. 94. Baseline DataTata Sky – Sales Mystery Shopping•  Ambience ( 4 parameters )•  Grooming ( 4 parameters )•  Opening the call (15 parameters )•  Developing/ Identifying the need ( 8 parameters )•  Proposing a solution ( 4 parameters )•  Objection Handling ( 5 parameters )•  Closing (4 parameters )•  Follow up ( 2 parameters )•  Process adherence ( 4 parameters )•  Telephone Related (1 parameters )
  95. 95. Baseline DataPublishing Firm – Score from emails, reportsParameters Pre-AssessmentGrammar 63.8 %Structure 85.4%Tone & Style 58.3%Average Score 70.2%
  96. 96. Data CollectionCategory 2
  97. 97. DevelopObjectives ofSolution (s)DevelopEvaluationPlans andBaseline DataCollectData DuringSolutionImplementationCollectData AfterSolutionImplementationLevel 2:LearningLevel 4:Business ImpactLevel 1: Reaction,Satisfaction, andPlanned ActionsLevel 3:Application/ImplementationEvaluation Planning Data CollectionStep 3 Step 4
  98. 98. 1.  Relevance to work2.  Important to success3.  Would recommend to others4.  Amount of new information5.  Method of delivery6.  Learning Environment7.  Planned ActionLevel 1 Target AreasPage 4.3
  99. 99. 1.  Skills2.  Knowledge3.  Capacity4.  Capability5.  AwarenessLevel 2
  100. 100. 1.  Test2.  Simulation3.  Business games4.  In-basket5.  Case Studies6.  Role playing7.  Assessment center8.  Self, team, facilitator assessmentTechniques for Measuring Learning
  101. 101. DevelopObjectives ofSolution (s)DevelopEvaluationPlans andBaseline DataCollectData DuringSolutionImplementationCollectData AfterSolutionImplementationLevel 4:Business ImpactLevel 3:Application/ImplementationEvaluation Planning Data CollectionStep 4
  102. 102. 1.  Skills - Demonstration of the new skills on the job2.  Knowledge - Take appropriate actions/ decisions based on theknowledge acquired3.  Capacity – Demonstrate ability on the job4.  Awareness – Demonstrate the required behaviors on the jobLevel 3 Target Areas
  103. 103. Increase / Decrease / Cost avoidance in:1.  Hard data2.  Soft DataLevel 4 Target Areas
  104. 104. A. SurveyB.  TestC. QuestionnaireD. InterviewE.  Focus GroupF.  ObservationG. Performance RecordsData Collection Instruments
  105. 105. S.No Instrument Level 1 Level 2 Level 3 Level 41 Survey2 Test3 Questionnaire4 Interview5 Focus Group6 Observation7 Performance Contracts8 Action PlanningData Collection Instruments
  106. 106. Page 4.8
  107. 107. Follow up Questionnaire content checklistPage 4.10
  108. 108. How do you improve the response ratefor the questionnaire?Page 4.21
  109. 109. Page 4.23Action Planning
  110. 110. Data AnalysisCategory 3
  111. 111. Isolate theEffectsConvert Datato MonetaryValueCalculate theReturn onInvestmentTabulate Costsof SolutionIdentifyIntangibleMeasuresGenerateImpactStudyLevel 5:ROIIntangible MeasuresData Analysis ReportingStep 5
  112. 112. 1.  Trend line analysis of performance data2.  Use of a control group arrangement3.  Estimate of the program’s impact (percent)•  Participants•  Supervisor•  Experts4.  Use of experts/previous studiesTechniques to Isolate the effects of Programs
  113. 113. 20%10%J F M A M J J A S O N D JREJECTRATES18.5% Pre Program Six-Month AverageProjected Average —Using Pre Data as a Base14.5%7% Post Program Six-MonthAverageCPI Program ConductedMONTHSUsing Trend Analysis
  114. 114. Two Conditions1.  If the training program would not been implemented, wouldthis trend continue on the same path established before thetraining?2.  No other variables and influences entered the process afterthe training was conducted
  115. 115. Control Group
  116. 116. Sales & Customer Service TrainingRetail Chain (130 stores)
  117. 117. No ProgramProgramMeasurementMeasurementControl GroupExperimental
  118. 118. Using Estimates•  Participants estimate•  Supervisor estimate•  Customer Input•  Experts Inputs
  119. 119. Wisdom of CrowdsFrancis Galton
  120. 120. Ox Weighted – 1,198 poundsCrowd guessed– 1,197 pounds
  121. 121. Sales %Attributable totrainingConfidence LevelERPS %Attributable totrainingConfidenceLevelTotal Average forthe Group42% 86% 42% 88%Corrected Group average for sales 36% of the increase in Sales of BoxesCorrected Group average forSubscription Recharge(EPRS)37% of Increase in Monthly Recharge( EPRS)Tata Sky used Supervisors Estimates
  122. 122. DataSourceTime Savingsin HoursAttributionLevel toTrainingConfidenceLevelAdjustedValueDecrease inSupervisormonitoringtimeSupervisors 1.2 hours / day 60% 100%21.5 days /yearProject &StrategicHead2 hours / week 60% 100%  7.8 days / yearDecrease inescalations ofpoor qualityemailProject Head - 60% 100% 4.2 escalationsPublishing Firm used Supervisors &Project Head Estimates
  123. 123. Case StudyNational BankPage 5.10
  124. 124. Exercise 5.5
  125. 125. Isolate theEffectsConvert Datato MonetaryValueCalculate theReturn onInvestmentTabulate Costsof SolutionIdentifyIntangibleMeasuresGenerateImpactStudyLevel 5:ROIIntangible MeasuresData Analysis ReportingStep 6
  126. 126. •  Converting output to contribution - standard value (profit/savings)•  Converting the cost of quality - standard value•  Converting employee’s time (using compensation)•  Using historical costs/savings•  Using internal and external experts•  Using data from external databases/studiesMethods of converting data to monetary value
  127. 127.  Service/Production Workers - Hourly (e.g. Courier) 40 - 70%Skilled Hourly (e.g. Machinist) 75 - 100%Clerical/Administrative (e.g. Scheduler) 50 - 80%Professional (e.g. Sales Representative, Nurse,Accountant) 75 - 125%Technical (e.g. Computer Technician) 100 - 150%Engineers (e.g. Chemical Engineer) 200 - 300%Specialists (e.g. Computer Software Designer) 200 - 400%Supervisors/Team Leaders (e.g. Section Supervisor) 100 - 150%Middle Managers (e.g. Department Manager) 125 - 200%Turnover Summary
  128. 128. Exit cost of previous employee Lost productivityRecruiting cost Quality problemsEmployee cost Customer dissatisfactionOrientation cost Loss of expertise/knowledgeTraining cost Supervisor’s time for turnoverWages and salaries while training Temporary replacement costsParameters of a Turnover
  129. 129. Converting Data to Monetary Values:Step 1 Define the unit of measure(Increase in sales, decrease in absenteeism, time saving)Step 2 Determine the value of each unitOne unit of sales, one cost of an absentee, one hour of time saving)Step 3 Calculate the change in performanceChange during the reporting periodStep 4 Determine the annual amount of changeStep 5 Calculate the total value of improvementThe value of one unit of improvement multiplied by the amount of annual change duringthe reporting period
  130. 130. Converting Data to Monetary Values:Evaluation of a Team-Building Program in a Manufacturing PlantStep 1 Define the unit of measureUnit of measure = one less grievanceStep 2 Determine the value of each unitUsing internal experts on the labour relations staff, the cost of an average grievance wasestimated to be 18,000, considering time and direct costStep 3 Calculate the change in performanceSix months after the program, total grievances per month had declined by 10.Supervisors isolated the effects of the program, determining that 7 of the 10 grievancereductions were because of the programStep 4 Determine the annual amount of changeUsing the adjusted value of 7 per month yields an annual improvement of 84Step 5 Calculate the total value of improvementAnnual value = 84 x 18,000= 15, 12,000
  131. 131. Converting Data to Monetary Values:Winning Every Walk-in – Tata SkyStep 1 Define the unit of measureOne new customer,Subscription recharge for a monthStep 2 Determine the value of each unitAverage revenue earned is Rs. 300 for a set top box(Finance dept. worked out).Increased subscription recharge per month= Rs. 21,27,878 (after deduction 70% cost)Step 3 Calculate the change in performanceAverage increment in sales boxes is 1016. Out of which 36% was attributed to training.(1016 x 36)/ 100 = 365The percentage increase in subscription recharge attributable to training is 37%.(2127878 x 37)/ 100 = 7,87,315Step 4 Determine the annual amount of changeAnnual value of increased sales in the set top box : 300 x 365 x 12 = 13,14,000Annual value of increased in subscription recharge : 7,87,315 x 12 = 94,47,780Step 5 Calculate the total value of improvementTotal benefit : 13,14,000 + 94,47,780 = 1,07,61,780
  132. 132. Exercise 5.21
  133. 133.   TURNOVER COSTS SUMMARYEntry Level - Hourly, Non Skilled (e.g. Fast Food Worker) 30 - 50%Service/Production Workers - Hourly (e.g. Courier) 40 - 70%Skilled Hourly (e.g. Machinist) 75 - 100%Clerical/Administrative (e.g. Scheduler) 50 - 80%Professional (e.g. Sales Representative, Nurse,Accountant) 75 - 125%Technical (e.g. Computer Technician) 100 - 150%Engineers (e.g. Chemical Engineer) 200 - 300%Specialists (e.g. Computer Software Designer) 200 - 400%Supervisors/Team Leaders (e.g. Section Supervisor) 100 - 150%Middle Managers (e.g. Department Manager) 125 - 200%
  134. 134. To Convert or Not to ConvertNoIs there astandardvalue?YesIs there aMethod toget there?Move toIntangibleBenefitsCan we getthere withminimumresources?NoYesNo NoCan weconvinceexecutive ofcredibility in2 minutes?Convertdata andadd tonumeratorYes YesAdd tonumeratorMove toIntangibleBenefitsMove toIntangibleBenefits
  135. 135. Isolate theEffectsConvert Datato MonetaryValueCalculate theReturn onInvestmentTabulate Costsof SolutionIdentifyIntangibleMeasuresGenerateImpactStudyLevel 5:ROIIntangible MeasuresData Analysis ReportingStep 7
  136. 136. Fully load the cost•  Needs Analysis cost•  Development cost•  Delivery cost•  Evaluation cost•  Others
  137. 137. S. No The Cost categoryValue( InIndianRupees)1. Facilitation fees for the Master trainer 2,00,0002. The cost for the content 12,00,0003. The cost for the Market survey 25,0004.Participants Salaries(As per managerial grades):M1-2 Participants( @ Rs.10000 per day) 40,000M2 – 2 Participants( @ 7000 per day) 28,000M3 – 2 Participants( @ 4000 per day) 16,000M4 – 10 Participants( @ 2800 per day) 56,000M5 – 22 Participants ( @ 1400 per day) 61,6005.Cost of internal venue for the six batches done for the Company staff (ROIs andthe ROMs).This included rent of venue and food.1,50,0006.Cost of the 25 training batches for the Experience Zone Employees (@ Rs.25000 per batch).This included rent of venue and food.6,25,0007.Cost of 6 refresher batches – required to tackle the employee churn (@ Rs.25000 per batch).1,50,000Total Costs for the training program 25,51,600Fully loaded Cost – Tata Sky
  138. 138. Isolate theEffectsConvert Datato MonetaryValueCalculate theReturn onInvestmentIdentifyIntangibleMeasuresGenerateImpactStudyLevel 5:ROIIntangible MeasuresData Analysis ReportingStep 8
  139. 139. Benefits/ Cost Ratio =Program BenefitsProgram CostROI =Program Benefits – Program CostsProgram CostsX 100Calculate BCR & ROI
  140. 140. Program Benefits = 1,07,58,691Program Costs = 25,51,600Benefits/ Cost Ratio =1,07,58,6912551600= 4.21ROI =1,07,58,691 – 25,51,60025,51,600= 321%X 100
  141. 141. Isolate theEffectsConvert Datato MonetaryValueCalculate theReturn onInvestmentTabulate Costsof SolutionIdentifyIntangibleMeasuresGenerateImpactStudyLevel 5:ROIData Analysis ReportingStep 9
  142. 142. Common Intangibles§  Brand Awareness§  Job satisfaction§  Organizational Commitment§  Work Climate§  Employee Engagement§  Employee Stress§  Employee Innovation§  Employee Networking§  Employee Collaboration§  Employee partnering§  Customer Satisfaction /Dissatisfaction§  Reputation§  Image§  Customer Complaints§  Customer Response Time§  Customer Loyalty§  Teamwork§  Cooperation§  Conflict§  Decisiveness§  Communication§  Accountability§  Alliances§  Awards§  Capability§  Capacity§  Creativity§  Clarity§  Caring§  Corporate SocialResponsibility§  Human Life§  Intellectual Capital§  Leadership§  Poverty§  Sustainability§  Work/Life Balance
  143. 143. •  The confidence of the Experience zone employee increased due to the training.Thisreflected in all the activities of the store.The morale of the experience zone employee aswell that of the owner increased.The effect was noticeable but since we could not measurethe difference in confidence pre and post training, this was reported as an intangible benefit.•  The better customer management at the stores reduced the incoming call load at thecontact center. As we could not use any method to isolate the effect we decided to keep itas an intangible method.•  Better handling of customers during the time of sale also ensured that they recharged forthe first month more regularly.This reduced the casual churn of the customers due to notrecharging their accounts in time. Here again since we could not find a way to isolate thisparticular effect of training we decided to list this as an intangible benefit.•  The ROIs got skilled on the job coaching.This will help in better channel management.Intangibles – Tata Sky
  144. 144. Isolate theEffectsConvert Datato MonetaryValueCalculate theReturn onInvestmentTabulate Costsof SolutionIdentifyIntangibleMeasuresGenerateImpactStudyLevel 5:ROIIntangible MeasuresData Analysis ReportingStep 10
  145. 145. •  General information•  Methodology for impact study•  Data analysis•  Costs•  Results•  Barriers and enablers•  Summary of findings•  Conclusions and recommendations•  ExhibitsComplete Report
  146. 146. Project Implementation Issues
  147. 147. Shyam SunderCofounder & Principalss@greenbooks.co.inwww.linkedin.com/in/shyamsunderawww.twitter.com/shyamsunderwww.greenbooks.co.inget started
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×