<ul><li>“ Not everything that can be counted counts, and not everything that counts can be counted” Albert Einstein </li></ul>
Best Practice in Terms of ROI <ul><li>Training magazine’s 2003 top 100 companies found that linking training to specific business metrics requires evaluation before, during, and after training initiatives ; </li></ul><ul><li>Many of the Top 100 companies ranked by Training magazine tie training initiatives directly to retention effort s; </li></ul><ul><li>Also use other metrics – sales; production etc; </li></ul><ul><li>ROI used as basis for continued improvement of training function; </li></ul><ul><li>Forecas t ROI. </li></ul>
The Case for ROI ? Accenture Allstate Insurance Apple Computer AT&T Bristol-Myers Squibb British Telecom Caremark Comcast Coors Brewing Co. Deloitte & Touché Delta Airlines DHL Worldwide Express Federal Express General Motors US Department of Defense US Department of Labor US National Security Agency US Department of Veteran Affairs US General Accountability Office US Office of Personnel Management NASA
How does your company determine training needs?
Question – we develop and implement training that Absolutely Un True= 1 ; Absolutely True = 5 Link needs to business solutions 1 2 3 4 5 Based on assessment of current vs. desired performance 1 2 3 4 5 Isolate application and business impact objectives 1 2 3 4 5 Ensures communication of expectations to stakeholders and participants 1 2 3 4 5 Ensures the work environment supports reinforcement of implementation and application 1 2 3 4 5 Builds partnerships with managers 1 2 3 4 5 Measures at more than reaction and learning level 1 2 3 4 5 Communicates the results to all stakeholders 1 2 3 4 5
How Does Your Organization Score 8-15 Activity based Little Focus on Results 24-31 Significant progress To Results Based 16-23 Moderate shift Or Shift in Certain Areas 32- 40 Fully Integrated Results Based System Which Individual Areas Do You Need to Address?
Activity Results No established business need Business case Little relation to performance Focus on improving performance/change behavior No/little focus on objectives/competencies Demonstrating competence over time No/Little focus on preparing the environment Prepare environment to ensure transfer and reinforcement No/Little focus on measuring impact and value System geared to preparing and proving the case Mainly input focused Sound balance between input and output
<ul><li>We need to move towards a results </li></ul><ul><li>based approach </li></ul><ul><li>But what should we put in place? </li></ul>
Evaluation Approach Element Description Reaction Happiness with learning interventions Learning Changes in knowledge, skills and attitudes Transfer/application Changes in on the job behaviors over time (competence) Business Impact Changes in impact variables ROI Intervention benefits in relation to cost Intangibles Intervention benefits in relation to policy and/or strategic direction and related factors
Evaluation Approach Administrator Business Partner Customer Focus Data Availability Value of Case (relative) Intangibles ROI Business Impact Application and implementation Learning Reaction Element Assessment Complexity Limited/Low Wide/High Consumer Client Readily Research Limited/Low Wide/Complex
Intervention Planning Framework Reaction Evaluation General need Learning Outcomes Learning Material Achieved? Assess More