Effectiveness of Organizational Training


Published on

The request to measure effectiveness of the training performed at an organization is not met by the "beauty contest" survey taken at the end of an activity. Moreover, since 85% of knowledge acquired by adults is lost in two weeks unless used, as reported by Jane Tippett in Nurses’ acquisition and retention of knowledge after trauma training, it is of fundamental importance that the gauge corresponds to the needs. In this presentation we describe a low tech yet highly effective method for measuring the improvement in productivity gained by training attendees. The method, used since last century in a large telecom organization, is based on some premises: training is only useful if aligned with job outcomes; training should be timely and not carried out solely for consuming the training budget; training objectives should be described as learning objectives, that is to say, what behavioral changes the training is attempting to achieve; managers are responsible for the skills and competencies of their employees.

Published in: Business, Technology
  • Be the first to comment

Effectiveness of Organizational Training

  1. 1. On the Effectiveness of Training Boria, Jorge Rubinstein, Viviana
  2. 2. Objectives <ul><li>At the end of this presentation, you will be able to… </li></ul><ul><ul><li>describe the relationship between competency models and organizational training </li></ul></ul><ul><ul><li>use a simple, low tech, yet powerful method to measure the effectiveness of a training program. </li></ul></ul><ul><ul><li>But only if you are paying attention.  </li></ul></ul>
  3. 3. The CMMI Sayeth… <ul><li>SP 2.3 Assess Training Effectiveness </li></ul><ul><ul><li>Assess the effectiveness of the organization’s training program. </li></ul></ul><ul><li>A process should exist to determine the effectiveness of training (i.e., how well the training is meeting the organization’s needs). </li></ul><ul><ul><li>in lay terms, Are we getting bang for the training buck? </li></ul></ul>
  4. 4. Examples of Methods Used <ul><li>Testing in the training context </li></ul><ul><li>Post-training surveys of training participants </li></ul><ul><li>Surveys of managers’ satisfaction with post-training effects </li></ul><ul><li>Assessment mechanisms embedded in courseware </li></ul>
  5. 5. Sub Practices <ul><li>Assess in-progress or completed projects to determine whether staff knowledge is adequate for performing project tasks. </li></ul><ul><li>Provide a mechanism for assessing the effectiveness of each training course with respect to established organizational, project, or individual learning (or performance) objectives. </li></ul><ul><li>Obtain student evaluations of how well training activities met their needs. </li></ul>difficult... which model? kinda sorta subjective...
  6. 6. Typical Work Products <ul><li>Training-effectiveness surveys </li></ul><ul><li>Training program performance assessments </li></ul><ul><li>Instructor evaluation forms </li></ul><ul><li>Training examinations </li></ul>
  7. 7. Training Plans managed & controlled Organizational Training Plan Requirements: Global skills needs Project A needs ... Project N needs Courses: Skill 1 date provider resources ... Skill n date provider resources Instructional stds. & procedures Course development schedule Project C Skills needed by job type Skill development strategy Training needs & schedule Training sources Project B Skills needed by job type Skill development strategy Training needs & schedule Training sources <ul><li> Project A </li></ul><ul><li>Skills needed by job type </li></ul><ul><li>Skill development strategy </li></ul><ul><li>Training needs & schedule </li></ul><ul><li>Training sources: </li></ul><ul><ul><li>- project </li></ul></ul><ul><ul><li>- training dept. </li></ul></ul><ul><ul><li>- professor </li></ul></ul><ul><ul><li>- vendor </li></ul></ul>Project training plans PP plan
  8. 8. Expected Behaviors and Training <ul><li>The purpose of training is for people to learn new behaviors. </li></ul><ul><ul><li>there is no sense in learning what you already know </li></ul></ul><ul><ul><li>the new behaviors should make you a better performer in the job </li></ul></ul><ul><li>This is exactly how today’s educators define their training goals </li></ul>
  9. 9. Measuring Effectiveness <ul><li>Effective training achieves educational goals that are aligned with organizational goals </li></ul><ul><ul><li>the problem with adult training is that it has to be Just In Time… </li></ul></ul><ul><ul><ul><li>we forget over 85% of new stuff in two weeks </li></ul></ul></ul><ul><ul><li>On The Job… </li></ul></ul><ul><ul><ul><li>show once, work together once, supervise once </li></ul></ul></ul><ul><ul><li>Just Enough Training! </li></ul></ul><ul><ul><ul><li>we tend to teach encyclopedically, i.e. how the Tres Amigos met to define UML when training for modeling skills </li></ul></ul></ul>our training methods are grossly ineffective
  10. 10. Ineffectiveness Established <ul><li>We train for the wrong reasons, at the wrong time, using inadequate methods </li></ul><ul><ul><li>see Bloom’s Taxonomy of Educational Objectives to find out more about choosing the wrong method </li></ul></ul><ul><ul><ul><li>we usually set up training that works only to the comprehension level and expect results at the application level </li></ul></ul></ul><ul><ul><ul><li>teaching adults how to ride a bicycle with PowerPoint slides </li></ul></ul></ul>
  11. 11. Institutionalized Coaching <ul><li>Train in multiple steps </li></ul><ul><ul><li>build common vocabulary and understanding </li></ul></ul><ul><ul><ul><li>stand up is ok, so is e-learning </li></ul></ul></ul><ul><ul><li>bring the trainer into the team </li></ul></ul><ul><ul><ul><li>individuals learn faster in teams </li></ul></ul></ul><ul><ul><ul><li>makes it OTJ training </li></ul></ul></ul><ul><ul><li>have the trainer perform the task for the team </li></ul></ul><ul><ul><ul><li>train to the problem at hand, JIT-JET </li></ul></ul></ul><ul><ul><li>keep the trainer in the team long enough to warrant the behavioral change </li></ul></ul><ul><ul><ul><li>share one, supervise one. </li></ul></ul></ul>
  12. 12. Back to Measuring Effectiveness <ul><li>We could test right after the class </li></ul><ul><ul><li>adults hate to be tested </li></ul></ul><ul><ul><ul><li>witness the LA certification program! </li></ul></ul></ul><ul><li>Or a few weeks after </li></ul><ul><ul><li>but by then the knowledge could have gone down the drain </li></ul></ul><ul><li>Or use coaching </li></ul><ul><ul><li>our preference </li></ul></ul><ul><li>Or use a closed loop to judge impact. </li></ul>
  13. 13. The Telecom Smart Loop Part 1 <ul><li>OT gathers input to create an Organizational Training Plan from all sectors </li></ul><ul><li>Using their competency model and strategic plans, OT selects candidates to attend classes </li></ul><ul><li>OT sends to each manager a detailed list of all subordinates that could benefit from attending each class </li></ul><ul><li>Managers select the attendees according to their strategic and tactical needs </li></ul>PLANNING AND SELECTION
  14. 14. Competency Modeling <ul><li>Competencies are behaviors that encompass the knowledge, skills, and attributes required for successful performance. </li></ul><ul><li>Competency modeling is the activity of determining the specific competencies that are characteristic of high performance and success in a given job. </li></ul>http://edweb.sdsu.edu/people/arossett/pie/interventions/career_1.htm
  15. 15. Competency and Data Modeling <ul><li>A job has a descriptor. </li></ul><ul><li>A descriptor has one or more competency statements </li></ul><ul><li>A competency has a context, an action to perform , an object to perform the action on, an expected result , and could have a degree of expected prowess. </li></ul><ul><ul><li>a competency describes a behavior </li></ul></ul>
  16. 16. Example From Requirements Engineer JD <ul><li>Interviews project initiators </li></ul><ul><li>Researches current systems related to the requested system or upgrades </li></ul><ul><li>Identifies stakeholders and scopes the requested product </li></ul><ul><li>Identifies user constituencies and documents them </li></ul><ul><li>Identifies and documents business needs for the requested product for each user constituency </li></ul><ul><li>Identifies risks and selects adequate techniques to manage them during the requirements activities </li></ul><ul><li>Describes the requested system behavior </li></ul><ul><li>… </li></ul>
  17. 17. The Telecom Smart Loop Part 2 <ul><li>OT sends out invitations to be completed by candidate attendees and their individual managers </li></ul><ul><ul><li>they have to commit to using the skills picked up in training within two months or there will be penalties </li></ul></ul><ul><ul><ul><li>training is free to the section, unless skills are not used </li></ul></ul></ul>COMMITMENT
  18. 18. The Telecom Smart Loop Part 3 <ul><li>Training is delivered </li></ul><ul><ul><li>students are very keen on picking up skills because they have committed to use them </li></ul></ul><ul><ul><ul><li>poor trainers are quickly detected and swiftly dealt with </li></ul></ul></ul><ul><ul><ul><li>classes have zero tolerance with nonsense </li></ul></ul></ul><ul><ul><ul><li>students do not spend time on email or texting when their career is on the line </li></ul></ul></ul>DELIVERY
  19. 19. The Telecom Smart Loop Part 4 <ul><li>OT sends separate evaluation forms to managers and attendees </li></ul><ul><ul><li>Was the attendee given the chance to use the skill? </li></ul></ul><ul><ul><li>Was there a significant and noticeable change in the behavior of the attendee? </li></ul></ul><ul><ul><li>Is there a material way of showing this improvement? </li></ul></ul><ul><ul><ul><li>no more “beauty contest” survey at the end of a class </li></ul></ul></ul><ul><ul><ul><li>a chance to measure effectiveness in dollars </li></ul></ul></ul>EVALUATION
  20. 20. Benefits of the Closed Loop <ul><li>No testing </li></ul><ul><li>Simple technology </li></ul><ul><li>Full alignment with organizational goals </li></ul><ul><li>Distributed locus of responsibility </li></ul><ul><li>Satisfaction guaranteed! </li></ul>
  21. 21. Reviewing Objectives <ul><li>Are you now able to… </li></ul><ul><ul><li>describe the relationship between competency models and organizational training? </li></ul></ul><ul><ul><li>use a simple, low tech, yet powerful method to measure the effectiveness of a training program? </li></ul></ul><ul><ul><li>Or was it that you were not paying attention?  </li></ul></ul>
  22. 22. Got Questions?
  23. 23. Coordinate Process Activities Process group Communication to affected groups <ul><li>Transfer learning: </li></ul><ul><ul><li>processes </li></ul></ul><ul><ul><li>methods </li></ul></ul><ul><ul><li>tools </li></ul></ul>Process training <ul><li>Org. process </li></ul><ul><li>Proj. process </li></ul>Organization’s standard software process Organization’s standard software process Organization’s standard software process Use and tailoring (Projects) Software process database