The document discusses the relationship between competency models and organizational training, emphasizing the need for effective methods to measure training effectiveness. It outlines various strategies for assessing training outcomes, including surveys and performance evaluations, and highlights the importance of aligning training goals with organizational objectives. Additionally, it introduces the 'telecom smart loop' as a closed-loop approach to ensure commitment and measure the impact of training in a structured manner.

1. On the Effectiveness of Training Boria, Jorge Rubinstein, Viviana

2. Objectives At the end of this presentation, you will be able to… describe the relationship between competency models and organizational training use a simple, low tech, yet powerful method to measure the effectiveness of a training program. But only if you are paying attention. 

3. The CMMI Sayeth… SP 2.3 Assess Training Effectiveness Assess the effectiveness of the organization’s training program. A process should exist to determine the effectiveness of training (i.e., how well the training is meeting the organization’s needs). in lay terms, Are we getting bang for the training buck?

4. Examples of Methods Used Testing in the training context Post-training surveys of training participants Surveys of managers’ satisfaction with post-training effects Assessment mechanisms embedded in courseware

5. Sub Practices Assess in-progress or completed projects to determine whether staff knowledge is adequate for performing project tasks. Provide a mechanism for assessing the effectiveness of each training course with respect to established organizational, project, or individual learning (or performance) objectives. Obtain student evaluations of how well training activities met their needs. difficult... which model? kinda sorta subjective...

6. Typical Work Products Training-effectiveness surveys Training program performance assessments Instructor evaluation forms Training examinations

7. Training Plans managed & controlled Organizational Training Plan Requirements: Global skills needs Project A needs ... Project N needs Courses: Skill 1 date provider resources ... Skill n date provider resources Instructional stds. & procedures Course development schedule Project C Skills needed by job type Skill development strategy Training needs & schedule Training sources Project B Skills needed by job type Skill development strategy Training needs & schedule Training sources Project A Skills needed by job type Skill development strategy Training needs & schedule Training sources: - project - training dept. - professor - vendor Project training plans PP plan

8. Expected Behaviors and Training The purpose of training is for people to learn new behaviors. there is no sense in learning what you already know the new behaviors should make you a better performer in the job This is exactly how today’s educators define their training goals

9. Measuring Effectiveness Effective training achieves educational goals that are aligned with organizational goals the problem with adult training is that it has to be Just In Time… we forget over 85% of new stuff in two weeks On The Job… show once, work together once, supervise once Just Enough Training! we tend to teach encyclopedically, i.e. how the Tres Amigos met to define UML when training for modeling skills our training methods are grossly ineffective

10. Ineffectiveness Established We train for the wrong reasons, at the wrong time, using inadequate methods see Bloom’s Taxonomy of Educational Objectives to find out more about choosing the wrong method we usually set up training that works only to the comprehension level and expect results at the application level teaching adults how to ride a bicycle with PowerPoint slides

11. Institutionalized Coaching Train in multiple steps build common vocabulary and understanding stand up is ok, so is e-learning bring the trainer into the team individuals learn faster in teams makes it OTJ training have the trainer perform the task for the team train to the problem at hand, JIT-JET keep the trainer in the team long enough to warrant the behavioral change share one, supervise one.

12. Back to Measuring Effectiveness We could test right after the class adults hate to be tested witness the LA certification program! Or a few weeks after but by then the knowledge could have gone down the drain Or use coaching our preference Or use a closed loop to judge impact.

13. The Telecom Smart Loop Part 1 OT gathers input to create an Organizational Training Plan from all sectors Using their competency model and strategic plans, OT selects candidates to attend classes OT sends to each manager a detailed list of all subordinates that could benefit from attending each class Managers select the attendees according to their strategic and tactical needs PLANNING AND SELECTION

14. Competency Modeling Competencies are behaviors that encompass the knowledge, skills, and attributes required for successful performance. Competency modeling is the activity of determining the specific competencies that are characteristic of high performance and success in a given job. http://edweb.sdsu.edu/people/arossett/pie/interventions/career_1.htm

15. Competency and Data Modeling A job has a descriptor. A descriptor has one or more competency statements A competency has a context, an action to perform , an object to perform the action on, an expected result , and could have a degree of expected prowess. a competency describes a behavior

16. Example From Requirements Engineer JD Interviews project initiators Researches current systems related to the requested system or upgrades Identifies stakeholders and scopes the requested product Identifies user constituencies and documents them Identifies and documents business needs for the requested product for each user constituency Identifies risks and selects adequate techniques to manage them during the requirements activities Describes the requested system behavior …

17. The Telecom Smart Loop Part 2 OT sends out invitations to be completed by candidate attendees and their individual managers they have to commit to using the skills picked up in training within two months or there will be penalties training is free to the section, unless skills are not used COMMITMENT

18. The Telecom Smart Loop Part 3 Training is delivered students are very keen on picking up skills because they have committed to use them poor trainers are quickly detected and swiftly dealt with classes have zero tolerance with nonsense students do not spend time on email or texting when their career is on the line DELIVERY

19. The Telecom Smart Loop Part 4 OT sends separate evaluation forms to managers and attendees Was the attendee given the chance to use the skill? Was there a significant and noticeable change in the behavior of the attendee? Is there a material way of showing this improvement? no more “beauty contest” survey at the end of a class a chance to measure effectiveness in dollars EVALUATION

20. Benefits of the Closed Loop No testing Simple technology Full alignment with organizational goals Distributed locus of responsibility Satisfaction guaranteed!

21. Reviewing Objectives Are you now able to… describe the relationship between competency models and organizational training? use a simple, low tech, yet powerful method to measure the effectiveness of a training program? Or was it that you were not paying attention? 

22. Got Questions?

23. Coordinate Process Activities Process group Communication to affected groups Transfer learning: processes methods tools Process training Org. process Proj. process Organization’s standard software process Organization’s standard software process Organization’s standard software process Use and tailoring (Projects) Software process database