Overview of the SAT Process


Published on

Published in: Business, Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Overview of the SAT Process

  1. 1. Overview of the SAT Process:The Systematic Approach toTrainingWhat is SAT?How does it improve performance?How is it implemented?What are the 5 phases of SAT?How is each stage conducted?Expert Mission on Workforce PlanningEGAT Thailand Johanna P. BishopDecember 2009 Behavioral Science Program Coordinator/ Professor Wilmington University, Delaware USA Doctoral Candidate The George Washington University Washington DC, USA
  2. 2. What is SAT?SAT refers to the Systematic Approach to Training- training that is based on a demonstrated need based on worker and/or plant performance
  3. 3. Why Use SAT?• Nuclear power is a complex industry that requires highly qualified workers• Employee competence and work complexity demands a high quality training program• Training is at the heart of learning in highly complex organizations• Workplace competence involves more than the technical skills – it involves understanding the interrelationships of the organization, process and worker• SAT process includes the involvement of technical experts, management, and training personnel
  4. 4. Advantages of SATSAT is accountable because it is measuredSAT provides continual feedback as the results of training are evaluatedSAT supports teamworkSAT is cost effectiveSAT permits effective management control because measurable objectives are established and performance is evaluated and documented regularly, those responsible for training can measure and correct performanceSAT is performance based training intended to improve worker performance and measures success on the basis of improved personnel and plant performance
  5. 5. 5 Phases of the SAT Process Analysis• SAT Process: 5 Phases Design 1. Analysis 2. Design Development 3. Development 4. Implementation Implementation 5. Evaluation Evaluation Feedback Loop
  6. 6. Analysis Identifies hazards and requirements of the job/ task; examines differences between desired and actual performance• Analysis is the first phase of the Instructional Design, or Systematic Approach to Training, process• In the analysis phase, the end goal is to know IF and WHY training is needed as an intervention to solve a performance problem• There are several types of analysis, and each one has a different focus – NEEDS ANALYSIS • JOB ANALYSIS • TASK ANALYSIS
  7. 7. Design A plan that guides the creation of all training materials and strategies• The Design Phase is the second phase of the Instructional Design, or Systematic Approach to Training process – The design phase uses the results from the analysis phase to design the overall structure of the training intervention – This is the phase where the training instructor takes all the information he/she now knows about the performance and the performer and designs how to best meet the needs for improving and achieving the desired performance• The outputs of the design phase are: – Approve the task list and performance standards – Select the training topics – Determine trainee entry level requirements – Develop performance objectives – Sequence the objectives – Select instructional methods, settings, and media
  8. 8. DevelopmentCreation of training documents and materials, props, and assessment tools• The Development Phase is the third phase of the Instructional Design, or Systematic Approach to Training process• The development phase uses the outputs from the design phase to actually create the training materials – In this phase, the instructor conducts research on the subject matter, resulting in developing lesson plans and training materials – Both resource materials for the instructor and lesson materials for the students are developed in this phase• The outputs of the development phase are: – Create a lesson plan – Create qualification instruments – Create training program description – Create lesson materials for the trainer – Create lesson materials for the trainees
  9. 9. Implementation Training is delivered and learning is assessed• The Implementation Phase is the fourth phase of the Instructional Design, or Systematic Approach to Training process• The implementation phase uses the outputs from the development phase to actually conduct a training tryout• In this phase, the instructor conducts a pilot teach on the subject matter• The outputs of the implementation phase are: – Implement the training plan – Conduct a training program tryout – Conduct evaluation of the training program tryout – Document the training
  10. 10. Evaluation Training effectiveness is evaluated and continuous improvement is implemented• The Evaluation Phase is the fifth phase of the Instructional Design, or Systematic Approach to Training process• The evaluation “…ensures training’s continuing ability to produce qualified employees” (INPO, 1993)• Good instructors constantly monitor students’ nonverbal feedback and perform in-training evaluations• This allows them to adjust as necessary while delivering the lesson plan• In addition, there is also the phase of collecting student feedback after the training is completed• Instructors who follow up by monitoring student performance once they are back on the job are able to gather even more evaluation information• The evaluation phase collects feedback and uses that information to redesign the lesson plan• The outputs of the evaluation phase are: • Monitor indicators • Analyze information • Initiate corrective actions
  11. 11. SAT Phases In TrainingNow let’s discuss each phase of the SAT process for the training function in more detail…
  12. 12. Analysis Phase• In the analysis phase, the end goal is to know IF and WHY Analysis training is needed as an intervention to solve a performance problem Design• There are several levels of analysis, and each one has a different focus Development – NEEDS ANALYSIS • JOB ANALYSIS • TASK ANALYSIS Implementation• Needs analysis is usually conducted by an in-house trainer or a consultant Evaluation• Person should be trained on how to conduct a needs analysis• Must have ability to be objective Feedback Loop and inquisitive
  13. 13. Analysis Phase Activities• Needs Analysis: purpose is to ensure that management initiatives, in addition to training, are considered in satisfying performance requirements and deficiencies• Needs analysis answers 4 basic questions: 1. What are workers doing? 2. What should workers be doing? 3. What is causing the gap, if any, between actual and desired performance? 4. Would training be part of the solution?
  14. 14. Analysis Phase Activities- 2• Needs analysis is conducted whenever a request is made for training – May be due to new job, task, equipment, change in process, performance, etc – May be due to change in job such as additional responsibilities (i.e. promotion from line to supervisor, or added scope of job or task) – May be due to changing standards in performance expectations, or performance deficiencies
  15. 15. Analysis Phase Activities- 3• Training can help if the needs analysis shows a lack of knowledge or proficiency (lack of practice)• The key role of the line in the analysis phase is to provide information related to the issues or causes
  16. 16. Analysis Phase Activities Outputs• Results of the Needs Analysis: – Clarification of • Management expectations • Possible management intervention • Administrative interventions such as revising procedures or changing job requirements • Training to increase knowledge, skills, abilities or to change attitudes • Verified task list
  17. 17. Conducting a Needs Analysis Needs Analysis process identifies a need for training• Needs Analysis: – Perform Fact Finding • Search for information as to the need for training – Does it exist for key positions? For tasks that impact plant safety and performance? Difficult tasks? Shortages of qualified personnel? Major changes in job scope? Regulatory requirements? Plant modifications? Evidence of performance deficiencies? – Assess needs • Assess training and management initiatives; is there a gap? – Submit training development recommendation – Develop work plan • A plan that clarifies the requirements, schedule, and budget needed to ensure controlled development
  18. 18. Conducting a Needs Analysis• Needs Analysis techniques include: – Conduct direct observation – Survey questionnaires – Consult with persons in key positions, and/or with specific knowledge – Review relevant literature – Conduct interviews – Conduct focus groups – Administer tests – Examine records & report studies – Review work samples
  19. 19. Job Analysis Procedures Performed WhenConduct Job Analysis Conducting Job Analysis• Job incumbents and their • Review and select existing supervisors share their job data knowledge and experience • Indentify work attitudes with training system • Analyze the job personnel to develop a • Select tasks and performance detailed list of tasks indicators for analysis required to perform a • Identify tasks and particular job, duty area, performance indicators for or task initial and continuing training
  20. 20. Task Analysis Procedures PerformedConduct Task Analysis During Task Analysis:• The process of dividing a • Prepare for task analysis unit of work (task) into • Perform procedural task manageable steps analysis• Identifies task standards • Perform paradigm task• Identifies conditions analysis under which the task must • Perform attitude analysis be performed • Check task analysis for• Identifies the skills and quality knowledge required by • Group common skills and employees who perform knowledge the task
  21. 21. Performing A Task With Precision
  22. 22. DIF Ratings Tasks are rated according to difficulty, importance, and frequency• Information is • Job incumbents collected from: provide input – SME input • Tasks needing training – Table top sessions are identified – Questionnaires • Line management – Interviews reviews / approves
  23. 23. Dif Ratings- 2 Difficulty Importance Frequency• How difficult is the • How important, or • Frequentjob? critical, is the job? • Potential for• Can it be mastered • What are the initial trainingwith training? consequences of • Infrequent inadequate • Potential for performance? continuing • Unit scram? training • Personnel injury? • Equipment damage?
  24. 24. Summary of Analysis Phase• Needs analysis identifies human resource requirements and performance deficiencies requiring training• Job analysis is conducted to define the requirements further by identifying tasks that comprise job performance• Task analysis is conducted on the tasks selected to identify task conditions, standards, elements, and skills and knowledge required to perform the task satisfactorily
  25. 25. Design Phase• Design is primarily a Training Department responsibility Analysis• Begins to form the foundation for the course material• If the needs analysis shows a Design deficiency in knowledge, then the best option is to conduct training in a classroom setting Development• If the needs analysis shows a deficiency in skill, then the best option would be skill-based Implementation training – Done in a lab, simulator, or in the plant Evaluation• Qualified training personnel in consultation with Subject-Matter- Experts Feedback Loop
  26. 26. Design Phase Activities1. Develop job performance measures2. Determine appropriate training settings3. Prepare learning objectives4. Develop test items5. Describe expected trainee entry-level skills and knowledge6. Organize learning objectives7. Construct tests8. Develop training plan
  27. 27. Design Phase Outputs• Determines: – An approved training summary or plan – Learning objectives – Examination, evaluation, and test-out requirements – A task to training matrix
  28. 28. Job Performance Measures• First step in the design phase is to develop the job performance measures, which are the performance tests prepared at the task level• JPM are used: – To measure adequacy of a plant employee’s performance of a task on the job or during training – To provide the basis for development of a terminal objective, which is a training goals expressed in terms of measurable trainee performance• JPMs are derived from tasks and task elements
  29. 29. Determine the Training Setting• After job performance measures are developed, determine the training setting – Classroom: determine by the need to impart large amount of information; large group of trainees; or other training settings not suitable – Lab or workshop: tasks require hands-on practice to achieve mastery – Formal-on-the-job training: small groups of trainees; qualified personnel are available to manage and control this training; no critical resource constraints on plant equipment – Simulator: high fidelity training is required for trainees to achieve mastery; problem diagnosis under stressful conditions is integral part of the training; teamwork part of task performance – Self-study: training does not require close supervision; all conditions of the training can be met in the training materials; task does not require extended periods to achieve mastery (does not require lots of feedback)
  30. 30. Prepare the Learning Objectives• Learning objectives describe what is to be learned in in terms of measurable trainee performance• Learning objectives define the content of the training program• A properly prepared learning objective consists of the following: 1. Statement of behavior (action) the trainee must exhibit 2. Conditions under which the action will take place 3. Standards for satisfactory performance
  31. 31. Levels of Learning Objectives Learning objectives must specify behavior, the conditions under which the action must take place, and the standards of satisfactory performanceTerminal Condition Behavior• Given a bicycle and bicycle tools, the trainee must remove the rear bicycle wheel and install a new wheel so as to make the bicycle fully operational so that it can be ridden without the wheel falling off. StandardEnabling Objectives Standard• Remove the rear wheel• Install the new rear wheel• Test drive the bicycle to make sure the wheel stays in place
  32. 32. Develop Test Items• A test item bank is compiled• These test items are used to prepare tests used during training• Test items must match up to the objectives of the lesson plan• Tests and test items must be kept secure
  33. 33. Describe Entry-Level Skills and Knowledge• Pretest is used to confirm individual trainee qualifications for entering the training program• May also be used to identify remedial training requirements• Progress test is used during training to measure trainee progress• Post-test evaluates trainee’s successful completion of the training program
  34. 34. What Skills & Knowledge Do WorkersNeed to Possess?
  35. 35. Organize Learning Objectives• Organize the learning objectives into a sequence• Effective sequencing ensures that the training can be completed in the shortest amount of time• Also helps trainees make the transition from one learning level to another
  36. 36. Construct Tests• Test bank prepared during test item development is used to prepare tests to be used during training• Tests used during training include pre-test, progress test, and post-test
  37. 37. Develop Training Plan• The training plan addresses the organizing, planning, and administering the training program• Provides guidance for integrating resources: – Trainees – Instructors – Support services – Facilities – Equipment• Plan should be flexible and provide for coordination from all units responsible for the training
  38. 38. Summary of the Design Phase• Activities from the design phase produce job performance measures from tasks selected for training• They are used to measure the adequacy of the plant’s performance• Training settings are selected to provide the appropriate environment for learning to occur
  39. 39. Development Phase• The development phase is where specific training materials are developed Analysis• The training setting, learning objectives, and the training plan Design are used to determine instructor and trainee activities and to select and/or develop supporting materials for the Development training program• Development phase is conducted Implementation by qualified training personnel who are skilled in developing learning materials Evaluation• They work with the Subject- Matter-Experts to define the specific tasks that must be mastered by the trainees Feedback Loop
  40. 40. The development phase tells us “how” we will teach the material• An output of the development phase is a lesson plan The lesson plan serves as a “training script” for the instructor• Also included are the development of instructor-led activities to help the learners learn and examination material that will help us determine, “did they learn?”
  41. 41. Development Phase Activities• There are 5 major development phase activities: 1. Specify learning activities 2. Develop training materials 3. Select training methods 4. Develop lesson plans 5. Conduct training program tryout
  42. 42. Development Phase Outputs• Approved training materials – Lesson plan – Presentation materials – Assessment tools
  43. 43. Summary of the Development Phase• Development phase activities address trainee learning experiences• It also addresses the learning activities used to accomplish the training objectives• High cost of training materials requires that close attention be given to the collection, review, and selection of existing training materials• Data acquired in the pilot teach of the training program is used to determine technical accuracy and training effectiveness and guides any needed revisions
  44. 44. Implementation Phase• Implementation is the process of putting training programs into operation Analysis• It is the culmination of systematic planning and development in which Design trainees and the utility derive benefits of previous efforts through effective training Development• Implementation phase is conducted by qualified training personnel who are Implementation skilled in developing learning materials• They work with the Subject- Evaluation Matter-Experts to define the specific tasks that must be mastered by the trainees Feedback Loop
  45. 45. Implementation Phase Activities• There are 4 major implementation phase activities: 1. Implement the training plan 2. Conduct training 3. Conduct in-training evaluation program effectiveness 4. Document training
  46. 46. Implementation Phase Outputs• Trained employees• Documented with: – Attendance records – Assessment results – Evidence of appropriate instructor qualification – Individual training records
  47. 47. Classroom Training at Salem/ HopeCreek
  48. 48. Lab/ Shop TrainingLab/ Shop Training
  49. 49. One-on-One Training
  50. 50. Summary of Implementation Phase• Implementation activities ensure that trainees, facilities, an d resources are available when training is scheduled to start• Instructors are prepared to conduct training• Trainees are tested to determine their knowledge levels• Lesson plans are followed and adhered to during training• Trainees are continually evaluated• Training is documented
  51. 51. Evaluation Phase• To ensure that the training function produces competent Analysis employees, training programs must be monitored, evaluated, and revised as Design necessary• Program evaluation focuses on the results of the training Development program, not the process of training• Evaluation is the Implementation determination of a training program’s effectiveness in producing competent Evaluation employees• Evaluation phase is conducted by line management and training Feedback Loop
  52. 52. Levels of Training Evaluation Ways to measure the results of training- the Kirkpatrick Model FOCUS HOW TO MEASUREWere business results Measures againstachieved? performance, or business, RESULTS goals Plant performanceWere the skills & processes Assessment toolstaught in training Observationsconsistent with those used BEHAVIOR 360 degree feedbackon the job? Behavior surveysDid participants learn the Mastery testsconcepts and skills taught KNOWLEDGE Verbal reviewin the program?Did trainees react favorably Participant attitudeto the training? REACTION Questionnaires Interviews
  53. 53. Evaluation Phase Activities• There are 3 major evaluation phase activities: 1. Monitor indicators 2. Analyze information 3. Initiate corrective actions
  54. 54. TECHNIQUES USED IN THE EVALUATION PHASE 1. Monitoring Indicators• Understanding plant performance is to also monitor indicators• Indicators may include: – Plant operating, maintenance and industrial safety experiences – Employee and supervisor feedback – Plant inspection, evaluation, and accreditation reports – Plant modifications and procedure changes – Industry operating and maintenance experiences – Regulatory developments – In-training evaluation data (instructors and trainees)
  55. 55. TECHNIQUES USED IN THE EVALUATION PHASE 2. Analyzing InformationBefore selecting an analysis • Construct Frequency method, ask these 3 Distributions: questions: • Collecting data which can be converted to countable data:1. What information is • Surveys needed? • Reports2. What type of • Trainee performance information will be • Construct FD using simple counting, averaging, and analyzed? graphing procedures3. What is the simplest • Will show how often method of analysis that particular events have will yield the occurred information required?
  56. 56. TECHNIQUES USED IN THE EVALUATION PHASE Analyzing Information- 2Exception Analysis Content Analysis• Used for reviewing data to • Depends upon the detect unacceptable variations from a predefined expertise and professional standard judgment of individuals• Steps to conducting: performing it 1. Define standard 2. Determine actual • Data is evaluated performance results subjectively to identify 3. Compare performance to possible training problems standard 4. Investigate below standard • Identify root causes performance to determine cause(s)
  57. 57. TECHNIQUES USED IN THE IMPLEMENTATION Initiating Corrective Actions• When analysis results confirm program effectiveness, no corrective actions are needed – Analysis results should be retained to document evaluation activities – Indicators should continue to be monitored• If performance discrepancy is discovered and analysis confirm that training can help, then action to correct the problem should be initiated
  58. 58. Evaluation Phase Outputs• Output of the evaluation phase serves as feedback (input) for the analysis, design, development, implementation phases so the training program can be modified and improved• Revised or current training materials, strategies, or settings• Records of evaluation results• Records of approved changes made to training and a description of the considerations or drivers of the changes
  59. 59. Summary of the Evaluation Phase• Evaluation consists of activities used to ensure training system capability to produce competent employees• “Closes the loop” on job and task analysis, training, and job performance• By monitoring key performance indicators, evaluation helps to confirm program effectiveness, detect deficiencies, initiate corrective actions, and maintain program quality
  60. 60. Line Management’s Role in Training Why line management must be involved in the SAT process…How line management is involved in the SAT process…
  61. 61. Management Observations of Line Workers Help to Ensure the Quality of TrainingKey Elements of management observations include: Make Substantive Comments Prepare for the Observation Conduct the Observation Follow-up/Coaching/Feedback
  62. 62. Training to Procedural Compliance
  63. 63. Process Students Managers & InstructorsKey Roles in Station Training Supervisors • Identify training • Communicate expectations •Internalize management expectations Analysis •Exhibit ownership of plant needs • Address performance shortfalls • Provide expert • Provide recommendations performance • Internalize SAT fundamentals •Apply human performance principles technical advice •Seek solutions to performance • Identify training needs • Participate in job shortfalls and task analysis •Exercise SAT fundamentals •Advocate valid training solutions • Participate in • Lead training committees •Integrate management expectations Design •Select training methods and media training committees •Approve learning objectives •Review tests and related materials •Maintain task-to-training matrix • Comment on •Approve selected training materials •Produce stimulating training materials Development •Maintain training materials current proposed training • Participate in • Observe training •Model management expectations Implementation •Emphasize operating experience scheduled training •Evaluate instructor performance •Master learning •Evaluate worker and staff performance •Deliver quality training and •Attend continuing training remediation objectives •Attend continuing training •Facilitate exercise critiques •Apply learning on •Prepare for presentations •Monitor task performance evaluations the job •Use approved materials •Enforce qualification standards •Award qualifications •Facilitate learning •Apply qualifications to work assignment •Evaluate students •Document results • Provide critical •Provide critical feedback •Embrace critical feedback Evaluation feedback •Sponsor self-assessments •Engage in critical, ongoing self- • Complete training •Ensure timely corrective actions assessments surveys •Integrate training and line indicators •Achieve timely closure on corrective • Initiate training •Support timely revisions actions requests •Provide oversight
  64. 64. Supervisor Oversight ReinforcesPerformance
  65. 65. Working on the Salem Water boxes
  66. 66. Management Observations of TrainingManagement Observations of Challenges to ManagementTraining Observations of Training• Management needs to • Demands on manager’s observe training and time complete training observation forms – Management involvement in training means managers also have to observe training – Managers need to document their observations of training for feedback purposes, and – Establishing record trail
  67. 67. Human Performance Management & training emphasize human performance… Human Error Traps Human Error ToolsTime pressure Effective communicationDistractions Questioning attitudeMultiple tasks Self checkingOverconfidence Peer checkingVague or interpretive KnowledgeguidanceMental stress Procedures Job briefing Turnover
  68. 68. SAT and the Importance of Trained and Qualified Personnel Train-the-Trainer Model… Based on maintaining qualifications…
  69. 69. Train-the-Trainer Model Developing qualified training instructors…• Subject-Matter-Expert • Master Training is trained to become a Instructor training instructor (Instructional• Must go through Initial Technologists/ Instructor Training to Learning become qualified as a Technologists) have training instructor higher education• Maintaining instructor degrees specializing in quals requires learning or Instructor Continuing development Training
  70. 70. Quality of Training Qualified instructors are the core of training…• Training is timely • Training is – Training occurs at an instructionally appropriate time to effective & challenging support changing job – Uses instructional performance needs techniques that involve students in the learning – Training sequence process and intellectually supports scheduled plant challenge students to evolutions and individual think and learn job assignments• Training materials are technically correct and up-to-date
  71. 71. Industry Guidance for the SAT ProcessINPO 85-006, Principles of Training System DevelopmentACAD 92-004, Guideline for the Conduct of Training and Qualification ActivitiesDOE-HDBK-1078-94 Training program handbook: A Systematic Approach to TrainingIAEA-TECDOC-1170 Analysis phase of Systematic Approach to Training (SAT) for nuclear plant personnel
  72. 72. Interconnected Performance Understanding SAT is important to all personnel; whether they are training instructors, developers, supervisors, or learners, or whether they are plant managers or supervisors Organization ProcessOptimumPerformance Worker