Surviving troubled times (villachica & stepich, 2010)

  • 1,175 views
Uploaded on

Surviving troubled times (villachica & stepich, 2010)

Surviving troubled times (villachica & stepich, 2010)

More in: Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,175
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Surviving Troubled Times: Five BestPractices for Training ProfessionalsSteven W. Villachica, CPT, PhD, and Donald A. Stepich, PhD With the current economic down-T he training business has always been cyclical, turn and signs of an emerging recovery, with the fortunes of human resources (HR) and executives are trying to determine how training professionals rising and falling with to best use their organizations’ fundsthe economic fortunes of the workplace. As of 2008, and resources. This may mean down- sizing human resource departmentstraining is a $56.2 billion industry (Training, 2008), and eliminating positions for trainingwith funding moving toward ‘‘programs that are either personnel. The authors offer five stra-required or directly related to a learner’s job or profes- tegies drawn from the professional lit- erature to survive these and futuresion. With tighter budgets, organizations are targeting trying times: (1) align efforts with orga-their efforts to meeting compliance requirements and nizational missions and business goals,improving skills that are highly specific to a learner’s (2) use training only when it addresses ajob’’ (p. 22). A recent survey sponsored by the Amer- gap between existing and desired per- formance arising from a lack of requisiteican Society for Training and Development (ASTD) skill, (3) craft instructional objectivesfound that workplace learning and performance bud- that describe exemplary job perfor-gets had ‘‘withstood the challenges of a difficult econ- mance, (4) create sound training pro-omy’’ between 2007 and 2008 (Paradise & Patel, 2009, grams that promote learning and transfer to the job, and (5) collaboratep. 4). This familiar situation sounds much like the one with sponsors and other stakeholdersthat Rosenberg (1982) described over 25 years ago: outside the training department to pro- mote transfer of training to the job. Today, more than ever, training professionals Training personnel who employ these strategies successfully may be able to must demonstrate the effectiveness of their pro- answer executives’ common question, grams, and, even more important, are held ac- ‘‘What have you done for me recently countable for their training decisions. Spending that matters?’’ money on unnecessary training, inappropriate training or training that doesn’t train can spell disaster for those who design such programs. Training departments can no longer afford the luxury of ‘‘training by whim.’’ Courses can no longer be reflections of an individual’s ‘‘idea’’ of what should be taught and how such training should be organized and delivered. (p. 44) The current economic downturn and nascent recovery also represent anopportunity for training professionals to refocus their efforts to improve thevalue they offer their organizations. In this case, it means looking for 93PERFORMANCE IMPROVEMENT QUARTERLY, 23(2) PP. 93–115& 2010 International Society for Performance ImprovementPublished online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/piq.20083
  • 2. opportunities to achieve results that matter to executives, managers, and other stakeholders who control budgets and resources. As depicted in Figure 1, this article argues that training professionals best survive troubled times and position themselves for recovery by producing on-the-job beha- vioral change and measurable performance results that meet business goals. To this end, training professionals should adopt several best practices that appear in the professional literature: 1. Align their efforts with organizational missions and business goals. 2. Use training only when it addresses a gap between existing and desired performance arising from a lack of requisite skill. 3. Craft instructional objectives that describe exemplary job perfor- mance. 4. Create sound training programs that promote learning and transfer to the job. 5. Collaborate with sponsors and other stakeholders outside the train- ing department to promote transfer of training to the job. Following these practices will help executives determine their returns on training investments, managers see on-the-job behavioral change that leverages the performance of exemplary personnel, and trainees confidently apply relevant skills and knowledge to the workplace. Executives, managers, and trainees who see the value of the training they complete will be less likely to cut training budgets and eliminate the positions of training professionals. Aligning Training With Strategic Business Objectives Aguinis and Kraiger (2009) maintained that training ‘‘for the sake of training . . . is not aligned with today’s business realities’’ (p. 466). Tannenbaum and Yukl (1992) noted that the need to link training to organizational strategy appears as a recurring theme in the practitioner literature. Paradise (2008) reported that strategic plans and corporate objectives drove most learning and performance activities of ASTD’s award-winning BEST organizations. Each of these authors makes the same point: training courses should support the strategic direction of the organization, and training objectives should align with organizational goals, as depicted in Figure 1. Providing unaligned training puts the training department itself at risk of irrelevance. Allen (2003) illustrated this situation with the following anecdote: ‘‘‘Guess what?’’’ one executive says to another: ‘Training did just fine with their reduced budget. I don’t see any difference, really. Of course, they’re complaining that they didn’t have enough resources to do it right, but it seems we’re getting by just as well as before. Maybe we can cut training a little more! It doesn’t seem to matter’’’ (p. 6).94 DOI: 10.1002/piq Performance Improvement Quarterly
  • 3. Note. From ‘‘First Principles of Instruction,’’ by M.D. Merrill, 2002, Educational Technology Research andDevelopment, 50, p. 45. Copyright 2002 by Springer. Reprinted with permission. FIGURE 1. FIVE BEST PRACTICES FOR TRAINING Reviewing the results of over 400 impact studies they had conducted,Phillips and Phillips (2002) identified 11 reasons that training and develop-ment efforts fail. The most common was a lack ofalignment with business needs. Organizations of-ten implement training for the wrong reasons, such Providing unalignedas trends, desires, or other perceived needs. With- training puts the trainingout a link to business goals and organizational department itself at risk ofmissions, training professionals cannot tie their irrelevance.efforts to any sort of improvement in the busi-ness—let alone executive expectations regarding a return on their invest-ments of scarce organizational resources. Executives tend to look at suchunaligned training as expenses to cut rather than an investment resulting inincreased revenues, decreased costs, more customers, improved quality, andthe like. Such unaligned training often suffers a lack of organizationalsponsorship throughout its life cycle—however long that may be. Duringtraining development, this situation can result in underfunding, lack ofaccess to subject matter expertise (exemplary performers), inadequateschedule and release time to complete the project, and inadequate buy-inof the changes that the training might bring to the job. Resources and funds tomaintain such unaligned training are rarely forthcoming. Unaligned trainingcan also adversely affect the credibility of the training department and thecareers of its personnel (Stone & Villachica, 2006). With no sponsorship forthe development of unaligned training, it becomes difficult, if not impossible,Volume 23, Number 2 / 2010 DOI: 10.1002/piq 95
  • 4. to demonstrate the overall impact of a training program to the sponsor who funded it—let alone the rest of the organization. A Continuum of Training Approaches Robinson and Robinson (1990) built on this notion of alignment todescribe a continuum of training approaches, with training professionals and their efforts lying somewhere between two extremes: training for activity and training for impact. Figure 2 compares the two approaches. When training professionals adhere to the activity approach, they often respond to a formal request for training or some sort of ‘‘symptom’’ that someone feels training should address: a perceived need for increased awareness, improved attitudes, or vague, nonspecific changes in behavior, for example. Training professionals adhering to the activity approach measure their effectiveness in terms of the speed of their response to a request for training and the number of requests they fill annually. That the response may be well intentioned but ill conceived is epiphenomenal. In the past, these professionals may have received reinforce- ments for increasing the variety and number of training programs they offered in the form of increasing staff, budget, and resources. With responsiveness and program variety as paramount concerns, such professionals typically do not: ~ Determine whether there is a gap between existing and desired job performance worth closing to solve a problem or seize an opportunity. ~ Align the request for training with business goals and organizational missions. ~ Ensure there is adequate sponsorship to assess the cause of the performance gap. ~ Ensure adequate sponsorship to create any resulting training, deliver it, and ensure its sustained transfer to the workplace. Consequently, the activity training these professionals create often consists of ‘‘information dumps,’’ ‘‘moral diatribes,’’ and other training ‘‘events’’ that have no relationship to what performers do in the workplace. Activity training typically occurs in isolation from the workplace, with managers providing little or no support for their subordinates to attend Source. Robinson and Robinson (1990). FIGURE 2. CONTINUUM OF TRAINING APPROACHES96 DOI: 10.1002/piq Performance Improvement Quarterly
  • 5. the training or apply what they learned on their jobs. Word spreads amongtrainees and managers that the training is boring, annoying, or irrelevant.Any evaluation of activity training often consists of a smile sheet measuringtrainee reactions to the training. Robinson and Robinson (1990) contend thatthis activity-based approach lends itself to criticism that training profes-sionals lack business savvy. Allen (2003) described a similar training approach: to-do-list projectswhere people in the organization are not doing what they should be doing.The training department (or someone else) consequently receives theassignment to create and deliver training. Someone sets a budget, and theclock begins to tick down to the rollout date for delivering the training. Thegoal is to get something done on time and within budget. After announcingthe availability of the training and delivering it, the training group can crossthe assignment off the to-do list and move on to other projects.Training for Impact At the other end of the continuum, training professionals could opt totrain for impact, tying their efforts to business needs. Such needs travel undera variety of aliases, including ‘‘business goals,’’ ‘‘initiatives, strategic businessobjectives,’’ ‘‘problems,’’ and ‘‘opportunities.’’ Regardless of the term, suchbusiness goals tend to be what keep executives and senior managementawake at night. In industry settings, an aligned training program links tobusiness goals. In government, military, and other nonprofit settings, alignedtraining links to organizational missions. When goals or missions are vagueor numerous, prudence suggests aligning training efforts where the largerorganization is spending its budget and committing its resources. A jointresearch project of the ASTD and the U.S. Department of Labor (Carnevale,Gainer, & Villet, 1990) recommended that training professionals shouldalign their efforts with larger organizational strategies and business goalsinvolving these areas: ~ Concentration of products, markets, and technologies in ways that increase market share, reduce operating costs, or create and maintain a market niche. ~ Gradual and steady growth through an emphasis on the organiza- tion’s strengths by developing markets, products, innovations, or joint ventures. ~ External growth through acquisition of other organizations to inte- grate new resources with existing capabilities or diversify holdings. ~ Disinvestment by trimming back and refocusing resources through retrenchment, turnaround, divestiture, or liquidation. Rivera (2007) also argued that training professionals should formulateways to help an organization’s executives and managers achieve theirbusiness goals. He suggests finding areas where training can leveragestrengths and shore up organizational weaknesses. Opportunities to aligntraining with business goals typically lie in areas listed in Table 1.Volume 23, Number 2 / 2010 DOI: 10.1002/piq 97
  • 6. TABLE 1 OPPORTUNITIES AND MEASURES APPEARING IN RIVERA (2007) FOR ALIGNING TRAINING WITH BUSINESS GOALS OPPORTUNITY FOR ALIGNMENT DEFINITION EXAMPLES Improved productivity Greater ability to produce an Output of goods, services, or increase in items that can other items in the organization’s immediately be exchanged for inventory cash or booked as future sellable assets Increased organizational A perceived future competitive Scalability of business processes, capacity and growth advantage based on talents the with the ratio of inputs and organization develops internally outputs compared to expected rather than acquires costs Improved ability to seize Excess organizational capacity Adapting, retooling, and opportunities within short and human competencies refocusing to enter markets time frames (organizational and the ability to quickly quickly and enjoy first-mover nimbleness) deploy them advantages Reduced scrap and rework Reduction of defective products Disposal costs, efficiency of or scrap to a level below either production processes, and the organization’s existing or minimization of accident rates industry benchmarks Increased customer satisfaction Overall product quality and Barriers to entry for competitors, customer satisfaction market share, justification for price premiums, and customer loyalty Increased employee satisfaction Employees’ attitudes about their Employee morale; willingness to and retention job placement and work perform; team cohesion; and lost challenges productivity due to accidents, sickness, and undocumented absences Increased time and focus on Increased percentage of billable Historical data for estimating value-producing goals project time or improved costs, cycle time, and resource accuracy of time reports, requirements maximized value creation, and associated time allocation Lower operational and Reduced operational risks due to Compensation of inconvenienced franchise risks employee negligence or customers, customer loss, delays carelessness and franchise risks in product launches, or the caused by employee recklessness, prevention of key deliverables sabotage, misconduct, or from reaching the customer malevolence Reduced poor management Improved speed and quality of Reduced direct and indirect costs decisions decisions, decreased cost of for the decision taken, the next obtaining information best course of action, and alternatives not taken Improved succession planning Reduced employee replacement Development time and cost costs and lost productivity— required to achieve a compounded over time if the predecessor’s level of position remains unfilled proficiency98 DOI: 10.1002/piq Performance Improvement Quarterly
  • 7. Alignment Facilitates Evaluation for Impact Aligning training with business goals and missions also facilitatesevaluation efforts to determine the impact of the training and the organiza-tion’s return on its expectations and monetary investment. A prevailingmisconception is that organizations should apply levels of evaluation in theKirkpatrick and Kirkpatrick (2006) and the Phillips and Phillips (2007)models in their listed order. In other words, practitioners evaluating atraining course would collect data that would enable them to conduct ananalysis of level 1 reactions, then a level 2 analysis of learning, then a level 3analysis of on-the-job behavior, followed by a level 4 analysis of theorganizational impact of the training, and, potentially, a level 5 analysis ofthe organization’s return on its training investment. Training professionals often expect to implement a program of evalua-tion using the same approach. They begin by creating level 1 reaction surveyswith the expectation that they will someday reach a point where they areconducting a significant number of level 4 and 5 evaluations. Theseexpectations remain largely unmet. Sugrue and Rivera (2005) reported thatorganizations in ASTD’s benchmarking forum measured trainee reactions in91% of programs, learning in 54% of programs, transfer to the job in 23% ofprograms, impact in 8% of programs, and return on investment (ROI) for 2%of all training programs. Among other causes, this drop-off of evaluation bylevel may indicate a widespread lack of training alignment, along with acorresponding lack of sponsorship for both the training and its evaluation. It is difficult, if not impossible, to observe transfer of learned skills on thejob, determine its impact, or calculate its return on investment when thetraining is unaligned with an organization’s mission or business goals.Unaligned training typically brings a dearth of organizational sponsorship,leaving training professionals largely unable to marshal the resourcesrequired to collect data, analyze them rigorously, and report trustworthyfindings to sponsors and shareholders. Rather than planning on implementing level 1 to 4 evaluations in thatorder, Kirkpatrick and Kirkpatrick (2006) argued that training professionalsshould begin their planning with desired results, determined in cooperationwith managers at various levels of the organization, then determine desiredbehaviors and their corresponding knowledge, skills, and attitudes. The lasttask is to present the training program in a way that enables both learning andfavorable reactions. This planning approach reverses the order of theevaluation levels. Kirkpatrick and Kirkpatrick (2009) recommended thattraining professionals work with executives and managers to negotiate keybusiness stakeholder expectations before completing other training devel-opment activities. Following this practice ensures what they call return onexpectations (ROE): ‘‘focusing training and reinforcement efforts on thestated bottom-line expectations of business partners, with the goal ofdelivering the key outcomes/results they expect’’ (p. 4). Phillips and Phillips(2007) maintained that organizations have shifted their focus from activitiesassociated with programs, projects, or processes to a focus on the resultsassociated with these activities. The process of determining an organization’sVolume 23, Number 2 / 2010 DOI: 10.1002/piq 99
  • 8. return on investment begins by defining the initial need and corresponding business goals for a project. In this way, training professionals position the effort for success by aligning its intended outcomes with business goals and missions—from the start of the effort. This focus on alignment enables training professionals to put processes in place to qualify training projects in ways that ensure their linkage to business goals. It also allows training professionals to identify a training sponsor who possesses signatory authority and an adequate span of control over the schedule, budget, and resources to develop, deliver, and maintain the training. Finally, alignment includes any realignment required to keep pace with organizational changes or unanticipated issues arising during the course of a training effort. Because of the importance of alignment and realignment, Villachica, Stone, and Endicott (2004) recommended adding an alignment phase to the standard instructional systems development (ISD) model. During this initial phase, a training department collaborates with a sponsor and other stake- holders to align the requests for training with business goals and organiza- tional missions, specify critical success factors for the training effort, and determine associated level 4 and 5 evaluation measures. Training profes- sionals can also use an alignment effort to specify these project components: ~ Roles for the sponsor, subject matter experts (exemplary performers), other stakeholders, and the training department ~ Responsibilities and estimated completion times ~ Deliverables ~ Review and approval cycles ~ Risk management strategies ~ Schedule Revising these alignment components as necessary facilitates ongoing realignment throughout the life of the training project. Closing Skill Gaps As shown in Figure 1, training worth an organization’s investment in money, time, and resources should address skill gaps. These gaps between existing and desired performance occur when people lack the skill and knowledge to perform. Phillips and Phillips’s (2002) second most common reason for the failure of training and development programs lay in a failure to recognize nontraining solutions. There is little or no payoff for developing and implementing the wrong solution, and in a variety of performance gaps, a lack of skill or knowledge is not the underlying cause. The behavioral engineering model (BEM) (Chevalier, 2003; Gilbert, 2007) specifies potential causes of gaps between existing and desired performance. Dean (1997) investigated the relative frequencies with which these causes occur. In group settings with over 1,000 participants from business100 DOI: 10.1002/piq Performance Improvement Quarterly
  • 9. TABLE 2 BEHAVIORAL ENGINEERING MODEL DEPICTING THE RELATIVE FREQUENCY OF CAUSES OF GAPS BETWEEN EXISTING AND DESIRED PERFORMANCE INFORMATION INSTRUMENTATION MOTIVES Environmental Data (35.3%): Lack of Resources (29.0%): Lack Incentives (11.3%): Lack factors standards, clear and of adequate tools, materials, of financial and adequate directions, processes, procedures, and nonfinancial incentives relevant and timely work environment for desired performance feedback Individual Knowledge (10.5%): Lack Capacity (7.5%): Lack of Motivation (6.3%): Lack factors of adequate skill and adequate hiring and of motive to work or knowledge, access to selection for job tasks; recruitment of people scientifically designed physical, mental, emotional unaligned with the training matching ability realities of the situation requirements of exemplary performanceSource. Gilbert (2007), Chevalier (2003), and Dean (1997).and industry, Dean would ask them to think about their jobs and then answerthe question, ‘‘Where’s my biggest performance block?’’ Participants alsoreceived an instruction sheet containing the six descriptions in Table 2, alongwith the question, ‘‘Improvement in which one of the following six areaswould enable you to do your job better?’’ Table 2 depicts both the BEM andthe relative frequencies Dean obtained for each of the potential causes of aperformance gap. Dean’s results indicated that causes other than a lack ofskill and knowledge account for 89.4% of all performance gaps, withenvironmental causes alone accounting for 75.6%. Training is the appro-priate solution for closing a performance gap only when the cause lies in alack of required skill or knowledge—about 10% of the time. As over 45% of all potential gaps arise from a lack of information (data andknowledge), training professionals need to determine whether informationto close performance gaps should reside in the environment, where peoplecould access the directions they need to perform their tasks using a job aid, atool, or some sort of performance support that acts as ‘‘a repository forinformation, processes, and perspectives that inform and guide planning andaction’’ (Rossett & Schafer, 2007, p. 2) or the heads of performers, wherepeople access internalized skills and knowledge learned in training andrecalled from memory. Organizations that use job aids or performancesupport to address performance gaps arising from a lack of access to data canobtain desired performances without incurring costly training development,delivery, and maintenance costs. Harless (1986) offered the flowchartappearing in Figure 3 illustrating how to determine whether a job aid ortraining is appropriate. Use of job aids limits training to situations in which the skill must residein the individual performer and training professionals have eliminated allother environmental causes of performance gaps. The resulting trainingfocuses on providing coaching and feedback opportunities for trainees tobuild and master skills and knowledge that they need to perform in ways thatVolume 23, Number 2 / 2010 DOI: 10.1002/piq 101
  • 10. Is task characterized by one or a combination of: Yes No Decide to develop pure Can the barrier be instruction-for-recall • Severe speed requirement? minimized or tolerated? (store in memory exclusively) • Precluding environment? • Psychosocial barrier? No Yes Is task performed frequently Yes Are there any reasons for No enough to justify recall training? development of a job aid? No Yes Decide to develop job aid to play some role in treating the skills/ knowledge deficit in the task.Source. Harless (1986). FIGURE 3. FLOWCHART FOR DETERMINING WHETHER A JOB AID OR TRAINING IS APPROPRIATE close performance gaps to meet organizational missions and business goals. Harless also notes one exception to this rule: the use of quasi-training in situations where preparing and distributing a job aid has not met with as much success as introducing it in training sessions describing when and how to use the job aid in the workplace. Such quasi-training focuses on providing trainees with practice using the job aids to complete job tasks and showing how absence of aids hinders work performance. Such quasi-training requires less time to create, deliver, and maintain than training to build fluent job performance based on recall of learned skill and knowledge. As illustrated in Table 3, sponsors and others requesting ‘‘training’’ may have quite different outcomes in mind—if they have any—because they do not use the term with the same rigor as training professionals do. Unfortu- nately, this lack of rigor can place training professionals at risk. For example, assume that a vice president of marketing has requested a training course to keep sales representatives motivated to reach a sales target for product A. Assume that product A is one of several products the organization makes, each of which has its own sales target and vice president of marketing. What happens when the incentives associated with product B are better than those associated with product A? Training will not motivate savvy reps to sell product A when they get more money selling the other. Worse yet, vice president A could end up resenting the training that she requested, perceiv- ing it as a line cost to her organization that failed to yield anticipated sales results and the bonus she expected. Well-intentioned responsiveness to such ill-informed requests for training creates ‘‘activity’’ or ‘‘to-do list’’ training that is a prescription for budget cuts in hard economic times. Compare this situation to one where the training department took the request, collected information to specify corresponding performance gaps102 DOI: 10.1002/piq Performance Improvement Quarterly
  • 11. TABLE 3 ILLUSTRATION OF OUTCOMES IN MIND FOR A TRAINING REQUEST A CLIENT REQUESTING ‘‘TRAINING’’ COULD REALLY TO ADDRESS A PERFORMANCE GAP BE ASKING FOR FOR THE PURPOSE OF ARISING FROM A LACK OF An orientation Providing information or Data (standards, directions, increasing awareness feedback) A ‘‘pick-me-up’’ or ‘‘keep- Motivating personnel to reach Incentives me-going’’ a goal or perform better Motives Feedback An activity to ‘‘check off’’ Complying with a regulation Data (standards, directions, feedback) Tools Incentives Skill or knowledge Capacity Motives A change in perspective Changing perceptions and Incentives attitudes Motives Capacity Career planning and Improved succession planning Skill or knowledge mentoring Capacity Motives A way to address an Solving the problem without Data (standards, directions, unspecified problem really knowing its causes feedback) Tools Incentives Skill or knowledge Capacity Motives Training Closing skill gaps Skill or knowledgeand their causes, and then worked with the vice presidents to align theirincentives and create hard-hitting training that help all sales reps close dealslike their best performers. The implication is that training professionalsshould understand the problem the training requester is trying to solvebefore creating a solution for it. This means that every training requestshould undergo some sort of front-end analyses (Hale, 2007; Harless, 1973;Robinson & Robinson, 2008; Rossett, 1987, 2009; Rummler, 2007) to do thefollowing: ~ Specify the gap between existing and desired performance using measurable terms. ~ Align the gap with the organization’s business goals or mission. ~ Determine whether the gap is worth closing. ~ Determine whether the gap arises from a lack of skill and knowledge rather than environmental or other potential causes.Volume 23, Number 2 / 2010 DOI: 10.1002/piq 103
  • 12. ~ Use training to address skill gaps or create appropriate nontraining performance improvement solutions (or partner with those outside the training department who can). Some would argue that compliance burdens pose an exception to the practice of conducting front-end analyses before creating requested training. Law, policy, standards, and regulations often mandate training, even though it may not address any particular performance gaps. Nonetheless, the mere existence of regulatory requirements does not free training departments from their obligation to conduct front-end analyses. In these situations, training personnel tasked with providing mandated training would still be wise to conduct a front-end analysis for the following reasons: ~ Aligning the required training with relevant regulations, as well as organizational missions and business goals. ~ Framing training within a larger context of exemplary job perfor- mance that includes nontraining solutions that work together to create organizational cultures and work environments that remove potential barriers to compliant behavior. ~ Focusing training on job skills that need continuous practice to be maintained or involve complex problem solving. ~ Identifying opportunities for job aids, cross-training, and modular- ization to minimize time spent in training while still meeting due diligence requirements. For example, in 1992 the National Association of Securities Dealers (NASD) found itself in a position of needing to provide compliance training for new federal regulations as well as new products and emerging practices (Villachica & Stone, 1999). In addition, the association was growing and needed to bring newly hired employees up to speed quickly. Based on the results of a needs analysis, the NASD opted to create a large-scale perfor- mance support system, which it called ‘‘Cornerstone.’’ An online database provided a means for keeping all employees current with new regulations and practices, thereby meeting any needs for awareness training. Performance support tools streamlined the investigation process. Online training intro- duced new employees to the mental models that experts used, with struc- tured on-the-job training and job aids providing opportunities to apply newly learned knowledge and skills in real situations. Although the NASD could have addressed required training by providing typical compliance training, it chose to create a larger system supporting the job performances of novice and expert examiners alike. In opting for this approach, the NASD met its required training obligations while it accomplished a number of other goals: ~ Reducing ramp-up time for newly hired examiners from 2.5 years to 1.0, a reduction of 60%. ~ Eliminating the time examiners otherwise spent creating and main- taining personal product libraries of widely varying quality.104 DOI: 10.1002/piq Performance Improvement Quarterly
  • 13. ~ Providing experienced examiners with new opportunities for career advancement ~ Improving job satisfaction ~ Increasing consistency in exam performance while decreasing rework Villachica and Stone (1999) note that Cornerstone produced an ROI of42.3%, with payoff in 2.2 years while reducing training delivery costs by 43%.Job-Focused Instructional Objectives Figure 1 indicates that instructional objectives for training should focuson the job. In addition, these objectives should arise from a task analysis of thejob and guide the creation of authentic assessments of job performance andpractice exercises that prepare people for the assessments. Mager (1962)argued that an instructional objective was an intent communicated in astatement describing a proposed change in a trainee. Objectives specify whattrainees should be able to do when they successfully complete a learningexperience. He contended that instructional objectives should consist ofthree parts: ~ A behavior specifying what trainees are able to do to demonstrate achievement of the objective. ~ Conditions specifying what is imposed on trainees when they are demonstrating their mastery of the objective. ~ Criteria specifying how well trainees should be able to demonstrate their achievement of the objective. Because the goal of training is to improve workplace performance inmeasurable ways, training objectives need to focus on the job regardless ofwhether they are called ‘‘behavioral objectives,’’ ‘‘performance objectives,’’‘‘learning outcomes,’’ or some other term. As depicted in Figure 4, this focusbegins with an analysis of job tasks that the training will address. This analysisdetails how exemplars actually perform their jobs. In addition to specifyingtasks, it describes exemplary performances that make up each task, as well asthe conditions under which the performance occurs and the criteria that the FIGURE 4. CONGRUENCE OF PERFORMANCE, CONDITIONS, AND CRITERIA ACROSS JOB TASKS, INSTRUCTIONAL OBJECTIVES, PRACTICE EXERCISES, AND ASSESSMENTSVolume 23, Number 2 / 2010 DOI: 10.1002/piq 105
  • 14. performance must meet to satisfy job requirements. Seen in this way, the task analysis specifies the components of exemplary performance. Instructional objectives, practice exercises, and assessments demonstrating mastery all flow from the job task. Performance, conditions, and criteria should remain congruent from the job task through all aspects of the training. Training professionals then use information from the task analysis to craft instructional objectives. In training situations, each component of the instructional objectives should have a distinct job focus: ~ Performance: What should people do on the job to perform in the same way that the organization’s exemplars do? ~ Conditions: Under what circumstances will they do that on the job (including cues that tell exemplars when to perform and the resources they use)? ~ Criteria: What defines doing work well on the job (the standards that exemplary performance meets to close the skill gap and meet the organization’s mission and business goals)? Table 4 depicts several job-focused instructional objectives for trainees in a variety of workplace settings. Such objectives serve several roles. First, they act as a contract among the training professionals who created them, the training sponsor, and the training stakeholders. Training accountability begins with objectives describing exactly how trained people should be performing their jobs when they return to the workplace. At this point, trainees will have the skill and knowledge to perform their jobs as the organization’s exemplary performers do. These objectives form the basis for posttraining observations of job performance (level 3 evaluation) and the collection of corresponding organizational measures of training’s impact and return on investment (levels 4 and 5). Second, job-focused instructional objectives act as a compass for instructional designers who are creating lean and effective training. Objec- tives guide instructional designers as they determine what to include and exclude from a training program. Objectives focused on job performance help instructional designers omit irrelevant content and activities that will require time and money to create but improve neither learning nor job performance. Third, job-focused objectives facilitate the transfer of learned performance to the job. This idea has deep roots. Thorndike and Woodworth (1901) argued that the transfer of learning is a function of the number of identical elements between two environments. Their findings indicate that the greater the number of shared elements, the more transfer will occur. Training professionals promote transfer of learning when they create learning environments that resemble the job—the closer, the better. This rule of thumb appears in the military adage, ‘‘Fight like you train and train like you fight.’’ When relevant content and practice exercises focus on the job, mastery of the instructional objective indicates that trainees possess the skill and knowledge to perform a job task to exemplar standards.106 DOI: 10.1002/piq Performance Improvement Quarterly
  • 15. TABLE 4 JOB-FOCUSED INSTRUCTIONAL OBJECTIVES FOR TRAINEES IN FOUR WORKPLACE SETTINGS PERFORMANCE CONDITIONS (GIVENS) CRITERIA Construction managers will Given a blueprint Within 5% of actual costs estimate material costs Using an estimation Within project budget spreadsheet Nurses will administer drugs Given a physician’s orders Right drug Given necessary equipment Right amount Using the prescription Right time administration software Right patient Right route (oral, intramuscular) Marketing personnel will Given a product launch Meet targeted goals create an ad campaign Given authorization Within schedule, scope, and budget Cartographers will label a map Using a map Right attributes Using a data set Appropriate fonts, text size, Using software colors Set of instructions Within 1% scaled accuracyNote. Each objective consists of a performance (including the performer), conditions, and criteria. Using this table formathelps training professionals keep their focus on the components of the objective and what learners will be able to do onthe job. This format also assists subject matter experts and other stakeholders in reviewing objectives. They can quicklyscan each cell for completeness and accuracy, rather than read long, convoluted sentences containing all the componentsof each objectiveCreating Sound Training Programs As depicted in Figure 1, training professionals should create strongtraining programs that promote learning and transfer to the job. One way todo this is to use an instructional model such as Merrill’s (2002) firstprinciples. Instructional models bring order to instruction, suggesting whatshould happen at the beginning of the instruction, at the end, and in between.Systematic instructional models have been around a long time. For example,the followers of Johann Friedrich Herbart (1776–1841), the acknowledgedfather of scientific pedagogy, developed a five-step teaching sequence basedon his methods (Clark, 1999): 1. Prepare the pupils to be ready for the new lesson. 2. Present the new lesson. 3. Associate the new lesson with ideas studied earlier. 4. Use examples to illustrate the lesson’s major points. 5. Test pupils to ensure they had learned the new lesson. Charles R. Allen of the Emergency Fleet Corporation of the U.S. ShippingBoard used the show-tell-do-check instructional sequence to train nearly500,000 workers during World War I (Miller, 1996). Surgeons still learn theircraft using a traditional ‘‘see one, do one, teach one’’ sequence, whereVolume 23, Number 2 / 2010 DOI: 10.1002/piq 107
  • 16. residents first observe a procedure performed by a more senior resident, then practice it on patients under the supervision of a more senior resident, and then teach the procedure to a less experienced resident (Velmahos et al., 2004). Synthesizing empirical research in the procedures of instruction with the human information processing model of learning and memory, Gagne ´ (1988) formulated nine events of instruction: 1. Gaining attention. 2. Informing the learner of the objective. 3. Stimulating recall of prior learning. 4. Presenting the stimulus. 5. Providing learning guidance. 6. Eliciting the performance. 7. Giving informative feedback. 8. Assessing performance. 9. Enhancing retention and transfer. (p. 11) ´ Building on the work of Gagne and other instructional design theorists, Merrill and colleagues (Merrill, 2002; Merrill, Barclay, & van Schaak, 2007) identified five underlying prescriptive principles of learning. Each corre- sponds to a phase of Merrill’s instructional sequence, with further guidance for each phase provided by specific corollaries. Together the principles and their corollaries provide a structure for sequencing instruction in ways that promote learning and transfer to the job. Merrill (2002) also discussed the use of all or part of the first principles and their corollaries in a variety of instructional theories. Table 5 describes Merrill’s principles and implications the authors derive for training professionals. Looking Outside the Training Department As depicted in Figure 1, training professionals should collaborate with sponsors and other stakeholders outside the training program in ways that promote transfer to the job. Until recently, Paradise (2008) could contend that business leaders ‘‘continue to recognize the need to financially support the learning function to leverage human capital to the fullest’’ (p. 6), with members of ASTD’s benchmarking forum and its BEST award winners spending more on average direct learning expenditure per employee than others ($1,609, $1,451, and $1,103, respectively, ASTD benchmarking forum, ASTD BEST award winners, and other ASTD survey respondents). Paradise and Patel (2009) reported that organizational investments in individual learning per employee remained relatively stable between 2007 and 2008 at $1,110 and $1,068, respectively. They maintained that this 3.8 percent decrease in light of other expense trimming was not dispropor- tionate. As of early 2010, annual 2009 employee expenditures on training remain unknown. Whatever amount this may be, the overall impact of this investment in training varies, with returns that executives, managers, and108 DOI: 10.1002/piq Performance Improvement Quarterly
  • 17. TABLE 5 MERRILL’S PRINCIPLES, PHASES IN AN INSTRUCTIONAL SEQUENCE, AND TRAINING IMPLICATIONS MERRILL’S FIRST PHASE PRINCIPLES TRAINING IMPLICATIONS Problem Learning is Ensure that the problems addressed in the training correspond centered promoted when to real-world tasks in the organization. learners are This typically means focusing on whole tasks rather than parts engaged in of them. solving real-world Increase task complexity and diversity over multiple lessons to problems. ensure that trainees who master the training have experienced an adequate sampling of task-based experiences to perform their jobs. Activation Learning is Relate what trainees know about organizational mission and promoted when strategic business objectives to the training they will complete. existing know- Relate the skill and knowledge trainees will master in the training ledge is activated to what they already know about their jobs, as well as their as a foundation improved on-the-job performance. for new Inform trainees that the skills, knowledge, and thought knowledge. processes they will build during training are like those that exemplary performers use to perform their jobs. Demonstration Learning is Employ examples and nonexamples for concept learning, (show me) promoted procedural demonstrations, process visualizations, and when new behavioral modeling depicting what exemplary performers knowledge is actually think and do on the job. demonstrated to Draw attention to any invisible thought processes and the learner. mental models that support situational interpretations, decision making, or problem solving on the job. Point out relevant similarities and differences across multiple demonstrations. Application Learning is Provide enough authentic, job-based practice to master an (let me) promoted when authentic, job-based assessment indicating mastery of the the learner applies instructional objective and performance of the corresponding new knowledge. task on the job at stated exemplary levels of performance. Provide scaffolding, coaching, and immediate feedback during the practices. For trainees to fly solo on the job, eliminate scaffolding and coaching on the assessment and provide feedback only afterward. Inform trainees that performing the assessment to stated standards means they can feel confident that they can perform their job tasks. Integration Learning is Provide opportunities for trainees to publicly demonstrate their promoted when new skills in ways that commit trainees to applying what they new knowledge is learned on the job. integrated into the Provide posttraining opportunities for trainees to continue learner’s world. discussing how they are applying what they learned in training to their jobs and ways to improve. Provide posttraining opportunities for managers and supervisors to monitor and reinforce the application of learned skills on the job. Ask managers and supervisors to use authentic in-class assessments from training (level 2 evaluation) to collect data indicating the transfer of training to the job (level 3 evaluation).Source. Merrill (2002).Volume 23, Number 2 / 2010 DOI: 10.1002/piq 109
  • 18. other training stakeholders may well find unacceptable. Georgenson (1982) estimated that only 10% of the information presented in training results in behavioral change on the job. In a study of 150 organizations, Saks and Belcourt (2006) reported that 62%, 44%, and 34% of employees apply training on the job immediately, six months, and one year after training. This suggests that transfer of training to the job is complex, with various researchers (Baldwin & Ford, 1988; Burke & Hutchins, 2007; Ford & Weissbein, 1997) maintaining that trainee characteristics, training design, and the work environment interact to influence learning and retention in ways that produce consistent performances on the job. Both Broad and Newstrom (1992) and Milheim (1994) suggested a variety of pretraining, training, and posttraining activities that facilitate transfer and persistence of newly learned skills on the job. As one example of collaboration, Broad and Newstrom (1992) describe a transfer partnership involving managers, trainers, and trainees ‘‘who have a strong interest in a particular training program and who have agreed to work together to support the full application of the training to the job’’ (p. 14). Partnerships for every high-priority program would employ strategies to reduce, remove, or control barriers to transfer before, during, and after training. Involving managers in transfer may represent a major change in many organizations. Including different levels of management represents another change. For example, supervisors may need to adjust schedules to accommodate training atten- dance and revise productivity expectations to allow trainees to apply what they learned in training to the job. Supervisors may need to assist trainees with course selection and enforce course prerequisites. Line managers who are funding training efforts may need to provide release time and incentives for exemplary performers to share their skills with the training developers who are conducting task analyses. Managers may also need to provide resources for the collection and analysis of level 3 and 4 evaluation data. Executives may need to see aggregated training data to gauge the overall returns on their training investments relative to their competitors and to organizations’ strategic directions. This kind of collaboration requires training professionals to look for opportunities to work with executives, managers, supervisors, and trainees to ensure that training transfers from the classroom to the job to the bottom line. Finding Opportunity in Trying Times The best practices described in this article appear in the instruction, training, performance improvement, and psychology literature. Nonethe- less, the empirical basis for some of these practices is stronger than others. In their integrative review investigating the transfer of training, Burke and Hutchins (2007) included three factors: learner, instructional design and delivery, and influences of the work environment. In some cases, the reviewed studies yielded strong or moderate relationships between the variables and transfer. In other cases, the authors noted that minimal110 DOI: 10.1002/piq Performance Improvement Quarterly
  • 19. research exists, and more is needed. A summary of the variables related to thefive practices described in this article appears in Table 6. While researchmight be limited, a lack of research does not mean that training professionalsshould eschew these best practices. Adhering to them enables trainingprofessionals to link their contributions to endeavors that executives thinkare important while demonstrating their due diligence in acting as stewardsof the funds they receive. Adopting these strategies also enables trainingpersonnel to document the benefits they bring to the larger organization tothe ‘‘various stakeholders in the process, including those who participate intraining, those who deliver it, and those who fund it’’ (Aguinis & Kraiger,2009, p. 464). Moving from order takers to partners in improved job performance willrequire training personnel to adopt appropriate short- and longer-termstrategies. In the short term, training personnel could focus on seizingopportunities involving low complexity and risks that offer large organiza-tional returns. One tactic may be to cut any training not directly aligned withorganizational missions or business goals. Another tactic could be eliminat-ing or minimizing training by providing job aids or online performancesupport (Rossett & Schafer, 2007). By closing gaps in performance arisingfrom a lack of access to data or tools, this tactic places necessary standards,guidance, feedback, process, and tools in the job environment rather than inthe long-term memory of the learner. Training personnel can typically createjob aids in less time than training, and online deployment decreasesdistribution costs while accommodating rapid revisions to volatile content.When people need coaching, practice, and feedback to master job tasks,training people to use a job aid typically requires less time than training torecall. A third tactic is to look for opportunities to partner with theorganization’s information technology (IT) group to eliminate or minimizetraining by creating online performance support. For example, rather thanproviding training for software systems, training personnel could partnerwith the IT group to write cue cards that provide step-by-step instructionsdescribing how to complete job tasks. Any necessary training would largelyserve the purpose of acculturation. It would focus on helping users becomecomfortable accessing and referring to the cue cards to perform their jobtasks and showing how performance without the cards is difficult (Harless,1986). In the longer term, completing the move from order taker to perfor-mance partner requires training personnel to reposition their efforts to focuson performance improvement. Seen this way, training becomes one of avariety of approaches for meeting this goal, a special case of improvingworkplace performance. Part of the long-term strategy will require trainingprofessionals to upgrade their own skill sets and find opportunities to employthem in ways that focus on performance. Most of these opportunities shouldaddress identifying and removing environmental barriers, as Dean (1997)suggests they are the sources of some 76% of gaps between existing anddesired performance. Other strategies are to create effective blends ofperformance-based training for gaps that truly result from a lack of skill orVolume 23, Number 2 / 2010 DOI: 10.1002/piq 111
  • 20. TABLE 6 DEGREE OF EMPIRICAL SUPPORT FOR THE FIVE BEST PRACTICES DESCRIBED IN THIS ARTICLE BURKE AND STRONG OR FIVE BEST PRACTICES TO HUTCHINS NUMBER OF MODERATE SURVIVE TRYING ECONOMIC (2007) REPORTED RELATIONSHIP MINIMAL TIMES VARIABLES STUDIES TO TRANSFER RESEARCH Align training efforts with Strategic link 3 | organizational missions and business goals Use training only when it addresses Needs analysis 6 | a gap between existing and desired performance arising from a lack of requisite skill Craft instructional objectives that Learning goals 16 | describe exemplary job and content performance relevance Create sound training programs Practice, 18 | that promote learning and transfer feedback, and to the job behavioral modeling Collaborate with sponsors and Transfer climate, 33 | other stakeholders outside the supervisory training department to promote support, peer transfer of training to the job support, opportunity to performSource. Burke and Hutchins (2007). knowledge and then partner with supervisors and managers in ways that facilitate transfer and measurably improved job performance. Still other long-term strategies involve institutionalizing exemplary performance throughout the organization. These strategies would partner training per- sonnel, executives, line managers, supervisors, exemplary performers, no- vices, and others in ways that create a culture of sustainable excellence throughout the organization. Executives trying to cut their operating costs to meet Wall Street expectations regarding profitability face tough questions about how they spend their monies. As they weigh their alternatives, they will be wondering, ‘‘What have you done for me recently that matters?’’ It will be far easier for them to cut mere order takers who deliver requested training than problem solvers who deliver improved job performance in ways that support business goals. As training professionals, the choice to become ‘‘strategic organiza- tional player[s]’’ is ours (Aguinis & Kraiger, 2009, p. 466). What will we do to survive these trying times and prepare ourselves for the next?112 DOI: 10.1002/piq Performance Improvement Quarterly
  • 21. ReferencesAguinis, H., & Kraiger, K. (2009). Benefits of training and development for individuals and teams, organizations, and society. Annual Review of Psychology, 60, 451–474.Allen, M. W. (2003). Michael Allen’s guide to e-learning: Building interactive, fun, and effective learning programs for any company. Hoboken, NJ: Wiley.Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for future research. Personnel Psychology, 41(1), 63–105.Broad, M. L., & Newstrom, J. W. (1992). Transfer of training: Action-packed strategies to ensure high payoff from training investments. Reading, MA: Addison-Wesley.Burke, L. A., & Hutchins, H. M. (2007). Training transfer: An integrative literature review. Human Resource Development Review, 6(3), 263–296.Carnevale, A. P., Gainer, L. J., & Villet, J. (1990). Training in America: The organization and strategic role of training. San Francisco: Jossey-Bass.Chevalier, R. (2003). Updating the behavior engineering model. Performance Improve- ment, 42(5), 8–14.Clark, D. (1999). Johann Friedrich Herbart (1776–1841). A time capsule of training and learning. Retrieved June 4, 2010, from http://www.nwlink.com/donclark/hrd/history/ herbart.htmlDean, P. J. (1997). Thomas F. Gilbert, Ph.D: Engineering performance improvement with or without training. In P. J. Dean & D. E. Ripley (Eds.), Performance improvement pathfinders: Models for organizational learning systems (pp. 45–64). Silver Spring, MD: International Society for Performance Improvement.Ford, J. K., & Weissbein, D. A. (1997). Transfer of training: An updated review and analysis. Performance Improvement Quarterly, 10(2), 22–41. ´,Gagne R. M. (1988). Mastery learning and instructional design. Performance Improvement Quarterly, 1(1), 7–18.Georgenson, D. L. (1982). The problem of transfer calls for partnership. Training and Development Journal, 36(10), 75–78.Gilbert, T. F. (2007). Human competence: Engineering worthy performance (Tribute ed.). San Francisco: Jossey-Bass/Pfeiffer.Hale, J. (2007). The performance consultant’s fieldbook: Tools and techniques for improving organizations and people (2nd ed.). San Francisco: Jossey-Bass/Pfieffer.Harless, J. H. (1973). An analysis of front-end analysis. Improving Human Performance: A Research Quarterly, 4, 229–244.Harless, J. H. (1986). Guiding performance with job aids. Introduction to performance technology (Vol. 1, pp. 106–124). Washington, DC: National Society for Performance and Instruction.Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating training programs: The four levels (3rd ed.). San Francisco: Berrett-Koehler.Kirkpatrick, J., & Kirkpatrick, W. K. (2009). The Kirkpatrick four levels: A fresh look after 50 years Retrieved from http://www.trainingmag.com/msg/kirkpatrick_white_paper_ reg.jsp 9Mager, R. F. (1962). Preparing instructional objectives. Palo Alto, CA: Fearon.Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and Development, 50(3), 43–59.Merrill, M. D., Barclay, M., & van Schaak, A. (2007). Prescriptive principles for instructional design. In J. M. Spector, M. D. Merrill, J. van Merrie ¨nboer, & M. P. Driscoll (Eds.), Handbook of research for educational communications and technology (pp. 173–184). New York: Taylor & Francis Group.Milheim, W. D. (1994). A comprehensive model for the transfer of training. Performance Improvement Quarterly, 7(2), 95–104.Miller, V. A. (1996). The history of training. In R. L. Craig (Ed.), The ASTD training and development handbook (4th ed., pp. 3–18). New York: McGraw-Hill.Paradise, A. (2008). 2008 state of the industry report: ASTD’s annual review of trends in workplace learning and performance. Alexandria, VA: ASTD.Volume 23, Number 2 / 2010 DOI: 10.1002/piq 113
  • 22. Paradise, A., & Patel, L. (2009). 2009 state of the industry report: ASTD’s annual review of trends in workplace learning and performance. Alexandria, VA: ASTD. Phillips, J. J., & Phillips, P. P. (2002). Eleven reasons why training & development fails y and what you can do about it. Training, 39(9), 79–85. Phillips, J. J., & Phillips, P. P. (2007). Show me the money: How to determine ROI in people, projects, and programs. San Francisco: Berrett-Koehler. Rivera, R. J. (2007). How to demonstrate value: Key measures every learning professional should know. In WLP scorecard: Why learning matters (pp. 17–24). Alexandria, VA: ASTD Press. Robinson, D. G., & Robinson, J. C. (1990). Training for impact: How to link training to business needs and measure the results. San Francisco: Jossey-Bass. Robinson, D. G., & Robinson, J. C. (2008). Performance consulting: A practical guide for HR and learning professionals (2nd ed.). San Francisco: Berrett-Koehler. Rosenberg, M. J. (1982). The ABCs of ISD. Training and Development Journal, 36(9), 44–50. Rossett, A. (1987). Training needs assessment. Upper Saddle River, NJ: Educational Technology Publications. Rossett, A. (2009). First things fast: A handbook for performance analysis (2nd ed.). San Francisco: Jossey-Bass. Rossett, A. & Schafer, L. (2007). Job aids and performance support: Moving from knowledge in the classroom to knowledge everywhere. San Francisco: Jossey-Bass/Pfeiffer. Rummler, G. A. (2007). Serious performance consulting according to Rummler. San Francisco: Jossey-Bass/Pfieffer. Saks, A. M., & Belcourt, M. (2006). An investigation of training activities and transfer of training in organizations. Human Resource Management, 45(4), 629–648. Stone, D. L., & Villachica, S. W. (2006). Lessons learned aligning projects: Cat herding 101. Paper presented at the ASTD TechKnowledge Conference, Denver, CO. Sugrue, B., & Rivera, R. J. (2005). 2005 state of the industry: ASTD’s annual review of trends in workplace learning and performance. Alexandria, VA: ASTD. Tannenbaum, S. I., & Yukl, G. (1992). Training and development in work organizations. Annual Review of Psychology, 43(1), 399–441. Thorndike, E. L., & Woodworth, R. S. (1901). The influence of improvement in one mental function upon the efficiency of other functions. Psychological Review, 8(247–261). Training. (2008, November/December). Guages and drivers: Training Magazine’s exclusive analysis of the U.S. training industry. Training, 48, 16–34. Velmahos, G. C., Toutouzas, K. G., Sillin, L. F., Chan, L., Clark, R. E., Theodorou, D., et al. (2004). Cognitive task analysis for teaching technical skills in an inanimate surgical skills laboratory. American Journal of Surgery, 187, 114–119. Villachica, S. W., & Stone, D. L. (1999). Appendix: Cornerstone—A case study of a large-scale performance support system. In P. J. Dean & D. E. Ripley (Eds.) Performance improvement interentions: Performance technologies in the workplace (Vol. 3, pp. 437–460). Washington, DC: International Society for Performance Improvement. Villachica, S. W., Stone, D. L., & Endicott, J. (2004). Project alignment: Ensuring successful development and implementation from day one. Performance Improvement, 43(10), 9–15. STEVEN W. VILLACHICA Steven W. Villachica, CPT, PhD, is an associate professor of instructional and performance technology at Boise State University. He completed his doctorate in educational technology at the University of Northern Colorado. E-mail: SteveVillachica@boisestate.edu114 DOI: 10.1002/piq Performance Improvement Quarterly
  • 23. DONALD A. STEPICH Donald A. Stepich, PhD, is an associate professor of instructional andperformance technology at Boise State University. He completed his doc-torate in education at Purdue University. E-mail: DStepich@boisestate.eduVolume 23, Number 2 / 2010 DOI: 10.1002/piq 115