Your SlideShare is downloading. ×
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Pfxwp01 monster curriculum1_12-0115
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Pfxwp01 monster curriculum1_12-0115

539

Published on

In this white paper, I lay out a framework for designing and developing large-scale curricula for workplace learning. I've successfully helped organizations tame their monster curriculum needs with …

In this white paper, I lay out a framework for designing and developing large-scale curricula for workplace learning. I've successfully helped organizations tame their monster curriculum needs with this approach.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
539
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  1. Monster Curriculum!Part 1: A Curriculum Development Framework White Paper Author: Gus Prestera, PhD Date: January 15, 2012 Citation: Prestera, G.E. (2012). Monster Curriculum! Part 1: A Curriculum Development Framework (white paper). Prestera FX, Inc.: Glen Mills, PA (retrieved from www.presterafx.com). PRESTERA FX, INC. | P.O. Box 5, Glen Mills, PA 19342 | 484.343.6474 | WWW.PRESTERAFX.COM
  2. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White PaperTABLE OF CONTENTSAbstract ......................................................................................................................... 2What is a Monster Curriculum?.................................................................................... 3 Types of Curricula ....................................................................................................................................... 3 A Curriculum Map....................................................................................................................................... 4 A Good Curriculum is More Than a Map...it’s about Proficiency ............................................................... 5 So What Then is a Good Curriculum? ......................................................................................................... 6What’s so Different? ..................................................................................................... 7 Size Matters ................................................................................................................................................ 8 Complexity Demands Attention ................................................................................................................. 8 Importance is a Double-Edged Sword ........................................................................................................ 8 Shelf Life Requires Long-Term Commitment .............................................................................................. 9A Curriculum Development Framework .................................................................... 10 Phases ....................................................................................................................................................... 11 Analysis Phase ...................................................................................................................................... 11 Design Phase......................................................................................................................................... 11 Buildout Phase ...................................................................................................................................... 11 Rollout Phase ........................................................................................................................................ 11 Maintenance Phase .............................................................................................................................. 12 Functions .................................................................................................................................................. 12 Program Management ......................................................................................................................... 13 Curriculum Design ................................................................................................................................ 13 Curriculum Development ..................................................................................................................... 16 Change Management ........................................................................................................................... 18 Marketing Communications ................................................................................................................. 19 Program Operations ............................................................................................................................. 20 Program Evaluation .............................................................................................................................. 21What’s Next? ............................................................................................................... 22Updated: 1/15/2012 PRESTERA FX Page 1
  3. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White PaperABSTRACTIt’s easy to assume that building a curriculum, or any other large-scale corporate training program consisting ofmultiple courses, is just like building a stand-alone course, only with multiple repetitions of the development cycle.Those of us who have seen the beast up close know that there are many differences between building a course, oreven series of courses, and building a large, integrated, cohesive curriculum. Some of these differences relate to thesize, complexity, importance, and shelf life of a monster training curriculum. In this paper, we will explore thosedifferences and their implications for training professionals. We will use that understanding to build a curriculumdevelopment framework that addresses the unique challenges a monster curriculum represents, while still being ableto scale down to smaller curriculum initiatives. This framework will address the entire process: analysis, design,buildout, rollout, and maintenance. It will also consider the various ongoing functions that are needed to support thecurriculum, including program management, curriculum design, and change management. Future papers will divedeeper into each element of this framework, so we ask the reader to consider this paper a foundation upon which wecan build an understanding of best practices.Updated: 1/15/2012 PRESTERA FX Page 2
  4. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White PaperWHAT IS A MONSTER CURRICULUM?A course is typically associated with one set of instructional goals and one subject matter. Often, weassociate a course with the knowledge set of a single expert—typically, that expert is the instructor. Acourse could last 30 minutes, a day, or maybe even a few months, but usually not much more than that.You might build a series of courses and group them into a program of related courses, such as a sellingskills program. A curriculum, on the other hand, is designed around the needs of a particular job ratherthan a particular skill. It involves many instructional goals and different subject matters pressed intoservice to suit the needs of a given audience. Depending on the depth and breadth of a curriculum, itcould take months and sometimes years to complete.TYPES OF CURRICULAThere are three common breeds of curricula found in workplace learning: Job-Specific Onboarding: A curriculum that teaches people new to a position everything they need to know in order to reach proficiency in that position and be allowed to operate independently in that role. Leadership Development: A curriculum that prepares future managers to succeed in their leadership responsibilities or one that trains current managers on their leadership responsibilities. Certifications: A curriculum that officially certifies a learner’s proficiency in a particular discipline, such as a Six Sigma Black Belt certification or a Certified Professional in Learning and Performance (CPLP) designation.Other types may be more specific to certain industries, such as a Corporate Integrity Agreement (CIA)Curriculum, which is commonly seen in the health sciences field. A new process, system, or method mayalso call for a large training program that borders on being considered a curriculum. Typically, though, aprogram teaches workers a specific set of things that they need to know; whereas a true curriculumattempts to teach them all the things they need to know in order to be proficient in a particular job. Anew machinist working at an assembly plant may be required to learn all of the company’s policies andprocedures. In addition, he/she may need to complete the OSHA-mandated safety program, which couldconsist of a series of courses. The safety program is one element, though, within a broader NewMachinist Curriculum that could consist of stand-alone courses on operating a particular piece ofequipment or entire programs that cover every aspect of the plant’s operations. In terms of a hierarchy,the course is a basic building block; multiple courses can make up a program; and multiple programs andcourses can contribute to a broader curriculum. Key Point: A curriculum is made up of building blocks, consisting of individual courses and groups of courses, which are often called programs.Updated: 1/15/2012 PRESTERA FX Page 3
  5. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White PaperA monster curriculum often pulls from many different sources within a training department’s body ofresources, and often from outside the training department. For example, a new First Time Supervisorleadership development curriculum could draw courses from an already-existing communications skillsprogram that includes courses on active listening, conducting presentations, and running effectivemeetings. It could also draw from a coaching program that includes courses on goal setting, providingfeedback, and conducting difficult performance conversations. Those programs and their courses couldbe utilized across a variety of different audience-specific curricula. The coaching program could, forexample, be used for the First Time Supervisor curriculum, and it could also be used for the DevelopingFuture Leaders curriculum. A curriculum incorporates whatever serves the needs of a given audience.Because of this propensity to draw from existing resources, instructional designers must often conduct acurriculum analysis to see what’s available and what can be utilized.A CURRICULUM MAPOne thing that all curricula worthy of the name have in common is that they are audience-specific. Whenwe speak of a curriculum, we refer to it in terms of the audience: the Store Manager Curriculum, theFranchisee Curriculum, the Shop Floor Supervisor Curriculum, or the Nanofabrication Lab AssistantCurriculum. A curriculum is not designed around a body of content but rather around the needs of aparticular audience. Key Point: All curricula worthy of the name are audience-specific.As we think of a curriculum framework, it’s helpful to consider courses as building blocks and programs asgroups of building blocks that have the same color or shape. Those building blocks can be organized,sequenced, and tailored to different audience-specific curricula. Visually, that can start to look like Figure1, where courses from the different programs are listed along the right side of the table; each columnrepresents the needs of a different audience; and courses/programs are selected based on theiralignment with those audience-specific needs. Taken altogether, this begins to form a curriculum map.Those different audiences could be different levels of managers, ranging from future leaders to first-timesupervisors, to middle managers, to department heads, to business unit heads, to C-suite executives.Some readers may object that even those audience segments may be too broad: maybe your organizationhas four variations of first-time supervisors depending on which part of the business they operate. In anycase, you are probably starting to observe one of the critical aspects of curriculum design, differentiatingaudience needs, or what instructional designers typically refer to as a job analysis.Updated: 1/15/2012 PRESTERA FX Page 4
  6. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White PaperFigure 1. Framework for a curriculum map that organizes building blocks by audience needs Audience W Audience X Audience Y Audience ZProgram 1  Course A  Course BProgram 2  Course C  Course DA GOOD CURRICULUM IS MORE THAN A MAP...IT’S ABOUT PROFICIENCYBased on what we’ve discussed so far, we might define a curriculum as a collection of courses andprograms organized in a sequence around the needs of a particular audience. That may be an adequateworking definition for what a curriculum IS, but it does not begin to capture what a good curriculumshould be. For that, we need to look at the intent of a curriculum.When an organization goes about creating a curriculum for a particular job, it is making a sizableinvestment of time, money, focus, and other resources. Why? The organization makes this investmentwith the expectation that training the individual worker in some systematic way is more effective thansimply allowing that worker to learn on the job through trial & error, osmosis, mimicry, or whatever othertechniques they pick up in the wild. We know that without formal training, workers can becomeproficient, assuming they have good role models, adequate resources, and good leaders: it just takes timeand patience. However, time and patience usually translate into mistakes that cost the organization interms of low productivity, lost revenue, increased waste, reduced quality, and unhappy customers. Themore time and patience are given, the greater the cost to the organization. Therefore, the intent of thecurriculum—from a business standpoint—is to minimize the time and costs associated with getting anewbie from the point where their performance on-the-job is a detriment to the organization to a pointwhere that individual is contributing to the organization positively with minimal errors and acceptableproductivity. Instructional designers refer to that end point as proficiency. In simplistic terms, the pointof a workplace curriculum is to minimize time-to-proficiency and cost-to-proficiency. What can we do toget an individual proficient on-the-job in the fastest, most efficient way possible? That’s the question thatshould drive all decisions that an instructional designer makes when building a curriculum.Updated: 1/15/2012 PRESTERA FX Page 5
  7. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White Paper Key Point: The point of a workplace curriculum is to minimize time-to-proficiency and cost-to-proficiency, so instructional designers need to consider: what is the fastest, most efficient way to get a worker proficient on-the-job?Note here that proficient on the job is not the same as proficient in the classroom, or what instructionaldesigners often refer to as mastery. Passing a paper-and-pencil test is, for many job skills, what passes formastery these days, but that is not sufficient evidence that the worker is proficient in doing the work onthe job. Even so, many curricula are built to stop at mastery and abandon learners to their own devices,once they have passed a paper-and-pencil test or once they have sat through a certain amount of seattime. We criticize schools for this, yet in the corporate sector we have copied the same practice.If the point of a corporate training curriculum—the reason why management invests in training—is on-the-job proficiency, then on-the-job proficiency is what we need to measure and use as our end point. Asan example, one of my financial services clients developed an authentic assessment of a financialadvisor’s critical skills, including the ability to analyze a client’s portfolio and offer soundrecommendations. New financial advisors—after completing their boot camp training—mustdemonstrate that they can perform their real-world job tasks under realistic conditions before they canstart talking with real clients. Even then, those rookie financial advisors are still operating within thecurriculum, under close scrutiny, and are on probation until they can demonstrate proficiency on-the-job.When your monster curriculum has teeth like that, it can truly have an impact on real-world performance.SO WHAT THEN IS A GOOD CURRICULUM?With time- and cost-to-proficiency in mind, below are some characteristics that distinguish effectivecurricula from ineffective ones: 1. Lean: Contains no fluff…all content and learning activities are designed to prepare the learner for on-the-job performance, and anything not relevant or essential for that purpose has been stripped out of the curriculum. 2. Practice: Decades of research confirm that practice does indeed make perfect, and in order to reach proficiency, learners must have ample opportunity to practice applying what they’ve learned to real-world tasks. 3. Context: Both the presentation of content and the practice exercises need to contain real-world examples, scenarios, challenges, and solutions, so that learners are prepared to go out and use what they’ve learned on-the-job more quickly. 4. Relevance: It’s not enough to have real-world context embedded into the learning: that context needs to be relevant for that particular audience and their work. Greater relevance leads to faster knowledge and skill acquisition; greater knowledge retention; faster recall; and greater on- the-job transfer.Updated: 1/15/2012 PRESTERA FX Page 6
  8. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White Paper 5. Integrated: Contains overlaps and points of intersection between courses, so that individual courses feel part of a larger whole. Though too much can create fluff, some redundancy among courses is important as it can reinforce critical information and skills and create synergies within the curriculum that would otherwise not be possible. 6. Progression: One course or program builds on the knowledge of previous courses and programs, creating a feeling for the learner that there is a logical progression to the curriculum. Progressions help to accelerate learning, because the learner is building new knowledge onto a strong foundation. 7. Cohesive: Content consistently echoes the same general school of thought and seems to be written in one voice, so the learner is able to organize, store, and later retrieve the information more rapidly and more accurately. Cohesion also helps to speed up learning. 8. Technology-Smart: Though instructional technologies can be cool and fun, the real value that they bring to workplace learning is the potential to speed up learning and reduce the cost of learning. Some can speed up learning by enabling learners to get more practice repetitions in a safe environment and more individualized feedback. They can reduce cost by reducing travel, and by enabling the expert/instructor to reach a larger audience at once (scalability). The smart use of technologies is another hallmark of well-designed curricula. 9. Synthesis: A curriculum involves a wide variety of content domains focused through the lens of one job. Therefore, it’s critical that the curriculum offer ample opportunities for learners to synthesize all they have learned—across all of the different courses and programs—into one integrated body of knowledge and skills. 10. Authentic Assessments: The criteria for proficiency need to be embodied in the form of an authentic assessment that enables learners to demonstrate that they can do the job under real- world conditions. This enables us to be more confident that the learner is truly proficient and it gives the curriculum a razor-sharp clarity of purpose.WHAT’S SO DIFFERENT?Throughout this paper, we have asserted that a curriculum is different than a program or course. Acourse is a building block that can be used for multiple audiences. A program is a collection of coursescentered on a particular subject (e.g., a product training program or a compliance training program). Thecurriculum is a collection of courses and programs filtered through the needs of a particular audience.There are some other characteristics that distinguish a curriculum, and they are significant because theydramatically influence how someone needs to go about the work of building a curriculum. Specifically,we’ll discuss four important characteristics: size, complexity, importance, and shelf life. Key Point: Size, complexity, importance, and shelf life differentiate curricula from training programs and courses.Updated: 1/15/2012 PRESTERA FX Page 7
  9. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White PaperSIZE MATTERSWhen you are trying to teach workers everything they need to know in order to be proficient in their jobs,each job-specific curriculum can get rather large. The number of individual courses involved couldnumber in the dozens, sometimes hundreds. To the extent that many of those courses need to be builtfrom scratch (some will be re-purposed from other curricula and some may be sourced off-the-shelf), thebuildout of the curriculum could require a large number of designers and developers; multiple, concurrentworkstreams; and a great deal of project management effort. Just as important will be the curriculumdesigner’s ability to see the entire forest through the many individual trees. Someone needs to have aclear vision of what the curriculum is intended to accomplish; how all of the curriculum elements will fittogether when finished; and what the learner experience should look like when navigating through thecurriculum. The ability to envision such a large, and often complex, design; the ability to communicatethat vision to the many contributors who will be responsible for building and/or implementing parts ofthe curriculum; and the ability to drive the team’s efforts toward the agreed-upon vision are all criticalskill sets for a curriculum designer.COMPLEXITY DEMANDS ATTENTIONCurricula can range in size, but even when they are small, they are often complex. By their nature,curricula include content from a variety of knowledge and skill domains. For example, training sales repsoften involves teaching them a great deal of product knowledge, but that curriculum likely also containsinter-personal communication, negotiation, proposal-writing, and selling skills training. It probably alsocontains compliance training, to make sure the reps know what they can and cannot do, by policy and bylaw. It may involve technology training to help them learn how to use their standard-issue tablets andtheir Customer Relationship Management (CRM) platform. The point is, a curriculum can involve a widevariety of content. In addition, those different courses may need to be built and delivered using differenttechnologies. In a curriculum, it is not unusual to see a combination of formats used, including classroomtraining, self-paced e-learning tutorials, online simulations, mobile learning modules, synchronous e-learning, paper-based workbooks, and online tests. It is also not unusual to see a combination of formaltraining and informal training, such as coaching, mentoring, secondments, and special assignments.When you combine the complexity of multiple content domains with the complexity of multiple deliveryformats, you can get a curriculum that is extremely complex to build, implement, and maintain. With thatin mind, it is easy to see why curriculum designers require unusually strong organizational skills as well asexperience with project management and/or program management.IMPORTANCE IS A DOUBLE-EDGED SWORDFor most organizations, building a curriculum is considered a capital expenditure and is seen as a strategiccompany initiative. Of course, not all jobs are equally important. At a brokerage firm, for example, theposition of Financial Advisor is considered mission-critical, because FAs are responsible for building clientrelationships; analyzing client needs; recommending solutions; and generating brokerage fees for thefirm. When the FAs perform well, the brokerage firm performs well. Other job families are important—maybe even critical to the success of the firm in their own way—but none are as critical as the FA jobfamily. Therefore, it’s no surprise that when a brokerage firm designs a curriculum for its FAs, that effortUpdated: 1/15/2012 PRESTERA FX Page 8
  10. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White Paperis going to generate a great deal of interest from all sorts of senior stakeholders. At other businesses, itwill be other job families that garner the lion’s share of the attention and resources.Stakeholder attention can be a good thing. It makes it easier to get funding when senior leadershipconsiders the curriculum mission-critical. It can also be easier to break through organizational barriersand gain cooperation. Obstacles that seem insurmountable on a smaller project are brushed aside by themomentum of a large curriculum initiative. However, with all that attention and support from seniorleadership comes greater scrutiny and higher expectations for achieving significant business resultsquickly. When a curriculum gets attention, every manager in the organization wants to be involved withthe initiative and wants to have input. No one wants to be left out in the cold. The fact that most largecurricula pull content and resources from across the organization only magnifies the effect. What doesthis mean for the curriculum designer? Active and proactive stakeholder management becomes anecessity. The curriculum buildout and implementation need to be managed like large organizationalchange efforts. The curriculum designer needs to be comfortable working with large numbers of seniorstakeholders, communicating effectively with them, and keeping the large egos from getting in the way ofprogress.SHELF LIFE REQUIRES LONG-TERM COMMITMENTA typical course has a shelf life of 1-3 years. It’s much like a hamster: every few years, you can go to thestore and get a new one. If so, then a curriculum is more like a dragon that starts small but gets very big,that lives for decades. Because of that, the curriculum’s owner needs to make arrangements to ensurethat the curriculum is cared for, maintained, and updated on a regular basis, over time. Maintenanceplanning should follow immediately on the heels of the curriculum’s rollout.A second implication of a curriculum’s long shelf life is that it will evolve and morph over time, andworkers trained today potentially will experience a significantly different curriculum than those trained ayear from now. This can create gaps in the workforce’s knowledge and skills. Indeed, most monstercurricula are constantly undergoing updates and improvements. What does this mean for the curriculumowner? As part of the curriculum design, we need to consider how we will update workers who havegone through prior versions of the curriculum and ensure that their legacy needs are tracked andaddressed, even as we are focusing our efforts on new workers starting the curriculum today.Updated: 1/15/2012 PRESTERA FX Page 9
  11. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White PaperA CURRICULUM DEVELOPMENT FRAMEWORKTo summarize, a curriculum development initiative is typically viewed by business managers as a strategic,capital investment, so there is a great deal of organizational pressure to succeed and lots of stakeholderinvolvement that needs to be managed. In addition, the size, complexity, and long shelf life of acurriculum demand greater attention to managing its buildout, rollout, and ongoing maintenance. Thecurriculum designer needs to think more strategically than a course designer and needs to be able to drivethe vision across a large number of concurrent workstreams. To accomplish this, the curriculum designerneeds to leverage a combination of consulting, design, and project management skills throughout the lifecycle of a curriculum initiative. If we put all of these ingredients into a blender, and pepper in trainingindustry best practices and my own experiences, we arrive at the Prestera Curriculum DevelopmentFramework shown in Figure 2.Figure 2. Prestera Curriculum Development Framework PHASES Analysis Design Buildout Rollout Maintenance Program Management Curriculum Design Curriculum DevelopmentFUNCTIONS Change Management Marketing Communication Program Operations Program EvaluationUpdated: 1/15/2012 PRESTERA FX Page 10
  12. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White PaperPHASESThe Prestera Curriculum Development Framework consists of phases and functions. The phases representdistinct segments of the overall initiative. We could think of each phase as its own project, if that helpsvisualize the work effort. There are five phases in the Prestera model: Analysis, Design, Buildout, Rollout,and Maintenance. Let’s take a walk through each of them.ANALYSIS PHASEDuring the Analysis Phase, we analyze the needs of the organization and the target audience(s) as well asconsider the resources initially available to satisfy those needs. From that analysis, we identify whatneeds to get done and how.DESIGN PHASEThe Analysis Phase quickly gives way to the Design Phase, where we architect the curriculum, mappingout all of the curriculum elements in sequence for each audience and each sub-audience. We alsodescribe the vision of what we would like the curriculum to look and feel like from the learner’sperspective. Based on this curriculum design strategy, we start to create a buildout plan that describeshow the curriculum components will be sourced and staged for implementation. During this DesignPhase, we will also consider how the curriculum will need to be positioned with stakeholders andaudience groups; what change management efforts will be required; what communications will beneeded; and how all of that should unfold such that the curriculum launch is set up for success; and moreimportantly, such that our learners are set up for success in their jobs.BUILDOUT PHASEAs the name suggests, the Buildout Phase is all about building the curriculum we’ve blueprinted in theDesign Phase. The curriculum buildout effort is likely to include some development of customcomponents, but it is just as likely to involve sourcing off-the-shelf components as well as re-purposingand customizing components that already exist within the organization. Therefore, buildout does notnecessarily mean development or production. It means sourcing, purchasing, developing, customizing,and assembling all of the curriculum components needed—and doing so in a way that ensurescohesiveness, consistency, and alignment with the original design vision. During this time, we are alsoputting together the plan for implementing the curriculum—the rollout plan. In the rollout plan, we willdefine a schedule for what rolls out, when, where, how, and to whom. The rollout plan includes thedistribution of marketing communications, course registration information, and the curriculumcomponents themselves.ROLLOUT PHASEThe Rollout Phase involves implementing the change management, communications, logistics, andcurriculum components for the first time. It can range from establishing an online Learning ManagementSystem (LMS) portal and populating it with courses and resources to conducting classroom training atmultiple locations with live instructors. You might ask: why not call it “implementation” instead ofUpdated: 1/15/2012 PRESTERA FX Page 11
  13. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White Paperrollout? Implementation refers broadly to the ongoing delivery of the curriculum, but the rollout is a one-time special event for each curriculum component, the time when it is first implemented. Carefulattention needs to be paid to the rollout of new curriculum components, because first impressions cancreate positive momentum for a curriculum or they can derail a curriculum’s progress (remember, seniorleadership is watching the monster closely). Typically, curriculum components will be piloted, revised,and then fully launched and marketed as part of the rollout. Also, note that not all curriculumcomponents will necessarily roll out at the same time. Version 1.0 of the curriculum could contain manyof the components, but then additional components are scheduled to come online in subsequent months.It is not unusual for it to take 3 months to a year to roll out a new curriculum in its entirety. In fact, Irecommend staging your development efforts with the idea that there will be multiple versions of thecurriculum. This takes some pressure off of the team; enables you to roll something out more quickly,then refine as you gain experience and feedback from the field. After the rollout is completed, thecurriculum continues to get implemented, but it then moves into the Maintenance Phase.MAINTENANCE PHASEDuring the Maintenance Phase, the curriculum continues being implemented. Sometimes, the curriculumneeds to be implemented with all legacy personnel within the target audience, which can involve amassive initial implementation to the hundreds or thousands of people in that audience. Once that grouphas been trained up, the implementation work settles down, focusing primarily on new hires. Even so,there will always be legacy needs to address, especially as the curriculum is continually updated andupgraded. For that reason, owners of monster curricula often segment their audiences into cohorts, sothey can track what each cohort has received and has not. This way, they can go back and provide“What’s New” training to cohorts that missed the opportunity to get that training while going through thecurriculum. When new content is added or content is changed, we need to consider those legacy needs.In addition to maintaining the implementation efforts, we need to develop a maintenance plan for thecurriculum, which describes what components need to be updated, when, how often, and by whom.Though unexpected needs for updates can occur, there should be some default schedule for reviewingcomponents to ensure that they are up-to-date. One other thing to consider is that a monster curriculumis like a long train with many railcars. Some of those railcars may have reached the Maintenance Phase,while others are just entering the Rollout Phase, and while still others are still in the Buildout Phase. Key Point: It’s rare for all components to be rolled out at once, and even more rare for them to be updated all at once.FUNCTIONSThe functions within the Prestera Curriculum Development Framework represent the various types ofwork that need to take place throughout the initiative. Again, if it helps to visualize the work effort, wecould think of each function as a person or team; or better yet, as a hat that someone needs to wear. Ona smaller curriculum, one person could wear many, perhaps even all of these hats. On a very large one,many people could share responsibility for just one of these hats.Updated: 1/15/2012 PRESTERA FX Page 12
  14. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White PaperPROGRAM MANAGEMENTThe Program Management function relates to all of the activities involved in managing contributors,timelines, budgets, reviewers, stakeholders, and risks. The function could be fulfilled by a single person,or it could be fulfilled by a team of managers responsible for the curriculum. For our purposes here, I willuse the Program Manager label to represent the person or the group of people ultimately responsible formanaging the work of building and implementing the curriculum. The Program Manager will have manyday-to-day oversight, coordination, risk management, and project communications responsibilittiesthroughout the initiative, but there are four critical planning pieces that a Program Manager needs to getright in order for the curriculum to succeed: the charter, buildout plan, rollout plan, and maintenanceplan. Charter. The charter is an agreement between the sponsor and the team responsible for building the curriculum, laying out what the requirements, resources, stakeholders, and constraints of the initiative are. What is the end point we’re trying to reach? Whether that’s captured in a document or not, the team needs to be clear on the desired outcomes. Buildout Plan. The buildout plan is a project plan—or more likely a collection of project plans— that describes how all of the curriculum components will be made ready for rollout. Some components will be purchased off the shelf; some built from scratch; some re-purposed; and some customized. Some components will be ready by January: others may come online in March. Some will involve outside vendors: some will require contributions from internal staff. All will require some planning. Rollout Plan. The rollout plan likewise is a project plan—or a collection of project plans—that describes how all of the components will be tested, reviewed, approved, piloted, finalized, staged, marketed, and implemented—as appropriate. The logistics involved in rolling out one component may be more complex than that of another component, so each requires due consideration. Maintenance Plan. The fourth critical program management deliverable is the maintenance plan. Once a curriculum component has been rolled out and has reached the status of “routine” implementation, it’s time to consider if/when that component needs to be reviewed for updates; how often; and by whom. If updates are made, how will legacy learners be informed of those changes. In other words, the maintenance plan describes how each curriculum component will be kept relevant and accurate over time.CURRICULUM DESIGNAs important as all the other functions are, if the curriculum design does not hit the mark, not much isgoing to go right. Instructional designers who focus on curriculum-level design work are often referred toas a Curriculum Architects, Program Architects, Instructional Design Architects, and Learning Strategists.As with the program management function, the curriculum design function can be performed by oneperson or a combination of people, so for our purposes, I’ll refer to this person or group as the Architect.Updated: 1/15/2012 PRESTERA FX Page 13
  15. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White PaperThe Architect’s job is to craft a vision of what the curriculum will look like and how learners willexperience it. This requires a great deal of creativity, conceptual thinking, and mental stamina, becausethere are so many moving parts that the Architect needs to be able to visualize over a long developmentcycle. Moreover, the Architect will need to communicate that vision in a compelling way to the sponsoras well as a wide variety of contributor and stakeholder groups. An architect of a building communicateshis/her vision for the construction through blueprints, models, materials lists, and other specificationdocuments. The architect walks around the construction site and makes sure that the carpenters, pipe-fitters, electricians, and other trades people are constructing the building according to the architect’sspecifications and in alignment with the architect’s vision. Similarly, our Architect must create a vivid setof curriculum design blueprints and specifications, and our Architect must inspect all of the curriculumcomponents at all stages of development to make sure that each one is consistent with the specifications;is integrated into a cohesive set of materials; and is in alignment with the broader vision of thecurriculum. What could be more important than that? To accomplish all that, here are the most criticaldeliverables that the Architect will be responsible for during the initiative, many of which are found in theCurriculum Design Strategy Document. Job, Curriculum, and Gap Analyses. There are three types of analysis normally associated with a curriculum development initiative. (1) The Architect will need to analyze the job itself in order to define what “proficient” looks like for that job. This primarily involves observing and interviewing people who know how to perform the job effectively. (2) The second type of analysis needed is called a curriculum analysis, and it involves reviewing all current curriculum components—or, in the absence of existing components—reviewing any components that could potentially be made available for this curriculum. The idea here is to take stock of what’s available; to determine its usefulness; and to figure out how it could or could not be leveraged in building the new curriculum. (3) The third type is a gap analysis, and it is normally only conducted when the curriculum is being built for the purpose of closing a perceived performance gap. You might ask: aren’t all curricula built for that purpose? No, not really. Most curricula are built for new hires, who have no performance history with the organization and therefore would not normally have known performance gaps. Sometimes, a curriculum is built for current employees, not for new hires. In that case, if the curriculum is being created in response to a perceived performance gap within that target audience, it makes sense to figure out what the difference is between what management considers optimal performance and what actual performance looks like in the field. Often, when we conduct a gap analysis, we find that if there is truly a significant performance gap, it is more likely to be caused by a lack of information, resources, role clarity, or incentives than by a lack of knowledge and skills (Rothwell, 1996). Even so, the organization may still choose to move forward with the curriculum, but hopefully armed with some non-training related performance improvement interventions. Curriculum Design Strategy. The strategy is a succinct explanation of how the curriculum is going to address the needs of the organization and the target audience. For one of my clients—a large financial services firm—the strategy boiled down to reducing early training costs by providing most of the bootcamp training online and on-the-job; culling the flock of new hires based on performance, keeping only those who show promise; and investing heavily into the development of those higher performers. The strategy sets the tone for all other design decisions, and helps to anchor all contributors and stakeholders to the initiative, even as itUpdated: 1/15/2012 PRESTERA FX Page 14
  16. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White Paper experiences many twists and turns. The Architect must develop the strategy, communicate it, and sell it. Curriculum Architecture: Most curricula these days involve a mixture of different types of content, approaches, and delivery formats. One curriculum may start with webinars and self- paced online tutorials, then proceed to online assessments, then classroom training, and then be capped off with coaching in the field. Another may involve a series of virtual classroom events, interspersed with team-based projects and readings assignments. Yet another may consist of quarterly classroom events, each followed by a series of weekly webinars, interspersed with e- learning tutorials, online simulations, e-books, vodcasts, and telephone check-ins with a regional training manager. Whatever that combination is, that’s at the heart of your curriculum architecture, and it will give your sponsor, contributors, and stakeholders a good sense of what the curriculum involves. Curriculum Map: The next level of granularity happens at the course level. What are all the courses and programs that a given audience needs to complete in a given period of time? For any curriculum, there could be several target audiences and sub-audiences, so expect that there will be multiple curriculum maps needed. Curriculum maps are sometimes called roadmaps, learning paths, and learning pathways. Regardless, they spell out what the desired route is for the learner from beginning to end. The Learner Experience. Though not always done, I recommend that the Architect go one more step and define the learner experience. What does the Architect expect the learner to see and feel as he/she is going through the curriculum? An architect of a building might describe a visitor entering the atrium of his/her building and experiencing a sense of awe at the openness of the space, the serenity of the waterfall in the lobby, the warm feelings elicited by the soft hues of the paint and plush carpeting. These types of descriptions are not reserved for artsy folks. Architects frequently speak in these terms when describing their buildings. Likewise, our Architect needs to be able to describe how, if applicable, the learners will be immersed in the learning because of the use of real-world scenarios and video-based problem-solving activities that kick off each instructional unit, and how they will refer to their workbook whenever they’re moving from component to component, so as not to get disoriented within the curriculum. Giving the team—especially the contributors—a sense of what the Architect’s vision of the learner experience will be like helps to get the team in alignment with each other and with the vision, so that the buildout can go more smoothly and the finished components can be more consistent in form. Recurring Themes and Tactics. In addition to describing how the learner will experience these instructional tactics that will be used throughout the curriculum, it is also valuable for the Architect to describe recurring content themes and tactics to be used by contributors in building out the components. For example, in a sales training curriculum I worked on, we identified in the upfront analysis that the most successful salespeople are not shy about asking for the business or for the referral, and so the idea of advancing the sale became a recurring theme, whether we were covering closing techniques or cold calling techniques. Giving the team recurring themes anchors all of the contributors and creates threads that can be weaved throughout theUpdated: 1/15/2012 PRESTERA FX Page 15
  17. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White Paper curriculum. Likewise, instructional tactics that will be get re-used throughout, such as case studies that will be leveraged multiple times during a curriculum, a common cast of characters that will be referenced, a process animation that will be used to introduce different parts of the process within multiple components: these are all examples of recurring instructional tactics. The more the Architect can anticipate the re-use of certain instructional elements and communicate that to contributors, the more consistency and cohesiveness is baked into the curriculum. Key Point: The Architect is primarily responsible for driving the design vision to all corners of the curriculum, making sure that all components are consistent, cohesive, and in alignment with that original vision.CURRICULUM DEVELOPMENTAs with the previously-described functions, Curriculum Development is something that can be done byone person, but it may involve a number of different contributors. In fact, for monster curricula, it caninvolve a large number of internal and external subject matter experts, instructional designers anddevelopers, and vendors of all shapes and sizes. Here I will use the term Developer to encompass all ofthese different players. Often, the Architect is one of the Developers, but he/she is not the onlyDeveloper. As soon as someone other than the Architect is building out curriculum components—someone who was not involved in crafting the original vision, strategy, and tactics—there is a risk that thecurriculum components will be disjointed. In fact, this is probably the biggest risk that needs to bemanaged throughout the life of a curriculum: with so many hands involved in the making of a curriculum,it’s very easy for it to become disjointed, looking more like a patchwork than a quilt. The Architect will ofcourse need to communicate frequently with the Developers before and during the buildout, and theArchitect will need to review all components to ensure alignment, consistency, and cohesiveness. Inaddition, there are several things that an Architect and the Developers can do to drive the developmentof a seamless curriculum: Standards. The Architect and Developers can collaborate to establish standards that can be applied to all of the components. These can include branding standards, related to the use of the company’s name and logo as well as the curriculum’s branding elements. Writing standards can also help to establish consistency around tone, voice, syntax, referencing, and formatting. 1 Agreeing on external standards, such as Strunk and White’s (2000) style book or Microsoft’s 2 (2004) technical writing standards, can save time while giving everyone a common reference point. Establishing and maintaining a lexicon, or glossary, of curriculum-specific terminology can also be helpful, so that everyone on the team is defining terms in the same way. That lexicon can later be published as its own curriculum component to help learners reference definitions as well. Overall, the more that the Architect can establish, document, and communicate quality1 Strunk, W. & White, E.B. (2000). The Elements of Style (fourth edition). Allyn and Bacon: Needham Heights, MA.2 Microsoft Corporation Editorial Style Board (2004). Manual of Style for Technical Publications (third edition). Microsoft: Redmond, WAUpdated: 1/15/2012 PRESTERA FX Page 16
  18. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White Paper standards to the various Developers, the easier it will be to maintain consistency, cohesiveness, and alignment. To the extent that the Developers are not aware of the standards, they will regress towards their own means, creating deviations from the Architect’s vision, which will then require the Architect to spend more time and energy correcting them. Templates. It is always a good investment of time and energy to create templates that can be re- used by all of the Developers. In the past, I have designed templates not only for the finished materials—for example, slide decks, facilitator guides, e-learning skins—I have also created templates for work products that get used during the buildout phase. For example, I ask the designer of each curriculum component to prepare a treatment plan (aka, design brief, design document, or design specification) that describes the component-level learning objectives, instructional tactics, and content structure/flow. The curriculum-level design document describes all of the high-level goals and objectives, global strategies and tactics, etc., so I just need the component designer to specify what’s unique about the design of that particular component. If we have 20 different components and each has a different designer, we could get back 20 very different treatment plans. To prevent that, I create a treatment plan template that tells the designers what information I need from them, in what format, and to what level of detail. This helps ensure some level of consistency and enables me as the Architect to compare oranges with oranges and do a better job of policing design consistency, cohesiveness, and alignment. As a general rule, then, whenever I suspect something is going to get used more than once, I create a template for it, so that I can impose consistency and control on the process. Libraries. When we have a slew of components getting built by different people at the same time, there are few efficiencies to be had unless we equip those various Developers with a centralized library of shared resources that they can all leverage. These can include image libraries, where we can re-use and re-purpose recurring graphic models, character photos, avatars, etc. Programming, or code, libraries enable Developers to borrow each other’s code more easily, so that new functionalities can be built on top of previously built ones and no one is re-inventing a functionality that already exists in the library. Content libraries for source documents and other subject matter inputs can help the various instructional designers educate each other, especially as it relates to recurring content themes. Tools, templates, examples, case studies, music, special effects, and any other elements within a curriculum that could be re-used across multiple Developer teams can and should be organized into shared libraries. The more information and resources we can share across the various teams, the more efficiently those teams can work and the better aligned their efforts will be with the Architect’s vision. Prototypes. For some things, the best way for the Architect to communicate his/her design vision is to create models, what e-learning developers often refer to as prototypes. A prototype can be a small version of a curriculum component that embodies the specifications and the spirit of what the Architect envisions. If a curriculum is heavily reliant on e-learning courses, the Architect might create a 5- to 15-minute module. If it relies heavily on classroom training, the Architect might create slides, guides, and handouts for a 90-minute segment. The benefit of creating prototypes is that they more concretely communicate specifications than the design documents alone; they help to reduce variances from the design vision; they provide the team with templates they can use; and they enable the team to test-run the development process,Updated: 1/15/2012 PRESTERA FX Page 17
  19. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White Paper working out the kinks before a larger batch of courses flows through it. Prototypes can also be used to get early feedback from pilot groups, so that adjustments can be made before mass producing content, saving time and money later. There are a lot of benefits to creating prototypes, but there is one drawback. Building prototypes can hold up the start of mass production. I’ve found it helpful to incorporate prototyping into the start of the buildout phase. I give myself a little head start, then build the prototype as quickly as I can, even as the other instructional designers are starting their courses. If I can show them the prototypes within a week or two of their start, that’s good enough to gain a lot of the alignment and consistency benefits. To gain speed, I can initially create the e-learning prototype in PowerPoint, building navigation and interactivity into it as best I can replicate, so that the team can more quickly see a working model of the course. Regardless of how you do it, consider building prototypes as a way of modeling how you want components designed.CHANGE MANAGEMENTMany curricula are established as part of a broader change effort. If the curriculum is for a new job—saya company establishes the role of Key Account Manager (KAM) within its Sales department—that’stypically a big organizational change for everyone who will come into contact with that new job. Forcustomers, they will need to understand what it means to have a KAM, rather than the regular AccountRepresentatives they had previously. For the Account Representatives, it could mean that some of themwill get pulled off of accounts that are now considered “key accounts” and given to these new KAMs. Forthe Sales Managers, how will they supervise and support KAMs differently than they do their other reps?For the Sales and Marketing staff, how will they support the KAMs differently than they do the reps? Forthe CEO of the company, will a direct phone call from a KAM mean something different? It probablyshould. Even with this relatively innocuous event of introducing a new KAM role into an existingorganization, there are a lot of potential change implications that ripple through the entire structure.Imagine now a situation where instead of adding a new job, the organization has decided to change theperformance expectations around an already-existing job. For example, let’s say that a manufacturingdivision decides that it wants its supervisors to spend 80% of their time working on the shop floor, directlysupervising the production line, and only 20% of their time working in the office doing paperwork andother administrative tasks. If the current state is that those supervisors spend 90% of their time in theoffice, that’s a huge change not only for the supervisors but also for the people on the production linewho are not used to seeing their supervisors very often and for the folks back at the office who are usedto seeing lots of administrative output coming from the supervisors. A seemingly small change in jobexpectations can have huge change implications that ripple out across the organization. With that, myassumption is that when I’m working on a curriculum design initiative, there is always going to be a needfor change management in order to help the organization transition from the current state to the newdesired state. I also try to remember that the curriculum itself is not the change. It is the battering ramfor the change effort: the change is actually going to happen (or not) outside of the curriculum, on theshop floor, in the offices, around the water cooler, in conversations, and in people’s minds. Key Point: The curriculum is not the change…it’s the battering ram for change. The change is actually going to happen…in people’s minds.Updated: 1/15/2012 PRESTERA FX Page 18
  20. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White PaperAs with the other functions, change management is not necessarily the responsibility of one person: it canbe shared within a team, though generally there is one person who is the inspirational leader of thatchange effort. Sometimes, that person is the sponsor but often it’s a leader who works for the sponsorand has accepted the mission of driving this change forward. For our sake here, I will refer to that personand any contributors and consultants helping that leader as the Change Agent.The Change Agent is annoying, because he/she represents a disruption to the safety and security of thestatus quo. Let’s face it, change is a hassle. Even if we accept and embrace change, most people directlyaffected by change consider it at best a necessary evil. For consultants like me, who support changeefforts, we cannot understand why anyone would be opposed to changing things for the better. WhenI’m starting to think that way, I remember a quote from the movie Jurassic Park, where chaos theorist Dr.Ian Malcom says: “Whats so great about discovery? Its a violent, penetrative act that scars what itexplores. What you call discovery, I call the rape of the natural world.” Now substitute the word “change”for “discovery.” When workers and managers have gotten used to doing things a certain way and gettingpredictable outcomes—even if they’re bad outcomes—they get comfort and a sense of safety from thenatural order of things—it’s like your favorite blanket on a cold winter night. The Change Agent is yourcollege roommate who pours cold beer on you and your blanket at 3:00 in the morning, after crawlinghome from a night at the bars. It’s not so much fun when you’re the one in the bed. You could dowithout the laughing and slurred speech too, and your hope is that your roommate will pass out beforedoing any more damage. Likewise, workers often wish their Change Agents would run out of steam andgo to bed. I try to remember those sentiments when dealing with change efforts, because it helps mestay humble when I think I’m a big shot change consultant.Change Management Plan. Whether it is a formal, written document or not, it is important for theChange Agent to have a plan. The Change Management Plan should define what the scope of the changeeffort entails; who the various stakeholders are; how they are expected to respond (e.g., apathy, relief,anxiety, opposition); and what techniques we plan to use in order to manage that response. A commonchange management technique, for example, is inclusion. If you think some people are going to opposean initiative, ask them to help you organize it. They then can’t oppose it because they’d be opposing theirown initiative. If they refuse to help you organize it, you can later combat their opposition by citing thefact that you gave them the opportunity to help you organize it. It’s a wonderfully sneaky technique thatoften does help reduce active opposition. There is still passive opposition, though, which is difficult tocombat when a person or group is really not in agreement with the change and does things to undermineyour efforts. I generally favor a more direct approach—tell them the truth. If a change is needed, lay outthe situation for your stakeholders, equipping them with the facts; offer your solution; and invite them toprovide input and alternative suggestions. Sometimes, you will get good ideas from them, and perhapseven, you might uncover a completely different approach. This form of inclusion is much more sincereand earnest. What matters, though, is that people understand why it is necessary to change; that theyappreciate that you are open and honest with them about it; and that they see their input heard,processed, and where appropriate incorporated into the overall solution.MARKETING COMMUNICATIONSWhether you agree that creating a new job or changing an existing job is worthy of change managementefforts or not, it is difficult to argue that a monster curriculum rollout doesn’t require a fair amount ofUpdated: 1/15/2012 PRESTERA FX Page 19
  21. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White Papermarketing communications effort. Whether you are re-designing an existing curriculum or rolling out anentirely new one, you need to generate awareness, at a minimum, from your target audience. Preferably,your communication efforts also generate interest in participating in the curriculum and/or supporting it.Possibly, your efforts can even help to generate some initial enthusiasm, enough to get the curriculumlaunched successfully, and then you hope that positive word-of-mouth advertising will sustain it overtime. Communication Plan. A good Communication Plan is going to define who the various message audience are; what their communication needs are before, during, and after the rollout; what messaging you’re going to put in front of them; when; and what communication vehicle you’re going to use to do that. Some of your communications may be targeted at the learners themselves, getting them pumped up that training is coming; recruiting participants for the pilot; inspiring them to register for courses during the rollout; and making sure that they have positive things to say about the curriculum after completing it. You might also need to communicate with the managers of the target learners, perhaps to ensure that those managers will allow their reports to sign up for training and take time off the line to complete the training. Maybe you need the managers to take an active role in the training, reinforcing new knowledge and skills. What about senior leadership—the folks who pay the bills for the curriculum? You need them to be aware of the curriculum and the impact it’s having on the organization, so that you can continue to get funding and perhaps even be able to request more funding in the future. Communication Pieces. Often, a curriculum will have its own branding, possibly a logo, a tagline, and a set of talking points. Some communication plans involve using that branding as a starting point to create communication pieces such as brochures, web sites, YouTube video advertisements, and special email blast announcements containing graphics and animations. Communication pieces don’t need to be flashy and expensive: a simple campaign of email communications can be very effective, so long as there has been some thought put into the needs of the audience, the messaging, and the timing. The marketing communications surrounding a curriculum are often overlooked by the Program Managers and Architects of the world, so it’s up to the Change Agent to realize the importance of reaching all of the stakeholder groups with appropriate messaging, so that the curriculum is successfully launched and has the kind of impact it is intended to have. For this reason, the Communication Plan is sometimes a subset of the Change Management Plan, and the Change Agent is often the person spearheading the design and rollout of communication pieces.PROGRAM OPERATIONSAll the great Program Management, Curriculum Design, Change Management, and MarketingCommunications can be for naught, if the curriculum falls apart at delivery. If the instructor doesn’t makeit to the classroom or poorly delivers the content; if the e-learning course doesn’t launch from the LMS; ifthe web conferencing technology crashes part-way through the webinar, all the great work that went intobuilding a great curriculum can be sabotaged. Having worked in training operations, I know theimportance of logistical planning and high attention to detail, but often Program Managers, Architects,and Change Agents are so consumed with the other aspects of the curriculum that they neglect the lowlytechnicalities of getting classroom materials printed on time, scheduling classrooms, configuring theUpdated: 1/15/2012 PRESTERA FX Page 20
  22. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White Papertechnology platforms, registering and tracking learners through their individual learning paths, and thethousands of other little things that add up to a successful curriculum. The activities of ProgramOperations rarely fall to a single person: it takes a village of training coordinators, LMS and portaladministrators, facilities personnel, IT personnel, instructors, printing professionals, and others toimplement a monster curriculum. For our purposes, I will refer to this motley collection as the Operators.I could provide a huge laundry list of the things they do in order to make a curriculum run smoothly, but Iwill save that for its own white paper. Key Point: You can have the nicest trains running on the nicest tracks, but if the train is late or doesn’t show, you’re going to lose your passengers.PROGRAM EVALUATIONLast but not least, we get to the Program Evaluation function. There are some aspects of programevaluation that are formative and ongoing—such as collecting and reviewing “smile sheet” data for eachcomponent, test scores, and program completion metrics—but there are some aspects that aresummative, intended to judge the effectiveness and worthiness of the curriculum. As with the otherfunctions, evaluation work can be the responsibility of one or more people. Some of them will design theevaluation strategy and create the evaluation instruments. Others will primarily collect the data and inputthem into some sort of database. Others will pull from that data to aggregate, analyze, and report on thecurriculum. Others will consume those reports and make decisions about how to interpret theinformation and about what, if any, modifications are needed. I will refer to this entire collection of datagatherers, analyzers, and decision-makers as the Auditors. Typically, the Auditors will consist of peopleinternal to the organization. Sometimes, they could be people from departments other than Training.For example, I once saw a curriculum audited by a business metrics professional who worked in theFinance department. Occasionally, a sponsor will bring in an outside auditing body. For example, I wasonce asked by the National Science Foundation to help evaluate a grant-funded nanotechnologycurriculum. Bringing in an outsider to evaluate a capital expenditure makes good business sense, and it’ssurprising that more senior leaders do not do this more often than they do, even when a monstercurriculum costs millions of dollars per year to operate. 3You may be familiar with Donald Kirkpatrick’s four levels of training evaluation: (1) reaction, or learnersatisfaction; (2) learning, or evidence of understanding and retention; (3) behavior, or transfer of learned 4behaviors to the workplace; (4) and results, or performance impact. More recently, Jack Phillips added afifth level, Return on Investment (ROI). For most small-scale training efforts, we collect level 1 andsometimes level 2 data: smile sheets and test scores. Some of my diehard training colleagues will collectand evaluate level 3 data, taking the time and effort to gauge the extent to which the knowledge and skillslearned in the classroom were successfully transferred on the job. I would be very happy to see this donefor all training efforts that are intended to change workplace behaviors. For a large-scale curriculum,3 Kirkpatrick D. L. (1959). Techniques for evaluating training programs. Journal of American Society of Training Directors, 13 (3): pp. 21 - 26.4 Phillips, J.J. (1996). Measuring the results of training. The ASTD Training & Development Handbook. Robert L. Craig (ed.). New York: McGraw-Hill.Updated: 1/15/2012 PRESTERA FX Page 21
  23. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework White Paperwhere capital investments and senior leadership are involved, level 4 and even level 5 program evaluationare not uncommon.Measuring impact generally requires capturing performance metric data before the curriculum is put inplace—back at the Analysis phase, to see what current performance looks like—and comparing it withpost-rollout data. It can also be done by comparing the performance of a cohort that has gone throughthe curriculum with one that has not, over the same period of time. In either case, the differencerepresents the impact, and when that impact is quantified in terms of dollars earned or saved andextrapolated to represent the total value gained of running the curriculum, the resulting number can bedivided by the total cost of the curriculum to come up with ROI. For example, if a sales trainingcurriculum nets an extra $100,000 per trained salesperson, representing additional first year revenue, andthe company spends $1 million training 50 salespeople, the ROI is $4 million, or 400% ($100k x 50 - $1m =$4m in profit; then $4m/$1m = 400%). That kind of information is immensely valuable in defending thecurriculum’s worth to the leaders who sponsor it and secure funding for it. More importantly, withoutthis kind of information, we put the curriculum at risk to be scaled back or eliminated when the timecomes to slim budgets and prioritize how resources are allocated. If we wait until then to start justifyingthe expenditures, it will likely be too late.WHAT’S NEXT?There it is, that’s the Prestera Curriculum Development Framework, five phases and seven functions toorganize our procedures, tools, and best practices. The framework can be scaled up for monstercurriculum initiatives and scaled down for small ones. Functions can be consolidated with one person,shared among a small team, or distributed across a large number of contributors. If you need to build orre-build a curriculum, this big picture framework will help you organize your people and your workactivities, ensuring that no major set of activities is overlooked. In subsequent white papers, I will divedeeper into each phase and function to describe procedures, tools, and best practices for how to applythe framework, providing examples and artifacts from real projects. In the meanwhile, please send meyour feedback, questions, suggestions, and any examples you’d care to share from your own curriculumdevelopment efforts. The more we share, the more our collective knowledge expands and the better wecan serve our learners. Gus Prestera is a Learning and Performance Strategist, working at the intersection of blended learning, curriculum development, and performance improvement. He holds a PhD in Instructional Systems, an MBA, and a Certified Performance Technologist designation and has over 15 years of experience designing, developing, and implementing large-scale training programs for a variety of businesses. If you have questions, suggestions, or feedback, please contact Gus at gus@presterafx.com.Updated: 1/15/2012 PRESTERA FX Page 22

×