Collaborative approach to competency based curriculum
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Collaborative approach to competency based curriculum

on

  • 705 views

 

Statistics

Views

Total Views
705
Views on SlideShare
703
Embed Views
2

Actions

Likes
0
Downloads
4
Comments
0

1 Embed 2

https://childrensboston.socialtext.net 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Collaborative approach to competency based curriculum Document Transcript

  • 1. Competency-Based EducationOne Specialty’s Collaborative Approach toCompetency-Based Curriculum DevelopmentDiane Kittredge, MD, Constance D. Baldwin, PhD, Miriam Bar-on, MD,R. Franklin Trimm, MD, and Patricia S. Beach, MDAbstractThe authors describe a seven-step incorporation of up-to-date information collected for the 2004 edition,consensus development process used to from the literature and national experts, demonstrate that both editions havecreate the two most recent editions of (4) responsive consultation with the been used by most residency programsthe Academic Pediatric Association’s national Pediatric Residency Review throughout the country. The authors(APA’s) educational guidelines for Committee on the latest accreditation believe that the multifaceted approachpediatric residency. The 1996 (printed) requirements, (5) wide distribution for to consensus development and theand 2004 (online) editions of the prepublication review, to obtain broad customizable design of the curricularguidelines were designed as flexible organizational buy-in and end-user tools in the APA’s guidelines are directlytools to help residency programs meet acceptance, (6) intensive dissemination associated with their broad nationalchanging accreditation requirements by and faculty development through use. These methods may help to guideproviding lists of goals and objectives multiple national workshops over several educators in other disciplines who areand objective-based evaluation tools. The years, and (7) careful evaluation of interested in developing andguidelines were developed in seven utilization and user feedback. implementing educational products forsteps: (1) centralized national leadership Representatives of all major national dissemination and use.combined with coordinated, organizations involved in pediatricdisseminated authorship, (2) clear education helped to refine the Acad Med. 2009; 84:1262–1268.definition of targeted users and repeated guidelines. User surveys conducted forassessment of their needs, (3) the 1996 edition, and Web site user dataS ince 1983, the Academic Pediatric changing environment of clinical experiences and normal newbornAssociation (APA) has engaged pediatric medicine and medical education.1–3 rotations for medical students andeducators nationwide in the collaborative Pressure to develop a better-articulated residents.8 In 1996, the APA completelydevelopment of three successive editions and more structured approach to rewrote the 1985 edition and publishedof educational guidelines for residency residency training led to revised an expanded Educational Guidelines fortraining. Each edition of these guidelines accreditation standards for residencies. Residency Training in General Pediatrics.2was shaped in purpose and scope by the In 1997, the Accreditation Council for This edition provided the first Graduate Medical Education (ACGME) comprehensive set of learning goals and mandated the use of written goals and objectives for the education of generalDr. Kittredge is professor of pediatrics, Dartmouth objectives for residency curricula.4 Later, pediatricians across all three years ofMedical School, Hanover, New Hampshire. the ACGME focused on strengthening postgraduate training. Finally, in 2004,Dr. Baldwin is professor of pediatrics, University of the evaluation of residents and required updated APA Educational Guidelines forRochester Medical Center, Rochester, New York. programs to certify residents’ Pediatric Residency were published onBefore 2005, she was professor, Departments of competence in six competency domains an interactive Web site.3 This editionPediatrics and Family Medicine, University of TexasMedical Branch, Galveston, Texas. by the completion of their training.5–7 offers residency programs a resource These changes challenged programs in all for building their own customized,Dr. Bar-on is associate dean for graduate medicaleducation and professor of pediatrics, University of medical disciplines to improve their competency-based curricularNevada School of Medicine, Reno, Nevada. Before curricula. documents, using interactive tools and a2006, she was professor of pediatrics, Loyola comprehensive database of goals andUniversity Stritch School of Medicine, Maywood,Illinois. The APA addressed these challenges objectives. by developing a curriculum resourceDr. Trimm is professor of pediatrics, University of for pediatric residencies that evolved In this article, we describe the seven-stepSouth Alabama College of Medicine, Mobile,Alabama. over three editions to become a national consensus development process comprehensive and flexible set of used to create the 1996 and 2004Dr. Beach is professor of pediatrics, University ofTexas Medical Branch, Galveston, Texas. In 2001, she tools. In 1985, the APA published the Guidelines and develop a communityjoined the team that helped prepare the educational first edition of the guidelines, of Guidelines users. The Guidelinesguidelines discussed in this article. Educational Guidelines for Training in developers were influenced by a nationalCorrespondence should be addressed to Dr. Kittredge, General/Ambulatory Pediatrics,1 which climate of collaboration around sharedDepartment of Pediatrics, Dartmouth Hitchcock outlined a minimum core of pediatric curriculum development that was visibleMedical Center, 1 Medical Center Drive, Lebanon, NH03756; telephone: (603) 653-6041; fax: (603) 653- knowledge, skills, and attitudes that in projects reported by several disciplines6050; e-mail: (diane.kittredge@hitchcock.org). should be taught in ambulatory in the 1990s. The Society of Teachers of1262 Academic Medicine, Vol. 84, No. 9 / September 2009
  • 2. Competency-Based EducationFamily Medicine (STFM) created a The Collaborative Development centralization of control was essential.collaborative family medicine clerkship Process: Key Steps and Lessons The core team developed templates andcurriculum in 19909; a pediatric clerkship Learned instructions for the section editorscurriculum was published in 1995 by the The collaborative development processes and reviewed and combined all theCouncil on Medical Student Education used to develop the 1996 and 2004 documents to ensure consistency inin Pediatrics (COMSEP)10 –11; and an editions of the Guidelines were similar. format and language across all sections.internal medicine clerkship curriculum The seven steps of this process are This was especially important for thewas published in 1995 by the Clerkship described below and summarized in complex tasks of Web site design,Directors in Internal Medicine.12–14 Table 1. In this article, we will focus database construction, and developmentCollaborative curriculum development is mainly on development of the 2004 of new curricular tools. These jobs werea common method of the STFM, which edition, which updated the content of the most efficiently conducted by a small,resembles the APA in its practice of 1996 edition and added subspecialty goals focused group.networking around national projects.15–18 and objectives. This edition is built on a Step 2: Clear definition of targeted usersUse of a highly collaborative process for large database of 334 goals with objectives and repeated assessment of their needsthe Guidelines was appropriate because the that can be accessed dynamically on thegoal was to develop a flexible product that Web; lists of goals and objectives can be Process. Needs assessment surveys wereall residency programs could readily adapt selected for every residency experience conducted before and during theto their own needs. Creation of a and downloaded as customizable development of both the 1996 and 2004standardized, prescriptive curriculum documents. The Web site also includes Guidelines editions, to ensure that the onscreen instructions, curricular tools content served the needs of a broad groupresource was not considered a useful end such as customizable evaluation forms of potential users and to enhance end-userpoint. This same strategy of flexibility was and templates for rotation planning, acceptance.adopted by the Clerkship Directors in and six tutorials on how to buildInternal Medicine for their curriculum.14 Surveys of pediatric program directors competency-based curricula that are downloadable for local adaptation. conducted in 1993, 1996, and 1999 pointedCarole Bland and colleagues, in an 19 out discrepancies between residencyinfluential article published in 2000, Step 1: Centralized, national leadership program performance and currentdescribed 35 features of successful combined with coordinated, accreditation requirements. These datacurricular change in medical schools disseminated authorship helped the team define the utility, content,which were drawn from a careful review and scope of the resources under Process. All editions of the Guidelinesof the educational and business literature. development. For example, the 1999 survey were official projects of the APAHer process includes numerous elements on programs’ uses of the 1996 Guidelines Education Committee. The Guidelinesthat resemble our collaborative process— identified demand for a more flexible, easy- project director for the 1996 and 2004for instance, creation of a cooperative to-customize document that could be editions (D.K.) was chair of the educationclimate, broad participation, strong committee from 1992 to 1995. For both accessed online. Hence, the 2004 editioncommunications, formative evaluation, editions, about 50 educators were divided was designed to be interactive at its point oftraining support, and effective guiding into writing and review subcommittees. access, so programs could select neededleadership— but she was not describing a Hence, the Guidelines were “owned” tools and customize them aftercurriculum development process that from the start by a large group of leaders downloading. The survey also showed thatcrossed institutional borders. Our project in pediatric education, who not only the previous edition was more useful forwas particularly ambitious because it contributed their expertise in the generalist than subspecialty rotations, sowas designed to serve more than 200 development and refinement process extensive subspecialty content was added.residency programs and to address the but also were able to assist with Finally, because the survey showed that newfull scope of three years of residency dissemination of the final product. ACGME requirements were still verytraining. None of the national challenging to many programs, tutorialscollaborative projects mentioned In 2000, substantial funding to produce and program planning and evaluation toolsabove have described their the 2004 edition enabled formation of a were added. national advisory board to garner thedevelopment process in clear steps with support of pediatric subspecialists and The core team also used workshops tosufficient detail to help other groups to facilitate an efficient nationwide gather continuing needs assessment data.implement their methods. collaborative process. The core team To guide development of the 2004 edition, recruited 10 section editors from the APA workshops were conducted in a computerTherefore, we wrote this article to Education Committee to manage writing classroom, so users could try out the Webdescribe a national consensus and review of content revisions. Other site interface, sample objectives in thedevelopment method that might serve as contributors were members of the APA Guidelines database, and suggest ways toa useful model for other disciplines. We and/or other organizations, or they were make the resource more useful.also demonstrate wide utilization of the paid consultants on competency-basedGuidelines, and we hypothesize that the accreditation, project evaluation, and Lessons learned. Cycles of needssystematic collaborative development of computer programming. assessment were essential to make thethis shared resource and its customizable product as responsive as possible to theformat have enhanced its acceptance, Lessons learned. While the revision needs of educators “in the trenches,”usefulness, and broad dissemination. process was intensely collaborative, especially because they were working inAcademic Medicine, Vol. 84, No. 9 / September 2009 1263
  • 3. Competency-Based Education Table 1 Collaborative Development of the Academic Pediatric Association’s Educational Guidelines for Residency Training: Key Points and Lessons Learned* Collaborative process step Purpose of step Key points and lessons learned Step 1: Centralized national • Efficiently coordinate a nationwide • Central leadership was especially important for the leadership combined with collaborative process complex tasks of Web site design, database construction, coordinated, disseminated and development of new curricular tools. authorship • Dissemination offered a range of content expertise representing end-user groups and enhanced buy-in. ................................................................................................................................................................................................................................................................................................................... Step 2: Clear definition of targeted • Create immediately useful product • Needs were measured before, during, and after the users and repeated assessment of project to guide both planning and implementation. their needs • End-user input facilitated meeting of end-user needs. • Multiple methods of data collection, including face-to- face feedback during pilot testing, were used. ................................................................................................................................................................................................................................................................................................................... Step 3: Incorporation of up-to-date • Align content with latest, best evidence • National experts, as well as the literature, were used to information from the literature and help identify latest and best evidence regarding medical national experts content and educational process. • Step 3 increased content validity. ................................................................................................................................................................................................................................................................................................................... Step 4: Responsive consultation • Align curricular content with residency • Alignment with requirements of the Accreditation with the national Pediatric program needs Council for Graduate Medical Education (ACGME) was Residency Review Committee on critical to later use of the resource, BUT: the latest accreditation • It was important to avoid acting as a mouthpiece of the requirements ACGME and to serve as an advocate for faculty. • The resource was not dated by too-close adherence to changeable requirements. • All semblance of prescriptiveness was avoided. ................................................................................................................................................................................................................................................................................................................... Step 5: Wide distribution for • Tap expertise nationwide to refine product • This was a low-cost, effective way to test innovations, prepublication review, to obtain and enhance user buy-in garner support, disseminate the product, and enlist future broad organizational buy-in and • Develop visibility and buy-in by professional users. end-user acceptance organizations ................................................................................................................................................................................................................................................................................................................... Step 6: Intensive dissemination and • Enhance wide visibility and utilization • Hands-on workshops were critical for beta-testing and faculty development through refinement of tools. national workshops • Encourage and guide use • This was a powerful, low-cost approach to faculty development. ................................................................................................................................................................................................................................................................................................................... Step 7: Careful evaluation of • Refine the products • Collection of data was carried out using multiple utilization and user feedback strategies (surveys, Web usage reports). • Enhance user buy-in and satisfaction • Evaluation by actual users of specific product functions was more informative to developers than global surveys.* The table summarizes key features and lessons learned by the authors in the development of the Academic Pediatric Association’s educational guidelines.2,3 the context of rapid change and new Lessons learned. This step is essential to team to address current and upcoming demands. Workshops served as focus earn credibility for a curricular resource. faculty needs, even before revised RRC groups and taught the development The amount of content to include in the requirements were published. For team more than mailed surveys did final product was an issue that we and our example, in developing the 2004 edition, about specific user needs and reactions targeted users debated extensively. We learning objectives from the 1996 edition to individual Guidelines components. decided, as did the internal medicine were revised by modifying the verbs to clerkship curriculum task force,14 that reflect performance (e.g., “analyze” and Step 3: Incorporation of up-to-date making our resource comprehensive rather “manage” rather than “discuss” and information from the literature and than abbreviated would facilitate local “explain”). Customizable evaluation tools national experts adaptation, even though the large volume were developed that could be built Process. During the 1996 Guidelines of information could be overwhelming. around a program’s selected list of development, the editorial team learning objectives and, thus, be specific conducted an extensive review of the Step 4: Responsive consultation with the and competency-based. Templates for literature on curriculum content, national Pediatric Residency Review program and rotation planning were also drawing on expert opinion about medical Committee on the latest accreditation created to help program directors content and educational methodologies. requirements organize their responses to new ACGME For the 2004 edition, the core team and Process. Throughout the development requirements.3 its consultants also reviewed literature on process for both editions, members of the educational change, competencies, national Pediatric Residency Review Lessons learned. Consultations with evaluation processes, and faculty Committee (RRC) informed the APA and the RRC were essential to adjust the development. Further content refinement other organizations about anticipated Guidelines to users’ needs. However, the resulted from the extensive expert review changes in accreditation requirements. team learned that keeping some distance process described in Step 5, below. This knowledge enabled the Guidelines from the RRC was important, given the 1264 Academic Medicine, Vol. 84, No. 9 / September 2009
  • 4. Competency-Based Educationfrustrations felt by educators during workshops and e-mail queries facilitated users were pediatric generalists. Thethose years of rapidly changing ACGME by links to online materials to review. major limitations to use of the Guidelinesrequirements. We carefully disclaimed a Modern information technology has were reported to be lack of time,prescriptive intent for the Guidelines greatly reduced the time and expense of resources, and faculty support. Althoughand consistently encouraged local this activity. few considered the format (14; 8%) orcustomization of the resources provided. content (2; 1%) to be a limiting factor, Step 6: Intensive dissemination and written comments indicated that theStep 5: Wide distribution for faculty development through national document was intimidating in volume,prepublication review, to obtain broad workshops and many respondents suggested onlineorganizational buy-in and end-user publication. Process. The 1996 and 2004 Guidelinesacceptance have been distributed free of charge andProcess. The APA Board and the core widely publicized in 14 well-attended In October 2005, 18 months afterGuidelines team recognized the value of a national workshops. In 2000 and 2007, publication of the 2004 Guidelines,national consensus-building effort pediatric residency programs were members of the APA, Association ofaround development of the Guidelines. invited to showcase their own innovative Pediatric Program Directors (APPD), andThe 1996 edition went through three curriculum development activities that Society of Adolescent Medicine weredrafts, and at each step broad input was implemented the Guidelines. In 2003 and surveyed using a commercial Web surveysought from APA members and other 2004, live demonstrations using a tool (SurveyMonkey; http://www.groups, including subspecialists and portable computer laboratory gave SurveyMonkey.com; accessed May 21,academic leaders. A prepublication draft participants the opportunity to explore 2009). Replies were received from 582was distributed in advance to participants new Web site functions and give instant respondents, who represented 171 of 204at a national workshop, generating feedback, and helped the core team (84%) ACGME-approved residencyvaluable practical feedback. A formal recruit beta-testers for more extensive programs. The data showed that 149 ofexternal review by all the major pediatric explorations. training programs (73%) were aware oforganizations involved in education the Guidelines and that 106 (62%) hadtook place prior to publication. The both logged onto the Web site and used Lessons learned. Faculty development isAPA explored the need for formal the Guidelines; many said they were likely one of the cornerstones of educationalendorsement of the document from to return at a later time for further use. change. While users like self-directedleadership organizations within The majority of respondents did not online tutorials,20 interactive workshopspediatrics, but these groups deemed report significant barriers in using the encouraged educators to use theformal approval to be unnecessary, Web site, but 145 (about 25%) of users Guidelines creatively and share what theybecause they had been involved in the commented on long downloaded had learned. These low-cost workshopsdevelopment process, and because the documents and difficulty manipulating informed potential users about theproduct was intended to be a flexible tool the tables. resource, taught them how to use it,for local adaptation, not a prescription helped to win their acceptance, andfor curriculum change. In December 2007, we extended this gathered their feedback so the tools could preliminary survey by evaluating online be optimized to meet educators’ needs.For development of the 2004 edition, a use of the 2004 Guidelines by registered We believe that the personal contactsimilar national consensus process for users between July 2005 and December achieved in workshops was instrumentalcontent review was formalized through 2007. A total of 1,747 registered Web site in making the resource more “friendly”the national advisory board. A new users came from 47 states and 33 foreign to users.challenge was developing users’ countries and represented all pediatricacceptance of and comfort with the web- residencies approved by the ACGME in Step 7: Careful evaluation of utilization 2008. In all, 8,754 files had beenbased platform. Guidelines section editors and user feedback downloaded by 188 of a total of 194served as alpha-testers, and we recruitedbeta-testers at annual workshops and Process. During development of the residency programs (97%). Ourcomputer laboratory demonstrations. 1996 and 2004 Guidelines, we surveyed companion study by Beach andBeta-testing was conducted in cycles users repeatedly to measure their use of colleagues,20 published in this issue ofthroughout two years as new functions the Guidelines and gather information Academic Medicine, provides more detailwere completed. Feedback from on satisfaction, barriers to use, and on how these users implemented thereviewers and workshop attendees led to suggestions for improvement. The results online resources.significant revisions of some Web site are summarized in List 1.Thesecomponents and the addition of several evaluations were approved by Dartmouth Lessons learned. Evaluation datanew tools. Medical School’s IRB. collected from users provided critical information to help us improve theLessons learned. Prepublication review To prepare for the 2004 edition, Guidelines during development. Anhas been included in the development of questionnaires mailed to educators at online survey tool vastly simplified themany national curricular resources. We all 195 ACGME-approved pediatric process of gathering these data, comparedfound that it was an effective way to test residency programs (in 1999) yielded 170 with mailed surveys, but data collectedinnovations, garner support, and enlist responses (program response rate, 87%). directly from the Web site were farfuture users. This process is relatively Among all programs, 131 (77%) reported more representative of users. Gettinginexpensive when conducted using that they had used the Guidelines; most overloaded program directors to respondAcademic Medicine, Vol. 84, No. 9 / September 2009 1265
  • 5. Competency-Based Education of specific Guidelines tools by those who List 1 had used them.20 Use of the 1996 and 2004 Editions of the Academic Pediatric Association’s Educational Guidelines for Residency, as Reported by Survey Respondents and Site Users* Discussion The 1996 Guidelines2: Model of collaboration The survey and respondents The collaborative development of the Guidelines is consistent with • Survey mailed November and December 1999 to 195 programs, which included all APPD listed pediatric residency programs in 1999. organizational practices first introduced in the business world. Peters and • 170 programs (87%) responded. Waterman,21 in In Search of Excellence: • 151 programs (77%) responded that they had used the guidelines. Lessons From America’s Best-Run Sample of written comments on needs Companies, emphasize the importance • Needs more specificity of staying close to the customer, listening to users, and promoting intense • Needs a functional index communications within an • Needs online format to allow updates organization—all principles that were Sample of written evaluation comments reflected in our collaborative process. They also describe the value of • Objectives lack detail “simultaneous loose-tight properties,” • Too much material in objectives that is, a combination of central control • Guidelines facilitated development of required curriculum with reasonable investment of and disseminated freedom of action, resources which also typified our process. Day22 • For next step, please develop materials to facilitate teaching and implementation wrote that the value of a business should • Was useful to develop ideas for grant writing be anchored in value offered to the customer, and Kotter’s23 eight steps for • In RRC preparation, guidelines helped us add more structure and substance to existing the change process emphasize the curriculum importance of coalition building and The 2004 Guidelines3: communication. Our goal was to build The survey and respondents not only a set of tools and resources but • Survey mailed October and November 2005* to 204 programs, which included all pediatric also a community of Guidelines users residency programs on the ACGME-approved list for 2005. who initially contributed to the • 171 programs (84%) responded. development of document content and Web site design and who later shared • 127 programs (62%) responded that they had used the guidelines. ideas about how to implement the Sample of written evaluation comments Guidelines in their programs. In our view, • I wish the “build your own” selections were more concise to avoid excessive editing after our most effective community-building download technique was conducting national • Standard predesigned goals and objectives were too short, but the “build your own” tables workshops; these simultaneously were too lengthy monitored user concerns, elicited • It is a wonderful resource and really helped us revise goals and objectives in competency format formative feedback, and provided faculty development. • Exceedingly helpful in giving my subspecialists ideas for goals, objectives, and ways to redesign their rotations with competence in mind Our collaborative process also drew on • I could never have gotten started building competency-based evaluations without the guidelines methods used by several concurrent site national curricular development projects. • I greatly appreciate the tutorials for faculty Development of the national curricular guidelines for family medicine • Fellowship directors have taken to this site and like its ease of use clerkships10 was funded by the Bureau of The 2004 Guidelines: Site User Data, May 2005 Through December 2007 Health Professions (BHPr) and published • Users of Web site from May 2005 through December 2007 were from 194 programs. in 1991. This project balanced central • These programs represented 100% of all ACGME-approved programs for 2008. control, provided by the BHPr, with disseminated review, facilitated by an • Site registrants from 188 programs (97%) downloaded files from the Web site. (No written comments are available.) advisory committee of representatives from national organizations with an* The survey was carried out 18 months after initial publication of the guidelines Web site in May 2004, and 6 interest in family medicine education. months after completion of site refinements in May 2005. (Superscripted numbers 2 and 3 refer to references 2 and 3 in the reference list.) We also drew inspiration from the COMSEP curriculum, which was also supported by the BHPr and published in to surveys has become increasingly collection worked best: we harvested 1995. This project employed an advisory difficult over the past decade. We decided utilization data from the Web site, and committee, conducted two national that a combined approach to data supplemented these data with evaluation surveys, and used an iterative process of 1266 Academic Medicine, Vol. 84, No. 9 / September 2009
  • 6. Competency-Based Educationreview by its future constituency.10 –11 A implementation, and evaluation. Our argue that our collaborative development1995 clerkship curriculum for internal process was consistent with all six of process, our customizable design, andmedicine was also developed Glassick’s27 criteria for the evaluation of our scholarly approach were probablycollaboratively.12 This BHPr-funded educational scholarship. The project additional important contributors to thedevelopment project, like ours, included team widespread use of the product.a collaborative process, an advisory • set clear goals for the project, carefully A causal connection betweenboard, and national surveys of clerkship identifying our purpose and our collaborative development anddirectors to help define curriculum targeted end-users in advance; widespread use of the Guidelines may becontent.13–14 Although these projects, andothers published more recently,24 –25 • made adequate preparation by implied by these associations, but itcollectively included methods which we conducting needs assessments and cannot be proven by the data available atadopted or adapted, none of them has literature reviews and consulting with this time. However, evaluation data frompublished a well-articulated model to RRC members and other experts; other national collaborative projectsdisseminate their approaches to other suggest similar associations. In pediatrics,groups. • used appropriate methods for team for example, the collaborative COMSEP building, collaborative development curriculum for medical students10 was and review, evaluation, faculty used by 90% of all U.S. medical schools.11Model of developmental flexibility development, and dissemination; The Web-based CLIPP project,28 –29The project’s emphasis on flexibility in which recruited more than 100 facultycurriculum design reflects developmental • demonstrated significant results through surveys, focus groups, and online data nationwide to author and peer review 31approaches to innovation and the teaching cases for pediatric clerkships, isevaluation of innovations that have collection; now licensed by more than 75% of U.S.evolved in industry during the past 20 • used effective communications, through medical schools and used by more thanyears. Peters and Waterman,21 and also workshops and publications, to keep 12,000 students (Leslie H. Fall, MD,Collins and Porras,26 advocate methods targeted users apprised of project associate professor of pediatrics,that avoid restrictive traditions and status, new online functions, evaluation Dartmouth Medical School, personalhierarchical power structures so that results, and implementation ideas; and communication, May 2008). Manyorganizations can adapt to unexpected educators recognize the “not inventedenvironmental challenges and • engaged in reflective critique to examine here” phenomenon, which typicallyopportunities. The Guidelines were our work before, during, and after limits the dissemination of educationalcreated in a rapidly and radically publication of the Guidelines. innovations across institutions. Thatchanging world of education and clinical these three educational projects have This process led to many enhancementspractice. We designed them to help all succeeded in overcoming this of the Guidelines as we worked to buildresidency programs deal with evolving parochialism may be attributable at least innovative tools to meet evolvingchanges by (1) developing the document in part to their collaborative needs—inventing, testing, and refining atby a collaborative process that engaged development. each stage of the development process.the intended users in a dynamic fashion,and (2) making the document flexible Widespread usage The 1996 and 2004 editions of theand adaptable, rather than prescriptive. Guidelines were created with generousFor the 2004 edition, technological Although our usage data are support from a large number of pediatricinnovations enabled us to create a highly impressive—188 programs (97%) have educators from within and outside theflexible resource that surpasses many used the 2004 edition—we recognize that APA. We believe that our effort garneredavailable Web-based educational tools by self-reported use of the tools via surveys national participation and enjoyed wideoffering users choices for selecting the and Web site usage data cannot measure acceptance at least in part because it wascontent and the format of curriculum how programs actually applied the tools carefully planned and conducted withdocuments. For example, educators can they obtained from the Guidelines. Nor extensive input from intended users and,build a customized list of learning can these data tell us how well the therefore, met their immediate need toobjectives for a rotation and then insert Guidelines helped faculty integrate move toward competency-basedthose objectives into custom-formatted competencies into their programs. To educational models with flexible toolsevaluation tools, in order to meet the address such key questions, more suitable for local adaptation. Ourrequirements of competency-based comprehensive outcome studies will be collaborative model, with its emphasiseducation. We believe that in today’s needed. Residents’ competence at the end on developmental flexibility andchallenging world of health professions of training and after entry into practice customizable products, attention toeducation, developmental and will be the best measure of the real faculty development, and adherence tointeractive, user-driven approaches to effectiveness of our educational resource, the scholarly criteria of Glassick, mayinnovation have great value to offer. the basis of an important study that is prove useful in other settings and for beyond the scope and time frame of this other disciplines. project.Scholarly approachAnother factor that may have enhanced The urgent need for curricular resources Acknowledgmentsdissemination and utilization of the to meet changing RRC requirements The authors of this article acknowledge theGuidelines was our use of a systematic, probably drove the extensive use of the generous assistance of individuals representingscholarly approach for development, 1996 and 2004 Guidelines, but we would many pediatric organizations in refining theAcademic Medicine, Vol. 84, No. 9 / September 2009 1267
  • 7. Competency-Based Education1996 and 2004 editions of the Guidelines. 6 Carraccio C, Wolfsthal SD, Englander R, cfm?event c.beginBrowseD&clearThese include the American Academy of Ferentz K, Martin C. Shifting paradigms: Selections 1&criteria curriculum%Pediatrics, APA, APPD, American Board From Flexner to competencies. Acad Med. 20resources%20project#156). Accessed Mayof Pediatrics, Association of Medical School 2002;27:361–367. 21, 2009.Pediatric Department Chairs, COMSEP, the 7 Sectish TC, Zalneraitis EL, Carraccio C, 18 STFM Group on Faculty Development.Federation of Pediatric Organizations, Pediatric Behrman RE. The state of pediatrics residency Faculty Development Resources. 2nd ed.RRC, Society of Adolescent Medicine, and training: A period of transformation of Available at: (http://www.fmdrl.org/index.Society for Developmental and Behavioral graduate medical education. Pediatrics. 2004; cfm?event c.beginBrowseD&clearSelectionsPediatrics. The authors also wish to thank the 114:832– 841. 1&criteria faculty%20development#2263). 8 Sahler OJZ, Lysaught JP, Greenberg LW, Accessed October 27, 2008.hard-working chapter editors of the 1996 Siegel BS, Caplan SE, Nelson KG. A survey of 19 Bland CJ, Starnaman S, Wersal L, Moorhead-Guidelines as well as the many contributors and undergraduate pediatric education: Progress Rosenberg L, Zonia S, Henry R. Curricularreviewers. Special thanks are offered to our in the 1980s? Am J Dis Child. 1988;142:519 –consultants for the 2004 online edition; members change in medical schools: How to succeed. 523.of the Guidelines’ national advisory board, who Acad Med. 2000;75:575–594. 9 National curricular guidelines for third-yearprovided guidance throughout development of 20 Beach PS, Bar-on M, Baldwin CD, Kittredge family medicine clerkships. The Society ofthis edition, under the able leadership of Kenneth Teachers of Family Medicine (STFM) D, Trimm RF, Henry R. Evaluation of the useRoberts; and the loyal and indispensable section Working Committee to Develop Curricular of an interactive, online resource foreditors for the 2004 Guidelines. Guidelines for a Third-Year Family Medicine competency-based curriculum development. Clerkship. Acad Med. 1991;66:534 –539. Acad Med. 2009;84:1269 –1275. 10 Ambulatory Pediatric Association; Council 21 Peters TJ, Waterman RH. In Search ofThe projects described in this article were on Medical Student Education in Pediatrics. Excellence: Lessons From America’s Best-Runsupported by the Academic Pediatric Association General Pediatric Clerkship Curriculum and Companies. New York, NY: Harper and Row,(1992–2007); the DHHS, Bureau of Health Resource Manual. McLean, Va: Ambulatory Publishers, Inc.; 1982.Professions, Contract 103HR940857 (1994 –1995); 22 Day GS. Market Driven Strategy: Processes Pediatric Association; 1995.the American Board of Pediatrics Foundation for Creating Value. New York, NY: The Free 11 Olson AL, Woodhead J, Berkow R, Kaufman(1996); the Pfizer Foundation (2000 –2001); and NM, Marshall SG. A national general Press; 1999.the Josiah Macy Jr. Foundation (2002–2004). pediatric clerkship curriculum: The process 23 Kotter JP. Leading Change. Boston, Mass: of development and implementation. Harvard Business School Press; 1996. Pediatrics. 2000;106(1 pt 2):216 –222. 24 Carroll RG. Design and evaluation of a 12 CDIM-SGIM Core Medicine Clerkship national set of learning objectives: TheReferences Curriculum. Available at: (http://www.im. medical physiology learning objectives 1 Education Committee of the Ambulatory org/Resources/Education/Students/Learning/ project. Adv Physiol Educ. 2001;25:2–7. Pediatric Association. Educational Guidelines Pages/CDIM-SGIMCoreMedicineClerkship 25 Bowen JL, Clark JM, Houston TK, et al. A for Training in General/Ambulatory Curriculum.aspx). Accessed May 21, 2009. national collaboration to disseminate Pediatrics. McLean, Va: Ambulatory Pediatric 13 Bass EB, Fortin AH, Morrison G, Wills S, skills for outpatient teaching in internal Association; 1985. Mumford LM, Goroll AH. National survey of medicine: Program description and 2 Kittredge D, Baldwin CD, Bar-on M, et al, clerkship directors in internal medicine on preliminary evaluation. Acad Med. 2006;81: eds. Educational Guidelines for Residency the competencies that should be addressed in 193–202. Training in General Pediatrics. McLean, Va: the medicine core clerkship. Am J Med. 1997; 26 Collins JC, Porras JL. Try a lot of stuff and Ambulatory Pediatric Association; 1996. 102:564 –571. keep what works. In: Built to Last. New York, 3 Kittredge D, Baldwin CD, Bar-on ME, Beach 14 Goroll AH, Morrison G, Bass EB, et al. NY: HarperCollins Publishers, Inc.; 1997. TS, Trimm RF, eds. APA Educational Reforming the core clerkship in internal Guidelines for Pediatric Residency. MedEd 27 Glassick CE. Boyer’s expanded definitions of medicine: The SGIM/CDIM Project. Ann PORTAL. Available at: (http://www.aamc.org/ Intern Med. 2001;134:30 –37. scholarship, the standards for assessing medportalID 1736). Accessed July 2, 2009. 15 Society of Teachers of Family Medicine scholarship, and the elusiveness of the 4 Accreditation Council for Graduate Medical (STFM) Web site. Available at: scholarship of teaching. Acad Med. 2000;75: Education. Program requirements for (http://www.stfm.org/about/index.cfm). 877– 880. residency education in pediatrics. In: Accessed May 21, 2009. 28 Fall LH, Berman NB, Smith S, White CB, Graduate Medical Education Directory, 16 Kligler B, Gordon A, Stuart M, Sierpina V. Woodhead JC, Olson AL. Multi-institutional 1996 –1997. Chicago, Ill: Accreditation Suggested curriculum guidelines on development and utilization of a computer- Council for Graduate Medical Education; complementary and alternative medicine: assisted learning program for the pediatrics 1996;200 –206. Recommendations of the Society of Teachers clerkship: The CLIPP Project. Acad Med. 5 Accreditation Council for Graduate Medical of Family Medicine Group on Alternative 2005;80:847– 855. Education, Outcome Project 2002–2011. Medicine. Fam Med. 1999;31:30 –33. 29 Computer-assisted Learning in Pediatrics Available at: (http://www.acgme.org/ 17 STFM Curriculum Resources Project. Program (CLIPP). Available at: (http://www. Outcome). Accessed May 21, 2009. Available at: (http://www.fmdrl.org/index. clippcases.org). Accessed May 21, 2009.1268 Academic Medicine, Vol. 84, No. 9 / September 2009