Medical Educator's Portfolios: How To Be Recognized and Promoted For Your Technology-Enhanced Medical Education Scholarship
Medical Educators Portfolios: How To Be Recognized and Promoted For Your Technology-Enhanced Medical Education Scholarship"Donna M. DAlessandro, M.D."firstname.lastname@example.org"Michael P. DAlessandro, M.D."email@example.com"Educational Informatics"www.educationalinformatics.org"www.slideshare.net/michaeldalessandro"Departments of Pediatrics and Radiology"University of Iowa College of Medicine / University of Iowa Childrens Hospital"
About Us"• Educational Informatics Laboratorys vision since 1989"• To improve patients care, outcomes and lives;"• By changing physicians knowledge, attitudes, and behaviors;"• Through the creation and evaluation of tools, techniques, and procedures that shiftlearning from the classroom and lecture hall to the point-of-care"• Strong record of innovation in medical education"• Resulting in grants, peer-reviewed publications, awards"• Serve 2 million learners / year ~ highest impact factors on Internet"• Experience on promotion + tenure committees at Department + College levels"
Workshop Outline"• What is a medical educators portfolio?"• What do you put in a medical educatorsportfolio?"• How do you include your technology-enhanced educational scholarship in amedical educators portfolio?"• How do you interpret what is in the medicaleducators portfolio? "
Introduction""The worst thing that can happen for a Promotionand Tenure committee is to not have data, and thesecond worst is to have a disorganized dossier.""- Interview comment from Simpson et. al. Acad Medicine, 2004 "• Our goal"• Help you to gather and organize your""Evidence of educational scholarly achievements""…and thus help you to manage your career"
Uses of the Medical Educators Portfolio"• For self-reﬂection and learning"• In the promotion and tenure process"• To assist in advising and mentoring others"• To empower diversity in scholarship"• For "overcoming universal entropy""
Some Questions"• Where are you from and what do you do?"• What is a question you want answered in thisworkshop?"
Some Deﬁnitions"• Scholarship deﬁnition"• In 1990, Ernest Boyer deﬁned scholarship in four domains:"• 1. Discovery - i.e. traditional research; "• 2. Integration - collaborative connections across disciplines; "• 3. Application - vital interaction between research and practice; "• 4. Teaching (educational scholarship) - the creation of new knowledgeabout teaching and learning in the presence of learners "• Educational Scholarship deﬁnition ""Educational scholarship refers to any material, product or resourceoriginally developed to fulﬁll a speciﬁc educational purpose that has beensuccessfully peer-reviewed and is subsequently made public throughappropriate dissemination for use by others." - AAMC Web site 2005"• Medical Educators Portfolio deﬁnition""Portfolio is a systematic collection of information documenting expertise inan area, usually incorporating multiple sources of information collectedover time to demonstrate excellence." - Seldin P. 1997. "
Example Medical Educators Portfolio"http://www.mededportal.org/publication/626"
Medical Educators Portfolio Contents"• Teaching"• Volume - detailed list of venues taught (e.g. classroom, clinic,laboratory), number / type of learners (e.g. medical students, nurses,patients), number of hours taught, teaching modality used (e.g. lecture,discussion, case-based presentation, etc.) "• Honors and awards - teacher of the year, unsolicited letters "• Assessment of learnersʼ performance - in-training / boardexaminations, OSCE results, pre / post assessments"• Learner assessments of oneʼs teaching - demonstration of basicteaching skills or as an excellent teacher - learner ratings, unsolicitedletters"• Peer assessment of teaching - demonstration of basic teaching skillsor as an excellent teacher, recognition as a national leader in education- peer evaluations, videotape evaluations, honors and awards,invitations for dissemination of teaching "• Curriculum development / innovation / evaluation - examples anddocumentation of ones best work with discussion of the philosophy,application and evaluation / outcomes of the curriculum or othereducational project"• Continuing education for oneself - participation at local or nationalmeetings to improve own skills "
Medical Educators Portfolio Contents"• Advising / Mentoring "• Volume of learner advising - list of venues of advising, number / typesof advisees, number of hours or length of time advising, advisingmodality (e.g. role model, formal advisor, etc.) "• Learner assessment of advising - advisor ratings, unsolicited letters"• Other evaluation of advising - advisees honors, publications,leadership positions etc. "• Mentorship of other faculty locally and nationally - national facultydevelopment program"• Research - supervision, mentoring, thesis / dissertation committees"• Educational Administration / Leadership"• Education committees - department, medical school, national includingoneʼs role, service dates, special projects "• Accreditation committees - including resident review committee,ACGME, GME, etc. "• Leadership positions - clerkship director, residency program director,dean, national organizations educational activities "• Curriculum development including role, dates of service, specialprojects (see also Teaching section)"
Medical Educators Portfolio Contents"• Dissemination / Scholarship of Teaching "• Peer-reviewed publications with contributions listed: conceptionand design, data acquisition, data analysis and interpretation,manuscript drafting, manuscript critical review, statistical expertise,senior author or other special designation "• Books / chapters "• Editorial services - reviewer, editorial board "• Invited / keynote presentations - locally, nationally "• National meeting presentations on educational topics "• Grants and contracts "• Peer recognition "• Personal Statement (at beginning or end)"• Philosophy of teaching"• Long term goals as an educator"
What You Need From Your Medical EducationWeb Site / App / Ebook for Your MedicalEducators Portfolio"• Why create a medical education Web site / Ebook / App?"• To teach + enhance your reputation as medical educator"• Is valid form of educational scholarship ~ Is a scholarly work"• Counts towards promotion / tenure if documented "• Documentation of scholarship"• Design ~ Build ~ Utilize ~ Evaluate ~ Reﬁne"• Traditional metrics for scholarly works"• Grants obtained"• Peer-reviewed articles published"• Evolving metrics for scholarly works ~ Web 1.0 metrics"• Peer reviews conducted by organizations"• AAMC MedEdPORTAL, ACGME, Merlot"• Reviews published in print journals"• Awards"• Bibliometric analysis / Impact factor"• Web statistics ~ overall usage: pages read / visitors "• Users comments (fan mail = true peer review) "
Three Step Approach To Data Gathering"• 1. Instrumenting your Web site / Ebook / App to gather all data"• Installing a Web statistics package for Web site"• SiteMeter - www.sitemeter.com"• Google Analytics - www.google.com/analytics"• Utilizing preexisting App Store statistics packages for Ebook / App"• Amazon Kindle / Apple iTunes / Google Play"• Installing a comment / feedback form"
Three Step Approach To Data Gathering"• 2. Web publicity and marketing"• Submit Web site to "• (Consider creating marketing Web site for Ebook / App)"• Search engines - google.com, www.bing.com"• Internet Archive - www.archive.org"• Internet directories - www.dmoz.org"• Health sciences directories - www.martindalecenter.com"• Health on the Net - www.hon.ch"• Add links to related articles in Wikis"• www.wikipedia.org, www.wikibooks.org"• Index in Open Education Resources repositories"• AAMC MedPortal, www.curriki.org, www.e-booksdirectory.com, www.lemill.net,www.merlot.org, www.oercommons.org, textbookrevolution.org, wlh.webhost.utexas.edu"• Send announcement"• groups.google.com (misc.education.medical), groups.yahoo.com, www.medscape.com,www.sermo.com, Facebook groups, Twitter, society mailing lists, friends / colleagues"• Check search engines for top 100 hits for related terms / keywords"• Ask these sites to trade links"• Send e-mails to journal reviewers"• Professional meetings ~ hand out ﬂyers ~ teach workshops"• News Alerts to track your progress - www.google.com/alerts"
Three Step Approach To Data Gathering"• 3. The regular maintenance, evaluation and continuousquality improvement of Web site / Ebook / App"• Usually most neglected step ~ Why most fail"• Evaluation is key - listen to your users"• Take the long view - commit to 10 years"• Keep it simple stupid! - in terms of design + technology"• Always have a backup of the Web site / Ebook / App"• Keep a diary from day one"• Yearly curation / scholarship harvest"• Reconﬁrm Step 1 - your mission / audience / goals / metrics"• Check - links / HTML / accessibility"• Preserve - archive, print, screen shot"• Measure evolving metrics - inbound links, Web statistics, bibliometricanalysis, downloads"• Measure Altmetrics - ImpactStory.org"• Search for reviews - LexisNexis, Google News Archive, GoogleScholar, Amazon Search Inside Books, Google Books"
Results of Evolving Metrics = Peer Review By Quantitative Acclimation (What You Need From Your Medical Educators Portfolio For Your Medical Education Web Site / App / Ebook)"www.pediatriceducation.org/reviews.html"
Altmetrics Web 2.0 metrics"• Bibliometrics = traditional techniques for measuringscholarly impact"• Citations, h-index, i10-index"• Altmetrics = alternative metrics of measuring scholarlyimpact"• Creation and study of new metrics based on the social Web foranalyzing and informing scholarship ~ Altmetrics.org"• Measures Web-driven scholarly interactions"• How often are research products cited, downloaded, shared, viewed,mentioned, blogged, tweeted, bookmarked, commented on, liked, etc."• Example research products being measured"• For published articles in PubMed, et.al"• For software in GitHub"• For dataset in Dryad or ﬁgshare"• For slides in SlideShare"• For any Web site or Web resource"
Your First Step Towards Altmetrics ImpactStory.org"5 minutes to set up if you have Google Scholar Proﬁle"Results of Altmetrics = Peer Review By Quantitative Acclimation"
Promotion Lessons Learned"• Understand the criteria for promotion"• Get on promotions committee to see criteria applied and to see howothers are viewed"• The arc of progress is crucial"• Your work should not be all over the place; Dr. X = Y"• It is a personality contest, it is about who you are"• A nice personality is given much more leeway than a malignant one"• Writing the essay"• Tell me a story"• You are writing a legal document"• You are making your case for promotion"• Dont be shy about your accomplishments"• Back up your claimed accomplishments with facts"• There are evil forces out there"• The only way to ﬁght evil is with facts and data that is wellorganized and presented"
The Essay"• Teaching"• Local - formal - lectures: undergraduate, graduate, continuing"• Local - informal: at conferences, in labs, in clinics / on wards"• National / International: lectures - continuing, visiting professor"• Service"• Clinical"• Responsibilities, % time, RVUs compared to others, clinical revenue"• Administrative"• Local - committees: departmental, hospital, college"• Local - courses, curricula"• National / International - committees, editorial boards"• Scholarship (only place to describe your scholarship)"• Papers"• Grants"• For each medical education Web site / App / Ebook"• Traditional and evolving metrics / altmetrics"• Have someone read it"• Make sure contents meet all requirements"• Have supplementary materials organized logically"
The Essay - Explaining New Metrics"Digital library scholarship can be measured using traditional metrics such as grants obtained for it andpeer reviewed articles published about it as well as evolving metrics such as peer reviews conductedby organizations such as the AAMC and ACGME, reviews published in journals, bibliometric analysis,overall usage of it, and comments from users."In traditional print publishing, through bibliometric analysis performed by Thomson-Scientiﬁc, eachjournal article and each journal can be assigned an impact factor, which is a measure of authority,based upon the number of citations each receives from other journal articles. Web publishing hasquickly evolved in an identical manner, and through bibliometric analysis each Web page and eachWeb site can be assigned an impact factor, which is a measure of authority, based upon the numberof links each receives from other Web pages. Additionally, it has been shown that the impact factor ofelectronic editions of print journals on the Web correlates signiﬁcantly to the impact factor of theprint edition of the journal."The top three search engines (Google, Baidu, Bing) account for 81% of all searches on the Internet, eachusing a unique and proprietary algorithm to perform their bibliometric analyses. The end result is thatthe Web pages and Web sites with the highest impact factors are listed ﬁrst in the results returned bya search engine for a given term."Today, the Internet is rapidly becoming the most common place that answers are sought to questions inmany domains, including medicine. Most users (60%) doing a search never look beyond the ﬁrst pageof search engine results (ﬁrst 10 hits) and an additional 30% of users never look beyond the ﬁrst threepages of results (ﬁrst 30 hits). Therefore, being on the ﬁrst page of results for a given search term is ahighly inﬂuential place to be."For the last several years, for the search engine terms "pediatrics," "general pediatrics," and "pediatriceducation" my digital libraries are consistently displayed on the ﬁrst page of search results / ﬁrst 10hits for each of the top 3 search engines. This means that for these search engine terms, my digitallibraries have the highest impact factors on the Internet and are therefore amongst the top 10authoritative Web sites on these subjects."
Questions on Construction"• Who has a medicaleducators portfolio /teaching portfolio? "• How did you construct yourportfolio?"• How do you maintain yourportfolio?"• What is in your medicaleducators portfolio?"• What else do you thinkshould be in a medicaleducators portfolio?"
Questions on Use"• Have you or anyone you know been promoted forcomputer-based medical education scholarship?"• If so, how did they do it?"• What problems do you already anticipate from yourinstitution?"• Who can you turn to for help?"• What problems have you already encountered with yourtechnology-enhanced scholarship and how have you ﬁxedthem?"• For the already promoted, what problems did you face andwhat words of wisdom can you give others?"• Other comments? "• Tips and tricks? Words of wisdom?"
Conclusion / Call For Action"• You are now armed with the knowledgenecessary to successfully create a medicaleducators portfolio"• Your plan of action"• Will you create or reﬁne a medical educatorsportfolio?"• What are / is your ﬁrst steps?"• When will have your ﬁrst step done?"• Put this by your computer…"
References For Medical Educators Portfolio"• A medical educators portfolio"• Educator Portfolio Template of the Academic Pediatric AssociationsEducational Scholars Program. "• Gusic ME, Chandran L, Baldwin CD, DAlessandro DM, Balmer DF.MedEdPORTAL; 2007. "• Available from: http://www.mededportal.org/publication/626"• Also at: http://www.ambpeds.org/education/educator_portfolio_template.cfm"• How to use the medical educators portfolio"• The Educator Portfolio: A Tool For Career Development."• Baldwin CD, Gusic ME, Chandran L. AAMC Faculty Vitae 2008."• Available from: http://www.aamc.org/members/gfa/faculty_vitae/148574/educator_portfolio.html"• Evaluation tool of the medical educators portfolio"• Evaluating the performance of medical educators: a novel analysis tool todemonstrate the quality and impact of educational activities. "• Chandran L, Gusic M, Baldwin C, Turner T, Zenni E, Lane JL, Balmer D, Bar-OnM, Rauch DA, Indyk D, Gruppen LD. Acad Med. 2009 Jan;84(1):58-66."
References"• American Association of Medical Colleges MedEdPortal. EducationalScholarship for Promotion and Tenure. Available from the Internet at:http://www.mededportal.org/download/190392/data/educational_scholarship_guide.pdf (rev. 2009, cited 4/2013)."• Boyer EL. Scholarship Reconsidered: Priorities for the Professoriate. 1990;The Carnegie Foundation the Advancement of Teaching: Princeton, NJ. "• Glassick CE, Huber MR, Maeroff GI. Scholarship Assessed: Evaluation ofthe Professoriate. 1997; San Francisco, CA: Jossey-Bass. "• Manion Fleming V, Schindler N, Martin GJ, DaRosa DA. Separate andEquitable Promotion Tracts for Clinician-Educators. JAMA. 2005; 294(9)1101-1104. "• Ruiz JG, Candler CS, Qadri SS, Roos BA. E-Learning as Evidence ofEducational Scholarship: A Survey of Chairs of Promotion and TenureCommittees at US Medical Schools. Acad Med. 2009; 84(1): 47-56."• Seldin P. The Teaching Portfolio: A Practical Guide to ImprovedPerformance and Promotion/Tenure Decisions. Second Edition. Bolton,MA: Anker Publishing Company, 1997. "• Simpson DE, Haﬂer J, Brown D, Wilkerson L. Documentation Systems forEducators Seeking Academic Promotion in U.S. Medical Schools. AcadMed. 2004; 79(8): 783-90. "To see a variety of medical education Web sites and Ebooks, visit www.educationalinformatics.org"To stay up-to-date on digital scholarship, visit ProfHacker at chronicle.com/blogs/profhacker/"