Your SlideShare is downloading. ×
ADEA 2010 Competency Based Evaluation Of Predental Applicants
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

ADEA 2010 Competency Based Evaluation Of Predental Applicants

1,414
views

Published on

New Idea Session on Competency-Based Evaluation of Predental Students from the data-driven evaluation side (predental advising and letters of evaluation).

New Idea Session on Competency-Based Evaluation of Predental Students from the data-driven evaluation side (predental advising and letters of evaluation).

Published in: Education, Technology

1 Comment
0 Likes
Statistics
Notes
  • Thank you for presenting at the recent 87th ADEA Annual Session & Exhibition in Washington, DC. Below is a your evaluation summary averages results based on the four outlined areas listed below.

    Ranking Scale: 4.0 – 1.0 with 4.0 being the Strongly Agree (the highest rating)

    Number of Responses 19

    Demonstrated Knowledge of the audience and the subject matter. = 3.58
    The content presented was relevant and delivered in a logical manner. = 3.26
    I would participate in another session led by this presenter. = 3.32
    Grand Total = 3.39
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Views
Total Views
1,414
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
1
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • http://www.nytimes.com/2010/02/15/education/15medschools.html
  • http://www.nytimes.com/2010/02/15/education/15medschools.html
  • http://www.aamc.org/data/facts/applicantmatriculant/table24-mcatgpa-grid-3yrs-app-accpt.htm
  • Wirtzberg quote: http://www.aamc.org/opi/holisticreview/resources/witzburg.pdf Taken from www.creid.ed.ac.uk/events/disabledlearners.../LTA_healey_roberts.ppt .
  • Royalty free photos: http://www.fotosearch.com/bigcomp.asp?path=CSP/CSP176/k1761429.jpgAAMC diversity self-assessment: http://www.aamc.org/opi/holisticreview/resources/selfassessmentworksheet.pdf
  • April 5, 2003, UNC ballroom dance club practice with RobertasMaleckis (personal copy: Emil Chuck)
  • Multiple intelligence theory
  • AtulGawande speaking at Politics and Prose, photo by Emil Chuck.
  • Transcript

    • 1. Can progress in “Holistic admissions be measured”?
      Emil T. Chuck, Ph.D.
      Health professions Advisor
      Term Assistant Professor of Biology
      George Mason University
      2010 ADEA Annual Session
      Saturday, February 27, 2010
      3:00 to 3:25 pm, Potomac 5 (New idea session)
      Some data shown are preliminary and still being collected for future evaluation.
      Competency-Based Evaluation of Predental Applicants
    • 2. Can progress in “Holistic admissions be measured”?
      Competency-Based Evaluation of Predental Applicants
      • Recognize how principles of holistic review are applied to predental advising for applicants and committee-based evaluations.
      • 3. Discuss whether the evaluation criteria fit professional competencies that are valued by the dental profession.
      • 4. Compare dental and medical admissions outcomes trends as measured by the holistic rubric.
    • How “the public” views admissions
      “Peter Allen applied to 30 medical schools after graduating from the University of Pittsburgh last year. Twenty-eight said no. …
      “His pre-med adviser told him that with his 3.3 grade-point average, he should apply only to osteopathic schools, but he persisted, and was admitted to The Commonwealth and New York Medical College in Valhalla, N.Y.”
      “Expecting a surge in U.S. medical schools” by AnemonaHartocollis (Feb 14, 2010 NYT)
      Peter Allen and Jessica Sidari, first-year students at The Commonwealth Medical College, a first-year school in Scranton, Pa
      Credit: Niko J. Kallianiotis for The New York Times
    • 5. How prehealth advisors(?) view admissions
      Comment #184: “I must laugh at the credentials that this white student has. As an advisor for pre-med students at an Ivy-league school, my Asian students with 3.7-3.8 GPAs in the sciences have a rough time even getting interviews. A 3.3 GPA is a laughable GPA and even more laughable coming from a school like Pitt.”
      “Expecting a surge in U.S. medical schools” by AnemonaHartocollis (Feb 14, 2010 NYT)
      Peter Allen and Jessica Sidari, first-year students at The Commonwealth Medical College, a first-year school in Scranton, Pa
      Credit: Niko J. Kallianiotis for The New York Times
    • 6. AAMC GPA/MCAT (2005-2007 accepted %)
    • 7. “The system produces the result for which it was designed.”Bob Witzberg MD, Boston University SOM, AAMC Holistic Review Project
    • 8. Holistic Review:Choosing from a pool of applicants
      Established substantive criteria for admissions that
      Reflect a justifiable balance among academic accomplishments and personal factors in applicants given the school’s mission-related goals.
      The school must have evidence for its admissions considerations with regard to:
      Academic background (major, GPA, letters, essays, etc.)
      Personal characteristics (background)
      Personal attributes (altruism, leadership, service)
      Personal experiences (relevant experiences, life experiences)
      Other criteria (please specify)
    • 9. Competencies and Learning
      April 5, 2003, UNC ballroom dance club practice with RobertasMaleckis, Credit: Emil Chuck (personal)
    • 10. Goals of competency-based evaluation
      Criteria should align with articulated technical standards of admissions (based on multiple personality theory).
      “Legal compliance” arguments tend to have more resistance to acceptance.
      Criteria should be transparent and independent of professional degree pursued by applicant to should reflect qualities desired for an interprofessional health care workforce.
      Can compare pools of applicants in the same year for different paths.
      Applicants should become familiar with competency-based evaluations as it will be a facet of their professional training and practice.
      Criteria should assist students with self-evaluation and references with confirmatory evaluation (360-evaluation)
      Quantitative measures should inform advising, program evaluation, and admissions outcomes.
    • 11. GMU Prehealth Evaluation Rubric
      Compare to technical standards of admissions
      Academic foundation (science and non-science breadth/rigor)
      Scholarship and lifelong learning (problem-solving, research, entrepreneurship)
      Social intelligence (interpersonal, teamwork, collegial)
      Personal awareness (maturity, ethics, resiliency)
      Managerial experience (time, financial, project, leadership)
      Aesthetic or observational skills (art, music, literature)
      Kinesthetic dexterity (lab methods, sculpture, electronics)
      Communications skills (written/oral English, other)
      Knowledge of the profession and health care system
      Evaluators usually cannot comment on specific fitwith institutional mission and culture!
    • 12. Competencies and Recommendations
      Newcomer: just starting out.
      Novice: follows instructions without questioning.
      Intermediate: memorizes steps, needs more practice.
      Proficient: can perform the tasks satisfactorily (minor difficulties).
      Competent: fluent; can improve the task/outcome by making minor adjustments.
      Expert: tests improvements to tasks in a systematic manner.
      Master: honored for exceptional skill development.
      No/Concern: general newcomer/novice to competency concepts.
      General: generally satisfactory/proficient in competencies
      Strong: shows additional development (competent) with competencies
      Enthusiastic: generally expert in competencies
      Highly: consistently honored (especially by peer review processes) to be exceptional
    • 13. Example self-assessment
      Awards declared: I have made the dean’s list multiple times.
    • 14. EY2011 preapplication pipeline
    • 15. Rubric self-assessment
      EY2011 Applicants (n=147)
      Most confident:personal, emotional resilience and intelligence
      Least confident: academic foundation; mentoring/advising relationships.
    • 16. Rubric self-assessment
      EY2011 Applicants (n=39) Predental
      Most confident:personal, emotional resilience and intelligence; observational skills; manual dexterity
      Least confident: academic foundation; mentoring/advising relationships.
    • 17. Solicited evaluators assessment
      PRELIMINARY DATA ONLY (n=52), for EY2011 applicants, as of 24 Feb 2010.
      Solicited evaluators may overrate applicants’ skills development.
      Solicited evaluators rarely can comment on manual dexterity skills.
    • 18. Committee evaluators assessment
      PRELIMINARY DATA ONLY (n=72), for EY2011 applicants, as of 26Feb 2010.
      Committee interviewers are more likely to detect lower-competency applicants (novice to proficient).
      Committee interviewers have more difficulty assessing manual dexterity & applicants’ collegial relationships with other advisors or faculty.
    • 19. Evaluators’ summary assessment
      Committee EvaluatorsPRELIMINARY DATA ONLY (n=72), for EY2011 applicants, as of 26 Feb 2010.
      Solicited Evaluators PRELIMINARY DATA ONLY (n=52), for EY2011 applicants, as of 24 Feb 2010.
    • 20. Solicited evaluators assessment
      PRELIMINARY DATA ONLY (n=53), for EY2011 applicants, as of 26 Feb 2010.
      Solicited evaluators have a relatively casual relationship or a knowledgeable relationship with applicants.
      5/39 evaluators consider themselves “mentors” to applicant.
    • 21. 360-assessment reveals calibration issues
      Applicants claim most confidence in their intrapersonal skills (personal resilience, hardship, evaluating criticism, faith).
      Applicants are least confident in academic skills and having strong mentoring relationships.
      Few solicited evaluators are “mentors” to applicants.
      Solicited evaluators will overrate/exaggerate the development of applicants’ skills (except for manual dexterity) compared to committee evaluators.
    • 22. “Percent chance of admission” EY 2009
    • 23. GPA applicants/accepted EY 2009
    • 24. Average GPA’s for EY 2009 applicants
      All admissions decisions (GMU)
    • 25. Average GPA’s for EY 2009 applicants
      All AADSAS 2009 admissions decisions (GMU)
    • 26. Average GPA’s for EY 2010 applicants
      PRELIMINARY admissions decisions (GMU) as of February 19, 2010
    • 27. Average GPA’s for AADSAS EY 2010 applicants
      PRELIMINARY AADSAS 2010 admissions decisions (GMU) as of February 19, 2010
    • 28. Translating rubric to evaluators
      Competency-Based Holistic Evaluation of Prehealth Applicants
      Chuck ET. The Advisor (NAAHP), June 2009.
      Available online as PDF
      http://www.slideshare.net/etchuck/advisor-june-2009-competency-based-holistic-evaluations
      Writing better evaluation letters (solicited and committee)
      Competencies as a checklist of topics to cover.
      Training admissions readers and interviewers.
      Compare competencies against technical standards and admissions criteria.
      Communicate and provide feedback (compare with any competency assessments after D1/D2/D4 ).
    • 29. Can holistic admissions be measured?
      Current challenges
      Applicants consider their competencies in their emotional maturity, integrity, and character as “expert”, as do solicited evaluators.
      Evaluators rate most applicants’ skills as “expert” though they do not usually have close mentoring relationships.
      Must better align evaluation rubric criteria with competency experiences/standards from professional education.
      Current opportunities with rubric
      “Research”-based standards of ideal health care professionals can be assessed, communicated, and “measured” with admissions outcomes.
      Improve prehealth advising and evaluation letters for applicants.
      Research any interprofessional (inter-institutional?) comparison of applicant holistic qualities.
      Introduce applicants to competency-based evaluation processes common in professional education and development.
    • 30.
    • 31. Trend towards holistic competencies
      AAMC Holistic Review Project
      National Postdoctoral Association Core Competencies
      AAMC/HHMI Scientific Foundations for Future Physicians
      Assessment projects throughout medical education
    • 32. It’s time for a checklist!
      Rubrics are checklists with an assessment angle.
    • 33. Committee evaluators assessment
      PRELIMINARY DATA ONLY (n=65), for EY2011 applicants, as of 24 Feb 2010.
      Majority (84.6%) of committee evaluators suggest that reapplicants should have another set of MSAC interviews.
    • 34. Average GPA’s for EY 2009 applicants
      All applicants (GMU)
    • 35. Recommendations for EY 2009 applicants
      All admissions decisions (GMU)
    • 36. Recommendations for EY 2010 applicants
      All admissions decisions (GMU) as of February 19, 2010
    • 37. Recommendations for EY 2010 applicants
      All admissions decisions (GMU) as of February 19, 2010

    ×