Presenting to the residency program director and her team, I covered the research found within the "during the interview" section. I also designed the slides and answered questions from the audience as necessary.

1. PGY-1 and PGY-2 Interview
Process and Candidate Success
A Recommended Plan
Amy Turner, Anna Zschuppe, Jacob Motley, Darby Friend,
Kendra Stevenson

2. Executive Summary
● Comprehensive evaluation of the selection process
● Ensure selection of high-talent individuals
Objective
Research
Data Analysis
Solutions
● JHACH application data and chi-squared tests
● JHACH interview characteristics and scores
● National statistics and characteristics
● Current practices within pharmacy residency programs
● Aligning selection process with organizational needs
● Track data records and adjust procedures as necessary

3. Objective
#1
To identify
potential
biases
Interview
Process
1. Before
2. During
3. After
Objective
#4
To
recommend
best practices
Objective
#2
To evaluate
scoring tool
accuracy
Objective
#3
To assess the
selection
process

4. Before the Interview
Process

5. Before the Interview Process
Summary of Research and Data Findings
● Researched application components that predict matching success
● Compared results to 2016 PGY-1 application data for JHACH
○ Chi-squared tests
○ Significance of p-values
● Identified limitations of research and data
● Suggested recommendations for PhORCAS table and applications
● Reviewed JHACH interview itinerary vs. national characteristics

6. Data Gathered
● Reviewed 31 PGY-1 applications and 7 PGY-2 applications from 2016
● Focused on PGY-1 2016 only
○ Consistent
○ Large enough to run data analysis
● Calculated some data points and others straight from applications

7. Limitations
Isolating Effects
● Unable to detect biases in college reputation
● No demographics on applications to identify age/race discrimination
Coverage of Applications
● Delving into only PGY-1 programs
● Only able to analyze 2016 data due to inconsistencies elsewhere
Data Collection
● Limited scope of data (less than 30 data points)
● High p-values calculated

8. Chi-Squared Tests Results
Chi-Squared Tests and P-Values of 2016 Application Dataset

9. Recommendations
Tweak Application Process
● Accept more applicants to increase future return on investment
● Computerize applications to extrapolate data easily
● Applications should ask demographic questions
Adjust PhORCAS Table
● Weights should reflect factors that other hospitals and programs use
● Implement constant 0-4 scale for each component
● Objectify custom fields definitions

10. Recommendations
Assess Recruitment Procedures → JHACH Itinerary
● Medical team rounds
● Facility tour
● Rounds in appropriate care unit
● Candidate presentations
● Break/lunch
● Panel interviews
○ Session #1: JHACH residents
○ Session #2: Pharmacy administration
○ Session #3: Preceptors
● Patient case with resident program director (RPD)

11. Recommendations
Great Interview Components
● Lunch
● One-on-one interviews
● Presentations
● SOAP note/patient case
Components to Consider
● Ones with high percentages
● Differentiating interview
process
Why?
Widely used and have <0.05 p-value

12. During the Interview
Process

13. During the Interview Process
Suggestions Based on Research Findings
● Part I: Evaluate desired qualities within interview procedures
● Part II: Understand current and prospective residents
○ Four-step behavioral interview process
○ Employee persona template
● Part III: Modify interviewing scores and rating systems
● Part IV: Train interviewers appropriately

14. Part I: Evaluating Desired Qualities
Summation of Interview Scores for PGY-1 2016

15. Research Takeaways
● Most components overlap with O*NET
characteristics
● Area of Issue: Program fit
○ Only quality doubled in interviews
○ “Like-me bias”: subconscious
preference due to similar personalities
○ Can use in case of a tiebreaker
Limitations to Results
● National results consider more than just
pharmaceutical occupations
● Some skills are not necessary
for industry
Part I: Evaluating Desired Qualities

16. Part II: Understanding Current & Future Residents
Gathering Research
● Look inwardly to receive feedback from employees
○ Formulate questions for interviews
○ Engage and interact with workers
○ Create long-term tracking processes
○ Connects to organizational strategy and mission
● Patterned Behavior Description Interview (PBDI)
○ Developed by Gallup, Inc.
○ Higher levels of predictive validity for selecting employees
○ Can decrease turnover and increase employee performance and retention
○ Can outline expectations for future residents from an internal
standpoint

17. Part II: Understanding Current & Future Residents
FOUR-STEP BEHAVIORAL INTERVIEW PROCESS
Step 1: Conduct Focus Groups and Workplace Evaluations
● Collect insights from key executives and top-performing employees
● Ask questions to understand an employee’s daily activities
○ “Describe the organization’s culture.”
○ “What do you like about your job?”
● Gain better understanding of job demands and expectations
Step 2: Study Group and Contrast Group Research
● Gauge behaviors of different employees and their behaviors
● Exit interview data between high and low/average performers
● Create employee persona template to depict a typical worker’s attributes

18. Part II: Understanding Current & Future Residents
FOUR-STEP BEHAVIORAL INTERVIEW PROCESS
Step 3: Development of Themes/Questions
● Look externally through hiring procedure
● Use identified qualities to evaluate individuals during interview process
● Cross-reference themes and qualities with their performance
● Keep desired skills in check with market
Step 4: Measuring Effectiveness
● Determine how well responses match wanted job behaviors
● Aggregate scores and use to create overall rating
● Adjust central themes as necessary

19. Part II: Understanding Current & Future Residents
Sample of Persona Template
Creating a Persona Template
● Beyond demographics and
psychographics
● Humanizes ideal candidates
by gauging emotional
triggers
● Mapping out a particular
person’s life and success
story
● Similar to a buyer persona
used by marketers

20. Part II: Employee Persona Template Example
Recommended Employee Persona Template for Johns Hopkins All Children’s Hospital
Application for JHACH
● Create ones for PGY-1 and PGY-2
● Display pictorially and in writing
How JHACH Will Benefit
● Help satisfy employee wants and needs
● Understand applicant audience from
micro-behavioral level
● Improve hiring and internal
communications process
● Align organizational strategy with
recruiting process

21. Part III: Modify Interviewing Scores and Rating Systems
Behaviorally Anchored Rating Scale (BARS)
● Descriptive definitions provided to objectively quantify subjective behaviors
○ Allows interviewers to be on the same page as to behaviors and answers
deserve what rating
○ Shift scale from 0-3 to 1-4
● JHACH’s issue with half points
○ Raters giving half points because scale is ambiguous
○ Through descriptions create clear cut levels for interviewers to rate in

22. Part III: Modify Interviewing Scores and Rating Systems
Sample Interview Rating Scale
Suggested Rating Scale

23. Part IV: Train Interviewers Appropriately
Research Findings
● Need to train interviewers on errors
● Avoid lecturing – makes interviewers aware but not hypersensitive to issues
● Implement interactive, two-way teaching process
○ Simulations
○ Group discussions
○ Viewing tapings of interviews
Application for JHACH
● Educate interviewers properly and uniformly
● Reduce inconsistencies in ratings
● Negate discrepancies if candidate has had previous contact

24. After the Interview
Process

25. After the Interview Process
Track Training Module Usage
● Identify correlations between learning modules and resident success
● Fulfill knowledge gap vital to the industry
● Study results to find areas that residents may need additional support
Monitor JHACH Strategies
● Evaluate interview questions as a whole
○ Reliable: Free from random error
○ Valid: Actually relates what is to be measured
● Adjust processes as necessary to changes within the industry
● Improve record keeping and data collection

26. Conclusion

27. Q&A