ACEM fellowship exams - an overview of marking

1. ACEM Fellowship Exams
A brief overview of the structure of
the exams and how they are marked

2. What this talk is not….
• Comprehensive study guide
• All encompassing list of tips and tricks to
preparing for study and ACEM exams
• Detailed insider knowledge and cumulated
experience of many successful candidates
If you want all that – you need to come to SCGH
ED Fellowship Teaching sessions Thursdays 1030

3. Denial of responsibility
• I am a
– DEMT
– ACEM examiner
– FEx OSCE working party member
– Standard setting panel member
• I have signed a
– Conflict of interest declaration

4. Two distinct Fellowship Exams
Written – 2 x 3 hour papers Clinical OSCE
• 3 x 60 minutes over 3 days
– 3 x 20 minute stations 1 day
– 6 x 10 minute stations 2 days
• 3 mins reading time 7
minutes exam
• 4 mins reading 16 mins
exam
• Double stations = double
marks
• Select Choice Questions
– 120 items
– Extended matching questions
– Multiple choice questions
• Short Answer questions
– 3 equal x 60 minute booklets
– Each question contains several
sub-questions, which require
concise, short answer
responses

5. Two Become One…
• Knowledge required
• Clinical experience required
• Type of thinking required
• You will hear the phrase ”consultant level” until
a) It drives you insane
b) Understand what people mean when they say it
c) Have succumbed to the 3 strikes you’re out rule
d) Are ready for retirement
– If you discover a good definition let me know

6. Now for some practice written
exam questions
Grab a pen and paper!

7. ACEM FEx written SCQ – MCQ examples

8. ACEM FEx written SCQ – EMQ examples

9. ACEM FEx written SAQ example
A 65 year old man presents to the emergency
department with a history of palpitations.
His vital signs are:
BP 105/60 mmHg
RR 26 /min

10. Answer the questions

11. Answer discussion
• MCQ
• EMQ
• SAQ
• How did you go?

12. Discussion on Answering
• Consultant level language
• Specific AND accurate
• No abbreviations
• Read the question
• Write your best answer
– Could it be better?
• You can and will and many people have failed
exams because of poor handwriting

13. Examples of FEEDBACK TO UNSUCCESSFUL
CANDIDATES
• Parts of the SAQ had no answer.
• A significant number of incorrect or irrelevant answers
were provided, leaving no space for the required correct
answers.
• The answer used vague or non-consultant level
terminology.
• Some of the mandatory answers were missing, meaning
the answer lacked prioritisation.
• The handwriting was illegible to the point it cost the
candidate marks.
• A response was provided or omitted which would cause
significant harm to the patient, thus the zero score rule was
applied to a sub question.

14. FEx Written results
• Exams are standard set
– Pass mark will reflect variation in difficulty
between exams
– Modified Angoff method
– Independent judgements are made of the
expected performance ‘just at standard’ candidate
for each question
• Standard setting determines cut score
• Cut score is politically correct way of saying pass mark
• “Nobody fails”

15. Recent ACEM FEx Written Results
• Exam reports available on website
Exam Candidates SCQ % rate SAQ % rate Overall pass
rate
2015.1 271 N/A N/A 88.%
2015.2 231 N/A N/A 60%
2016.1 198 71% 76% 64%
2016.6 259 62% 64% 50%
2017.1 201 61% 55% 47%

16. FEx Clinical OSCE
• Candidates assessed across multiple domains
– Curriculum framework
• Each station 2 - 4 domains
• Scenario/tasks given outside examination room
• Performance against observed and assessed by
examiner

17. OSCE exam advice
• The intent of the OSCE examination is to assess candidates
at a level consistent with a newly graduated FACEM
– Professional behaviours and interactions
– Communication, psychomotor and other relevant skills
– The application of medical expertise knowledge to various
clinical scenarios
• It is a performance based assessment
– Application of knowledge
• How it is applied matters…
• Wear consultant shoes to the exam

18. Now for a practice OSCE realtime
We are really going to do it…..
I need a volunteer!

19. 2016.1B STATION 15
MANAGEMENT OF A LIFE-THREATENING LOWER GI BLEED
CANDIDATE INSTRUCTIONS
You are working as a consultant in a tertiary Emergency Department. The ED junior registrar (first year
Advanced Trainee) seeks your advice about the management of 74 year-old Peter Smith, who presented
with a large-volume PR bleed.
Mr Smith has a background of atrial fibrillation and is on dabigatran. The registrar has completed the
initial assessment and will provide the history, examination and relevant investigations to date.
Your tasks are to:
• Establish key issues from the patient’s clinical assessment.
• Interpret available laboratory results.
• Advise the registrar on further assessment and management of Mr Smith.
You will not be required to see the patient or perform any examination.
This OSCE will assess the following domains:
• Medical Expertise
• Communication

20. 10 minutes later……..
Take a deep breath
How did that go?
Tell me what you did well…
Tell me blah blah blah

21. OSCE Scoring and Results
• Standard setting occurs
– Borderline regression method
– Uses individual candidates domain scores and
global score in the calculation
– Single pass score for exam is created
– No set number of stations required
• Feedback given to unsuccessful candidates

22. Recent ACEM FEx OSCE Results

23. Domain Frequencies
Increase in number to obtain
more data on candidate
performance
See exam reports for more info

24. Fellowship exam Resources
- some of my favourites
• https://acem.org.au/Education-Training/Specialist-Training-Assessment-
and-Exams/Fellowship-Examination.aspx
• https://lifeinthefastlane.com/exams/acem-fellowship/
• http://acemfex.adelaideemergencyphysicians.com/
• http://www.shakem.co.nz/part2/overview
• http://www.eduacute.com.au/simulation-blog/hitchhikers-guide-to-the-
acem-fellowship-written-exam/
• https://emergencypedia.com/
• http://www.gcs16.com/
• http://www.edexam.com.au/
• There are many others

25. Final advice
• Take all advice you get (including this) with a grain
of salt
– NO ONE is an authority on how YOU will best pass the
exam
• You need to study hard and be very well prepared
– Follow a program
• For some candidates – doing a paid course might
be worthwhile
– But there is a reason these courses exist
• Its only an exam