The document discusses the development of an online assessment tool called ACE to help allied health students at the University of Sydney improve their clinical communication skills. It describes how weak communication skills can negatively impact patient care. The ACE tool includes a diagnostic pre-test and an intensive learning program completed during winter break with simulated assessment tasks using standardized patients to practice oral and written clinical communication. Student feedback indicated the client interview role-plays and assessments were the most helpful aspects of the program in helping them improve their skills.

1. The University of Sydney
ACE: an online assessment to help develop complex
communication competency of allied health students
Tina Barclay

2. The University of Sydney
Students gain real-world, hands-on training
through clinical placements within our extensive
network of clinical partners.
Faculty of Health Sciences, University of Sydney
4500 clinical
placements
undertaken
each year
Placements are
at the centre of
our curriculum
As allied health
professionals, our
graduates are changing
the health landscape and
working to improve health
and wellbeing for
individuals, families and
communities worldwide.
Our disciplines:
• behavioural and social
sciences in health
• diagnostic radiography
• exercise and sports
science
• occupational therapy
• physiotherapy
• rehabilitation
counselling
• speech pathology.
3669 students
80% domestic
20% international
90% full time
67% female
73% under 24

3. The University of SydneyThe University of Sydney
“In addition to language proficiency, clinical
communication involves high level language skills (e.g.
verbal and written clinical reasoning) in order to support
the clinical management processes of consultation,
diagnosis and intervention planning and delivery.
Health professionals also need to sensitively adjust
their communication for professional, patient, family
and student audiences.
Weakness in communication skills may
potentially result in patient mismanagement,
poor patient satisfaction and student failure.”
The pitch:
Communication skills are an essential graduate quality for health
professionals, weak communication skills can result in failure.
A systematic review of assessment and intervention
strategies for effective clinical communication in
culturally and linguistically diverse students.
Chan A1, Purcell A1, Power E1.
Empathise

4. The University of SydneyThe University of Sydney Page 4
“I lacked the English skills to engage in a
conversation which included clinical reasoning,
providing my opinion as to why I did what I did,
those sorts of things. I realised had had this
challenge during my clinical placements, when I
realised I wasn’t doing well in clinical placement.
My supervisors informed me about this difficulty. I
had difficulties engaging with clients and with
other health professionals too during
placements.”
3rd year occupational therapy student
Understanding the problem:
Characteristics of poor clinical communication skills
Empathise

5. The University of Sydney
Lack of
assertiveness
Afraid to say,
reluctant to ask
Not always present
May be vague or
vacant
Unsure how to
interrupt/direct a
conversation
Poor eye contact
Limited empathy
Hard to understand
or cannot be
understood by
unfamiliar listeners
Highly Anxious
Defensive or
reluctant to take on-
board advice
Little change
following feedback
Poor self-reflection
skills
Poor self-
organisation
Poor written
language skills
Failing to listen to
clients
Missing relevant
information in an
interview
Overusing jargon
and technical terms
Not able to match
communication and
behaviour to the
needs of the client
Understanding the problem:
Characteristics of poor clinical communication skills
Some student have some or many of these
Empathise

6. The University of SydneyThe University of Sydney
Understanding the problem:
Characteristics of poor clinical communication skills
Empathise

7. The University of Sydney
ACE: Accelerate Communication Excellence
Diagnostic assessments are pre-tests and barometers of current learning. Similar to formative assessment in that
it examines the knowledge and skills that a student has already learnt. ACE is a 0% barrier task, pass/fail for the unit
of study. Failure means that the first placement is postponed to later in the first year.
ACE online diagnostic assessment tool ACE intensive learning support program

8. The University of Sydney
Assemble a team: use our combined experience to build an
online diagnostic assessment and support program
Learning Technologists & Web Dev
• Emerging full-stack web developers
• Creative coding, rapid prototyping, data
analytics and visualisations, DevOps
• Design thinking and learning design
Academic and clinical placement
• Research
• Assessment designs
• Learning activities
• Intervention strategies

9. The University of SydneyThe University of Sydney Page 9
Review solutions for MVP:
Scaffolding for Clinical Success
Purcell, A. Bourne, E., Sokkar, C., Blackford, J., Lewis, S., Chapparo, C. & Power, E. (2011)
University of Sydney Widening Participation Grant (Curriculum Renewal)
A lecture/tutorial activity followed
by a short answer test and viva
assessment
• Record a patient's case
history – short answer test
• Conduct a client/patient
interview - vIva
Ideate
Define

10. The University of SydneyThe University of Sydney Page 10
Purcell, A. Bourne, E., Sokkar, C., Blackford, J., Lewis, S., Chapparo, C. & Power, E. (2011)
University of Sydney Widening Participation Grant (Curriculum Renewal)
KEEP
• The essence of the viva assessment
• Speaking under time pressure
• The urgency of writing answering
questions immediately after watching a
video
• Active listening and accurate
recording
• A fast and natural writing component
• Thinking of and expressing an idea
under time pressure
Review solutions for MVP:
Scaffolding for Clinical Success
Ideate
Define

11. Page 11The University of Sydney
GO FURTHER
– From paper to online
– Speed up marking and feedback
– Focus on good user experience &
design
– A timed sequence of activities
– Track progress of students
– Provide more data and analytics
– Peer ratings - students see what
others in the cohort answered
DefineReview solutions for MVP:
Scaffolding for Clinical Success

12. The University of Sydney
We started this in late 2015 early 2016.
At that time our VLE’s design and learning tools were stuck
• We did not have a good way for students to record videos in a browser
that we could assess
• We couldn’t really easily include beautiful, simple web design in our VLE
• Our VLE was passive and content hard to access
• Students needed to login or be directed deep through the page structure,
rather than to a page.
Review solutions for MVP: Define

13. The University of Sydney
Ideate
Test

14. Page 14The University of Sydney
• Simple - Time pressure - Forward only sequence - Read the instructions
carefully
• Simulation writing and speaking in the moment – post activity survey of
confidence
Common strategies used in the ACE learning design
Ideate
Test

15. Page 15The University of Sydney
• Simple - Time pressure - Forward only sequence - Read the instructions
carefully
• Simulation writing and speaking in the moment – post activity survey of
confidence
Common strategies used in the ACE learning design
Ideate
Test

16. Page 16The University of Sydney
• Simple - Time pressure - Forward only sequence - Read the instructions
carefully
• Simulation writing and speaking in the moment – post activity survey of
confidence
Common strategies used in the ACE learning design
Ideate
Test

17. The University of Sydney
Features and challenges: online marking
Ideate
Prototype
Test

18. The University of Sydney
Features and challenges: editable like a VLE
Ideate
Prototype
Test

19. The University of Sydney
Features and challenges: visualise data
Ideate
Prototype
Test

20. The University of Sydney
Seeing - Recognising poor communication
behaviour
Nonverbal behaviours :
Eye contact and looking at your client while they
speak
Open posture and sitting upright
Verbal behaviours :
Speaking clearly
Using a loud voice
Active listening :
Observing what people are saying with their body,
eyes and voice
Reflecting back what you hear
Acknowledge and identify how a person feels
Asking questions to understand more
Features and challenges: ideas that almost work
Ideate
Prototype
Test

21. The University of Sydney
Features and challenges: ideas that almost work
Ideate
Prototype
Test

22. The University of SydneyThe University of Sydney
Test
How do our students experience ACE?

23. The University of Sydney
Test
Empathise
How do some students experience ACE?
• Unsure, unclear and even terrified
• Very aware of the high stakes
• “There are so many other tasks to be
done as an university student so I feel
overwhelmed about the ACE module. I
have heard that if I fail, I wont be able to
visit home during the July break.”
• “I felt ashamed about failing the ACE.”

24. The University of Sydney
1. Importance of the front page for describing
the assessment and setting expectations,
without giving the assessment away
2. Better access to support: triage of IT,
professional and academic staff
3. Better communicate results and feedback
4. Need for a in-system approach to academic
integrity
Test
Empathise
User experience design improvements

25. The University of Sydney
Empathise
Define
Ideate
Prototype
Test
Understand the
problem: artefacts,
documents, videos,
assessment, learning
activities, as well as
stories and recording
student reflections of
their own journeys.
Look at the data:
Education impact
Student satisfaction
User experience
research
Start creating &
building:
build fast, test quickly,
go again.
Agree the scope of
the problem to be
solved:
Take everything you
have collected and
synthesize them to
define the core
problems you will
aim to solve
Find alternative
ways to view the
problem:
identify innovative
solutions
Review solutions and
technologies
Our approach to DOING, DIALOGUE & DATA
Collaborative iterative design - design thinking – project methodologies – UX research + other data

26. The University of Sydney
• Mandatory for students assessed with the weakest
communication skills
• Intensive program completed in the Winter break
• Simulated assessment tasks with standardized
patients
• Oral and written assessment tasks
What have been the best aspects of this unit of
study?
• the client interview role play activity and assessment
was mentioned positively in 56% comments made by
students from 2017 to 2019.
• Simulated learning & assessment tasks
• Practicing authentic tasks
• Role playing in a simulated medical environment
• Working with peers and great teachers
3%
0%
6%
4%
0% 0%
2015 2016
Percentage of fail grades by
degree 2016
Undergraduate Occupational Therapy
Postgraduate Occupational Therapy
Undergraduate Physiotherapy
Did it work?
ACE Immersive & educational impact

27. The University of Sydney