[Re]evaluating practice: measuring what is important, to inform improvements to student outcomes

RESPECT. DRIVE. ADVENTUROUS. GENUINE. RIGOUR. CONVICTION.
[Re]evaluating practice: measuring what is important,
to inform improvements to student outcomes
Maria Spies
Dr Margot McNeill
Dr Suneeti Rekhari
Karen Woo

Outline
1. Overview – what matters for student success and retention?
2. Retention Driver Tree
3. Immediate interventions and Pilots
4. Medium & long term success factors
5. 3 ‘Essentials’ for a successful Retention project

THREE GROUPED FACTORS CONTRIBUTING TO THE STUDENT
EXPERIENCE
First term/year experience; Academic Support; Peer
Support; Teaching and Learning Approaches;
Assessment and Curriculum Design; Teacher
engagement; Classroom practices; Academic skills
embedding; Teachers knowing students
Institutional Conditions
Academic Preparation; Learning skills;
Enrolment Choice; Demographics;
Family/Peer Support; Aptitude; Learner
efficacy
Attributes & Experiences Student Behaviours
Study habits; Peer involvement;
Interaction with teachers; Time on task;
Motivation;
Student
Experience

FACTORS LEADING TO STUDENT DROP OUT
1. Poor preparation for tertiary education
2. Weak institutional and/or course match
3. Unsatisfactory academic experience
4. Lack of social integration
5. Personal circumstances
6. Financial issues

RETENTION DRIVER TREE
5
Retention is an interplay of more than 39 possible levers that work together to retain a student.

Student retention is a core component of
Navitas’ strategic goal of delivering world
class quality through excellent student
outcomes.
Retention is very complex and requires a
number of interconnected levers and
actions. The retention driver tree
summarises the complexity of theses
relationships.
From this project, the HIU lever was
recognised as a strategy that was feasible,
trackable and could afford quick wins that
would give us the most ‘bang for our buck’ .
RETENTION DRIVER TREE PILOT – S3 2017
6
Our focus should prioritise feasibility and impact.
Retention Driver Tree
Typically:
- first trimester/year subjects
- traditionally high failure rates
- low student satisfaction, compulsory, with complex and/ foundational
concepts and low immediate applicability to career outcomes
- high student numbers.
Overview High Impact Unit Intervention

RESPECT. DRIVE. ADVENTUROUS. GENUINE. RIGOUR. CONVICTION. 7
Current state analysis
RETENTION HIGH IMPACT UNIT PILOTS
6 ‘high impact units’ with a target to increase pass rates by 10%
Aug - Sept Oct - Dec Jan - Feb Mar - Apr
Develop & agree on interventions
Design & implement
immediate interventions
Ongoing immediate intervention,
initiate mid-term interventions
Preliminary analysis & results
Final evaluation &
recommendations
Skilled and supportive teachers
• Teachers have a varying view of their
roles
• Little uptake of PD opportunities
• No framework to guide reflective
practice which affects quality and
results
Assessments
 Exams are weighed heavily (up to
60%) and consistently reap the worst
results.
 No evidence of developmental
assessment design, students perform
worse in consecutive tasks instead of
improving
 No significant relationships were
found in attendance and grades in
the units
 Few examples of assessment
templates, examples or rubrics used
to guide student work
Integrated skills development
• Little evidence of integrated skills
development in the curriculum
although most teachers in interviews
cited the need for students to
develop academic skills and for
international students to improve
their English communication
Timely and affirming feedback
• Class observation shows little
feedback provided in class
• Student evaluation instruments are
not aligned with TEQSA standards
on quality and timeliness of feedback
• There is little evidence of
developmental feedback for
assessment
Curriculum
• Unit outlines do not communicate
relevance & alignment to outcomes
• Most units make use of digital
resources, some still rely heavily on
textbooks – which are often not
purchased
Active & engaging online
• Reliance on print resources
• Little evidence of alignment between
online resources and on campus
classes
• Online systems such as Moodle need
to be better leveraged and designed
for clarity
Active & engaging classroom
• Observations indicate low student
engagement in many classes
• Only one unit met the target of 70:30
teacher talk ratio. One unit had 100%
teacher talk

SHORT TERM INTERVENTIONS
23/07/2018 8
Prioritised as actionable ‘quick wins’
Timely & affirming feedback
 Ensure that feedback given to students is
personalised, constructive, aligned and
future focused.
 Ensure that feedback is given at least one
week before the next task is due, so
students have time to incorporate
feedback into the next task.
Active & engaging classroom
 Lesson plans are developed to ensure at
least 30% of class time is active learning
and that class time is utilised to the full
potential*
 Teacher participation in the Learning
Design course in T3
 PD to develop a range of questioning
techniques used to elicit info and check
understanding
 Attendance is monitored & followed up
Active & engaging online
 Teacher participation in the Learning
Design course in T3
 Review learning design of the online
spaces to encourage engagement outside
of class time and to support digital literacy
Well designed assessments
 Implement developmental assessment
design, where assessments are
sequenced such that they scaffold
students’ skills development
 Implement low-stakes first assessment,
due by week 4, are worth no more than
10%, with a target average grade of
75%.
 Review all exams and assessment tasks
with an average grade <50%.
 All exams have at least one (preferably
two) practice sessions in class.*
 All major tasks have scaffolding
designed into previous weeks.*
 All assessments are explained in class
with reference to the marking rubric, and
started in class.*
 Lesson plans include opportunities for
formative feedback to be given to
students.
 Ensure that all tasks include clear
assessment criteria and rubrics to
provide comprehensive feedback to
students.*
Skilled & supportive teachers
 Embed academic skills and digital literacy
into the curriculum
 Establish explicit criteria on English and
skills on each assessment rubric
 Track students’ performance in skills
across tasks
Integrated skills development
 Review unit outlines to ensure the
purpose of the unit and of each task is
clear to the students*
 Work with teachers to create a stronger
narrative that would resonate with the
students*
 Ensure all students have access to the
textbook, if necessary included in fees
Structured & relevant curriculum
 Teacher PD on internationalising the
classroom including strategies for
questioning and concept checking
 PD for staff to equip them for strategies
for continual improvements and reflection

ACTIVE AND ENGAGING CLASSROOM ENVIRONMENT
23/07/2018 9
Active learning (30% per hour) is a core component of our student experience
Project objectives Early insights & indicators
Targeted actions
Lesson plans are developed to
ensure at least 30% of class time
is active learning and that class
time utilised to full potential
Chunked interactive assignment instructions so that students
have a visual way to access assessment information –
supported by in-class activities & practice
Disengaged classes,
high teacher talk time
Teachers resistant to
change due to time
req’d to re-develop
plans
WHAT
Work with teachers
to increase active
learning in their
lesson plans, focus
on easy strategies
that won’t take up
too much time
HOW
Active participation in
class increases
students understanding
of material and
supports soft-skill
capability &
employability
development e.g. group
work, public speaking
WHY

STRUCTURED AND RELEVANT CURRICULUM
23/07/2018 10
Alignment scaffolds student learning and provides a consistent and coherent experience
Targeted actions
Re-designing
course material
to reduce
cognitive load on
students – this
includes
‘chunking’ in-
class material,
the unit Moodle
site is
redesigned to
clarify content
and activities.
Review unit outline to ensure the purpose of
the unit and each task is clear to the
students
Work with teachers to create a stronger
narrative that resonates with students
Observations and
course content (unit
guides) show that
relevance and
alignment is not made
for Navitas students
WHAT
Re-work course
content to reduce
cognitive load and
ensure content is
relevant for
students who are of
a non-English
speaking, low
academic ability
HOW
Students, particularly
the Navitas cohort,
require clarity & strong
linkages to their
learning pathway &
experiences to solidify
understanding
WHY

WELL-DESIGNED ASSESSMENTS
23/07/2018 11
Guide and affirm student learning, with developmental tasks and rubrics & models
Targeted actions
Grading system is clearly
articulated and shows HOW to
improve in the future
Practice exam opportunities
prior to final
Develop standards based
rubrics to guide improvement
Models and templates to
demonstrate requirements
Provide practice
opportunities
and clearer,
standards based
rubrics and
examples on
how to succeed
HOW
Formative
feedback and
developmental
assessment
design assists in
building student
confidence and
expertise
WHY
Students do not
have the
foundation
capability to
succeed in
assessment
tasks – more
guidance is
needed
WHAT
Past submissions leveraged with teacher
comments to demonstrate working
example of a high grade assessment
Collated, chunked and visual resources
for assessment success is made
available

To
E.G. 1 - AUTHENTIC PRACTICE USING ONLINE MCQ
12
From: hand-written
quiz which took a
week to mark and
bore no similarity to
the ‘real’ process
To: practice quiz
on the LMS site,
simulating test
conditions, with
immediate
feedback.
3%
23%
0%
5%
10%
15%
20%
25%
91-100% score
Results: Multiple choice quiz
score over 91%
Before intervention (%) After intervention (%)

To
E.G. 2 - PRACTICE FOR TASK # ONE
13
From: low
submission rates for
lengthy and
challenging first task
(attrition risk!)
To: H5P tool, for
students to
practice with
chunked sub-
sections, and
peer discussion
in class.
60%
96%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of submissions
Results: First assessment
submissions
Before intervention After intervention

To
E.G. 3 – RUBRICS AND MARKING GUIDES
23/07/2018 14
From: no rubrics to
guide student work
or marking guides
for consistency
among markers
To: a maths-
based calculation
general rubric + a
detailed marking
guide for the
teaching staff
34%
70%
0%
20%
40%
60%
80%
100%
Average mark
Results: Second assessment average
mark
Before intervention After intervention

Policy
• Clear KPIs in position
descriptions
• Track, monitor and follow
up on attendance
• Teachers maintain currency
and contemporary teaching
practice
• Minimum standards for
quality and digitally
enhanced L&T
• Assessment policy
• Classroom mgmt policy to
guide student activity in
class
Process
• Efficient reporting on
students, attendance, pass
rates, and student
satisfaction
• Review student evaluation
of unit and teacher
instruments to support
quality assurance
• Review classroom spaces
based on learning
requirements
• A range of feedback
mechanisms be employed
University Partner
Engagement
• Redesign of curriculum
topics or sequence
• Redesign of major
assessment task
• Continuing and consistent
tracer data
MEDIUM TERM INTERVENTIONS
23/07/2018 15
Stakeholder engagement to drive policy, procedure and process changes
We need longer term thinking in some aspects and consistency across program, and to manage the teaching practice in a continuous improvements. These
interventions require involvement of multiple stakeholders and changes in policy.

Culture
• Culture at the college is
supportive of student
engagement and learning.
• Culture is professional and
reflective of the Navitas value of
‘respect’.
• Supporting a culture of
continuous improvement and a
focus on academic outcomes.
• Networks across Navitas college
teaching similar courses for
sharing good practice.
Capacity
• Specific and aligned strategies
for building teaching capacity.
• Supporting scholarly approaches
to teaching practice.
• Developing academic leadership
at all levels.
• Building digital skills across the
workforce, from data
management and reporting to
teaching with technology.
LONG TERM INTERVENTIONS
23/07/2018 16
Ensuring sustainability into Business as Usual
These are longer term recommendations, such as adaptations to culture and the development of academic leadership. Similarly, PD and
performance management take time to have an impact.

Key data not available in
some courses. Differences in
measure definitions hinders
assessing retention. An
overarching framework that
reflects regulatory standards
is required.
ESSENTIALS FOR A SUCCESSFUL RETENTION PROJECT
23/07/2018 17
Joint Academic &
Business Leadership
Attention to Data &
Measurement
Time to Implement:
Staying the course
Top-down sponsorship is
required for interventions to
be successful. This is
particularly required where
teacher characteristics and
capability has a driving impact
on student retention
Resistance to change due to
the time it takes to develop
and implement interventions.
Frameworks, resources,
polices and templates
leveraged across colleges
will reduce this impact.

Questions and Discussion

[Re]evaluating practice: measuring what is important, to inform improvements to student outcomes

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to [Re]evaluating practice: measuring what is important, to inform improvements to student outcomes

Similar to [Re]evaluating practice: measuring what is important, to inform improvements to student outcomes (20)

More from LearningandTeaching

More from LearningandTeaching (20)

Recently uploaded

Recently uploaded (20)

[Re]evaluating practice: measuring what is important, to inform improvements to student outcomes