Understanding and improving Level 3 Value Added Performance in the Further Education Sector

RCU, 3 Tustin Court, Port Way, Preston, PR2 2YQ
Tel: 01772 734855 email: clee@rcu.co.uk
AoC Teaching, Learning and Assessment Conference
Understanding and improving Level 3
value added performance
19thth January 2016
Presented by Richard Boniface (RCU Ltd)
Martin Laverick (Harlow College)

Session overview
1. Introduction
2. Feedback on analysis of national level data
3. Value added at Harlow College – making it work in practice
4. Activity and discussion

Two new progress measures:
 Value added measure for Academic and Applied General qualifications –
based on L3VA
 Completion and Attainment measure for Tech Levels and Technical
Certificates (at Level 2)
 Both measures will show the progress of students compared to others
taking the same subject nationally
In addition:
 the value added measure will compare students with the same starting
points
 The completion and attainment measure will treat non completion as a
fail.
These measures will form the basis for new progress based
minimum standards
New Progress Measures (2016)

Analysis of National Level 3
Value Added Data
Richard Boniface
RCU Ltd.

• Level 3 value added score for each qualification taken by a
learner is:
Actual Level 3 performance MINUS Predicted Level 3 performance
• The predicted score is derived from a statistical model based
on the performance of other learners who took the same
qualification and had the same Key Stage 4 points score;
• The published data aggregates learner level data to give
scores by qualification, qualification type and institution
(academic and vocational);
• The baseline Key Stage 4 score was calculated at age 15 for
16,17 year old completers but age 16 for 18 year old
completers.
Analysis of Level 3 Value Added Data

Academic Value Added Scores
Provider Type
Number of
Entries
Value Added Score
Lower
Confidence
Limit
Upper
Confidence
Limit
Academy 358,046 0.03 0.03 0.03
Sixth form college 213,704 0.00 0.00 0.01
Independent school 124,777 0.12 0.12 0.12
Community school 91,557 -0.02 -0.02 -0.01
Voluntary aided school 59,208 0.01 0.00 0.01
General further education college 45,489 -0.13 -0.14 -0.13
Foundation school 43,728 -0.01 -0.02 -0.01
Sponsored academy 37,164 -0.09 -0.10 -0.09
Tertiary college 32,349 0.01 0.00 0.01
Voluntary controlled school 9,149 -0.01 -0.02 0.00
City technology college 1,316 -0.04 -0.06 -0.01
Free school - mainstream 772 -0.09 -0.11 -0.06
Agriculture and Horticulture College 701 -0.10 -0.13 -0.07
Free school - 16-19 628 0.30 0.26 0.34
Free school - UTC (university technology college) 202 -0.26 -0.33 -0.19
Art, design and performing arts college 183 -0.05 -0.12 0.02
Free school - studio school 102 -0.24 -0.31 -0.17

Academic Value Added Scores
0%
10%
20%
30%
40%
50%
60%
>-1and<=-0.8
>-0.8and<=-0.6
>-0.6and<=-0.4
>-0.4and<=-0.2
>-0.2and<=0
>0and<=0.2
>0.2and<=0.4
>0.4and<=0.6
>0.6and<=0.8
>0.8and<=1
PercentageofProvidersinValueAddedBand
Value Added Score
All
Academy
Sixth Form College
Community School
GFE/Tertiary

Vocational Value Added Scores
Provider Type
Number of
Entries
Value Added Score
Lower
Confidence
Limit
Upper
Confidence
Limit
General further education college 73,750 -0.14 -0.14 -0.14
Academy 24,571 0.16 0.15 0.16
Sixth form college 22,017 0.14 0.13 0.14
Tertiary college 16,685 0.04 0.03 0.05
Sponsored academy 14,480 0.08 0.07 0.09
Community school 11,606 0.14 0.13 0.15
Foundation school 6,129 0.14 0.12 0.15
Voluntary aided school 4,837 0.20 0.19 0.21
Agriculture and Horticulture College 3,207 0.03 0.01 0.04
Voluntary controlled school 694 0.11 0.08 0.14
Independent school 611 0.22 0.19 0.26
Art, design and performing arts college 561 0.37 0.33 0.40
General further education college (special) 536 0.03 -0.02 0.07
City technology college 532 0.38 0.35 0.42
Free school - UTC (university technology college) 168 0.31 0.25 0.38
Free school - studio school 104 0.27 0.19 0.36
Free school - mainstream 13 0.32 0.08 0.56

0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
>-1.6and<=-1.4
>-1.4and<=-1.2
>-1.2and<=-1
>-1and<=-0.8
>-0.8and<=-0.6
>-0.6and<=-0.4
>-0.4and<=-0.2
>-0.2and<=0
>0and<=0.2
>0.2and<=0.4
>0.4and<=0.6
>0.6and<=0.8
>0.8and<=1
>1and<=1.2
>1.2and<=1.5
PercentageofProvidersinValueAddedBand
Value Added Score
All
Academy
Sixth Form College
Community School
GFE/Tertiary
Vocational Value Added Scores

VA Scores by Qualification Type (GFE)
GFE/Tertiary
Qualification Type
Number of
Entries
Value Added Score
Lower
Confidence
Limit
Upper
Confidence
Limit
A Level 53,197 -0.06 -0.06 -0.05
EDEXCEL BTEC Level 3 Extended Diploma 47,842 -0.05 -0.06 -0.05
AS Level (Not Continued to A2) 21,698 -0.03 -0.03 -0.02
Edexcel BTEC Level 3 90-credit Diploma 15,107 -0.05 -0.06 -0.04
EDEXCEL BTEC Level 3 Subsidiary Diploma 10,726 -0.29 -0.30 -0.28
EDEXCEL BTEC Level 3 Diploma 6,764 -0.33 -0.34 -0.32
CACHE Level 3 Diploma in Child Care and Education 1,809 -0.06 -0.08 -0.03
BTEC National Diploma 1,700 -0.13 -0.15 -0.10
Extended Project (Diploma) 1,433 -0.01 -0.04 0.02
EDEXCEL BTEC Level 3 Certificate 1,046 -0.13 -0.15 -0.10
Level 3 Diploma (ABC or UAL) 615 0.12 0.07 0.16
Applied GCE Single Award 610 -0.08 -0.11 -0.04
CACHE Level 3 Extended Diploma 498 -0.00 -0.03 0.02
UAL Level 3 Extended Diploma 483 -0.15 -0.20 -0.09
L3 Foundation Diploma in Art and Design 413 -0.05 -0.11 -0.00
UAL Level 3 Diploma 315 0.00 -0.06 0.06
City & Guilds Level 3 Extended Diploma 291 0.04 0.01 0.08
IFS Level 3 Diploma in Financial Studies 258 -0.28 -0.33 -0.23
IFS Level 3 Certificate 236 -0.05 -0.10 0.01
City & Guilds Level 3 Diploma 196 0.08 0.01 0.14
CACHE Level 3 Certificate in Child Care and Education 185 0.04 -0.02 0.10
Pearson BTEC Level 3 Diploma in Beauty Therapy (QCF) 173 -0.05 -0.11 0.01
Applied GCE Double Award 172 -0.37 -0.43 -0.31
Applied GCE AS level 166 -0.04 -0.10 0.02
Level 3 Subsidiary Diploma (RSL or City & Guilds) 155 0.16 0.10 0.22
International Baccalaureate 139 0.00 -0.05 0.06
IFS Level 3 Certificate in Financial Studies 96 0.24 0.14 0.34
ABC Level 3 Foundation Diploma 74 0.05 -0.08 0.18
BTEC National Award 73 -1.31 -1.41 -1.20
OCR Level 3 Cambridge Technical Extended Diploma 70 -0.49 -0.60 -0.38
BTEC National Certificate 64 0.54 0.41 0.67
City & Guilds Level 3 Subsidiary Diploma 54 -0.02 -0.10 0.05
OCR Level 3 Cambridge Technical Introductory Diploma 30 -0.09 -0.25 0.08
Pre-U Principal Subject 16 -0.05 -0.21 0.11
Pearson BTEC Level 3 National Award 15 0.28 0.09 0.48
OCR Level 3 Cambridge Technical Diploma 13 0.19 -0.02 0.41
Free Standing Maths 8 0.24 -0.15 0.64
OCR National Certificate 7 -0.97 -1.27 -0.67

VA Scores by Qualification Type (SFC)
Qualification Type
Number of
Entries
Value Added Score
Lower
Confidence
Limit
Upper
Confidence
Limit
A Level 139,358 0.02 0.01 0.02
AS Level (Not Continued to A2) 58,852 -0.01 -0.01 -0.00
Extended Project (Diploma) 8,799 0.10 0.07 0.13
EDEXCEL BTEC Level 3 Subsidiary Diploma 7,583 0.17 0.16 0.17
EDEXCEL BTEC Level 3 Extended Diploma 7,148 0.15 0.14 0.15
Applied GCE Single Award 4,330 0.13 0.09 0.16
EDEXCEL BTEC Level 3 Diploma 3,639 0.20 0.19 0.20
Applied GCE AS level 1,205 0.11 0.05 0.17
EDEXCEL BTEC Level 3 Certificate 1,078 0.18 0.16 0.21
Applied GCE Double Award 882 0.09 0.04 0.15
Edexcel BTEC Level 3 90-credit Diploma 792 0.17 0.16 0.17
IFS Level 3 Diploma in Financial Studies 377 0.18 0.13 0.23
BTEC National Diploma 283 0.36 0.34 0.39
CACHE Level 3 Diploma in Child Care and Education 183 0.16 0.14 0.19
International Baccalaureate 153 0.01 -0.04 0.07
OCR Level 3 Cambridge Technical Introductory Diploma 149 0.03 -0.14 0.19
OCR National Diploma 146 -0.12 -0.12 -0.12
OCR Level 3 Cambridge Technical Diploma 104 -0.11 -0.32 0.10
City & Guilds Level 3 Diploma 43 -0.06 -0.12 0.01
IFS Level 3 Certificate 36 0.34 0.28 0.39
BTEC National Award 33 0.71 0.61 0.82
UAL Level 3 Extended Diploma 33 0.24 0.18 0.29
IFS Level 3 Certificate in Financial Studies 31 0.66 0.56 0.76
L3 Foundation Diploma in Art and Design 26 0.23 0.18 0.28
Pearson BTEC Level 3 National Award 25 0.20 0.01 0.40
Level 3 Diploma (ABC or UAL) 16 0.02 -0.02 0.07
Free Standing Maths 12 -1.27 -1.66 -0.87
CACHE Level 3 Extended Diploma 10 0.41 0.39 0.44
BTEC National Certificate 7 -0.72 -0.85 -0.59
City & Guilds Level 3 Subsidiary Diploma 7 0.17 0.10 0.24
ABC Level 3 Foundation Diploma 7 0.31 0.19 0.44

Academic Vocational
College Value Added Scores by Region
-0.30 -0.20 -0.10 0.00 0.10 0.20 0.30
North East
West Midlands
East Midlands
London
Yorkshire and The Humber
South East
East of England
North West
South West
Value Added Score
-0.30 -0.20 -0.10 0.00 0.10 0.20 0.30
London
Yorkshire and The Humber
East Midlands
West Midlands
East of England
North East
South West
South East
North West
Value Added Score

Value Added Scores by Deprivation
Academic Vocational
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
0 10 20 30 40 50 60
ValueAddedScore
IMD 2010 16-18 Classbased L3 Deprivation Score
GFE / Tertiary College
Sixth form college
Land-based/Art and
Design Colleges
Least Deprived Most Deprived
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
0 10 20 30 40 50 60
ValueAddedScore
IMD 2010 16-18 Classbased L3 Deprivation Score
GFE / Tertiary College
Sixth form college
Land-based/Art and
Design Colleges
Least Deprived Most Deprived

Value Added Scores by OFSTED Grades
Academic Vocational
-0.20 -0.10 0.00 0.10 0.20
Outstanding
Good
Satisfactory/
Requires Improvement
Inadequate
Value Added Score
-0.20 -0.10 0.00 0.10 0.20
Outstanding
Good
Satisfactory/
Requires Improvement
Inadequate
Value Added Score

• Schools were withdrawing students who they expected to do
badly, improving their headline Value Added scores;
• Level 3 Value Added measure does not take into account the
impact of deprivation;
• The lack of a common Key Stage 4 baseline year
disadvantages GFEs with 18 year olds and does not show the
true value that colleges add;
• Statistical model may not be accurate for learners with
relatively low KS4 scores;
• Small sixth forms benefit from a ‘shrinkage factor;’
• Value Added scores included qualifications taken at other
institutions.
Issues Raised from Analysis

• Common baseline introduced - only specific qualifications
achieved during Key Stage 4 will be included;
• New methodology for predicting scores to be used which
should be more accurate for students with low prior
attainment (20 bands rather than multi-level modelling);
• Shrinkage factor removed;
• Introduction of a separate retention measure;
• Progress measure only applies to Academic and Applied
General qualifications;
• Level 3 value added scores will be reported separately for
each provider if a learner changes institution.
Changes for 2016

Harlow College
Martin Laverick
Director of Quality Standards and Assessment

• Values around
– A Levels +0.30
– Vocational +0.30
• A brief history
– Starting Point – Sport
Harlow College Experience

• Discuss and write down three things that are
stopping your organisation from achieving
high value added
– Caveat – these must be internal factors
• By 9 minutes each table should list these
factors on separate post-it notes (min of 3)
• At 10 minutes stick them on the wall.
Activity

• Set a challenging target grade for the student based
on KS4/GCSE results
– Predicted plus one?
– Target based on data from DfE, ALIS, ALPS, etc.
– Does the accuracy of the target matter?
• Don’t get hung up on the correlation at individual student level.
– New Assessment Rules QCF and NQF Edexcel
• Negotiate a student target
Our Process

• Are your tutors/lecturers and managers held
accountable?
– How?
• Do you have a central tracking system?
– Advice - have something simple that works
• In year progress
– Which groups underperformed last year.
– Where are they are now?
– Can you predict the influence of other changes?
Monitor Student Progress

What evidence?

At student level?
Note: the headings for the
third and fourth columns
should be:
Personal Target Points
Harlow College Target Points

• Staff
– Teaching to the higher grade
– Not accepting the “just pass”
– Other reasons
• Students
– The basics
• Attendance (>85% will cost you a grade)
– For every negative VA.
Changing Mindsets

• Daily targets that build on prior knowledge,
are personalised/differentiated, and include
support and intervention, as well as, stretch
and challenge.
• Unit Days - students go home when s/he has
met the differentiated SMART target(s).
One Approach

• Data/Data Capture
• Data Arrives too late
• Teachers don’t understand
• Not understanding what a distinction profile like
• Lack of Stretch and Challenge/personalised targets
• Vocational teachers understanding of calculated
• Value Added high achievers
• Too much focus on success – lack of aspirations
• Feedback on work doesn’t help learners improve
• Not collecting quals on entry
Summary and Questions

Understanding and improving Level 3 Value Added Performance in the Further Education Sector

Recommended

Recommended

More Related Content

Similar to Understanding and improving Level 3 Value Added Performance in the Further Education Sector

Similar to Understanding and improving Level 3 Value Added Performance in the Further Education Sector (20)

Recently uploaded

Recently uploaded (20)

Understanding and improving Level 3 Value Added Performance in the Further Education Sector

Editor's Notes