This presentation from the OECD Disrupted Futures 2023: International lessons on how schools can best equip students for their working lives conference looks at Enabling effective transitions for all youth “Reduced youth disengagement post-16 via improved school-level careers provision in England”. Presented by Chris Percy and Laura Hawksworth. Discover the videos and other sessions from the OECD Disrupted Futures 2023 conference at https://www.oecd.org/education/career-readiness/conferences-webinars/disrupted-futures-2023.htm Find out more about our work on Career Readiness https://www.oecd.org/education/career-readiness/

1. Reduced youth
disengagement post-
16 via improved
school-level careers
provision in England
Chris Percy & Laura Hawksworth
The Careers & Enterprise Company
OECD conference Disrupted Futures
2 June 2023

2. Context: Post-16 participation static at 94%
of students 2014/15 to 2020/21 (latest data)
Source: Dept for Educ; Mar 2023 updated data https://explore-education-statistics.service.gov.uk/find-statistics/key-stage-4-destination-measures/
Data notes: Destinations sustained to March following completion of the Sept-Aug academic year 11 (Key Stage 4), when most students turned 16 and did national GCSE exams.
State funded mainstream educated students only. 2020/21 n=551k.
3 cohorts analysed in this study

3. Topic: The link between Gatsby Benchmark
achievement and post-16 participation
Source: Gatsby Charitable Foundation. (2014). Good Career Guidance; Dept for Educ. (2017). Careers strategy: making the most of everyone’s skills and talents
Self-assessed scores via online Compass tool for 8 BMs
1. A stable careers programme
2. Learning from career and labour market information
3. Addressing the needs of each pupil
4. Linking curriculum learning to careers
5. Encounters with employers and employees
6. Experiences of workplaces
7. Encounters with further and higher education
8. Personal guidance
 Average score from 0%-100%, weighting each BM equally

4. Method: Regression on observational data,
with controls to mitigate endogeneity
• School-level data
• Non-private (fee-paying) schools only
• With a big enough Year 11 cohort in the
relevant year that DfE release data
• Use all years of data available at the time of
initiating analysis in mid-2022, i.e. those with
both Gatsby BM data & destinations data
• Require control variable data available
• Final N=5453 schools
(2016/17: 435; 2017/18: 2382; 2018/19: 2636)
• Dependent variable: the % of the cohort is
confirmed sustained EET status post-16)
• Model: Generalised linear model with logit
link function and binomially distributed DV,
robust SEs and finite popn correction
• Controls build up :
Year control (i.e. time / school cohort effects)
+ School structure (school type, sixth form
presence, gender mix, whether selective)
+ School location (region, rurality, local area
unemployment / job density, opportunity area)
+ School intake features (cohort size, FSM rate)
+ School grades (Attainment 8, Progress 8)
+ School quality indicator (Ofsted grade)
Sample selection Analysis approach

5. Result: 7.3%pts non-EET rate improves to
6.7%pts in schools with a top careers score
Model description Sample size
(schools)
P-value Average non-EET rate
for zero vs full
benchmarks **
Change in
rate for
top score
(1) Direct GLM
– No control variables
5949 [0.000]* 9.2% 7.5% 1.7 %pts
(2) Main model GLM
– Control variables
5453 *** [0.006]* 7.3% 6.7% 0.7 %pts
(3) OLS regression (FYI)
– Control variables
(no FPC; normal SE)
5453 [0.015] 7.5% 6.6% 0.9 %pts
* Bootstrapped standard errors on the non-FPC dataset produce an SE estimate within 10% of the robust standard errors on the non-FPC dataset using 200-250 replications, with bootstrapped SEs
being almost identical in (1) and 10% lower in (2); ** Being one minus the confirmed EET rate, as the original dependent variable; i.e. those confirmed NEET combined with those with unknown
destinations. Marginal effects calculated based on other variable values aligning with the actual dataset distribution; *** Outlier analysis performed on the headline specification identifies few
outliers by the 4/N Cook’s D heuristic (n=2), whose removal marginally increases the coefficient size. Arbitrarily dropping the top 1% of outliers reduces the coefficient to 0.08 (p-value 0.01).

6. By route: Most consistent drivers are more
apprentices & fewer untraced students
Route breakdown
(Model 2)
Sample size P-value on careers
provision
Avg change in % participation on the
specified route from zero to full
benchmarks
Further Education (FE) 5450 0.066 38.3% - 36.9% -1.4 %pts
School Sixth Form (SSF) 5427 0.110 32.7% - 33.7% +1.0 %pts
Sixth Form College (SFC) 5307 0.546 11.9% - 12.3% +0.4 %pts
Apprenticeships 5383 0.000 3.6% - 4.2% +0.6 %pts
Work 5390 0.919 3.4% - 3.5% +0.1 %pts
Unknown destination 5053 0.001 1.5% - 1.1% -0.4 %pts
Secondary drivers: more students into school sixth forms; fewer into FE colleges

7. By group: Twice the relationship strength in
the 25% most econ. disadvantaged schools
Model description Cohort Sample
size
P-value Average non-EET rate for
zero vs full benchmarks
Range
(1) Direct GLM
– No control variables
Full 5949 [0.000] 9.2% 7.5% 1.7 %pts
Econ. dis. 1555 [0.001] 18.1% 14.2% 3.9 %pts
(2) Main model GLM
– Control variables
Full 5453 [0.006] 7.3% 6.7% 0.7 %pts
Econ. dis. 1159 [0.023] 13.8% 11.8% 2.0 %pts
(3) OLS regression (FYI)
- Control variables
Full 5453 [0.015] 7.5% 6.6% 0.9 %pts
Econ. dis. 1159 [0.053] 14.0% 11.6% 2.4 %pts

8. Limitations and possible extensions
Key limitation Details Possible extensions
Causality • Cross-sectional
observational data
• Causality requires DAG /
heterogeneity assumptions
• Further controls (theory-led as credible confounders)
• Model selection effects/matching (but results typically similar given
available data)
• Longitudinal panel analysis (but may require 6+ years of data)
Alternative
specifications
• Results have proven robust
to specification examined
to date but other options
exist
• Highest priority alternatives have been examined. Others:
• SEM for BMs as proxy for quality of provision as latent var.
• Compare BMs with less vs more theoretical endogeneity or are more
targeted to Year 11 NEETs
• MI to adjust for missing data
Improved model
precision
• Some control variables co-
vary or may inefficiently
used
• Test the inclusion of squared terms and removing or collapsing highly
correlated variables
• Improved precision particularly important for y-o-y analysis
Different route
baseline risk
• Switch institution or leave educ for work may be inherently higher risk (something new often is) even if
the right choice for some (better than staying, better long-term outcomes albeit rocky in short-term)
• We might adjust for this level of baseline risk to avoid “penalising” institutions more in such situations
However other outcomes as higher priority: e.g. motivation/confidence, post-18 destinations, sense of fit within
and contribution to society, course switching/completion/regret, longer-term labour market outcomes, …

9. Broader CEC context & reflections
More details: https://www.careersandenterprise.co.uk/who-we-are/what-we-do/our-impact-2021-22/

10. Summary and Q&A
Full reports: (1) Percy, C. (2023). Technical note: Further analysis on post-16 destinations for the 2016/17 to 2018/19 cohorts. London: The Careers & Enterprise Company
(2) Percy, C., & Tanner, E. (2021). The benefits of Gatsby Benchmark achievement for post-16 destinations. London: The Careers & Enterprise Company.
https://www.careersandenterprise.co.uk/our-evidence/evidence-and-reports/the-benefits-of-gatsby-benchmark-achievement-for-post-16-destinations/
• England gvnt policy asks schools to do the Gatsby
Benchmarks (BM) of good careers provision
• Schools that report higher BM achievement have
better confirmed sustained EET rates post-16
(students in education, employment, or training)
• Mostly via more apprentices and fewer unknowns &
more school sixth forms and fewer FE students
• 6.7% non-EET rate vs 7.3% for full vs zero BMs
(p-value < 0.01, 2016/17-2018/19 cohorts; with
diverse controls in place for school context)
• The relationship with career guidance is twice as
strong in the top 25% most disadvantaged schools