Track 16. Doctoral Consortium Authors: Adriana Gamazo, Susana Olmos-Migueláñez and Fernando Martínez-Abad https://youtu.be/aoLXMeBHKjA

1. Multilevel models for the
assessment of school
effectiveness using PISA
scores
Gamazo, Adriana; Olmos-Migueláñez, Susana;
Martínez-Abad, Fernando
University of Salamanca

2. Introduction
Rising interest
in the study of school and
student factors
Models explaining:
 Academic performance
 School effectiveness

3. International
European
Regional
National
Standardised tests

4. Standardised
Tests
Academic
Performance
Context
Questionnaire
Students
School
Usual standardised test structure

5. Student context factors (socio-economic
background, gender, country of origin,
grade repetition…)
School factors that can be intervened
(relationships with parents/community,
methodology, organisation, evaluation
methods…)
School context factors (resources, rate of
males/females, rate of grade repeaters…)
PISA School score composition

6. OUR RESEARCH AIMS TO…
• develop multilevel models in order to eliminate the influence of context in
PISA scores
• detect schools with high and low effectiveness within the Autonomous
Community of Castile and Leon
• study the factors associated with high- and low-performing students
So, why is this important?

7. ESCS -1 -0,5 0 0,5 1
600
550
500
450
400
350
300
PISA score
Regression diagram: Pisa score – Socio-econocmic level (ESCS)

8. ESCS -1 -0,5 0 0,5 1
600
550
500
450
400
350
300
PISA score
Score above
OECD average
Selection based on PISA alone

9. ESCS -1 -0,5 0 0,5 1
600
550
500
450
400
350
300
PISA score
Most effective
schools
Selection based on context

10. ESCS -1 -0,5 0 0,5 1
600
550
500
450
400
350
300
PISA score
Most effective
schools
Least effective
schools
Selection based on context

11. QUALITATIVE
Case study Practice analysis
QUANTITATIVE
School selection Factor analysis
METHODOLOGY

12. QUANTITATIVE PHASE
Sample:
• Schools and students selected for
PISA 2015
• Castile and Leon, Spain
Sample:
• Schools and students selected for
PISA 2015
• Castile and Leon, Spain
Methodology:
• Transversal
• Hierarchical linear models

13. Sort the schools
according to their
value added
Select the most and
least effective
schools
QUANTITATIVE PHASE – EXPECTED OUTCOMES
&

14. QUALITATIVE PHASE
Sample:
• Schools selected in quantitative
phase
Methodology:
• Case study
• Interviews, group discussions,
observation…

15. QUALITATIVE PHASE
Most effective schools Least effective schools
In-depht study of
Leadership
Teaching
methodology
Time
management
Assessment
Guidance and
counselling

16. QUANTITATIVE PHASE – EXPECTED OUTCOMES
Practices associated
with school
effectiveness
Practices associated
with school
ineffectiveness
Seemingly irrelevant or
unclear practices

17. What do we hope to achieve in the end?
Policymakers
School heads Teachers
Understand school efficiency and which factors help and hinder it

18. Dissertation status
Analysis phase Conclusions
Literature
review
Database
preparation
Pilot
analysis
Result
discussion
Previous phase
Quantitative Qualitative
December 2016
PISA 2015

19. Limitations of the research
Longitudinal
vs.
transversal
Sample
vs.
census information

20. THANK YOU VERY MUCH FOR LISTENING!