Slides from the Learning Circle on 12th March

1. Early Childhood Education Study
Progress Report
Juliane Hencke, Caroline Sharp, Tony Bertram, Chris Pascal
IEA Standing Committee – 5 October 2013

2. Content of Presentation
• Study Overview
• Study Management
• Summary of 1st NRC Meeting
• Conceptual Diagram
• Sampling Design
• Research Questions
• Instruments
• Study Milestones and Next Steps
2

3. Study Overview
• ECES is a cross-national study…
– Exploring, describing, and analysing provision of early
childhood education (ECE)
– Analyzing how ECE contributes to children’s outcomes
• Aims to…
– Provide a framework for countries to benchmark their ECE
systems in an international context
– Examine the relationships between policy aims, provision,
staffing, parental satisfaction and child outcomes
– Investigate the relationships between characteristics of
ECE and children’s competencies
3

4. What do we provide with ECES?
• Country profiles in terms of system characteristics
• Descriptives showing what are the characteristics of
ECE in each country on different levels (system, ECE
settings, practitioners, parents, children)
• Once we understand the characteristics, what are the
relationships between the different levels within and
across countries
• What are the outcome differences in terms of different
provider types and how do they relate to the outlined
policies
• Identify approaches to facilitate these variations
4

5. What makes ECES unique?
• Diversity of countries
• Covers different levels of ECE:
intended - implemented - achieved policies
• Covers diverse consumer perspectives (settings,
practitioners, parents)
• Provide various outcome measures based on an
international framework
• Examines key structural elements from provision to
outcome including policy, providers, parents and
children
5

6. Study Management
• IEA Data Processing and Research Center (IEA DPC):
Study coordination, practitioner questionnaire,
sampling, data processing, analysis
• National Foundation for Educational Research (NFER):
Study framework, ECE setting and home questionnaire
• Centre for Research in Early Childhood (CREC):
Policy questionnaire and child assessment
• IEA Secretariat: Translation verification, quality
monitoring
6

7. Study Management
• Joint Management Committee (JMC)
– Regular meetings of international study consortium
• Project Advisory Committee (PAC)
– Expert advice on framework refinement, instrument
development and reporting
• National Research Coordinators (NRCs)
– Regular meetings to contribute to and to review key
elements of the study
7

8. Participants at the 1st NRC Meeting
8
Austria Denmark Norway
Bahrain England Poland
Belgium (Flemish) Estonia Russian Federation
Brazil Germany Slovenia
Canada Hong Kong Spain
Chile Italy Thailand
Croatia Lithuania Turkey (did not show up)
Czech Republic New Zealand USA (online)

9. Summary of 1st NRC Meeting
• Child outcomes are the aspect of greatest interest to
country representatives
• Framework needs to be re-organized, so that the
assessment module is the focus of the study
• Research questions need to more directed to outcome
measures
• There should be a greater focus on pedagogic practice
• Interest was expressed in administering the
assessment to younger children (as national option?)
9

10. Conceptual Diagram
10
National/ Regional/ Local
Policies
ECE Setting
• Structural characteristics
• Leadership and quality
processes
Practitioners
• Practitioner
characteristics
• Process and
engagement
Family and Child
• Characteristics
• Home Learning
Environment
Child Outcomes
1. Social and
emotional
2. Dispositional
and Learning
3. Physical
4. Language and
literacy
5. Numeracy
Parental
Satisfaction

11. Phase 1
Phase 2
Sampling Design
ECE Settings
of focused children (200)
Practitioners
of focused children
(15 or all per setting; about 1,500)
Focus: Children attending center-based education
and care in the year before entering ISCED 1
Child Assessment
Children
(15 or all per setting;
minimum: 3,000)
Policy Questionnaire (1)
Parents/Carers
of sampled children
Ideally:
PSU, first
sampling
stage
Second
sampling
stage

12. ECES Research Questions
Study focus is on how ECE contributes to children’s outcomes
1. What variations exist between and within countries in
outcomes for children attending ECE?
2. What aspects of ECE settings and education systems are
related to outcomes for children’s early learning and
development?
a. System context, policy aims and QA for ECE
b. ECE setting characteristics, aims and resources
c. ECE staff qualifications and CPD
d. ECE staff practices in working with children
e. ECE setting leadership and QA systems
f. Communication with and involvement of families
12

13. ECES Research Questions
3. What aspects of children’s personal and social
background and family context are related to
children's outcomes?
a. To what extent do these relationships differ within and
between countries?
b. Are there particular aspects of the ECE system or setting
that mediate these relationships?
13

14. ECES Research Questions
4. What is the relationship between parental
satisfaction with ECE and key features of the setting?
1. How is parental satisfaction with ECE related to family and
child characteristics?
2. How is parental satisfaction with their child’s ECE setting
related to child outcomes?
5. To what extent do ECE systems differ in their
availability, reach and inclusion of children in the
period between birth and ISCED 1?
14

15. Instruments
Phase 1
• Policy Questionnaire
Phase 2
• Assessment module evaluating different aspects of
children’s development and learning
– Practitioner ratings (?)
– Assessment tasks
• ECES includes comprehensive set of questionnaires
– ECE Setting Questionnaire
– Practitioner Questionnaire
– Home Questionnaire
15

16. Policy Questionnaire
• To capture evidence on the wider policy context for
ECE in each participating country from birth to age of
compulsory schooling.
• Completed by NRC (or country designate) to provide
robust national level data (qualitative and quantitative)
which will address the study research questions.
16

17. Identification of Key Learning Domains, Sub-Domains and
Indicators
• Theoretical Grounding: Vygotskian socio-constructivist
approach to learning which accepts children‘s experience
and environment shape learning outcomes;
• Empirical Grounding: thorough review of international
evidence to identify those early learning outcomes which
are asssociated significantly with later achievement and
attainment and which are widely acknowledged as
important at end of ISCED 0, and which can be assessed
reliably, rigorously and efficiently.
17
Assessment Module

18. 18
Learning Domains, Sub-Domains, Indicators
Social &
Emotional
Competence
Dispositional and
Learning
Competence
Physical
Competence
Language &
Literacy
Competence
Numeracy
Competence
a. Sociability
b. Emotional literacy and well being
a. Attitudes and mindsets
b. Underpinning learning skills
a. Gross motor skills
b. Fine motor skills
a. Early language skills
b. Early literacy skills
a. Number concepts and operations
b. Spatial concepts
• Social competence
• Self-regulation
• Self-concept
• Exploratory drive and curiosity
• Involvement/ interest
• Persistence
• Locomotion
• Coordination
• Manipulation and handling
• Speaking and communication
• Story picture sequencing
• Comprehension
• Simple number computation
• Problem solving
• Language of spatial objects
Domains Sub-Domains Indicators

19. • Psychometrically robust and reliable
• Inclusive of 5 Learning Domains and able to
discriminate developmental differences
• Graduated assessment (possible 3-7 year span, but
probably most 5-6 years)
• Ethical, respectful of children’s rights and well being
• Interactive, meaningful and attractive items
• Easy to train and implement
• Adaptable to different cultural contexts and low
resource settings
19
Assessment Design Considerations

20. Two assessment tools which will be used as a blended
assessment approach for all 5 learning domains to
ensure reliability and efficiency in the assessment
process:
1. Direct Assessment Tasks (DAT) (by trained External
Administrator)
2. Practitioner Rating (PR) (by practitioner who is
familiar to the children)
20
Assessment Tools

21. DATPR
21
Social & Emotional
Dispositional and
Learning
Physical
Language &
Literacy
Numeracy
Sub-Domain 1a
Sub-Domain 1b
Sub-Domain 2a
Sub-Domain 2b
Sub-Domain 3a
Sub-Domain 3b
Sub-Domain 4a
Sub-Domain 4b
Sub-Domain 5a
Sub-Domain 5b
I 1a1
I 1a2
I 1b1
I 2a1
I 2a2
I 2b1
I 3a1
I 3a2
I 3b1
I 4a1
I 4a2
I 4b1
I 5a1
I 5a2
I 5b1
         
         
         
         
         
         
         
         
         
  
  
  
  
  
         
         
         
         
         
         
Assessment Design

22. • DAT administered by trained External Administrator: 30
mins per child
• PR administered by setting practitioner: 15 mins per
child
• Clear and standardised instructions for assessment
administration for DAT and PR
• 15 sampled children per setting assessed
• Assessment undertaken in 2 week period in setting
during a three month window before the end of ISCED 0
• IEA QA procedures applied
22
Administration of Assessment

23. • No ranking of countries but could for example, draw
comparisons against countries with similar models
of/approaches toECE
• Separate scales for 5 Learning Domains
• Linking of outcomes evidence to in country and between
country systemic, structural and process evidence
• Country profile using
radar diagrams with
country mean scores,
for example:
23
Reporting of Assessment
0
1
2
3
4
5
6
7
8
Social and
Emotional
Competence
Dispositional
and Learning
Competence
Physical
Competence
Language and
Literacy
Competence
Numeracy
Competence
Country A
Country B

24. ECE Setting Questionnaire
• To help gain a deeper understanding of key aspects of ECE
from the perspective of those responsible for
leading/managing ECE settings.
• It will be completed by the leaders/managers of the
sampled settings (approximately 200 per country).
• It will take approximately 45 minutes to complete and
can be administered on paper or online.
• The questions will capture information on setting
characteristics and all three study themes.
24

25. Practitioner Questionnaire
• To help gain a deeper understanding of key aspects of
ECE from the perspective of practitioners
• Focus on pedagogical action and the quality of the
settings
• Target population: All staff directly responsible for the
education and care of children in the last year of ISCED
Level 0
• It will take approximately 45 minutes to complete and
can be administered on paper or online.
25

26. Home Questionnaire
• The Home Questionnaire will enable the study to report
on the expectations, experiences and views of parentsof
children currently attending ECE in the year before ISCED
1.
• It will be completed by the parent/primary caregiver of
the sampled children (approximately 3000 parents per
country).
• It will take approximately 30 minutes to complete and can
be administered on paper or online.
• The questions will capture information on family/child
characteristics and will focus on all three study themes.
26

27. Study Publications
• Study Framework
– Conceptual underpinnings
– To be published after Field Trial (Feb 2016)
• International Reports
– Results from international data collection summarized in four
thematic reports
– Published in 2015, 2017 and 2018
• Technical Report
– Information on methods and procedures
• International Database and User Guide
– To be released in 2018
27

28. Study Milestones – Phase 1
• 2013
– Jun 10-11: 1st Project Advisory Committee meeting
(Birmingham/ UK)
– Jul 29-Aug 1: 1st National Research Coordinator meeting
(Hamburg/ Germany)
– Nov/Dec: Pilot Policy Questionnaire
• 2014
– Apr/ May: Main Study Policy Questionnaire
• 2015
– Feb: Publish Report 1:
Outcomes of Policy Questionnaire
28

29. Study Milestones – Phase 2
• 2014
– Jan 9-10: PAC meeting (Hamburg, Germany)
– Feb 24-28: 2nd NRC Meeting (Rome, Italy)
– Sep 15-19: 3rd NRC Meeting (TBA)
– Nov: Data Management Seminar (Hamburg, Germany)
• 2015
– Apr/ May: Field Test
• 2016
– Feb: Publish Study Framework
– Apr/May: Main Study Northern Hemisphere Countries
– Sep/Oct: Main Study Southern Hemisphere Countries
29

30. Study Milestones – Phase 2
• 2017
– Oct: Publish Report 2: Outcomes Questionnaires
• 2018
– Feb: Publish Report 3: Assessment Outcomes
– Sep: Publish Report 4: Final Report
– Nov: Release International Database
30