The document discusses the role of large-scale assessments, specifically PIRLS and TIMSS, in evaluating and improving educational systems. It highlights how these assessments measure student achievement in mathematics, science, and reading, while providing insights for researchers, policy makers, and educators. The document also outlines the IEA's objectives and methodologies in conducting these assessments across various countries.

1. Using Large-Scale Assessments to
Improve Schools
Andrés Sandoval-Hernández – IEA DPC
International Conference “Improve the school”
Naples, May 14-15, 2015

2. Contents
• About the IEA DPC
• Introduction to PIRLS and TIMSS
• Benchmarking function
– International Reports PIRLS and TIMSS 2011
– Report on relationships between achievement in Math, Science and
Reading
• Analytic function
– School effectiveness factors for socially advantaged and disadvantaged
students in European countries
• Evaluation tools for teachers
– Use of released items

3. About the IEA DPC
Who we are
• International Association for the
Evaluation of Educational Achievement
- Data Processing and Research Center
• Located in Hamburg, Germany
• Association of countries
What we do
• International Large scale assessments
in education
• Main studies: TIMSS and PIRLS (2011)
• Major activities are comparative
educational studies, research and
training
• Almost 70 country members

4. Introduction to PIRLS and TIMSS
Major postulates of the IEA studies:
– The quality of an educational system should be measured
by what students know
– What students know is related to the characteristics of the
System
– Assessments should be school based and curriculum
driven
What do PIRLS and TIMSS measure?
– Achievement in Mathematics, Science and Reading
– Student characteristics and learning environment (family,
school, context)

5. Introduction to PIRLS and TIMSS
Who uses this information?
1. Researchers
2. Educational policy makers
3. Teachers and principals
4. Parents

6. • Areepattamannil, S. (2012). Science self-beliefs and science achievement of adolescents in Gulf Cooperation Council countries. Educational Studies,
38(1), 13–17. Doi:10.1080/03055698.2011.567058
• Areepattamannil, S. (2012). First- and second-generation immigrant adolescents’ multidimensional mathematics and science self-concepts and their
achievement in mathematics and science. International Journal of Science & Mathematics Education, 10(3), 695–716.
• Bakker, S. (2012). Introduction of external, independent testing in “new countries”: Successes and defeats of the introduction of modern
educational assessment techniques in former soviet and socialist countries. Educational Measurement: Issues & Practice, 31(2), 38–44.
Doi:10.1111/j.1745-3992.2012.00234.x
• Chen, Y.-H. (2012). Cognitive diagnosis of mathematics performance between rural and urban students in Taiwan. Assessment in Education:
Principles, Policy & Practice, 19(2), 193–209. Doi:10.1080/0969594x.2011.560562
• Cho, I. (2012). The effect of teacher–student gender matching: Evidence from OECD countries. Economics of Education Review, 31(3), 54–67.
Doi:10.1016/j.econedurev.2012.02.002
• Choi, K., Choi, T., & Mcaninch, M. (2012). A comparative investigation of the presence of psychological conditions in high achieving eighth graders
from TIMSS 2007 mathematics. ZDM, 44(2), 189–199. Doi:10.1007/S11858-012-0401-6
• Dodeen, H., Abdelfattah, F., Shumrani, S., & Hilal, M. A. (2012). The effects of teachers’ qualifications, practices, and perceptions on student
achievement in TIMSS mathematics: a comparison of two countries. International Journal of Testing, 12(1), 61–77.
Doi:10.1080/15305058.2011.621568
• Gültekin, S., & Demirtaşli, N. Ç. (2012). Çoktan seçmeli, açik uçlu ve karma testlerden sağlanan bilginin madde tepki kuramina dayali olarak
karşilaştirilmasi. (in Turkish). Ilkogretim Online, 11(1), 251–263.
• Gür, B. S., Çelik, Z., & Özoğlu, M. (2012). Policy options for turkey: a critique of the interpretation and utilization of PISA results in Turkey. Journal of
Education Policy, 27(1), 1–21. Doi:10.1080/02680939.2011.595509
• House, J. D. (2012). Motivational effects of computers and classroom instruction for science learning: an examination of eighth-grade students in
the United States and Korea in the TIMSS 2007 Assessment. International Journal of Instructional Media, 39(2), 169–179.
• House, J. D., & Telese, J. A. (2012). Effects of mathematics lesson activities and computer use on algebra achievement of eighth-grade students in
the United States and japan: findings from the TIMSS 2007 Assessment. International Journal of Instructional Media, 39(1), 69–81.
• Jianjun Wang. (2012). A comparative study of science achievement and self-concept during a historical transition. International Education Studies,
5(1), 137–146. Doi:10.5539/ies.v5n1p137
• Kwon, O., & Ju, M.-K. (2012). Standards for professionalization of mathematics teachers: policy, curricula, and national teacher employment test in
Korea. ZDM, 44(2), 211–222. Doi:10.1007/s11858-012-0384-3
• Lloyd, D., & Pratt, A. (2012). The new OFSTED framework - Implications for teaching and learning. Teaching Business & Economics, 16(2), 28–29.
• Männamaa, M., Kikas, E., Peets, K., & Palu, A. (2012). Cognitive correlates of math skills in third-grade students. Educational Psychology, 32(1), 21–
44. Doi:10.1080/01443410.2011.621713
• Park, H., Lawson, D., & Williams, H. E. (2012). Relations between technology, parent education, self-confidence, and academic aspiration of Hispanic
immigrant students. Journal of Educational Computing Research, 46(3), 255–265.
• Schmidt, W. H. (2012). At the precipice: The story of mathematics education in the United States. Peabody Journal of Education, 87(1), 133–156.
Doi:10.1080/0161956x.2012.642280
• Serrano, A. M. (2012). A cross-cultural investigation into how tasks influence seatwork activities in mathematics lessons. Teaching & Teacher
Education, 28(6), 806–817. Doi:10.1016/j.tate.2012.03.001
• Spencer, J. A. (2012). Views from the black of the math classroom. Dissent, 59(1), 76–80.
American Behavioral Scientist, 56(7), 941–960. Doi:10.1177/0002764211408861
• Yoshino, A. (2012). The relationship between self-concept and achievement in TIMSS 2007: a comparison between American and Japanese
students. International Review of Education / Internationale Zeitschrift für Erziehungswissenschaft, 58(2), 199–219. Doi:10.1007/s11159-012-
9283-7
• Zhu, Y., & Leung, F. (2012). Homework and mathematics achievement in Hong Kong: evidence from the TIMSS 2003. International Journal of
Science & Mathematics Education, 10(4), 907–925.
• Bagata, B., Geske, A., & Kiselova, R. (2012). Using the IEA TIMSS tests to compare pupils´ Science Education Achievements at regional and
school levels. Journal of Baltic Science Education, 3(1).
• Cavanagh, S. (2012). Complex policy options abound amid international comparisons. Education Week, 31(16), 6–10.
• Chen, S. F., Lin, C. Y., Wang, J. R., Lin, S. W., & Kao, H. L. (2012). A cross-grade comparison to examine the context effect on the relationships
among family resources, school climate, learning participation, science attitude, and science achievement based on TIMSS 2003 in Taiwan.
International Journal of Science Education, 1-18.
• Falch, T., & Fischer, J. A. V. (2012). Public sector decentralization and school performance: International evidence. Economics Letters, 114(3),
276–279.
• Sparks, S. D. (2012). Matching up states, countries gives fresh take on performance. Education Week, 31(16), 36–38.
• Wang, Z., Osterlind, S., & Bergin, D. (N.D.). Building mathematics achievement models in four countries using TIMSS 2003. International Journal
of Science and Mathematics Education, 1–28. Doi:10.1007/s10763-011-9328-6
• Brozo, W. G., Valtin, R., Garbe, C., Sulkunen, S., Shiel, G., & Pandian, A. (2012). A report from the PISA/PIRLS task force: member country
highlights from PISA 2009. Reading Today, 29(6), 11–13.
• Caro, D. H., & Lenkeit, J. (2012). An analytical approach to study educational inequalities: 10 hypothesis tests in PIRLS 2006. International
Journal of Research & Method in Education, 35(1), 3–30. Doi:10.1080/1743727x.2012.666718
• Thiel, O. (2012). Socio-economic diversity and mathematical competences. European Early Childhood Education Research Journal, 20(1), 61–
81. Doi:10.1080/1350293x.2012.650012
• Van Staden, S., & Howie, S. (2012). Reading between the lines: contributing factors that affect grade 5 student reading performance as
measured across South Africa’s 11 languages. Educational Research & Evaluation, 18(1), 85–98. Doi:10.1080/13803611.2011.641270
• Caro, D. H., & Mirazchiyski, P. (2012). Socioeconomic gradients in eastern European countries: Evidence from PIRLS 2006. European
Educational Research Journal, 11(1), 96–110.
• Cavanagh, S. (2012). Complex policy options abound amid international comparisons. Education Week, 31(16), 6–10
• Blömeke, S., Suhl, U., Kaiser, G., & Döhrmann, M. (2012). Family background, entry selectivity and opportunities to learn: what matters in
primary teacher education? An international comparison of fifteen countries. Teaching & Teacher Education, 28(1), 44–55.
Doi:10.1016/j.tate.2011.08.006
• Sawchuk, S. (2012). Teacher training has key role to play. Education Week, 31(16), 14.
• Döhrmann, M., Kaiser, G., & Blömeke, S. (2012). The conceptualization of mathematics competencies in the International Teacher Education
Study TEDS-M. ZDM, 44(3), 325–340. Doi:10.1007/s11858-012-0432-z
• Felbrich, A., Kaiser, G., & Schmotz, C. (2012). The cultural dimension of beliefs: An investigation of future primary teachers’ epistemological
beliefs concerning the nature of mathematics in 15 countries. ZDM, 44(3), 355–366. Doi: 10.1007/s11858-012-0418-x
• König, J., & Blömeke, S. (2012). Future teachers’ general pedagogical knowledge from a comparative perspective: Does school experience
matter? ZDM, 44(3), 341–354. Doi: 10.1007/s11858-012-0394-1
• Blömeke, S., & Delaney, S. (2012). Assessment of teacher knowledge across countries: A review of the state of research. ZDM, 44(3), 223–247.
Doi:10.1007/s11858-012-0429-7
• Schmidt, W. H. (2012). At the precipice: the story of mathematics education in the United States. Peabody Journal of Education, 87(1), 133–
156. Doi:10.1080/0161956x.2012.642280
• Blömeke, S., & Kaiser, G. (2012). Homogeneity or Heterogeneity? Profiles of opportunities to learn in primary teacher education and their
relationship to cultural context and outcomes. ZDM, 44(3), 249–264. Doi:10.1007/s11858-011-0378-6
• Cole, Y. (2012). Assessing elemental validity: the transfer and use of mathematical knowledge for teaching measures in Ghana. ZDM, 44(3),
415–426. Doi:10.1007/s11858-012-0380-7
• Delaney, S. (2012). A validation study of the use of mathematical knowledge for teaching measures in Ireland. ZDM, 44(3), 427–441.
Doi:10.1007/s11858-012-0382-5
• Fauskanger, J., Jakobsen, A., Mosvold, R., & Bjuland, R. (2012). Analysis of psychometric properties as part of an iterative adaptation process of
MKT items for use in other countries. ZDM, 44(3), 387–399. Doi:10.1007/s11858-012-0403-4
• Senk, S., Tatto, M., Reckase, M., Rowley, G., Peck, R., & Bankov, K. (2012). Knowledge of future primary teachers for teaching mathematics: An
international comparative study. ZDM, 44(3), 307–324. Doi: 10.1007/s11858-012-0400-7
• Barber, C., & Torney-Purta, J. (2012). Comparing attitudes in the 1999 and 2009 IEA Civic and Citizenship Education Studies: Opportunities and
limitations illustrated in five countries. Journal of Social Science Education, 11(1), 47–63.
• Cosgrove, J., & Gilleece, L. (2012). An international perspective on civic participation in irish post-primary schools: Results from ICCS. Irish
Educational Studies, 1–19. Doi:10.1080/03323315.2012.664948
• Gilleece, L., & Cosgrove, J. (2012). Student civic participation in school: what makes a difference in Ireland? Education, Citizenship and Social
Justice. Doi:10.1177/1746197912448715
• Ince, B. (2012). Citizenship education in turkey: inclusive or exclusive. Oxford Review of Education, 38(2), 115–131.
Doi:10.1080/03054985.2011.651314
• Liljestrand, J. (2012). Citizenship education in discussions concerning political issues. Utbildning & Demokrati, 21(1), 77–95.
• Ljunggren, C., & Unemar Öst, I. (2012). Teachers’ dealings with controversial issues – a typology from the 2009 IEA/ICCS study. PCS – Politics,
Culture and Socialization, 2(3).
• Quaynor, L. J. (2012). Citizenship education in post-conflict contexts: a review of the literature. Education, Citizenship and Social Justice, 7(1),
33–57. Doi:10.1177/1746197911432593
• Kennedy, K. J. (2012). Asian students’ citizenship values and their relationship to civic understanding: An exploratory study comparing Thai and
Hong Kong students. Research in Comparative and International Education, 7(2), 248–259.
• Cosgrove, J., & Gilleece, L. (2012). An international perspective on civic participation in Irish post-primary schools: Results from ICCS. Irish
Educational Studies, 31(4), 377–395.
• Gilleece, L., & Cosgrove, J. (2012). Student civic participation in school: What makes a difference in Ireland? Education, Citizenship & Social
Justice, 7(3), 225–239.
• Macháček, L. (2012). Education of the school youth to democratic citizenship in Slovakia. Mokinių Demokratinio Pilietiškumo Ugdymas
Slovakijoje., (107), 110–117.
• House, J. D. (2012). Effects of computer engagement and classroom instructional activities on reading achievement of a national sample of
Chinese- American elementary- school students: results from the TIMSS 2006 assessment. International Journal of Instructional Media, 39(4),
345–354.
• Johansson, S., Myrberg, E., & Rosén, M. (2012). Teachers and tests: assessing pupils’ reading achievement in primary schools. Educational
Research & Evaluation, 18(8), 693–711.
• Lee, M., & Hallinger, P. (2012). National contexts influencing principals’ time use and allocation: Economic development, societal culture, and
educational system. School Effectiveness & School Improvement, 23(4), 461–482.
• Sparfeldt, J. R., Kimmel, R., Löwenkamp, L., Steingräber, A., & Rost, D. H. (2012). Not read, but nevertheless solved? Three experiments on
PIRLS multiple choice reading comprehension test items. Educational Assessment, 17(4), 214–232.
• Abu-Hilal, M. M., Abdelfattah, F. A., Alshumrani, S. A., Abduljabbar, A. S., & Marsh, H. W. (N.D.). Construct validity of self-concept in TIMSS’s
student background questionnaire: a test of separation and conflation of cognitive and affective dimensions of self-concept among Saudi
eighth graders. European Journal of Psychology of Education, 1–20. Doi:10.1007/s10212-012-0162-1
among family resources, school climate, learning participation, science attitude, and science achievement based on TIMSS 2003in Taiwan.
International Journal of Science Education, 34(14), 2089–2106. Doi:10.1080/09500693.2012.701352
• Glynn, S. M. (2012). International Assessment: A RASCH model and teachers’ evaluation of TIMSS science achievement items. Journal of
Research in Science Teaching, 49(10), 1321–1344.
• Hansson, Å. (2012). The meaning of mathematics instruction in multilingual classrooms: Analyzing the importance of responsibility for learning.
Educational Studies in Mathematics, 81(1), 103–125.
• Hee Kyung Hong. (2012). Trends in mathematics and science performance in 18 countries: Multiple regression analysis of the cohort effects of
TIMSS 1995-2007. Tendências no desempenho em matemáticas e ciências em 18 países: análise de regressão múltipla de efeitos de coorte no
TIMSS 1995-2007., 20(33), 1–20.
• House, J. D. (2012b). Science achievement of elementary-school students in the United States and Japan in TIMSS 2007: An assessment of the
effects of technology engagement and classroom lesson activities. International Journal of Instructional Media, 39(3), 263–274.
• Louis, R., & Mistele, J. (2012). The differences in scores and self-efficacy by student gender in mathematics and science. International Journal of
Science & Mathematics Education, 10(5), 1163–1190.
• Mesic, V. (2012). Identifying country-specific cultures of physics education: A differential item functioning approach. International Journal of
Science Education, 34(16), 2483–2500.
• Mohammadpour, E. (2012a). Factors accounting for mathematics achievement of Singaporean eighth-graders. Asia-Pacific Education
Researcher, 21(3), 507–518.
• Mohammadpour, E. (2012b). A multilevel study on trends in Malaysian secondary school students’ science achievement and associated school
and student predictors. Science Education, 96(6), 1013–1046.
• Ng, K. T., Lay, Y. F., Areepattamannil, S., Treagust, D. F., & Chandrasegaran, A. L. (2012). Relationship between affect and achievement in
science and mathematics in Malaysia and Singapore. Research in Science & Technological Education, 30(3), 225–237.
• Rutkowski, D., Rutkowski, L., & Plucker, J. A. (2012). Trends in education excellence gaps: A 12-year international perspective via the multilevel
model for change. High Ability Studies, 23(2), 143–166.
• Vandecandelaere, M., Speybroeck, S., Vanlaar, G., De Fraine, B., & Van Damme, J. (2012). Learning environment and students’ mathematics
attitude. Studies in Educational Evaluation, 38(3/4), 107–120.
• Wang, Z., Osterlind, S. J., & Bergin, D. A. (2012). Building mathematics achievement models in four countries using TIMSS 2003. International
Journal of Science and Mathematics Education, 10(5), 1215–1242. Doi:10.1007/s10763-011-9328-6
• Wiseman, A. W., & Anderson, E. (2012). ICT-Integrated Education and National Innovation Systems in the Gulf Cooperation Council (GCC)
countries. Computers & Education, 59(2), 607–618.
• Yu, C. H. (2012). Examining the relationships among academic self-concept, instrumental motivation, and TIMSS 2007 science scores: A cross-
cultural comparison of five East Asian countries/regions and the United States. Educational Research & Evaluation, 18(8), 713–731.
• Blómeke, S. (2012). Content, Professional Preparation, and Teaching Methods: How Diverse Is Teacher Education across Countries? Contenido,
preparación profesional y métodos de enseñanza: ¿Cuan diversa es la educación de los profesores entre países?, 56(4), 684–714.
• Chudgar, A., Luschei, T. F., & Yisu Zhou. (2013). Science and Mathematics Achievement and the Importance of Classroom Composition:
Multicounty Analysis Using TIMSS 2007. American Journal of Education, 119(2), 295–316.
• Hopland, A. O. (2013). School Facilities and Student Achievement in Industrial Countries: Evidence from the TIMSS. International Education
Studies, 6(3), 162–171.
• Peterson, P. E. (2013). Revelations from the TIMSS. Education Next, 13(2), 5–5.
School Students’ Science Achievement. School Science & Mathematics, 113(2), 69–79.
• Turgut, G. (2013). International Tests and the U.S. Educational Reforms: Can Success Be Replicated? Clearing House, 86(2), 64–73.
• Hao, S., & Johnson, R. L. (2013). Teachers’ classroom assessment practices and fourth-graders’ reading literacy achievements: An international
study. Teaching & Teacher Education, 29, 53–63.
• Sandilands, D., Oliveri, M. E., Zumbo, B. D., & Ercikan, K. (2013). Investigating Sources of Differential Item Functioning in International Large-
Scale Assessments Using a Confirmatory Approach. International Journal of Testing, 13(2), 152–174.
• Cho,Y., & Kim, T. (2013). Asian Civic Values: A Cross-Cultural Comparison of Three East Asian Societies. Asia-Pacific Education Researcher, 22(1),
21–31.
• Geske, A., & Cekse, I. (2013). The Influence of Civic and Citizenship Education Achievements on the Development of Students’ Citizenship
Attitudes. Problems of Education in the 21st Century, 52, 21–34.
• Hooghe, M., & Dassonneville, R. (2013). Voters and Candidates of the Future: The Intention of Electoral Participation among Adolescents in 22
European Countries. Young, 21(1), 1–28.
• Kaiser, G., & Blömeke, S. (2013). Learning from the Eastern and the Western debate: The case of mathematics teacher education. ZDM, 45(1),
7–19.
• Andrews, P. (2013). What Does PISA Performance Tell Us about Mathematics Teaching Quality? Case studies from Finland and Flanders. Oxford
Studies in Comparative Education, 23(1), 99–115.
• Daniel, E. G. S. (2013). Asia Pacific science education in a knowledge society. Asia Pacific Journal of Education, 33(2), 170–182.
• Drent, M., Meelissen, M. R. M., & Van Der Kleij, F. M. (2013). The contribution of TIMSS to the link between school and classroom factors and
student achievement. Journal of Curriculum Studies, 45(2), 198–224.
• Hansson, Å., & Gustafsson, J.-E. (2013). Measurement Invariance of Socioeconomic Status across Migrational Background. Scandinavian Journal
of Educational Research, 57(2), 148–166.
• Jen, T.-H., Lee, C.-D., Chien, C.-L., Hsu, Y.-S., & Chen, K.-M. (2013). Perceived Social Relationships and Science Learning Outcomes for Taiwanese
Eighth Graders: Structural Equation Modeling with a Complex Sampling Consideration. International Journal of Science & Mathematics
Education, 11(3), 575–600.
• Larson, M. R., & Leinwand, S. (2013). Prepare for More Realistic Testing Results. Mathematics Teaching in the Middle School, 18(9), 524–526.
• Machin, S., McNally, S., & Wyness, G. (2013). Educational attainment across the UK nations: performance, inequality and evidence. Educational
Research, 55(2), 139–164.
• Mills, J. D., & Holloway, C. E. (2013). The development of statistical literacy skills in the eighth grade: exploring the TIMSS data to evaluate
student achievement and teacher characteristics in the United States. Educational Research & Evaluation, 19(4), 323–345.
• Pahlke, E., Hyde, J. S., & Mertz, J. E. (2013). The Effects of Single-Sex Compared With Coeducational Schooling on Mathematics and Science
Achievement: Data from Korea. Journal of Educational Psychology, 105(2), 444–452.
Scale Assessments Using a Confirmatory Approach. International Journal of Testing, 13(2), 152–174.
• Barber, C., Fennelly, K., & Torney-Purta, J. (2013). Nationalism and Support for Immigrants’ Rights Among Adolescents in 25 Countries. Applied
Developmental Science, 17(2), 60–75.
• Lauglo, J.013). Do more knowledgeable adolescents have more rationally based civic attitudes? Analysis of 38 countries. Educational
Psychology, 33(3), 255–275.
• Schulz, W., Fraillon, J., & Ainley, J. (2013). Measuring young people’s understanding of civics and citizenship in a cross-national study.
Educational Psychology, 33(3), 327–349.
• Gustafsson, J.-E. (2013). Causal inference in educational effectiveness research: a comparison of three methods to investigate effects of
homework on student achievement. School Effectiveness & School Improvement, 24(3), 275–295.
• Marsh, H. W. &Amp;Rft Au=Abduljabbar. (2013). Factorial, convergent, and discriminant validity of TIMSS math and science motivation
measures: A comparison of Arab and Anglo-Saxon countries. The Journal of Educational Psychology, 105(1), 108–128
• Woolard, J. C. (2013). Prelude to the Common Core: Internationally Benchmarking a State’s Math Standards. Educational Policy, 27(4), 615–
644.A three-level multilevel analysis of Singaporean eighth-graders science achievement. Learning & Individual Differences, 26, 212–220.
• Blömeke, S., Suhl, U., & Döhrmann, M. (2013). Assessing Strengths and Weaknesses of Teacher Knowledge in Asia, Eastern Europe, and
Western Countries: Differential Item Functioning in TEDS-M. International Journal of Science & Mathematics Education, 11(4), 795–817.
• Cañadas, M., Gómez, P., & Rico, L. (2013). Structure of Primary Mathematics Teacher Education Programs in Spain. International Journal of
Science & Mathematics Education, 11(4), 879–894.
• Hsieh, F.-J. (2013). Strengthening the Conceptualization of Mathematics Pedagogical Content Knowledge for International Studies: A Taiwanese
Perspective. International Journal of Science & Mathematics Education, 11(4), 923–947.
• Hsieh, F.-J., Wong, K., & Wang, T.-Y. (2013). Are Taiwanese and Singaporean Future Teachers Similar in Their Mathematics-Related Teaching
Competencies? International Journal of Science & Mathematics Education, 11(4), 819–846.
• Laschke, C. (2013). Effects of Future Mathematics Teachers’ Affective, Cognitive and Socio-Demographic Characteristics on Their Knowledge at
the End of the Teacher Education in Germany and Taiwan. International Journal of Science & Mathematics Education, 11(4), 895–921.
• Wang, T.-Y., & Tang, S.-J. (2013). Profiles of Opportunities to Learn for Teds-M Future Secondary Mathematics Teachers. International Journal
of Science & Mathematics Education, 11(4), 847–877.
• Zuckerman, G. A., Kovaleva, G. S., & Kuznetsova, M. I. (2013). Between PIRLS and PISA: The advancement of reading literacy in a 10–15-year-
old cohort. Learning & Individual Differences, 26, 64–73.
• Nemeržitski, S., Loogma, K., Heinla, E., & Eisenschmidt, E. (2013). Constructing model of teachers’ innovative behaviour in school environment.
Teachers & Teaching, 19(4), 398–418.
How do they use the information? What do they do
with it?

7. Benchmarking function:
International Reports
PIRLS and TIMSS 2011

8. PIRLS International Report
Progress in International Reading Literacy Study

9. PIRLS International Report
Further increases in performance than decreases
- Compared to 2001, ten
countries increased their
average.
Compared to 2006,
thirteen countries
increased their average.
Only four countries
recorded declines:
Bulgaria, Lithuania, the
Netherlands and Sweden.

10. TIMSS International Report (Mathematics 4th grade)
Trends in International Mathematics and Science Study

11. TIMSS International Report (Mathematics 8th grade)
Trends in International Mathematics and Science Study

12. TIMSS International Report
More increases than decreases in progress in 4th grade, but not 8th grade
- In 4th grade, of the 17 countries with comparable data, thirteen
recorded increases and only three recorded decreases.
- In 8th grade, of the 25 countries with comparable data, nine
recorded increases and eleven recorded decreases.

13. 13
TIMSS International Report
Increases in Mathematic Achievement (8th grade) in Italy between 1999 and 2011

14. 14
TIMSS International Report
Increases in Mathematic Achievement (8th grade) in Singapore and Italy between 1999 and 2011
• Singapore: Increase of 6 Score Points between 1999 and 2011
• Italy: Increase of 16 Score Points between 1999 and 2011
604 605 593
611
479 484 480 495
0
100
200
300
400
500
600
700
1999 2003 2007 2011
Mathematic Achievement from 1999 to 2011
Singapore
Italy

15. TIMSS International Report (Science 4th grade)
Trends in International Mathematics and Science Study

16. TIMSS International Report (Science 8th grade)
Trends in International Mathematics and Science Study

17. TIMSS International Report
More increases than decreases in performance particularly in the 4th grade
- In 4th grade, of the 17 countries with comparable data, eight
reported increases and only one reported decreases.
- In 8th grade, of the 25 countries with comparable data, eleven
reported increases and six reported decreases.

18. 18
TIMSS International Report
Results in Science Achievement (8th grade) in Italy between 1999 and 2011 remain stable

19. 19
TIMSS International Report
Increases in Science Achievement (8th grade) in Singapore and Italy between 1999 and 2011
568 578 567
590
493 491 495 501
0
100
200
300
400
500
600
700
1999 2003 2007 2011
Science Achievement from 1999 to 2011
Singapore
Italy
• Singapore: Increase of 22 Score Points between 1999 and 2011
• Italy: Increase of 8 Score Points between 1999 and 2011

20. Benchmarking function:
Report on relationships between
achievement in Math, Science and Reading
20

21. Relationships between achievement in Mathematics, Science and
Reading
• The analysis includes only countries that applied
PIRLS and TIMSS to the same students (4th grade)
• It is an edited volume with four chapters
– Profiles of achievement in mathematics, reading and
science
– The impact of reading skill achievement in TIMSS (math
and science)
– Effective schools in mathematics, reading and science
– The effects of home background on academic achievement

22. Relationships between achievement in Mathematics, Science and Reading
Achievement Profiles

23. Relationships between achievement in Mathematics, Science and Reading
Achievement Profiles

24. Relationships between achievement in Mathematics, Science and Reading
Achievement Profiles

25. Relationships between achievement in Mathematics, Science and Reading
Achievement Profiles

26. Analytic function:
A Comparison of School Effectiveness
Factors for Socially Advantaged and
Disadvantaged Students in ten European
Countries in TIMSS-2011
26

27. Objectives
• To what extent an Educational Effectiveness Research model works similarly
for socially advantaged and disadvantaged students?
Methods
• Using HLM analyses on the TIMSS 2011 dataset, the fit of the EER model is
evaluated in two sub-samples:
1. Advantaged Students
2. Disadvantaged Students
• comparing the results across ten European countries, including Italy
Results
• EER model fit better the data of advantaged students in most countries and
in both levels (i.e. student and school level)
A Comparison of School Effectiveness Factors for Socially Advantaged and
Disadvantaged Students

28. Control
• Resources for learning at
home
• Early reading and math
skills
Analysis
• Security and order
atmosphere
• Support for academic
success
• Adequate infrastructure
• Trained teachers
• Well-equipped classrooms
• Effective instruction
Relationships between achievement in Mathematics, Science and Reading
Effective schools in Mathematics, Science and Reading

29. 29
A Comparison of School Effectiveness Factors for Socially Advantaged and
Disadvantaged Students
Summary of results for all countries
Results for Italy:
• Overall, the EER model fits the sample of advantaged
students better (on both levels), but…

30. Evaluation tools for teachers:
Use of released items
30

31. Use of released items
• Teachers can use released items in different ways
– To evaluate specific processes, such as processes for Reading
Comprehension:
- Make direct inferences
- Locate and obtain explicit information
- Interpret and integrate ideas and information
– Teachers can give the items to the students and grade them
using the corresponding guides
– Teachers can even compare the percentage of students who
responded correctly with the international average

32. For feedback on the teaching-learning process
– When the concepts of the class match the items released,
the teacher can use the same procedure to get immediate
feedback on their performance
To understand misunderstandings or confusions
– The teacher can analyze incorrect or partially correct answers
in its class to understand possible confusion and reinforce a
specific topic
To identify individual difficulties
– As a basis for planning remedial actions
Use of released items

33. Use of released items
Example for schools: Reading comprehension
Enemy Pie
Written by Derek Munson,
Illustrated by Calahan King

34. Use of released items
Example for schools: Mathematics

35. Use of released items
Example for schools: Science

36. Publications:
Mullis, I.V.S., Martin, M.O., Foy, P., & Drucker, K.T. (2012). PIRLS 2011 International Results in Reading. Chestnut Hill,
MA: TIMSS & PIRLS International Study Center, Boston College.
Mullis, I.V.S., Martin, M.O., Foy, P., & Arora, A. (2012). TIMSS 2011 International Results in Mathematics. Chestnut
Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
Martin, M.O., Mullis, I.V.S., Foy, P., & Stanco, G.M. (2012). TIMSS 2011 International Results in Science. Chestnut
Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
Martin, M.O. & Mullis, I.V.S. (Eds.). (2013). TIMSS and PIRLS 2011: Relationships among Reading, Mathematics, and
Science Achievement at the Fourth Grade Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston
College.
Sandoval-Hernandez, A., Castejon-Company, A. & Aghakasari, P. (2015). A Comparison of School Effectiveness
Factors for Socially Advantaged and Disadvantaged Students in ten European Countries in TIMSS 2011. Šolsko polje
25 (3-4), 61-96.
Nielsen, T. & Angell, C. (2014). The importance of discourse and attitude in learning astronomy: A mixed methods
approach to illuminate the results of the TIMSS 2011 survey. NorDiNa 10(1), 16-31.
More Information:
www.iea-dpc.de | www.iea.nl
FB: www.facebook.com/IEARANDA