Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Experiences in assessing early learning achievement in international contexts (2011)


Published on

Jindani, F., & Shallwani, S. (May, 2011). Experiences in assessing early learning achievement in international contexts: adapting an ‘international’ tool or ‘locally’ developing a new one? Paper presented at the Annual Conference of the Comparative and International Education Society, Montreal.

Abstract: In recent decades, countries around the world have aimed to increase primary school enrolment rates (Filmer, Hasan, & Pritchett, 2006). However, school quality varies widely and many children in school are not learning effectively. For example, a learning achievement study in India found that almost half of fifth grade children were unable to read at second grade proficiency levels (Pratham, 2007). If children do not establish basic literacy skills in the first few years of education, they are at a severe disadvantage for the remainder of their schooling and lives. In order to understand and improve the quality of education and students’ learning, it is necessary to have indicators and standards for early learning achievement, literacy in particular, at both local and global levels.
Some researchers and evaluators have aimed to develop and adapt standard international indicators for use in various contexts around the world. However, others, coming from a social-constructivist perspective, have argued that such tools are products of the Western world and inappropriate for diverse contexts of the world. In this paper, we compare and contrast the two perspectives with literature and research evidence, as well as our own experiences in developing indicators of learning achievement in international contexts. We share the experience of adapting an ‘international’ tool (the Early Grades Reading Assessment) for the Cambodian context and the experience of developing a ‘local’ Learning Achievement Tool in Pakistan.
We also deconstruct notions of ‘international’ and ‘local’, and critically reflect on our role precisely as the international researchers in these contexts. We argue that standards must be established both at local and international levels through reflection and explicit discussion about contexts, social positions, values, and purpose (Myers, 2004), and that these standards must then be continuously reflected upon and subject to change over time and place.

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

Experiences in assessing early learning achievement in international contexts (2011)

  1. 1. Adapting an international tool or locally developing a new one? Farah Jindani Sadaf Shallwani University of Toronto (OISE)Jindani, F., & Shallwani, S. (May, 2011). Experiences in assessing early learningachievement in international contexts: adapting an ‘international’ tool or ‘locally’developing a new one? Paper presented at the Annual Conference of theComparative and International Education Society, Montreal.
  2. 2. Outline of Presentation Background Experiences adapting an international tool Experiences developing a local tool Reflections A suggested framework for moving forward
  3. 3. Background Higher enrollment rates globally Concerns: Completion rates Levels of learning Uganda: 46% of third graders and 50% of sixth graders are not reaching expected levels of competency in literacy India: 47% of Grade 5 children could not read a story text at a Grade 2 level of difficulty
  4. 4. Background Where is the system failing? Need indicators and standards to assess the situation, identify areas needing intervention, and monitoring progress Need for indicators that are reliable, contextually meaningful, and socially relevant Two approaches: adapting an international tool or developing a new tool
  5. 5. Cambodian Context Ratanakiri is home to a number of ethnic minority groups; Khmer 33% of population Ethnic minority children have trouble accessing relevant and meaningful education CARE long history of education programs in Cambodia HCEP (2002): Bilingual model of education
  6. 6. Adapting an International Tool Most assessments do not assess reading skills before fourth grade Lack knowledge tested by assessments or lack basic reading and comprehension skills? Remedial instruction Most assessments based on word recognition tasks
  7. 7. International Tool RDI International (2007) with support from World Bank and USAID developed Early Grades Reading Assessment (EGRA) EGRA oral assessment designed to assess skills known to predict literacy: Recognizing letters of alphabet Reading simple words Understanding sentences and paragraphs
  8. 8. EGRAQuick to conductHas been adapted in numerous internationalcontextsSalient for testing Khmer literacy in HCEP in RTKbecause of bilingual education
  9. 9. Adapting EGRA EGRA toolkit developed by writer; shared with team Participatory approach Cross-referencing between English and Khmer during adaptation by research team Content validity of national curriculum and community school texts Sound recognition section on EGRA Independent work by team and would then come back together
  10. 10. Experiences in Field Variability amongst data EGRA based literacy assessment tool completed within time frame (20 minutes) Time frame per section deemed too short (60 seconds) for children Curriculum validity; tested grade 4 students
  11. 11. Lessons Learned Organizational issues in relation to context Content issues (straight translation between languages) Benefits and supports
  12. 12. Pakistan RCC programme – early childhood development programme Project of the Aga Khan Foundation, Pakistan; funded by the Dutch government Research study Programme and comparison schools in four provinces Assessing outcomes on school-level, classroom-level, and child-level indicators Child-level: learning achievement in early primary
  13. 13. Learning Achievement Tool Decision to develop a tool locally Concerns over appropriateness of using tools developed in other contexts Desire to have a tool that assesses children’s learning according to local benchmarks / standards Process of development Identified National Curriculum benchmarks – English literacy, Urdu literacy, and numeracy Reviewed textbooks from different regions Drafted questions/tasks Collaborative process of reviewing, piloting, and revising each task
  14. 14. Experiences in the fieldAdministration Training, field practice, field supervision Two RAs at a school, 1 RA with up to 5 children Consent Counterbalanced order Explain each question with example to child (individually) – give time, prompt at least twice If child does not to 3 questions in a row, move on to next portion Took about 20 minutes to administer
  15. 15. Experiences in the fieldChallenges Difficult at first to administer Fine line between explaining and leading RA bias – desire for child to perform wellStrategies to overcome Developed standard guidelines for RAs to use in explaining the task and examples Field supervision and supportIssues of research ethics with children
  16. 16. Lessons learned Test development process Difficulties developing questions relevant to varying contexts of Pakistan Need to assess reliability and validity Importance of standard implementation of tool Children from programme schools outperformed children in comparison schools
  17. 17. Reflections International-local: not binary Tools Different contexts within country Role of international researcher Power/presumed expertise Internal struggles on decisions (e.g., presumed credibility of ‘international’ tools) How do we create/adapt tools to assess learning achievement that are contextually meaningful and credible?
  18. 18. A framework for moving forward Quality in early childhood (Myers, 2004) Ongoing process of discussion and dialogue in the centre – all stakeholders Discuss ‘what kind of society do we want?’ ‘what kind of citizens are needed for that society?’ Derive areas agreement, construct major categories and indicators Use research and experience Make value positions and assumptions explicitly Distinguish purpose of using instruments Ongoing – society is not static