Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

2011 TESOL Seminar 4: Assessment for ESL learners: What, why and for whom?


Published on

2011 TESOL Seminar 4: Assessment for ESL learners: What, why and for whom?
Prof Tim McNamara, Faculty of Arts, School of Language and Linguistics, The University of Melbourne

Published in: Education
  • Be the first to comment

2011 TESOL Seminar 4: Assessment for ESL learners: What, why and for whom?

  1. 1. Classroom based assessment: The teacher's perspective Tim McNamara [email_address]
  2. 2. <ul><li>What are the most pressing issues in assessment that you face? </li></ul><ul><li>Agree on two in order of priority </li></ul>
  3. 3. Outline <ul><li>What is assessment? </li></ul><ul><li>Standards-based assessment and the ESL teacher </li></ul><ul><li>Example: ESL standards in Victoria </li></ul><ul><li>Study of implementation of standards </li></ul><ul><li>Study of classroom based assessment </li></ul><ul><li>Impact of external tests (NAPLAN; NCLB) </li></ul>
  4. 4. What is assessment? (1) Assessment - internal <ul><li>Assessment is a process of </li></ul><ul><ul><li>observing evidence from performance and </li></ul></ul><ul><ul><li>interpreting it </li></ul></ul><ul><ul><li>so that we can reach conclusions about what individuals know or can do and </li></ul></ul><ul><ul><li>in order to make decisions based on these conclusions </li></ul></ul>
  5. 5. Assessment provides evidence only, and relies on interpretation <ul><li>Assessment only gives us evidence, not certainty </li></ul><ul><li>We interpret the evidence - we draw inferences - we reach conclusions and make claims about students </li></ul><ul><li>e.g. ‘this student can follow a normal classroom lesson in maths’ </li></ul><ul><li>We make decisions about students based on these inferences </li></ul><ul><li>e.g. ‘this student no longer needs ESL support’ </li></ul><ul><li>These claims and decisions can be wrong - just like with detectives or doctors </li></ul>
  6. 6. Validation: Checking on our inferences <ul><li>When the claims and decisions we make about test takers matter a lot, we need to check that they are defensible </li></ul><ul><li>We need to provide reasoning that links the evidence (the score) to the claim </li></ul><ul><li>We need to think about competing explanations for the score - patterns in the score data, student accounts, evidence from judges can be used to investigate these issues </li></ul><ul><li>This investigation is known as validation </li></ul><ul><li>Has parallel in the investigation of inferences in legal trials </li></ul>
  7. 7. What is assessment? (2) Assessment - external <ul><li>Standards are key methods for managing education systems – funding, student pathways, accountability </li></ul>
  8. 8. Format of standards <ul><li>Standards are typically formulated as an ordered series of statements about levels of achievement or stages of development. </li></ul><ul><li>(There may be multiple sets of ordered statements for different aspects of language development) </li></ul>
  9. 9. Possible functions of standards <ul><li>Planning : to act as a series of objectives of goals for teaching and learning; involve clear and specific statements of teaching aims </li></ul><ul><li>Professional understanding : to inform teachers about the typical progress of learning; more complex statements include contextual and interpretative information in order to help the teacher understand more fully the nature of the emergent ability in the learner </li></ul><ul><li>Accountability : to act as statements of learning outcomes for administrative purposes - tends to be dominant function </li></ul>
  10. 10. Structure of ESL standards, Victorian Essential Learning Standards (VELS) STAGE BANDS (YEARS OF SCHOOLING) A1 Lower primary (Years P to 2) A2 B1 Upper primary (Years 3 to 6) B2 B3 S1 Secondary (Years 7 to 10) S2 S3 S4 BL Little literacy, middle/upper primary SL Little literacy, secondary
  11. 11. ESL Companion to VELS
  12. 12. Relationship to the English standards <ul><li>Using the English Standards will not be an accurate reflection of an ESL student’s learning, and can indicate that there are problems in the student’s learning progress, when in fact the student is actually making adequate progress learning English as their second or subsequent language. </li></ul><ul><li>As ESL students learn more English, their learning begins to correspond to the English Standards. If an assessment of an ESL student on the English Standards is within acceptable year level parameters, then it is time to start using the English Standards for assessing and reporting progress. If not, then the ESL Standards should continue to be used. </li></ul>
  13. 14. ESL Developmental Continuum
  14. 15. STAGE BL - Reading BL beginning (B0.1) Students beginning to work towards the standard at BL are new to English and new to literacy. They may have little or no experience of formal learning and do not have literacy strategies already acquired through developing literacy in a first language. They will be able to draw on general learning skills and strategies that they have used to function effectively in their own communities, for example observing, memorising, classifying. They may be reluctant to participate in reading activities. They may recognise their own language in writing, if it has a written form, and may recognise that English print is different from their own language.  BL progressing towards (B0.2) Students progressing towards the standard at BL show interest in print and recognise some environmental print including their name. They can recognise and name some letters. They watch and listen as texts are read aloud to them but may not join in. They rely on peer or teacher support to complete structured activities. They show an interest in books and focus on illustrations. They demonstrate reading-like behaviour such as holding a book, sitting and looking at a book, turning pages and looking at pictures. They are starting to build a bank of English words they recognise, such as environmental print. They read some short, familiar texts that have been well-introduced in class. BL Standard (B0.3) At Stage BL , students read a wide range of familiar, short, simple, repetitive, fictional and everyday texts, and complete simple, structured activities based on them. They retell a simple familiar story, and sequence a simple familiar process with sentences and pictures. They show early understanding that texts are written for a variety of purposes. They show beginning understanding of the sound/symbol relationships of English. They read some familiar words and phrases in context, and recognise, can name, and know the sounds some common letters and letter groups usually make. They recognise that meaning is carried by intonation, and they listen for key words and for repetition of words and phrases in texts read aloud. They focus on illustrations and other non-print features when reading. They use word lists and personal dictionaries to assist them to read new words.
  15. 16. STAGE BL – Speaking and Listening BL beginning (B0.1) Students beginning to work towards the standard at BL have very little or no oral English. They do not respond meaningfully to English. They will join in activities, watching and copying what other students do in the classroom but may not speak. They may spontaneously repeat words or phrases without understanding their meaning. They may not speak in the classroom except to same language peers. They may initially attempt to communicate with the teacher using their own language. They are likely to listen to extended texts in English with visual support. BL progressing towards (B0.2) Students progressing towards the standard at BL are settling into situations where English is the dominant language. They begin to understand that communication with teachers and peers needs to be conducted in English. They begin to learn the very basic oral English needed to manage learning in an English-speaking classroom, where the teacher adapts spoken texts to assist the students. Through their first language experiences, they understand that different forms of language and levels of politeness are used in different situations and contexts. They begin to adapt their limited, emerging English language resources to respond to new communicative and functional demands. They recognise the importance of non-verbal communication. They begin to become familiar with patterns in the sounds, intonation, rhythm, grammar and meaning of English. BL Standard (B0.3) At Stage BL , students communicate simply but effectively in familiar, basic social and classroom contexts, using simple formulaic and creative structures. They learn through English, well supported by context. They contribute relatively complex ideas through simple English, and use simple English to respond to the ideas of others. Students’ English is characterised by varying grammatical accuracy, a short ‘telegraphic’ structure, simple subject/verb/object construction and overgeneralisation of rules. They use common adjectives to describe or add emphasis. They use repetitive grammar patterns copied from stories, songs, rhymes or the media. Students’ pronunciation, stress and intonation are comprehensible, but carry elements of first language pronunciation. They use some basic communication strategies, asking for repetition, and questioning to check understanding, clarify or confirm. They use some basic strategies to initiate and sustain simple conversations in English, restating, repeating or re-pronouncing as appropriate. 
  16. 17. Indicators of progress – 4 aspects: <ul><li>Texts and responses to texts focuses on producing and responding to oral English texts used for social interaction and in the school context across the curriculum. </li></ul><ul><li>Cultural conventions of language use focuses on understanding and using spoken English in a variety of contexts and identifying how different contexts affect the way spoken English is used and interpreted. </li></ul><ul><li>Linguistic structures and features focuses on control over the structures and features of spoken English. </li></ul><ul><li>Maintaining and negotiating communication focuses on the strategies students at this stage typically use to speak in and learn English. </li></ul>
  17. 18. Teaching strategies – Speaking and Listening <ul><li>Listening cloze </li></ul><ul><li>Reaching a consensus </li></ul><ul><li>Barrier games </li></ul><ul><li>Dictogloss </li></ul><ul><li>Running dictation </li></ul><ul><li>Pair dictation </li></ul><ul><li>Language experience </li></ul><ul><li>Role play </li></ul><ul><li>Picture stories </li></ul><ul><li>Surveys </li></ul><ul><li>Tell me </li></ul>
  18. 19. Teaching strategies – Writing <ul><li>Scaffolding writing using the curriculum cycle </li></ul><ul><li>Language experience </li></ul><ul><li>Dictogloss </li></ul><ul><li>Pair dictation </li></ul><ul><li>Running dictation </li></ul><ul><li>Deconstruction </li></ul><ul><li>Joint construction </li></ul>
  19. 22. How will evidence be collected to know what standard a student is at? <ul><li>Using teachers as the agents: </li></ul><ul><li>Formal assessment </li></ul><ul><ul><li>Teacher develops own instruments (less reliable) </li></ul></ul><ul><ul><li>Teacher draws on task bank developed by language testers (more reliable) </li></ul></ul><ul><li>Informal assessment </li></ul><ul><ul><li>Teacher uses naturally occurring assessment opportunities to form picture of child’s standard </li></ul></ul><ul><li>Teacher reports standard of each students </li></ul><ul><li>Using external assessment : </li></ul><ul><li>tests developed by language testers (more reliable) </li></ul>
  20. 23. The assessment pyramid <ul><li>LEVELS </li></ul><ul><li>(NUMBERED ) </li></ul><ul><li>LEVEL </li></ul><ul><li>SUMMARIES </li></ul><ul><li>DIMENSION DESCRIPTIONS </li></ul><ul><li>WITHIN EACH DIMENSION, EXAMPLES PROVIDED </li></ul><ul><li>[ADVICE TO TEACHERS] DETAILED EXAMPLES </li></ul><ul><li>TEACHER CHOOSES ACTIVITY & CRITERIA </li></ul>
  21. 24. Competing demands in standards-based assessment Validity demands Managerialist demands Teacher/ learner demands <ul><li>Intellectual defensibility of construct </li></ul><ul><li>Evidence of Reliability </li></ul><ul><li>Other validity evidence </li></ul><ul><li>Concern for consequences </li></ul><ul><li>Reporting </li></ul><ul><li>Accountability </li></ul><ul><li>Meaningfulness in instructional process </li></ul><ul><li>Facilitation of learning </li></ul><ul><li>Enhanced quality of teaching </li></ul><ul><li>Minimization of administrative burden on teachers </li></ul>
  22. 25. Study of use of ESL assessment frameworks by teachers in Victoria <ul><li>Macqueen, S., L. Harding & C. Elder (2011) Investigating the validity of the ESL VELS student data. Report to DEECD. </li></ul><ul><li>Interview study, 21 representatives from 9 schools </li></ul><ul><li>All participants were teachers in Victorian primary or secondary schools or language centres with responsibility for assessing English as a Second Language (ESL) students. </li></ul><ul><li>The teachers ranged from fairly new graduates to very experienced teachers, including teachers with support, co-ordination and management roles. </li></ul><ul><li>Schools from different geographical regions were selected </li></ul>
  23. 26. Hill (2010) study of assessment cultures <ul><li>Compared assessment cultures of 2 language classrooms in Melbourne: </li></ul><ul><li>Yr 6 (last year of primary) </li></ul><ul><li>Yr 7 (first year of secondary) </li></ul><ul><li>as part of ‘transition’ study </li></ul><ul><li>Year-long ethnographic study of two experienced teachers of Indonesian </li></ul>
  24. 27. Classroom based assessment - definition <ul><li>Any reflection by teachers (and/or learners) on the qualities of a learner’s (or group of learners’) work, and the use of that information by teachers (and/or learners) for teaching, learning (feedback), reporting, management or socialization purposes. </li></ul>
  25. 28. ‘ Assessment opportunity’ <ul><li>Any action, interaction or artifact (planned or unplanned, deliberate or unconscious, explicit or embedded) which has the potential to provide information on the qualities of a learner’s (or group of learners’) performance. </li></ul>
  26. 29. 1. WHAT DO LANGUAGE TEACHERS DO? 1.1 Planning Assessment Is there planning for assessment? How detailed is planning? What is its intended relationship to instruction? How does it relate to external standards and frameworks? 1.2 Framing Assessment Is assessment made explicit to learners? How is this done? 1.3 Conducting Assessment What opportunities does the classroom provide for assessment? Does assessment tend to focus on the class, group/pairs of students or individuals? 1.4 Using Assessment Data How is assessment-related information used? Teaching Learning (feedback) Person-referenced Task-referenced Confirmatory Explanatory Corrective Reporting Management Socialization
  27. 30. 2. WHAT DO TEACHERS LOOK FOR? What information about valued enterprises, qualities and standards is available 2.1 In Advance in written/ verbal instructions and/or assessment rubrics? 2.2 In Feedback in written and/or verbal feedback? 2.3 In Reporting in reporting deliberations and/or in written reports?
  28. 31. 3. WHAT THEORY OR 'STANDARDS' DO THEY USE? 3. Teacher Theories & Beliefs <ul><li>What does the data reveal about teachers’ beliefs about </li></ul><ul><li>the subject or content area </li></ul><ul><li>second language learning and teaching </li></ul><ul><li>the nature of assessment? </li></ul>4. DO LEARNERS SHARE THE SAME UNDERSTANDINGS? 4. Learner Theories & Beliefs <ul><li>What does the data reveal about learners’ beliefs about </li></ul><ul><li>the subject or content area </li></ul><ul><li>second language learning and teaching </li></ul><ul><li>the nature of assessment? </li></ul>
  29. 32. NAPLAN – ACTA submission <ul><li>We submit that NAPLAN assessments currently do not permit the reporting of quality information about EAL/D students’ progress or school performance to parents, principals, school communities and systems because: </li></ul><ul><li>1. the test data are not disaggregated to reveal EAL/D learners’ performance </li></ul><ul><li>2. the tests are inaccurate, inconsistent and lack validity for EAL/D learners </li></ul><ul><li>3. the focus on English literacy fails to provide a basis for innovation and quality teaching of EAL/D learners, and is narrowing and distorting the teaching EAL/D learners receive. </li></ul>
  30. 33. <ul><li>We recognise that accurate, quality data is essential to implementing National Partnership agreements and the ongoing development of effective policies and programs. </li></ul><ul><li>1. Without data on how EAL/D learners are performing on NAPLAN tests, it is difficult to sustain claims for provision that is appropriate to their educational needs and, at the national level, impossible to monitor the effectiveness of any such provision. </li></ul><ul><li>2. Failure to identify these learners within the NAPLAN reporting regime is having adverse flow-on effects for their educational experiences and outcomes. </li></ul>
  31. 34. <ul><li>Identifying EAL/D learning needs requires reputable, specific-purpose EAL/D assessment tools and frameworks that are appropriate for use in Australian schools. EAL/D assessment frameworks now exist in all states, territories and systems, and can be used with confidence as a basis for the immediate disaggregation of EAL/D learners’ NAPLAN results. In fact, Australia has led the world in developing assessment methodologies for EAL/D learners in schools, and some have been used as models internationally. </li></ul>
  32. 35. US: No Child Left Behind <ul><li>Strict, controversial measures of managing educational system through tests </li></ul><ul><li>EAL/D achievement measured separately </li></ul><ul><li>Accommodations allowed in content area assessments </li></ul><ul><ul><li>translation </li></ul></ul><ul><ul><li>dictionaries </li></ul></ul><ul><ul><li>longer time </li></ul></ul>
  33. 36. New EAL/D frameworks for NCLB (Bailey & Huang 2011; Llosa 2011) <ul><li>WIDA Consortium: </li></ul><ul><li>separate strands for English in relation to content areas: </li></ul><ul><ul><li>language arts </li></ul></ul><ul><ul><li>mathematics </li></ul></ul><ul><ul><li>science </li></ul></ul><ul><ul><li>social studies </li></ul></ul>