My Qs: Stemming from our Readings and from my interviewsArticle Qs: After all, they are both concerned with the development of language ability, and its measurements, prima facie.
Here is also where we start to see limitations in their approach…remember their research question, and their approach becomes….
Diagnostic testing can provide useful information for placement or even proficiency purposes, the authors claim, but the reverse is certainly not true. So diagnostic tests have been misused often for language placements. Theyhttp://www.lancs.ac.uk/researchenterprise/dialang/about
Language Orthography: alphabetic or non-alphabetic
Please see your handouts for full details on the methodology used
I’m not crazy about tests, but I do want to know how to assess and prepare my students should they ever need to take these tests and how they can improve their quality and choices in education
Article presentation fall 2011
SLA RESEARCH,READING, AND TESTING WITH ALEX TEMPLETON
OVERVIEW• Leading Questions J. Charles Alderson, Ph.D. Lancaster University. Professor of• Summary/Foci Linguistics and English Language• Study Education. Center for Research in Language Education (link)• Methods Ari Huhta, Lic.Ph. (Ph.D. in Pedagogy)• Results University of Jyväskylä. Researcher in• Limitations Language Assessment. Center for Applied Linguistics (link)• Implications• Relevance Can research into the diagnostic testing of reading in a second or• Concerns (So what?) foreign language contribute to SLA research? EUROSLA Yearbook 11 (2011). John Benjamins Publishing Company. (link)
LEADING QUESTIONSMy Questions Article Questions• Why is English generally • How can SLA theory taught in Asian countries towards performing on and language testing tests? contribute to each• Why are Hispanic ELL’s other? (among many others) in America not generally • Can research into the taught English for tests? diagnostic testing of• Are those tests even reading in a S/FL congruent with what is contribute to SLA taught internationally?• Are those tests based on research? SLA theory and/or practice?
ARTICLE SUMMARY• Hypotheses• In theory, SLA should: • Offer insights into the construct of reading in S/FL • Testing should be able to base diagnostic tests of S/FL READING on those insights• In practice, SLA does not clearly do this, so: • Examine the potential synergy between SLA and FL testing • Report on progress of 3 interrelated projects into S/FL Reading
TheoryFOCUS: TESTINGTo better understandlanguage developmentin specific skill areas to: Design PedagogyDevelop suitable andtruly diagnostic tests;To identify and defineconstructs in SLA forbasis test design. Testing Assessment
FOCUS: TESTING •Readiness for •To place learnerProblems with given target situation w/o at suitable level within a givenmisuse of curriculum ref. language curriculumdiagnostictests:No Proficiency Placement Testing Testingindividualizedfeedback onproblem areasDo not addressfull range of Progress Achievement Testing TestingpotentialOr causes of •Assess how much •Assess how muchproblems in a student has a student has learned during or learned during orlearning at the end of a at the end of a given curriculum given curriculumlanguage
FOCUS: READING PROFICIENCY• Is reading in a foreign language a reading problem or a language problem? Motivation Age• Authors say the Language orthography Cognitive variables evidence suggests it as a language Background Vocabulary knowledge Size problem Transfer of• Authors claim L1- Nature of reading Syntax text from L1 to the S/FL reading-theory ignores this
FOCUS: CLAIMS• Authors claim little is known about: • How S/FL reading develops • How to identify strengths and weaknesses • Which abilities contribute most to the development of overall S/FL reading performance • How teachers can best facilitate reading abilities
METHODOLOGY: PREPARATION• Author’s Lit. review on SLA in reading:• Baddeley and de Jong’s (2006) Executive Control model (p. 34) [Cognitive-information processing]• Alderson’s (2000) Situation Model of Reader Interpretation (p. 35)[Cognitive-connectionism]• Just and Carpenter’s (1992) Capacity Constrained Reader Model (p. 36) [Cognitive-ACT]• Stanovich’s (2000) Interactive Compensatory Model (p. 36) [Input-interaction]• Chen and Vellutino’s (1997) Simple View of Reading (p. 36) and Carver’s (2000) Rauding Model (pp. 36-37) [Input-Interaction]• REFUTE Goodman’s (1967,1996) Psycholinguistic Guessing Game (p. 37)
METHODOLOGY: STUDY• Three interrelated research projects analyzing three instruments, all on going, and all being compared to each other: • Project 1 Test- Reading in the language instruction in PISA (Programme for International Student Assessment) • Project 2 Test – Reading in L2 in the Pearson Tests of English Academic and General • Project 1 & 2 addressed the same research questions with the same test item analyses methodologies• Project 3 Test – Reading and Writing in S/FL in Finland (DIALUKI) • Studied reading and possible causes of reading problems in L1 and S/FL, cross-sectional and longitudinal, with three different age groups of S/FL learners.
PROJECT 1 STUDY• Project 1- Reading and Language Instruction in PISA investigates reading in the language of instruction – usually L1 (achievement)• Asked, “what makes reading items in the PISA tests difficult”? (p. 40) • 1) To investigate the project in itself for validity • 2) 10 variables as potential predictors of item or task difficulty • 3) Survey of a test of 100 items given to experts • 4) Participants – 4 expert evaluators ranked each test item on a liker scale according to variables • 5) Ratings of variables refined over a number of sessions • 6) Ratings of variables were compared to item difficulty values • 7) Ratings discussed, ratings of items adjusted in meetings
PROJECT 2 STUDY• Project 2 Test – Reading in L2 in the Pearson Tests of English Academic and General investigates reading in a S/FL (proficiency)• Like TOEFL, designed to identify those students who are likely to be able to study successfully in English- medium Universities • Same research questions as Project 1 • Same methods used in Project 1 • 5 experts • Bayesian LLTM+e analysis • Eye-tracking equipment!
PROJECT 3 STUDY• DIALUKI – Finnish means “Diagnosing reading and writing” in a second or foreign language (p. 45) • Can diagnostic measures of L1 reading and writing predict difficulties in S/FL reading and writing? • How does S/FL proficiency in reading and writing develop in psycholinguistic and linguistic terms? • Which features or combinations of features characterize different CEFR (Common European Framework for Languages) proficiency levels?• Language Testers, applied linguists, and psychologists• 3 phases over 2 years (2010-2012)• Main groups were Finnish learners of English, and immigrants learning Finnish in Finland.
PROJECT 3 (CONT.)• All three [phases] aim to develop diagnostic tools and procedures for second and foreign language reading and writing that: • can assess the strengths and weaknesses in learners current proficiency and, • make valid predictions about the rate, quality, sequence and ultimate level of attainment of their learning.• Phase 1 (2010 to 2011) – to select the best predictors for further study• Phase 2 (2011) – examines reading training in reading- related skills through computerized learning games (used for treating dyslexia)• Phase 3 (2011 to 2012) – development of literacy skills and the relationship to diagnostic measures
RESULTSSLA Researchers Test Developers• Driven by theory and • Driven by measurement and pedagogy that is vastly performance evolving • Interested in assessing stable• Need to improve designing language ability tests with validity and • Performance across test reliability measures task facets would benefit• More research on variability SLA of S/FL performance in • Diagnostics in S/FL reading assessment measures could widen scope of SLA• Refer to reading and research comprehension processes in • Hope to set agenda for SLA language testing lit. research into S/FL reading, offering set research Q’s and methods
LIMITATIONS• Diagnosis & diagnostic testing definition?• Small =N for Projects 1 & 2• Regional• Participant experts• Research design Bias • 1st author was principle investigator of all three projects, 2nd author for Project 3• Arguably, Project 3 says more about cognitive constructs than it does about S/FL reading constructs (p. 49)
LIMITATIONS• Claim pedagogy & theory change in “week’s time”• (Hypocrisy) • Advocate synergy: reactive, not proactive • “We have not incorporated specific aptitude measures into research to date" (pg. 50).• Respective progress in both fields, please.
RELEVANCE TO TEXT• Mitchell and Myles (2004) agree that there is more empirical evidence needed about transfer and interlanguage development.• Alderson and Huhta (2011) address a possible “threshold of S/FLA proficiency found beyond reading ability transferable from the L1 to reading in the L2, and that this varies with the relevance of S/FL reader’s background knowledge, the nature of the text, and so on” (p. 38).• Alderson and Huhta (2011) are hopeful that synergies of theory-driven testing will emerge out of their research question (p. 49).• Alderson and Huhta (2011) and Mitchell and Myles, all agree about the complexity that the author’s present about language learning, “it is very difficult to predict in SLA what makes people learn faster and better than others, and involves other factors and variables”, (p. 38).
CONCERNS (SO WHAT?)• Researchers, test-makers, and practitioners all need to communicate via congruent and agreeable theorem• Communicative trends in pedagogy require communicative testing assessment• Tests can provide evidence for program/dept. support for you, your students, and your colleagues• More support provides more accessible testing materials to launch ESL/EFL students into workforce and colleges• Theory always connects to pedagogy, and we should always be able to articulate it to those who will be testing our students• Think about where you want to teach, how much will testing guide or control your pedagogy?• Don’t repeat the flaws of standardized testing in public American education• Know how to properly assess students based on supported theory, if you don’t like the tests that are out there!