Tblt Presentation

7,220 views

Published on

Published in: Technology, Education
0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
7,220
On SlideShare
0
From Embeds
0
Number of Embeds
22
Actions
Shares
0
Downloads
269
Comments
0
Likes
10
Embeds 0
No embeds

No notes for slide

Tblt Presentation

  1. 1. John M. Norris University of Hawai´i Task-Based Language Teaching 2005
  2. 2. This Powerpoint presentation is copyrighted by John M. Norris © September 23, 2005 Citation information: Norris, J. M. (2005, September). The essential roles of assessment, measurement, and evaluation in task-based language teaching . Plenary address presented at the First International Conference on Task-Based Language Teaching, University of Leuven, Belgium (September 23, 2005).
  3. 3. 0:00 9:00 3:00 6:00 12:00 21:00 15:00 18:00 What time is it in Leuven ? What time is it in Hawaii ? “ Here and Now” condition “ There and… ” condition Now
  4. 4. 0:00 9:00 3:00 6:00 12:00 21:00 15:00 18:00 What is the time difference between Leuven & Hawaii? <ul><li>Leuven </li></ul><ul><li>Hawaii </li></ul><ul><li>= 12hrs </li></ul>Task condition for presentation = “ Here _and There_ and Now”
  5. 5. measurement, The essential roles of assessment, and evaluation in task-based language teaching
  6. 6. TBLT 2005: What’s at stake? ??? Language Education ??? Does it work? for whom, when, why? What’s it worth? to whom, when, why? How does it work? for whom, when, why? Can it be improved? by whom, when, why?
  7. 7. TBLT 2005: What’s at stake? … another L2 teaching method bandwagon… The critics say… … cognitive-interactionist SLA hegemony… … won’t work with _____ learners… … not possible in _____ program settings… … too much task, not enough language… … not happening in practice… T B L T
  8. 8. Setting the record straight: TBLT at work TARGET LANGUAGES English Dutch French Czech Chinese Arabic Ukranian Japanese Russian Spanish Korean German
  9. 9. Setting the record straight: TBLT at work PROGRAM TYPES Second Language Foreign Language Vocational Education General Education Local Innovation Small Classes National Ed. Policy Special Purposes Adult Child University Requirement Large Classes Beginning Learners Advanced Learners Face-to- Face Online
  10. 10. Setting the record straight: TBLT foundations “ All genuine education terminates in discipline, but it proceeds by engaging the mind in activities worthwhile for their own sake.” Dewey (1933), pp. 86-87
  11. 11. Setting the record straight: TBLT foundations “ This discussion rejects the doctrine that students should first learn passively, and then, having learned, should apply knowledge. […] In fact, the applications are part of the knowledge. For the very meaning of things known is wrapped up in their relationships beyond themselves. Thus, unapplied knowledge is knowledge shorn of its meaning.” Whitehead, 1947, pp. 218-219
  12. 12. Setting the record straight: TBLT foundations “ For starters, this means having students do tasks, or at least meaningful simulations, that experts do in the various disciplines. Second, it means teaching them to think in ways that experts do when they perform these tasks.” Sternberg (2003), p. 5
  13. 13. Setting the record straight: Other task-based domains Medical school Nursing Curriculum by design Successful intelligence Environmental studies Social work Task-Based Practice Music education
  14. 14. Setting the record straight: So what is TBLT? Task-Based Language Teaching Rationales and Principles Education Philosophy Psychology SLA Language teaching Sociopolitical values L2 Education Programs Learner needs Assessment Curriculum Instruction Teacher development Evaluation Materials
  15. 15. The potential of task-based practice: One example William Reid Reid. W. J. (1992). Task strategies: An empirical approach to social work practice . New York: Columbia University Press. Social work problems : truancy, aging, delinquency, substance abuse, family health
  16. 16. Task-Centered Practice Empirical Justification Empirical Implementation Theory Learning, behavior, cognition Research Psychosocial change, Problem solving Evaluation Case work practice, Intervention outcomes Needs Identify, agree, prioritize Action Plan, rehearse, attempt, feedback Assessment Task accomplishment, Problem change Development Practitioner-friendly methods Improvement Field-tested findings The potential of task-based practice: One example
  17. 17. The potential of task-based practice: One example “ Assessment, process, and outcome data are systematically collected in each case by means of structured recording guidelines. Continuing research is used to improve the model.” (Reid, 1992, p. 132). Development Improvement Field-testing
  18. 18. “ [Reid] was the single most significant figure in social work research” Shaw (2004) p. 113 The potential of task-based practice: One example
  19. 19. Task-Based Language Teaching: A(nother) method for language teaching. The potential of task-based practice: TBLT
  20. 20. Task-Based Language Teaching: A(nother) method for language education practice. The potential of task-based practice: TBLT
  21. 21. Task-Based Language Teaching: An empirical approach to language education practice. The potential of task-based practice: TBLT Assessment Measurement Evaluation
  22. 22. The Role of Measurement in TBLT Interpreting research constructs… … to understand task-based learning
  23. 23. Measurement in TBLT: Responding to research purposes What is measurement? … trustworthy definition, elicitation, observation, analysis, and interpretation of research constructs…
  24. 24. Measurement in TBLT: Responding to research purposes Measurement should… Illuminate task-based learning… L2 dvlpmt. in task, over tasks Noticing, integrating, automatizing Proficiency, motivation, aptitude Timing, sequencing, embedding TBI FonF, modeling, feedback Reliable reporting of TBL variables Synthesis of how TBL works <ul><li>Define clear learning outcomes </li></ul><ul><li>Identify specific learning processes </li></ul><ul><li>Clarify learner individual differences </li></ul><ul><li>Indicate effects of context features </li></ul><ul><li>Articulate instructional processes </li></ul><ul><li>Enable communication of findings </li></ul><ul><li>Foster accumulation of knowledge </li></ul>
  25. 25. Measurement in TBLT: Responding to research purposes “ Perhaps the best recommendation would be that, with research in this area still developing techniques and understandings, the role of A/L work will be to offer suggestions for relevant variables and the standardization of operationalizations of variables…” Tavakoli & Skehan, 2005, p. 273
  26. 26. In short… TBLT <ul><li>Theory/research </li></ul><ul><li>Constructs </li></ul><ul><li>Learner variables </li></ul><ul><li>Setting variables </li></ul><ul><li>Learning process </li></ul><ul><li>Instruction process </li></ul><ul><li>Learning outcomes </li></ul>Practice Empirical Measurement Measurement in TBLT: Responding to research purposes How does TBLT work, for whom, when, why?
  27. 27. Measurement in TBLT: Problems of practice Interpreting research constructs No construct definition Little standardization Missing measures No attention to reliability Impoverished analyses Insufficient reporting Understanding task-based learning <ul><li>Inaccurate interpretation of </li></ul><ul><li>task-related variables </li></ul><ul><li>2. Massive over-generalization </li></ul><ul><li>of measurement ‘findings’ </li></ul><ul><li>3. Minimal meaning for </li></ul><ul><li>scope, scale, setting of </li></ul><ul><li>task-based practice </li></ul>
  28. 28. Task influence on L2 performance (adapted from: Skehan, 2001) Measurement in TBLT: Problems of practice 0 + 0 Transformations (planning) 0 + 0 Outcomes + 0 + Structure - + + Dialogic v. Monologic + 0 + Familiarity Fluency Complexity Accuracy Task characteristic
  29. 29. Task influence on L2 performance (adapted from: Skehan, 2001) Measurement in TBLT: Problems of practice 0 + 0 Transformations (planning) 0 + 0 Outcomes + 0 + Structure - + + Dialogic v. Monologic + 0 + Familiarity Fluency Complexity Accuracy Task characteristic
  30. 30. “… giving learners the opportunity to plan before a task is done consistently produces greater complexity of language…” (Skehan, 2003, pp. 395-96) Measurement in TBLT: Problems of practice “ To sum up, both modality and group condition affected complexity at the level of syntax…” (Ellis & Yuan, 2005, p. 184)
  31. 31. Measurement in TBLT: Problems of practice Task condition Planning L2 performance outcome Complexification Modality? writing/speaking Complexity? Subordination Complexity? Other measures Scope of measure? duration/timing Planning? Focus, structure, pressure Learner proficiency? “ L-H”, “N-I-A”, “450-600” Task features? Type, difficulty, form, mono-dialogic, interlocutors ?
  32. 32. Measurement in TBLT: Problems of practice Task condition + strategic planning L2 performance outcome Complexification Complexity? Clauses per C-Unit Learner proficiency? TOEFL: 427-670 Elder & Iwashita (2005) X X X
  33. 33. Measurement in TBLT: Problems of practice Task condition + strategic planning L2 performance outcome Complexification Complexity? Words/T-Unit Clauses per T-Unit Total Subordination Learner proficiency? TOEFL: 420-480 TOEFL: 510-580 TOEFL: 560-610 Kawauichi (2005)
  34. 34. Measurement in TBLT: An empirical approach to practice In what ways are what kinds of planning associated with what kinds of L2 performance for what kinds of L2 users under what conditions…and why?
  35. 35. Measurement in TBLT: An empirical approach to practice “ ... descriptive longitudinal studies which establish norms of performance for particular processes and phenomena in L2 acquisition…” Norris & Ortega, 2003, p. 738 “… attempting to ‘measure’ acquisition without a sound descriptive basis for meaningful differences in particular acquisition-related behaviors would be akin to timing a runner’s performance over a mile without knowing how many times around the track a mile happens to be…”
  36. 36. Measurement in TBLT: An empirical approach to practice Georgetown University German Department: “ Syntactic complexity development in task-based writing” Longitudinal (23) + Cross-sectional (329) Extended and extemporaneous writing tasks Multiple measures, including Multiple measures of SC End-of-curricular levels Across 4 years
  37. 37. Syntactic complexity in curriculum-embedded writing assessment: LONGITUDINAL Global compexification   Phrasal elaboration Subordination 
  38. 38. Syntactic complexity in curriculum-embedded writing assessment: CROSS-SECTIONAL Global compexification  Subordination   Phrasal elaboration
  39. 39. Measurement in TBLT: An empirical approach to practice Implications for measuring syntactic complexity in task-based research <ul><li>Syntactic complexity is multi-faceted; so, measures are multivariate </li></ul><ul><li>Complexification is a long-term developmental process, best understood in situ </li></ul><ul><li>At different learner proficiency/curricular levels, complexification occurs in distinct ways </li></ul>Synthesis of research findings on syntactic complexity See: Ortega (2000, 2003)
  40. 40. In short… TBLT in situ <ul><li>Theory/research </li></ul><ul><li>Constructs </li></ul><ul><li>Learner variables </li></ul><ul><li>Setting variables </li></ul><ul><li>Learning process </li></ul><ul><li>Instruction process </li></ul><ul><li>Learning outcomes </li></ul>Practice Empirical Measurement Measurement in TBLT: Responding to research purposes How does TBLT work, for whom, when, why?
  41. 41. The Role of Assessment in TBLT Informing decisions and actions… … to enable task-based learning
  42. 42. Assessment in TBLT: Responding to educative purposes What is assessment? … the use of tests and other procedures for gathering trustworthy information about learners and making decisions and taking actions on the basis of that information, in support of teaching and learning…
  43. 43. Assessment in TBLT: Responding to educative purposes <ul><li>Ensure learner-relevant instruction </li></ul><ul><li>Support classroom teaching </li></ul><ul><li>Motivate and focus learners </li></ul><ul><li>Target clear learning outcomes </li></ul><ul><li>Identify learning gaps </li></ul><ul><li>Transform curriculum </li></ul><ul><li>Counter imposed testing practice </li></ul><ul><li>Foster range of learning outcomes </li></ul><ul><li>Certify valued accomplishments </li></ul>Educative assessment should… Enable task-based learning… Needs analysis, intake procedures Feedback on TB pedagogy effects Interesting, meaningful tests Genre, task performance expectation Feedback on task performance Process-product articulation Internal TBA v. external mandate Language + content + task abilities Profile individual abilities in L2 use
  44. 44. Assessment in TBLT: Responding to educative purposes “ In other words, by focusing on task-based assessment—here writing—faculty faced their own assumptions about long-term development in instructed L2 learning and appropriate learning outcomes at particular stages in that sequence.” Byrnes (2002), p. 428
  45. 45. Assessment in TBLT: Responding to educative purposes “… the quality of task completion gave teachers a good understanding of what students did and did not know—often, a better understanding than most tests could provide.” Leaver & Kaplan (2004), p. 52
  46. 46. Assessment in TBLT: Responding to educative purposes “ Focusing on what one can do instead of the competencies one has yields a more direct link with language performance that can be observed in the real world.” Gysen & van Avermaet (2005), p. 51
  47. 47. Assessment in TBLT: Responding to educative purposes In short… TBLT <ul><li>Implementation </li></ul><ul><li>Curriculum </li></ul><ul><li>Instruction </li></ul><ul><li>Materials </li></ul><ul><li>Learners </li></ul><ul><li>Teachers </li></ul><ul><li>Outcomes </li></ul><ul><li>ETC. </li></ul>Practice Empirical Assessment Does TBLT work, and how can we enable it?
  48. 48. Assessment in TBLT: Problems of practice Tensions in “how to” assess… L2 competence v. task ability Extrapolation v. prediction Construct v. consequence Control v. understand Prescribe v. respond Enabling task-based learning <ul><li>Irrelevant for many educative </li></ul><ul><li>uses in classrooms/programs </li></ul><ul><li>Ignorant of interconnections </li></ul><ul><li>between language and task/ </li></ul><ul><li>genre/content </li></ul><ul><li>Irresponsible to alignment of </li></ul><ul><li>assessment with C&I </li></ul><ul><li>Incompatible with values and </li></ul><ul><li>goals of task-based education </li></ul>Dichotomy v. multiplicity
  49. 49. Assessment in TBLT: Problems of practice “… the inferences we want to make are about underlying ‘language ability’ or ‘capacity for language use’ or ‘ability for use’…” Bachman (2002), p. 454 “… assessments can have many different functions. What is appropriate for one assessment purpose may be inappropriate for another…” Council of Europe (2001), p. 180
  50. 50. Advanced intercultural evaluation: An online French course Starkey & Osler (2001) Common European Framework: Language+ Pluriculturalism, Anti-racism <ul><li>Online advanced French </li></ul><ul><li>learning objectives : </li></ul><ul><li>Language development </li></ul><ul><li>Social/cultural awareness </li></ul><ul><li>Intercultural evaluation </li></ul><ul><li>Online advanced French </li></ul><ul><li>teacher assessments </li></ul><ul><li>(writing feedback): </li></ul><ul><li>Language development </li></ul><ul><li>Social/cultural awareness </li></ul><ul><li>Intercultural evaluation </li></ul>Assessment in TBLT: Problems of practice
  51. 51. Assessment in TBLT: Problems of practice “… is not for the test designer to decide or prescribe. Rather, it should be the test users and stakeholders who inform exactly what needs to be done in the name of language assessment, and it is up to us to enable them to do so.” Norris (2002), p. 344
  52. 52. “ 1. The intended use(s) for task-based assessment within the language programme must be specified…” Long & Norris (2000), p. 600 Assessment in TBLT: An empirical approach to practice
  53. 53. Specifying intended assessment use From: Norris (2000)
  54. 54. Assessment in TBLT: Examples in practice, Specification process <ul><li>Specifying intended uses for assessment in the Georgetown University German Department </li></ul><ul><li>Clarifying the purposes for assessment, from classroom- to program-level, by… </li></ul><ul><li>Investigating existing assessment practices (via interviews, surveys, observations) </li></ul><ul><li>Establishing consensus on the roles to be played by assessment (via meetings, workshops, negotiation among intended users/stakeholders) </li></ul>
  55. 55. Assessment in TBLT: Examples in practice, Specification products Intended assessment use specification Assessment policies Use specs for all tests Priorities for Development Assessment Constraints Quizzes Exams Performance Internal External Time Money Ethics Training Unknowns (Urgent!) Placement Learning outcomes External Accountability Developing Implementing Articulating Communicating Evaluating
  56. 56. Assessment in TBLT: Examples in practice, Use specification Intended use Who : Teachers, curriculum developers, researchers What : Learner written task/genre performance abilities at the end of each curricular level Why : Understanding student achievement of targeted task abilities for improvement of C&I Impact : Feasible (revised?) curricular expectations supported by effective (improved?) pedagogy <ul><li>Constraints </li></ul><ul><li>Explicitness of curricular expectations </li></ul><ul><li>Availability/agreement of ‘prototypical’ performance tasks by curricular level </li></ul><ul><li>Competing uses for assessment (feedback) </li></ul><ul><li>End-of-semester timing </li></ul><ul><li>Learner investment, understanding of task expectations </li></ul><ul><li>Task versus content </li></ul>Task-based writing assessment
  57. 57. Task Content L conventions L focus Assignment 1 Assignment 2 Process writing 4-6 tasks per semester Assignment 3 Assignment 4 Prototypical Performance Writing Task: Level I Level II Level III Level IV Curricular level/genre learning expectations Feedback loop Explicit criteria Assessment in TBLT: Examples in practice, Instruments
  58. 58. Assessment in TBLT: Examples in practice, Impact <ul><li>Curriculum : much greater specificity in learning objectives and articulation, overall and for each level/semester </li></ul><ul><li>Courses : revised/improved pedagogic focus, consistency </li></ul><ul><li>Teachers : greater awareness of learner abilities and learning trajectories, enhanced clarity of grading/feedback purpose </li></ul><ul><li>Learners : enhanced understanding of learning targets & expectations, supportive role of assessment </li></ul><ul><li>Assessment : clearer articulation with C&I, development of similar assessment systems (e.g., speaking tasks) </li></ul><ul><li>Program : explicit statement of valued learning outcomes in the form of genres, tasks, content, and language </li></ul>
  59. 59. Assessment in TBLT: Responding to educative purposes In short… TBLT in situ <ul><li>Implementation </li></ul><ul><li>Curriculum </li></ul><ul><li>Instruction </li></ul><ul><li>Materials </li></ul><ul><li>Learners </li></ul><ul><li>Teachers </li></ul><ul><li>Outcomes </li></ul><ul><li>ETC. </li></ul>Practice Empirical Assessment Does TBLT work, and how can we enable it?
  60. 60. The Role of Evaluation in TBLT Articulating empirical practice… … to improve task-based education … with practical value
  61. 61. Evaluation in TBLT: Responding to program purposes What is evaluation? … gathering trustworthy information about program elements in order to understand, demonstrate, and improve program value…
  62. 62. In short… TBLT in situ <ul><li>Society </li></ul><ul><li>Values </li></ul><ul><li>Needs </li></ul><ul><li>Resources </li></ul><ul><li>Stakeholders </li></ul><ul><li>Funders </li></ul><ul><li>Educators </li></ul><ul><li>ETC. </li></ul>Practice Empirical Evaluation What is TBLT worth, and how can we improve it? Evaluation in TBLT: Responding to program purposes
  63. 63. Evaluation in TBLT: Responding to program purposes “ Given the checkered history of prescriptions for language teaching, the likelihood that all 10 TBLT MPs will turn out to have been well founded is minimal. Thus, designers … who draw upon these or other sources should build careful evaluation components into their delivery systems.” Doughty & Long (2003), p. 68 “ In contrast to the extensive research into tasks…there have been no more than a handful of evaluations of task-based teaching” Ellis (2003), p. 323
  64. 64. Evaluation in TBLT: Problems of practice Nunan (1989) Crookes & Gass (1993) Willis (1996) Skehan (1998) Lee (2000) Bygate, Skehan, & Swain (2001) Ellis (2003) Leaver & Willis (2004) Nunan (2005) Overview books on TBLT… … with chapters on program evaluation… Ellis (2003)
  65. 65. Evaluation in TBLT: Problems of practice <ul><li>Bangalore Project </li></ul><ul><li>Evaluation: </li></ul><ul><li>Task-base program </li></ul><ul><li>Large-scale effort </li></ul><ul><li>External evaluator </li></ul>(see Beretta, 1990a, 1990b; Beretta & Davies, 1985) Crisis of conscience Evaluation practice? Evaluation purpose? Evaluation impact? Theory v. practice???
  66. 66. Evaluation in TBLT: An empirical approach to practice Patton (1996) Utilization-focused Evaluation Intended uses Intended users Questions + Methods Understand Improve Educate Demonstrate worth Hold accountable Test theory Empower Teachers Administrators Curriculum writers Learners Parents/public Researchers Funders Values clarification Implementation Process-product Experimentation Synthesis
  67. 67. Evaluation in TBLT: Examples in practice Norris (2004) Evaluation of a US college German FL assessment program Van den Branden (to appear) Evaluation of task-based teacher development in Flanders
  68. 68. Assessment, measurement, and Evaluation in TBLT: Essential roles TBLT in situ Measurement Theory, Research Assessment Imple-mentation Evaluation Society, Value EMPIRICAL PRACTICE
  69. 69. http://www2.hawaii.edu/~jnorris [email_address] See link at: www.tblt.org For more information: Contact me
  70. 70. See you in Hawaii for TBLT 2007!
  71. 71. http://www2.hawaii.edu/~jnorris [email_address] See link at: www.tblt.org
  72. 72. REFERENCES Please see the following web site for a complete pdf file of references and resources from this presentation: http://www2.hawaii.edu/~jnorris/TBLT%20references.pdf

×