Timothy Curnow And Anthony Liddicoat 2008

825 views

Published on

Session B - H6-03

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
825
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Timothy Curnow And Anthony Liddicoat 2008

  1. 1. ASSESSMENT AS LEARNING: ENGAGING STUDENTS IN ACADEMIC LITERACY IN THEIR FIRST SEMESTER Timothy Jowan Curnow & Anthony J. Liddicoat Research Centre for Languages and Cultures University of South Australia
  2. 2. REDESIGNING ASSESSMENT <ul><li>Redesign applied linguistics courses, including the assessment </li></ul><ul><li>Integrate academic literacy of native English speakers and others into teaching, learning and assessment </li></ul><ul><li>Focus on academic literacy doesn’t entail a lack of focus on disciplinary content; the two can be integrated </li></ul>
  3. 3. A MODEL OF THE ASSESSMENT PROCESS Source, Scarino, 2006) Conceptualising Judging Validating Eliciting What to assess How to judge How to justify How to elicit The “ability” or “knowing” of interest Criteria for judging performance, including 1. aspects of competence 2. performance analysis Validation Tasks/ procedures that operationalise the construct
  4. 4. ASSESSMENT IN PRACTICE <ul><li>Most thought is given to elicitation, very little to anything else. </li></ul><ul><ul><li>In terms of this model, many assessment codes of practice deal only with elicitation (and only with the amount of elicitation) </li></ul></ul><ul><ul><li>Note that a lack of clarity about the conceptualisation of what is being assessed has a direct impact on all other assessment processes </li></ul></ul>
  5. 5. RETHINKING ASSESSMENT IN THE APPLIED LINGUISTICS MAJOR <ul><li>Design of an overarching assessment approach for all core courses in applied linguistics </li></ul><ul><li>Process: </li></ul><ul><ul><li>Conceptualising what learnings we wanted to achieve across the program </li></ul></ul><ul><ul><li>Conceptualising how these interrelated </li></ul></ul><ul><ul><li>Conceptualising what tasks would: </li></ul></ul><ul><ul><ul><li>allow those learnings to be put into practice </li></ul></ul></ul><ul><ul><ul><li>give evidence of nature of students’ learning </li></ul></ul></ul><ul><ul><ul><li>develop students knowledge of the discipline as well as demonstrating their ability to use this knowledge </li></ul></ul></ul><ul><ul><li>Designing tasks (eliciting) </li></ul></ul><ul><ul><li>Designing assessment criteria (judging) </li></ul></ul>
  6. 6. WHAT LEARNING DID WE WANT TO SEE? <ul><li>Not simply “body of knowledge” but creation/use of knowledge in different ways. E.g. </li></ul><ul><ul><li>acquiring knowledge from text </li></ul></ul><ul><ul><li>linking argument and evidence </li></ul></ul><ul><ul><li>evaluating information according to purpose </li></ul></ul><ul><ul><li>…… .. </li></ul></ul><ul><li>Focus on transformation of knowledge not of reproduction of knowledge. E.g. </li></ul><ul><ul><li>using knowledge for new purposes, not simply recall </li></ul></ul><ul><ul><li>making connections between different texts, experiences, etc. </li></ul></ul><ul><ul><li>…… </li></ul></ul>
  7. 7. WHAT LEARNING DID WE WANT TO SEE? <ul><li>Key dimensions determined by team as being </li></ul><ul><ul><li>Critical reading of research </li></ul></ul><ul><ul><li>Analysis of research writing </li></ul></ul><ul><ul><li>Synthesis of research from multiple sources </li></ul></ul><ul><ul><li>Constructing an argument using the research of others </li></ul></ul><ul><ul><li>Analysing language data </li></ul></ul><ul><ul><li>Constructing an argument from language examples </li></ul></ul><ul><ul><li>Understanding the process of research development </li></ul></ul><ul><ul><li>Designing and implementing research projects </li></ul></ul><ul><ul><li>Communicating research findings </li></ul></ul><ul><li>Divided between courses in the major </li></ul><ul><ul><li>1-4: LANG 1056 </li></ul></ul><ul><ul><li>5-6: LANG 1055 </li></ul></ul><ul><ul><li>7-9: Later courses </li></ul></ul>
  8. 8. HOW DID THIS AFFECT OUR APPROACH TO ASSESSMENT? <ul><li>Realisation that some assessment tasks did not have a clear link either to knowledge or its use, for example: </li></ul><ul><ul><li>Reading tasks </li></ul></ul><ul><ul><ul><li>Purpose: “get the buggers to read” </li></ul></ul></ul><ul><ul><ul><ul><li>Reading had no purpose other than having been completed </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Questions mainly showed comprehension of the text (i.e. they confirmed the text had been read) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>The tasks had no connection to each other or to larger themes </li></ul></ul></ul></ul><ul><ul><li>Oral presentations on texts </li></ul></ul><ul><ul><ul><li>Purpose: ??? (something oral might be good? Task “inherited”) </li></ul></ul></ul><ul><ul><ul><ul><li>Presentations were boring, for tutors and for the students – simple rehearsals of information </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Students had nothing to say about the texts because they had no reason to say anything </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Students had nothing to say about the texts because they had no framework in which to develop any ideas about it </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Very hard to mark (criteria when you don’t have a purpose?) </li></ul></ul></ul></ul>
  9. 9. HOW DID THIS AFFECT OUR APPROACH TO ASSESSMENT? <ul><li>New assessment plans have </li></ul><ul><ul><li>a reduced diversity of task types, at least for first year courses </li></ul></ul><ul><ul><li>greater coherence between individual assessment tasks </li></ul></ul><ul><ul><ul><li>cumulative development across tasks </li></ul></ul></ul><ul><ul><ul><li>development towards a clear end point, with each task contributing to the end point </li></ul></ul></ul><ul><ul><li>clear links between teaching/learning needs in class and assessment tasks </li></ul></ul>
  10. 10. ASSESSMENT IN LANG 1056 ‘LANGUAGE AND CULTURE’ <ul><li>Developmental focus on essay writing and reading/using research writing </li></ul><ul><li>Academic literacy focus combined with content focus </li></ul><ul><li>Discrete literacy focus for each assessment task </li></ul><ul><li>Cumulative learning developed through assessment </li></ul>
  11. 11. ASSESSMENT IN LANG 1056 <ul><li>Critical reading task </li></ul><ul><ul><li>guided reading task based on a single set research article </li></ul></ul><ul><ul><li>emphasis on identifying and analysing arguments and evidence </li></ul></ul><ul><ul><li>emphasis on abstracting information across components of the research article </li></ul></ul>
  12. 12. ASSESSMENT IN LANG 1056 <ul><li>Analytic reading task </li></ul><ul><ul><li>focused essay-like response based on a single reading – using a research article to address a new question </li></ul></ul><ul><ul><li>essay topic related to reading but not about the reading – knowledge transformation </li></ul></ul><ul><ul><li>emphasis on locating and using information for new purposes </li></ul></ul><ul><ul><li>requires critical reading as starting point </li></ul></ul>
  13. 13. ANALYTIC READING <ul><li>Text: </li></ul><ul><ul><li>Wierzbicka , A. (1991) Cross-cultural pragmatics . The Hague: Mouton de Gruyter. Ch 2: “Different cultures, different languages, different speech acts” </li></ul></ul><ul><li>Task: </li></ul><ul><ul><li>Your task is to write an analytic response to Wierzbicka’s paper which addresses the following topic: </li></ul></ul><ul><ul><ul><li>The meaning of an utterance in a language is not simply the result of the meanings of the words in the utterance, but is fundamentally influenced by the cultural context in which it is uttered . </li></ul></ul></ul>
  14. 14. ASSESSMENT IN LANG 1056 <ul><li>Analytic reading task </li></ul><ul><ul><li>focused essay-like response based on a single reading – using a research article to address a new question </li></ul></ul><ul><ul><li>essay topic related to reading but not about the reading – knowledge transformation </li></ul></ul><ul><ul><li>emphasis on locating and using information for new purposes </li></ul></ul><ul><ul><li>requires critical reading as starting point </li></ul></ul>
  15. 15. ASSESSMENT IN LANG 1056 <ul><li>Synthetic reading task </li></ul><ul><ul><li>focused essay-like response based on a set of readings </li></ul></ul><ul><ul><li>essay topic related to readings – knowledge transformation </li></ul></ul><ul><ul><li>emphasis on using multiple sources to address a question </li></ul></ul><ul><ul><li>emphasis on synthesising arguments and evidence from multiple sources </li></ul></ul><ul><ul><li>requires critical reading and analytic use of reading as starting point </li></ul></ul>
  16. 16. SYNTHETIC TASK <ul><li>Texts: </li></ul><ul><ul><li>Nelson, et al. (2002). Cross-Cultural Pragmatics: Strategy Use in Egyptian Arabic and American English Refusals. Applied Linguistics 23(2):163-189. </li></ul></ul><ul><ul><li>Hickey, L. (2005). Politeness in Spain: Thanks but no “thanks”. In L. Hickey and M. Stewart (eds). Politeness in Europe. Clevedon: Multilingual Matters. </li></ul></ul><ul><ul><li>Haugh, M. (2003). Revisiting the conceptualisation of politeness in English and Japanese. Multilingua 23(1-2): 85-110. </li></ul></ul><ul><li>Task: </li></ul><ul><ul><ul><li>Using only these readings, answer the following question in a brief essay format: </li></ul></ul></ul><ul><ul><ul><ul><li>While all cultures value politeness, what it means to be polite varies from culture to culture. </li></ul></ul></ul></ul>
  17. 17. ASSESSMENT IN LANG 1056 <ul><li>Synthetic reading task </li></ul><ul><ul><li>focused essay-like response based on a set of readings </li></ul></ul><ul><ul><li>essay topic related to readings – knowledge transformation </li></ul></ul><ul><ul><li>emphasis on using multiple sources to address a question </li></ul></ul><ul><ul><li>emphasis on synthesising arguments and evidence from multiple sources </li></ul></ul><ul><ul><li>requires critical reading and analytic use of reading as starting point </li></ul></ul>
  18. 18. ASSESSMENT IN LANG 1056 <ul><li>Essay </li></ul><ul><ul><li>conventional essay task </li></ul></ul><ul><ul><li>requires critical reading and analytic and synthetic use of readings as starting point </li></ul></ul><ul><ul><li>involves locating own sources </li></ul></ul>
  19. 19. ASSESSMENT IN LANG 1055 <ul><li>Focus of course is on micro-analysis of language data </li></ul><ul><li>Academic literacy focus: </li></ul><ul><ul><li>analysing language data </li></ul></ul><ul><ul><li>constructing an argument based on language examples </li></ul></ul><ul><li>Previous assessment items included a ‘conventional’ essay, requiring finding own references </li></ul><ul><li>Research literature important and much exposure in course, but didn’t need assessment item since focus of LANG 1056 </li></ul>
  20. 20. ASSESSMENT IN LANG 1055 <ul><li>Language analysis tasks </li></ul><ul><ul><li>One task per course module </li></ul></ul><ul><ul><ul><li>semantics </li></ul></ul></ul><ul><ul><ul><li>‘ grammar’ (morphology and syntax) </li></ul></ul></ul><ul><ul><ul><li>phonetics and phonology </li></ul></ul></ul><ul><ul><li>Modelled in tutorials </li></ul></ul><ul><ul><ul><li>spot patterns in the data </li></ul></ul></ul><ul><ul><ul><li>develop rules to account for the patterns </li></ul></ul></ul><ul><ul><ul><li>(choosing between competing analyses) </li></ul></ul></ul><ul><ul><li>Assessment criteria include </li></ul></ul><ul><ul><ul><li>treating the data as the unit for analysis </li></ul></ul></ul><ul><ul><ul><li>analysis justified on the basis of the data </li></ul></ul></ul>
  21. 21. ASSESSMENT IN LANG 1055 <ul><li>‘ Data analysis and justification’ essay </li></ul><ul><ul><li>replaced previous ‘conventional’ essay </li></ul></ul><ul><ul><li>half-way through course </li></ul></ul><ul><ul><li>structure modelled on previous six or seven readings </li></ul></ul><ul><ul><li>students choose one of eight languages (they’re nearly all studying a language also, strong engagement) </li></ul></ul><ul><ul><li>given 30-40 example sentences </li></ul></ul><ul><ul><li>initial analysis, then an explicit choice, each path with advantages and disadvantages </li></ul></ul><ul><ul><li>students explicitly told there is no single right answer </li></ul></ul><ul><ul><li>need to give evidence for their particular answer, they are assessed on their justification, and how they argue that their analysis is superior to alternatives </li></ul></ul>
  22. 22. CONCLUSIONS <ul><li>Important to begin from what you want students to learn </li></ul><ul><li>‘ Whole-of-major’ approach </li></ul><ul><li>Students engaged with professional research literature </li></ul><ul><li>Academic literacy through discipline-specific content </li></ul><ul><li>Effect on students somewhat unclear, since most of these students are only now finishing their first year, but some tentative outcomes </li></ul>
  23. 23. OUTCOMES? <ul><li>Course evaluation for LANG 1056 </li></ul><ul><ul><li>Did the course make you feel more confident in dealing with reading applied linguistics research? Resounding ‘yes’. </li></ul></ul><ul><ul><li>Did the course develop your understanding of the types of writing required in linguistics? Resounding ‘yes’. </li></ul></ul><ul><ul><li>“ I came into the course with little knowledge on reading linguistics research and now I feel as if I can read research papers confidently” </li></ul></ul><ul><ul><li>“ I went back over the first reading of the course and I discovered that I could actually understand what the article was trying to say, whereas before, I had much trouble understanding” </li></ul></ul><ul><li>Students’ results </li></ul><ul><ul><li>LANG 1056 essays brought total marks up </li></ul></ul><ul><ul><li>Some LANG 1055 essays publishable in terms of structure and arguments </li></ul></ul>

×