Assessing Intercultural language learning


Published on

Assessing intercultural language learning, handout from Joe van Dalen's presentation at AFMLTA conference in Sydney 2009

Published in: Education, Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Assessing Intercultural language learning

  1. 1. ASSESSMENT OF INTERCULTURAL LEARNING Introduction to the project: In 2007 and 2008, I was part of a project conducted by the Research Centre for Languages and Cultural Education at the University of South Australia, under the leadership of Angela Scarino and Tony Liddicoat. The project was on Assessing Intercultural Learning, funded by the Australian Research Council. I was one of three practising teachers from the School of Languages in Adelaide involved in this project along with about 6 other teachers from a number of primary and secondary schools across Adelaide, representing the government, Catholic and Independent sectors. I wish to stress that any views and conclusions related by me today are not necessarily those of the researchers. They will no doubt have drawn their own conclusions. These comments are merely personal observations as a participant in that ARC project although I will draw on some thinking and considerations from the research team as we discussed the challenges and the concepts of assessment as a group throughout the two phases of the investigation. It became evident very quickly to the team and participants involved how complex this aspect of teaching and learning in the classroom actually is as far as intercultural learning is concerned. The Intercultural Language Teaching and Learning in Practice (ILTLP) Project mobilised many teachers of languages into considering the development of intercultural programs in their classrooms, to view language learning and teaching as a relationship between language and culture. It has required teachers and students to consider or re-evaluate their own cultural positions with respect to the linguistic and cultural material with which they work. Adopting an intercultural orientation, however, has also required us to consider the assessment of an intercultural capability on the basis that assessment and learning are inextricably linked. Specifically, we have needed to consider what indicators of intercultural learning might look like and how we might assess such learning. We needed to consider what evidence to collect, and in a practical sense, what can be 1
  2. 2. collected in the classroom, in order to make any valid judgements of intercultural learning. Since the teachers involved in this project represented such varied levels of teaching in the classroom, challenging questions were raised about the kinds of intercultural learning that might take place in a year 2 or 3 class as opposed to a middle school classroom or yet again, in a senior school classroom. Are there different assessment procedures that would be appropriate for different cohorts? The Process As participants in the ILTLP project will know, everyone had first to reflect on his or her own cultural position, to clarify cultural identity and then to conceptualise what the intercultural meant. Broadly speaking, and to state the now fairly obvious, participants understood intercultural teaching and learning to be developing an awareness and understanding of linguistic and cultural diversity, of seeing themselves and their students as both language users and as cultural beings. The intercultural works from the premise that both language and culture affects communication and relationships, and that an awareness of behaviour and attitudes towards language and culture, of the multiple interpretations in interaction that are possible, and of the nature of the resource materials we select in the classroom, are very important considerations. The first challenge I and others in the team experienced in this project was something of the obvious - to decide exactly what we wanted students to learn conceptually, and whether the planned interactions were going to elicit the sought after evidence of intercultural learning. In other words, what kinds of formative and summative assessment tasks were going to be useful in determining whether intercultural learning had taken place? It is worthwhile highlighting the diversity of views held by participants in this project. During initial discussions in the first round, some believed that it was simply not possible to assess intercultural learning in conventional ways, arguing that a student’s development of the intercultural is very much tied up with the 2
  3. 3. growth of the individual, the socio-cultural development of the student, if you will. This growth is organic, continuous, and unpredictable, manifesting at different stages for different students. It is a development linked to the whole notion of learning, and even learning for learning, which could only be arbitrarily “assessed” for something like intercultural understanding. On the other hand, given that all learning is along a developmental continuum, even though not always predictably linear, it seems not out of the question to be able to take a snapshot along this continuum, to examine tasks or performances which can conceivably provide evidence of intercultural learning. Increasingly, we now find that writers of curricula at systems levels are now including the notion of the intercultural and, rightly or wrongly, believe that it is a capacity that can be ticked off as a performance standard within the context of conventional approaches to assessment tasks, much in the same way as higher order thinking or treatment of a topic in depth might be able to be ticked off. The tension between these two polarities, that is, the views that the intercultural is assessable or not assessable, certainly makes this area of investigation challenging, not the least from a practical point of view of the daily teaching routines within the classroom. It is not possible to give you a detailed description of the kinds of units of learning planned by the teachers of Italian, Spanish, Chinese and French involved. However, in the first phase of the research, there were “topics” for want of a better description that ranged from asking year 2 students of Italian to deal with the formalities of greetings, and what it is like to live in Rome for Year 10s. There were excursions into the culture of the indigenous of Mexico in a Beginners Spanish class which included adult learners, contrasting these with the treatment of the indigenous in this country. Year 9 students of Chinese looked at schooling while the role of women in China was examined by a class of year 11s. All the while, teachers aimed to get students to be clear on their cultural position while examining the culture of others. This was asking students to demonstrate a sophisticated skill. 3
  4. 4. The French teachers organised activities which included looking at the Parisian Banlieue in a year 12 class, the Pari-roller event in the context of Youth Culture in a year 10/11 class, and the nature of lunch in Australia contrasted with La Cantine in a French école primaire for a group of primary year 5 and 6 group. If I can highlight Kathy Moore’s endeavour, the primary school French colleague with a group of year 6 primary school students who only have 50 or so minutes per week of French, who focused on La Cantine. The purpose of the assessment was for students to recognize:  that there is not one way to have lunch and  that ‘lunchtime’ means different things to different people There was an initial discussion in English about the students’ understanding of what a canteen is, comparing experiences from their primary school and other schools within Australia and overseas. Interactive tasks included:  An initial discussion in English about their understanding of what is a canteen, comparing experiences from Glen Osmond Primary and other schools within Australia and overseas  A written task in English responding to: Describe how you imagine la cantine in a French school to be.  Spoken responses in French of meal preferences in a French cantine.  Written examples in French of meal preferences taken from a French school menu  Written self-reflections about lunchtime in France and at Glen Osmond Primary School, after working with several French texts. Students then had to write reflections to the following questions: 1. Imagine you are going to spend a week or so with a family in France. You will be going to school with them. What will it be like at lunchtime? 4
  5. 5. 2. A French student is visiting Glen Osmond for a short time and will be with your class. What do you need to tell them about lunchtime in your school in order for them to feel comfortable? 3. Where would you prefer to spend your lunchtime? Explain your reasons. The latter tasks clearly required students to apply the knowledge gained of the other culture and put themselves in a position in which they had to be aware of their own cultural practices and values. Kathy reported several instances where some students had an “ah ha” moment, as they came to realise such marked differences between French and Australian lunch routines, as well as pondering the origins of word used. I started the first round of trialling taking the perhaps naïve viewpoint that the intercultural can be gathered principally in the traditional form of a summative task, one I had to consider undertaking, given the time constraints at my disposal in the course. My aim was to: • To assess the students’ writing ability in French as a response to a written text (a SACE required task). • To assess the students’ understanding of the cultural context of the text to which they must respond. • To assess the students’ appreciation of their own cultural perspective in writing the response. After a series of interactive and comprehension activities, and reflecting on what students had gained from these, the final task was for students to write a response in French to a letter by a citizen complaining about roller-bladers riding on the footpaths, endangering others and preventing the complainant, in particular, from walking his dog in peace. (This complaint was authentic as it was also listed on the Pari-roller website.) Students had to examine this complaint as an Australian outsider, being mindful of their own cultural position and that of French participants. This was an explicit criterion. Not all students were able to manage the task in the way I had hoped. 5
  6. 6. The re-assessment Now, you can see in these activities that they are practicable enough in the classroom, of reasonable interest and authenticity to students, and forming the basis of plenty of language learning regardless of whether or not anything specifically of intercultural value could emerge from them. And it was precisely this fact that was the basis of a whole raft of questions that were raised in the evaluation of the first round of the trial. It is a somewhat sobering experience to be grilled on why we did the things we did? Did we elicit the kinds of evidence we were expecting? If not, why not? What did we actually want students to learn from an intercultural perspective? While some of my students were able to respond in articulate and interesting ways, there were others who could not interpret the task as required, even after one-on- one discussions. I had several students from Africa in the group who were finding adjusting to Australian ways of learning challenging enough! A Chinese teacher who looked at the role of women in Chinese society with a group of year 11 girls also had some sophisticated responses from the students. However, she, like many of us, found that she had to revisit the notion of the intercultural and ask herself whether students could actually go beyond the obvious comparison between cultures and sometimes sophisticated statement of opinion from a linguistic point of view. Where was the evidence that students were able to interpret the differences and similarities and take an intercultural position? Many participants also found that there was a greater amount of time spent discussing ideas in English with the students rather than in the target language in order to get across what the students should aim to achieve. In other words, we found that classroom time readily focused on the culture and rather than the language, or conversely, focused on the language at the expense of the culture. Here entered a tricky balancing act. 6
  7. 7. In evaluating what had occurred during the trial and after the first round, the researchers were formulating the view that the traditional ways of measuring achievement, as I had certainly adopted, were insufficient for assessing the intercultural. The implication for us was to shift from the tasks created for the class to emphasising what have been described as “data points”, that is, from an emphasis from stand alone procedures, or structured activities (tests or tasks) to the totality of formal and informal judgements made in interaction. As participants, we were asked to submit a range of these data points, along with a description of the interaction, what we hoped would be achieved in each interaction, and the evidence that arose out of it. Interactions could be written tasks in [language] or English, recordings of conversations and interactions, formative assessment snapshots etc. A common observation made by several participants was the following:  The trouble is that sometimes you get an insight from a student through spontaneous and unplanned conversation. It is difficult to formally call that a data point for collecting evidence, at least, at a practical level in the classroom, even though, in a sense, it is actually a data point at which some “evidence” emerges. How to practically capture them is the challenge. The second round In other words, going into the second round of trials, we were looking at assessment from a different perspective. Assessment was viewed more as something socio-cultural, interactive and as activity rather than as a single event, procedure, task or episode. Assessment became the developmental path and the inquiry for both teachers and learners. And as assessment was reconceptualized, the process of eliciting evidence, validating and judging it changed. In the literature, assessment is described as being of three kinds, broadly speaking. There are the inherent assessments, that is, those that happen informally and non- verbally in all situations. I have already mentioned that many commented students 7
  8. 8. made off the cuff remarks that actually demonstrated an intercultural understanding but which were not captured in any formal way although teachers wished they had been able to do so. There are also the discursive assessments, that is, those that occur when members of a social group talk about what they are doing in an evaluative way. Lastly there are the documentary assessments that occur when activities are evaluated according to a scheme that produces numbers and symbols (in other words the formative and summative assessments with which we are familiar). To reorientate our thinking, Angela Scarino cited a quotation from Delandshere, 2002, that read: “We are moving from an educational practice of assessment where we have defined a priori what we are looking for, to an educational practice where we are participating in activities in which we formulate representations to better understand and transform the world around us. If our purpose is to understand and support learning and knowing and to make inferences about these phenomena, then it seems that the ideas of inquiry – open, critical, dialogic – rather than assessment (as is currently understood) would be more helpful.” Our second round of trialling assessment, therefore, was mindful of the idea that one experience, whether a text, a DVD, an image and so on, is just one experience and will never provide a complete picture. For students to decentre, that is to see culture and language from the inside and the outside, is the more critical factor. The data we intended to collect would need to be recordings of interactions that happened from moment to moment (an extremely tricky procedure), surveys and interviews and other forms of self-reporting, and also our own recordings of the things we had observed happening in the classroom. Of course, samples of written or spoken tasks would also be a feature. In the assessment process, we were continuing to move forward with key questions that were posed from the outset of this research project: • How do we move from the cultural to the intercultural? 8
  9. 9. • What exactly do we want students to learn? What language is needed to do so? • Why is it important? • What data or evidence captures the learning? • What are the questions we need to ask students to progress to the intercultural? • What processes will we use to gather this? How can we connect the “data” points? To capture the data, the focus was more on the assessment than the content, on how students were developing. In my own class, I certainly found it necessary to focus more on the progress of a small group of individuals rather than the whole class of 26 students. I have had to interrogate more vigorously the relationship between language and culture and to interrogate the students about this connection. For example, given the text, why is the structure before you the way it is, why is this or that vocabulary or language used, what is not said? Rather than focus on the end product, the focus had fallen more on the process of thinking in getting there. We were looking more for cultural positioning from students and asking students to respond as themselves rather than as an imaginary person or in an imaginary situation. (Exams are full of questions that ask students to imagine themselves to be this or that person or to be in this or that situation rather than responding to their own feelings, values and knowledge directly.) In the second phase of research, I had a class composed of a different mix of students: 4 year 10 students (I boy and 3 girls), 23 year 11 students (10 males and 13 females). Of the 10 males, there were 6 Africans boys, 1 German, 1 Laotian and 2 Anglo-Australians. Of 13 females, there were 3 German girls, 2 Africans, 4 Anglo- Australian girls and 4 Anglo-Australian adults. Given the size of the class I decided to focus 10 of these students for collecting data and representing a cross section of male and female students and their backgrounds. 9
  10. 10. The topic we studied was multiculturalism, broadly speaking, with some emphasis of the prohibition of the wearing of the hijab in French schools. There were 9 major activities which included students keeping a journal which they could write in English or French (some of the Africans found French easier), recordings made of some interactive tasks, oral reports in French on multiculturalism, comprehension exercises on various texts including writing personal opinions, and a final written task symbolising a cumulative end-point. All these activities served as data points for evidence. There were also the required moments of explicit grammar teaching and the introduction and revision of vocabulary. Just to highlight two activities: the first and the most revealing was one in which students were asked to answer a series of questions about the culture they most identify with, and then to talk to a person in the class they had not talked to share their answers. You can imagine the intensity and amazement of Aussie students who learnt about Kenyan or Congolese wedding ceremonies or attitudes, or adults talking to young Burundi boys about Australian insights. And as often happens in the classroom, I was slow enough not to realise that I had not adequately catered for recording some of the interactions!! Nevertheless, this was a genuine intercultural moment, so to speak, since students suddenly had insights into the other and their own cultural headsets. For the culminating activity, students were presented with two sets of photos and had to choose one set, to write a response in French by interpreting these pictures and their feelings, and try to express their cultural point of view. Before this task, and in fact during the series of activities, I had to explicitly remind students what we were aiming to achieve, that is, to explain again what the intercultural actually entailed. What students were asked to do was quite challenging, abstract in some ways, and requiring them have an adult headset. One student only actually analysed the language written across the photo of a Muslim girl wearing the hijab. The girls wearing the hijab in the tricolor was better understood in terms of the cultural context. 10
  11. 11. As a general observation of interaction in the class, I found that the African students participated less publicly in whole class discussions while the Anglo- Australian girls were the most vocal. The class as a whole was fully engaged, however, and many expressed an appreciation of the opportunity to deal with the topic in the manner that we did. On the question of multiculturalism the African and German students thought Australia was a multicultural society while the Anglo-Australian students thought Australia was on the way of being one but that there was still quite a bit of intolerance. Being involved in this project has certainly changed my view of how I teach and it is difficult not to see every resource used now in terms of a cultural context and the potential for some intercultural understanding. However, I do see on-going problems with assessing intercultural learning; while curriculum designers increasingly frame the intercultural component into formulaic assessment, I feel uncomfortable about it being something easily accounted for in performance. The reality is that we are dealing with a student’s development and internal state of cognition that is not easily accounted for at a given moment. Only the most articulate, perhaps, will be able to express their insight within a one off assessment piece. I find it difficult to evaluate how much my explicit teaching of the teaching of the intercultural actually determined the responses rather than there being an unexpected and valuable “intercultural” revelation from the students. Maybe it does not matter. I was certainly taken aback when one of the students in a piece of writing, which was in the form of a dialogue, had one of the characters remark: “Oh you are an interesting intercultural person!” And so here we have a case of the concept already having become what I call a commodity, something that is packaged, and in that sense, potentially removed from authentic interaction. Certainly, intercultural teaching and learning is not an easy recipe or methodology that is a panacea for learning language and culture, and for going beyond cultural 11
  12. 12. comparisons. It requires a different kind of alertness and different kind of questioning from the teachers to progress the intercultural learning in the students. It will take some experience to capture the right bits of evidence to show that education, or self-education in a broad sense, has sparked young minds into looking at themselves and others in an exciting way. Of course, good teachers have always had elements of the intercultural in the classroom. Good teachers have always managed to ask the right questions to progress their students’ learning. Through the experience of this research project, I hope to join them. Joe van Dalen Acting Deputy Principal School of Languages 255 Torrens Road West Croydon SA 5008 Mob: 0421 954 216 July 10, 2009 12