Assessing Intercultural Capability: teacher and researcher perspectives Presentation by Angela Scarino and Melissa Gould-Drakeley at AFMLTA conference in Sydney 2009, Dialogue Discourse Diversity

1. Assessing intercultural capability:
teacher and researcher perspectives
AFMLTA Biennial National Languages Conference
Sydney, 9-12 July 2009
Angela Scarino
Research Centre for Languages and Cultures, University of South Australia
email: angela.scarino@unisa.edu.au
Melissa Gould-Drakeley
Macarthur Anglican School
email: mgoulddrakeley&mas.nsw.edu.au

2. The issue: uncertainties
 uncertainty pertaining to the construct
What exactly is this ‘intercultural capability?
 uncertainty pertaining to the nature of assessment
How do we assess intercultural understanding in language
learning? Is it embedded within or separate from assessing
language?
→ these questions evidence deeply held assumptions about
the nature of language and language learning and the
nature of assessment
2

3. ILTLP investigations: teacher-researcher
collaboration
• developing a long-term program
• feedback → incorporation of feedback
• year long investigation; ongoing data gathering (e.g. student logs)
• classroom observation; interview with Melissa; interview with
students.
3

4. The assessment cycle
Conceptualising  Eliciting  Judging 
Validating
(what to assess) (how to elicit) (how to appraise) (how to
justify)
(Scarino 2009)
4

5. Traditional and alternative assessment
 Traditional: testing content through objective procedures as a single
event; a measurement focus
 Alternative: diverse meanings
(1) the centrality of the classroom (teaching practice and
learning process)
(2) the active role played by students/learners in the
assessment process
(3) a heightened view of process
(4) outcomes charasterized by summaries of learner
competencies which are detailed, descriptive,
informative (Fox 2008:102)
5

6. Interview
AS: What does the intercultural actually mean to you now that you’ve played with it for a few
years……?
M: …..it’s really meaningful communication. And I suppose, when I’ve looked at my program, I think
that the things I’ve changed significantly is now that my program is now a program in which my
students feature…. So it’s that idea, I think you might have quoted it, that it’s a peopled program and
that the whole idea of it’s being not only just turning the page and moving on, it’s actually looking at
the students themselves…..and I think for me actually a big change too.
But what is different for me is the intra-cultural…… and really coming to terms with it, looking at each
student’s background, and knowledge, and experience, and absolutely everything they bring to the
classroom and how we are all moving together and yet on a separate journey…… and that, if I don’t
understand and they don’t understand their intra-cultural journey, they never will understand the
intercultural. And to me that’s the difference in the way I teach….. so for me, I suppose it’s the intra-
cultural…that it’s been the big shift in the way I present things…………
(Melissa Gould-Drakeley interviewed by Angela Scarino and Leo Papademetre,
September 2007)
6

7. Conceptualising: multidimensional nature
of language
“there are different types of language knowledge and mastering
one type is no guarantee for mastering another……..”
(Shohamy 1996:152)
7

8. Eliciting intercultural capability
Assessing intercultural capability involves several dimensions:
 interactions (spoken and written) in the target language in which students
negotiate meaning through interpreting and using language in diverse
contexts, involving people from diverse linguistic and cultural backgrounds;
in these interactions what is of interest is the accuracy, fluency and
complexity of language used in the exchange as well as how students
negotiate meaning in interaction and how they manage the variability that the
particular context of communication demands;
 eliciting understanding of the way peoples’ enculturation affects how they
see and interpret the world, interact and communicate; understanding how
peoples’ own language(s) and culture(s) come into play in exchanging
meaning; understanding how they themselves are already situated in their
own language(s) and culture(s), and the same for others, and recognising
that this experience is dynamic, ever-developing.
8

9. Example 1: scope and sequence
9

10. Example 1: scope and sequence (cont.)
10

11. Example 1: scope and sequence (cont.)
11

12. Example 2: questioning
Questions used in this module include:
Is religion part of your identity? Is it an important aspect of your life? Do your parents (or
others) expect you to follow a religion?
What influence does Macarthur Anglican School have on your identity (if any)? Does going to a
Christian school have an impact on who you are?
Do you think religion is an important aspect of the lives of Indonesians? What influences your
point of view on this? Consider the previous modules covered.
What do you know about religion in Indonesia?
From your readings/viewings/interactions, what do you think about arranged marriages, or
elopement or interfaith marriage, etc in Indonesia? How does this relate back to the
information you have learnt on ethnic diversity in Indonesia? Consider your situation. Will
you have an arranged marriage? Is it possible for you to marry someone from a different
religious/ethnic background? Is age a factor?
What connections have you made to previous modules?
→ questions enmeshed in continuous classroom interactions.
12

13. Example 2: questioning (cont.)
Teacher input and facilitation is necessary, they:
 encourage noticing and analysing of language and culture (including
visuals);
 manage the discussion of controversial issues and diverse perspectives;
 challenge the naturalness of culture and students’ own culture being the
norm;
 help dispel stereotypes;
 encourage students to ask themselves questions: Why do I think that?
Where does that idea come from?
 provide sensitive feedback;
 provide time for guided student reflection;
 encourage students to interpret themselves.
13

14. Example 3: Excerpts from Becky’s
learning log
“Learning about Mudik, in which everyone has the right to go home to celebrate the
breaking of the fast, no matter who they are or what ‘status’ they are astounds me. This
is because it is such a contrast to what I have learnt so far to be the hierarchical culture
of every day Indonesian life, displaying the huge importance this celebration holds in
Indonesian society.” This shows that the student is questioning her knowledge.
“…….. it was interesting to note that the inclusive words and phrases like “kita”
(inclusive form of ‘we’), “kita semua” (we all) “setiap orang” (every person), “bersama”
(together), which I have seen several times in reading, writing, speaking and listening,
are used so consistently throughout many texts we have studied in Indonesian. To me,
this displays a clear reflection of the Indonesian culture and their aims as a nation and
individual islands/ societies ie the national determination to work together showing unity
[the Indonesian motto is Unity in Diversity.] These words employ a sense of unity and
inclusiveness. I need to be very aware of this when I interact with an Indonesian.”
14

15. Example 4: assessment task (interview
and email)
The assessment task was as follows:
Students interviewed a young Indonesian male Muslim, Ucy, (who was on exchange in
Australia for 3 weeks) about his religious practice and the special occasions associated
with his religion. Students then imagined they were living in Indonesia with Ucy’s family
and witnessed a special occasion. Using the information from their interview, they
described and reflected on the special occasion in the form of an email to a close friend.
Criteria for judging performance in the interview:
recognise and employ language appropriate to interviewee and social context
relevance and depth of questions
clarity of expression (pronunciation, intonation, stress)
accuracy of vocabulary and sentence structures
variety and appropriateness of vocabulary and sentence structures
capacity to maintain a conversation (comprehension, communication strategies)
15

16. Example 4: assessment task (interview
and email) (cont.)
Sample student email response
“sebelum aku datang ke sini aku kira bahwa agama Islam nggak menerima agama lain.”
(Before I came here I thought that Islam didn’t accept other religions) …. Di Australia aku
berprasangka tentang semua aspek agama Islam. Aku berpendapat bahwa mereka nggak
membolehkan aku ikut perayaan mereka. Akhirnya, Ucy berkata: ‘tergantung dari kamu.’ (In
Australia I was prejudiced about all aspects of Islam. I thought that they wouldn’t allow me to
take part in their celebrations. Finally Ucy said: It’s up to you.” (Alana) Alana had asked Ucy in
her interview if, as a Christian, she would be able to participate in the fasting month of
Ramadan.
Ucapan yang selalu aku dengar adalah ‘Selamat Hari Raya. Maaf lahir batin.’ Ucapan ini
menarik; penting bahwa yang muda mengunjungi yang tua untuk sungkem kepada mereka
dan minta maaf. Aku ngaak sungkem, sedangkan Ucy dan kakaknya sungkem kepada
ortunya. Bisa kamu membayangkannya?’ The expression that I always heard was “Happy
Hari Raya. Please forgive me in body and soul.” This expression is interesting; it’s important
for young people to kneel before their parents and request forgiveness. I didn’t kneel whereas
Ucy and his older brother knelt before their parents. Can you believe it? (Laura)
16

17. Judging performance
A framework for judging intercultural capability:
 the nature and scope of the interaction
─ understanding the task, text, concept or participation in interaction in vivo
─ understanding assumptions/diverse perspectives
─ responding to others: noticing, explaining, connecting, applying, etc;
 understanding the process of interpretation/themselves as interpreters and
their ability to reflect;
and, within a long-term perspective to consider:
 the connections that students make in all of the above in understanding
languages and cultures in social life
(see also Scarino & Liddicoat, 2009:75-76)
17

18. Validation: video interview data
 Has taking an intercultural approach to your learning helped you learn more about your own culture?
Yes absolutely. Language reflects culture and culture reflects language – the two are inseparable. Um, as I said before OFC [Object
Focus Construction], in studying OFC, I’ve learnt that my culture is a very concerning Western society whereas in Indonesia,
they’re not.” Becky
 Has taking the intercultural approach helped you learn in a deeper way?
I’m not just looking at vocabulary, not just learning how to speak the language, I’m not just learning the what but I’m learning the how
and the why. Through focusing on my intercultural understanding, I’m gaining a better understanding of the culture in Indonesia
and my own culture. (Becky)
I feel like I am learning at deeper level because I’m trying to look beyond and like trying to understand more about them rather than
what they look like and how they talk and everything… language is in culture and culture is language and they both tie into together
and um when I visited Malaysia it really was an eye-opener for me to really connect with how they interact with everyone.”
(Candice)
 How do you feel about reflecting on your learning?
Yeah I really enjoy that, um, it’s time when I can really reinforce what I learn on paper. Then when it’s on paper I can then go back
and look over it therefore I won’t forget it and it allows me to deepen my understanding and knowledge of what I’ve learned, not
simply forget it. (Becky)
 How is the intercultural approach different to other ways you have learned?
“I think the major thing for me is this idea that - you go into a Maths classroom, it’s here’s a formulae, make sure you know what the
heck to do with it. You go into an English classroom – here’s a play, make sure you know it backwards, intercultural learning, yeah
again, it’s thinking of things on a deeper level.” (Frank)
18

19. Conclusion
Assessing an intercultural capability involves an invitation:
 to reconceptualise language, culture, learning;
 to reconceptualise the nature of evidence of learning;
 to expand the ways of gathering evidence and the kinds of evidence
gathered;
 to create a new culture of learning and assessment.
19

20. References
Birenbaum, M. (1996) Assessment 2000; towards a pluralistic approach to assessment. In Birenbaum, M. and Dochy, F. (eds).
Alternatives in assessment of achievements, learning processes and prior knowledge. Boston, MA. Kluwer Academic
Publishers, pp.3-29.
Black, P. & Jones, J. (2006) Formative assessment and the learning and teaching of MFL: sharing the languages learning road
map with the learners. Language Learning Journal 34, 1, 4-9.
Fox, J. (2008) Alternative assessment. In Shohamy, E. & Hornberger, N.H. Encyclopedia of language and education (second
edition). Volume 7. Language testing and assessment. New York. Springer. Chapter7, pp.97-110.
Gipps, C. (1999) Sociocultural aspects of assessment. Review of Research in Education. 24, pp.355-392.
Liddicoat, A., Papademetre, L., Scarino, A. & Kohler, M. (2003) A report on intercultural language learning. Australian
Government Department of Ducation, Science and Training, Canberra. http://www.curriculum.edu.au/nalsas/reports02.htm
Scarino, A. (2009) Assessing intercultural capability in learning languages: some issues and considerations. Language Teaching
42,1,67-80.
Scarino, A. & Liddicoat, A.J. (2009) Teaching and Learning Languages: a guide. Carlton South, Victoria. Curriculum
Corporation.
Scarino, A., Liddicoat, A.J., Crichton, J., Curnow, T.J., Kohler, M., Loechel, K., Mercurio, N., Morgan, AM., Papademetre, L. &
Scrimgeour, A. (2008) Professional Standards Project Professional Learning Program. www.pspl.unisa.edu.au
Scarino, A., Liddicoat, A.J., Carr, J., Crichton, J., Crozet, C., Dellit, J., Kohler, M., Loechel, K., Mercurio, N., Morgan, AM.,
Papademetre, L. & Scrimgeour, A. (2007) The Intercultural Language Teaching and Learning in Practice (ILTLP) Project.
www.iltlp.unisa.edu.au
Shohamy, E. (1996) Language testing. Matching assessment procedures with language knowledge. In Birenbaum, M. and
Dochy, F. (eds). Alternatives in assessment of achievements, learning processes and prior knowledge. Boston, MA. Kluwer
Academic Publishers, pp.143-159.
Teasdale, A. and Leung, C. (2000) Teacher assessment and psychometric theory: a case of paradigm crossing? Language
Testing. 17, 2, pp.163-184.
20