0
DO INDONESIANS MAKE MOBILE CALLS AND DO RUSSIANS  GO ON HOLIDAY? NHTV Breda  – january 2008 Katholieke Universiteit Leuven...
 
Content of this presentation <ul><li>Purpose of this presentation: to be aware of cultural bias in the classroom </li></ul...
The Certificate of Dutch as a Foreign Language <ul><li>CNaVT (Certificaat Nederlands als Vreemde Taal) </li></ul><ul><li>a...
Assess as direct as possible <ul><li>Based on a needs analysis </li></ul><ul><li>authenticity: 6 exams represent 6 differe...
The effect of culture: cultural bias <ul><li>Definition of cultural bias: </li></ul><ul><li>Scores of groups are distorted...
THE EFFECT OF CULTURE ON DIFFERENT LEVELS <ul><li>(Van Baalen, Beheydt & van Kalsbeek, 2003) </li></ul><ul><li>On the leve...
1. WORDS and EXPRESSIONS <ul><li>   verpotteren </li></ul><ul><li>   flitspaalgedrag </li></ul><ul><li>speeddaten </li><...
2. IMAGES
 
3. SUBJECTS
4. KNOWLEDGE OF THE WORLD example 1 <ul><li>You want to organize a party for your birthday.  </li></ul><ul><li>Describe wh...
4. KNOWLEDGE OF THE WORLD example 2 vs.
5. RELATION EXAMINER-STUDENT
6. CLASSROOM PRACTICE/ TEST SITUATION
MINI TEST <ul><li>Read the test and answer the following question. </li></ul><ul><li>Does this test contain cultural bias ...
A TENSION <ul><li>Validity in the sense that we want our exams to measure language proficiency. </li></ul>Reliability in t...
DIF ANALYSIS DIF Differential Item Functioning = the statistical view of cultural bias
CNaVT item bias research Example 1 Country: Indonesia Item: Question in form ‘ What kind of object have you lost?’
CNaVT item bias research Example 2 Country: Russia Item: Question from friend in letter  (…) ‘But I often think back with ...
How does the CNaVT manage the effect of culture? <ul><li>awareness of  what  we test: what (concerning culture) is relevan...
BUT… CNaVT is a test of Dutch as a foreign language!  <ul><li>Difference between test situation and the classroom  </li></...
Tips: How to make students interculturally competent?  <ul><li>Three important principles: </li></ul><ul><li>Perspective <...
DISCUSSIE / VRAGEN <ul><li>What is your experience with cultural bias in your own classroom/test practice? Can you give an...
LITERATUURVERWIJZINGEN <ul><li>Van Baalen, C., Beheydt, L. & Van Kalsbeek, A. (2003).  Cultuur in taal. Interculturele vaa...
DO INDONESIAN MAKE MOBILE CALLS AND DO RUSSIAN  GO ON HOLIDAY? NHTV Breda  – januar 2008 Katholieke Universiteit Leuven Un...
Upcoming SlideShare
Loading in...5
×

Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?

738

Published on

Do Indonesians make mobile calls and do Russians go on holiday? Cultural determination in tests. By academic staff of the University of Amsterdam and the Catholic University of Leuven

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
738
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Transcript of "Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?"

    1. 1. DO INDONESIANS MAKE MOBILE CALLS AND DO RUSSIANS GO ON HOLIDAY? NHTV Breda – january 2008 Katholieke Universiteit Leuven Universiteit van Amsterdam E.Teijsse-Jongkind @uva.nl [email_address] The effect of culture on testing The Certificate Dutch as a Foreign Language
    2. 3. Content of this presentation <ul><li>Purpose of this presentation: to be aware of cultural bias in the classroom </li></ul><ul><li>Background of the CNaVT (Certificaat Nederlands als Vreemde Taal) </li></ul><ul><li>Definition of cultural bias and the effect of culture on different levels </li></ul><ul><li>Task: Does this test contain cultural bias? </li></ul><ul><li>How does the CNaVT manage the effect of culture in the CNaVT-exams? </li></ul><ul><li>Discussion: own experiences with cultural bias </li></ul>
    3. 4. The Certificate of Dutch as a Foreign Language <ul><li>CNaVT (Certificaat Nederlands als Vreemde Taal) </li></ul><ul><li>administration in more than 40 countries around the world: from Madrid to Jakarta and from Warsaw to Rio de Janeiro </li></ul><ul><li>2500 candidates each year </li></ul>
    4. 5. Assess as direct as possible <ul><li>Based on a needs analysis </li></ul><ul><li>authenticity: 6 exams represent 6 different domains </li></ul><ul><li>task-based: exam tasks represent real life language use situations </li></ul>
    5. 6. The effect of culture: cultural bias <ul><li>Definition of cultural bias: </li></ul><ul><li>Scores of groups are distorted, because the items favour some groups over others. This distortion is not caused by a difference in (language) proficiency. </li></ul>
    6. 7. THE EFFECT OF CULTURE ON DIFFERENT LEVELS <ul><li>(Van Baalen, Beheydt & van Kalsbeek, 2003) </li></ul><ul><li>On the level of: </li></ul><ul><li>1. Words and expressions </li></ul><ul><li>2. Images </li></ul><ul><li>3. Subjects and concepts </li></ul><ul><li>Knowledge of the world </li></ul><ul><li>Relation examiner - candidate </li></ul><ul><li>Classroom / test situation </li></ul>
    7. 8. 1. WORDS and EXPRESSIONS <ul><li> verpotteren </li></ul><ul><li> flitspaalgedrag </li></ul><ul><li>speeddaten </li></ul><ul><li> (zit)bank vs. zetel </li></ul><ul><li> lopen vs. stappen </li></ul><ul><li> namiddag </li></ul><ul><li> iets afkloppen/ hout vasthouden </li></ul>
    8. 9. 2. IMAGES
    9. 11. 3. SUBJECTS
    10. 12. 4. KNOWLEDGE OF THE WORLD example 1 <ul><li>You want to organize a party for your birthday. </li></ul><ul><li>Describe what you want to your friend. </li></ul>
    11. 13. 4. KNOWLEDGE OF THE WORLD example 2 vs.
    12. 14. 5. RELATION EXAMINER-STUDENT
    13. 15. 6. CLASSROOM PRACTICE/ TEST SITUATION
    14. 16. MINI TEST <ul><li>Read the test and answer the following question. </li></ul><ul><li>Does this test contain cultural bias for some groups of students? Which levels of cultural bias can appear? Why? </li></ul>
    15. 17. A TENSION <ul><li>Validity in the sense that we want our exams to measure language proficiency. </li></ul>Reliability in the sense that we want to be able to state that our exams are not influenced by other factors like the culture of the candidate.
    16. 18. DIF ANALYSIS DIF Differential Item Functioning = the statistical view of cultural bias
    17. 19. CNaVT item bias research Example 1 Country: Indonesia Item: Question in form ‘ What kind of object have you lost?’
    18. 20. CNaVT item bias research Example 2 Country: Russia Item: Question from friend in letter (…) ‘But I often think back with nostalgia about my holiday in your country. (…) And you? What did you do during your holidays? Did you travel? Where to? Or did you stay at home and what did you do?’ (…)
    19. 21. How does the CNaVT manage the effect of culture? <ul><li>awareness of what we test: what (concerning culture) is relevant, what is irrelevant </li></ul><ul><li>generate input on possible cultural bias from different sources in different stadia </li></ul><ul><li>In case of suspicion of irrelevance: DIF analysis </li></ul>
    20. 22. BUT… CNaVT is a test of Dutch as a foreign language! <ul><li>Difference between test situation and the classroom </li></ul><ul><li>Cultural bias is not something that has to be avoided in the classroom </li></ul><ul><li>But: teachers should use cultural miscommunications to learn from it </li></ul><ul><li>Intercultural competence! </li></ul><ul><li>Teacher and test developer: recognize and be aware of the effect of culture </li></ul>
    21. 23. Tips: How to make students interculturally competent? <ul><li>Three important principles: </li></ul><ul><li>Perspective </li></ul><ul><li>Context </li></ul><ul><li>Dialogue </li></ul>
    22. 24. DISCUSSIE / VRAGEN <ul><li>What is your experience with cultural bias in your own classroom/test practice? Can you give an example? How do you manage? </li></ul>
    23. 25. LITERATUURVERWIJZINGEN <ul><li>Van Baalen, C., Beheydt, L. & Van Kalsbeek, A. (2003). Cultuur in taal. Interculturele vaardigheden voor docenten Nederlands aan anderstaligen . Utrecht: Nederlands centrum buitenlanders, 167p. </li></ul>
    24. 26. DO INDONESIAN MAKE MOBILE CALLS AND DO RUSSIAN GO ON HOLIDAY? NHTV Breda – januar 2008 Katholieke Universiteit Leuven Universiteit van Amsterdam E.Teijsse-Jongkind @uva.nl [email_address] Language testing and culture The certificate of Dutch as a foreign language
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×