CEFcult: online assessment of oral language skills in an intercultural workplace Ana Beaven and Lucia Livatino EUROCALL 20...
Recommendations and standards <ul><li>Recommendations of the ELAN Project (2006), “Effects on the European Union Economy o...
The WebCEF project (2006-2009) <ul><li>Development of an online spoken language assessment tool for learners of various Eu...
The CEFcult project (2009-2011) <ul><li>Description </li></ul><ul><ul><li>Development of an online assessment environment ...
Users <ul><ul><li>Target groups: </li></ul></ul><ul><ul><li>undergraduates and graduates who are preparing themselves for ...
Partners <ul><li>AVNet - K.U.Leuven (Belgium)  Coordinator </li></ul><ul><li>Open Universiteit Nederland (The Netherlands)...
Outputs <ul><li>Web environment:  a modular, customizable environment that allows for assessment of oral and intercultural...
Design and Development phase <ul><li>Partner meetings in Nov.2009 and June 2010. </li></ul><ul><li>Creation of project web...
 
The web-based environment <ul><li>Choice of scenarios. </li></ul><ul><li>Recording of samples (using FlashVlog or uploadin...
Scenarios and tasks <ul><li>Scenarios : applying for a job, working in a multinatinal team, etc. </li></ul><ul><li>Tasks :...
Assessment scales <ul><li>Common European Framework of Reference (CEFR) </li></ul><ul><ul><li>Linguistic competence (oral ...
Assessment scales <ul><li>Intercultural Competence Assessment (INCA) </li></ul><ul><ul><li>Openness: respect for otherness...
Example from the scenario “Working in a multinational team  in different European countries” <ul><li>Task 2 “Explaining a ...
Example from the scenario “Working in a multinational team  in different European countries” <ul><li>Intercultural compete...
Interaction: Information exchange  (CEFR 4.4.3) Can exchange, check and confirm accumulated factual information on familia...
Descriptors from INCA When people communicate in ways I do not understand I try in an unsystematic way to take part, but h...
Scenarios and tasks: issues <ul><li>Privacy : Private / public samples </li></ul><ul><li>Stakes : high/low </li></ul><ul><...
Thank you Ana Beaven  [email_address] Lucia Livatino  [email_address]
Upcoming SlideShare
Loading in …5
×

CEFcult Eurocall 2010

661 views

Published on

This was the presentation of the European project CEFcult at EUROCALL in Bordeaux, France, Sept. 2010

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
661
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • ELAN: Effects on the European Economy of Shortages of Foreign Language Skills in Enterprise was commissioned by the Directorate General for Education and Culture of the European Commission in December 2005 and undertaken by CILT, the UK National Centre for Languages in collaboration with InterAct International and an international team of researchers. Its objective was to provide the Commission and decision-takers in Member States with practical information and analysis of the use of language skills by SMEs and the impact on business performance. The study has shown that language skills can significantly improve export success for European companies. Small and medium sized enterprises (SMEs) which have a languages strategy and invest in staff with language skills are shown to be able to achieve 44% more export sales than those which don&apos;t. Analysis of the survey responses identified some key elements of language management which were associated with strong export performance, and there could be very significant gains to the EU economy if all exporting SMEs employed these techniques ELAN Report (2006), sponsored by the EU (European Commission Directorate General for Education and Culture), into the effects of language skills on the economy of Europe. Recommendations: General recommendations: 3.Improve Business-Education links in relation to languages. Identify and disseminate models of successful collaboration between Business and Education especially, but not exclusively, directed towards the promotion of language skills. Recommendations relating to SMEs and businesses: 8.Support businesses to become more expert at managing language skills and in applying the four elements of language management which are associated with improved export performance. Four elements of language management were found to be associated with successful export performance: having a language strategy, appointing native speakers, recruiting staff with language skills and using translators/interpreters.
  • Inca Project 2004, Leonardo da Vinci – Framework and Scales for assessment of Intercultural Competence in the workplace
  • CEFcult Eurocall 2010

    1. 1. CEFcult: online assessment of oral language skills in an intercultural workplace Ana Beaven and Lucia Livatino EUROCALL 2010 - Bordeaux
    2. 2. Recommendations and standards <ul><li>Recommendations of the ELAN Project (2006), “Effects on the European Union Economy of Shortages of Foreign Language Skills in Enterprise” commissioned by the Directorate General for Education and Culture of the European Commission . </li></ul>Businesses should be encouraged […] to support education and training programmes linking languages and enterprise, working with schools, colleges and universities. <ul><li>“ National Occupational Standards for Intercultural Working”, CILT, The National Centre for Languages, UK (2008). </li></ul>
    3. 3. The WebCEF project (2006-2009) <ul><li>Development of an online spoken language assessment tool for learners of various European languages, teachers of secondary school and HE, teacher trainers and trainees in Europe. </li></ul>
    4. 4. The CEFcult project (2009-2011) <ul><li>Description </li></ul><ul><ul><li>Development of an online assessment environment for oral language skills and intercultural communicative competence. </li></ul></ul><ul><li>Funding body </li></ul><ul><ul><li>The Lifelong Learning Programme of the Education, Audiovisual & Culture Executive Agency of the European Commission (key activity 2: Languages). </li></ul></ul>
    5. 5. Users <ul><ul><li>Target groups: </li></ul></ul><ul><ul><li>undergraduates and graduates who are preparing themselves for first-time employment </li></ul></ul><ul><ul><li>Participants in vocational training </li></ul></ul><ul><ul><li>managers and employees </li></ul></ul><ul><ul><li>language teachers </li></ul></ul><ul><ul><li>in-company trainers </li></ul></ul><ul><ul><li>educators </li></ul></ul>
    6. 6. Partners <ul><li>AVNet - K.U.Leuven (Belgium) Coordinator </li></ul><ul><li>Open Universiteit Nederland (The Netherlands) </li></ul><ul><li>Technische Universität Dresden (Germany) </li></ul><ul><li>Internationale Hogeschool Leuven (Belgium) </li></ul><ul><li>Université de Savoie (France) </li></ul><ul><li>Università di Bologna (Italy) </li></ul><ul><li>Uniwersytet Łódzki (Poland) </li></ul><ul><li>Helsingin Yliopisto (Finland) </li></ul><ul><li>Euneos Oy (Finland) </li></ul><ul><li>Jyväskylän Yliopisto (Finland) </li></ul><ul><li>The Open University (UK) </li></ul><ul><li>Fontys Hogescholen (The Netherlands) </li></ul>
    7. 7. Outputs <ul><li>Web environment: a modular, customizable environment that allows for assessment of oral and intercultural skills. </li></ul><ul><li>Scenarios: a set of authentic communication tasks through which users can assess their oral and intercultural skills in different European languages. </li></ul><ul><li>Training materials: for self-, peer- and expert assessment in education and enterprises. </li></ul><ul><li>Guidelines: research-supported guidelines for foreign language teaching and educational policy. </li></ul><ul><li>Collaborative assessment: viable and tested models for collaborative assessment among higher education institutions, enterprises, and various stakeholders. </li></ul>
    8. 8. Design and Development phase <ul><li>Partner meetings in Nov.2009 and June 2010. </li></ul><ul><li>Creation of project website. </li></ul><ul><li>Development of a prototype of the web-based environment for assessment. </li></ul><ul><li>Selection of rating scales. </li></ul><ul><li>Definition of scenarios and tasks that require the activation of both language and intercultural competences. </li></ul><ul><li>Scheduling and specification of pilots (HE and enterprise). </li></ul><ul><li>Initial development of the social media aspects of the assessment environment. </li></ul><ul><li>Appointment of advisory board. </li></ul><ul><li>Dissemination. </li></ul>
    9. 10. The web-based environment <ul><li>Choice of scenarios. </li></ul><ul><li>Recording of samples (using FlashVlog or uploading of samples). </li></ul><ul><li>Assessment of samples (self-, peer- or expert assessment). </li></ul><ul><li>Assessment environment will be enhanced by available web 2.0 applications, advancing a number of user-driven functionalities (communities of practice; e-portfolio). </li></ul>
    10. 11. Scenarios and tasks <ul><li>Scenarios : applying for a job, working in a multinatinal team, etc. </li></ul><ul><li>Tasks : eliciting linguistic and intercultural performance. </li></ul><ul><li>Response format : sustained monologues, interactions, etc </li></ul><ul><li>Assessment : </li></ul><ul><li>CEFR </li></ul><ul><li>INCA </li></ul>
    11. 12. Assessment scales <ul><li>Common European Framework of Reference (CEFR) </li></ul><ul><ul><li>Linguistic competence (oral production and interaction): range, accuracy, fluency, coherence, interaction. </li></ul></ul><ul><ul><li>Sociolinguistic competence </li></ul></ul><ul><ul><li>Pragmatic competence </li></ul></ul>
    12. 13. Assessment scales <ul><li>Intercultural Competence Assessment (INCA) </li></ul><ul><ul><li>Openness: respect for otherness, tolerance of ambiguity. </li></ul></ul><ul><ul><li>Knowledge: knowledge discovery, empathy. </li></ul></ul><ul><ul><li>Adaptability: behavioural flexibility, communicative awareness. </li></ul></ul>
    13. 14. Example from the scenario “Working in a multinational team in different European countries” <ul><li>Task 2 “Explaining a procedure” </li></ul><ul><li>Integrated Speaking Task (recording time: 2-3 minutes) </li></ul><ul><li>In this task, you will first be asked to watch a video clip in which a person explains a procedure to a colleague who has never done it before and who does not speak English well. </li></ul><ul><li>Once you have watched the clip, reflect on the following points: </li></ul><ul><ul><li>whether the communication was successful or not; </li></ul></ul><ul><ul><li>how you can tell; </li></ul></ul><ul><ul><li>what were the positive and negative aspects and why; </li></ul></ul><ul><ul><li>what, if anything, could have been done to improve the interaction. </li></ul></ul><ul><li>Now, when you are ready, record yourself as you explain the same procedure, taking into account your reflections. Pretend you are talking to an interlocutor who does not speak English well or, if you have a partner, record yourself explaining the procedure to him/her. </li></ul>
    14. 15. Example from the scenario “Working in a multinational team in different European countries” <ul><li>Intercultural competence tested: Adaptability (Communicative awareness). </li></ul><ul><li>Linguistic competence tested: Speaking. </li></ul><ul><ul><li>Interaction: Information exchange (CEFR 4.4.3). </li></ul></ul><ul><li>Pragmatic competence tested: Discourse competence (flexibility). (CEFR 5.2.3.1). </li></ul>
    15. 16. Interaction: Information exchange (CEFR 4.4.3) Can exchange, check and confirm accumulated factual information on familiar routine and non-routine matters within his/her field with some confidence. Can describe how to do something, giving detailed instructions. Can summarise and give his or her opinion about a short story, article, talk, discussion, interview, or documentary and answer further questions of detail. Can find out and pass on straightforward factual information. Can ask for and follow detailed directions. Can obtain more detailed information. B1 Can understand and exchange complex information and advice on the full range of matters related to his/her occupational role. Can pass on detailed information reliably. Can give a clear, detailed description of how to carry out a procedure. Can synthesise and report information and arguments from a number of sources. B2
    16. 17. Descriptors from INCA When people communicate in ways I do not understand I try in an unsystematic way to take part, but hope that they will eventually adapt to the way I communicate. I know that others may communicate in ways I am not familiar with. My behaviour is influenced by principles that guide me and I often plan for eventualities, including ambiguous situations. I seek to achieve good communication both by making my own conventions clearer and by adopting those of others. When there is, or might be, a problem with communication, I quite often find ways around it, e.g. using gesture, re-explaining, simplifying etc. I am aware of a number of useful strategies for dealing with common communication problems When ambiguous situations arise, I can usually clarify or otherwise deal with them to the benefit of the group. I use my communication strategies to prevent, solve and mediate problems arising from differences in language or other communication conventions. I have a good overall understanding of the kinds of communicative difficulties that can arise in an intercultural context and of a wide range of strategies for resolving them. When ambiguous situations arise, I can usually clarify or otherwise deal with them to the benefit of the group. I use my communication strategies to prevent, solve and mediate problems arising from differences in language or other communication conventions. I have a good overall understanding of the kinds of communicative difficulties that can arise in an intercultural context and of a wide range of strategies for resolving them. 3. Full My behaviour is influenced by principles that guide me and I often plan for eventualities, including ambiguous situations. I seek to achieve good communication both by making my own conventions clearer and by adopting those of others. When there is, or might be, a problem with communication, I quite often find ways around it, e.g. using gesture, re-explaining, simplifying etc. I am aware of a number of useful strategies for dealing with common communication problems 2. Intermediate When people communicate in ways I do not understand I try in an unsystematic way to take part, but hope that they will eventually adapt to the way I communicate. I know that others may communicate in ways I am not familiar with. 1. Basic Adaptability (Communicative awareness)
    17. 18. Scenarios and tasks: issues <ul><li>Privacy : Private / public samples </li></ul><ul><li>Stakes : high/low </li></ul><ul><li>Languages : Target language, mother tongue, lingua franca </li></ul><ul><li>Cultures : culture-general/culture-specific </li></ul><ul><li>Assessment : Integrated / separated language and culture assessment </li></ul><ul><li>Roles : Assessee /assessor </li></ul><ul><li>Scales : INCA: need for operational descriptors/examples? </li></ul><ul><li>Technical issues. </li></ul><ul><li>Other issues? </li></ul>
    18. 19. Thank you Ana Beaven [email_address] Lucia Livatino [email_address]

    ×