Your SlideShare is downloading. ×
Surveying English Language Assessment Practices in International Plurilingual Organizations
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Surveying English Language Assessment Practices in International Plurilingual Organizations

1,119
views

Published on

Dr. Kathi Bailey, professor of applied linguistics at the Monterey Institute of International Studies, presents at the 2011 Language Teaching Research Colloquium in Ann Arbor, MI.

Dr. Kathi Bailey, professor of applied linguistics at the Monterey Institute of International Studies, presents at the 2011 Language Teaching Research Colloquium in Ann Arbor, MI.

Published in: Education, Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,119
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
22
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Workforce Language Assessment in the 21 st Century Knowledge Economy: A Quality Management Perspective
  • 2. Presenters/Authors & Discussant
    • Nick Saville and Mike Milanovic
    • University of Cambridge ESOL Examinations
    • Ryan Damerow, Courtney Pahl, & Kathi Bailey
    • Monterey Institute of International Studies & TIRF
    • Mary Ann Christison
    • University of Utah
    • Discussant: Jean Turner
    • Monterey Institute of International Studies
  • 3. Surveying English Language Assessment Practices in International Plurilingual Organizations
    • Ryan Damerow (TIRF)
    • Courtney Pahl
    • Monterey Institute of International Studies
    • Kathi Bailey (MIIS & TIRF)
  • 4. TIRF
    • The International Research Foundation
    • for English Language Education
    • Promoting research and best practices to improve the use of English in the 21 st century global knowledge economy
  • 5. TIRF
    • Survey based in part on
    • research commissioned
    • and published by TIRF
    • in 2009 – available as a
    • free downloadable PDF
    • at TIRF’s website:
    • www.tirfonline.org
  • 6. Sampling and Respondents
    • Questionnaire designed and distributed with
    • SurveyMonkey (convenience sampling)
    • Monterey Institute alumni
    • Anaheim University alumni and students
    • International Chambers of Commerce
    • Global Finance Association
    • English Career magazine (Taiwan)
    • Individuals working in international organizations
    • N = 243 responses to date
  • 7. Organizational Sectors Represented Indicate the sector that best describes your organization. Please check only one.
  • 8. Organizational Sectors Represented
    • The “ Other” category Included the following:
    • International NGOs, government & diplomacy, law and legal practice, technology, arms control, grant making, maritime activities, translation & interpretation
    • And some responses that we thought were already covered by our existing categories: automobile manufacturing, accounting, and telecommunications
  • 9. In what countries does your organization currently operate?
    • Afghanistan, Algeria, Angola, Argentina, Australia, Austria, Azerbaijan, Bahrain, Bangladesh, Belgium, Belize, Benin, Bolivia, Brazil, Burkina Faso, Burundi, Camaroon, Cambodia, Canada, Central African Republic, Chile, China, Colombia, Costa Rica, Czech Republic, Denmark, Dominican Republic, Ecuador, Egypt, El Salvador, Estonia, Ethiopia, Finland, France, Gabon, Ghana, Germany, Georgia, Guatemala, Haiti, Holland, Honduras, India, Indonesia, Iran, Iraq, Ireland, Israel, Italy, Japan, Jordan, Kazakhstan,
  • 10. In what countries does your organization currently operate?
    • Kenya, Korea, Kosovo, Kyrgystan, Latvia, Lebanon, Liberia, Libya, Lithuania, Madagascar, Malaysia, Mali, Mexico, Morocco, Mozambique, Namibia, Nepal, New Zealand, Nicaragua, Nigeria, Norway, Oman, Panama, Peru, Philippines, Poland, Portugal, Russia, Rwanda, Saudi Arabia, Senegal, Serbia, Singapore, South Africa, Spain, Sri Lanka, Sudan, Sweden, Taiwan, Tanzania, Thailand, Togo, Turkmenistan, UAE, Uganda, UK, Ukraine, USA, Venezuela, Vietnam, Yemen, & Zambia.
  • 11. In what countries does your organization currently operate?
    • “ Too many to name”
    • “ Worldwide”
    • “ Globally”
    • “ Every country of the Caribbean”
    • “ In 32/ 36/ over 40/ over 60/ 72/ 80/ 87/ 90/ 120/ over 160/ 167/ over 190 countries”
  • 12. Languages used to communicate within your organization
    • Arabic
    • Cantonese
    • Dutch
    • English
    • French
    • German
    • Italian
    • Japanese
    • Korean
    • Mandarin
    • Portuguese
    • Russian
    • Spanish
    • Other (please specify)
  • 13. What is the primary language used by your employees in those locations to communicate within your international organization?
  • 14. Other Languages Reported
    • Albanian, Amharic, Armenian, Azerbaijani, Bahasa
    • Melu, Bangla, Catalan, Creole, Czech, Danish, Dari,
    • Estonian, Farsi, Finnish, Georgian, Gujarati,
    • Hebrew, Hindi, Hokkien, Khmer, Kirundi,
    • Kinyarwanda, Macedonian, Malay, Marahti,
    • Melanesian Pidgin, Mongolian, Norwegian,
    • Pashtu, Polish, Serbo-Croation, Slovak, Slovenian,
    • Swahili, Swedish, Tamil, Turkish, Ukrainian, Urdu,
    • Vietnamese, & African and Indian languages
  • 15. Languages used to communicate with clients
    • Arabic
    • Cantonese
    • Dutch
    • English
    • French
    • German
    • Italian
    • Japanese
    • Korean
    • Mandarin
    • Portuguese
    • Russian
    • Spanish
    • Other (please specify)
  • 16. What is the primary language used by your employees in those locations to communicate with clients of your international organization?
  • 17. Other Languages Used with Clients
    • All of the “other” languages listed above and also Bulgarian, Hungarian, Malagasy, Malayam, Kannada, Kazakh, Kyrgyz, Lozi, Quechua, Tajik, Telegu, and “too many to name.”
  • 18. Research Question #1
    • 1. What are the necessary English language skills individuals must possess to be successful as competent employees and leaders in international plurilingual organizations in the 21 st century knowledge-based economy?
  • 19. English Speaking & Listening Skills
    • Interacting in face-to-face meetings 88.5%
    • Telephone conversations 88.1%
    • Listening during face-to-face meetings 87.2%
    • Listening to presentations 79.8%
    • Giving presentations 78.2%
    • Listening to teleconferences 73.3%
    • Running face-to-face meetings 72.4%
    • International travel 67.9%
  • 20. English Speaking & Listening Skills
    • Presenting during teleconferences 66.3%
    • Negotiations (for example, a contract) 65.8%
    • Socializing 63.8%
    • Talking with people at
    • exhibitions & tradeshows 53.9%
    • Conducting interviews 48.1%
    • Being interviewed 47.7%
    • Other (please specify) 15.2%
  • 21. English Writing Skills
    • Writing e-mails 95.9%
    • Writing correspondence 81.5%
    • Writing reports 75.3%
    • Writing proposals 69.1%
    • Writing memos 60.9%
    • Writing newsletters 37.0%
    • Writing manuals 35.8%
    • Other (please specify) 14.4%
  • 22. English Reading Skills
    • Reading e-mails 97.1%
    • Reading correspondence 86.4%
    • Reading reports 86.4%
    • Reading proposals 73.7%
    • Reading memos 73.3%
    • Reading manuals 65.8%
    • Reading newsletters 60.5%
    • Other 9.5%
  • 23. Research Question #2
    • What assessment procedures and instruments are used to determine whether such individuals possess
    • the necessary skills
    • to accomplish those roles
    • successfully?
  • 24. Do you have English language requirements for any of your employees?
    • No 9.5%
    • Yes 81.0%
    • Don't know 9.5%
  • 25. If you have English language requirements, do they vary by position?
    • No 25.3%
    • Yes 59.1%
    • Don't know 15.6%
  • 26. Are bilingual/plurilingual applicants hired over monolingual applicants with equal skill sets (other than their language abilities)?
    • No 7.8%
    • Yes 70.0%
    • Don't know 22.2%
  • 27. Are bilingual/plurilingual employees paid more than monolingual employees in the same positions?
    • No 41.2%
    • Yes 17.3%
    • Don't know 41.6%
  • 28. Do you test the English proficiency of any of your employees using any commercially developed test(s) ?
    • No 60.9%
    • Yes 13.2% (n = 32)
    • Don't know 26.3%
    • If yes, what tests…? (TOEIC was identified 6 times and TOEFL four times; UN Proficiency Exam twice; government OPI, and UniCredit Group were each mentioned once.)
  • 29. Do you test the English proficiency of your employees using any in-house procedure(s) ?
    • No 37.4%
    • Yes 36.6% (n = 89)
    • Don't know 27.6%
    • If yes, what procedures are used?
    • In-house interviews,
    • some writing samples,
    • speech recognition tool (1),
    • & translating documents (1)
  • 30. Do you test the English proficiency of your employees using any in-house procedure(s) ?
    • “ There is no formal procedure, but the application and interview process can weed out potential contractors whose English abilities are weak.”
    • “ We don't use any official test, but through the interview process and writing samples we are able to assess language ability.”
    • “ Financial test in English.”
  • 31. Do you test the English proficiency of your employees using any in-house procedure(s) ?
    • “ The HR department develops language tests for English (or the relevant language for the project) depending on the position.”
    • “ No specific procedures except conversation.”
    • “ We talk to them. If they
    • speak English well enough
    • to communicate with
    • management, they pass.”
  • 32. If you use any in-house testing procedure(s), what training is given to the test administrators?
    • There were 75 responses to this item. Among them
    • 26 said “None”
    • 14 said they didn’t know or weren’t sure.
    • 5 said “Not applicable.”
  • 33. If you use any in-house testing procedure(s), what training is given to the test administrators?
    • “ None; testing would be very simple.”
    • “ The interviewer is the experienced one.”
    • “ The administrators do the tests based on their experience in the area tested.”
    • “ The interviewer must speak English fluently.”
    • “ The Japanese manager simply uses his own personal criteria of what defines ‘good English.’”
    • “ Most are PhDs in linguistics.”
  • 34. Think about some employees in your organization who are bilingual or plurilingual. Have the bilingual or plurilingual skills of those employees had a positive impact on your organization?
    • No 4.5%
    • Yes 82.3%
    • Don't know 13.2%
  • 35. Think about some employees in your organization who are NOT bilingual or plurilingual. Has the lack of bilingual or plurilingual skills of those employees had a negative impact on your organization?
    • No 40.7%
    • Yes 27.2%
    • Don't know 32.1%
  • 36. Does your organization support employees in the improvement of their second or foreign language skills?
    • No 28.0%
    • Yes 59.3%
    • Don't know 14.0%
  • 37. Does your organization calculate return on investment for language development?
    • No 54.5%
    • Yes 7.1%
    • Don't know 39.6%
  • 38. If you answered YES to the previous question, please check all the methods for calculating return on investment that apply:
    • Change in quality of work 71.4%
    • Change in time needed to complete tasks 71.4%
    • Change in quantity or output of work 42.9%
    • Change documented by experts (e.g.,
    • supervisors, personnel managers, etc.) 42.9%
    • Change in costs 35.7%
    • Other (please specify) 21.4%
  • 39. Closing Comments
    • In Summary
    • 70% of the respondents said that bilingual or plurilingual applicants get hiring preference.
    • 81% said there are English requirements for employees.
    • Only 13.2 % of the respondents report using commercially developed language tests.
  • 40. Closing Comments
    • 59.3% of the respondents reported that their organization does provide language development support for employees
    • BUT
    • Only 7.1% said that their organization calculates the Return on
    • Investment for such training.
  • 41. Closing Comments
    • About 95% of the respondents report that
    • English is used
    • as a language of communication within the organization
    • as a language of communication with clients
  • 42. Closing Comments
    • Over one-third report that their organization uses some in-house procedure(s) to assess employees’ English proficiency.
    • Of the 75 respondents who report using some kind of in-house procedure, at least one-third (n = 26) reported no training is provided for test administrators.
  • 43. Closing Comments
    • Over 75% say English is the primary language
    • used to communicate within the organization and
    • about two-thirds say English is the primary
    • language used to communicate with clients.
    • Relatively few organizations in this
    • small data set report using TOEIC
    • or other professionally developed
    • language tests.
  • 44. Closing Comments
    • Following Nick’s and Mike’s presentation about Quality Management Systems (QMS) in language assessment, it seems that little attention is paid to the quality of language assessment by many of these international organizations.
    • Many business schools use the QMS framework – which may be a reason for language testers to use it as a talking point in business contexts.
  • 45. Closing Comments
    • Limitations to the Study:
    • 1. Qualitative data not yet fully analyzed
    • More data forthcoming
    • Convenience sampling
    • 4. Regional and sector distribution uneven
    • 5. What to do with the “don’t know” answers?
    • 6. Exclude responses from language-based groups?
    • 7. Individual respondents vs. organizations as the source of data -- What to do with multiple responses from the same organization?
  • 46. Closing Comments
    • For reference lists on various language
    • assessment topics and one on language in
    • business, industry and the professions,
    • please visit www.tirfonline.org
    • Click on “Access to Resources”
    • and then on “References” (downloadable Word
    • docs in APA format – please share with others).