Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Assessment Issues in Workplace ESL Instructional Programs

1,482 views

Published on

Dr. MaryAnn Christison, professor at the University of Utah, presents at the 2011 Language Teaching Research Colloquium in Ann Arbor, MI.

Published in: Education, Technology
  • Be the first to comment

Assessment Issues in Workplace ESL Instructional Programs

  1. 1. Symposium Workforce Assessment in the 21 st Century Knowledge Economy: A Quality Management Perspective June 25, 2011
  2. 2. Assessment Issues in Workplace ESL Instructional Programs <ul><li>MaryAnn Christison </li></ul><ul><li>Dept. of Linguistics/Urban Institute for Teacher Education </li></ul><ul><li>University of Utah </li></ul><ul><li>TIRF Board of Trustees </li></ul>
  3. 3. <ul><li>The International Research Foundation </li></ul><ul><li>for English Language Education </li></ul><ul><li>Promoting research and best practices to improve the use of English in the 21 st century global knowledge economy </li></ul><ul><li>www.tirfonline.org </li></ul>
  4. 4. A Backdrop for the Paper <ul><li>Personal background </li></ul><ul><ul><li>Instructional design, teacher education, and classroom-based assessment and research </li></ul></ul><ul><ul><li>Experiences in workplace ESL include </li></ul></ul><ul><ul><ul><li>identifying what type of language instruction is needed by a given company (via a needs assessment), </li></ul></ul></ul><ul><ul><ul><li>designing language instruction with goals that meet domain specific workplace needs, and </li></ul></ul></ul><ul><ul><ul><li>determining whether the language instruction is meeting the goals. </li></ul></ul></ul><ul><ul><li>Issues identified derive from practical experiences in workplace language contexts </li></ul></ul>
  5. 5. Five Issues <ul><li>The issues </li></ul><ul><li>are not meant to comprise an exhaustive list, </li></ul><ul><li>are based on my own and others’ experiences, and </li></ul><ul><li>draw on data from my own research notes. </li></ul>
  6. 6. The Five Issues <ul><ul><li>1. Communicating to laypersons about the complex nature of language acquisition </li></ul></ul><ul><ul><li>2. Moderating learner and employer expectations about language </li></ul></ul><ul><ul><li>3. Connecting language training to workers’ on the job performances </li></ul></ul><ul><ul><li>4. Using assessment data in decision-making </li></ul></ul><ul><ul><li>5. Understanding the research available on workplace language instruction </li></ul></ul>
  7. 7. Issue #1 <ul><li>Communicating to laypersons about the complex nature of language acquisition </li></ul><ul><li>Example Quotes </li></ul><ul><ul><li>“ So, how long do you think it will take?” </li></ul></ul><ul><ul><li>“ Okay, so the good news is that I’ve been able to get funding for the next six weeks [so that these employees to develop English language proficiency].” </li></ul></ul><ul><ul><li>“ I’m really under the gun here. Of course, you know that I really support this program, but we just don’t understand why is it taking so long?” </li></ul></ul><ul><ul><li>“ My son learned Italian in one summer, so I just don’t understand why can’t some of these people cannot speak English after two years?” </li></ul></ul><ul><ul><li>“ I’m sorry to have to be the one to tell you this, but we have to cancel the program; it is too expensive and taking too long.” </li></ul></ul>
  8. 8. Issue #1 <ul><li>Difficulties in communicating to laypersons about the complex nature of language acquisition </li></ul><ul><ul><li>Defining language constructs </li></ul></ul><ul><ul><li>Language vs. other competencies </li></ul></ul><ul><ul><li>Time and SLA </li></ul></ul><ul><ul><li>Time and money </li></ul></ul><ul><ul><li>Myths about language </li></ul></ul><ul><ul><li>Questions that we can answer about SLA are often not questions laypersons are asking—there’s a cultural gap. </li></ul></ul>
  9. 9. Issue #2 <ul><li>The difficulty in moderating employer and employee expectations </li></ul><ul><li>My first encounter </li></ul><ul><ul><ul><li>Diminishing U.S. federal funding for workplace literacy in the mid to late 90s underscored the importance of accountability for workplace language educators. </li></ul></ul></ul><ul><ul><ul><li>Sponsoring companies and businesses had different educational expectations for language programs than educational institutions and governmental agencies. Businesses and companies often compared language learning to the acquisition of non-language skills and wanted similar results relative to time. </li></ul></ul></ul>
  10. 10. Issue #2 <ul><li>The difficulty in moderating employer and employee expectations </li></ul><ul><ul><li>Quotes from a hotel manager </li></ul></ul><ul><ul><ul><li>“I know . . ., but we had that a . . that a . . .consultant come in last week for a workshop--six hours--, and now I think all of the employees can now use the software.” </li></ul></ul></ul><ul><ul><ul><li>“I think huh . . Mr. [X] still cannot enter this information and run this program, but that’s English.” </li></ul></ul></ul>
  11. 11. Issue #2 <ul><li>Companies are often not aware of their employees’ (and potential employees’) expectations about assessment practices. </li></ul><ul><ul><li>Quotes from potential flight attendants in an online chat room </li></ul></ul><ul><ul><li>“ Does anyone know how they [will] evaluate us?” </li></ul></ul><ul><ul><li>“ Do you know if we will we have to read passenger instructions?” </li></ul></ul><ul><ul><li>“ Is there a spontaneous interview?” </li></ul></ul><ul><ul><li>“ Someone said there was a simulation or such and how can we prepare for that? Would they tell us, do you think?” </li></ul></ul><ul><ul><li>“ I’d like to talk to current crews, so I maybe can compare my language with theirs.” </li></ul></ul>
  12. 12. Issue #2 <ul><ul><li>Companies can benefit from language assessment experts. Companies can improve skills in the administration of assessments and communicating about these assessments to potential employees. </li></ul></ul>
  13. 13. Issue #3 <ul><li>Connecting language training to workers’ performances on the job </li></ul><ul><ul><li>The beginnings </li></ul></ul><ul><ul><ul><li>General English on-site </li></ul></ul></ul><ul><ul><ul><li>General Business English </li></ul></ul></ul><ul><ul><ul><li>Domain specific English (frequency vs. importance) </li></ul></ul></ul><ul><ul><li>Language tasks related to workplace tasks </li></ul></ul><ul><ul><li>Quotes from Call Centre online chat rooms </li></ul></ul><ul><ul><ul><li>“ The telephone role plays [were] very good for us.” </li></ul></ul></ul><ul><ul><ul><li>“ The interactive tasks are good I think, so we must talk to others and make good impressions for our customers.” </li></ul></ul></ul>
  14. 14. Issue #4 <ul><ul><li>Using language assessment data in decision making. </li></ul></ul><ul><ul><ul><li>Speaker A: “So, why don’t we talk about the results, OK? . . . Hm. . . I got your analysis yesterday. You know, a . . we might not be so happy with some things. Ah. . Hm. . Ok, so, for example, I think you know Mr. [X] and how good he is, but it seems that according to your, ya know test, hm . . . he didn’t do so well. OK. . . . a . . .So, it would be difficult to give him this hm . . promotion, and ah . . . Well, this needs to happen. I think we need to discuss a . . Hm. . . how to adapt these scores so that this happens. . . .” </li></ul></ul></ul><ul><ul><ul><li>Speaker B (consultant):“Ah. . . I don’t think I’m really clear on what you want me to do” </li></ul></ul></ul>
  15. 15. Issue #5 <ul><ul><li>Understanding the research available on workplace instruction. </li></ul></ul><ul><ul><li>Call Centre X reported a lack of success in using commercially available general business English tests within this industry. English tests provide a score for proficiency and do not break down the result in a diagnostic profile. </li></ul></ul><ul><ul><li>Call centres reported that the general business tests were not valid indicators of what they were testing for. The scores did not carry meaning in terms of performance on the job. </li></ul></ul><ul><ul><li>Neither the cost nor the long turn-around time met their needs. </li></ul></ul>
  16. 16. Five Issues <ul><ul><li>Communicating to laypersons about the complex nature of language acquisition </li></ul></ul><ul><ul><li>Moderating learner and employer expectations about language </li></ul></ul><ul><ul><li>Connecting language training to workers’ on the job performances </li></ul></ul><ul><ul><li>Using assessment data in decision-making </li></ul></ul><ul><ul><li>Understanding the research available on workplace language instruction </li></ul></ul><ul><ul><li>contact </li></ul></ul><ul><ul><li>[email_address] </li></ul></ul>

×