Successfully reported this slideshow.
Assessing	
  Language	
  	
  
Using	
  Computer	
  
Technology	
  
Dr. Volker Hegelheimer
Iowa State University
Agenda	
  
• Acknowledgements	
  	
  
• Context	
  	
  
• Audience	
  	
  
• Outline	
  of	
  Webinar	
  Goals	
  
• Langu...
Acknowledgements:	
  	
  
Ahmet	
  Dursun	
  
	
  	
  	
  	
  	
  Erin	
  Todey	
  
	
  	
  	
  	
  	
  Hye-­‐Won	
  Lee	
...
Context	
  
• Iowa	
  State	
  University	
  
• Teaching	
  and	
  Tes<ng	
  Context	
  
• Technology	
  for	
  Use	
  wit...
Current	
  loca<on	
  
Webinar	
  Polls
• When	
  you	
  see	
  a	
  slide	
  with	
  “Poll”	
  in	
  the	
  <tle,	
  
please	
  click	
  on	
  t...
Poll:	
  Webinar	
  Par<cipants?	
  
Ques<on	
  #1:	
  What	
  level	
  do	
  you	
  teach?	
  
A.	
  Elementary	
  
B.	
 ...
Teaching	
  &	
  Tes<ng	
  Context	
  at	
  ISU	
  
•  Faculty	
  	
  
•  8	
  Tenure	
  track,	
  1	
  post	
  doc,	
  2	...
Poll:	
  Webinar	
  Par<cipants?	
  
Ques<on	
  #2:	
  What	
  language	
  do	
  you	
  teach?	
  
A.	
  English	
  (as	
 ...
Languages	
  Taught	
  at	
  ISU	
  
•  My	
  Context:	
  ESL	
  
•  Other	
  Languages	
  
o  Chinese	
  
o  Spanish	
  
...
Poll:	
  Technology	
  Availability?	
  
Ques<on	
  #3:	
  What	
  is	
  the	
  availability	
  of	
  technology	
  
(Inte...
312	
  Ross	
  Hall	
  
Poll:	
  Technology	
  Used?	
  
Ques<on	
  #4:	
  Which	
  of	
  these	
  technologies	
  have	
  
you	
  used	
  in	
  l...
Technology	
  Used	
  at	
  ISU	
  
•  Google	
  Docs	
  
•  Variety	
  of	
  internet	
  sources	
  used	
  in	
  classes...
Poll:	
  Technology	
  Used	
  for	
  
Crea<ng	
  Assessments?	
  
Ques<on	
  #5:	
  What	
  experience	
  do	
  you	
  ha...
Poll:	
  What	
  Assessments?	
  
Ques<on	
  #6:	
  What	
  language	
  assessments	
  are	
  you	
  
crea<ng?	
  
	
  A.	...
Chat	
  Box:	
  Computer	
  Technology	
  	
  	
  
Experience	
  
• In	
  the	
  chat	
  box,	
  please	
  briefly	
  share...
Webinar	
  Goals	
  
•  Introduce	
  computer	
  technologies	
  useful	
  for	
  
assessment	
  of	
  the	
  four	
  skil...
Outline	
  of	
  the	
  Webinar	
  
Skill	
  being	
  
assessed	
  
Computer	
  technologies	
  
introduced	
  
Reading	
 ...
Fundamental	
  Concepts	
  in	
  
Assessment	
  
•  Validity	
  
o  Is	
  the	
  test	
  measuring	
  what	
  we	
  want	
...
“Construct?”	
  What	
  Construct?	
  	
  
•  The	
  "ability"	
  or	
  skills	
  that	
  we	
  want	
  to	
  measure	
  
...
Assessment	
  Types	
  
Forma<ve	
  Assessment:	
  	
  
Occurs	
  while	
  students	
  are	
  in	
  
the	
  learning	
  pr...
Ques<on	
  Time	
  -­‐	
  3	
  minutes!	
  
Please	
  post	
  ques<ons	
  in	
  the	
  chat	
  box	
  
Computer	
  Assisted	
  Language	
  
Tes<ng	
  (CALT)	
  	
  
•  Promises	
  
•  Convenience	
  	
  
•  Consistency	
  	
 ...
Computer	
  Assisted	
  Language	
  
Tes<ng	
  	
  
• Challenges	
  
•  Different	
  test	
  performance	
  
•  New	
  task...
CALT	
  –	
  Assessing	
  Reading	
  	
  
Interac<ve/Non-­‐interac<ve	
  
e-­‐books	
  
(iBooks)	
  
Mul<media	
  Texts	
 ...
CALT	
  –	
  Assessing	
  Reading	
  	
  
Google	
  Forms	
  
RESOURCE	
  
• Free	
  and	
  easy	
  to	
  use	
  
• Mul<pl...
Response	
  capture	
  
CALT	
  –	
  Assessing	
  Reading	
  	
  
Google	
  Forms:	
  	
  
Sample	
  Reading	
  Ac<vity	
  
RESOURCE	
  
Click	
  ...
Assessing	
  Reading:	
  Challenges	
  
•  The	
  process	
  of	
  interac<ng	
  with	
  the	
  text	
  is	
  dynamic,	
  ...
Assessing	
  Reading:	
  Promises	
  
• Convenience	
  
• Mul<media	
  	
  
• Consistency	
  	
  
CALT	
  –	
  Assessing	
  Reading:	
  
Ques<ons?	
  
Please	
  post	
  ques<ons	
  you	
  may	
  have	
  about	
  
assessi...
CALT	
  –	
  Assessing	
  Listening
• Audio	
  vs.	
  video-­‐based	
  listening	
  assessment	
  
• Technologies	
  for	
...
Audio	
  vs.	
  video-­‐based	
  listening	
  	
  	
  
assessment	
  
1.	
  The	
  effect	
  of	
  visuals	
  on	
  L2	
  l...
CALT	
  –	
  Assessing	
  Listening	
  
Sample	
  implementa<on	
  of	
  “Poll	
  Everywhere”	
  	
  
h_ps://www.youtube.c...
Assessing	
  Listening:	
  Challenges	
  
1.  Listening	
  is	
  widely	
  used	
  but	
  under-­‐researched	
  
(Harding,...
Assessing	
  Listening:	
  Promises	
  
1.  Ubiquitous	
  access	
  to	
  authen<c	
  mul<modal	
  
listening	
  texts	
  ...
Ques<ons?	
  
What	
  ques<ons	
  do	
  you	
  have	
  about	
  assessing	
  
listening	
  using	
  computer	
  technologi...
CALT	
  –	
  Assessing	
  Speaking	
  
Storing	
  speech	
  
(VoiceThread,	
  Web	
  
poriolios,	
  Podcasts)	
  
Evalua<n...
Poll:	
  VoiceThread?
• Ques<on	
  #7:	
  Have	
  you	
  used	
  VoiceThread?	
  	
  
	
  	
  	
  	
  A.	
  Yes	
  
	
  	
...
CALT	
  –	
  Assessing	
  Speaking	
  
VoiceThread	
  
RESOURCE	
  
•  A	
  mul<-­‐media	
  tool	
  that	
  allows	
  you	...
Poll:	
  Google	
  Voice	
  Search?
• Ques<on	
  #8:	
  Have	
  you	
  used	
  Google	
  Voice	
  
Search?	
  	
  
	
  	
 ...
CALT	
  –	
  Assessing	
  Speaking	
  
Google	
  Voice	
  Search	
  
RESOURCE	
  
•  Speaking,	
  instead	
  of	
  typing,...
CALT	
  –	
  Assessing	
  Speaking	
  
Google	
  Voice	
  Search:	
  
Sample	
  Pronuncia<on	
  Assessment	
  
RESOURCE	
 ...
Speech-enabled Web Search
Speaking	
  Assessment:	
  Challenges	
  
•  "The	
  ability	
  to	
  speak	
  a	
  foreign	
  language	
  is	
  without	
...
Speaking	
  Assessment:	
  Promises	
  
•  Improved	
  and	
  Ubiquitous	
  Automa<c	
  Speech	
  
Recogni<on	
  
•  Brows...
Ques<ons?	
  
What	
  ques<ons	
  do	
  you	
  have	
  about	
  assessing	
  
speaking	
  using	
  computer	
  technologie...
CALT	
  –	
  Assessing	
  Wri<ng	
  
RESOURCES	
  
Internet	
  Technologies	
  
(Google	
  Docs,	
  Wiki,	
  Blog,	
  e-­‐...
Poll:	
  Automated	
  Wri<ng	
  
Evalua<on	
  (AWE)?
• Ques<on	
  #9:	
  Have	
  you	
  used	
  an	
  AWE	
  system?	
  	
...
AWE	
  Systems	
  (Selec<on)	
  	
  
• Criterion	
  
o  hbps://www.youtube.com/watch?v=PMgZvKNXfzs	
  	
  
• MyAccess	
  
...
Poll:	
  Google	
  Docs?
• Ques<on	
  #10:	
  Have	
  you	
  used	
  Google	
  Docs?	
  	
  
	
  	
  	
  	
  A.	
  Yes	
  ...
CALT	
  –	
  Assessing	
  Wri<ng	
  
Google	
  Docs	
  
RESOURCE	
  
• Plaiorm	
  for	
  collabora<ve	
  wri<ng	
  
• Uses...
Poll:	
  MS	
  Word?
• Ques<on	
  #11:	
  Do	
  you	
  know	
  about	
  the	
  following	
  
features	
  in	
  MS	
  Word?...
CALT	
  –	
  Assessing	
  Wri<ng	
  
Microso`	
  Word	
  
RESOURCE	
  
• Useful	
  features	
  
o  Track	
  Changes	
  
o ...
Dran	
  1	
  –	
  Default	
  seong	
  
Dran	
  1	
  –	
  Language	
  set	
  to	
  German	
  
Dran	
  2	
  
Dran	
  1	
  vs.	
  Final	
  Dran	
  
Challenges	
  with	
  wri<ng	
  assessment	
  
• Teacher	
  workload	
  (Ferris	
  &	
  Hedgcock,	
  2005)	
  
• Teacher	
...
Wri<ng	
  Assessment:	
  Promises	
  
•  Increasingly	
  powerful	
  automated	
  wri<ng	
  evalua<on	
  
tools	
  	
  
• ...
Questions?
What	
  ques<ons	
  do	
  you	
  have	
  about	
  assessing	
  
wri<ng	
  using	
  computer	
  technologies?	
 ...
Overall Questions?
What	
  overall	
  ques<ons	
  do	
  you	
  have	
  about	
  the	
  
content	
  we	
  covered	
  in	
  ...
Thank	
  you	
  
Contact	
  informa<on:	
  	
  
Volker	
  Hegelheimer	
  –	
  volkerh@iastate.edu	
  
Image	
  Credits	
  
"Apple	
  in	
  Educa<on".	
  (2013).	
  Apple.	
  h_p://www.apple.com/educa<on/labs/	
  
Jiang,	
  Y...
Assessing Language Using Computer Technology with Dr. Dr. Volker H. Hegelheimer
Assessing Language Using Computer Technology with Dr. Dr. Volker H. Hegelheimer
Assessing Language Using Computer Technology with Dr. Dr. Volker H. Hegelheimer
Assessing Language Using Computer Technology with Dr. Dr. Volker H. Hegelheimer
Assessing Language Using Computer Technology with Dr. Dr. Volker H. Hegelheimer
Assessing Language Using Computer Technology with Dr. Dr. Volker H. Hegelheimer
Assessing Language Using Computer Technology with Dr. Dr. Volker H. Hegelheimer
Assessing Language Using Computer Technology with Dr. Dr. Volker H. Hegelheimer
Assessing Language Using Computer Technology with Dr. Dr. Volker H. Hegelheimer
Upcoming SlideShare
Loading in …5
×

Assessing Language Using Computer Technology with Dr. Dr. Volker H. Hegelheimer

909 views

Published on

From the CALPER/LARC Testing and Assessment Webinar Series
Download the handouts and ppt: https://larc.sdsu.edu/archived-events/
View the recording: http://vimeo.com/63784749

Presentation Description:
While the use technology in language teaching classrooms has become increasingly more common, fewer teachers are comfortable using technology as they assess leaner progress. In this webinar on assessing language using computer technology, I will outline ways in which technology can be used to help inform the assessment of language learners.

Webinar Date: April 10, 2013

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

Assessing Language Using Computer Technology with Dr. Dr. Volker H. Hegelheimer

  1. 1. Assessing  Language     Using  Computer   Technology   Dr. Volker Hegelheimer Iowa State University
  2. 2. Agenda   • Acknowledgements     • Context     • Audience     • Outline  of  Webinar  Goals   • Language  Tes<ng  Terminology/Concepts   • Technology   • General  approach   o  Demonstrate,  then  respond  to  ques<ons    
  3. 3. Acknowledgements:     Ahmet  Dursun            Erin  Todey            Hye-­‐Won  Lee       Ruslan  Suvorov  
  4. 4. Context   • Iowa  State  University   • Teaching  and  Tes<ng  Context   • Technology  for  Use  with  Assessing  4  Skills     • Plug!  Mark  your  calendars  (11th  Annual   Technology  for  Second  Language  Learning   conference  –  TSLL,  Oct.  18-­‐19,  2013;  Technology   and  Teaching  Wri<ng  for  Academic  Disciplines)   –  with  John  Flowerdew!    
  5. 5. Current  loca<on  
  6. 6. Webinar  Polls • When  you  see  a  slide  with  “Poll”  in  the  <tle,   please  click  on  the  poll  bu_on  under  your  name   on  the  le`  hand  side  of  the  screen.     • Let’s  try  that  next  
  7. 7. Poll:  Webinar  Par<cipants?   Ques<on  #1:  What  level  do  you  teach?   A.  Elementary   B.  Middle  School/Junior  High   C.  High  School   D.  Post-­‐secondary  (universi<es,  community   colleges)  
  8. 8. Teaching  &  Tes<ng  Context  at  ISU   •  Faculty     •  8  Tenure  track,  1  post  doc,  2  lecturers,  55  graduate   students  (38  PhD,  17  MA)   •  Pre-­‐university   •  Intensive  English  Program   o  Reading,  Wri<ng,  Oral  Communica<on,  Grammar   •  University   •  ESL  Wri<ng  Courses     •  ESL  Listening  and  Reading  Strategies  Courses   •  Interna<onal  TA  Training   •  "First  Year  Composi<on”  
  9. 9. Poll:  Webinar  Par<cipants?   Ques<on  #2:  What  language  do  you  teach?   A.  English  (as  a  second/foreign  language)   B.  Spanish   C.  French   D.  Mandarin  Chinese   •  Please  post  other  languages  that  you  teach  in   the  chat  box.  
  10. 10. Languages  Taught  at  ISU   •  My  Context:  ESL   •  Other  Languages   o  Chinese   o  Spanish   o  French   o  Arabic   o  German   o  Russian  
  11. 11. Poll:  Technology  Availability?   Ques<on  #3:  What  is  the  availability  of  technology   (Internet,  computers,  etc.)  in  your  teaching   context?   A.  1  to  1  Computer-­‐Student   B.  Computer  Lab(s)     C.  Limited  Technology   D.  No  technology  
  12. 12. 312  Ross  Hall  
  13. 13. Poll:  Technology  Used?   Ques<on  #4:  Which  of  these  technologies  have   you  used  in  language  teaching?    Select  all  that   apply.     A.  MS  Word     B.  Google  Docs   C.  e-­‐Poriolios   D.  Automated  Wri<ng  Evalua<on   •  Please  post  any  other  technologies  in  the  chat   box.  
  14. 14. Technology  Used  at  ISU   •  Google  Docs   •  Variety  of  internet  sources  used  in  classes  and  in   student  research   o  Youtube   o  VoiceThread   o  Audacity   •  Assessment  context   •  English  Placement  Test     •  ~  850  non-­‐na<ve  English  speaking  students  annually        
  15. 15. Poll:  Technology  Used  for   Crea<ng  Assessments?   Ques<on  #5:  What  experience  do  you  have  with   crea<ng  language  assessments?    A.  Not  much.  This  is  pre_y  new  to  me.    B.  Some.    C.  Quite  a  bit.    D.  I'm  an  expert!  
  16. 16. Poll:  What  Assessments?   Ques<on  #6:  What  language  assessments  are  you   crea<ng?    A.  Mostly  Classroom  Assessments    B.  Program  Assessments    C.  Medium  Stakes  Assessments    D.  High  Stakes  Assessments  
  17. 17. Chat  Box:  Computer  Technology       Experience   • In  the  chat  box,  please  briefly  share  your   experience(s)  with  using  computer  technologies   in  the  classroom  for:   1.  Teaching   2.  Assessment   One  sentence,  2  min.  ac<vity    
  18. 18. Webinar  Goals   •  Introduce  computer  technologies  useful  for   assessment  of  the  four  skills   •  Provide  specific  examples  of  technologies  used   for  language  assessment   •  Discuss  the  promises  and  challenges  of  using   technology  for  assessing  different  skills  
  19. 19. Outline  of  the  Webinar   Skill  being   assessed   Computer  technologies   introduced   Reading   •  Google  Forms   Listening   •  PollEverywhere   •  Google  Forms   Speaking   •  Google  Voice  Search   Wri<ng   •  Automated  Wri<ng   Evalua<on   •  Microso`  Word   •  Google  Docs   • Technology   introduc<on     • Experience   • Promises  and   challenges  for   language  tes<ng  
  20. 20. Fundamental  Concepts  in   Assessment   •  Validity   o  Is  the  test  measuring  what  we  want  it  to  measure?   •  Reliability   o  Consistency  of  measurement   •  Authen<city   o  Similarity  of  test  tasks  to  real  life  tasks   •  Washback   o  Effect  of  the  test  on  teaching  &  learning  processes   Adapted  from:  Bachman,  L.F.,  &  Palmer,  A.S.  (1996).  Language  Tes*ng  in  Prac*ce.   Oxford:  Oxford  University  Press.    
  21. 21. “Construct?”  What  Construct?     •  The  "ability"  or  skills  that  we  want  to  measure   •  Example:  The  construct  of  wri<ng   o  What  are  we  measuring?     lexical  knowledge     grammar     textual  competence     accuracy     fluency     syntac<c  control     rhetorical  control   Adapted  from  Hamp-­‐Lyons,  L.  (1991).  "Scoring  procedures  for  ESL  contexts."  In  L.  Hamp-­‐Lyons  (ed.)  Assessing  Second  Language   Wri*ng  in  Academic  Contexts.  Norwood,  NJ:  Ablex,  241-­‐276.  
  22. 22. Assessment  Types   Forma<ve  Assessment:     Occurs  while  students  are  in   the  learning  process   Summa<ve  Assessment:   Occurs  at  the  end  of  a  program   or  unit  (i.e.  a`er  the  learning   process)   •  Diagnos<c  tests   •  Placement  tests   •  Progress  tests   •  End  of  unit  tests   •  Achievement  tests   •  Semester  final   •  State/na<onal  tests  
  23. 23. Ques<on  Time  -­‐  3  minutes!   Please  post  ques<ons  in  the  chat  box  
  24. 24. Computer  Assisted  Language   Tes<ng  (CALT)     •  Promises   •  Convenience     •  Consistency     •  Mul<media  capabili<es   •  Adaptable  input     •  Natural  language  processing   Adopted  from:  Chapelle,  C.  A.  &  Douglas,  D.  (2006).  Assessing  language  through   computer  technology.  Cambridge,  UK:  Cambridge  University  Press.  
  25. 25. Computer  Assisted  Language   Tes<ng     • Challenges   •  Different  test  performance   •  New  task  types   •  Limita<ons  due  to  adap<ve  item  selec<on   •  Inaccurate  automa<c  response  scoring   •  Compromised  security     •  Nega<ve  consequences     Adopted  from:  Chapelle,  C.  A.  &  Douglas,  D.  (2006).  Assessing  language  through   computer  technology.  Cambridge,  UK:  Cambridge  University  Press.  
  26. 26. CALT  –  Assessing  Reading     Interac<ve/Non-­‐interac<ve   e-­‐books   (iBooks)   Mul<media  Texts   Annotated    Web-­‐based  Texts       (Hypertext  Links)   RESOURCES   Google  Forms  
  27. 27. CALT  –  Assessing  Reading     Google  Forms   RESOURCE   • Free  and  easy  to  use   • Mul<ple-­‐ques<on  formats   • “Data”  collection   • Responses  are  automa<cally  sent  to  the   spreadsheet  
  28. 28. Response  capture  
  29. 29. CALT  –  Assessing  Reading     Google  Forms:     Sample  Reading  Ac<vity   RESOURCE   Click  on  the  link  to  view  the  reading  ac<vity:   h_ps://docs.google.com/forms/d/1qSo2F0uymbe8MKBSnnXaegFNeA6LUjnIaWpx4C6zB4A/viewform  
  30. 30. Assessing  Reading:  Challenges   •  The  process  of  interac<ng  with  the  text  is  dynamic,   variable  and  different  for  the  same  learners  on  the   same  text  at  different  <mes  (Alderson,  2006).       •  Background  or  prior  knowledge  may  play  a   significant  role  in  text  comprehension.   •  It  is  hard  to  match  test  takers'  reading  strategies   with  the  ones  they  need  to  develop/use  for  the  test   items.     •  Reading  encompasses  many  skills  that  complement   one  another  in  many  ways  but  assessing  these  skills   integra<vely  is  hard.  
  31. 31. Assessing  Reading:  Promises   • Convenience   • Mul<media     • Consistency    
  32. 32. CALT  –  Assessing  Reading:   Ques<ons?   Please  post  ques<ons  you  may  have  about   assessing  reading  using  computer  technologies  in   the  chat  box     3-­‐5  min.    
  33. 33. CALT  –  Assessing  Listening • Audio  vs.  video-­‐based  listening  assessment   • Technologies  for  assessing  L2  listening   • Challenges  with  listening  assessment  
  34. 34. Audio  vs.  video-­‐based  listening       assessment   1.  The  effect  of  visuals  on  L2  listening       comprehension  is  unclear   2.  Authen<city  vs.  prac<cal  constraints   3.  Context  vs.  content  visuals   4.  Note-­‐taking  during  listening  assessment  
  35. 35. CALT  –  Assessing  Listening   Sample  implementa<on  of  “Poll  Everywhere”     h_ps://www.youtube.com/watch?v=ih1ARCbIljE   Google  Forms   •     
  36. 36. Assessing  Listening:  Challenges   1.  Listening  is  widely  used  but  under-­‐researched   (Harding,  2012).   2.  Issues  with  the  defini<on  of  listening:   a.  Processing  of  auditory  informa<on   b.  Processing  of  auditory  AND  visual  informa<on   3.  Listening  is  mul<dimensional.   4.  Listening  comprehension  process  “disputed”.     5.  Classifica<on  of  the  types  of  listening.   6.  Different  listening  constructs.  
  37. 37. Assessing  Listening:  Promises   1.  Ubiquitous  access  to  authen<c  mul<modal   listening  texts   2.  Listener  control     1.  Repe<<on   2.  Speed  
  38. 38. Ques<ons?   What  ques<ons  do  you  have  about  assessing   listening  using  computer  technologies?  Please   post  ques<ons  in  the  chat  box.       3-­‐5  min.    
  39. 39. CALT  –  Assessing  Speaking   Storing  speech   (VoiceThread,  Web   poriolios,  Podcasts)   Evalua<ng  speech   Speech  recogni<on  applica<on   (Google  Voice  Search,  Vlingo)   Speech  recogni<on  so`ware   (Dragon  Naturally  Speaking,   Dictate)   Online  course  w/speech   recogni<on  system   (EnglishCentral)   RESOURCES   Capturing  speech   (Audacity) Enabling  speech   (Skype)  
  40. 40. Poll:  VoiceThread? • Ques<on  #7:  Have  you  used  VoiceThread?            A.  Yes          B.  No   • Please  respond.    If  you  have  used  it,  please  type   in  the  chat  box  how  you’ve  used  it.      
  41. 41. CALT  –  Assessing  Speaking   VoiceThread   RESOURCE   •  A  mul<-­‐media  tool  that  allows  you  to  upload,  share  and  discuss   documents,  presenta<ons,  images,  audio  files  and  videos   •  Can  be  repurposed  as  a  plaiorm:   o  To  provide  an  input  material  such  as  aural/wri_en  prompts,   pictures  to  describe  and  audios/videos  to  summarize  for   elici<ng  a  sufficient  amount  of  ratable  speech  samples     o  And  to  collect  one's  spoken  response  to  the  input  in  either   an  audio  or  a  video  format   •  Logis<c  benefits:  keep  inputs,  responses,  and  feedback  together   in  the  same  interface   •  Appropriate  for  low-­‐stakes  assessment  
  42. 42. Poll:  Google  Voice  Search? • Ques<on  #8:  Have  you  used  Google  Voice   Search?            A.  Yes          B.  No   • Please  respond  and  those  who  have  used  it,   please  type  in  the  chat  box  when  and  how  you’ve   used  it.      
  43. 43. CALT  –  Assessing  Speaking   Google  Voice  Search   RESOURCE   •  Speaking,  instead  of  typing,  one's  search  queries   •  Can  be  repurposed  to  evaluate  pronuncia<on  as  part  of   the  speaking  construct   •  Provide  "textualized"  feedback  on  what  one  pronounces   (Lee,  2011);  e.g.  meant  to  say  ”coal”  (/koʊl/),  but   received  "cool”  (/kul/)  as  a  search  result               no<ce  a  gap  in  one's  /oʊ/  vs  /u/   •  Requires  a  li_le  bit  of  explicit  training  on  how  to   interpret  "textualized"  feedback  and  pick  out   problema<c  sounds  
  44. 44. CALT  –  Assessing  Speaking   Google  Voice  Search:   Sample  Pronuncia<on  Assessment   RESOURCE   1.  If  the  computer  you  are  using  now  to  par<cipate  in  this  webinar   is  equipped  with  a  built-­‐in  microphone,  go  to   h_ps://www.google.com.  Ideally  use  Google  Chrome   If  not,  but  a  smartphone  is  currently  available  near  you,  use  the   device  and  go  to  h_ps://www.google.com.   2.  Click  the        in  the  search  bar  and  read  the  following  words  one   by  one:     bathroom,  children,  district,  first,  hungry,  increase,  learning,  pancreas,  strategy   3.  How  many  words  did  you  get  right?  What  are  the  words  you   got  wrong,  and  what  search  results  did  you  get  instead?  Type   your  answers  in  the  chat  box.  
  45. 45. Speech-enabled Web Search
  46. 46. Speaking  Assessment:  Challenges   •  "The  ability  to  speak  a  foreign  language  is  without   doubt  the  most  prized  language  skill,  and  rightly   so  ...  Yet,  tes<ng  the  ability  to  speak  a  foreign   language  is  perhaps  the  least  developed  and  the   least  prac<ced  in  the  language  tes<ng  field."  (Lado,   1961,  p.  239)   •  Similar  ques<ons  remain  (Fulcher,  2012):   •  Accented  speech  recogni<on    
  47. 47. Speaking  Assessment:  Promises   •  Improved  and  Ubiquitous  Automa<c  Speech   Recogni<on   •  Browsers   •  Smart  phones   •  Cars  etc.     •   Improved  accuracy  rates  
  48. 48. Ques<ons?   What  ques<ons  do  you  have  about  assessing   speaking  using  computer  technologies?   Please  post  ques<ons  in  the  chat  box.     3-­‐5  min.    
  49. 49. CALT  –  Assessing  Wri<ng   RESOURCES   Internet  Technologies   (Google  Docs,  Wiki,  Blog,  e-­‐books,  Poriolios,  AWE)   So`ware   (Microso`  Word)  
  50. 50. Poll:  Automated  Wri<ng   Evalua<on  (AWE)? • Ques<on  #9:  Have  you  used  an  AWE  system?            A.  Yes          B.  No   • Please  respond  to  the  poll  and  type  in  the  chat   box  how  you’ve  used  it  (e.g.,  as  part  of  teaching,   for  yourself,  etc.)        
  51. 51. AWE  Systems  (Selec<on)     • Criterion   o  hbps://www.youtube.com/watch?v=PMgZvKNXfzs     • MyAccess   o  hbps://www.youtube.com/watch?v=SOTkmTnRZts   • FOLIO     o  hbps://www.youtube.com/watch?v=ZLaLVWXjw04    
  52. 52. Poll:  Google  Docs? • Ques<on  #10:  Have  you  used  Google  Docs?            A.  Yes          B.  No   • Please  respond  to  the  poll  and  type  in  the  chat   box  how  you’ve  used  it  (e.g.,  as  part  of  teaching,   for  yourself,  etc.)        
  53. 53. CALT  –  Assessing  Wri<ng   Google  Docs   RESOURCE   • Plaiorm  for  collabora<ve  wri<ng   • Uses  cloud  technology  to  store  documents   • Easy  sharing   • Revision  history   • Comment  func<on     • Many  of  the  same  features  a  MS  Word  
  54. 54. Poll:  MS  Word? • Ques<on  #11:  Do  you  know  about  the  following   features  in  MS  Word?     A.  Sezng  the  target  language   B.  Track  Changes   C.  Comparing  documents   D.  All  of  the  above   • Please  respond  to  the  poll  
  55. 55. CALT  –  Assessing  Wri<ng   Microso`  Word   RESOURCE   • Useful  features   o  Track  Changes   o  Merge  &  Compare   o  Comments  
  56. 56. Dran  1  –  Default  seong  
  57. 57. Dran  1  –  Language  set  to  German  
  58. 58. Dran  2  
  59. 59. Dran  1  vs.  Final  Dran  
  60. 60. Challenges  with  wri<ng  assessment   • Teacher  workload  (Ferris  &  Hedgcock,  2005)   • Teacher  feedback     o  Mismatch  between  what  teacher  reports  as  best   prac<ce  and  the  actual  prac<ce  (Lee,  2009)   • Issues  with  validity  and  reliability  (Hamp-­‐ Lyons,  1991;  Weigle,  2002)   • Concerns  with  AWE  assessment  (Condon,   2012;  Perelman,  2012)  
  61. 61. Wri<ng  Assessment:  Promises   •  Increasingly  powerful  automated  wri<ng  evalua<on   tools     •  Free  use!   •  E.g.,  EdEX  (Grading  Essays  and  the  College  Level  –  NYT,  April  4,   2013)    
  62. 62. Questions? What  ques<ons  do  you  have  about  assessing   wri<ng  using  computer  technologies?     3-­‐5  min.    
  63. 63. Overall Questions? What  overall  ques<ons  do  you  have  about  the   content  we  covered  in  the  webinar?  
  64. 64. Thank  you   Contact  informa<on:     Volker  Hegelheimer  –  volkerh@iastate.edu  
  65. 65. Image  Credits   "Apple  in  Educa<on".  (2013).  Apple.  h_p://www.apple.com/educa<on/labs/   Jiang,  Yifan.  (2013).  "Beau<ful  Day".  Iowa  State  University  Photostream.  h_p:// photostream.iastate.edu/photo/view/id/1259/sort/newest   Jiang,  Yifan.  (2013).  "Lay  where  they're  laying".  Iowa  State  University  Photostream.   h_p://photostream.iastate.edu/photo/view/id/1263/sort/highest-­‐rated   "Iceni  Technology  Blog".  (2013).  Iceni  Technology.  h_p://www.iceni.com/blog/ translate-­‐your-­‐pdf-­‐with-­‐google-­‐docs/   Masud,  La<f.  (2012).  "Campanile  in  spring!".  Iowa  State  University  Photostream.   h_p://photostream.iastate.edu/photo/view/id/991/sort/newest   "MS  Word  Help".  (2013).  MicrosoC  Word.  h_p://www.mswordhelp.com/   Vanberkum,  Nicholas.  (2012).  "First  Snowfall  of  the  Year".  Iowa  State  University   Photostream.  h_p://photostream.iastate.edu/photo/view/id/1224/sort/highest-­‐rated   Widlowski,  Cort.  (2013).  "Beardshear  just  a`er  sunset".  Iowa  State  University   Photostream.  h_p://photostream.iastate.edu/photo/view/id/1255/sort/newest   Zhang,  Lei.  (2012).  "Memorial  Union  Fountain  at  4pm".  Iowa  State  University   Photostream.  h_p://photostream.iastate.edu/photo/view/id/1211/sort/newest  

×