Testing speaking

   Workshop by Michel Fandel, Christiane Klein and Romy Schmit
                                                18 October 2012
What are the challenges?
•   Time consuming
•   Costly in terms of man-power
•   Difficult to administer
•   Design
•   Documenting the performance
•   Assessing
•   Marking criteria
Why test speaking?
• We are required to test speaking by the
  authorities.

• We have to give marks for speaking and want to
  be able to justify our marks.

• Speaking is an important skill: we want to
  encourage learners to practice speaking and to
  give them meaningful feedback on their
  performances.
Why test language learning? (1)
‘Tests are also important instruments of public
policy. National examinations, for example, are
used to ensure that learners at educational
institutions across the country are held to the
same standards.’
                            Douglas, D. [2010:2]
Why test language learning? (2)
‘Tests also allow other stakeholders, including
programme administrators, parents, admissions
officers and prospective employers, to be assured
that learners are progressing according to some
generally accepted standard or have achieved a
requisite level of competence in their second
language’.
                           Douglas, D. [2010:1-2]
Why test language learning? (3)

‘Perhaps the most important reason is fairness.
We like to make sure all students are treated the
same, to give each of them equal opportunity to
show us what they’ve learned and what they can
do with the language they’ve learned. Tests allow
us to present all our students with the same
instructions and the same input under the same
conditions.’
                             Douglas, D. [2010:1]
What do we test?
What is ‘speaking’?
What do we test?
What is ‘speaking’?
What do we test?
What is ‘speaking’?
What do we test?
What is ‘speaking’?
What do we test?
What is ‘speaking’?

       Interactive    Productive
        Informal       Informal
      Spontaneous     Rehearsed



       Interactive
                      Productive
         Formal
                       Formal
      Spontaneous
                      Rehearsed
       Rehearsed
What do we test?
What is ‘speaking’?
A1                                                             C2
     Formulaic       Interactive   Productive   Entertaining
                      Informal      Informal
                    Spontaneous    Rehearsed
                     Interactive
                                   Productive
                      Informal
                                    Formal
                    Spontaneous                 Professional
                                   Rehearsed
     Professional    Rehearsed                    Academic
C2                                                             C2
What is the place and the use of a
scale like the CEFR?


• Theoretical model = simplification of reality
• Identifies levels of performance
• Presents these levels in a logical order

• No representation of individual language
  learners actual progress

• Cf.: language scales.pdf
How do we test?                                 Higly standardized tests
How do we assess?                               e.g. Certifying tests
                Warm up                                          Main tasks

                                                   “Long” monologue (description, presentation)

                                                             Vocabulary,& structure
                                                                 Organisation
                                                                Pronunciation

  Interview (can be prepared & rehearsed)
Short turns, basic simple language and issues

             First impression:
            Maybe pronunciation


                                                       More complex, in-depth exchange:



                                                               Interactive speaking
How do we test?   Regular classroom observations
How do we assess? Electronic portfolio


  Issues
  • Fairness
  • Number of observations or samples
  • Documentation and administration
  • Getting the whole picture
  • Giving feedback and justifying mark
Marking grids
Cf.: LUXEMBOURG MARKING DESCRIPTORS
FOR SPEAKING ACROSS THE CEFR
LEVELS.pdf
Bibliography and Sources (1)
• Csépes, I. & Együd, G. (2003) Into Europe – The
  Speaking Handbook. British Council – Teleki
  László Foundation
 The Into Europe series can be downloaded from:
 http://www.lancs.ac.uk/fass/projects/examreform/
• Douglas, D. (2010) Understanding Language
  Testing. Hodder Education
• Luoma, S. (2004) Assessing Speaking. CUP
Bibliography and Sources (2)
 • http://www.cambridgeesol.org/index.html
   http://research.cambridgeesol.org/fitness-
    purpose/examples-speaking-tests
 • http://www.cityandguildsenglish.com/
   http://www.cityandguildsenglish.com/SESOL
   http://www.cityandguildsenglish.com/Sample_spo
    ken_examination_videos
 • http://www.ielts.org/
   http://www.ielts.org/test_takers_information/test_
    sample/speaking_sample.aspx
 • http://www.trinitycollege.co.uk/site/?id=369
   http://www.trinitycollege.co.uk/site/?id=2071
   http://www.trinitycollege.co.uk/site/?id=1861

Testing speaking

  • 1.
    Testing speaking Workshop by Michel Fandel, Christiane Klein and Romy Schmit 18 October 2012
  • 2.
    What are thechallenges? • Time consuming • Costly in terms of man-power • Difficult to administer • Design • Documenting the performance • Assessing • Marking criteria
  • 3.
    Why test speaking? •We are required to test speaking by the authorities. • We have to give marks for speaking and want to be able to justify our marks. • Speaking is an important skill: we want to encourage learners to practice speaking and to give them meaningful feedback on their performances.
  • 4.
    Why test languagelearning? (1) ‘Tests are also important instruments of public policy. National examinations, for example, are used to ensure that learners at educational institutions across the country are held to the same standards.’ Douglas, D. [2010:2]
  • 5.
    Why test languagelearning? (2) ‘Tests also allow other stakeholders, including programme administrators, parents, admissions officers and prospective employers, to be assured that learners are progressing according to some generally accepted standard or have achieved a requisite level of competence in their second language’. Douglas, D. [2010:1-2]
  • 6.
    Why test languagelearning? (3) ‘Perhaps the most important reason is fairness. We like to make sure all students are treated the same, to give each of them equal opportunity to show us what they’ve learned and what they can do with the language they’ve learned. Tests allow us to present all our students with the same instructions and the same input under the same conditions.’ Douglas, D. [2010:1]
  • 7.
    What do wetest? What is ‘speaking’?
  • 8.
    What do wetest? What is ‘speaking’?
  • 9.
    What do wetest? What is ‘speaking’?
  • 10.
    What do wetest? What is ‘speaking’?
  • 11.
    What do wetest? What is ‘speaking’? Interactive Productive Informal Informal Spontaneous Rehearsed Interactive Productive Formal Formal Spontaneous Rehearsed Rehearsed
  • 12.
    What do wetest? What is ‘speaking’? A1 C2 Formulaic Interactive Productive Entertaining Informal Informal Spontaneous Rehearsed Interactive Productive Informal Formal Spontaneous Professional Rehearsed Professional Rehearsed Academic C2 C2
  • 13.
    What is theplace and the use of a scale like the CEFR? • Theoretical model = simplification of reality • Identifies levels of performance • Presents these levels in a logical order • No representation of individual language learners actual progress • Cf.: language scales.pdf
  • 14.
    How do wetest? Higly standardized tests How do we assess? e.g. Certifying tests Warm up Main tasks “Long” monologue (description, presentation) Vocabulary,& structure Organisation Pronunciation Interview (can be prepared & rehearsed) Short turns, basic simple language and issues First impression: Maybe pronunciation More complex, in-depth exchange: Interactive speaking
  • 15.
    How do wetest? Regular classroom observations How do we assess? Electronic portfolio Issues • Fairness • Number of observations or samples • Documentation and administration • Getting the whole picture • Giving feedback and justifying mark
  • 16.
    Marking grids Cf.: LUXEMBOURGMARKING DESCRIPTORS FOR SPEAKING ACROSS THE CEFR LEVELS.pdf
  • 17.
    Bibliography and Sources(1) • Csépes, I. & Együd, G. (2003) Into Europe – The Speaking Handbook. British Council – Teleki László Foundation The Into Europe series can be downloaded from: http://www.lancs.ac.uk/fass/projects/examreform/ • Douglas, D. (2010) Understanding Language Testing. Hodder Education • Luoma, S. (2004) Assessing Speaking. CUP
  • 18.
    Bibliography and Sources(2) • http://www.cambridgeesol.org/index.html http://research.cambridgeesol.org/fitness- purpose/examples-speaking-tests • http://www.cityandguildsenglish.com/ http://www.cityandguildsenglish.com/SESOL http://www.cityandguildsenglish.com/Sample_spo ken_examination_videos • http://www.ielts.org/ http://www.ielts.org/test_takers_information/test_ sample/speaking_sample.aspx • http://www.trinitycollege.co.uk/site/?id=369 http://www.trinitycollege.co.uk/site/?id=2071 http://www.trinitycollege.co.uk/site/?id=1861

Editor's Notes

  • #3 CHALLENGES – WE ARE AWARE OF THEM; WE HAVE NO PERFECT SOLUTIONS; WE PROPOSE IDEAS; INVITE YOU TO SHARE YOUR EXPERIENCE AND EXPERTISE WITH UStime consuming: 25 students x 7 minutes = 175 minutes (+/- 4 lessons)costly in terms of man-powerdifficult to administer whilst being in charge of a whole classinterlocutingassessingdesign => what are we testing? how is supposed to be tested?documenting the performance: justification for the markassessing real timelatermarking criteria => how objective are they?
  • #4 LAWThese marks are of interest to various stakeholders apart from the test takers: parents, prospective employers …How frequently do you use your foreign languages and which skills do you use the most?
  • #5 LAWTestssetstandardse.g. Final examse.g. Épreuvescommunes
  • #6 TESTS SET STANDARDS => FAIRNESSe.g. Proficiency tests => topredictwhatlearnerswillbeableto do withthelanguage in communicativesituationsoutsidetheclassroome.g. Achievement tests => todetermineiflearnersaremakingadequateprogress (syllabusbased tests)
  • #7 TESTS SET STANDARDS => FAIRNESSReliabilitySecond opinion (evaluation vs. assessment)
  • #8 InteractiveInformalSpontaneous
  • #9 ProductiveInformalPrepared
  • #10 InteractiveFormalSpontaneous / prepared
  • #11 ProductiveFormalPrepared
  • #16 Number of observations or samplesDocumentation and administrationrecordingscanweaskstudents to do thishow authenticisspeaking to berecordedGetting the wholepicture => several observations (5-6)Giving feedback and justifying mark => markinggrids