Assessing Spoken Language

1. Testing & Assessment*
Different Types of Speaking Test
* The most spectacular, empowering, life-affirming and ultimately
satisfying module ever. Also prone to exaggeration and outright lies.
So we agree! I’m king forever and we
put Ralph to the sword!

2. During this session you will…
• Get experience of doing two types of speaking
test;
• Consider a number of different speaking tests
and their comparative merits and
disadvantages;
• Have reverse engineered a speaking test to
determine what it aimed to assess;
• Have shed between 60,000 – 80,000 skin cells
(Please clean up after yourselves)

3. Part 2
• Describe something you own which is very
important to you. You should say:
• Where you got it from
• How long you have had it
• What you use it for
• And explain why it is important to you.

4. Doing a Speaking Test
• A local school is going to introduce a new
competitive sport. Discuss all the options and
then decide which two sports from the list below
you think are best for the school.
Table tennis Canoeing
Rugby Cycling
Hockey Gymnastics
Crocodile wrestling Catch the javelin

5. Basic Types of Speaking Test (Brown, 2004)
• Imitative
– Parroting word or phrase e.g. minimal pairs
• Intensive
– Short utterances to demonstrate grammatical, phrasal,
lexical or phonological ability
• Responsive
– Short and limited conversation (stimulus – response)
• Interactive
– Transactional & interactional speaking
• Extensive
– Long turns (e.g. speeches, presentations, monologues etc)

6. Different Oral Examinations
• Look at the materials from different speaking
tests
– What types of speaking does the test contain?
– What are the stages of the examination?
– What do you think are the criteria candidates are
marked on?
• What do you think are
– the merits of each examination?
– Any problems that candidates or examiners might
have with the test?

7. TASK: Stages of an interactive
speaking test
• Watch the video (C2 – CPE examination)
• What stages does the interview go through?
• What is the purpose of each stage?
• What is the role of the interlocutor?

8. Testing individually vs in pairs
• Which examinations do their oral components
one-to-one (i.e. Examiner to one candidate)
• Which examinations have two candidates
tested at the same time (i.e. One interlocutor,
one assessor, two candidates)
• What are some of the advantages and
drawbacks of each method?

9. Testing in Pairs
Advantages (Norton, 2000)
• Candidates are more relaxed
• Candidates use each other as a resource
• More varied patterns of interaction possible
• Interaction in test = positive washback on classroom
• Interactive tests – consistent with communicative classroom
Drawbacks (Foot, 1999)
• Candidates may not perform better
• ‘catching’ anxiety off a nervous partner
• Interacting with a stranger
• Lack of data of how factors (e.g. Age, language ability, L1 etc
impact on performance)

10. Testing in Pairs/Groups
• Quick and efficient (Bonk & Ockey, 2003)
• Less burden on examiners (Folland & Robertson, 1976)
• More consistent test administration (Ockey, 2001)
• Elicits richer samples (French, 2003)
• Symmetrical contribution to the interaction (Van Lier, 1989)
• Positive student perception (Fulcher, 1996; Ockey, 2001)
• Positive washback on classroom (Hilsdon, 1995)
• Three as the optimum number (Nakatsuhara, Forthcoming)

11. The Stages of an Interactive Oral Test
(Canale (1984))
• Warm up
• Level check
– Pre-planned questions
– Elicit an appropriate range of grammatical categories,
discourse structures, vocabulary use or socio-linguistic
factors)
• Probe
– Discover the widest extent of candidate’s abilities,
• Wind-down

12. (Brown 2004)
Successful Oral Tests require…
• Clearly specified administrative procedures
(practicality)
• Questions and probes based on the purpose of the
assessment (validity)
• Eliciting sufficient (in terms of quantity and quality)
oral production from the candidate (validity &
reliability)
• Consistent, workable scoring (reliability)

13. Situation
• You are preparing a speaking examination for
students exiting a university preparation
programme. Students should exit at B2
(Roughly IELTS 6.5/Band A of FCE)
• Work in your group and develop an overview
of the speaking test you would use to assess
these learners.

14. Example One: Discussing which topics to present
about their country to an audience of young people
Student A: [continuing after 64 seconds]...and
young people interested festivals and
celebrations so I think it can be this.
[Slight pause] What do you think?
Student B: I am agree with you. I think festivals
and celebrations important. Young
people like. And food. I think food is
good. [continues for 25 seconds]

15. Example Two: Discussing where to take
their fellow students on holiday
Student B: For me, I’d take the weekend by the sea.
Student C: So you, er, so you will take the weekend
by the sea.
Student B: [by the sea.]
Student B: And I think lots of people agree with me.
Student C: Well, yeah. This summertime will be
Student B: [Yeah.]
Student C: nice but because the UK doesn’t have nice
beaches or days
Student B: = I don’t mean in the UK.
Student C: Ah! You want to go to Spain?
Student B: = Anywhere!

16. Interaction
• Are students turn-taking or taking turns?
(Morris-Adams, 2005)
• Turn-taking “How...people know when to
speak in conversations so they don’t all talk at
the same time.” (Dornyei and Thurell, 1994)
• Backchannelling
• Conversation strategies (e.g. ‘well’ for thinking
time, floor holding strategies)
• Discourse management

17. The Tests
• B2
• Interlocutor & assessor
• 2 candidates
– Stage 1: Individual ‘warm-
up’ questions to candidates
(3 mins)
– Stage 2: Task. Cs given two
question prompts on same
topic to discuss together (4
mins)
– Stage 3: 3-way discussion –
follow up questions based on
topic from stage 2 (5 mins)
• C1
• Interlocutor & assessor
• 2 Candidates
– Stage 1: Individual ‘warm-
up’ questions to candidates
(4 mins)
– Stage 2: Task. Cs given a
problem & five options to
discuss (5 mins)
– Stage 3: 3-way discussion –
follow up questions based on
topic from stage 2 (5 mins)

18. Criteria for Levels B2 and C1
• Grammar
• Vocabulary
• Pronunciation
• Discourse Management
• Interactive Ability

19. CEFR – Spoken Interaction
(Self Assessment p.26-27)
• B2
• “Interact with a degree of fluency and
spontaneity...take an active part in
discussion...”
• C1
• “Express [him/herself] fluently and
spontaneously without too much searching for
expressions...relate contribution...skilfully to
those of other speakers”

20. Feedback
• 66% of teachers rated speaking as either 3rd or 4th out of the 4
skills for time spent on in class
• 75% of students ranked speaking as being 1st or 2nd out of the
4 skills for time spent on in class
• Teachers tended to feel students needed most help with
pronunciation, grammar , lexis
• Learners ranked backchannelling, discourse management and
lexis highest
• Teachers tended to be sceptical about whether students
listened to each other / responded to each other in class
• 75% of learners felt the did not respond to what each other
said in class

21. Teachers felt…
• the types of interaction in the test overlapped with types of
interaction in the classroom
• Part 2 tasks encouraged genuine interaction & turntaking
• Uncertain whether students had actually listened &
responded to each other in the tests
• Students had used a range of communicative strategies
Students felt...
• they had listened and responded to each other in the test
(100%)
• Uncertain as to whether they had negotiated meaning in the
test (50%)
• They could see a connection btw the test and the classroom
activities they did (100%)

22. Issues
• Teachers & students see connection btw interactive speaking
skills & the test
• Teachers know their students use some conversation
strategies (but this is limited?)
• Teachers don’t feel their learners are genuinely interacting
and therefore don’t feel learners listen or respond to each
other
• Learners feel that they listen to each other in class but that
they don’t necessarily respond or negotiate meaning
• Students feel that they do listen & respond in test but that
they aren’t necessarily negotiating meaning

23. Teacher Comments
• “I could have assigned more time to speaking/oral skills.”
• “More natural turn-taking, hedging, playing for time,
backchannelling etc needs to be introduced in the
classroom.”
• “Sts from L1 backgrounds are clearly not taught this kind
of authentic discourse…”
Student comments
• “confidence and feeling easy/free to talk is very
important to me”
• “I’ll really appreciate…an opportunity which I can have a
one by one chat time with my tutor each week.”
• “having a partner from [different country] didn’t make
the test easier”

24. • “The language classroom does not offer too
many opportunities for students to develop
their awareness of turn-taking or to practice
turn-taking skills” (Dorneyei and Thurell, 1994)

25. Proposed Ways of Fostering Interaction
• Awareness raising (Jones, 2009)
– authentic models
– Analysis of transcript & elicit key features (e.g.
backchannelling, latching etc)
• Ensure that there is a goal to the speaking task at B2
• Make explicit to learners the marking criteria for the test
– Interaction & discourse management
– Emphasising the connection with classroom activities

26. • Task Based Learning
– Clear outcome providing purpose for learner
interaction
– Pre-task preparation (awareness raising tasks)
– Reporting back (fb from tutor or observer)

27. A Validity Problems with Controlled
Interactive Tests
• Valdman (1988)
• forces test-takers into a closed system
– Interlocutor has full control
– Candidates unable to nominate topics, switch
formality levels or display full range of stylistic
maneuver
• “Therefore, this type of test can only inform
us how learners deal with an artificial social
imposition”

28. Testing Extensive Speaking
• Oral Presentations
• Cued-picture story-telling
• Retelling a story
• Translation of extended prose
TASK
• What do you think are the merits of each of
these task types? What/how would you go
about assessing each?

29. References
Brown, H.,D., 2004,, Language Assessment: Principles and Classroom Practice, New York:Longman
Clifton, J., 2006, Facilitator Talk, ELT Journal, 60/2
Council of Europe, 2001, Common European Framework of Reference for Languages: learning, teaching,
assessment, Cambridge: CUP
Dornyei and Thurell, 1994, Teaching conversational skills intensively: course content and rationale, ELT
Journal 48/1 Oxford: OUP
Jones, C., 2007, Teaching Spoken Grammar: Is noticing the best option?, Modern English Techer 16/4
Morris-Adams, M. 2005, Talking in turn or turn-taking?, Conference paper at TESOL Spain 2005
Norton, J., 2005, The paired format in the Cambridge speaking tests, ELT Journal, 59/4
Nunn, R., Designing rating scales for small-group interaction, ELT Journal 54/2 Oxford: OUP
H. Sacks, E.A. Schegloff and G. Jefferson , A simplest systematics for the organization of turn-taking in
conversation. Language 50 (1974), pp. 696–735
Van Lier, L., 1988, The Classroom and the Language Learner, London: Longman

30. Homework
• Reading: Waller, 2009, Are you listening to me:
Interaction in the Lancashire Examinations, in
Recent Approaches to Teaching and Assessing
Speaking, Mader, J. & Urkun, Z. (eds), IATEFL
Testing and Evaluation SiG