The importance and basic type of speaking, micro- and macro skills of speaking, and how to observing the performance and designed the assessment tasks

1. EFL Learners

3. It is simply the ability to parrot
back a word or phrase or a sentence.

4. It is the production of short stretches of oral language.
Examples include directed response tasks, reading
aloud, sentence and dialogue completion, limited
picture-cued tasks.

5. The tasks include interaction and test comprehension
but at the limited level of short conversations, standard
greetings, small talk, requests, and comments.

6. The length and complexity of the interaction are more
in interactive tasks than in responsive ones. The task
sometimes includes multiple exchanges and/or multiple
participants.

7. The tasks include speeches, oral presentations, and
story-telling. Oral interaction from listeners is either
highly limited or ruled out altogether.

8.  What tasks to assess imitative speaking?
 Typical – listen and repeat.
 What kinds of prompts would you use?
 Examples: Test-takers hear:
 beat/bit bat/vat
 I bought a boat yesterday.
 The glow of the candle is growing.
 Test-takers repeat the stimulus.
 What tasks to assess imitative speaking?
 Typical – listen and repeat.
 What kinds of prompts would you use?
 Examples: Test-takers hear:
 beat/bit bat/vat
 I bought a boat yesterday.
 The glow of the candle is growing.
 Test-takers repeat the stimulus.

9.  How would you score these tasks?
 Example scoring scale:
 2 acceptable pronunciation.
 1 comprehensible, partially correct.
 0 silence, seriously incorrect.
 How would you score these tasks?
 Example scoring scale:
 2 acceptable pronunciation.
 1 comprehensible, partially correct.
 0 silence, seriously incorrect.

10.  Could you design a test that is done over the
phone?
 Where the test-taker talks to a computer?
 What tasks could you include to test all the
five basic types of speaking?
 Let’s look and listen at a test called
PHONEPASS (Versant English Test – now
offered by Pearson Education, Inc.)
http://www.ordinate.com/samples/english.jsp

11.  What do you think about this test? Use the
five basic criteria.

12.  What speaking skills can be assessed/inferred
from the PHONEPASS test?
 Examples?

13.  What range of tasks can be used to assess
intensive speaking?
 List some with your partner…

14.  Typical: Directed Response Tasks
 What kinds of prompts would you use?
 Examples: Test-takers are directed to
respond:
 Tell me he went home.
 Tell me that you like rock music.
 Tell me that you aren’t interested in tennis.
 Tell him to come to my office at noon.
 Remind him what time it is.

15.  Typical: Reading aloud
 What kinds of prompts did PHONEPASS use?
 What other tasks could you design? List two
other tasks with your partner…

16.  Reading a scripted dialogue.
 Reading sentences containing minimal pairs.
 Examples:
 The man beat his dog.
 The man bit his dog.
 Reading information from a table or chart.
 Reading a scripted dialogue.
 Reading sentences containing minimal pairs.
 Examples:
 The man beat his dog.
 The man bit his dog.
 Reading information from a table or chart.

17.  What elements of speech would you assess?
List two of the major ones.
 Typical:
 Pronunciation
 Fluency

18.  Pronunciation: what scale would you design
for scoring?
 Example:
 0.0—0.4 frequent errors and unintelligible.
 0.5—1.4 occasionally unintelligible.
 1.5—2.4 some errors but intelligible.
 2.5—3.0 occasional errors but always intelligible.
 Pronunciation: what scale would you design
for scoring?
 Example:
 0.0—0.4 frequent errors and unintelligible.
 0.5—1.4 occasionally unintelligible.
 1.5—2.4 some errors but intelligible.
 2.5—3.0 occasional errors but always intelligible.

19.  Fluency: what scale would you design for
scoring?
 Example:
0.0– 0.4 slow, hesitant, and unintelligible.
0.5– 1.4 non-native pauses and flow that
interferes with intelligibility.
1.5- 2.4 non-native pauses but the flow is
intelligible.
2.5- 3.0 smooth and effortless.
 Fluency: what scale would you design for
scoring?
 Example:
0.0– 0.4 slow, hesitant, and unintelligible.
0.5– 1.4 non-native pauses and flow that
interferes with intelligibility.
1.5- 2.4 non-native pauses but the flow is
intelligible.
2.5- 3.0 smooth and effortless.

20.  Look at the PHONEPASS sample score report
and answer these questions.
1. What is the range of possible scores?
2. What are the different bands?
3. What did the ‘sample’ candidate score?
4. What does the score tell the candidate?
5. How does PHONEPASS measure up against
other speaking tests?

21. What are the advantages?
 Comparisons between students are quite
simple.
 Tests are easy to prepare and to administer.
 Predictable output, practicality, and
reliability in scoring.
What are the advantages?
 Comparisons between students are quite
simple.
 Tests are easy to prepare and to administer.
 Predictable output, practicality, and
reliability in scoring.

22. What are the disadvantages?
 It is not ‘really’ authentic is it?
 Exceptions
 A parent reading to a child,
 sharing a story (news) with someone
 giving a scripted oral presentation.
 It is not communicative in real contexts.

23.  Let’s take a look at some speaking task
samples from the TOEFL iBT TEST.
 As you listen to the sample, make a note of
the
 similarities and differences to the PHONEPASS
test
 what tasks are used to assess the different types
of speaking
See research paper for background of test design.
You can download the full sample iBT test.

24.  First, test-takers are given time to read
through the dialogue to get its gist, then the
tape/teacher produces one part orally and
the test-taker responds.
 Advantages?
 More time to anticipate an answer
 No potential ambiguity created by aural
misunderstanding (oral interview).

25.  TOEFL iBT identifies three broad areas to
assess in their scoring standards. What are
they?
 D E L I V E R Y
 L A N G U A G E U S E
 T O P I C D E V E L O P M E N T
 TOEFL iBT identifies three broad areas to
assess in their scoring standards. What are
they?
 D E L I V E R Y
 L A N G U A G E U S E
 T O P I C D E V E L O P M E N T

26.  Oral Interview:
 a test administrator and a test-taker
 direct face-to-face exchange
 proceed through a protocol of questions and
directives.
 It can vary in length from 5 to 45 minutes,
depending on purpose and context.
 Placement interviews may need only 5 minutes
 Proficiency Interviews may require an hour.
 Oral Interview:
?
 a test administrator and a test-taker
 direct face-to-face exchange
 proceed through a protocol of questions and
directives.
 It can vary in length from 5 to 45 minutes,
depending on purpose and context.
 Placement interviews may need only 5 minutes
 Proficiency Interviews may require an hour.

27.  Let’s take a look at the IELTS TEST, part 1
and IELTS TEST, part2.
 As you listen to the sample, make a note of
the
 the differences between IELTS and PHONEPASS
 the role of the ‘examiner’
 what different tasks are used, and why

28.  IELTS has five broad areas in scoring:
 C O M P R E H E N S I O N
 G R A M M A R
 V O C A B U L A R Y
 P R O N U N C I A T I O N
 C O M M U N I C A T I V E C O M P E T E N C E
 IELTS has five broad areas in scoring:
 C O M P R E H E N S I O N
 G R A M M A R
 V O C A B U L A R Y
 P R O N U N C I A T I O N
 C O M M U N I C A T I V E C O M P E T E N C E

29.  Let’s take a look at the IELTS TEST, part 2.
 As you listen to the sample, try to assess the
speaker using the scoring grid provided.
 Note your score for each aspect
 Compare with your partner at the end

30. How do these relate to the 5 criteria?
 Clear administrative PRACTICAL
procedures
 Focusing the questions and VALID
probes on the purpose of
the assessment
 Biased for best AUTHENTIC & WASHBACK
performance
 Creating a consistent, RELIABLE
workable scoring system
How do these relate to the 5 criteria?
 Clear administrative PRACTICAL
procedures
 Focusing the questions and VALID
probes on the purpose of
the assessment
 Biased for best AUTHENTIC & WASHBACK
performance
 Creating a consistent, RELIABLE
workable scoring system

31.  Let’s take a look at another totally different
approach to assessing speaking.
 Linguistic profiling suggests that language
development (first or second) follows a standard
schedule.
 Speech samples collected from learners
 Analysed to locate the patterns
 Patterns matched from the sample to the regularities
of the standard development schedule.
See RAPID PROFILE as an example:
http://groups.uni-paderborn.de/rapidprofile/

36.  For your reference, what follows is a
summary of various tasks that can be used to
assess different aspects of speaking.
 These have been summarized from Brown
(2003) Language Assessment Principles.

37. Question and Answer
Examples:
1. What is this called in English? ( to elicit a
predetermined correct response)
2. What are the steps governments should
take, if any, to stem the rate of de-
forestation in tropical countries? ( given
more opportunity to produce meaningful
language in response)

38. 1. What do you think about the weather today?
2. Why did you choose your academic major?
3. Personal questions:
a. Have you ever been to the U. S. before?
b. What other countries have you visited?
c. Why did you go there? What did you like best
about it?

39. Examples:
 how to operate an appliance
 how to put a bookshelf together, or
 how to create a dish.
Scoring: based on
(1) Comprehensibility
(2) Specified grammatical/discourse categories.

40. Test-takers hear:
1. Describe how to make a typical dish
2. What’s a good recipe for making _____?
3. How do you access email on a PC computer?
4. How do I get from ___ to ____ in your city?
 Test-takers respond.

41. The task should require the test-taker to
produce at least 5 or 6 sentences.
 Use familiar topics and test linguistic
competence.
 Paraphrasing, e.g. paraphrasing a story or a
phone message

42. 1. elicit short stretches of output
2. the criterion being assessed:
a. Is it a listening task more than production?
b. Does it test short-term memory rather than
linguistic ability?
c. How does the teacher determine scoring of
responses?

43. Four stages: Warm-up, Level check, Probe,
and Wind-down.
1. Warm-up
 the interviewer directs mutual
introductions, helps the test-taker become
comfortable with the situation, apprises
the format, and reduces anxieties.
2. Level check
 Through preplanned Qs, the test-takers
respond using expected forms and functions.
Linguistic target criteria are scored.

44. 3. Probe:
 In this phase, test-takers go to the heights of
their ability and extend beyond the limits of
the interviewer’s expectation.
 Through probe questions, the interviewer
discovers the test-taker’s proficiency. At the
lower levels of proficiency, probe items may
demand a higher range of vocabulary and
grammar than predicted. At the higher levels,
probe items will ask the t-t to give an opinion,
to recount a narrative or to respond to
questions.

45. 4. Wind-down
 the interviewer encourages the test-taker to
relax with some easy questions, sets the t-t’s
mind at ease, and provides information about
when and where to obtain the results of the
interview. This part is not scored.

46. 1. Warm-up:
How are you?/What’s your name?/What country
are you from?/Let me tell your about this
interview.
2. Level check:
How long have you been in this city?/tell me about
your family./What is your major?/How long have
you been working at your degree?/What are your
hobbies or interests?/Why do you like your hobby?
What is your favorite food?/Tell me about your
exciting experience you’ve had.

47. 3. Probe:
What are your goals for learning English in this
program?/Describe your academic field to me.
What do you like or dislike about it?/Describe
someone you greatly respect, and tell me why you
respect that person./If you were [president, prime
minister] of your country, what would you like to
change about your country?

48. 4. Wind-down:
Did you feel okay about this interview?/You’ll get
your results from this interview next week./Do
you have any question to ask?/It was interesting to
talk with you. Best wishes.

49.  A picture-cued stimulus requires a
description from the test-taker. It may elicit
a word, a phrase, a story, or incident.
 Scoring scale for intensive tasks:
2 comprehensible; acceptable target form
1 comprehensible; partially correct
0 silence; or seriously incorrect

50.  Translation is a communicative device in
contexts where English is not a native lang.
 English can be called on to be interpreted as
a second language.
 Conditions may vary from an instant
translation of a native word, phrase, or
sentence to a translation of longer texts.
 Advantages: the control of the output &
easily specified scoring.

51.  It is a popular pedagogical activity in
communicative language-teaching classes.
 The test administrator must determine
the assessment objectives of the role play,
then devise a scoring technique that
pinpoints those objectives.
 Examples: “Pretend that you’re a tourist
asking me for directions”, “You are buying
a necklace from me in a flea market, and
want a lower price”.

52.  As informal techniques to assess learners,
D & C offer a level of authenticity and
spontaneity that other assessment
techniques may not provide:
 clarifying, questioning, paraphrasing,
intonation patterns, body language, eye
contact, and other sociolinguistic factors

53.  Extensive speaking tasks are frequently
variations on monologues, usually with
minimal verbal interaction.
 Oral Presentations:
 Examples: presenting a report, a paper, a
marketing plan, a sales idea, a design of a new
product, or a method.

54. Rules for effective assessment: (a) specify the
criterion, (b) set appropriate tasks, Elicit
optimal output, and (d) establish practical,
reliable scoring procedures.
 Oral presentation checklist
3 excellent 2 good 1 fair 0 poor
 Content:
 The purpose or objective of the presentation
was accomplished.
 The introduction was lively and got my attention.
 The main idea or point was clearly stated toward
the beginning. The supporting points were clearly
expressed and supported well by facts and
argument.
 The conclusion restated the main idea or purpose.

55.  Delivery
 The speaker used gestures and body language
well.
 The speaker maintained eye contact with the
audience.
 The speaker’s language was natural and fluent.
 The volume of speech was appropriate.
 The rate of speech was appropriate.
 The pronunciation was clear and comprehensible.
 The grammar was correct and didn’t prevent
understanding.
 Used visual aids, handouts, etc., effectively.
 Showed enthusiasm and interest.
 Responded to audience questions well.

56.  At this level, a picture/a series of
pictures is used as a stimulus for a longer
story or description.
 The objective of eliciting narrative
discourse needs to be clear. (p. 181) (Tell
& use the P. tense)
 For example, are you testing for oral
vocabulary, (girl, telephone, wet) for
time relatives (before, after, when), for
sentence connectors (then, so), for past
tense of irregular verbs (woke, drank,
rang), or for fluency in general?
 Criteria for scoring need to be clear.

57.  Test-takers hear /read a story or news event
that they are asked to retell.
 It differs from the paraphrasing task
discussed above in that it is a longer stretch
of discourse and a different genre.

58.  Longer texts are presented for the test-
taker to read in the native language and
then translate into English.
 Texts vary in forms: dialogue, directions,
play, movie, etc.
 Advantages: the control of the content,
vocabulary, the grammatical and
discourse features.
 Disadvantages: a highly specialized skill is
needed.