1. The document outlines a proposed task-based language assessment for students who have completed the first unit of a task-based English course called Widgets. 2. It describes the Widgets course and discusses task-based language teaching and assessment. 3. The proposed assessment includes 3 tasks - a role-play conversation, a listening and note-taking task, and a presentation - to evaluate students' language ability and performance of tasks similar to those in the Widgets course.

1. Michael Roberts, Cathy Taylor, Veronika Timpe

2. Outline
1. Our task
2. Widgets
3. TBLT & TBLA
4. Approach to TBLA
5. Needs Analysis
6. Test design and purpose
7. The tasks and their assessment criteria
8. Limitations
9. References
2

3. Our task
Design a task-based language test for a group of students who have done
the first unit of the task-based course in practical English Widgets
+ Appendix C: Video Scripts
-design decisions
-Instrument(s)
-tasks will the students have to do
-judgment criteria
-generalisability, task difficulty, and task variability
3

4. Widgets
http://www.widgets-inc.com/teacher/tblt.php
“What makes Widgets truly different from other language
teaching textbooks is its focus on creating a believable,
“real-life” English-speaking environment in the classroom.
Students imagine that they are new employees at an
exciting international company, Widgets Incorporated,
where they must work together in small groups to perform
various practical, yet fun, tasks.”
4

5. Widgets (2)
Initial reservations with the course:
 Are the tasks truly authentic when they often centre round make-believe, weird products
We did not find these fantastical products either fun or motivating
Language is given before a task to be used in that task (not TBL)
There is little evidence of language being fed in when the need arises (this is TBL)
TBL is only one type of methodology for teaching language.
Shouldn’t it be presented with others to provide a balanced syllabus?
Is it desirable that a course book confines itself to one methodology?
5

6. TBLT & TBLA
“Task-based language assessment takes the task itself as the fundamental
unit of analysis motivating item selection, test instrument construction, and
the rating of task performance. Task-based assessment does not simply
utilize the real-world task as a means for eliciting particular components of
the language system, which are then measured or evaluated; instead, the
construct of interest is performance of the task itself.”
Brown, Hudson Bonk and Norris
“Many traditional methodologies begin by teaching grammatical forms and
then go on to set communicative activities in which they believe learners
will be able to use these forms. The initial aim of TBT is to encourage
learners to engage in meaning with the language resources they already
have. This makes learners acutely aware of what they need to learn.”
Dave and Jane Willis
6

7. Approach to TBLA
7
1. Specify intended uses for the assessment: who, what, why, impact
2. Select and analyze key target tasks/features from needs analysis
3. Design tests and items: authentic elicitation, evaluation conditions
4. Determine real-world criteria for rating task performance qualities
5. Pilot-test and revise instruments and procedures (raters/rating)
6. Evaluate validity in terms of intended uses, and especially impact
on teaching and learning.
(Long & Norris 2000)

8. 1) Intended assessment use
8
Who?
Test takers:
-12-40 Asian students
-between 16-24 years old
“Stake holders” :
-Teachers
-parents
What?
Similar tasks that the students have
been exposed to/ dealt with in the
teaching
Impact?
-Washback effect
-Low stakes
Why?
The purpose of the test is to gather
information about the student learning,
i.e. whether the students, after having
completed the first unit in Widgets, are
able to perform these tasks and how
language is used among them.
Moreover, the assessment might
support further teaching and learning.

9. 2) Key target tasks & features from NA
9
Can do statements from Widgets

10. 10
2) Key target tasks & features from NA (2)
Objectives from Widgets

11. 3) & 4) Test design & tasks
11
Importance of task assessment being based on tasks known or familiar to the
students
Achievement test
Transfer of the tasks from the context of the company (as in the unit)
into a different contextual framework: biannual trade fair
assessment spin-off to unit 1

12. Tasks and their assessment criteria
Task 1 – speaking (role play):
Task title: Catching up over coffee
Theme: Chat
Level: intermediate
Time frame: 3-5 minutes per pair
Responds to: water cooler chat
Communicative Mode: Role Play
Description of task:
As an employee of Widgets Inc. you are attending a biannual trade fair in New
York where you meet people from other companies. You are acquainted with most
of them. On the first day during a conference break you meet a former colleague
of yours who now works at another Widgets subsidiary. You two enjoy the chance
to catch up socially over coffee. In pairs exchange information about events in
your lives over the last 12 months. The prompt cards provided will help you find
topics to talk about (e.g. work, holidays etc.).
12

13. 13
Tasks and their assessment criteria (2)
Task 1 – assessment criteria
a) Task-dependent criteria (Norris et al 2002)
appropriateness of questions
ability to produce follow-up questions that show the speaker A has listened to speaker B
ability to respond to follow-up questions appropriately rather than beginning a new topic
ability to maintain dialogue and begin a new topic when necessary
b) Task-independent criteria
appropriateness of vocabulary and structure for intermediate
discourse markers
turn taking elements
rejoinders

14. 14
Tasks and their assessment criteria (3)
Task 1 – Scoring
5-0 scale
Reason: reflects the self-assessment task at the end of stage 1

15. Tasks and their assessment criteria (4)
Task 2.1 – listening and note taking:
Task title: Competing products
Theme: Advertisement
Level: Intermediate
Time frame: depends on the length of recordings (approx. 6-7 min)
Responds to: “Water cooler chat” & “That’s my department”
Communicative Mode: Aural reception/ information gap filling
Description of task:
As a conference participant you decide to listen to the presentation of three products from a
competitor. You and your company have heard some things of the products before, but are
still missing more detailed information about the products. You listen to the presentation to
find out more details about the product and to tell your superiors more about the products
later on. Therefore, you will listen to the presentations twice. While listening to the
presentation for the first time, take notes on the form provided. When listening a second
time make notes that you will need to tell your boss more about the competitive products.
15

16. 16
Form for information gap task 2.1

17. 17
Tasks and their assessment criteria (5)
Task 2.1 – assessment criteria
a) Task-dependent criteria (Norris et al. 2002)
-Completeness of information
-Listening for detail/ keyword/ gist
-I can take short notes in English
b) Task-independent criteria
-Alternative spellings

18. 18
Tasks and their assessment criteria (6)
Task 2.1 – Scoring
a) Information gap
-Point for each blank
-Cutoff: 3 correct
b) Note taking
- Will be taken into consideration in the context of the presentation
 Task 2.1 = scaffolding for the following presentational task

19. Tasks and their assessment criteria (7)
Task 2.2 - presentation:
Task title: Product presentation
Theme: Marketing
Level: intermediate
Time frame: Preparation 20 min; task 3-5 min
Responds to: “Sell, sell, sell”
Communicative Mode: Presentational
Description of task:
You have returned to your company site from the conference. Your superiors are
eager to know what you learned and which of the products you consider to be the
most likely competitor. Go back to your notes from the listening task and prepare
a short presentation for your group on the product you consider to be the most
competitive one for Widgets.
19

20. 20
Tasks and their assessment criteria (8)
Task 2.2 – assessment criteria
a) Task-dependent criteria (Norris et al. 2002)
-speaks smoothly without any pauses
-structure of the presentation: introduction, description, good and bad points, why it's a
competitor, conclusion
-quality of delivery - longer turns, lack of hesitation, clarity of pronunciation
- does the presentation content reflect a success of the listening
-b) Task-independent criteria
-signpostings
-should we include body language/gestures?

21. 21
Tasks and their assessment criteria (9)
Task 3 – Scoring
-Scale numerical 5-0
-Descriptors such as from excellent to not sufficient

22. Limitations
22
General limitations:
Not Constructed the materials
Interlocutor frame for assessment
Outline of task, rather than creation of tasks
Descriptors on rating scales need to be considered
Concerns expressed in the literature  no easy answers
Limitations based on Widgets:
No clear language objectives outlined in Can do statements
Level of learners is not well-defined
Can Do statements are very general
suggestion to have an overall written report on the entire assessment series

23. References
Elder, C., Iwashita, N. & McNamara, T. (2002). Estimating the difficulty of oral proficiency tasks: what does
the test-taker have to offer? Language Testing, 19 (4), 347 – 348.
Long, M. H., & Norris, J. M. (2000). Task-based teaching and assessment. In M. Byram (Ed.), Encyclopedia of
language teaching (pp. 597-603). London: Routledge.
Norris, J. M., Brown, J. D., Hudson, T. D., & Bonk, W. (2002). Examinee abilities and
task difficulty in task-based second language performance assessment. Language
Testing, 19(4), 395-418.
Wigglesworth, G. (2008). Task and performance based assessment. In S. Shohamy & N. H. Hornberger
(Eds.), Encyclopedia of language and education (2nd Ed.), Volume 7: Language testing and assessment
(pp.111-122). Springer Science+Business Media.
Willis, D & Willis, J. (2007) Doing task-based Teaching, Oxford
http://flenj.org/CAPS/?page=parent
23