SlideShare a Scribd company logo
1 of 77
How do we assess task-
based performance?
Recent advances in
language assessment
LARC/CALPER webinar
April 3, 2014
John M. Norris – Georgetown University
Copyright © John M. Norris, 2014: Please cite as:
Norris, J. M. (2014, April). How do we assess task-based performance? Invited
LARC/CALPER testing and assessment webinar.
Plan:
0. What is assessment and why do we do it?
1. Traditions, alternatives  Task-based assessment
2. The range of uses for task-based language assessment
3. Issues in assessing task-based language performance
4. Some challenges and opportunities in TBLA
Task-based language assessment: the elicitation and evaluation
of language use (across all modalities) for expressing and
interpreting meaning, within a well-defined communicative
context (and audience), for a clear purpose, towards a valued
goal or outcome.
What is assessment?
What isn’t assessment (in language education)?
CEFR
B1
What is assessment and why to we do it?
Robert Mislevy (2004)
Assessment is itself a particular kind of narrative:
• an evidentiary argument
• about aspects of what students know and can do,
• based on a handful of particular things they have said, done, or made.
What is educational assessment?
Which narratives do we want to tell in language assessment?
2. The European Union and the United
States: Possible Comparisons and Lessons
You have been invited to give a lecture at
the Society for German-American Relations
in Weimar next month. For your
contribution, the organizers are looking for
a presentation on the much-discussed topic
“The European Union and the Founding of
the United States – A Model?” Your talk is
limited to 15 minutes, to be followed by
discussion of your remarks.
What is assessment and why do it?
1. Fill in the missing letters to
create complete words that make
sense in the passage.
This is an example C-test passage.
Starting wi_____ the sec____ word
o____ this sent____, the la____
half fr____ each consec____ word
h____ been del____. C-tests a____
typically comp____ of mult____
texts wh____ become increa____
difficult. Ea____ text usu____
contains bet____ 20 a____ 25
it____. So, wh____ do y____ think
ge____ tested on a C-test?
Which is the ‘better’
language assessment?
The ‘correct’ answer:
IT DEPENDS ON THE INTENDED USE of the TEST!
Specifying intended uses for assessment (Norris, 2000)
INTENDED
ASSESSMENT
USE
WHO?
Test
Users
WHAT?
Test
Information
IMPACT?
Test
Consequences
WHY?
Test
Purposes
The ‘correct’ answer:
IT DEPENDS ON THE INTENDED USE of the TEST!
2. The European Union and the United
States: Possible Comparisons and Lessons
You have been invited to give a lecture at
the Society for German-American Relations
in Weimar next month. For your
contribution, the organizers are looking for
a presentation on the much-discussed topic
“The European Union and the Founding of
the United States – A Model?” Your talk is
limited to 15 minutes, to be followed by
discussion of your remarks.
What is assessment and why do it?
1. Fill in the missing letters to
create complete words that make
sense in the passage.
This is an example C-test passage.
Starting wi_____ the sec____ word
o____ this sent____, the la____
half fr____ each consec____ word
h____ been del____. C-tests a____
typically comp____ of mult____
texts wh____ become increa____
difficult. Ea____ text usu____
contains bet____ 20 a____ 25
it____. So, wh____ do y____ think
ge____ tested on a C-test?
Intended use: quick
estimate of language
proficiency, e.g., for use in
placing students into
language courses…not task-
based assessment.
Which is the ‘better’
language assessment?
2. The European Union and the United
States: Possible Comparisons and Lessons
You have been invited to give a lecture at
the Society for German-American Relations
in Weimar next month. For your
contribution, the organizers are looking for
a presentation on the much-discussed topic
“The European Union and the Founding of
the United States – A Model?” Your talk is
limited to 15 minutes, to be followed by
discussion of your remarks.
What is assessment and why do it?
1. Fill in the missing letters to
create complete words that make
sense in the passage.
This is an example C-test passage.
Starting wi_____ the sec____ word
o____ this sent____, the la____
half fr____ each consec____ word
h____ been del____. C-tests a____
typically comp____ of mult____
texts wh____ become increa____
difficult. Ea____ text usu____
contains bet____ 20 a____ 25
it____. So, wh____ do y____ think
ge____ tested on a C-test?
Intended use: quick
estimate of language
proficiency, e.g., for use in
placing students into
language courses…not task-
based assessment.
Intended use: extended,
authentic, and integrated
performance with the
target language, e.g., for
end-of-unit achievement
testing and feedback…yes,
task-based assessment.
Which is the ‘better’
language assessment?
Making decisions
about students
admissions
placement course credit
fulfilling requirements
grading /achievement
advancement/retention
certification of
abilities
awarding
funding
ETC.
INDIVIDUAL
DECISION-
MAKING
EDUCATIONAL USES FOR ASSESSMENT
Making decisions
about students
admissions
placement course credit
fulfilling requirements
grading /achievement
advancement/retention
certification of
abilities
awarding
funding
ETC.
Informing classroom
teaching and learning
diagnosing learner
needs
motivating
students
understanding
learning outcomes
improving
instruction
defining
learning
expectations focusing students’
learning/study
ETC.
INDIVIDUAL
DECISION-
MAKING
CLASSROOM
FEEDBACK
EDUCATIONAL USES FOR ASSESSMENT
Making decisions
about students
admissions
placement course credit
fulfilling requirements
grading /achievement
advancement/retention
certification of
abilities
awarding
funding
ETC.
Informing classroom
teaching and learning
diagnosing learner
needs
motivating
students
understanding
learning outcomes
improving
instruction
defining
learning
expectations focusing students’
learning/study
ETC.
Improving, ensuring,
demonstrating quality
demonstrating
student learning
clarifying program
mission/needs
revising curriculum/
pedagogy improving teacher
performance
communicating beyond,
about the program
responding to
program review
ETC.
INDIVIDUAL
DECISION-
MAKING
CLASSROOM
FEEDBACK
PROGRAM
EVALUATION
aligning, articulating
program practices
EDUCATIONAL USES FOR ASSESSMENT
Making decisions
about students
admissions
placement course credit
fulfilling requirements
grading /achievement
advancement/retention
certification of
abilities
awarding
funding
ETC.
Informing classroom
teaching and learning
diagnosing learner
needs
motivating
students
understanding
learning outcomes
improving
instruction
defining
learning
expectations focusing students’
learning/study
ETC.
Improving, ensuring,
demonstrating quality
demonstrating
student learning
clarifying program
mission/needs
revising curriculum/
pedagogy improving teacher
performance
communicating beyond,
about the program
responding to
program review
ETC.
INDIVIDUAL
DECISION-
MAKING
CLASSROOM
FEEDBACK
PROGRAM
EVALUATION
aligning, articulating
program practices
EDUCATIONAL USES FOR ASSESSMENTWhat different kinds
of assessment are
needed to meet
these different uses?
How might tasks
play a role?
METHODS
MCT
Matching
Fill-in-the-blank
Short answer
Cloze/C-test
Performance
assessment
Interviews
Observations
Essays
Portfolios
True-false
Self-assessment
What are the possibilities for assessment?
Projects
Peer-assessment
Rubrics
Alternatives in assessment: The move to task
Early interest in ‘tasks’ for assessment
“If we want to find out how well a person can perform a task, we
can put him to work at that task, and observe how well he does it
and the quality and quantity of the product he turns out.”
Cureton (1950): “Validity”
e.g., in language testing:
Foreign Service Institute Oral Interview (1950s)
Using tasks to elicit a holistic estimate of
functional language ability, from 0 (no ability) to
5 (educated native speaker ability)
Standardized testing and psychometrics
But, the primary tradition of assessment in the 20th century:
Large-scale standardized testing: minimal competency, accountability
emphasis on efficiency, technology (e.g., scoring), discrete-point
primary concern with psychometrics, norm-referencing
validity = narrowly focused on construct representation, reliability
e.g., in language testing:
Test of English as a Foreign Language (1964)*
MCT: vocabulary, reading, listening, structure,
grammar (emphasis on testing
language knowledge)
*But see later iterations of TOEFL…
Influence of traditional testing
When we ask teachers
what an assessment looks
like, what do they tell us?
• True-false
• Matching
• Multiple choice
• Fill-in the blank
• Exam
• Test
• Giving grades
• Accountability
• ETC.
Influence of traditional testing
When we ask teachers
what an assessment looks
like, what do they tell us?
Washback of testing traditions:
Narrowing teaching and learning
Misrepresenting assessment
Disconnect between teaching-
testing-learning
• True-false
• Matching
• Multiple choice
• Fill-in the blank
• Exam
• Test
• Giving grades
• Accountability
• ETC.
“It is the simplification of competencies to what can be
‘objectively’ tested which has a detrimental effect: the learning
of trivial facts, the reduction of subtle understanding to
generalizations and stereotypes, the lack of attention to
interaction and engagement because these are not tested.”
e.g., Michael Byram (1997): Need for change in intercultural assessment
Challenges to ‘traditional’ testing
Reactions to ‘traditional’ testing
Portfolios
Performance-based
assessments
Essays
CBTs Self-assessments
Interviews
Rating scales,
rubrics
Alternative
assessments
1980s-90s
•Authenticity
•Performance
•Can-do (vs. know)
•Classroom
•Washback
http://www.learner.org/libraries/tfl/assessment/analyze.html?
pop=yes&pid=2003
The move to task as assessment medium
Grant Wiggins (1998): Educative assessment
The primary assessment medium should be authentic
tasks which have learners demonstrate what they can
do with what they are learning; enables meaningful
integration of teaching, learning, and assessment
assessment becomes educative (informative) rather
than just an authoritarian mechanism for control.
The move to task-based language assessment
“…the process of evaluating, in relation to a set of
explicitly stated criteria, the quality of the
communicative performances elicited from learners as
part of goal-directed, meaning-focused language use
requiring the integration of skills and knowledge”
Geoff Brindley (1994): Task-centered assessment
aligned with CLT, TBLT movements in pedagogy
emphasis on authenticity of L2 use
validity = capturing ability for use in valued tasks
classroom applications obvious, but beyond?
Operationalizing task-based language assessment
“…genuinely task-based language assessment takes the task itself
as the fundamental unit of analysis, motivating item selection,
test instrument construction, and the rating of task performance”
Long & Norris (2000): Task-based language assessment
1. Specify intended uses for the assessment: who, what, why, impact
2. Select and analyze key target tasks/features from needs analysis
3. Design tests and items: emphasize authentic performances
4. Determine real-world criteria for rating task performance qualities
5. Pilot-test and revise instruments and procedures (raters/rating)
6. Evaluate validity in terms of intended uses, and especially
impact on teaching and learning.
Norris et al. (1998)
Tasks respond to diverse needs/uses in assessment
• Focus on what L2 users can actually do with the language
• Integrate language form and meaning into what gets assessed
• Provide useful feedback to teachers, learners, curriculum
• ‘Wash back’ on teaching and learning by making outcomes ‘real’
• Counter negative influence of traditional knowledge assessment
• Align testing with new, evolving language pedagogies
Task-based language assessment: the elicitation and evaluation
of language use (across all modalities) for expressing and
interpreting meaning, within a well-defined communicative
context (and audience), for a clear purpose, towards a valued
goal or outcome.
Why task-based language assessment?
Up Next:
How do we assess task-based performance
in language classes and programs?
Issues, examples, suggestions… But First:
Any Questions?
Task-based assessment in language classes and programs
Issues
1. Which tasks do we assess?
2. How do we elicit task-based performance?
3. What gets assessed in task performance?
4. How do we score task performance?
1. Which tasks do we assess?
e.g.: EAP pragmatics
assessment (Youn,
2009)
1. Write a
recommendation
request e-mail
to professor
2. Write an e-mail
to potential employer
to send application
packet
3. Write an e-mail
to refuse professor’s
request
4. Write constructive
comments on cover letter
written by classmate
5. Give oral peer-feedback
on classmate’s request
e-mail to professor
6. Interacting with
classmates in situations of
making suggestions and
disagreement
7. Interacting with a professor
in situations of
making requests and refusal
Why
these
tasks?
1. Which tasks do we assess?
Why
these
tasks?Which tasks?
Based on needs analysis for specific English
language program context (individual &
interactive tasks)
High priority tasks reflecting pragmatic
challenges for college-level international
students
Each task serves as end-of-unit achievement
assessment
Combined, all of the tasks can be used for
diagnostic assessment
You have been given the opportunity of a lifetime to apply for an athletic
training camp in [e.g., Spain], tuition free! This camp trains young people in all
sports, from the extreme (snowboarding, bicycle motor cross, rollerblading) to
team sports of all kinds (basketball to volleyball). You name it, they help you
train for it! To be accepted into the camp, all applicants have to convince the
admissions office that they have good exercise and nutrition habits. First, you
will read about health and nutrition from the perspective of the Spanish –
speaking world. Then you will discuss your eating and exercise regimen with
your partner to compare your nutrition and exercise, perhaps event to get some
ideas. You will then write your application letter to the summer camp
describing your nutrition and training regimen, convincing them that you are
well prepared for the camp and need to be accepted.
1. Which tasks do we assess?
e.g., Integrated performance assessment in US schools
Overall introduction to a sequence of interrelated
pedagogic/assessment tasks. Note the distinct task phases.
Consortium for assessing performance standards
http://flenj.org/CAPS/?page=parent
1. Which tasks do we assess?
e.g., Integrated performance assessment in US schools
Consortium for assessing performance standards
http://flenj.org/CAPS/?page=parent
Individual reading task
Pair discussion task
Letter writing task
Why
these
tasks?
1. Which tasks do we assess?
e.g., Integrated performance assessment in US schools
Consortium for assessing performance standards
http://flenj.org/CAPS/?page=parent
Individual reading task
Pair discussion task
Letter writing task
Self-assessment:
understanding task
content, language
Peer-assessment:
feedback on
development and
expression of ideas
Teacher-assessment:
effectiveness of letter,
language use
Why
these
tasks?
1. Which tasks do we assess?
Which tasks?
Based on the U.S. Standards for Foreign Language
Learning, by specific age/grade level and proficiency
3 integrated tasks—Interpretive, interpersonal,
presentational—reflect distinct modes of performance
Tasks are closely integrated with what is being taught:
this is why you are learning what you are learning
Tasks provide the opportunity for self-, peer-, and
teacher-assessment and feedback
Assessment raises awareness about effectiveness of
teaching and learning
See Adair-Hauck, et al. (2006)
Why
these
tasks?
2. How do we elicit task-based performance?
The European Union and the United States:
Possible Comparisons and Lessons
Task
You have been invited to give a lecture at the Society for German-American Relations in
Weimar next month. For your contribution, the organizers are looking for a presentation on
the much-discussed topic “The European Union and the Founding of the United States – A
Model?” Your talk is limited to 15 minutes, to be followed by discussion of your remarks.
Please observe your position as a speaker to this particular audience. While you should expect
good background knowledge from your German audience regarding the situation in the EU, you
primarily contribute the perspective of an American who is familiar with the foundational
history of the USA and subsequent development of the county. In its particularities this is most
likely to be less familiar to your audience. Therefore, it is precisely those aspects that you
must bring to the awareness of your audience in order to link them to the present situation in
Europe. In other words, continuously keep in mind this occasion for your speech, this starting
point for your presentation, and this intention of your talk.
e.g.: Georgetown German: Prototypical Performance Writing Task
Level IV course, “Text in Context”
What does this
description
accomplish?
Example: CNaVT
Certificate in Dutch as
a Foreign Language
Tourist domain
Listening comprehension task
“Visit to the museum”
Item: Identify the paintings
based on the descriptions you
hear from an audio museum
guide.
What is the
purpose of
the paintings?
Task Title: Recycling
Theme: Environment
Level: Novice-Mid (Focus Age Group: 11 – 14)
Communicative Mode: Presentational
Time Frame: One 42 minute period or more depending on the class and the
type of presentations chosen.
Description of Task:
Now that you’ve learned about saving the planet and the need to reuse and
recycle, you want to share with your family your ideas for reusing some throw
away things. You have cleverly thought of a few uses for reusing ____________.
Now you will share your new ideas. Create and develop your idea and choose
one from the following types of presentations to present it.
Continued on next page…
Consortium for assessing performance standards
http://flenj.org/CAPS/?page=parent
e.g.: assessing young/beginning learners, US schools
2. How do we elicit task-based performance?
Title: ¿Qué puedo hacer con_________?
(What can I do with_______?)
Include the following in your presentation:
• four or more ways of re-using something
• specific uses
• descriptions/elaboration
Three options, you choose:
Power point presentation
Poster
Live oral 3-D presentation
Materials Needed: computers, markers, poster board, realia
Why give the
student these
options?
2. How do we elicit task-based performance?
2. How do we elicit task-based performance?
Task-based performance elicitation?
Give a purpose for communicating (motivate)
For interactive/presentational tasks, indicate or
provide an audience
Provide a structure or procedures for doing the task
Utilize authentic input and other realia for situating
the task in a meaningful context
Indicate desired outcomes or products of the task
Where feasible, encourage ownership over the task
through individual choice in its accomplishment
3. What gets assessed in task performance?
Test task: Order pizza for the class.
Take orders
Place orders
Task
accomplishment?
3. What gets assessed in task performance?
Test task: Order pizza for the class.
XTake orders
Place orders
Task
accomplishment?
3. What gets assessed in task performance?
e.g.: classroom assessments for middle-school learners in Flanders
(Jordens, 2007)
Type task: (from learning goals)
Pupils can report to the teacher
on a subject, previously
discussed at school
Test task:
Pupils report to the
teacher on a film about
scuba diving, the report
being based on
photographs extracted
from the film.
e.g.: classroom assessments for middle-school learners in Flanders
Parameters for
assessment
What does pupil say?
Content relevance, completeness
Clarity
Organization
How does pupil say it?
Fluency
Articulation
Manner of speaking
Variation and extensiveness
Accuracy: word/sentence
Interaction skills
3. What gets assessed in task performance?
How is this
rubric used?
Give a presentation in Spanish
about your life and interests.
Indicate how these activities
reflect your personality. Include
some possibilities for your future
in terms of career, study, travel,
pursuing personal interests.
e.g.: performance assessment for 8th graders in US school
3. What gets assessed in task performance?
Source:
http://depts.washington.edu/mellwa/Events/20081105
/sandrock_ipa_handout.pdf
EXCEEDS
EXPECTATIONS
MEETS
EXPECTATIONS
DOES NOT MEET
EXPECTATIONS
Do we understand you?
(Comprehensibility)
My message is easily
understood by my
audience.
I am able to
communicate my
message by reproducing
memorized material.
I rely heavily on
visuals to convey my
message.
How do I use the Spanish
language?
(Language Control &
Vocabulary Use)
I can write/speak in
simple sentences
correctly.
My presentation is rich
in appropriate
vocabulary.
I demonstrate some
accuracy in written/oral
presentation when I
reproduce memorized
words and phrases.
My vocabulary reveals
basic information.
My presentation is
correct only at the
word level.
My vocabulary is
limited and / or
repetitive.
I use English.
How do I organize the
presentation?
(Communication
strategies)
My presentation is
supported with many
enriched examples.
My presentation has
some visuals/realia and
examples.
My presentation lacks
organization.
How well do I impact the
audience?
(Impact)
I use visuals/ realia and
gestures, and tone of
voice to maintain
audience’s attention.
I use some visuals/realia
and gestures to maintain
audience’s attention. My
tone of voice is
acceptable.
I use a small amount of
effort to maintain
audience’s attention.
What is the
purpose of these
criteria?
Criteria for judging performance?
Consider real-world criteria for task success or
accomplishment, including as appropriate:
Performance products: Does something happen, does
something get produced?
Performance steps: Do critical elements of the task happen?
Performance qualities: linguistic and paralinguistic features
(e.g., accuracy, fluency, gesture, pragmatics), content
(e.g., meaning), other attributes (e.g., visual support,
efficiency of task completion)
Performance levels: How well does the task performance
meet expectations, standards, etc.?
3. What gets assessed in task performance?
e.g., Georgetown Univ. German writing assessments
Writing task 1
______
______________
______________
______________
_____
Writing task 2
______
______________
______________
______________
_____
Writing task 3
______
______________
______________
______________
_____
Writing task 4
______
______________
______________
______________
_____
Writing task 5
______
______________
______________
______________
_____
End-of-level
Prototypical task
_________
_________________
_________________
_________________
_________________
_________________
____
4. How do we score task performance?
Final
Grade
Feedback:
Rating rubric,
In-text
Initial Grade
Task –based
assignment
Level scoring
rubric
Writing task v. 1
______
______________
______________
______________
_____
Writing task v. 2
______
______________
______________
______________
_____
4. How do we score task performance?
Formative assessment:
Process writing
Providing rich
feedback to enable
learning
Teacher
Students
End-of-level
Prototypical task
_________
_________________
_________________
_________________
_________________
_________________
____
Example
Journalistic
portrait: The
Vietnamese
immigrant
experience in
contemporary
Germany
2 raters from the
curricular level
+
2 raters from
another level
4. How do we score task performance?
Summative assessment:
Performance writing
Providing reliable
judgments of language +
task abilities
Scoring and score reporting for different purposes?
Consider how ‘scores’ will be used before deciding how to
score:
Formative assessment: Use public criteria and scales, and
encourage self-, peer-, teacher-scoring for feedback
Summative assessment: Use well-defined criteria and scales,
and multiple, trained raters in scoring for judgment
Holistic scoring (overall judgment) provides and indication of
task success or accomplishment or completion
Analytic scoring (by categories and qualities) provides a
useful basis for remediation
Rich and detailed score reporting enables learning from
assessment (e.g., in person, score + justification)
4. How do we score task performance?
Up Next:
The range of current uses for
task-based language assessment
But First:
Additional Questions?
Tasks as targets: Policies & standards
V. Workplace Tasks: Reading
--Scan basic charts, tables, maps or schedules for information. (5)
--Find routine information on the computer screen/scanner screen/computerized
display screen, if available. (6)
--Read information in the reception/appointment book to find available
openings for a new appointment. (6)
--Read a checklist to verify if all the steps in the procedure have been completed.
(6)
--Follow one page of clear familiar instructions. (7)
--Read a reminder or complaint letter/memo, take appropriate action. (7)
--Scan complex charts, tables and schedules for several specific pieces of
information for comparison/contrast. (7)
--Follow instructions on evacuation procedures, fire drills, or on using simple
machinery/equipment.(7)
--Use plain language manual with familiar topic and content in own field of
knowledge to find specific information. (8)
ETC…
Who should be able
to do these tasks?
For example…
Tasks as targets: Policies & standards
V. Workplace Tasks: Reading
--Scan basic charts, tables, maps or schedules for information. (5)
--Find routine information on the computer screen/scanner screen/computerized
display screen, if available. (6)
--Read information in the reception/appointment book to find available openings
for a new appointment. (6)
--Read a checklist to verify if all the steps in the procedure have been completed.
(6)
--Follow one page of clear familiar instructions. (7)
--Read a reminder or complaint letter/memo, take appropriate action. (7)
--Scan complex charts, tables and schedules for several specific pieces of
information for comparison/contrast. (7)
--Follow instructions on evacuation procedures, fire drills, or on using simple
machinery/equipment.(7)
--Use plain language manual with familiar topic and content in own field of
knowledge to find specific information. (8)
ETC…
Who should be able
to do these tasks?
Canadian Language Benchmarks:
English Language Demands for Nursing
For example…
Tasks as targets: Policies & standards
(e.g., Canadian Language Benchmarks)
“The CLB is task-based: • In syllabus design, tasks are considered
to be basic building blocks.• Tasks in language learning promote
the integration of all aspects of communicative competence, and
multilevel language processing.”
CEFR
europe
CLB
canada
ACTFL SFLL
U.S.
TESOL/NCATE
teacher development
Why?
Tasks provide meaningful targets: Ability to use the language
Washback on teaching and learning
TOC
hong kong
Tasks in large-scale proficiency assessment
4. Integrated Speaking: Read a passage from a psychology textbook and listen
to the lecture that follows it. Then answer the question.
Reading: Flow
In psychology, the feeling of complete and energized focus in an activity is
called flow. People who enter a state of flow lose their sense of time and have
a feeling of great satisfaction. They become completely involved in an activity
for its own sake rather than for what may result from the activity, such as
money or prestige. Contrary to expectation, flow usually happens not during
relaxing moments of leisure and entertainment, but when we are actively
involved in a difficult enterprise, in a task that stretches our mental or
physical abilities.
THEN
For example…
Listen to the Lecture (Male professor): I think this will help you get a picture of
what your textbook is describing. I had a friend who taught in the physics
department, Professor Jones, he retired last year. . . . Anyway, I remember . . . this
was a few years ago . . . I remember passing by a classroom early one morning just
as he was leaving, and he looked terrible: his clothes were all rumpled, and he
looked like he hadn’t slept all night. And I asked if he was OK. I was surprised when
he said that he never felt better, that he was totally happy. He had spent the entire
night in the classroom working on a mathematics puzzle. He didn’t stop to eat
dinner; he didn’t stop to sleep . . . or even rest. He was that involved in solving the
puzzle. And it didn’t even have anything to do with his teaching or research; he had
just come across this puzzle accidentally, I think in a mathematics journal, and it
just really interested him, so he worked furiously all night and covered the
blackboards in the classroom with equations and numbers and never realized that
time was passing by.
Question: Explain flow and how the example used by the professor illustrates the
concept.
Tasks in large-scale proficiency assessment
Q: Who should be
able to do these
tasks?
For example…
Lecture (Male professor): I think this will help you get a picture of what your
textbook is describing. I had a friend who taught in the physics department,
Professor Jones, he retired last year. . . . Anyway, I remember . . . this was a few
years ago . . . I remember passing by a classroom early one morning just as he was
leaving, and he looked terrible: his clothes were all rumpled, and he looked like he
hadn’t slept all night. And I asked if he was OK. I was surprised when he said that he
never felt better, that he was totally happy. He had spent the entire night in the
classroom working on a mathematics puzzle. He didn’t stop to eat dinner; he didn’t
stop to sleep . . . or even rest. He was that involved in solving the puzzle. And it
didn’t even have anything to do with his teaching or research; he had just come
across this puzzle accidentally, I think in a mathematics journal, and it just really
interested him, so he worked furiously all night and covered the blackboards in the
classroom with equations and numbers and never realized that time was passing by.
Question: Explain flow and how the example used by the professor illustrates the
concept.
Tasks in large-scale proficiency assessment
Q: Who should be
able to do these
tasks?
Test of English as a Foreign Language:
English for Academic Purposes
Example…
Tasks in large-scale proficiency assessment
(e.g., TOEFL iBT)
“Working directly with higher education institutions, we
designed the TOEFL test to simulate actual academic
classroom tasks.”
TOEFL iBT
U.S. CNaVT
Holland/Belgium
IELTS
U.K.
NMET
China
AMES
Australia
Why?
Tasks provide improved interpretations, construct validity
Washback on teaching and learning
Tasks in professional language certification
Example…
Tasks in professional language certification
http://www.anglo-continental.com/en/uk/forms/Aviation/Part%202%20%20ready.mp3
Q: Who should be
able to do these
tasks?
Example…
International Civil Aviation Organization:
English for Aviation Purposes
Tasks in professional language certification
http://www.anglo-continental.com/en/uk/forms/Aviation/Part%202%20%20ready.mp3
For example…
Tasks in professional language certification
(e.g., ICAO, 2007)
“Because of the high stakes involved, pilots and controllers deserve to
be tested in a context similar to that in which they work and test
content should therefore be relevant to their roles in the work-place”
FBI
Translation
CLB-
Nursing
NCATE
L2 teaching
English
Aviation
Why?
Tasks demonstrate ‘on-the-job’ abilities for high-stakes decisions
Washback on teaching and learning
OET
Medical
Overview of task
In groups, learners design a brochure for the Hong Kong Tourism Board describing
4 attractions in Hong Kong which would appeal to young people of their own age.
Writing a Tourist Brochure
Imagine the Hong Kong Tourism Board has asked your class to design a brochure
that would be of interest to young visitors of your own age. In groups of 4, design
the brochure describing FOUR sites suitable to young people of your own age
coming to Hong Kong. Complete this task by following the steps below:
Step 1: Group Task.
Discuss in your groups which sites young people would want to visit in Hong Kong. Choose
one site each to investigate. For homework find out as much as you can about the site,
where the site is, when it is open, what one can see / do there, what the facilities are, how
one gets there, etc. Bring this information to the next class.
For example…
Tasks in classrooms and programs
Step 2: Group Task.
Exchange information with your group members. Tell them about the site you have found out
about. Then decide how you are going to present the information in your brochure, what
order you want to put your sites in, what illustrations you need, what title you want to give
the brochure, etc.
Step 3: Individual Task.
Write a description about your chosen site (120 words). Remember to say why it is
interesting. Proofread it carefully, then hand it to your teacher.
Step 4: Group Task.
In your groups edit your work based on your teacher’s comments. Then put together your
brochure. Your brochures will be assessed on the following basis:
• Task fulfilment: would your selected sites appeal to young people?
• Accuracy of language and information provided: is the brochure written in good English? Is
the information provided accurate?
• Attractiveness of final written submission: is your brochure really attractive? Can you
make it more appealing?
For example…
Tasks in classrooms and programs
Q: Who should be
able to do these
tasks?
Step 2: Group Task.
Exchange information with your group members. Tell them about the site you have found out
about. Then decide how you are going to present the information in your brochure, what
order you want to put your sites in, what illustrations you need, what title you want to give
the brochure, etc.
Step 3: Individual Task.
Write a description about your chosen site (120 words). Remember to say why it is
interesting. Proofread it carefully, then hand it to your teacher.
Step 4: Group Task.
In your groups edit your work based on your teacher’s comments. Then put together your
brochure. Your brochures will be assessed on the following basis:
• Task fulfilment: would your selected sites appeal to young people?
• Accuracy of language and information provided: is the brochure written in good English? Is
the information provided accurate?
• Attractiveness of final written submission: is your brochure really attractive? Can you
make it more appealing?
Task-Based Assessment for English Language Learning:
Hong Kong, Secondary Schooling
Example…
Tasks in classrooms and programs
Q: Who should be
able to do these
tasks?
Tasks in classrooms and programs
(e.g., Hong Kong Special Administrative Region)
“Task-based assessment for English language learning at secondary level: One of
the key principles underpinning the school English language curriculum in Hong
Kong is that of task-based language teaching. In this package we introduce the
features of task-based assessment (TBA) within a school-based curriculum…”
http://cd1.edb.hkedcity.net/cd/eng/TBA_Eng_Sec/index.html
Local
practice
TBLT
research
Can-do
e.g., DIALANG
Teacher
guides ACTFL
IPAs
Why?
Tasks enable feedback on learning, teaching, curriculum
Washback on teaching and learning
Task-based language assessment in practice
Administrators
Staff
Classroom:
TBLAssessment
For Learning
Program: Tasks
as summative
indicators of
teaching
effectiveness
Profession: Tasks
for certification
of important
abilities
Large-scale: Tasks
for demonstrating
proficiency in societal
domains (e.g.,
education)
Standards:
Tasks as valued
targets for
learning
Some challenges and opportunities
In Task-Based Language Assessment…
2002 (19.4)
• How do we interpret task-based L2 performance?
authentic criteria, proficiency, L2 production?
• Can task-based performances be rated reliably?
analytic/holistic rating scales, rater training?
• What is the nature of task difficulty?
cognitive complexity, learner individual diffs?
• Can we generalize and/or extrapolate based on tasks?
domain-sampling, ECD, underlying constructs?
• How does task-based assessment impact instruction?
teacher, learner, curricular consequences?
Basic validity questions for TBLA
Norris (2008)
“Fundamentally…assessments are only good insofar as their use
does good, in terms of supporting educational efforts and
outcomes, and it is my intent that validity evaluation helps to
ensure that they do.”
How do we interpret task-based L2 performance?
Can do a
specific
task
Has a
certain
proficiency
Displays X
language
qualities
High-fidelity
task simulation
or replication
Test of aviation
English
Domain
sampling, ‘type’
tasks
TOEFL, CNAVT
Implicit,
spontaneous
elicitation
Classroom
practice,
SLA research
Can task-based performances be rated reliably?
YES!!!
Individual task
accomplishment:
single task
Highly specified
indigenous criteria,
steps, etc.
Teacher, tester,
learner
awareness-raising
Ability for L2
use:
multiple/type
tasks
Cross-task
holistic/analytic
rating scales
Rater training
What is the nature of task difficulty?
Why
is this
task
so
hard?
Task features
e.g., number of
elements, number of
steps, planning time,
structure, reasoning
requirements, code
frequency/complexity
Task conditions
e.g., number of
participants,
open/closed task,
convergent/divergent,
familiarity, gender,
status
Individual differences
e.g., task familiarity,
motivation, testing
stakes, aptitude,
personality
More difficult =
better or worse
performance???
Can we generalize and/or extrapolate based on tasks?
Of course we can  and we do it all the time!
Driving testseries of tasks
(1-time!)
Generalizationcan drive a car
Extrapolation
can negotiate a roundabout in
London???
…be careful!
Can we generalize and/or extrapolate based on tasks?
1. Listening comprehension?
2. Listening comprehension in a particular domain?
3. Listening comprehension on a specific task?
1. Adequate
construct
definition
2. Adequate
domain
sampling3. Adequate
task
representation
How does task-based assessment impact instruction?
Local impact (Georgetown German):
“So I think it has filtered down to smaller writing
assignments. First of all so that students get used to it
and it’s a coherent system. Whereas before we would
say, ‘Okay, now write one page about your family and
your Heimat’. Now we actually make up a sheet and
use the same features.”
Global impact (TOEFL)
“…considerable changes occurred in the
amount of attention the teachers paid to
the development of speaking and to the
integration of different skills.”
In sum, task-based performances and assessments may not (should not) meet all
purposes for assessment within language education, but they probably offer a lot of
advantages for certain purposes:
Task-based performances tell us things that other item types do not (e.g., ability to
accomplish particular tasks under particular conditions)
Task-based performances offer rich opportunities for eliciting and observing
language use with meaningful content in context (there is always more to language
use than just language, and tasks capture that)
Task-based performances can reveal multiple aspects of language ability and/or
development within a single task (e.g., accuracy, complexity, fluency; content
knowledge; procedural knowledge; expertise)
Task-based performances are in many cases the most indicative targeted
expectations of language classes and programs (if not, then what exactly are we
teaching?)
Task-based assessments provide relevant and comprehensive frames of reference
for learners and teachers to generate feedback on form-function-meaning
relationships (formative, learning-oriented uses)
Task-based assessments communicate to teachers and learners and others what is
valued within a given educational setting (motivational, educative uses)
Task-based assessments may impact curriculum and educational systems, causing a
rethinking of what and how teaching is occurring (washback uses)
Task-based assessments generate interpretable data regarding the worthwhile
outcomes of language learning programs (summative uses, e.g., for accountability,
certification, program evaluation)
Why task-based assessment?
Tasks are useful:
• align to instructional practices and values
• provide rich samples of language use data
• inform how well we teach and learn
• show what L2 users can do, and motivate them to learn
Task-based assessments can:
• improve accuracy of interpretations
• ensure abilities to use language for specific purposes
• guide educational practices
• counter negative washback of other assessments
• enable a variety of uses, decisions, actions
Q: Task-based assessment is a lot of work, and we can make
somewhat accurate assessments of learners’ abilities and
knowledge with short-cut tests; why should we bother with TBLA?
Thank
you!

More Related Content

What's hot

task based language teaching TBLT
task based language teaching TBLTtask based language teaching TBLT
task based language teaching TBLTMohammed Imad
 
Chapter 5( standards based assessment)
Chapter 5( standards based assessment)Chapter 5( standards based assessment)
Chapter 5( standards based assessment)Kheang Sokheng
 
English for Specific Purposes
English for Specific PurposesEnglish for Specific Purposes
English for Specific PurposesRoj Eusala
 
Stages of test development and common test techniques (1)
Stages of test development and common test techniques (1)Stages of test development and common test techniques (1)
Stages of test development and common test techniques (1)Maury Martinez
 
Communicative language teaching
Communicative language teachingCommunicative language teaching
Communicative language teachingCris Baràbas
 
Contrastive Analysis Hypothesis
Contrastive Analysis HypothesisContrastive Analysis Hypothesis
Contrastive Analysis HypothesisShona Whyte
 
ASSESSMENT: DISCRETE POINT TEST, INTEGRATIVE TESTING, PERFORMANCE-BASED ASSES...
ASSESSMENT: DISCRETE POINT TEST, INTEGRATIVE TESTING, PERFORMANCE-BASED ASSES...ASSESSMENT: DISCRETE POINT TEST, INTEGRATIVE TESTING, PERFORMANCE-BASED ASSES...
ASSESSMENT: DISCRETE POINT TEST, INTEGRATIVE TESTING, PERFORMANCE-BASED ASSES...A. Tenry Lawangen Aspat Colle
 
Module VIII Testing the Productive Skills
Module VIII Testing the Productive SkillsModule VIII Testing the Productive Skills
Module VIII Testing the Productive SkillsAnnalynInsic
 
Chapter 7(assessing speaking )
Chapter 7(assessing speaking )Chapter 7(assessing speaking )
Chapter 7(assessing speaking )Kheang Sokheng
 
First language acquisition (behaviourism)
First language acquisition (behaviourism)First language acquisition (behaviourism)
First language acquisition (behaviourism)Valeria Roldán
 
discrete-point and integrative testing
discrete-point and integrative testingdiscrete-point and integrative testing
discrete-point and integrative testingindriyatul munawaroh
 
Assessing grammar & vocabulary
Assessing grammar & vocabularyAssessing grammar & vocabulary
Assessing grammar & vocabularyMusfera Nara Vadia
 
Chapter 6( assessing listening)
Chapter 6( assessing listening)Chapter 6( assessing listening)
Chapter 6( assessing listening)Kheang Sokheng
 
Evaluating ELT Materials; Adapting Materials; Technology in ELT
Evaluating ELT Materials; Adapting Materials; Technology in ELTEvaluating ELT Materials; Adapting Materials; Technology in ELT
Evaluating ELT Materials; Adapting Materials; Technology in ELTRBLmadev Class 2018
 

What's hot (20)

task based language teaching TBLT
task based language teaching TBLTtask based language teaching TBLT
task based language teaching TBLT
 
Chapter 5( standards based assessment)
Chapter 5( standards based assessment)Chapter 5( standards based assessment)
Chapter 5( standards based assessment)
 
English for Specific Purposes
English for Specific PurposesEnglish for Specific Purposes
English for Specific Purposes
 
Stages of test development and common test techniques (1)
Stages of test development and common test techniques (1)Stages of test development and common test techniques (1)
Stages of test development and common test techniques (1)
 
Communicative language teaching
Communicative language teachingCommunicative language teaching
Communicative language teaching
 
Contrastive Analysis Hypothesis
Contrastive Analysis HypothesisContrastive Analysis Hypothesis
Contrastive Analysis Hypothesis
 
ASSESSMENT: DISCRETE POINT TEST, INTEGRATIVE TESTING, PERFORMANCE-BASED ASSES...
ASSESSMENT: DISCRETE POINT TEST, INTEGRATIVE TESTING, PERFORMANCE-BASED ASSES...ASSESSMENT: DISCRETE POINT TEST, INTEGRATIVE TESTING, PERFORMANCE-BASED ASSES...
ASSESSMENT: DISCRETE POINT TEST, INTEGRATIVE TESTING, PERFORMANCE-BASED ASSES...
 
Notional functional syllabus
Notional functional syllabusNotional functional syllabus
Notional functional syllabus
 
The Nature of Approaches and Methods in Language
The Nature of Approaches and Methods in LanguageThe Nature of Approaches and Methods in Language
The Nature of Approaches and Methods in Language
 
Module VIII Testing the Productive Skills
Module VIII Testing the Productive SkillsModule VIII Testing the Productive Skills
Module VIII Testing the Productive Skills
 
Chapter 7(assessing speaking )
Chapter 7(assessing speaking )Chapter 7(assessing speaking )
Chapter 7(assessing speaking )
 
First language acquisition (behaviourism)
First language acquisition (behaviourism)First language acquisition (behaviourism)
First language acquisition (behaviourism)
 
Testing listening slide
Testing listening slideTesting listening slide
Testing listening slide
 
discrete-point and integrative testing
discrete-point and integrative testingdiscrete-point and integrative testing
discrete-point and integrative testing
 
Assessing grammar & vocabulary
Assessing grammar & vocabularyAssessing grammar & vocabulary
Assessing grammar & vocabulary
 
SKILL BASED SYLLABUS
SKILL BASED SYLLABUSSKILL BASED SYLLABUS
SKILL BASED SYLLABUS
 
Language testing
Language testingLanguage testing
Language testing
 
Chapter 6( assessing listening)
Chapter 6( assessing listening)Chapter 6( assessing listening)
Chapter 6( assessing listening)
 
Evaluating ELT Materials; Adapting Materials; Technology in ELT
Evaluating ELT Materials; Adapting Materials; Technology in ELTEvaluating ELT Materials; Adapting Materials; Technology in ELT
Evaluating ELT Materials; Adapting Materials; Technology in ELT
 
Testing of listening
Testing of listeningTesting of listening
Testing of listening
 

Viewers also liked

Self-assessment in language learning
Self-assessment in language learningSelf-assessment in language learning
Self-assessment in language learningwillyccardoso
 
Language Assessment - Standards-Based Assessment by EFL Learners
Language Assessment - Standards-Based Assessment by EFL LearnersLanguage Assessment - Standards-Based Assessment by EFL Learners
Language Assessment - Standards-Based Assessment by EFL LearnersEFL Learning
 
Introduction to Language Assessment by Brown
Introduction to Language Assessment by BrownIntroduction to Language Assessment by Brown
Introduction to Language Assessment by BrownEFL Learning
 
Assessment & Language Learning
Assessment & Language LearningAssessment & Language Learning
Assessment & Language LearningBishara Adam
 
Language Assessment Types
Language Assessment TypesLanguage Assessment Types
Language Assessment TypesOla Sayed Ahmed
 
technology in standardized language assessment
technology in standardized  language assessmenttechnology in standardized  language assessment
technology in standardized language assessmentHuang YaLi
 
Language Assessment - Beyond Test-Alternatives Assessment by EFL Learners
Language Assessment - Beyond Test-Alternatives Assessment by EFL LearnersLanguage Assessment - Beyond Test-Alternatives Assessment by EFL Learners
Language Assessment - Beyond Test-Alternatives Assessment by EFL LearnersEFL Learning
 
Oral language assessment
Oral language assessmentOral language assessment
Oral language assessmentyonfreddygarzon
 
Types of language assessment
Types of language assessmentTypes of language assessment
Types of language assessmentFrancis Calubayan
 
Language Assessment_Formal and Informal
Language Assessment_Formal and InformalLanguage Assessment_Formal and Informal
Language Assessment_Formal and InformalMæäSii Mööì
 
Language Assessment Principles and Issues
Language Assessment Principles and IssuesLanguage Assessment Principles and Issues
Language Assessment Principles and IssuesMaury Martinez
 
Principles of language assessment
Principles of language assessmentPrinciples of language assessment
Principles of language assessmentAstrid Caballero
 
Informal Assessment
Informal AssessmentInformal Assessment
Informal Assessmentsongoten77
 
Portfolio Based Language Assessment (PBLA): Maximizing Assessment for Learning
Portfolio Based Language Assessment (PBLA): Maximizing Assessment for LearningPortfolio Based Language Assessment (PBLA): Maximizing Assessment for Learning
Portfolio Based Language Assessment (PBLA): Maximizing Assessment for LearningJoanne Pettis
 
Types of Language Assessment
Types of Language AssessmentTypes of Language Assessment
Types of Language AssessmentJhun Ar Ar Ramos
 
Principles of Language Assessment
Principles of Language AssessmentPrinciples of Language Assessment
Principles of Language AssessmentA Faiz
 

Viewers also liked (20)

Tim McNamara
Tim McNamara   Tim McNamara
Tim McNamara
 
Self-assessment in language learning
Self-assessment in language learningSelf-assessment in language learning
Self-assessment in language learning
 
Language Assessment - Standards-Based Assessment by EFL Learners
Language Assessment - Standards-Based Assessment by EFL LearnersLanguage Assessment - Standards-Based Assessment by EFL Learners
Language Assessment - Standards-Based Assessment by EFL Learners
 
Developing Rubrics for Language Assessment with Dr. JD Brown
Developing Rubrics for Language Assessment with Dr. JD BrownDeveloping Rubrics for Language Assessment with Dr. JD Brown
Developing Rubrics for Language Assessment with Dr. JD Brown
 
Introduction to Language Assessment by Brown
Introduction to Language Assessment by BrownIntroduction to Language Assessment by Brown
Introduction to Language Assessment by Brown
 
Assessment & Language Learning
Assessment & Language LearningAssessment & Language Learning
Assessment & Language Learning
 
Language Assessment Types
Language Assessment TypesLanguage Assessment Types
Language Assessment Types
 
technology in standardized language assessment
technology in standardized  language assessmenttechnology in standardized  language assessment
technology in standardized language assessment
 
Language Assessment - Beyond Test-Alternatives Assessment by EFL Learners
Language Assessment - Beyond Test-Alternatives Assessment by EFL LearnersLanguage Assessment - Beyond Test-Alternatives Assessment by EFL Learners
Language Assessment - Beyond Test-Alternatives Assessment by EFL Learners
 
Beyond tests alternatives in assessment
Beyond tests alternatives in assessmentBeyond tests alternatives in assessment
Beyond tests alternatives in assessment
 
Oral language assessment
Oral language assessmentOral language assessment
Oral language assessment
 
Types of language assessment
Types of language assessmentTypes of language assessment
Types of language assessment
 
Language Assessment_Formal and Informal
Language Assessment_Formal and InformalLanguage Assessment_Formal and Informal
Language Assessment_Formal and Informal
 
Language Assessment Principles and Issues
Language Assessment Principles and IssuesLanguage Assessment Principles and Issues
Language Assessment Principles and Issues
 
Principles of language assessment
Principles of language assessmentPrinciples of language assessment
Principles of language assessment
 
Informal Assessment
Informal AssessmentInformal Assessment
Informal Assessment
 
Portfolio Based Language Assessment (PBLA): Maximizing Assessment for Learning
Portfolio Based Language Assessment (PBLA): Maximizing Assessment for LearningPortfolio Based Language Assessment (PBLA): Maximizing Assessment for Learning
Portfolio Based Language Assessment (PBLA): Maximizing Assessment for Learning
 
Types of Assessment
Types of AssessmentTypes of Assessment
Types of Assessment
 
Types of Language Assessment
Types of Language AssessmentTypes of Language Assessment
Types of Language Assessment
 
Principles of Language Assessment
Principles of Language AssessmentPrinciples of Language Assessment
Principles of Language Assessment
 

Similar to How do we assess task-based performance? with Dr. John Norris

Fundamental concepts and principles in Language Testing
Fundamental concepts and principles in Language TestingFundamental concepts and principles in Language Testing
Fundamental concepts and principles in Language TestingPhạm Phúc Khánh Minh
 
Language Testing :kinds of tests
Language Testing :kinds of testsLanguage Testing :kinds of tests
Language Testing :kinds of testsahmedabbas1121
 
Summary of all the chapters
Summary of all the chaptersSummary of all the chapters
Summary of all the chapterskashmasardar
 
Assessment of oral skills roleplay
Assessment of oral skills roleplayAssessment of oral skills roleplay
Assessment of oral skills roleplayLourdes Pomposo
 
Assesing speaking skills
Assesing speaking skillsAssesing speaking skills
Assesing speaking skillssyed ahmed
 
Language Assessment - Assessing Speaking byEFL Learners
Language Assessment - Assessing Speaking byEFL LearnersLanguage Assessment - Assessing Speaking byEFL Learners
Language Assessment - Assessing Speaking byEFL LearnersEFL Learning
 
Assessment &testing in the classroom
Assessment &testing in the classroomAssessment &testing in the classroom
Assessment &testing in the classroomCidher89
 
Assessment &testing in the classroom
Assessment &testing in the classroomAssessment &testing in the classroom
Assessment &testing in the classroomCidher89
 
Designing Calssroom Language Test and Test Methods
Designing Calssroom Language Test and Test MethodsDesigning Calssroom Language Test and Test Methods
Designing Calssroom Language Test and Test MethodsIin Widya Lestari
 
Integrating Performance Assessment into World Language Learning and Teaching ...
Integrating Performance Assessment into World Language Learning and Teaching ...Integrating Performance Assessment into World Language Learning and Teaching ...
Integrating Performance Assessment into World Language Learning and Teaching ...Language Acquisition Resource Center
 
(4b) Tsl641 Principles For Call Evaluation (2)
(4b) Tsl641   Principles For Call Evaluation (2)(4b) Tsl641   Principles For Call Evaluation (2)
(4b) Tsl641 Principles For Call Evaluation (2)Izaham
 
Task based language test
Task based language testTask based language test
Task based language testVronit82
 
Unit1(testing, assessing, and teaching)
Unit1(testing, assessing, and teaching)Unit1(testing, assessing, and teaching)
Unit1(testing, assessing, and teaching)Kheang Sokheng
 
Reliability of pronunciation tests seminar
Reliability of pronunciation tests seminarReliability of pronunciation tests seminar
Reliability of pronunciation tests seminarIrina K
 
Constructing rubrics to assess productive skills
Constructing rubrics to assess productive skillsConstructing rubrics to assess productive skills
Constructing rubrics to assess productive skillsjohan cuya garcía
 
UTPL-LENGUAGE TESTING-I-BIMESTRE-(OCTUBRE 2011-FEBRERO 2012)
UTPL-LENGUAGE TESTING-I-BIMESTRE-(OCTUBRE 2011-FEBRERO 2012)UTPL-LENGUAGE TESTING-I-BIMESTRE-(OCTUBRE 2011-FEBRERO 2012)
UTPL-LENGUAGE TESTING-I-BIMESTRE-(OCTUBRE 2011-FEBRERO 2012)Videoconferencias UTPL
 
Why Test Language for Specific Purposes?
Why Test Language for Specific Purposes?Why Test Language for Specific Purposes?
Why Test Language for Specific Purposes?Parisa Mehran
 

Similar to How do we assess task-based performance? with Dr. John Norris (20)

Fundamental concepts and principles in Language Testing
Fundamental concepts and principles in Language TestingFundamental concepts and principles in Language Testing
Fundamental concepts and principles in Language Testing
 
Assessing speaking
Assessing speakingAssessing speaking
Assessing speaking
 
Language Testing :kinds of tests
Language Testing :kinds of testsLanguage Testing :kinds of tests
Language Testing :kinds of tests
 
Summary of all the chapters
Summary of all the chaptersSummary of all the chapters
Summary of all the chapters
 
50 3 10_norris
50 3 10_norris50 3 10_norris
50 3 10_norris
 
A AND E.ppt
A AND E.pptA AND E.ppt
A AND E.ppt
 
Assessment of oral skills roleplay
Assessment of oral skills roleplayAssessment of oral skills roleplay
Assessment of oral skills roleplay
 
Assesing speaking skills
Assesing speaking skillsAssesing speaking skills
Assesing speaking skills
 
Language Assessment - Assessing Speaking byEFL Learners
Language Assessment - Assessing Speaking byEFL LearnersLanguage Assessment - Assessing Speaking byEFL Learners
Language Assessment - Assessing Speaking byEFL Learners
 
Assessment &testing in the classroom
Assessment &testing in the classroomAssessment &testing in the classroom
Assessment &testing in the classroom
 
Assessment &testing in the classroom
Assessment &testing in the classroomAssessment &testing in the classroom
Assessment &testing in the classroom
 
Designing Calssroom Language Test and Test Methods
Designing Calssroom Language Test and Test MethodsDesigning Calssroom Language Test and Test Methods
Designing Calssroom Language Test and Test Methods
 
Integrating Performance Assessment into World Language Learning and Teaching ...
Integrating Performance Assessment into World Language Learning and Teaching ...Integrating Performance Assessment into World Language Learning and Teaching ...
Integrating Performance Assessment into World Language Learning and Teaching ...
 
(4b) Tsl641 Principles For Call Evaluation (2)
(4b) Tsl641   Principles For Call Evaluation (2)(4b) Tsl641   Principles For Call Evaluation (2)
(4b) Tsl641 Principles For Call Evaluation (2)
 
Task based language test
Task based language testTask based language test
Task based language test
 
Unit1(testing, assessing, and teaching)
Unit1(testing, assessing, and teaching)Unit1(testing, assessing, and teaching)
Unit1(testing, assessing, and teaching)
 
Reliability of pronunciation tests seminar
Reliability of pronunciation tests seminarReliability of pronunciation tests seminar
Reliability of pronunciation tests seminar
 
Constructing rubrics to assess productive skills
Constructing rubrics to assess productive skillsConstructing rubrics to assess productive skills
Constructing rubrics to assess productive skills
 
UTPL-LENGUAGE TESTING-I-BIMESTRE-(OCTUBRE 2011-FEBRERO 2012)
UTPL-LENGUAGE TESTING-I-BIMESTRE-(OCTUBRE 2011-FEBRERO 2012)UTPL-LENGUAGE TESTING-I-BIMESTRE-(OCTUBRE 2011-FEBRERO 2012)
UTPL-LENGUAGE TESTING-I-BIMESTRE-(OCTUBRE 2011-FEBRERO 2012)
 
Why Test Language for Specific Purposes?
Why Test Language for Specific Purposes?Why Test Language for Specific Purposes?
Why Test Language for Specific Purposes?
 

More from Language Acquisition Resource Center

Designing Writing Assessments and Rubrics with Dr. Deborah Crusan
Designing Writing Assessments and Rubrics with Dr. Deborah Crusan Designing Writing Assessments and Rubrics with Dr. Deborah Crusan
Designing Writing Assessments and Rubrics with Dr. Deborah Crusan Language Acquisition Resource Center
 
Assessing Language Using Computer Technology with Dr. Dr. Volker H. Hegelheimer
Assessing Language Using Computer Technology with Dr. Dr. Volker H. HegelheimerAssessing Language Using Computer Technology with Dr. Dr. Volker H. Hegelheimer
Assessing Language Using Computer Technology with Dr. Dr. Volker H. HegelheimerLanguage Acquisition Resource Center
 
Pathways to Excellence: Using Backward Design Principles for Instruction and ...
Pathways to Excellence: Using Backward Design Principles for Instruction and ...Pathways to Excellence: Using Backward Design Principles for Instruction and ...
Pathways to Excellence: Using Backward Design Principles for Instruction and ...Language Acquisition Resource Center
 
Diagnosing Strengths and Weaknesses of Foreign/Second Language Readers with D...
Diagnosing Strengths and Weaknesses of Foreign/Second Language Readers with D...Diagnosing Strengths and Weaknesses of Foreign/Second Language Readers with D...
Diagnosing Strengths and Weaknesses of Foreign/Second Language Readers with D...Language Acquisition Resource Center
 
Foreign Language Classroom Assessment in Support of Teaching and Learning wit...
Foreign Language Classroom Assessment in Support of Teaching and Learning wit...Foreign Language Classroom Assessment in Support of Teaching and Learning wit...
Foreign Language Classroom Assessment in Support of Teaching and Learning wit...Language Acquisition Resource Center
 
NO PROGRAMMING NECESSARY: Tools for creating and sharing online language reso...
NO PROGRAMMING NECESSARY: Tools for creating and sharing online language reso...NO PROGRAMMING NECESSARY: Tools for creating and sharing online language reso...
NO PROGRAMMING NECESSARY: Tools for creating and sharing online language reso...Language Acquisition Resource Center
 
What’s So Epic About This Legend? Blending Technologies to Teach Language and...
What’s So Epic About This Legend? Blending Technologies to Teach Language and...What’s So Epic About This Legend? Blending Technologies to Teach Language and...
What’s So Epic About This Legend? Blending Technologies to Teach Language and...Language Acquisition Resource Center
 
Don't Put Your Cell Phone Away: Polleverywhere and Google Voice for the Langu...
Don't Put Your Cell Phone Away: Polleverywhere and Google Voice for the Langu...Don't Put Your Cell Phone Away: Polleverywhere and Google Voice for the Langu...
Don't Put Your Cell Phone Away: Polleverywhere and Google Voice for the Langu...Language Acquisition Resource Center
 

More from Language Acquisition Resource Center (14)

Assessing Vocabulary with Dr. John Read
Assessing Vocabulary with Dr. John ReadAssessing Vocabulary with Dr. John Read
Assessing Vocabulary with Dr. John Read
 
Designing Writing Assessments and Rubrics with Dr. Deborah Crusan
Designing Writing Assessments and Rubrics with Dr. Deborah Crusan Designing Writing Assessments and Rubrics with Dr. Deborah Crusan
Designing Writing Assessments and Rubrics with Dr. Deborah Crusan
 
Assessing Language Using Computer Technology with Dr. Dr. Volker H. Hegelheimer
Assessing Language Using Computer Technology with Dr. Dr. Volker H. HegelheimerAssessing Language Using Computer Technology with Dr. Dr. Volker H. Hegelheimer
Assessing Language Using Computer Technology with Dr. Dr. Volker H. Hegelheimer
 
Pathways to Excellence: Using Backward Design Principles for Instruction and ...
Pathways to Excellence: Using Backward Design Principles for Instruction and ...Pathways to Excellence: Using Backward Design Principles for Instruction and ...
Pathways to Excellence: Using Backward Design Principles for Instruction and ...
 
Diagnosing Strengths and Weaknesses of Foreign/Second Language Readers with D...
Diagnosing Strengths and Weaknesses of Foreign/Second Language Readers with D...Diagnosing Strengths and Weaknesses of Foreign/Second Language Readers with D...
Diagnosing Strengths and Weaknesses of Foreign/Second Language Readers with D...
 
Assessing Listening with Dr. Larry Vandergrift
Assessing Listening with Dr. Larry VandergriftAssessing Listening with Dr. Larry Vandergrift
Assessing Listening with Dr. Larry Vandergrift
 
Foreign Language Classroom Assessment in Support of Teaching and Learning wit...
Foreign Language Classroom Assessment in Support of Teaching and Learning wit...Foreign Language Classroom Assessment in Support of Teaching and Learning wit...
Foreign Language Classroom Assessment in Support of Teaching and Learning wit...
 
NO PROGRAMMING NECESSARY: Tools for creating and sharing online language reso...
NO PROGRAMMING NECESSARY: Tools for creating and sharing online language reso...NO PROGRAMMING NECESSARY: Tools for creating and sharing online language reso...
NO PROGRAMMING NECESSARY: Tools for creating and sharing online language reso...
 
Using MOBILE PHONES for Language Teaching and Learning
Using MOBILE PHONES for Language Teaching and LearningUsing MOBILE PHONES for Language Teaching and Learning
Using MOBILE PHONES for Language Teaching and Learning
 
What’s So Epic About This Legend? Blending Technologies to Teach Language and...
What’s So Epic About This Legend? Blending Technologies to Teach Language and...What’s So Epic About This Legend? Blending Technologies to Teach Language and...
What’s So Epic About This Legend? Blending Technologies to Teach Language and...
 
LARC ILR at Ed presentation
LARC ILR at Ed presentationLARC ILR at Ed presentation
LARC ILR at Ed presentation
 
Don't Put Your Cell Phone Away: Polleverywhere and Google Voice for the Langu...
Don't Put Your Cell Phone Away: Polleverywhere and Google Voice for the Langu...Don't Put Your Cell Phone Away: Polleverywhere and Google Voice for the Langu...
Don't Put Your Cell Phone Away: Polleverywhere and Google Voice for the Langu...
 
Introducing Project GO - ROTC Language and Culture Project
Introducing Project GO - ROTC Language and Culture ProjectIntroducing Project GO - ROTC Language and Culture Project
Introducing Project GO - ROTC Language and Culture Project
 
Project GO - The ROTC Language & Culture Project
Project GO - The ROTC Language & Culture ProjectProject GO - The ROTC Language & Culture Project
Project GO - The ROTC Language & Culture Project
 

Recently uploaded

ICS2208 Lecture6 Notes for SL spaces.pdf
ICS2208 Lecture6 Notes for SL spaces.pdfICS2208 Lecture6 Notes for SL spaces.pdf
ICS2208 Lecture6 Notes for SL spaces.pdfVanessa Camilleri
 
Dust Of Snow By Robert Frost Class-X English CBSE
Dust Of Snow By Robert Frost Class-X English CBSEDust Of Snow By Robert Frost Class-X English CBSE
Dust Of Snow By Robert Frost Class-X English CBSEaurabinda banchhor
 
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdfGrade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdfJemuel Francisco
 
Choosing the Right CBSE School A Comprehensive Guide for Parents
Choosing the Right CBSE School A Comprehensive Guide for ParentsChoosing the Right CBSE School A Comprehensive Guide for Parents
Choosing the Right CBSE School A Comprehensive Guide for Parentsnavabharathschool99
 
Q4-PPT-Music9_Lesson-1-Romantic-Opera.pptx
Q4-PPT-Music9_Lesson-1-Romantic-Opera.pptxQ4-PPT-Music9_Lesson-1-Romantic-Opera.pptx
Q4-PPT-Music9_Lesson-1-Romantic-Opera.pptxlancelewisportillo
 
EMBODO Lesson Plan Grade 9 Law of Sines.docx
EMBODO Lesson Plan Grade 9 Law of Sines.docxEMBODO Lesson Plan Grade 9 Law of Sines.docx
EMBODO Lesson Plan Grade 9 Law of Sines.docxElton John Embodo
 
Measures of Position DECILES for ungrouped data
Measures of Position DECILES for ungrouped dataMeasures of Position DECILES for ungrouped data
Measures of Position DECILES for ungrouped dataBabyAnnMotar
 
Concurrency Control in Database Management system
Concurrency Control in Database Management systemConcurrency Control in Database Management system
Concurrency Control in Database Management systemChristalin Nelson
 
ENG 5 Q4 WEEk 1 DAY 1 Restate sentences heard in one’s own words. Use appropr...
ENG 5 Q4 WEEk 1 DAY 1 Restate sentences heard in one’s own words. Use appropr...ENG 5 Q4 WEEk 1 DAY 1 Restate sentences heard in one’s own words. Use appropr...
ENG 5 Q4 WEEk 1 DAY 1 Restate sentences heard in one’s own words. Use appropr...JojoEDelaCruz
 
Textual Evidence in Reading and Writing of SHS
Textual Evidence in Reading and Writing of SHSTextual Evidence in Reading and Writing of SHS
Textual Evidence in Reading and Writing of SHSMae Pangan
 
ROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptxROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptxVanesaIglesias10
 
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptxMULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptxAnupkumar Sharma
 
How to Add Barcode on PDF Report in Odoo 17
How to Add Barcode on PDF Report in Odoo 17How to Add Barcode on PDF Report in Odoo 17
How to Add Barcode on PDF Report in Odoo 17Celine George
 
How to do quick user assign in kanban in Odoo 17 ERP
How to do quick user assign in kanban in Odoo 17 ERPHow to do quick user assign in kanban in Odoo 17 ERP
How to do quick user assign in kanban in Odoo 17 ERPCeline George
 
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxINTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxHumphrey A Beña
 
ANG SEKTOR NG agrikultura.pptx QUARTER 4
ANG SEKTOR NG agrikultura.pptx QUARTER 4ANG SEKTOR NG agrikultura.pptx QUARTER 4
ANG SEKTOR NG agrikultura.pptx QUARTER 4MiaBumagat1
 
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTS
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTSGRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTS
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTSJoshuaGantuangco2
 

Recently uploaded (20)

ICS2208 Lecture6 Notes for SL spaces.pdf
ICS2208 Lecture6 Notes for SL spaces.pdfICS2208 Lecture6 Notes for SL spaces.pdf
ICS2208 Lecture6 Notes for SL spaces.pdf
 
Paradigm shift in nursing research by RS MEHTA
Paradigm shift in nursing research by RS MEHTAParadigm shift in nursing research by RS MEHTA
Paradigm shift in nursing research by RS MEHTA
 
Dust Of Snow By Robert Frost Class-X English CBSE
Dust Of Snow By Robert Frost Class-X English CBSEDust Of Snow By Robert Frost Class-X English CBSE
Dust Of Snow By Robert Frost Class-X English CBSE
 
FINALS_OF_LEFT_ON_C'N_EL_DORADO_2024.pptx
FINALS_OF_LEFT_ON_C'N_EL_DORADO_2024.pptxFINALS_OF_LEFT_ON_C'N_EL_DORADO_2024.pptx
FINALS_OF_LEFT_ON_C'N_EL_DORADO_2024.pptx
 
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptxINCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
 
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdfGrade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
 
Choosing the Right CBSE School A Comprehensive Guide for Parents
Choosing the Right CBSE School A Comprehensive Guide for ParentsChoosing the Right CBSE School A Comprehensive Guide for Parents
Choosing the Right CBSE School A Comprehensive Guide for Parents
 
Q4-PPT-Music9_Lesson-1-Romantic-Opera.pptx
Q4-PPT-Music9_Lesson-1-Romantic-Opera.pptxQ4-PPT-Music9_Lesson-1-Romantic-Opera.pptx
Q4-PPT-Music9_Lesson-1-Romantic-Opera.pptx
 
EMBODO Lesson Plan Grade 9 Law of Sines.docx
EMBODO Lesson Plan Grade 9 Law of Sines.docxEMBODO Lesson Plan Grade 9 Law of Sines.docx
EMBODO Lesson Plan Grade 9 Law of Sines.docx
 
Measures of Position DECILES for ungrouped data
Measures of Position DECILES for ungrouped dataMeasures of Position DECILES for ungrouped data
Measures of Position DECILES for ungrouped data
 
Concurrency Control in Database Management system
Concurrency Control in Database Management systemConcurrency Control in Database Management system
Concurrency Control in Database Management system
 
ENG 5 Q4 WEEk 1 DAY 1 Restate sentences heard in one’s own words. Use appropr...
ENG 5 Q4 WEEk 1 DAY 1 Restate sentences heard in one’s own words. Use appropr...ENG 5 Q4 WEEk 1 DAY 1 Restate sentences heard in one’s own words. Use appropr...
ENG 5 Q4 WEEk 1 DAY 1 Restate sentences heard in one’s own words. Use appropr...
 
Textual Evidence in Reading and Writing of SHS
Textual Evidence in Reading and Writing of SHSTextual Evidence in Reading and Writing of SHS
Textual Evidence in Reading and Writing of SHS
 
ROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptxROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptx
 
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptxMULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
 
How to Add Barcode on PDF Report in Odoo 17
How to Add Barcode on PDF Report in Odoo 17How to Add Barcode on PDF Report in Odoo 17
How to Add Barcode on PDF Report in Odoo 17
 
How to do quick user assign in kanban in Odoo 17 ERP
How to do quick user assign in kanban in Odoo 17 ERPHow to do quick user assign in kanban in Odoo 17 ERP
How to do quick user assign in kanban in Odoo 17 ERP
 
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxINTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
 
ANG SEKTOR NG agrikultura.pptx QUARTER 4
ANG SEKTOR NG agrikultura.pptx QUARTER 4ANG SEKTOR NG agrikultura.pptx QUARTER 4
ANG SEKTOR NG agrikultura.pptx QUARTER 4
 
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTS
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTSGRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTS
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTS
 

How do we assess task-based performance? with Dr. John Norris

  • 1. How do we assess task- based performance? Recent advances in language assessment LARC/CALPER webinar April 3, 2014 John M. Norris – Georgetown University Copyright © John M. Norris, 2014: Please cite as: Norris, J. M. (2014, April). How do we assess task-based performance? Invited LARC/CALPER testing and assessment webinar.
  • 2. Plan: 0. What is assessment and why do we do it? 1. Traditions, alternatives  Task-based assessment 2. The range of uses for task-based language assessment 3. Issues in assessing task-based language performance 4. Some challenges and opportunities in TBLA Task-based language assessment: the elicitation and evaluation of language use (across all modalities) for expressing and interpreting meaning, within a well-defined communicative context (and audience), for a clear purpose, towards a valued goal or outcome.
  • 3. What is assessment? What isn’t assessment (in language education)? CEFR B1
  • 4. What is assessment and why to we do it? Robert Mislevy (2004) Assessment is itself a particular kind of narrative: • an evidentiary argument • about aspects of what students know and can do, • based on a handful of particular things they have said, done, or made. What is educational assessment? Which narratives do we want to tell in language assessment?
  • 5. 2. The European Union and the United States: Possible Comparisons and Lessons You have been invited to give a lecture at the Society for German-American Relations in Weimar next month. For your contribution, the organizers are looking for a presentation on the much-discussed topic “The European Union and the Founding of the United States – A Model?” Your talk is limited to 15 minutes, to be followed by discussion of your remarks. What is assessment and why do it? 1. Fill in the missing letters to create complete words that make sense in the passage. This is an example C-test passage. Starting wi_____ the sec____ word o____ this sent____, the la____ half fr____ each consec____ word h____ been del____. C-tests a____ typically comp____ of mult____ texts wh____ become increa____ difficult. Ea____ text usu____ contains bet____ 20 a____ 25 it____. So, wh____ do y____ think ge____ tested on a C-test? Which is the ‘better’ language assessment?
  • 6. The ‘correct’ answer: IT DEPENDS ON THE INTENDED USE of the TEST!
  • 7. Specifying intended uses for assessment (Norris, 2000) INTENDED ASSESSMENT USE WHO? Test Users WHAT? Test Information IMPACT? Test Consequences WHY? Test Purposes The ‘correct’ answer: IT DEPENDS ON THE INTENDED USE of the TEST!
  • 8. 2. The European Union and the United States: Possible Comparisons and Lessons You have been invited to give a lecture at the Society for German-American Relations in Weimar next month. For your contribution, the organizers are looking for a presentation on the much-discussed topic “The European Union and the Founding of the United States – A Model?” Your talk is limited to 15 minutes, to be followed by discussion of your remarks. What is assessment and why do it? 1. Fill in the missing letters to create complete words that make sense in the passage. This is an example C-test passage. Starting wi_____ the sec____ word o____ this sent____, the la____ half fr____ each consec____ word h____ been del____. C-tests a____ typically comp____ of mult____ texts wh____ become increa____ difficult. Ea____ text usu____ contains bet____ 20 a____ 25 it____. So, wh____ do y____ think ge____ tested on a C-test? Intended use: quick estimate of language proficiency, e.g., for use in placing students into language courses…not task- based assessment. Which is the ‘better’ language assessment?
  • 9. 2. The European Union and the United States: Possible Comparisons and Lessons You have been invited to give a lecture at the Society for German-American Relations in Weimar next month. For your contribution, the organizers are looking for a presentation on the much-discussed topic “The European Union and the Founding of the United States – A Model?” Your talk is limited to 15 minutes, to be followed by discussion of your remarks. What is assessment and why do it? 1. Fill in the missing letters to create complete words that make sense in the passage. This is an example C-test passage. Starting wi_____ the sec____ word o____ this sent____, the la____ half fr____ each consec____ word h____ been del____. C-tests a____ typically comp____ of mult____ texts wh____ become increa____ difficult. Ea____ text usu____ contains bet____ 20 a____ 25 it____. So, wh____ do y____ think ge____ tested on a C-test? Intended use: quick estimate of language proficiency, e.g., for use in placing students into language courses…not task- based assessment. Intended use: extended, authentic, and integrated performance with the target language, e.g., for end-of-unit achievement testing and feedback…yes, task-based assessment. Which is the ‘better’ language assessment?
  • 10. Making decisions about students admissions placement course credit fulfilling requirements grading /achievement advancement/retention certification of abilities awarding funding ETC. INDIVIDUAL DECISION- MAKING EDUCATIONAL USES FOR ASSESSMENT
  • 11. Making decisions about students admissions placement course credit fulfilling requirements grading /achievement advancement/retention certification of abilities awarding funding ETC. Informing classroom teaching and learning diagnosing learner needs motivating students understanding learning outcomes improving instruction defining learning expectations focusing students’ learning/study ETC. INDIVIDUAL DECISION- MAKING CLASSROOM FEEDBACK EDUCATIONAL USES FOR ASSESSMENT
  • 12. Making decisions about students admissions placement course credit fulfilling requirements grading /achievement advancement/retention certification of abilities awarding funding ETC. Informing classroom teaching and learning diagnosing learner needs motivating students understanding learning outcomes improving instruction defining learning expectations focusing students’ learning/study ETC. Improving, ensuring, demonstrating quality demonstrating student learning clarifying program mission/needs revising curriculum/ pedagogy improving teacher performance communicating beyond, about the program responding to program review ETC. INDIVIDUAL DECISION- MAKING CLASSROOM FEEDBACK PROGRAM EVALUATION aligning, articulating program practices EDUCATIONAL USES FOR ASSESSMENT
  • 13. Making decisions about students admissions placement course credit fulfilling requirements grading /achievement advancement/retention certification of abilities awarding funding ETC. Informing classroom teaching and learning diagnosing learner needs motivating students understanding learning outcomes improving instruction defining learning expectations focusing students’ learning/study ETC. Improving, ensuring, demonstrating quality demonstrating student learning clarifying program mission/needs revising curriculum/ pedagogy improving teacher performance communicating beyond, about the program responding to program review ETC. INDIVIDUAL DECISION- MAKING CLASSROOM FEEDBACK PROGRAM EVALUATION aligning, articulating program practices EDUCATIONAL USES FOR ASSESSMENTWhat different kinds of assessment are needed to meet these different uses? How might tasks play a role?
  • 15. Alternatives in assessment: The move to task
  • 16. Early interest in ‘tasks’ for assessment “If we want to find out how well a person can perform a task, we can put him to work at that task, and observe how well he does it and the quality and quantity of the product he turns out.” Cureton (1950): “Validity” e.g., in language testing: Foreign Service Institute Oral Interview (1950s) Using tasks to elicit a holistic estimate of functional language ability, from 0 (no ability) to 5 (educated native speaker ability)
  • 17. Standardized testing and psychometrics But, the primary tradition of assessment in the 20th century: Large-scale standardized testing: minimal competency, accountability emphasis on efficiency, technology (e.g., scoring), discrete-point primary concern with psychometrics, norm-referencing validity = narrowly focused on construct representation, reliability e.g., in language testing: Test of English as a Foreign Language (1964)* MCT: vocabulary, reading, listening, structure, grammar (emphasis on testing language knowledge) *But see later iterations of TOEFL…
  • 18. Influence of traditional testing When we ask teachers what an assessment looks like, what do they tell us? • True-false • Matching • Multiple choice • Fill-in the blank • Exam • Test • Giving grades • Accountability • ETC.
  • 19. Influence of traditional testing When we ask teachers what an assessment looks like, what do they tell us? Washback of testing traditions: Narrowing teaching and learning Misrepresenting assessment Disconnect between teaching- testing-learning • True-false • Matching • Multiple choice • Fill-in the blank • Exam • Test • Giving grades • Accountability • ETC.
  • 20. “It is the simplification of competencies to what can be ‘objectively’ tested which has a detrimental effect: the learning of trivial facts, the reduction of subtle understanding to generalizations and stereotypes, the lack of attention to interaction and engagement because these are not tested.” e.g., Michael Byram (1997): Need for change in intercultural assessment Challenges to ‘traditional’ testing
  • 21. Reactions to ‘traditional’ testing Portfolios Performance-based assessments Essays CBTs Self-assessments Interviews Rating scales, rubrics Alternative assessments 1980s-90s •Authenticity •Performance •Can-do (vs. know) •Classroom •Washback
  • 22. http://www.learner.org/libraries/tfl/assessment/analyze.html? pop=yes&pid=2003 The move to task as assessment medium Grant Wiggins (1998): Educative assessment The primary assessment medium should be authentic tasks which have learners demonstrate what they can do with what they are learning; enables meaningful integration of teaching, learning, and assessment assessment becomes educative (informative) rather than just an authoritarian mechanism for control.
  • 23. The move to task-based language assessment “…the process of evaluating, in relation to a set of explicitly stated criteria, the quality of the communicative performances elicited from learners as part of goal-directed, meaning-focused language use requiring the integration of skills and knowledge” Geoff Brindley (1994): Task-centered assessment aligned with CLT, TBLT movements in pedagogy emphasis on authenticity of L2 use validity = capturing ability for use in valued tasks classroom applications obvious, but beyond?
  • 24. Operationalizing task-based language assessment “…genuinely task-based language assessment takes the task itself as the fundamental unit of analysis, motivating item selection, test instrument construction, and the rating of task performance” Long & Norris (2000): Task-based language assessment 1. Specify intended uses for the assessment: who, what, why, impact 2. Select and analyze key target tasks/features from needs analysis 3. Design tests and items: emphasize authentic performances 4. Determine real-world criteria for rating task performance qualities 5. Pilot-test and revise instruments and procedures (raters/rating) 6. Evaluate validity in terms of intended uses, and especially impact on teaching and learning.
  • 25. Norris et al. (1998) Tasks respond to diverse needs/uses in assessment • Focus on what L2 users can actually do with the language • Integrate language form and meaning into what gets assessed • Provide useful feedback to teachers, learners, curriculum • ‘Wash back’ on teaching and learning by making outcomes ‘real’ • Counter negative influence of traditional knowledge assessment • Align testing with new, evolving language pedagogies Task-based language assessment: the elicitation and evaluation of language use (across all modalities) for expressing and interpreting meaning, within a well-defined communicative context (and audience), for a clear purpose, towards a valued goal or outcome. Why task-based language assessment?
  • 26. Up Next: How do we assess task-based performance in language classes and programs? Issues, examples, suggestions… But First: Any Questions?
  • 27. Task-based assessment in language classes and programs Issues 1. Which tasks do we assess? 2. How do we elicit task-based performance? 3. What gets assessed in task performance? 4. How do we score task performance?
  • 28. 1. Which tasks do we assess? e.g.: EAP pragmatics assessment (Youn, 2009) 1. Write a recommendation request e-mail to professor 2. Write an e-mail to potential employer to send application packet 3. Write an e-mail to refuse professor’s request 4. Write constructive comments on cover letter written by classmate 5. Give oral peer-feedback on classmate’s request e-mail to professor 6. Interacting with classmates in situations of making suggestions and disagreement 7. Interacting with a professor in situations of making requests and refusal Why these tasks?
  • 29. 1. Which tasks do we assess? Why these tasks?Which tasks? Based on needs analysis for specific English language program context (individual & interactive tasks) High priority tasks reflecting pragmatic challenges for college-level international students Each task serves as end-of-unit achievement assessment Combined, all of the tasks can be used for diagnostic assessment
  • 30. You have been given the opportunity of a lifetime to apply for an athletic training camp in [e.g., Spain], tuition free! This camp trains young people in all sports, from the extreme (snowboarding, bicycle motor cross, rollerblading) to team sports of all kinds (basketball to volleyball). You name it, they help you train for it! To be accepted into the camp, all applicants have to convince the admissions office that they have good exercise and nutrition habits. First, you will read about health and nutrition from the perspective of the Spanish – speaking world. Then you will discuss your eating and exercise regimen with your partner to compare your nutrition and exercise, perhaps event to get some ideas. You will then write your application letter to the summer camp describing your nutrition and training regimen, convincing them that you are well prepared for the camp and need to be accepted. 1. Which tasks do we assess? e.g., Integrated performance assessment in US schools Overall introduction to a sequence of interrelated pedagogic/assessment tasks. Note the distinct task phases. Consortium for assessing performance standards http://flenj.org/CAPS/?page=parent
  • 31. 1. Which tasks do we assess? e.g., Integrated performance assessment in US schools Consortium for assessing performance standards http://flenj.org/CAPS/?page=parent Individual reading task Pair discussion task Letter writing task Why these tasks?
  • 32. 1. Which tasks do we assess? e.g., Integrated performance assessment in US schools Consortium for assessing performance standards http://flenj.org/CAPS/?page=parent Individual reading task Pair discussion task Letter writing task Self-assessment: understanding task content, language Peer-assessment: feedback on development and expression of ideas Teacher-assessment: effectiveness of letter, language use Why these tasks?
  • 33. 1. Which tasks do we assess? Which tasks? Based on the U.S. Standards for Foreign Language Learning, by specific age/grade level and proficiency 3 integrated tasks—Interpretive, interpersonal, presentational—reflect distinct modes of performance Tasks are closely integrated with what is being taught: this is why you are learning what you are learning Tasks provide the opportunity for self-, peer-, and teacher-assessment and feedback Assessment raises awareness about effectiveness of teaching and learning See Adair-Hauck, et al. (2006) Why these tasks?
  • 34. 2. How do we elicit task-based performance? The European Union and the United States: Possible Comparisons and Lessons Task You have been invited to give a lecture at the Society for German-American Relations in Weimar next month. For your contribution, the organizers are looking for a presentation on the much-discussed topic “The European Union and the Founding of the United States – A Model?” Your talk is limited to 15 minutes, to be followed by discussion of your remarks. Please observe your position as a speaker to this particular audience. While you should expect good background knowledge from your German audience regarding the situation in the EU, you primarily contribute the perspective of an American who is familiar with the foundational history of the USA and subsequent development of the county. In its particularities this is most likely to be less familiar to your audience. Therefore, it is precisely those aspects that you must bring to the awareness of your audience in order to link them to the present situation in Europe. In other words, continuously keep in mind this occasion for your speech, this starting point for your presentation, and this intention of your talk. e.g.: Georgetown German: Prototypical Performance Writing Task Level IV course, “Text in Context” What does this description accomplish?
  • 35. Example: CNaVT Certificate in Dutch as a Foreign Language Tourist domain Listening comprehension task “Visit to the museum” Item: Identify the paintings based on the descriptions you hear from an audio museum guide. What is the purpose of the paintings?
  • 36. Task Title: Recycling Theme: Environment Level: Novice-Mid (Focus Age Group: 11 – 14) Communicative Mode: Presentational Time Frame: One 42 minute period or more depending on the class and the type of presentations chosen. Description of Task: Now that you’ve learned about saving the planet and the need to reuse and recycle, you want to share with your family your ideas for reusing some throw away things. You have cleverly thought of a few uses for reusing ____________. Now you will share your new ideas. Create and develop your idea and choose one from the following types of presentations to present it. Continued on next page… Consortium for assessing performance standards http://flenj.org/CAPS/?page=parent e.g.: assessing young/beginning learners, US schools 2. How do we elicit task-based performance?
  • 37. Title: ¿Qué puedo hacer con_________? (What can I do with_______?) Include the following in your presentation: • four or more ways of re-using something • specific uses • descriptions/elaboration Three options, you choose: Power point presentation Poster Live oral 3-D presentation Materials Needed: computers, markers, poster board, realia Why give the student these options? 2. How do we elicit task-based performance?
  • 38. 2. How do we elicit task-based performance? Task-based performance elicitation? Give a purpose for communicating (motivate) For interactive/presentational tasks, indicate or provide an audience Provide a structure or procedures for doing the task Utilize authentic input and other realia for situating the task in a meaningful context Indicate desired outcomes or products of the task Where feasible, encourage ownership over the task through individual choice in its accomplishment
  • 39. 3. What gets assessed in task performance? Test task: Order pizza for the class. Take orders Place orders Task accomplishment?
  • 40. 3. What gets assessed in task performance? Test task: Order pizza for the class. XTake orders Place orders Task accomplishment?
  • 41. 3. What gets assessed in task performance? e.g.: classroom assessments for middle-school learners in Flanders (Jordens, 2007) Type task: (from learning goals) Pupils can report to the teacher on a subject, previously discussed at school Test task: Pupils report to the teacher on a film about scuba diving, the report being based on photographs extracted from the film.
  • 42. e.g.: classroom assessments for middle-school learners in Flanders Parameters for assessment What does pupil say? Content relevance, completeness Clarity Organization How does pupil say it? Fluency Articulation Manner of speaking Variation and extensiveness Accuracy: word/sentence Interaction skills 3. What gets assessed in task performance? How is this rubric used?
  • 43. Give a presentation in Spanish about your life and interests. Indicate how these activities reflect your personality. Include some possibilities for your future in terms of career, study, travel, pursuing personal interests. e.g.: performance assessment for 8th graders in US school 3. What gets assessed in task performance? Source: http://depts.washington.edu/mellwa/Events/20081105 /sandrock_ipa_handout.pdf
  • 44. EXCEEDS EXPECTATIONS MEETS EXPECTATIONS DOES NOT MEET EXPECTATIONS Do we understand you? (Comprehensibility) My message is easily understood by my audience. I am able to communicate my message by reproducing memorized material. I rely heavily on visuals to convey my message. How do I use the Spanish language? (Language Control & Vocabulary Use) I can write/speak in simple sentences correctly. My presentation is rich in appropriate vocabulary. I demonstrate some accuracy in written/oral presentation when I reproduce memorized words and phrases. My vocabulary reveals basic information. My presentation is correct only at the word level. My vocabulary is limited and / or repetitive. I use English. How do I organize the presentation? (Communication strategies) My presentation is supported with many enriched examples. My presentation has some visuals/realia and examples. My presentation lacks organization. How well do I impact the audience? (Impact) I use visuals/ realia and gestures, and tone of voice to maintain audience’s attention. I use some visuals/realia and gestures to maintain audience’s attention. My tone of voice is acceptable. I use a small amount of effort to maintain audience’s attention. What is the purpose of these criteria?
  • 45. Criteria for judging performance? Consider real-world criteria for task success or accomplishment, including as appropriate: Performance products: Does something happen, does something get produced? Performance steps: Do critical elements of the task happen? Performance qualities: linguistic and paralinguistic features (e.g., accuracy, fluency, gesture, pragmatics), content (e.g., meaning), other attributes (e.g., visual support, efficiency of task completion) Performance levels: How well does the task performance meet expectations, standards, etc.? 3. What gets assessed in task performance?
  • 46. e.g., Georgetown Univ. German writing assessments Writing task 1 ______ ______________ ______________ ______________ _____ Writing task 2 ______ ______________ ______________ ______________ _____ Writing task 3 ______ ______________ ______________ ______________ _____ Writing task 4 ______ ______________ ______________ ______________ _____ Writing task 5 ______ ______________ ______________ ______________ _____ End-of-level Prototypical task _________ _________________ _________________ _________________ _________________ _________________ ____ 4. How do we score task performance?
  • 47. Final Grade Feedback: Rating rubric, In-text Initial Grade Task –based assignment Level scoring rubric Writing task v. 1 ______ ______________ ______________ ______________ _____ Writing task v. 2 ______ ______________ ______________ ______________ _____ 4. How do we score task performance? Formative assessment: Process writing Providing rich feedback to enable learning Teacher Students
  • 48. End-of-level Prototypical task _________ _________________ _________________ _________________ _________________ _________________ ____ Example Journalistic portrait: The Vietnamese immigrant experience in contemporary Germany 2 raters from the curricular level + 2 raters from another level 4. How do we score task performance? Summative assessment: Performance writing Providing reliable judgments of language + task abilities
  • 49. Scoring and score reporting for different purposes? Consider how ‘scores’ will be used before deciding how to score: Formative assessment: Use public criteria and scales, and encourage self-, peer-, teacher-scoring for feedback Summative assessment: Use well-defined criteria and scales, and multiple, trained raters in scoring for judgment Holistic scoring (overall judgment) provides and indication of task success or accomplishment or completion Analytic scoring (by categories and qualities) provides a useful basis for remediation Rich and detailed score reporting enables learning from assessment (e.g., in person, score + justification) 4. How do we score task performance?
  • 50. Up Next: The range of current uses for task-based language assessment But First: Additional Questions?
  • 51. Tasks as targets: Policies & standards V. Workplace Tasks: Reading --Scan basic charts, tables, maps or schedules for information. (5) --Find routine information on the computer screen/scanner screen/computerized display screen, if available. (6) --Read information in the reception/appointment book to find available openings for a new appointment. (6) --Read a checklist to verify if all the steps in the procedure have been completed. (6) --Follow one page of clear familiar instructions. (7) --Read a reminder or complaint letter/memo, take appropriate action. (7) --Scan complex charts, tables and schedules for several specific pieces of information for comparison/contrast. (7) --Follow instructions on evacuation procedures, fire drills, or on using simple machinery/equipment.(7) --Use plain language manual with familiar topic and content in own field of knowledge to find specific information. (8) ETC… Who should be able to do these tasks? For example…
  • 52. Tasks as targets: Policies & standards V. Workplace Tasks: Reading --Scan basic charts, tables, maps or schedules for information. (5) --Find routine information on the computer screen/scanner screen/computerized display screen, if available. (6) --Read information in the reception/appointment book to find available openings for a new appointment. (6) --Read a checklist to verify if all the steps in the procedure have been completed. (6) --Follow one page of clear familiar instructions. (7) --Read a reminder or complaint letter/memo, take appropriate action. (7) --Scan complex charts, tables and schedules for several specific pieces of information for comparison/contrast. (7) --Follow instructions on evacuation procedures, fire drills, or on using simple machinery/equipment.(7) --Use plain language manual with familiar topic and content in own field of knowledge to find specific information. (8) ETC… Who should be able to do these tasks? Canadian Language Benchmarks: English Language Demands for Nursing For example…
  • 53. Tasks as targets: Policies & standards (e.g., Canadian Language Benchmarks) “The CLB is task-based: • In syllabus design, tasks are considered to be basic building blocks.• Tasks in language learning promote the integration of all aspects of communicative competence, and multilevel language processing.” CEFR europe CLB canada ACTFL SFLL U.S. TESOL/NCATE teacher development Why? Tasks provide meaningful targets: Ability to use the language Washback on teaching and learning TOC hong kong
  • 54. Tasks in large-scale proficiency assessment 4. Integrated Speaking: Read a passage from a psychology textbook and listen to the lecture that follows it. Then answer the question. Reading: Flow In psychology, the feeling of complete and energized focus in an activity is called flow. People who enter a state of flow lose their sense of time and have a feeling of great satisfaction. They become completely involved in an activity for its own sake rather than for what may result from the activity, such as money or prestige. Contrary to expectation, flow usually happens not during relaxing moments of leisure and entertainment, but when we are actively involved in a difficult enterprise, in a task that stretches our mental or physical abilities. THEN For example…
  • 55. Listen to the Lecture (Male professor): I think this will help you get a picture of what your textbook is describing. I had a friend who taught in the physics department, Professor Jones, he retired last year. . . . Anyway, I remember . . . this was a few years ago . . . I remember passing by a classroom early one morning just as he was leaving, and he looked terrible: his clothes were all rumpled, and he looked like he hadn’t slept all night. And I asked if he was OK. I was surprised when he said that he never felt better, that he was totally happy. He had spent the entire night in the classroom working on a mathematics puzzle. He didn’t stop to eat dinner; he didn’t stop to sleep . . . or even rest. He was that involved in solving the puzzle. And it didn’t even have anything to do with his teaching or research; he had just come across this puzzle accidentally, I think in a mathematics journal, and it just really interested him, so he worked furiously all night and covered the blackboards in the classroom with equations and numbers and never realized that time was passing by. Question: Explain flow and how the example used by the professor illustrates the concept. Tasks in large-scale proficiency assessment Q: Who should be able to do these tasks? For example…
  • 56. Lecture (Male professor): I think this will help you get a picture of what your textbook is describing. I had a friend who taught in the physics department, Professor Jones, he retired last year. . . . Anyway, I remember . . . this was a few years ago . . . I remember passing by a classroom early one morning just as he was leaving, and he looked terrible: his clothes were all rumpled, and he looked like he hadn’t slept all night. And I asked if he was OK. I was surprised when he said that he never felt better, that he was totally happy. He had spent the entire night in the classroom working on a mathematics puzzle. He didn’t stop to eat dinner; he didn’t stop to sleep . . . or even rest. He was that involved in solving the puzzle. And it didn’t even have anything to do with his teaching or research; he had just come across this puzzle accidentally, I think in a mathematics journal, and it just really interested him, so he worked furiously all night and covered the blackboards in the classroom with equations and numbers and never realized that time was passing by. Question: Explain flow and how the example used by the professor illustrates the concept. Tasks in large-scale proficiency assessment Q: Who should be able to do these tasks? Test of English as a Foreign Language: English for Academic Purposes Example…
  • 57. Tasks in large-scale proficiency assessment (e.g., TOEFL iBT) “Working directly with higher education institutions, we designed the TOEFL test to simulate actual academic classroom tasks.” TOEFL iBT U.S. CNaVT Holland/Belgium IELTS U.K. NMET China AMES Australia Why? Tasks provide improved interpretations, construct validity Washback on teaching and learning
  • 58. Tasks in professional language certification Example…
  • 59. Tasks in professional language certification http://www.anglo-continental.com/en/uk/forms/Aviation/Part%202%20%20ready.mp3 Q: Who should be able to do these tasks? Example…
  • 60. International Civil Aviation Organization: English for Aviation Purposes Tasks in professional language certification http://www.anglo-continental.com/en/uk/forms/Aviation/Part%202%20%20ready.mp3 For example…
  • 61. Tasks in professional language certification (e.g., ICAO, 2007) “Because of the high stakes involved, pilots and controllers deserve to be tested in a context similar to that in which they work and test content should therefore be relevant to their roles in the work-place” FBI Translation CLB- Nursing NCATE L2 teaching English Aviation Why? Tasks demonstrate ‘on-the-job’ abilities for high-stakes decisions Washback on teaching and learning OET Medical
  • 62. Overview of task In groups, learners design a brochure for the Hong Kong Tourism Board describing 4 attractions in Hong Kong which would appeal to young people of their own age. Writing a Tourist Brochure Imagine the Hong Kong Tourism Board has asked your class to design a brochure that would be of interest to young visitors of your own age. In groups of 4, design the brochure describing FOUR sites suitable to young people of your own age coming to Hong Kong. Complete this task by following the steps below: Step 1: Group Task. Discuss in your groups which sites young people would want to visit in Hong Kong. Choose one site each to investigate. For homework find out as much as you can about the site, where the site is, when it is open, what one can see / do there, what the facilities are, how one gets there, etc. Bring this information to the next class. For example… Tasks in classrooms and programs
  • 63. Step 2: Group Task. Exchange information with your group members. Tell them about the site you have found out about. Then decide how you are going to present the information in your brochure, what order you want to put your sites in, what illustrations you need, what title you want to give the brochure, etc. Step 3: Individual Task. Write a description about your chosen site (120 words). Remember to say why it is interesting. Proofread it carefully, then hand it to your teacher. Step 4: Group Task. In your groups edit your work based on your teacher’s comments. Then put together your brochure. Your brochures will be assessed on the following basis: • Task fulfilment: would your selected sites appeal to young people? • Accuracy of language and information provided: is the brochure written in good English? Is the information provided accurate? • Attractiveness of final written submission: is your brochure really attractive? Can you make it more appealing? For example… Tasks in classrooms and programs Q: Who should be able to do these tasks?
  • 64. Step 2: Group Task. Exchange information with your group members. Tell them about the site you have found out about. Then decide how you are going to present the information in your brochure, what order you want to put your sites in, what illustrations you need, what title you want to give the brochure, etc. Step 3: Individual Task. Write a description about your chosen site (120 words). Remember to say why it is interesting. Proofread it carefully, then hand it to your teacher. Step 4: Group Task. In your groups edit your work based on your teacher’s comments. Then put together your brochure. Your brochures will be assessed on the following basis: • Task fulfilment: would your selected sites appeal to young people? • Accuracy of language and information provided: is the brochure written in good English? Is the information provided accurate? • Attractiveness of final written submission: is your brochure really attractive? Can you make it more appealing? Task-Based Assessment for English Language Learning: Hong Kong, Secondary Schooling Example… Tasks in classrooms and programs Q: Who should be able to do these tasks?
  • 65. Tasks in classrooms and programs (e.g., Hong Kong Special Administrative Region) “Task-based assessment for English language learning at secondary level: One of the key principles underpinning the school English language curriculum in Hong Kong is that of task-based language teaching. In this package we introduce the features of task-based assessment (TBA) within a school-based curriculum…” http://cd1.edb.hkedcity.net/cd/eng/TBA_Eng_Sec/index.html Local practice TBLT research Can-do e.g., DIALANG Teacher guides ACTFL IPAs Why? Tasks enable feedback on learning, teaching, curriculum Washback on teaching and learning
  • 66. Task-based language assessment in practice Administrators Staff Classroom: TBLAssessment For Learning Program: Tasks as summative indicators of teaching effectiveness Profession: Tasks for certification of important abilities Large-scale: Tasks for demonstrating proficiency in societal domains (e.g., education) Standards: Tasks as valued targets for learning
  • 67. Some challenges and opportunities In Task-Based Language Assessment…
  • 68. 2002 (19.4) • How do we interpret task-based L2 performance? authentic criteria, proficiency, L2 production? • Can task-based performances be rated reliably? analytic/holistic rating scales, rater training? • What is the nature of task difficulty? cognitive complexity, learner individual diffs? • Can we generalize and/or extrapolate based on tasks? domain-sampling, ECD, underlying constructs? • How does task-based assessment impact instruction? teacher, learner, curricular consequences? Basic validity questions for TBLA Norris (2008) “Fundamentally…assessments are only good insofar as their use does good, in terms of supporting educational efforts and outcomes, and it is my intent that validity evaluation helps to ensure that they do.”
  • 69. How do we interpret task-based L2 performance? Can do a specific task Has a certain proficiency Displays X language qualities High-fidelity task simulation or replication Test of aviation English Domain sampling, ‘type’ tasks TOEFL, CNAVT Implicit, spontaneous elicitation Classroom practice, SLA research
  • 70. Can task-based performances be rated reliably? YES!!! Individual task accomplishment: single task Highly specified indigenous criteria, steps, etc. Teacher, tester, learner awareness-raising Ability for L2 use: multiple/type tasks Cross-task holistic/analytic rating scales Rater training
  • 71. What is the nature of task difficulty? Why is this task so hard? Task features e.g., number of elements, number of steps, planning time, structure, reasoning requirements, code frequency/complexity Task conditions e.g., number of participants, open/closed task, convergent/divergent, familiarity, gender, status Individual differences e.g., task familiarity, motivation, testing stakes, aptitude, personality More difficult = better or worse performance???
  • 72. Can we generalize and/or extrapolate based on tasks? Of course we can  and we do it all the time! Driving testseries of tasks (1-time!) Generalizationcan drive a car Extrapolation can negotiate a roundabout in London??? …be careful!
  • 73. Can we generalize and/or extrapolate based on tasks? 1. Listening comprehension? 2. Listening comprehension in a particular domain? 3. Listening comprehension on a specific task? 1. Adequate construct definition 2. Adequate domain sampling3. Adequate task representation
  • 74. How does task-based assessment impact instruction? Local impact (Georgetown German): “So I think it has filtered down to smaller writing assignments. First of all so that students get used to it and it’s a coherent system. Whereas before we would say, ‘Okay, now write one page about your family and your Heimat’. Now we actually make up a sheet and use the same features.” Global impact (TOEFL) “…considerable changes occurred in the amount of attention the teachers paid to the development of speaking and to the integration of different skills.”
  • 75. In sum, task-based performances and assessments may not (should not) meet all purposes for assessment within language education, but they probably offer a lot of advantages for certain purposes: Task-based performances tell us things that other item types do not (e.g., ability to accomplish particular tasks under particular conditions) Task-based performances offer rich opportunities for eliciting and observing language use with meaningful content in context (there is always more to language use than just language, and tasks capture that) Task-based performances can reveal multiple aspects of language ability and/or development within a single task (e.g., accuracy, complexity, fluency; content knowledge; procedural knowledge; expertise) Task-based performances are in many cases the most indicative targeted expectations of language classes and programs (if not, then what exactly are we teaching?) Task-based assessments provide relevant and comprehensive frames of reference for learners and teachers to generate feedback on form-function-meaning relationships (formative, learning-oriented uses) Task-based assessments communicate to teachers and learners and others what is valued within a given educational setting (motivational, educative uses) Task-based assessments may impact curriculum and educational systems, causing a rethinking of what and how teaching is occurring (washback uses) Task-based assessments generate interpretable data regarding the worthwhile outcomes of language learning programs (summative uses, e.g., for accountability, certification, program evaluation)
  • 76. Why task-based assessment? Tasks are useful: • align to instructional practices and values • provide rich samples of language use data • inform how well we teach and learn • show what L2 users can do, and motivate them to learn Task-based assessments can: • improve accuracy of interpretations • ensure abilities to use language for specific purposes • guide educational practices • counter negative washback of other assessments • enable a variety of uses, decisions, actions Q: Task-based assessment is a lot of work, and we can make somewhat accurate assessments of learners’ abilities and knowledge with short-cut tests; why should we bother with TBLA?