2. Ideas for Integrated-
skills Assessment
JACET International Conference
August 31, 2013
Kahoko Matsumoto
(Tokai University)
3. 3
1. Increasing Importance of
Integrated-skills Assessment
1. From general theories of competence
- Canale & Swain (1980)
Grammatical, Discoursal, Sociolinguistic, Strategic
- Backman (1990)
Organizational(grammatical/textual) and
Pragmatic(illocutional/sociolinguistic)
2. From what is required in the globalizing
world (=21st Century Skills)
- to form/express one’s views/opinions based on
received information (input-output connection +
critical thinking)
5. 5
Applying the 3 aspects of Inbar-Lourie’s (2008) “Language
Assessment Literacy:
- ‘Why’ refers to the rationale behind the testing. → to produce
future workers/citizens with real-life communicative skills
- ‘What’ deals with the current theories regarding assessment
and the validity and reliability of the same. → to create
valid and reliable integrated-skills tests to measure authentic
communicative skills, especially in everyday practice
- ‘How’ can look at test construction or development or the
role of assessment in a language curriculum.→choice of
methods/tasks that fit the purpose (Why), are valid and
reliable,(What), and also doable/manageable.
6. 2.TOEFL-iBT Speaking Part
Not interactional, but statistically reliable.
2 independent tasks and 4 integrated ones
Integrated tasks
- Reading + Listening → Speaking
(2 genres)=summarizing main points
- Listening →Speaking
(2 genres)=expressing opinions based on
summary
Cf: Writing part: one independent task and one
integrate one (Reading+Listening→Writing)
6
7. <The rubric for the Integrated
Speaking Items>
Score 4
1. General Description:
The response fulfills the demand of the task,
with at most minor lapses in completeness.
It is highly intelligible and exhibits sustained
coherent discourse. A response at this level is
characterized by all of the following:
7
8. <The Rubric for the Integrated
Speaking Items>
Score 4
2. Delivery:
Speech is generally clear, fluid and sustained.
It may include minor lapses or minor
difficulties with pronunciation or intonation.
Pace may vary at times as speaker attempts to
recall information. Overall intelligibility
remains high.
8
9. <The rubric for the Integrated
Speaking Items>
Score 4
3. Language Use:
The response demonstrates good control of
basic and complex grammatical structures
that allow for coherent, efficient (automatic)
expression of relevant ideas. Contains
generally effective word choice. Though some
minor (or systematic) errors or imprecise use
may be noticeable, they do not require listener
effort (or obscure meaning)
9
10. <The rubric for the Integrated
Speaking Items>
Score 4
4. Topical Development:
The response presents a clear progression of
ideas and conveys the relevant information
required by the task. It includes appropriate
details though it may have minor errors or
minor omissions.
10
11. 3. Making Integrated-skills
Tests
●Difficulty
Hard to create a balanced rubric
- Output is the only observable result.
- Detailed, strategic parts of receptive skills
(reading/listening skills) cannot be obtained.
For speaking, hard to get a complete record
- limited resource
- intelligibility of speech
- unlike writing, process-oriented method
hasn’t been established. 11
12. 3. Making Integrated-skills
Tests
●Benefit
We can motivate students toward the
acquisition of real-life communicative abilities
(positive washback effect)
Easier to establish the validity to what has been
taught (input-output connection), and we can
capitalize on the advantages of criterion-based
assessment.
We can avoid too much testing, and use the
class time for more meaningful activities. 12
13. 13
4. Choosing Appropriate
Methods/Tasks
1) performance test
- recitation/translation
- short response to a stimulus/question
- discourse completion
- summary creation
- expression of an opinion
- free writing/speaking in response to a prompt
* individual vs. interactive tasks
2) integrated-skills test
- controlled
- project/process-based assessment
* formative vs. summative assessment
Consideration of
validity and practicality
14. 14
<How: development of a solid rubric>
1) Content/ability to cover→validity
2) Assessment criteria
3) Nature of task
4) Students’ ability range→difficulty level
5) Weighting among measured aspects (global vs.
local abilities)
15. 4. Examples
1. Controlled tasks
-a paired test based on the situations dealt
with listening practice (required class)
-a Q&A and summary test about reading
in a computer room (regular elective)
-a spoken discourse completion test
about the pragmatic situations learned
(academic elective)
-Presentation based on a given material
(ESP class)
20. Reference
Backman, L.F. (1990). Fundamental Considerations in Language Testing.
Oxford: Oxford University Press.
Canale, M and M. Swain. (1980). Theoretical bases of communicative
approaches to second language teaching and testing. Applied
Linguistics 1, 1-47.
Cummins, J. (1979) Cognitive/academic language proficiency, linguistic
interdependence, the optimum age question and some other matters.
Working Papers on Bilingualism, No. 19, 121-129.
Inbar-Lourie. O. (2008). Constructing a language assessment knowledge
base: A focus on language assessment courses. Language Testing,
25(3), 385-402.
Matsumoto, K. (2011). Studies on the Correlations of Listening Ability
and Productive Abilities. In JACET Testing SIG (eds.) Studies on L2
Listening: Teaching and Assessing. 69-84.
OECD (2009). 21st Century Skills and Competencies for New Millennium
Learners in OECD Countries.
Taylor, L. (2009). Developing assessment literacy. Annual Review of
Applied Linguistics. 29, 21-36.
Warschauer, M. (2000). The changing global economy and the future of
the English teaching, TESOL Quarterly, 32, 511-535. 20
21. Thank you for listening!
For questions and comments:
mkahoko@tsc.u-tokai.ac.jp
21
