The document discusses issues related to testing interlanguage pragmatic ability. It defines pragmatics and places interlanguage pragmatics within a model of communicative language ability. The document outlines considerations for validity in testing pragmatic ability, including construct validity and content validity. It also discusses common issues in interlanguage pragmatics like misunderstandings between speakers from different cultures or language backgrounds. The document concludes by discussing components that could be included in a test of interlanguage pragmatic ability, such as speech acts, implicatures, and routines.

1. Tarbiat Modares University
Faculty of Humanities
Department of English Language Teaching
Investigating Interlanguage
Pragmatic Ability: What Are We Testing?
(SAYOKO YAMASHITA)
Course: Applied Linguistics in Teaching Languages
Instructor: Dr. Tajeddin
Presenter: Adil Irahyyim

2. Abbreviations:
 NS = native speaker
 TL = target language
 TLU = target language use
 DCT = discourse completion test
 NOVA= nonverbal ability
SAYOKO

3. Introduction
‘The primary purpose of a language test is to provide a measure that
we can interpret as an indicator of an individual’s language ability’
(Bachman & Palmer, 1996: 23). This is also true for testing learners’
pragmatic ability or interlanguage pragmatics, which Kasper and
Blum- Kulka (1993: 3) defined as ‘a non-native speaker’s use and
acquisition of linguistic action patterns in a second language’.

4. Theoretical Overview
Pragmatic Competence
 Pragmatics is defined as ‘the study of people’s comprehension and production of
linguistic action in context’ (Kasper & Blum-Kulka, 1993: 3). A more specific
definition can be given as ‘pragmatics is the study of language from the point of
view of users, especially of the choices they make, the constraints they encounter in
using language in social interaction and the effects their use of language has on other
participants in the act of communication’ (Crystal, 1985: 240).
 interlanguage pragmatics is placed in a model of communicative language ability,
which Bachman (1990: 84) described as ‘consisting of both knowledge, or
competence, and the capacity for implementing, or executing that competence in
appropriate, contextualized communicative language use’.

5.  Bachman proposed to include three components in a model of communicative language
ability: 1- language competence 2- strategic competence 3- psychophysiological
mechanisms.
 1- language competence:
language competence
pragmatic
competenceorganizational
competence
Grammatical
competence
textual
competence
illocutionary
competence
sociolinguistic
competence

6.  The components under pragmatic competence guided the
theoretical direction for the measurement of interlanguage
pragmatics. Bachman described illocutionary competence
in reference to the theory of speech acts and language
functions, functions further categorized as ideational,
manipulative or interactional, heuristic and imaginative
functions.
 Sociolinguistic competence. Bachman (1990) listed
three sensitivities and an ability sensitivity to differences in
dialect or variety, sensitivity to differences in register,
sensitivity to naturalness and ability to interpret cultural
references and figures of speech.
pragmatic
competence
illocutionary
competence
sociolinguistic
competence
Pragmatic ability for the language learner then is to be able to
comprehend the pragmalinguistic action as a listener and also be
able to produce it as a speaker in a target language (TL) and
following its cultural norms, using one’s own pragmatic
knowledge of a TL.

7. Testing
 Testing is one type of measure according to Bachman (1990). While
measurement is the process of quantifying the characteristics of
persons according to explicit procedures and rules such as different
types of measures, including rankings, ratings and tests, and its
quantification involves the assigning of numbers, letter grades or
labels, such as excellent or good.
 As one type of measurement, a test is designed to elicit a specific sample of
an individual’s behavior, and ‘the value of tests lies in their capability for
eliciting the specific kinds of behavior that the test user can interpret as
evidence of the attributes or abilities which are of interest’.
 When developing and evaluating a test of interlanguage pragmatics, the
consideration of the validity, or a question of ‘does the test measure what
it is intended to measure?’ is important.

8.  Among several ways of evaluating validity, construct validation is a process of
investigating what a test measures and in construct validation one validates a
test against a theory (Palmer & Groot, 1981). ‘Construct validity is used to
refer to the extent to which we can interpret a given test score as an indicator
of the ability
 Bachman and Palmer further expressed that we need to determine the extent
to which the test task corresponds to tasks in the Target Language Use (TLU)
domain or ‘authenticity’, and to determine the extent to which the test task
engages the test taker’s areas of language ability, or ‘interactiveness’.
 TLU domain involves other areas of language knowledge, as well as
strategies, and may involve topical knowledge and affective responses as well.
 Content validity is another important area to be considered in test
Content validity is the process of investigating whether the selection of tasks
one observes in a test is representative of the larger set of tasks of which the
test is assumed to be a sample (Palmer & Groot, 1981: 2).

9. Interlanguage Pragmatics
Issues
 Misunderstandings between speakers
Misunderstandings are a central issue in cross-
cultural or interlanguage pragmatics, which occur
both between NSs, and a NS and a learner.
Cross-cultural pragmatic misunderstanding
occurs between people from different cultural
backgrounds.
 Misunderstandings between a NS and a learner
can naturally occur very often due to the learner’s
weak understanding of the target
culturepragmatics, or due to differences in their
cultural backgrounds.

10. Interlanguage Pragmatics Issues
 Participating in a conversation as a listener:
Listeners’ responses are called backchannels. Many researchers have
studied the functions of backchannels used in English. While Orestrom (1983)
took it as a supportive function (i.e. the non-primary speaker sends a signal to
the primary speaker to show his or her understanding or agreement), Erickson
(1979) and Schegloff (1982) described them as the interactional functions and
contextual cues that contribute to conversation. The non-primary speaker
serves to pass an opportunity to produce a full turn or regulative function
(Schegloff, 1982).
Understanding metalanguage and metapragmatics is another important
ability of a listener. Metalanguage is language which comments on, describes
or examines what happens at the level of language itself.
The pragmatic ability of language users not only deals with whether or not
they can produce pragmatic expressions appropriately or adequately, but also
concerns how a receiver or listener interprets their utterances. If speakers are
concerned about a recipient’s possible misunderstanding, they may try to

11. Interlanguage Pragmatics
Issues
 Understanding the unsaid and assessing
the unsaid:
Speaker hinting (using a hinting word or
expression or something unsaid) is expecting
that the listener knows what the speaker
wants (Gibbs, 1983). When cultures differ, the
extent to how explicitly people express
themselves differs. Consequently, how well
people understand what is said also differs.
Example:
Hayata (1999) reported a case of an
encounter of a Japanese person and a
French person. She described a
situation when the French person
turned off a light in a university
hallway in Paris where the Japanese
person was sitting on a bench. The
Japanese person was upset, thinking
that the French person did not care
about other people nearby when he
turned off the light. Another Japanese
person living in France for over 30
years laughed when she heard her
complaint and said, French people
will shout in such a case, ‘‘‘Don’t
turn off the light! I am here!’’Why
didn’t you shout him?’

12. Interlanguage Pragmatics Issues
 Avoiding a speech act to accommodate a target culture norm:
Yamashita (1996) reported a number of examples in which participants in
her study purposely avoided producing expected speech acts suggested
in her discourse completion tests (DCTs).
 Nonverbal behaviors:
Whether or not we actually say something, nonverbal actions such as
gestures including hand waves, head nods, facial expressions or eye
movements can mean as much as verbal utterances alone or even more.
Nonverbal behaviors play an important role in daily life and academic
contexts. The Gesture Test (Gestest) for assessing the comprehension of
English gestures and the Nonverbal Ability Scales (NOVA) (see
Jungheim, 1995) for assessing nonverbal behavior in conversations were
developed within the communicative competence framework.

13. Interlanguage Pragmatics
Issues
 Estimating the level of a learner’s language proficiency :
Learners’ levels of language proficiency are usually assessed by
scores on a proficiency test, a placement test or comprehensive
examination administered by an institution. These traditionally
measure a general language ability which includes knowledge of
grammar, morphology, semantics, syntax and phonology, or skill
categories such as listening, speaking, reading and writing.
Pragmalinguistics is not yet regularly included in these tests. This is
partly because the theories of communicative competence and
communicative language teaching have not been fully developed and
rigorous empirical studies need to be carried out.
When we design a proficiency test, we need to ask ourselves, should we
include a pragmalinguistic section, and if the answer is ‘yes’, how?

14. Components of a Test of Interlanguage Pragmatic
Ability
 Roever (2005) chose three components for his web-based language
(pragmatics) test, namely: 1- speech acts 2- implicatures , and 3-
routines.
1. speech acts :
Most studies have focused on a particular speech act to examine how
people realize each speech act and tried to find variations in strategies
used by the participants.
2. implicatures:
Conversational implicature was discussed by Grice (1975) with the
notion of the Cooperative Principle. Participants in a conversation are
expected to make their contributions to the conversation informative
(quantity), truthful (quality), relevant (relation) and clear (manner).

15. Components of a Test of Interlanguage Pragmatic
Ability
3. Pragmatic routines:
Routine ‘is a property of utterances or an expression that is
appropriate to a situation of a certain kind or a strategy which is
appropriate relative to certain communicative ends’ (Coulmas,
1981). Many routines are universal phenomena such as
greetings. However, according to different cultures, there are
variations which are for learners difficult to understand and
sometimes cause misunderstanding (Coulmas, 1981). Being
able to master and use routines of a TL are beneficial for
learners as they provide speech with a natural and proficient
flavor, and often such routine expressions such as ‘What does it
mean?’ or ‘Please speak slowly’ help solve learners’ recurrent
communication problems.

16. MethodsofTestingPragmaticAbility Discourse completion
tests (DCTs)
Multiple-choice tests
Roleplays
Picture prompts
Video prompts
Methods of Testing Pragmatic Ability

17. Methods of Testing Pragmatic
Ability
 Discourse completion tests (DCTs):
The most popular method in cross-cultural and interlanguage
pragmatics is known as DCTs.
 Some negative characteristics of this method:
(1) the waffling effect or verbosity
(2) the differences of the intended speech acts and elicited speech
acts
(3) differences of the length of oral responses and written DCT
responses
(4) DCTs’ misguiding written descriptions
 Despite the above weaknesses, DCTs have been regarded as very effective
tools for crosscultural and interlanguage pragmatics studies due to the fact
that they can be used to gather a large amount of data easily and the data
obtained from this method have been considered as compatible with
naturalspeech occurrences.

18. Methods of Testing Pragmatic Ability
 Multiple-choice tests:
Multiple-choice tests are used quite often in traditional
grammar tests, as their strongest feature is the ability to
test easily in a short period of time and to facilitate
analysis. Tanaka and Kawade (1982) used multiple-
choice questions containing a picture of each situation,
a written description of each situation and six polite
responses or choices for each situation. Their multiple-
choice design worked for their own research purposes,
as it was designed for studying politeness levels of
requests, it does not reflect differences in
pragmaticbased strategies.

19. Methods of Testing Pragmatic
Ability
 Roleplays:
Roleplays are possible to simulate conversational turns and to have
the interlocutor apply conversational pressures that are not present in
a DCT (Cohen & Olshtain, 1994).
 The advantage of using the roleplay method in testing pragmatics is
that the full discourse context and sequential organization in terms
of negotiation of meaning, the strategy choice and politeness
investment can be examined (Kasper & Dahl, 1991).
 The problem of using a roleplay in an actual testing situation is that
it is time consuming and requires interlocutor training.

20. Methods of Testing Pragmatic
Ability
 Picture prompts:
Picture prompts, including
illustrations and cartoons, have
been used to elicit interlanguage
pragmatic features instead of
using written descriptions. Picture
prompt interlanguage pragmatics
tests are particularly useful when
young learners’ second language
acquisition is investigated.

21. Methods of Testing Pragmatic
Ability
 Video prompts:
Video prompts have been used for different purposes, such as (1) to
grasp a learner’s beliefs toward a target culture pragmatic norm in
general; (2) to comprehend a learner’s interpretation of target culture
pragmatics; and (3) to elicit a learner’s pragmatic production. It contains
various aspects of interlanguage interaction, not only verbal including
voice tones and prosodic (e.g. ‘uhmmm’), but also conversational
partners’ facial expressions, gestures and silent pauses, as well as visual
cues and background to the situation.. Naturally there are also
weaknesses in using video prompt.
A specialist will be required to produce such videos, and the budget
will be more than that of paper and pencil tests due to the need for
equipment to show it to test takers. Evaluation standards should also be