Summary on article by Ch. Alderson & J.Banerjee regarding Language Testing & Assessment. Diferent types of language testing, authors, hypothesis, conclusions and expectations. Ethics, politics and standards impact on language testing.

1. My summary on
“State-of-the-Art Review
Language testing & assessment
(Part 1)”
Yill Garcia
Universidad Vizcaya de las Américas
Mc Millan Diploma
April 4rth, 2015.

2. A dim light on language testing:
Washback
Ethics in
language
testing
Politics
Standards in
testing
National
tests
LSP testing
Computer-
based
testing
Self-
assessment
Alternative
assessment
Assessing
young
learners
2

3. Washback
 It refers to the impact that tests have on teaching and learning.
 Alderson and Wall (1993) were among the first to introduce the “test
washback” term in language education and proposed a research
agenda of such impact upon tests.
 Three example hypothesis are:
 A) Tests will have washback on what teachers teach (content).
 B) Tests have also impact on how teachers teach (methodology).
 C) High-stake tests (tests with important consequences) will have
more impact than low-stake tests.
3

4.  Alderson and Hamp-Lyons (1996)
show that teachers may change
the way they teach when
reaching towards a test.
 In this case, TOEFL-Test of English
as a Foreign Language.
 Also, the nature of the change
varies from teacher to teacher
(Watanabe’s findings 1996).
 The complex washback
phenomenon is slowly getting
recognition; influenced by many
factors, even by the nature of
such test.
 No major studies have yet been
carried out into effect of test
preparation on test performance,
given the prevalence, for high-
stakes tests, at least, about test
preparation courses.
4

5.  What may be most important is
not the objective difficulty of the
test, but the student’s perception
of difficulty, concluded
Watanabe.
 Wall (2000) summarizes research
findings which show that test
design is only one of the factors
affecting washback, and lists as
factors influencing the nature of
the test washback.
 These include assessing the
feasibility of examination reform
by a “baseline study” (Weir &
Roberts, 1994, Fekete et al, 1999).
 Policy makers and test designers
should be aware that tests on
their own will not have a positive
impact if the materials and
practices they are based on have
not been effective.
5

6. Ethics in language testing
 Messick (1994) argues that all testing involves making value
judgements, and therefore language testing is open to a critical
discussion of whose values are being represented and served; this in
turn leads to a consideration of ethical conduct.
 A number of case studies have been presented recently illustrating
the use and misuse of language tests.
 One case was Chen and Henning (1985), who compared
international student’s performance on the UCLA (University of
California, Los Angeles) English as a Second Language Placement
Test, and discovered that a number of items were biased in favor of
Spanish speaking students and against Chinese speaking students.
6

7.  The International Language Testing
Association (ILTA) has recently developed
a Code of Ethics, which is a set of
principles which draws upon moral
philosophy and strives to guide good
professional conduct.
 The code is clear: testers must follow
ethical practices and have a moral
responsibility to do so.
McNamara (1998)
concludes that we are
likely to see a broadening
of issues involved in
language testing research,
at least, the following
disciplines and fields:
philosophy, especially
ethics and the
epistemology of social
science; critical theory;
policy analysis; program
evaluation, and
innovation theory.
7

8. Politics
 Politics can be defined as action, or activities, to achieve
power or to use power, and as beliefs about government,
attitudes to power, and to the use of power.
 Brindley (1998,2001) describes the political use of test-based
assessment for reasons of public accountability, often in the
context of national frameworks, standards or benchmarking.
 National educational policy often involves innovations in
testing in order to influence the curriculum, or in order to open
up or restrict access to education and employment, and
even to influence immigration opportunities.
8

9.  “Testing is too important to be left
to testers” -Alderson, 2001a.
 Politics can be seen as
methods, tactics, intrigue,
maneuvering, within institutions
which are themselves not
political, but commercial,
financial and educational.
 Alderson (1999) argues that
politics with a small “p”
includes not only institutional
politics, but also personal
politics; the motivation of the
actors themselves and their
agendas. And personal politics
can influence both test
development and test use.
9

10. Standards in testing
 “Standardized test” appoints to a test whose difficulty level is known,
it has been adequately piloted and analyzed, the results of which
can be compared with those of a norming population.
 Standardized tests are typically norm referenced tests. Within this last
context ‘standards’ is equivalent to ‘norms’.
 The Council of Europe’s Common European Framework is seen as
independent of any possible vested interest and it has a long
pedigree originated over 25 years ago in the development of the
Threshold level and wide guaranteed acceptability across Europe.
10

11.  Today, it is clear that the Common
European Framework has become
more influential due to the
growing need for international
recognition of certificates in
Europe, to guarantee educational
and employment mobility.
 The Framework is a compendium
of what is known about language
learning, language use and
language proficiency.
 It is an essential guide to syllabus
construction, development of test
specifications and rating criteria.
 The Framework is bound to be used
for materials design, textbook
production and teacher education.
 Calibrating the new tests against
the Framework is essential.
11

12. National tests
 The development of national language tests continues to be the
focus of many publications, although many are either simply
descriptions of test development or discussions of controversies,
rather than reports on research done in connection with test
development.
 Jansen & Peer (1999) in the Netherlands, reports a study of the
recently introduced use of dictionaries in Dutch foreign language
examinations and shows that dictionary use does not have any
significant effect on text scores.
 Pupils are very positive about being allowed to use dictionaries,
claiming that it reduces anxiety and enhances their text
comprehension.
12

13.  The value of local descriptive
publications serve many needs,
including necessary publicity for
reform work, helping teachers
to understand developments.
 Publication can serve political
as well as professional and
academic purposes.
 Standard setting data can
reveal what levels are achieved
by the school population.
Examples of comparisons are:
 A) Those who started learning
the language early with late-
starters.
 B) Those studying a first foreign
language with those studying
the same language as their
second or third foreign
language, and so on.
 Studies have been used in in-
service and pre-service teacher
education.
13

14.  They can also show the effect of
innovations, and help language
educators to understand how to
do things more effectively.
 Washback studies are used in
teacher training in order to
influence test preparation
practices, but also to encourage
teachers to reflect on the
reasons for their practices.
14

15. LSP Testing
 The development of Language for Specific Purpose (LSP) testing
can be traced back to the Temporary Registration Assessment
Board (TRAB), introduced by the British General Medical Council in
1976.
 Douglas (1997,2000) identifies two aspects that distinguish LSP
testing from general purpose testing.
 The first is the authenticity of the tasks, i.e., the test tasks share key
features with the tasks that a test taker might encounter in the
target language use situation.
 The second distinguishing feature of LSP testing is the interaction
between language knowledge and specific content language.
15

16.  The real challenge to the field
is in identifying when it is
absolutely necessary to know
how well someone can
communicate in a specific
context or if the information
being sought is equally
obtainable through a general
purpose language test.
 It must be noted, however,
that, because of the need for
in-depth analysis of the target
language use situation, LSP
tests are time-consuming and
expensive to produce.
 Douglas (2000) stands firmly by
claims made earlier in the
decade that in highly field-
specific language contexts, a
field-specific language test is a
better predictor of
performance than a general
purpose test (Douglas &
Selinker, 1992).
16

17. Computer based testing
Alderson (1996) points out that computers have much to offer
language testing:
a) Test construction.
b) Test compilation.
c) Response capture.
d) Test scoring.
e) Result calculation and delivery.
f) Test analysis.
g) Storing tests and details of candidates.
That is, computers can be used at all stages in the test development
and administration process.
17

18.  The commonest use of
computers in language testing
is to deliver tests adaptively
(Young, 1996).
 Some advantages:
 a) First, candidates are
presented with items at their
level of ability and are not
faced with items that are too
easy or too difficult.
 b) Second, computer
adaptive tests (CAT’s) are
typically quicker to deliver,
and security is less a problem
since candidates are
presented with different items.
18

19.  In the Reading section candidates are required to select a
word, phrase, sentence or paragraph in the text itself and
other questions ask candidates to insert a sentence where it
fits best.
 One advantage is that candidates can see the result of their
choice in context, before making a final decision.
19

20.  DIALANG is a suite of computer
based diagnostic tests (funded
by the European Union) which
are available over the internet.
 DIALANG uses self assessment as
an integral part of diagnosis.
 User’s self ratings are combined
with objective test results in
order to identify a suitable
difficult test for the user.
 DIALANG users get immediate
feedback on scores, test results
and their self-assessment.
 Alderson (2000c) argues the
need for a research agenda,
which would address the
challenge of the opportunities
afforded by computer based
testing.
20

21. Self-assessment
In the 1980’s the interest in self-assessment increased.
Introduction of self-assessment was viewed as promising by many,
especially in formative assessment contexts (Oscarson, 1989).
It was considered to encourage increasing learner awareness,
helping learners to:
a) Gain confidence in their own judgement.
b) Acquire a view of evaluation that covers the whole learning
process.
c) See errors as something helpful.
21

22. 22
 Carton (1993) discusses how self-
assessment can become part of the
learning process.
 He describes his use of questionnaires
to encourage learners to reflect on
their learning objectives and preferred
modes of learning.
 Oscarson (1997) sums up progress in
this area by reminding us that
research in self-assessment is still
relatively new.
 Self-assessment must be introduced
slowly and learners need to be
guided and supported.
 In multicultural groups, the cultural
influences must be considered as
well.

23. Alternative assessment
 It usually means assessment procedures which are less formal than
traditional testing.
 They are gathered over a period of time rather than being taken at
one point in time.
 Usually, they are formative rather than summative in function.
 They are often low-stakes in terms of consequences, and are
claimed to have beneficial washback effects.
23

24.  Although portfolio assessment in
other subject areas is not new, in
foreign language education
portfolios have a major
innovation, supposedly
overcoming the drawbacks of
traditional assessment.
 A typical example is Padilla
(1996) who describes the design
and implementation of portfolio
assessment in Japanese,
Chinese, Korean and Russian, to
assess growth in foreign
language proficiency.
 They make a number of
practical recommendations to
assist teachers wishing to use
portfolio in progress
assessment.
24

25.  Many of the accounts of alternative assessment are for classroom
based assessment, often for assessing progress through a program
of instruction.
 Brown and Hudson (1998), instead of the “alternative assessment”,
they propose the term “alternatives in assessment”, pointing out
that there are many different testing methods available for
assessing student learning and achievement.
25

26.  Hamayan (1995), argues that
consistency in the application of
alternative assessment is still a
problem.
 That mechanisms for thorough
self criticism and evaluation of
alternative assessment
procedures are lacking.
 That some degree of
standardization of such
procedures will be needed if
they are to be used for high-
stakes assessment.
 And, that the financial and
logistic viability of such
procedures remains to be
demonstrated.
26

27. Assessing young learners
 The assessment of young learners term dates back to the 1960’s.
 Ages between 5 and 12 are usually in account.
 We consider recent developments in the assessment of young
learners, an area where it is often argued that alternative
assessment procedures are more appropriate than formal testing
procedures.
27

28.  It could be due to three factors:
 1st., second language teaching
(English in particular) to children in
the pre-primary and primary age
groups both within mainstream
education and by commercial
organizations, has gotten dusty.
 2nd., it is recognized that classrooms
have become multicultural and,
mainly in the context of Australia,
Canada, USA and the UK, many
learners are speakers of English as an
additional/second language (rather
than heritage English speakers).
 3rd., the decade has seen an
increased proliferation, within
mainstream education, of teaching
and learning standards.
28

29.  On the other hand, studies show
a desire to understand how
teachers implement assessment
(Gatullo, 2000) as well as a need
for inducting teachers into
assessment practices in contexts
where is no tradition of
assessment (Hasselgren, 1998).
 It seems that the field has moved
forward in its understanding of the
assessment needs of young
learners, yet it has been pressed
back by economic
considerations.
29

30. 30
Bibliography
Alderson, C., & Banerjee, J. (2001). Language testing and assessment
(Part I). State-of-the-Art, 213-235.