This document outlines various topics related to language testing, including types of tests, approaches to testing, validity and reliability, and achieving beneficial backwash effects. It discusses proficiency tests, achievement tests, and diagnostic tests. It also covers direct and indirect testing, norm-referenced and criterion-referenced testing, and objective and subjective testing. Validity is defined as accurately measuring the intended abilities, while reliability is consistency of results. Achieving beneficial backwash means testing abilities you want to foster and ensuring students and teachers understand the test.
Types of tests: proficiency, achievement, diagnostic, placement
Types of testing: direct vs indirect tests, discrete point vs integrative tests, criterion-referenced vs norm-referenced tests, objective vs subjective tests
kinds of tests and testing
proficiency tests- achievement tests, diagnostics test, placement tests, direct and indirect test, discrete point and intergrative testing, norm-referenced and criterion testing, objective testing and subjective testing, computer adapting testing
Types of tests: proficiency, achievement, diagnostic, placement
Types of testing: direct vs indirect tests, discrete point vs integrative tests, criterion-referenced vs norm-referenced tests, objective vs subjective tests
kinds of tests and testing
proficiency tests- achievement tests, diagnostics test, placement tests, direct and indirect test, discrete point and intergrative testing, norm-referenced and criterion testing, objective testing and subjective testing, computer adapting testing
Topic: Qualities of a Good Test
Student Name: Amna Mishal
Class: B.Ed. (Hons) Elementary
Project Name: “Young Teachers' Professional Development (TPD)"
"Project Founder: Prof. Dr. Amjad Ali Arain
Faculty of Education, University of Sindh, Pakistan
A brief summary of the Test Methods and Test Facets affecting testing performance (Source: Fundamental Considerations in Language Testing - Lyle F. Bachman)
Product Syllabus : product syllabuses are those in which the focus is on the knowledge and skills which learners should gain as a result of instruction.
4.2. process syllabuses are those which focus on the learning experiences themselves.
. Synthetic syllabus: segment the target language into discrete linguistic items.
Different parts of language are taught separately.
4.4 . Analytic Syllabi: focus on the learner and his needs and on the kinds of linguistic
performance necessary to achieve those goals .
4.5. Type A: This type deals with what should be learned in a second language classroom.
4.6. Type B : Consider the question of how a second language should be learned.
Topic: Qualities of a Good Test
Student Name: Amna Mishal
Class: B.Ed. (Hons) Elementary
Project Name: “Young Teachers' Professional Development (TPD)"
"Project Founder: Prof. Dr. Amjad Ali Arain
Faculty of Education, University of Sindh, Pakistan
A brief summary of the Test Methods and Test Facets affecting testing performance (Source: Fundamental Considerations in Language Testing - Lyle F. Bachman)
Product Syllabus : product syllabuses are those in which the focus is on the knowledge and skills which learners should gain as a result of instruction.
4.2. process syllabuses are those which focus on the learning experiences themselves.
. Synthetic syllabus: segment the target language into discrete linguistic items.
Different parts of language are taught separately.
4.4 . Analytic Syllabi: focus on the learner and his needs and on the kinds of linguistic
performance necessary to achieve those goals .
4.5. Type A: This type deals with what should be learned in a second language classroom.
4.6. Type B : Consider the question of how a second language should be learned.
This is a copy of Prof Ed 3: Assessment in learning 1. Created by Dr. Ariel Mabansag. This will introduce you on how to utilize various assessment tools necessary in teaching.
Overview of Assessment
It is an integral part of instruction, as it determines whether or not the goals of education are being met.
3 criteria of assessment
Validity
Reliability
Practicality
(Farhady,2012)
Assessment
Assessment information is needed by administrators, teachers, staff developers, students, and parents to assist in determining appropriate program placements and instructional activities as well as in monitoring student progress. (O’Malley,1994)
Assessment Purposes of ELL Students
Screening and identification
Placement
Reclassification or exit
Monitoring Student Progress
Program Evaluation
Accountability
(O’Malley,1994)
Normal Labour/ Stages of Labour/ Mechanism of LabourWasim Ak
Normal labor is also termed spontaneous labor, defined as the natural physiological process through which the fetus, placenta, and membranes are expelled from the uterus through the birth canal at term (37 to 42 weeks
Unit 8 - Information and Communication Technology (Paper I).pdfThiyagu K
This slides describes the basic concepts of ICT, basics of Email, Emerging Technology and Digital Initiatives in Education. This presentations aligns with the UGC Paper I syllabus.
Operation “Blue Star” is the only event in the history of Independent India where the state went into war with its own people. Even after about 40 years it is not clear if it was culmination of states anger over people of the region, a political game of power or start of dictatorial chapter in the democratic setup.
The people of Punjab felt alienated from main stream due to denial of their just demands during a long democratic struggle since independence. As it happen all over the word, it led to militant struggle with great loss of lives of military, police and civilian personnel. Killing of Indira Gandhi and massacre of innocent Sikhs in Delhi and other India cities was also associated with this movement.
Read| The latest issue of The Challenger is here! We are thrilled to announce that our school paper has qualified for the NATIONAL SCHOOLS PRESS CONFERENCE (NSPC) 2024. Thank you for your unwavering support and trust. Dive into the stories that made us stand out!
Francesca Gottschalk - How can education support child empowerment.pptxEduSkills OECD
Francesca Gottschalk from the OECD’s Centre for Educational Research and Innovation presents at the Ask an Expert Webinar: How can education support child empowerment?
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
2024.06.01 Introducing a competency framework for languag learning materials ...Sandy Millin
http://sandymillin.wordpress.com/iateflwebinar2024
Published classroom materials form the basis of syllabuses, drive teacher professional development, and have a potentially huge influence on learners, teachers and education systems. All teachers also create their own materials, whether a few sentences on a blackboard, a highly-structured fully-realised online course, or anything in between. Despite this, the knowledge and skills needed to create effective language learning materials are rarely part of teacher training, and are mostly learnt by trial and error.
Knowledge and skills frameworks, generally called competency frameworks, for ELT teachers, trainers and managers have existed for a few years now. However, until I created one for my MA dissertation, there wasn’t one drawing together what we need to know and do to be able to effectively produce language learning materials.
This webinar will introduce you to my framework, highlighting the key competencies I identified from my research. It will also show how anybody involved in language teaching (any language, not just English!), teacher training, managing schools or developing language learning materials can benefit from using the framework.
Acetabularia Information For Class 9 .docxvaibhavrinwa19
Acetabularia acetabulum is a single-celled green alga that in its vegetative state is morphologically differentiated into a basal rhizoid and an axially elongated stalk, which bears whorls of branching hairs. The single diploid nucleus resides in the rhizoid.
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
Synthetic Fiber Construction in lab .pptxPavel ( NSTU)
Synthetic fiber production is a fascinating and complex field that blends chemistry, engineering, and environmental science. By understanding these aspects, students can gain a comprehensive view of synthetic fiber production, its impact on society and the environment, and the potential for future innovations. Synthetic fibers play a crucial role in modern society, impacting various aspects of daily life, industry, and the environment. ynthetic fibers are integral to modern life, offering a range of benefits from cost-effectiveness and versatility to innovative applications and performance characteristics. While they pose environmental challenges, ongoing research and development aim to create more sustainable and eco-friendly alternatives. Understanding the importance of synthetic fibers helps in appreciating their role in the economy, industry, and daily life, while also emphasizing the need for sustainable practices and innovation.
2. Outline
• Testing as Problem Solving
• Kinds of Testing
• Approaches to Testing
• Validity and Reliability
• Achieving Beneficial Backwash
• Stages of Test Construction
• Test Techniques for Testing Overall Ability
• Testing Writing
• Testing Reading
• Testing Listening
• Testing Grammar and Vocabulary
• Test Administration
3. Testing As Problem Solving!
• No Best Test or Technique
– A test which proves ideal for one purpose may be useless for
another; a technique which works well in one situation can be
entire inappropriate in another
• We Want Tests that…
1. Consistently and accurately measure the abilities we want to
measure
2. Have a beneficial effect on teaching
3. Are practical – economical in terms of time and money
4. Practicality
• Practicality
– All tests cost time and money – to prepare,
administer, score, and interpret. Time and money
are in limited supply!
• Our basic challenge is to
– Develop tests which
• (1) are Valid and Reliable,
• (2) have a Beneficial Backwash Effect on teaching, and
• (3) are Practical!
5. Kinds of Testing
• Proficiency Test
• Achievement Test
• Diagnostic Test
• Placement Test
• Direct Test
• Indirect Test
• Norm Referenced Test
• Criterion Referenced Test
• Discrete Point
• Integrative Test
• Objective Test
• Subjective Test
• Communicative Language Test
6. Kinds of Testing
• Proficiency Tests: Used to test a student’s general ability with the
language
• Achievement Tests: Used to test how well the students are at
achieving the objectives of the course. Most teachers are involved in
the preparation and use of these.
• Diagnostic Tests: Used to identify students’ strengths and
weaknesses. Intended to ascertain what further teaching is
necessary.
• Placement Tests: Used to place students at the stage of the teaching
program most appropriate to their abilities. Typically, they assign
students to classes at different levels.
7. Achievement Tests:
Progress and Final
• Final Achievement Test
– Administered at the end of a course of study.
– Intended to measure course contents and/or
objectives.
• Progress Achievement Test
– Administered during a course of study.
– Measures the progress the students are making
towards course objectives.
8. Final Achievement Tests
• Syllabus-Content Approach
– Based directly on a detailed course syllabus or on
books or other material used.
– Obvious Appeal: test contains only what it is
thought that the students have encountered –
and thus can be considered, at least, a fair test.
– Disadvantage: if the syllabus is badly designed, or
books and other material are badly chosen, then
the results of the test can be very misleading.
9. Final Achievement Tests
• Course-Objective Approach
– Based directly on the objectives of the course.
– Obvious Appeal:
• Compels course designers to be explicit about
objectives.
• Makes it possible for performance on the test to show
just how far students have achieved those objectives.
• Puts pressure on those responsible for the syllabus and
for the selection of books and materials to ensure that
these are consistent with the course objectives.
10. Final Achievement Tests
• Ideally Speaking
– Course content will meet the objectives and a
test would be hence based on both the content
and the objectives!
– “If a test is based on the content of a poor or
inappropriate course, the students taking it will
be misled as to the extent of their achievement
and the quality of the course.”
Arthur Hughes, Testing for Language Teachers,
1989.
11. Progress Achievement Tests
• Progress Achievement Tests are intended to
measure the progress students are making.
– Repeatedly administer final achievement tests
and the – hopefully – increasing scores will
indicate the progress being made.
– Establish a series of well-defined short-term
objectives on which to test or quiz the students.
12. Approaches to Testing
• Direct vs. Indirect Testing
• Discrete Point vs. Integrative
• Norm-referenced vs. Criterion-referenced
• Objective vs. Subjective Testing
13. Approaches to Testing
Direct vs. Indirect Testing
• Direct Testing requires the test taker to
perform precisely the skill we wish to
measure. For example, if we want to know
how well a student writes essays, then we
get them to write an essay.
• Indirect Testing makes an attempt to
measure the sub-skills which underlie the
skills in which we are interested.
14. Approaches to Testing Benefits
of Direct Testing
• Direct Testing
– Is easier to carry out with productive skills of
reading and writing
– Relatively straightforward to create the
conditions we want to test
– Assessment and Interpretation of students’
performance is also straightforward
– Practice for the test involves practice of the skills
we wish to foster – helpful backwash!
15. Approaches to Testing
Benefits and Pitfalls of Indirect Testing
• Indirect Testing
– Offers possibility of testing a representative
sample of a finite number of abilities (e.g.
vocabulary, grammatical structures) which
underlie a potentially indefinitely large number of
manifestations of them.
– Danger is in that the mastery of the underlying
micro-skills does not always lead to mastery of
larger skills from which these emanate.
16. Approaches to Testing
Direct vs. Indirect Testing
• Ideally speaking
– we should have a combination of both!
– which should lead to beneficial backwash in that
the teaching would hence focus on both the
greater skills and the micro-skills that underlie
them.
17. Approaches to Testing
Discrete Point vs. Integrative Testing
• Discrete Point Testing
– entails testing one element at a time, element by
element.
– Could be vocabulary or grammatical structures.
• Integrative Testing
– Entails having the test taker combine many
language elements in the completion of some
task.
– Could be writing a composition, taking lecture
notes, giving directions, etc.
18. Approaches to Testing
Norm-referenced vs.
Criterion-referenced Testing
• Norm-referenced Testing
– Places a student in a percentage category.
– Relates one candidate’s performance to that of other
candidates.
– Seeks a bell-shaped curve in student assessment.
• Criterion-referenced Testing
– Test what students can actually do with the language.
– Hence, it is possible for all students to get As if they are all able
to meet the criteria.
– Motivates Students to perform “up-to-standard” rather than
trying to be “better” than other students.
19. Approaches to Testing
Subjective Testing vs. Objective Testing
• Subjective Testing
– Judgement is required on the part of the scorer.
– Different degrees of Subjectivity in Scoring.
– Complexity increases subjectivity – the scoring of
a composition being more subjective compared
to short-answer responses.
• Objective Testing
– No Judgement is required on the part of the
scorer.
– Multiple Choice, Fill-in-the-blank
20. Validity and Reliability
• Validity: a test is said to be valid if it
measures accurately the abilities it is
intended to measure
• Reliability: a test is said to be reliable if it
provides consistent results no matter how
many times the students take it
21. Validity and Reliability Validity:
Four Factors
• Content Validity:
– Content is representative of all the language skills, structures,
vocabulary, etc. with which it is intended to test.
• Criterion-related Validity:
– Where the results of a shorter test – given for practical reasons –
corresponds to the results obtained from a longer more complete
test.
• Construct Validity:
– The test measures exactly the ability it is intended to measure.
Construct refers to an underlying trait or ability hypothesized in
language learning theory. Becomes an important consideration in
indirect testing of abilities or the testing of sub-abilities like guessing
the meaning of unknown words.
• Face Validity:
– An examination has face validity if it seems as if it is measuring what
it is supposed to be measuring.
22. Validity and Reliability Reliability:
Two Components
• Test Reliability:
– That a score on a test will be approximately the
same no matter how many times a student takes
it.
• Scorer Reliability:
– When the test is objective, the scoring requires
no judgment, and the scores should always be
the same.
– When the test is subjective, the scoring requires
judgment, and the scores will not be the same.
23. How to Make Tests More Reliable!
• Test for enough independent samples of behavior and allow for as many fresh
starts as possible
• Do not allow test takers too much freedom. Restrict and specify their range of
possible answers.
• Write unambiguous items
• Provide clear and explicit instructions
• Ensure that tests are well laid out and perfectly legible
• Make sure candidates are familiar with format and test-taking procedures
• Provide uniform and non-distracting conditions of administration
• Use items that permit scoring which is objective as possible
• Make comparisons between candidates as direct as possible
• Provide a detailed scoring key
• Train scorers
• Agree on acceptable responses and appropriate scores at the outset of scoring
• Identify test takers by number, not name
• Employ multiple, independent scoring
24. Achieving Beneficial Backwash /
Washback
• Test abilities whose development we want
fostered
• Sample widely and unpredictably
• Use both direct and indirect testing
• Make testing criterion-referenced
• Base achievement tests on objectives
• Ensure test is known and understood by both
teachers and students
• Provide assistance to teachers
25. Achieving Beneficial Backwash /
Washback
• Test abilities whose development we want
fostered
– For example, if we want to develop
“Communicative Competence” than we need to
test aspects of Communicative Competence.
– Don’t just test what is easiest to test.
– Certain abilities should be given sufficient
“weight” in relation to other abilities.
26. Achieving Beneficial Backwash /
Washback
• Sample widely and unpredictably
– Tests can normally only measure a sample of the
language. Therefore the sample taken should
represent as much as possible the full scope of
what is specified.
– For example, if the TOEFL writing test were to
only test (1) compare and contrast, and (2)
problem and solution, then much preparation
would be limited to only these two types of tasks
while others would be ignored.
27. Achieving Beneficial Backwash /
Washback
• Use both direct and indirect testing
– Test the larger skills directly
– Test the micro-skills (making up those larger
skills) indirectly
28. Achieving Beneficial Backwash /
Washback
• Make testing criterion-referenced
– If students know what they have to do and to what degree to
succeed, they will have a clear picture of what they need to do
in order to achieve.
– They will know that if they perform the tasks at the criterion
level, then they will be successful on the test, regardless of how
the other students perform.
– Both of the above are motivating for the students.
– Also possible to have a series of Criterion-referenced tests,
each representing a different level of proficiency. Students
must complete the majority of tasks successfully in order to
“pass” the test and move onto the next level of proficiency.
29. Achieving Beneficial Backwash /
Washback
• Base achievement tests on objectives
– Will provide truer picture of what has actually
been achieved
30. Achieving Beneficial Backwash /
Washback
• Ensure test is known and understood by
students and teachers
– Teachers and students should understand what
the test demands.
– The test’s rationale, its specifications, and sample
items should be made available to everyone
concerned with the preparation for the test.
– Increases test reliability.
31. Achieving Beneficial Backwash /
Washback
• Provide assistance to teachers
– The introduction of a new test can make new demands on
teachers
• If a long-standing test on grammatical structure and vocabulary is
to be replaced with a test of a much more communicative nature,
it is possible that many teachers may feel that they do not know
how to teach communicative skills. Of course, the reason the
communicative test may have been introduced in the first place
was to encourage communicative language teaching. Hence, the
teachers will also need guidance and training in how (and why) to
do this. If these are not given, the test will not achieve its desired
effect and will more likely result in chaos and disaffection.
32. Stages of Test Construction
• Statement of the Problem
• Providing a Solution to the Problem
– Writing Specifications for the Test
– Writing the Test
– Pretesting
33. Stages of Test Construction
Statement of the Problem
• Statement of the Problem
– Be clear about what one wants to know and why!
• What kind of test is most appropriate?
• What is the precise purpose?
• What abilities are to be tested?
• How detailed must the results be?
• How accurate must the results be?
• How important is backwash?
• What are the constraints (unavailability of expertise,
facilities, time [for construction, administration, and
scoring])?
34. Stages of Test Construction
Providing a Solution to the Problem
• Providing a Solution to the Problem
– Once the problem is clear, then steps can be
taken to solve it.
– Efforts should be made to gather information on
similar tests designed for similar situations. If
possible, samples should be obtained. Should not
be copied, but rather used to suggest possibilities,
since there is no need to “reinvent the wheel.”
35. Stages of Test Construction
Writing Specifications for the Test
• Writing Specifications for the Test
– Content
• Operations
• Types of Text
• Addressees
• Topics
– Format and Timing
– Criterial Levels of Performance
– Scoring Procedures
36. Stages of Test Construction
Writing Specifications for the Test
• Content
– Refers not to the content of a single, particular
version of the test, but to the entire potential
content of any number of versions.
– Samples of this content should appear in
individual versions of the test.
– The fuller the information on content available,
the less arbitrary should the decisions be as to
what should appear on any version of the test.
37. Stages of Test Construction
Writing Specifications for the Test
• Content
– The content will vary depending on the type of test. A grammar test
(e.g. structures) will be different than one that tests communicative
functions (e.g. ordering in a restaurant or asking for directions).
– Some things to consider:
• Operations: tasks students will have to be able to carry out (e.g. in reading,
skimming and scanning, etc.).
• Types of Text: (e.g. in writing, letters, forms, academic essays, etc.).
• Addressees: the people the test-taker is expected to be able to speak or
write to; or the people for whom reading and listening are primarily
intended (for example, native-speaker university students).
• Topics: topics should be selected according to their suitability for the test
takers and the type of test.
38. Stages of Test Construction
Writing Specifications for the Test
• Format and Timing
– Should specify test structure and item
types/elicitation procedures, with examples.
– Should state how much weight in scoring will be
allocated to each component.
39. Stages of Test Construction
Writing Specifications for the Test
• Criterial Levels of Performance
– The required levels of performance for different
levels of success should be specified. For example,
to demonstrate mastery, 80 % of the items must
be responded to correctly.
– It may entail a complex rubric including the
following: accuracy, appropriacy, range of
expression, flexibility, size of utterances.
40. Stages of Test Construction
Writing Specifications for the Test
• Scoring Procedures
– Most relevant when scoring is subjective.
– Test constructors should be clear as to how they
will achieve high scorer reliability.
41. Stages of Test Construction
Writing the Test
• Sampling
– Choose widely from whole area of content.
– Succeeding versions of test should sample widely
and unpredictably.
42. Stages of Test Construction
Writing the Test
• Item Writing and Moderation
– Writing of successful items is difficult.
– Some items will have to be rejected – others
reworked.
• Best way is through teamwork!
• Item writers must be open to, and ready to accept
criticism.
– Critical questions:
• Is the task perfectly clear?
• Is there more than one possible correct answer?
• Do test takers have enough time to perform the tasks?
43. Stages of Test Construction
Writing the Test
• Writing and Moderation of Scoring Key
– When there is only one correct response, this is
quite straightforward.
– When there are alternative acceptable responses,
which may be awarded different scores, or where
partial credit may be given for incomplete
responses, greater care should be given.
44. Stages of Test Construction
Pretesting
• Pretesting
– Even after careful moderation, there may be some problems
with the test.
– Obviously better if these problems can be identified before the
test is administered to the group for which it is intended.
– Pretesting is often not feasible. Group may not be available or
may put security of test at risk.
– Problems that become apparent during administration and
scoring should be noted and corrections made for the next
time the test is given.
45. Test Techniques for
Testing Overall Ability
• Definition: Test Techniques
– Means of eliciting behavior from test takers which
inform us about their language abilities.
• We need test techniques which
– elicit valid and reliable behavior regarding ability
in which we are interested;
– will elicit behavior which will be reliably scored;
– are economical; and
– have a positive backwash effect.
46. Test Techniques for Testing Overall Ability
Multiple Choice
• Multiple Choice
– Advantages
• Scoring is reliable and can be done rapidly and economically,
• Possible to include many more items than would otherwise be
possible in a given period of time – making the test more reliable.
– Disadvantages
• Tests only recognition knowledge
• Guessing may have a considerable but unknowable effect on test
scores
• Technique severely restricts what can be tested
• It is very difficult to write successful items
• Backwash may be harmful
• Cheating may be facilititated.
47. Test Techniques for Testing Overall Ability
Multiple Choice
• Multiple Choice
– Hence, it is
• Best suited for relatively infrequent testing of large
numbers of individuals,
• Should be limited in institutional testing to particular
tasks which lend themselves very well to the multiple
choice format (e.g. reading or listening comprehension).
• Institutions should avoid excessive, indiscriminate, and
potentially harmful use of the technique.
48. Test Techniques for Testing Overall Ability
Cloze (Fill in the Blanks)
• Cloze
– A cloze test is essentially a fill-in-the-blank test.
However, initially, after a lead-in every seventh
word or so was deleted and the test taker was
asked to attempt to replace the original words.
– A better and more reliable method is to carefully
choose which words to delete from a passage.
– Can be used with a tape-recorded oral passage to
indirectly test oral ability.
49. Test Techniques for Testing Overall Ability
Cloze (Fill in the Blanks)
• Advice for Cloze Tests
– Passages should be at the appropriate level.
– Should be of the appropriate style of text.
– Deletions should be made every 8th to 10th word after a few
sentences of uninterrupted text.
– Passage should be tried out on native speakers and range of
acceptable answers determined.
– Clear instructions should be provided and students should
initially be encouraged to read through the passage first.
– The layout should facilitate scoring.
– Test takers should have had an opportunity to become familiar
with this technique beforehand.
50. Test Techniques for Testing Overall Ability
The C-Test
– A variety of the C-Test
– Instead of whole words it is the second half of
every word that is deleted.
– Advantages over the cloze test are
• Only exact scoring is necessary
• Shorter (and so more) passages are possible
– A wider range of topics, styles, and levels of ability is possible.
• In comparison to a Cloze, a C-Test of 100 items takes
little space and not nearly so much time to complete
(since candidates do not have to read so much text).
51. Test Techniques for Testing Overall Ability
The C-Test
• Disadvantage
– Puzzle-like nature
– May end up rather testing one’s ability to figure
out puzzles than in testing language ability.
• However,
– Research seems to indicate that it gives a rough
estimate of overall language ability.
52. Test Techniques for Testing Overall Ability
Dictation
• Initially dismissed as…
– …hopelessly misguided.
• However, orthodoxy was challenged.
– Research showed high correlations between scores on dictation
tests and scores on longer more complex tests.
– Candidates hear a stream of sound which had to be decoded
into a succession of words, stored, and recreated on paper.
– Ability to identify words from context was now seen as a very
desirable quality, one which distinguished between learners at
different levels.
53. Test Techniques for Testing Overall Ability
Dictation
• Dictation tests are…
– …in prediction of overall ability have the advantage of involving
listening ability.
– …easy to create and administer.
• However, they are …
– … not easy to score and…
– …they are time-consuming.
– With poorer students scoring becomes tedious.
– Partial-dictation may be considered as a better alternative since
it is easier for both the test taker and the scorer.
54. Testing Writing
• Best way to test writing ability is to get test
takers to write.
1. Set writing tasks that are representative of the
population of tasks that we should expect the
students to be able to perform.
2. Tasks should elicit samples of writing which are
truly representative of the student’s ability to
write.
3. Samples of writing must be scored reliably.
55. Testing Writing
Setting the Tasks
• Specify Appropriate Tasks and Select a Sample
– Need to be clear at the outset the tasks students should be
able to perform.
– Should be identified in test specifications.
• Example: (Basic Level) Operations, types of text, addressees,
topics.
– Operations: Expressions of thanks, opinions, apology, etc.
– Text Types: Form: Type -- Letter: Announcement; Postcard:
Description; Note: Narration; Form: Comment
– Addressees: Acquaintances/Colleagues/Sales Clerks, etc.
– Topics: Social Interaction with Native and Non-native Speakers of
English; Dealing with Official and Semi-official bodies; Shopping and
Using Services; Visiting Places of Interest, etc.
56. Testing Writing
Setting the Tasks
• Obtain Samples that Properly Represent
Each Candidate’s Ability
– Set as many tasks as are feasible.
• Offer test takers as many fresh starts as possible –
each task can represent a fresh start.
• Reason for including as many different tasks as is
possible.
• Must be balanced with Practicality.
• Depends on the purpose of the test.
57. Testing Writing
Setting the Tasks
• Obtain Samples that Properly Represent
Each Candidate’s Ability
– Test Only Writing Ability, and Nothing Else
• Is not an Intelligence or a Knowledge Test.
• Make the Instructions Short and Simple. Reading
Ability can hence interfere with measuring Writing
Ability.
– Make Use of Illustrations.
– Restrict What Candidates are Allowed to do
• Writing tasks should be well-defined: test takers
should know exactly what it is they are required to do.
58. Testing Writing
Setting the Tasks
• Set Tasks Which Can Be Reliably Scored
– Set as many tasks as possible
– Restrict what test takers can do
– Give no choice of tasks: makes comparisons
between test takers easier
– Ensure long enough samples for reliable
judgments.
59. Testing Writing
Setting the Tasks
• Obtain Reliable Scoring of Writing
– Holistic Scoring:
• Also known as “Impressionistic Scoring.”
• Involves the assignment of a single score to a sample
of writing on the basis of an overall impression.
• Very Rapid.
– Analytic Scoring:
• Methods of scoring which require a separate score for
each of a number of aspects.
60. Testing Oral Ability
• We want to set tasks which are
representative of the population of oral
tasks that we expect test takers to be able
to perform.
• Hence, the tasks should elicit behavior
which is truly representative of the test
taker’s ability and which can be scored
validly and reliably.
61. Testing Oral Ability
Setting the Tasks
• Specify Appropriate Tasks
– Content
• Operations (Expressing, Narrating, Eliciting, etc.).
• Types of Text (Dialogue, Multi-participant Interactions
[face-to-face and also telephone])
• Addressees
• Topics
– Format
• Interview
• Interaction with Peers
• Response to tape-recordings
62. Testing Oral Ability
Setting the Tasks
• Obtaining Appropriate Samples and the Reliable
Judging of Tests:
– Advice for Oral Tests
• Make tests as long as possible
• Include a wide sample of specified content
• Plan test carefully
• Give test taker as many “fresh starts’ as possible
• Set only tasks and topics that would not cause the test taker
difficulty in their own language
• Choose a quiet room with good acoustics
• Put test takers at ease
• Interviewer should not talk too much. Let the test taker do the
talking
63. Testing Oral Ability
Setting the Tasks
• Elicitation Techniques
– Questions and Requests for Information
– Pictures (for eliciting descriptions)
– Role Play
– Interpreting
– Discussion
– Tape-recorded Stimuli (e.g. language lab)
– Imitation (i.e. repetition)
64. Testing Oral Ability
Setting the Tasks
• Elicitation Techniques:
• NOT RECOMMENDED
– Prepared Monologue
– Reading Aloud
65. Testing Oral Ability
Setting the Tasks
• Obtaining Valid and Reliable Scoring
– Scoring will be valid and reliable only if
• Appropriate descriptions of criteria levels are written
out and scorers are trained to use them.
• Irrelevant features of performance are ignored.
• There is more than one scorer for each performance.
66. Testing Reading
Specifying What Test Takers Should Be Able to Do
• Content
– Operations: Macro
» Scanning text to locate specific information
» Skimming text to obtain the gist
» Identifying stages of an argument
» Identifying examples in support of an argument
– Operations: Micro
» Identifying referents of pronouns
» Using context to guess meaning of unfamiliar words
» Understanding relations between parts of text
» Understanding grammatical structures and meanings of
words
67. Testing Reading
Specifying What Test Takers Should Be Able to Do
• Content
– Types of Text
» Textbook, Novel, Magazine,
Newspaper, Letter, Poem, etc.
– Addressees: Implied
– Topics: General
68. Testing Reading
Setting the Tasks
• Selecting Texts
– Try to select a representative sample as possible
– Choose texts of appropriate length for the required task
– Include as many passages as possible giving test takers as
many “fresh starts” as possible
– For testing scanning, use pieces with lots of discrete pieces of
information
– Choose interesting pieces but not ones which will overly excite
or disturb
– Avoid texts which are made up of test takers’ general
knowledge
– Do not use texts which students have already read
69. Testing Reading
Setting the Tasks
• Writing Items: Possible Techniques
– Multiple Choice (with or without pictures)
– Unique Answer (only one possible answer: e.g. answer to a
question or fill in the blank)
– Short Answer
– Guided Short Answers (students fill in the blanks).
– Summary Cloze: the reading passage is summarized by the
tester with gaps left in the summary for completion by the test
taker.
– Information Transfer: test taker shows completion of reading
task by (1) supplying simple information in a table, (2)
following a route on a map, (3) labeling a picture, etc.
70. Testing Reading
Setting the Tasks
• Writing Items: Possible Techniques
– Identifying Order of Events, Topics, or Arguments
– Identifying Referents: (e.g. “What does the word ‘it’ [line 25]
refer to?” _____________
– Guessing the meaning of unfamiliar words from context
71. Testing Reading
Setting the Tasks
• Procedures for Writing Items
– Careful reading of text with specified operations in mind.
– Determining what tasks are appropriate
– Writing Draft Items
– Paragraph numbers and line numbers added if necessary
– Should be checked by colleagues
72. Testing Listening
• There are times when no speaker is called for such as
when listening to the radio, listening to lectures, or
listening to announcements. Therefore, listening can
be separated from speaking. There are other times
when it is inconvenient to test speaking and testing
listening can have a backwash effect on oral skills.
73. Testing Listening
Specifying What Test Takers Should Be Able to Do
• Content
– Operations: Macro
» Listening for specific information
» Obtaining the gist of what is being said
» Following directions
» Following instructions
– Operations: Micro
» Interpretation of intonation patterns (recognition of
sarcasm, etc.)
» Recognition of function of structures (such as interrogative
as request).
74. Testing Listening
Specifying What Test Takers Should Be Able to Do
• Content
– Types of Texts
» Monologue
» Dialogue
» Multi-participant
» Announcement, Lecture, Instructions, Directions
– Addressees
» General public, Students, Young Children, etc.
– Topics
» General Terms
75. Testing Listening
Setting the Tasks
• Selecting Samples of Speech
– Native of Non-native Speech
• Writing Items: Possible Techniques
– Multiple Choice: choices need to be kept short and simple
– Short Answer
– Information Transfer
– Note Taking: Students respond to questions after talk
– Partial Dictation: when no other listening test practical
– Recordings or Live Presentations?
• Scoring the Listening Test
– Receptive Skill: No need to deduct points for errors in
grammar or spelling
76. Testing Grammar and Vocabulary
Testing Grammar
• Why test grammar?
– Recently, there has been the argument that it is language skills
that need to be tested and not the structures that underlie
these.
– Hence, there is more to any skill than the sum of its parts.
– The backwash effect of testing skills directly are preferable to
tests which encourage the learning of grammatical structures
in isolation – with no apparent need to use them.
– However, most large-scale proficiency tests DO retain a
grammar section and there is good cause to include grammar
sections in institutional achievement, diagnostic, and
placement tests – since most teach “grammar” in one guise or
another.
77. Testing Grammar and Vocabulary
Testing Grammar
• Why test grammar?
– Grammatical ability – or rather lack of it – does set limits to
what can be achieved in the way of skills performance.
– In order to place students in the most appropriate class for
their level having some inkling of their ability to use and
understand grammatical abilities should be very useful.
– Diagnostically, knowing a students strengths and weaknesses
with regards to grammar, should also help a teacher design
more effective lessons in the classroom.
78. Testing Grammar and Vocabulary
Testing Grammar
• Writing Specifications
– For achievement tests, where the grammatical structures to be
covered are listed, specification of content should be quite
straightforward.
– When there is no such listing, then it must be inferred from
the textbook/materials being used in the course.
• Sampling
– Selecting widely from the structures specified should give the
test content validity.
– Should also take into account what are regarded as the more
important structures.
– Should NOT focus on structures which are easiest to test.
79. Testing Grammar and Vocabulary
Testing Grammar
• Writing Items
– Multiple choice is not a good choice for testing grammar.
– Paraphrase, Completion, and Modified Cloze are more
appropriate techniques for testing grammar.
» They share the quality of requiring students to supply
grammatical structures appropriately rather than just
recognizing their correct use.
80. Testing Grammar and Vocabulary
Testing Grammar
• Scoring Production Grammar Tests
– Points should only be awarded for what each item is testing.
– Nothing should be deducted for non-grammatical errors or
errors in grammar not being tested.
» For example, a test taker should not be penalized for
missing the -s on the third-person singular when the item
being tested is relative pronouns.
– If two elements are being tested at the same time, then points
can be assigned to each item. Alternatively, it can be stipulated
that both elements have to be correct for any points to be
awarded.
81. Testing Grammar and Vocabulary
Testing Vocabulary
• Why test vocabulary?
– Clear knowledge of vocabulary is essential to the development
and demonstration of linguistic skill.
82. Testing Grammar and Vocabulary
Testing Vocabulary
• Writing Specifications
– All vocabulary items introduced to the students in class should
be included in the specifications.
– Items should be grouped according to their relative
importance.
» Recently, the lexical approach born out of corpus linguistics
has produced word (and word group) lists listing
frequencies in which these words appear in print or media.
• Sampling
– Words can be grouped according to their frequency and
usefulness. Words can be taken out of these randomly – with
more being selected from groups containing the more
frequent and more useful words.
83. Testing Grammar and Vocabulary
Testing Vocabulary
• Item Writing
– Recognition. This is one testing problem for which multiple choice is a
useful technique.
» Synonyms: Test takers choose the correct synonym
» Definitions: Test takers choose the correct definition
» Gap filling: Test takers choose the correct item to go into the gap in
a sentence
– Production. Difficult to use in Proficiency Tests. Recommended for
Achievement Tests only.
» Pictures: Test takers write the names of items to match its picture.
» Definitions: Test takers write the lexical item for the its definition
» Gap filling: Test takers write the lexical item in a sentence where
the word has been deleted.
84. Testing Grammar and Vocabulary
Postscript
• While Grammar and Vocabulary DO contribute to
communicative skills, they are rarely to be regarded as
ends in themselves.
• Hence, it is essential that tests should not accord them
too much importance.
• To do otherwise would be to create a backwash effect
undermining the achievement of teaching and learning
objectives in a communicate classroom.
85. Test Administration
Preparation
• Materials and Equipment
– Organize the printing of test booklets in plenty of time.
– If previously used test booklets are to be used, check to make sure there are no marks left
by previous candidates.
– Number all the test material consecutively .
– Make sure there are sufficient keys for scorers.
– Check to make sure all equipment is working correctly.
• Examiners
– All examiners should receive detailed instructions which should be gone over at least the
day before the exam. An attempt should be made to cover all eventualities.
– Examiners should practice directions they will need to read out to test takers.
– Examiners should familiarize themselves with any equipment they may have to use.
– Examiners who need to read out loud for listening should practice.
– Oral examiners should be thoroughly familiar with procedures and rating system.
• Invigilators (or proctors)
– Detailed instructions should be prepared for invigilators.
86. Test Administration
Preparation
• Candidates/Test Takers
– Every test taker should be given full instructions beforehand (e.g.
place, time, materials, etc.).
– There should be an examination number for each candidate.
• Rooms
– Rooms should be quiet and large enough to accommodate the
intended number of test takers.
– There should be sufficient space between candidates to prevent
copying.
– For listening tests, rooms must have satisfactory acoustics.
– The layout of room should be arranged well in advance.
– Ideally, there should be a clock visible to all test takers.
87. Test Administration
Administration
– Test takers should be required to arrive well before the intended starting time.
– Test takers arriving late should not be admitted to the room.
– Identity of test takers should be checked.
– Should be seated to prevent cheating.
– Clear instructions should be given by the examiner.
– Test materials should be distributed individually to test takers by the
invigilators.
– Examiner should instruct test takers to provide the required details
(examination number and date, etc.) on the answer sheet or test booklet.
– Test should be timed precisely making sure everyone starts on and finishes on
time.
– Once the test has begun, invigilators should unobtrusively monitor the
behavior of the candidates and deal with any irregularities as laid down in
their instructions.
– During the test, test takers should only be allowed to leave the room one at a
time – and preferably accompanied by an invigilator.
– Invigilators should make sure test takers stop work immediately when they are
told to do so. Test takers should remain in place until all material are collected
and their numbers checked.