1. Criterion-Referenced Diagnostic and Achievement Tests

2. This last realization led directly to the development of criterion referenced tests that were based on the objectives of our courses. These tests were developed by the teachers in each skill area to directly reflect the objectives of their courses and the types of activities that were going on in those courses. Therefore, the test could be considered very much like the final examinations that teachers almost everywhere develop for their courses.

3. We set out to create two forms of the test for each course we could use them as a pre-test and post-test in each session. We aimed at having three forms of each rest for each course so that we could use them as pre-test, midterm, and post test. When these tests were fully implemented, we were in a position to give the students diagnostic information objective-by-objective based on pre test, diagnostic progress objective-by-objective based at the midterm, and achievement information objective-by-objective at the end of the course.

4. One problem we had to deal with was the practice effect. Brown (1988, p.35) defines the practice effect as “the potential influence of the measures on each other.” In this case, simply having taken one of the tests was likely to improve the students’ performances on any subsequent administrations of the same test. We took the view that the tests should be considered part of the teaching and that the practice effect could be considered part of the learning experience. We wanted to minimize the effect that remembering specific test questions would have on subsequent tests. We did not want any student taking exactly the same form of any test more than once.

5. To minimize the impact of the practice effect, we used what is called a counterbalanced design for administering the tests. We developed such an elaborate system of tests because we felt that it was important to provide valuable diagnostic, progress, and achievement information to the students and their teachers, and to do so without causing a direct practice effect by allowing students to take the same test twice. We also recognized that it is important to have pre test-post test information in developing CRT’s so that the effectiveness of the items can be studied and the tests can be improved.

6. Benefits of a Sound Testing Program

7. These criterion-referenced tests were not easy to develop. In fact, a tremendous amount of work was involved. However, a determined group of teachers did create these tests, and did so in multiple forms. The payoff from all this work was enormous, especially with regard to gathering information on all the elements of the curriculum design process.

8. First, the tests helped us to closely examine our perceptions of the students’ needs. Second, after we discovered that some objectives did not need to be taught, we had the freedom to concentrate instead on the remaining objectives or to add new objectives designed to meet more advanced needs. Perhaps we were succumbing to what Tumposky (1984) sarcastically calls the “cult of efficiency,” but frankly I do not understand why attempting to be relatively efficient is a bad thing.

9. Third, changing the objectives due to what we learned from the tests naturally led to rethinking our materials and teaching strategies to meet the newly perceived needs of the students. The fourth and last benefit gained from our testing program was that whenever we needed to focus on program evaluation, we had a great deal of information ready to be presented. We had information about the overall proficiency of our students, as well as information about what and how much the students were learning in the classroom.

10. Initial Screening Procedures

11. Before students are admitted to UHM, they are carefully screened by the Office of Admissions and Records. The students’ previous academic records, letters of recommendation, and TOEFL scores are reviewed and only those students with total scores of 500 or higher are accepted for admissions to UHM. This information, including each students’ TOEFL subtest and total scores, is transmitted to the ELI. If their scores are above 600, students are notified that they are exempted from any ELI requirement. Those students who have scored between 500 and 599 are notified that they must clear the ELI immediately upon arrival at UHM. In this way, the initial screening procedures narrow the range of English proficiencies with which the ELI must deal.

12. Placement Procedures

13. In most cases, however, students who scored between 500 and 599 on the TOEFL are required to take the ELI Placement Test (ELIPT) as soon as they arrive on campus. We require this test for three reasons: Because we want more detailed information than is provided by TOEFL scores Because we want specific information about how the students will fit into our particular language programs Because we are also interested in getting information that is more recent than their TOEFL scores

14. Placement procedures are particularly important in programs, like our ELI, that have different tracks and level. Recall that we have four tracks, each of which is focused on one skill. Within these tracks there are up to three levels. So the placement decisions must be focused on the skills involved. The ELIPT is a three-hour test battery consisting of six subtests: Academic Listening Test (ALT), Dictation (DCT), Reading Comprehension Test (RCT), Cloze Procedures (CLZ), Writing Sample (WTS), and Academic Writing Test (AWT)

15. The Academic Listening Test and Dictation are used to place students into our listening skill courses. The Reading Comprehension Test and Cloze Procedure are used for the reading skill courses. The Academic Writing Test and Writing Sample are employed for placing students into proper writing skill level. In each skill area, one of the subtests is discrete-point, that is, it tests the discrete bits and pieces of the language, and one is integrative, that is, it requires the student to integrate and use a variety of different language skills.

16. First-Week Assessment Procedures

17. During the first week of instruction, all ELI teachers administer a criterion-referenced test designed specifically to test the objectives of their courses. The teachers are also asked to keep a close watch on their students to see if any have been misplaced. When teachers find students who seem to be in the wrong level, they consult with the ELI director. Interviews with the students are conducted to find out what they want to do, and they are advised about what we think should happen.

18. Achievement Procedures

19. At the end of each semester, evaluation report forms are filled out by the teachers about the performance of every student. Teachers are asked specifically what ELI course level the students should take during the next semester. On the basis of classroom performance and the student’s score on the criterion-referenced final achievement test, the teachers can suggest that students skip one level or be exempted from any further study in that particular skill area. Again, interviews with the ELI director are arranged, and the students are advised on the course of action that we think would be most appropriate.

20. Criterion-Referenced Diagnostic and Achievement Procedures

21. There are also tests within each of the courses in the ELI courses that are designed to test the specific objectives of that course. Unlike the criterion-referenced tests developed for the GELC program, the tests in the ELI only exist in two forms.

22. The two forms of each test are used in a counterbalanced design for the course pre test and post test. These tests, like those at GELC, have proven useful for giving information to the students and their teachers, as well as for assessing the accuracy of our perceptions of students’ needs and the appropriateness of the course goals and objectives.

23. This whole system of tests, including proficiency, placement, diagnosis, and achievement, is made relatively efficient by instituting the role of lead teacher for a teacher who is given 50 percent release time to develop, analyze, revise, score, and report the results of ELI tests.

24. For the criterion-referenced diagnostic and achievement tests, the lead teacher has proven essential in rallying the teachers to review and revise each of the forms for each course and getting the tests to the teachers on time for use in class.

25. ELI Testing Program

26. We sincerely hope that the vast majority of the students who are served by the procedures discussed above are correctly classified, placed, diagnosed, and promoted. Decisions are made by human beings, and even when they are based on seemingly scientific information in the form of test scores, human judgments can be wrong.

27. All this takes a little more effort on the part of the administrators and teachers, but the benefits gained from effective and humane testing procedures accrue to all students teachers, and administrators alike.

28. One particularly useful side effect of having a complete testing system, or program, is that information gained from one type of test can sometimes be utilized to improve the other types. Information gained from the CRT Achievement Tests can prove useful in revising the placement procedures so that they more closely match that which is being taught in the classrooms. Brown (1989b) describes one such process. The CRT subtest of the ELIPT was made to more closely match the kinds of approach, syllabus, techniques, and exercises that actually go on in the reading courses. The procedures and statistics involved in the process are explained step-by-step in the article cited.

29. Yes!! The End?! (ha-ha..) THANK YOU^^ GOD Bless! J.K.A.S