Published on

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. Dennis McGeehan EDU 510 Educational Assessment Final Paper Dennis McGeehan EDU 510 Educational Assessment Dr. White December 8, 2007
  2. 2. Dennis McGeehan EDU 510 Educational Assessment The diligent pursuit of improved assessment measures as reflected in the efforts of the NSF TEXTBOOK encourages the discovery and implementation of improved assessment methodology. Current assessment trends reflect the discoveries that have led to refinements in many testing aspects such as test content and delivery which are be addressed by this paper. Many current articles reflect problems identified in the standardized testing process. Caesar and Kohler (2007) and Meeks (2006) reflect concerns to adapt test content to special concerns for the needs of test subjects and Popham (2007) argues for instructional sensitivity for accurate assessment based teacher evaluation while the other studies by Stansbury and Kymes (2007) and Nicol (2007) reflect adjustments in assessment delivery which promise improved assessment results. Popham (2007) believes most high stakes tests have no direct relationship to what is taught in the classroom. He terms the current tests instructionally insensitive. Since high stakes testing seeks to determine the accountability of teachers, the tests should be linked to classroom instruction so that effective and ineffective instruction can be identified. The author considers most accountability testing as a reflection of students’ socioeconomic status more than an accurate depiction of effective classroom instruction. Under these conditions, he believes teachers are more likely to engage in test preparation than teaching the curriculum. To remedy this condition, Popham (2007) proposes standards to determine instructional sensitivity as identified by two categories of evidence, judgmental and empirical. Judgmental evidence involves the use of trained judges to evaluate test attributes, while empirical evidence is collected from students’ test 2
  3. 3. Dennis McGeehan EDU 510 Educational Assessment scores in two groups, those taught and not taught with the test material. Judgmental evidence would carry more weight than empirical. Traits to be judged include the quantity and quality of curricular items assessed, the ratio per curricular item and the instructional sensitivity of test items. Ratings should have a quantitative base. To be effective, the curricular items assessed must be a controllable number, so that limitations are placed on what needs to be assessed. These limitations allow teachers to concentrate on goals that are most vital. Teachers must also have a clear understanding of what needs to be taught so they will know what areas to concentrate their efforts. Teachers must also know the intent of what the test proposes to measure, and if are adequate questions on the test to effectively measure a curricular aim. The fourth and final evaluative trait, according to Popham (2007), is a gauge of the instructional sensitivity of the test. The test should not be a reflection of socioeconomic status or a student’s inherent skills, but should reflect the effects of the instruction they have received. If the purpose of these tests is to assess teacher accountability then the tests should be subject to thorough review for instructional accountability. Caesar and Kohler (2007) investigated the instances where alternative testing could be effectively substituted in standardized tests for bilingual students. Data for their study was collected by survey of speech-language pathologists affiliated with the schools. The researchers discovered in most instances the pathologists administered formal standardized tests to this group whereas, in fairness, alternative testing should be the more likely course. Data showed the determinate for the speech-language pathologists’ decisions were usually their employment setting rather than their professional training 3
  4. 4. Dennis McGeehan EDU 510 Educational Assessment which would appear a likely concession to unspecified political or social pressure. The root causes for concession themselves serve as a reason for further study of bilingual student testing. The authors conclude more attention should be paid to training and information dissemination to these pathologists in order to increase the awareness and sensitivity of the needs of bilingual students and their access to the alternative testing methods that would be preferable under such testing conditions. If fairness and objectivity are the goals of large scale assessment, then provisions must be made to accommodate psychological and physical impairments of some students. Meek offers insight to the conditions for high stakes testing of special education students through a qualitative account of the experiences of one student, Dominic. As federal and state legislation have addressed disability through legislation, notably the Americans with Disabilities Act or 1991, accommodations must be made to ensure fair treatment of those with special needs. In this case Dominic and other special education students are expected to complete the same testing as students without learning disabilities. Dominic has learning and emotional limitations; however, he has demonstrated academic progress to the extent he is expected to participate in testing. Meek describes the elaborate preparations made in consideration for the special students’ needs. These special students are trying, but Meek (2006) believes the state of California should reassess the purpose of requiring the same testing for them. Statistics reflect her concern as approximately one third of students like Dominic pass the state of California assessments. The results show that most special education students can not 4
  5. 5. Dennis McGeehan EDU 510 Educational Assessment contend with the rigors of the high stakes testing as currently administered. The duration and complexity of this testing are beyond their abilities. The author believes extensive modifications must be made to the testing to accommodate students with special needs. Suggested improvements include a provision to test special education students outside their grade level, most likely at a lower level than their peers, and revised structure to administer a limited portion of a test that remains within their capabilities to accomplish. Also special education students testing could be limited to their strengths and away from their established deficiencies so as not to severely impact overall test results for their school districts. Adaptation of large scale assessments for special needs of test subjects and for instructional sensitivity promise greater assessment accuracy. Efforts to enrich the learning experience through improvements to the assessment delivery system are addressed by the research of Stansbury and Kymes (2007) and Nicol (2007) on the application of electronic portfolios. Stansbury and Kymes study the effect of the portfolio and the student teacher’s experience with it as a learning tool on the teacher’s appraisal of its utility for classroom assessment. The portfolio itself applies to Mezirow’s theory of transformative learning. The student teacher can experientially link the value of the portfolio as a student and a teacher since institutions like Oklahoma State University (OSU) have incorporated technology as a learning process into the teaching curriculum. OSU students must create their own electronic portfolios for assessment. The authors saw merit in an exploration of whether the OSU students later continued the electronic portfolio project as teachers and 5
  6. 6. Dennis McGeehan EDU 510 Educational Assessment whether they considered the portfolio process itself an experience in transformational learning. Study subjects reported they were somewhat more likely to use an electronic portfolio than previously and that they had experienced some degree of transformational learning; however, the researchers expressed surprise upon the discovery that the likelihood for electronic portfolio assessment would be used was only marginal. Apprehension some OSU students had toward technology had been removed once they had applied technology as an assessment tool. Students who expressed some discomfort asking for assistance with technology reported more confidence after their experiences with the portfolio. Dr. Nicol’s concern for academic attrition in higher education focuses on how to best address reasons for student attrition, particularly attrition based on the student’s experience with an academic program and the student’s failure to manage program requirements. Doctor Nicol serves as Deputy-Director for the Centre for Academic Practice and Learning Enhancement at the University of Strathclyde, Scotland. Nicol (2007) believes greater formative assessment opportunities with support from information and telecommunication technology can moderate student attrition although he concludes further research will be needed to support an underlying link. With higher course enrollments of more diverse students driving larger student-teacher ratios, emphasis has changed from formative to summative assessment in the first year classes. This shift in course management caused by the higher student enrollments has put greater emphasis on grades and less focus on long-term learning. 6
  7. 7. Dennis McGeehan EDU 510 Educational Assessment Under these classroom conditions, teachers need adequate technology support for formative assessment to enrich the learning experience and to permit adequate feedback to student assignments. Properly deployed information and telecommunication technology (termed ICT) can enhance the quality of instruction and restructure these courses so that knowledge that comes with assessment and reflection rather than grades becomes most important to first year students. Nicol offers two examples of formative assessment made possible by ICT support as developed under Scotland’s Re-engineering Assessment Practices (REAP) project. The first example involves and introductory psychology class with 550 registrants. While 550 students seems high for an introductory college course by any standard, we learn there may be as many as 800 students in a first year foundation course. A class this size presents an immediate formative assessment problem accentuated by all students’ desire for test results as quickly as possible, certainly no longer that the week proceeding a test. To introduce feedback in the assessments, the course implemented a WebCT discussion board for student and teacher interaction. The second example, a mechanical engineering class, incorporates peer instruction (Mazur 1997) through an electronic voting system (EVS) that tallies multiple choice question responses. EVS tallies and reports student responses to produce a bar chart reporting the distribution of answers which leads to further student discussion of the question in search of the correct response. This example, like the previous one, engages the students in a highly engaged, scaffolding approach to learning. This approach is further enhanced through the use of certainty based marking (CBM) which requires the student report his or her confidence level in the response as 7
  8. 8. Dennis McGeehan EDU 510 Educational Assessment well as the answer chosen. CBM enhances learning through metacognitive thinking. Later, the instructor will inform students of the correct answer which offers an opportunity for reflection. Current trends like those addressed by this paper reflect the totality of concern for accurate evaluation of both student performance and teacher appraisal. The learning experience can be enhanced by its effective measurement through efforts to continuously improve the content and delivery of assessments. Caesar, L.G. and Kohler, P.D. (2007). The state of school-based bilingual assessment: actual practice versus recommended guidelines. Language, Speech and Hearing Services in Schools, 38(3), 191-200. Meek, C. (2006). From the inside out: a look at testing special education students. Phi Delta Kappan, 88(4), 293-297. Popham, W.J. (2007) Instructional insensitivity of tests: accountability’s dire drawback. Phi Delta Kappan, 89(2),146-155 Stansbury, S.L. and Kymes, A.D. (2007). Transformative learning through “teaching with technology” electronic portfolios. Journal of Adolescent and Adult Literacy, 50(6), 488- 496. Nicol, D. (2007). Laying a foundation for lifelong learning: case studies of e-assessment in large 1st -year classes. British Journal of Educational Technology, 38(4), 668-678. 8