Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
Uploaded bySteveMwenye
PPTX, PDF8 views

Comprehensive Guide to Testing, Measurement, and Evaluation in Education

Explore key concepts, purposes, types, and methods of testing, measurement, assessment, and evaluation to enhance student learning and instructional effectiveness.

Education◦
Download to read offline
TESTING, MEASUREMENT & EVALUATION
Aim(s) of the Study • To provide students with knowledge of testing, measurement and evaluation for the development and management of different ways of measuring student’s learning behaviour.
Course Objectives By the end of the course students should be able to:- • Define the terms testing, measurement and evaluation • Explain reasons for testing, measuring and evaluating students’ learning behaviours • Describe ways of measuring students’ learning achievements. • Construct tests • Administer tests • Interpret test scores correctly • Explain factors affecting student performance in relation to interpretation of test results. • Relate measurement concept to guidance and counseling
Topics of Study • Meaning of testing, measurement and evaluation • Purposes of evaluation • Assessment and classroom tests • Purpose of assessment • The concept of continuous assessment • Taxonomy of educational objectives • Old and modern practices of assessment • Tests in the classroom
Topics of Study Cont’d Types of tests, characteristics of good tests and test • Types of tests • Characteristics of good tests • Basic principles of test construction • Different kinds of tests to be constructed • Test contingencies, validity and reliability • Test contingencies • Test validity • Reliability of tests
Topics of Study Cont’d Administering tests, test scoring and interpretation of scores • Factors of test administering • Scoring of tests • Objective scoring • Using test results • Distribution and measures of central tendency • Measures of variability • Derived scores • Standard score or z-score • Course credit system and grade points •
PrescribedTexts • Gronlund N. & Cauneron I.J. (2004) Assessment Of Student Achievement. Toronto : Pearson • Hopkins K.D. & Stanley J.C. (1984) Educational and Psychological Measurement and Evaluation York : Holt Reinhart • Nitko A.J. (2000) Educational Assessment Of Students Merrill : Engleured Cloffs
Recommended Readings • Elbel R.L. & Frisbie D.A. (1991) Essential of Educational Measurement. NewYork : Holt, Reinhart AndWinston • Gronlund N.E. & Linn R.L. (1990) Measurement and Evaluation in Teaching. London : Macmillan Publications Co. • KubiszynT. & Borich G. (1993) EducationalTesting and Measurement. NewYork: Harper Collins College Publishers • A.J. (1983) Education Tests And Measurement:An Introduction. NewYork : Harcourt Brace Jovanovich, Inc • Ogumniyi A.B. (1984) Educational Measurement & Evaluation. Ikeja : Longman • Wiersma W. & Jurs S.G. (1990) Educational Measurement And Testing. Boston : Allyn & Bacon
Meanings of Testing, Measurement, Assessment and Evaluation
OBJECTIVE By the end of the lesson, you should be able to distinguish clearly between test, measurement, assessment and evaluation.
Measurement • Measurement is a process of assigning numerals to objects, quantities or events in order to give quantitative meaning to such qualities. • It is the process of quantifying the degree to which someone or something possesses a given characteristic, quality or feature. • Measurement process involves four steps 1. Define the characteristics that you want to measure. 2. Select the appropriate test.This may also mean to select the appropriate testing instrument. 3. Administer the test. 4. Collect and record the measurement from the test.
Measurement Cont’d • What do you measure in a classroom? • Individual differences on a specific trait (behavior). • How do you get measurement data?
Testing • Test: an instrument or activity used to accumulate data on a person’s ability to perform a specified task. • Data: the translation of behavior into a numerical or verbal descriptor which is then recorded in written form. • Testing is a technique for obtaining information.
Assessment • Assessment refers to all the processes and products which are used to describe the nature and the extent of pupils’ learning. • This also takes cognisance of the degree of correspondence of such learning with the objectives of instruction.
Evaluation • It is the estimation of the worth of a thing, process or programme in order to reach meaningful decisions about that thing, process or programme. • The process of making judgments about the results of measurement in terms of the purpose of the measurement. • Evaluation often includes recommendations for constructive action. • Teachers evaluate how well or to what extent their students are achieving instructional outcomes. • Teachers also evaluate students at the end of the year to determine whether they are suitably prepared for the work expected at the next grade level. • Good evaluation requires data gathering for informed decision making
Evaluation cont’d Steps involved in making an evaluation; 1. Define the objective or the purpose of the test. 2. Measure the performance or administer the test. 3. Find or develop a standard. 4. Compare a person’s performance on the test to a standard. 5. Make the evaluation then discuss and distribute the results in the most appropriate manner. REVIEW: Distinguish clearly between testing, assessment, measurement and evaluation.
Evaluation
Objective By the end of the lesson, you should be able to explain the purposes of evaluation in education.
Evaluation • Evaluation is concerned with the application of measurement findings, and implies some judgement. • It often includes recommendations for constructive action. • It calls for evidence of effectiveness, suitability, or goodness of the programme. • In other words, evaluation is the estimation of the worth of a thing, process or programme in order to reach meaningful decisions about that thing, process or programme. • A systematic process of determining what the actual outcomes are but it also involves judgement of desirability of whatever outcomes are demonstrated.
The Purposes of Evaluation • To determine the relative effectiveness of the programme in terms of students’ behavioural output; • To make reliable decisions about educational planning; • To ascertain the worth of time, energy and resources invested in a programme; • To identify students’ growth or lack of growth in acquiring desirable knowledge, skills, attitudes and societal values; • To help teachers determine the effectiveness of their teaching techniques and learning materials; • To help motivate students to want to learn more as they discover their progress or lack of progress in given tasks; • To encourage students to develop a sense of discipline and systematic study habits; • To provide educational administrators with adequate information about teachers’ effectiveness and school need; • To acquaint parents or guardians with their children’s performances;
The Purposes of Evaluation Cont’d • To identify problems that might hinder or prevent the achievement of set goals; • To predict the general trend in the development of the teaching-learning process; • To ensure an economical and efficient management of scarce resources; • To provide an objective basis for determining the promotion of students from one class to another as well as the award of certificates; • To provide a just basis for determining at what level of education the possessor of a certificate should enter a career.
Evaluation Cont/… •Types of evaluation • Formative evaluation • Summative evaluation
Formative Evaluation • The purpose of formative evaluation is to find out whether after a learning experience, students are able to do what they were previously unable to do. • Its ultimate goal is usually to help students perform well at the end of a programme. • Formative evaluation enables the teacher to: • Draw more reliable inference about his students than an external assessor, although he may not be as objective as the latter; • Identify the levels of cognitive process of his students; • Choose the most suitable teaching techniques and materials;
Formative Evaluation Formative evaluation also enables the teacher to: • Determine the feasibility of a programme within the classroom setting; • Determine areas needing modifications or improvement in the teaching-learning process; and • Determine to a great extent the outcome of summative evaluation. (Ogunniyi, 1984) Thus, formative evaluation attempts to: • Identify the content (i.e. Knowledge or skill) which has not been mastered by the students; • Appraise the level of cognitive abilities such as memorization, classification, comparison, analysis, explanation, quantification, application and so on; and • Specify the relationships between content and levels of cognitive abilities.
Summative Evaluation • Summative evaluation often attempts to determine the extent the broad objectives of a programme have been achieved • Some of the underlying assumptions of summative evaluation are that: • The programme’s objectives are achievable; • The teaching-learning process has been conducted efficiently; • The teacher-student-material interactions have been conducive to learning; • The teaching techniques, learning materials and audio-visual aids are adequate and have been judiciously dispensed; and • There is uniformity in classroom conditions for all learners. • REVIEW: With appropriate examples distinguish between formative and summative evaluation
Assessment and ClassroomTests Objective By the end of the lesson, you should be able to give the purposes of assessment.
Purposes of Classroom Assessment • Instructional Functions • Administrative Functions • Guidance Functions • Research And Evaluation • Instructional functions • Determination of level of achievement • Effectiveness of the teacher, teaching method, learning situation and instructional materials • Motivating the child by showing him his progress i.e. Success breeds success. • It can be used to predict students performance in novel situations.
Purposes of Classroom Assessment • Administrative functions • Assessing can serve as communication of information when data collected are used in reports to parents • It could form the basis upon which streaming, grading, selection and placement are based. • Making appropriate decisions and recommendations on curricula packages and curricula activities. • Guidance functions • assessment can be of value in diagnosing an individual’s special aptitudes and abilities. • diagnostic data about individual students will show the pupils’ strength, weaknesses and interests. • it can also help to decide on which method to use or what remedial activities that are necessary. • parents and pupils can also be rightly guided in terms of career choice.
Purposes of Classroom Assessment • Research and Evaluation • Assessments are useful for programme evaluation and research • Outcome measures are necessary to determine whether an innovative programme is better or poorer than the conventional one in facilitating the attainment of specific curricular objectives.
Assessment and ClassroomTests Objective: the end of the lesson, you should be able to compare old and modern assessment practices
Old Practices of Assessment • Comprised mainly of tests and examinations administered periodically, either fortnightly or monthly.Terminal assessment were administered at the end of term, year or course; hence its being called a ‘one-shot’ assessment. • Problems associated with this type of assessment • It put so much into a single examination. • It was unable to cover all that was taught within the period the examination covered. • Schools depended so much on the result to determine the fate of pupils. • It caused a lot of emotional strains on the students. • It was limited to students cognitive gains only. • Its administration periodically, tested knowledge only.
Old Assessment • Problems associated with this type of assessment • Learning and teaching were regarded as separate processes in which, only learning could be assessed. • It did not reveal students’ weakness early enough to enable teacher to help students overcome them. • Achievements were as a result of comparing marks obtained. • It created unhealthy competition which led to all forms of malpractices.
Modern Assessment • This is a method of improving teaching and learning processes. • It forms the basis for guidance and counselling in the school. • Teaching and learning are mutually related. • Teaching is assessed when learning is and vice-versa. • Assessment is an integral and indispensable part of the teaching-learning process. • The attainment of objectives of teaching and learning can be perceived and confirmed through continuous assessment. • It evaluates students in areas of learning other than the cognitive
Concept of Continuous Assessment • For any form of assessment to be able to serve the above functions, it cannot be a one shot kind of assessment. It has to be an on-going exercise throughout the teaching and learning processes. This is why continuations assessment is advocated in the classroom. • “A mechanism whereby the final grading of a student in the cognitive, affective and psychomotor domains of behaviour takes account in a systematic way, of all his performances during a given period of schooling”. • It involves the use of a great variety of models of evaluation for the purpose of finding and improving the learning and performance of the students.”
Concept of Continuous Assessment • Continuous assessment, means assessing or weighing performance of students periodically to be able to determine progress made in teaching-learning activities. • Continuous assessment tests can be done daily, weekly, monthly, depending on the goals of teaching and learning.
Continuous Assessment • Advantages of CA • It provides useful information about the academic progress of the learner; • It makes the learner to keep on working in a progressive manner; • It informs the teacher about the teaching-learning effectiveness achieved; • It gives a true picture of the student academic performance since it is a continuous process rather than one duration type of test which may be affected by many variables such as sickness, fatigue, stress, etc; and • It makes learning an active rather than a passive process.
• Characteristics of CA • In most cases, continuous assessment tests are periodical, systematic, and well planned.They should not be tests organized in a haphazard manner. • Continuous assessment tests can be in any form. They may be oral, written, practical, announced, or unannounced, multiple choice objective, essay, or subjective and so on. • Continuous assessment tests are often based on what has been learnt within a particular period.Thus, they should be a series of tests. • Invariably, continuous assessment tests are designed and produced by the classroom teacher. • All continuous assessment tests should meet the criteria for a good test: e.G., Validity and reliability.
Taxonomy of Educational Objectives Objective: by the end of the lesson, you should be able to explain bloom’s taxonomy of educational objectives.
Taxonomy of Educational Objectives • Classification of educational objectives • Cognitive; • Affective; and • Psychomotor domains • The focus of assessment is on these three domains of educational objectives.
Cognitive Domain Involves remembering previously learnt matter Has the following levels • Knowledge of specifics Remember previously learned information. • Comprehension Demonstrate an understanding of the facts. • Application Apply knowledge to actual situations. • Analysis Break down objects or ideas into simpler parts and find evidence to support generalization • Synthesis Compile component ideas into a new whole or propose alternative solutions. • Evaluation Make and defend judgments based on internal evidence or external criteria.
Affective Domain • Relates to interests, appreciation, attitudes and values • These learning outcomes include feelings, beliefs, attitudes, interests, social relationships etc.Which, at times are referred to as personality traits. • Some of these that can be assessed indirectly include: Honesty - truthfulness, trustworthiness, dependability, faithfulness etc. Respect - tolerance, respect for parents, elders, teachings, constituted authority, peoples’ feelings. Obedience - to people and law Self-control - temperamental stability, non-aggression, use of decent language etc. Social relationship - kindness, leadership and social skills.
Affective Domain Cont’d • The most appropriate instrument for assessment here is observation. • Others like self-reporting inventories; questionnaires, interviews; rating scales, projective technique and sociometric technique may as well be used as the occasion demands. • Note that the purpose of assessing students’ personality traits in the school is to give feedback to the students to help them adjust in the right direction rather than the assignment of grades.
Psychomotor Domain • Deals with motor and manipulative skills • These learning outcomes cannot be assessed through achievement tests or class work. • The learning outcomes stretch from handling of writing materials to activities in drama, practicals, laboratory activities, technical subjects, games and athletics. • Note: elements of cognitive behaviour are also present in these activities because for one to do something well, one must know how since the activities are based on knowledge.
Types of Tests • Diagnostic Tests • AchievementTests • Placement Tests • Predictive Tests • Aptitude Tests • Standardise Tests EducationalTesting, Measurement and Evaluation
Different Kinds of Tests • Diagnostic tests • Usually administered on an individual basis. • These tests are very good for determining problems in learning for individual students. • Diagnostic tests also bring out areas of difficulty of a student for the purpose of remediation. • Typically, these types of tests are administered by the school psychologist, counsellor, or trained in its administration and scoring.
Different Kinds of Tests • Achievement tests • For measuring the achievement of a candidate in a particular course either during or at the end of the course. • Placement tests • For placing students at a particular level, school, or college. • Predictive tests • Designed to be able to predict the learning outcomes of the candidate. A predictive test is able to predict or forecast that if the candidate is able to • Pass a particular test, he/she will be able to carry out a particular task, skill, course, action, or programme.
DIFFERENT KINDS OF TESTS • Aptitude tests Are designed to determine the aptitude (fitness) of a student for a particular task, course, programme, job, etc. • Standardized tests: Are any of the above mentioned tests that have been tried out with large groups of individuals, whose scores provide standard norms or reference points for interpreting any scores that anybody who writes the tests has attained. Standardized tests are to be administered in a standard manner under uniform positions. They are tested and re-tested and have been proved to produce valid or reliable scores.
Characteristics of GoodTests • A good test should be valid: by this we mean it should measure what it is supposed to measure or be suitable for the purpose for which it is intended. • A good test should be reliable: reliability simply means measuring what it purports to measure consistently. On a reliable test, you can be confident that someone will get more or less the same score on different occasions or when it is used by different people. • A good test must be capable of accurate measurement of the academic ability of the learner: a good test should give a true picture of the learner. It should point out clearly areas that are learnt and areas not learnt. All being equal, a good test should isolate the good from the bad. A good student should not fail a good test, while a poor student passes with flying colours.
Characteristics of GoodTests • A good test must represent teaching-learning objectives and goals: the test should be conscious of the objectives of learning and objectives of testing. For example, if the objective of learning is to master a particular skill and apply the skill, testing should be directed towards the mastery and application of the skill. • Test materials must be properly and systematically selected: the test materials must be selected in such a way that they cover the syllabus, teaching course outlines or the subject area.The materials should be of mixed difficulty levels (not too easy or too difficult) which represent the specific targeted learners’ needs that were identified at the beginning of the course.
Characteristics of GoodTests • Variety is also a characteristic of a good test. • This includes a variety of test type: multiple choice tests, subjective tests and so on. It also includes variety of tasks and so on. It also includes variety of tasks within each test: writing, reading, speaking, listening, re-writing, transcoding, solving, organizing and presenting extended information, interpreting, black filling, matching, extracting points, distinguishing, identifying, constructing, producing, designing, etc. In most cases, both the tasks and the materials to be used in the tests should be real to the life situation of what the learner is being trained for.
Types ofTest Items • Structured/ select response items • Multiple choice • Matching response • True-false response • Unstructured/supply response items • Essays • Restricted response e.g., Short essay; fill in; completion type • Extended response/long essay • Mixed variety • Performance tasks • Numerical/mathematics/calculations/equations/problem solving
Basic Principles of Test Construction
Writing Test Items General guidelines for writing test items • Begin test development with table of specifications. Table of specifications (or test blueprint) identifies the objectives and skills which are to be tested and the relative weight on the test given to each. Chart shows the relationship between objectives, content, and bloom’s taxonomy. Rows specify major categories of content. Columns represent six levels of bloom’s cognitive domain. Cells are filled in with the number or percentage of items to be developed for that content and at that level.
SAMPLE TABLE OF SPECIFICATIONS • TWO-WAY TABLE OF SPECIFICATIONS NEED TO CONSIDER AND DECIDE: • LENGTH OF TEST • WEIGHT TO BE GIVEN TO EACH OBJECTIVE • WEIGHT TO BE GIVEN TO EACH LEVEL OF TAXONOMY (BLOOM’S TAXONOMY) • ESTIMATE NUMBER OF ITEMS IN EACH CELL Content/Outcomes Knowledge (20%) Comprehension (40%) Application (40%) Totals Fundamentals of assessment (25%) 2 4 4 10 Planning test (25%) 2 4 4 10 Scoring and analysing tests (25%) 2 4 4 10 Interpretation and reporting test scores (12.5%) 1 2 2 5 Reliability of test scores (12.5%) 1 2 2 5 Total 8 16 16 40
Constructing Test Items Two general categories of test items 1. Objective items Require students to select the correct response from several alternatives or to supply a word or short phrase to answer a question or complete a statement. 2. Subjective items Permit the student to organise and present an original answer
General Characteristics of ObjectiveTest Items  Objective test items: those with a single correct response; regardless of who scores a set of responses, an identical score will be obtained. • Objective implies that subjective judgments of the scorer do not influence an individual’s score. • Also known as “selected-response” and “structured-response” items. • Include multiple-choice, matching, and alternate-choice items. • Typically assess lower-level skills such as knowledge, comprehension, and application (higher-order items are much more difficult to write).  Objective test items (continued) • Are relatively easy to administer, score, and analyze. • Writing high-quality items does require substantial time. • Although subjectivity is removed from the scoring process, substantial degree of subjectivity exists in determination of content to be covered by items. • Guessing is a distinct possibility. • Poor readers may be unjustly penalized.
General Characteristics of ObjectiveTest Items  General guidelines for writing objective test items Writing Objective Items: General Guidelines 1. Objective test items should cover important content and skills. 2. The reading level and vocabulary of each item should be as elementary as possible. 3. Each objective item should be stated in an unambiguous manner, and confusing sentence structure and wording should be avoided. 4. Objective items should not consist of verbatim statements or phrases lifted from the text. 5. Clues to the correct answer should not be provided.
General Characteristics of ObjectiveTest Items  General guidelines for formatting objective tests Formatting ObjectiveTests: General Guidelines 1. Vary the types of items that appear on classroom tests. 2. Group items similar in format together so that each type appears in a separate section. 3. Each section should be preceded by clear directions. 4. Within each section, order the items from easiest to most difficult. 5. Although all item types will not appear on every test, they should be arranged in the following order: true-false, matching, short answer, multiple-choice, and essay. 6. Provide adequate space for students to respond to each item. 7. Avoid splitting an item between two pages.
General Characteristics of ObjectiveTest Items  General guidelines for writing objective test items (continued) • Provide assurance that tests cover a representative and accurate sample of the content. • Option of using published objective tests. Consider a basic question: Do the items match the instruction provided to students? Textbook must be carefully examined for alignment. • Scoring is relatively easy. Use an answer key to arrive at the total number of items the student answered correctly.
Types of Objective Test Items  Multiple-Choice Items • Basic format consists of a stem and responses (one of which is correct; others are called distractors). • Stem may be written as either a question or an incomplete statement. • Three to five options. • Useful for assessing recall of facts and application of knowledge. • Can be used to assess higher-order thinking skills (much more difficult to write).
Types of ObjectiveTest Items  Multiple-Choice Items (continued) • Guidelines for development: Creating Multiple-choice Items: General Guidelines 1. The Stem Should Clearly Present The Problem To Be Addressed By The Student. 2. All Response Options Should Be Parallel In Type Of Content. 3. Options Should AvoidThe Use Of RepetitiveWords. 4. Adjectives Or AdverbsThat Substantially AlterThe Meaning Of A Stem Or Option Should Be Emphasized. 5. All Distractors In The Response Set Should Be Plausible. 6. The Grammar In Each Option Should Be Consistent With The Stem. 7. Items Should AvoidThe Inclusion Of “All Of The Above,” “None OfThe Above,” Or Any Equivalents As Response Options. 8. The Use Of AbsoluteTerms Should Be Avoided. 9. Items Should Remain Independent Of One Another. 10. Avoid Nongrammatical Clues Such As A Key Term Appearing In Both The Stem And Correct
Types of ObjectiveTest Items  Multiple-Choice Items (continued) • Advantages Allow a test to comprehensively and efficiently sample the content domain. Can be used in virtually all subject areas. Can be scored relatively quickly. Scoring is an objective process. Can provide diagnostic information. • Limitations Susceptible to guessing. Can be quite time consuming to construct.
Types of ObjectiveTest Items  Matching Items • Basic format consists of two lists (stimuli and responses). • Should be a one-to-one correspondence between members of the two lists. • Considered to be a special case of multiple-choice item. • Can assess knowledge and comprehension skills. • Especially useful in measuring understanding of concepts or terms that are interrelated.
Types of ObjectiveTest Items  Matching Items (continued) • Guidelines for development: Creating Matching Items: General Guidelines 1.The lists should be homogeneous. 2.The directions (i.e., the basis for matching) must be made clear. 3.Avoid “perfect matching” by placing more items in the response list than in the stimulus list. 4.Use relatively short lists of stimuli and responses. 5.Place longer phrases in the stimulus list and shorter ones in the response list. 6.Arrange the lists in some logical order.
Types of ObjectiveTest Items  Matching Items (continued) • Advantages Permit efficient assessment of related facts, ideas, and concepts. Relatively easy to construct. Basically a combination of multiple-choice items using the same set of responses. Scoring is relatively easy. • Limitations Require large amount of related concepts or ideas. Very difficult to design for higher-order skills.
Types of ObjectiveTest Items  Alternate-Choice Items • Essentially a special case of multiple-choice items where options are limited to only two choices. • Most popular type is the true-false item. • Variations might include “correct–not correct,” “yes–no,” and “fact– opinion.” • Can be effective if written carefully (despite negative press). • Tend to overestimate student achievement since students have a 50% chance of guessing the correct answer.
Types of ObjectiveTest Items  Alternate-Choice Items (continued) • Guidelines for development: Creating Alternate-Choice Items: General Guidelines 1.Avoid the use of absolute terms and other specific determiners. 2.Avoid testing trivial knowledge. 3.Items should be stated positively; if a negative must be used, underline, bold, or italicize the term. 4.Roughly half of the test items should be keyed true and half false. 5.True statements and false statements should be of equal length. 6.True-false items should be entirely true or entirely false.
Types of ObjectiveTest Items  Alternate-Choice Items (continued) • Advantages Relatively quick to construct, answer, and score. Can be scored efficiently and objectively. • Limitations Highly susceptible to guessing. Predominantly lower-level skills are appropriately assessed with these items.
Types of ObjectiveTest Items  Alternate-Choice Items (continued) • Variations Yes-no item Correction true-false item—requires students to rewrite any statement that is false. Embedded alternate-choice item—presents a series of alternate-choice items within a paragraph. Multiple true-false item—hybrid between multiple-choice and alternate- choice items (multiple true-false statements, each using the same stem).
TYPES OFTEST ITEMS SUBJECTIVETEST ITEMS
General Characteristics of Subjective Test Items  Subjective test items:Those items that typically do not have a single correct response. • Subjective implies that subjective judgments of the scorer are an integral part of the scoring process. • Also known as “free-response,” “constructed-response,” and “supply-type” items. • Include essay items. • Require students to produce what they know, as opposed to merely recognizing the best response from a set of options.
General Characteristics of Subjective Test Items  Subjective test items (continued) • Allow students to search for ideas and concepts that are not restricted to a predetermined set of responses. • Relatively easy to construct. • Time consuming for students to answer. • Can be time consuming and quite subjective to score, depending on the format.
Types of SubjectiveTest Items  Essay Items • Questions or prompts that require students to write paragraphs or develop themes as responses. • Involve a wider variety of thinking skills (students must recall, select, organize, and apply). • Responses may range from a few sentences (restricted-response items) to several pages (extended-response items); range along a continuum… Restricted- response Extended- response
Types of SubjectiveTest Items  Essay Items (continued) • Appropriately used to assess complex thinking skills (i.e., analysis, synthesis, and evaluation). • Sometimes used exclusively or as part of a larger performance assessment. • Not appropriately used to assess lower-level skills. • Scoring is complex and typically unreliable (due to influence of response characteristics). • Scoring can be quite time consuming.
Types of SubjectiveTest Items  Essay Items (continued) • Guidelines for development: Creating Essay Items: General Guidelines 1. Essays should consist of the application of essential knowledge to new situations. 2. Essays should present a clear and focused task to students. 3. Specify for students the desired length, time limits, and evaluative criteria. 4. Develop a model response.
Types of Subjective Test Items  Essay Items (continued) • Advantages  Can be used to elicit a wide variety of responses.  Students are permitted to create their own responses.  Probability of guessing is substantially reduced. • Limitations  Very time consuming to score.  Content coverage may be restricted.
Assembling,Administering, Scoring and Analysing Classroom Tests
AssemblingTests • Components That Constitute Test Assembly: • Packaging The Test • Reproducing The Test
Packaging Guidelines • Group together all items of similar format • Arrange test items from easy to hard • Space the items for easy reading • Keep items and options on the same page • Position illustrations near descriptions • Check your answer key • Determine how students record answers • Provide space for name and date • Check test directions • Proofread the test
Reproducing the Test • Know the photocopying machine • Specify copying instructions • File original test
Administering the Test • Suggestions to help students psychologically prepare for the test: • Maintain a positive attitude • Maximize achievement motivation • Equalize advantages • Avoid surprises • Clarify the rules • Rotate distributions • Remind students to check their copies • Monitor students • Minimize distractions • Give time warnings • Collect tests uniformly
Scoring the Test • General suggestions to save scoring time and improve scoring accuracy and consistency • Prepare an answers key • Check the answer key • Score blindly • Check machine-scored answer sheets • Check scoring • Record scores
Analysing the Test • Classroom tests can include inappropriate, invalid or otherwise deficient items. • Item analysis is used to identify items that are deficient in some way. • Item analysis may result in improving some items, and eliminating others. • Item analysis is a technique that enables the examiner to assess the quality or utility of an item.
Item Analysis • The item analysis is an important phase in the development of an exam programme. • In this phase statistical methods are used to identify any test items that are not working well. If an item is too easy, too difficult, failing to show a difference between skilled and unskilled examinees, or even scored incorrectly, an item analysis will reveal it. • The two most common statistics reported in an item analysis are the item difficulty, which is a measure of the proportion of examinees who responded to an item correctly and the item discrimination, which is a measure of how well the item discriminates between examinees who are knowledgeable in the content area and those who are not. • An additional analysis that is often reported is the distractor analysis.The distractor analysis provides a measure of how well each of the incorrect options contributes to the quality of a multiple choice item. Once the item analysis information is available, an item review is often conducted.
Analysing the Test • Key words • Key • Correct option in a multiple-choice item. • Distractor • Incorrect option in a multiple-choice item. • Difficulty index (p) • Proportion of students who answered the item correctly. • Discrimination index (d) • Measure of the extent to which a test item discriminates or differentiates between students who do well on the overall test and those who do not do well on the overall test.
Item Difficulty Index • The item difficulty index is one of the most useful, and most frequently reported, item analysis statistics. • It is a measure of the proportion of examinees who answered the item correctly; for this reason it is frequently called the p-value. As the proportion of examinees who got the item right, the p-value might more properly be called the item easiness index, rather than the item difficulty. • It can range between 0.0 and 1.0, with a higher value indicating that a greater proportion of examinees responded to the item correctly, and it was thus an easier item. • With their emphasis on mastery-testing, many items on an exam form will have p-values of .9 or above. • On the other hand, are designed to be harder overall and to spread out the examinees' scores.Thus, many of the items will have difficulty indexes between .4 and .6.
Analysing the Test • Suppose your students chose the options to a four alternative multiple- choice item the following number of times: A b c* d 3 0 18 9 • B was not a very good option or distractor, because no one chose it. • More than half the class answered the item correctly.
Analysing the Test • Formula for computing p, the item’s difficulty index (this is first step in item analysis) • The item was moderately difficult (60% of the class got it right). • Option b ought to be modified or replaced
Item Discrimination Index • The item discrimination index is a measure of how well an item is able to distinguish between examinees who are knowledgeable and those who are not, or between masters and non-masters. • There are actually several ways to compute an item discrimination, but one of the most common is the point-biserial correlation. • This statistic looks at the relationship between an examinee's performance on the given item (correct or incorrect) and the examinee's score on the overall test. • For an item that is highly discriminating, in general the examinees who responded to the item correctly also did well on the test, while in general the examinees who responded to the item incorrectly also tended to do poorly on the overall test.
• The possible range of the discrimination index is -1.0 to 1.0; however, if an item has a discrimination below 0.0, it suggests a problem. • When an item is discriminating negatively, overall the most knowledgeable examinees are getting the item wrong and the least knowledgeable examinees are getting the item right. • A negative discrimination index may indicate that the item is measuring something other than what the rest of the test is measuring. • More often, it is a sign that the item has been mis-keyed.
• When interpreting the value of a discrimination it is important to be aware that there is a relationship between an item's difficulty index and its discrimination index. • If an item has a very high (or very low) p-value, the potential value of the discrimination index will be much less than if the item has a mid-range p-value. In other words, if an item is either very easy or very hard, it is not likely to be very discriminating. • A typical test, with many high item p-values, may have most item discriminations in the range of 0.0 to 0.3. • A useful approach when reviewing a set of item discrimination indexes is to also view each item's p-value at the same time. • For example, if a given item has a discrimination index below .1, but the item's p-value is greater than .9, you may interpret the item as being easy for almost the entire set of examinees, and probably for that reason not providing much discrimination between high ability and low ability examinees
Analysing the Test • Discrimination index • Steps for determining discrimination index (D): 1. Arrange the papers from highest to lowest score. 2. Separate the papers into an upper and a lower group based on total test scores. Do so by including half of your papers in each group. 3. For each item, count the number in the upper group and the number in the lower group that chose each alternative. 4. Record your information for each item in the following form:
Analysing the Test Example for item X (class size = 30) Options A B C D Upper 1 0 11 3 Lower 2 0 7 6 5. Compute D, the discrimination index, using the following formula:
Analysing the Test • Plugging in our numbers in the example we arrive at • The discrimination index (D) is 0.267, which is positive. More students who did well on the overall test answered the item correctly than students who did poorly on the overall test.
Scoring Essay Type Tests
Outline What Constitutes a Good or Acceptable Answer As a Scoring Key This Calls For a Marking Scheme
Marking Scheme • Marking scheme provides the criteria for making judgments about the different levels of student performance and the marks to be awarded. • Mainly concerned with “supply response item format”, e.g., Essay questions. Why A Marking Scheme? • Provides a clear basis for distinguishing between different levels of achievement. • Supports consistency in marking, both over the time of the marking and where more than one marker is involved. • Provides the basis for useful feedback to students on their demonstrated achievement
Checklist For a Good Marking Scheme • Provide clear criteria to identify and differentiate students’ achievement of the course objectives • Reflect the relative importance and difficulty of parts of a task where it consists of a number of parts • Describe the range of student achievement in relation to what is expected to be demonstrated • Indicate the quality of response required to gain marks or a sub-range of marks
Two Major Approaches of Scoring • Holistic scoring • Analytic scoring
Holistic Scoring • One global, holistic score for a product or behaviour • Method where a piece of writing is evaluated for its overall quality. • Requires the examiner to evaluate the work as a whole. • Assigns a single score to represent a weighting of the whole work. • The score is based on a general impression • Often accomplished by placing essays in designated piles that represent different degrees of quality • Most appropriate for extended-response essays in which responses are not limited and are generally long