By: Lauren Chadwick ED 450Monday September 9, 2012
BellworkO Please use a pen or pencil and draw 2 appointment slots Once you are finished on the plate I gave open the folded paper you when you and fill in the blank entered the with the type of word classroom: written on the outside. 1 2 Put paper in box.
SLO’sO Students will … O Understand reliability in assessment. O Describe the 3 types of reliability evidence. O Explain the importance of standard error of measurement.
Let’s have some fun! O Everyone get up and stand in a circle!
Reliability = ConsistencyO Reliability is an O 2 important ways important criteria reliability may be useful: used when judging O How reliable are standardized tests. your assessments for you and your students? O Parents may ask how reliable standardized tests may be.
3 Types of Reliability Evidence O Stability O Alternate Form O Internal Consistency
Stability ReliabilityO Consistency of test O Example: Midterm on results over time TuesdayO The goal is to see O A masked man grabs your briefcase close scores in containing the students’ tests! O The next day you tell the students tests administered that their tests were stolen and that even if they are they need to re-take the test! given on different O Because there has been no new occasions. information taught since the last test, you expect that the test scores would be similar. O So, if you were able to see that the scores were not similar, then your exam would be judged to have little stability reliability.
Test-RetestO To see how stable an assessment’s results are over time, we usually test students, then in a week or two retest with the same form of assessment.O It is important though that there is no significant event taking place in between the two assessments. Example: field trip, movie, speaker- exposing student to test-relevant information.
So, how are scores computed?Correlation Coefficient (r) So…..O Reflects the degree of O …if the test–retest r similarity between for two sets of test scores on two tests/ scores were .84 thisO Ranges from: would show that the +1.0 to -1.0 students’ performance on both tests were similar.(close to 1.0 = strong relationship)(closer to 0 = weak relationship) O If the r was instead .23 this would show that the results were not as similar.
Classification- ConsistencyO To decide which students Student Test 1 Test 2 would be exempted from further assessment. Jill 84% 99%O Example: If you decide on Jack 65% 82% an 80% answers correct required for exemption, then on a test- retest approach, the teacher would find the % of students who scored ≥ 80% on both assessments.
How to determine Classification-ConsistencyA % of students exempt42%B % of students not exempt46%C % of students different classification12%D % of classification – consistency ( A + B)---------------------------------------------------------------------------------------------- 88% Classification- Consistency of that test
Alternate-Form ReliabilityO Simply put: O Usually found in settings where a student is testing to O Providing an receive his/her diploma. equivalent retest O BUT keep in mind when you are reviewing a for students in state-developed test another form other which claims to have than the first and equivalent forms, that failed assessment. those tests assess the same information as the original.
Internal Consistency ReliabilityO Reflects the degree O Example: of consistency of the items on a given 20 item test on problem test. solving: O Homogeneous Students who are strongO Each of the test’s problem solvers should get most of the 20 items items should be correct assessing the same variable.
FYIO Tests containing items O Tests containing items which a students can where students can either be right or wrong be given different such as multiple- grades such as in choice, the most commonly used essay items, the most approaches are the: common approach is: Kuder-Richardson Cronbach’s procedures coefficient alpha
Just a thought…O The more times you taste from a pot of soup, the more accurate will be your estimate of what the ingredients are.O One taste might fool you, twenty tastes will give you a much better idea of what’s in the pot.O More items on a test will yield more reliable estimates of understanding.
Choose one…Stability Reliability ≠ Alternate-Form Reliability ≠ Internal Consistency Reliability These three forms of reliability evidence should not be used together. Instead, keep in mind the kind of educational decision linked to the results of the assessment.
Standard Error of MeasurementO Supplies information of test consistency of an individual student by estimating their score consistency. O Problem: Teacher see’s student results from a pervious IQ test. They expect that the student will perform on that exact levelO SEM helps to remind teachers that scores earned on commercial or classroom tests are not always exact on student’s understanding.O Another area where SEM is used is in reporting of student performance on state tests.
60 item Math test O Jack answers 53Classificatio # correctn out of the 60Advanced 54 – 60 questions correctly,Proficient 44 – 53 so what would heBasic 37 – 43 be classified as?Below Basic 36 and below a proficient student.
Appointment Time!Everyone find a classmate for your 1st appointment.You have 3 minutes to decide whether the following isstability, alternate-form, or internal consistency reliability.Suppose you were trying to show that the items onan educational assessment all measured the samevariable. What type of reliability evidence wouldyou use? Internal consistency reliability
Appointment Time!Everyone find a classmate for your 2nd appointment.You have 3 minutes to decide whether the following is stability,alternate-form, or internal consistency reliability.A new nationally standardized reading test forjunior high students has been created, but the testdevelopers want to make sure that the threedifferent forms of tests are performing in essentiallythe same way. What sort of reliability evidenceshould they rely on? Alternate-form reliability
Why do you need to know?O You may be called on O Understanding to explain to parents standard error of the meaning of their measurement will child’s test scores. be helpful when you need to explain toO At least be parents and knowledgeable about students how to the meaning of make sense of their reliability in educational scores on assessments. accountability tests.
Check your understanding..O Do you understand assessment reliability?O Do you know the 3 types of reliability evidence?O Can you explain the importance of standard error of measurement?