There is a mistake on the slide that talks about confidence intervals. I accidentally refer to a confidence interval for a score of 150 as being from 160 to 170....that's not possible! A confidence interval goes from below the actual score to above the score so I should have said 140 to 160. Sorry!
2.
Do Now• Download the ―Score reports‖ file from the website Choose a score report based on your grade level and/or interest.• Look over the report and try to find any of the following. Label them on the score report. • Raw score • Standard/scaled score • Confidence intervals or score range • Standard error of measure • Percentile/Percentile rank • Grade/age equivalent• Highlight other scores/words you encounter that you are not familiar with.
3.
ObjectivesRWBAT…• Define terms related to standardized assessment• Read a standardized assessment score report to gain information about a student• Translate a score report into student and parent-friendly language• Describe how accommodations are used in district/state assessments
4.
What are standardized assessments?• Also known as Formal Assessments• Designed by people who are experts in test construction and distributed by companies• Test administration kept consistent• Objective• Strict standards for scoring• Often multiple choice
5.
Types of Formal assessments• Aptitude—used to predict performance in the future (ex. ACT, IQ tests)• Achievement—used to evaluate academic skills (ex. ISAT) Reasons for administering achievement tests: • Determine eligibility for special education • Identify students in need of support • Compare students across schools, districts, etc. • Evaluate schools, teachers, curriculum, etc.
6.
Norm vs. Criterion Referenced• Norm-referenced assessments compare students with others who took the test. This group is called the norm or reference group.• Criterion-referenced assessments compare students with a pre-determined set of skills. These tests do not compare students with each other but focus on describing a individual student’s academic performance.
7.
Reporting ScoresWays to report scores on standardized tests:• Raw score• Standard/scale score• Percentile• Stanine• Confidence intervals• Cut scores
8.
Helpful Terms• Mean • average score (*used the most)• Median • the middle score• Mode • the score the occurs the most often• Standard deviation • measure of variability, shows the average deviation of scores from the mean• Example: Find the mean, median, and mode • 60, 78, 80, 80, 85, 92, 96
9.
ExampleGiven scores of 60, 78, 80, 80, 85, 92, 96 Mean = 81.6 Median = 80 Mode = 80 Standard Deviation = 11.6
10.
The Normal Curve• The ―normal distribution‖ is a distribution of scores that are distributed evenly around the mean score. It describes many naturally occurring physical and social phenomena. Many scores will cluster around the mean. Fewer scores fall at the end points or at the higher and lower occurrences.• Sometimes referred to as a ―bell curve‖
12.
Properties of the Normal Distribution• The mean is the midpoint of the distribution.• The mean, the median, and the mode are all at the same point.• 68% of all scores are located in the area of plus or minus one standard deviation from the mean.• About 16% of the scores are higher or lower than one standard deviation above the mean.• Only 2% of the scores are more than two standard deviations above or below the mean.• Example: If you scored 700 on the SAT math test, you did well, because only about 2% of the people who take the test make that score.
13.
Raw Score• The number of questions answered correctly• Always reference the number of items (questions) when interpreting the raw score• Never try to interpret raw scores across subtests or between tests because the subtests & tests differ in difficulty and in the number of questions asked• Example: A CPS reading benchmark test gives raw scores out of 45
14.
Standard Score• Also sometimes called a scaled score• Way of converting raw scores to a number that makes more sense (i.e. does not take into account the number of items)• Uses the mean and standard deviation to convert scores.• T-scores and Z-scores are each types of standard score.• Many psychological tests use standard scores with a mean of 100 and a SD of 15 (ex. IQ tests— 100 is average)
15.
Percentile Rank• These scores show the percentage of students in a normed sample who score at, below, or above a particular raw score.• A student scoring better than three-fourths of the students in the normed sample would be at the 75th percentile or have a percentile rank of 75. A percentile rank of 50 means the student scored as well, or better than 50% of the students in the normed sample.• Different from % correct. Percents are only used with raw scores.
16.
Stanine• Divides the normal curve into 9 sections.• Range from stanine 1 (lowest) to stanine 9 (highest)
17.
Grade Equivalent Scores (GE)• GE scores are obtained from separately normed samples for each grade level.• They are reported in numbers such as 3.8, 5.4, 11.2. The whole number equals the grade and the decimal the month.• GE scores are often misleading and misinterpreted. For example, if a 7th grade student has a GE of 10, should she be promoted to the 10th grade? • No! A GE score of 10 does not mean that the student can do 10th grade level work. This score means that the 7th grade student scored similar to what a 10th grade student would have scored on the 7th grade test. The high scores may simply represent superior mastery of material at the 7th grade level.
18.
Confidence Interval• This is a ―standard error band‖ within which a student’s score can fall. It is sometimes referred to as a score range on assessment reports.• It is based on the standard error of measurement (SEM), and is an estimation made by test developers who take into consideration how much a student’s score would vary over repeated testing sessions.• You might see this as a shaded area surrounding a graph of scores
19.
Cut Scores• Some tests use cut scores to divide students into classification groups (e.g. meeting standards, below standards, etc.)• Cut scores are normally use the standard/scale score instead of percentiles• Examples: • http://www.isbe.state.il.us/assessment/htmls/psae_cut_points.htm • http://www.isbe.state.il.us/assessment/pdfs/2013/isat/cut- scores13.pdf
20.
RIT Scores• Part of the NWEA/MAPS test• Basically a standard score used by the test makers• Grade-independent: a score of 235 is the same no matter what grade so a 4th grade student and a 5th grade student both with a score of 235 are on the same level• Equal intervals between scores: a student who goes from 150 to 170 has made the same amount of growth as a student who goes from 170 to 190.• Presented in Descartes Ranges: way of using cut scores to put students in different groups for instructional planning purposes. Descartes ranges connected to criterion information.
21.
Standardized Assessment Reports• Review your score report: • Any new terms/scores you can identify? • Lingering questions?
22.
Communicating Assessment Results withParents and StudentsTyesha, a 7th grade student in your class, just took astandardized assessment and received the score reportbelow. Her mother emails you asking you to explain whatthe different numbers mean.Tyesha’s Standardized Test Scores for Math Standard score 182 Grade equivalent 5.5 Percentile Rank (district normed) 43 Stanine 5
23.
Standard score 182Her score on the test converted to some scale used by the testmakers. That does not mean that she got 182 questions correct.(It would help to show the parent the scale used for the test –what does 182 mean?) Grade equivalent 5.5Tyesha scored as well as a fifth grader in the fourth month ofschool would have scored on this 7th grade test. It doesn’t meanshe’s at the 5th grade level in math but it does mean she is a bitbelow where she should be. Percentile Rank (district normed) 43Tyesha scored better than 43% of the students who took thetest. She is slightly below the average. Stanine 5The stanine divides scores into 9 groups. Five is the middlestanine so Tyesha’s score is in the middle range.What can we say overall?
24.
Communicating Assessment Results withParents and Students• Avoid using statistical terms – standard deviation, mean, etc.• Avoid terms that could be triggers for parents – bell curve, ―normal‖ curve• Keep things simple – ex. You don’t have to explain how we get a standard score• Be positive but also honest!• Talk about next steps – what do the results tell you about where you (and the student) need to go
25.
Evaluating Standardized TestsAll forms of assessment should have evidence ofreliability and validity.• Reliability refers to the consistency of the test results, or how likely it is that a person will get the same score each time the test is taken.• Validity refers to the adequacy and appropriateness of interpretations based on the test resultsIt is also important to consider who is the in thenorming group.• How do the characteristics (i.e. race, gender, geographic region, disability) of the population used as the norming sample compare with your population of students
26.
ReliabilityHow reliability is judged (done by testmanufacturers):• Test-retest: give a the same test at two different times• Alternate form: give two versions of the test• Split-half: break test into two sections• Internal consistency – statistical measure of reliability• Interrater: have two different people score a test (only works for tests that involve ratings of performance, i.e. not multiple choice)
27.
Validity• Content • How well a test’s items reflect a particular body of knowledge and skill • Ex. Does the math section on the 3rd grade ISAT reflect what is covered in 3rd grade math?• Criterion • How well a test predicts a student’s future behavior or compare to current performance on other measures • Ex. How well does the ACT predict college success?• Construct • How well a test measures some internal attribute of a person • Ex. If a test is measuring student verbal reasoning, how is that term defined/measured?• Consequence • What decisions are being made based on the test results • Do the test results accomplish their intended purposes and avoid unintended consequences?
28.
Validity of standardized assessments forstudents with disabilities• Issues with construct-irrelevant variance - Is the test measuring something other than it intends? For example, if a math test has a heavy reading load, does it accurately measure the math ability of a student with a severe reading disability?• Use of accommodations – Do the accommodations being used impact the test in such a way that is no longer measuring what they intend to?• Consequential validity – Are the test results being used appropriately for students with disabilities? For example, if a student is being retained based on ISAT results, is that appropriate for a student with a severe learning disability?• Predictive power – If a test is designed to predict future performance (e.g. the SAT and college grades) does it still hold true for SWD?
29.
Accommodations for District/StateAssessments• Remember the 5 types: • Presentation • Response • Setting • Timing • Scheduling• Located in Section 10c of the IEP
30.
Accommodations for District/StateAssessments• Not always the same as accommodations used for classroom assessments. Different accommodations are allowed for different tests.• For a student to receive accommodations on standardized tests they must be listed in the IEP and have been used previously by the student for classroom assessments. (Ex. You can’t create new accommodations just for the ISAT if a student hasn’t used them in classroom assessments.)
31.
Allowable Accommodations• Each district/state test has accommodations that they allow to be used by students with disabilities. They also may have procedures for schools to apply and get approval for individual students to use accommodations on an assessment.• The school counselor or case manager usually handles these accommodations.• http://www.isbe.state.il.us/assessment/pdfs/2013/IEP_504 _guidance.pdf
32.
Accommodations on the NWEA• NWEA lays out guideline for accommodations: http://www.nwea.org/node/4615• Not as strict as ISAT and PSAE since they are not used for high-stakes decisions at the district or state level.• Download the accommodations checklist (on the web page above) to keep track of acc used for students
33.
Accommodations on the ISAT• List of allowable accommodations for SWD and ELLs(English Language Learners): • http://www.isbe.state.il.us/assessment/pdfs/ISAT_accommodations _grid.pdf
34.
Accommodations on the PSAE (ACT)• The ACT breaks accommodations into two types: • ACT accommodations – if a student uses these accommodations their scores can still be reported to colleges • State allowed accommodations – these are more intensive acc that a student may use if the test is going to be used for state accountability purposes (like the PSAE in Illinois). If a student uses state allowed acc their ACT scores cannot be reported to colleges.• Differences between ACT and State Allowed acc: • http://www.isbe.state.il.us/assessment/pdfs/2009/ACT_Apvd_State_Al wd.pdf• More detailed info: look at p. 1 for the difference between ACT and State allowed accommodations, p. 2 for examples, and p. 6 for specific disabilities and accommodations allowed • http://www.isbe.state.il.us/assessment/pdfs/2013/psae/accom- procedures13.pdf
35.
Illinois Alternate Assessment• For students with most severe cognitive disabilities for whom taking the ISAT or PSAE even with accommodations is not appropriate• Determined by the IEP team • http://www.isbe.state.il.us/assessment/pdfs/2012/iaa/partic_guidelin es_pres.pdf• Administered in a one-on-one setting. Student given prompts (questions) and evaluated on the amount of support they need to respond to the prompt • http://www.isbe.state.il.us/assessment/pdfs/2013/iaa/test_admin_pr es.pdf
36.
For more information• ISBE info on standardized tests: http://www.isbe.state.il.us/assessment/• Information about tests for parents (good review of info): http://www.pbs.org/wgbh/pages/frontline/shows/schools/et c/guide.html
37.
Next steps…Module 6 activity:• Download module 6 activity from the website. Choose the activity that corresponds with your teaching assignment (primary or secondary). You do not need to do both!• Complete module 6 activity by typing directly on the document. Save and upload to your google folder by 11:59pm on 3/24.Course feedback:• Click on the link for the google form to give feedback on the course
38.
Finishing the course• This is the last module!• Remaining assignments: • IEP Assessment Report Review – due by 11:59pm on 3/24 • Adapting assessments assignment – Due by 11:59pm on 3/29!