SlideShare a Scribd company logo

Technical report page 558 560

•
•
D
davewoods
1 of 651
Download to read offline
California Department of Education Statewide Assessment Division California Standards Tests Technical Report Spring 2009 Administration Submitted March 1, 2010 Educational Testing Service Contract No. 5417
STAR Program Table of Contents (Summarized) Chapter 1: Introduction......................................................................................................................................... 1 Background ........................................................................................................................................................................... 1 Test Purpose.......................................................................................................................................................................... 1 Test Content .......................................................................................................................................................................... 2 Intended Population .............................................................................................................................................................. 2 Intended Use and Purpose of Test Scores.......................................................................................................................... 3 Testing Window ..................................................................................................................................................................... 3 Significant STAR Developments in 2009 ............................................................................................................................. 4 Limitations of the Assessment ............................................................................................................................................ 4 Groups and Organizations Involved with the STAR Program ........................................................................................... 5 Overview of the Technical Report ........................................................................................................................................ 5 References ............................................................................................................................................................................. 7 Chapter 2: An Overview of CST Processes ........................................................................................................ 8 Item Development ................................................................................................................................................................. 8 Test Variations, Accommodations, and Modifications .................................................................................................... 11 Scores .................................................................................................................................................................................. 12 References ........................................................................................................................................................................... 20 Chapter 3: Item Development........................................................................................................................... 196 Rules for Item Development ............................................................................................................................................. 196 Selection of Item Writers .................................................................................................................................................. 198 Item Review Process ......................................................................................................................................................... 199 Field Testing ...................................................................................................................................................................... 203 Data Review Meetings ....................................................................................................................................................... 205 Item Banking ...................................................................................................................................................................... 206 References ......................................................................................................................................................................... 207 Chapter 4: Test Assembly ................................................................................................................................ 208 Test Length ........................................................................................................................................................................ 208 Rules for Item Selection.................................................................................................................................................... 208 Reference ........................................................................................................................................................................... 213 Appendix 4.B—Cluster Targets ........................................................................................................................................ 223 Chapter 5: Test Administration ........................................................................................................................ 253 Test Security and Confidentiality ..................................................................................................................................... 253 Procedures to Maintain Standardization ......................................................................................................................... 258 Accommodations for Students with Disabilities ............................................................................................................ 260 Demographic Data Corrections ........................................................................................................................................ 261 Testing Irregularities ......................................................................................................................................................... 261 Test Administration Incidents .......................................................................................................................................... 261 References ......................................................................................................................................................................... 262 Chapter 6: Performance Standards ................................................................................................................. 263 Background ....................................................................................................................................................................... 263 Standard Setting Procedure ............................................................................................................................................. 263 Results ............................................................................................................................................................................... 265 References ......................................................................................................................................................................... 268 Chapter 7: Scoring and Reporting ................................................................................................................... 269 Procedures for Maintaining and Retrieving Individual Scores ...................................................................................... 269 Types of Scores and Subscores ...................................................................................................................................... 270 Score Verification Procedures ......................................................................................................................................... 271 Overview of Score Aggregation Procedures .................................................................................................................. 274 Reports to be Produced and Scores for Each Report .................................................................................................... 278 Criteria for Interpreting Test Scores ................................................................................................................................ 280 Criteria for Interpreting Score Reports ............................................................................................................................ 280 Reference ........................................................................................................................................................................... 281 Chapter 8: Analyses .......................................................................................................................................... 372 Samples Used for the Analyses ....................................................................................................................................... 372 Classical Item Analyses .................................................................................................................................................... 373 Reliability Analyses ........................................................................................................................................................... 375 Decision Classification Analyses .................................................................................................................................... 379 Validity Evidence ............................................................................................................................................................... 382 Page i
STAR Program IRT Analyses ...................................................................................................................................................................... 394 DIF Analyses ...................................................................................................................................................................... 399 References ......................................................................................................................................................................... 401 Chapter 9: Quality Control Procedures ...........................................................................................................609 Quality Control of Item Development .............................................................................................................................. 609 Quality Control of the Item Bank ...................................................................................................................................... 611 Quality Control of Test Materials ..................................................................................................................................... 611 Quality Control of Scanning ............................................................................................................................................. 612 Quality Control of Image Editing ...................................................................................................................................... 613 Quality Control of Answer Document Processing and Scoring .................................................................................... 613 Quality Control of Psychometric Processes ................................................................................................................... 615 Quality Control of Reporting ............................................................................................................................................ 617 Chapter 10: Historical Comparisons ............................................................................................................... 618 Examinee Performance ..................................................................................................................................................... 619 Test Characteristics .......................................................................................................................................................... 620 References ......................................................................................................................................................................... 621 Page ii
STAR Program Table of Contents (Full) Acronyms and Initialisms Used in the CST Technical Report.............................................................................. xiv Chapter 1: Introduction......................................................................................................................................... 1 Background ........................................................................................................................................................................... 1 Test Purpose.......................................................................................................................................................................... 1 Test Content .......................................................................................................................................................................... 2 Intended Population .............................................................................................................................................................. 2 Intended Use and Purpose of Test Scores.......................................................................................................................... 3 Testing Window ..................................................................................................................................................................... 3 Significant STAR Developments in 2009 ............................................................................................................................. 4 Elimination of California Achievement Tests, Sixth Edition Survey (CAT/6 Survey) .............................................................. 4 Release of CST Exemplars .................................................................................................................................................... 4 Limitations of the Assessment ............................................................................................................................................ 4 Score Interpretation ............................................................................................................................................................... 4 Out-of-Level Testing .............................................................................................................................................................. 4 Score Comparison ................................................................................................................................................................. 4 Groups and Organizations Involved with the STAR Program ........................................................................................... 5 State Board of Education (SBE)............................................................................................................................................. 5 California Department of Education (CDE) ............................................................................................................................ 5 Contractors ............................................................................................................................................................................ 5 Overview of the Technical Report ........................................................................................................................................ 5 References ............................................................................................................................................................................. 7 Chapter 2: An Overview of CST Processes ........................................................................................................ 8 Item Development ................................................................................................................................................................. 8 Item Formats .......................................................................................................................................................................... 8 Item Development Specifications ........................................................................................................................................... 8 Item Banking .......................................................................................................................................................................... 8 Item Refresh Rate and Released Test Questions (RTQ) ....................................................................................................... 9 Test Length ............................................................................................................................................................................ 9 Test Blueprint ......................................................................................................................................................................... 9 Content Rules and Item Selection.......................................................................................................................................... 9 Psychometric Criteria ........................................................................................................................................................... 10 Test Security and Confidentiality.......................................................................................................................................... 10 Procedures to Maintain Standardization .............................................................................................................................. 11 Test Variations, Accommodations, and Modifications .................................................................................................... 11 Test Variations, Accommodation and Modification Summaries ........................................................................................... 12 Scores .................................................................................................................................................................................. 12 Individual Scores.................................................................................................................................................................. 13 Group Scores ....................................................................................................................................................................... 13 Calibration ............................................................................................................................................................................ 14 Scaling ................................................................................................................................................................................. 15 True Score Equating ............................................................................................................................................................ 15 Equating Samples ................................................................................................................................................................ 17 Equating the Braille Versions of the CSTs ........................................................................................................................... 18 Scaling the CSTs for Integrated/Coordinated Science ......................................................................................................... 18 References ........................................................................................................................................................................... 20 Appendix 2.A—CST Item and Estimated Time Chart ....................................................................................................... 21 Appendix 2.B—Reporting Clusters .................................................................................................................................... 22 English–Language Arts ........................................................................................................................................................ 22 History–Social Science ........................................................................................................................................................ 24 Mathematics......................................................................................................................................................................... 24 Science ................................................................................................................................................................................ 27 Appendix 2.C—2009 Test Variations, Accommodations, and Modifications ................................................................. 32 Appendix 2.D—Special Services Summary Tables .......................................................................................................... 35 Chapter 3: Item Development........................................................................................................................... 196 Rules for Item Development ............................................................................................................................................. 196 Item Development Specifications ....................................................................................................................................... 196 Expected Item Ratio ........................................................................................................................................................... 197 Selection of Item Writers .................................................................................................................................................. 198 Criteria for Selecting Item Writers ...................................................................................................................................... 198 Item Writer Training ........................................................................................................................................................... 198 Page iii
STAR Program Item Review Process ......................................................................................................................................................... 199 Contractor Review ............................................................................................................................................................. 199 Content Expert Reviews .................................................................................................................................................... 200 Statewide Pupil Assessment Review (SPAR) Panel .......................................................................................................... 203 Field Testing ...................................................................................................................................................................... 203 Stand-Alone Field Testing .................................................................................................................................................. 203 Embedded Field-Test Items ............................................................................................................................................... 204 Data Review Meetings ....................................................................................................................................................... 205 Item Banking ...................................................................................................................................................................... 206 References ......................................................................................................................................................................... 207 Chapter 4: Test Assembly ................................................................................................................................ 208 Test Length ........................................................................................................................................................................ 208 Rules for Item Selection.................................................................................................................................................... 208 Test Blueprint ..................................................................................................................................................................... 208 Content Rules and Item Selection...................................................................................................................................... 208 Psychometric Criteria ......................................................................................................................................................... 209 Projected Psychometric Properties of the Assembled Tests .............................................................................................. 211 Rules for Item Sequence and Layout ................................................................................................................................. 212 Reference ........................................................................................................................................................................... 213 Appendix 4.A—Technical Characteristics....................................................................................................................... 214 Appendix 4.B—Cluster Targets ........................................................................................................................................ 223 Chapter 5: Test Administration ........................................................................................................................ 253 Test Security and Confidentiality ..................................................................................................................................... 253 ETS’s Office of Testing Integrity (OTI) ............................................................................................................................... 253 Test Development .............................................................................................................................................................. 253 Item Review ....................................................................................................................................................................... 253 Item Bank ........................................................................................................................................................................... 254 Transfer of Forms and Items to the CDE ........................................................................................................................... 254 Firewall .............................................................................................................................................................................. 254 Printing............................................................................................................................................................................... 255 Test Administration ............................................................................................................................................................ 255 Test Delivery ...................................................................................................................................................................... 255 Processing and Scoring ..................................................................................................................................................... 256 Data Management ............................................................................................................................................................. 256 Transfer of Scores via Secure Data Exchange .................................................................................................................. 257 Statistical Analysis ............................................................................................................................................................. 257 Reporting and Posting Results........................................................................................................................................... 257 Student Confidentiality ....................................................................................................................................................... 257 Test Results ....................................................................................................................................................................... 257 Procedures to Maintain Standardization ......................................................................................................................... 258 Test Administrators ............................................................................................................................................................ 258 Directions for Administration (DFA) .................................................................................................................................... 259 District and Test Site Coordinator Manual ......................................................................................................................... 259 STAR Management System Manuals ................................................................................................................................ 259 Test Booklets ..................................................................................................................................................................... 260 Accommodations for Students with Disabilities ............................................................................................................ 260 Identification ....................................................................................................................................................................... 260 Scoring............................................................................................................................................................................... 261 Demographic Data Corrections ........................................................................................................................................ 261 Testing Irregularities ......................................................................................................................................................... 261 Test Administration Incidents .......................................................................................................................................... 261 References ......................................................................................................................................................................... 262 Chapter 6: Performance Standards ................................................................................................................. 263 Background ....................................................................................................................................................................... 263 Standard Setting Procedure ............................................................................................................................................. 263 Standard Setting Methodologies ........................................................................................................................................ 264 Results ............................................................................................................................................................................... 265 References ......................................................................................................................................................................... 268 Chapter 7: Scoring and Reporting ................................................................................................................... 269 Procedures for Maintaining and Retrieving Individual Scores ...................................................................................... 269 Scoring and Reporting Specifications ................................................................................................................................ 269 Scanning and Scoring ........................................................................................................................................................ 270 Types of Scores and Subscores ...................................................................................................................................... 270 Raw Score ......................................................................................................................................................................... 270 Page iv
STAR Program Subscore............................................................................................................................................................................ 270 Scale Score........................................................................................................................................................................ 270 Performance Levels ........................................................................................................................................................... 270 Writing Score ..................................................................................................................................................................... 271 Score Verification Procedures ......................................................................................................................................... 271 Scoring Key Verification Process ....................................................................................................................................... 271 Monitoring and Quality Control of Writing Scoring ............................................................................................................. 272 Score Verification Process ................................................................................................................................................. 274 Overview of Score Aggregation Procedures .................................................................................................................. 274 Individual Scores................................................................................................................................................................ 274 Reports to be Produced and Scores for Each Report .................................................................................................... 278 Types of Score Reports ..................................................................................................................................................... 278 Score Report Contents ...................................................................................................................................................... 278 Score Report Applications.................................................................................................................................................. 279 Criteria for Interpreting Test Scores ................................................................................................................................ 280 Criteria for Interpreting Score Reports ............................................................................................................................ 280 Reference ........................................................................................................................................................................... 281 Appendix 7.A—ELA for Writing (Grades Four and Seven) Rubrics .............................................................................. 282 Grade Four Scoring Rubric ................................................................................................................................................ 282 Grade Seven Scoring Rubric ............................................................................................................................................. 285 Appendix 7.B—Scale Score Distributions....................................................................................................................... 288 Appendix 7.C—Demographic Summaries ....................................................................................................................... 292 Appendix 7.D—Types of Score Reports .......................................................................................................................... 368 Chapter 8: Analyses .......................................................................................................................................... 372 Samples Used for the Analyses ....................................................................................................................................... 372 Classical Item Analyses .................................................................................................................................................... 373 Multiple-Choice Items ........................................................................................................................................................ 373 Writing Tasks ..................................................................................................................................................................... 374 Reliability Analyses ........................................................................................................................................................... 375 Intercorrelations, Reliabilities, and SEMs for Reporting Clusters ....................................................................................... 377 Subgroup Reliabilities and SEMs ....................................................................................................................................... 377 Conditional Standard Errors of Measurement .................................................................................................................... 377 Decision Classification Analyses .................................................................................................................................... 379 Writing Score Reliability ..................................................................................................................................................... 380 Prompt and Rater Agreement Summary ............................................................................................................................ 381 Validity Evidence ............................................................................................................................................................... 382 Purpose of the CST ........................................................................................................................................................... 382 The Constructs to Be Measured ........................................................................................................................................ 382 The Scores Generated and the Interpretations and Uses of these Scores ........................................................................ 383 Intended Test Population(s) ............................................................................................................................................... 383 Validity Evidence Collected ................................................................................................................................................ 384 Evidence Based on Response Processes ......................................................................................................................... 391 Evidence Based on Internal Structure ................................................................................................................................ 392 Evidence Based on Consequences of Testing................................................................................................................... 394 IRT Analyses ...................................................................................................................................................................... 394 IRT Model-Data Fit Analyses ............................................................................................................................................. 394 Model Fit Assessment Results ........................................................................................................................................... 396 Evaluation of Scaling ......................................................................................................................................................... 396 Summaries of Scaled IRT b-values.................................................................................................................................... 398 Post-Equating Results ....................................................................................................................................................... 398 DIF Analyses ...................................................................................................................................................................... 399 References ......................................................................................................................................................................... 401 Appendix 8.A—Classical Analyses .................................................................................................................................. 403 Appendix 8.B—Reliability Analyses ................................................................................................................................ 415 Appendix 8.C—Validity Analyses .................................................................................................................................... 497 Appendix 8.D—IRT Analyses ........................................................................................................................................... 541 Scaling and Post-Equating Results .................................................................................................................................... 557 Appendix 8.E—DIF Analyses............................................................................................................................................ 579 Chapter 9: Quality Control Procedures ...........................................................................................................609 Quality Control of Item Development .............................................................................................................................. 609 Item Specifications ............................................................................................................................................................. 609 Item Writers........................................................................................................................................................................ 609 Internal Contractor Reviews ............................................................................................................................................... 609 Assessment Review Panel (ARP) Review ......................................................................................................................... 610 Statewide Pupil Assessment Review (SPAR) Panel Review ............................................................................................. 610 Page v
STAR Program Data Review of Field Tested Items .................................................................................................................................... 610 Quality Control of the Item Bank ...................................................................................................................................... 611 Quality Control of Test Materials ..................................................................................................................................... 611 Collecting Test Materials .................................................................................................................................................... 611 Processing Test Materials .................................................................................................................................................. 612 Quality Control of Scanning ............................................................................................................................................. 612 Post-scanning Edits ........................................................................................................................................................... 612 Quality Control of Image Editing ...................................................................................................................................... 613 Quality Control of Answer Document Processing and Scoring .................................................................................... 613 Accountability of Answer Documents ................................................................................................................................. 613 Processing of Answer Documents ..................................................................................................................................... 613 Scoring and Reporting Specifications ................................................................................................................................ 614 Matching Information on CST Answer Documents ............................................................................................................ 614 Matching Multiple-choice and Writing Scores for ELA Grades Four and Seven ................................................................ 614 Storing Answer Documents................................................................................................................................................ 614 Quality Control of Psychometric Processes ................................................................................................................... 615 Score Key Verification Procedures .................................................................................................................................... 615 Quality Control of Item Analyses, DIF and Equating Process ............................................................................................ 615 Score Verification Process ................................................................................................................................................. 616 Offloads to Test Development............................................................................................................................................ 616 Quality Control of Reporting ............................................................................................................................................ 617 Excluding Student Scores from Summary Reports ............................................................................................................ 617 Chapter 10: Historical Comparisons ............................................................................................................... 618 Examinee Performance ..................................................................................................................................................... 619 Test Characteristics .......................................................................................................................................................... 620 References ......................................................................................................................................................................... 621 Appendix 10.A—Historical Comparisons Tables ........................................................................................................... 622 Appendix 10.B—Historical Comparisons Tables ........................................................................................................... 632 Tables Table 2.1 Subgroup Definitions ........................................................................................................................................... 13 Table 2.2 Scale Scores Ranges for Performance Levels .................................................................................................... 16 Table 2.3 Sample Selected to Establish Reference Form ................................................................................................... 17 Table 2.4 Proficiency-Level Cut Points for Science End-of-Course CSTs .......................................................................... 19 Table 2.5 Computation of Proficiency-Level Cut Points on Integrated/Coordinated Science 1 ........................................... 19 Table 2.C.1 Matrix of Test Variations, Accommodations, and Modifications ........................................................................ 32 Table 2.C.2 Matrix of Test Variations for English Learners .................................................................................................. 34 Table 2.D. 1 Special Services Summary for ELA, Grades Two and Three ......................................................................... 35 Table 2.D.2 Special Services Summary for ELA, Grade Four............................................................................................. 40 Table 2.D.3 Special Services Summary for ELA, Grades Five and Six .............................................................................. 45 Table 2.D.4 Special Services Summary for ELA, Grade Seven .......................................................................................... 49 Table 2.D.5 Special Services Summary for ELA, Grades Eight and Nine ........................................................................... 54 Table 2.D.6 Special Services Summary for ELA, Grades Ten and Eleven ......................................................................... 58 Table 2.D.7 Special Services Summary for History–Social Science, Grade Eight .............................................................. 62 Table 2.D.8 Special Services Summary for World History .................................................................................................. 67 Table 2.D.9 Special Services Summary for History–Social Science, Grade Eleven ........................................................... 72 Table 2.D.10 Special Services Summary for Mathematics, Grades Two and Three........................................................... 77 Table 2.D.11 Special Services Summary for Mathematics, Grades Four and Five............................................................. 82 Table 2.D.12 Special Services Summary for Mathematics, Grades Six and Seven............................................................ 87 Table 2.D.13 Special Services Summary for General Mathematics.................................................................................... 92 Table 2.D.14 Special Services Summary for Algebra I ....................................................................................................... 97 Table 2.D.15 Special Services Summary for Geometry .................................................................................................... 102 Table 2.D.16 Special Services Summary for Algebra II .................................................................................................... 107 Table 2.D.17 Special Services Summary for Summative High School Mathematics ........................................................ 112 Table 2.D.18 Special Services Summary for Integrated Mathematics 1 ........................................................................... 118 Table 2.D.19 Special Services Summary for Integrated Mathematics 2 ........................................................................... 123 Table 2.D.20 Special Services Summary for Integrated Mathematics 3 ........................................................................... 128 Table 2.D.21 Special Services Summary for Science, Grade Five ................................................................................... 133 Table 2.D.22 Special Services Summary for Science, Grade Eight .................................................................................. 138 Table 2.D.23 Special Services Summary for Grade Ten Life Science .............................................................................. 143 Table 2.D.24 Special Services Summary for Biology ........................................................................................................ 148 Table 2.D.25 Special Services Summary for Chemistry ................................................................................................... 154 Table 2.D.26 Special Services Summary for Earth Science ............................................................................................. 160 Table 2.D.27 Special Services Summary for Physics ....................................................................................................... 166 Table 2.D.28 Special Services Summary for Integrated/Coordinated Science 1 .............................................................. 172 Page vi
STAR Program Table 2.D.29 Special Services Summary for Integrated/Coordinated Science 2 .............................................................. 178 Table 2.D.37 Special Services Summary for Integrated/Coordinated Science 3 ............................................................... 184 Table 2.D.30 Special Services Summary for Integrated/Coordinated Science 4 .............................................................. 190 Table 3.1 Field-test Percentages for the CSTs ................................................................................................................. 198 Table 3.2 CST ARP Member Qualifications, by subject and total ..................................................................................... 201 Table 3.3 Summary of Items and Forms Presented in 2009 CSTs ................................................................................... 204 Table 4.1 Target Statistical Specifications for the CSTs ................................................................................................... 210 Table 4.A.1 Summary of 2009 CST Projected Technical Characteristics ......................................................................... 214 Table 4.A.2 Summary of 2009 CST Projected Statistical Attributes .................................................................................. 215 Table 6.1 Scale Scores Ranges for Performance Levels .................................................................................................. 265 Table 6.2 Percentage of Examinees in Each Performance Level ..................................................................................... 266 Table 7.1 Mean and Standard Deviation of Raw and Scale Scores for CSTs .................................................................. 275 Table 7.2 Percentage of Examinees in Proficiency Levels for CSTs ................................................................................ 276 Table 7.3 Subgroup Definitions ......................................................................................................................................... 277 Table 7.B.1 Distribution of CST Scale Scores for ELA ...................................................................................................... 288 Table 7.B.2 Distribution of CST Scale Scores for History–Social Science ........................................................................ 288 Table 7.B.3 Distribution of CST Scale Scores for Mathematics, Grades Two through Seven .......................................... 289 Table 7.B.4 Distribution of Scale Scores for Mathematics End-of-Course CSTs .............................................................. 289 Table 7.B.5 Distribution of CST Scale Scores for Science ................................................................................................ 290 Table 7.B.6 Distribution of CST Scale Scores for a Grade-Specific Population ................................................................ 291 Table 7.C.1 Demographic Summary for ELA, Grade Two ................................................................................................ 292 Table 7.C.2 Demographic Summary for ELA, Grade Three .............................................................................................. 294 Table 7.C.3 Demographic Summary for ELA, Grade Four ................................................................................................ 296 Table 7.C.4 Demographic Summary for ELA, Grade Five ................................................................................................ 298 Table 7.C.5 Demographic Summary for ELA, Grade Six .................................................................................................. 300 Table 7.C.6 Demographic Summary for ELA, Grade Seven ............................................................................................. 302 Table 7.C.7 Demographic Summary for ELA, Grade Eight ............................................................................................... 304 Table 7.C.8 Demographic Summary for ELA, Grade Nine ................................................................................................ 306 Table 7.C.9 Demographic Summary for ELA, Grade Ten ................................................................................................. 308 Table 7.C.10 Demographic Summary for ELA, Grade Eleven .......................................................................................... 310 Table 7.C.11 Demographic Summary for History–Social Science, Grade Eight ............................................................... 312 Table 7.C.12 Demographic Summary for World History ................................................................................................... 314 Table 7.C.13 Demographic Summary for History–Social Science, Grade Eleven ............................................................ 316 Table 7.C.14 Demographic Summary for Mathematics, Grade Two ................................................................................. 318 Table 7.C.15 Demographic Summary for Mathematics, Grade Three .............................................................................. 320 Table 7.C.16 Demographic Summary for Mathematics, Grade Four ................................................................................ 322 Table 7.C.17 Demographic Summary for Mathematics, Grade Five ................................................................................. 324 Table 7.C.18 Demographic Summary for Mathematics, Grade Six................................................................................... 326 Table 7.C.19 Demographic Summary for Mathematics, Grade Seven ............................................................................. 328 Table 7.C.20 Demographic Summary for General Mathematics ....................................................................................... 330 Table 7.C.21 Demographic Summary for Algebra I .......................................................................................................... 332 Table 7.C.22 Demographic Summary for Geometry ......................................................................................................... 334 Table 7.C.23 Demographic Summary for Algebra II ......................................................................................................... 336 Table 7.C.24 Demographic Summary for Summative High School Mathematics ............................................................. 338 Table 7.C.25 Demographic Summary for Integrated Mathematics 1 ................................................................................ 340 Table 7.C.26 Demographic Summary for Integrated Mathematics 2 ................................................................................ 342 Table 7.C.27 Demographic Summary for Integrated Mathematics 3 ................................................................................ 344 Table 7.C.28 Demographic Summary for Science, Grade Five ........................................................................................ 346 Table 7.C.29 Demographic Summary for Science, Grade Eight ....................................................................................... 348 Table 7.C.30 Demographic Summary for Grade Ten Life Science ................................................................................... 350 Table 7.C.31 Demographic Summary for Biology ............................................................................................................. 352 Table 7.C.32 Demographic Summary for Chemistry ......................................................................................................... 354 Table 7.C.33 Demographic Summary for Earth Science .................................................................................................. 356 Table 7.C.34 Demographic Summary for Physics ............................................................................................................ 358 Table 7.C.35 Demographic Summary for Integrated/Coordinated Science 1 ................................................................... 360 Table 7.C.36 Demographic Summary for Integrated/Coordinated Science 2 ................................................................... 362 Table 7.C.37 Demographic Summary for Integrated/Coordinated Science 3 ................................................................... 364 Table 7.C.38 Demographic Summary for Integrated/Coordinated Science 4 ................................................................... 366 Table 7.D.1 Score Reports Reflecting CST Results .......................................................................................................... 368 Table 8.1 Mean and Median Proportion Correct and Point-Biserial .................................................................................. 374 Table 8.2 Reliabilities and Standard Errors of Measurement for the CSTs ....................................................................... 376 Table 8.3 Scale Score CSEM at Cut Points ...................................................................................................................... 378 Table 8.4 2009 CST Content Area Correlations (All Valid Scores) ................................................................................... 388 Table 8.5 Evaluation of Common Items between New and Reference Test Forms .......................................................... 397 Table 8.A.1 Item-by-item p-value and Point-Biserial for ELA ............................................................................................ 403 Table 8.A.2 Item-by-item p-value and Point-Biserial for History–Social Science .............................................................. 405 Page vii
STAR Program Table 8.A.3 Item-by-item p-value and Point-Biserial for Mathematics ............................................................................... 406 Table 8.A.4 Item-by-item p-value and Point-Biserial for Mathematics [continued] ............................................................ 407 Table 8.A.5 Item-by-item p-value and Point-Biserial for Science ...................................................................................... 408 Table 8.A.6 Item-by-item p-value and Point-Biserial for Science [continued] .................................................................... 409 Table 8.A.7 Item-by-item p-value and Point-Biserial for Grade-Specific CSTs ................................................................. 411 Table 8.A.8 Distribution of Essay Scores for ELA (Grade Four)—Overall and by Subgroup (all %) ................................. 412 Table 8.A.9 Distribution of Essay Scores for ELA (Grade Seven)—Overall and by Subgroup (all %) .............................. 412 Table 8.A.10 Mean Scores for ELA (Grade Four Essay)—Overall and by Subgroup ....................................................... 413 Table 8.A.11 Mean Scores for ELA (Grade Seven Essay)—Overall and by Subgroup .................................................... 413 Table 8.A.12 Effect Sizes for ELA (Grades Four and Seven Essay)—by Subgroup ......................................................... 414 Table 8.B.1 Reliabilities and Correlations for ELA ............................................................................................................ 415 Table 8.B.2 Reliabilities and Correlations for History–Social Science ............................................................................... 416 Table 8.B.3 Subscore Reliabilities and Correlations for Mathematics ............................................................................... 417 Table 8.B.4 Subscore Reliabilities and Correlations for Science ...................................................................................... 420 Table 8.B.5 Reliabilities and Correlations for Grade-Specific Tests .................................................................................. 422 Table 8.B.6 Reliabilities and SEMs for the CSTs by Gender ............................................................................................ 423 Table 8.B.7 Reliabilities and SEMs for the CSTs by Economic Status ............................................................................. 424 Table 8.B.8 Reliabilities and SEMs for the CSTs by Special Services .............................................................................. 425 Table 8.B.9 Reliabilities and SEMs for the CSTs by English Fluency ............................................................................... 426 Table 8.B.10 Reliabilities and SEMs for the CSTs by English Fluency ............................................................................. 427 Table 8.B.11 Reliabilities and SEMs for the CSTs by Primary Ethnicity ........................................................................... 428 Table 8.B.12 Reliabilities and SEMs for the CSTs by Primary Ethnicity ........................................................................... 429 Table 8.B.13 Reliabilities and SEMs for the CSTs by Primary Ethnicity ........................................................................... 430 Table 8.B.14 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Not Economically Disadvantaged ................ 431 Table 8.B.15 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Not Economically Disadvantaged ................ 432 Table 8.B.16 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Not Economically Disadvantaged ................ 433 Table 8.B.17 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Economically Disadvantaged ....................... 434 Table 8.B.18 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Economically Disadvantaged ....................... 435 Table 8.B.19 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Economically Disadvantaged ....................... 436 Table 8.B.20 Subscore Reliabilities and SEM for ELA by Gender/Economic Status ........................................................ 437 Table 8.B.21 Subscore Reliabilities and SEM for History–Social Science by Gender/Economic Status ........................... 438 Table 8.B.22 Subscore Reliabilities and SEM for Mathematics by Gender/Economic Status ........................................... 439 Table 8.B.23 Subscore Reliabilities and SEM for Science by Gender/Economic Status .................................................. 441 Table 8.B.24 Subscore Reliabilities and SEM for Grade-Specific CSTs by Gender/Economic Status.............................. 443 Table 8.B.25 Subscore Reliabilities and SEM for ELA by Special Services/English Fluency ........................................... 444 Table 8.B.26 Subscore Reliabilities and SEM for History–Social Science by Special Services/English Fluency .............. 446 Table 8.B.27 Subscore Reliabilities and SEM for Mathematics by Special Services/English Fluency .............................. 447 Table 8.B.28 Subscore Reliabilities and SEM for Science by Special Services/English Fluency...................................... 450 Table 8.B.29 Subscore Reliabilities and SEM for Grade-Specific CSTs by Special Services/English Fluency ................. 452 Table 8.B.30 Subscore Reliabilities and SEM for ELA by Primary Ethnicity ..................................................................... 453 Table 8.B.31 Subscore Reliabilities and SEM for History–Social Science by Primary Ethnicity........................................ 455 Table 8.B.32 Subscore Reliabilities and SEM for Mathematics by Primary Ethnicity ........................................................ 456 Table 8.B.33 Subscore Reliabilities and SEM for Science by Primary Ethnicity ............................................................... 459 Table 8.B.34 Subscore Reliabilities and SEM for Grade-Specific CSTs by Primary Ethnicity........................................... 461 Table 8.B.35 Subscore Reliabilities and SEM for ELA by Primary Ethnicity-for-Not Economically Disadvantaged .......... 462 Table 8.B.36 Subscore Reliabilities and SEM for History–Social Science by Primary Ethnicity-for-Not Economically Disadvantaged .......................................................................................................................................... 464 Table 8.B.37 Subscore Reliabilities and SEM for Mathematics by Primary Ethnicity-for-Not Economically Disadvantaged ................................................................................................................................................................ 465 Table 8.B.38 Subscore Reliabilities and SEM for Science by Primary Ethnicity-for-Not Economically Disadvantaged ................................................................................................................................................................ 468 Table 8.B.39 Subscore Reliabilities and SEM for Grade-Specific CSTs by Primary Ethnicity-for-Not Economically Disadvantaged ................................................................................................................................................................ 470 Table 8.B.40 Subscore Reliabilities and SEM for ELA by Primary Ethnicity-for-Economically Disadvantaged ................. 471 Table 8.B.41 Subscore Reliabilities and SEM for History–Social Science by Primary Ethnicity-for-Economically Disadvantaged ................................................................................................................................................................ 473 Table 8.B.42 Subscore Reliabilities and SEM for Mathematics by Primary Ethnicity-for-Economically Disadvantaged ................................................................................................................................................................ 474 Table 8.B.43 Subscore Reliabilities and SEM for Science by Primary Ethnicity-for-Economically Disadvantaged ........... 477 Table 8.B.44 Subscore Reliabilities and SEM for Grade-Specific CSTs by Primary Ethnicity-for- Economically Disadvantaged .......................................................................................................................................... 479 Table 8.B.45 Reliability of Classification for ELA, Grade Two ........................................................................................... 480 Table 8.B.46 Reliability of Classification for ELA, Grade Three ........................................................................................ 480 Table 8.B.47 Reliability of Classification for ELA, Grade Four (Reading only) .................................................................. 480 Table 8.B.48 Reliability of Classification for ELA, Grade Four (Reading and Writing) ...................................................... 481 Table 8.B.49 Reliability of Classification for ELA, Grade Five ........................................................................................... 481 Page viii
STAR Program Table 8.B.50 Reliability of Classification for ELA, Grade Six............................................................................................. 481 Table 8.B.51 Reliability of Classification for ELA, Grade Seven (Reading only) ............................................................... 482 Table 8.B.52 Reliability of Classification for ELA, Grade Seven (Reading and Writing) .................................................... 482 Table 8.B.53 Reliability of Classification for ELA, Grade Eight ......................................................................................... 482 Table 8.B.54 Reliability of Classification for ELA, Grade Nine .......................................................................................... 483 Table 8.B.55 Reliability of Classification for ELA, Grade Ten ........................................................................................... 483 Table 8.B.56 Reliability of Classification for ELA, Grade Eleven....................................................................................... 483 Table 8.B.57 Reliability of Classification for History–Social Science, Grade Eight ............................................................ 484 Table 8.B.58 Reliability of Classification for World History ................................................................................................ 484 Table 8.B.59 Reliability of Classification for History–Social Science, Grade Eleven ......................................................... 484 Table 8.B.60 Reliability of Classification for Mathematics, Grade Two ............................................................................. 485 Table 8.B.61 Reliability of Classification for Mathematics, Grade Three ........................................................................... 485 Table 8.B.62 Reliability of Classification for Mathematics, Grade Four ............................................................................. 485 Table 8.B.63 Reliability of Classification for Mathematics, Grade Five ............................................................................. 486 Table 8.B.64 Reliability of Classification for Mathematics, Grade Six ............................................................................... 486 Table 8.B.65 Reliability of Classification for Mathematics, Grade Seven .......................................................................... 486 Table 8.B.66 Reliability of Classification for General Mathematics ................................................................................... 487 Table 8.B.67 Reliability of Classification for Algebra I ....................................................................................................... 487 Table 8.B.68 Reliability of Classification for Geometry ..................................................................................................... 487 Table 8.B.69 Reliability of Classification for Algebra II ...................................................................................................... 488 Table 8.B.70 Reliability of Classification for Summative High School Mathematics .......................................................... 488 Table 8.B.71 Reliability of Classification for Integrated Mathematics 1 ............................................................................. 488 Table 8.B.72 Reliability of Classification for Integrated Mathematics 2 ............................................................................. 489 Table 8.B.73 Reliability of Classification for Integrated Mathematics 3 ............................................................................. 489 Table 8.B.74 Reliability of Classification for Science, Grade Five ..................................................................................... 489 Table 8.B.75 Reliability of Classification for Science, Grade Eight ................................................................................... 490 Table 8.B.76 Reliability of Classification for Grade 10 Life Science .................................................................................. 490 Table 8.B.77 Reliability of Classification for Biology ......................................................................................................... 490 Table 8.B.78 Reliability of Classification for Chemistry ..................................................................................................... 491 Table 8.B.79 Reliability of Classification for Earth Science ............................................................................................... 491 Table 8.B.80 Reliability of Classification for Physics ......................................................................................................... 491 Table 8.B.81 Reliability of Classification for Integrated/Coordinated Science 1 ................................................................ 492 Table 8.B.82 Reliability of Classification for Integrated/Coordinated Science 2 ................................................................ 492 Table 8.B.83 Reliability of Classification for Integrated/Coordinated Science 3 ................................................................ 492 Table 8.B.84 Reliability of Classification for Integrated/Coordinated Science 4 ................................................................ 493 Table 8.B.85 Reliability of Classification for Algebra I – Grade 8 ...................................................................................... 493 Table 8.B.86 Reliability of Classification for Geometry – Grade 9..................................................................................... 493 Table 8.B.87 Reliability of Classification for Algebra II – Grade 10 ................................................................................... 494 Table 8.B.88 Reliability of Classification for Summative High School Mathematics – Grade 11 ....................................... 494 Table 8.B.89 Reliability of Classification for World History-Grade 10 ................................................................................ 494 Table 8.B.90 Inter-Rater Agreement Analyses for ELA (Grade Four) ............................................................................... 495 Table 8.B.91 Inter-Rater Agreement Analyses for (ELA Grade Seven) ............................................................................ 495 Table 8.B.92 Descriptive Statistics for the Ratings by the Two Raters ............................................................................. 495 Table 8.B.93 Generalizability Analyses for Grade Four Essay—[(Person: Essay) x Rater] .............................................. 496 Table 8.B.94 Generalizability Analyses for Grade Seven Essay—[(Person: Essay) x Rater] ........................................... 496 Table 8.C.1 2009 CST Content Area Correlations (Male) ................................................................................................. 497 Table 8.C.2 2009 CST Content Area Correlations (Female) ............................................................................................. 499 Table 8.C.3 2009 CST Content Area Correlations (Gender unknown) ............................................................................. 501 Table 8.C.4 2009 CST Content Area Correlations (American Indian) ............................................................................... 503 Table 8.C.5 2009 CST Content Area Correlations (Asian American) ................................................................................ 505 Table 8.C.6 2009 CST Content Area Correlations (Pacific Islander) ................................................................................ 507 Table 8.C.7 2009 CST Content Area Correlations (Filipino) ............................................................................................. 509 Table 8.C.8 2009 CST Content Area Correlations (Hispanic) ........................................................................................... 511 Table 8.C.9 2009 CST Content Area Correlations (African American) ............................................................................. 513 Table 8.C.10 2009 CST Content Area Correlations (White).............................................................................................. 515 Table 8.C.11 2009 CST Content Area Correlations (Ethnicity unknown) .......................................................................... 517 Table 8.C.12 2009 CST Content Area Correlations (English Only) ................................................................................... 519 Table 8.C.13 2009 CST Content Area Correlations (Initially Fluent English Proficient) .................................................... 521 Table 8.C.14 2009 CST Content Area Correlations (English Learner) .............................................................................. 523 Table 8.C.15 2009 CST Content Area Correlations (Reclassified Fluent English Proficient) ............................................ 525 Table 8.C.16 2009 CST Content Area Correlations (English Fluency unknown) .............................................................. 527 Table 8.C.17 2009 CST Content Area Correlations (Not Economically Disadvantaged) .................................................. 529 Table 8.C.18 2009 CST Content Area Correlations (Economically Disadvantaged) ......................................................... 531 Table 8.C.19 2009 CST Content Area Correlations (Unknown Economic Status) ............................................................ 533 Table 8.C.20 2009 CST Content Area Correlations (No Special Education Services) ...................................................... 535 Table 8.C.21 2009 CST Content Area Correlations (Special Education Services) ........................................................... 537 Page ix
STAR Program Table 8.C.22 2009 CST Content Area Correlations (Special Education unknown) ........................................................... 539 Table 8.D.1 IRT Model Data Fit Distribution for ELA ......................................................................................................... 541 Table 8.D.2 IRT Model Data Fit Distribution for History–Social Science ........................................................................... 541 Table 8.D.3 IRT Model Data Fit Distribution for Mathematics, Grades Two through Seven ............................................. 541 Table 8.D.4 IRT Model Data Fit Distribution for Mathematics End-of-Course Tests ......................................................... 541 Table 8.D.5 IRT Model Data Fit Distribution for Science ................................................................................................... 542 Table 8.D.6 IRT Model Data Fit Distribution for ELA (field-test items) .............................................................................. 542 Table 8.D.7 IRT Model Data Fit Distribution for History–Social Science (field-test items) ................................................ 542 Table 8.D.8 IRT Model Data Fit Distribution for Mathematics, Grades Two through Seven (field-test items) ................... 542 Table 8.D.9 IRT Model Data Fit Distribution for Mathematics End-of-Course Tests (field-test items) ............................... 543 Table 8.D.10 IRT Model Data Fit Distribution for Science (field-test items) ...................................................................... 543 Table 8.D.11 IRT b-values for ELA, Grade Two................................................................................................................ 543 Table 8.D.12 IRT b-values for ELA, Grade Three ............................................................................................................. 543 Table 8.D.13 IRT b-values for ELA, Grade Four ............................................................................................................... 544 Table 8.D.14 IRT b-values for ELA, Grade Five................................................................................................................ 544 Table 8.D.15 IRT b-values for ELA, Grade Six ................................................................................................................. 544 Table 8.D.16 IRT b-values for ELA, Grade Seven ............................................................................................................ 544 Table 8.D.17 IRT b-values for ELA, Grade Eight .............................................................................................................. 545 Table 8.D.18 IRT b-values for ELA, Grade Nine ............................................................................................................... 545 Table 8.D.19 IRT b-values for ELA, Grade Ten ................................................................................................................ 545 Table 8.D.20 IRT b-values for ELA, Grade Eleven ........................................................................................................... 545 Table 8.D.21 IRT b-values for History–Social Science, Grade Eight ................................................................................ 546 Table 8.D.22 IRT b-values for World History..................................................................................................................... 546 Table 8.D.23 IRT b-values for History–Social Science Grade Eleven ............................................................................... 546 Table 8.D.24 IRT b-values for Mathematics, Grade Two .................................................................................................. 546 Table 8.D.25 IRT b-values for Mathematics, Grade Three ............................................................................................... 547 Table 8.D.26 IRT b-values for Mathematics, Grade Four ................................................................................................. 547 Table 8.D.27 IRT b-values for Mathematics, Grade Five .................................................................................................. 547 Table 8.D.28 IRT b-values for Mathematics, Grade Six .................................................................................................... 547 Table 8.D.29 IRT b-values for Mathematics, Grade Seven ............................................................................................... 548 Table 8.D.30 IRT b-values for General Mathematics ........................................................................................................ 548 Table 8.D.31 IRT b-values for Algebra I ............................................................................................................................ 548 Table 8.D.32 IRT b-values for Geometry .......................................................................................................................... 548 Table 8.D.33 IRT b-values for Algebra II ........................................................................................................................... 549 Table 8.D.34 IRT b-values for Summative High School Mathematics ............................................................................... 549 Table 8.D.35 IRT b-values for Integrated Mathematics 1 .................................................................................................. 549 Table 8.D.36 IRT b-values for Integrated Mathematics 2 .................................................................................................. 549 Table 8.D.37 IRT b-values for Integrated Mathematics 3 .................................................................................................. 550 Table 8.D.38 IRT b-values for Science, Grade Five.......................................................................................................... 550 Table 8.D.39 IRT b-values for Science, Grade Eight ........................................................................................................ 550 Table 8.D.40 IRT b-values for Grade 10 Life Science ....................................................................................................... 550 Table 8.D.41 IRT b-values for Biology .............................................................................................................................. 551 Table 8.D.42 IRT b-values for Chemistry .......................................................................................................................... 551 Table 8.D.43 IRT b-values for Earth Science .................................................................................................................... 551 Table 8.D.44 IRT b-values for Physics .............................................................................................................................. 551 Table 8.D.45 Distribution of IRT b-values for ELA ............................................................................................................ 552 Table 8.D.46 Distribution of IRT b-values for History–Social Science ............................................................................... 552 Table 8.D.47 Distribution of IRT b-values for Mathematics, Grades Two through Seven ................................................. 553 Table 8.D.48 Distribution of IRT b-values for Mathematics End-of-Course CSTs ............................................................. 553 Table 8.D.49 Distribution of IRT b-values for Science ...................................................................................................... 554 Table 8.D.50 Distribution of IRT b-values for ELA (field-test items) .................................................................................. 554 Table 8.D.51 Distribution of IRT b-values for History–Social Science (field-test items) .................................................... 555 Table 8.D.52 Distribution of IRT b-values for Mathematics, Grades Two through Seven (field-test items) ....................... 555 Table 8.D.53 Distribution of IRT b-values for Mathematics End-of-Course CSTs (field-test items) .................................. 556 Table 8.D.54 Distribution of IRT b-values for Science (field-test items) ............................................................................ 556 Table 8.D.55 2009 New Conversions for ELA, Grades Two and Three ............................................................................ 557 Table 8.D.56 New Conversions for ELA, Grade Four (with and without essay) ................................................................ 558 Table 8.D.57 2009 New Conversions for ELA, Grades Five and Six ................................................................................ 559 Table 8.D.58 2009 New Conversions for ELA, Grade Seven (with and without essay) .................................................... 560 Table 8.D.59 2009 New Conversions for ELA, Grades Eight and Nine ............................................................................ 561 Table 8.D.60 2009 New Conversions for ELA, Grades Ten and Eleven ........................................................................... 562 Table 8.D.61 2009 New Conversions for History–Social Science, Grade Eight and World History .................................. 563 Table 8.D.62 2009 New Conversion for History–Social Science, Grade Eleven ............................................................... 564 Table 8.D.63 2009 New Conversions for Mathematics, Grades Two and Three .............................................................. 565 Table 8.D.64 2009 New Conversions for Mathematics, Grades Four and Five ................................................................ 566 Table 8.D.65 2009 New Conversions for Mathematics, Grades Six and Seven ............................................................... 567 Page x
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560
Technical report page 558 560

Recommended

THE IMPACT OF SOCIALMEDIA ON ENTREPRENEURIAL NETWORKS
THE IMPACT OF SOCIALMEDIA ON ENTREPRENEURIAL NETWORKSTHE IMPACT OF SOCIALMEDIA ON ENTREPRENEURIAL NETWORKS
THE IMPACT OF SOCIALMEDIA ON ENTREPRENEURIAL NETWORKSDebashish Mandal
 
Learning to live together
Learning to live togetherLearning to live together
Learning to live togetherLittle Daisy
 
Monitoring and Supervision Procedures for Exploration and Development Activit...
Monitoring and Supervision Procedures for Exploration and Development Activit...Monitoring and Supervision Procedures for Exploration and Development Activit...
Monitoring and Supervision Procedures for Exploration and Development Activit...Nicholas Varilone
 
2014 National Senior Certificate Examination Diagnostic report
2014 National Senior Certificate Examination Diagnostic report2014 National Senior Certificate Examination Diagnostic report
2014 National Senior Certificate Examination Diagnostic reportRene Kotze
 
194825214 science-tests
194825214 science-tests194825214 science-tests
194825214 science-testsธีรพงศ์ อ่อนอก
 
Wa early childhood_handbook
Wa early childhood_handbookWa early childhood_handbook
Wa early childhood_handbookshawnamali
 
WelDest Primary Research Report
WelDest Primary Research ReportWelDest Primary Research Report
WelDest Primary Research ReportWelDest Project (Erasmus LLP)
 
Reading stages
Reading stagesReading stages
Reading stagesscaredstraight
 

Recommended

THE IMPACT OF SOCIALMEDIA ON ENTREPRENEURIAL NETWORKS
THE IMPACT OF SOCIALMEDIA ON ENTREPRENEURIAL NETWORKSTHE IMPACT OF SOCIALMEDIA ON ENTREPRENEURIAL NETWORKS
THE IMPACT OF SOCIALMEDIA ON ENTREPRENEURIAL NETWORKSDebashish Mandal
 
Learning to live together
Learning to live togetherLearning to live together
Learning to live togetherLittle Daisy
 
Monitoring and Supervision Procedures for Exploration and Development Activit...
Monitoring and Supervision Procedures for Exploration and Development Activit...Monitoring and Supervision Procedures for Exploration and Development Activit...
Monitoring and Supervision Procedures for Exploration and Development Activit...Nicholas Varilone
 
2014 National Senior Certificate Examination Diagnostic report
2014 National Senior Certificate Examination Diagnostic report2014 National Senior Certificate Examination Diagnostic report
2014 National Senior Certificate Examination Diagnostic reportRene Kotze
 
194825214 science-tests
194825214 science-tests194825214 science-tests
194825214 science-testsธีรพงศ์ อ่อนอก
 
Wa early childhood_handbook
Wa early childhood_handbookWa early childhood_handbook
Wa early childhood_handbookshawnamali
 
WelDest Primary Research Report
WelDest Primary Research ReportWelDest Primary Research Report
WelDest Primary Research ReportWelDest Project (Erasmus LLP)
 
Reading stages
Reading stagesReading stages
Reading stagesscaredstraight
 
Reform
ReformReform
ReformWellesley Public Schools
 
Presentación slides-portfolio
Presentación slides-portfolioPresentación slides-portfolio
Presentación slides-portfolioDavicho2013
 
Developing The Mongolian Wolf Spirit In Football
Developing The Mongolian Wolf Spirit In FootballDeveloping The Mongolian Wolf Spirit In Football
Developing The Mongolian Wolf Spirit In FootballMichele Ferraris
 
Presentación slides-portfolio
Presentación slides-portfolioPresentación slides-portfolio
Presentación slides-portfolioDavicho2013
 
6th math plc analysis
6th math plc analysis6th math plc analysis
6th math plc analysisdavewoods
 
Footprint template
Footprint templateFootprint template
Footprint templatedavewoods
 
Api and calculator power point
Api and calculator power pointApi and calculator power point
Api and calculator power pointdavewoods
 
Api calculator 07_08_practice_school_data
Api calculator 07_08_practice_school_dataApi calculator 07_08_practice_school_data
Api calculator 07_08_practice_school_datadavewoods
 
Successful
SuccessfulSuccessful
Successfulscaredstraight
 
Install program
Install programInstall program
Install programHuudhoed Al-Lael
 
BETT 2015: Moodle and Microsoft, Better Together
BETT 2015: Moodle and Microsoft, Better TogetherBETT 2015: Moodle and Microsoft, Better Together
BETT 2015: Moodle and Microsoft, Better TogetherDoug Mahugh
 
2011 presentation powerpoint
2011 presentation powerpoint2011 presentation powerpoint
2011 presentation powerpointdavewoods
 
12 power words
12 power words12 power words
12 power wordsdavewoods
 
12 power words quiz
12 power words quiz12 power words quiz
12 power words quizdavewoods
 
Templos n.
Templos n. Templos n.
Templos n. Ivone Janete Katzer
 
Testing tips strategies ppt
Testing tips strategies pptTesting tips strategies ppt
Testing tips strategies pptdavewoods
 
Perso je-dis-bravo-mr-harkins11
Perso je-dis-bravo-mr-harkins11Perso je-dis-bravo-mr-harkins11
Perso je-dis-bravo-mr-harkins11Ivone Janete Katzer
 
Test taking vocabulary
Test taking vocabularyTest taking vocabulary
Test taking vocabularydavewoods
 
Stages of Reading Development
Stages of Reading DevelopmentStages of Reading Development
Stages of Reading DevelopmentKarisse Ramoso
 
Developmental Reading
Developmental ReadingDevelopmental Reading
Developmental Readingalyn mae valledor
 
Maint language final
Maint language finalMaint language final
Maint language finalLeilaHannane
 
Lesson 5...Guide
Lesson 5...GuideLesson 5...Guide
Lesson 5...Guidebhushan Nehete
 

More Related Content

Viewers also liked

Presentación slides-portfolio
Presentación slides-portfolioPresentación slides-portfolio
Presentación slides-portfolioDavicho2013
 
Developing The Mongolian Wolf Spirit In Football
Developing The Mongolian Wolf Spirit In FootballDeveloping The Mongolian Wolf Spirit In Football
Developing The Mongolian Wolf Spirit In FootballMichele Ferraris
 
Presentación slides-portfolio
Presentación slides-portfolioPresentación slides-portfolio
Presentación slides-portfolioDavicho2013
 
6th math plc analysis
6th math plc analysis6th math plc analysis
6th math plc analysisdavewoods
 
Footprint template
Footprint templateFootprint template
Footprint templatedavewoods
 
Api and calculator power point
Api and calculator power pointApi and calculator power point
Api and calculator power pointdavewoods
 
Api calculator 07_08_practice_school_data
Api calculator 07_08_practice_school_dataApi calculator 07_08_practice_school_data
Api calculator 07_08_practice_school_datadavewoods
 
BETT 2015: Moodle and Microsoft, Better Together
BETT 2015: Moodle and Microsoft, Better TogetherBETT 2015: Moodle and Microsoft, Better Together
BETT 2015: Moodle and Microsoft, Better TogetherDoug Mahugh
 
2011 presentation powerpoint
2011 presentation powerpoint2011 presentation powerpoint
2011 presentation powerpointdavewoods
 
12 power words
12 power words12 power words
12 power wordsdavewoods
 
12 power words quiz
12 power words quiz12 power words quiz
12 power words quizdavewoods
 
Testing tips strategies ppt
Testing tips strategies pptTesting tips strategies ppt
Testing tips strategies pptdavewoods
 
Test taking vocabulary
Test taking vocabularyTest taking vocabulary
Test taking vocabularydavewoods
 
Stages of Reading Development
Stages of Reading DevelopmentStages of Reading Development
Stages of Reading DevelopmentKarisse Ramoso
 

Viewers also liked (20)

Reform
ReformReform
Reform
 
Presentación slides-portfolio
Presentación slides-portfolioPresentación slides-portfolio
Presentación slides-portfolio
 
Developing The Mongolian Wolf Spirit In Football
Developing The Mongolian Wolf Spirit In FootballDeveloping The Mongolian Wolf Spirit In Football
Developing The Mongolian Wolf Spirit In Football
 
Presentación slides-portfolio
Presentación slides-portfolioPresentación slides-portfolio
Presentación slides-portfolio
 
6th math plc analysis
6th math plc analysis6th math plc analysis
6th math plc analysis
 
Footprint template
Footprint templateFootprint template
Footprint template
 
Api and calculator power point
Api and calculator power pointApi and calculator power point
Api and calculator power point
 
Api calculator 07_08_practice_school_data
Api calculator 07_08_practice_school_dataApi calculator 07_08_practice_school_data
Api calculator 07_08_practice_school_data
 
Successful
SuccessfulSuccessful
Successful
 
Install program
Install programInstall program
Install program
 
BETT 2015: Moodle and Microsoft, Better Together
BETT 2015: Moodle and Microsoft, Better TogetherBETT 2015: Moodle and Microsoft, Better Together
BETT 2015: Moodle and Microsoft, Better Together
 
2011 presentation powerpoint
2011 presentation powerpoint2011 presentation powerpoint
2011 presentation powerpoint
 
12 power words
12 power words12 power words
12 power words
 
12 power words quiz
12 power words quiz12 power words quiz
12 power words quiz
 
Templos n.
Templos n. Templos n.
Templos n.
 
Testing tips strategies ppt
Testing tips strategies pptTesting tips strategies ppt
Testing tips strategies ppt
 
Perso je-dis-bravo-mr-harkins11
Perso je-dis-bravo-mr-harkins11Perso je-dis-bravo-mr-harkins11
Perso je-dis-bravo-mr-harkins11
 
Test taking vocabulary
Test taking vocabularyTest taking vocabulary
Test taking vocabulary
 
Stages of Reading Development
Stages of Reading DevelopmentStages of Reading Development
Stages of Reading Development
 
Developmental Reading
Developmental ReadingDevelopmental Reading
Developmental Reading
 

Similar to Technical report page 558 560

Maint language final
Maint language finalMaint language final
Maint language finalLeilaHannane
 
Pages from asnt snt tc-1 a-2016
Pages from asnt snt tc-1 a-2016Pages from asnt snt tc-1 a-2016
Pages from asnt snt tc-1 a-2016david guo
 
Emergency Planning Independent Study 235.b
Emergency Planning Independent Study 235.b Emergency Planning Independent Study 235.b
Emergency Planning Independent Study 235.b MerrileeDelvalle969
 
Emergency planning independent study 235.b
Emergency planning independent study 235.b Emergency planning independent study 235.b
Emergency planning independent study 235.b ronak56
 
Approaches and implications of eLearning Adoption in Relation to Academic Sta...
Approaches and implications of eLearning Adoption in Relation to Academic Sta...Approaches and implications of eLearning Adoption in Relation to Academic Sta...
Approaches and implications of eLearning Adoption in Relation to Academic Sta...Nancy Ideker
 
Basic Thinking Tool for E-Services Planning
Basic Thinking Tool for E-Services PlanningBasic Thinking Tool for E-Services Planning
Basic Thinking Tool for E-Services PlanningJohn Macasio
 
Project Control System Manual
Project Control System ManualProject Control System Manual
Project Control System Manualharoldtaylor1113
 
Montana Videoconferencing Strategic Plan Final Report June 12 ...
Montana Videoconferencing Strategic Plan Final Report June 12 ...Montana Videoconferencing Strategic Plan Final Report June 12 ...
Montana Videoconferencing Strategic Plan Final Report June 12 ...Videoguy
 
Citrus College - NASA SL Criticla Design Review
Citrus College - NASA SL Criticla Design ReviewCitrus College - NASA SL Criticla Design Review
Citrus College - NASA SL Criticla Design ReviewJoseph Molina
 
Trimble total station help
Trimble total station helpTrimble total station help
Trimble total station helpGonçalo Beja
 
Honolulu Emergency Services Audit
Honolulu Emergency Services AuditHonolulu Emergency Services Audit
Honolulu Emergency Services AuditHonolulu Civil Beat
 
Tensioned Building Construction - Testing and Validation
Tensioned Building Construction - Testing and ValidationTensioned Building Construction - Testing and Validation
Tensioned Building Construction - Testing and ValidationRobert Lewis
 

Similar to Technical report page 558 560 (20)

Maint language final
Maint language finalMaint language final
Maint language final
 
Lesson 5...Guide
Lesson 5...GuideLesson 5...Guide
Lesson 5...Guide
 
Pages from asnt snt tc-1 a-2016
Pages from asnt snt tc-1 a-2016Pages from asnt snt tc-1 a-2016
Pages from asnt snt tc-1 a-2016
 
Emergency Planning Independent Study 235.b
Emergency Planning Independent Study 235.b Emergency Planning Independent Study 235.b
Emergency Planning Independent Study 235.b
 
Emergency planning independent study 235.b
Emergency planning independent study 235.b Emergency planning independent study 235.b
Emergency planning independent study 235.b
 
Approaches and implications of eLearning Adoption in Relation to Academic Sta...
Approaches and implications of eLearning Adoption in Relation to Academic Sta...Approaches and implications of eLearning Adoption in Relation to Academic Sta...
Approaches and implications of eLearning Adoption in Relation to Academic Sta...
 
Basic Thinking Tool for E-Services Planning
Basic Thinking Tool for E-Services PlanningBasic Thinking Tool for E-Services Planning
Basic Thinking Tool for E-Services Planning
 
BW Power User TG R4a-1
BW Power User TG R4a-1BW Power User TG R4a-1
BW Power User TG R4a-1
 
Thesis writing
Thesis writingThesis writing
Thesis writing
 
Project Control System Manual
Project Control System ManualProject Control System Manual
Project Control System Manual
 
Montana Videoconferencing Strategic Plan Final Report June 12 ...
Montana Videoconferencing Strategic Plan Final Report June 12 ...Montana Videoconferencing Strategic Plan Final Report June 12 ...
Montana Videoconferencing Strategic Plan Final Report June 12 ...
 
Evaluating Your Program
Evaluating Your ProgramEvaluating Your Program
Evaluating Your Program
 
Citrus College - NASA SL Criticla Design Review
Citrus College - NASA SL Criticla Design ReviewCitrus College - NASA SL Criticla Design Review
Citrus College - NASA SL Criticla Design Review
 
Slr kitchenham
Slr kitchenhamSlr kitchenham
Slr kitchenham
 
Trimble total station help
Trimble total station helpTrimble total station help
Trimble total station help
 
Honolulu Emergency Services Audit
Honolulu Emergency Services AuditHonolulu Emergency Services Audit
Honolulu Emergency Services Audit
 
Sdm Manual
Sdm ManualSdm Manual
Sdm Manual
 
Rand rr2504
Rand rr2504Rand rr2504
Rand rr2504
 
CSTE_CBOK_V6-2
CSTE_CBOK_V6-2CSTE_CBOK_V6-2
CSTE_CBOK_V6-2
 
Tensioned Building Construction - Testing and Validation
Tensioned Building Construction - Testing and ValidationTensioned Building Construction - Testing and Validation
Tensioned Building Construction - Testing and Validation
 

More from davewoods

Alphabet tally sheet
Alphabet tally sheetAlphabet tally sheet
Alphabet tally sheetdavewoods
 
Alphabet pyramid2
Alphabet pyramid2Alphabet pyramid2
Alphabet pyramid2davewoods
 
Alphabet collaboration activity
Alphabet collaboration activityAlphabet collaboration activity
Alphabet collaboration activitydavewoods
 
Test thinking strategy
Test thinking strategyTest thinking strategy
Test thinking strategydavewoods
 
Star student report
Star student reportStar student report
Star student reportdavewoods
 
Star card parent letter
Star card parent letterStar card parent letter
Star card parent letterdavewoods
 
Released test question activity
Released test question activityReleased test question activity
Released test question activitydavewoods
 
Bar footprint template
Bar footprint templateBar footprint template
Bar footprint templatedavewoods
 

More from davewoods (8)

Alphabet tally sheet
Alphabet tally sheetAlphabet tally sheet
Alphabet tally sheet
 
Alphabet pyramid2
Alphabet pyramid2Alphabet pyramid2
Alphabet pyramid2
 
Alphabet collaboration activity
Alphabet collaboration activityAlphabet collaboration activity
Alphabet collaboration activity
 
Test thinking strategy
Test thinking strategyTest thinking strategy
Test thinking strategy
 
Star student report
Star student reportStar student report
Star student report
 
Star card parent letter
Star card parent letterStar card parent letter
Star card parent letter
 
Released test question activity
Released test question activityReleased test question activity
Released test question activity
 
Bar footprint template
Bar footprint templateBar footprint template
Bar footprint template
 

Technical report page 558 560

  • 1. California Department of Education Statewide Assessment Division California Standards Tests Technical Report Spring 2009 Administration Submitted March 1, 2010 Educational Testing Service Contract No. 5417
  • 2.
  • 3. STAR Program Table of Contents (Summarized) Chapter 1: Introduction......................................................................................................................................... 1 Background ........................................................................................................................................................................... 1 Test Purpose.......................................................................................................................................................................... 1 Test Content .......................................................................................................................................................................... 2 Intended Population .............................................................................................................................................................. 2 Intended Use and Purpose of Test Scores.......................................................................................................................... 3 Testing Window ..................................................................................................................................................................... 3 Significant STAR Developments in 2009 ............................................................................................................................. 4 Limitations of the Assessment ............................................................................................................................................ 4 Groups and Organizations Involved with the STAR Program ........................................................................................... 5 Overview of the Technical Report ........................................................................................................................................ 5 References ............................................................................................................................................................................. 7 Chapter 2: An Overview of CST Processes ........................................................................................................ 8 Item Development ................................................................................................................................................................. 8 Test Variations, Accommodations, and Modifications .................................................................................................... 11 Scores .................................................................................................................................................................................. 12 References ........................................................................................................................................................................... 20 Chapter 3: Item Development........................................................................................................................... 196 Rules for Item Development ............................................................................................................................................. 196 Selection of Item Writers .................................................................................................................................................. 198 Item Review Process ......................................................................................................................................................... 199 Field Testing ...................................................................................................................................................................... 203 Data Review Meetings ....................................................................................................................................................... 205 Item Banking ...................................................................................................................................................................... 206 References ......................................................................................................................................................................... 207 Chapter 4: Test Assembly ................................................................................................................................ 208 Test Length ........................................................................................................................................................................ 208 Rules for Item Selection.................................................................................................................................................... 208 Reference ........................................................................................................................................................................... 213 Appendix 4.B—Cluster Targets ........................................................................................................................................ 223 Chapter 5: Test Administration ........................................................................................................................ 253 Test Security and Confidentiality ..................................................................................................................................... 253 Procedures to Maintain Standardization ......................................................................................................................... 258 Accommodations for Students with Disabilities ............................................................................................................ 260 Demographic Data Corrections ........................................................................................................................................ 261 Testing Irregularities ......................................................................................................................................................... 261 Test Administration Incidents .......................................................................................................................................... 261 References ......................................................................................................................................................................... 262 Chapter 6: Performance Standards ................................................................................................................. 263 Background ....................................................................................................................................................................... 263 Standard Setting Procedure ............................................................................................................................................. 263 Results ............................................................................................................................................................................... 265 References ......................................................................................................................................................................... 268 Chapter 7: Scoring and Reporting ................................................................................................................... 269 Procedures for Maintaining and Retrieving Individual Scores ...................................................................................... 269 Types of Scores and Subscores ...................................................................................................................................... 270 Score Verification Procedures ......................................................................................................................................... 271 Overview of Score Aggregation Procedures .................................................................................................................. 274 Reports to be Produced and Scores for Each Report .................................................................................................... 278 Criteria for Interpreting Test Scores ................................................................................................................................ 280 Criteria for Interpreting Score Reports ............................................................................................................................ 280 Reference ........................................................................................................................................................................... 281 Chapter 8: Analyses .......................................................................................................................................... 372 Samples Used for the Analyses ....................................................................................................................................... 372 Classical Item Analyses .................................................................................................................................................... 373 Reliability Analyses ........................................................................................................................................................... 375 Decision Classification Analyses .................................................................................................................................... 379 Validity Evidence ............................................................................................................................................................... 382 Page i
  • 4. STAR Program IRT Analyses ...................................................................................................................................................................... 394 DIF Analyses ...................................................................................................................................................................... 399 References ......................................................................................................................................................................... 401 Chapter 9: Quality Control Procedures ...........................................................................................................609 Quality Control of Item Development .............................................................................................................................. 609 Quality Control of the Item Bank ...................................................................................................................................... 611 Quality Control of Test Materials ..................................................................................................................................... 611 Quality Control of Scanning ............................................................................................................................................. 612 Quality Control of Image Editing ...................................................................................................................................... 613 Quality Control of Answer Document Processing and Scoring .................................................................................... 613 Quality Control of Psychometric Processes ................................................................................................................... 615 Quality Control of Reporting ............................................................................................................................................ 617 Chapter 10: Historical Comparisons ............................................................................................................... 618 Examinee Performance ..................................................................................................................................................... 619 Test Characteristics .......................................................................................................................................................... 620 References ......................................................................................................................................................................... 621 Page ii
  • 5. STAR Program Table of Contents (Full) Acronyms and Initialisms Used in the CST Technical Report.............................................................................. xiv Chapter 1: Introduction......................................................................................................................................... 1 Background ........................................................................................................................................................................... 1 Test Purpose.......................................................................................................................................................................... 1 Test Content .......................................................................................................................................................................... 2 Intended Population .............................................................................................................................................................. 2 Intended Use and Purpose of Test Scores.......................................................................................................................... 3 Testing Window ..................................................................................................................................................................... 3 Significant STAR Developments in 2009 ............................................................................................................................. 4 Elimination of California Achievement Tests, Sixth Edition Survey (CAT/6 Survey) .............................................................. 4 Release of CST Exemplars .................................................................................................................................................... 4 Limitations of the Assessment ............................................................................................................................................ 4 Score Interpretation ............................................................................................................................................................... 4 Out-of-Level Testing .............................................................................................................................................................. 4 Score Comparison ................................................................................................................................................................. 4 Groups and Organizations Involved with the STAR Program ........................................................................................... 5 State Board of Education (SBE)............................................................................................................................................. 5 California Department of Education (CDE) ............................................................................................................................ 5 Contractors ............................................................................................................................................................................ 5 Overview of the Technical Report ........................................................................................................................................ 5 References ............................................................................................................................................................................. 7 Chapter 2: An Overview of CST Processes ........................................................................................................ 8 Item Development ................................................................................................................................................................. 8 Item Formats .......................................................................................................................................................................... 8 Item Development Specifications ........................................................................................................................................... 8 Item Banking .......................................................................................................................................................................... 8 Item Refresh Rate and Released Test Questions (RTQ) ....................................................................................................... 9 Test Length ............................................................................................................................................................................ 9 Test Blueprint ......................................................................................................................................................................... 9 Content Rules and Item Selection.......................................................................................................................................... 9 Psychometric Criteria ........................................................................................................................................................... 10 Test Security and Confidentiality.......................................................................................................................................... 10 Procedures to Maintain Standardization .............................................................................................................................. 11 Test Variations, Accommodations, and Modifications .................................................................................................... 11 Test Variations, Accommodation and Modification Summaries ........................................................................................... 12 Scores .................................................................................................................................................................................. 12 Individual Scores.................................................................................................................................................................. 13 Group Scores ....................................................................................................................................................................... 13 Calibration ............................................................................................................................................................................ 14 Scaling ................................................................................................................................................................................. 15 True Score Equating ............................................................................................................................................................ 15 Equating Samples ................................................................................................................................................................ 17 Equating the Braille Versions of the CSTs ........................................................................................................................... 18 Scaling the CSTs for Integrated/Coordinated Science ......................................................................................................... 18 References ........................................................................................................................................................................... 20 Appendix 2.A—CST Item and Estimated Time Chart ....................................................................................................... 21 Appendix 2.B—Reporting Clusters .................................................................................................................................... 22 English–Language Arts ........................................................................................................................................................ 22 History–Social Science ........................................................................................................................................................ 24 Mathematics......................................................................................................................................................................... 24 Science ................................................................................................................................................................................ 27 Appendix 2.C—2009 Test Variations, Accommodations, and Modifications ................................................................. 32 Appendix 2.D—Special Services Summary Tables .......................................................................................................... 35 Chapter 3: Item Development........................................................................................................................... 196 Rules for Item Development ............................................................................................................................................. 196 Item Development Specifications ....................................................................................................................................... 196 Expected Item Ratio ........................................................................................................................................................... 197 Selection of Item Writers .................................................................................................................................................. 198 Criteria for Selecting Item Writers ...................................................................................................................................... 198 Item Writer Training ........................................................................................................................................................... 198 Page iii
  • 6. STAR Program Item Review Process ......................................................................................................................................................... 199 Contractor Review ............................................................................................................................................................. 199 Content Expert Reviews .................................................................................................................................................... 200 Statewide Pupil Assessment Review (SPAR) Panel .......................................................................................................... 203 Field Testing ...................................................................................................................................................................... 203 Stand-Alone Field Testing .................................................................................................................................................. 203 Embedded Field-Test Items ............................................................................................................................................... 204 Data Review Meetings ....................................................................................................................................................... 205 Item Banking ...................................................................................................................................................................... 206 References ......................................................................................................................................................................... 207 Chapter 4: Test Assembly ................................................................................................................................ 208 Test Length ........................................................................................................................................................................ 208 Rules for Item Selection.................................................................................................................................................... 208 Test Blueprint ..................................................................................................................................................................... 208 Content Rules and Item Selection...................................................................................................................................... 208 Psychometric Criteria ......................................................................................................................................................... 209 Projected Psychometric Properties of the Assembled Tests .............................................................................................. 211 Rules for Item Sequence and Layout ................................................................................................................................. 212 Reference ........................................................................................................................................................................... 213 Appendix 4.A—Technical Characteristics....................................................................................................................... 214 Appendix 4.B—Cluster Targets ........................................................................................................................................ 223 Chapter 5: Test Administration ........................................................................................................................ 253 Test Security and Confidentiality ..................................................................................................................................... 253 ETS’s Office of Testing Integrity (OTI) ............................................................................................................................... 253 Test Development .............................................................................................................................................................. 253 Item Review ....................................................................................................................................................................... 253 Item Bank ........................................................................................................................................................................... 254 Transfer of Forms and Items to the CDE ........................................................................................................................... 254 Firewall .............................................................................................................................................................................. 254 Printing............................................................................................................................................................................... 255 Test Administration ............................................................................................................................................................ 255 Test Delivery ...................................................................................................................................................................... 255 Processing and Scoring ..................................................................................................................................................... 256 Data Management ............................................................................................................................................................. 256 Transfer of Scores via Secure Data Exchange .................................................................................................................. 257 Statistical Analysis ............................................................................................................................................................. 257 Reporting and Posting Results........................................................................................................................................... 257 Student Confidentiality ....................................................................................................................................................... 257 Test Results ....................................................................................................................................................................... 257 Procedures to Maintain Standardization ......................................................................................................................... 258 Test Administrators ............................................................................................................................................................ 258 Directions for Administration (DFA) .................................................................................................................................... 259 District and Test Site Coordinator Manual ......................................................................................................................... 259 STAR Management System Manuals ................................................................................................................................ 259 Test Booklets ..................................................................................................................................................................... 260 Accommodations for Students with Disabilities ............................................................................................................ 260 Identification ....................................................................................................................................................................... 260 Scoring............................................................................................................................................................................... 261 Demographic Data Corrections ........................................................................................................................................ 261 Testing Irregularities ......................................................................................................................................................... 261 Test Administration Incidents .......................................................................................................................................... 261 References ......................................................................................................................................................................... 262 Chapter 6: Performance Standards ................................................................................................................. 263 Background ....................................................................................................................................................................... 263 Standard Setting Procedure ............................................................................................................................................. 263 Standard Setting Methodologies ........................................................................................................................................ 264 Results ............................................................................................................................................................................... 265 References ......................................................................................................................................................................... 268 Chapter 7: Scoring and Reporting ................................................................................................................... 269 Procedures for Maintaining and Retrieving Individual Scores ...................................................................................... 269 Scoring and Reporting Specifications ................................................................................................................................ 269 Scanning and Scoring ........................................................................................................................................................ 270 Types of Scores and Subscores ...................................................................................................................................... 270 Raw Score ......................................................................................................................................................................... 270 Page iv
  • 7. STAR Program Subscore............................................................................................................................................................................ 270 Scale Score........................................................................................................................................................................ 270 Performance Levels ........................................................................................................................................................... 270 Writing Score ..................................................................................................................................................................... 271 Score Verification Procedures ......................................................................................................................................... 271 Scoring Key Verification Process ....................................................................................................................................... 271 Monitoring and Quality Control of Writing Scoring ............................................................................................................. 272 Score Verification Process ................................................................................................................................................. 274 Overview of Score Aggregation Procedures .................................................................................................................. 274 Individual Scores................................................................................................................................................................ 274 Reports to be Produced and Scores for Each Report .................................................................................................... 278 Types of Score Reports ..................................................................................................................................................... 278 Score Report Contents ...................................................................................................................................................... 278 Score Report Applications.................................................................................................................................................. 279 Criteria for Interpreting Test Scores ................................................................................................................................ 280 Criteria for Interpreting Score Reports ............................................................................................................................ 280 Reference ........................................................................................................................................................................... 281 Appendix 7.A—ELA for Writing (Grades Four and Seven) Rubrics .............................................................................. 282 Grade Four Scoring Rubric ................................................................................................................................................ 282 Grade Seven Scoring Rubric ............................................................................................................................................. 285 Appendix 7.B—Scale Score Distributions....................................................................................................................... 288 Appendix 7.C—Demographic Summaries ....................................................................................................................... 292 Appendix 7.D—Types of Score Reports .......................................................................................................................... 368 Chapter 8: Analyses .......................................................................................................................................... 372 Samples Used for the Analyses ....................................................................................................................................... 372 Classical Item Analyses .................................................................................................................................................... 373 Multiple-Choice Items ........................................................................................................................................................ 373 Writing Tasks ..................................................................................................................................................................... 374 Reliability Analyses ........................................................................................................................................................... 375 Intercorrelations, Reliabilities, and SEMs for Reporting Clusters ....................................................................................... 377 Subgroup Reliabilities and SEMs ....................................................................................................................................... 377 Conditional Standard Errors of Measurement .................................................................................................................... 377 Decision Classification Analyses .................................................................................................................................... 379 Writing Score Reliability ..................................................................................................................................................... 380 Prompt and Rater Agreement Summary ............................................................................................................................ 381 Validity Evidence ............................................................................................................................................................... 382 Purpose of the CST ........................................................................................................................................................... 382 The Constructs to Be Measured ........................................................................................................................................ 382 The Scores Generated and the Interpretations and Uses of these Scores ........................................................................ 383 Intended Test Population(s) ............................................................................................................................................... 383 Validity Evidence Collected ................................................................................................................................................ 384 Evidence Based on Response Processes ......................................................................................................................... 391 Evidence Based on Internal Structure ................................................................................................................................ 392 Evidence Based on Consequences of Testing................................................................................................................... 394 IRT Analyses ...................................................................................................................................................................... 394 IRT Model-Data Fit Analyses ............................................................................................................................................. 394 Model Fit Assessment Results ........................................................................................................................................... 396 Evaluation of Scaling ......................................................................................................................................................... 396 Summaries of Scaled IRT b-values.................................................................................................................................... 398 Post-Equating Results ....................................................................................................................................................... 398 DIF Analyses ...................................................................................................................................................................... 399 References ......................................................................................................................................................................... 401 Appendix 8.A—Classical Analyses .................................................................................................................................. 403 Appendix 8.B—Reliability Analyses ................................................................................................................................ 415 Appendix 8.C—Validity Analyses .................................................................................................................................... 497 Appendix 8.D—IRT Analyses ........................................................................................................................................... 541 Scaling and Post-Equating Results .................................................................................................................................... 557 Appendix 8.E—DIF Analyses............................................................................................................................................ 579 Chapter 9: Quality Control Procedures ...........................................................................................................609 Quality Control of Item Development .............................................................................................................................. 609 Item Specifications ............................................................................................................................................................. 609 Item Writers........................................................................................................................................................................ 609 Internal Contractor Reviews ............................................................................................................................................... 609 Assessment Review Panel (ARP) Review ......................................................................................................................... 610 Statewide Pupil Assessment Review (SPAR) Panel Review ............................................................................................. 610 Page v
  • 8. STAR Program Data Review of Field Tested Items .................................................................................................................................... 610 Quality Control of the Item Bank ...................................................................................................................................... 611 Quality Control of Test Materials ..................................................................................................................................... 611 Collecting Test Materials .................................................................................................................................................... 611 Processing Test Materials .................................................................................................................................................. 612 Quality Control of Scanning ............................................................................................................................................. 612 Post-scanning Edits ........................................................................................................................................................... 612 Quality Control of Image Editing ...................................................................................................................................... 613 Quality Control of Answer Document Processing and Scoring .................................................................................... 613 Accountability of Answer Documents ................................................................................................................................. 613 Processing of Answer Documents ..................................................................................................................................... 613 Scoring and Reporting Specifications ................................................................................................................................ 614 Matching Information on CST Answer Documents ............................................................................................................ 614 Matching Multiple-choice and Writing Scores for ELA Grades Four and Seven ................................................................ 614 Storing Answer Documents................................................................................................................................................ 614 Quality Control of Psychometric Processes ................................................................................................................... 615 Score Key Verification Procedures .................................................................................................................................... 615 Quality Control of Item Analyses, DIF and Equating Process ............................................................................................ 615 Score Verification Process ................................................................................................................................................. 616 Offloads to Test Development............................................................................................................................................ 616 Quality Control of Reporting ............................................................................................................................................ 617 Excluding Student Scores from Summary Reports ............................................................................................................ 617 Chapter 10: Historical Comparisons ............................................................................................................... 618 Examinee Performance ..................................................................................................................................................... 619 Test Characteristics .......................................................................................................................................................... 620 References ......................................................................................................................................................................... 621 Appendix 10.A—Historical Comparisons Tables ........................................................................................................... 622 Appendix 10.B—Historical Comparisons Tables ........................................................................................................... 632 Tables Table 2.1 Subgroup Definitions ........................................................................................................................................... 13 Table 2.2 Scale Scores Ranges for Performance Levels .................................................................................................... 16 Table 2.3 Sample Selected to Establish Reference Form ................................................................................................... 17 Table 2.4 Proficiency-Level Cut Points for Science End-of-Course CSTs .......................................................................... 19 Table 2.5 Computation of Proficiency-Level Cut Points on Integrated/Coordinated Science 1 ........................................... 19 Table 2.C.1 Matrix of Test Variations, Accommodations, and Modifications ........................................................................ 32 Table 2.C.2 Matrix of Test Variations for English Learners .................................................................................................. 34 Table 2.D. 1 Special Services Summary for ELA, Grades Two and Three ......................................................................... 35 Table 2.D.2 Special Services Summary for ELA, Grade Four............................................................................................. 40 Table 2.D.3 Special Services Summary for ELA, Grades Five and Six .............................................................................. 45 Table 2.D.4 Special Services Summary for ELA, Grade Seven .......................................................................................... 49 Table 2.D.5 Special Services Summary for ELA, Grades Eight and Nine ........................................................................... 54 Table 2.D.6 Special Services Summary for ELA, Grades Ten and Eleven ......................................................................... 58 Table 2.D.7 Special Services Summary for History–Social Science, Grade Eight .............................................................. 62 Table 2.D.8 Special Services Summary for World History .................................................................................................. 67 Table 2.D.9 Special Services Summary for History–Social Science, Grade Eleven ........................................................... 72 Table 2.D.10 Special Services Summary for Mathematics, Grades Two and Three........................................................... 77 Table 2.D.11 Special Services Summary for Mathematics, Grades Four and Five............................................................. 82 Table 2.D.12 Special Services Summary for Mathematics, Grades Six and Seven............................................................ 87 Table 2.D.13 Special Services Summary for General Mathematics.................................................................................... 92 Table 2.D.14 Special Services Summary for Algebra I ....................................................................................................... 97 Table 2.D.15 Special Services Summary for Geometry .................................................................................................... 102 Table 2.D.16 Special Services Summary for Algebra II .................................................................................................... 107 Table 2.D.17 Special Services Summary for Summative High School Mathematics ........................................................ 112 Table 2.D.18 Special Services Summary for Integrated Mathematics 1 ........................................................................... 118 Table 2.D.19 Special Services Summary for Integrated Mathematics 2 ........................................................................... 123 Table 2.D.20 Special Services Summary for Integrated Mathematics 3 ........................................................................... 128 Table 2.D.21 Special Services Summary for Science, Grade Five ................................................................................... 133 Table 2.D.22 Special Services Summary for Science, Grade Eight .................................................................................. 138 Table 2.D.23 Special Services Summary for Grade Ten Life Science .............................................................................. 143 Table 2.D.24 Special Services Summary for Biology ........................................................................................................ 148 Table 2.D.25 Special Services Summary for Chemistry ................................................................................................... 154 Table 2.D.26 Special Services Summary for Earth Science ............................................................................................. 160 Table 2.D.27 Special Services Summary for Physics ....................................................................................................... 166 Table 2.D.28 Special Services Summary for Integrated/Coordinated Science 1 .............................................................. 172 Page vi
  • 9. STAR Program Table 2.D.29 Special Services Summary for Integrated/Coordinated Science 2 .............................................................. 178 Table 2.D.37 Special Services Summary for Integrated/Coordinated Science 3 ............................................................... 184 Table 2.D.30 Special Services Summary for Integrated/Coordinated Science 4 .............................................................. 190 Table 3.1 Field-test Percentages for the CSTs ................................................................................................................. 198 Table 3.2 CST ARP Member Qualifications, by subject and total ..................................................................................... 201 Table 3.3 Summary of Items and Forms Presented in 2009 CSTs ................................................................................... 204 Table 4.1 Target Statistical Specifications for the CSTs ................................................................................................... 210 Table 4.A.1 Summary of 2009 CST Projected Technical Characteristics ......................................................................... 214 Table 4.A.2 Summary of 2009 CST Projected Statistical Attributes .................................................................................. 215 Table 6.1 Scale Scores Ranges for Performance Levels .................................................................................................. 265 Table 6.2 Percentage of Examinees in Each Performance Level ..................................................................................... 266 Table 7.1 Mean and Standard Deviation of Raw and Scale Scores for CSTs .................................................................. 275 Table 7.2 Percentage of Examinees in Proficiency Levels for CSTs ................................................................................ 276 Table 7.3 Subgroup Definitions ......................................................................................................................................... 277 Table 7.B.1 Distribution of CST Scale Scores for ELA ...................................................................................................... 288 Table 7.B.2 Distribution of CST Scale Scores for History–Social Science ........................................................................ 288 Table 7.B.3 Distribution of CST Scale Scores for Mathematics, Grades Two through Seven .......................................... 289 Table 7.B.4 Distribution of Scale Scores for Mathematics End-of-Course CSTs .............................................................. 289 Table 7.B.5 Distribution of CST Scale Scores for Science ................................................................................................ 290 Table 7.B.6 Distribution of CST Scale Scores for a Grade-Specific Population ................................................................ 291 Table 7.C.1 Demographic Summary for ELA, Grade Two ................................................................................................ 292 Table 7.C.2 Demographic Summary for ELA, Grade Three .............................................................................................. 294 Table 7.C.3 Demographic Summary for ELA, Grade Four ................................................................................................ 296 Table 7.C.4 Demographic Summary for ELA, Grade Five ................................................................................................ 298 Table 7.C.5 Demographic Summary for ELA, Grade Six .................................................................................................. 300 Table 7.C.6 Demographic Summary for ELA, Grade Seven ............................................................................................. 302 Table 7.C.7 Demographic Summary for ELA, Grade Eight ............................................................................................... 304 Table 7.C.8 Demographic Summary for ELA, Grade Nine ................................................................................................ 306 Table 7.C.9 Demographic Summary for ELA, Grade Ten ................................................................................................. 308 Table 7.C.10 Demographic Summary for ELA, Grade Eleven .......................................................................................... 310 Table 7.C.11 Demographic Summary for History–Social Science, Grade Eight ............................................................... 312 Table 7.C.12 Demographic Summary for World History ................................................................................................... 314 Table 7.C.13 Demographic Summary for History–Social Science, Grade Eleven ............................................................ 316 Table 7.C.14 Demographic Summary for Mathematics, Grade Two ................................................................................. 318 Table 7.C.15 Demographic Summary for Mathematics, Grade Three .............................................................................. 320 Table 7.C.16 Demographic Summary for Mathematics, Grade Four ................................................................................ 322 Table 7.C.17 Demographic Summary for Mathematics, Grade Five ................................................................................. 324 Table 7.C.18 Demographic Summary for Mathematics, Grade Six................................................................................... 326 Table 7.C.19 Demographic Summary for Mathematics, Grade Seven ............................................................................. 328 Table 7.C.20 Demographic Summary for General Mathematics ....................................................................................... 330 Table 7.C.21 Demographic Summary for Algebra I .......................................................................................................... 332 Table 7.C.22 Demographic Summary for Geometry ......................................................................................................... 334 Table 7.C.23 Demographic Summary for Algebra II ......................................................................................................... 336 Table 7.C.24 Demographic Summary for Summative High School Mathematics ............................................................. 338 Table 7.C.25 Demographic Summary for Integrated Mathematics 1 ................................................................................ 340 Table 7.C.26 Demographic Summary for Integrated Mathematics 2 ................................................................................ 342 Table 7.C.27 Demographic Summary for Integrated Mathematics 3 ................................................................................ 344 Table 7.C.28 Demographic Summary for Science, Grade Five ........................................................................................ 346 Table 7.C.29 Demographic Summary for Science, Grade Eight ....................................................................................... 348 Table 7.C.30 Demographic Summary for Grade Ten Life Science ................................................................................... 350 Table 7.C.31 Demographic Summary for Biology ............................................................................................................. 352 Table 7.C.32 Demographic Summary for Chemistry ......................................................................................................... 354 Table 7.C.33 Demographic Summary for Earth Science .................................................................................................. 356 Table 7.C.34 Demographic Summary for Physics ............................................................................................................ 358 Table 7.C.35 Demographic Summary for Integrated/Coordinated Science 1 ................................................................... 360 Table 7.C.36 Demographic Summary for Integrated/Coordinated Science 2 ................................................................... 362 Table 7.C.37 Demographic Summary for Integrated/Coordinated Science 3 ................................................................... 364 Table 7.C.38 Demographic Summary for Integrated/Coordinated Science 4 ................................................................... 366 Table 7.D.1 Score Reports Reflecting CST Results .......................................................................................................... 368 Table 8.1 Mean and Median Proportion Correct and Point-Biserial .................................................................................. 374 Table 8.2 Reliabilities and Standard Errors of Measurement for the CSTs ....................................................................... 376 Table 8.3 Scale Score CSEM at Cut Points ...................................................................................................................... 378 Table 8.4 2009 CST Content Area Correlations (All Valid Scores) ................................................................................... 388 Table 8.5 Evaluation of Common Items between New and Reference Test Forms .......................................................... 397 Table 8.A.1 Item-by-item p-value and Point-Biserial for ELA ............................................................................................ 403 Table 8.A.2 Item-by-item p-value and Point-Biserial for History–Social Science .............................................................. 405 Page vii
  • 10. STAR Program Table 8.A.3 Item-by-item p-value and Point-Biserial for Mathematics ............................................................................... 406 Table 8.A.4 Item-by-item p-value and Point-Biserial for Mathematics [continued] ............................................................ 407 Table 8.A.5 Item-by-item p-value and Point-Biserial for Science ...................................................................................... 408 Table 8.A.6 Item-by-item p-value and Point-Biserial for Science [continued] .................................................................... 409 Table 8.A.7 Item-by-item p-value and Point-Biserial for Grade-Specific CSTs ................................................................. 411 Table 8.A.8 Distribution of Essay Scores for ELA (Grade Four)—Overall and by Subgroup (all %) ................................. 412 Table 8.A.9 Distribution of Essay Scores for ELA (Grade Seven)—Overall and by Subgroup (all %) .............................. 412 Table 8.A.10 Mean Scores for ELA (Grade Four Essay)—Overall and by Subgroup ....................................................... 413 Table 8.A.11 Mean Scores for ELA (Grade Seven Essay)—Overall and by Subgroup .................................................... 413 Table 8.A.12 Effect Sizes for ELA (Grades Four and Seven Essay)—by Subgroup ......................................................... 414 Table 8.B.1 Reliabilities and Correlations for ELA ............................................................................................................ 415 Table 8.B.2 Reliabilities and Correlations for History–Social Science ............................................................................... 416 Table 8.B.3 Subscore Reliabilities and Correlations for Mathematics ............................................................................... 417 Table 8.B.4 Subscore Reliabilities and Correlations for Science ...................................................................................... 420 Table 8.B.5 Reliabilities and Correlations for Grade-Specific Tests .................................................................................. 422 Table 8.B.6 Reliabilities and SEMs for the CSTs by Gender ............................................................................................ 423 Table 8.B.7 Reliabilities and SEMs for the CSTs by Economic Status ............................................................................. 424 Table 8.B.8 Reliabilities and SEMs for the CSTs by Special Services .............................................................................. 425 Table 8.B.9 Reliabilities and SEMs for the CSTs by English Fluency ............................................................................... 426 Table 8.B.10 Reliabilities and SEMs for the CSTs by English Fluency ............................................................................. 427 Table 8.B.11 Reliabilities and SEMs for the CSTs by Primary Ethnicity ........................................................................... 428 Table 8.B.12 Reliabilities and SEMs for the CSTs by Primary Ethnicity ........................................................................... 429 Table 8.B.13 Reliabilities and SEMs for the CSTs by Primary Ethnicity ........................................................................... 430 Table 8.B.14 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Not Economically Disadvantaged ................ 431 Table 8.B.15 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Not Economically Disadvantaged ................ 432 Table 8.B.16 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Not Economically Disadvantaged ................ 433 Table 8.B.17 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Economically Disadvantaged ....................... 434 Table 8.B.18 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Economically Disadvantaged ....................... 435 Table 8.B.19 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Economically Disadvantaged ....................... 436 Table 8.B.20 Subscore Reliabilities and SEM for ELA by Gender/Economic Status ........................................................ 437 Table 8.B.21 Subscore Reliabilities and SEM for History–Social Science by Gender/Economic Status ........................... 438 Table 8.B.22 Subscore Reliabilities and SEM for Mathematics by Gender/Economic Status ........................................... 439 Table 8.B.23 Subscore Reliabilities and SEM for Science by Gender/Economic Status .................................................. 441 Table 8.B.24 Subscore Reliabilities and SEM for Grade-Specific CSTs by Gender/Economic Status.............................. 443 Table 8.B.25 Subscore Reliabilities and SEM for ELA by Special Services/English Fluency ........................................... 444 Table 8.B.26 Subscore Reliabilities and SEM for History–Social Science by Special Services/English Fluency .............. 446 Table 8.B.27 Subscore Reliabilities and SEM for Mathematics by Special Services/English Fluency .............................. 447 Table 8.B.28 Subscore Reliabilities and SEM for Science by Special Services/English Fluency...................................... 450 Table 8.B.29 Subscore Reliabilities and SEM for Grade-Specific CSTs by Special Services/English Fluency ................. 452 Table 8.B.30 Subscore Reliabilities and SEM for ELA by Primary Ethnicity ..................................................................... 453 Table 8.B.31 Subscore Reliabilities and SEM for History–Social Science by Primary Ethnicity........................................ 455 Table 8.B.32 Subscore Reliabilities and SEM for Mathematics by Primary Ethnicity ........................................................ 456 Table 8.B.33 Subscore Reliabilities and SEM for Science by Primary Ethnicity ............................................................... 459 Table 8.B.34 Subscore Reliabilities and SEM for Grade-Specific CSTs by Primary Ethnicity........................................... 461 Table 8.B.35 Subscore Reliabilities and SEM for ELA by Primary Ethnicity-for-Not Economically Disadvantaged .......... 462 Table 8.B.36 Subscore Reliabilities and SEM for History–Social Science by Primary Ethnicity-for-Not Economically Disadvantaged .......................................................................................................................................... 464 Table 8.B.37 Subscore Reliabilities and SEM for Mathematics by Primary Ethnicity-for-Not Economically Disadvantaged ................................................................................................................................................................ 465 Table 8.B.38 Subscore Reliabilities and SEM for Science by Primary Ethnicity-for-Not Economically Disadvantaged ................................................................................................................................................................ 468 Table 8.B.39 Subscore Reliabilities and SEM for Grade-Specific CSTs by Primary Ethnicity-for-Not Economically Disadvantaged ................................................................................................................................................................ 470 Table 8.B.40 Subscore Reliabilities and SEM for ELA by Primary Ethnicity-for-Economically Disadvantaged ................. 471 Table 8.B.41 Subscore Reliabilities and SEM for History–Social Science by Primary Ethnicity-for-Economically Disadvantaged ................................................................................................................................................................ 473 Table 8.B.42 Subscore Reliabilities and SEM for Mathematics by Primary Ethnicity-for-Economically Disadvantaged ................................................................................................................................................................ 474 Table 8.B.43 Subscore Reliabilities and SEM for Science by Primary Ethnicity-for-Economically Disadvantaged ........... 477 Table 8.B.44 Subscore Reliabilities and SEM for Grade-Specific CSTs by Primary Ethnicity-for- Economically Disadvantaged .......................................................................................................................................... 479 Table 8.B.45 Reliability of Classification for ELA, Grade Two ........................................................................................... 480 Table 8.B.46 Reliability of Classification for ELA, Grade Three ........................................................................................ 480 Table 8.B.47 Reliability of Classification for ELA, Grade Four (Reading only) .................................................................. 480 Table 8.B.48 Reliability of Classification for ELA, Grade Four (Reading and Writing) ...................................................... 481 Table 8.B.49 Reliability of Classification for ELA, Grade Five ........................................................................................... 481 Page viii
  • 11. STAR Program Table 8.B.50 Reliability of Classification for ELA, Grade Six............................................................................................. 481 Table 8.B.51 Reliability of Classification for ELA, Grade Seven (Reading only) ............................................................... 482 Table 8.B.52 Reliability of Classification for ELA, Grade Seven (Reading and Writing) .................................................... 482 Table 8.B.53 Reliability of Classification for ELA, Grade Eight ......................................................................................... 482 Table 8.B.54 Reliability of Classification for ELA, Grade Nine .......................................................................................... 483 Table 8.B.55 Reliability of Classification for ELA, Grade Ten ........................................................................................... 483 Table 8.B.56 Reliability of Classification for ELA, Grade Eleven....................................................................................... 483 Table 8.B.57 Reliability of Classification for History–Social Science, Grade Eight ............................................................ 484 Table 8.B.58 Reliability of Classification for World History ................................................................................................ 484 Table 8.B.59 Reliability of Classification for History–Social Science, Grade Eleven ......................................................... 484 Table 8.B.60 Reliability of Classification for Mathematics, Grade Two ............................................................................. 485 Table 8.B.61 Reliability of Classification for Mathematics, Grade Three ........................................................................... 485 Table 8.B.62 Reliability of Classification for Mathematics, Grade Four ............................................................................. 485 Table 8.B.63 Reliability of Classification for Mathematics, Grade Five ............................................................................. 486 Table 8.B.64 Reliability of Classification for Mathematics, Grade Six ............................................................................... 486 Table 8.B.65 Reliability of Classification for Mathematics, Grade Seven .......................................................................... 486 Table 8.B.66 Reliability of Classification for General Mathematics ................................................................................... 487 Table 8.B.67 Reliability of Classification for Algebra I ....................................................................................................... 487 Table 8.B.68 Reliability of Classification for Geometry ..................................................................................................... 487 Table 8.B.69 Reliability of Classification for Algebra II ...................................................................................................... 488 Table 8.B.70 Reliability of Classification for Summative High School Mathematics .......................................................... 488 Table 8.B.71 Reliability of Classification for Integrated Mathematics 1 ............................................................................. 488 Table 8.B.72 Reliability of Classification for Integrated Mathematics 2 ............................................................................. 489 Table 8.B.73 Reliability of Classification for Integrated Mathematics 3 ............................................................................. 489 Table 8.B.74 Reliability of Classification for Science, Grade Five ..................................................................................... 489 Table 8.B.75 Reliability of Classification for Science, Grade Eight ................................................................................... 490 Table 8.B.76 Reliability of Classification for Grade 10 Life Science .................................................................................. 490 Table 8.B.77 Reliability of Classification for Biology ......................................................................................................... 490 Table 8.B.78 Reliability of Classification for Chemistry ..................................................................................................... 491 Table 8.B.79 Reliability of Classification for Earth Science ............................................................................................... 491 Table 8.B.80 Reliability of Classification for Physics ......................................................................................................... 491 Table 8.B.81 Reliability of Classification for Integrated/Coordinated Science 1 ................................................................ 492 Table 8.B.82 Reliability of Classification for Integrated/Coordinated Science 2 ................................................................ 492 Table 8.B.83 Reliability of Classification for Integrated/Coordinated Science 3 ................................................................ 492 Table 8.B.84 Reliability of Classification for Integrated/Coordinated Science 4 ................................................................ 493 Table 8.B.85 Reliability of Classification for Algebra I – Grade 8 ...................................................................................... 493 Table 8.B.86 Reliability of Classification for Geometry – Grade 9..................................................................................... 493 Table 8.B.87 Reliability of Classification for Algebra II – Grade 10 ................................................................................... 494 Table 8.B.88 Reliability of Classification for Summative High School Mathematics – Grade 11 ....................................... 494 Table 8.B.89 Reliability of Classification for World History-Grade 10 ................................................................................ 494 Table 8.B.90 Inter-Rater Agreement Analyses for ELA (Grade Four) ............................................................................... 495 Table 8.B.91 Inter-Rater Agreement Analyses for (ELA Grade Seven) ............................................................................ 495 Table 8.B.92 Descriptive Statistics for the Ratings by the Two Raters ............................................................................. 495 Table 8.B.93 Generalizability Analyses for Grade Four Essay—[(Person: Essay) x Rater] .............................................. 496 Table 8.B.94 Generalizability Analyses for Grade Seven Essay—[(Person: Essay) x Rater] ........................................... 496 Table 8.C.1 2009 CST Content Area Correlations (Male) ................................................................................................. 497 Table 8.C.2 2009 CST Content Area Correlations (Female) ............................................................................................. 499 Table 8.C.3 2009 CST Content Area Correlations (Gender unknown) ............................................................................. 501 Table 8.C.4 2009 CST Content Area Correlations (American Indian) ............................................................................... 503 Table 8.C.5 2009 CST Content Area Correlations (Asian American) ................................................................................ 505 Table 8.C.6 2009 CST Content Area Correlations (Pacific Islander) ................................................................................ 507 Table 8.C.7 2009 CST Content Area Correlations (Filipino) ............................................................................................. 509 Table 8.C.8 2009 CST Content Area Correlations (Hispanic) ........................................................................................... 511 Table 8.C.9 2009 CST Content Area Correlations (African American) ............................................................................. 513 Table 8.C.10 2009 CST Content Area Correlations (White).............................................................................................. 515 Table 8.C.11 2009 CST Content Area Correlations (Ethnicity unknown) .......................................................................... 517 Table 8.C.12 2009 CST Content Area Correlations (English Only) ................................................................................... 519 Table 8.C.13 2009 CST Content Area Correlations (Initially Fluent English Proficient) .................................................... 521 Table 8.C.14 2009 CST Content Area Correlations (English Learner) .............................................................................. 523 Table 8.C.15 2009 CST Content Area Correlations (Reclassified Fluent English Proficient) ............................................ 525 Table 8.C.16 2009 CST Content Area Correlations (English Fluency unknown) .............................................................. 527 Table 8.C.17 2009 CST Content Area Correlations (Not Economically Disadvantaged) .................................................. 529 Table 8.C.18 2009 CST Content Area Correlations (Economically Disadvantaged) ......................................................... 531 Table 8.C.19 2009 CST Content Area Correlations (Unknown Economic Status) ............................................................ 533 Table 8.C.20 2009 CST Content Area Correlations (No Special Education Services) ...................................................... 535 Table 8.C.21 2009 CST Content Area Correlations (Special Education Services) ........................................................... 537 Page ix
  • 12. STAR Program Table 8.C.22 2009 CST Content Area Correlations (Special Education unknown) ........................................................... 539 Table 8.D.1 IRT Model Data Fit Distribution for ELA ......................................................................................................... 541 Table 8.D.2 IRT Model Data Fit Distribution for History–Social Science ........................................................................... 541 Table 8.D.3 IRT Model Data Fit Distribution for Mathematics, Grades Two through Seven ............................................. 541 Table 8.D.4 IRT Model Data Fit Distribution for Mathematics End-of-Course Tests ......................................................... 541 Table 8.D.5 IRT Model Data Fit Distribution for Science ................................................................................................... 542 Table 8.D.6 IRT Model Data Fit Distribution for ELA (field-test items) .............................................................................. 542 Table 8.D.7 IRT Model Data Fit Distribution for History–Social Science (field-test items) ................................................ 542 Table 8.D.8 IRT Model Data Fit Distribution for Mathematics, Grades Two through Seven (field-test items) ................... 542 Table 8.D.9 IRT Model Data Fit Distribution for Mathematics End-of-Course Tests (field-test items) ............................... 543 Table 8.D.10 IRT Model Data Fit Distribution for Science (field-test items) ...................................................................... 543 Table 8.D.11 IRT b-values for ELA, Grade Two................................................................................................................ 543 Table 8.D.12 IRT b-values for ELA, Grade Three ............................................................................................................. 543 Table 8.D.13 IRT b-values for ELA, Grade Four ............................................................................................................... 544 Table 8.D.14 IRT b-values for ELA, Grade Five................................................................................................................ 544 Table 8.D.15 IRT b-values for ELA, Grade Six ................................................................................................................. 544 Table 8.D.16 IRT b-values for ELA, Grade Seven ............................................................................................................ 544 Table 8.D.17 IRT b-values for ELA, Grade Eight .............................................................................................................. 545 Table 8.D.18 IRT b-values for ELA, Grade Nine ............................................................................................................... 545 Table 8.D.19 IRT b-values for ELA, Grade Ten ................................................................................................................ 545 Table 8.D.20 IRT b-values for ELA, Grade Eleven ........................................................................................................... 545 Table 8.D.21 IRT b-values for History–Social Science, Grade Eight ................................................................................ 546 Table 8.D.22 IRT b-values for World History..................................................................................................................... 546 Table 8.D.23 IRT b-values for History–Social Science Grade Eleven ............................................................................... 546 Table 8.D.24 IRT b-values for Mathematics, Grade Two .................................................................................................. 546 Table 8.D.25 IRT b-values for Mathematics, Grade Three ............................................................................................... 547 Table 8.D.26 IRT b-values for Mathematics, Grade Four ................................................................................................. 547 Table 8.D.27 IRT b-values for Mathematics, Grade Five .................................................................................................. 547 Table 8.D.28 IRT b-values for Mathematics, Grade Six .................................................................................................... 547 Table 8.D.29 IRT b-values for Mathematics, Grade Seven ............................................................................................... 548 Table 8.D.30 IRT b-values for General Mathematics ........................................................................................................ 548 Table 8.D.31 IRT b-values for Algebra I ............................................................................................................................ 548 Table 8.D.32 IRT b-values for Geometry .......................................................................................................................... 548 Table 8.D.33 IRT b-values for Algebra II ........................................................................................................................... 549 Table 8.D.34 IRT b-values for Summative High School Mathematics ............................................................................... 549 Table 8.D.35 IRT b-values for Integrated Mathematics 1 .................................................................................................. 549 Table 8.D.36 IRT b-values for Integrated Mathematics 2 .................................................................................................. 549 Table 8.D.37 IRT b-values for Integrated Mathematics 3 .................................................................................................. 550 Table 8.D.38 IRT b-values for Science, Grade Five.......................................................................................................... 550 Table 8.D.39 IRT b-values for Science, Grade Eight ........................................................................................................ 550 Table 8.D.40 IRT b-values for Grade 10 Life Science ....................................................................................................... 550 Table 8.D.41 IRT b-values for Biology .............................................................................................................................. 551 Table 8.D.42 IRT b-values for Chemistry .......................................................................................................................... 551 Table 8.D.43 IRT b-values for Earth Science .................................................................................................................... 551 Table 8.D.44 IRT b-values for Physics .............................................................................................................................. 551 Table 8.D.45 Distribution of IRT b-values for ELA ............................................................................................................ 552 Table 8.D.46 Distribution of IRT b-values for History–Social Science ............................................................................... 552 Table 8.D.47 Distribution of IRT b-values for Mathematics, Grades Two through Seven ................................................. 553 Table 8.D.48 Distribution of IRT b-values for Mathematics End-of-Course CSTs ............................................................. 553 Table 8.D.49 Distribution of IRT b-values for Science ...................................................................................................... 554 Table 8.D.50 Distribution of IRT b-values for ELA (field-test items) .................................................................................. 554 Table 8.D.51 Distribution of IRT b-values for History–Social Science (field-test items) .................................................... 555 Table 8.D.52 Distribution of IRT b-values for Mathematics, Grades Two through Seven (field-test items) ....................... 555 Table 8.D.53 Distribution of IRT b-values for Mathematics End-of-Course CSTs (field-test items) .................................. 556 Table 8.D.54 Distribution of IRT b-values for Science (field-test items) ............................................................................ 556 Table 8.D.55 2009 New Conversions for ELA, Grades Two and Three ............................................................................ 557 Table 8.D.56 New Conversions for ELA, Grade Four (with and without essay) ................................................................ 558 Table 8.D.57 2009 New Conversions for ELA, Grades Five and Six ................................................................................ 559 Table 8.D.58 2009 New Conversions for ELA, Grade Seven (with and without essay) .................................................... 560 Table 8.D.59 2009 New Conversions for ELA, Grades Eight and Nine ............................................................................ 561 Table 8.D.60 2009 New Conversions for ELA, Grades Ten and Eleven ........................................................................... 562 Table 8.D.61 2009 New Conversions for History–Social Science, Grade Eight and World History .................................. 563 Table 8.D.62 2009 New Conversion for History–Social Science, Grade Eleven ............................................................... 564 Table 8.D.63 2009 New Conversions for Mathematics, Grades Two and Three .............................................................. 565 Table 8.D.64 2009 New Conversions for Mathematics, Grades Four and Five ................................................................ 566 Table 8.D.65 2009 New Conversions for Mathematics, Grades Six and Seven ............................................................... 567 Page x