2. 2
Abstract
• Evidence-based item development approaches require
item authors to make explicit how items provide validity
evidence to support claims.
• Advances in technology require increased consideration
of accessibility and interoperability during item authoring.
• Refined item development processes are needed to
meet these requirements.
3. 3
Why now?
• Market demand for transparency
• Evidence requirements articulated by CCSS
assessment consortia
• Technical advances in assessment (e.g. CAT, diagnostic
models)
• Interoperability requirements (accessibility, tagging)
• Increased need for cost-effective development
6. 6
CTB CADDS
• Serves as a bridge between standards and assessment
• Provides opportunities for systematic unpacking of
standards for item development
• Documents steps in reaching assessment goals
• Maintains focus at all steps on evidence to support
interpretations and uses of test information
7. 7
CTB CADDS
• Details information about the content to be assessed
• Describes item types that will be used
• Detailed specifications for each item/task
– cognitive tasks and processes
– proficiency level targets
• Increased direction for item writers
8. 8
CADDS
1. Define the intended inferences and decisions to be
based on test scores.
2. Define the achievement construct.
3. Draft performance level descriptors (expectations of
students).
4. Define the evidence to be elicited by the item pool.
9. 9
CADDS
5. Complete item writer assignments to meet the item pool
specification requirements.
6. Complete item creation (authoring and editing).
7. Field test assessment items and tasks.
8. Implement the operational test.
11. 11
CTB CADDS
1. Define intended inferences and uses of the assessment
data.
– Adoption of CCSS has created need for transition assessments.
– Test designs are incorporating new item types.
– Reporting requirements are changing.
– “Alignment” issues abound.
12. 12
CTB CADDS
2. Define the test construct(s) that will become
assessment targets.
– What cognitive tasks are required by the standard?
– How do we consider students progression of learning?
– What is the instructional context for a given standard?
– How do we incorporate performance levels?
13. 13
CTB CADDS
3. Develop initial proficiency level descriptors to guide
development and interpretation of test scores.
– Identify source of PLDS.
– Define the role of PLDs in item specifications and authoring.
14. 14
CTB CADDS
4. Define the evidence.
– Item pool or test blueprint
– Specifications
– Item/task templates
– Instructions to item writers
15. 15
CTB CADDS
5. Develop items and performance tasks based on
specifications.
– specifications templates
– cognitive task frameworks
– tagging for accessibility and interoperability
16. 16
CTB CADDS
6. Refine items and tasks through collaborative review by
stakeholders.
7. Field test items and tasks in appropriate small- or large-
scale settings.
8. Implement the operational test and continue the design
and specification validation process.
22. 22
Specifications Elements
• Rules for source materials
• Rules for item or task problem
• Rules for response requirements
• Administration requirements
• Accessibility requirements
• Administrator directions
• Technology requirements
23. 23
Item Authoring
• Clearly defined elements for authors
– construct evidence requirements
– parameters for student responses
– options for item type, cognitive demand, other variable factors
• Articulated focus on evidence, validity and accessibility
25. 25
Item Development Implications
• Explicit articulation of evidence to support claims
• Rigorous specifications development
• More direction for item writers
• Clearer distinction among item attributes
• Refinement of item development plans
27. 27
Element Specification
Performance description Students will demonstrate their ability to evaluate texts in
multiple presentations and compare their presentations of
ideas.
Performance Level
Descriptors
Proficient, possibly Advanced
Grade level placement 6
Assessment targets CCSS 6.RI-7. Integrate information presented in different
media or formats (e.g., visually, quantitatively) as well as in
words to develop a coherent understanding of a topic or
issue.
CCSS 6.RI-2. Determine a central idea of a text and how it is
conveyed through particular details
Cognitive Rigor Analyze and Evaluate (Bloom)
DOK Level 3: Strategic Thinking and Reasoning (Webb)
28. 28
Element Specification
Problem/processing type ELA/Literacy Claim 1: Students can read closely and critically
to comprehend a range of increasingly complex literary and
informational texts.
Central ideas: summarize central ideas
Analyze relationships: analyze how presentation format
reveals author interpretation of a topic/idea
Analyze or interpret author’s craft
Evaluate text to determine central ideas
Rules for Source
Materials:
Stimulus must meet the grade 6 text specifications.
Text stimuli must be a well-crafted informational text that has
a clear central idea.
Visual or quantitative text should have a connection to the
informational text, such as shared theme, details, or ideas.
Connection does not have to be immediately transparent
(e.g., a text about storms and a photo of a storm), but it
should be discernible by grade 6 students.
Text should be readable in 5-7 minutes.
Visually impaired students will be provided with a description
of the explicit details of the visual stimuli, but the description
will not present judgments about what the visual may
suggest
29. 29
Element Specification
Rules for item or task
problem
The item will focus on how each medium presents similar
information and will require an analysis of both texts.
The visual stimulus will be presented before the item stem,
and students will be directed to, “Look at the [stimulus].”
The stem will explicitly direct students to consider both texts.
The stem will pose a question for students to consider
regarding both texts.
All parts of the item must use grade-appropriate language.
Rules for response
requirements
The distractors will be plausible to students who have
misunderstood or misinterpreted the texts. A distractor may
oRelate to one text only
oMisrepresent the central idea of one or both texts
All answer choices must use grade-appropriate language.
30. 30
Which statement best captures an idea that the reader can infer from both the
photograph and President Obama’s speech?
a. It is important to stay focused to reach the top.
b. To do well, becoming physically fit is necessary.
c. When in doubt, seeking help is a sign of strength.
d. It takes effort and determination to achieve goals.
31. 31
Look at the table Claire made to organize quadrilaterals by their attributes.
