RBT and DOK Alignment for Practitioners
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

RBT and DOK Alignment for Practitioners

  • 2,752 views
Uploaded on

Staff training clinic on alignment of test items and state standards using both Revised Bloom's Taxonomy and Depth of Knowledge levels, May 2009

Staff training clinic on alignment of test items and state standards using both Revised Bloom's Taxonomy and Depth of Knowledge levels, May 2009

More in: Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
2,752
On Slideshare
2,744
From Embeds
8
Number of Embeds
1

Actions

Shares
Downloads
181
Comments
0
Likes
0

Embeds 8

http://www.linkedin.com 8

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Practical Alignment for Item / Test Developers: Revised Bloom’s Taxonomy (RBT) and Depth of Knowledge Levels (DOK) Staff Trainings, 2009 Jerrie W. Brown, Content Lead for Social Studies Assessments North Carolina State University Center for Urban & Community Services Technical Outreach for Public Schools (TOPS)
  • 2.
    • “ Lack of excellence in American schools is not caused by ineffective teaching, but mostly by misaligning what teachers teach, what they intend to teach, and what they assess as having been taught.”
    • -Prof. S. Alan Cohen, 1987
    • University of San Francisco
    “ Alignment also helps ensure the validity of test results. If standards and tests are not aligned, results might provide a misleading picture of how well students are attaining standards.” -Robert Rothman, 2003 Annenberg Institute for School Reform
  • 3. What is alignment? The No Child Left Behind Act of 2001 (NCLB) requires that each state align assessments to content standards. Once the enacted (instruction), intended (state standards), and assessed (state tests) curricula have been measured, questions can be asked about the extent to which content is similar across them. To the extent content is the same, they are said to be aligned. J. Brown, NCSU, 2009
  • 4. Defining Alignment
    • If the content assessed is exactly the same as the content represented in the standards, alignment is perfect.
    • There are two ways in which alignment can be less than perfect:
      • (1) Content in the standards may not be assessed, and
      • (2) content assessed may not be in the standards.
    J. Brown, NCSU, 2009
  • 5. Alignment Procedure
    • Review alignment criteria of Revised Bloom’s Taxonomy (RBT) and Depth of Knowledge (DOK) levels
    • Rate standards
    • Rate test items
    • Aggregate individual ratings
    • Determine degree of RBT-based alignment
    • Determine degree of DOK-based alignment
    J. Brown, NCSU, 2009
  • 6. Alignment Dimensions
    • DOK – Two Dimensions
    • Depth of Knowledge Levels
    • Content Topics or Strands
    • (Webb, 2002)
    J. Brown, NCSU, 2009
    • RBT – Two Dimensions
    • Cognitive Processes
    • Knowledge Types
    • (Anderson, 2001)
    6. Create 5. Evaluate 4. Analyze 3. Apply 2. Understand 1. Remember Content Topic/Strand A 4. Extended Thinking 3. Strategic Thinking 2. Skill/Concept 1. Recall A. Factual B. Conceptual C. Procedural D. Meta-Cognitive Content Topic/Strand B Content Topic/Strand C (Note: achieves content validity) (Note: achieves content validity and instructional validity)
  • 7. Revised Bloom’s Taxonomy (RBT) Table for _____________________________ (Subject / Grade Level) Adapted from Anderson, Lorin W. and David R. Krathwohl, et al., (2001). J. Brown, NCSU, 2009 Knowledge Dimension (Lower Order = 0% ) Cognitive Process Dimension (Higher Order = 0% ) 1. Remember 2. Understand 3. Apply 4. Analyze 5. Evaluate 6. Create A. Factual Knowledge (0%) B. Conceptual Knowledge (0%) C. Procedural Knowledge (0%) D. Meta- Cognitive Knowledge (0%)
  • 8. Depth of Knowledge (DOK) Levels for _____________________________ (Subject / Grade Level) Adapted from Norm Webb, (2002). J. Brown, NCSU, 2009 Content Categories Depth of Knowledge Levels Categorical Concurrence Yes / No 1. Recall 2. Skill/Concept 3. Strategic Thinking 4. Extended Thinking A. Topic/Strand __________ (Specify) B. Topic/Strand __________ (Specify) C. Topic/Strand __________ (Specify)
  • 9. RBT’s Knowledge Dimension: J. Brown, NCSU, 2009 Major Types Definitions A Factual The basic elements students must know to be acquainted with a discipline or solve problems in it B Conceptual The interrelationships among the basic elements within a larger structure that enable them to function together C Procedural How to do something, methods of inquiry, and criteria for using skills, algorithms, techniques, and methods D Metacognitive Knowledge of cognition in general as well as awareness and knowledge of one’s own cognition
  • 10. RBT’s Knowledge Dimension: J. Brown, NCSU, 2009 Major Types Subtypes A Factual AA. Knowledge of terminology AB. Knowledge of specific details and elements B Conceptual BA. Knowledge of classifications and categories BB. Knowledge of principles and generalizations BC. Knowledge of theories, models, and structures C Procedural CA. Knowledge of subject-specific skills and algorithms CB. Knowledge of subject-specific techniques and methods CC. Knowledge of criteria for determining when to use appropriate procedures D Metacognitive DA. Strategic knowledge DB. Knowledge about cognitive tasks, including appropriate contextual and conditional knowledge DC. Self-knowledge
  • 11. RBT’s Cognitive Process Dimension: J. Brown, NCSU, 2009 Cognitive Process Categories Definitions 1 Remember Retrieve relevant knowledge from long-term memory 2 Understand Construct meaning from instructional messages, including oral, written, and graphic communication 3 Apply Carry out or use a procedure in a given situation 4 Analyze Break material down into its constituent parts and determine how the parts relate to one another and to an overall structure or purpose 5 Evaluate Make judgments based on criteria and standards 6 Create Put elements together to form a coherent or functional whole; reorganize elements into a new pattern or structure
  • 12. RBT’s Cognitive Process Dimension: J. Brown, NCSU, 2009 Cognitive Process Categories Cognitive Processes 1 Remember 1.1 Recognizing 1.2 Recalling 2 Understand 2.1 Interpreting 2.2 Exemplifying 2.3 Classifying 2.4 Summarizing 2.5 Inferring 2.6 Comparing 2.7 Explaining 3 Apply 3.1 Executing 3.2 Implementing 4 Analyze 4.1 Differentiating 4.2 Organizing 4.3 Attributing 5 Evaluate 5.1 Checking 5.2 Critiquing 6 Create 6.1 Generating 6.2 Planning 6.3 Producing
  • 13.
    • Norm Webb’s Specific Alignment Criteria
    • Content Focus
      • A. Categorical Concurrence: between goals and assessment is met if the same or consistent categories of content appear in both documents
      • B. Depth-of-Knowledge Consistency: at least 50% of the items corresponding to a goal had to be at or above the level of knowledge of the goal
      • C. Range-of-Knowledge Correspondence: fifty percent of the objectives for a goal must have at least one related assessment item
      • D. Balance of Representation: the degree to which one objective is given more emphasis on the assessment than another
    Formal Alignment Studies and Depth of Knowledge Levels (DOKs) J. Brown, NCSU, 2009
  • 14. Depth of Knowledge Levels
    • Norm Webb (2002) Generic DOKs
    • Level 1: Recall
      • Recall of a fact, information, or procedure.
    • Level 2: Skill/Concept
      • Use information or conceptual knowledge, two or more steps, etc.
    • Level 3: Strategic Thinking
      • Requires reasoning, developing plan or a sequence of steps, some complexity, more than one possible answer.
    • Level 4: Extended Thinking
      • Requires an investigation, time to think and process multiple conditions of the problem.
    J. Brown, NCSU, 2009
  • 15. Subject-Specific DOKs J. Brown, NCSU, 2009 Science Social Studies Math Level 1 Recall and Reproduction Recall of Information Recall Level 2 Skills and Concepts Basic Reasoning Skill/ Concept Level 3 Strategic Thinking Complex Reasoning Strategic Thinking Level 4 Extended Thinking Extended Reasoning Extended Thinking
  • 16. Reading DOKs J. Brown, NCSU, 2009
    • Level 1 :
    • Support ideas by reference to details in the text.
    • Use a dictionary to find the meaning of words.
    • Identify figurative language in a reading passage.
    • Level 2:
    • Use context cues to identify the meaning of unfamiliar words.
    • Predict a logical outcome based on information in a reading selection.
    • Identify and summarize the major events in a narrative.
    • Level 3:
    • Determine the author’s purpose and describe how it affects the interpretation of a reading selection.
    • Summarize information from multiple sources to address a specific topic.
    • Analyze and describe the characteristics of various types of literature.
    • Level 4:
    • Analyze and synthesize information from multiple sources.
    • Examine and explain alternative perspectives across a variety of sources.
    • Describe and illustrate how common themes are found across texts from different cultures.
  • 17. Getting Started
    • Resources needed:
      • Knowledge types (a-d)
      • Cognitive processes (1-6)
      • Depth of knowledge levels (1-4)
      • Standards ratings forms
      • Test items ratings forms
      • RBT blank taxonomy tables
    J. Brown, NCSU, 2009
  • 18. Rating WA State Standards J. Brown, NCSU, 2009
  • 19. Plotting Standards J. Brown, NCSU, 2009 Knowledge Dimension (Lower Order = 0% ) Cognitive Process Dimension (Higher Order = 0% ) 1. Remember 2. Understand 3. Apply 4. Analyze 5. Evaluate 6. Create A. Factual Knowledge (0%) 10 1, 5 9 B. Conceptual Knowledge (0%) 2, 3 8 6 C. Procedural Knowledge (0%) 7 4 D. Meta- Cognitive Knowledge (0%)
  • 20. Rating ‘WASL’ Test Items J. Brown, NCSU, 2009
  • 21. Plotting Test Items J. Brown, NCSU, 2009 Knowledge Dimension (Lower Order = 0% ) Cognitive Process Dimension (Higher Order = 0% ) 1. Remember 2. Understand 3. Apply 4. Analyze 5. Evaluate 6. Create A. Factual Knowledge (0%) 10 1, 5 9 B. Conceptual Knowledge (0%) 2, 3 8 6 C. Procedural Knowledge (0%) 7 4 D. Meta- Cognitive Knowledge (0%)
  • 22. Individual and Group Results
  • 23. Thank You for Participating! FOR MORE INFORMATION : Mr. Jerrie W. Brown, Consultant North Carolina State University Center for Urban and Community Services Technical Outreach for Public Schools 1500 Blue Ridge Road Raleigh, N. Carolina 27607 Tel: (919) 515-1125 Email: [email_address] Website: http://www.cuacs.ncsu.edu