Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

CAAVES Universal Design In Assessment

1,023 views

Published on

Applying Principles of Universal Design to Assessment Item Modification
by Peter A. Beddow III
Vanderbilt University
Nashville, TN
June 2008

Published in: Education, Design, Technology
  • Be the first to comment

  • Be the first to like this

CAAVES Universal Design In Assessment

  1. 1. Applying Principles of Universal Design to Assessment Item Modification Peter A. Beddow III Vanderbilt University Nashville, TN June 2008
  2. 2. Background <ul><li>Definition of Universal Design: </li></ul><ul><ul><li>The design of products and environments to be usable by all people, to the greatest extent possible, without the need for adaptation or specialized design. </li></ul></ul><ul><ul><ul><li>(Center for Universal Design, NC State University, 2008; http://www.design.ncsu.edu/cud/about_ud/udprincipleshtmlformat.html#top) </li></ul></ul></ul>
  3. 3. Universal Design (cont.) <ul><li>Equitable use </li></ul><ul><li>Flexibility in use </li></ul><ul><li>Simple and intuitive </li></ul><ul><li>Perceptible information </li></ul><ul><li>Tolerance for error </li></ul><ul><li>Low physical effort </li></ul><ul><li>Size and space for approach and use </li></ul><ul><ul><li>(Center for Universal Design, 2008) </li></ul></ul>
  4. 4. Universal Design (cont.) <ul><li>Principles of Universal Design are intended to maintain equity among user groups while providing the same or equivalent means of use for all users; </li></ul><ul><li>Ideally, services, products, or systems employing universal design principles: </li></ul><ul><ul><li>Eliminate unecessary complexity; </li></ul></ul><ul><ul><li>Arrange information consistent with its importance; </li></ul></ul><ul><ul><li>Use prompting and feedback during and after task completion; </li></ul></ul><ul><ul><li>Present information redundantly (visual, verbal, tactile); </li></ul></ul><ul><ul><li>Maximize legibility and construct; </li></ul></ul><ul><ul><li>Simplify directions and instructions; </li></ul></ul><ul><ul><li>Reduce the potential for error. </li></ul></ul>
  5. 5. Item Accessibility and Modification Guide An Instrument for Designing and Modifying Assessment Items
  6. 6. Purpose of the Instrument <ul><li>The Item Accessibility and Modification Guide (Beddow, Kettler, & Elliott, 2007) was designed to facilitate the analysis of test item features to enhance meaningful responses from, and provide equal access for, all students; </li></ul><ul><li>The instrument can be used: </li></ul><ul><ul><li>To aid in examination and modification of existing items; </li></ul></ul><ul><ul><li>To influence the design of new items; or </li></ul></ul><ul><ul><li>As a training tool for item writers. </li></ul></ul>
  7. 7. Influence <ul><li>The Item Accessibility and Modification Guide was directly influenced by National Center for Educational Outcomes publications, specifically: </li></ul><ul><ul><li>NCEO’s Policy Directions 18 (Johnstone, Thurlow, Moore, & Altman, 2006) which provides expert review recommendations for test item. </li></ul></ul><ul><ul><li>Section 508 guidelines. </li></ul></ul><ul><li>To date there has been no effort to provide a rating instrument to assist in the consideration of universally designed assessment items. </li></ul>
  8. 8. Organization <ul><li>The Item Accessibility and Modification Guide consists of two sections: </li></ul><ul><ul><li>Considerations for Universally Designed Assessment Items (7 categories, 38 items ); </li></ul></ul><ul><ul><li>Computer-based Tests: Considerations for Universally Designed Items ( 5 categories, 26 items ). </li></ul></ul>
  9. 9. Procedure <ul><li>The rater is asked to record the item being reviewed, his/her name, and a brief description of the construct the item is intended to measure. </li></ul><ul><li>For each accessibility item, the rater is asked to check Yes or No (or N/A if the question is not applicable to the target item). </li></ul><ul><li>For each category, the rater is asked to circle the number that best represents his/her summative evaluation of the item’s accessibility for that category based on checklist responses: </li></ul><ul><ul><li>0 = The item is Not Accessible </li></ul></ul><ul><ul><li>1 = The item is Minimally Accessible </li></ul></ul><ul><ul><li>2 = The item is Moderately Accessible </li></ul></ul><ul><ul><li>3 = The item is Maximally Accessible </li></ul></ul><ul><li>The Total Item Accessibility Rating is found by totalling overall evaluation ratings for the item across categories. </li></ul><ul><ul><li>It should be noted that the overall rating is intended as a simple way to categorize items by high, medium, or low accessibility, as opposed to providing a definitive indicator of accessibility that is based on research data. </li></ul></ul>
  10. 10. Considerations for Universally Designed Assessment Items <ul><li>7 categories: </li></ul><ul><ul><li>Content Relevance (2 items); </li></ul></ul><ul><ul><li>Text Content (10 items); </li></ul></ul><ul><ul><li>Text Economy/Conciseness (3 items); </li></ul></ul><ul><ul><li>Text Appearance (7 items); </li></ul></ul><ul><ul><li>Visual Content (if applicable; 7 items); </li></ul></ul><ul><ul><li>Fairness for Subgroups (8 items); and </li></ul></ul><ul><ul><li>Format Flexibility (5 items). </li></ul></ul>
  11. 11. Considerations for Universally Designed Assessment Items
  12. 12. Considerations for Computer-based Test Items <ul><li>5 categories: </li></ul><ul><ul><li>Layout and Design (6 items); </li></ul></ul><ul><ul><li>Navigation (5 items); </li></ul></ul><ul><ul><li>Screen Reader Considerations (5 items); </li></ul></ul><ul><ul><li>Test Specific Options (4 items); and </li></ul></ul><ul><ul><li>Computer Capabilities (6 items). </li></ul></ul>
  13. 13. Considerations for Computer-based Test Items
  14. 14. The Item Accessibility and Modification Guide <ul><li>The instrument is available electronically at: </li></ul><ul><ul><li>http://www.vanderbilt.edu/lsi/caavespage/Item_Accessibility_and_Modification_Guide.pdf </li></ul></ul><ul><li>Manuscript: </li></ul><ul><ul><li>http://www.vanderbilt.edu/lsi/caavespage/Item_Accessibility_and_Modification_Guide_Manuscript.pdf </li></ul></ul>
  15. 15. Thanks for listening. Peter A. Beddow III [email_address] CAAVES Item Mod Data 2008

×