Digital Learning

1. DEVELOPING HIGH QUALITY
ASSESSMENT ITEMS
Region 7 Digital Learning Academy

2. Dr. LaTricia Townsend
Research Scholar
Educational Evaluation
NCSU Friday Institute
@tricia_townsend

3. HTTP://REGION7DLA.WIKISPACES.COM/

4. LOOKING BACK
o Activity-Table Share: Use of Session
1 Information Reflection
⚫How have you used information
gained from Session 1 in your
practice?
⚫How have you shared the
information from Session 1 in your
school or district?

5. LOOKING FORWARD
Take the poll about how you obtain
assessment items

6. ITEM DEVELOPMENT PROCESS
Curriculum Standards

7. ALIGNMENT
Curriculum
Standards
Depth and
Type
Thinking

8. ACCESS CURRICULAR DOCUMENTS
Activity- Take a moment to locate
curriculum standards materials
Common Core
⚫ ELA and Math
Essential Standards
Next Generation Science Standards
Unpacking documents completed by
school/district
Curriculum Maps/Pacing Guides
Begin to think about which ones you want
to address today

9. REVISED BLOOM’S TAXONOMY-KNOWLEDGE
PROCESS DIMENSION
 FACTUAL KNOWLEDGE-basic elements students must
know to be acquainted with a discipline or solve problems in
it
 CONCEPTUAL KNOWLEDGE-The interrelationships among
the basic elements within a larger structure that enable them
to function together
 PROCEDURAL KNOWLEDGE-How to do
something, methods of inquiry, and criteria for using
skills, algorithms, techniques, and methods
 METACOGNITIVE KNOWLEDGE-Knowledge of cognition in
general as well as awareness and knowledge of one's own
cognition

10. REVISED BLOOM’S TAXONOMY-COGNITIVE
PROCESS DIMENSION
 Revised Bloom’s Taxonomy According to Homer
Simpson

11. WEBB’S DEPTH OF KNOWLEDGE
 Level 1: Recall
⚫ Recall a fact, information, or procedure
 Level 2: Skill/Concept
⚫ Use information or conceptual knowledge, two or
more steps
 Level 3: Strategic Thinking
⚫ Requires reasoning, developing a plan or a
sequence of steps, some complexity
 Level 4: Extended Thinking
⚫ Requires an investigation, time to think and
process multiple conditions of the problem.

13. PRACTICE ALIGNMENT
o Activity-Write items (open ended- that align
to your content area and Revised Bloom’s
Taxonomy/Webb’s Depth of Knowledge)
• Write four questions to address each of
the four levels.
• Record your responses using the tool of
your choice for your own records.
• Share your responses on the Google
Spreadsheet

14. PRACTICE ALIGNMENT
o Activity-Find a partner who teaches in the same
grade span in a different district. Share your
questions and discuss how you might approach
the content differently.
o Reflect #R7DLA

15. MULTIPLE CHOICE ITEM GUIDELINES
Content
General
Format
Structure
•Stem
•Foil
Accessibility
•SWD and LEP
Bias/Sensitivity

16. MULTIPLE CHOICE TERMINOLOGY
A. bluebird
B. cardinal
C. robin
D. sparrow
Which is the state bird of North Stem
Carolina?
Distractor
Foils
Distractor
Key
Distractor

17. MULTIPLE CHOICE GUIDELINES-CONTENT
 Base items on curriculum standards and
DOK.
 Keep the testing vocabulary consistent with
grade level.
 Measure a single
concept, principle, procedure, or
competency.
 Write items with a mixture of difficulty levels-
Easy, Medium, Hard

18. MULTIPLE CHOICE GUIDELINES-CONTENT
 Choose to assess important information, not trivial
material.
POOR ITEM
How many different elements make up the Noble Gases?
A. 3
B. 4
C. 5
D. 6
IMPROVED ITEM
What common property is shared by all the noble gases?
A. They are highly reactive and must be stored in air-tight containers.
B. They are inert, and do not easily form chemical compounds.
C. They are metallic in nature, and have basic oxides.
D. When mixed with water, they form a toxic blue-green solution.

19. MULTIPLE CHOICE GUIDELINES-GENERAL
 Test the content, not the test-taking or reading ability of
students-avoid tricky or misleading items.
 Avoid errors of grammar, abbreviations, punctuation and
spelling.
 Do not use contractions.
 Do not use first or second person.
 Do not use humor.

20. MULTIPLE CHOICE GUIDELINES-FORMAT
 Write clear, concise instructions.
 Format foils vertically.
 Use space in a balanced manner to produce an easy to
follow format

21. MULTIPLE CHOICE GUIDELINES-STRUCTURE
 Avoid writing complex format multiple-choice items.
Which of the following are fruits?
1. Tomatoes
2. Tomatillos
3. Habanero peppers
A. 1 & 2
B. 2 & 3
C. 1 & 3
D. 1, 2, & 3

22. MULTIPLE CHOICE GUIDELINES-STRUCTURE
 Do not use fill-in-the blank format. Pose a question.
 Be brief . Wordiness in the stem can clue or confuse the student. (“window-
dressing”)
POOR ITEM
High temperatures and heavy rainfall characterize a humid climate. People who live in this kind of
climate usually complain of problems dealing with heavy amounts of perspiration. Even moderately warm
days are uncomfortable for most people in this climate. Which biome is being described?
A. savanna
B. taiga
C. tundra
D. tropical rainforest
IMPROVED ITEM
Which term describes a biome with high temperatures and heavy rainfall?
A. savanna
B. taiga
C. tundra
D. tropical rainforest

23. MULTIPLE CHOICE GUIDELINES-STEM
 Word the stems positively. Do not use any negative
phrasing, such as words NOT or EXCEPT.
 Do not write stems that ask for or express opinions.
 Write the items so that the central idea and any common
elements are included in the stem instead of the foils.
 Place the interrogative as close to the foils as possible.
 Use qualifying words in a stem, when needed, to
emphasize the “best answer.” Words such as
best, most likely, and most appropriate are bold and
italicized.

24. MULTIPLE CHOICE GUIDELINES- FOIL
DEVELOPMENT
 Each item must contain four foils (A,B,C,D)
 Order the answer choices in a logical order, such as
ascending or descending value for numbers, the
order in which the words appear in a
chart, chronological order, etc.
 Each item written should contain foils that are
independent and not overlapping.
 To the extent possible, foils for an item should be
homogeneous in content and length.

25. MULTIPLE CHOICE GUIDELINES- FOIL
DEVELOPMENT
 Do not use the following as foils: all of the
above, none of the above, I don’t know.
 Word the foils positively, avoid any negative
phrasing.
 Avoid providing clues to the correct
response.
 Avoid writing items in which phrases in the
stem are repeated in the foils.

26. MULTIPLE CHOICE GUIDELINES- FOIL
DEVELOPMENT
 Avoid absolutes such as ALWAYS, NEVER,
TOTALLY, and ABSOLUTELY should not be
used when writing items.
 Qualifiers such as best, most likely,
approximately, etc. should be bold and italic.
 Present one and only one best (correct)
answer.
 Use the common errors made by students as
distractors.

27. MULTIPLE CHOICE GUIDELINES- FOIL
DEVELOPMENT
 Use plausible distractors. The best
(correct) answer must clearly be the best
(correct) answer and the incorrect
responses must clearly be inferior to the
best (correct) answer. No distractor
should be obviously wrong.
POOR ITEM
Which example best describes the adhesion
property of water?
A. drops of water on the outside of a glass
B. sugar dissolving in water
C. ice floating in water
D. melting of aluminum

28. MULTIPLE CHOICE GUIDELINES- FOIL
DEVELOPMENT
Use plausible distractors.
IMPROVED ITEM
Which example best describes the adhesion
property of water?
A. drops of water on the outside of a glass
B. sugar dissolving in water
C. ice floating in water
D. water changing from a liquid to a gas

29. MULTIPLE CHOICE GUIDELINES-UNIVERSAL
DESIGN
 Make items accessible and usable by students of all
abilities, including students with disabilities and students
with limited English proficiency.
 Use simple language and sentence structure.’
 Do not use colloquial language.

30. MULTIPLE CHOICE GUIDELINES-UNIVERSAL
DESIGN
 Limit the use of homonyms, homophones, and
homographs
fair
1.Adjective. Meaning lovely or beautiful, when describing
someone
 2.Adjective. Meaning of light complexion, or light colored (blonde)
hair
 3.Adjective. Meaning reasonable, equitable, free from self-interest
 3. Noun. Meaning a gathering of people to sell and barter
goods, or to enjoy entertainment, and also a travelling show of
rides, stalls and sideshows

31. SENSITIVITY AND BIAS
 Do not use stereotypes based on gender, race,
religion, socio-economic status, or other factors.
 Respect diversity by including people from all walks
of life in use of names and artwork.
 Ensure items do not highlight socio-economic
disparities.
 Avoid espousing religious and political beliefs.

32. REVISING AND WRITING QUESTIONS
o Activity-- Would you change any of the
questions that you posed, initially? Make any
necessary changes.
o Create 3 Multiple Choice Questions- Record the
questions using the same tool you used before
and share your responses on the Google
Spreadsheet
o Reflect #R7DLA

33. SESSION 2 PRODUCT IDEA BRAINSTORM
 Activity- Table Talk-With your team, discuss what
your product for School Net will be for Session 2.
What is the proposed product? How can you align
the Session 2 product with your Session 1 product?
How will it be disseminated in your school/district?
 Share your ideas to wiki Padlet.

34. ASSESSING ITEMS
o Activity- Review Questions
o Find a trio
o Use the item review document to gauge the quality of
each of the items
o What changes would you make to the items?
Reflect #R7DLA

35. AFTERNOON WORK TASKS
 Explore online resources
 Develop Session 2 Resource
 Reflect #R7DLA

36. ONLINE RESOURCES/SCHOOL NET
 Activity- Explore the resources presented. How
could these be used in your instructional delivery or
assessment program? Are there other resources
you would like to share? Share your ideas to
Padlet.

37. SESSION 2 PRODUCT
 Activity-Work with your district team to develop the
product. In which ways can you connect Session 1
and Session 2? How will you disseminate the
information to your school/district to build capacity?
 Complete the School Net upload form

38. HTTP://REGION7DLA.WIKISPACES.COM/