Assessing Change in
Changing Times
Justin Reich
Tom Daccord
EdTechTeacher.org
M.A.S.S. Webinar Series
Where are we going?
• Preparing students for success in career,
college and citizenship
• Nurturing 21st century skills
– Expert Thinking
– Complex Communication
– New Media Literacy
Managing System Change
towards 21st century skills
1. Change has to be backwards planned
(and logic models are the tool)
2. Change has to happen in the instructional
core (led by/with teacher-leaders)
3. 21st century skills will primarily be
evaluated through performances of
understanding (so rubrics will be central)
Backwards Planning
Select learning goals
What do you want students to be able to do?
Design assessment tasks
How will students demonstrate their developing mastery of those goals?
Develop lesson activities
How will you prepare students to master the goals and
succeed on the assessment task?
Logic Model
Learning Goals
Performance
Assessments
Skill
Benchmarks
Learning
Activities
Instructional
Support
Causality
Design
WHAT DO YOU WANT
STUDENTS TO BE ABLE TO
DO?
Logic Model
Learning Goals
New Media/
Technology
Literacy
Expert Thinking
Complex
Communication
Logic Model
Learning Goals
New Media/
Technology
Literacy
Information/
Search Literacy
Productivity
Software
Persuasive
Multimedia
Communication
Multimodal
Literacy
Digital
Citizenship/
Internet Safety
Logic Model
Learning Goals
New Media/
Technology
Literacy
Information/
Search Literacy
Logic Model
Learning Goals
New Media/
Technology
Literacy
Information/
Search Literacy
Identifying
information
needs
Assessing
Sources
Finding
Sources
Crediting
Sources
Using
Sources
Logic Model
Learning Goals
New Media/
Technology
Literacy
Information/
Search Literacy
Performance
Assessments
Who is on your team?
• District vs. building?
• Asst. Sup. for curriculum and instruction?
• Principals? Asst. Principal for academics?
• Academic Technology? Library/media?
• Curriculum specialists?
• Teacher-leaders!
HOW WILL STUDENTS
DEMONSTRATE THEIR
DEVELOPING MASTERY OF
THOSE GOALS?
Logic Model
Learning Goals
New Media/
Technology
Literacy
Information/
Search Literacy
Performance
Assessments
Logic Model
Learning Goals
New Media/
Technology
Literacy
Information/
Search Literacy
Performance
Assessments
Science Fair
Project
Geography
Country
Presentation
Historical
Artist
Portfolio
ELA Personal
History
Project
Logic Model
Learning Goals
New Media/
Technology
Literacy
Information/
Search Literacy
Performance
Assessments
Science Fair
Project
Geography
Country
Presentation
Historical
Artist
Portfolio
ELA Personal
History
Project
Research
Journal
Annotated
Bibliography
Common
Elements
Logic Model
Learning Goals
New Media/
Technology
Literacy
Information/
Search Literacy
Performance
Assessments
Science Fair
Project
Geography
Country
Presentation
Historical
Artist
Portfolio
ELA Personal
History
Project
Research
Journal
Annotated
Bibliography
Common
Elements
Skill
Benchmarks
•Identify source
needs
•Find and use
credible sources
•Use multiple
source types
•Appropriate
crediting/citing
Building/District Rubric Co-Creation
• Collect examples and exemplars from
throughout the building
• Collect best practices online (Classroom
2.0 ning, Rubistar.4teachers.org)
• Collect and analyze student work
• Instructional Rounds (Elmore, City, Teitel) focused
on information literacy instruction
• Pilot, Pilot, Pilot!
Skill
Benchmarks
Exceeds
Standard (4)
Meets
Standard (3)
Approaching
Standard (2)
Getting
Started (1)
•Identify
source needs
Research
journal shows
original/creative
search strategy
and keywords
RJ shows
effective search
strategy and
diverse
keywords
RJ shows basic
search strategy
and some
keywords
RJ missing
search strategy
or keywords
Find and use
credible
sources
Annotated
Bibliography
shows
thoughtful
evaluation of
credibility
An Bib shows
many credible
sites, with some
evaluation of
credibility
AnBib shows
some credible
sites, with
limited
evaluation of
credibility
AnBib shows a
few well chosen
sites, but many
sites lack
credibility or no
evaluation
•Use multiple
source types
AnBib includes
great diversity
of source types
AnBib includes
multiple source
types
AnBib includes
two types of
sources
AnBib includes
one type of
source
Appropriate
crediting/citing
Complex
sources cited
correctly
Sources cited
correctly in MLA
format
Sources cited in
MLA format with
errors
Sources
credited without
format
Using Data to Address Instruction
Identify a
Problem of
Practice
Gather data
about the
problem
Examine
Instruction
Develop an
Action Plan
Act and
Assess
Further Reading: Data Wise and Data Wise in Action
HOW WILL YOU PREPARE
STUDENTS TO MASTER THE
GOALS AND SUCCEED ON
THE ASSESSMENT TASK?
Logic Model
Learning Goals
New Media/
Technology
Literacy
Information/
Search Literacy
Performance
Assessments
Science Fair
Project
Geography
Country
Presentation
Historical
Artist
Portfolio
ELA Personal
History
Project
Skill
Benchmarks
•Identify source
needs
•Find and use
credible sources
•Use multiple
source types
•Appropriate
crediting/citing
Learning
Activities
•Identifying search
keywords
•Advanced search
engine strategies
•Search engines
vs. search
directories
•Web site
evaluation
•Proactive search
strategies
•Citation and
crediting
•Subject specific
skills
Curriculum Mapping
• Where are these lesson activities taught?
• What necessary skills are not taught
anywhere?
• Where should they be taught?
• Are we teaching skills in similar ways?
Logic Model
Performance
Assessments
Science Fair
Project
Geography
Country
Presentation
Historical
Artist
Portfolio
ELA Personal
History
Project
Skill
Benchmarks
•Identify source
needs
•Find and use
credible sources
•Use multiple
source types
•Appropriate
crediting/citing
Learning
Activities
•Identifying search
keywords
•Advanced search
engine strategies
•Search engines
vs. search
directories
•Web site
evaluation
•Proactive search
strategies
•Citation and
crediting
•Subject specific
skills
Instructional
Support
•District rubric
creation
•District data
analysis
•Technology
access
• Common
planning time
•Professional
Development
•Coaching
•Model projects
•Model lessons
•Observational
protocols
•Curriculum
mapping
Information/
Search Literacy
Using Data to Address Instruction
Identify a
Problem of
Practice
Gather data
about the
problem
Examine
Instruction
Develop an
Action Plan
Act and
Assess
Further Reading: Data Wise and Data Wise in Action
Learning Goals
Performance
Assessments
Skill
Benchmarks
Learning
Activities
Instructional
Support
Leveraging Technology for
Assessment
• As your teaching/learning activities go
online, it becomes easier to
share/aggregate data
– Publish student work projects
– Publish teaching materials
– Use electronic grading
Assessing Change in
Changing Times
• Plan backwards from goals with logic models
• Measure change the only place it matters: in
the instructional core
• Use rubrics to gather quantitative data about
performance assessments
• Use that assessment data to drive decisions
about learning activity and instructional
support
• “It’s the hardest work I’ve ever done in my
career. We’re trying to effect change at
scale, and we have to ‘play on two playing
fields’ at once. We’re still being judged by
the criteria for AYP and state
accountability, while holding ourselves to a
much higher standard. We have to
succeed at both. It’s hard, but it’s the right
work to be doing.” –Jim Merrill, Virginia Beach (quoted in
Tony Wagner’s Global Achievement Gap)
Visit us at EdTechTeacher.org!
justin@edtechteacher.org
tom@edtechteacher.org
Cycle of Experiment and Experience
Fear - Growth+
Institutional Capacity+
Experiment
Review
(Experience)Plan

Assessing Change in Changing Time

  • 1.
    Assessing Change in ChangingTimes Justin Reich Tom Daccord EdTechTeacher.org M.A.S.S. Webinar Series
  • 3.
    Where are wegoing? • Preparing students for success in career, college and citizenship • Nurturing 21st century skills – Expert Thinking – Complex Communication – New Media Literacy
  • 5.
    Managing System Change towards21st century skills 1. Change has to be backwards planned (and logic models are the tool) 2. Change has to happen in the instructional core (led by/with teacher-leaders) 3. 21st century skills will primarily be evaluated through performances of understanding (so rubrics will be central)
  • 6.
    Backwards Planning Select learninggoals What do you want students to be able to do? Design assessment tasks How will students demonstrate their developing mastery of those goals? Develop lesson activities How will you prepare students to master the goals and succeed on the assessment task?
  • 8.
  • 9.
    WHAT DO YOUWANT STUDENTS TO BE ABLE TO DO?
  • 10.
    Logic Model Learning Goals NewMedia/ Technology Literacy Expert Thinking Complex Communication
  • 11.
    Logic Model Learning Goals NewMedia/ Technology Literacy Information/ Search Literacy Productivity Software Persuasive Multimedia Communication Multimodal Literacy Digital Citizenship/ Internet Safety
  • 12.
    Logic Model Learning Goals NewMedia/ Technology Literacy Information/ Search Literacy
  • 13.
    Logic Model Learning Goals NewMedia/ Technology Literacy Information/ Search Literacy Identifying information needs Assessing Sources Finding Sources Crediting Sources Using Sources
  • 14.
    Logic Model Learning Goals NewMedia/ Technology Literacy Information/ Search Literacy Performance Assessments
  • 15.
    Who is onyour team? • District vs. building? • Asst. Sup. for curriculum and instruction? • Principals? Asst. Principal for academics? • Academic Technology? Library/media? • Curriculum specialists? • Teacher-leaders!
  • 16.
    HOW WILL STUDENTS DEMONSTRATETHEIR DEVELOPING MASTERY OF THOSE GOALS?
  • 17.
    Logic Model Learning Goals NewMedia/ Technology Literacy Information/ Search Literacy Performance Assessments
  • 18.
    Logic Model Learning Goals NewMedia/ Technology Literacy Information/ Search Literacy Performance Assessments Science Fair Project Geography Country Presentation Historical Artist Portfolio ELA Personal History Project
  • 19.
    Logic Model Learning Goals NewMedia/ Technology Literacy Information/ Search Literacy Performance Assessments Science Fair Project Geography Country Presentation Historical Artist Portfolio ELA Personal History Project Research Journal Annotated Bibliography Common Elements
  • 20.
    Logic Model Learning Goals NewMedia/ Technology Literacy Information/ Search Literacy Performance Assessments Science Fair Project Geography Country Presentation Historical Artist Portfolio ELA Personal History Project Research Journal Annotated Bibliography Common Elements Skill Benchmarks •Identify source needs •Find and use credible sources •Use multiple source types •Appropriate crediting/citing
  • 21.
    Building/District Rubric Co-Creation •Collect examples and exemplars from throughout the building • Collect best practices online (Classroom 2.0 ning, Rubistar.4teachers.org) • Collect and analyze student work • Instructional Rounds (Elmore, City, Teitel) focused on information literacy instruction • Pilot, Pilot, Pilot!
  • 22.
    Skill Benchmarks Exceeds Standard (4) Meets Standard (3) Approaching Standard(2) Getting Started (1) •Identify source needs Research journal shows original/creative search strategy and keywords RJ shows effective search strategy and diverse keywords RJ shows basic search strategy and some keywords RJ missing search strategy or keywords Find and use credible sources Annotated Bibliography shows thoughtful evaluation of credibility An Bib shows many credible sites, with some evaluation of credibility AnBib shows some credible sites, with limited evaluation of credibility AnBib shows a few well chosen sites, but many sites lack credibility or no evaluation •Use multiple source types AnBib includes great diversity of source types AnBib includes multiple source types AnBib includes two types of sources AnBib includes one type of source Appropriate crediting/citing Complex sources cited correctly Sources cited correctly in MLA format Sources cited in MLA format with errors Sources credited without format
  • 23.
    Using Data toAddress Instruction Identify a Problem of Practice Gather data about the problem Examine Instruction Develop an Action Plan Act and Assess Further Reading: Data Wise and Data Wise in Action
  • 24.
    HOW WILL YOUPREPARE STUDENTS TO MASTER THE GOALS AND SUCCEED ON THE ASSESSMENT TASK?
  • 25.
    Logic Model Learning Goals NewMedia/ Technology Literacy Information/ Search Literacy Performance Assessments Science Fair Project Geography Country Presentation Historical Artist Portfolio ELA Personal History Project Skill Benchmarks •Identify source needs •Find and use credible sources •Use multiple source types •Appropriate crediting/citing Learning Activities •Identifying search keywords •Advanced search engine strategies •Search engines vs. search directories •Web site evaluation •Proactive search strategies •Citation and crediting •Subject specific skills
  • 26.
    Curriculum Mapping • Whereare these lesson activities taught? • What necessary skills are not taught anywhere? • Where should they be taught? • Are we teaching skills in similar ways?
  • 27.
    Logic Model Performance Assessments Science Fair Project Geography Country Presentation Historical Artist Portfolio ELAPersonal History Project Skill Benchmarks •Identify source needs •Find and use credible sources •Use multiple source types •Appropriate crediting/citing Learning Activities •Identifying search keywords •Advanced search engine strategies •Search engines vs. search directories •Web site evaluation •Proactive search strategies •Citation and crediting •Subject specific skills Instructional Support •District rubric creation •District data analysis •Technology access • Common planning time •Professional Development •Coaching •Model projects •Model lessons •Observational protocols •Curriculum mapping Information/ Search Literacy
  • 28.
    Using Data toAddress Instruction Identify a Problem of Practice Gather data about the problem Examine Instruction Develop an Action Plan Act and Assess Further Reading: Data Wise and Data Wise in Action Learning Goals Performance Assessments Skill Benchmarks Learning Activities Instructional Support
  • 29.
    Leveraging Technology for Assessment •As your teaching/learning activities go online, it becomes easier to share/aggregate data – Publish student work projects – Publish teaching materials – Use electronic grading
  • 30.
    Assessing Change in ChangingTimes • Plan backwards from goals with logic models • Measure change the only place it matters: in the instructional core • Use rubrics to gather quantitative data about performance assessments • Use that assessment data to drive decisions about learning activity and instructional support
  • 31.
    • “It’s thehardest work I’ve ever done in my career. We’re trying to effect change at scale, and we have to ‘play on two playing fields’ at once. We’re still being judged by the criteria for AYP and state accountability, while holding ourselves to a much higher standard. We have to succeed at both. It’s hard, but it’s the right work to be doing.” –Jim Merrill, Virginia Beach (quoted in Tony Wagner’s Global Achievement Gap)
  • 32.
    Visit us atEdTechTeacher.org! justin@edtechteacher.org tom@edtechteacher.org
  • 33.
    Cycle of Experimentand Experience Fear - Growth+ Institutional Capacity+ Experiment Review (Experience)Plan

Editor's Notes

  • #2 ChrisDede, Learning Technologies as Fire vs. Medicine
  • #5 Once you know where you want to go, you need to evaluate whether or not you are making progress getting there … Yogi Berra, “We’re lost but we’re making great time!”
  • #6 The goal of this session is to be to backwards plan from our goals
  • #9 Draw Arrow
  • #13 Picked this one because I actually think it’s a great place to start thinking about system wide 21st century skill development
  • #14 Talking in Verbs
  • #18 Type some school challenges
  • #19 Nature of 21st C skills is that they are distributed across the subject areas