1. Finding Coherence
Focus, Measure, & Connect
The Measure Sessions!
January 20th, 2015
http://digitallearningforallnow.com
http://www.slideshare.net/jpcostasr
costa@educationconnection.org
Jonathan P. Costa
2.
3. Three Principles of Coherence
Measure
what you
value
Value
what you
measure
Priority
Student
Learning
Priority
Adult
Learning
Priority
Systems
Learning
Student
Learning
Adult
Learning
Systems
Learning
MeasureFocus Connections
4. Coherence Pathways
G = Goals P = Practices M= Measures Jonathan P. Costa S= Students A = Adults O = District/Building
Mission
To develop in all
children the
knowledge, skills,
attitudes and
values...
Theory of Action
Focus
Measure
Connect
7. Align Assessments With Goals for Learning
Appropriate
Assessment
Instruments
Recall & recognition
based quizzes, tests,
and activities. Multiple
choice, matching, fill
In the blanks, etc.
(SAT/AP/Exams)
Checklists,
analytic rubrics,
or other agreed upon
skill standards
(AP/Smarter B/Exams)
Holistic and,
analytic rubrics,
or other agreed upon
skill standards
(Portfolios, Demonstrations,
Exhibitions, Etc)
Content
(Declarative)
Facts
“Define this term”
Content Skills
(Procedural)
Discrete Skills
“Factor this equation”
Meta Skills
(Contextual)
Applied Understandings
“Skilled Information Consumer”
Type of
Knowledge
Desired
Appropriate
Assessment
Standards/Scales
Take this quiz, test,
or other appropriate
content focused
assessment instrument.
Answering/responding to
classroom or text based
problems, experiments, and
writing assignments.
Complex projects,
real time explorations,
authentic problem based
skill applications.
Appropriate
Assessment
Tasks
Percentages of
right and wrong
answers on an agreed
upon numeric or letter
grade scale.
1-100/A-B-C-D
Specific criteria tied
to an agreed upon
numeric or
descriptive scale.
1-100/Pass-Fail
Insufficient-Excellent
Specific criteria tied
to an agreed upon
descriptive scale.
Insufficient-Sufficient
Proficient-Excellent
Overlap between columns can and does occur.
8. Reflections on Assessment…
Other data points: Required, by grade, department, standards aligned, type of administration, time required, testing window, vendor, ROI
9. Reflections on Assessment…
To be Effective, Formative or Summative Assessments Should Be:
1. Generally Reliable (as opposed to statistically so)
2. Valid (reflects what it is intended to measure)
3. Helps the learner improve his/her performance (formative mostly)
4. Can be tracked and analyzed over time (summative mostly)
Building Technical Capacity
• Constructing effective rubrics – measuring what you value
• Scoring protocols – build capacity to judge student work
As a General Rule:
• The goal is the reduce the scoring range – to increase the reliability of the feedback
• Transparency with the standards – sharing standards with the learners and providing them with
effective feedback to improve performance.
11. Pareto Is Not a New Idea
I served with General Washington in the Virginia Legislature and
with Dr. Franklin in Congress. I never heard either of them speak
to any but the main point that was to decide the question. They laid
their shoulders to the great points, knowing that the little ones would
follow themselves.
Thomas Jefferson
12. The Biggest Challenge
Getting together and building
the structures from which to
give students the feedback they
need to ensure success and
accountability in these priority
skill areas is the hardest part of
this work.
Please write 5 words that
describe this winter’s weather in
Connecticut.
13. Keep this shift in mind…
- From -
View Judge Explain
- To -
Establish a reliable standard; then…
View Analyze Discuss
14. It’s our natural instinct…
Think back to the five words I asked you to
write to DESCRIBE the winter weather in
Connecticut…
15. Pay close attention to the language.
Judgment - Descriptive
“reliable” vs. “corroborated by at least
one other source”
“effective” “1st person narrative”
To help this shift…
16. Narrowing the Range
• As a general rule, descriptive language is
preferable to judgmental language for the
purpose of narrowing the range of scores.
• The more judgmental the language, the
greater the time investment required for
calibration and the more dependent on
personnel the process.
• The goal is find the balance time – descriptive
enough to effectively communicate without
being too restrictive or unrealistic.
17. Getting Some Descriptive Practice
What is it – why is it seen that way?
Element
Spring day…
Turkey sandwich…
Student engagement…
Written summary…
Argument with evidence…
More DescriptiveMore Judgmental
18. Making an Argument Based on Evidence
Item Insufficient Sufficient Proficient Excellent
Stated Position
Rationale
Supporting
Evidence
Closing
Statement
19. Something to
Think About
• There is no shortcut to the amount of time
required to use standards and assessment
effectively.
• You either have to define the standard from
scratch through descriptive language or you
will have to define it through working with
your peers to develop anchor sets/models of
student work.
• The best strategy is probably a combination
of both approaches.
20. Scoring Practice
• Match evidence in the student work to the descriptors in the rubric and
scoring guide.
• Scoring decisions must be based on particular evidence in the student
work. Be able to point to the evidence in the student work that is
relevant to scoring fairly and consistently.
• You must be indifferent to the student – about what is demonstrated
not about what you believe was intended.
