Next Gen. Assessment: Assessing &
Measuring What Matters Most
Jonathan E. Martin
Head of School
St. Gregory College Prep
w...
Not everything that can be counted counts, and
not everything that counts can be counted.
Albert Einstein
Data must inform...
Appreciating Howard Levin, Urban School, San
Francisco:
“We didn’t require data to know that our students were
learning mo...
CAIS ERB
I spent 9 years opposing the california
association’s mandated erb testing, saying it
was an inappropriate requir...
begin with the end in mind
backward design
www.21k12blog.net
Balancing the Scorecard
Our schools will be evaluated and measured,
whether we like it or not, especially with SAT
scores ...
what gets measured gets done
www.21k12blog.net
the measurement is the message
www.21k12blog.net
5 suggested tools
internal
Expanded reporting, particularly with 21st c. skills
Pat Bassett’s Demonstrations of Learning
e...
expanding student reporting
assessing and reporting on desired outcomes
www.21k12blog.net
Sources for skill sets
Partnership for 21st c. Skills:
• creativity,
• critical thinking,
• collaboration,
• communication...
www.21k12blog.net
St. Gregory’s “egg”
• Effort
• Integrity
• Leadership
• Communication skills
– Writing
– Speaking
– Listening
– Digital co...
• Thinking skills
– Inquisitiveness
– Analytic thinking
– Synthetic thinking
– Critical thinking
• Innovation skills
– Cre...
The Egg: a work in progress
Just tacking to report card not successful
enough.
Embedding it instead in advisory, conferenc...
Pat Bassett’s call for “demonstrations of
learning”
Define the skills of your graduates, and required
“exhibitions” or dem...
Examples from Bassett
www.21k12blog.net
•Conduct a fluent conversation in a foreign language about a
piece of writing in t...
Wouldn’t it be great if each school in a
competitive set defined their lists differently,
driven by unique mission, and fa...
What are/ would be your school’s
demonstrations of learning?
Discuss.
www.21k12blog.net
external assessment : measuring what matters
www.21k12blog.net
Compromise independence?
Discuss.
www.21k12blog.net
Does your educational mission include:
Meaningful student engagement in learning?
Supportive school culture and strong com...
Let’s measure mission:
www.21k12blog.net
To improve accountability.
In accreditation
To Boards
To parents
To funders
www.21k12blog.net
For marketing and communications.
www.21k12blog.net
To improve learning in the 21st century!
www.21k12blog.net
NAIS monograph: july 2010
Student Outcomes that Measure the School’s Value Added
“Independent schools are now being challe...
3 assessments featured in nais monograph
hssse
map
cwra
www.21k12blog.net
High School Survey of Student Engagement
HSSSE
www.21k12blog.net
Task: review slides, draw inferences about
learning at St. Gregory.
Offer suggested action items for improved
student enga...
Secretary Duncan’s call for assessment 2.0
www.21k12blog.net
Duncan:
• Almost everywhere I went, I heard people express concern
that the curriculum had narrowed as more educators
“tau...
duncan:
I believe the impact of this next generation of
assessments in the classroom will be
dramatic—and that the new ass...
computer adaptive assessment
www.21k12blog.net
duncan:
One-shot, year-end bubble tests administered on a single day,
too often lead to a dummying down of curriculum and
...
Northwest Evaluation Association
Measurement of Academic Progress
www.21k12blog.net
St. Gregory teachers discuss the MAP
video
www.21k12blog.net
Duncan:
One of the biggest frustrations of teachers with existing assessments is
that they fail to test higher-order reaso...
Council for Aid to Education’s
College and Work Readiness Assessment
CWRA
www.21k12blog.net
St. Gregory students discuss the CWRA
video
www.21k12blog.net
Example Performance Task:
Source: Council for Aid to Education, CWRA Institutional
Report
You advise Pat Williams, the pre...
Documents or Artifacts:
• Newspaper article about the accident
• Federal Accident Report on in-flight breakups in single-e...
Questions:
• Do the available data tend to support or refute
the claim that the type of wing on the SwiftAir
235 leads to ...
Discuss
What skills must students demonstrate to
answer effectively?
www.21k12blog.net
CWRA tasks require that students integrate:
critical thinking,
analytic reasoning,
problem solving,
and written communicat...
50th percentile
performance
College
freshmen
StG freshmen StG seniors
Mean Raw Score 1070 1107 1235
Mean Percentile
(compa...
St. Gregory seniors
Mean Raw Score Points Increase 128
Percentile Increase, on the College
Freshman scale
30 percentile po...
Bringing CWRA style assessment
of higher order thinking skills
to classroom evaluation and instruction.
www.21k12blog.net
performance task assessment
It is an empty exercise to assess student learning
without providing a means to adjust teachin...
Real-world scenario
Authentic, complex process
Higher-order thinking
Authentic performance
Transparent evaluation criteria...
Performance task assessment
considering an issue from multiple perspectives
critically examining evidence
valuing claims t...
Q & A
www.21k12blog.net
Thank you.
www.21k12blog.net
Upcoming SlideShare
Loading in …5
×

Next gen. assessments of learning

1,612 views

Published on

Presentation at NEIT 2010

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,612
On SlideShare
0
From Embeds
0
Number of Embeds
1,085
Actions
Shares
0
Downloads
15
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Next gen. assessments of learning

  1. 1. Next Gen. Assessment: Assessing & Measuring What Matters Most Jonathan E. Martin Head of School St. Gregory College Prep www.21k12blog.net
  2. 2. Not everything that can be counted counts, and not everything that counts can be counted. Albert Einstein Data must inform, not replace, judgment. www.21k12blog.net
  3. 3. Appreciating Howard Levin, Urban School, San Francisco: “We didn’t require data to know that our students were learning more and more powerfully with laptops” www.21k12blog.net
  4. 4. CAIS ERB I spent 9 years opposing the california association’s mandated erb testing, saying it was an inappropriate requirement for independent schools. Instead, I argued, in CAIS, let’s be the nation’s leader in assessment excellence, more broadly defined. www.21k12blog.net
  5. 5. begin with the end in mind backward design www.21k12blog.net
  6. 6. Balancing the Scorecard Our schools will be evaluated and measured, whether we like it or not, especially with SAT scores and college lists. We need to take the initiative, inside schools, to broaden the measurements and balance the scorecard. www.21k12blog.net
  7. 7. what gets measured gets done www.21k12blog.net
  8. 8. the measurement is the message www.21k12blog.net
  9. 9. 5 suggested tools internal Expanded reporting, particularly with 21st c. skills Pat Bassett’s Demonstrations of Learning external High School Survey of Student Engagement (hssse) NWEA’s Measurement of Academic Progress (MAP) College and Work Readiness Assessment (CWRA) www.21k12blog.net
  10. 10. expanding student reporting assessing and reporting on desired outcomes www.21k12blog.net
  11. 11. Sources for skill sets Partnership for 21st c. Skills: • creativity, • critical thinking, • collaboration, • communication Tony Wagner’s Seven Survival Skills NAIS commission’s list of essential 21st c. capacities www.21k12blog.net
  12. 12. www.21k12blog.net
  13. 13. St. Gregory’s “egg” • Effort • Integrity • Leadership • Communication skills – Writing – Speaking – Listening – Digital communications www.21k12blog.net
  14. 14. • Thinking skills – Inquisitiveness – Analytic thinking – Synthetic thinking – Critical thinking • Innovation skills – Creativity and ingenuity – Adaptability – Initiative – Experimentation and risk-taking • Complex, real-world, problem-solving www.21k12blog.net
  15. 15. The Egg: a work in progress Just tacking to report card not successful enough. Embedding it instead in advisory, conferencing, goal-setting, and self-assessment. www.21k12blog.net
  16. 16. Pat Bassett’s call for “demonstrations of learning” Define the skills of your graduates, and required “exhibitions” or demonstrations of those skills for graduation. www.21k12blog.net
  17. 17. Examples from Bassett www.21k12blog.net •Conduct a fluent conversation in a foreign language about a piece of writing in that language. •Write a cogent and persuasive opinion piece on a matter of public importance. •Declaim with passion and from memory a passage that is meaningful — of one’s own or from the culture’s literature or history. •Produce or perform a work of art.
  18. 18. Wouldn’t it be great if each school in a competitive set defined their lists differently, driven by unique mission, and families chose schools by their defined outcomes (and demonstrated success at those outcomes?) www.21k12blog.net
  19. 19. What are/ would be your school’s demonstrations of learning? Discuss. www.21k12blog.net
  20. 20. external assessment : measuring what matters www.21k12blog.net
  21. 21. Compromise independence? Discuss. www.21k12blog.net
  22. 22. Does your educational mission include: Meaningful student engagement in learning? Supportive school culture and strong community? Personalized learning for breadth of student abilities in core subjects? Development of higher order thinking skills, and effective problem-solving skills? www.21k12blog.net
  23. 23. Let’s measure mission: www.21k12blog.net
  24. 24. To improve accountability. In accreditation To Boards To parents To funders www.21k12blog.net
  25. 25. For marketing and communications. www.21k12blog.net
  26. 26. To improve learning in the 21st century! www.21k12blog.net
  27. 27. NAIS monograph: july 2010 Student Outcomes that Measure the School’s Value Added “Independent schools are now being challenged to show how their educational model generates what it does and how it is worth the investment. Consider using a tool or tools that best fits your school, mission, and community, knowing that the primary purpose for doing so is institutional assessment and improvement. You will be stronger and more financially viable for doing so.” www.21k12blog.net
  28. 28. 3 assessments featured in nais monograph hssse map cwra www.21k12blog.net
  29. 29. High School Survey of Student Engagement HSSSE www.21k12blog.net
  30. 30. Task: review slides, draw inferences about learning at St. Gregory. Offer suggested action items for improved student engagement at St. Gregory based on data. www.21k12blog.net
  31. 31. Secretary Duncan’s call for assessment 2.0 www.21k12blog.net
  32. 32. Duncan: • Almost everywhere I went, I heard people express concern that the curriculum had narrowed as more educators “taught to the test. Existing state assessments in mathematics and English often fail to capture the full spectrum of what students know and can do. Students, parents, and educators know there is more to a sound education than picking the right selection for a multiple choice question. • It’s for all these reasons that shortly after taking office, President Obama called on the nation’s governors and state education chiefs “to develop standards and assessments that don’t simply measure whether students can fill in a bubble on a test, but whether they possess 21st century skills like problem-solving and critical thinking and entrepreneurship and creativity.“ www.21k12blog.net
  33. 33. duncan: I believe the impact of this next generation of assessments in the classroom will be dramatic—and that the new assessments will support learning and instructional practices that teachers have long hungered for themselves. www.21k12blog.net
  34. 34. computer adaptive assessment www.21k12blog.net
  35. 35. duncan: One-shot, year-end bubble tests administered on a single day, too often lead to a dummying down of curriculum and instruction throughout the course of the entire school year. In short, most of the assessment done in schools today is after the fact and designed to indicate only whether students have learned. Not enough is being done to assess students’ thinking as they learn to boost and enrich learning, and track student growth. For the first time, state assessments will make widespread use of smart technology. They will provide students with realistic, complex performance tasks, immediate feedback, computer adaptive testing, and incorporate accommodations for a range of students. www.21k12blog.net
  36. 36. Northwest Evaluation Association Measurement of Academic Progress www.21k12blog.net
  37. 37. St. Gregory teachers discuss the MAP video www.21k12blog.net
  38. 38. Duncan: One of the biggest frustrations of teachers with existing assessments is that they fail to test higher-order reasoning and writing skills, and thus fail to show what students know and can do. For the first time, many teachers will have the state assessments they have longed for– tests of critical thinking skills and complex student learning that are not just fill-in-the-bubble tests of basic skills but support good teaching in the classroom. The new assessments will better measure the higher-order thinking skills so vital to success in the global economy of the 21st century and the future of American prosperity. To be on track today for college and careers, students need to show that they can analyze and solve complex problems, communicate clearly, synthesize information, apply knowledge, and generalize learning to other settings. www.21k12blog.net
  39. 39. Council for Aid to Education’s College and Work Readiness Assessment CWRA www.21k12blog.net
  40. 40. St. Gregory students discuss the CWRA video www.21k12blog.net
  41. 41. Example Performance Task: Source: Council for Aid to Education, CWRA Institutional Report You advise Pat Williams, the president of DynaTech, a company that makes precision electronic instruments and navigational equipment. Sally Evans, a member of DynaTech’s sales force, recommended that DynaTech buy a small private plane (a SwiftAir 235) that she and other members of the sales force could use to visit customers. Pat was about to approve the purchase when there was an accident involving a SwiftAir 235. Your document library contains the following materials: www.21k12blog.net
  42. 42. Documents or Artifacts: • Newspaper article about the accident • Federal Accident Report on in-flight breakups in single-engine planes • Internal Correspondence (Pat’s e-mail to you and Sally’s e-mail to Pat) • Charts relating to SwiftAir’s performance characteristics • Excerpt from magazine article comparing SwiftAir 235 to similar planes • Pictures and descriptions of SwiftAir Models 180 and 235 www.21k12blog.net
  43. 43. Questions: • Do the available data tend to support or refute the claim that the type of wing on the SwiftAir 235 leads to more in-flight breakups? • What is the basis for your conclusion? • What other factors might have contributed to the accident and should be taken into account? • What is your preliminary recommendation about whether or not DynaTech should buy the plane and what is the basis for this recommendation? www.21k12blog.net
  44. 44. Discuss What skills must students demonstrate to answer effectively? www.21k12blog.net
  45. 45. CWRA tasks require that students integrate: critical thinking, analytic reasoning, problem solving, and written communication skills. The holistic integration of these skills on the CWRA tasks mirrors the requirements of serious thinking and writing tasks faced in life outside of the classroom. www.21k12blog.net
  46. 46. 50th percentile performance College freshmen StG freshmen StG seniors Mean Raw Score 1070 1107 1235 Mean Percentile (compared to college freshmen) 50 [by definition] 67 97 www.21k12blog.net
  47. 47. St. Gregory seniors Mean Raw Score Points Increase 128 Percentile Increase, on the College Freshman scale 30 percentile points (67th to 97th) Median Standard Deviation Increase .92 (compared to .51 at other high schools) www.21k12blog.net
  48. 48. Bringing CWRA style assessment of higher order thinking skills to classroom evaluation and instruction. www.21k12blog.net
  49. 49. performance task assessment It is an empty exercise to assess student learning without providing a means to adjust teaching in response to deficiencies revealed through the information gleaned from that assessments. Marc Chun: Taking Teaching to the (Performance) Task. www.21k12blog.net
  50. 50. Real-world scenario Authentic, complex process Higher-order thinking Authentic performance Transparent evaluation criteria www.21k12blog.net
  51. 51. Performance task assessment considering an issue from multiple perspectives critically examining evidence valuing claims that are backed by appropriate and adequate evidence, reasoning objectively and dispassionately arriving at informed judgments and decisions. www.21k12blog.net
  52. 52. Q & A www.21k12blog.net
  53. 53. Thank you. www.21k12blog.net

×