Video<br />Assessment 2.0: <br />New & Next-Gen Measurements of Learning<br />Jonathan Martin<br />Head of School<br />St....
“How do we measure what we value rather than value what we measure?”<br />
How do we ensure we don’t constrain our teachers? <br />
What about Students?<br />
Less Actually, More Uncertainty<br />
Commission on  Accreditation Criterion 13<br />The standards require a school to provide <br />evidence of a thoughtful pr...
Balance the Portfolio<br />
 Demonstrations of Learning: 	“What you do, not what you know, is the ultimate test of education.”  ~PFB Tweet<br />Conduc...
Character and 21st c. skills report cards <br />www.21k12blog.net<br />
St. Gregory’s “egg”<br />Character<br />Integrity <br />Leadership<br />Positive role model<br />Facilitating Collaboratio...
High Tech High Digital Portfolios<br />New Tech Network Digital Portfolios<br />Digital Portfolios<br />
New and Next Gen. External Measurements of learning<br />
The questions<br />Does the assessment measure what we value?<br />Does it constrain creativity, narrow curriculum, or res...
In “Crime Reduction” students act as a consultant to Mayor Pat Stone who is running for reelection. <br />As Mayor Stone’s...
Crime Statistics<br />Research Brief<br />
Critical thinking<br />Analytic reasoning<br />Problem solving<br />Written communication<br />
Three Metrics<br />Internal growth at a school<br />Comparison to other participating schools<br />College readiness<br />
http://www.youtube.com/watch?v=Y4qgDXUrc4w<br />
CWRA: the good and the bad<br />Great Goals: A test worth teaching to; measuring what we value.<br />Does Not Constrain or...
Note<br />Experiment with developing internally designed, administered, and assessed performance task assessments.  <br />...
Measures of Academic Progress™(MAP)<br />
Traditional/Conventional Standardized Testing Updated<br />Computer Adaptive Assessment<br />More frequent (3x) than once ...
Sample Math Question<br /><ul><li>Notice the calculator.
To use it, click the numbers with your mouse.</li></li></ul><li>Sample Reading Question<br /><ul><li>Read the passage and ...
Click the <Go on> button</li></li></ul><li>Measures what we value?   Yes and no.   More basic skills than higher order thi...
What Excites & Engages Me?<br />
Measures what we value. <br />Does Not Narrow or Constrain<br />Not painful for students, but does fatigue in subsequent a...
student, teacher and parent surveys collect data and generate reports focused on school climate, classroom conditions, and...
Tripod Seven Cs Sample Survey Items<br />Control<br />Challenge<br /><ul><li>Students in this class treat the teacher with...
My classmates behave the way the teacher wants them to
Our class stays busy and doesn’t waste time
My teacher asks students to explain more about the answers they give.
My teacher doesn’t let people give up when the work gets hard.
In this class, we learn to correct our mistakes.
My teacher wants us to use our thinking skills, not just memorize things.</li></ul>Care<br />Control<br />Clarify<br />Cha...
STUDENTACHIEVEMENT OUTCOMES<br />The Seven C’s<br />What Teachers Do (What Students Experience)<br />Caring about students...
Agreement with selected statements by students in 25th and 75th percentile classrooms from the MET secondary school sample...
Beyond test scores: Healthy dispositions and experiencing a high quality of life in school<br /><ul><li>Happiness in class
Good behavior  in class
Health vs. unhealthy responses to social pressures </li></ul>   (e.g., holding back, hiding effort, help avoidance, etc.)<...
Academic engagement/effort
Satisfaction with performance and progress</li></ul>46<br />
What Works in the Classroom? Ask the Students<br />Teachers whose students described them as skillful at maintaining class...
Example: Happiness Index<br /> This class is a happy place for me to be.<br /> This class feels like a happy family.<br />...
Tripod Thoughts<br />Measures What We Value<br />Does not narrow. <br />Not painful for students, but a large task to admi...
Information Literacy: “To be information literate, a person must be able to recognize when information is needed and have ...
Define<br />Access<br />Digital Fluency:Skillful use of information via technology<br />Evaluate<br />Manage<br />Integrat...
The ETS iSkills™ Assessment<br />Features<br /><ul><li>   Interactive simulations – not multiple choice
   Vendor neutral
   14 real-world, problem-solving scenarios</li></ul> Copyright © 2011 by Educational Testing Service. All rights reserve...
Measuring How We Integrate<br />Task Length: 4-minute duration<br />Task Type Integrate: Interpret and represent informati...
In this task, examinees summarize information from a variety of sources and then draw conclusions from their summary.<br />
Examinees are presented with an information need and three different types of information sources.<br />
After filling in the table, examinees must interpret the summary to rank the three sources correctly.<br />
58<br />Institutional Skill Area Report<br />
59<br />Confidential and Proprietary. Copyright ©2006 Educational Testing Service.<br />Aggregate Task Performance Feedba...
Upcoming SlideShare
Loading in …5
×

Assessment 2.0 for Canadian Accredited Independent Schools Heads

4,483 views

Published on

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

Assessment 2.0 for Canadian Accredited Independent Schools Heads

  1. 1. Video<br />Assessment 2.0: <br />New & Next-Gen Measurements of Learning<br />Jonathan Martin<br />Head of School<br />St. Gregory College Prep<br />Tucson, AZ<br />www.21k12blog.net<br />Video<br />
  2. 2.
  3. 3. “How do we measure what we value rather than value what we measure?”<br />
  4. 4. How do we ensure we don’t constrain our teachers? <br />
  5. 5. What about Students?<br />
  6. 6.
  7. 7.
  8. 8. Less Actually, More Uncertainty<br />
  9. 9. Commission on Accreditation Criterion 13<br />The standards require a school to provide <br />evidence of a thoughtful process, respectful <br />of its mission, for the collection and use <br />in school decision-making of data (both <br />external and internal) about student learning<br />
  10. 10.
  11. 11. Balance the Portfolio<br />
  12. 12.
  13. 13. Demonstrations of Learning: “What you do, not what you know, is the ultimate test of education.” ~PFB Tweet<br />Conduct a fluent conversation in a foreign language about of piece of writing in that language.<br />Declaim with passion and from memory a passage that is meaningful, of one’s own or from the culture’s literature or history.<br />Invent a machine or program a robot capable of performing a difficult physical task.<br />
  14. 14. Character and 21st c. skills report cards <br />www.21k12blog.net<br />
  15. 15. St. Gregory’s “egg”<br />Character<br />Integrity <br />Leadership<br />Positive role model<br />Facilitating Collaboration<br />Scholarship<br />Inquisitiveness<br />Analytic thinking<br />Synthetic thinking<br />Critical thinking<br />Innovation  <br />Adaptability  <br />Initiative   <br />Experimentation<br />www.21k12blog.net<br />
  16. 16.
  17. 17. High Tech High Digital Portfolios<br />New Tech Network Digital Portfolios<br />Digital Portfolios<br />
  18. 18. New and Next Gen. External Measurements of learning<br />
  19. 19. The questions<br />Does the assessment measure what we value?<br />Does it constrain creativity, narrow curriculum, or restrict teacher latitude?<br />Are our students meaningfully engaged in the experience of the assessment? <br />Does it provide data we can use for accountability, communications, and continuous improvement of learning? <br />Is it affordable? <br />
  20. 20.
  21. 21. In “Crime Reduction” students act as a consultant to Mayor Pat Stone who is running for reelection. <br />As Mayor Stone’s consultant, students are asked to weigh evidence to analyze the strengths and weaknesses of two policy initiatives. <br />
  22. 22. Crime Statistics<br />Research Brief<br />
  23. 23.
  24. 24. Critical thinking<br />Analytic reasoning<br />Problem solving<br />Written communication<br />
  25. 25.
  26. 26.
  27. 27. Three Metrics<br />Internal growth at a school<br />Comparison to other participating schools<br />College readiness<br />
  28. 28.
  29. 29. http://www.youtube.com/watch?v=Y4qgDXUrc4w<br />
  30. 30. CWRA: the good and the bad<br />Great Goals: A test worth teaching to; measuring what we value.<br />Does Not Constrain or Narrow Teaching or Diminish creativity & innovation. <br />Authentic assessment: Positive testing experience for students. <br />Student motivation can be an issue. <br />Results: Good for communications and accountability, not detailed enough for continuous improvement or student differentiation. <br />Pricy<br />
  31. 31. Note<br />Experiment with developing internally designed, administered, and assessed performance task assessments. <br />Some schools and districts are finding this enormously valuable for teachers, such as Virginia Beach. <br />CLA in the Classroom provides workshops on this.<br />
  32. 32. Measures of Academic Progress™(MAP)<br />
  33. 33. Traditional/Conventional Standardized Testing Updated<br />Computer Adaptive Assessment<br />More frequent (3x) than once a year<br />More rapid feedback to teachers<br />More detailed analysis of student proficiency<br />
  34. 34. Sample Math Question<br /><ul><li>Notice the calculator.
  35. 35. To use it, click the numbers with your mouse.</li></li></ul><li>Sample Reading Question<br /><ul><li>Read the passage and select the best answer
  36. 36. Click the <Go on> button</li></li></ul><li>Measures what we value? Yes and no. More basic skills than higher order thinking skills. <br />It can narrow teaching and curriculum, perhaps diminish creativity and innovation, by its emphases. <br />Test does conform itself to each student’s individual proficiency level. <br />Student experience of testing, 3x a year, certainly wears on kids. <br />Results are very comprehensive and detailed, and can be very useful for continuous improvement and differentiation, but big learning curve. <br />Moderately priced. <br />
  37. 37.
  38. 38.
  39. 39.
  40. 40. What Excites & Engages Me?<br />
  41. 41. Measures what we value. <br />Does Not Narrow or Constrain<br />Not painful for students, but does fatigue in subsequent administrations. <br />Results useful for accountability and communications, but limited in value for continuous improvement. <br />Highly affordable.<br />Middle School Survey (MSSSE) coming soon. <br />
  42. 42. student, teacher and parent surveys collect data and generate reports focused on school climate, classroom conditions, and student engagement<br />42<br />
  43. 43. Tripod Seven Cs Sample Survey Items<br />Control<br />Challenge<br /><ul><li>Students in this class treat the teacher with respect
  44. 44. My classmates behave the way the teacher wants them to
  45. 45. Our class stays busy and doesn’t waste time
  46. 46. My teacher asks students to explain more about the answers they give.
  47. 47. My teacher doesn’t let people give up when the work gets hard.
  48. 48. In this class, we learn to correct our mistakes.
  49. 49. My teacher wants us to use our thinking skills, not just memorize things.</li></ul>Care<br />Control<br />Clarify<br />Challenge<br />Captivate<br />Confer<br />Consolidate<br />
  50. 50. STUDENTACHIEVEMENT OUTCOMES<br />The Seven C’s<br />What Teachers Do (What Students Experience)<br />Caring about students (Encouragement and Support)<br />Captivating students (Learning seems Interesting and Relevant<br />Conferring with students (Students Sense their Ideas are Respected)<br />Controlling behavior (Culture of Cooperation and Peer Support)<br />Clarifying lessons (Success Seems Feasible)<br />Challenging students (Press for Effort, Perseverance and Rigor)<br />Consolidating knowledge (Ideas get Connected and Integrated)<br />Student Engagement Targets<br />(Both Individual and Collective)<br />Trust (vs. Mistrust)<br />Cooperation (vs. Resistance)<br />Ambitiousness (vs. Ambivalence)<br />Diligence (vs. Discouragement or Disengagement)<br />Satisfaction and Efficacy (vs. Disappointment and Failure)<br /> Teacher Professional Learning (PLCs)<br />Content Knowledge<br /> Pedagogic Skill Relationship-Building Skills<br />The Tripod<br />
  51. 51. Agreement with selected statements by students in 25th and 75th percentile classrooms from the MET secondary school sample. ( From among 2985 classrooms, each with at least 5 students responding) <br />
  52. 52. Beyond test scores: Healthy dispositions and experiencing a high quality of life in school<br /><ul><li>Happiness in class
  53. 53. Good behavior in class
  54. 54. Health vs. unhealthy responses to social pressures </li></ul> (e.g., holding back, hiding effort, help avoidance, etc.)<br /><ul><li>Self-consciousness about being perceived as smart or not by peers
  55. 55. Academic engagement/effort
  56. 56. Satisfaction with performance and progress</li></ul>46<br />
  57. 57. What Works in the Classroom? Ask the Students<br />Teachers whose students described them as skillful at maintaining classroom order, at focusing their instruction and at helping their charges learn from their mistakes are often the same teachers whose students learn the most in the course of a year, as measured by gains on standardized test scores, according to a progress report on the research.<br />
  58. 58. Example: Happiness Index<br /> This class is a happy place for me to be.<br /> This class feels like a happy family.<br /> I feel respected in this class.<br /> Being in this class makes me feel angry.<br />The next slide shows the relationship to the Seven C’s for this measure of happiness. <br />48<br />
  59. 59.
  60. 60. Tripod Thoughts<br />Measures What We Value<br />Does not narrow. <br />Not painful for students, but a large task to administer surveys. <br />Seems it can provide valuable data for both communications and continuous improvement. <br />Pricing varies.<br />
  61. 61. Information Literacy: “To be information literate, a person must be able to recognize when information is needed and have the ability to locate, evaluate and use effectively the needed information …” (ACRL, 1989)<br />Digital Fluency= information literacy + digital environments<br />
  62. 62. Define<br />Access<br />Digital Fluency:Skillful use of information via technology<br />Evaluate<br />Manage<br />Integrate<br />Create<br />Communicate<br /> Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS). <br />
  63. 63. The ETS iSkills™ Assessment<br />Features<br /><ul><li> Interactive simulations – not multiple choice
  64. 64. Vendor neutral
  65. 65. 14 real-world, problem-solving scenarios</li></ul> Copyright © 2011 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are all registered trademarks of Educational Testing Service (ETS). <br />
  66. 66. Measuring How We Integrate<br />Task Length: 4-minute duration<br />Task Type Integrate: Interpret and represent information in an ICT context, including synthesizing, summarizing, comparing, and contrasting information from multiple sources.<br />Within the task, the test taker is asked to…<br />Summarize information from three different types of sources<br />Compare the information to reach a conclusion<br />
  67. 67. In this task, examinees summarize information from a variety of sources and then draw conclusions from their summary.<br />
  68. 68. Examinees are presented with an information need and three different types of information sources.<br />
  69. 69. After filling in the table, examinees must interpret the summary to rank the three sources correctly.<br />
  70. 70. 58<br />Institutional Skill Area Report<br />
  71. 71. 59<br />Confidential and Proprietary. Copyright ©2006 Educational Testing Service.<br />Aggregate Task Performance Feedback Report<br />
  72. 72. Measures what we value? Open question. <br />Narrows teaching: No<br />Experience for students: Challenging, but the online dynamic may be engaging for some. <br />Detailed data seems very useful for communications and continuous improvement. <br />Price moderate.<br />
  73. 73. PISA-Based Testing for Schools<br />Have you ever wanted to compare your school’s effectiveness with the world’s best national educational systems?<br />
  74. 74. EXAMPLE: International comparisons of your school’s performance: PISA 2009 Countries and Economies<br />Section II. Your School’s Results in an international context<br />School ABCD<br />Reading: 512<br />Mathematics: 513<br />Science: 507<br />
  75. 75. EXAMPLE: Your school’s performance in the context of PISA results for Canada (with socio-economic background)<br />Section II. Your school’s results in an international context<br />Studentperformance<br />School ABCD<br />Reading: 512<br />School performance and schools’ socio-economic background<br />Private school<br /> Public school in rural area<br /> Public school in urban area<br />Student performance and students’ socio-economic background withinschools<br />700<br />Your School<br />493<br />Index calculated from student questionnaires<br />300<br />PISA Index of socio-economic background<br />Advantage<br />Disadvantage<br />
  76. 76. Blood donation is essential.<br />There is no product that can fully substitute for human blood. Blood donation is thus irreplaceable and essential to save lives. <br />In France, each year, 500,000 patients benefit from a blood transfusion.<br />The instruments for taking the blood are sterile and single-use (syringe, tubes, bags).<br />There is no risk in giving your blood.<br />Blood donation:<br />It is the best-known kind of donation, and takes from 45 minutes to 1 hour.<br />A 450-ml bag is taken as well as some small samples on which tests and checks will be done.<br />- A man can give his blood five times a year, a woman three times. <br />- Donors can be from 18 to 65 years old.<br />An 8-week interval is compulsory between each donation.<br />BLOOD DONATION Notice<br />
  77. 77. Question 1<br />Level 2 item – 81.2% of students across OECD can perform tasks at least at this level<br />
  78. 78. Proposed Structure of School Reports (Preliminary)<br />Contents of print-ready reports will include:<br />Understanding your school’s results from the PISA-Based Test for Schools assessment<br />Your school’s results in an international context<br />Additional insights from international PISA results<br />Annexes<br />Summary of PISA-Based Test for Schools assessment questions (items, units and response-types)<br />Summary Description of PISA Assessment Frameworks<br />Will review option for schools to electronically explore their confidential results online<br />
  79. 79. PISA-Based Test for Schools Pilot:<br />is...<br />An assessment tool to support international benchmarking efforts of schools and school networks<br />Meant as a tool to assess competencies, skills and application of knowledge of 15-year-old students - comparable to international PISA results<br />Based on the same assessment frameworks as the main PISA assessment<br />To be available by choice and ‘on-demand’ <br />A tool that links schools and practitioners to comparative and international policy discussions <br />is not...<br />A mandated standardised assessment <br />Meant as a substitute or alternative to the main PISA assessment<br />Meant to inform day-to-day instruction of teachers<br />Meant to align 100% with country-specific standards or curriculum (i.e. “Common Core” although considerable alignment is expected <br />Meant as a formative evaluation tool<br />
  80. 80. PISA-Based Test for Schools Pilot and Post-Pilot<br />
  81. 81. Find out more about thePISA-Based Test for Schools:<br />pisabasedtestforschools@OECD.org<br />It is important to distinguish clearly between the PISA-Based Test for Schools pilot and the main PISA 2012 assessment also taking place in the following months. The two assessments are separate and independent exercises with different goals.<br />The next cycle of the main PISA assessment will occur in 2012. Results for participating countries and economies are expected to be published in December 2013. Schools cannot participate in both the main 2012 PISA assessment and the pilot round of PISA-Based Test for Schools.Participating in the national sample of the main PISA 2012 study should be the priority.<br />
  82. 82. PISA Based Testing for Schools<br />The good and the bad: <br />Measures what we value: application of knowledge. <br />Don‘t think it constrains. <br />Great general data for communications and accountability, but maybe not enough detailed reporting. <br />Student experience: unsure. <br />May be expensive. <br />
  83. 83. www.21k12blog.net<br />Twitter: @JonathanEMartin<br />

×