Assessing Change in Changing Times


Published on

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Assessing Change in Changing Times

  1. 1. Did we change? Justin Reich Doctoral Student, Harvard Graduate School of Education Co-Director, @bjfr Blog:
  2. 2. Using Data to Address Instruction Identify a Problem of Practice Gather data about the problem Examine Instruction Develop an Action Plan Act and Assess Further Reading: Data Wise and Data Wise in Action
  3. 3. The lower school at the American School of Brussels is working on increasing connections between home and school and having parents more aware of and involved in children’s classroom experiences. They are requiring all elementary teachers to create blogs to help build these connections. How could you assess, short term and long term, whether this initiative is working? Scenario 1
  4. 4. How would you test French Fry Quality at McDonalds?
  5. 5. The Fakename School District has a Wordpress install for classroom blogging. Teachers have been asked to consider Will Richardson’s definition of blogging, emphasizing student autonomy, creativity, and attentiveness to a real audience Bud Hunt wanted to write his MA thesis about Scenario 2
  6. 6. What did Bud do? • He took a sample of all of his school’s blog posts • He read them • He categorized them • He drew some conclusions about the kinds of learning taking place based on that analysis
  7. 7. American School of Doha has a 6-12 one-to- one program. They are implementing a 1-1 program in the fifth grade. The tech directors want to assess the efficacy of their intervention. What questions do you have? What could we do? Scenario 3
  8. 8. What does awesome look like?
  9. 9. The Newmarket High School has just purchased 300 iPads to allow them to pilot a 1 to 1 program in the ninth grade. Teachers have been asked to create “21st century learning environments,” and they can use the iPads however they wish. How could you assess whether this pilot should expand to the entire high school? Scenario 4
  10. 10. Using Data to Address Instruction Identify a Problem of Practice Gather data about the problem Examine Instruction Develop an Action Plan Act and Assess Further Reading: Data Wise and Data Wise in Action
  11. 11. Six Steps for Research 1. Ask a Question 2. Define Indicators 3. Identify a Sample 4. Identify a Method 5. Gather Data 6. Analyze the Results
  12. 12. Ask a Question Good questions ask about how technology changes people’s experience of learning. Two helpful prompts: What is the learning goal before technology? Is technology supposed to change the learning goal?
  13. 13. Define Indicators • Patrick Larkin said he wants students to be more engaged. What are indicators of engagement? • Chris Lehmann said he wants students to be able to ask good questions. What are indicators that students ask good questions?
  14. 14. Identify a Sample • Whole Population • Convenience Sample- whoever you can get • Purposive Sample- a few, well-chosen people • Random sample- often the best method
  15. 15. Identify a Method • Online Survey • Interview (phone or in person) • Classroom observation (online or in person) • Teaching Materials Review (online or in print) • Looking at Student Work (online or in print) • Web Analytics • Test Score Analysis
  16. 16. Gather Data • Structured Data Collection Allows for Structured Data Analysis – Ask all of your random sample of parents the same four questions, and you can compare their answers • Unstructured Data Collection Allows for Surprising Insight
  17. 17. Analyze the Results • Carefully crunch the numbers OR • Just look at stuff and see what you think • How will the data make your school better?
  18. 18. Rules • Write your name on the back of your cards • Gain points by presenting research plans • Start Research with a Question Card • All cards only go face up, cards only connect on lines • Extend Research with Sample, sampling, method, and other cards • All current cardholders must agree to add a new card • Cards only add constraints, anything else is allowed • Call Justin when you present, total points times multiplier is score
  19. 19. Presentations • What is the question? • What are the indicators? • What methods will allow you to assess those indicators? • How will the data gathered help you answer to the question? • What could you do with your results to improve the school?
  20. 20. Final Thoughts • Identify learning goals connected to technology investments and interventions • Measure progress towards those goals • Draw random samples rather than assessing everyone – (and actually make them random) • Leverage partnerships with universities and other organizations • Use assessment data to reflect upon and improve instruction
  21. 21. Visit us at!