PBL in Economics

4,179 views
4,011 views

Published on

A federally-funded, randomized controlled trial showing efficacy of PBL use in economics with professional development

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,179
On SlideShare
0
From Embeds
0
Number of Embeds
2,041
Actions
Shares
0
Downloads
14
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

PBL in Economics

  1. 1. Effects of Problem Based Economics on High School Economics Instruction<br />Neal D. FinkelsteinRegional Educational Laboratory, West (REL West), WestEd<br />Jason RavitzBuck Institute for Education<br />AERA Annual Meeting – New Orleans, LAApril 14, 2011<br />1<br />
  2. 2. Neal D. FinkelsteinSenior Research ScientistREL West)WestEd<br />December 8, 2010<br />2<br />
  3. 3. The case for economic literacy is obvious. High school graduates will be making economic choices all their lives, as breadwinners and consumers, and as citizens and voters. A wide range of people will be bombarded with economic information and misinformation for their entire lives. They will need some capacity for critical judgment. They will need it whether or not they go to college. <br />James Tobin, Yale Economist and Nobel Laureate <br />Wall Street Journal, July 9, 1986 <br />3<br />
  4. 4. Purpose<br />Assess student-level impacts of a problem-based instructional approach to high school economics.<br />Tests the effectiveness of Buck Institute for Education curriculum on student learning of economics content and problem solving skills.<br />Within-school randomized controlled trial in urban and rural high schools in California and Arizona during the 2007/08 school year.<br />4<br />
  5. 5. Why Study Economics Instruction?<br />For decades, economists, educators, and business and government leaders have advocated for economic literacy as a component in school curricula.<br />In general, high school economics does not help students understand our economic system.<br />Most teachers are not adequately prepared to teach economics.<br />48 states include content standards in economics. 40 require implementation of standards, 23 require testing, and 17 require an economics course for graduation.<br />5<br />
  6. 6. What is Problem-Based Economics Instruction?<br />Each of five curriculum modules is built around a case study well-suited to student-driven problem solving.<br />Teachers use economic problems to help students learn to contextualize, understand, and solve problems.<br />Teaching guide lays out all aspects of instruction<br />including problem statement, placement in curriculum, concepts taught, objectives, resources materials and do’s and don’ts.<br />Teachers in the intervention group receive professional development introducing the curriculum.<br />Led by expert teachers with experience using the PBE units<br />6<br />
  7. 7. Students are often challenged <br />in high school Economics <br />courses because:<br />O Economics concepts can be <br /> abstract<br />O They don’t see relevance to <br /> real life<br />O They are bored<br />O All of the Above<br />7<br />
  8. 8. “Anyone…? Anyone…?”<br />8<br />
  9. 9. The Curriculum<br />Organizes curriculum and instruction around<br /><ul><li> Carefully crafted situations
  10. 10. Adapted from real-world issues
  11. 11. With more than one possible solution</li></ul>Students work in teams to gather, apply and demonstrate knowledge and skills as they develop and present solutions.<br />9<br />
  12. 12. <ul><li> Units on micro- & macro-economics (7 total, 5 used in study)
  13. 13. Aligned with National Content </li></ul> Standards in Economics and state <br /> standards<br /><ul><li> Units take from one to three weeks
  14. 14. Typically taught in stand-alone high </li></ul> school Economics courses<br />10<br />
  15. 15. HOW EACH PBE UNIT IS <br />STRUCTURED:<br />Project Launch (do NOT pre-teach!)<br />Framing the Inquiry (Know/Need <br /> to Know List & Driving Question)<br />3. Problem-Solving & Learning Activities<br />4. Presentation, Assessment, and Debrief<br />11<br />
  16. 16. WHAT IS INCLUDED IN EACH <br />PBE UNIT: <br /><ul><li> guidance on teaching with PBL
  17. 17. “Make More Money?” activity
  18. 18. step-by-step directions (with Project Log </li></ul> prompts and Content Teaching Notes) <br /><ul><li> student handout masters
  19. 19. content background material for </li></ul> teacher mini-lectures & lessons<br /><ul><li> rubrics & exemplars for assessment
  20. 20. multiple-choice test </li></ul>12<br />
  21. 21. PBE UNIT: <br />RUNNING IN PLACE<br /><ul><li> As grad students in Economics, students </li></ul> are asked for advice by an entrepreneur <br /> who wants to start a new shoe company<br /><ul><li> Students hear from consumers and </li></ul> producers<br /><ul><li> Students learn about factors of </li></ul> production, circular flow<br /><ul><li> Students create visuals and make an oral </li></ul> presentation of their recommendations <br />13<br />
  22. 22. WHAT TEACHERS SAY ABOUT PBE: <br />“Making economics more engaging and relevant for students is possible!”<br />“I have really enjoyed teaching with this curriculum; having a central problem to refer back to throughout the unit helps with teaching the concept.”<br />“I just finished using PBE for the second time. The material that my kids are putting together is amazing. These are inner-city kids and the results are beyond belief. They love your material. I have been teaching for over 30 years and the lessons are undoubtedly the best I have ever used.”<br />14<br />
  23. 23. Conceptual Framework<br />Student achievement outcomes are of primary importance and are hypothesized to be mediated by changes in teacher knowledge and pedagogical practice.<br />15<br />
  24. 24. Conceptual Framework<br />Logic model for the study of high school instruction with Problem-based Economics<br />Teacher Professional Development<br />Classroom Instruction<br />Student Performance<br />Content<br /><ul><li>Review of economic theory and related applications; curriculum reivew</li></ul>Pedagogy<br /><ul><li>Problem-based approach to teaching and support strategies for students</li></ul>Curriculum<br /><ul><li>Fluent presentation of curriculum; scaffolding for students</li></ul>Engagement<br /><ul><li>Problem-solving strategies that build economic content and reinforce core analytic thinking</li></ul>Concepts<br /><ul><li>Broad-based literacy in micro- and macroeconomic concepts</li></ul>Problem Solving<br /><ul><li>Ability to apply concepts and analytic approaches to real-world problems</li></ul>16<br />
  25. 25. Research Questions<br />Does PBE change students’ content knowledge in economics?<br />Does PBE change students’ problem-solving skills in economics<br />Does PBE change teachers’ content knowledge of economics?<br />Does use of PBE change economics teachers’ instructional practices<br />Does the use of PBE change teachers’ satisfaction with teaching materials and methods used to teach economics?<br />17<br />
  26. 26. Key Outcomes to be Studied and Their Measures<br />The primary outcome measure is content knowledge gains for students in economics measured by the Council for Economic Education’s Test of Economic Literacy.<br />Student problem-solving skills are measured with open-response performance assessment of applied economic concepts developed by UCLA CRESST.<br />Teachers’ content knowledge was also measured using the Test of Economic Literacy. Pedagogical practices and teacher satisfaction were measured using a series of survey measures.<br />18<br />
  27. 27. Key Study Characteristics<br />Study characteristics and data collection schedule<br />19<br />
  28. 28. Teacher Participants<br />Allocation<br />Retention<br />Data Collected & Analyzed<br />Allocated to intervention group: n = 64<br /><ul><li>44 teachers from 44 singleton schools
  29. 29. 20 teachers from 16 schools with 2 or more participating teachers.</li></ul>Included: n = 42<br /><ul><li>28 teachers from 28 singleton schools
  30. 30. 14 teachers from 11 schools with 2 or more participating teachers.</li></ul>Not Included: n = 22<br /><ul><li>16 teachers from 16 singleton schools
  31. 31. 6 teachers from 6 schools with 2 or more participating teachers.</li></ul>Baseline Measures (Summer 2007)<br /><ul><li>Teacher content knowledge in economics (n = 41)
  32. 32. Pedagogical practices used (n = 39)
  33. 33. Satisfaction with teaching materials & methods (n = 40)</li></ul>Outcome Measures (June 2008)<br /><ul><li>Teacher content knowedge in economics (n = 38)
  34. 34. Pedagogical practices used (n = 38)
  35. 35. Satisfaction with materials & methods (n = 37)</li></ul>Number of recruited teachers: n = 128 teachers from 106 schools<br />Number of teachers randomized: n = 128 teachers from 106 schools<br />Allocated to control group:n = 64<br /><ul><li>46 teachers from 46 singleton schools
  36. 36. 18 teachers from 15 schools with 2 or more participating teachers.</li></ul>Included: n = 41<br /><ul><li>31 teachers from 31 singleton schools
  37. 37. 10 teachers from 9 schools with 2 or more participating teachers.</li></ul>Not Included: n = 23<br /><ul><li>15 teachers from 15 singleton schools
  38. 38. 8 teachers from 8 schools with 2 or more participating teachers.</li></ul>Baseline Measures (Summer 2007)<br /><ul><li>Teacher content knowledge in economics (n = 29)
  39. 39. Pedagogical practices used (n = 25)
  40. 40. Satisfaction with teaching materials & methods (n = 29)</li></ul>Outcome Measures (June 2008)<br /><ul><li>Teacher content knowedge in economics (n = 34)
  41. 41. Pedagogical practices used (n = 35)
  42. 42. Satisfaction with materials & methods (n = 35)</li></ul>Excluded: 0 teachers<br />20<br />
  43. 43. Teacher Participants Providing Student-Level Data <br />Number of teachers remained in the study: n = 42<br />In fall 2007:<br /><ul><li>38 teachers returned student-level data
  44. 44. 3 teachers did not return any student-level data</li></ul>In spring 2008:<br /><ul><li>35 teachers returned student-level data
  45. 45. 6 teachers (including those who dropped out of the study) did not return any student-level data</li></ul>Allocated to intervention group: n = 64<br />Attrition: n = 22<br />Number of teachers recruited and randomized: <br />n = 128 teachers<br />Total number of teachers who provided student-level data in spring 2008: n = 64 teachers<br />Number of teachers remained in the study: n = 41<br />In fall 2007:<br /><ul><li>38 teachers returned student-level data
  46. 46. 3 teachers did not return any student-level data</li></ul>In spring 2008:<br /><ul><li>29 teachers returned student-level data
  47. 47. 12 teachers (including those who dropped out of the study) did not return any student-level data</li></ul>Allocated to control group:n = 64<br />Attrition: n = 23<br />21<br />
  48. 48. Student Participants<br />Experimental Condition<br />Data Collected & Analyzed<br />In intervention group (35 teachers):<br /><ul><li>Included: n = 2,502
  49. 49. Not Included: n =0</li></ul>Baseline Measures (January 2008)<br /><ul><li>Student content knowledge in economicsNon-missing: n = 2,232 (98%)Missing: n = 270 (11%)Number of teachers = 35</li></ul>Outcome Measures (June 2008)<br /><ul><li>Student content knowledge in economicsIncluded: n = 2,178 (87%)Not Included: n = 324 (13%)Number of teachers = 35
  50. 50. Student problem-solving skillsIncluded: n = 1,918 (77%)Not Included: n = 584 (23%)Number of teachers = 33</li></ul>Number of students consented to participate in the study & their data available to the research team: <br />n = 4,350 students from 64 teachers<br />In control group (29 teachers):<br /><ul><li>Included: n = 1,848
  51. 51. Not Included: n = 0</li></ul>Baseline Measures (January 2008)<br /><ul><li>Student content knowledge in economicsNon-missing: n = 1,589 (86%)Missing: n = 259(14%)Number of teachers = 29</li></ul>Outcome Measures (June 2008)<br /><ul><li>Student content knowledge in economicsIncluded: n = 1,574 (85%)Not Included: n = 274 (15%)Number of teachers = 29
  52. 52. Student problem-solving skillsIncluded: n = 1,497 (81%)Not Included: n = 351 (19%)Number of teachers = 29</li></ul>22<br />
  53. 53. Data Analysis Methods<br /><ul><li>The analyses for this study compare outcomes for students and teachers in the treatment group with their counterparts in the control group after the economics course has been completed.
  54. 54. For student outcomes, the analyses involve fitting conditional multilevel regression models (HLM) to account for the nesting of students within teachers. </li></ul>23<br />
  55. 55. Data Analysis Methods<br /><ul><li>A random effect for teachers is included in the model to account for the nesting of student observations within teachers. Fixed effects include treatment status, baseline (pre-test) measures of outcome variables, and other student and teacher-level covariates.
  56. 56. For teacher outcomes, single-level regression models that include treatment status along with certain covariates (such as pre-test measures of outcome variables and teacher background information) are used to estimate program impacts.</li></ul>24<br />
  57. 57. Student-Level Findings<br />Statistically significant finding: students whose teachers had received professional development and support in PBE outscored their control group peers on the TEL by an average of 2.6 test items.<br />25<br />
  58. 58. Student-Level Findings<br />The outcomes on student measures of problem-solving skills and application to real world economic dilemmas also showed significant differences in favor of the intervention group.<br />26<br />
  59. 59. Teacher-Level Findings<br /><ul><li>No statistically significant difference between the interventions and control groups on teachers’ knowledge of economics.
  60. 60. No statistically significant difference in teachers’ pedagogical style with the survey measures used.
  61. 61. Statistically significant differences in favor of the intervention group teachers on a measure of satisfaction with the teaching materials and methods.</li></ul>27<br />[1] Source: www.knowhow2go.org<br />
  62. 62. Effects of Problem Based Economics on High School Economics Instruction<br />http://ies.ed.gov/ncee/edlabs/regions/west/pdf/REL_20104012.pdf<br />28<br />

×