Utah State University Using a Concept Inventory to Inform the Design of Instruction and Software Doug Holton [email_addres...
About Me <ul><li>Instructional Technology & Learning Sciences @ USU </li></ul><ul><li>Background in engineering and scienc...
The Problem <ul><li>The problem we worked on in the past: </li></ul><ul><li>Identifying difficulties and misconceptions st...
Interviews <ul><li>We interviewed instructors – what concepts did students have the most difficulty with? </li></ul><ul><l...
Invariants <ul><li>Ohm's Law </li></ul><ul><li>Kirchoff's Current and Voltage Laws </li></ul><ul><li>Effective resistance ...
Misconceptions, Difficulties <ul><li>“ empty pipe” - wires are empty when circuit is “off” </li></ul><ul><li>“ current con...
AC/DC Concept Inventory <ul><li>Designed a multiple choice test </li></ul><ul><li>boiled it down to 20 items </li></ul><ul...
Example Questions <ul><li>(DEMO) Concept Inventory </li></ul><ul><li>Why do the lights come on so fast when you flip the s...
Other Concept Inventories <ul><li>Signals & Systems </li></ul><ul><li>Electronics </li></ul><ul><li>DIRECT – high school c...
AC/DC Inventory Results <ul><li>Across many schools, students were only getting about 55%-65% correct on these 20 test que...
Using a Concept Inventory to Improve Instruction <ul><li>” We believe concept inventories can be critical change agents fo...
Dynamic Assessment <ul><li>First strategy was to give students help while they took the test. </li></ul><ul><li>This is ca...
Results from Inductor Tool <ul><li>Our web-based dynamic assessment tool:  Inductor </li></ul><ul><li>Students learned whi...
Animated Circuit Simulation <ul><li>Because of #2, I looked into finding/creating better resources with which students cou...
MBL Research <ul><li>Modifications inspired by MBL (microcomputer-based labs) research. </li></ul><ul><li>Example: Using d...
Real-time Reactive Control <ul><li>Tried the same thing by modifying the circuit simulation. </li></ul><ul><li>Added real-...
Results from Simulation <ul><li>The results were similar to MBL. </li></ul><ul><li>Students were given concept inventory a...
Caveat <ul><li>BUT, the gains seen were only on half of the test questions. </li></ul><ul><li>What was in common with thos...
Other 10 questions <ul><li>The other 10 questions often involved students' lack of distinctions between different things, ...
Contrasting Cases <ul><li>The third strategy we are looking into is based on perceptual theory about how people learn to m...
Contrasting Cases examples <ul><li>In the next slides, ask yourself – what do you notice in the left box? </li></ul>
What do you see in left box?
What did you notice? A Circle Now try it again What do you notice in the left box?
What do you see in left box?
What did you notice? A Smaller Circle You noticed the size now And perhaps the white color too Try It Again
What do you see in left box?
What did you notice? A Circle in the Middle Now the position of the circle Is more salient (Demo circuit examples)
Future Plans <ul><li>Software – release new circuit simulation this summer called  nodicity </li></ul><ul><li>Research – t...
Take Away Messages <ul><li>Recommend you try giving your own pre-tests and post-tests to see gains for yourself. Works wit...
Thank You, Questions? [email_address] http://edtechdev.blogspot.com Doug Holton Department of Instructional Technology  & ...
Upcoming SlideShare
Loading in …5
×

Using a Concept Inventory to Inform the Design of Instruction and Software

2,039 views

Published on

Presentation to Rocky Mountain ASEE 2009 Conference - http://ateng.tc.uvu.edu/aseerms2009/

Published in: Education, Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,039
On SlideShare
0
From Embeds
0
Number of Embeds
270
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Using a Concept Inventory to Inform the Design of Instruction and Software

  1. 1. Utah State University Using a Concept Inventory to Inform the Design of Instruction and Software Doug Holton [email_address] Department of Instructional Technology & Learning Sciences - http://itls.usu.edu/
  2. 2. About Me <ul><li>Instructional Technology & Learning Sciences @ USU </li></ul><ul><li>Background in engineering and science education research and development: </li></ul><ul><li>Learning by Design middle school design challenges @ GaTech </li></ul><ul><li>VaNTH bioengineering education @ Vanderbilt </li></ul><ul><li>ONR funded project to investigate the difficulties students have understanding electrical circuits. </li></ul>
  3. 3. The Problem <ul><li>The problem we worked on in the past: </li></ul><ul><li>Identifying difficulties and misconceptions students have when learning about electrical circuits . </li></ul><ul><li>Much pre-existing research, but it was only on the simplest of circuits (bulbs/batteries). </li></ul><ul><li>We extended it to AC, capacitors, inductors, RLC, etc. Undergraduate level. </li></ul>
  4. 4. Interviews <ul><li>We interviewed instructors – what concepts did students have the most difficulty with? </li></ul><ul><li>Interviewed students – Probed their thinking while they worked on circuit problems. </li></ul><ul><li>Identified core invariant principles, and a list of misconceptions. </li></ul>
  5. 5. Invariants <ul><li>Ohm's Law </li></ul><ul><li>Kirchoff's Current and Voltage Laws </li></ul><ul><li>Effective resistance – series/parallel </li></ul><ul><li>Charge & impedance of capacitor </li></ul><ul><li>Inductor and impedance, flux </li></ul><ul><li>Power </li></ul>
  6. 6. Misconceptions, Difficulties <ul><li>“ empty pipe” - wires are empty when circuit is “off” </li></ul><ul><li>“ current consumption” - current is consumed as it travels through wires </li></ul><ul><li>more resistance means more power </li></ul><ul><li>AC is spatial rather than temporal </li></ul><ul><li>ignore negative part of AC </li></ul><ul><li>confuse high and low pass filters </li></ul><ul><li>don't distinguish caps & inductors </li></ul>
  7. 7. AC/DC Concept Inventory <ul><li>Designed a multiple choice test </li></ul><ul><li>boiled it down to 20 items </li></ul><ul><li>Correct choices target invariants </li></ul><ul><li>Incorrect choices target misconceptions </li></ul><ul><li>Improved its reliability - test-wiseness </li></ul><ul><li>See ASEE 2008 Paper – Holton, Verma, & Biswas </li></ul><ul><li>“ Assessing Student Difficulties in Understanding the Behavior of AC and DC Circuits” </li></ul>
  8. 8. Example Questions <ul><li>(DEMO) Concept Inventory </li></ul><ul><li>Why do the lights come on so fast when you flip the switch? </li></ul><ul><li>What happens in a basic battery, bulb, capacitor circuit? </li></ul><ul><li>Understanding a basic filter circuit </li></ul>
  9. 9. Other Concept Inventories <ul><li>Signals & Systems </li></ul><ul><li>Electronics </li></ul><ul><li>DIRECT – high school circuits </li></ul><ul><li>Statistics, Biology, Mechanics </li></ul><ul><li>Statics, Dynamics, Materials </li></ul><ul><li>Thermodynamics </li></ul><ul><li>Force Concept Inventory </li></ul><ul><li>Google “concept inventory” </li></ul>
  10. 10. AC/DC Inventory Results <ul><li>Across many schools, students were only getting about 55%-65% correct on these 20 test questions even after they had completed 1-2 undergraduatecircuits classes. </li></ul><ul><li>Did worse on questions involving AC, capacitors, inductors – the dynamic components and circuits </li></ul>
  11. 11. Using a Concept Inventory to Improve Instruction <ul><li>” We believe concept inventories can be critical change agents for transformation of pedagogy and learning in STEM disciplines” (Concept Inventory Central) </li></ul><ul><li>Goal is to see how we can improve student performance on these questions. </li></ul>
  12. 12. Dynamic Assessment <ul><li>First strategy was to give students help while they took the test. </li></ul><ul><li>This is called dynamic assessment </li></ul><ul><li>Students selected the answer, and then selected the principle/invariant. </li></ul><ul><li>If incorrect – received access to resources for learning: </li></ul><ul><ul><li>video explanations </li></ul></ul><ul><ul><li>links to website resources </li></ul></ul>
  13. 13. Results from Inductor Tool <ul><li>Our web-based dynamic assessment tool: Inductor </li></ul><ul><li>Students learned while taking this dynamic assessment </li></ul><ul><li>But they were not motivated, saw declining performance </li></ul><ul><li>Problems: </li></ul><ul><ul><li>It still was just a test </li></ul></ul><ul><ul><li>The resources were not that good or consistent </li></ul></ul>
  14. 14. Animated Circuit Simulation <ul><li>Because of #2, I looked into finding/creating better resources with which students could understand the behavior of circuits. </li></ul><ul><li>Found an animated circuit simulation which I modified for use in a study. </li></ul>
  15. 15. MBL Research <ul><li>Modifications inspired by MBL (microcomputer-based labs) research. </li></ul><ul><li>Example: Using distance sensor, a student moves a car back and forth, and see its position/speed/acceleration graphed in real-time </li></ul><ul><li>Very effective – students better understood motion graphs in a matter of minutes </li></ul><ul><li>Key aspects 1) real-time 2) haptic </li></ul>
  16. 16. Real-time Reactive Control <ul><li>Tried the same thing by modifying the circuit simulation. </li></ul><ul><li>Added real-time reactive control (enactive interface) </li></ul><ul><li>You can “wiggle” the voltage </li></ul><ul><li>and see its effects immediately. </li></ul><ul><li>(DEMO) </li></ul>
  17. 17. Results from Simulation <ul><li>The results were similar to MBL. </li></ul><ul><li>Students were given concept inventory as pre-test, tutored with simulation, then given post-test. </li></ul><ul><li>Students showed significant gains after just a half hour tutoring session. </li></ul><ul><li>With 'turning on lights' question, students went from 20% correct to 65%, even though nothing about lights was mentioned during tutoring. </li></ul>
  18. 18. Caveat <ul><li>BUT, the gains seen were only on half of the test questions. </li></ul><ul><li>What was in common with those 10? </li></ul><ul><li>They forced you to consider the behavior of current over time </li></ul><ul><li>(Examples of temporal/non-temporal questions) </li></ul><ul><li>Students improved from 53% to 73% on temporal questions. </li></ul><ul><li>Only 68% to 71% on non-temporal. </li></ul>
  19. 19. Other 10 questions <ul><li>The other 10 questions often involved students' lack of distinctions between different things, such as voltage vs. current, power vs. resistance, capacitors vs. inductors, high-pass vs. low-pass filters, etc. </li></ul><ul><li>(Show example questions) </li></ul>
  20. 20. Contrasting Cases <ul><li>The third strategy we are looking into is based on perceptual theory about how people learn to make distinctions and notice differences. </li></ul><ul><li>Researchers found that if you show one item, people don't notice many features, but if you show 2 side by side, people attend to the differences between the two. </li></ul><ul><li>(DEMO – simulation & contrasts) </li></ul>
  21. 21. Contrasting Cases examples <ul><li>In the next slides, ask yourself – what do you notice in the left box? </li></ul>
  22. 22. What do you see in left box?
  23. 23. What did you notice? A Circle Now try it again What do you notice in the left box?
  24. 24. What do you see in left box?
  25. 25. What did you notice? A Smaller Circle You noticed the size now And perhaps the white color too Try It Again
  26. 26. What do you see in left box?
  27. 27. What did you notice? A Circle in the Middle Now the position of the circle Is more salient (Demo circuit examples)
  28. 28. Future Plans <ul><li>Software – release new circuit simulation this summer called nodicity </li></ul><ul><li>Research – test contrasting cases strategy, combine other strategies with simulation in the context of a class </li></ul><ul><li>Pilot work in high schools </li></ul><ul><li>Mainly collaborating with Amit Verma (Texas), Oenardi Lawanto, and Paul Schreuders </li></ul>
  29. 29. Take Away Messages <ul><li>Recommend you try giving your own pre-tests and post-tests to see gains for yourself. Works with regular tests </li></ul><ul><li>Try incorporating animated simulations and visualizations – they can have great effects on student learning and help them understand the behavior that underlies the equations. </li></ul><ul><li>Check out the book How People Learn for more on other strategies. </li></ul>
  30. 30. Thank You, Questions? [email_address] http://edtechdev.blogspot.com Doug Holton Department of Instructional Technology & Learning Sciences http://itls.usu.edu/

×