What Physics Knowledge Is Assessed in TIMSS Advanced 2008?


Published on

Slide from a talk at the AAAS Annual Meeting 2/19/11

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

What Physics Knowledge Is Assessed in TIMSS Advanced 2008?

  1. 1. What Physics Knowledge Is Assessedin TIMSS Advanced 2008? Chad Orzel Department of Physics and Astronomy Union College Schenectady, NY
  2. 2. SummaryTIMSS Advanced 2008 Overview: 9 countries (Armenia, Iran, Italy, Lebanon, Netherlands, Norway, Russia, Slovenia, Sweden) Approx 8200 students 39 Physics questions (37 scored) 4 Content Domains: Mechanics Electricity and Magnetism Atomic and Nuclear Physics Heat and Temperature
  3. 3. Content AssessmentOverall Approach: ―Would this make our job easier?‖Does content tested correspond to what students need to be prepared for college physics?Compare to: Introductory College Physics State High School Curricula (US) Physics Education Research
  4. 4. Introductory College PhysicsStandard Course Format: 2-semester survey calculus-based First Semester: Second Semester: Primarily mechanics Primarily E&M
  5. 5. Standard Curriculum: First Semester PART 1. 1 Measurement. Vectors/ 2 Motion Along a Straight Line. Kinematics 3 Vectors. 4 Motion in Two and Three Dimensions. 5 Force and Motion — I.Newton’s Laws 6 Force and Motion — II. 7 Kinetic Energy and Work. Energy 8 Potential Energy and Conservation of Energy. Momentum 9 Center of Mass and Linear Momentum. Angular 10 Rotation. 11 Rolling, Torque, and Angular Momentum. Momentum PART 2. 12 Equilibrium and Elasticity.Applied Forces 13 Gravitation. 14 Fluids. 15 Oscillations. Oscillations/ 16 Waves — I. Waves 17 Waves — II. 18 Temperature, Heat, and the First Law of Thermodynamics. Heat/ 19 The Kinetic Theory of Gases. Temperature 20 Entropy and the Second Law of Thermodynamics.
  6. 6. Standard Curriculum: Second Semester 21 Electric Charge.Electric Forces/ 22 Electric Fields. Fields 23 Gauss’ Law. 24 Electric Potential. Electric 25 Capacitance. 26 Current and Resistance. Circuits 27 Circuits. 28 Magnetic Fields. Magnetism 29 Magnetic Fields Due to Currents. 30 Induction and Inductance. 31 Electromagnetic Oscillations and Electro- Alternating Current. magnetics 32 Maxwell’s Equations; Magnetism of Matter. 38 Photons and Matter Waves. 33 Electromagnetic Waves. 39 More About Matter Waves. 34 Images. Optics Atomic/ 40 All About Atoms. 35 Interference. Nuclear 41 Conduction of Electricity in Solids. 36 Diffraction. 42 Nuclear Physics. 37 Relativity. 43 Energy from the Nucleus. 44 Quarks, Leptons, and the Big Bang.
  7. 7. Matter and Interactions: First Semester Chapter 1 Interactions and Motion. Chapter 2 The Momentum Principle. Momentum/ Chapter 3 The Fundamental Interactions. Force Chapter 4 Contact Interactions. Chapter 5 Rate of Change of Momentum. Chapter 6 The Energy Principle. Energy Chapter 7 Internal Energy. Chapter 8 Energy Quantization. Chapter 9 Multiparticle Systems.Complex Systems Chapter 10 Collisions. Ang. Momentum Chapter 11 Angular Momentum. Chapter 12 Entropy: Limits on the Possible.Heat/ Temperature Chapter 13 Gases and Engines. Similar set of topics, focus on fundamental principles Integrates modern physics, computation with VPython
  8. 8. Matter and Interactions: First Semester Chapter 14 Electric Field. Chapter 15 Electric Fields and Matter. Electric Field Chapter 16 Electric Field of Distributed Charges. Energy Chapter 17 Electric Potential. Magnetism Chapter 18 Magnetic Field. Chapter 19 Electric Field and Circuits. Circuits Chapter 20 Circuit Elements. Magnetism Chapter 21 Magnetic Force. Chapter 22 Patterns of Field in Space.Electromagnetics Chapter 23 Faradays Law. Chapter 24 Electromagnetic Radiation. Atomic/Nuclear Chapter 25 Waves and Particles. Similar set of topics, focus on fundamental principles Integrates modern physics, computation with VPython
  9. 9. Six Ideas: First SemesterUnit C: Conservation Laws Constrain Interactions Momentum, Energy, Angular Momentum ~4wks Some thermodynamicsUnit N: The Laws of Physics are Universal Newton’s Laws, Forces, Dynamics Oscillations/ Waves ~4wksUnit R: The Laws of Physics are Frame-Independent Special Relativity ~3wks
  10. 10. Six Ideas: Second SemesterUnit E: Electric and Magnetic Fields are Unified Electricity, Magnetism ~4wks Wave motionUnit Q: Particles Behave like Waves Classical Optics Quantum Physics ~4wks Atomic/NuclearUnit T: Some Processes are Irreversible Heat, Temperature, Thermodynamics ~3wks
  11. 11. Introductory College PhysicsCentral, common principles: Mechanics: Electromagnetism: Newton’s Laws Coulomb’s Law/ Electric Fields Momentum Magnetic Fields Energy Electric Circuits Angular Momentum Maxwell’s Equations Atomic/ Nuclear: Thermodynamics Particle Nature of Light Heat and Temperature Wave Nature of Matter Kinetic Theory/ Ideal Gases Atomic Structure Entropy Nuclear Processes Heat Engines/ Efficiency Helpful for entering students to have seen at least some of these
  12. 12. State CurriculaNorth Carolina StandardsMechanics: ~144/180 classes Kinematics: 48 Newton’s Laws: 31 Momentum: 14 Energy: 27 Waves: 24Electricity and Magnetism: ~27/180 classes Coulomb’s Law, E-field: 7 Electric Circuits: 20Good coverage of mechanics sparser coverage of E&M, no H&T or A&N
  13. 13. State CurriculaNew York: ―The student will be able to…‖ Mechanics: 25 items ―…verify Newton’s Second Law for linear motion‖ ―…describe and explain the exchange between potential energy, kinetic energy, and internal energy for simple mechanical systems, such as a pendulum, a roller coaster, a spring, a freely falling object‖ ―… recognize and describe conversions among different forms of energy in real or hypothetical devices such as a motor, a generator, a photocell, a battery.‖ Kinematics: 4 Newton’s Laws: 4 Momentum: 1 Energy: 7 Waves: 9 (includes EM waves/optics)
  14. 14. State CurriculaNew York: ―The student will be able to…‖ Electricity and Magnetism: 8 items ―… use measurements to determine the resistance of a circuit element‖ ―… draw and interpret circuit diagrams which include voltmeters and ammeters‖ ―… map the magnetic field of a permanent magnet, indicating the direction of the field between the N (north-seeking) and S (south-seeking) poles‖ Circuits: 7 Magnetic Field: 1 Atomic/Nuclear: 2 items ―… interpret energy-level diagrams‖ ―… correlate spectral lines with an energy-level diagram‖
  15. 15. State CurriculaCalifornia: ―Students know…‖ Mechanics: 22 items ―… that when forces are balanced, no acceleration occurs; thus an object continues to move at a constant speed or stays at rest (Newton’s first law). ‖ ―… applying a force to an object perpendicular to the direction of its motion causes the object to change direction but not speed (e.g., Earth’s gravitational force causes a satellite in a circular orbit to change direction but not speed).‖ Kinematics: 2 Newton’s Laws: 7 Momentum: 4 Energy: 3 Waves: 6 (includes EM waves/optics)
  16. 16. State CurriculaCalifornia: ―Students know…‖ Electricity and Magnetism: 11 items ―… how to predict the voltage or current in simple direct current (DC) electric circuits constructed from batteries, wires, resistors, and capacitors. ‖ ―… magnetic materials and electric currents (moving electric charges) are sources of magnetic fields and are subject to forces arising from the magnetic fields of other sources.‖ Coulomb’s Law, E-Fields: 2 Circuits: 4 Magnetic Field: 5 Heat and Thermodynamics: 7 items ―… that the work done by a heat engine that is working in a cycle is the difference between the heat flow into the engine at high temperature and the heat flow out at a lower temperature (first law of thermodynamics) and that this is an example of the law of conservation of energy.
  17. 17. State Curricula SummaryOverall Picture:  Good, fairly complete coverage of mechanics  Selected topics in Electricity and Magnetism Tend to emphasize circuits over fields  Highly variable coverage of other areas Extensive coverage in some states, none at all in othersTend to assume the first-year college students have seen nothing but mechanics, very basic E&M
  18. 18. Physics Education ResearchActive and well-established within physics departmentsNumerous research-based subject testsFCI: Force Concept InventoryFMCE: Force and Motion Conceptual EvaluationCSEM: Conceptual Survey of Electricity and Magnetism
  19. 19. Conceptual Tests Research shows that students can learn to solve problems by rote without fully understanding the underlying concepts In the circuit at right, find the current 8V 1WCalculation: through the 2 ohm resistor and the potential 75% correct difference between points a and b. 2W b a 12 V 1W In the circuit at the left, explain what will happen to the following variables when the switch is closed: * The current through the battery Concept: *The brightness of the bulbs 40% correct * The voltage drop across the bulbs * The total power dissipated From Peer Instruction by Eric Mazur
  20. 20. TIMSS Question BreakdownContent Domain distribution: Mechanics: 12 questions (11 scored) Electricity and Magnetism: 12 questions Atomic and Nuclear: 8 questions (7 scored) Heat and Temperature: 7 questions Fairly close to distribution of high school and intro college curricula Primary emphasis on Mechanics and E&M
  21. 21. TIMSS Mechanics12 Questions (11 scored) Kinematics: 3 Newton’s Laws: 4 Momentum: 1 Energy: 1 Waves: 2Primarily computational, rather than conceptualEnergy, momentum underrepresented
  22. 22. Conceptual MechanicsTIMSS question: FCI question: 50.8% correct (all countries)  Lowest of Mechanics MC (62% avg) Consistent with known results from Physics Education Research
  23. 23. TIMSS Energy Ei  E f Ki  U s 1 2 1 2 mv  kxVery information dense question 2 2 m Need to know KE, PE, cons. of E x v  0.125m k 46.8% correct (international average)Would be good to have additional energy question(s) to identify problem areas
  24. 24. TIMSS E&M12 Questions Electric Field: 3 Circuits/Conduction: 4 Magnetic force: 1 Radiation: 3Good mix of calculational/ conceptual questionsGood connections to mechanicsNo questions dealing with energy or magnetic fieldsRadiation questions over-emphasized
  25. 25. E&M RadiationThree radiation questions in E&M section 50.1%Very fact-based, almost triviaSeems excessive, especially given omission of energy 45.1% and magnetic field questions 38.5%
  26. 26. E&M MechanicsGood questions making conceptual linkage between E&M and mechanicsConceptual: Both Coulomb’s Law and Newton’s 3rd 22.3% correct– Lowest E&M MCReasoning: Both Lorentz force on moving charge and centripetal acceleration 9.8% correct– Lowest in data set
  27. 27. Conceptual E&MTIMSS Question CSEM Question 41.4%  Highest in E&M free-response
  28. 28. TIMSS Atomic and Nuclear8 Questions (7 scored) Atomic Structure: 4 Radioactive Decay: 3 Photoelectric Effect: 1 Large variance in curricula in this area makes assessment difficult Questions generally reasonable No questions on atomic spectra, or nuclear fission/ fusion
  29. 29. Atomic StructureSubjective impression: too many atomic structure questions 76%2/3 questions about notation 70.9% 40.7%
  30. 30. TIMSS Heat and Temperature7 Questions: Heat Capacity: 3 Radiation: 3 Ideal Gas Law: 1 Large variance in curricula in this area makes assessment difficult Notable omissions: kinetic theory, heat engines 2nd Law of Thermodynamics probably most important idea from this area of physics  not tested
  31. 31. TIMSS Global WarmingTwo questions with a global warming theme(Just an observation, not a complaint…) 66.6% 51.1%
  32. 32. Overall ResultsScores by section: Scores by type: Mult. Choice Free Resp. Mechanics: 51% Mechanics: 50% 34% E&M: 37% E&M: 45% 27% H&T: 49% H&T: 52% 28% A&N: 51% A&N: 58% 28%(Median % getting full credit, (Mean % getting full credit,averaged over all questions) averaged over all questions)Of note:  Ned. has top scores in Mech, H&T, A&N, Arm in E&M  Italy last or 2nd to last in all 4, Leb. last in E&M, H&T
  33. 33. Gender BreakdownScores by section: Scores by type: Mult. Choice Free Resp. Mechanics: -3.6% Mechanics: -9% -6% E&M: -4.0% E&M: -13% -9% H&T: -6.2% H&T: -16% -12% A&N: -2.5% A&N: -6% -2%(F % full credit- M% full credit (F%-M% as %-age of avg. scoreaveraged over all questions) averaged over all questions of type)Of note:  Lebanon: avg(F)>avg(M) for all sections, Armenia 3/4 Iran, Italy, Rus., Ned., Nor.: all have M>F for all questions on 2/4 sections
  34. 34. ConclusionsTIMSS Physics subjects areas generally aligned with standard curricula Primary emphasis on Newtonian mechanics and E&MSome overemphasis/ omissions Too much: Not enough: Radiation Energy/ momentum Atomic Structure Magnetism Kinetic theory/ entropyTIMSS Physics questions emphasize calculations over concepts PER shows these aren’t necessarily the same Some good conceptual questions, would be good to see more