1. The document discusses analyzing data from 2000 student responses to the Force Concept Inventory (FCI), which is a 30-question multiple choice assessment of conceptual understanding of force. 2. Exploratory factor analysis was used to analyze the data and identify latent concepts represented in the FCI. Five factors were identified that group the questions according to conceptual themes related to Newton's laws of motion. 3. Preliminary item response analysis found that questions relating to Newton's third law had significantly higher difficulty than other questions. Some students got third law questions correct but questions relating to the first and second laws incorrect, indicating anomalous results.

1. The Concept of Force and the
Mechanics Curriculum
Terry Scott Statistician
Not a Statistician Andrew Gray
Daniel Schumayer
Minion

2. The Force Concept Inventory
• Concept Inventory rather than a “test”.
• 30 multichoice questions.
• Very good distractors:
– Guessing success very low
– Reliability unusually high
– Pre and post test design seems to work well

3. The Force Concept Inventory

4. FCI - Uses
• Standard uses:
– Evaluate development of conceptual
understanding.
– Evaluate teaching methods.
• Our research:
– Search for conceptual coherence.
– Understand associations in student mind.
– Inform teaching method.
– Suggest future research.

5. Data Collection
• FCI delivered online (Blackboard)
• PHSI191 – Large Health Science service
course.
• 2000 student responses collected over two
years, 2008 and 2009.
• Not obligatory.
• Data checked for disengaged responses
– 50 response sets removed.

6. Data Analysis
• Exploratory Factor Analysis
– Construct correlation matrix (using tetrachoric
correlations)
– Find eigenvalues/eigenvectors.
• What does this mean?
– Factors = latent traits/concepts
– Factors represent student understanding of force.
• How do students group questions, which questions are
about the same thing?
• How does this compare with the way experts group
questions?

7. Interpretation
• 5 factors found in the analysis.
• These factors are not orthogonal
– Correlations between factors – useful
information
• Clear that there is conceptual coherence
in the data.
• Better score = greater conceptual
coherence?
– No evidence, not enough data

8. The Factors
• Factor 1 – Identification of forces.
• Factor 2 – Newton’s first law, linear and
curved paths.
• Factor 3 – Newton’s second law +
necessary concepts
(acceleration, velocity).
• Factor 4 – Newton’s first law with zero net
force.
• Factor 5 – Newton’s third law.

9. Factor 1 – Identification of
Forces.

10. Factor 2 – Newton’s First

11. Factor 3 – Newton’s Second

12. Factor 4 – Newton’s First Again

13. Third Law/First Law conflation –
Qu.16

14. Kinematics?
• Authors of the FCI included a number of
questions solely about kinematics
concepts.
• We find no factor containing only
kinematics questions.
• Kinematics questions are distributed
“where they are needed” in factors about
force laws.

15. Item Response Analysis -
Preliminary
• Preliminary results
• Show third law questions significantly
higher difficulty that other questions
• Marginal analysis
– Assess student ability, predict the questions
they will get right.
– Anomalous numbers of students get 3rd law
questions right but 1st and 2nd law questions
wrong.