The study compared student conceptual knowledge gains in chemistry and physics when taught with and without visualizations. It found no significant differences in overall learning gains between the two groups. For physics classes, male students saw slightly higher gains when using visualizations compared to females. The researchers concluded that while visualizations did not decrease learning, their effectiveness was not clearly proven and excessive claims about their benefits should be avoided. They plan to conduct a follow up study with more focus on the types and uses of visualizations.

1. David Geelan, The University of Queensland Michelle Mukherjee, Queensland University of Technology

2. Overview 12 Chemistry classes and 10 physics classes Comparison of student conceptual knowledge gains when taught with and without visualisations Sex, learning style and academic ability as further variables

3. Background Lots of good evidence that students enjoy learning with visualisations Lots of teachers adopting them, lots of money being spent developing, hosting and sharing them Not much good quality quantitative evidence of their educational effectiveness, particularly at the high school level

4. Design The students completed a pre-test and post-test of conceptual understanding, based on the Force Concept Inventory and the Chemistry Concept Inventory Multiple-choice items with common student misconceptions as distracters Cross-over experimental design: students completed one topic with visualisations and one without

5. Teaching Comparison Teachers taught the physics and chemistry topics either with or without using scientific visualisations Non-visualisation cases were not necessarily just lectures, and included demonstrations and other activities After post-test, most classes did use the visualisations

6. Examples

8. Results - Overall 79 physics, 78 chemistry 34 male, 123 female No significant difference t(512) = -1.48, p = .14 Treatment Gain Mean SD No visualisation (N=157) 1.19 2.26 Visualisation (N=157) 1.58 2.39

9. Results - Physics 80 physics No significant difference t(158)=-1.58, p=.116 Treatment Gain Mean SD No visualisation (N=157) .95 2.22 Visualisation (N=157) 1.53 2.38

10. Results - Chemistry 129 chemistry No significant difference t(256)=-.538, p=.59 Treatment Gain Mean SD No visualisation (N=157) 1.74 2.67 Visualisation (N=157) 1.92 2.65

11. Results - Sex Chemistry: no significant difference Physics: significant difference at p<.05 level: (t(78)=2.37, p=.02) moderate effect size (Cohen’s d=0.54)

12. Results – Academic Achievement Chemistry: no significant difference Physics: no significant difference

13. Results – Learning Styles Very small differences noted for physics with a slight advantage for kinesthetic learners (not visual learners) Construct is very shaky and so was the measurement Can’t get published with this measure included Therefore this facet discarded

14. Conclusion ‘ First, do no harm’: While there were no large benefits for conceptual learning observed, there was also no decrease in conceptual learning Given the other benefits of student enjoyment and engagement, use of visualisations is probably justified Excessive effectiveness claims should be avoided

15. The Next Study Applying for ARC Discovery grant: If successful study will start in 2012 Many detail variables in relation to types of visualisations and ways they were used Still focused on classroom-based research More qualitative approach to students’ learning/thinking while learning with visualisations

16. Contact Please do get in touch if you have questions, suggestions, solutions or are doing related work: [email_address] Michelle Mukherjee will be reporting some results from a related study here on Friday