Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The tricky relationship between research and practice


Published on

Talk at HCS conference

Published in: Education
  • Be the first to comment

  • Be the first to like this

The tricky relationship between research and practice

  1. 1. The tricky relationship between research and practice 29 June 2018 Christian Bokhove HCS conference
  2. 2. Evidence informed education • Evidence-based medicine • Evidence-based practice • Evidence-informed education
  3. 3. "Evidence-based practice is about making decisions through the conscientious, explicit and judicious use of the best available evidence from multiple sources by … Asking: Translating a practical issue or problem into an answerable question Acquiring: Systematically searching and retrieving the evidence Appraising: Critically judging the trustworthiness and relevance of the evidence Aggregating: Weighing and and pulling together the evidence Applying: Incorporating the evidence in the decision-making process Assessing: Evaluating the outcome of the decision taken . … to increase the likelihood of a favorable outcome."
  4. 4. Evidence-informed still means ‘based on scientific research’ but we’re in the field of learning sciences, which is an interdisciplinary science • Sociology • Cognitive psychology • Etc.. “dealing with so many variables that are extremely hard to (all) control.”
  5. 5. Some age-old issues 1 • Cause and effect • Hume’s Skepticism and Kant • Correlation and causation • Falsification • Popper and Kuhn • Does one study falsify a body of research? • Context • Berliner: ‘the hardest science of them all’
  6. 6. Some age-old issues 2 • A lesser form of knowledge? • Labaree: soft knowledge. • Negative: lower status, weaker authority, push quantification. • Positive: knowledge for practice, free from consumer pressures, no disciplinary boundaries, general public.
  7. 7. Some age-old issues 3 • Quantification and measurement • To improve credibility, origins ‘statistics’ • But could destroy local practical knowledge • What are we measuring (fMRI, SES, mindset, load) This requires a vast knowledge base…
  8. 8. But there are tensions with this
  9. 9. “Modal models for prospective teachers should use folk psychological terms whenever doing so will not mislead. For example, the core properties of working memory can be conveyed with the intuitive observations that there is a mental “space” for thinking, that this space is limited, and that it can be occupied by things perceived in the environment and/or things from the long-term memory.” But the key question then becomes: when misleading? (Willingham, 2017)
  10. 10.
  11. 11. A thought-provoking example • Everyone loves Cognitive Load Theory • It is good science not bad science • So how can you be critical of it? • Being a Skeptic • Being skeptical does not mean I disagree with it • It forces people to reflect on their thinking • Let me ‘strip it and flip it’ (Willingham, 2012) REDUCE COGNITIVE LOAD This is often interpreted as to ‘take away’ something
  12. 12. Load from words R I S D E S Add a P and I can make SPIDERS A unit with meaning Load interacts with prior knowledge and schemas.
  13. 13. Load from picturebooks research (Flack & Horst, 2017)
  14. 14. Rest of slide Intentionally left blank to not impose too much load. Scale originates from Paas (1992)
  15. 15. Feedback might reduce load “Across the three experiments–with different problem-solving tasks and participant populations– we found that subjective ratings of effort investment were significantly higher after negative than after positive feedback; ratings given without feedback fell in between. These findings show that feedback valence alters perceived effort investment (possibly via task perceptions or affect), which can be problematic when effort is measured as an indicator of cognitive load.” (Raaijmakers et al., 2017)
  16. 16. Moment of measuring… “The findings from Experiment 1 (between-subjects) and 2 (within-subjects), using different arrangements of simple and complex tasks, showed that a single rating after a series of tasks resulted in a higher mental effort score than the average of ratings provided immediately after every task. A similar result was obtained in Experiment 3 with series of complex tasks, but not with simple tasks. Experiment 4 showed that knowing beforehand that mental effort rating will be required after completing all tasks results in lower scores, but average retrospective ratings per task still differed from a single retrospective rating.” (Van Gog et al, 2012)
  17. 17. Message • NOT that we can’t get something out of this, but… • “It’s complex” • Messages get distorted as they propagate through the system. • So even a statement like “REDUCE COGNITIVE LOAD” must be unpicked.
  18. 18. How do misconceptions start? “examples of cases in which entrepreneurs have knowingly set out to mislead educators are difficult to find.” (Howard-Jones, 2014, p. 817) “more likely that such interventions originate from uninformed interpretations of genuine scientific facts and are promoted by victims of their own wishful thinking who hold a “sincere but deluded fixation on some eccentric theory that the holder is absolutely sure will revolutionize science and society” (Howard- Jones, 2014, p. 817)
  19. 19. Perpetuated • Cultural conditions e.g. differences in terminology and language • Also check Lilienfeld et al. (2015, 2017) for lists with psychological terms to avoid and pairs of confusion • Counter-evidence difficult to access • Untestable • Biases • Complex
  20. 20. Robin-Garcia et al. (2017) • Dentistry • “tweeting about scholarly articles represents curating and informing about state-of-the-art appears not to be realized in practice.” • “Simplistic and naïve use of social media data risks damaging the scientific enterprise, misleading both authors and consumers of scientific literature.” • (Now check the paper, it says much more, and it has its own limitations)
  21. 21. Do teachers need research literacy skills? • Ideally, yes, as much as possible • But it is clear this can be quite challenging for all kinds of reasons, ranging from time, knowledge, skills constraints. So why not *work* with Academics?
  22. 22. Academia interested in links? • This is very likely • Universities seek impact • Researchers seek research participants • It is good for ecological validity that schools and classrooms participate. • But because of all kinds of privacy and safegaurding reasons increasingly difficult. • Societal responsiblity
  23. 23. Example collaborations
  24. 24. PhD work • Finished in 2011 • Worked with 12 teachers in 9 schools in the Netherlands • Algebraic skills training with technology (Bokhove & Drijvers, 2010)
  25. 25. MC-squared project • Design and develop a new genre of authorable e-book, which we call 'the c-book' (c for creative) • Creative Mathematical Thinking (CMT) • Initiate a ‘Community of Interest’ (CoI) • A community of interest consists of several stakeholders from various ‘Communities of Practice. • England, Spain, Greece, France • Within these, teachers who co-design and use resources for teaching, can contribute to their own professional development. • Social Creativity • UK CoI: learning analytics and feedback (e.g. Fischer, 2001; Wenger, 1998)
  26. 26. Building blocks
  27. 27. Building blocks
  28. 28. Summary - chances • So despite the challenges it can be done… • Needs action from both sides (I know: workload! But benefits could outweigh the ‘cost’) • But we can benefit from each other… schools gain a research culture, universities gain societal impact
  29. 29. In the meantime • Try to follow up sources as much as possible • Refrain from too firm a position, until you feel you have reviewed a fair amount of material, from different actors, might be a good strategy. Sorry – this is just hard work, and I understand that practitioners do not always have this time. • Be mindful of over-simplifications. • Follow the facts and, if one simplifies, be aware of the limitations or what it leaves out. See what I said about Willingham (2017). • Be cautious about developing policy based on new claims. • Research findings should be accompanied by a clear scope and disclaimer with regard to claims. No silver bullet. • Look up a researcher • They don’t bite.
  30. 30. Thank you • • Twitter: @cbokhove • Website: • Want to collaborate? • Please let me know. There are only two types of people in the world: those who believe in false dichotomies, and penguins.
  31. 31. Selected references Bokhove, C., & Drijvers, P. (2010). Digital tools for algebra education: criteria and evaluation. International Journal of Computers for Mathematical Learning, 15(1), 45-62. Online first. Fischer, G. (2001). Communities of interest: learning through the interaction of multiple knowledge systems. In the Proceedings of the 24th IRIS Conference. S. Bjornestad, R. Moe, A. Morch, A. Opdahl (Eds.) (pp. 1-14). August 2001, Ulvik, Department of Information Science, Bergen, Norway. Flack, Z. M., & Horst, J.S. (2017). Two sides to every story: Children learn words better from one storybook page at a time. Infant and Child Development, 27(1) Howard-Jones, P. (2014). Neuroscience and education: myths and messages. Nature Reviews Neuroscience, 15(12), 817-824. Mintrop, R. (2016). Design-Based School Improvement: A Practical Guide for Education Leaders. Cambridge. Harvard Education Press. Paas, F. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84(4), 429-434. Raaijmakers, S. F., Baars, M., Schaap, L., Paas, F. & Van Gog, T. (2017). Effects of performance feedback valence on perceptions of invested mental effort. Learning and Instruction, 51 36-46. Robinson-Garcia, N., Costas, R., Isett, K., Melkers, J., & Hicks, D. (2017). The unbearable emptiness of tweeting—About journal articles. PLOS one. Van Gog, T., Kirschner, F., Kester, L., & Paas, F. (2012). Timing and frequency of mental effort measurement: Evidence in favour of repeated measures. Applied Cognitive Psychology, 26, 833-839. Wenger, E. (1998). Communities of Practice: Learning, Meaning, Identity. Cambridge University Press. Willingham, D. T. (2012). When can you trust the experts: How to tell good science from bad in education . Willingham, D. T. (2017). A Mental Model of the Learner: Teaching the Basic Science of Educational Psychology to Future Teachers. Mind, Brain and Education, 11(4), 166-175.