The recent move towards more 'evidence informed teaching' has seemingly increased the interest of schools in researchers and academia, and of academia in schools. This development presents us with several questions. How can we best organise this relationship? Do teachers need research literacy skills? Why would academia be interested in cultivating teacher-researcher links? This talk will explore these questions and give several examples of research projects, most concerning mathematics in secondary education, where fruitful collaboration between me as researcher and classroom teachers, brought benefits to the school and higher education. I will try to convey what opportunities and chances there are in working together, but also -in light of workload discussions- some of the risks.
The tricky relationship between research and practice
The tricky relationship
between research and
29 June 2018
Evidence informed education
• Evidence-based medicine
• Evidence-based practice
• Evidence-informed education
"Evidence-based practice is about making
decisions through the conscientious, explicit
and judicious use of the best available
evidence from multiple sources by …
Translating a practical issue or problem into an
Acquiring: Systematically searching and retrieving the evidence
Critically judging the trustworthiness and relevance of the
Aggregating: Weighing and and pulling together the evidence
Incorporating the evidence in the decision-making
Assessing: Evaluating the outcome of the decision taken
… to increase the likelihood of a favorable outcome."
Evidence-informed still means ‘based on scientific
research’ but we’re in the field of learning sciences,
which is an interdisciplinary science
• Cognitive psychology
“dealing with so many variables that are extremely
hard to (all) control.”
Some age-old issues 1
• Cause and effect
• Hume’s Skepticism and Kant
• Correlation and causation
• Popper and Kuhn
• Does one study falsify a body of research?
• Berliner: ‘the hardest science of them all’
Some age-old issues 2
• A lesser form of knowledge?
• Labaree: soft knowledge.
• Negative: lower status, weaker authority, push
• Positive: knowledge for practice, free from consumer
pressures, no disciplinary boundaries, general public.
Some age-old issues 3
• Quantification and measurement
• To improve credibility, origins ‘statistics’
• But could destroy local practical knowledge
• What are we measuring (fMRI, SES, mindset, load)
This requires a vast knowledge base…
“Modal models for prospective teachers should use folk
psychological terms whenever doing so will not
mislead. For example, the core properties of working
memory can be conveyed with the intuitive
observations that there is a mental “space” for thinking,
that this space is limited, and that it can be occupied by
things perceived in the environment and/or things from
the long-term memory.”
But the key question then becomes: when misleading?
A thought-provoking example
• Everyone loves Cognitive Load Theory
• It is good science not bad science
• So how can you be critical of it?
• Being a Skeptic
• Being skeptical does not mean I disagree with it
• It forces people to reflect on their thinking
• Let me ‘strip it and flip it’ (Willingham, 2012)
REDUCE COGNITIVE LOAD
This is often interpreted as to ‘take away’
Load from words
R I S D E S
Add a P and I can make SPIDERS
A unit with meaning
Load interacts with prior knowledge and schemas.
Load from picturebooks research
(Flack & Horst, 2017)
Rest of slide Intentionally left blank to not impose too much load.
Scale originates from Paas (1992)
Feedback might reduce load
“Across the three experiments–with different
problem-solving tasks and participant populations–
we found that subjective ratings of effort
investment were significantly higher after negative
than after positive feedback; ratings given without
feedback fell in between. These findings show that
feedback valence alters perceived effort investment
(possibly via task perceptions or affect), which can be
problematic when effort is measured as an indicator
of cognitive load.”
(Raaijmakers et al., 2017)
Moment of measuring…
“The findings from Experiment 1 (between-subjects) and
2 (within-subjects), using different arrangements of
simple and complex tasks, showed that a single rating
after a series of tasks resulted in a higher mental effort
score than the average of ratings provided immediately
after every task. A similar result was obtained in
Experiment 3 with series of complex tasks, but not with
simple tasks. Experiment 4 showed that knowing
beforehand that mental effort rating will be required
after completing all tasks results in lower scores, but
average retrospective ratings per task still differed from a
single retrospective rating.”
(Van Gog et al, 2012)
• NOT that we can’t get something out of this, but…
• “It’s complex”
• Messages get distorted as they propagate through
• So even a statement like “REDUCE COGNITIVE
LOAD” must be unpicked.
How do misconceptions start?
“examples of cases in which entrepreneurs have
knowingly set out to mislead educators are difficult
to find.” (Howard-Jones, 2014, p. 817)
“more likely that such interventions originate from
uninformed interpretations of genuine scientific facts
and are promoted by victims of their own wishful
thinking who hold a “sincere but deluded fixation on
some eccentric theory that the holder is absolutely
sure will revolutionize science and society” (Howard-
Jones, 2014, p. 817)
• Cultural conditions e.g.
• Also check Lilienfeld et al.
(2015, 2017) for lists with
psychological terms to
avoid and pairs of
difficult to access
Robin-Garcia et al. (2017)
• “tweeting about scholarly articles
represents curating and
informing about state-of-the-art
appears not to be realized in
• “Simplistic and naïve use of
social media data risks damaging
the scientific enterprise,
misleading both authors and
consumers of scientific
• (Now check the paper, it says
much more, and it has its own
Do teachers need research
• Ideally, yes, as much as possible
• But it is clear this can be quite challenging for all
kinds of reasons, ranging from time, knowledge,
So why not *work* with Academics?
Academia interested in links?
• This is very likely
• Universities seek impact
• Researchers seek research participants
• It is good for ecological validity that schools and
• But because of all kinds of privacy and safegaurding
reasons increasingly difficult.
• Societal responsiblity
• Finished in 2011
• Worked with 12 teachers in 9 schools in the
• Algebraic skills training with technology
(Bokhove & Drijvers, 2010)
• Design and develop a new genre of authorable e-book,
which we call 'the c-book' (c for creative)
• Creative Mathematical Thinking (CMT)
• Initiate a ‘Community of Interest’ (CoI)
• A community of interest consists of several
stakeholders from various ‘Communities of Practice.
• England, Spain, Greece, France
• Within these, teachers who co-design and use
resources for teaching, can contribute to their own
• Social Creativity
• UK CoI: learning analytics and feedback
(e.g. Fischer, 2001; Wenger, 1998)
Summary - chances
• So despite the challenges it can be done…
• Needs action from both sides (I know: workload!
But benefits could outweigh the ‘cost’)
• But we can benefit from each other… schools gain a
research culture, universities gain societal impact
In the meantime
• Try to follow up sources as much as possible
• Refrain from too firm a position, until you feel you have reviewed a
fair amount of material, from different actors, might be a good
strategy. Sorry – this is just hard work, and I understand that
practitioners do not always have this time.
• Be mindful of over-simplifications.
• Follow the facts and, if one simplifies, be aware of the limitations or
what it leaves out. See what I said about Willingham (2017).
• Be cautious about developing policy based on new claims.
• Research findings should be accompanied by a clear scope and
disclaimer with regard to claims. No silver bullet.
• Look up a researcher
• They don’t bite.
• Twitter: @cbokhove
• Website: www.bokhove.net
• Want to collaborate?
• Please let me know.
There are only two types of
people in the world: those
who believe in false
dichotomies, and penguins.
Bokhove, C., & Drijvers, P. (2010). Digital tools for algebra education: criteria and evaluation. International Journal of Computers for
Mathematical Learning, 15(1), 45-62. Online first.
Fischer, G. (2001). Communities of interest: learning through the interaction of multiple knowledge systems. In the Proceedings of the
24th IRIS Conference. S. Bjornestad, R. Moe, A. Morch, A. Opdahl (Eds.) (pp. 1-14). August 2001, Ulvik, Department of Information
Science, Bergen, Norway.
Flack, Z. M., & Horst, J.S. (2017). Two sides to every story: Children learn words better from one storybook page at a time. Infant and
Child Development, 27(1)
Howard-Jones, P. (2014). Neuroscience and education: myths and messages. Nature Reviews Neuroscience, 15(12), 817-824.
Mintrop, R. (2016). Design-Based School Improvement: A Practical Guide for Education Leaders. Cambridge. Harvard Education Press.
Paas, F. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of
Educational Psychology, 84(4), 429-434.
Raaijmakers, S. F., Baars, M., Schaap, L., Paas, F. & Van Gog, T. (2017). Effects of performance feedback valence on perceptions of
invested mental effort. Learning and Instruction, 51 36-46.
Robinson-Garcia, N., Costas, R., Isett, K., Melkers, J., & Hicks, D. (2017). The unbearable emptiness of tweeting—About journal articles.
PLOS one. https://doi.org/10.1371/journal.pone.0183551
Van Gog, T., Kirschner, F., Kester, L., & Paas, F. (2012). Timing and frequency of mental effort measurement: Evidence in favour of
repeated measures. Applied Cognitive Psychology, 26, 833-839.
Wenger, E. (1998). Communities of Practice: Learning, Meaning, Identity. Cambridge University Press.
Willingham, D. T. (2012). When can you trust the experts: How to tell good science from bad in education .
Willingham, D. T. (2017). A Mental Model of the Learner: Teaching the Basic Science of Educational Psychology to Future Teachers.
Mind, Brain and Education, 11(4), 166-175.