Paper by Alden Rivers & Richardson (2014) presented at the 7th International Higher Education Close Up Conference, 21-23 July, 2014, Lancaster, UK

1. Exploring data driven learning
design for negotiating troublesome
concepts
Bethany Alden Rivers, The University of Northampton
John Richardson, The Open University

2. Epistemic beliefs
are our ideas about
‘knowledge and
knowing’.
(Hofer, 2002, p. 3)

3. ‘…how students themselves think
about knowledge, learning and
teaching is a primary factor
influencing their experience of
higher education itself.’
(Richardson, 2013, p. 192).

4. Universities need to care more
about developing learners’
epistemologies.
(Lucas & Tan, 2013)

5. The problem…
Researchers need better ways of understanding epistemic
beliefs (Schraw, et al., 2002).
‘Well-validated quantitative instruments that could be used
to measure epistemological development in large samples
of students are still lacking’ (Richardson, 2013, p. 201).

6. Knowing and Reasoning Inventory (KARI)
Measure of Epistemological Review
(Baxter Magolda, 1992)
Argumentative Reasoning
(Kuhn, 1992)
Reasoning Knowing+

7. Theoretical framework
Reasoning
(Kuhn & Weinstock, 2002)
Knowing
(Baxter Magolda, 1996)
Realist
Absolutist
Multiplist Transitional Knowing
Independent Knowing
Contextual Knowing
Absolute Knowing
Evaluativist

8. Creating the KARI
• Converting qualitative interview protocol into
quantitative ‘statements’.
• 5-point Likert scale
• 3 demographic questions: age, gender, educational
background
• 12 questions related to Reasoning
• 20 questions related to Knowing

9. Examples of KARI statements for
Reasoning
• Experts know what causes prisoners to return to
crime.
• It is possible to have more than one point of view
as to what causes prisoners to return to crime.
• Compared to the average person, I know a lot
about what causes prisoners to return to crime.

10. Examples of KARI statements for Knowing
• The goal of learning is to come up with my own
perspective on things. (Role of the Learner)
• My classmates play a very important role in my
own learning. (Role of Peers)
• Not everything my instructor says is true because
we all have our own beliefs about things. (Nature
of Knowledge)

12. Validating the KARI
• Phase 1: 77 students at the University of Northampton in
May 2014
• Excellent inter-reliability among the Reasoning and among
the Knowing questions
• Some interesting and statistically significant relationship
between Reasoning, Knowing and Demographics

13. What can an epistemological profile
allow us to do?

14. Our storyline
A learner’s conceptual development is ‘shrouded in
distinctive, epistemic modes of reasoning and explanation’
(Baillie et al., 2013, p. 234).
A greater awareness of students’ beliefs about the nature of
knowledge can illuminate students’ conceptual
development (Buehl & Alexander, 2001).

15. Exploratory workshop
• 10 members of staff at the University of Northampton
• Academic staff, learning technologists, learning designers, heads of learning
and quality
• Theoretical frame: Threshold Concepts Framework (Meyer & Land, 2003)
How might KARI data be used by learning designers
to promote university students’ conceptual
development?

16. Three activities
 Identify a threshold concept
 List three activities for grasping this threshold
concept (see Baillie et al.’s 2014 Capability model)
 Consider how a student’s KARI profile may
influence the design of these activities

18. Theme 1: Threshold concept is a
threshold concept.
My own approach to ‘learning design’ failed
to account for ‘pre-liminal variation’.
For further reflection: how can those who
influence course design become more
familiar with conceptual development
frames, such as the TCF?

19. Theme 2: Learning design is for the
cohort, not for the individual.
“I would not create different
scenarios. I'd use exactly the same
activities for both students. The
output and subsequent outcome of
a single "learning situation" would
be different, perhaps, but the
scenario for learning would be the
same.”

20. Theme 3: Visualisations of data need to
be more meaningful.
“there is too much new
information for some folk
[sic] to take in”.

21. Next steps?
• Rework the KARI and continue to validate the instrument
• Try new visualisations—what does a strong profile look
like compared to a developing or weak profile?
• Learn more about how such data may or may not be
useful to staff or students

22. References
Baxter Magolda, M. (1992) Knowing and Reasoning in College: Gender-Related Patterns in
Students’ Intellectual Development, San Francisco, Jossey-Bass.
Hofer, B. K. (2002) ‘Personal epistemology as a psychological and educational construct: an
introduction’, in Hofer, B. K. and Pintrich, P. R. (eds.) Personal Epistemology: The Psychology of
Beliefs about Knowledge and Knowing, Mahwah, NJ, Erlbaum.
Lucas, U. and Tan, P. L. (2013) ‘Developing a capacity to engage in critical reflection: students’
‘ways of knowing’ within an undergraduate business and accounting programme’, Studies in
Higher Education, vol. 38, no. 1, pp. 104-123.
Kuhn, D. (1991) The Skills of Argument, Cambridge, Cambridge University Press.
Richardson, J. T. E. (2013) Epistemological Development, Educational Research Review, vol. 9, pp.
191-206.
Schraw, G., Bendixen, L. D., & Dunkle, M. E. (2002). ‘Development and validation of the Epistemic
Belief Inventory (EBI)’. In B. K. Hofer & P. R. Pintrich (Eds.),Personal epistemology: The psychology
of beliefs about knowledge and knowing (pp. 261–275). Mahwah, NJ: Erlbaum.

23. Bethany Alden Rivers
Bethany.Alden@northampton.ac.uk
John Richardson
John.T.E.Richardson@open.ac.uk
Twitter: @Alden_Rivers