The document provides an overview of research methods in open education. It begins with defining what research methods are on a basic level and then delves deeper into philosophical foundations such as ontology, epistemology, and metaphysics. It discusses how open approaches may differ from traditional methods both as the object of study and in the methods used. Insights from members of the GO-GN network are shared, including the types of methods used and a proposed structure for a research methods report. The document seeks feedback on the content and aims to support doctoral researchers in open education.

1. Research Methods in Open
Education
GO-GN Workshop
31st March 2020
Dr. Robert Farrow
Open Education Research Hub
Institute of Educational Technology
The Open University, UK
rob.farrow@open.ac.uk
@philosopher1978

2. 2
0
1 What is research method?
A “simple” and a complicated answer0
2 Philosophical foundations
Ontology, epistemology, metaphysics and
method
0
3 The difference open makes
Open approaches to method and open as an
object of study
0
4 Insights
Reflections shared by GO-GN members0
5 Research methods report
Next steps
STRUCTURE
Why do this work?
Rationale & ambition
0
6

3. Why do this work?
Rationale and ambition

4. 4
RATIONALE & AMBITION
WHY DO THIS WORK?
If you’ve been to a face-to-face seminar or workshop you’ll know that we offer people some one-on-one time to
discuss any issues privately; concerns about method are perhaps the most common issue that people don’t always
feel like discussing in a group.
Why might this be the case?
• Methodology is hard, and often pushes you outside your comfort zone
• Learners are rarely encouraged to critically engage with method before doctoral study
• People writing PhDs can suffer from imposter syndrome, or feel like they are expected to know all this stuff by
now
But openness also plays a part:
• Openness as object of study vs. openness as method
• Open practices challenge some of our assumptions about how to do research
• Different cultures can be more receptive to different kinds of research

5. 5
RATIONALE & AMBITION
WHY DO THIS WORK?
The GO-GN staff are experienced PhD supervisors and researchers. We know that method can be challenging
when researching open education!
One of the things that is challenging about method is that you can feel an obligation to be really innovative; or at
least to have mastered a particular approach.
Innovation in open education research is of course of interest to us as a community, but it’s also perfectly legitimate
to use established methods with new data. You don’t have to reinvent the wheel!
However, many of us are interested in whether we need new methods to study openness, including drawing in
perspectives from a wide range of disciplines.
The idea behind this effort is to draw on the experiences of people within the network to build understanding of
research methods in open so that everyone in the network (and beyond) might benefit.

6. What is research method?
A “simple” and a complicated answer

7. 7
SIMPLE ANSWER
WHAT IS RESEARCH METHOD?
Method:
• Approach or technique for conducting research
• How a study is completed
• How data was collected and analyzed
• Testing a hypothesis
• Supports a claim to knowledge or validity

8. 8
COMPLEX ANSWER
WHAT IS RESEARCH METHOD?
Method
Methodology
Theory
Philosophical Foundations
Techniques used
Systematization
Theoretical stance
Ontology, metaphysics, epistemology

9. Philosophical foundations
Ontology, epistemology, metaphysics and method

10. 10
ONTOLOGY
PHILOSOPHICAL FOUNDATIONS
Concept: “logical discourse about being”
Focus: What is there? What types of things are there? How can we describe existence? What is real?
Key terms: existence; categorization; differentiation; concretization; change; relation
In Philosophy, fundamental questions about existence is typically posed as an ontological investigation (and
has been since Classical times)
Domain ontology describes concepts and articles relevant to a particular discipline (e.g. a branch of science) or
(e.g. a branch of science) or line of inquiry
Interface ontology describes concepts and articles relevant to several disciplines
Ontology is often considered to be a sub-domain of metaphysics.

11. 11
METAPHYSICS
PHILOSOPHICAL FOUNDATIONS
Concept: literally “after Physics” or “above and beyond the physical”
Focus: What is the fundamental nature of reality? What kind of relations do different parts of reality have?
Key terms: explaining relationships, states of being; phenomena; categories of being (e.g. spatio-temporality;
minds; identity; necessity & possibility; freedom; essence etc.)
In Philosophy, metaphysics is a long-contested term which many have suggested is meaningless in the face of
modern (quantitative) science. Many of the traditional domains of metaphysics are not considered to be
scientific.
Some have suggested that metaphysics itself is (and always was) an impossibility.
Although highly abstract, most contemporary philosophers of science understand metaphysics as a necessary
part of science. See https://www.iep.utm.edu/met-scie/

12. 12
EPISTEMOLOGY
PHILOSOPHICAL FOUNDATIONS
Concept: “logical discourse about knowledge”
Focus: the study of knowledge and methods used to arrive at knowledge
Key terms: validity; justified belief; rationality; truth; cognition
Epistemology has a history as long as Philosophy, and in many ways represents the foundation of other forms
of knowledge. How can we claim to know anything at all? How can we meet the challenges of skepticism about
our beliefs? What does it mean to know something?
These are philosophically deep waters… yet remain a fundamental part of research (even if we often try to just
make it past unscathed).
Epistemology is the aspect most closely connected to method as they are both concerned with knowledge
creation (broadly construed).

13. 13
EPISTEMOLOGY & METHOD
PHILOSOPHICAL FOUNDATIONS
There is an obvious connection between ontology (what we believe there is) and epistemology (how we think
we know things) – it is typically not explored explicitly in research.
Research methods are essentially epistemologies – by following a certain process we support our claims about
the thing(s) we have been researching.
Poorly followed methods undermine our claims to have produced new knowledge.
Inappropriately chosen methods undermine our claims to have produced new knowledge.
Philosophically, this can be understood as “cognitive success” and “cognitive failure” (SEP, 2020).
Research methods are essentially stripped down, purpose-specific epistemologies

14. 14
THREE SCHOOLS OF THOUGHT
PHILOSOPHICAL FOUNDATIONS
See also Pham (2018); Ryan & Sfar-Gandoura (2018); Raddon (n.d.)
In practice approaches often overlap - but is open education aligned with one more than the others?
Paradigm Ontology Epistemology Observation Goal
Positivist Empiricism Skews objective,
quantitative
Privileged (Law-like)
Explanation’
instrumental
knowledge
Interpretivist Socially
constructed;
Relativism
Skews subjective,
qualitative
Embedded Understanding the
whole; meaning
Critical Realism Post-positivist,
Interpretivist,
Normative
Deconstructive-
discursive
Emancipation,
reveal domination

15. 15
APPROACHES ASSOCIATED WITH RESEARCH PARADIGMS
PHILOSOPHICAL FOUNDATIONS
Positivist Methods: Document coding; Experimental & Quasi-experimental design;
Isolating & measuring variables; Statistical analysis; Structured interviews; Surveys
Interpretivist Methods: Case Studies; Conversational analysis; Delphi; Description;
Document analysis; Interviews; Focus Groups; Grounded theory,
Thematic analysis
Critical Methods: Action research; Critical ethnology; Deconstruction; Dialectics;
Field research; Textual analysis
In additional to supporting specific methods, research paradigms also influence
nature of research, the researcher-participant relationship and how the role of the
interests) is understood.
In practice, work often happens in an interdisciplinary way, and/or uses mixed
When it comes to assembling the data into a whole and triangulating great care
Wehrden et al. (2019)

16. 16
Mittelmeier (2017)

17. 17
POSTPOSITIVIST CRITIQUE OF STATISTICAL VALIDITY
PHILOSOPHICAL FOUNDATIONS
https://aeon.co/essays/it-s-time-for-science-to-abandon-the-term-statistically-significant

18. 18
KUHN (1962) THE STRUCTURE OF SCIENTIFIC REVOLUTIONS
PHILOSOPHICAL FOUNDATIONS
Paradigm
Change
Normal /
Post-
revolutionary
Science
Model DriftModel Crisis
Model
Revolution

19. 19
GEORGE E. P. BOX ON VALIDITY
PHILOSOPHICAL FOUNDATIONS
“Essentially, all models are
wrong but some are useful”
(1976)

20. The difference open makes
Open approaches to method and open as an object of study

21. 21
MATURITY OF THE FIELD
THE DIFFERENCE OPEN MAKES
In recent years we have seen examples which support the idea that open education research is reaching a phase of
maturity and being recognized as a field in itself.
• Literature review
• Meta-analysis
• Discourse analysis
• Theoretical Analysis (e.g. concept of “open”)
• Importance of positionality
• Awareness of axiology?
See e.g. Jordan & Weller (2017)

22. 22
METHODOLOGICAL INNOVATION
THE DIFFERENCE OPEN MAKES
As a relatively new field, open education research has been obliged to establish its credentials scientifically.
Perhaps the best example of this are the OER efficacy studies (e.g. Hilton, 2016) which take a highly statistical
approach to understanding the impact of OER.
But what examples are there of innovation in research methods in open education?
• Use of open data
• Building tools for others to use
• Radical transparency
• Explicit claims to social justice
• Technological innovation

23. Insights
Reflections shared by GO-GN members

24. 24
OVERVIEW OF RESEARCH METHODS (WELLER, FARROW & PITT, 2019)
GO-GN INSIGHTS
Method n % (n=28)
Case studies 5 17.9%
Concept mapping 1 3.6%
Content analysis 4 14.3%
Critical discourse
analysis
2 7.1%
Design-based research 2 7.1%
Developmental
evaluation
1 3.6%
Educational data mining 1 3.6%
Ethnography 4 14.3%
Focus Groups 1 3.6%

25. 25
OVERVIEW OF RESEARCH METHODS (WELLER, FARROW & PITT, 2019)
GO-GN INSIGHTS
Method n % (n=28)
Grounded theory 2 7.1%
Institutional analysis 1 3.6%
Interviews 6 21.4%
Narrative inquiry 1 3.6%
Phenomenography 2 7.1%
Phenomenology 1 3.6%
Questionaire/Survey 6 21.4%
Social Network Analysis 1 3.6%
Systematic review 1 3.6%

26. 26
APPROACHES USED WITHIN THE NETWORK
GO-GN INSIGHTS
Quantitative: Demographics; Document analysis; KPI tracking; Mapping; Pattern
recognition; Quality metrics
HCI/Computer Science: Accessibility auditing; Learning Analytics; Social
Network Analysis; Virtual Learning Environment; UX
Qualitative: Axial coding; Biographical method; Focus groups; Interviews; Holistic
analysis; Interpretative research; Observations; Phenomenography; Phenomenology;
Interdisciplinary: Evaluation; Synthesis of existing research
Theoretical Lens/Frameworks: Action Research; Connectivism; Content Analysis;
Communities of Practice; Complexity Science; Critical realism; Enion Probability
Netography; Open Educational Practices; Phenomenology; Realist social theory;
determination Theory; Social Justice; Socio-technical perspective; System analysis;
Topologies; Value Creation Stories

27. Research Methods Report
Coming soon – but are we missing anything?

28. 28
NEXT STEPS
GO-GN INSIGHTS
The original plan was to work on the report collaboratively at the workshop to improve the draft before opening
it up for comments as a Google document
Today, we are asking for feedback on the proposed structure and content instead, but we will still invite
comments on the draft once it is more or less complete
The key thing today is making sure we are covering the right kind of content and supporting people in the right
ways
We also have some prompts for reflection/discussion

29. 29
PROPOSED STRUCTURE
GO-GN METHODS REPORT
INTRODUCTION
RESEARCH METHOD TYPOLOGIES
KEY ISSUES IN OPEN EDUCATION RESEARCH
GO-GN RESEARCH METHODS
GO-GN RESEARCH INSIGHTS
USEFUL RESOURCES
REFERENCES
(... ANYTHING MISSING ?)

30. 30
REFLECTION PROMPTS
RESEARCH METHODS REPORT
How do you frame your research? What motivates it?
• Describing what is happening (e.g. learner diversity in MOOCs)
• Identifying patterns (e.g. how are networks changing learner interactions?)
• Challenging existing narratives (e.g. ‘digital native’)
• Focus on something overlooked (e.g. importance of sociocultural factors)
• Supporting professional practice (e.g. educator development)
• Developing new theories
• Describing new trends (e.g. open education)
• Refine/redefine roles (e.g. MOOCs)
What will be the value of answering your research question? Can open approaches enhance or add value?
• Directly influencing practice
• Producing tools
• Data for re-use
• Open access publication
• Developing networks

31. THANK YOU
rob.farrow@open.ac.uk
@philosopher1978
go-gn.net
oerhub.net

32. 32
FURTHER READING
REFERENCES &
Box, G. E. P. (1976). Science and Statistics. Journal of the American Statistical Association, Vol. 71, No. 356 (Dec., 1976), pp. 791-799.
https://www.jstor.org/stable/2286841
Creswell, JW. (2003). Research Design: Qualitative and Quantitative and Mixed Method Approaches, Thousand Oaks, CA: Sage Publications.
Crotty, M. (1998) The Foundations of Social Research : Meaning and Perspective in the Research Process, London: Sage Publications.
Hilton, J. (2016). Open educational resources and college textbook choices: a review of research on efficacy and
perceptions. Education Tech Research Dev 64, 573–590. https://doi.org/10.1007/s11423-016-9434-9
Jordan, K. and Weller, M. (2017). Openness and education: a beginner’s guide. Global OER Graduate Network.
May, T. and Williams, M. (eds.) (1998) Knowing the Social World, Buckingham: Open University Press.
Mittelmeier, Jenna (2017). Supporting Intercultural Collaborations In Blended And Online Settings: A Randomised Control
Trial Of Internationalised Academic Content. PhD thesis. The Open University.
Pham, Lan. (2018). A Review of key paradigms: positivism, interpretivism and critical inquiry.. 10.13140/RG.2.2.13995.54569.
Punch, K. (2005) Introduction to Social Research: Quantitative and qualitative approaches, London: Sage Publications.
Raddon, A. (n.d.). Early Stage Research Training: Epistemology & Ontology in Social Science Research. University of Leicester.
https://www2.le.ac.uk/colleges/ssah/documents/research-training-presentations/EpistFeb10.pdf
Ryan, G. & Sfar-Gandoura, H. (2018). Introduction to positivism, interpretivism and critical theory. Nurse Researcher,
25(4) pp. 41–49.
SEP (2020). Epistemology. Stanford Encyclopaedia of Philosophy. https://plato.stanford.edu/entries/epistemology/
von Wehrden, H., Guimarães, M.H., Bina, O. et al. Interdisciplinary and transdisciplinary research: finding the common
ground of multi-faceted concepts. Sustain Sci 14, 875–888 (2019). https://doi.org/10.1007/s11625-018-0594-x
Weller, M., Farrow, R. & Pitt, R. (2019). GO-GN: Lessons in Building an Open Research Community. Pan-Commonwealth Forum 9
(PCF9). Murrayfield, Edinburgh. http://oasis.col.org/handle/11599/3336

33. 33
A NETWORK TO SUPPORT DOCTORAL RESEARCH IN OPEN EDUCATION
GLOBAL OER GRADUATE NETWORK
http://go-gn.net/