This document discusses key concepts related to research quality and methodology. It begins by defining research as a disciplined attempt to address questions or solve problems through collecting and analyzing primary data. Educational research is distinctive in that it focuses on understanding learning and teaching quality. The document outlines several features of quality in educational research, including rigor of the research process, trustworthiness, usefulness of implications, and originality. It also discusses the role of values and the researcher in research. Different research approaches like quantitative, qualitative, and mixed methods are examined, along with related issues around generalizability, validity, and reliability. Philosophies of research and the influences on social research are also addressed.

1. THINKING ABOUT RESEARCH AND
RESEARCH QUALITY IN YOUR
ACADEMIC WORK.
Presentation for the annual Oxford-Cambridge
Exchange
Pam Sammons and Linda Bakkum

2. WHAT IS RESEARCH?
 Research is a disciplined attempt to address questions or solve problems
through the collection and analysis of primary data for the purpose of
description, explanation, generalization and prediction (Anderson 1998, p 6)
 The nature of the subject matter determines what kind of research is valid or
relevant (Pring 2000, p 6)
 Reasoning: deductive (Aristotle) formal steps of logic
 inductive (Bacon) empirical evidence for verification
 Inductive-deductive moving from observations to hypotheses then back to
implications (backwards & forwards).
 Subjective belief must be checked against objective reality, research is self-
correcting.

3. WHAT IS DISTINCTIVE ABOUT EDUCATIONAL RESEARCH?
• The distancing of theory from practice is associated with public and policy
scepticism about value of educational research
• Need for clarity in defining key terms identified from your literature review and as
used in your study e.g. ‘good’ ‘effective’ c’ompetent’ teacher, what it means to be
an ‘educated’ person
• Need to attend to the ‘logic of the discourse’ the rules implicit in the use of
particular words and those to which they are logically related
• For Dewey “education concerned the development of the distinctively human
capacities of ‘knowing’ ‘understanding, ‘judging’; ‘behaving intelligently’ “(Pring
2000, p 12)
• What that makes your study distinctive in relation to the field of education?

4. KEY FEATURES OF EDUCATIONAL RESEARCH
• The attempt to make sense of the activities, policies and institutions which, through the
organisation of learning, help to transform the capacities of people to live a fuller, more
distinctively human life.
• The distinctive focus of educational research must be upon the quality of learning and thereby
teaching
• Much writing sets up a false dichotomy between different research traditions
• Variety in approaches to educational research is desirable, depending on questions explored
and philosophical position
• Is it the ‘real’ world that we observe – or one interpreted through my own personal & subjective
scheme of things?
• What is the connection between language and the world language is used to describe? After
Pring (2000)
• All links to notion of clarity in writing and argument & demonstrating
critical engagement with substantive, theoretical & methodological
literature

5. WHAT IS...?
 A research design is “an integrated statement of and
justification for the technical decisions involved in
planning a research project” (Blaikie, “Designing Social
Research”, p. 15).
 A research project is a temporary organisation that is
created with the purpose of carrying out systematic and
rigorous enquiry to address a particular problem arising
from a gap in knowledge (a theoretical puzzle, a
pragmatic need etc).

6. FEATURES OF QUALITY IN (EDUCATION)
RESEARCH
 Rigour of research process
 Trustworthiness
 Reliability/ validity
 Usefulness – implications for research
methodology, for policy &/practice in education
 Originality
 Contribution to theory?
How can you demonstrate rigour in
these areas in your study?

7. WHAT IS PHILOSOPHY OF EDUCATIONAL
RESEARCH?
A “second order activity” which explores
 the beliefs about the nature of (social) reality or of a
phenomenon (including self and other – “what exists,
what it looks like, what units make it up and how these
units interact with each other”) - ontology
 the beliefs about the nature of educational research
knowledge (and its relationships to other kinds of
knowledge) - epistemology
 the beliefs about principles and values (including the
right, the good and the virtuous) in the practice of
educational research - axiology
(see D. Bridges, 2003, p. 15; N.Blaikie, 2000, p. 8)

8. WHAT IS/ARE YOUR...
Quantitative?
Qualitative?
Mixed Methods?
•Positivist ?
•Post-positivist?
•Critical Theory
•Constructivism
•Pragmatic?
HOW DOES THIS AFFECT YOUR CHOICE OF:
• RESEARCH AIMS & QUESTIONS?
• RESEARCH DESIGN & METHODOLOGY?
...Ontological position?
...beliefs about epistemology?

9. QUANTITATIVE VERSUS QUALITATIVE
 Some researchers have argued that it may be
appropriate to think of Qualitative & Quantitative as
being on a continuum Gray and Densten (1998),
Tashakkori & Teddlie 2003
 ‘Qualitative and quantitative choices viewed as polar
opposites may be viewed as a ‘false dualism’ (Frazer
1995)
 Can you clarify & justify your own view and approach in
your study?
 How has your view evolved over the course of your PhD
research?

10. Pragmatism supports the use of both QUAL & QUAN
methods in the same study & rejects the either/or
incompatibility thesis
It considers the research questions to be more important
than either the method or paradigm that underlies the
method – the dictatorship of the RQ
Pragmatism
avoids the use of metaphysical concepts eg ‘truth’
‘reality’
Pragmatism presents a very practical & applied philosophy
After Tashakkori & Teddlie 2003 p 20-21
Pragmatism as the Foundation for MM Research

11. RESEARCH QUESTIONS
 The ‘big’ research question: one over-arching
question
 The sub-questions which help to guide your enquiry
Characteristics of Good Research Questions
 Clarity
 Empirical focus
 Accessible evidence
 Manageable
 Awareness of
assumptions
 Awareness of implicit
values
 Awareness of political
implications
 Related to previous
research
 Significant
 Ethical
 Practical use (relevant)
 ‘fun’ (interesting to you)
source: Ingrid Lunt.

12. CLARITY
 The question(s) in your study should be
answerable i.e can be illuminated or addressed by
your methodology (you are looking to find the
answer to a genuine question)
 The question should be intelligible to the reader
who may not be an ‘expert’ in your topic
(understandable)
 The questions should offer the prospect of making
an ‘original contribution to knowledge’ in some way
(methodologically theoretically empirically etc)
 Are the terms clearly defined?
 Are the questions precise?
source: Ingrid Lunt.

13. EMPIRICAL FOCUS
 Require that you generate data to answer question
 Lead you to determine methods of enquiry and data
collection
 NB it is usually most appropriate for methods to follow
questions; different types of questions will lead to
different approaches to research and methods of data
collection, but this is not always the case
Reflect on your own Research aims/RQs:
 How have they evolved over the course of your study
 How far have they driven your choice of design &
methodology and the specific methods you are using?
source: Ingrid Lunt.

14. SIGNIFICANT
 Is there a clear rationale for the question?
 So what?
 Does this question matter?
 Why is it of interest and to whom?
source: Ingrid Lunt.

15. POSSIBLE AIMS & OBJECTIVES
 Description: what does it look like (what, when, where,
who)?
 Explanation: why did it happen?
 Prediction: what is to be expected?
 Understanding: how is it grasped in human experience?
 Interpretation: what does it mean?
 Prescription: how ought it be?
 Change and emancipation: how can it be transformed
for the better?
 Critique and disruption: what are the limitations and
hidden assumptions? How can these assumptions be
challenged/ interrupted?
 Etc. (e.g., exploration, demonstration, classification)

16. AIMS AND CLAIMS KINDS OF RESEARCH
QUESTION
EXAMPLES OF RESEARCH
Explanatory What is the relationship
between?
Survey, experiment
Explanatory
Descriptive
Prescriptive
What happens if . . . ? Experiment, participatory
research, action research
Descriptive
Explanatory
‘What’ and ‘why’? Mixed methods research
Explanatory
Descriptive
What happened in the past/
how to make sense of the
past?
Historical research
Understanding
Interpretative
How can we understand a
situation?
Ethnographic and interpretive/
Case study
Critique
Emancipatory
How to disrupt convention and
empower participants?
Critical approaches
As by Alis Oancea.

17. SOME INFLUENCES ON SOCIAL RESEARCH
• These can affect choice of
research topic, formulation of
research questions, choice of
methods, choice of research
design and instruments, ethics,
sample & process of data
collection, interpretation of data
and findings, conclusions,
reporting and dissemination
• Need be self-reflective, and to
exhibit reflexivity about the part
played by the researcher’s own
values and their potential
influence on research process
and outcomes
• Existing knowledge base on
topic, is this a new topic of
interest? (generation or testing
of theory more appropriate?),
resources available,
availability/interest of
participants
• All social research is a coming
together of the ideal and the
feasible
Values Practical Considerations
As by Alis Oancea.

18. ROLE OF VALUES & OF RESEARCHER
• The value determined nature of enquiry in anti- positivist
research such as Critical theory and Constructivism, Advocacy
and activism encouraged, researcher transformative intellectual
or passionate participant
• What can be known is mediated by interaction between
investigator and subject of investigation
• For constructivists there are multiple realities, that depend on the
individuals or groups holding constructions, constructions may
change/be altered and thus so can ‘realities’
• Researcher and subject are interactively linked and findings are
created through hermeneutical and dialectical techniques and are
relative
• Aims to critique & transform (critical theory) or to understand &
reconstruct, subject to continuous revisions.

19. HOW VALUES MAY INFLUENCE SOCIAL RESEARCH
• Choice of research area
• Formulation of research questions
• Choice of method
• Formulation of research design & data collection techniques
• Implementation of data collection
• Interpretation of data
• Conclusions drawn

20. BODIES OF KNOWLEDGE
• Theories, propositions and explanations accumulated
through enquiry, criticism, argument and counter
argument. What has survived testing and
criticism…public property. Their credentials depend
upon their being open to public challenge and
refutation.
• Any body of knowledge can only be provisional and
is open to further challenge through criticism.. The
link between knowledge & certainty is broken.
• Disciplined, critical and reflective thinking is the mark
of educational research, at odds with unquestioning
‘common sense’ beliefs.

21. POINTS TO ESTABLISH IN EXAMINING DIFFERENT RESEARCH
APPROACHES & IN CRITICAL READING OF RESEARCH
• Research assumptions - are they explicit?
• Aims – explanation or understanding
• The subjective-objective dimension
• Role and definitions of theory
• Doing research/reading research
• Theoretical and empirical domains
• Values and interpretation
• Use of findings/ audience
• Stages in the development of enquiry

22. ISSUES IN QUALITATIVE RESEARCH
 Dependability,
consistency,
comprehensiveness,
‘checkability’, empathy,
uniqueness, explanatory
and descriptive potential,
confirmability,
“neutrality”, applicability,
transferability
Generalisability Validity Reliability
 Often concerns:
honesty, credibility,
richness, authenticity,
depth, scope,
subjectivity, strength
of feeling, capturing
uniqueness,
idiographic
statements, fidelity to
participants’ accounts
 Enriching understanding
and generating theory
 Fuzzy Generalisations
 Falsification
 Using extreme
(most/least likely to
fit theory), atypical,
and critical cases
As by Alis Oancea.

23. STRATEGIES FOR...
 Careful, sometimes
strategic selection of
cases
 Intense participation and
effort to develop valid and
rich descriptions
 Challenging theories,
conventional wisdom,
and prior assumptions
 Letting the case “talk
back” – sensitivity to
diversity, uniqueness,
history and context
 Good preparation for fieldwork
 Piloting and peer and
participant debriefing
 Justification of decisions (e.g.
transcription; recording; types
of questions; extent of
‘mapping’ and ‘summarising’ in
case presentation etc.)
 Awareness of transcriber
selectivity and other limitations
 Independent audits and audit
trails
 Multiple coders
Generalisability Reliability
As by Alis Oancea.

24. STRATEGIES FOR VALIDITY
 Prolonged engagement
in the field
 Persistent observation
 Rich and thick
description
 Leaving an audit trail
 Reflexive diaries
 Respondent validation
 Peer debriefing
 Checking for researcher
effects
 Making
contrast/comparisons
 Ruling out spurious
relations
 Following up surprises
 Using extreme cases
 Assessing rival
explanations
 Triangulation
 Back translation
As by Alis Oancea.

25. SOME ISSUES IN QUANTITATIVE RESEARCH
 Fundamentally
concerned with the
reliability of measures.
 Stability
 Dependability
 Replicability
 Internal reliability
 Inter-observer
consistency
Generalisability Validity Reliability
 Measurement valididy,
face validity,
concurrent validity
predictive validity,
construct validity,
convergent validity
 Role of confidence
intervals

 Can findings be
generalised outside
the sample?
 Importance of sample
 Concept of statistical
probability

26. STRATEGIES FOR...
 Test – Retest
 Chronbach Alpha
 Multiple coders
 Consider the consistency
of your observations.
 Controllable, predictable,
consistent, replicable.
(Cohen et al. 2007)
Generalisability Validity Reliability
 Appropriate
instrumentation,
 Appropriate treatment
of statistical data
 Careful sampling
 At best strive to
minimize invalidity and
maximize validity
 Careful sample
selection.
 Random selection can
be useful because of
known properties.
 Be cautious with
making inferences.

27. MIXED METHODS APPROACHES
 All the same problems as with
Quant and Qual!
But also:
 Design choice
 Data synthesis
 Can your data inform one
another?
 Two separate studies?
 Quant and Qual findings dont
match?
 Skill and confidence in both
research approaches?
 Should be more than the sum
of its parts.
 Careful design of each (Qual
and Quant) component.
 Think about how your data
might be used to inform one
another.
 Explore what the combined set
of findings indicate.
 If not confident with a particular
method, hit the books, ask for
help!
Issues Strategies

28. A Dynamic Conceptual Model for MM research
• QUAN MIXED METHOD QUAL
• Sphere of Concepts (abstract operations) Purposes Questions
• Deductive Qs ............................................ Inductive Qs
• Objective purpose ............................................ Subjective purpose
• Value neutral ............................................ Value informed
• Politically neutral ............................................. Transformative
• Experiential sphere (concrete observations & operations) Data
Observation
• Numerical data .............................................. Narrative data
• Structured process .............................................. Emergent process
• Statistical analysis ............................................... Content analysis
• Sphere of Influence (abstract explanations & understandings)
Theories Explanations Inferences
• Deductive logic ............................................... Inductive logic
• Objective inference ............................................... Subjective inference
• Value neutral ............................................... Value involved
• Politically neutral ............................................... Transformative
•
after Tashakkori & Teddlie 2003

29. MM Designs characterised by
 Multiple positions along each attribute traditionally assumed to distinguish
QUAN & QUAL eg they have both confirmatory and exploratory research
questions
 They are near the end of one continuum on one attribute ( eg inductive
questions but near the other end of the continuum on another attribute eg
statistical analysis)
Multiple Method Designs (more than 1 method or more than 1 world view
A. Multi method designs ( more than 1 method but restricted to within 1
world view (eg Quan/Quan or Qual/Qual)
B. Mixed methods designs (use of QUAL & QUAN)
 Mixed method research (occurs only in methods stage of a study)
 Mixed model research (can occur in all stages of a study )
after Tashakkori & Teddlie 2003

30. MM Designs characterised by
• Multiple positions along each attribute traditionally assumed to distinguish QUAN &
QUAL eg they have both confirmatory & exploratory research questions
• They are near the end of one continuum on one attribute ( eg inductive questions but
near the other end of the continuum on another attribute eg statistical analysis )
1. Multiple Method Designs (more than 1 method or more than 1 world view)
A. Multi method designs ( more than 1 method but restricted to within 1 world view eg
Quan/Quan or Qual/Qual)
B. Mixed methods Designs (use of Quan & Qual methods/data collection/analysis
strategies)
1. Mixed Method research (occurs in the methods stage of study only)
2. Mixed Model research (can occur in all stages of a study)
– Concurrent Mixed Method design one kind of question simultaneously
addressed by collecting & analysing QUAN & QUAL data then one type
inference made from both sources
– Concurrent mixed Model 2 strands of research with both types of question, both
types of data & both types of analysis then both types of inferences are pulled
together to create meta-inferences at the end
after Tashakkori & Teddlie 2003

31. Purpose / Question
Data Collection
Data Analysis
Meta - Inference
Concurrent Mixed Model Design (Fig 26.6 p688)
Inference
Inference
Data Analysis
Data Collection
Purpose /
Question

32. Purpose / Question
Data Collection
Data Analysis
Meta - Inference
Sequential Mixed Model Design (Fig 26.8 p688)
Inference
Inference
Data Analysis
Data Collection
Purpose /
Question

33. Purpose / Question
Data Collection
Data Analysis
Meta - Inference
Fully Integrated Mixed Model Design (Fig 26.11)
Inference
Inference
Data Analysis
Data Collection
Purpose /
Question

34.  Your research will be informed by your
readings.
 Critical reading of the literature is a major
part of good research!

35. HOW TO READ RESEARCH ARTICLES CRITICALLY (AN
APPRECIATION OF STRENGTHS & WEAKNESSES/LIMITATIONS)
 Identify research aims/questions?
 Identify nature/type of study (scholarly review, empirical
work, new or secondary analysis)
 Identify ontological position, epistemological &
methodological assumptions
 Is researcher’s value position explicit?
 Identify location, date, sample, methods used
 Examine use of theory, deductive? Inductive?
 Are analysis methods clearly explained ?
 Are conclusions appropriately supported by evidence ?
 What are the implications for policy/practice?
.

36. FINAL COMMENTS
 No study can be ‘perfect’
 Research rigour is about clarity of research process throughout
 Justifying your choices, design, interpretations, conclusions
 Persuasion of arguments re original contribution
 Awareness of strengths & limitations
 How your research fits into existing body of knowledge
 Implications for policy practice, future directions for research
 Your viva involves an oral ‘defence’ a justification of
the rigour of your research to probe your
understanding and ‘ownership’ of your study
 It is helpful to practice thinking, talking about and
presenting your study with special attention to
demonstrating rigour