The document discusses the importance of rigor and robustness in research. It aims to provide insight into the meaning of rigor and how it can be maintained throughout the different stages of the research process, regardless of the research approach or paradigm used. Specifically, it discusses strategies for achieving rigor in areas like sampling techniques, data collection, data analysis, and reporting findings. Maintaining rigor is important for producing robust and reliable research results.

1. Rigour and Robustness
in Research
Dr Jennifer Loke
PHD; MBA; MSC; PGDIP; BN; ADVDIP, SFHEA;
LPE; RN
© J. Loke 2014

2. Aims
• To gain insight to the meaning of rigour and
its importance for robustness in all research.
• To gain understanding on how rigour can be
maintained to achieve robustness in the
different stages of the research process of
the different research approaches
(paradigms).
• To appreciate the strategies to achieve rigour
in the different stages of research.
© J. Loke 2014

3. Learning Outcomes
At the end of the session the student will be able
to:
o understand the meaning and the importance of rigour in
research;
o understand how rigour is judged in the different research
paradigms;
o describe ways to achieving rigour in research studies;
o identify some of the common pitfalls in the process of
achieving rigour in research;
o highlight research knowledge and skills as appropriate to
address these pitfalls.
© J. Loke 2014

4. © J. Loke 2014

5. By John Stevens
PUBLISHED:MailOnline 14:03, 20 June 2012 | UPDATED: 12:27, 21 June 2012
One McBurger with lies, please: Fast-food firm's own video reveals why items
on the menu never look like what you are served
© J. Loke 2014

6. © J. Loke 2014
The quality of being extremely
thorough and careful
Rigour Robustness

7. © J. Loke 2014

8. Replicability
not
reproducibility
Trouble at the lab. The
Economist. October 2013
© J. Loke 2014

9. Types of Research
QUANTITATIVE QUALITATIVEMixed-Methods
© J. Loke 2014

10. Research paradigms
Quantitative designs Qualitative approachesMixed methodology
approaches
 LOKE, J. Jan2012
Located in a continuum

11. The goal of Quantitative research
© J. Loke 2014
Generally, to quantify the problem and
understand how prevalent it is by looking
for projectable results to a larger
population

12. Rigour in Quantitative Research
ReliabilityValidity
© J. Loke 2014

13. The goal of Qualitative research
© J. Loke 2014
Generally, to define the problem or
develop an approach to the problem.
Aim to contribute to an ongoing dialogue,
in the form of debate, discussion or
argument about a phenomenon

14. Rigour in Qualitative Research
Credibility
Trustworthiness
© J. Loke 2014
Dependability

15. Rigour in Qualitative Research – drawing parallels with
quantitative research
Reliability
Validity
Credibility
Trustworthiness
© J. Loke 2014
Validity
Dependability
Reliability

16. Strategies
to achieve rigour and robustness
Research approach/paradigm
• Methodological approach
• Sampling techniques
• Data collection
• Data analysis

17. Strategies
to achieve rigour and robustness
Research approach/paradigm
• Professional/Personal bias
– Implicitly expressed (quantitative research)
– Explicitly expressed (qualitative research)

18. Rigour through proper selection of research
paradigm/methodological approaches
• Personal/Professional Bias – prevent highlighting
some issues while obscuring others?
• Address ethical aspects of research – which
sometimes can be compromised due to political
reasons
Reflexivity –
qualitative research
© J. Loke 2014

19. Strategies
to achieve rigour and robustness
Methodological approach
Quantitative vs Qualitative

20. Research methodologies
Randomised control Trial
(RCT)
Quasi RCT (cluster/group
randomisation
Survey-based approach
Discourse analytic work:
Corpus analytic work
Grounded theory
Discourse analytic work
Life history
Semiotic analytic work
Ethnography
Ethno-methodology
Phenomenology
Quantitative
in nature
Qualitative
in nature
 LOKE, J. Jan2012

21. Selecting an appropriate research design
• Quantitative
• Qualitative
– Triangulation of theoretical framework
• Mixed-Methods - approach
Credibility
Theoretical
framework 1
Theoretical
framework 2
Data Sources
© J. Loke 2014

22. Strategies
to achieve rigour and robustness
Sampling Techniques
Participants selection
Document selection
Event selection
Representativeness

23. Strategies
to achieve rigour and robustness
Data collection
Participants selection
Document selection
Event selection
what, where, when and how

24. Data collection in research
© J. Loke 2014
Common techniques for producing data
Indirect
Observations
Direct
Observations
Elicitation
Techniques
Mixed Methods
Continuous
or Spot
monitoring
Scientific tools,
Unstructured, Semi-
Structured,
Structured, Mixed
Case studies,
Participants
Observations
What
Technique?

25. Strategies to achieve rigour
© J. Loke 2014
• Knowledge and experience of data
collector
• Appropriateness of the collection
methods for the research objectives
and settings
• Relationship between data collector
and data (participants/
texts/historical documents)
• Addressing ethical issues

26. Consider the different quantitative and
qualitative techniques in Data Collection
• Data Collection
Reliability
Data
Source 1
Data
Source 3
Data
Source 2
Credibility
Observer
2
Observer
3
Observer
1
Trustworthiness
Observer
Participant/
member
checking
Data
source
© J. Loke 2014
Where/When/How

27. Strategies
to achieve rigour and robustness
Data Analysis
Depends on the research
approach

28. Key qualitative and quantitative distinctions
DATA
Analysis Quantitative Qualitative
Quantitative Statistical and mathematical
analysis of the numerical
data as presented
Assigning meaning of words
based on scoring system
E.g. Content analysis, word
counts, free lists, pile sorts
Qualitative Search for and presentation
of meaning in results of
quantitative processing
Interpretative test studies
E.g. Hermeneutics,
Grounded Theory,
Phenomenology
© J. Loke 2014

29. Consider the different quantitative and qualitative
techniques in Data Analysis
• Appropriate types of statistical tests for different types
of quantitative data
• Appropriate level of statistical tests based on the size of
data
• Appropriate analytic framework for the different
theoretical framework
• Triangulation of analytic framework
Reliability
Data Source
1
Data Source
3
Data Source
2
© J. Loke 2014

30. Rigour for Robustness
Analyses involve attending to inconsistencies and
diversity
Present the richness of detail in data,
researcher’s assumptions must be clarified
upfront and context of the study must be
explained
Validation through reference to coherence and
fruitfulness of findings, participants’ orientation and
to new research problems
Seek out negative/deviant cases
© J. Loke 2014

31. Rigour through reporting on findings
and making conclusions
• Personal/Professional Bias – prevent
highlighting some issues while obscuring
others?
• Address ethical aspects of research – which
sometimes can be compromised due to political
reasons
Reflexivity –
qualitative research
© J. Loke 2014

32. Ways to address rigour
• Consider the quantitative and
qualitative techniques in:
a. Data sources/collection
b. Data analysis
© J. Loke 2014
 Selecting an appropriate research
design
*Research
Question

33. Further readings
• http://www.economist.com/news/briefing/21588057-scientists-think-science-self-
correcting-alarming-degree-it-not-trouble
• http://cogprints.org/7691/7/ICMLws09.pdf
• http://www.sonoma.edu/users/k/koshar/n300/rigor_table.htm
• Tobin, G. A., Begley, C. M. 2004. Methodological rigour within a qualitative framework.
Journal of Advanced Nursing. 48(4), 388-96.
© J. Loke 2014