2. post-positivism, critical theory, participatory, constructivism, pragmatism, social science
research typology 114 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES
Volume 3, Issue 2, August 2009 Call for mixed analysis: A philosophical framework for
combining qualitative and quantitative approaches T he process of mixed research –
involving ‘mix[ing] or combin[ing] quantitative and qualitative research techniques,
methods, approaches, concepts or language into a single study’ (Johnson & Onwuegbuzie
2004: 17) – has developed a long way since Campbell and Fiske (1959) coined the term
multiple operationalism, wherein more than one method is used as part of a validation
process to help ensure that the variance explained culminates from the underlying
phenomenon or trait and is not a function of the method. However, most of the biggest
formalized developments in mixed research have occurred within the last 25 years, being
given impetus in the 1980s by several prominent researchers (e.g. Brewer & Hunter 1989;
Bryman 1988; Greene, Caracelli, & Graham 1989; Jick 1983; Kidder & Fine 1987; Louis
1982; Madey 1982; Mark & Shotland 1987; Maxwell, Bashook, & Sandlow 1986; Phelan
1987; Rossman & Wilson 1985), who called for the integration of quantitative and
qualitative approaches. However, the last five years have witnessed a significant increase in
the number of mixed research studies, marked by a handbook on mixed methods
(Tashakkori & Teddlie 2003); several mixed research textbooks (e.g. Bergman 2008;
Creswell & Plano Clark 2007; Greene 2007; Johnson & Christensen 2008; Plano Clark &
Creswell 2007; Ridenour & Newman 2008; Teddlie & Tashakkori 2009); several mixed
research articles contained in methodological handbooks (e.g. Teddlie, Tashakkori, &
Johnson 2008); mixed research articles contained in encyclopedias (e.g. Onwuegbuzie
2007); two journals devoted to mixed research (i.e. Journal of Mixed Methods Research,
International Journal of Multiple Research Approaches); several journals now routinely
publishing mixed research (e.g. Field Methods, Educational Evaluation & Policy Analysis,
Quality & Quantity, Evaluation, Evaluation Practice, Research in Nursing & Health, Research
in the Schools, The Qualitative Report); and websites (e.g. http://www.fiu.edu/~bridges/),
conferences (e.g. http://www.mixedmethods.leeds.ac.uk/) and workshops (Creswell &
Plano Clark 2008; Mertens 2008; O’Cathain 2008; Onwuegbuzie & Collins 2008;
Onwuegbuzie, Slate, Leech, & Collins 2008) devoted to mixed research, and by special issues
(Gorard & Smith 2006; Johnson 2006; O’Cathain & Collins 2009) with another underway
(Leech & Onwuegbuzie, 2010). These and other sources have helped to increase the
visibility of mixed research. Greene (2008: 8) recently asked the following questions: • ‘Is
mixed methods social inquiry a distinctive methodology? • Is the field moving in that
direction? • What is needed for mixed methods to become a distinctive methodology?
According to Greene (2006, 2008), the development of a methodological or research
paradigm (i.e. qualitative, quantitative and mixed research) in the social and behavioral
sciences requires a thorough critique of four interrelated but conceptually distinct domains:
(i) philosophical assumptions and stances (i.e. what are the core philosophical or
epistemological assumptions of the methodology?); (ii) inquiry logics (i.e. what traditionally
is called methodology and refers to broad inquiry purposes and questions, logic, procedures
and designs, quality standards and writing and reporting forms that guide the researcher’s
gaze); (iii) guidelines for research practice (i.e. specific strategies and tools that are used to
3. conduct research; the how to component of research methodology); (iv) sociopolitical
commitments (i.e. interests, commitments and power relations surrounding the location in
society in which an inquiry is situated; proclamation of values-based rationales and
meanings for the practice of social and behavioral science research in society). Together,
Greene’s four domains provide a cohesive and interactive framework and an array Volume
3, Issue 2, August 2009 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES
115 Anthony J Onwuegbuzie, R Burke Johnson and Kathleen MT Collins of practical
guidelines for a methodological or research paradigm. Although these domains have been
more fully developed with respect to both the quantitative and qualitative research
paradigms, this is not the case for the field of mixed research. Indeed, not all of Greene’s
(2006) domains have been fully articulated and developed in mixed research (Greene 2006,
2008). Consequently, more theoretical, conceptual and practical work is needed in the area
of mixed research. In recent years, much has been written about most of the 13 distinct,
interactive, iterative steps of the mixed research process identified by Collins, Onwuegbuzie
and Sutton (2006), namely: (a) determining the mixed goal of the study (b) formulating the
mixed research objective(s) (c) determining the rationale(s) for mixing quantitative and
qualitative approaches (d) determining the purpose(s) for mixing quantitative and
qualitative approaches (e) determining the mixed research question(s) (f ) selecting the
mixed sampling design (g) selecting the mixed research design (h) collecting quantitative
and qualitative data (i) transforming and analyzing the quantitative and qualitative data (j)
legitimating the data sets and mixed research findings (k) interpreting the mixed research
findings (l) writing the mixed research report (m) reformulating the mixed research
question(s). However, despite the fact that the mixed analysis step is considered by
beginning researchers to be the most difficult step of the mixed research process, because
typically this step necessitates competence in conducting both quantitative and qualitative
data analyses, it is one of the least developed areas in the mixed research literature, with
relatively few published articles on mixed analysis (e.g. Bazeley 2003 2006; Caracelli &
Greene 1993; Chi 1997; Greene et al. 1989; Lee & Greene 2007; Li, Marquart, & Zercher
2000; Onwuegbuzie 2003; 116 Onwuegbuzie & Dickinson 2008; Onwuegbuzie & Leech
2004, 2006; Onwuegbuzie, Slate, Leech & Collins 2007; Onwuegbuzie, Slate, Leech, & Collins
2009; Onwuegbuzie & Teddlie 2003; Sandelowski 2000, 2001; Teddlie et al. 2008).
Currently, published mixed research textbooks (e.g. Bergman 2008; Creswell & Plano Clark
2007; Greene 2007; Ridenour & Newman 2008) – although groundbreaking – contain at
most one chapter on mixed analysis. Thus, as noted by Greene (2008: 14): ‘There has also
been some work in the area of integrated mixed methods data analysis, although this work
has not yet cohered into a widely accepted framework or set of ideas’. Moreover, when
discussing mixed analysis strategies, none of these authors discuss the philosophical
underpinnings. Yet, by linking mixed analysis techniques to philosophical assumptions and
stances, an iterative, interactive and dynamic linkage is provided among Greene’s (2006,
2008) four domains. With this in mind, the present article is a first attempt explicitly to
provide a philosophical justification for conducting mixed analyses. First, we present
several recent typologies of analyses in social science research that incorporate both
monomethod (i.e. purely quantitative research or purely qualitative research) and mixed
4. research studies. Second, we discuss what has been referred to as the fundamental principle
of data analysis, wherein both qualitative and quantitative data analysis techniques are
shaped by an attempt to analyze data in a way that yields at least one of five types of
generalizations. Third, building on the frameworks of Denzin and Lincoln (2005), Heron and
Reason (1997) and Johnson and Onwuegbuzie (2004), we compare and contrast three
qualitative-based paradigms (i.e. constructivism, critical theory, participatory), one
quantitative-based paradigm (i.e. postpositivism) and one mixed research-based paradigm
(i.e. pragmatism) with respect to three axiomatic components (i.e. ontological,
epistemological and methodological foundations) and seven issues INTERNATIONAL
JOURNAL OF MULTIPLE RESEARCH APPROACHES Volume 3, Issue 2, August 2009 Call for
mixed analysis: A philosophical framework for combining qualitative and quantitative
approaches (i.e. nature of knowledge, knowledge accumulation, goodness or quality criteria,
values, ethics, inquirer posture and training). Also, we link each paradigm to data analysis
strategies. Fourth, we illustrate similarities in goals between some qualitative and
quantitative analyses; in so doing, we deconstruct the strong claim that analysis must be
either qualitative or quantitative and illustrate that regardless of perspective (e.g.
postpositivist or constructivist), both qualitative and quantitative data can be jointly
analyzed. Finally, we compare and contrast 11 mixed research paradigms/worldviews,
linking them to mixed analysis strategies. TYPOLOGY OF ANALYSES IN SOCIAL SCIENCE
RESEARCH Onwuegbuzie et al. (2007) outlined the concept of monotype data, which
represents the use of one data type (e.g. qualitative data) that is available for analysis – in
contrast to multitype data wherein both types of data (i.e. qualitative and quantitative data)
are collected and thus are available for analysis. Onwuegbuzie et al. (2007) coined the
phrase monoanalysis to denote when one class of analysis (e.g. qualitative analysis) is used
to analyze one data analysis type (e.g. qualitative data) – as opposed to multianalysis,
wherein both classes of analyses (i.e. qualitative analysis and quantitative analysis) are used
to analyze one or more data analysis types. For example, a quantitative researcher might
use multiple regression (Fox 1997) to examine which variables predict some quantitative
outcome of interest. Alternatively, a qualitative researcher might use the method of
constant comparison (Glaser & Strauss 1967) to analyze responses to open-ended interview
questions. Three dimensional framework for qualitative and quantitative analyses
Onwuegbuzie et al. (2009) also conceptualized a typology for classifying qualitative and
quantitative analysis techniques. Specifically, these authors presented a three-dimensional
representation for classifying and organizing both qualitative and quantitative analyses,
which involves reframing qualitative and quantitative analyses as a case-oriented, variable-
oriented, or process/experienceoriented analyses. 1. Case-oriented analyses are analyses
that focus primarily or exclusively on the selected case(s), wherein the goal is to analyze
and interpret the meanings, experiences, attitudes, opinions, or the like of one or more
persons – with a tendency toward particularizing and analytical generalizations. Although
case-oriented analyses are best suited for identifying patterns common to one or a
relatively small number of cases – and thus lend themselves better to qualitative research in
general and qualitative analyses in particular – this class of analyses can be used for any
number of cases and, as such, also is pertinent for quantitative research, leading to the use
5. of quantitative analysis techniques such as single-subject analyses, descriptive analyses and
profile analyses (Onwuegbuzie et al. 2009). 2. In contrast, variable-oriented analyses
involve identifying relationships – often probabilistic in nature – among entities that are
treated as variables such that this class of analysis tends to be conceptual and theory-
centered from the onset and has a proclivity toward external generalizations. Therefore,
variable-oriented analyses, whose ‘“building blocks” are variables and their
intercorrelations, rather than cases’ (Miles & Huberman 1994: 174), are more apt for
quantitative research in general and quantitative analyses in particular. However, although
the use of large and representative samples often facilitates identification of relationships
among variables, small samples also can serve this purpose, making variable-oriented
analyses also relevant for qualitative data along with the use of qualitative analysis
techniques (e.g. examining themes that cut across cases). 3. Finally, process/experience-
oriented analyses involve evaluating processes or experiences Volume 3, Issue 2, August
2009 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES 117 Anthony J
Onwuegbuzie, R Burke Johnson and Kathleen MT Collins pertaining to one or more cases
within a specific context over time, with processes tending to be associated with variables
and experiences tending to be associated with people (i.e. cases). Because each of these
three analysis orientations represent a continuum rather than a dichotomy (e.g. variable-
oriented analyses are conceptualized by Onwuegbuzie et al. (2009) as falling on a
particularistic–universalistic continuum, classifying the extent to which the metainferences
stemming from the variable-oriented analysis can be generalized), this three-dimensional
framework supports Johnson and Onwuegbuzie’s (2004: 20) assertion that ‘the possible
number of ways that studies can involve mixing is very large because of the many potential
classification dimensions’. Cross-over mixed analysis Onwuegbuzie and Combs (2009),
building on the works of Greene (2008) and Onwuegbuzie and Teddlie (2003), outlined the
concept of cross-over mixed analyses, which involves using one or more analysis types
associated with one tradition (e.g. quantitative analysis) to analyze data associated with a
different tradition (e.g. qualitative data). Such an analysis can be used to reduce, display,
transform, correlate, consolidate, compare, integrate, assert, or import data. Table 1
presents the cross-over mixed analysis types and strategies that Onwuegbuzie and Combs
(2009) identified. Cross-over mixed analyses are distinct from other types of mixed
analyses (i.e. non-cross-over mixed analyses) such as parallel mixed analysis, wherein both
quantitative and qualitative data analyses are conducted separately, neither type of analysis
builds on or interacts with the other during the data analysis stage and the findings from
each type of analysis are neither compared nor consolidated until both sets of data analyses
have been completed. Whereas non-cross-over mixed analyses involve collection of both
types of data and the analysis conducted per data set represents the same paradigmatic
tradition (i.e. either quantita118 tive or qualitative) – ‘within-paradigm analysis’
(Onwuegbuzie et al. 2007: 12) – cross-over mixed analyses involve a between-paradigm
analysis, which involves ‘an analysis technique more associated with one traditional
paradigm (e.g. quantitative) to analyze data that originally represented the type of data
collected associated with the other traditional paradigm (e.g. qualitative)’ (Onwuegbuzie et
al. 2007: 12). Thus, cross-over mixed analyses involve more integration of qualitative and
6. quantitative analyses than do other types of mixed analyses because they involve the mixing
or combining of qualitative- and quantitative-based paradigmatic assumptions and stances
(e.g. using exploratory factor analysis to examine the structure of themes that emerged from
a qualitative analysis; cf. Onwuegbuzie 2003), which involves either maintaining an
analytical-philosophical stance that the human mind/perception and mathematical
algorithms can be used sequentially to examine patterns in qualitative data or adopting an
analytical-philosophical stance that transcends the stances underlying both paradigms (e.g.
assuming that data saturation [i.e. qualitative information repeats itself such that no new or
relevant information seem to emerge pertaining to a category and the category
development is well established and validated; Morse 1995] and reliability [i.e. consistency
or repeatability of participants’ quantitative responses] represent parallel constructs). This
distinction between cross-over mixed analyses and non-cross-over mixed analyses is
important because certain epistemological, ontological, axiological and methodological
stances might lend themselves more to conducting cross-over mixed analyses than do other
stances. Building on the works of Onwuegbuzie et al. (2007), Onwuegbuzie et al. (2009) and
Onwuegbuzie and Combs (2009), before conducting an analysis, a researcher explicitly or
implicitly makes the following six decisions: (a) the number of data types that will be
analyzed – yielding either monotype data (i.e. use of one data type, namely: qualitative data
or quantitative data) or multitype data INTERNATIONAL JOURNAL OF MULTIPLE
RESEARCH APPROACHES Volume 3, Issue 2, August 2009 Call for mixed analysis: A
philosophical framework for combining qualitative and quantitative approaches TABLE 1: C
ROSS -O VER (M IXED ) A NALYSES S TRATEGIES Analysis Step Cross-Case Analysis Strategy
Integrated Data Reduction Reducing the dimensionality of qualitative data/findings using
quantitative analysis (e.g. exploratory factor analysis of qualitative data) and/or
quantitative data/findings (e.g. thematic analysis of quantitative data) (Onwuegbuzie 2003;
Onwuegbuzie and Teddlie 2003) Integrated Data Display Visually presenting both
qualitative and quantitative results within the same display (Lee and Greene 2007;
Onwuegbuzie and Dickinson 2008) Data Transformation Converting quantitative data into
data that can be analyzed qualitatively (i.e. qualitizing data; Tashakkori and Teddlie 1998)
and/or qualitative data into numerical codes that can be analyzed statistically (i.e.
quantitizing data; Tashakkori and Teddlie 1998) Data Correlation Correlating qualitative
data with quantitized data and/or quantitative data with qualitized data (Onwuegbuzie and
Teddlie 2003) Data Consolidation Combining or merging multiple data sets to create new or
consolidated codes, variables, or data sets (Louis 1982; Onwuegbuzie and Teddlie 2003)
Data Comparison Comparing qualitative and quantitative data/findings (Onwuegbuzie and
Teddlie 2003) Data Integration Integrating qualitative and quantitative data/findings either
into a coherent whole or two separate sets (i.e. qualitative and quantitative) of coherent
wholes (McConney, Rudd and Ayres 2002; Onwuegbuzie and Teddlie 2003) Warranted
Assertion Analysis Data Importation Reviewing all qualitative and quantitative data to yield
meta-inferences (Smith 1997) Utilizing follow-up findings from qualitative analysis to
inform the quantitative analysis (e.g. qualitative contrasting case analysis, qualitative
residual analysis, qualitative followup interaction analysis and qualitative internal
replication analysis; Li et al. 2000; Onwuegbuzie and Teddlie 2003) or follow-up findings
7. from quantitative analysis to inform the qualitative analysis (e.g. quantitative extreme case
analysis, quantitative negative case analysis; Onwuegbuzie and Teddlie 2003) (i.e. use of
both data types, namely: qualitative data and quantitative data); (b) the number of data
analysis types that will be used – yielding monoanalysis (i.e. use of one data analysis type,
namely: qualitative data analysis or quantitative data analysis) or multianalysis (i.e. use of
both data analysis types, namely: qualitative data analysis and quantitative data analysis);
(c) the analysis emphasis of interest – comprising case-oriented analyses (i.e. analyses that
focus primarily or exclusively on the selected case(s)), variable-oriented analyses (i.e.
identifying relationships among entities that are conceived as variables) and/or
process/experience-oriented analyses (i.e. evaluating processes or experiences pertaining
to one or more cases within a specific context over time); (d) whether or not analysis types
associated with one tradition will be used to analyze data associated with a different
tradition (i.e. cross-over mixed analysis vs. non-crossover mixed analysis); (e) whether the
qualitative and quantitative analyses will be carried out concurrently (i.e. results stemming
from one data analysis phase do not inform the results stemming from the other phase) or
sequentially (i.e. the qualitative analyses are conducted first, which then inform the
subsequent quantitative analyses, or vice versa); (f ) whether the qualitative or quantitative
analyses will be given priority (i.e. quantitative analyses carry the most weight or the
qualitative phase carry the most weight), or whether they will be assigned equal status.
Monomethod studies involve the use of monotype data, monoanalysis and non-cross-over
Volume 3, Issue 2, August 2009 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH
APPROACHES 119 Anthony J Onwuegbuzie, R Burke Johnson and Kathleen MT Collins
mixed analysis, thereby making the fifth (i.e. time orientation of analyses) and sixth (i.e.
priority of analyses) decisions irrelevant. Monomethod studies also include a decision as to
the analysis emphasis. In contrast, in mixed research, consideration of these six decisions
yields a large variety of possible combinations (i.e. except the aforementioned combination
associated with monomethod research) that is impossible to capture completely in any
single typology. FUNDAMENTAL PRINCIPLE OF DATA ANALYSIS Onwuegbuzie et al. (2009)
outlined what they refer to as the fundamental principle of data analysis,1 wherein both
qualitative and quantitative data analysis techniques are shaped by an attempt to analyze
data in a way that yields at least one of the following five types of generalizations: external
(statistical)2 generalizations, internal (statistical)3 generalizations, analytical
generalizations, case-tocase transfer and/or naturalistic generalizations. External
(statistical) generalization involves making generalizations, inferences, or predictions on
data obtained from a representative statistical (i.e. optimally random) sample to the
population from which the sample was drawn (i.e. universalistic generalizability).
Contrastingly, internal (statistical) generalization involves making generalizations,
inferences, or predictions on data obtained from one or more representative or elite
participants (e.g. key informants, politically important cases, sub-sample members) to the
sample from which the participant(s) was selected (i.e. particularistic generalizability). It
should be noted that internal (statistical) generalization can stem from 1 2 3 120
qualitative, quantitative, or mixed analyses. Conversely, with analytic generalizations ‘the
investigator is striving to generalize a particular set of [case study] results to some broader
8. theory’ (Yin 2009: 43). In other words, analytical generalizations are ‘applied to wider
theory on the basis of how selected cases “fit” with general constructs’ (Curtis, Gesler,
Smith, & Washburn 2000: 1002) (i.e. particularistic generalizability). Case-to-case transfer
involves making generalizations or inferences from one case to another (similar) case
(Firestone 1993; Kennedy 1979; Miles & Huberman 1994) (i.e. particularistic
generalizability). Finally, with naturalistic generalization, the readers of the article (i.e.
consumers of the findings) make generalizations entirely, or at least in part, from their
personal or vicarious experiences (Stake 2005), such that meanings stem from personal
experience and are adapted and reified by repeated encounter (Stake 1980; Stake &
Trumbull 1982). Rather than the researcher making this sort of generalization, it is the
reader who generalizes (often based on proximal similarity of the case data to the reader’s
focal context of interest). LINKING PARADIGMS TO DATA ANALYSIS STRATEGIES Over the
years, in an attempt to distinguish qualitative-based paradigms from quantitative-based
paradigms and to demonstrate that qualitative research represents a distinct tradition,
several eminent qualitative researchers have presented frameworks that contrast
qualitative-based paradigms (e.g. constructivism, critical theory, participatory) and
quantitative-based paradigms (i.e. [logical] positivism and postpositivism). Indu-
Onwuegbuzie et al. (2009) used the word fundamental because this principle is pertinent
(i.e. fundamental) to qualitative, quantitative and mixed analyses. Unlike Onwuegbuzie et al.
(2009), we place the word ‘statistical’ in parentheses to denote the fact that external
generalization can arise predominantly or exclusively from either quantitative or
qualitative analyses. The term ‘statistical’ is used to denote the possible assumption that the
sample was representative in some way (e.g. probabilistically, vicariously) of the larger
group from which the sample was drawn. Unlike Onwuegbuzie et al. (2009), as is the case
for external generalizations, we place the word ‘statistical’ in parentheses to denote the fact
that internal generalization can arise predominantly or exclusively from either quantitative
or qualitative analyses. Again, the term ‘statistical’ is used to denote the possible
assumption that the sample was representative in some way (e.g. probabilistically,
vicariously) of the larger group from which the sample was drawn. INTERNATIONAL
JOURNAL OF MULTIPLE RESEARCH APPROACHES Volume 3, Issue 2, August 2009 Call for
mixed analysis: A philosophical framework for combining qualitative and quantitative
approaches bitably, the most popularized framework is that developed by Guba (1990) and
extended, more recently, by Denzin and Lincoln (2005). Denzin and Lincoln outlined their
views of the axiomatic nature of paradigms and the issues they believed were most
fundamental to differentiating the paradigms by contrasting three qualitative-based
paradigms (i.e. constructivism, critical theory, participatory) and two quantitative-based
paradigms (i.e. [logical] positivism and postpositivism) with respect to three axiomatic
components (i.e. ontological, epistemological and methodological foundations) and seven
issues (i.e. nature of knowledge, knowledge accumulation, goodness or quality criteria,
values, ethics, inquirer posture and training). In Table 2, we build on Denzin and Lincoln’s
(2005) table of axioms and issues. Specifically, Table 2 contains Denzin and Lincoln’s (2005:
193-194) axioms of the paradigms of postpositivism and constructivism, as well as axioms
of the participatory paradigms outlined by Heron and Reason (1997) (which was also
9. included in Denzin and Lincoln’s [2005] table). Table 2 also contains our proposed
pragmatist paradigm, which incorporates some of the major concepts outlined in Johnson
and Onwuegbuzie’s (2004) seminal article on mixed research. However, Table 2 does not
include the axioms of the paradigm of positivism because this paradigm, which incorporates
several movements (e.g. analytical philosophy, logical atomism, logical empiricism and
semantics), was discredited shortly after World War II.4 As noted by Johnson and
Onwuegbuzie (2004): Positivism is a poor choice for labeling quantitative researchers today
because positivism has long been replaced by newer philosophies of science (Yu 2003). The
term is more of a straw man (easily knocked down) for attack than standing for any actual
practicing researchers. 4 A term that better represents today’s practicing quantitative
researchers is postpositivism (Phillips & Burbules 2000: 24). In this table, we have included
two additional issues to Denzin and Lincoln’s seven issues – namely, qualitative analysis
and quantitative analysis – in an attempt to take the first step toward linking data analysis
techniques to philosophical assumptions and stances. In Figure 1, we reproduce
Onwuegbuzie et al.’s (2009) typology for classifying qualitative and quantitative analysis
techniques. This figure presents an array of analytic techniques undertaken in quantitative
and qualitative research as a function of the following three analysis emphases: case-
oriented analyses, variable-oriented analyses and process/ experience-oriented analyses.
Although this list is by no means exhaustive, it does capture most of the major qualitative
and quantitative analytical techniques (Onwuegbuzie et al. 2009). Postpositivist paradigm
Often under-emphasized in the literature, postpositivism is a rejection or modification of
several core tenets of positivism. Although postpositivist researchers believe that there is
an independent reality that can be studied, they assert that all observation is inherently
theory-laden and fallible and that all theory can be modified. They also believe that, as a
result of their cultural experiences and worldviews, people are always partially biased in
their objective perceptions of reality. Given these inherent biases and perceptions and
observations that are fallible, ensuing constructions are imperfect (Phillips & Burbules
2000). Thus, postpositivist researchers assert that we can only approximate the truth of
reality but can never explain it perfectly or completely. Notwithstanding, postpositivist
researchers believe that objectivity can be approximated by triangulating Disturbingly,
many authors appear to be unaware of the six major tenets of positivism, namely: (a) an
emphasis on verification, (b) pro-observation, (c) anti-cause, (d) downplaying explanation,
(e) anti-theoretical entities and (f ) anti-metaphysics (Hacking 1983). Volume 3, Issue 2,
August 2009 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES 121 122
External replication and external statistical generalization Knowledge accumulation
Historical revisionism; generalization by similarity; internal statistical generalization;
analytical generalization; case-to-case transfer; naturalistic generalization
Structural/historical insights Individual and collective reconstructions that may unite
around consensus Nonfalsified hypotheses that are probably facts or laws Nature of
knowledge Elaborate reconstructions; vicarious experience; internal statistical
generalization; analytical generalization; case-to-case transfer; naturalistic generalization
Critical discourse Detailed, rich and thick (empathic) description, written directly and
somewhat informally Rhetorical neutrality, involving formal writing style using impersonal
10. passive voice and technical terminology, in which establishing and describing social laws is
the major focus; may include qualitative methods Rhetorical In communities of inquiry
contained in communities of practice Entrenched epistemolog-ical emphasis on practical
knowing and critical subjectivity Use of language based on shared experiential context
Political participation in collaborative action research; emphasis on practical
Dialogic/dialectical Hermeneutical/ dialectical; impossible to differentiate fully causes and
effects; inductive reasoning; timeand context-free generalizations are neither desirable nor
possible Time- and context-free generalizations are desirable and possible and real causes
of social scientific outcomes can be determined reliably and validly via quantitative (and
sometimes qualitative) methods Methodology Experiential, propositional and practical
knowing; co-created findings Transactional/subjectivist; value-mediated findings
Subjective knower and known are not separable; Transactional/ subjectivist; co-created
findings/meaning Researchers should eliminate their biases, remain emotionally detached
and uninvolved with the objects of study and test or empirically justify their stated
hypotheses Epistemology Participatory b Subjective-objective reality co-created by mind
and given world order Critical Theory a D ISTINGUISHING CHARACTERISTICS Virtual
reality influenced by social, political, cultural, ethnic, racial, economic and gender values
that evolve over time AND Multiple contradictory, but equally valid accounts of the same
phenomenon representing multiple realities Constructivisma,c CONTEMPORARY R
ESEARCH PARADIGMS Social science inquiry should be objective Postpositivism a,c OF
Ontology Paradigmatic Element TABLE 2: U NDERLYING B ELIEF S YSTEMS Follows
dynamic homeostatic process of belief, doubt, inquiry, modified belief, new doubt, new
inquiry, in an infinite loop, where the person or researcher (and research community)
constantly tries to improve upon past understandings in a way that fits and works in the
world in which he or she operates; internal statistical generalization; analytical
generalization; case-to-case transfer; naturalistic generalization Intersubjectivity, emic and
etic viewpoints; respect for nomological and ideographic knowledge; Use of both
impersonal passive voice and technical terminology, as well as rich and thick (empathic)
description Thoughtful/ dialectical eclecticism and pluralism of methods and perspectives;
determine what works and solves individual and social problems Knowledge is both
constructed and based on the reality of the world we experience and live in; justification
comes via warranted assertability Multiple realities (i.e. subjective, objective,
intersubjective); rejects traditional dualisms (e.g. subjectivism vs. objectivism; facts vs.
values); high regard for the reality and influence of the inner world of human experience in
action; current truth, meaning and knowledge are tentative and changing Pragmatismc
Anthony J Onwuegbuzie, R Burke Johnson and Kathleen MT Collins INTERNATIONAL
JOURNAL OF MULTIPLE RESEARCH APPROACHES Volume 3, Issue 2, August 2009 All forms
of qualitative analyses Descriptive statistics; most, if not all, forms of inferential statistics
that lead to internal (statistical) generalizations and external (statistical) generalizations
Resocialization; qualitative and quantitative; history; values of altruism, empowerment and
liberation All forms of qualitative analyses Descriptive statistics; some inferential statistics
that lead to internal (statistical) generalization but not to external (statistical)
generalization Technical; quantitative and qualitative; substantive theories Some forms of
11. qualitative analysis are possible, especially qualitative analyses that generate numbers as
part of the findings such as word count and classical content analysis All forms of
descriptive and inferential statistics, with an ultimate goal of making external (statistical)
generalizations Training Qualitative analysis Quantitative analysis c b Extracted from
Denzin and Lincoln (2005, pp. 195-196) Extracted from Heron and Reason (1997) Extracted
from Johnson and Onwuegbuzie (2004, pp. 14, 18-20) ‘Transformative intellectual’ as
advocate and activist Passionate participant as facilitator of multivoice reconstruction
Objective scientist and informer of decision makers, policy makers and change agents
Inquirer posture a Intrinsic; moral proclivity toward revelation Intrinsic; process proclivity
toward revelation Extrinsic Ethics Resocialization; qualitative and quantitative; history;
values of altruism, empowerment and liberation Research is value-bound; formative; seeks
to reveal injustice Research is value-bound Research is value-free Critical Theory a All
forms of qualitative analyses All forms of descriptive and inferential statistics Descriptive
statistics; inferential statistics that lead to both internal (statistical) generalizations and
external (statistical) generalizations Qualitative, quantitative, mixed research; substantive
theories; values of altruism, empowerment and liberation Offers the pragmatic method for
solving traditional philosophica dualisms as well as for making methodological choices
Extrinsic and intrinsic; justification comes in the form of warranted assertability Takes an
explicitly value-oriented approach to research that is derived from cultural values;
specifically endorses shared values such as democracy, freedom, equality and progress.
Reliability, internal validity, external validity, objectivity; Trustworthiness, dependability,
confirmability, transferability; authenticity Pragmatismc All forms of qualitative analyses
Researchers, who learn via active engagement in study, need emotional competence,
democratic disposition and skills Primary voice manifest via aware self-reflective action;
secondary voices in revealing theory, narrative, etc. Intrinsic; moral proclivity toward
revelation Research is value-bound Congruence of experiential, presentational,
propositional and practical knowing leads to action to transform the world Participatory b
D ISTINGUISHING CHARACTERISTICS (Continued) Historical situatedness; reduction of
ignorance and misperceptions; involve participants in knowledge construction and
validation AND Values (i.e. Axiology) Trustworthiness, dependability, confirmability,
transferability; authenticity Constructivisma,c CONTEMPORARY R ESEARCH PARADIGMS
Reliability, internal validity, external validity, objectivity Postpositivism a,c OF Goodness or
quality criteria Paradigmatic Element TABLE 2: U NDERLYING B ELIEF S YSTEMS Call for
mixed analysis: A philosophical framework for combining qualitative and quantitative
approaches Volume 3, Issue 2, August 2009 INTERNATIONAL JOURNAL OF MULTIPLE
RESEARCH APPROACHES 123 Anthony J Onwuegbuzie, R Burke Johnson and Kathleen MT
Collins Phase Case-oriented Variable-oriented Process/Experience-oriented Quantitative
Descriptive Analyses (e.g. measures of central tendency, variability, position) Cluster
Analysis Q Methodology Time Series Analysis Profile Analysis Panel Data Analysis Single-
Subject Analysis Classical Test Theory Item Response Theory Multidimensional Scaling
Hazard Proportional Hazards Model Descriptive Analyses Correlation Analysis Independent
t-tests Dependent t-tests Analysis of Variance Analysis of Covariance Multiple Analysis of
Variance Multiple Analysis of Covariance Multiple Regression (Multivariate) Logistic
12. Regression Descriptive/Predictive Discriminant Analysis Log-Linear Analysis Canonical
Correlation Analysis Path Analysis Structural Equation Modeling Hierarchical Linear
Modeling Correspondence Analysis Multidimensional Scaling Exploratory/Confirmatory
Factor Analysis Time Series Analysis Classical Test Theory Item Response Theory
Descriptive Analyses (e.g. measures of central tendency, variability, position) Dependent t-
tests Time Series Analysis Profile Analysis Panel Data Analysis Single-Subject Analysis
Classical Test Theory Item Response Theory Repeated Measures Analysis of Variance
Repeated Measures Analysis of Covariance Survival Analysis Path Analysis Structural
Equation Modeling Hierarchical Linear Modeling Qualitative Word Count Keywords-in-
Context Classical Content Analysis Secondary Data Analysis Taxonomic Analysis
Componential Analysis Text Mining Qualitative Comparative Analysis Semantic Network
Analysis Cognitive Map Analysis Causal Network Analysis Conceptually Ordered Matrix
Analysis Case-ordered Matrix/Network Analysis Time-ordered Matrix/Network Analysis
Variable-by-Variable Matrix Analysis Predictor-Outcome Matrix Analysis Explanatory Effect
Matrix Analysis Method of Constant Comparison Word Count Keywords-in-Context Classical
Content Analysis Domain Analysis Taxonomic Analysis Componential Analysis Conversation
Analysis Discourse Analysis Secondary Data Analysis Text Mining Narrative Analysis
Manifest Content Analysis Latent Content Analysis Qualitative Comparative Analysis
Semantic Network Analysis Cognitive Map Analysis Causal Network Analysis Time-ordered
Matrix/Network Analysis Method of Constant Comparison Word Count Keywords-in-
Context Classical Content Analysis Domain Analysis Taxonomic Analysis Componential
Analysis Conversation Analysis Discourse Analysis Secondary Data Analysis Membership
Categorization Analysis Narrative Analysis Semiotics Manifest Content Analysis Latent
Content Analysis Text Mining Qualitative Comparative Analysis Micro-interlocutor Analysis
Partially Ordered Matrix Analysis Time-ordered Matrix/Network Analysis Note: All
quantitative analyses above include non-parametric counterparts. F IGURE 1: T HREE -
DIMENSIONAL MATRIX INDICATING ANALYTICAL TECHNIQUES AS A FUNCTION OF
APPROACH (i.e. quantitative vs. qualitative) and analysis emphasis (i.e. case-oriented vs.
variable-oriented vs. process/experience-oriented). Reproduced with permission:
International Journal of Multiple Research Approaches (2009) 3(1): 27 [Figure 4]. across
these multiple fallible perspectives (i.e. triangulation of method, data and theory). Karl
Popper (1994) identified three worlds that postpositivist researchers can address: (a) an
objective 124 physical external world (World 1); (b) an interpretative, subjective inner
world (World 2); and (c) the theory world where humans mentally and physically represent
the first two worlds (World INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH
APPROACHES Volume 3, Issue 2, August 2009 Call for mixed analysis: A philosophical
framework for combining qualitative and quantitative approaches 3). Popper’s views of
knowledge and reality are just one example of complexity and nuance commonly expressed
by quantitative researchers that is not represented in simple statements found in paradigm
comparison tables popular in the research literature. Belief in the fallibility of observations
renders statistics in general, and inferential statistics in particular, as suited to
postpositivist research due to its emphasis on assigning probabilities (e.g. pvalues, levels of
confidence or error) to observed findings. Thus, as well as utilizing descriptive statistics,
13. postpositivist researchers use the whole array of inferential statistics for making external
(statistical) generalizations (cf. Figure 1). Contrary to popular depictions of quantitative
research as being deductive (and qualitative as inductive), inferential statistical
generalizations are inductive. Postpositivist researchers also utilize some qualitative
analysis techniques, especially those that yield frequency data such as Word Count and
Classical Content Analysis (cf. Leech & Onwuegbuzie 2007, 2008). Postpositivist
researchers also employ qualitative data analysis techniques that help them develop
quantitative instruments. Constructivist paradigm As can be seen in Table 2, constructivist
researchers (e.g. radical constructivists, cognitive constructivists, cultural constructivists,
social constructivists/constructionists, communal constructivists, critical constructivists,
genetic epistemology) often claim to believe that multiple, contradictory, but equally valid
accounts of the same phenomenon (i.e. multiple realities) can exist. Thus, it is somewhat
contradictory that those who ascribe to strong relativism or strong constructivism do not
view the use of quantitative methods in general and quantitative analysis in particular as
representing one such valid account of a phenomenon – albeit not their preferred account.
However, the fact that constructivists tend to believe that time- and context-free
generalizations are neither desirable nor possible, likely renders inappropriate the use of
inferential statis- tics such as the sample mean for the purpose of making external
(statistical) generalizations across populations (i.e. claiming that a statistical parameter
applies broadly to nearly everyone in a population). Generalizing to a population (e.g.
claiming that the sample statistic such as the sample mean is a reasonable representation of
the corresponding population parameter) should be less controversial. This distinction of
generalizing across versus generalizing to a population (Cook & Campbell 1979) seldom is
addressed in the paradigm debates. Parameter estimation would probably be of more
interest to a social constructivist than a radical or cognitive constructivist. Descriptive
statistics has long been an important part of ethnography in anthropology as ethnographers
include quantitative descriptors to complement narrative description. Descriptive statistics,
more generally, can be used to enhance the qualitative researcher’s quest for detailed
description. Sechrest and Sidani note (1995: 79) ‘qualitative researchers regularly use
terms such as ‘many,’ ‘most,’ ‘frequently,’ ‘several,’ ‘never,’ and so on. These terms are
fundamentally quantitative.’ Qualitative researchers also can obtain more meaning by
obtaining counts of words in addition to their narrative descriptions (Sandelowski 2001).
For example, in examining the lived experience in the classroom of Johnny, a child with
attention deficit hyperactivity disorder, rather than telling readers that Johnny left his seat
on many occasions during a class, it would be informative for the qualitative researcher to
note that Johnny left his seat six times during the course of 30 minutes. This way, the
readers can (if contextual detail is provided) decide whether six incidences of out-ofseat
behavior are significant/meaningful and also will be in a better position to decide whether
to make naturalistic generalizations. Consistent with our recommendation for qualitative
researchers to use descriptive statistics more frequently where applicable, more than one
half a century ago, Barton and Lazarsfeld (1955) advocated the use of what they coined
quasi-statistics in qualitative research. According to these Volume 3, Issue 2, August 2009
INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES 125 Anthony J
14. Onwuegbuzie, R Burke Johnson and Kathleen MT Collins authors, quasi-statistics pertain to
the use of descriptive statistics that can be extracted from qualitative data. Interestingly, the
prominent symbolic interactionist Howard Becker (1970: 81-82) contended ‘one of the
greatest faults in most observational case studies has been their failure to make explicit the
quasi-statistical basis of their conclusions.’. Further, Joseph Maxwell (1996: 95) noted that:
Quasi-statistics not only allow you to test and support claims that are inherently
quantitative, but also enable you to assess the amount of evidence in your data that bears on
a particular conclusion or threat, such as how many discrepant instances exist and from
how many different sources they were obtained. [emphasis in original] Interestingly,
Becker, Geer, Hughes and Strauss (1961/1977) provided more than 50 tables and graphs in
their qualitative works. These tables and graphs complemented the narrative descriptions
of their qualitative data. Internal (statistical) generalizations also are possible in
constructivist research. In particular, it is not unusual for constructivist researchers to
utilize key informants to provide them with an insider’s understanding and to provide
information that the researcher is unable to experience. This often includes both qualitative
and quantitative information. For example, the informant might state what many members
believe or describe as characteristics of many (or only a few) people in the group. These
numbers would contribute to internal generalizations. In particular, inferential statistics
can be used to make internal (statistical) generalizations. Educational ethnographers
Margaret LeCompte and Jude Preissle (1993) discuss enumeration in ethnographic data
analysis in some depth in their qualitative research textbook. The fact that many computer-
assisted qualitative data analysis software (CAQDAS) programs allow data to be imported to
statistical software programs (e.g. Excel, SIMSTAT) supports our 126 assertion that
descriptive and inferential statistical analyses are an option for constructivist researchers
should they deem it appropriate (e.g. based on their philosophical stance and research
question(s)) to make internal (statistical) generalizations. These statistical analysis tools
also can be used to bolster analytical generalizations. One of the most popular methods of
qualitative research, grounded theory, is premised on the development of theory for the
purposes of generalization. Descriptive and inferential statistics can be used to facilitate
rich and detailed description, and to assess and enhance trustworthiness, dependability,
confirmability, transferability and authenticity. Consistent with this assertion, Denzin and
Lincoln (2005) state that the training of constructivists includes both qualitative and
quantitative techniques (cf. Table 2). Whatever findings emerge from descriptive and/or
inferential statistical analyses utilized, it should be noted that for constructivists, these
findings represent just one of the multiple valid accounts of the phenomenon. Critical
theory paradigm Critical theory researchers seek to understand the relationship between
societal structures (e.g. economic, political) and ideological patterns of thought that impede
a person or group from identifying, confronting and addressing unjust social systems.
Simply put, critical theory researchers primarily are interested in social change as it
emerges in relation to social struggle. Moreover, critical theory researchers operate under
the assumption that the knowledge gleaned from their research represents an initial step
toward addressing social injustices and promoting social change. As such, critical theory
research represents a form of transformative research by providing emancipatory
15. knowledge that identifies the contradictions that are masked or distorted by our everyday
thoughts and perceptions (Lather 1986). Whereas many other types of qualitative
researchers (e.g. phenomenological researchers, constructivist researchers) are interested
in the meanings that people attach to INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH
APPROACHES Volume 3, Issue 2, August 2009 Call for mixed analysis: A philosophical
framework for combining qualitative and quantitative approaches their own actions, critical
theory researchers aim to place such actions in a broader context that is framed by social,
economic, political and ideological forces that have previously not been identified or
acknowledged. That is, critical theory researchers are more concerned with the forces that
limit actions rather than the actions themselves. Thus, critical theory researchers tend to
embrace a more etic (i.e. outsider’s) stance than an emic (i.e. insider’s) stance. Although it
might appear that critical theory research is similar to postpositivist researchers – where
critical theory researchers seek cause-andeffect relationships wherein the actions of
individuals or groups are influenced directly by social, economic, political and/or
ideological variables – many critical theory researchers would consider this strategy as
being inappropriately reductionistic. Rather, many postmodern critical theory researchers
believe that it is not possible to predict reliably how these variables determine actions.
Further, instead of focusing on foundational criteria to justify its research findings, critical
theory researchers contend that, by attending to the role of power in social systems, their
analyses occur at the meta-theoretical level. Drawing upon research from other paradigms,
critical theory researchers identify power structures and their processes that are typically
ignored in both postpositivist and constructivist research. Like constructivist researchers,
critical theory researchers believe that meaning and language are socially constructed and
are neither time- nor context-free, although critical theory researchers assert that social
injustice has real (i.e. objective, material) consequences. Also, like constructivist
researchers, critical theory researchers primarily assess their findings with respect to the
community of researchers to which they belong (i.e. authoritative consensus). Critical
theory researchers contend that the subjective/objective dualism camouflages the ways in
which both standpoints are constrained by power dynamics and social forces. However,
critical theory researchers endorse subjectivism inasmuch as the subjective knower and
known cannot be separated and that knowledge is subjective (culturally and historically
embedded) and constructed on the basis of power issues (Lather 2006); yet, they reject the
stance that all analyses are relative, instead, believing that rational analysis is essential to
social justice. Thus, many critical theorists endorse critical realism (Morrow & Brown
1994). Critical theory researchers believe that constructivists place too much weight on
people’s perceptions and too little emphasis on the more complex social forces that shape
and constrain experiences, events and actions. As such, critical theory researchers deem
constructivism to be just as much at risk as is postpositivism of uncritically bolstering the
status quo of social injustice. According to Habermas, a leading contemporary proponent of
critical theory, critical theory involves the following three types of knowledge: (a) the
empirical-analytic sciences (i.e. including both the natural sciences and social sciences,
which seek to manipulate knowledge for the purposes of prediction and control over nature
and social structure); (b) historical-hermeneutic sciences (i.e. including the cultural and
16. human sciences, which seek to understand communication among and within social
groups); and (c) critical theory (i.e. which seeks freedom from oppression and social justice)
(Blaikie 1993). Thus, Habermas advocates the use of both the method of the empirical-
analytic sciences, historical-hermeneutic sciences and critical theory. However, critical
theory researchers believe that critical theory utilizes but transcends both the empirical-
analytic and historical-hermeneutic sciences inasmuch as it provides knowledge that is
subjected to rigorous processes of free and rationale discourse, which prevents it from
being socially distorted (Blaikie 1993). This suggests that the ontological, epistemological
and axiological stances of critical theorist researchers do not prevent them from using, as
appropriate, all forms of qualitative analysis techniques and quantitative analysis
techniques that include both descriptive and inferential statistics (cf. Figure 1). Volume 3,
Issue 2, August 2009 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES
127 Anthony J Onwuegbuzie, R Burke Johnson and Kathleen MT Collins As noted by Morrow
and Brown (1994: 200), ‘critical theory has no basis for a priori rejection of any particular
methods or techniques as such’. However, whatever analysis is used, critical theory
researchers make no claims that their analyses are objective in the manner claimed by some
postpositivists. Participatory paradigm. As noted by Heron and Reason (1997), proponents
of the participatory paradigm subscribe to an experiential subjective–objective reality that
stems from a co-creative interaction between the given cosmos and the way the mind
connects with it. Thus, what can be known about the given cosmos is that it represents a
subjectively articulated world, whose objectivity reflects how it is intersubjectively formed
by the knower. A participative worldview acknowledges the following four interdependent
ways of knowing: experiential (i.e. direct encounter), presentational (i.e. emerges from and
grounded on experiential knowing), propositional (i.e. conceptually knowing that
something or other is the case) and practical (i.e. knowing how to do something) (Heron &
Reason 1997). Of these, propositional knowing is most compatible with statistical analyses.
In particular, propositional knowing represents knowledge that comes to the fore by
describing some person, group, entity, location, situation, process, or the like. Simply put, it
represents knowledge of facts. This way of knowing is articulated via declarations and
theories that stem from mastery of concepts that language provides. This suggests that
propositional knowledge can be expressed not only via qualitative analysis techniques but
also via an array of quantitative techniques that include both descriptive and inferential
statistics. For example, inferential statistics could be used to attach levels of certainty (i.e.
probability) to knowing that something is the case. Interestingly, as noted by Heron and
Reason (1997), intervention research represents a form of participative inquiry that
includes statistical analyses (cf. Fryer & Feather 1994). 128 When studying a group or
community, participatory researchers might use inferential techniques to test theory that
represents propositional knowledge. Confirmation of a theory then would lead the
participatory researcher to make external (statistical) generalizations from the participants
to the group or community from which the participants were selected. Alternatively, when
studying key informants or sub-sample members, a participatory researcher might use
inferential statistics to make internal (statistical) generalizations to assess propositional
knowledge. Alternatively still, when studying one or a few people, then only descriptive
17. statistics might be appropriate. The quest for practical knowing also invites statistical
analyses. In particular, practical knowing assumes a conceptual understanding of standards
and principles of practice, presentational stylishness and experiential grounding in the
situation within which the action takes place. In this respect, participatory research
coincides with pragmatist research, which endorses a strong and practical empiricism, in
which both qualitative and qualitative data are collected and analyzed to generate and to
test theories against observations of the natural world. As such, participatory research does
not appear to invalidate the use of descriptive or inferential statistical analyses and thus all
analyses in Figure 1 are available to participatory researchers. Pragmatist paradigm The
pragmatist paradigm offers an epistemological justification (i.e. via pragmatic epistemic
principles and standards) and logic (i.e. combining approaches that help researchers
optimally frame, examine and provide tentative answers to one’s research question[s]) for
mixing approaches and methods (Johnson, Onwuegbuzie, & Turner 2007; Onwuegbuzie &
Leech 2005). Further, a pragmatist would reject the incompatibility thesis that qualitative
and quantitative research are fully incompatible and cannot, in any useful way, be used in
combination in social or behavioral research. A serious problem with the incompati-
INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES Volume 3, Issue 2,
August 2009 Call for mixed analysis: A philosophical framework for combining qualitative
and quantitative approaches bility thesis is that it is an a priori argument; it seems to be
based on rationalistic, foundational, deductive logic (which oftentimes is said to have been
rejected in qualitative research). The incompatibility thesis does not appear to be based on
observation of social and behavioral research (i.e. through an examination of how
researchers actually conduct their research). In pragmatist research, research paradigms
can remain separate, but they also can be mixed or combined into another research
paradigm (i.e. mixed research). Johnson et al. (2007: 129) asserts that pragmatist
researchers may ascribe to the view of mixed methods research is a research paradigm that:
(a) partners with the philosophy of pragmatism in one of its forms (left, right, middle); (b)
follows the logic of mixed methods research (including the logic of the fundamental
principle and any other useful logics imported from qualitative or quantitative research that
are helpful for producing defensible and usable research findings); (c) relies on qualitative
and quantitative viewpoints, data collection, analysis and inference techniques combined
according to the logic of mixed methods research to address one’s research question(s); and
(d) is cognizant, appreciative and inclusive of local and broader sociopolitical realities,
resources and needs. The mixed methods research paradigm offers an important approach
for generating important research questions and providing warranted answers to those
questions. This type of research should be used when the nexus of contingencies in a
situation, in relation to one’s research question(s), suggests that mixed methods research is
likely to provide superior research findings and outcomes. As noted in Table 2, pragmatist
researchers can use the whole range of qualitative analyses and quantitative (i.e. descriptive
and inferential analytical techniques) in an attempt to fulfill one or more of five mixed
research purposes identified by Greene et al. (1989): tri-angulation (i.e. comparing findings
from quantitative data with qualitative results in hopes of convergence); complementarity
(i.e. seeking elaboration, illustration, enhancement and clarification of the results from one
18. method with findings from the other method); development (i.e. using the results from one
method to help inform the other method); initiation (i.e. discovering paradoxes and
contradictions that culminate in a re-framing of the research question); and expansion (i.e.
expanding the breadth and range of a study by using multiple methods for different study
phases). Decisions made regarding these five mixed research purposes and the resulting
mixed research designs help pragmatist researchers to determine which of three types of
mixed analysis should be undertaken: a parallel mixed analysis (i.e. findings obtained from
both analysis phases are interpreted separately), concurrent mixed analysis (i.e. results
stemming from one data analysis phase do not inform the results stemming from the other
phase), or sequential mixed analysis (i.e. the qualitative analysis phase is conducted first,
which then informs the subsequent quantitative analysis phase, or vice versa). In particular,
if the purpose of mixing is complementarity, then all three families of mixed analyses (i.e.
parallel, concurrent and sequential) can be used. If triangulation or initiation represents the
purpose, then both parallel and concurrent mixed analyses are viable. If development is the
purpose, then concurrent and sequential mixed analyses are appropriate. Finally, if the
purpose of mixing analyses is expansion then a sequential mixed analysis is pertinent
(Onwuegbuzie et al. 2007). The purposes of mixed research and the resulting kinds of
analyses also will help pragmatist researchers determine the number of data types that will
be analyzed (i.e. monotype data vs. multitype data), the number of data analysis types that
will be used (i.e. monoanalysis vs. multianalysis), the analysis emphasis of interest (i.e. case-
oriented analyses, variable-oriented analyses Volume 3, Issue 2, August 2009
INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES 129 Anthony J
Onwuegbuzie, R Burke Johnson and Kathleen MT Collins and/or process/experience-
oriented analyses), whether or not analysis types associated with one tradition will be used
to analyze data associated with a different tradition (i.e. cross-over mixed analysis vs. non-
cross-over mixed analysis) and whether the qualitative or quantitative analyses will be
given priority, or whether they will be assigned equal status. Pragmatist researchers also
can use inferential statistics to make internal (statistical) generalizations. For instance, in
the field of linguistics, political science and the like, it is not unusual for researchers
analyzing qualitative data to conduct sequence analyses wherein hypotheses regarding
patterns and trends in the qualitative data are tested (i.e. using p-values, confidence
intervals and/or effect sizes). As an example, these researchers might test whether themes
occurring among the participants statistically and/or practically significantly tend to
emerge in a particular order. QDA Miner 3.0 is a computer-assisted qualitative data analysis
software (CAQDAS) program that allows researchers to conduct qualitative analyses, such
as thematic analysis and also to sequence analyses and other statistical and visualization
tools such as clustering, multidimensional scaling, heatmaps and correspondence analysis.
The fact that CAQDAS programs increasingly are allowing inferential statistical analyses to
be conducted supports our assertion that inferential statistical analyses are an option that
analysts of qualitative data have should they deem it appropriate to make statistical (i.e.
internal and to a lesser degree external) generalizations. These inferential statistical
analysis tools also can be used to bolster analytical generalizations. The descriptive and
inferential statistics could be used not only to facilitate rich and detailed description but
19. also could be used to assess and enhance trustworthiness, dependability, confirmability,
transferability and authenticity. For pragmatist research, having a postpositivist orientation
does not prevent a researcher from conducting qualitative analyses – especially analy130
ses such as word count and content analysis that involve, in part, the counting of words,
codes, categories, or other aspects of the qualitative data. Such analyses can form part of
quantitative-dominant mixed analyses (cf. Johnson et al. 2007), in which the analyst adopts
a postpositivist stance, while, at the same time, believing that the inclusion of qualitative
data and approaches are likely to enhance the findings. Similarly, having a constructivist
orientation – or any other qualitativebased orientation – does not prevent a researcher
from conducting quantitative analyses – especially descriptive statistical analyses that do
not involve the analyst making inferences beyond the research participants at hand, which
is typically not the goal of qualitative researchers. Such analyses can form part of
qualitative-dominant mixed analyses (cf. Johnson et al. 2007), in which the researcher takes
a constructivist-poststructuralist-critical stance with respect to the mixed analysis process,
while, at the same time, deeming the addition of quantitative data and approaches as helpful
in providing richer data and interpretations. Commonalities regarding data analysis
strategies across paradigms From Table 2, the table of axioms and issues, it can be seen that
the ontological, epistemological and methodological assumptions and stances representing
all five paradigms allow the conduct of both quantitative and qualitative analyses – at least
to a small degree – with postpositivist and constructivist paradigms having the least
potential to use analytical techniques that belong to a different paradigm, the critical theory
and the participatory paradigms having excellent potential to use analytical techniques that
belong to a different paradigm and the pragmatist paradigm, almost by definition, having
the greatest potential. That the ontological, epistemological and methodological
assumptions and stances representing all five paradigms legitimate both quantitative and
qualitative analyses to be undertaken is especially apparent when one INTERNATIONAL
JOURNAL OF MULTIPLE RESEARCH APPROACHES Volume 3, Issue 2, August 2009 Call for
mixed analysis: A philosophical framework for combining qualitative and quantitative
approaches examines the similarities in goals between many quantitative and qualitative
analyses rather than emphasizing the differences. 5 First and foremost, both quantitative
and qualitative researchers analyze empirical observations (i.e. data coming from personal
experience, observation, or experiment) to address research questions (Johnson &
Onwuegbuzie 2004). Sechrest and Sidani (1995: 78) note that both sets of researchers
‘describe their data, construct explanatory arguments from their data and speculate about
why the outcomes they observed happened as they did.’. At the level of data analysis, for
example, numeric data are similar to textual/visual data inasmuch as they represent
(descriptive) codes that characterize the person’s meanings, beliefs, attitudes and so on;
thus, both data types might be available for collection and analysis regardless of the
research paradigm involved, depending on the research question(s). Indeed, both numeric
data and textual/visual data can be used to facilitate any of the following types of coding:
inductive coding, deductive coding, abductive coding, interpretive coding, open coding, axial
coding and selective coding (cf. Miles & Huberman 1994). As another example, thematic
analysis (qualitative technique) and exploratory factor analysis (quantitative technique)
20. have very similar goals, namely, to reduce the dimensionality of the raw data. This
similarity allows mixed researchers to use analyses associated with one paradigm on data
associated with another paradigm. For instance, it is not unusual for a mixed 5 6 researcher
to factor analyze themes that emerged from textual data (cf. Onwuegbuzie 2003)6 or to
undertake a profile analysis of a set of quantitative measures stemming from one or more
cases (Tashakkori & Teddlie 1998). Statistical factor analysis is an exploratory data analysis
technique that searches for sets of variables that are similar to one another but are different
from the other sets of variables found in the data. The researcher must label the factors that
emerge from the data; that is, the research must give a name to the factors and interpret
what the named factors mean. The naming part of factor analysis is an example of the use of
qualitative coding within a technique traditionally viewed as quantitative. Factor analysis,
we argue, has both quantitative and qualitative elements and, therefore, is an inherently
mixed analysis procedure. An even more compelling explanation for our assertion – that the
ontological, epistemological and methodological assumptions and stances representing all
five paradigms allow both quantitative and qualitative analyses to be undertaken – stems
from the nature of qualitative and quantitative analyses themselves. Many of the core
analytical techniques that are associated with both qualitative and quantitative paradigms
are not as pure as is contended by proponents of monomethod research. For instance, with
respect to qualitative research, the concepts of data saturation, informational redundancy
and/or theoretical saturation (i.e. when no new or relevant information seem to emerge
pertaining to a category and the It is legitimate and useful to examine and emphasize the
differences between qualitative and quantitative research. We should remain aware,
however, that this emphasis glosses over the many, sometimes great, differences within
qualitative and quantitative research. Furthermore, our point here is that it also is
legitimate to examine similarities that sometimes are present in applications of qualitative
and quantitative research. The exploratory factor analysis is conducted after converting the
themes to a ‘1’ if the theme is present for the research participant and a ‘0’ if the theme is
not present for the research participant. This conversion yields what Onwuegbuzie (2003)
refers to as an inter-respondent matrix (i.e. participant x theme matrix), consisting only of
0s and 1s that is subsequently converted to a matrix of bivariate associations among the
responses pertaining to each of the emergent themes. These bivariate associations then are
converted to tetrachoric correlation coefficients because the themes had been quantitized
to dichotomous data (i.e. ‘0’ vs. ‘1’) and tetrachoric correlation coefficients are appropriate
to use when one is determining the relationship between two (artificial) dichotomous
variables. The matrix of tetrachoric correlation coefficients then becomes the basis of the
exploratory factor analysis (see, for example Onwuegbuzie, Witcher, Collins, Filer,
Wiedmaier and Moore 2007). Volume 3, Issue 2, August 2009 INTERNATIONAL JOURNAL
OF MULTIPLE RESEARCH APPROACHES 131 Anthony J Onwuegbuzie, R Burke Johnson and
Kathleen MT Collins category development is well established and validated; Flick 1998;
Lincoln & Guba 1985; Morse 1995; Strauss & Corbin 1990), although inherently
interpretivist, have quantitative undertones. These concepts incorporate quantitative
assumptions – including the idea of internal replication (i.e. findings that are replicated by
other participants in the study). In order to reach conclusions about saturation or
21. informational redundancy, some kind of formal (i.e. conscious) or informal (e.g.
subconscious) assessment of degree or amount typically takes place to determine how
exhaustive the data are. Also, any conclusion on the part of the qualitative researcher that
saturation or informational redundancy has taken place is accompanied by some degree of
confidence or even certainty (i.e. high probability) – whether or not this confidence is
estimated (which is a strategy that a postpositivist researcher likely would pursue). As the
early modern continental philosopher Immanuel Kant (1781/1998) pointed out, the
categories of qualitative and quantitative are necessarily part of human thought and the
conclusions we construct about entities that are important to us. With respect to
quantitative research, techniques such as exploratory factor analysis and cluster analysis,
although inherently postpositivist, have constructivist leanings inasmuch as for any given
dataset, there are myriad mathematical solutions (i.e. factor/cluster structures) that can be
constructed and the analyst has to make sense of the selected mathematical solution (i.e.
meaningmaking). Indeed, there are numerous strategies (e.g. principal component analysis
vs. factor analysis; correlation matrix vs. variance–covariance matrix; maximum likelihood
vs. unweighted least squares vs. generalized least squares vs. principle axis factoring vs.
alpha factoring vs. image factoring; eigenvalues; trace; scree plot; parallel analysis; number
of iterations; orthogonal vs. oblique rotation; factor pattern matrix vs. factor structure
matrix; communality estimates; internal replication techniques such as bootstrapping,
jackknifing and cross-validation; cf. Henson, Capraro, & 132 Capraro 2004; Henson &
Roberts 2006; Hetzel 1996; Onwuegbuzie & Daniel 2003) and criteria that are available for
deciding on the most appropriate factor structure. With so many decisions to make, two or
more analysts easily can arrive at different final factor solutions, which indicates that
exploratory factor analysts serve, to a significant degree, as research instruments
themselves – a concept that some postpositivist researchers might be reluctant to
acknowledge due to a stance that social science inquiry should be objective and that
postpositivist researchers should eliminate all biases (cf. Table 2). Interestingly, the role of
postpositivist researcher-as-instrument is not restricted to analyses that are exploratory in
nature (e.g. exploratory factor analysis, cluster analysis, multidimensional scaling) but also
include analyses that involve null hypothesis significance testing (NHST). In fact, the more
sophisticated the NHST-based analysis, the more subjective decisions positivist analyst has
to make, making him/her serve as a research instrument to an even larger degree. For
example, structural equation modeling (SEM; e.g. Schumacker & Lomax 1996) and
hierarchical linear modeling (HLM; e.g. Bryk & Raudenbush 1992) typically involve the
analyst making numerous subjective decisions, including selecting how many and which
models to test and selecting from the numerous goodness-of-fit indices and criteria
available. Typically, the theoretical models are literally created by the researcher (after an
inductive analysis of past research, current theory, hunches, etc.). Thus, it is difficult for
advocates of any of the research paradigms to claim justifiably a one-to-one correspondence
between ontology/epistemology and type of analysis (i.e. qualitative vs. quantitative).
Moreover, we contend that pragmatist researchers who hold a mixture of philosophical
positions (i.e. belonging to both quantitative and qualitative traditions) find it natural to
combine statistical analyses with an array of qualitative analyses. In order to accomplish
22. such an embed- INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES
Volume 3, Issue 2, August 2009 Call for mixed analysis: A philosophical framework for
combining qualitative and quantitative approaches ded analysis, the mixed researcher has
to make Gestalt switches from a quantitative lens to a qualitative lens and vice versa, going
back and forth, multiple times (Kuhn 1962). According to Onwuegbuzie and Johnson
(2006), this series of switching yields a new or consolidated analysis – such as cross-over
mixed analyses – which, if effective, can yield more fully mixed meta-inferences that
incorporate a strong use of both quantitative and qualitative assumptions and stances and
that represent the strongest paradigmatic mixing. More generally in the mixed research
literature, this purposeful switching is called the dialectical approach (Greene 2007;
Johnson 2008). This approach can rely on a single individual trained in both qualitative and
quantitative research or, if no qualified individual is available, it can rely on a research team
composed of at least one qualitative and one quantitative researcher. A recent example of a
qualitative–quantitative research team is found in Corden and Hirst (2008). Mixed research
philosophical paradigms Pragmatism is only one of many stances that underlie mixed
research (Greene 2007, 2008; Johnson et al. 2007). Current stances include the pragmatism-
of-the-middle philosophy (Johnson & Onwuegbuzie 2004), pragmatism-of-the-right
philosophy (Putnam 2002; Rescher 2000), pragmatism-of-the-left philosophy (Maxcy 2003;
Rorty 1991), the anti-conflationist philosophy (Bryman 1992; Hammersley 1992; Layder
1993; Roberts 2002), critical realist orientation (Houston 2001; Maxwell 2004; McEvoy &
Richards 2003, 2006), the dialectical stance (Greene 2008; Greene & Caracelli 1997;
Maxwell & Loomis 2003), complementary strengths stance (Brewer & Hunter 1989; Morse
2003), transformativeemancipatory stance (Mertens 2003), a-paradigmatic stance (Patton
2002; Reichardt & Cook 1979), substantive theory stance (Chen 2006) and, most recently,
communities of practice stance (Denscombe 2008). Table 3 provides one way of
representing the major mixed research stances. In this table, these leading mixed research
philosophical paradigms are presented (left column), alongside a summary of their major
stances (middle column) and the core analyses that are associated with these stances (right
column). This table represents a first attempt to link mixed research paradigms and stances
to mixed analysis strategies. However, clearly, more work is needed here to enhance our
understanding of the mixed analysis choices made by mixed researchers and thus address
Greene’s (2008: 13) important question: ‘how do the assumptions and stances…influence
inquiry decisions?’. CONCLUSIONS The present article represents a first attempt, albeit a
tentative one, explicitly to provide a philosophical justification for conducting mixed
analyses. As noted by Greene (2008: 12): ‘Our thinking about the nature and role of
philosophical assumptions in our practice needs to make more practical sense, as well as
offer possibilities to practitioners not yet envisioned, which is one key role of mixed
methods theory’. Although we recognize that much more work is needed in this area, we
hope that our article will motivate constructive and positive dialogues among mixed
researchers and researchers representing qualitative- and quantitative-based paradigms
(Onwuegbuzie 2002), so that, regardless of our differences in philosophical assumptions
and stances, we all come to view all researchers as belonging to communities of interest (cf.
Fischer 2001). Further, we hope that our article will motivate others either to refute or to
23. build on the ideas and concepts we have presented. To the extent that our ideas and
concepts make practical sense, we hope that our article will help better situate mixed
analyses in the philosophy of science, thereby promoting mixed research as a distinctive
methodology – consistent with the call made by Greene (2006). Volume 3, Issue 2, August
2009 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES 133 Anthony J
Onwuegbuzie, R Burke Johnson and Kathleen MT Collins TABLE 3: M IXED R ESEARCH
PARADIGMS Paradigm/ Worldview Pragmatism-ofthe-middle philosophy Pragmatism-
ofthe-right Pragmatism-ofthe-left Anti-conflationist Critical realist Dialectical stance
Complementary strengths Transformativeemancipatory A-paradigmatic Substantive theory
Communities of practice AND W ORLDVIEWS M IXED A NALYSIS A SSUMPTIONS Stance
Offers a practical and outcome-oriented method of inquiry that is based on action and leads,
iteratively, to further action and the elimination of doubt; paradigms routinely are mixed
(Johnson and Onwuegbuzie 2004) Holding a moderately strong form of realism and a weak
form of pluralism (Johnson et al. 2007) Antirealism and strong pluralism (Johnson et al.
2007) Methodology should not be conflated with technical aspects of method because the
same method can be used by researchers with different ontological/epistemological
stances; adoption of a more principled approach when combining methods – only
appropriate to combine methods if a common ontological/epistemological stance can be
maintained (McEvoy and Richards 2003) Mix of critical theory and a multilevel, discursive
social scientific realism (Maxwell 2004) Dialogical engagement with paradigm differences
that generatively produce new knowledge and insights (Greene 2007). Use of ‘dialectical
pragmatism’ (i.e. examine qualitative and quantitative stances fully and dialectically and
produce a combination solution that works best for the research question) (Teddlie and
Johnson 2009) Paradigms are not necessarily incompatible but are substantively different;
thus, methods used for different paradigms should be kept separate to preserve
paradigmatic and methodological integrity (Greene 2007) Emancipatory, participatory and
anti-discriminatory research that focuses directly on the lives, experiences and perceptions
of marginalized persons or groups (Mertens 2003) Paradigms are logically independent and
thus can be mixed; but although they are useful for reflection, they do not shape practical
research decisions; rather, practical characteristics and issues related to the underlying
context and problem drive these decisions (Greene 2007) Paradigms may be embedded or
intertwined with substantive theories; yet, substantive issues and conceptual theories drive
the mixed research, not paradigms (Greene 2007) Consistent with pragmatist philosophy
but accommodates variations and inconsistencies that prevail within mixed research by
promoting a diversity of researchers, allowing paradigms to operate at different levels,
incorporating group influences on methodological decisions, shifting debates about
paradigms to level of practice and research culture and allowing methods to be chosen
based on their practical value for addressing a research problem (Denscombe 2008)
References Barton, A. and Lazarsfeld, F. (1955) Some functions of qualitative data analysis
in sociological research. Sociologica, 1, 321-361. Bazeley, P. (2003) Computerized data
analysis for mixed methods research. In A. Tashakkori and C. Teddlie (Eds.), Handbook of
mixed methods in social and behavioral research (pp. 385-422) Thousand Oaks CA: Sage.
134 AND THEIR Core Mixed Analysis Strategies Sequential, concurrent, cross-over
24. Sequential, concurrent, cross-over Sequential, concurrent, cross-over Sequential Sequential,
concurrent Sequential, concurrent, cross-over Parallel Sequential, concurrent, cross-over
Sequential, concurrent, cross-over Sequential, concurrent Sequential, concurrent, cross-
over Bazeley, P. (2006) The contribution of computer software to integrating qualitative
and quantitative data and analyses. Research in the Schools, 13(1), 64-74. Becker, H. S.
(1970) Sociological work: Method and substance. New Brunswick NJ: Transaction Books.
Becker, H. S., Geer, B., Hughes, E. C. and Strauss, A. L. (1977) Boys in white: Student culture
in medical school. New Brunswick NJ: Transaction INTERNATIONAL JOURNAL OF
MULTIPLE RESEARCH APPROACHES Volume 3, Issue 2, August 2009 Call for mixed
analysis: A philosophical framework for combining qualitative and quantitative approaches
Books. (Original work published by University of Chicago Press 1961) Bergman, M. (Ed.)
(2008) Advances in mixed methods research. Theories and applications. Thousand Oaks CA:
Sage. Blaikie, N. (1993) Approaches to social enquiry. Cambridge: Polity. Brewer, J. and
Hunter, A. (1989) Multimethod research. Thousand Oaks CA: Sage. Bryman, A. (1988)
Quantity and quality in social research. London: Urwin Hyman. Bryman, A. (1992)
Quantitative and qualitative research: Further reflections on their integration. In J. Brannen
(Ed.), Mixing methods: Qualitative and quantitative research (pp. 89-111) Aldershot:
Avebury Press. Bryk, A. S. and Raudenbush, S. W. (1992) Hierarchical linear models:
Applications and data analysis methods. Newbury Park CA: Sage. Campbell, D. T. and Fiske,
D. W. (1959) Convergent and discriminant validation by the multitrait-multimethod matrix.
Psychological Bulletin, 56, 81-105. Caracelli, V. W. and Greene, J. C. (1993) Data analysis
strategies for mixed-method evaluation designs. Educational Evaluation and Policy Analysis,
15 195-207. Chen, H. T. (2006) A theory-driven evaluation perspective on mixed methods
research. Research in the Schools, 13(1), 75-83. Chi, M. T. H. (1997) Quantifying qualitative
analyses of verbal data: A practical guide. The Journal of the Learning Sciences, 6, 271-315.
Collins, K. M. T., Onwuegbuzie, A. J. and Sutton, I. L. (2006) A model incorporating the
rationale and purpose for conducting mixed methods research in special education and
beyond. Learning Disabilities: A Contemporary Journal, 4, 67-100. Cook, T. D. and Campbell,
D. T. (1979) Quasiexperimentation: Design and analysis issues for field settings. Chicago:
Rand McNally. Corden, A. and Hirst, M. (2008) Implementing a mixed methods approach to
explore the financial implications of death of a life partner. Journal of Mixed Methods, 2,
208-220. Creswell, J. W. and Plano Clark, V. L. (2007) Designing and conducting mixed
methods research. Thousand Oaks CA: Sage. Creswell, J. W. and Plano Clark, V. L. (2008)
Conducting Mixed Methods Research and the Practice of Research. Workshop presented at
the annual Mixed Methods Conference. Cambridge UK. Curtis, S., Gesler, W., Smith, G. and
Washburn, S. (2000) Approaches to sampling and case selection in qualitative research:
Examples in the geography of health. Social Science and Medicine, 50, 1001-1014.
Denscombe, M. (2008) Communities of practice: A research paradigm for the mixed
methods approach. Journal of Mixed Methods Research, 2, 270-283. Denzin, N. K. and
Lincoln, Y. S. (2005) Paradigmatic controversies, contradictions and emerging confluences.
In N. K. Denzin and Y. S. Lincoln (Eds.), Sage handbook of qualitative research (2nd edn, pp.
191-216) Thousand Oaks CA: Sage. Firestone, W. A. (1993) Alternative arguments for
generalizing from data, as applied to qualitative research. Educational Researcher, 22(4),
25. 16-23. Fischer, G. (2001) Communities of interest: Learning through the interaction of
multiple knowledge systems. In S. Bjornestad, R. Moe, A. Morch and A. Opdahl (Eds.),
Proceedings of the 24th IRIS Conference, Ulvik, Norway. Flick, U. (1998) An introduction to
qualitative research: Theory, method and applications. London: Sage. Fox, J. (1997) Applied
regression analysis, linear models and related methods. Thousand Oaks CA: Sage. Fryer, D.
and Feather, N. T. (1994) Intervention techniques. In C. Cassell and G. Symon (Eds.),
Qualitative methods in organizational research (pp. 230-247) London: Sage. Glaser, B. G.
and Strauss, A. L. (1967) The discovery of grounded theory: Strategies for qualitative
research. Chicago: Aldine. Gorard, S. and Smith, E. (Eds.) (2006) Special Issue on ‘Combining
numbers with narratives.’ Evaluation and Research in Education 19(2) Retrieved March 28
2009, from http://www.multilingual-matters .net/erie/019/2/ Greene, J. C. (2006) Toward
a methodology of mixed methods social inquiry. Research in the Schools, 13(1), 93-98.
Greene, J. C. (2007) Mixed methods in social inquiry. San Francisco: Jossey-Bass. Greene, J. C.
(2008) Is mixed methods social inquiry a distinctive methodology? Journal of Mixed
Methods Research, 2, 7-22. Greene, J. C. and Caracelli, V. J. (1997) Defining and describing
the paradigm issue in mixedmethod evaluation. In J. C. Greene and V. J. Caracelli (Eds.),
Advances in mixed-method evaluation: The challenges and benefits of integrating diverse
paradigms (New Directions for Evaluation, No. 74, pp. 5-17) San Francisco: Jossey-Bass.
Greene, J. C., Caracelli, V. J. and Graham, W. F. (1989) Toward a conceptual framework for
Volume 3, Issue 2, August 2009 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH
APPROACHES 135 Anthony J Onwuegbuzie, R Burke Johnson and Kathleen MT Collins
mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11, 255-274.
Guba, E. G. (1990) The alternative paradigm dialog. In E. G. Guba (Ed.), The paradigm dialog
(pp. 17-27) Newbury Park CA: Sage. Hacking, I. (1983) Representing and intervening:
Introductory topics in the philosophy of natural science. New York: Cambridge University
Press. Hammersley, M. (1992) Deconstructing the qualitative–quantitative divide. In J.
Brannen (Ed.) Mixing methods: Qualitative and quantitative research (pp. 39-55) Aldershot:
Avebury Press. Henson, R. K., Capraro, R. M. and Capraro, M. M. (2004) Reporting practice
and use of exploratory factor analysis in educational research journals: Errors and
explanation. Research in the Schools, 11(2), 61-72. Henson, R. K. and Roberts, J. K. (2006)
Use of exploratory factor analysis in published research. Educational and Psychological
Measurement, 66, 393-416. Heron, J. and Reason, P. (1997) A participatory inquiry
paradigm. Qualitative Inquiry, 3, 274-294. Hetzel, R. D. (1996) A primer on factor analysis
with comments on patterns of practice and reporting. In B. Thompson (Ed.), Advances in
social science methodology (Vol. 4, pp. 175-206) Greenwich CT: JAI Press. Houston, S.
(2001) Beyond social constructionism: Critical realism and social work. British Journal of
Social Work, 31, 845-861. Jick, T. D. (1983) Mixing qualitative and quantitative methods:
Triangulation in action. In J. Vam Maanen (Ed.), Qualitative methodology (pp. 135-148)
Thousand Oaks CA: Sage. Johnson, R. B. (2006) Special issue on ‘New directions in mixed
methods research.’ Research in the Schools, 13(1) Retrieved July 8 2008, from
http://www.msera.org/rits_131.htm Johnson, R. B. (2008) Editorial: Living with tensions.
Journal of Mixed Methods Research, 2, 202-207. Johnson, R. B. and Christensen, L. B. (2008)
Educational research: Quantitative, qualitative and mixed approaches (3rd edn) Thousand
26. Oaks CA: Sage. Johnson, R. B. and Onwuegbuzie, A. J. (2004) Mixed methods research: A
research paradigm whose time has come. Educational Researcher, 33(7), 14-26. Johnson, R.
B., Onwuegbuzie, A. J. and Turner, L. A. (2007) Toward a definition of mixed methods
research. Journal of Mixed Methods Research, 1, 112-133. Kant, I. (1781/1998) The
Cambridge edition of the 136 works of Immanuel Kant: Critique of pure reason. Cambridge
UK: Cambridge University Press. Kennedy, M. (1979) Generalizing from single case studies.
Evaluation Quarterly, 3, 661-678. Kidder, L. H. and Fine, M. (1987) Qualitative and
quantitative methods: When stories converge. In M. M. Mark and R. L. Shotland (Eds.),
Multiple methods in program evaluation (New Directions of Program Evaluation, No. 35, pp.
57-75) San Francisco: Jossey-Bass. Kuhn, T. S. (1962) The structure of scientific revolutions.
Chicago: University of Chicago Press. Lather, P. (1986) Research as praxis. Harvard
Educational Review, 56, 257-277. Lather, P. (2006) Paradigm proliferation as a good thing
to think with: Teaching research in education as a wild profusion. International Journal of
Qualitative Studies 19(1), 35-57. Layder, D. (1993) New strategies in social research: An
introduction and guide. Cambridge UK: Polity Press. LeCompte, M. D. and Preissle, J. (1993)
Ethnography and qualitative design in educational research (2nd edn) San Diego CA:
Academic Press. Lee, Y-j. and Greene, J. C. (2007) The predictive validity of an ESL
placement test: A mixed methods approach. Journal of Mixed Methods Research, 1, 366-389.
Leech, N. L. and Onwuegbuzie, A. J. (2007) An array of qualitative data analysis tools: A call
for qualitative data analysis triangulation. School Psychology Quarterly, 22, 557-584. Leech,
N. L. and Onwuegbuzie, A. J. (2008) Qualitative data analysis: A compendium of techniques
and a framework for selection for school psychology research and beyond. School
Psychology Quarterly, 23, 587-604. Leech, N. L. and Onwuegbuzie, A. J. (Eds) (2010) Special
issue on ‘Teaching mixed methods.’ International Journal of Multiple Research Approaches,
4(1), in press. Li, S., Marquart, J. M. and Zercher, C. (2000) Conceptual issues and analytical
strategies in mixed-method studies of preschool inclusion. Journal of Early Intervention, 23,
116-132. Lincoln, Y. S. and Guba, E. G. (1985) Naturalistic inquiry. Beverly Hills CA: Sage.
Louis, K. S. (1982) Sociologist as sleuth: Integrating methods in the RDU study. American
Behavioral Scientist, 26(1), 101-120. Madey, D. L. (1982) Some benefits of integrating
qualitative and quantitative methods in program evaluation, with some illustrations.
Educational Evaluation and Policy Analysis, 4, 223-236. INTERNATIONAL JOURNAL OF
MULTIPLE RESEARCH APPROACHES Volume 3, Issue 2, August 2009 Call for mixed
analysis: A philosophical framework for combining qualitative and quantitative approaches
Mark, M. M. and Shotland, R. L. (1987) Alternative models for the use of multiple methods.
In M. M. Mark and R. L. Shotland (Eds.), Multiple methods in program evaluation (New
Directions of Program Evaluation, No. 35 pp. 95-100) San Francisco: Jossey-Bass. Maxcy, S. J.
(2003) Pragmatic threads in mixed methods research in the social sciences: The search for
multiple modes of inquiry and the end of the philosophy of formalism. In A. Tashakkori and
C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 5189)
Thousand Oaks CA: Sage. Maxwell, J. A. (1996) Qualitative research design. Newbury Park
CA: Sage. Maxwell, J. A. (2004, April) Realism as a stance for mixed methods research. Paper
presented at the annual meeting of the American Educational Research Association, San
Diego CA. Maxwell, J. A., Bashook, G. and Sandlow, C. J. (1986) Combining ethnographic and
27. experimental methods in educational evaluation. In D. M. Fetterman and M. A. Pittman
(Eds.), Educational evaluation: Ethnography in theory, practice and politics (pp. 121-143)
Thousand Oaks CA: Sage. Maxwell, J. A. and Loomis, D. M. (2003) Mixed methods design: An
alternative approach. In A. Tashakkori and C. Teddlie (Eds.), Handbook of mixed methods in
social and behavioral research (pp. 241-272) Thousand Oaks CA: Sage. McConney, A., Rudd,
A. and Ayres, R. (2002) Getting to the bottom line: A method for synthesizing findings
within mixed-method program evaluations. American Journal of Evaluation, 23, 121-140.
McEvoy, P. and Richards, D. (2003) Critical realism: A way forward for evaluation research
in nursing? Journal of Advanced Nursing, 43, 411-420. McEvoy, P. and Richards, D. (2006) A
critical realist rationale for using a combination of quantitative and qualitative methods.
Journal of Research in Nursing, 11, 66-78. Mertens, D. (2003) Mixed methods and the
politics of human research: The transformativeemancipatory perspective. In A. Tashakkori
and C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp.
135-164) Thousand Oaks CA: Sage. Mertens, D. (2008, July) Transformative mixed methods
research. Workshop presented at the annual mixed methods conference. Cambridge UK.
Miles, M. B. and Huberman, A. M. (1994) Qualitative data analysis: An expanded sourcebook
(2nd edn) Thousand Oaks CA: Sage. Morrow, R. A. and Brown, D. D. (1994) Critical theory
and methodology (Contemporary Social Theory) Thousand Oaks CA: Sage. Morse, J. M.
(1995) The significance of saturation. Qualitative Health Research, 5, 147- 149. Morse, J. M.
(2003) Principles of mixed methods and multimethod research design. In A. Tashakkori and
C. Teddlie (Eds), Handbook of mixed methods in social and behavioral research (pp. 189-
208) Thousand Oaks CA: Sage. O’Cathain, A. (2008, July) ‘An…
![post-positivism, critical theory, participatory, constructivism, pragmatism, social science
research typology 114 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES
Volume 3, Issue 2, August 2009 Call for mixed analysis: A philosophical framework for
combining qualitative and quantitative approaches T he process of mixed research –
involving ‘mix[ing] or combin[ing] quantitative and qualitative research techniques,
methods, approaches, concepts or language into a single study’ (Johnson & Onwuegbuzie
2004: 17) – has developed a long way since Campbell and Fiske (1959) coined the term
multiple operationalism, wherein more than one method is used as part of a validation
process to help ensure that the variance explained culminates from the underlying
phenomenon or trait and is not a function of the method. However, most of the biggest
formalized developments in mixed research have occurred within the last 25 years, being
given impetus in the 1980s by several prominent researchers (e.g. Brewer & Hunter 1989;
Bryman 1988; Greene, Caracelli, & Graham 1989; Jick 1983; Kidder & Fine 1987; Louis
1982; Madey 1982; Mark & Shotland 1987; Maxwell, Bashook, & Sandlow 1986; Phelan
1987; Rossman & Wilson 1985), who called for the integration of quantitative and
qualitative approaches. However, the last five years have witnessed a significant increase in
the number of mixed research studies, marked by a handbook on mixed methods
(Tashakkori & Teddlie 2003); several mixed research textbooks (e.g. Bergman 2008;
Creswell & Plano Clark 2007; Greene 2007; Johnson & Christensen 2008; Plano Clark &
Creswell 2007; Ridenour & Newman 2008; Teddlie & Tashakkori 2009); several mixed
research articles contained in methodological handbooks (e.g. Teddlie, Tashakkori, &
Johnson 2008); mixed research articles contained in encyclopedias (e.g. Onwuegbuzie
2007); two journals devoted to mixed research (i.e. Journal of Mixed Methods Research,
International Journal of Multiple Research Approaches); several journals now routinely
publishing mixed research (e.g. Field Methods, Educational Evaluation & Policy Analysis,
Quality & Quantity, Evaluation, Evaluation Practice, Research in Nursing & Health, Research
in the Schools, The Qualitative Report); and websites (e.g. http://www.fiu.edu/~bridges/),
conferences (e.g. http://www.mixedmethods.leeds.ac.uk/) and workshops (Creswell &
Plano Clark 2008; Mertens 2008; O’Cathain 2008; Onwuegbuzie & Collins 2008;
Onwuegbuzie, Slate, Leech, & Collins 2008) devoted to mixed research, and by special issues
(Gorard & Smith 2006; Johnson 2006; O’Cathain & Collins 2009) with another underway
(Leech & Onwuegbuzie, 2010). These and other sources have helped to increase the
visibility of mixed research. Greene (2008: 8) recently asked the following questions: • ‘Is
mixed methods social inquiry a distinctive methodology? • Is the field moving in that
direction? • What is needed for mixed methods to become a distinctive methodology?
According to Greene (2006, 2008), the development of a methodological or research
paradigm (i.e. qualitative, quantitative and mixed research) in the social and behavioral
sciences requires a thorough critique of four interrelated but conceptually distinct domains:
(i) philosophical assumptions and stances (i.e. what are the core philosophical or
epistemological assumptions of the methodology?); (ii) inquiry logics (i.e. what traditionally
is called methodology and refers to broad inquiry purposes and questions, logic, procedures
and designs, quality standards and writing and reporting forms that guide the researcher’s
gaze); (iii) guidelines for research practice (i.e. specific strategies and tools that are used to](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)