Mixing Methods Tutorial


Published on

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Mixing Methods Tutorial

  1. 1. HRM4710 Qualitative Research Methods Week 23 Thinking about ‘triangulation’ & other issues of mixing methods Dr. Maria Kapsali (m.kapsali@imperial.ac.uk) Imperial College London with Dr. Uracha Chatrakul Na Ayudhya
  2. 2. Learning Objectives <ul><li>By the end of this workshop, you should be able to: </li></ul><ul><li>Critically discuss the arguments for & against multi-strategy research </li></ul><ul><li>Appreciate the range of possibilities in engaging in multi-strategy research, including triangulation </li></ul><ul><li>Understand what is meant by ‘mixing methods in a qualitatively driven way’ as outlined by Mason (2006) </li></ul><ul><li>Critically reflect on the appropriateness of multi-strategy research for your own research interests </li></ul>
  3. 3. Contents <ul><li>Part 1 Theory: Qualitative research, Multi-level strategy and triangulation </li></ul><ul><li>Part 2 Types of triangulation + Activity </li></ul><ul><li>Part 3 Practical example </li></ul>
  4. 4. Part 1: Theory Types of qualitative research Case study Attempts to shed light on a phenomena by studying indepth a single case example of the phenomena.  The case can be an individual person, an event, a group, or an institution. Grounded theory Theory is developed inductively from a corpus of data acquired by a participant-observer. Phenomenology Describes the structures of experience as they present themselves to consciousness, without recourse to theory, deduction, or assumptions from other disciplines Ethnography Focuses on the sociology of meaning through close field observation of sociocultural phenomena. Typically, the ethnographer focuses on a community. Historical Systematic collection and objective evaluation of data related to past occurrences in order to test hypotheses concerning causes, effects, or trends of these events that may help to explain present events and anticipate future events. (Gay, 1996)
  5. 5. Multi-strategy research <ul><li>Refers to research that combines qualitative & quantitative research </li></ul><ul><li>On the surface, this can be seen as a straightforward way of breaking down the qual-quant divide </li></ul><ul><li>But if we dig deeper, this approach is not without difficulties </li></ul>
  6. 6. Make a list: <ul><li>What are some of the advantages? </li></ul><ul><li>What are some of the difficulties? </li></ul>
  7. 7. The Argument Against Multi-strategy Research (I) <ul><li>The embedded methods argument: </li></ul><ul><ul><li>Research methods carry epistemological and ontological commitments </li></ul></ul><ul><ul><li>Thus multi-strategy research is not feasible or even desirable </li></ul></ul><ul><li>Why? </li></ul><ul><ul><li>Integrating research strategies could lead to danger of ignoring assumptions underlying research methods </li></ul></ul><ul><ul><li>Could result in transforming “qualitative inquiry into a procedural variation of quantitative inquiry” </li></ul></ul><ul><ul><li>(Smith & Heshusius, 1986: 8) </li></ul></ul>
  8. 8. The Argument Against Multi-strategy Research (II) <ul><li>The paradigm argument: </li></ul><ul><ul><li>Quantitative and qualitative research are separate, incommensurable paradigms </li></ul></ul><ul><ul><li>Thus even when combined they are incompatible: the integration is only at a superficial level and within a single paradigm </li></ul></ul>
  9. 9. Your view: <ul><li>Do you see data as just data? </li></ul><ul><li>Your reflection on this shapes how you engage in the multi-strategy research debate </li></ul>
  10. 10. Two Versions of the Debate About Quantitative and Qualitative Research <ul><li>An epistemological version: </li></ul><ul><ul><li>as in the embedded methods argument and the paradigm argument, multi-strategy research is not possible </li></ul></ul><ul><ul><li>A technical version: </li></ul></ul><ul><ul><ul><li>Emphasises the strengths of data collection and data analysis techniques associated with quantitative and qualitative research and sees these as capable of being fused </li></ul></ul></ul><ul><ul><ul><li>Research methods are perceived as autonomous and compatible </li></ul></ul></ul>
  11. 11. Your view: <ul><li>Think about ways in which multi-strategy research can take place </li></ul><ul><li>In particular, think about how and why </li></ul><ul><li>The ‘why’ question compels us to provide a rationale for the choices we make if and when we choose to combine methods </li></ul>
  12. 12. Subjectivist Projection of human imagination Social construction Contextual field of information Concrete process Concrete structure Objectivist Different assumptions about reality from Morgan and Smircich (1980).   Quantitative vs Qualititive Researcher is independent vs Researcher is involved Large samples vs Small samples Testing theories vs Generating theories Experimental design vs Fieldwork methods Verification vs Falsification   The choices of research design (Hassard, 2002).
  13. 13. Theory Empirical Generalization Deduction, induction and Retroduction representation of how deduction, induction and Retroduction work through the empirics and theory (Alvesson and Skoldberg, 1994). Empirical Deduction Induction Retroduction The four types of scientific abstraction – logical reasoning (as in Bertilsson, 2004). Ontology and epistemology Deduction Logical inferences from major and minor premises Formally correct but sometimes empirically flawed- it depends on the correctness of premises Induction The ability to form probability statements when all conditions being equal the higher past frequency the higher the probability Inductive thinking is susceptible to habit and expectation Abduction The act of seeing something anew to connect a case with an unexpected rule related to vague common sense Abductive reasoning is Perceptual judgments and their clarity is questionable Retroduction Aims to specify the necessary and sufficient causes and conditions to be produced or reproduced, for the phenomenon to come into existence.
  14. 14. Triangulation – popular strategy for mixing methods <ul><li>It is based on the assumption that “any bias inherent in particular data sources, investigator, and method would be neutralised when used in conjunction with other data sources, investigators and methods” </li></ul><ul><li>(Creswell, 1994: 174) </li></ul><ul><li>It assumes that data from different methods will corroborate one another, where the choice of methods is intended to investigate a single social phenomenon from different vantage points </li></ul><ul><li>(Denzin,1970; Brannen,2005) </li></ul>
  15. 15. Combining methods & data: many possible outcomes <ul><li>“ Data collected from different methods cannot be simply added together to produce a unitary or rounded reality” (Brannen 2005: 176) </li></ul><ul><ul><li>Triangulation (see corroboration) is just one of many possible outcomes </li></ul></ul><ul><ul><li>Corroboration: The ‘same result’ are derived from both qualitative and quantitative methods </li></ul></ul><ul><ul><li>Elaboration: The qualitative data analysis exemplifies how the quantitative findings apply in particular cases. </li></ul></ul><ul><ul><li>Complementarity: The qualitative and quantitative results differ but together they generate insights. </li></ul></ul><ul><ul><li>Contradiction: Where qualitative data and quantitative findings conflict. </li></ul></ul><ul><ul><li>(Morgan, 1998, cited in Bryman, 2001; Hammersley, 1996) </li></ul></ul>
  16. 16. Part 2: Types of Triangulation –strategy for mixing methods <ul><li>It is based on the assumption that “any bias inherent in particular data sources, investigator, and method would be neutralised when used in conjunction with other data sources, investigators and methods” </li></ul><ul><li>(Creswell, 1994: 174) </li></ul><ul><li>It assumes that data from different methods will corroborate one another, where the choice of methods is intended to investigate a single social phenomenon from different vantage points </li></ul><ul><li>(Denzin, 1970; Brannen, 2005) </li></ul>
  17. 17. Types of Triangulation <ul><li>Methods Triangulation </li></ul><ul><ul><li>One researcher using two or more research techniques (within and between QUAN-QUAL techniques); </li></ul></ul><ul><ul><li>Two or more researchers using the same research technique; </li></ul></ul><ul><ul><li>Two or more researchers using two or more research techniques. </li></ul></ul><ul><li>Theoretical Triangulation </li></ul><ul><ul><li>Looking at the research situation from different theoretical perspectives. </li></ul></ul><ul><li>Data Triangulation </li></ul><ul><ul><li>Combining qualitative and quantitative data within the same method. </li></ul></ul>The types of data these three methods generate are field notes, audio (and sometimes video) recordings, and transcripts.
  18. 18. Mixed Methods Strategies <ul><li>What is the implementation sequence of the QUAN and QUAL data collection in the proposed study? </li></ul><ul><li>What priority will be given to the QUAN and QUAL data collection and analysis? </li></ul><ul><li>At what stage in the research project will the QUAN and QUAL data and findings be integrated ? </li></ul>
  19. 19. 1. Implementation <ul><li>Sequential </li></ul><ul><ul><li>The researcher collects both the QUAN and QUAL data in phases </li></ul></ul><ul><li>Concurrent </li></ul><ul><ul><li>The researcher collects both the QUAN and QUAL data at the same time </li></ul></ul>
  20. 20. 2. Priority <ul><li>Equal priority </li></ul><ul><ul><li>Same weight is given to QUAN and QUAL </li></ul></ul><ul><li>Dominant and less-dominant priority </li></ul><ul><ul><li>Priority for either QUAN or QUAL </li></ul></ul>
  21. 21. 3. Integration <ul><li>Integration of QUAN and QUAL data might occur at several stages in the process of research: </li></ul><ul><ul><li>Data collection </li></ul></ul><ul><ul><li>Data analysis </li></ul></ul><ul><ul><li>Interpretation </li></ul></ul><ul><ul><li>Or combination of places </li></ul></ul>
  22. 22. Alternative Strategies and Visual Models <ul><li>Sequential Explanatory Strategy </li></ul><ul><ul><li>Use QUAL results to assist in explaining and interpreting the findings of a primarily QUAN study. </li></ul></ul><ul><ul><li>Especially useful when unexpected results arise from QUAN study. </li></ul></ul>
  23. 23. Alternative Strategies and Visual Models (cont.) <ul><li>Sequential Explanatory Design </li></ul>Qual QUAN Data Collection QUAN Data Analysis qual Data Collection qual Data Analysis Interpretation of Entire Analysis QUAN <ul><ul><li>Use QUAN data and results in the interpretation of QUAL findings. </li></ul></ul><ul><ul><li>Focus of this model is on exploring a phenomenon. </li></ul></ul>
  24. 24. Alternative Strategies and Visual Models (cont.) <ul><li>Sequential Exploratory Design </li></ul>QUAL quan QUAL Data Collection QUAL Data Analysis quan Data Collection quan Data Analysis Interpretation of Entire Analysis
  25. 25. Alternative Strategies and Visual Models (cont.) <ul><li>Concurrent Triangulation Strategy </li></ul>QUAN QUAL QUAN Data Collection QUAN Data Analysis QUAL Data Collection QUAL Data Analysis Data results compared <ul><ul><li>Selected as a model when the researcher uses two different methods in an attempt to confirm, cross-validate, or corroborate findings within a single study. </li></ul></ul>
  26. 26. Alternative Strategies and Visual Models (cont.) <ul><li>Concurrent Nested Strategy </li></ul>Qual Quan QUAN QUAL Analysis of Findings Analysis of Findings <ul><ul><li>Often used to gain broader perspectives as a result of using the different methods </li></ul></ul><ul><ul><li>May be employed to study different groups or levels </li></ul></ul>
  27. 27. Other qualitative strategies <ul><li>Multiple case studies, quantification of qualitative data </li></ul><ul><li>Within qualitative methods triangulation </li></ul><ul><li>Participant observation is appropriate for collecting data on naturally occurring behaviours in their usual contexts. </li></ul><ul><li>In-depth interviews are optimal for collecting data on individuals’ personal histories, perspectives, and experiences, particularly when sensitive topics are being explored. </li></ul><ul><li>Focus groups are effective in eliciting data on the cultural norms of a group and in generating broad overviews of issues of concern to the cultural groups or subgroups represented. </li></ul>
  28. 28. Activity : think of your own research <ul><li>What is my ontology </li></ul><ul><li>What is my epistemology </li></ul><ul><li>What are my methods </li></ul><ul><li>What type of data am I going to use </li></ul><ul><li>What kind of answer do I look for: what, how or why </li></ul>
  29. 29. Part 3 Example: qualitative multiple case studies using Critical Realism <ul><li>My Research Question: Why and how projects succeed or otherwise </li></ul><ul><li>Why and how needs in depth rich data </li></ul><ul><li>Qualitative rich data usually are collected through qualitative interviews-focus groups etc and analysed within case studies </li></ul>Problem : Generalizability is low in case studies Problem : I needed to generalize the causes of an extended observed phenomenon in quantitative research
  30. 30. Choices <ul><li>Wouldn’t it be good to generalize from many case studies like as if from a survey </li></ul><ul><li>Finding patterns of behaviour and explanation through many case studies strengthen the argument placed by evidence </li></ul>Problem : Finding patterns in case studies? Horrific ! Blending ontologies and epistemologies – very tricky indeed !
  31. 31. Research design Research Question: What is the effect of Policy on Project Management? Method: Multiple, explanatory comparative cross cases Instrumentation: Human In-depth, Semi-structured interviews Purpose: Building model Understand social interpretations of a phenomenon and built a model to represent it. Data: Subjective Data are perceptions of the people in the environment. Orientation: Causality Find how the issues involved interact. Focus: Holistic A total or complete picture is sought. Reality: Dynamic Reality changes with changes in people’s perceptions. Viewpoint: Insider Reality is what people perceive it to be Results: Valid The focus is on design and procedures to gain &quot;real,&quot; &quot;rich,&quot; and &quot;deep&quot; data. Analysis: Retroduction Deductively collecting inductively analyzing and abductively concluding and theorizing
  32. 32. Normative, Descriptive and Systems theories in Project Management Stakeholderism vs boundary roles Policy Project Management Operational Change Public Policy Implementation Strategy public sector - Institutions Managing projects in change – planned vs emergent approaches Relationality
  33. 33. Table 6.1: The ontological assumptions of CR (Downward, 2008: 314). The ontology of CR (the author from Modell, 2005) Core Ontological Assumptions Reality as a concrete process Assumptions About Human Nature Man as an adaptor Basic Epistemological Stance To study systems, process, change Favored Metaphors Organism Real Actual Empirical Contingent conditions Triggers Observation Experience Intrinsic objects Mechanisms Events and tendencies Patterns may or may not fire may or may not be observable
  34. 34. Real Actual Empirical Contingent conditions Instruments Observation Experience of participants Policy Mechanisms Programme actions and contexts Project Management
  35. 35. Epistemology Basic ideas Role of theory What is real is not given. The world has structure (there are levels of reality) and emergent structures. People’s involvement with structures is transformational. Subject matter has to reflect both its meaningfulness to actors and their location in a given network of relationships and structures. Knowledge is dualistic. Theory is a conjecture about the connectedness of events and the causal sequences produced by generative mechanisms. Nature of explanation Method of study   Something is explained if it is allocated a place at the end of a causal sequence. There may be multiple causes of a single event, co-variation and feedback The aim is to produce a good theory which accurately identifies causal mechanisms. The ways these work themselves out in given cases will be complicated. Multiple data is required   Ackroyd (2004: 150-1) on the characteristics of CR
  36. 36. <ul><li>17 multiple case studies </li></ul><ul><li>Could quantify results but didn’t </li></ul><ul><li>Instead I made a conceptual model based on the common causal explanations directly within and across embedded case studies </li></ul><ul><li>I linked the data to the literatures using the framework of open systems </li></ul>
  37. 37. Mixing methods in a qualitatively driven way… Jennifer Mason (2006)
  38. 38. Mixing methods in a qualitative driven way (Mason, 2006) <ul><li>Contributes to the debate of multi-strategy research by arguing “the value of mixed-methods approaches for researching questions about social experiences and lived realities” (Mason, 2006: 9) </li></ul><ul><li>Makes a case for looking at mixed-methods as “multi-dimensional research that transcend or even subvert the so-called qualitative divide” (Mason, 2006: 9) </li></ul><ul><li>“ social experience and lived realities and multi-dimensional... our understandings are impoverished and may be inadequate if we view these phenomena only along a single dimension” (Mason, 2006: 10) </li></ul>
  39. 39. ‘ Meshing’ methods? (I) <ul><li>Mason (2006) proposes that instead of talking about ‘integrating’ methods & data, perhaps it is more useful to describe mixing methods as ‘linking’ or ‘meshing’ processes </li></ul><ul><li>“ But how can this be done without sinking into a relativist mire, where we have many different and fragmented descriptions of social experience, but no real explanation of anything? On the face of it, mixed- methods approaches are trapped between the devil and the deep blue sea” </li></ul><ul><ul><ul><ul><ul><li>(Mason, 2006: 20) </li></ul></ul></ul></ul></ul>
  40. 40. ‘ Meshing’ methods? (II) <ul><li>“ I think the answer lies in how we construct our explanations and what we expect them to do. Explanations do not have to be internally consensual and neatly consistent to have meaning and to have the capacity to explain. Indeed, if the social world is multi-dimensional, then surely our explanations need to be likewise?” (Mason, 2006: 20) </li></ul>
  41. 41. ‘ Meshing’ methods? (III) Essentially, Mason (2006) argues for going beyond divides, including qual-quant; micro-macro; global-local, socio-cultural-individual in order to acknowledge the multi-dimensionality of contexts & how they intersect to shape social experience & lived realities
  42. 42. Qualitative derived principles for mixing methods (Mason, 2006: 21-22) (I) <ul><li>Underpinned by a qualitative constructivist approach & the aim of understanding the how and why of social experience & lived realities </li></ul><ul><li>A questioning, reflexive, and non-accepting approach to research design & practice </li></ul><ul><li>Recognising the validity (legitimacy) of more than one approach </li></ul><ul><li>A flexible, creative approach </li></ul>
  43. 43. Qualitative derived principles for mixing methods (Mason, 2006: 21-22) (II) <ul><li>Celebrating richness, depth, complexity, and nuance (through embracing a range of data types & sources, including ‘quantitative’ understandings) </li></ul><ul><li>A reflexive approach to what it is that data represent & how they constitute knowledge </li></ul><ul><ul><li>this involves questions about the contexts/situatedness of the social phenomena / processes being investigated & extent to which the methods used can provide knowledge about them </li></ul></ul><ul><ul><li>E.g. Are they to be found in people’s behaviours, practices, imaginations, in physical or visual environments, in norms or discourses, etc.? </li></ul></ul>
  44. 44. Questions?
  45. 45. <ul><li>Reading </li></ul><ul><li>Bryman and Bell (2007): Chapter 25: Mixed methods research: Combining quantitative and qualitative research </li></ul><ul><li>You might also want to look at: </li></ul><ul><li>Brannen, J. (2005). Mixing Methods: The Entry of Qualitative and Quantitative Approaches into the Research Process. Int. J. Social Research Methology , 8(3), 173-184. </li></ul><ul><li>Bryman, A. (2007). Barriers to Integrating Quantitative and Qualitative Research. Journal of Mixed Methods Research, 1(1), 8-22. </li></ul><ul><li>Mason, J. (2006). Mixing methods in a qualitatively driven way. Qualitative Research , 6(1), 9-25. </li></ul>
  46. 46. Other references <ul><li>Campbell, D.T. and Fiske, D.W. (1959): “Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix”, Psychological Bulletin, iss.2, pp.81-105. </li></ul><ul><li>Cherryholmes, C.H. (1993): “Notes on Pragmatism and Scientific Realism”, Educational Researcher, vol.21, iss.6, pp.13-17. </li></ul><ul><li>Creswell, J.W. (1994): Research Design: Qualitative & Quantitative Approaches, (Sage Publications: U.S.A.). </li></ul><ul><li>Creswell, J.W. (2003): Research Design: Qualitative, Quantitative and Mixed Methods Approaches, 2nd ed., (Sage: U.K.). </li></ul><ul><li>Datta, L. (1994): “Paradigm Wars: A Basis for Peaceful Coexistence and Beyond”, in Reichardt, C.S. and Rallis, S.F. (eds.): The Qualitative-Quantitative Debate: New Perspectives, pp.53-70, (Jossey-Bass: San Francisco). </li></ul>
  47. 47. ...... references ....... <ul><li>Denzin, N.K. (1978): The Research Act: A Theoretical Introduction to Sociological Methods, (McGraw-Hill: New York). </li></ul><ul><li>Fielding, N.G. and Fielding, J.L. (1986): Linking Data, (Sage Publications Inc.: London). </li></ul><ul><li>Flick, U. (1991): “Triangulation”, in Flick, U., Kardoff, E., Keupp, H., Rosenstiel, L., and Wolff, S. (eds.): Handbuch Qualitative Sozialforschung, pp.432-434, (Psychologie Verlags Union: Munich). </li></ul><ul><li>Flick, U. (1992): “Triangulation Revisited: Strategy of Validation or Alternative?”, Journal of Theory of Social Behaviour, iss.2, pp.175-197. </li></ul><ul><li>Flick, U. (1998): An Introduction to Qualitative Research, (Sage: U.S.A.). </li></ul><ul><li>Silverman, D. (2000): Doing Qualitative Research: A Practical Handbook, (Sage Publications Inc.: London). </li></ul><ul><li>Tashakkori, A. and Teddlie, C. (1998): Mixed Methodology: Combining Qualitative and Quantitative Approaches, Applied Social Research Methods Series, vol.46, (Sage Publications, London). </li></ul><ul><li>Webb, E.J., Campbell, D.T., Schwartz, R.D., and Sechrest, L. (1966): Unobtrusive Measures: Nonreactive Research in the Social Sciences, (Rand McNally: Chicago). </li></ul>
  48. 48. ...... references .......... <ul><li>Greene, J.C., Caracelli, V.J., and Graham, W.F. (1989): “Toward a Conceptual Framework for Mixed-Method Evaluation Designs”, Educational Evaluation and Policy Analysis, vol.11, iss.3, pp.255-274. </li></ul><ul><li>Jick, T.D. (1979): “Mixing Qualitative and Quantitative Methods: Triangulation in Action”, Administrative Science Quarterly, vol.24, iss.4, pp.602-611. </li></ul><ul><li>Johnson, R.B. and Onwuegbuzie, A.J. (2004): “Mixed Methods Research: A Research Paradigm Whose Time Has Come”, Educational Researcher, vol.33, iss.7, pp.14-26. </li></ul><ul><li>Lamnek, S. (1995): Qualitative Sozialforschung, Band 1: Methodologie, (Psychologie Verlags Union: Germany). </li></ul><ul><li>Murphy, J.P. (1990): Pragmatism: From Peirce to Davidson, (Westview Press: Oxford). </li></ul>