Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Maryam Bolouri
Mixed method
Research
Toward New Designs
in
Applied Linguistics
Hashemi &Babaii
MLJ 97,4, 2013
Movement in inquiry in AL
• Emergence of Mixed method
• Driving forces behind this shift:
1.Advances in natural science
2....
Statement of problem:
• How these 2 methods can be integrated
usefully in a way to enhance the research
tools?
• How these...
Purpose of the study:
• This study attempted to be a first move to
study
1.The current status of mixed methods
research in...
Review of literature
• Mixed method definition:
1.Simple definition by Bergman (2008): the combination
of at least one qua...
Mixed method origins
• Mixed method research has evolved from mono method
research.
• The first instances of mixing design...
Theoretical issues in mixed
method research
• Incompatibility thesis
• commensurability validity
• Pragmatist position (di...
Methodological issues:
in mixed method research
• Bazeley (2004): corruption of methods
“It more that using a variety of m...
Mixed method validity issues:
(Onwuegbuzie &Johnson, 2006): legitimation
• Complementary strength and non-overlapping
weak...
Methodological issues:
in mixed method research
• (Tashakkori & Teddlie, 2003): Inference
• A researcher’s construction of...
Research questions:
1. What kinds of research designs are used
when method mixing is utilized in AL
research?
2. What kind...
Methodology
1. Data gathering
• How to select a representative sample
of all journals?
2. Data analysis
• What frameworks ...
Data gathering
• 7 comprehensive international peer reviewed
AL journals
• Over a period of 14 years 1995---2008
• Journal...
Data gathering (cont.)
• The first four sources yielded a comprehensive
list of 64 journals
• Expert judgment
• This exclu...
The final list of Journals
1. Applied linguistics-AL
2. English for specific purpose-ESP
3. Language learning-LL
4. Langua...
The process of gathering the
data sources
• The electronic versions
• Screening phase (abstracts and
methodology)
• Search...
Review of Data gathering
Data Analysis
1. Rigorous qualitative content analysis
(rich description of mixing in the process of data
collection and d...
Creswell et al. (2008)
analysis of research designs
• Concurrent designs and Sequential designs
• Concurrent triangulation...
Collins et al. (2007)
2 dimensional model of mixed methods
sampling
• Time orientation of components
(concurrent or sequen...
Data Analysis (cont.)
• Content analysis based on FWs
• Consider the arguments of validity and
inference quality
Tashakkor...
Data Analysis (cont.)
Design quality
1. Design suitability
2. Design adequacy
3. Within design
consistency
4. Analytic ade...
What is Integrative efficacy?
• The degree to which inferences made in
each strand of mixed methods study are
effectively ...
extended definition of Tashkkori
& Creswell (2007a)
1. Mixing should happen at different
stages of a particular study
2. M...
The reliability of the coding
• 1) An independent coder with qualitative
background coded a sample of articles
( 10% of th...
Review of Data Analysis
Research question1: What kinds of research
designs are used when method mixing is
utilized in AL research?
Research question1: What kinds of research
designs are used when method mixing is
utilized in AL research?
• The direct us...
Research question2: What kinds of sampling
designs are used in mixed methods research in
the field of AL?
Research question3: How are general
conclusions or meta-inferences developed in
this type of research?
• The analyses of d...
Discussions:
1- Use of mixed research design
Concurrent triangulation design (66%)
1. Providing the supplementary data in ...
Discussions:
1- Use of mixed research design
• Sequential exploratory design:
• development of questionnaire tests invento...
Discussions
2-Use of mixed sampling design
This study (2013)
• Concurrent Identical designs
• multilevel concurrent>>>
Con...
Discussions :
3-Inference quality
• Why there was scant attention to the concept of
integrative efficacy among the article...
Implications of the study
• 1) To explore complicated process and outcomes
• 2) Concurrent triangulation designs (DA, GA, ...
Implications of the study
(Cont.)
• Sequential exploratory designs: validating
evaluation criteria such as tests, surveys,...
Summary
• Innovative mixed method can be used in
design and sampling in AL based on
• Research purpose
• Requirements of t...
Penny for thoughts
• What are the evaluative criteria for
investigating the integration quality in AL
research?
• What is ...
Thank You
for your
patience 
Bolouri qualitative method
Upcoming SlideShare
Loading in …5
×

Bolouri qualitative method

365 views

Published on

a review of hashemi Babae

Published in: Design
  • I can advise you this service - ⇒ www.WritePaper.info ⇐ Bought essay here. No problem.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Bolouri qualitative method

  1. 1. Maryam Bolouri Mixed method Research Toward New Designs in Applied Linguistics Hashemi &Babaii MLJ 97,4, 2013
  2. 2. Movement in inquiry in AL • Emergence of Mixed method • Driving forces behind this shift: 1.Advances in natural science 2.Development and challenges of social and educational research 3.Criticism against the dichotomy of qualitative and quantitive research methods
  3. 3. Statement of problem: • How these 2 methods can be integrated usefully in a way to enhance the research tools? • How these 2 methods can be integrated usefully in a way to enhance the quality of obtained findings?
  4. 4. Purpose of the study: • This study attempted to be a first move to study 1.The current status of mixed methods research in AL 2.Explore the nature of mixed research designs, sampling designs and the quality of inferences that can be obtained
  5. 5. Review of literature • Mixed method definition: 1.Simple definition by Bergman (2008): the combination of at least one qualitative and at least one quantitive component in a single research project or program 2.Technical definition by Hanson et. al (2005): the collection, analysis and integration of quantitive and qualitative data in a single or multiphase study 3.Extended definition by Tashakkori & Creswell (2007a): research in which the investigator collects, and analyzes data integrate the findings and draws inferences using both qualitative and quantitive approaches in a single study or program of inquiry
  6. 6. Mixed method origins • Mixed method research has evolved from mono method research. • The first instances of mixing designs can be traced back to 1800s. • It has been influenced by the emergence of the “Triangulation” • Navigation: taking bearings on 2 landmarks in order to locate one’s position • Land surveying: determine the position of point C if we know the positions of points A and B • Original definition: the use of multiple forms of qualitative research methods in favor of richly developed multidimensional descriptions of the world
  7. 7. Theoretical issues in mixed method research • Incompatibility thesis • commensurability validity • Pragmatist position (dialectical, balanced, transformative or critical interpretive view): • From practice oriented perspective. It is not a major hindrance, the value is evident and they can coexist in a dialectical relationship. • The findings can enhance the understanding. • The absent one would remain partially answered or wholly unanswered questions.
  8. 8. Methodological issues: in mixed method research • Bazeley (2004): corruption of methods “It more that using a variety of methods….it is an attempt to serve the dual purposes of generalization and in- depth understanding…” • High quality mixed method research: Integrations happens to RQ, units of analysis, sample, instrumentation, data collection methods and analytic strategies
  9. 9. Mixed method validity issues: (Onwuegbuzie &Johnson, 2006): legitimation • Complementary strength and non-overlapping weakness of both are combined, so the validity assessment is important and complex. • (Tashakkori & Teddlie, 2008): inference quality • It gathers info that result in meta inferences that neither of method can do alone. They are an overall conclusion through integration. Methodological issues: in mixed method research
  10. 10. Methodological issues: in mixed method research • (Tashakkori & Teddlie, 2003): Inference • A researcher’s construction of the relationship among people, events, and variables as well as the his/her construction of respondent’s perceptions, behaviors, and feelings, and how there relate to each other in a coherent and systematic manner
  11. 11. Research questions: 1. What kinds of research designs are used when method mixing is utilized in AL research? 2. What kinds of sampling designs are used in mixed methods research in the field of AL? 3. How are general conclusions or meta inferences developed in this type of research?
  12. 12. Methodology 1. Data gathering • How to select a representative sample of all journals? 2. Data analysis • What frameworks and typologies inform data analysis?
  13. 13. Data gathering • 7 comprehensive international peer reviewed AL journals • Over a period of 14 years 1995---2008 • Journal selection: based on 5 sources: 1. The list of professional journals published by MLJ- Weber& Campbell, 2004 2. Evaluation of AL journals -Egbert (2007) 3. Examination of the frequency of the appearance of ELT journals selected forpresentation in lg teaching btw 1996- 2002-Jung, (2004) 4. A study on the use of qualitative and quantitative methods in 332 articles overa 7 yearperiod- Lazarton (2000) 5. Expert judgment (2 PhDholders with research experience of
  14. 14. Data gathering (cont.) • The first four sources yielded a comprehensive list of 64 journals • Expert judgment • This exclusion was based on the following five reasons: 1.An indirect relationship to the field of AL such as: computers and compositions, British journal of educational psychology. 2.Publishing in lg otherthan English 3.Insufficient info in the methodology section of articles 4.Lesserprominence on the international scene
  15. 15. The final list of Journals 1. Applied linguistics-AL 2. English for specific purpose-ESP 3. Language learning-LL 4. Language teaching research-LTR 5. Language testing-LT 6. The modern language journal-MLJ 7. TESOL quarterly-TQ
  16. 16. The process of gathering the data sources • The electronic versions • Screening phase (abstracts and methodology) • Search function of adobe reader A total of 273 articles were identified
  17. 17. Review of Data gathering
  18. 18. Data Analysis 1. Rigorous qualitative content analysis (rich description of mixing in the process of data collection and data analysis) Content analysis was based on both: • Manifest content: the concrete terms used for coding • Latent content: the coder’s interpretation of the content 2. Make use of code-sheets and respective categories based on three FWs
  19. 19. Creswell et al. (2008) analysis of research designs • Concurrent designs and Sequential designs • Concurrent triangulation design (parallel) • Concurrent embedded design( btw pre and post tests) • Sequential explanatory design : quan->qual • Sequential exploratory design: qual->quan • Sequential embedded design: before and after the intervention--qual
  20. 20. Collins et al. (2007) 2 dimensional model of mixed methods sampling • Time orientation of components (concurrent or sequential) • Relationship of the qualitative and quantitative samples 1.Identical 2.Multilevel 3.Nested 4.Parallel
  21. 21. Data Analysis (cont.) • Content analysis based on FWs • Consider the arguments of validity and inference quality Tashakkori and Teddlie’s (2008) integrative model of inference quality in mixed method • It consists of 2 broad categories of 1.Design quality 2.Integrative rigor
  22. 22. Data Analysis (cont.) Design quality 1. Design suitability 2. Design adequacy 3. Within design consistency 4. Analytic adequacy Integrative rigor 1. Interpretive consistency 2. Theoretical consistency 3. Interpretive agreement 4. Interpretive distinctiveness 5. Integrative efficacy
  23. 23. What is Integrative efficacy? • The degree to which inferences made in each strand of mixed methods study are effectively integrated into a theoretically consistent meta-inference. • Simply speaking: effective integration of both qual and quan findings
  24. 24. extended definition of Tashkkori & Creswell (2007a) 1. Mixing should happen at different stages of a particular study 2. Meta inference must be developed based on data integration both qual and quan It led to the exclusion of 68 articles and the final sample included 205 articles
  25. 25. The reliability of the coding • 1) An independent coder with qualitative background coded a sample of articles ( 10% of the sample 20 articles) • 2) Cohen’s Kappa was calculated . It was • 0.82 for coding of research designs • 0.71 for coding of sampling designs. • 3) These indices were checked against Altman (1991) criteria and proved to be acceptable.
  26. 26. Review of Data Analysis
  27. 27. Research question1: What kinds of research designs are used when method mixing is utilized in AL research?
  28. 28. Research question1: What kinds of research designs are used when method mixing is utilized in AL research? • The direct use of mixed designs was directly acknowledged in only a few studies • . Triangulation at the level of analysis is referred to as triangulation by analyses (68 articles out of 136 articles that used triangulation) • The term triangulation was used With different wording such as: Multiple data source, multiple measures, qualitative and quantitive data • the emergence of mixed method designs that were not in Creswell et al. typology (7 articles) next table
  29. 29. Research question2: What kinds of sampling designs are used in mixed methods research in the field of AL?
  30. 30. Research question3: How are general conclusions or meta-inferences developed in this type of research? • The analyses of discussion and conclusion sections revealed that data integration and consolidation were not presented • These meta-inferences were offered by researchers in order to: 1.enrich their discussions and conclusions 2.present informative summaries of the findings
  31. 31. Discussions: 1- Use of mixed research design Concurrent triangulation design (66%) 1. Providing the supplementary data in multilevel D.A 2. Needs analysis 3. Program evaluation (collect data from various sources or by multiple methods) Lg Testing and ESP journals (Need assessment /program evaluation) Adopting a particular design could be a practical considerations Concurrent embedded designs(MLJ and lg learning) Sequential explanatory design
  32. 32. Discussions: 1- Use of mixed research design • Sequential exploratory design: • development of questionnaire tests inventories • validating the measurement instruments • Sequential embedded designs • examine cross sections of the test development and validation process. • ethnographical studies in order to develop patterns in a more rigorous way.
  33. 33. Discussions 2-Use of mixed sampling design This study (2013) • Concurrent Identical designs • multilevel concurrent>>> Concurrent nested • Why: • different samples of articles in terms of disciplines, size, purpose, rationale for mixing • Why concurrent identical: • Practicality, feasibility, time span Collins et al (2007) • Concurrent Identical designs • Concurrent nested>>>multilevel concurrent
  34. 34. Discussions : 3-Inference quality • Why there was scant attention to the concept of integrative efficacy among the articles? • The concept hasn’t been introduced appropriately to AL research • It hasn’t been widely used in AL research • It can be explained by Tashakkori and Creswell distinction btw • Methodology: integration of the 2 approaches throughout the study • Method: collecting and analyzing different types of qual and quan data
  35. 35. Implications of the study • 1) To explore complicated process and outcomes • 2) Concurrent triangulation designs (DA, GA, NA, …) • 3) Concurrent embedded design (Lg development or a process within an experiment) • 4) Sequential explanatory designs: • To explore a subsample from large samples used in qual phase • Analysis of outliers-extreme cases-inconsistent components or incompatible subsystems • Further analysis of performance on quan measurement instruments and evaluate DIF items after statistical analyses
  36. 36. Implications of the study (Cont.) • Sequential exploratory designs: validating evaluation criteria such as tests, surveys, questionnaire, scales or band descriptors • Sequential embedded design: longitudinal studies of development process in SLA analyze cross sections of test development and validation process to investigate social aspects of lg learning such as T ID, L ID, T D, L D, interaction patterns,…
  37. 37. Summary • Innovative mixed method can be used in design and sampling in AL based on • Research purpose • Requirements of the research context High quality integration in mixed method depends crucially on design quality and quality of meta-inferences • This can happen as result of integration of qual and quan components in all of the stages of the study
  38. 38. Penny for thoughts • What are the evaluative criteria for investigating the integration quality in AL research? • What is the tendency of some journals in use of particular designs? (As it was not the main focus of this study, more accurate accounts of it would require further research.)
  39. 39. Thank You for your patience 

×