Quality in qualitative research the role of the software’s in quality assurance


Published on

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Quality in qualitative research the role of the software’s in quality assurance

  1. 1. Computer-Aided Qualitative Research Europe 7 & 8 Oct 2010, Lisbon For more information about our events, please visit: http://www.merlien.org
  2. 2. 3rd European workshop on COMPUTER-AIDED QUALITATIVE RESEARCH 2010 Quality in Qualitative Research: The role of the software’s in Quality Assurance SÍLVIA SILVA SARA RAMOS ISCTE-IUL
  3. 3. OVERVIEW Main points of our presentation: • Aims • Background • Literature review and results • Quality and CAQDAS Additionally : - Distribution of Some Examples - Distribution of List of Main References
  4. 4. AIM To present a review of quality criteria in Qualitative Research and explore and conclude about the potential roles that software packages may play in quality assurance.
  5. 5. Specific goals: (1) Identify the criteria used (2) Identify the most common/consensual criteria; (3) Link quality to CAQDAS (4) Contribute for debate and consensus (?)
  6. 6. Background • In the last thirty years several authors proposed criteria for considering when approaching the issue of Quality in Qualitative Research (QR). • It is somehow recognized that we are far from having shared quality assumptions
  7. 7. Background: Examples on well-known criteria presented in books • Guba and Lincoln (1985) Trustworthiness : • Transferability ; • Credibility ; • Dependability; • Confirmability • Bauer & Gaskell (2000) Accountability: • Confidence • Relevance • Flick (2007/9) • Transparency, • Documentation • Writing
  8. 8. Research accountability (Gaskell & Bauer, 2000) • Confidence (c) Triangulation and (results represent the Reflexivity (c) reality studied) Transparency and • Relevance (r) procedural clarity (c) (utility and importance) Corpus construction (c,r) Thick description (c,r) Local surprise (r) Communicative validation (r)
  9. 9. Background: Why to focus on Quality • The relevance of quality criteria: epistemological issues and practical issues • A well-written description of the rigour in research analysis should convince readers that the study findings are credible and trustworthy (Belgrave et al., 2002). Moreover: • QR Internal needs (development and proliferation) (Flick,2007) • QR external challenges (e.g. competition: publication; funding; teaching and curriculum planning) (Flick,2007)
  10. 10. The big question: One size fits all?
  11. 11. Literature Review: Search Approach (I) • Covering last 20 years • Databases: - Psycharticles; - ABI/INFORM; - ISI - Google academics • Search in the Abstracts and Titles
  12. 12. Literature Review: Search Approach (II) • Papers focusing Quality Issues/Criteria in Qualitative Research • Keywords used: Quality+ Qualitative Research/Methods; Validity+ Qualitative Research/Methods; Rigor/our + Qualitative Research/Methods; Trustworthiness +Qualitative Research/Methods; all the above with Computer Programs, Software and CAQDAS • Results: > 100 papers ; >30 specifically focusing quality as the main topic
  13. 13. Literature Review: Main Characteristics • Huge progressive increase of papers about Quality Issues in QR or QS in the last 20 years • Journals: Either general (Methods in General; Methods in QR); either in specific fields (mainly: health; business; education; communication)
  14. 14. Literature Review: Main Characteristics Therefore: Papers focusing QR Quality in specific research fields VS generic criteria and recommendations • Papers departing from validity & reliability and framing them in the QR VS papers against that assumptions
  15. 15. Literature Review: Main Findings Main types of papers focus: • Criteria used by Journals and Referees • Specific Criteria for specific QR • Generic Criteria for all QR • Techniques for assuring quality: Audit; triangulation; … • Quality and CAQDAS Most cited Authors: • Guba and Lincoln (1985)
  16. 16. LR Results – Journals and Referees criteria • Papers focusing specific Criteria used by Journals and Referees to sustain Qualitative Papers Review Examples: • Academy of Management Journal Editors (2002, 2004 & 2009) • Savall et al. (2008) • Crescentini & Mainard (2009)
  17. 17. Academy of Management Journal 2009 Editors: Tips for Writing Qualitative Papers 1. Make sure your paper includes “the basics”: • Discuss why this research is needed • Are you building a new theory or elaborating existing theory? • Why did you choose this context and this “unit of analysis”? • How did you get from your data to your findings? 2. Show data in a smart fashion 3. Think about using/organizing figures 4. Think about telling a story 5. Consider “modeling” someone whose style you like who consistently publishes qualitative
  18. 18. 10 (generic) Criteria used by Reviewers in an European Management Journal Savall et al. (2008) • Rigor • Formulation • Coherency • Originality • Relevance • Explication • Positioning • Contribution • Rationale • Delimitation
  19. 19. LR Results – Specific Criteria • Papers focusing specific Criteria for specific Qualitative Approaches: epistemological adequate criteria • For instance: adequate to the the type of analysis: Content Analysis (e.g., Lombard et al., 2002); Grounded Theory (e.g., Chiovitti & Piran, 2002; Elliot et al., 2005); Discourse Analysis (e.g., Nixon, 2007 )
  20. 20. LR Results – Specific Criteria • For instance: Characterizing the Philosophy and Politics of Quality in QR. Distinguishing: review of quality indicators attached to - Foundational; - Quasi-Foundational; - Non-Foundational QR (Amis & Silk, 2008 )
  21. 21. LR Results - Generic Criteria or Recomendations I • Papers focusing specific Criteria for generic/transversal Qualitative Approaches: criteria that applies to all approaches
  22. 22. LR Results - Generic Criteria or Recomendations II Authors Criteria Akkerman et al. •Visibility •AUDITING (2008) •Compreehensibility •Acceptability Morrow (2005) •Social validity Guidelines for Writing QR •Subjectivity and Reflexivity •Adequacy of data •Adequacy of interpretation Shank & •Investigative depth Villela(2004) •Interpretative adequacy •Illuminative fertility •Participatory accountability Whitemore et al., •Primary Criteria (for all QR) Primary criteria: Credibility; (2001) •Secondary Criteria Authenticity; Integrity and •Tecnhiques Criticality Techniques: Design consideration; Data generating; Analytic; Presentation
  23. 23. Meyrick (2008): Good Qualitative Research
  24. 24. Rolfe (2006) “the commonly perceived quantitative–qualitative dichotomy is in fact a continuum which requires a continuum of quality criteria, or to recognize that each study is individual and unique, and that the task of producing frameworks and predetermined criteria for assessing the quality of research studies is futile.” “individual judgments of individual studies”
  25. 25. Quality Objects • Theory, Method and • Adequate Epistemological methodological Coherence approach considering the analytic grounding • Design and Report of the Research General: • Data Collection • Transparency • Data Analysis • Reflexivity
  26. 26. LR Results – Quality and CAQDAS • Papers focusing the role of CAQDAS on assuring Rigor and Quality
  27. 27. LR Results – Quality and CAQDAS Authors Focus Lu & Shulman (2008) CAQDAS and Rigour and Flexibility and the use of CAT (Coding Analysis Toolkit) Rambaree (2007) Rigour in Qualitative Social Research: The Use of a CAQDAS Atlas.ti example Sin (2007) CAQDAS for achieving Transparency Illustration with NVivo Sinkovics et al. (2008) CAQDAS for achieving trustworthiness of QR in Business Research Westphal (2000) N4 and Trustworthiness: Searching for negative evidence – Easier to Find Inconsistency Linking Data and conclusions and theory Conducting Coding Checks Audit Trails and Conducting Audits Detailed reporting
  28. 28. What Software Do (Gibbs, Lewins & Silver, 2005; Lewins & Silver,2009) • Structure work (access and organization of all project elements) • 'Closeness to data' interactivity (quick access to source data files) • Explore data • Code and Retrieve Functionality • Project Management and Data Organisation • Search and interrogating the database • Writing tools - Memos, comments and annotations, • Output - Reports to view a hard copy or export to another package.
  29. 29. Advantages (Gibbs, Lewins & Silver, 2005; Lewins & Silver,2009) • Organised and controllable data set • Support for coding • Searching Text and codes • Support for comparative analysis • Models, networks and diagrams • Interface with quantitative data
  30. 30. Qualitative Softwares – Possible contributions for quality (I) •Data and Coding: More Easy to Be sure that all data are analysed (assure that participants perspectives are covered) and code and retrieve approach more complete and rigorous Reporting: More adequate support for reporting results and allowing simple and complex results
  31. 31. Qualitative Softwares – Possible contributions for quality (II) •Good support for assisting a systematic analysis approach: allows and simplifies the use of “equivalent” procedures to all the data •TRANSPARENCE and Reflexivity : Allowing a more easy access to the data analysis (e.g., Codes; Quotations; Memos); methodological analytic decisions, and reflexions (Memos, Comments) •Facilitation of TRIANGULATION and AUDIT: Simplifies the combination of different sources of data (or other type of triangulation); team work; …
  32. 32. Qualitative Softwares – Possible contributions for quality (III) Besides: allows the researchers in assuring quality when following most of the specific recommendations But: it can also be misused and give “overconfidence”
  33. 33. Conclusions (I) • There is not a single criteria solution adequate for all “one best way” • Quality issues cover both theoretical issues and technical problems • There is an ethical obligation to demonstrate “Rigor” and Integrity of research • Quality Reflexion is needed but we are still far from consensus: We suggest some debate on this
  34. 34. Conclusions (II) Anyway: • Debate and establishing “criteria” about Quality of QR contributes for a bigger awareness of methodological decisions during QR (Seale, 1999) • CAQDAS may play an important role but it always depend on it is used
  35. 35. Conclusions (III) Myth 94: qualitative researchers will agree about validity Sparkes (2001)
  36. 36. Our Future work on this subject • First Paper in Preparation • Project about the Quality of Qualitative Research and CAQDAS(without funding at the moment) in the beginning; looking forward possible cooperation/network for proposal to be submitted to the Portuguese Science Foundation
  37. 37. References A list with the Main References will be distributed
  38. 38. Computer-Aided Qualitative Research Europe 7 & 8 Oct 2010, Lisbon For more information about our events, please visit: http://www.merlien.org