Qualitative research, lab report overview, and review of lectures 1 to 7

17,884 views

Published on

This lecture introduces qualitative research and qualitative analysis, overviews the lab report tasks, and summarises Lectures 1 to 7. See also http://ucspace.canberra.edu.au/pages/viewpage.action?pageId=57409703

Published in: Education, Technology, Business
1 Comment
34 Likes
Statistics
Notes
No Downloads
Views
Total views
17,884
On SlideShare
0
From Embeds
0
Number of Embeds
711
Actions
Shares
0
Downloads
0
Comments
1
Likes
34
Embeds 0
No embeds

No notes for slide
  • 7126/6667 Survey Research & Design in Psychology Semester 1, 2011, University of Canberra, ACT, Australia James T. Neill Home page: http://ucspace.canberra.edu.au/display/7126 Lecture page: http://en.wikiversity.org/wiki/Survey_research_methods_and_design_in_psychology http://ucspace.canberra.edu.au/pages/viewpage.action?pageId=57409703 Image source http://www.wordle.net/gallery/wrdl/803807/Qualitative_research License: CC-by-A 3.0 Description: This lecture introduces qualitative research and qualitative analysis, overviews the lab report tasks, and summarises Lectures 1 to 7.
  • Image source: http://commons.wikimedia.org/wiki/File:Information_icon4.svg License: Public domain
  • Image source: http://commons.wikimedia.org/wiki/File:Information_icon4.svg License: Public domain What is qualitative research? Qualitative analysis - How to analyse qualitative data?
  • Image name: Concept for augmented reality mobile phone Image source: http://www.flickr.com/photos/22385963@N00/310039863/ Image author: Leonard Low, http://www.flickr.com/people/leonardlow/ License: CC-by-A 2.0 http://creativecommons.org/licenses/by/2.0/deed.en
  • Based on: http://wilderdom.com/research/QualitativeVersusQuantitativeResearch.html
  • There are also other qualitative data analysis techniques such as narative analysis and discourse analysis.
  • Discourse analysis
  • Discourse analysis
  • Discourse analysis
  • Discourse analysis
  • Discourse analysis
  • Discourse analysis
  • Discourse analysis
  • Discourse analysis
  • Discourse analysis
  • Discourse analysis
  • Discourse analysis
  • Discourse analysis
  • Discourse analysis
  • Discourse analysis
  • Discourse analysis
  • See also Codebook Development for Team-Based Qualitative Analysis ( http://www.cdc.gov/hiv/software/pubs/codebook.pdf )
  • Image source: James Neill, Creative Commons Attribution-Share Alike 2.5 Australia, http://creativecommons.org/licenses/by-sa/2.5/au/ 14 new variables were created, one for each category. If a case’s response had indicated a quality, it was coded as having a 1 for the corresponding variable. Similar Values was the most commonly chosen characteristic. Percentage of responses - proportion of given response in relation to total responses Percentage of cases refers to the proportion of a given response in relation to no. of valid cases (respondents). For a step-by-step tutorial, see Generating New Variables in SPSS: The Multiple Response Command ( http://ftp2.arts.unsw.edu.au/argyrous/extra_chapters/SPSSMultipleResponseCommand.pdf )
  • Image source: James Neill, Creative Commons Attribution-Share Alike 2.5 Australia, http://creativecommons.org/licenses/by-sa/2.5/au/ Demo multiple response coding
  • Image source: James Neill, Creative Commons Attribution-Share Alike 2.5 Australia, http://creativecommons.org/licenses/by-sa/2.5/au/ Analyse – Tables – Multiple Response Sets or Analyse – Multiple Response – Define Sets
  • Analyse – Multiple Response - Frequencies
  • Image source: James Neill, Creative Commons Attribution-Share Alike 2.5 Australia, http://creativecommons.org/licenses/by-sa/2.5/au/ Output after each item has been given value labels
  • Did a meaningful picture of the phenomenon under study emerge? Describe theoretical concepts, relationships between concepts, and integration of relationships among meanings that emerged from the data in order to yield a meaningful picture of the phenomenon under study. Trustworthiness / rigour of the qualitative findings: What strategies were employed to help ensure rigour and trustworthiness of findings? (and what are the limitations)
  • Image source: http://commons.wikimedia.org/wiki/File:Information_icon4.svg License: Public domain What is qualitative research? Qualitative analysis - How to analyse qualitative data?
  • Image source: http://commons.wikimedia.org/wiki/File:Information_icon4.svg License: Public domain Image source: http://www.flickr.com/photos/aparejador/3184382650/ By aparejador http://www.flickr.com/photos/aparejador/ License: Creative Commons by Attribution 2.0 http://creativecommons.org/licenses/by/2.0/deed.en
  • Qualitative research, lab report overview, and review of lectures 1 to 7

    1. 1. Lecture 7 Survey Research & Design in Psychology James Neill, 2011 Qualitative Research Lab Report Overview Review of Lectures 1 to 7
    2. 2. Overview <ul><li>Qualitative research & analysis </li></ul><ul><li>Lab report overview </li></ul><ul><li>Review of Lectures 1 to 7 </li></ul>
    3. 3. Qualitative research & analysis <ul><li>What is qualitative research? </li><ul><li>Methods
    4. 4. Open-ended questions </li></ul><li>How to analyse qualitative data? </li><ul><li>Qualitatively
    5. 5. Quantitatively </li></ul></ul>
    6. 6. <ul><li>Braun & Clarke (2006). Using thematic analysis in psychology [article]
    7. 7. Neill (2009). Qualitative analysis [online article]
    8. 8. Taylor-Powell & Renner (2003). Analzying qualitative data. [online article] </li></ul>Readings
    9. 9. What is qualitative research? &quot; All research ultimately has a qualitative grounding &quot; - Donald Campbell
    10. 10. What is qualitative research? <ul><li>The primary intent of qualitative research is to “listen” to or learn from the participants/clients themselves about the topic of interest.
    11. 11. Qualitative data can consist of: </li><ul><li>Words (e.g., from interviews),
    12. 12. Pictures (e.g., video), or
    13. 13. Objects (e.g., an artifact) </li></ul></ul>
    14. 14. Nature of qualitative research <ul><li>Aims for a complete, rich, detailed description.
    15. 15. Often recommended during earlier phases of research.
    16. 16. Researcher may only know roughly in advance what he/she is looking for.
    17. 17. Design may emerge as the study unfolds.
    18. 18. Researcher becomes subjectively immersed in measuring and interpreting the subject matter.
    19. 19. Less objective and generalisable than quantitative. </li></ul>
    20. 20. Qualitative data gathering methods <ul><li>Historical
    21. 21. Participant observation
    22. 22. Focus groups
    23. 23. Interviews
    24. 24. Surveys </li></ul>
    25. 25. Survey research: Closed vs. open-ended questions <ul><li>Close-ended </li><ul><li>researcher provides respondent with limited response options. </li></ul><li>Open-ended </li><ul><li>respondents can formulate their own response. </li><ul><li>e.g., “What are the reasons you donate blood?” ____________________________ </li></ul></ul></ul>
    26. 26. <ul><li>Close-ended </li><ul><li>easier to analyse </li></ul><li>Open-ended </li><ul><li>useful in exploratory, pilot or pre-testing stage of research
    27. 27. useful for complex issues
    28. 28. richer data collected
    29. 29. can be less biased than close-ended
    30. 30. useful for instrument development and validation </li></ul></ul>Survey research: Close vs. open-ended questions
    31. 31. Qualitative data analysis &quot;[Qualitative] data analysis is the process of bringing order, structure and meaning to the mass of collected data. It is a messy, ambiguous, time-consuming, creative, and fascinating process. It does not proceed in a linear fashion; it is not neat. Qualitative data analysis is a search for general statements about relationships among categories of data.&quot; Marshall and Rossman (1990, p. 111)
    32. 32. Qualitative data analysis 1. Thematic analysis (treat as qualitative data ) - identify underlying themes - describe themes with illustrative quotes 2. Content analysis (treat as quantitative data ) <ul><ul><li>convert words into data by coding
    33. 33. analyse frequencies and percentages </li></ul></ul>
    34. 34. Thematic analysis: Treat as qualitative data <ul><li>Transcribe data
    35. 35. Read data
    36. 36. Develop themes (patterns)
    37. 37. Code
    38. 38. Analyse & illustrate themes and relationships
    39. 39. Report </li><ul><li>summarise themes in words
    40. 40. use illustrative quotes </li></ul></ul>
    41. 41. Phases of thematic analysis (Braun & Clarke, 2006) <ul><li>Familiarise yourself with your data
    42. 42. Generating initial codes
    43. 43. Searching for themes
    44. 44. Reviewing themes
    45. 45. Defining and naming themes
    46. 46. Producing the report </li></ul>
    47. 47. Generating initial codes (Braun & Clarke, 2006) Figure 1. Data extract, with codes applied (from Clarke et al., 2006) (from Braun & Clarke (2006, p. 88)) Decisions about initial coding should based on the research question being addressed.
    48. 48. Initial thematic map (Braun & Clarke, 2006) Figure 2. Initial thematic map, showing five main themes (final analysis presented in Braun and Wilkinson, 2003) (from Braun & Clarke (2006, p. 90))
    49. 49. Developed thematic map (Braun & Clarke, 2006) Figure 3. Developed thematic map, showing three main themes (final analysis presented in Braun and Wilkinson, 2003) (from Braun & Clarke (2006, p. 90))
    50. 50. Final thematic map (Braun & Clarke, 2006) Figure 4. Final thematic map, showing final two main themes (see Braun and Wilkinson, 2003) (from Braun & Clarke (2006, p. 91))
    51. 51. What makes good thematic analysis? (Braun & Clarke, 2006) <ul><li>Transcription </li><ul><li>The data have been transcribed to an appropriate level of detail, and the transcripts have been checked against the tapes for ‘accuracy’. </li></ul></ul>
    52. 52. What makes good thematic analysis? (Braun & Clarke, 2006) <ul><li>Coding </li></ul><ul><ul><li>Each data item has been given equal attention in the coding process.
    53. 53. Themes have not been generated from a few vivid examples (an anecdotal approach), but instead the coding process has been thorough, inclusive and comprehensive.
    54. 54. All relevant extracts for all each theme have been collated.
    55. 55. Themes have been checked against each other and back to the original data set.
    56. 56. Themes are internally coherent, consistent, and distinctive. </li></ul></ul>
    57. 57. What makes good thematic analysis? (Braun & Clarke, 2006) <ul><li>Analysis </li></ul><ul><ul><li>Data have been analysed / interpreted, made sense of / rather than just paraphrased or described.
    58. 58. Analysis and data match each other / the extracts illustrate the analytic claims.
    59. 59. Analysis tells a convincing and well-organized story about the data and topic.
    60. 60. A good balance between analytic narrative and illustrative extracts is provided. </li></ul></ul>
    61. 61. What makes good thematic analysis? (Braun & Clarke, 2006) <ul><li>Overall </li></ul><ul><ul><li>Enough time has been allocated to complete all phases of the analysis adequately, without rushing a phase or giving it a once-over-lightly. </li></ul></ul>
    62. 62. What makes good thematic analysis? (Braun & Clarke, 2006) <ul><li>Written report </li></ul><ul><ul><li>The assumptions about, and specific approach to, thematic analysis are clearly explicated.
    63. 63. There is a good fit between what you claim you do, and what you show you have done / ie, described method and reported analysis are consistent.
    64. 64. The language and concepts used in the report are consistent with the epistemological position of the analysis.
    65. 65. The researcher is positioned as active in the research process; themes do not just ‘emerge’. to analyse any data </li></ul></ul>
    66. 66. Pitfalls to avoid in thematic analysis (Braun & Clarke, 2006) <ul><li>Failure to analyse any data
    67. 67. Questions used as themes
    68. 68. Weak or unconvincing analysis
    69. 69. Mismatch between data & themes
    70. 70. Mismatch between theory & themes </li></ul>
    71. 71. Disadvantages of thematic analysis <ul><li>Focus – can be too flexibile – analysis needs focus and clear thinking
    72. 72. Tends to be descriptive (limited use for hypothesis testing)
    73. 73. Analyses across cases (not within)
    74. 74. Can be labour intensive
    75. 75. Subject to researcher bias
    76. 76. Can have limited generalisability
    77. 77. Difficult to establish reliability & validity </li></ul>Note : Many of the disadvantages depend more on poorly conducted analyses or inappropriate research questions than the method itself.
    78. 78. Advantages of thematic analysis <ul><li>Relatively easy & quick method to learn, & do. Accessible to researchers with little or no experience of qualitative research.
    79. 79. Results are generally accessible to educated general public.
    80. 80. Useful method for working within participatory research paradigm, with participants as collaborators </li></ul>
    81. 81. Advantages of thematic analysis <ul><li>Can usefully summarize key features of a large body of data, &/or offer a 'thick description' of the data set
    82. 82. Can highlight similarities & differences across the data set.
    83. 83. Can generate unanticipated insights .
    84. 84. Allows for social as well as psychological interpretions of data.
    85. 85. Can be useful for producing qualitative analyses suited to informing policy development . </li></ul>
    86. 86. Content analysis: Treat as quantitative data <ul><li>Coding - convert response into numerical code.
    87. 87. e.g., “What is you occupation?” - code responses into categories such as: </li></ul><ul><ul><li>Professional
    88. 88. Clerical/administrative
    89. 89. Skilled manual
    90. 90. Unskilled manual
    91. 91. Unemployed/other </li></ul></ul>
    92. 92. Content analysis: Coding “ Why do you donate blood?” <ul><li>Could code responses into categories such as: </li></ul><ul><ul><li>Social
    93. 93. Protective
    94. 94. Understanding
    95. 95. Values
    96. 96. Enhancement </li></ul></ul><ul><li>Choice of coding categories may be informed by theoretical literature </li></ul>
    97. 97. Content analysis: Coding <ul><li>Establish objectives for coding frame
    98. 98. Read through all responses
    99. 99. Develop list of common categories
    100. 100. Try not to have too many categories - look to combine similar categories
    101. 101. To increase reliability and validity , at least 2 researchers develop coding separately, then work together on final coding, code separately, compare responses and analyse inter-rater reliability </li></ul>
    102. 102. Content analysis: Inter-rater reliability <ul><li>Inter-rater reliability – percentage of times that raters agree on the category each response goes into. </li></ul><ul><li>Can use a statistic called Cohen’s kappa ( κ ) to assess inter-rater reliability – interpret like a correlation coefficient. </li></ul>
    103. 103. Content analysis: Multiple response analysis For analysis of open-ended questions with multiple responses : <ul><li>e.g., 364 uni students were asked to list up to 4 characteristics they thought were important in a romantic partner.
    104. 104. 14 response themes were identified. 4 new variables were created (resp1, resp2, resp3, resp4) and codes 1 to 14 were used to code each case's responses. </li></ul>
    105. 105. Content analysis: Multiple response analysis
    106. 106. <ul><li>This example involves responses to an open-ended question about why the person goes to university.
    107. 107. Up to two written responses were allowed (resp1 and resp2).
    108. 108. The qualitative responses were coded into four categories: Parents, Friends, Career, Avoid Work. </li></ul>Content analysis: Multiple response analysis
    109. 109. Content analysis: Multiple response analysis
    110. 110. Content analysis: Multiple response analysis
    111. 111. Content analysis: Multiple response analysis
    112. 112. <ul><li>The most commonly cited motivating for going to university was “friends”, cited by a half of all participants (52%) and accounting for over a third of all responses (34%).
    113. 113. The next two most popular reasons were “avoid work” and “parents” cited by 25% and 24% of respondents respectively. “Career” was cited by 16% of respondents. </li></ul>Content analysis: Multiple response analysis
    114. 114. Writing up qualitative analysis: Method <ul><li>Create a decision trail (could someone replicate your methodology?)
    115. 115. Explain rationale for choice of analysis method
    116. 116. Explain development of the thematic categories
    117. 117. Explain coding </li></ul>
    118. 118. Writing up qualitative analysis: Results <ul><li>Thematic maps and/or descriptive statistics (& graphs if appropriate)
    119. 119. Theme names and descriptions, possibly with illustrative quotes
    120. 120. Inferential statistics and effect sizes (if appropriate) </li></ul>
    121. 121. Writing up qualitative analysis: Discussion <ul><li>Did a meaningful picture of the phenomenon under study emerge?
    122. 122. Trustworthiness / rigour of the qualitative findings.
    123. 123. Consistency between qualitative and quantitative findings.
    124. 124. Contributions of findings to theory and practice . </li></ul>
    125. 125. Qualitative data analysis: Summary <ul><li>Open-ended data can be analysed as qualitative (thematic analysis) or converted to quantitative data (content analysis) .
    126. 126. Code then analyse the data , seeking to identify patterns as answers to the research question(s) .
    127. 127. Describe your decision-making to provide a rigorous ‘audit-trail’ </li></ul>
    128. 128. Student questions ?
    129. 129. Lab report guidelines & marking criteria
    130. 130. Lab report guidelines and marking criteria: Key documents <ul><li>The two key links are: </li><ul><li>Lab report guidelines
    131. 131. Lab report marking criteria </li></ul><li>Also see the Lab Report section on Moodle </li></ul>
    132. 132. Mark weighting Introduction ( 10% ) Method ( 15% ) Results ( 45% ) <ul><ul><li>Qualitative (10%)
    133. 133. EFA (15%)
    134. 134. MLR (10%)
    135. 135. Advanced ANOVA (10%) </li></ul></ul>Discussion ( 30% ) Present at least one of each of these 4 types of analyses.
    136. 136. Overall research question(s) Your lab report should be guided by at least one central, clearly stated research question : e.g., To what extent do university students' time management skills account for their degree of satisfaction with university teaching and learning ?
    137. 137. Specific research questions and hypotheses <ul><li>The next step is to identify specific hypotheses or research questions (RQ) : e.g., What do students report as the most dissatisfactory aspects of university? (Qualitative – RQ)
    138. 138. Create at least one hypothesis or research question for each major analysis. </li></ul>
    139. 139. Specific research questions and hypotheses: Examples <ul><li>What are the underlying dimensions of university student satisfaction? (EFA – RQ)
    140. 140. To what extent do self-reported time management skills predict students' satisfaction with university education and teaching? (MLR – RQ) </li></ul>
    141. 141. Specific research questions and hypotheses: Examples It is predicted that: <ul><li>males and females will have similar levels of education and teaching satisfaction
    142. 142. those who are coping with university are more satisfied
    143. 143. there is no interaction between gender and coping. (Factorial ANOVA - Hypothesis) </li></ul>
    144. 144. Specific research questions and hypotheses: Examples <ul><li>What aspects of University of Canberra are its students most satisfied with? (Qualitative - RQ) </li></ul>
    145. 145. Lab report: Sources of data You can use any of the following sources of data for your lab report: <ul><li>Background
    146. 146. Satisfaction – ? factors
    147. 147. Time management - ? factors </li></ul>
    148. 148. Lab report: Exploratory Factor analysis <ul><li>Analyse either satisfaction or time management (or both – but only report fully on one in the Results – briefly summarise the other one)
    149. 149. Include internal consistency, creation composite scores, and correlation between factors. Descriptive stats for factors can be covered later, e.g., in the ANOVA. </li></ul>
    150. 150. Lab report: Multiple linear regression <ul><li>MLR should involve at least three predictors (IVs) with one DV.
    151. 151. Include correlations, R 2 and regression coefficients </li></ul>
    152. 152. Lab report: Advanced ANOVA An advanced ANOVA can be any of: <ul><li>One-way repeated measures ANOVA
    153. 153. Mixed ANOVA
    154. 154. ANCOVA
    155. 155. MANOVA </li></ul>
    156. 156. Lab report: Qualitative analysis <ul><li>Take a qualitative (thematic analysis) or quantitative (content analysis) approach to the analysis of either the set of least or the set of most satisfied responses (or both). </li></ul>
    157. 157. Student questions ?
    158. 158. Review of Lectures 1 to 7
    159. 159. What we've covered <ul><li>Purposes and types of research
    160. 160. Survey design </li><ul><li>Question types
    161. 161. Designing good questions
    162. 162. Response formats
    163. 163. Sampling </li></ul><li>Univariate descriptives & graphing </li><ul><li>For each LOM
    164. 164. Properties of the normal distribution </li></ul></ul>
    165. 165. <ul><li>Correlation </li><ul><li>Types for each LOM
    166. 166. Scatterplots – Estimating correlation & outliers </li></ul><li>Exploratory factor analysis </li><ul><li>Assumptions
    167. 167. Types and purposes
    168. 168. Deciding # of factors and items to keep/drop </li></ul><li>Psychometrics </li><ul><li>Internal reliability
    169. 169. Composite scores
    170. 170. Factor correlations </li></ul></ul>What we've covered
    171. 171. <ul><li>Qualitative </li><ul><li>Qualitative vs. quantitative research
    172. 172. Thematic analysis vs. content analysis </li></ul></ul>What we've covered
    173. 173. <ul><li>Multiple Linear Regression (W9-10)
    174. 174. ANOVA (W10-11)
    175. 175. Effect sizes and power (W12)
    176. 176. Review (W14)
    177. 177. Lab report due (W15) </li></ul>What's next?
    178. 178. References <ul><li>Allen, P. & Bennett, K. (2008). SPSS for the health and behavioural sciences . South Melbourne, Victoria, Australia: Thomson.
    179. 179. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology , 3 (2), 77-101. doi:10.1191/1478088706qp063oa
    180. 180. Braun, V. and Wilkinson, S. 2003: Liability or asset? Women talk about the vagina. Psychology of Women Section Review , 5 , 28-42.
    181. 181. Francis, G. (2007). Introduction to SPSS for Windows: v. 15.0 and 14.0 with Notes for Studentware (5th ed.). Sydney: Pearson Education.
    182. 182. Howell, D. C. (2007). Statistical methods for psychology (6th ed.). Belmont, CA: Wadsworth.
    183. 183. Marshall, C. & Rossman, G. B (1989). Designing qualitative research . Newbury Park, CA: Sage.
    184. 184. Neill, J. T. (2009). Qualitative analysis . (Wilderdom) http://wilderdom.com/courses/surveyresearch/assessment/labreport/QualitativeAnalysis.html
    185. 185. Taylor-Powell, E. & Renner, M. (2003). Analyzing qualitative data . University of Wisconsin-Extension. http://learningstore.uwex.edu/assets/pdfs/G3658-12.PDF </li></ul>
    186. 186. Open Office Impress <ul><li>This presentation was made using Open Office Impress.
    187. 187. Free and open source software.
    188. 188. http://www.openoffice.org/product/impress.html </li></ul>

    ×