Successfully reported this slideshow.
Lecture 7 Survey Research & Design in Psychology James Neill,  2011 Qualitative Research  Lab Report Overview Review of Le...
Overview <ul><li>Qualitative research & analysis </li></ul><ul><li>Lab report overview </li></ul><ul><li>Review of Lecture...
Qualitative  research & analysis <ul><li>What is qualitative research? </li><ul><li>Methods
Open-ended questions </li></ul><li>How to analyse qualitative data? </li><ul><li>Qualitatively
Quantitatively </li></ul></ul>
<ul><li>Braun & Clarke (2006).  Using thematic analysis in psychology [article]
Neill (2009).  Qualitative analysis [online article]
Taylor-Powell & Renner (2003).  Analzying qualitative data. [online article] </li></ul>Readings
What is qualitative research? &quot; All research ultimately has  a qualitative grounding &quot; - Donald Campbell
What is qualitative research? <ul><li>The primary intent of  qualitative research  is to “listen” to or learn from the par...
Qualitative data  can consist of: </li><ul><li>Words (e.g., from interviews),
Pictures (e.g., video), or
Objects (e.g., an artifact) </li></ul></ul>
Nature of qualitative research <ul><li>Aims for a complete, rich, detailed description.
Often recommended during earlier phases of research.
Researcher may only know roughly in advance what he/she is looking for.
Design may emerge as the study unfolds.
Researcher becomes subjectively immersed in measuring and interpreting the subject matter.
Less objective and generalisable than quantitative. </li></ul>
Qualitative data gathering methods <ul><li>Historical
Participant observation
Focus groups
Interviews
Surveys </li></ul>
Survey research:  Closed vs. open-ended questions <ul><li>Close-ended </li><ul><li>researcher provides respondent with lim...
<ul><li>Close-ended </li><ul><li>easier to analyse </li></ul><li>Open-ended </li><ul><li>useful in exploratory, pilot or p...
useful for complex issues
richer data collected
can be less biased than close-ended
useful for instrument development and validation </li></ul></ul>Survey research: Close vs. open-ended questions
Qualitative data analysis &quot;[Qualitative] data analysis is the process of bringing order, structure and meaning to the...
Qualitative data analysis 1.  Thematic analysis   (treat as  qualitative data ) - identify underlying themes - describe th...
analyse frequencies and percentages </li></ul></ul>
Thematic analysis: Treat as qualitative data <ul><li>Transcribe data
Read data
Develop themes (patterns)
Code
Analyse & illustrate themes and relationships
Report </li><ul><li>summarise themes in words
use illustrative quotes </li></ul></ul>
Phases of thematic analysis (Braun & Clarke, 2006) <ul><li>Familiarise  yourself with your data
Generating  initial codes
Searching  for themes
Reviewing  themes
Defining  and naming themes
Producing  the report </li></ul>
Generating initial codes (Braun & Clarke, 2006) Figure 1. Data extract, with codes applied (from Clarke et al., 2006) (fro...
Initial thematic map (Braun & Clarke, 2006) Figure 2. Initial thematic map, showing five main themes (final analysis present...
Developed thematic map (Braun & Clarke, 2006) Figure 3. Developed thematic map, showing three main themes (final analysis p...
Final thematic map (Braun & Clarke, 2006) Figure 4. Final thematic map, showing final two main themes (see Braun and Wilkin...
What makes good thematic analysis? (Braun & Clarke, 2006) <ul><li>Transcription </li><ul><li>The data have been transcribe...
What makes good thematic analysis? (Braun & Clarke, 2006) <ul><li>Coding </li></ul><ul><ul><li>Each data item has been giv...
Themes have not been generated from a few vivid examples (an anecdotal approach), but instead the coding process has been ...
All relevant extracts for all each theme have been collated.
Themes have been checked against each other and back to the original data set.
Themes are internally coherent, consistent, and distinctive. </li></ul></ul>
What makes good thematic analysis? (Braun & Clarke, 2006) <ul><li>Analysis </li></ul><ul><ul><li>Data have been analysed /...
Analysis and data match each other / the extracts illustrate the analytic claims.
Analysis tells a convincing and well-organized story about the data and topic.
A good balance between analytic narrative and illustrative extracts is provided. </li></ul></ul>
What makes good thematic analysis? (Braun & Clarke, 2006) <ul><li>Overall </li></ul><ul><ul><li>Enough time has been alloc...
What makes good thematic analysis? (Braun & Clarke, 2006) <ul><li>Written report </li></ul><ul><ul><li>The assumptions abo...
There is a good fit between what you claim you do, and what you show you have done / ie, described method and reported ana...
The language and concepts used in the report are consistent with the epistemological position of the analysis.
Upcoming SlideShare
Loading in …5
×

Qualitative research, lab report overview, and review of lectures 1 to 7

18,342 views

Published on

This lecture introduces qualitative research and qualitative analysis, overviews the lab report tasks, and summarises Lectures 1 to 7. See also http://ucspace.canberra.edu.au/pages/viewpage.action?pageId=57409703

Published in: Education, Technology, Business

Qualitative research, lab report overview, and review of lectures 1 to 7

  1. 1. Lecture 7 Survey Research & Design in Psychology James Neill, 2011 Qualitative Research Lab Report Overview Review of Lectures 1 to 7
  2. 2. Overview <ul><li>Qualitative research & analysis </li></ul><ul><li>Lab report overview </li></ul><ul><li>Review of Lectures 1 to 7 </li></ul>
  3. 3. Qualitative research & analysis <ul><li>What is qualitative research? </li><ul><li>Methods
  4. 4. Open-ended questions </li></ul><li>How to analyse qualitative data? </li><ul><li>Qualitatively
  5. 5. Quantitatively </li></ul></ul>
  6. 6. <ul><li>Braun & Clarke (2006). Using thematic analysis in psychology [article]
  7. 7. Neill (2009). Qualitative analysis [online article]
  8. 8. Taylor-Powell & Renner (2003). Analzying qualitative data. [online article] </li></ul>Readings
  9. 9. What is qualitative research? &quot; All research ultimately has a qualitative grounding &quot; - Donald Campbell
  10. 10. What is qualitative research? <ul><li>The primary intent of qualitative research is to “listen” to or learn from the participants/clients themselves about the topic of interest.
  11. 11. Qualitative data can consist of: </li><ul><li>Words (e.g., from interviews),
  12. 12. Pictures (e.g., video), or
  13. 13. Objects (e.g., an artifact) </li></ul></ul>
  14. 14. Nature of qualitative research <ul><li>Aims for a complete, rich, detailed description.
  15. 15. Often recommended during earlier phases of research.
  16. 16. Researcher may only know roughly in advance what he/she is looking for.
  17. 17. Design may emerge as the study unfolds.
  18. 18. Researcher becomes subjectively immersed in measuring and interpreting the subject matter.
  19. 19. Less objective and generalisable than quantitative. </li></ul>
  20. 20. Qualitative data gathering methods <ul><li>Historical
  21. 21. Participant observation
  22. 22. Focus groups
  23. 23. Interviews
  24. 24. Surveys </li></ul>
  25. 25. Survey research: Closed vs. open-ended questions <ul><li>Close-ended </li><ul><li>researcher provides respondent with limited response options. </li></ul><li>Open-ended </li><ul><li>respondents can formulate their own response. </li><ul><li>e.g., “What are the reasons you donate blood?” ____________________________ </li></ul></ul></ul>
  26. 26. <ul><li>Close-ended </li><ul><li>easier to analyse </li></ul><li>Open-ended </li><ul><li>useful in exploratory, pilot or pre-testing stage of research
  27. 27. useful for complex issues
  28. 28. richer data collected
  29. 29. can be less biased than close-ended
  30. 30. useful for instrument development and validation </li></ul></ul>Survey research: Close vs. open-ended questions
  31. 31. Qualitative data analysis &quot;[Qualitative] data analysis is the process of bringing order, structure and meaning to the mass of collected data. It is a messy, ambiguous, time-consuming, creative, and fascinating process. It does not proceed in a linear fashion; it is not neat. Qualitative data analysis is a search for general statements about relationships among categories of data.&quot; Marshall and Rossman (1990, p. 111)
  32. 32. Qualitative data analysis 1. Thematic analysis (treat as qualitative data ) - identify underlying themes - describe themes with illustrative quotes 2. Content analysis (treat as quantitative data ) <ul><ul><li>convert words into data by coding
  33. 33. analyse frequencies and percentages </li></ul></ul>
  34. 34. Thematic analysis: Treat as qualitative data <ul><li>Transcribe data
  35. 35. Read data
  36. 36. Develop themes (patterns)
  37. 37. Code
  38. 38. Analyse & illustrate themes and relationships
  39. 39. Report </li><ul><li>summarise themes in words
  40. 40. use illustrative quotes </li></ul></ul>
  41. 41. Phases of thematic analysis (Braun & Clarke, 2006) <ul><li>Familiarise yourself with your data
  42. 42. Generating initial codes
  43. 43. Searching for themes
  44. 44. Reviewing themes
  45. 45. Defining and naming themes
  46. 46. Producing the report </li></ul>
  47. 47. Generating initial codes (Braun & Clarke, 2006) Figure 1. Data extract, with codes applied (from Clarke et al., 2006) (from Braun & Clarke (2006, p. 88)) Decisions about initial coding should based on the research question being addressed.
  48. 48. Initial thematic map (Braun & Clarke, 2006) Figure 2. Initial thematic map, showing five main themes (final analysis presented in Braun and Wilkinson, 2003) (from Braun & Clarke (2006, p. 90))
  49. 49. Developed thematic map (Braun & Clarke, 2006) Figure 3. Developed thematic map, showing three main themes (final analysis presented in Braun and Wilkinson, 2003) (from Braun & Clarke (2006, p. 90))
  50. 50. Final thematic map (Braun & Clarke, 2006) Figure 4. Final thematic map, showing final two main themes (see Braun and Wilkinson, 2003) (from Braun & Clarke (2006, p. 91))
  51. 51. What makes good thematic analysis? (Braun & Clarke, 2006) <ul><li>Transcription </li><ul><li>The data have been transcribed to an appropriate level of detail, and the transcripts have been checked against the tapes for ‘accuracy’. </li></ul></ul>
  52. 52. What makes good thematic analysis? (Braun & Clarke, 2006) <ul><li>Coding </li></ul><ul><ul><li>Each data item has been given equal attention in the coding process.
  53. 53. Themes have not been generated from a few vivid examples (an anecdotal approach), but instead the coding process has been thorough, inclusive and comprehensive.
  54. 54. All relevant extracts for all each theme have been collated.
  55. 55. Themes have been checked against each other and back to the original data set.
  56. 56. Themes are internally coherent, consistent, and distinctive. </li></ul></ul>
  57. 57. What makes good thematic analysis? (Braun & Clarke, 2006) <ul><li>Analysis </li></ul><ul><ul><li>Data have been analysed / interpreted, made sense of / rather than just paraphrased or described.
  58. 58. Analysis and data match each other / the extracts illustrate the analytic claims.
  59. 59. Analysis tells a convincing and well-organized story about the data and topic.
  60. 60. A good balance between analytic narrative and illustrative extracts is provided. </li></ul></ul>
  61. 61. What makes good thematic analysis? (Braun & Clarke, 2006) <ul><li>Overall </li></ul><ul><ul><li>Enough time has been allocated to complete all phases of the analysis adequately, without rushing a phase or giving it a once-over-lightly. </li></ul></ul>
  62. 62. What makes good thematic analysis? (Braun & Clarke, 2006) <ul><li>Written report </li></ul><ul><ul><li>The assumptions about, and specific approach to, thematic analysis are clearly explicated.
  63. 63. There is a good fit between what you claim you do, and what you show you have done / ie, described method and reported analysis are consistent.
  64. 64. The language and concepts used in the report are consistent with the epistemological position of the analysis.
  65. 65. The researcher is positioned as active in the research process; themes do not just ‘emerge’. to analyse any data </li></ul></ul>
  66. 66. Pitfalls to avoid in thematic analysis (Braun & Clarke, 2006) <ul><li>Failure to analyse any data
  67. 67. Questions used as themes
  68. 68. Weak or unconvincing analysis
  69. 69. Mismatch between data & themes
  70. 70. Mismatch between theory & themes </li></ul>
  71. 71. Disadvantages of thematic analysis <ul><li>Focus – can be too flexibile – analysis needs focus and clear thinking
  72. 72. Tends to be descriptive (limited use for hypothesis testing)
  73. 73. Analyses across cases (not within)
  74. 74. Can be labour intensive
  75. 75. Subject to researcher bias
  76. 76. Can have limited generalisability
  77. 77. Difficult to establish reliability & validity </li></ul>Note : Many of the disadvantages depend more on poorly conducted analyses or inappropriate research questions than the method itself.
  78. 78. Advantages of thematic analysis <ul><li>Relatively easy & quick method to learn, & do. Accessible to researchers with little or no experience of qualitative research.
  79. 79. Results are generally accessible to educated general public.
  80. 80. Useful method for working within participatory research paradigm, with participants as collaborators </li></ul>
  81. 81. Advantages of thematic analysis <ul><li>Can usefully summarize key features of a large body of data, &/or offer a 'thick description' of the data set
  82. 82. Can highlight similarities & differences across the data set.
  83. 83. Can generate unanticipated insights .
  84. 84. Allows for social as well as psychological interpretions of data.
  85. 85. Can be useful for producing qualitative analyses suited to informing policy development . </li></ul>
  86. 86. Content analysis: Treat as quantitative data <ul><li>Coding - convert response into numerical code.
  87. 87. e.g., “What is you occupation?” - code responses into categories such as: </li></ul><ul><ul><li>Professional
  88. 88. Clerical/administrative
  89. 89. Skilled manual
  90. 90. Unskilled manual
  91. 91. Unemployed/other </li></ul></ul>
  92. 92. Content analysis: Coding “ Why do you donate blood?” <ul><li>Could code responses into categories such as: </li></ul><ul><ul><li>Social
  93. 93. Protective
  94. 94. Understanding
  95. 95. Values
  96. 96. Enhancement </li></ul></ul><ul><li>Choice of coding categories may be informed by theoretical literature </li></ul>
  97. 97. Content analysis: Coding <ul><li>Establish objectives for coding frame
  98. 98. Read through all responses
  99. 99. Develop list of common categories
  100. 100. Try not to have too many categories - look to combine similar categories
  101. 101. To increase reliability and validity , at least 2 researchers develop coding separately, then work together on final coding, code separately, compare responses and analyse inter-rater reliability </li></ul>
  102. 102. Content analysis: Inter-rater reliability <ul><li>Inter-rater reliability – percentage of times that raters agree on the category each response goes into. </li></ul><ul><li>Can use a statistic called Cohen’s kappa ( κ ) to assess inter-rater reliability – interpret like a correlation coefficient. </li></ul>
  103. 103. Content analysis: Multiple response analysis For analysis of open-ended questions with multiple responses : <ul><li>e.g., 364 uni students were asked to list up to 4 characteristics they thought were important in a romantic partner.
  104. 104. 14 response themes were identified. 4 new variables were created (resp1, resp2, resp3, resp4) and codes 1 to 14 were used to code each case's responses. </li></ul>
  105. 105. Content analysis: Multiple response analysis
  106. 106. <ul><li>This example involves responses to an open-ended question about why the person goes to university.
  107. 107. Up to two written responses were allowed (resp1 and resp2).
  108. 108. The qualitative responses were coded into four categories: Parents, Friends, Career, Avoid Work. </li></ul>Content analysis: Multiple response analysis
  109. 109. Content analysis: Multiple response analysis
  110. 110. Content analysis: Multiple response analysis
  111. 111. Content analysis: Multiple response analysis
  112. 112. <ul><li>The most commonly cited motivating for going to university was “friends”, cited by a half of all participants (52%) and accounting for over a third of all responses (34%).
  113. 113. The next two most popular reasons were “avoid work” and “parents” cited by 25% and 24% of respondents respectively. “Career” was cited by 16% of respondents. </li></ul>Content analysis: Multiple response analysis
  114. 114. Writing up qualitative analysis: Method <ul><li>Create a decision trail (could someone replicate your methodology?)
  115. 115. Explain rationale for choice of analysis method
  116. 116. Explain development of the thematic categories
  117. 117. Explain coding </li></ul>
  118. 118. Writing up qualitative analysis: Results <ul><li>Thematic maps and/or descriptive statistics (& graphs if appropriate)
  119. 119. Theme names and descriptions, possibly with illustrative quotes
  120. 120. Inferential statistics and effect sizes (if appropriate) </li></ul>
  121. 121. Writing up qualitative analysis: Discussion <ul><li>Did a meaningful picture of the phenomenon under study emerge?
  122. 122. Trustworthiness / rigour of the qualitative findings.
  123. 123. Consistency between qualitative and quantitative findings.
  124. 124. Contributions of findings to theory and practice . </li></ul>
  125. 125. Qualitative data analysis: Summary <ul><li>Open-ended data can be analysed as qualitative (thematic analysis) or converted to quantitative data (content analysis) .
  126. 126. Code then analyse the data , seeking to identify patterns as answers to the research question(s) .
  127. 127. Describe your decision-making to provide a rigorous ‘audit-trail’ </li></ul>
  128. 128. Student questions ?
  129. 129. Lab report guidelines & marking criteria
  130. 130. Lab report guidelines and marking criteria: Key documents <ul><li>The two key links are: </li><ul><li>Lab report guidelines
  131. 131. Lab report marking criteria </li></ul><li>Also see the Lab Report section on Moodle </li></ul>
  132. 132. Mark weighting Introduction ( 10% ) Method ( 15% ) Results ( 45% ) <ul><ul><li>Qualitative (10%)
  133. 133. EFA (15%)
  134. 134. MLR (10%)
  135. 135. Advanced ANOVA (10%) </li></ul></ul>Discussion ( 30% ) Present at least one of each of these 4 types of analyses.
  136. 136. Overall research question(s) Your lab report should be guided by at least one central, clearly stated research question : e.g., To what extent do university students' time management skills account for their degree of satisfaction with university teaching and learning ?
  137. 137. Specific research questions and hypotheses <ul><li>The next step is to identify specific hypotheses or research questions (RQ) : e.g., What do students report as the most dissatisfactory aspects of university? (Qualitative – RQ)
  138. 138. Create at least one hypothesis or research question for each major analysis. </li></ul>
  139. 139. Specific research questions and hypotheses: Examples <ul><li>What are the underlying dimensions of university student satisfaction? (EFA – RQ)
  140. 140. To what extent do self-reported time management skills predict students' satisfaction with university education and teaching? (MLR – RQ) </li></ul>
  141. 141. Specific research questions and hypotheses: Examples It is predicted that: <ul><li>males and females will have similar levels of education and teaching satisfaction
  142. 142. those who are coping with university are more satisfied
  143. 143. there is no interaction between gender and coping. (Factorial ANOVA - Hypothesis) </li></ul>
  144. 144. Specific research questions and hypotheses: Examples <ul><li>What aspects of University of Canberra are its students most satisfied with? (Qualitative - RQ) </li></ul>
  145. 145. Lab report: Sources of data You can use any of the following sources of data for your lab report: <ul><li>Background
  146. 146. Satisfaction – ? factors
  147. 147. Time management - ? factors </li></ul>
  148. 148. Lab report: Exploratory Factor analysis <ul><li>Analyse either satisfaction or time management (or both – but only report fully on one in the Results – briefly summarise the other one)
  149. 149. Include internal consistency, creation composite scores, and correlation between factors. Descriptive stats for factors can be covered later, e.g., in the ANOVA. </li></ul>
  150. 150. Lab report: Multiple linear regression <ul><li>MLR should involve at least three predictors (IVs) with one DV.
  151. 151. Include correlations, R 2 and regression coefficients </li></ul>
  152. 152. Lab report: Advanced ANOVA An advanced ANOVA can be any of: <ul><li>One-way repeated measures ANOVA
  153. 153. Mixed ANOVA
  154. 154. ANCOVA
  155. 155. MANOVA </li></ul>
  156. 156. Lab report: Qualitative analysis <ul><li>Take a qualitative (thematic analysis) or quantitative (content analysis) approach to the analysis of either the set of least or the set of most satisfied responses (or both). </li></ul>
  157. 157. Student questions ?
  158. 158. Review of Lectures 1 to 7
  159. 159. What we've covered <ul><li>Purposes and types of research
  160. 160. Survey design </li><ul><li>Question types
  161. 161. Designing good questions
  162. 162. Response formats
  163. 163. Sampling </li></ul><li>Univariate descriptives & graphing </li><ul><li>For each LOM
  164. 164. Properties of the normal distribution </li></ul></ul>
  165. 165. <ul><li>Correlation </li><ul><li>Types for each LOM
  166. 166. Scatterplots – Estimating correlation & outliers </li></ul><li>Exploratory factor analysis </li><ul><li>Assumptions
  167. 167. Types and purposes
  168. 168. Deciding # of factors and items to keep/drop </li></ul><li>Psychometrics </li><ul><li>Internal reliability
  169. 169. Composite scores
  170. 170. Factor correlations </li></ul></ul>What we've covered
  171. 171. <ul><li>Qualitative </li><ul><li>Qualitative vs. quantitative research
  172. 172. Thematic analysis vs. content analysis </li></ul></ul>What we've covered
  173. 173. <ul><li>Multiple Linear Regression (W9-10)
  174. 174. ANOVA (W10-11)
  175. 175. Effect sizes and power (W12)
  176. 176. Review (W14)
  177. 177. Lab report due (W15) </li></ul>What's next?
  178. 178. References <ul><li>Allen, P. & Bennett, K. (2008). SPSS for the health and behavioural sciences . South Melbourne, Victoria, Australia: Thomson.
  179. 179. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology , 3 (2), 77-101. doi:10.1191/1478088706qp063oa
  180. 180. Braun, V. and Wilkinson, S. 2003: Liability or asset? Women talk about the vagina. Psychology of Women Section Review , 5 , 28-42.
  181. 181. Francis, G. (2007). Introduction to SPSS for Windows: v. 15.0 and 14.0 with Notes for Studentware (5th ed.). Sydney: Pearson Education.
  182. 182. Howell, D. C. (2007). Statistical methods for psychology (6th ed.). Belmont, CA: Wadsworth.
  183. 183. Marshall, C. & Rossman, G. B (1989). Designing qualitative research . Newbury Park, CA: Sage.
  184. 184. Neill, J. T. (2009). Qualitative analysis . (Wilderdom) http://wilderdom.com/courses/surveyresearch/assessment/labreport/QualitativeAnalysis.html
  185. 185. Taylor-Powell, E. & Renner, M. (2003). Analyzing qualitative data . University of Wisconsin-Extension. http://learningstore.uwex.edu/assets/pdfs/G3658-12.PDF </li></ul>
  186. 186. Open Office Impress <ul><li>This presentation was made using Open Office Impress.
  187. 187. Free and open source software.
  188. 188. http://www.openoffice.org/product/impress.html </li></ul>

×