Qualitative research, lab report overview, and review of lectures 1 to 7

Lecture 7 Survey Research & Design in Psychology James Neill, 2011 Qualitative Research Lab Report Overview Review of Lectures 1 to 7

Overview
Qualitative research & analysis
Lab report overview
Review of Lectures 1 to 7

Qualitative research & analysis
What is qualitative research?
Methods
Open-ended questions
How to analyse qualitative data?
Qualitatively
Quantitatively

Braun & Clarke (2006). Using thematic analysis in psychology [article]
Neill (2009). Qualitative analysis [online article]
Taylor-Powell & Renner (2003). Analzying qualitative data. [online article]
Readings

What is qualitative research? " All research ultimately has a qualitative grounding " - Donald Campbell

What is qualitative research?
The primary intent of qualitative research is to “listen” to or learn from the participants/clients themselves about the topic of interest.
Qualitative data can consist of:
Words (e.g., from interviews),
Pictures (e.g., video), or
Objects (e.g., an artifact)

Nature of qualitative research
Aims for a complete, rich, detailed description.
Often recommended during earlier phases of research.
Researcher may only know roughly in advance what he/she is looking for.
Design may emerge as the study unfolds.
Researcher becomes subjectively immersed in measuring and interpreting the subject matter.
Less objective and generalisable than quantitative.

Qualitative data gathering methods
Historical
Participant observation
Focus groups
Interviews
Surveys

Survey research: Closed vs. open-ended questions
Close-ended
researcher provides respondent with limited response options.
Open-ended
respondents can formulate their own response.
e.g., “What are the reasons you donate blood?” ____________________________

Close-ended
easier to analyse
Open-ended
useful in exploratory, pilot or pre-testing stage of research
useful for complex issues
richer data collected
can be less biased than close-ended
useful for instrument development and validation
Survey research: Close vs. open-ended questions

Qualitative data analysis "[Qualitative] data analysis is the process of bringing order, structure and meaning to the mass of collected data. It is a messy, ambiguous, time-consuming, creative, and fascinating process. It does not proceed in a linear fashion; it is not neat. Qualitative data analysis is a search for general statements about relationships among categories of data." Marshall and Rossman (1990, p. 111)

Qualitative data analysis 1. Thematic analysis (treat as qualitative data ) - identify underlying themes - describe themes with illustrative quotes 2. Content analysis (treat as quantitative data )
convert words into data by coding
analyse frequencies and percentages

Thematic analysis: Treat as qualitative data
Transcribe data
Read data
Develop themes (patterns)
Code
Analyse & illustrate themes and relationships
Report
summarise themes in words
use illustrative quotes

Phases of thematic analysis (Braun & Clarke, 2006)
Familiarise yourself with your data
Generating initial codes
Searching for themes
Reviewing themes
Defining and naming themes
Producing the report

Generating initial codes (Braun & Clarke, 2006) Figure 1. Data extract, with codes applied (from Clarke et al., 2006) (from Braun & Clarke (2006, p. 88)) Decisions about initial coding should based on the research question being addressed.

Initial thematic map (Braun & Clarke, 2006) Figure 2. Initial thematic map, showing ﬁve main themes (ﬁnal analysis presented in Braun and Wilkinson, 2003) (from Braun & Clarke (2006, p. 90))

Developed thematic map (Braun & Clarke, 2006) Figure 3. Developed thematic map, showing three main themes (ﬁnal analysis presented in Braun and Wilkinson, 2003) (from Braun & Clarke (2006, p. 90))

Final thematic map (Braun & Clarke, 2006) Figure 4. Final thematic map, showing ﬁnal two main themes (see Braun and Wilkinson, 2003) (from Braun & Clarke (2006, p. 91))

What makes good thematic analysis? (Braun & Clarke, 2006)
Transcription
The data have been transcribed to an appropriate level of detail, and the transcripts have been checked against the tapes for ‘accuracy’.

Coding
Each data item has been given equal attention in the coding process.
Themes have not been generated from a few vivid examples (an anecdotal approach), but instead the coding process has been thorough, inclusive and comprehensive.
All relevant extracts for all each theme have been collated.
Themes have been checked against each other and back to the original data set.
Themes are internally coherent, consistent, and distinctive.

Analysis
Data have been analysed / interpreted, made sense of / rather than just paraphrased or described.
Analysis and data match each other / the extracts illustrate the analytic claims.
Analysis tells a convincing and well-organized story about the data and topic.
A good balance between analytic narrative and illustrative extracts is provided.

Overall
Enough time has been allocated to complete all phases of the analysis adequately, without rushing a phase or giving it a once-over-lightly.

Written report
The assumptions about, and specific approach to, thematic analysis are clearly explicated.
There is a good fit between what you claim you do, and what you show you have done / ie, described method and reported analysis are consistent.
The language and concepts used in the report are consistent with the epistemological position of the analysis.
The researcher is positioned as active in the research process; themes do not just ‘emerge’. to analyse any data

Pitfalls to avoid in thematic analysis (Braun & Clarke, 2006)
Failure to analyse any data
Questions used as themes
Weak or unconvincing analysis
Mismatch between data & themes
Mismatch between theory & themes

Disadvantages of thematic analysis
Focus – can be too flexibile – analysis needs focus and clear thinking
Tends to be descriptive (limited use for hypothesis testing)
Analyses across cases (not within)
Can be labour intensive
Subject to researcher bias
Can have limited generalisability
Difficult to establish reliability & validity
Note : Many of the disadvantages depend more on poorly conducted analyses or inappropriate research questions than the method itself.

Advantages of thematic analysis
Relatively easy & quick method to learn, & do. Accessible to researchers with little or no experience of qualitative research.
Results are generally accessible to educated general public.
Useful method for working within participatory research paradigm, with participants as collaborators

Advantages of thematic analysis
Can usefully summarize key features of a large body of data, &/or offer a 'thick description' of the data set
Can highlight similarities & differences across the data set.
Can generate unanticipated insights .
Allows for social as well as psychological interpretions of data.
Can be useful for producing qualitative analyses suited to informing policy development .

Content analysis: Treat as quantitative data
Coding - convert response into numerical code.
e.g., “What is you occupation?” - code responses into categories such as:
Professional
Clerical/administrative
Skilled manual
Unskilled manual
Unemployed/other

Content analysis: Coding “ Why do you donate blood?”
Could code responses into categories such as:
Social
Protective
Understanding
Values
Enhancement
Choice of coding categories may be informed by theoretical literature

Content analysis: Coding
Establish objectives for coding frame
Read through all responses
Develop list of common categories
Try not to have too many categories - look to combine similar categories
To increase reliability and validity , at least 2 researchers develop coding separately, then work together on final coding, code separately, compare responses and analyse inter-rater reliability

Content analysis: Inter-rater reliability
Inter-rater reliability – percentage of times that raters agree on the category each response goes into.
Can use a statistic called Cohen’s kappa ( κ ) to assess inter-rater reliability – interpret like a correlation coefficient.

Content analysis: Multiple response analysis For analysis of open-ended questions with multiple responses :
e.g., 364 uni students were asked to list up to 4 characteristics they thought were important in a romantic partner.
14 response themes were identified. 4 new variables were created (resp1, resp2, resp3, resp4) and codes 1 to 14 were used to code each case's responses.

Content analysis: Multiple response analysis

This example involves responses to an open-ended question about why the person goes to university.
Up to two written responses were allowed (resp1 and resp2).
The qualitative responses were coded into four categories: Parents, Friends, Career, Avoid Work.

The most commonly cited motivating for going to university was “friends”, cited by a half of all participants (52%) and accounting for over a third of all responses (34%).
The next two most popular reasons were “avoid work” and “parents” cited by 25% and 24% of respondents respectively. “Career” was cited by 16% of respondents.

Writing up qualitative analysis: Method
Create a decision trail (could someone replicate your methodology?)
Explain rationale for choice of analysis method
Explain development of the thematic categories
Explain coding

Writing up qualitative analysis: Results
Thematic maps and/or descriptive statistics (& graphs if appropriate)
Theme names and descriptions, possibly with illustrative quotes
Inferential statistics and effect sizes (if appropriate)

Writing up qualitative analysis: Discussion
Did a meaningful picture of the phenomenon under study emerge?
Trustworthiness / rigour of the qualitative findings.
Consistency between qualitative and quantitative findings.
Contributions of findings to theory and practice .

Qualitative data analysis: Summary
Open-ended data can be analysed as qualitative (thematic analysis) or converted to quantitative data (content analysis) .
Code then analyse the data , seeking to identify patterns as answers to the research question(s) .
Describe your decision-making to provide a rigorous ‘audit-trail’

Student questions ?

Lab report guidelines & marking criteria

Lab report guidelines and marking criteria: Key documents
The two key links are:
Lab report guidelines
Lab report marking criteria
Also see the Lab Report section on Moodle

Mark weighting Introduction ( 10% ) Method ( 15% ) Results ( 45% )
Qualitative (10%)
EFA (15%)
MLR (10%)
Advanced ANOVA (10%)
Discussion ( 30% ) Present at least one of each of these 4 types of analyses.

Overall research question(s) Your lab report should be guided by at least one central, clearly stated research question : e.g., To what extent do university students' time management skills account for their degree of satisfaction with university teaching and learning ?

Specific research questions and hypotheses
The next step is to identify specific hypotheses or research questions (RQ) : e.g., What do students report as the most dissatisfactory aspects of university? (Qualitative – RQ)
Create at least one hypothesis or research question for each major analysis.

Specific research questions and hypotheses: Examples
What are the underlying dimensions of university student satisfaction? (EFA – RQ)
To what extent do self-reported time management skills predict students' satisfaction with university education and teaching? (MLR – RQ)

Specific research questions and hypotheses: Examples It is predicted that:
males and females will have similar levels of education and teaching satisfaction
those who are coping with university are more satisfied
there is no interaction between gender and coping. (Factorial ANOVA - Hypothesis)

Specific research questions and hypotheses: Examples
What aspects of University of Canberra are its students most satisfied with? (Qualitative - RQ)

Lab report: Sources of data You can use any of the following sources of data for your lab report:
Background
Satisfaction – ? factors
Time management - ? factors

Lab report: Exploratory Factor analysis
Analyse either satisfaction or time management (or both – but only report fully on one in the Results – briefly summarise the other one)
Include internal consistency, creation composite scores, and correlation between factors. Descriptive stats for factors can be covered later, e.g., in the ANOVA.

Lab report: Multiple linear regression
MLR should involve at least three predictors (IVs) with one DV.
Include correlations, R 2 and regression coefficients

Lab report: Advanced ANOVA An advanced ANOVA can be any of:
One-way repeated measures ANOVA
Mixed ANOVA
ANCOVA
MANOVA

Lab report: Qualitative analysis
Take a qualitative (thematic analysis) or quantitative (content analysis) approach to the analysis of either the set of least or the set of most satisfied responses (or both).

Student questions ?

Review of Lectures 1 to 7

What we've covered
Purposes and types of research
Survey design
Question types
Designing good questions
Response formats
Sampling
Univariate descriptives & graphing
For each LOM
Properties of the normal distribution

Correlation
Types for each LOM
Scatterplots – Estimating correlation & outliers
Exploratory factor analysis
Assumptions
Types and purposes
Deciding # of factors and items to keep/drop
Psychometrics
Internal reliability
Composite scores
Factor correlations
What we've covered

Qualitative
Qualitative vs. quantitative research
Thematic analysis vs. content analysis
What we've covered

Multiple Linear Regression (W9-10)
ANOVA (W10-11)
Effect sizes and power (W12)
Review (W14)
Lab report due (W15)
What's next?

References
Allen, P. & Bennett, K. (2008). SPSS for the health and behavioural sciences . South Melbourne, Victoria, Australia: Thomson.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology , 3 (2), 77-101. doi:10.1191/1478088706qp063oa
Braun, V. and Wilkinson, S. 2003: Liability or asset? Women talk about the vagina. Psychology of Women Section Review , 5 , 28-42.
Francis, G. (2007). Introduction to SPSS for Windows: v. 15.0 and 14.0 with Notes for Studentware (5th ed.). Sydney: Pearson Education.
Howell, D. C. (2007). Statistical methods for psychology (6th ed.). Belmont, CA: Wadsworth.
Marshall, C. & Rossman, G. B (1989). Designing qualitative research . Newbury Park, CA: Sage.
Neill, J. T. (2009). Qualitative analysis . (Wilderdom) http://wilderdom.com/courses/surveyresearch/assessment/labreport/QualitativeAnalysis.html
Taylor-Powell, E. & Renner, M. (2003). Analyzing qualitative data . University of Wisconsin-Extension. http://learningstore.uwex.edu/assets/pdfs/G3658-12.PDF

Open Office Impress
This presentation was made using Open Office Impress.
Free and open source software.
http://www.openoffice.org/product/impress.html

http://ucspace.canberra.edu.au	492
http://www.slideshare.net	63
http://elearning.rmutl.ac.th	4
https://learn.uark.edu	1
http://translate.googleusercontent.com	1

Qualitative research, lab report overview, and review of lectures 1 to 7

by James Neill

Accessibility

Categories

Upload Details

Usage Rights

5 Embeds 561

Statistics

Qualitative research, lab report overview, and review of lectures 1 to 7 Presentation Transcript