Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
RTI International

Intro to Focus Groups
Emily Geisen
Amanda Wilmot
10/30/2013

RTI International is a trade name of Resea...
RTI International

Ground Rules





Cell phones turned to vibrate or off
Refreshments and restrooms any time
Several ...
RTI International

Course Outline
The role and use of focus groups
2. Focus group demonstration
3. Developing the protocol...
RTI International

1. The Role and Use of Focus Groups

RTI International is a trade name of Research Triangle Institute.
...
RTI International

What is a Focus Group?

“Focus group interviews typically have five characteristics
or features: (a) pe...
RTI International

Examples




A group of coal miners sit around a small conference
room discussing mine safety. They a...
RTI International

What is a Focus Group, again?


Researchers have a topic they want to know about:
–
–



These topics...
RTI International

Uses of Focus Groups


Exploration and Discovery
–
–
–



Context and Depth
–
–
–



Collect info th...
RTI International

Uses of Focus Groups*, Continued


Stand-alone data collection method, or
 Supplement other qualitati...
RTI International

Focus Groups for Survey Research


Identify topics/themes that are of interest to population
–
–



G...
RTI International

Strengths of Focus Groups (over other
qual./quan. methods)


If you‟re unfamiliar with a topic, you ca...
RTI International

Limitations of Focus Groups


Some topics are too sensitive or controversial to be
discussed in a grou...
RTI International

Thinking about your research…


How are you going to use focus groups?
–
–
–

Exploration and discover...
RTI International

Questions/Discussion
RTI International

2. Focus Group Demonstration

RTI International is a trade name of Research Triangle Institute.

www.rt...
RTI International

3. Developing the Protocol Guide

RTI International is a trade name of Research Triangle Institute.

ww...
RTI International

The What, Why, Who, How, Where
What Review research objectives
Who Determine number and type of partici...
RTI International

The What, Why, Who, How, Where
What Review research objectives
Who Determine number and type of partici...
RTI International

What’s the Focus Group About: Review
Research Objectives






Start with the overall research obj...
RTI International

Review Research Objectives: Example
Step

Personal Health Records Example

Study goals:

Learn about he...
RTI International

Review Research Objectives: Your Turn
Step
Study goals:

What do you already know
about the topic:

Wha...
RTI International

Why: Identify Major Discussion Topics



Once you identify what you want to learn, identify major
dis...
RTI International

Identify Major Discussion Topics: Example


Why aren‟t more consumers using electronic PHRs to
manage ...
RTI International

How: Scripting Questions


Avoid yes/no or short-answer questions, unless:





Open-ended question...
RTI International

Scripting Questions: Example 1
Topic: How do consumers currently manage their health
information?
 Bad...
RTI International

Scripting Questions: Example 2


Bad: Why do you use the mobile device you
have?



Better: What kind...
RTI International

Other aspects of Protocol Guide




Welcome/Introduction
Icebreaker
Opening (or Engagement) question...
RTI International

Welcome/Introduction


Welcome
–
–



Informed consent (if required)
–
–
–



Introduce yourself, no...
RTI International

Icebreaker




Sometimes it‟s helpful to start with an ice-breaker
Ask participant to provide their ...
RTI International

Intro Questions


Introduce Topic
–
–



Provide any needed context
–
–
–



Goal/Purpose of the stu...
RTI International

Pre/Post Questionnaires


Pre questionnaire:
–
–



Post questionnaire:
–
–



Use to collect inform...
RTI International

Putting it all together


Allow time for welcome, explaining the purpose of the
study, and consent pro...
RTI International

Determining Question Flow
What order would you put these questions in?
1. Describe the benefits of usin...
RTI International

Question Flow
2. How did you decide what kind of mobile device to use at
work?
1. Describe the benefits...
RTI International

Protocol Guide Examples: Handouts





Handout 1: Mine Contractors Protocol Guide
Handout 2: Health...
RTI International

Individual Practice (15 minutes)
Use your study or class example
 Identify 1-2 major discussion topics...
RTI International

Questions/Discussion
RTI International

4. Moderating Focus Groups

RTI International is a trade name of Research Triangle Institute.

www.rti....
RTI International

Understanding the group Process

Forming
Testing and dependence
Dependence on the leader

Storming
Intr...
RTI International

The Ideal Moderator:


Has adequate knowledge of topic
 Has good communication skills
 Is similar to...
RTI International

What the Moderator Does




Welcome/Introductions
Leads discussion
Concludes discussion
RTI International

Welcome/Introduction


Make participants feel comfortable
 Read or paraphrase this welcome/introducti...
RTI International

Leads Discussion


Provides unobtrusive control
 Manages group dynamics
 Asks spontaneous probes as ...
RTI International

Provide Unobtrusive Control


Know when to stop talking
–



Learn to focus the discussion
–
–



Yo...
RTI International

Tips for Managing Group Dynamics


Dominant talkers:
–
–



Ramblers/Off-topic:
–
–



Don‟t look at...
RTI International

Tips on Spontaneous Probing


Leading probes:
–
–
–



So you had a hard time with that then?
You did...
RTI International

When you hear yourself asking a leading
question, balance it
“So you think
that‟s difficult
then?”

Lea...
RTI International

Tips on Neutral Feedback






Monitor your reaction to participants/discussion
Be careful of nonv...
RTI International

Summarize/Verify Information


As needed, summarize/paraphrase what has been said
–
–



See if anyth...
RTI International

Conclusion / Wrap-up



Be sure to leave time for wrap-up, especially for
sensitive topics or intense...
RTI International

Dealing with Difficult Situations






Too many people show up
Participants bring their children
...
RTI International

Plan Your Response…


20 minutes into the focus group, and Sarah has not said
anything…



You can‟t ...
RTI International

Role of Note-Taker










Can Handle logistics & refreshments
Collects signed informed cons...
RTI International

Role of Co-Moderator


Not required, but can be useful in some situations
 Balance out strengths/weak...
RTI International

Moderating style


Different styles for different people
–
–



Spontaneous flow
Follow guide strictl...
RTI International

Group Practice Moderating (30 min)

RTI International is a trade name of Research Triangle Institute.

...
RTI International

Questions/Discussion
RTI International

5. Recruiting and Scheduling
(review on your own)

RTI International is a trade name of Research Triang...
RTI International

At Least a Month Before Focus Group



Decide on what you‟re testing
Decide on the participants (see ...
RTI International

How to Find Your Participants



Frame/list if available
Advertisements: Craigslist, flyers, newspape...
RTI International

What to Pay Participants?


Are participants allowed to accept monetary incentives?
–
–





Federa...
RTI International

Recruiting Tips


Recruit extras due to no-shows or cancellations
–



Schedule sessions about 3-4 we...
RTI International

Testing Location



At a minimum, use a large conference room (at your
organization‟s offices, hotel ...
RTI International

Observation Room Example
RTI International

Create a Virtual Observation Room using
screen-sharing software


Examples: Skype, Go to Meeting



F...
RTI International

Note-Taking






Recommend having a note-taker and audio or videorecording the session
Note-taker...
RTI International

At Least Three Weeks Before Focus Group




Get started on recruitment
Organize your incentives
Deve...
RTI International

One Week Before the Focus Group



Final your focus group guide
Organize roles in the test:
–
–
–
–

...
RTI International

Prepare your materials





Develop consent forms, screeners
Instructions/directions for participan...
RTI International

The Day Before / Morning Of


Send out reminders:
–
–



Phone or email to respondents
Email to obser...
RTI International

6. Participant Selection

RTI International is a trade name of Research Triangle Institute.

www.rti.or...
RTI International

Purpose of the session


To understand the principles of participant selection for
focus group researc...
RTI International

Focus groups: a qualitative data collection method





Focus groups are a qualitative data collecti...
RTI International

Sampling for qualitative research



The design of a sampling strategy for qualitative
research is as ...
RTI International

Informing the design of a qualitative sampling strategy














What are the researc...
RTI International

Sampling technique
Quantitative sampling
 Probability sampling



Members of the research population...
RTI International

Sample criteria


What characteristics will need to be reflected in the
sample population to ensure br...
RTI International

Sample size


Small sample sizes for qualitative research
There is no need for scale because there is ...
RTI International

Sampling frames
A sampling frame is a list that identifies units within the
target population
Frame eva...
RTI International

Sampling frames


Existing frames
– Administrative sources
– Survey samples



Constructed frames
– D...
RTI International

Constructed frames


Focussed enumeration
 Snowballing
 Screening questionnaires
 Organisations
 A...
RTI International

Study about attitudes and behaviors surrounding
dental attendance
Selection criteria initially consider...
RTI International

Prioritised selection criteria

Primary criteria
Dental attendance pattern
Age
Gender
Region
Family Uni...
RTI International

Sample Matrix
Area 1 of 6

Group 1
18-29

Group 2
30-44

Group 3
45+

4 men
1-2 regular attenders
1-2 i...
RTI International

Any Questions?
RTI International

Excercise





Groups
Designing a sampling strategy for focus groups on the
use of electronic perso...
RTI International

7. Data Management and analysis

RTI International is a trade name of Research Triangle Institute.

www...
RTI International

Purpose of the session


To understand the principles of good qualitative focus
group analysis



To ...
RTI International

Acknowledgement


Some of the slides contributed by the UK National
Centre for Social Research (NatCen...
RTI International

Qualitative data


Focus group recording



Focus group transcription



Notes



Admin data



Ph...
RTI International

Transcription


Transcription is time consuming
 The focus group transcription should capture the
dis...
RTI International

Different approaches to data management, analysis and
reporting



Summary report



Qualitative anal...
RTI International

Nature of qualitative analysis


Quantitative research

Qualitative research

–

- Creating categories...
RTI International

Aim of analysis - analytical outputs
Categories of things (thematic analysis)



Reasons for gambling...
RTI International

The nature of qualitative data
The data are:


Voluminous



Unwieldy



Multi-stranded with overlap...
RTI International

7.1 Data management

“Chaos is merely order waiting to be deciphered”
Jose Saramago
RTI International

What is Framework?
Developed by Jane Ritchie and Liz Spencer in the 1980s at NatCen
Social Research

Ke...
RTI International

Objectives of data management

Primary objective
Re-order

Make data accessible

Introduction to Framew...
RTI International

Data management
•

Data management involves ‘cutting’ up data

•

Aim is to create useful ‘piles’ of da...
RTI International

Theme-based approach

Crime

Introduction to Framework

Contact with
CJS

Impact

© NatCen Learning
RTI International

Case-based approach
Wendy
Bob
Jim
Robbery

Introduction to Framework

Contact
with police

Financial
im...
RTI International

Framework is a case and theme based approach

Name

Crime

Contact with
CJS

Impacts

Jim

Bob

Wendy

...
RTI International

The Framework approach
Familiarisation
Read proposal and transcripts
Review topic guide and field notes...
RTI International

Stage 1: Data familiarization process
•

Familiarize yourself with the data by reading
through transcri...
RTI International

Stage 2: Creating the ‘framework’
Multitude of potential descriptive categories
 different kinds and l...
RTI International

Frame construction: individual data


One chart represents one main theme



Chart each individual ca...
RTI International

Frame construction: group data


Chart as a whole group
–
–



One group per page, tracing individual...
RTI International

Before contact with employment scheme
Respondent

Employment
situation/
activity at time
of contact

Em...
RTI International

Stage 3: Testing the analytic framework


Testing framework is important
–

multi-stranded & overlappi...
RTI International

Data management coding / indexing
Example of analytical framework index for sexual identity project
1. ...
RTI International

Developing an analytical framework exercise


Groups to discuss analytic framework that could be used
...
RTI International

Stage 4: Summarising data (‘charting’)

• Condensing data
•

retains richness and flavour of data

• Pr...
RTI International
RTI International
RTI International

Introduction to Framework

© NatCen Learning
RTI International

Introduction to Framework

© NatCen Learning
RTI International

Entering data (‘charting’)
Process
 theme by theme (only when indexed)
 by transcript
Guidelines for ...
RTI International

Data Management: Good practice
Reading the transcript beforehand
 structure of account and nature of t...
RTI International

7.2 Interpretation

“All meanings, we know, depend on the key of interpretation”
George Elliot
RTI International

Descriptive or explanatory analysis


Descriptive accounts



Explanatory accounts
RTI International

There are three key steps…
Detection
 familiarisation
 extraction (highlight/ summarise)

Categorisat...
RTI International

Categorization and classification
Lone parents orientation to the Labor Market
•

Relevant dimensions
...
RTI International

Categorization and classification
Classification of sexual identity
 Relevant dimensions
–
–
–
–



C...
RTI International

Associations and explanations


Linkages between:
–
–
–



Two attitudes
Attitude and behavior
Circum...
RTI International

Making explanations


Informed by
–
–
–



Involves
–
–
–




Hunches and hypothesis
Reflections du...
RTI International

Levels of classification and interpretation
In summary:
 Descriptive categories
–



Factors, reasons...
RTI International

Any Questions?
RTI International

8. Reporting the Findings

RTI International is a trade name of Research Triangle Institute.

www.rti.o...
RTI International

Purpose of session


To provide a basic overview of the principles involved in
reporting qualitative f...
RTI International

Aims of the final report


To complete and document the research process
 To inform key stakeholders ...
RTI International

Basic principles for writing


Tell the story
 Structure the report
 Know your reader
 Write in pla...
RTI International

Reporting the Findings







Revisit the objectives (ensure the report covers these)
Consider th...
RTI International

Structuring the report


Title
 Contents
 Executive summary
 Introduction / Background
 Methodolog...
RTI International

Nature of the reporting



Descriptive



Explanatory
RTI International

Defining the boundaries







Research rationale
Methodology
– Sampling strategy
– Data collection...
RTI International

Using Quotations


Avoid over-use
 Avoid under-use
 Present range and balance
 Edit sparingly
 Sho...
RTI International

Quotation examples





“I don‟t feel you have to be out in all contexts…” [Gay
man, 35-44]
“The Gov...
RTI International

Quotation examples




“I think it depends quite a lot on the way you live and
how old you are. For e...
RTI International

Is a quotation necessary?
The actual presence of the question was considered to
have an important purpo...
RTI International

Avoiding quantitative language




Avoid tendency to use numbers
Do not discuss qualitative results ...
RTI International

Principles to remember


Present balanced findings
 Present grounded conclusions (avoid assumptions)
...
RTI International

Editing and reviewing


Allow some time before editing (if possible)
 Review the content
 Check accu...
RTI International

Any Questions?
RTI International

9. Drawing Wider Inferences

RTI International is a trade name of Research Triangle Institute.

www.rti...
RTI International

Purpose of the session


To understand whether wider inference can be made
when using qualitative data
RTI International

Concerns


Small samples



Not statistically representative



Interviews/discussions not standardi...
RTI International

Generalization


Can qualitative findings be generalized beyond the
sample and context of the study re...
RTI International

Representational generalization
Meaning
 Inference to parent or sampled population
 Whether range and...
RTI International

Validity
Validity: Accuracy of representation


Has the researcher understood the issues
from the resp...
RTI International

Reliability
Reliability: Replicability


Internal reliability
–



Extent to which assessments and ju...
RTI International

Drawing wider inference
Depends on:

Quality of sampling
–
Reflects diversity of sample
population
–
I...
RTI International

Validation of the inference


Validation
– Comparison and check of fit
– Deviant cases analysis
– Memb...
RTI International

In summary


Validity: do the data and findings accurately reflect the
behavior , views etc of respond...
RTI International

Any Questions?
RTI International

10. Qualitative Research Ethics

RTI International is a trade name of Research Triangle Institute.

www...
RTI International

Qualitative Research Ethics


Check the legal regulations for the State / Country
 Check your organis...
RTI International

References and Reading List

RTI International is a trade name of Research Triangle Institute.

www.rti...
RTI International

Reading List









Quality in Qualitative Evaluation: A Framework for assessing research
evid...
RTI International

Other Resources






http://www.eiu.edu/~ihec/KruegerFocusGroupInterviews.pdf
http://www.tgci.com...
RTI International

Final Questions and Discussion
Upcoming SlideShare
Loading in …5
×

Introduction to Focus Groups, Odum Institute, October 30

1,477 views

Published on

An introduction to conducting focus groups for social science research. The course includes information on developing protocol guides, moderating focus groups, analyzing results, and reporting findings.

Published in: Technology, Business
  • Be the first to comment

Introduction to Focus Groups, Odum Institute, October 30

  1. 1. RTI International Intro to Focus Groups Emily Geisen Amanda Wilmot 10/30/2013 RTI International is a trade name of Research Triangle Institute. www.rti.org
  2. 2. RTI International Ground Rules     Cell phones turned to vibrate or off Refreshments and restrooms any time Several bio-breaks throughout Questions are encouraged at end of each section
  3. 3. RTI International Course Outline The role and use of focus groups 2. Focus group demonstration 3. Developing the protocol guide 4. Moderating focus groups 5. Recruiting and scheduling 6. Participant selection 7. Data management/analysis 8. Reporting findings 9. Drawing wider inferences 10. Qualitative Research Ethics 1.
  4. 4. RTI International 1. The Role and Use of Focus Groups RTI International is a trade name of Research Triangle Institute. www.rti.org
  5. 5. RTI International What is a Focus Group? “Focus group interviews typically have five characteristics or features: (a) people, who (b) possess certain characteristics, (c) provide data (d) of a qualitative nature (e) in a focused discussion.” -Focus Groups: A Practical Guide for Applied Research (Krueger)
  6. 6. RTI International Examples   A group of coal miners sit around a small conference room discussing mine safety. They all agree that safety is important, but some argue that existing safety regulations are too unrealistic or impractical to be implemented. A group of employees sit in a conference room discussing access to email and other work functions from mobile devices. One participant says answering emails on his phone will save him time. Another participant worries that her supervisor will expect her to reply to emails at all hours if she has access on her phone.
  7. 7. RTI International What is a Focus Group, again?  Researchers have a topic they want to know about: – –  These topics are communicated to a group of participants to discuss – –  Why aren‟t coal miners following safety precautions? What concerns do employees have about mobile email access? Pre-scripted questions to generate discussion Targeted moderation to focus the discussion Researchers summarize and interpret what they have learned
  8. 8. RTI International Uses of Focus Groups  Exploration and Discovery – – –  Context and Depth – – –  Collect info that will be used to guide development of a survey Collect info for a needs assessment Test a new product before development Adding new topics or population groups to a survey Delving deeper into a survey topic Testing a product with a new audience Interpretation – Deciding how to use survey results – Discuss applications with end users -Focus Group Guidebook (Morgan)
  9. 9. RTI International Uses of Focus Groups*, Continued  Stand-alone data collection method, or  Supplement other qualitative or quantitative research methods – – – – In-depth personal interviews Survey development Needs assessment Product development
  10. 10. RTI International Focus Groups for Survey Research  Identify topics/themes that are of interest to population – –  Get more detail on a topic so that you don‟t have to ask open-ended questions –  Use this to add questions on these topic/themes Cut questions that are not relevant, not applicable, or show no variation Which of the following are reasons why you do not have health insurance? Too expensive, No health problems, … Get reaction to survey materials such as advanced letters, envelopes, etc.  Explore methods to boost cooperation: what type of incentives would people prefer?
  11. 11. RTI International Strengths of Focus Groups (over other qual./quan. methods)  If you‟re unfamiliar with a topic, you can learn a lot quickly (often used as the first step in research)  Can be used to observe interactions or group dynamics  Generate hypotheses (that can be tested using quantitative methods)  Cheaper/quicker than other methods
  12. 12. RTI International Limitations of Focus Groups  Some topics are too sensitive or controversial to be discussed in a group setting (social desirability)  Discussions can get easily sidetracked/derailed  Limited generalizability of findings/results  Cannot quantify findings or determine statistical significance
  13. 13. RTI International Thinking about your research…  How are you going to use focus groups? – – – Exploration and discovery? How so? Context and depth? How so? Interpretation? How so?  Will your focus groups be stand-alone or will you use them with other research methods?  What are the strengths/weaknesses of using focus groups for your research?
  14. 14. RTI International Questions/Discussion
  15. 15. RTI International 2. Focus Group Demonstration RTI International is a trade name of Research Triangle Institute. www.rti.org
  16. 16. RTI International 3. Developing the Protocol Guide RTI International is a trade name of Research Triangle Institute. www.rti.org
  17. 17. RTI International The What, Why, Who, How, Where What Review research objectives Who Determine number and type of participants Recruit and schedule participants Where Determine test location and equipment Why Identify major topic areas of interest How Script focus group questions Develop focus group guide
  18. 18. RTI International The What, Why, Who, How, Where What Review research objectives Who Determine number and type of participants Recruit and schedule participants Where Determine test location and equipment Why Identify major topic areas of interest How Script focus group questions Develop protocol guide
  19. 19. RTI International What’s the Focus Group About: Review Research Objectives      Start with the overall research objectives / study goals Identify what do you already know about the topic Determine what new information you want to learn Determine what information you want to explore in more detail Decide how are you going to use this information
  20. 20. RTI International Review Research Objectives: Example Step Personal Health Records Example Study goals: Learn about healthcare consumers understanding, experiences, needs, and concerns about the ways technology can be used within the healthcare system What do you already know about the topic: Research has shown that use of electronic PHRs can save consumers money and improve health, yet they are not being used extensively. What do we want to learn through focus groups: Why aren‟t more consumers using electronic PHRs to manage their health information? How will we use this info: Provide recommendations for improving PHRs and promoting awareness.
  21. 21. RTI International Review Research Objectives: Your Turn Step Study goals: What do you already know about the topic: What do we want to learn through focus groups: How will we use this info: Your Study
  22. 22. RTI International Why: Identify Major Discussion Topics   Once you identify what you want to learn, identify major discussion topics (brainstorm) Why are we doing these focus groups: – – –   What is the most important question your research must answer? What‟s the next most important question? And so on … Consult with project team members as needed Consider how topic areas may need to differ by participant selection
  23. 23. RTI International Identify Major Discussion Topics: Example  Why aren‟t more consumers using electronic PHRs to manage their health information?  Consumers who have not used electronic PHRs      How familiar are consumers with electronic PHRs? How do consumers currently manage their health information? What are the most important factors consumers consider when deciding how to manage their health information What are their biggest concerns/fears about using PHRs? Narrow list of discussion topics to about 4 major topic questions per hour (more if questions are more specific)
  24. 24. RTI International How: Scripting Questions  Avoid yes/no or short-answer questions, unless:    Open-ended questions generate discussion, however: – – –    Open-ended follow up is planned (e.g., why/why not?) Used to tally group (e.g., those in favor/opposed) Keep questions focused, one-dimensional Be careful of “Why?” questions Use scripted probes to focus or narrow topics Questions should be easily comprehensible: use familiar words/terms Questions should be reflective, not hypothetical Move from general to more specific
  25. 25. RTI International Scripting Questions: Example 1 Topic: How do consumers currently manage their health information?  Bad: How do you manage your family‟s health information?  Good: How do you currently maintain or store your and your family‟s medical records, such as records of vaccinations or prescriptions? – – Do you keep paper or electronic records? How does this differ for you compared to your children (or elderly parents)?
  26. 26. RTI International Scripting Questions: Example 2  Bad: Why do you use the mobile device you have?  Better: What kind of mobile device do you use at work?  Best: How did you decide what kind of mobile device to use at work?
  27. 27. RTI International Other aspects of Protocol Guide    Welcome/Introduction Icebreaker Opening (or Engagement) questions: – –   Exploration questions (main study questions) Exit questions: – –  Easy and inviting, comfortable to discuss Introduces the topic of discussion, provides context/background Used to wrap-up the discussion Check to see if there is anything else people would like to mention that they did not get a chance to Pre/Post Questionnaires
  28. 28. RTI International Welcome/Introduction  Welcome – –  Informed consent (if required) – – –  Introduce yourself, note-taker AND people behind the glass Make participants feel comfortable Read or summarize consent, ask participants to sign Note audio-recording if applicable Explain confidentiality Go over ground rules – – – Refreshments, restrooms, cell phones off or on vibrate No right or wrong answers Like to hear from everyone
  29. 29. RTI International Icebreaker    Sometimes it‟s helpful to start with an ice-breaker Ask participant to provide their name (first name only) Have them answer an easy question – – –  Participants should be able to answer it briefly, quickly Should not be sensitive Can be related to focus group topic or unrelated Example – – Employee focus group: How long at company? Department/Division? Focus group of moms: How many children and their ages
  30. 30. RTI International Intro Questions  Introduce Topic – –  Provide any needed context – – –  Goal/Purpose of the study Why participants were selected (if not obvious) For focus group on Personal Health Records, we had to explain what PHRs were Show participants related materials or products Provide background Opening Question – – Should be easy/simple to answer Non-sensitive
  31. 31. RTI International Pre/Post Questionnaires  Pre questionnaire: – –  Post questionnaire: – –  Use to collect information about your participants without wasting valuable time during the focus group Use to determine how familiar/knowledgeable participants are with topic at start of focus group Can be same as pre-questionnaire to monitor changes in opinion or knowledge Can be a handy way to summarize participants‟ opinions on discussion topics Keep questionnaires short (5 minutes)
  32. 32. RTI International Putting it all together  Allow time for welcome, explaining the purpose of the study, and consent procedures (if required) –  Allow time for introductory or ice-breaker question –  At least 5 minutes Time for your main discussion questions – –  At least 5 minutes About 4 question topics (per hour) 10-15 minutes per question Exit/Wrap-up – At least 5 minutes
  33. 33. RTI International Determining Question Flow What order would you put these questions in? 1. Describe the benefits of using a mobile device for the work you do? 2. How did you decide what kind of mobile device to use at work? 3. What are the biggest limitations or barriers to using a mobile device for the work you do? 4. How often do you use your mobile device for the work you do?
  34. 34. RTI International Question Flow 2. How did you decide what kind of mobile device to use at work? 1. Describe the benefits of using a mobile device for the work you do? 4. How often do you use your mobile device for the work you do? 3. What are the biggest limitations or barriers to using a mobile device for the work you do?
  35. 35. RTI International Protocol Guide Examples: Handouts     Handout 1: Mine Contractors Protocol Guide Handout 2: Healthcare Consumers Protocol Guide Handout 3: Tobacco Panel Protocol Guide Handout 4: Post questionnaire
  36. 36. RTI International Individual Practice (15 minutes) Use your study or class example  Identify 1-2 major discussion topics for your study  Prepare an introductory question  Prepare 1-2 discussion questions with probes as needed
  37. 37. RTI International Questions/Discussion
  38. 38. RTI International 4. Moderating Focus Groups RTI International is a trade name of Research Triangle Institute. www.rti.org
  39. 39. RTI International Understanding the group Process Forming Testing and dependence Dependence on the leader Storming Intragroup conflict Criticism Norming Development of group cohesion Optimism Performing Functional role relatedness Cohesiveness Adjourning (Mourning) Termination, Separation Model of group phases (Ritchie and Lewis 2003)
  40. 40. RTI International The Ideal Moderator:  Has adequate knowledge of topic  Has good communication skills  Is similar to / can build a rapport with participants  Can maintain balanced viewpoint  Is prepared for surprises  Reports accurately, even if it‟s uncomfortable  Has Experience with: – group dynamics – neutral probing/feedback
  41. 41. RTI International What the Moderator Does    Welcome/Introductions Leads discussion Concludes discussion
  42. 42. RTI International Welcome/Introduction  Make participants feel comfortable  Read or paraphrase this welcome/introduction in protocol guide  Make sure to collect informed consent  Notify participants of observers and/or audio recording  Go over any ground rules  Let participants get refreshments and get situated before starting actual focus group discussion
  43. 43. RTI International Leads Discussion  Provides unobtrusive control  Manages group dynamics  Asks spontaneous probes as needed to gather more information  Provides unbiased feedback  Summarize/verify information  Manages time
  44. 44. RTI International Provide Unobtrusive Control  Know when to stop talking –  Learn to focus the discussion – –  You said X, can you tell me more about that? Note topics said, that you want to circle around to Manage the flow of the discussion – –  Don‟t constantly interject, let participants discuss back and forth Don‟t have to follow script, allow natural segues If a topic you plan to cover later is brought up, indicate that you‟ll talk more about that later. Encourage different point of views – – Has anyone had a different experience? Does anyone view it differently?
  45. 45. RTI International Tips for Managing Group Dynamics  Dominant talkers: – –  Ramblers/Off-topic: – –  Don‟t look at them when you ask a question. (Politely) cut them off: “Thank you Amanda. Does anyone else have an experience they‟d like to share?” Avoid eye contact after so long off topic, look down or at clock Be prepared to interject at next pause Shy participants: – – Maximize eye contact Call by name: I‟ve heard from some of you on this and I want to hear from the rest of you. Sarah, what is your experience with X?
  46. 46. RTI International Tips on Spontaneous Probing  Leading probes: – – –  So you had a hard time with that then? You didn‟t want to learn more about that? Was that difficult for you? Neutral probing: – – – – Can you tell me more about that? Would you explain that further? Can you give me an example? Is there anything else?
  47. 47. RTI International When you hear yourself asking a leading question, balance it “So you think that‟s difficult then?” Leading question 47 “...or was it easy?” Balanced question
  48. 48. RTI International Tips on Neutral Feedback      Monitor your reaction to participants/discussion Be careful of nonverbal cues (leaning back, crossed arms) Limit head nodding: shows you‟re listening, but also implies agreement Provide quick verbal responses periodically (uh-huh, yes, mmm, okay, go on) Avoid loaded feedback (which implies agreement) – – Loaded feedback: “That‟s good!” “Excellent.” Neutral feedback: “That‟s helpful” “That‟s interesting.”
  49. 49. RTI International Summarize/Verify Information  As needed, summarize/paraphrase what has been said – –  See if anything has been missed? – –  Did I miss anything? Does anyone have anything else to add? Probe on any subtopics you didn‟t cover –  I understand that work-life balance is your most important concern. Safety is critical, but some of the precautions you‟re asked to follow are pointless or unrealistic. Is that correct? We talked about X, but I‟m also interested in Y Move on to next topic
  50. 50. RTI International Conclusion / Wrap-up   Be sure to leave time for wrap-up, especially for sensitive topics or intense discussions Ask participants to identify most important issues – –  Summarize major themes (ask note-taker for help) – –   Of everything we have discussed, what is the most important? If you could make one request to X, what would it be? Ask if its an adequate summary Ask if anything has been missed, anything to add Check with observers to see if they have any questions/clarifications Pay participants!
  51. 51. RTI International Dealing with Difficult Situations      Too many people show up Participants bring their children Only a few attend Group is reluctant to talk Can‟t get them to stop talking
  52. 52. RTI International Plan Your Response…  20 minutes into the focus group, and Sarah has not said anything…  You can‟t get Joe to stop talking…  John mentions something interesting, but you don‟t want to interrupt because there‟s a good discussion going…
  53. 53. RTI International Role of Note-Taker          Can Handle logistics & refreshments Collects signed informed consent (if required) Takes careful notes Does not participate in discussion Can recap major themes at end of discussion (used before wrap-up question) Monitors recording equipment Liaison between moderator and observers/clients Debriefs with moderator after session Assist with analysis and reports
  54. 54. RTI International Role of Co-Moderator  Not required, but can be useful in some situations  Balance out strengths/weaknesses in moderator  Use to match moderator (without being obvious)  Switch leading focus group (good for long or intense focus groups)  Support leader by keeping on track, recapping major themes, etc.
  55. 55. RTI International Moderating style  Different styles for different people – –  Spontaneous flow Follow guide strictly Adapt to your style, but plan ahead – – Spontaneous: Be sure to note what you‟ve covered and what you haven‟t so that you don‟t forget anything. Strict: “That‟s a very interesting point, and we‟re going to get to that in a minute, but right now I want to focus on X:
  56. 56. RTI International Group Practice Moderating (30 min) RTI International is a trade name of Research Triangle Institute. www.rti.org
  57. 57. RTI International Questions/Discussion
  58. 58. RTI International 5. Recruiting and Scheduling (review on your own) RTI International is a trade name of Research Triangle Institute. www.rti.org
  59. 59. RTI International At Least a Month Before Focus Group   Decide on what you‟re testing Decide on the participants (see Section 6) : – – –  Who you want to recruit How many How you will find them Plan the test date – – Find a date when your stakeholders can observe Decide on your location
  60. 60. RTI International How to Find Your Participants   Frame/list if available Advertisements: Craigslist, flyers, newspapers – –  Participant databases/recruiting orgs – –  Pros: quick, easy, cheap Cons: yields younger, higher educated users Pros: quick and easy, pay per user recruited Cons: can lead to “professional respondents” Other sources: word of mouth; sample lists; clubs, churches, and societies; snowball recruiting
  61. 61. RTI International What to Pay Participants?  Are participants allowed to accept monetary incentives? – –    Federal employees Participants who are being paid by their employer Is it appropriate for your organization to pay participants? Is topic interesting/important enough that incentives are not necessary How much should you pay them? – – – Trade-off between incentive amount and recruitment effort Tailor amount to population and location Will participants need to pay for parking?
  62. 62. RTI International Recruiting Tips  Recruit extras due to no-shows or cancellations –  Schedule sessions about 3-4 weeks ahead – –  Example: Recruit 12 to ensure at least 8 show up Any earlier and they may forget or make new plans Any later and you may not be able to recruit enough participants Send them an email or letter confirmation  Remind them the day before (by email or phone)
  63. 63. RTI International Testing Location   At a minimum, use a large conference room (at your organization‟s offices, hotel conference room) For numerous focus groups: formal observation rooms are nice (see picture on next slide) – – –  One-way mirror so you can see them, but they can‟t see you Microphones in ceiling to pick up voices, but not other sounds (paper shuffling) Video-taping capability Rent a focus group facility – Usually very nice but can be expensive ($1,000 for 2-hour session)
  64. 64. RTI International Observation Room Example
  65. 65. RTI International Create a Virtual Observation Room using screen-sharing software  Examples: Skype, Go to Meeting  Fosters collaboration – –  Improved schedule – –  Can accommodate observers from any location Facilitate discussions in conference setting Stakeholders get information immediately No waiting for recorded videos or report Cheaper: Inexpensive compared to travel costs
  66. 66. RTI International Note-Taking      Recommend having a note-taker and audio or videorecording the session Note-takers should be in the room or observation room For audio-recording, consider getting additional microphones so that you can hear everyone Video-recording can be helpful, but makes some participants uncomfortable Participants usually forget they‟re being recorded/ observed after 5-10 minutes
  67. 67. RTI International At Least Three Weeks Before Focus Group    Get started on recruitment Organize your incentives Develop your focus group guide
  68. 68. RTI International One Week Before the Focus Group   Final your focus group guide Organize roles in the test: – – – –   Meet and greet Observers/Stakeholders Moderator Note-takers Do a practice run on any equipment Arrange any refreshments – – – At a minimum provide beverages Tailor food to participants / time of day Nothing messy or loud
  69. 69. RTI International Prepare your materials     Develop consent forms, screeners Instructions/directions for participants Prepare any visual materials for participants Pretest/posttest questionnaires
  70. 70. RTI International The Day Before / Morning Of  Send out reminders: – –  Phone or email to respondents Email to observers, stakeholders, note-takers Equipment/Facility – – – – Make sure the room you‟ll use is tidy Make sure your meet/greet person has the final list of participants‟ names Incentives are available Check any equipment (video/audio recording)
  71. 71. RTI International 6. Participant Selection RTI International is a trade name of Research Triangle Institute. www.rti.org
  72. 72. RTI International Purpose of the session  To understand the principles of participant selection for focus group research  To consider the practicalities of participant selection for focus group research
  73. 73. RTI International Focus groups: a qualitative data collection method    Focus groups are a qualitative data collection method When reporting the findings the rationale should be provided for the research along with the methods used Is the research… – – – – Justified Rigorous Systematic Transparent
  74. 74. RTI International Sampling for qualitative research  The design of a sampling strategy for qualitative research is as important as that for quantitative research  Qualitative research uses non-probability sampling
  75. 75. RTI International Informing the design of a qualitative sampling strategy              What are the research objectives? What is the scope of the research? Who is out of scope and should be excluded from the sample? Who is in scope and should be included in the sample? What is the budget? What is the reporting time period? What sampling technique will be employed? How are the data to be analyzed? What data collection methods should be employed? What are the sample criteria? What size should the sample be? What will be used as the sampling frame? How are potential respondents/participants recruited?
  76. 76. RTI International Sampling technique Quantitative sampling  Probability sampling   Members of the research population are chosen at random and have a known probability of selection The aim is to produce a statistically representative sample Qualitative sampling  Purposive non-probability sampling    The number of people interviewed is less important that the criteria used to select them Members of the research population are chosen on the basis of their characteristics to reflect breadth and diversity of the research population We do not aim to produce a statistically representative sample or draw statistical inference
  77. 77. RTI International Sample criteria  What characteristics will need to be reflected in the sample population to ensure breadth and diversity?  Criteria used may be based on demographic characteristics or behaviours or attitudes  Some criteria may be considered more important than others
  78. 78. RTI International Sample size  Small sample sizes for qualitative research There is no need for scale because there is no need for statistical inference  Sample size determinants:     Heterogeneous or homogenous nature of sample population Number of selected criterion Scale   10 to 50 for one-to-one investigation 40 to 100 for group interview
  79. 79. RTI International Sampling frames A sampling frame is a list that identifies units within the target population Frame evaluation:  Comprehensive  Sufficient numbers  Geographical dispersion  Respondent contact details correct
  80. 80. RTI International Sampling frames  Existing frames – Administrative sources – Survey samples  Constructed frames – Direct and/or indirect methods
  81. 81. RTI International Constructed frames  Focussed enumeration  Snowballing  Screening questionnaires  Organisations  Advertisements
  82. 82. RTI International Study about attitudes and behaviors surrounding dental attendance Selection criteria initially considered           Age to ensure demographic balance Gender because patterns of attendance differ between men and women Family Unit composition because attendance by others in the family might influence the respondents attitudes or behaviours Employment activity because attendance might be affected by time constraints during working hours Income as a known factor affecting dental attendance patterns Regional location as dental attendance varies across the country Ethnic origin as may influence attitudes or behaviours Type of area as urban/rural location may affect attitudes and behaviours Dental health to explore how attitudes vary among people with different dental health Current pattern of dental attendance (regular/irregular/occasional) for comparative analysis Illustration from Ritchie J and Lewis J „Qualitative Research Practice‟ (2006)
  83. 83. RTI International Prioritised selection criteria Primary criteria Dental attendance pattern Age Gender Region Family Unit Employment activity Income Secondary criteria Dental Health Ethnic Origin Type of Area Illustration from Ritchie J and Lewis J „Qualitative Research Practice‟ (2006)
  84. 84. RTI International Sample Matrix Area 1 of 6 Group 1 18-29 Group 2 30-44 Group 3 45+ 4 men 1-2 regular attenders 1-2 irregular attenders 1 occasional attender 4 men 1 regular attender 1-2 irregular attenders 1-2 occasional attenders 4 men 1-2 regular attenders 1 irregular attender 1-2 occasional attenders 4 women 1-2 regular attenders 1-2 irregular attenders 1 occasional attender 4 women 1 regular attender 1-2 irregular attenders 1-2 occasional attenders 4 women 1-2 regular attenders 1 irregular attender 1-2 occasional attenders Spread of family unit type and employment activity Illustration from Ritchie J and Lewis J „Qualitative Research Practice‟ (2006)
  85. 85. RTI International Any Questions?
  86. 86. RTI International Excercise     Groups Designing a sampling strategy for focus groups on the use of electronic personal health records 15 mins Feedback
  87. 87. RTI International 7. Data Management and analysis RTI International is a trade name of Research Triangle Institute. www.rti.org
  88. 88. RTI International Purpose of the session  To understand the principles of good qualitative focus group analysis  To consider the practicalities of analysing qualitative data
  89. 89. RTI International Acknowledgement  Some of the slides contributed by the UK National Centre for Social Research (NatCen).  The Framework methodology was developed by Jane Ritchie and Liz Spencer in the 1980s at NatCen.  http://www.natcen.ac.uk/events-and-training/ourtraining
  90. 90. RTI International Qualitative data  Focus group recording  Focus group transcription  Notes  Admin data  Photographs, videos etc
  91. 91. RTI International Transcription  Transcription is time consuming  The focus group transcription should capture the discussion in it‟s entirety  Focus groups are more difficult to transcribe than one-toone interviews as there are multiple participants to distinguish between  It may be important for the analysis for the transcriber to be able to identify each participant‟s words. The transcriber will use the notetaker‟s notes to help determine who is speaking  Check on quality of the transcription  Anonymize transcript
  92. 92. RTI International Different approaches to data management, analysis and reporting  Summary report  Qualitative analysis
  93. 93. RTI International Nature of qualitative analysis  Quantitative research Qualitative research – - Creating categories – – Fixed categories aiming to enumerate Requires enumeration in order to be meaningful Test hypotheses and provide explanation - Developing explanations - Does not need enumerating - Range and diversity key
  94. 94. RTI International Aim of analysis - analytical outputs Categories of things (thematic analysis)   Reasons for gambling Sources of debt advice Categories of people or processes (typologies)   Types of parents of children with learning difficulties Types of welfare benefit claimants Explanations of attitudes, choices or impacts (explanatory analysis)  Factors influencing how people save for retirement  What helps achieve positive outcomes from an employment programme
  95. 95. RTI International The nature of qualitative data The data are:  Voluminous  Unwieldy  Multi-stranded with overlapping categories and theme
  96. 96. RTI International 7.1 Data management “Chaos is merely order waiting to be deciphered” Jose Saramago
  97. 97. RTI International What is Framework? Developed by Jane Ritchie and Liz Spencer in the 1980s at NatCen Social Research Key characteristics  Approach to data management which facilitates case and theme based analysis of qualitative data  Case and theme based approach  Matrix display  Reduces data through summarisation and synthesis  Retains links to original data  Output allows comprehensive and transparent data analysis Introduction to Framework © NatCen Learning
  98. 98. RTI International Objectives of data management Primary objective Re-order Make data accessible Introduction to Framework Secondary objective Reduce, prioritize © NatCen Learning
  99. 99. RTI International Data management • Data management involves ‘cutting’ up data • Aim is to create useful ‘piles’ of data • Three ways you can ‘cut’ data  Thematically  By case  By case and theme Introduction to Framework © NatCen Learning
  100. 100. RTI International Theme-based approach Crime Introduction to Framework Contact with CJS Impact © NatCen Learning
  101. 101. RTI International Case-based approach Wendy Bob Jim Robbery Introduction to Framework Contact with police Financial impact © NatCen Learning
  102. 102. RTI International Framework is a case and theme based approach Name Crime Contact with CJS Impacts Jim Bob Wendy Introduction to Framework © NatCen Learning
  103. 103. RTI International The Framework approach Familiarisation Read proposal and transcripts Review topic guide and field notes Identify descriptive categories Develop working analytical framework of key themes and sub-themes Indexing Pilot Charting Apply numerical series to framework, label data sources and revise framework Chart a few transcripts using working framework and the revise Charting Summarise/synthesise data Investigation and interpretation Reporting findings Introduction to Framework © NatCen Learning
  104. 104. RTI International Stage 1: Data familiarization process • Familiarize yourself with the data by reading through transcripts and other documentation • Immerse yourself in the data • Begin to construct your analytical framework Introduction to Framework © NatCen Learning
  105. 105. RTI International Stage 2: Creating the ‘framework’ Multitude of potential descriptive categories  different kinds and levels of category Choice of categories depends on  interview guide  research questions  emergent themes  recurrence across cases Data management  interpretation Introduction to Framework © NatCen Learning
  106. 106. RTI International Frame construction: individual data  One chart represents one main theme  Chart each individual case – – – Each individual as one row Subthemes in each column Give cases same space on each thematic chart
  107. 107. RTI International Frame construction: group data  Chart as a whole group – –  One group per page, tracing individual cases – – –  Each group as one case Several groups on each thematic chart Order cases in the same way Give cases same space on each thematic chart Don‟t be alarmed if every cell is not filled for each case Guidelines for entering data – – May be useful to keep a record of group process Levels of participation, context in which comments made, how views evolve/develop/change
  108. 108. RTI International Before contact with employment scheme Respondent Employment situation/ activity at time of contact Employment history/work activity in past Hopes & aims regarding work before contact Barriers perceived in achieving aims Efforts made to overcome barriers before contacting NDLP Perception of needs before contact Overview of financial situation & views about at time Other/Misc #1 #2 #3 #4 A chart with sub-categories Introduction to Framework © NatCen Learning
  109. 109. RTI International Stage 3: Testing the analytic framework  Testing framework is important – multi-stranded & overlapping categories – different experiences  Test by coding or indexing – apply numerical code to categories and sub-categories – code using software or write codes in margin of transcript  Test by piloting –  enter data from small number of transcripts Checking for: overlap / gaps
  110. 110. RTI International Data management coding / indexing Example of analytical framework index for sexual identity project 1. Demographic details 2. Defining sexual identity 2.1 Conceptualization 2.2 Salience 2.3 Self categorization 2.4 Changes over time 3. Language used 3.1 Terminology used 3.2 Categories used 4. Acceptability 4.1 Acceptance by society 4.2 Understanding purpose of questioning 4.3 Trust in data security 4.4 Trust in data collector 5. Other
  111. 111. RTI International Developing an analytical framework exercise  Groups to discuss analytic framework that could be used for the morning focus group session  Create main themes and subthemes  Post-it notes  15 mins  Feedback – group spokesperson
  112. 112. RTI International Stage 4: Summarising data (‘charting’) • Condensing data • retains richness and flavour of data • Process • theme by theme (if indexed) or by transcript • Takes practice – an art Introduction to Framework © NatCen Learning
  113. 113. RTI International
  114. 114. RTI International
  115. 115. RTI International Introduction to Framework © NatCen Learning
  116. 116. RTI International Introduction to Framework © NatCen Learning
  117. 117. RTI International Entering data (‘charting’) Process  theme by theme (only when indexed)  by transcript Guidelines for entering data  summarise material from transcripts/other data sources  retain language  mark but don‟t recite quotations  note page references / create links between summaries and the original transcript within CAQDAS software  use agreed abbreviations/conventions  avoid repetition by cross referencing data Introduction to Framework © NatCen Learning
  118. 118. RTI International Data Management: Good practice Reading the transcript beforehand  structure of account and nature of the data  repetition, clarification, contradiction Reading chart afterwards  clarity  balance and emphasis  check blank cells Use ‘other’ column for interpretative notes  note „flavor‟ of discussion  note overarching issues  alert others to important issues Introduction to Framework © NatCen Learning
  119. 119. RTI International 7.2 Interpretation “All meanings, we know, depend on the key of interpretation” George Elliot
  120. 120. RTI International Descriptive or explanatory analysis  Descriptive accounts  Explanatory accounts
  121. 121. RTI International There are three key steps… Detection  familiarisation  extraction (highlight/ summarise) Categorisation  creating meaningful conceptual boxes  assigning data Classification  creating higher order categories  assessing relationships between categories Introduction to Framework © NatCen Learning
  122. 122. RTI International Categorization and classification Lone parents orientation to the Labor Market • Relevant dimensions    • Attitudes to work Barriers to work Stage of job search activity Typology      Work not currently an option Beginning to think about work Personal issue barriers Labor market barriers Close to work Introduction to Framework © NatCen Learning
  123. 123. RTI International Categorization and classification Classification of sexual identity  Relevant dimensions – – – –  Conceptualization Salience Self-categorization Changes over time Typology – – – Latent identifiers Conscious identifiers Reluctant identifiers (2)
  124. 124. RTI International Associations and explanations  Linkages between: – – –  Two attitudes Attitude and behavior Circumstances and need Verified through explanation – – Explicit respondent accounts Implicit identified by the researcher
  125. 125. RTI International Making explanations  Informed by – – –  Involves – – –   Hunches and hypothesis Reflections during field work and analysis Other research or theories Detailed within case analysis Comparison between cases Repeated interrogation of data Comprehensive Expect multiplicity
  126. 126. RTI International Levels of classification and interpretation In summary:  Descriptive categories –  Factors, reasons, impacts Classifications or typologies  Explained associations  Meaning  Theory/Strategy generation
  127. 127. RTI International Any Questions?
  128. 128. RTI International 8. Reporting the Findings RTI International is a trade name of Research Triangle Institute. www.rti.org
  129. 129. RTI International Purpose of session  To provide a basic overview of the principles involved in reporting qualitative findings and constructing a final report.
  130. 130. RTI International Aims of the final report  To complete and document the research process  To inform key stakeholders of the findings  To communicate the findings in a clear and coherent way  To guide readers in the interpretation of findings (avoid misinterpretations)
  131. 131. RTI International Basic principles for writing  Tell the story  Structure the report  Know your reader  Write in plain English  Use visual representation
  132. 132. RTI International Reporting the Findings       Revisit the objectives (ensure the report covers these) Consider the audience Check report format Consider time available Consider length of the report (focus on relevant points only) Report findings in past tense
  133. 133. RTI International Structuring the report  Title  Contents  Executive summary  Introduction / Background  Methodology  Findings  Conclusions  Recommendations  Appendices e.g. sample matrix, interview guide
  134. 134. RTI International Nature of the reporting  Descriptive  Explanatory
  135. 135. RTI International Defining the boundaries     Research rationale Methodology – Sampling strategy – Data collection – Method of analysis – What can or cannot be inferred form the findings Examples of relevant documentation – Interview guide – Contact letter – Screening questionnaire – Sample matrix Substantiate conclusions by grounding the data in the findings – Examples – Quotations – Case illustrations
  136. 136. RTI International Using Quotations  Avoid over-use  Avoid under-use  Present range and balance  Edit sparingly  Should provide illustration only  Amplify but do not repeat a point  Avoid reliance on a few articulate respondents  Avoid very long quotations  Identify relevant characteristics  Preserve participant confidentiality
  137. 137. RTI International Quotation examples    “I don‟t feel you have to be out in all contexts…” [Gay man, 35-44] “The Government uses statistics to show what it wants to show, for example unemployment, they‟ve used all sorts of different measures for employment to show it‟s going down…rather than it necessarily going down itself.” [MHE Wales] “They don‟t XXX care about us. It‟s everyone for themselves” [Male, 18-24, Central London]
  138. 138. RTI International Quotation examples   “I think it depends quite a lot on the way you live and how old you are. For example, I was born down [South West county], and my parents being extremely religious, have very definite views on sexuality.” [Bisexual woman, aged 35-44] “I‟ve lived in [Muslim country], nobody would ever admit that [being gay], so they all choose to be heterosexual. Somebody‟s not been here that long, it‟s even more of an issue.” [Heterosexual woman, aged 35-44]
  139. 139. RTI International Is a quotation necessary? The actual presence of the question was considered to have an important purpose in its own right. It was thought that as the questioning became more commonplace it would demonstrate to the wider public that non-heterosexuality was unremarkable. “It‟s having it on there, it‟s beginning to usualise the question, it‟s a cultural shift, it‟s enabling people to gradually recognise it‟s no bloody big deal. But if we don‟t have it there, we are setting up this whole process of we must be quiet about these people and all the rest of it.” [Gay/lesbian woman, 55+]
  140. 140. RTI International Avoiding quantitative language    Avoid tendency to use numbers Do not discuss qualitative results in terms of proportions, percentages or statistics Indicate strength of finding by discussing in terms of: –  A common finding, A recurring problem, Problems were observed…. etc Use diagrams for illustration
  141. 141. RTI International Principles to remember  Present balanced findings  Present grounded conclusions (avoid assumptions)  Distinguish between findings and your own conclusions  Provide commentary alongside the analysis (do not leave the reader to do this for you)  Support with evidence from other research findings (if appropriate)
  142. 142. RTI International Editing and reviewing  Allow some time before editing (if possible)  Review the content  Check accuracy of statements  Peer Review  Proof read
  143. 143. RTI International Any Questions?
  144. 144. RTI International 9. Drawing Wider Inferences RTI International is a trade name of Research Triangle Institute. www.rti.org
  145. 145. RTI International Purpose of the session  To understand whether wider inference can be made when using qualitative data
  146. 146. RTI International Concerns  Small samples  Not statistically representative  Interviews/discussions not standardized  Proximity of researcher / too personal  Biased reporting
  147. 147. RTI International Generalization  Can qualitative findings be generalized beyond the sample and context of the study research?  Different definitions of „generalization‟. – Representational generalization.
  148. 148. RTI International Representational generalization Meaning  Inference to parent or sampled population  Whether range and diversity (experiences, explanations etc) can be matched with the sampled population  Whether list (of experiences, explanations etc) is inclusive of what would be found in the sampled population Basis  Depends on validity and reliability of findings
  149. 149. RTI International Validity Validity: Accuracy of representation  Has the researcher understood the issues from the respondent‟s perspective?  Have the issues been fully articulated and explained  Are interpretations underpinned by the data Alternative terms: Credibility and Plausibility
  150. 150. RTI International Reliability Reliability: Replicability  Internal reliability –  Extent to which assessments and judgments are replicated between researchers External reliability – Extent to which findings would be replicated if the study were repeated with the same or different sample Alternative terms: Confirmability, Consistency, Dependability
  151. 151. RTI International Drawing wider inference Depends on:  Quality of sampling – Reflects diversity of sample population – Includes all key constituents – Includes outliers and a-typical cases  Quality of data collection – Free from interference: neutral and objective – Probing of meaning – Exploration of all relevant explanations    Quality of analysis – Systematic, comprehensive and inclusive – Within and between case analysis – Displays diversity Quality of interpretation and reporting – Comprehensive – Clear – Underpinned by data – Displays multiplicity of accounts and explanations – Non-quantitative Documentation – Transparency – Research process
  152. 152. RTI International Validation of the inference  Validation – Comparison and check of fit – Deviant cases analysis – Member or respondent validation – Triangulation
  153. 153. RTI International In summary  Validity: do the data and findings accurately reflect the behavior , views etc of respondents? – –   Content validity: the respondent‟s expression or articulation of the issue Validity of interpretation: the researchers interpretation of the issue Reliability: would the data and findings be replicated if the study were repeated? Representational generalization: are the data and findings applicable to the wider population from which the sample is drawn?
  154. 154. RTI International Any Questions?
  155. 155. RTI International 10. Qualitative Research Ethics RTI International is a trade name of Research Triangle Institute. www.rti.org
  156. 156. RTI International Qualitative Research Ethics  Check the legal regulations for the State / Country  Check your organisations Institutional Review Board (IRB) policies  Understanding “informed consent” (verbal/written)  Understanding “confidentiality”  Basic principles of good social research – –  Participant safety (Physical/Emotional/Psychological) Researcher safety Recompense
  157. 157. RTI International References and Reading List RTI International is a trade name of Research Triangle Institute. www.rti.org
  158. 158. RTI International Reading List        Quality in Qualitative Evaluation: A Framework for assessing research evidence. A quality framework. A report produced on behalf of the Cabinet Office by Liz Spencer, Jane Ritchie, Jane Lewis and Lucy Dillon, National Centre for Social Research. ISBN: 07715 04465 8. August 2003. Government Chief Social Researcher‟s Office Crown Copyright 2003. Assessing the Quality of Qualitative Research. Patient Education and Counseling 90 (2013) 1–3. Qualitative Research Practice: A guide for Social Science Students and Researchers. Edited by Jane Ritchie and Jane Lewis. SAGE 2003. Focus Groups: Theory and Practice. D. W. Stewart and P.N. Shamdasani. Applied Social Research Methods, Volume 20. Focus Groups as Qualitative Research. David Morgan. Sage. 1989. Focus Groups: A Practical Guide for Applied Research. Richard Krueger. Sage. 1989. What Are Qualitative Research Ethics? Rose Wiles. Bloomsbury Academic. 2013.
  159. 159. RTI International Other Resources      http://www.eiu.edu/~ihec/KruegerFocusGroupInterviews.pdf http://www.tgci.com/magazine/How%20to%20Conduct% 20a%20Focus%20Group.pdf http://managementhelp.org/businessresearch/focusgroups.htm http://assessment.aas.duke.edu/documents/How_to_Co nduct_a_Focus_Group.pdf http://www.rowan.edu/colleges/chss/facultystaff/focusgro uptoolkit.pdf
  160. 160. RTI International Final Questions and Discussion

×