Pursuing research question with qualitative methods
Pursuing Your Research Question
With Qualitative Methods
Gail Matthews-DeNatale
Simmons College
You are here … but
where to begin? What
do you do now?
A Word on “Qualitative”
Core Beliefs
• The client/learner perspective
is different from our own
• Need to understand the life
patterns of those whom we
serve/teach
• It’s worth taking the time to
talk with, observe, and try to
“walk the life” of these people
• If we involve clients/learners
prior to (and during) the
development process, the
end result is improved
Related Terms
(What’s in a Name?)
• Ethnographic Research
• Participatory Design
• Contextual Inquiry
• Action Research
Example: Simmons SSW ePortfolios
Focus group with
pilot student
participants
How would you describe an
ePortfolio to someone who
has never seen one?
What was easiest about the
experience … what was
most challenging?
With whom have you shared
your ePortfolio?
What advice would you give to
an incoming student who
has never created an
ePortfolio?
Example: Brighton U Usability Lab
People often forget
the steps they took,
where they got lost.
Ask participants to do
certain tasks (e.g.,
find something on a
web site), then
observe them in
action.
Note:
Lab is desirable, but not essential.
Example: Simmons Blended Learning
Online Journal Writing Prompts
• What did you learn about yourself as a teacher this week?
What aspects of your course would you describe as a
success and why?
• What challenges are you experiencing as a teacher in this
course? What are your quandaries or puzzlements?
• What appear to be the learning accomplishments and
challenges that your students have experienced so far in
the course?
• Looking at how you designed and deployed your course,
what things are you glad that you took the time to do?
What things would you do differently next time?
Thinking “Out of the Box”
• Photo elicitation (participant-created)
• Role Play / Participant Observation
• Relational Mapping
• Paper Prototyping
• Shadowing
Photo Elicitation
Use the camera provided to take pictures of
• Your favorite place to study
• The computer you use
in the library
• One picture of the libraries
to show to a freshman
• All the stuff you take to class
• The place you keep your books
• Your favorite person or people to study with
• A place in the library where you feel lost
• The night before a big assignment is due
Source: Foster and Gibbons, Studying Students (2007)
Role Play
“What about designing
something that isn’t physical,
something like a service or
experience, interactions over
time? The best way to get a
feeling for flaws in your design
is to act it out. A place where I
think the effort is really worth
while, is where people are
wrestling with quite serious
problems: things like education,
security, finance, or health.”
Tim Brown (CEO of IDEO)
Participant Observation
“Another way to [deepen
understanding] is to put
ourselves through an
experience that we are
designing for. To project
ourselves into an experience.”
Tim Brown (CEO of IDEO)
15 Minute Breakout (Record on Flipchart)
• Which methods are relevant to your group’s
research? What other strategies come to mind?
• How might you use several methods in
conjunction with one another to gather a rich,
triangulated, set of data?
• Of the methods you deem relevant, which ones
could you implement without additional support or
training?
• What additional resources will you need, and who
on your campus might be able to provide that
support/training?
Interviewing: Asking Good Questions
"I've got two pieces of bad news about that
experimental English comp course where students
used computer conferencing.
First, over the course of the semester, the
experimental group showed no progress in abilities
to compose an essay.
The second piece of bad news is that the control
group, taught by traditional methods, showed no
progress either."
From Steve Ehrman, “Asking the Right Questions”
Interviewing Truisms
No such thing as a “perfect” interview
Record practice interviews, review and critique with
a trusted friend, person with expertise, etc.
The better you listen, the more articulate your
interviewees will become
Practicing with recording equipment beforehand is
like carrying an umbrella
Your last interview will be orders of magnitude better
than your first – plan accordingly
Sequence Matters
Begin with “easy” low stakes questions to develop
comfort and rapport
Cluster questions according to themes, but only ask
one question at a time
Question clusters should begin with a general query,
each question becomes increasingly specific
Don’t go lock-step through your questions
Consider using pre-interview exercises (e.g.,
mapping) to “prime the pump”
Wording and Pacing Matters
Whenever possible, use open-ended questions
(questions for which there isn’t a yes/no answer)
Beware of questions that betray your assumptions
(As a math major, in what ways are your needs for
the writing lab diminished?)
Good follow-up questions are the mark of a pro
(for example? … in what way? … when was that?)
Silence is a strategy – if you wait and appear to be
interested, most interviewees will elaborate
Individually, take five minutes to jot down interview
questions that would be relevant to your group’s
project.
As a group, pool questions – what patterns or
clusters emerge?
Select 5-10 questions that could be used in a one-
on-one interviews for your project. Please also note
who you would be interviewing (e.g., students in x
year or y department, adjunct faculty, pre-tenure
faculty, post-tenure faculty, etc.)
15 Minute Breakout (Record on Flipchart)
Action research means integrating your research into the
development of your project.
Note that worked with the results of two open-ended questions last time. Looking for patterns in large data sets.
Caveat:
Caveat:
Sloan Funded
Includes both Quantitative and Qualitative (Triangulation)
Pre-Course Peer Assessment Using Checklist
Pre- Post- Attitudes and Assumptions Survey (students)
CLASSE in Last Week
Faculty First Semester Online Journals
Videotaped Exit Interviews with Faculty
Data Used for Two Purposes
Improve Courses (Claudia Morner – improve her presence)
Create Media-Rich Case Studies
http://at.simmons.edu/blendedlearning/pages/learnhow/casestudies/morner/interview_alone.php
MAKE SURE YOU CLICK ON “Crafting Discourse”
Bottom image depicts doctors and nurses acting out a service scenario
One of the IDEO Staff putting himself through the experience of being admitted as an ER patient.
Reviewing the video, say 20 minutes of this [click]
Points to the significant difference between the experience/perspective of medical staff and the incoming patient.
In what higher education scenarios might this be useful?
Experiencing the process of class enrollment from the student perspective
What other examples come to mind?? [ask the group]
Example from Lowell Community Matters
Insert picture of map
Map – identify neighborhoods
Newcomers interview longterm residents and vice-versa
The Lowell exercise could be used between first year students and seniors to look for patterns of acculturation and acclimation
Or, [CLICK] Rochester study – asked a group of students to record their movements one day on a map
Example – GA Tech library – night staff map the position of movable furniture to assess differences in how day/night students use the space
From the University of Rochester Study, Student feedback on paper version of website
Paper prototype of a museum exhibit. Museums have experimented with temporary signage that is revised and moved in response to visitor input.
For example, to understand the software training needs of students, staff, etc, get permission to shadow several people.