1279 QUALITATIVE RESEARCH METHODS A COMPARISON BETWEEN .docx

1279
QUALITATIVE RESEARCH METHODS: A COMPARISON
BETWEEN FOCUS-GROUP AND IN-DEPTH INTERVIEW
Zaharia Rodica Milena
Bucharest University of Eonomics, Romania, Faculty of
International Business and Economics,
[email protected], Tel.:+40 21 319 19 90, tel. +40 0722179201
Grundey Dainora
Vilnius University, Kaunas Faculty of Humanities,
[email protected], Tel.: +370 37 425 462,
Fax: +370 37 423 222
Stancu Alin
Bucharest University of Economics, Faculty of Marketing,
[email protected], Tel.:+40 21 319 19 90
Qualitative research methods tend to be used more and more in
academic research. The cost for these
methods is quite low and the results may be very interesting and
useful for many fields of study. However,
the utility and the characteristic of qualitative research methods
differ from subject to subject and from
discipline to discipline. This paper comes close to a comparison
of two qualitative research methods
(focus-group and in-depth interview) used in investigating the

opinion of academics, analyzing by
comparison the results founded in a research conducted in the
Bucharest University of Economics using
focus group and in-depth interviews. The conclusions of the
study reveal that apart of the limits states in
the literature, there are other elements that can contribute to
obtaining unrealistic results.
Key words: Qualitative research methods, focus group, in-depth
interview, academic research
Introduction
Rapid social change and the diversity of the world have
contributed on a large scale to the diversification
of research methods. Limits of quantitative research methods
have determined orientation to the qualitative
instruments which are more reliable in certain circumstances.
There are many virtues of qualitative
research that determine a lot of researchers to manifest
preference for these kinds of methods: are the
correct choice of appropriate methods and theories, the
recognition and analysis of different perspectives,
the researchers’ reflections on their research as part of the
process of knowledge production, and the
variety of approaches and methods (Flick, 2002:4). Qualitative
research explain how it may be useful for
exploring “why” rather “how many”.
There are various types of instruments used to collect data for
qualitative research. Focus groups and in-
depth interviews are among the most utilize instruments that
researchers are using in collecting their data.
Focus group implies a group discussion in order to identify
perceptions, thoughts and impressions of a

selected group of people regarding a specific topic of
investigations (Kairuz, Crump and O’Brien, 2007).
Discussion should be perceived by the participants as no-
threatening and free to express any kind of
opinion, no matter if this opinion is shared or not by the other
participants. Focus groups generate valuable
information, especially when the participants represent small
groups of interest, ignored by the quantitative
research or when the area of investigation.
The in-depth interview is a technique designed to elicit a vivid
picture of the participant’s perspective on
the research topic. During in-depth interviews, the person being
interviewed is considered the expert and
the interviewer is considered the student. The researcher’s
interviewing techniques are motivated by the
desire to learn everything the participant can share about the
research topic.
In depth interview is an effective qualitative method for getting
people to talk about their personal feelings,
opinions, and experiences. It is also an opportunity to gain
insight into how people interpret and order the
world. We can accomplish this by being attentive to the causal
explanations.
The results obtained through these two qualitative methods
varied (among others) according to the subject
investigated. As a general rule, in depth interviews, the
participants are more confident, more relaxed and
they feel more encouraged to express the deepest thoughts about
a certain subject. In focus group the
1280

participants act according to their personality; it is the risk that
in some situation those with a week
personality to follow those with a stronger personality. There
are also some situations when the issue
discussed is incommode and the participants are not confident
in expressing their real opinions. They are
more preoccupied by the image that the other participants will
build up on them that to express what they
really think about that subject.
The case study
This study tries to identify the limits of using focus group
comparing with in-depth interview in identifying
the opinions of academics regarding some aspects of academic
research. The aspects evaluated through
these two qualitative methods were related to the connection
between academic research and business
environment and to the performance of the academic research.
Both issues are quite very important for Romanian academics
and universities management. The
performance of academic research (through the results of the
research process) is perceived as almost
similar with the performance of the university.
Cooperation between academic research and business
environment is considered to be one of the proofs
that academic research is in the benefit of society, a
confirmation of the market utility of universities.
Academic research is considered to be efficient and social
valuable if it offers solution to real problems that
companies confront on. Students seem that are not willing to
sacrifice functional expertise in favor of
generalist expertise. (Schelfhaudt and Crittenden, 2005)

From universities point of view, collaboration between business
environment and universities is an
important source of casuistic for the teaching process. It is also
an important source of financial resources
for universities, and many times sponsorship activity is
determined by the implication of business
environment in the academic activity. There are cases when
industries developed in a perfect harmony with
universities (Silicon Valley).
From the students’ point of view, those universities with strong
connection in business environment are
very attractive. It is a proof of the fact that they will have better
opportunities to find a job and a
confirmation that higher education does not offer only
theoretical knowledge, but also practical skills and
abilities.
In Romania it is still believed that universities are more
oriented to theoretical approach than to practical
direction. Universities considered that business environment
does not have positive reception for academic
research. Business environment is not convinced that
universities have necessary abilities to find correct
answers for their problems.
Methodology
The purpose of this paper is to present the differences between
focus group and in depth interview using
these qualitative methods on a subject regarding the academics’
evaluation of the performance of academic
research and the relation between business environment and
academic research. Both focus group and in-
depth interviews were conducted in the same university,
following the same conversation guide. Some of
the academics interviewed by in depth interview method were

present in the focus group. The structure of
focus group was similar with that of in-depth interviews.
Academics were grouped in three categories,
according to age and involvement in research. Group A, below
30, group B between 30 and 50 (with two
sub-groups, B1 with little involvement in research and B2 with
involvement in research) and group C, over
50 (also with 2 sub-groups: C1, with little involvement in
research and C2 with involvement in research).
The structure of the conversation guide was around three major
themes: what do you consider about the
present evaluation system of academic research, why the
performance of academic research in Romania is
considered to be low and how do you appreciate the relation
between academic research and business
environment. In order to discuss the comparison between these
two qualitative methods were selected two
questions: the first one, what do you consider about the present
evaluation system of academic research,
and the last one, how do you appreciate the relation between
academic research and business environment.
The reason of selecting these two questions is given by the fact
that the first two question offer similar
information from this paper’s goal point of view.
Also, literature review was used in order to determine the place
of this study among other researches. At
the international level it is a constant preoccupation in
analyzing both the virtues and the limits of focus
1281
group and in-depth interview and some of these studies were

consulted in order to sustain the conclusions
of the present paper.
Major findings
What do you consider about the present evaluation system of
academic research?
On this question, the answers gained in focus group and in in-
depth interview were quite different. If both
in-depth interviews and focus group all of the participants
complained about the present evaluation system,
the motives of complains were different.
The participants’ attitude in focus group was rather favorable to
the research aspects. Nobody mentioned in
the focus group pecuniary aspects. All critics expressed by the
participants in focus group were related to
the relativity of the criteria established by the Ministry “why
they are asking for publication in ISI journals,
when we don’t have such journals in Romania” (X12), to the
high pressure that exist on the professors
“students are asking for a good professor, evaluation system
asks for a good researcher. It is quite difficult
to be good on both” (X3, X9, X11). All participants in focus
groups agreed that publication should be a
criterion of evaluation, but not in this way. A lot of vague
aspects were revealed, as “originality of the
research”, or “the contribution to the field”, criteria which are
difficult to evaluate and nobody explained
what it mean.
In in-depth interviews the critics were very acid. The attitude
against research activity was evident. “When
I was hired, nobody told me that I supposed to be a researcher. I
was thinking that I will be a professor”
(X9, X6). “Which is the difference between research institute

and universities? We should educate people,
not to research” (X12, X10). Financial aspects were also among
those reasons invoke the most. “They
should first look to our wages and then they should ask us to
have similar performances as the western
academics” (X2, X7, X5).
The differences between the attitude expressed in focus group
and in in-depth interviews have different
reasons. One reason is given to the fact that all academics
acknowledge that research is an important part of
the academic activity and they don’t want to admit “in public”
that they do not agree with it. The
complaints are related to the way of establishing criteria not
with the criteria. In in-dept interview, they felt
more confident and they expressed their concerning and their
rejection regarding an activity which is
considered too difficult or unnecessary (for some) from their
point of view.
Another reason is given by the differences between generations.
Those over 50 are more reserved than
those under 30. But those over 50 have more official authority,
so they abstain to express negative opinion
in order to preserve their image.
How do you appreciate the relation between academic research
and business environment?
This issue is one on which focus group and in-depth interviews
were convergent. Both in focus-group and
in in-depth interviews a certain defensive attitude was revealed.
The academics consider that business
environment does not offer credit to academic research. “They
are not interested in what can we offer.
Managers are suspicious; they consider us a sort of spy (X1)”.
‘If you know someone in a company, than

you have chances to develop some relation with them, otherwise
is almost impossible”(X2). “Let’s be
realistic. What can we offer? Why should they be interested in
our researchers’ abilities?” (X7) “They
don’t need research; they definitely don’t need academic
research”(X5).
Other opinions consider business environments hostile to
collaboration with academic environment. “Our
students have many difficulties in finding companies for
internship and those which accept students ignore
them and do not allowed students to be involved in their current
job. “All the time, we receive the same
answer from the students when we ask them: what have you
done in the company? I answer to the phone; I
typed something on computer, things like these….(X6)”
From in-depth interviews, many additional comments appeared
in link with the relation between academic
environment and business environment. Almost all of them were
in the same spirit of malfunction relation,
which has roots both in the communist regime mentality and in
the perception that universities are mostly
oriented towards theory, not towards practice. “In communism,
the collaboration between so called
‘business’ and universities were compulsory and nobody could
say no, therefore, it was a formal
cooperation, especially in the economic field. Maybe for
engineering it was different but for us, it was
1282
totally formal, without any kind of practical results.”(X20,X24,
X2) “Companies do not appreciate

academic environment because they consider that universities
offer only pure theory. Many students are
working today and they are saying that what we are teaching
them doesn’t apply in practice”(X31).
Why on this question the results were similar? Why the both
methods lead to the same
conclusion, that business environment is hostile and is not
interested in the research that could
be provided by academics?
One explanation states in the fact that the collaboration between
business environment and academics is a
difficult one. Every academics confronted with this difficult
cooperation between universities and business
environment, mostly through students’ internship activity. As
one of the interviewed academics said, it is
somehow frustrating to ask the students all the time the same
question: “What have you gained from your
internship and to receive the same answer over and over again:
“Almost nothing because they didn’t
allowed me to involve in their business”. It is a real situation
given by the insufficient maturity of business
environment, on one side, and the struggle of academics to
prove their business utility on the other side.
We have to accept that Romania is still learning market
economy and all the actors are in the middle of a
radical transformation process. Business environment in
Romania, in general, gives no real attention to the
research activity. Many things are done “per se”. Romanian
companies are to poor and multinationals
prefer to involve specialized companies in their researches, not
academic environment. This is not a secret,
and academics are aware of this aspect. Therefore, the same
opinions are reflected by the focus-groups and
in-depth interviews.

Another explanation could be offered by the question itself.
Apart from the other two themes, this one is
about “somebody else”. It is a subject that concerns all of us,
but it involves another “entity”, an abstract
one, “business environment”. In focus group nobody was
worried that his/her opinion is misunderstood or
that the expressed opinions will have any kind of negative
consequences. Furthermore, as all of them
expressed the same thoughts, there was no fear of a “bad” image
perceived by the other participants.
Another motive could be given by the fact that the cooperation
with business environment is not necessary
assimilated with the personal academic performance. If an
academic does not publish, this could be
interpreted as a lack of skill or a low performance, but not being
involved in cooperation with business
environment could be the results of many external factors,
which does not necessary should be related with
personal abilities or personal performance.
Conclusions
The use of qualitative research methods is considered to be a
solution in investigating “Why?” especially
when the need to generalize the results is not necessary.
Qualitative research methods are also preferable
when the investigation is oriented to determine motivation,
perceptions or believes.
Focus-group and in-depth interview are among the most used
methods. There are some advantages offered
by these instruments: low costs and valuable information, which
are difficult to obtain from a quantitative
research.
The inconvenient of these methods are determine by their limits

in generalization the results. Also the
results can be altered if the instruments are used in an improper
manner. This case study shows that
interviews are also especially appropriate for addressing
sensitive topics that people might be reluctant to
discuss in a group. The question regarding the present
evaluation system of academic research was a
sensitive one and the results demonstrated that on question like
this in-depth interviews are more indicated.
On the second question the results were similar using both
methods because the subject discussed was
suitable for a group discussion too.
References
1. Aaker D.A; Kumar, V.; Day, G. (2001), Marketing Research,
Seventh Edition, John Wiley&Sons,
Inc. New York
2. Burns, A.C; Bush, R.F (2002), Marketing Research, Third
Edition, prentice Hall, New Jersey
3. Flick, U (2002), An introduction to Qualitative research,
SAGE Publication, London
1283
4. Kairuz, T, Crump K, and O’Brien, A. (2007), “Tools for data
collection and analysis”,The
Pharmaceutical Journal, Vol. 278, pp. 371-377, (referred on
02/04/2008), available from
www.pjonline.com
5. Kvale S. (1996), Interviews: An Introduction to Qualitative

Research Interviewing. London: Sage
Publications
6. Qualitative Research Methods: A Data Collector’s Field
Guide, (referred on 12/04/2008) available
from
ttp://www.fhi.org/nr/rdonlyres/eprvylcljxmbssmuizj3rcnycde2pn
lowtoa7nxedbjl23jqjpitifimwv6v
2tmsmyocepirpjpfrd/interviews1.pdf
7. Rubin HJ, Rubin IS. (1995), Qualitative Interviewing: The
Art of Hearing Data. London: Sage
Publications
8. Schelfhaudt, K; Crittenden, V.L. (2005), “Specialist or
generalist: Views from academia and
industry”, Journal of Business Research, 58, pp. 946– 954,
(referred on 02/03/2008), available
from Elsevier
9. Simon-Kumar, R. (2005), "Is Qualitative Research also
Quality Research? Debating the limits of
Critical Scholarship" Paper presented at the annual meeting of
the International Studies
Association, Hilton Hawaiian Village, Honolulu, Hawaii,
(referred on 22/03/2008), available from
http://www.allacademic.com/meta/p72159_index.html
Copyright of Annals of the University of Oradea, Economic
Science Series is the property of Annals of the

University of Oradea, Economic Science Series and its content
may not be copied or emailed to multiple sites or
posted to a listserv without the copyright holder's express
written permission. However, users may print,
download, or email articles for individual use.
Integrating Multiple
Qualitative Research
Methods (or Avoiding the
Precariousness of a One-
Legged Stool)*
Amy L. Hall
The George Washington University
Ray C. Rist
The World Bank
ABSTRACT
This article suggests that market research is enhanced when
multiple qualitative methods are combined in a triangulated
approach to examining marketing questions. The article begins
with
a case study that illustrates how a qualitative study can, by
itself, be
used as a basis for making marketing decisions, when
methodological triangulation is employed to guard against
unreliable or invalid results. Following the case study is a
presentation of the three legs of the qualitative stool—
interviews

(including focus groups), observations, and document analysis.
Each
method is described, with a discussion of inherent strengths and
weaknesses. The article ends with an argument for increasing
the
use of integrated, triangulated qualitative studies in the field of
market research. © 1999 John Wiley & Sons, Inc.
*The views expressed here are those of the authors, and no
endorsement by the World Bank Group
is intended or should be inferred.
Psychology & Marketing Vol. 16(4):291-304 (July 1999)
' © 1999 John Wiley & Sons, Inc. CCC 0742-6046/99/040291-
14
291
A CASE STUDYi
In the early 1990s, a leading bicycle helmet manufacturer,
BikeSafe,
found itself losing market share in the bicycle helmet industry.
BikeSafe
had long prided itself on a no-nonsense approach to bicycle
safety. The
company produced a nigged helmet, shaped for maximum
protection
and formed of white plastic for greater visibility. BikeSafe sold
helmets
in sizes suitable for adults and older children. BikeSafe enjoyed
a solid
reputation in the industry for proAdding a safe product at an
affordable

price, but now their competitors were doing better while their
business
was stagnant. What had changed? The CEO of BikeSafe called
on his
marketing department to provide some answers.
The market researchers at BikeSafe did what many market
research-
ers do: They organized several focus groups. One focus group
consisted
of customers who had previously purchased BikeSafe helmets.
The sec-
ond group was made up of a dozen local high school students
and a third
group included members of a biking club. The first group,
previous cus-
tomers, reported that they were quite satisfied with their bicycle
hel-
mets and would consider buying another BikeSafe product in
the future.
The group of students yielded little information, reporting that
bicycle
helmets were not "cool" and that they preferred not to wear
helmets at
all. The final group, the biking club members, generally focused
on
safety factors and seemed impressed by the safety rating of
BikeSafe's
helmets. Their discussion, however, quickly deteriorated into a
conver-
sation about the group's upcoming ride. In the end, the
reseai'chers
knew only a little more than when they started, and clearly not
enough.
They did learn that the members of the focus group that had
previously

purchased BikeSafe helmets were generally satisfied, but that
did not
address the question of why they were not attracting as many
new buy-
ers. What went wrong? Why did the focus groups not jdeld
better re-
sults?
The focus groups conducted by the BikeSafe researchers were
subject
to some common weaknesses of this qualitative method. One
researcher
observed that the group consisting of previous purchasers
included
mostly retired people. The researchers realized too late that
because the
focus group was held during a workday morning, they excluded
most
working-age people. The high school group probably faced the
common
problem of peer pressure. Once a single, loud student voiced the
opinion
that it was not acceptable to wear a bicycle helmet, no other
student
was willing to go against the tide. Finally, the researcher
conducting
the biking club's focus group was inexperienced with the
technique and
was not able to keep the discussion focused. Although the focus
groups
failed to provide answers to BikeSafe's problems, they did yield
the im-
'The case study has been created by the authors. It does not
refer to actual persons or an existing
company.

292 HALL AND RIST
portant information that the purchasers participating in the first
focus
group were generally satisfied. The researchers decided to focus
next on
how to reach more purchasers.
The researchers decided to reach into the qualitative tool kit for
an-
other technique. They decided to do some document analysis.
BikeSafe
included a survey on their warranty cards that approximately
50% of
purchasers returned. They began by examining these surveys.
What
became immediately apparent was that for the last few years,
most of
their helmets had been purchased by older consumers. They
were sell-
ing very few helmets to young adults and teenagers.
Additionally, the
factor most commonly cited for deciding to buy their product as
opposed
to another brand was the safety reputation of BikeSafe's
helmets. The
researchers concluded that the company should continue to
emphasize
the safety features of their helmets, but needed to find a way to
recap-
ture younger consumers.
Reviewing advertisements in several biking and sports

magazines
from the previous 6 months revealed some additional answers.
The
BikeSafe researchers noticed that most helmet ads of their
competitors
were aimed at the young consumer. The ads often showed young
adults
engaged in mountain biking and other adventure sports.
Previously,
BikeSafe had exclusively targeted the biking community and
casual bi-
cycle riders. Examination of their competitors' ads revealed that
they
were missing those consumers who purchased a helmet to wear
while
in-line skating, skateboarding, and playing other dangerous
sports.
They also noticed the snazzy colors and sleek designs of their
competi-
tors' helmets. The BikeSafe researchers had a new idea about
the au-
dience their own ads should be targeting. But before investing
in a new
advertising campaign, they wanted to gather additional
information. An
important thing the ads could not reveal was how popular these
other
sports had become and how many new potential consumers this
repre-
sented. The researchers pulled out that last qualitative tool—
observa-
tion.
The BikeSafe researchers decided to get a first-hand look at
how pop-
ular sports such as in-line skating and skateboarding had

become. They
chose several parks in the area and on three consecutive
weekends made
observations about helmet use. They paid particular attention to
how
many people rode bikes and what types of bikes were ridden,
and how
many people were skating or skateboarding. Of all these they
noted the
approximate ages of the people, whether or not they were
wearing a
helmet, and if a helmet was worn, what it looked like. At the
end of the
three weeks, they concluded that their initial impressions from
studjdng
the ads had been correct; more people were engaging in in-line
skating
and skateboarding, and the age range of helmet wearers seen
most fre-
quently was approximately 20-40 years old. Something they
concluded
from their observations that had not been revealed by either of
the other
two techniques was that more small children were wearing
bicycle hel-
INTEGRATING METHODS 293
mets. BikeSafe had never targeted small children as consumers;
they
did not even make hicycle helmets to fit small children, hecause
children
rarely rode hicycles that were hig or fast enough to really cause
injury.

Yet it was ohvious from their ohservations that many small chil-
dren riding bicycles with their parents were indeed wearing
safety
helmets.
The researchers finished up their project hy going hack to
interviews.
They returned to their focus group lists and invited some of the
people
from the focus groups in for individual interviews. They made
sure to
schedule some interviews on weekends so that recent purchasers
who
worked during the week would he included. The interviewees
supported
their conclusions that they should expand their marketing to
include all
age groups and should design a marketing campaign selling
their hel-
mets as sports helmets, suitable for a variety of sports, rather
than
simply bicycle helmets. Interviewees also supported the
conclusion
that consumers want safety with style—colorful helmets with
sleek
designs.
BikeSafe learned from their research. Within 2 years, BikeSafe
had
regained their market share of the helmet industry. They had
intro-
duced a new line of helmets that were designed with safety and
style in
mind. The helmets were produced in a greater variety of sizes to
fit ages
"one to ninety-nine," as their new ads stated. An advertising

campaign
had combined an emphasis on style with the company's already
strong
reputation for safety. The ads showed people in several age
ranges wear-
ing BikeSafe helmets while cycling, skating, and skateboarding.
There
were special ads focused on teenagers.
The company profited from their research. The researchers
learned
as well. They realized that focus groups alone cannot provide a
full,
accurate picture of a market situation. Like a one-legged stool,
infor-
mation gleaned from the initial focus groups would have led to
shaky
conclusions; in their case, there simply was not enough
information for
strong conclusions. By adding other research techniques, like
adding
legs to a stool, they firmed up their conclusions and were on
much surer
ground when they presented their results to BikeSafe's CEO.
The com-
pany benefited greatly as a result.
Market researchers have long relied on focus groups as the
mainstay
of their qualitative techniques. Occasionally other techniques
might be
used as well, but it is still rare that market researchers rely on a
qual-
itative study that combines all three of the legs of qualitative
research
into a triangulated approach. As the researchers from BikeSafe

learned,
when the three techniques are combined, they can provide valid,
reliable
conclusions upon which to base marketing decisions. The
balance of this
article aims to (a) demonstrate that there is an array of
qualitative
methods that can be applied to marketing questions and
problems, and
(b) outline strengths and weaknesses of each method.
294 HALL AND RIST
THE QUALITATIVE TOOL KIT
Every good craftsman likes having good tools with which to
work. This
article presumes that marketing researchers are no different, and
sug-
gests another look at the many tools available in the qualitative
meth-
ods tool kit, ones that may not be currently used to tbeir fullest.
Mar-
keting researchers have long been using a few select qualitative
tools,
namely, focus groups and some observation methods. But these
are
largely used in an exploratory fashion and are not often
integrated into
a multimethod approach to discovery.
For anyone using a new tool, or one with which they are only
slightly
familiar, it is important to become acquainted with any

guidelines per-
taining to that particular tool. True, rudimentary tools need only
rudi-
mentary instructions; but more sophisticated tools need more
careful
discussion of their use and abuse. No less the case with the
careful
application of qualitative metbods to marketing research.
Furtber, it is
important to understand a central reality of qualitative
researcb—its
strength lies in tbe concurrent use of multiple tools. Tbus, it is
not ap-
propriate to presume that if one knows how to use one or more
tools,
one knows bow to conduct qualitative researcb. It is the
interactive and
simultaneous use of metbods (and an analysis that takes account
of tbis
interaction) that is unique to qualitative researcb. Stated
differently,
the key to good qualitative research is metbodological
triangulation.
Triangulation is like using a three-legged stool. Remove one
leg, and
tbe stool is mucb less reliable. It is likely to wobble and
collapse under
one's weight. Remove a second leg and tbe stool becomes very
precarious
indeed. So it is witb qualitative research. Tbe three legs of
qualitative
research methods are interviewing, observation, and document
anal-
ysis.
With tbe use of a multimetbod approach tbat combines these

three
major data-collection strategies employed in qualitative
research, one
bas a basis for drawing conclusions witb strong validity.
Relying on only
one or two of tbe methods leaves one subject to questionable
validity.
LeCompte and Preissle (1993) note tbat tbe researcher can
"pinpoint
the accuracy of conclusions drawn by triangulating with several
sources
of data" (p. 48). Tbe underlying assumption here is
straightforward—
using multiple data sources can deepen our understanding and
bence
is advantageous in comparison to using a single method.
The conventional use of the term qualitative methods in
marketing
has come to imply tbe use of small-group interviews, generally
focus
groups. Tbe emphasis is on only one leg of the stool. Limiting
oneself to
focus groups misses the array of other metbods tbat are
contained in
the qualitative metbods tool kit. Marketing researcb will benefit
from
an expanded use of qualitative methods in a multimethod
approach to
learning about market demand, brand loyalty, consumers'
preferences

and needs, consumer behavior, and why it is that what
consumers say
is often not what they do.
A study that exemplifies the use of these three techniques in
con-
junction with one another was recently published in the Harvard
Busi-
ness Review. In "Making Differences Matter: A New Paradigm
for Man-
aging Diversity," Thomas and Ely (1996) examined diversity in
the
workplace and based their analyses on "interviews, surveys,
archival
data, and observation" (p. 85). Their use of multiple methods
facilitated
the exploration of why diversity is so difficult to manage, why
it is sel-
dom honestly discussed, why the disconnects between values
and be-
haviors caused dysfunctional performances in the organizations,
and
why official pronouncements within the organizations were
ignored if
not outright mocked. The basis for their development of a new
approach
to managing diversity was grounded in the multiple qualitative
data
sources they employed. It is hard to imagine that a sensitive
topic like
managing diversity could be fully understood with just
interviews, or
just the review of official documents in the organizations, or
just watch-
ing behaviors in the cafeteria or in the boardrooms.

THE LOGIC AND RATIONALE OF TRIANGULATION
The term triangulation is taken from land surveying, where the
sur-
veyor uses the sightings of two landmarks to locate his own, the
third,
position. The logic of triangulation is that
no single method ever adequately solves the problem of rival
causal
factors . . . Because each method reveals different aspects of
empir-
ical reality, multiple methods of observations must be
employed. This
is termed triangulation. I now offer as a final methodological
rule the
principle that multiple methods should be used in every
investigation.
(Patton, 1990, p. 187)
Denzin (1978) identifies four types of triangulation: data
triangula-
tion, investigator triangulation, theory triangulation, and
methodolog-
ical triangulation. Data triangulation refers to the gathering of
data at
different points in time and from different sources. Investigator
trian-
gulation is the use of multiple researchers to study the same
research
question or the same setting, presuming that different
researchers will
bring different perspectives, thinking, and analysis to the table,
thus
strengthening the final assessment. Triangulating theory stresses
that

the research should examine the phenomenon from different
theoretical
vantage points to see which would be the most robust in helping
to
clarify and explain what has been studied. Methodological
triangulation
refers to the use of multiple methods to gain the most complete
and
detailed data possible on the phenomenon.
296 HALL AND RIST
Any strategy of triangulation that adds a system of
theoretical/meth-
odological checks and halances to a study lends strength to that
study.
As Patton (1990) observed, "studies that use only one method
are more
vulnerable to errors linked to that particular method . . . than
studies
that use multiple methods in which different types of data
provide cross-
data validity checks" (p. 188). It is our view that a key means to
strengthen market research is to expand the range of
methodologies
used for the gathering of both factual and perceptual
information, and
our focus here is on methodological triangulation.
BALANCING STRENGTHS AND WEAKNESSES
Each of the three major types of data collection (interviews,
observation,
and document analysis) has unique strengths and weaknesses. It

is in
the combination of these strengths and in the compensation for
the
weaknesses that the intellectual and methodological power of
qualita-
tive research becomes apparent. In returning to the roots of
triangula-
tion, the use of multiple methods gives a more precise and
accurate fix
on the research question than does any one method by itself.
A brief summary of the strengths and weaknesses of the three
key
methodological approaches suggests why triangulation is so
central.
First, the use of focus groups, perhaps the central interviewing
tech-
nique of the market research community, "allows people to
discuss their
true feelings, anxieties, and frustrations as well as the depth of
their
convictions in their own words" (Zikmund, 1982, p. 123).
Accurate and
truthful responses in these sensitive areas are hard to gather in
any
research effort, and trjdng to gain them from paper and pencil
tests is
woefully inadequate. With regard to observation, Aaker, Kumar,
and
Day (1995) suggested that "observation may be the least
expensive and
most accurate method of collecting purely behavioral data such
as in-
store traffic patterns or traffic passing a certain point on a
highway
system" (p. 189). Observation also allows the researcher to

personally
see and verify that a particular behavior or interchange did or
did not
actually happen—independent of the respondent's perception of
that
event. The third tool, document analysis, can be used to
"analyze written
material into meaningful units, using carefully applied rules"
(Aaker et
al., p. 190). What individuals say is not always what they do.
Thus there
is a need to turn to this additional source of information—the
written
account—which may come in the form of memos, letters,
personal bi-
ographies, diaries, legal briefs, annual corporate reports, or
other doc-
uments.
With the strengths of these three methodological approaches,
how-
ever, come weaknesses. For example, a shortcoming of focus
groups is
that "without a sensitive and effective moderator, a self-
appointed par-
ticipant may dominate the session" (Zikmund, 1982, p. 127).
Relying
only on interviewing leaves the researcher vulnerable to
selective recall,
self-delusion, perceptual distortions, memory loss from the
respondent,

and subjectivity in the researcher's recording and interpreting of
the
data. Relying only on the second tool, observation, means that
the ac-
tor's understanding and motivation for behavior may not be
understood
by the observer. Inference in this circumstance as to why
someone did
as they did is risky and often just plain wrong. Document
analysis is
also subject to error, as it relies solely on the researcher's
interpretation
of what is in the document being analyzed. Conducting research
when
one's only source of information is historical documents (pity
the poor
historian) involves all the problems of selective deposit of the
material
(not all that is know is written down, let alone then saved),
selective
survival (remember the 18-minute gap in the Nixon White
House tapes),
and selective retrieval (not all that survives is found.)
INTERVIEWS
Interviews can be divided into three basic categories: the
individual in-
terview, the small-group or focus-group interview, and the
large-group
interview. Each of these techniques has strengths and liabilities,
ben-
efits and costs. The emphasis here will be on the first and
second of the
three types, those of individual and small-group or focus-group
inter-

viewing, because these are the most applicable to consumer
research.
The Individual In-Depth Interview
Aaker et al. (1995) described individual in-depth interviews as
"inter-
views that are conducted face to face with the respondent in
which the
subject matter of the interview is explored in detail" (p. 176).
Calder
(1994) noted several advantages of the individual interview,
including
those related to the amount of in-depth information obtained
and the
moderator's control of interview timing and related variables.
Another
advantage of the individual interview is that there is not the
potential
problem of group conformity. People may be more spontaneous
in an
individual interview. There is also the benefit that comes from a
face-
to-face interaction; up-close observation of body language, tone
of voice,
reaction to distractions, reaction to the interview setting, and
personal
appearance are all contextual factors that can assist the
interviewer in
analyzing the interview data. One-on-one interaction may also
sensitize
the interviewer to the existence of resistance to questions, and
efforts
to be evasive. There is also the ability to follow up and probe
incomplete,
vague, or ambiguous responses.

The primary weaknesses of the individual interview technique
are
logistical in nature. Individual interviews are both time
consuming and
expensive. A group interview of 20 consumers may require one
moder-
298 HALL AND RIST
ator and perhaps 2 hours of time. Individual interviews of the
same 20
people would require considerably more resources. In some
cases, the
information obtained from tbe individual interview may be
worth the
expense, but often the same information can be obtained
through the
less expensive foeus-group method.
A second shortcoming of the individual interview is that often
the
consumer is more comfortable expressing his or her opinions in
a sup-
portive group environment than in an individual interview.
Individual
interviews can create a feeling of being isolated or singled out
by the
interviewer. Another possible shortcoming stems from the
possibility of
the interviewer and interviewee being in conflict or disliking
one an-
other. In such cases, it is not likely that the person being
interviewed

will participate in an open and cooperative fashion.
The Focus Group
The focus group is probably the most widely used qualitative
technique
in marketing research. In fact, "qualitative research is
epitomized in
marketing practice by the focus group interview" (Calder, 1994,
p. 50).
The focus or small-group interview is essentially a data-
gathering tech-
nique where the interviewer/moderator focuses the attention of
a group
of 7-12 persons on a deflned set of topics with the intent being
for the
members of the group to discuss the issues among themselves.
The goal
is to learn the views and values of those involved on the topics
pre-
sented. In general, a group is selected and a moderator begins a
discus-
sion, allowing the group participants to pick up the flow and,
hopefully,
express themselves as freely as possible. The focus-group
process can
alter slightly depending on the speciflc nature of the group.
Focus groups share a number of strengths, including ones
related to
cost and time. Generally speaking, focus groups are clearly
lower on
both counts in comparison to individual interviews.
Additionally, being
in a group compels the individuals to focus on the topic at hand
if they

are to communicate appropriately in the situation. Another
strength of
focus-group analysis is that the voice of a number of persons on
a topic
may lend more credibility to a point of view than that which
flows from
an equal number of individual interviews. On a related note, the
inter-
actions in the group can 3deld interesting insights as well. A
report based
on a focus group of the phenomenological approach is also
likely to de-
scribe the group discussion without engaging in extensive
technical or
scientiflc jargon. One listing of the advantages of focus groups
included:
sjTiergism, snowballing, stimulation, security, spontaneity,
serendipity,
specialization, scientiflc scrutiny, structure, and speed
(Zikmund, 1982,
pp. 124-126). And although they do not start with the letter s,
the mat-
ters of cost, time, logistics, and coverage could well be added to
Zik-
mund's list.
There are, however, three important disadvantages to the focus-
group
methodology. First, the quality of the effort rests heavily on the
shoul-
ders of the moderator. Second, focus groups present a sampling

problem.
How to select 7-12 people, according to what criteria, with what
avail-
ability, with what previous experience in focus groups, and with
what
relations to others in tbe group are all matters that can cause tbe
dy-
namics of tbe group to become dysfunctional and tbe data less
tban
useful. Tbird, tbere is no ability to generalize from tbe material,
nor is
it easy to quickly summarize tbe discussions. A 90-minute focus
group
can easily generate 30 pages of typed notes. Working tbrougb
all tbis
material to reacb tbe core conclusions is difficult and time
consuming.
In addition, using tbe focus group as tbe single source of data
can lead
to erroneous conclusions.
Clearly, botb focus-group interviews and individual interviews
bave
strengtbs and weaknesses tbat tbe researcber must weigb against
tbe
expected results. Wbetber one or botb interview tecbniques are
used in
a study, tbe researcber sbould consider bow to combine
interviews witb
metbods from tbe otber two categories presented bere:
observation and
document analysis.
OBSERVATION
A second major type of qualitative researcb metbodology is

observation.
Zikmund (1982) describes two types of observation
tecbniques—unob-
trusive observation and visible observation. Wben one is
performing
unobtrusive observation, tbe intent is to collect data on bebavior
or in-
teractions witbout tbe subject's knowledge. Tbis approacb migbt
be
most appropriate wben tbe bebavior being observed would make
tbe
subject self-conscious, tbus leading tbe subject to cbange bis or
ber be-
bavior. It is also appropriate in tbose situations wbere tbere is
no viable
alternative. In tbe visible-observation case, tbe subject is aware
of tbe
observation taking place. Anotber way to cbaracterize
observations is
participant or nonparticipant. Participant observation requires
tbe ob-
server to get close enougb to tbose being observed to acbieve a
level of
comfort witb tbem, tbus allowing unimpeded observations and
tbe re-
cording of information. For example, a researcber migbt join a
consumer
rigbts organization to get an insider's look at bow participants
feel and
interact. In nonparticipant observation, tbe researcber remains
apart
from tbe subject(s) being studied, in a manner similar to
unobtrusive
observation. Witb all of tbese metbods, tbe researcber observes
ongoing
bebavior of tbe subject(s), interpreting meaning from wbat is

observed.
Zikmund (1982) suggests six types of bebavior can be observed.
First,
tbe researcber is able to witness pbysical actions. Do sboppers
stop to
look at sale ads if tbey are posted at tbe entrance of a store?
Verbal and
expressive bebavior can also be noted. How do sboppers interact
witb
store personnel wben a sale item is out of stock? Spatial
relations and
300 HALL AND RIST
locations can provide important information about consumers,
as can
temporal patterns. Do posted sale ads get more attention if
posted out-
side the store, or if more signs are posted in aisles? How much
time do
shoppers spend examining ads if they are posted outdoors, as
opposed
to inside the store? The sixth area of observation includes
verbal re-
cords, or the contents of advertisements. In this article, the
latter ob-
servational technique is addressed under document analysis.
As with interviewing, observation has both strengths and
weaknesses
associated with it. Observation can be a relatively inexpensive
yet ac-
curate means of gathering purely behavioral data. A major

strength of
the observation method is that because the data from the
observations
are recorded as they occur, there is less distortion than in
interviews,
where an important strategy is to encourage recall in the person
being
interviewed. In an interview, the researcher generally has to
rely on the
memory and honesty of the respondent. In a group interview,
this can
be even more critical if a respondent fears the reaction of others
in the
group. Observation, especially unobtrusive observation, negates
this
concern. The technique is not without weaknesses, however.
There are several disadvantages associated with observation
tech-
niques in general. First, behavior may take place out of sight. A
dissat-
isfied customer, for example, may not express much emotion
while in
the store, a public area, but may become very angry upon
arriving home.
Because observers can seldom consistently be in the right place
at the
right time, the observational picture on any phenomenon is
always in-
complete. Third, the researcher must accurately record the data,
with-
out injecting subjectivity into the recording. This can be a
difficult task.
One strategy for guarding against this in an emotionally charged
area
is for researchers to put all their own reactions, emotions, and

feelings
into a separate section at the end of the observational record.
Fourth,
the researcher must accurately interpret the meaning of the
observa-
tions. There are multiple interpretations often possible from
even a sim-
ple gesture or sigh. Getting the interpretation right is not easy.
Partic-
ipant observation carries with it the possible disadvantage that
subjects
may change their behavior as a result of the observation. An
ethical
consideration that itself could be the subject of another article
concerns
consent in observational research. Observations and interviews
are nat-
urally complementary as they allow the researcher to explore
the inter-
relation of words and deeds. Adding a third technique,
document anal-
ysis, allows for triangulation and thus still stronger grounds for
valid
interpretation.
DOCUMENT ANALYSIS
"Human behavior often leaves traces, and the study of those
traces can
tell us a lot" (Bernard, 1995, p. 332). Key among these are the
records.

documents, letters, newspapers, television programming, books,
diaries,
and e-mail messages that trace peoples' lives. This study of the
traces
we leave behind is termed document analysis. Document
analysis aims
to take information stored in such sources as noted above and
abstract
from it key themes, strategies, values, messages, and the like.
Coding
schemes are frequently used to systematically record and then
analyze
the collected information.
Document analysis can provide a wealth of information to
marketing
researchers. Individuals keep records, businesses keep records,
orga-
nizations keep records, and governments keep records. Maxine
Margolis
used advertisements in the Ladies Home Journal to trace "the
trans-
formation of the middle-class homemaker from an employer of
servants
to a direct user of household products" (Bernard, p. 341), surely
a topic
of interest to marketers. Document analysis, like observation,
can be
inexpensive. It is also unobtrusive, so it is not subject to
selective mem-
ory or social bias. But not all documents are equal. Some have
more
credibility and trustworthiness than do others.
Four categories of contemporary (as opposed to historical)
documents

can be suggested, moving from higher to lower levels of
reliability. They
are (a) contemporary records, (b) contemporary individual
accounts, (c)
reconstructed materials, and (d) reports contained in secondary
docu-
ments. For market researchers, the difficulty comes in that the
two cat-
egories most often used [(b) and (d)] are vulnerable to
distortion and
misunderstanding. But as noted, in conjunction with other forms
of
qualitative data discussed earlier, the use of documents is an
integral
part of a more comprehensive and thorough understanding of
the mo-
tivations and beliefs of individuals.
In the first category are contemporary records, such as
government
legislation and regulations, business and legal papers, office
memoran-
dums, minutes of meetings, and notes to the file. Materials in
this cat-
egory are almost always dated, and will state who wrote the
document.
They are often written very close to the actual happening of an
event.
Contemporary records are often written by one or more
participants, for
the purpose of keeping track of important issues, ideas, or
agreements,
so that they may be referred to and used by those same
participants in
later actions. Many materials in this class are reviewed for their
accu-

racy by all concerned parties. This process of verification may
be quite
formal, as in the reading and signing of legal documents, the
approval
of minutes of meetings, or in the review of important office
memoranda
before they are issued.
The second class of materials are contemporary individual
accounts,
such as what one finds in letters, diaries, personal notes from
meetings
or discussions, or notes in one's personal files. Materials here
are gen-
erally as contemporary as those in class one, but they are
prepared by
only one individual without the review, consultation, or
approval of oth-
302 HALL AND RIST
ers. Because of the personal, even potentially biased nature of
this ma-
terial, the credibility is less than class one.
The third class of materials are those that are constructed after
the
event and separated from that event by some period of time.
Recon-
structing a complete and sequenced description of the actions,
conver-
sations, asides, proposals, and so on of even two persons in a
conver-
sation is difficult; trying to capture a large group discussion or

the
activities of a board meeting is even more so. There is also the
matter
that with time, memory recedes and becomes less precise. The
old say-
ing that "We remember 10 percent of what we read, 20 percent
of what
we hear, and 30 percent of what we see" may not be precise in
the per-
centages, but the implications of reduced recollection are real
for the
trustworthiness of any document prepared some time after the
fact.
The fourth class of documents are those that contain elements of
con-
temporary reports (and not contemporary records as found in
class one).
Reports might be those from a state or the national census
bureau.
Newspaper accounts, surveys and opinion polls, and
investigative ac-
counts also fall into this category. Material in this category may
be gen-
erated on the one hand by statistical sampling (surveys or polls)
and on
the other by highly individualistic requirements (investigation
or news-
paper story). In either case, it is hard to know what meaning to
make
of the information, for it is aggregated, summarized, and
sometimes
reduced to statistical figures, leaving interpretation to a
probability
analysis.

It is evident from this discussion that document analysis does
have
pitfalls, based on the quality of the document being analyzed.
But there
is another domain of problems that also needs to be considered.
Specif-
ically, document analysis invariably involves the coding of the
material
being examined. Who makes up the code, and how closely it is
applied
to the material, can be problematic. Coding and analysis are
also subject
to cultural variability, which can invalidate findings if not
considered
(Bernard, 1995). Thus, use of documents in qualitative market
research
requires careful attention to the source of the document, the
intent in
using the document, and the coding scheme designed to classify
the
information within it.
BRINGING TOGETHER INTERVIEWS, OBSERVATIONS,
AND DOCUMENT ANALYSIS
The market researcher who employs multiple qualitative
research
methods is the researcher who is taking out an insurance policy
on the
accuracy of the answer to the research question. The basic
premise of
this article is not that more research is inherently good. We are
not
discussing moral precepts. Rather, the proposition is that a
coordinated.

triangulated strategy emplo3dng different approaches to the
study of a
research question can heighten the certainty with which that
question
is answered. The design and execution of research studies
inevitably
involve trade-offs among different constraints, for example,
scope, cost,
time, specificity, level of desired accuracy, and so on. The
emphasis here
is that when researchers tend to put all their qualitative
methodological
eggs in one basket (e.g., focus groups), there are risks. But
accuracy and
reliability can be enhanced by expanding the range of
qualitative re-
search methods used in any study. These methods exist and are
widely
employed in other sectors of qualitative research. Market
researchers
also have the choice to use these methods or not.
REFERENCES
Aaker, D. A., Kumar, V., & Day, G. S. (1995). Marketing
research (5th ed.).
New York: John Wiley & Sons.
Bernard, H. R. (1995). Research methods in anthropology:
Qualitative and
quantitative approaches (2nd ed.). Walnut Creek, CA: AltaMira
Press.

Calder, B. J. (1994). Qualitative marketing research. In R. P.
Bagozzi (Ed.),
Principles of marketing research (pp. 50-72). Cambridge, MA:
Basil Black-
well Ltd.
Denzin, N. K. (1978). The research act: A theoretical
introduction to sociological
methods (2nd ed.). New York: McGraw-Hill Book Company.
Denzin, N. K., & Lincoln, Y. S. (Eds.) (1994). Handbook of
qualitative research.
Thousand Oaks, CA: Sage.
Eisner, E. W. (1991). The enlightened eye: Qualitative inquiry
and the en-
hancement of educational practice. New York: Macmillan.
Fontana, A., & Frey, J. H. (1994). Interviewing: The art of
science. In N. K.
Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative
research (pp. 3 6 1 -
376). Thousand Oaks, CA: Sage.
LeCompte, M. D., & Preissle, J. (1993). Ethnography and
qualitative design in
educational research (2nd ed.). New York: Academic Press.
Patton, M. Q. (1990). Qualitative evaluation and research
methods (2nd ed.).
Newbury Park, CA: Sage Publications.
Thomas, D. A., & Ely, R. J. (1996, September-October) Making
differences
matter: A new paradigm for managing diversity. Harvard

Business Review,
74, 79-90.
Zikmund, W. G. (1982). Exploring marketing research (4th ed.).
Orlando, FL:
Dryden Press.
Correspondence regarding this article should be sent to: Ray C.
Rist, The World
Bank, 1818 H. St. NW, Washington, DC 20433
([email protected]).
304 HALL AND RIST

1279 QUALITATIVE RESEARCH METHODS A COMPARISON BETWEEN .docx

Recommended

Recommended

More Related Content

Similar to 1279 QUALITATIVE RESEARCH METHODS A COMPARISON BETWEEN .docx

Similar to 1279 QUALITATIVE RESEARCH METHODS A COMPARISON BETWEEN .docx (14)

More from moggdede

More from moggdede (20)

Recently uploaded

Recently uploaded (20)

1279 QUALITATIVE RESEARCH METHODS A COMPARISON BETWEEN .docx