MBA 5652, Research Methods 1 Course Learning Outcomes.docx

MBA 5652, Research Methods 1
Course Learning Outcomes for Unit III
Upon completion of this unit, students should be able to:
1. Discuss the importance of research-based decision making in
a company.
1.1 Summarize how qualitative methods of research aid in
decision making of a discipline.
4. Describe how qualitative research methods are used.
4.1 Describe specific qualitative methods that could be used
within a discipline.
4.2 Explain the effectiveness of qualitative methods of research
within a discipline.
7. Appraise current research tools used in business and industry.
Reading Assignment
Hall, A. L., & Rist, R. C. (1999). Integrating multiple
qualitative research methods (or avoiding the
precariousness of a one-legged stool). Psychology & Marketing,

16(4), 291-304. Retrieved from
https://libraryresources.columbiasouthern.edu/login?url=http://s
earch.ebscohost.com/login.aspx?direc
t=true&db=bth&AN=11580160&site=ehost-live&scope=site
Hegney, D., & Chan, T.W. (2010). Ethical challenges in the
conduct of qualitative research. Nurse
Researcher, 18(1), 4-7. Retrieved from
earch.proquest.com.libraryresources.c
olumbiasouthern.edu/docview/817118115?accountid=33337
Milena, Z. R., Dainora, G., & Alin, S. (2008). Qualitative
research methods: A comparison between focus-
group and in-depth interview. Annals of the University of
Oradea, Economic Science Series, 17(4),
1279-1283. Retrieved from
earch.ebscohost.com/login.aspx?direc
t=true&db=bth&AN=48755863&site=ehost-live&scope=site
Tolhurst, H., & Dean, S. (2004). Using teleconferencing to
enable general practitioner participation in focus
groups. Primary Health Care Research & Development, 5(1), 1-
4. Retrieved from
earch.proquest.com.libraryresources.c
olumbiasouthern.edu/docview/222746946?accountid=33337

Unit Lesson
The unit lessons for this course are presented through
interactive presentations. Each slide has audio that
accompanies it. Once the narration for each slide is complete,
the presentation will automatically advance to
the next slide. If you would like to stop the presentation on any
slide, click the pause button on the bottom left
hand side of the presentation. When you are finished watching
the presentation, simply close the browser
window.
To view the lesson, click on the link below:
http://columbiasouthern.adobeconnect.com/mba5652_unitiii/
UNIT III STUDY GUIDE
Qualitative Research
Methods
earch.ebscohost.com/login.aspx?direct=true&db=bth&AN=1158
0160&site=ehost-live&scope=site
earch.proquest.com.libraryresources.columbiasouthern.edu/docv
iew/817118115?accountid=33337

http://columbiasouthern.adobeconnect.com/mba5652_unitiii/
MBA 5652, Research Methods 2
UNIT x STUDY GUIDE
Title
The presentation contains closed captioning if needed. To learn
how to enable this feature, click here to
view a guide on navigating and using the presentation. To print
the presentation with a transcript for each
slide, click here.
Suggested Reading
This video will provide a more in-depth look at qualitative
research, methods of this type of research, and
case studies involving qualitative research.

Axman, L. [TSNRP]. (2012, May 7). Qualitative research
methods, case study research [Video file]. Retrieved
from https://www.youtube.com/watch?v=6Sg0M8Fn99E
This one-hour video of a lecture given by Dr. Roberta E.
Goldman was presented during the Harvard Catalyst
lecture series. This video will explain different qualitative
methods that can be used in public and primary
health care studies.
Brown University. (2011). Qualitative research for public health
and clinical investigation [Video file]. Retrieved
from https://www.youtube.com/watch?v=M7PCn_zId4Q
https://online.columbiasouthern.edu/CSU_Content/courses/Busi
ness/MBA/MBA5652/15M/Adobe_Presenter_guide.pdf
https://online.columbiasouthern.edu/CSU_Content/courses/Busi
ness/MBA/MBA5652/15M/Unit_III_Lesson_PDF.pdf
https://www.youtube.com/watch?v=6Sg0M8Fn99E%20
https://www.youtube.com/watch?v=M7PCn_zId4Q
Primary Health Care Research and Development 2004; 5: 1–4
Networking
Using teleconferencing to enable general
practitioner participation in focus groups

Helen Tolhurst School of Medical Practice and Population
Health, Faculty of Health, University of Newcastle,
Australia and Sarah Dean Faculty of Medicine, University of
Sydney, Australia
Focus groups are used increasingly in general practice research,
especially where a
qualitative method is required to answer exploratory questions.
Recently teleconfer-
encing has been used to conduct focus groups in research on
rural general prac-
titioners, where distance makes face-to-face focus groups dif�
cult and where anon-
ymity of participants is important. This paper discusses
advantages and
disadvantages of using teleconferencing as a means of
undertaking focus groups and
provides examples of research where this has been used. We
conclude that focus
groups, conducted using teleconferencing, are an appropriate
method in research on
general practitioners, and can be particularly useful, when
conducting research in
rural and remote areas, and when anonymity of participants is
desirable.
Key words: focus group; general practice; research; rural;
teleconference
Introduction
Focus groups have become a common method
in general practice research (Cho et al., 1995;
Schattner et al., 1993). Research has shown gen-
eral practitioners often mention time constraints as
a barrier to their participation in research studies

(Gray et al., 2001; Jowett et al., 2000). As the par-
ticipation in face-to-face focus groups is a time
consuming activity for participants it would be
expected that minimizing the time commitment
expected of participants would assist in over-
coming this barrier. This is particularly the case
when the research is being conducted in rural and
remote areas where attendance at face-to-face
focus groups involves travel to distant locations. A
Norwegian study showed that female general prac-
titioners (GPs) regarded their family commitments
as a barrier to participation in research (Forde and
Evenson, 1991). Teleconference focus groups are
Address for correspondence: Dr Helen Tolhurst, Rural Research
Fellow, Discipline of General Practice (Newbolds), School of
Medical Practice and Population Health, Faculty of Health, Uni-
versity of Newcastle, University Drive, Callaghan, NSW 2308,
Australia. Email: [email protected]
ÓArnold 2004 10.1191/1463423604pc190xx
a method which can provide GPs with the opport-
unity to participate without the need to travel to
a particular location and which could potentially
facilitate female GPs’ participation in some
research while also meeting their family commit-
ments.
Recently the focus group method has been
adapted to use teleconferencing for health care
research (Appleby et al., 1999; Tolhurst et al.,
1997; Tolhurst et al., 1999; Tudiver et al., 2001;
White et al., 1994; White and Thomson, 1995). We
discuss the use of teleconference focus groups for
research on GPs.

Teleconference focus groups as
research methodology
Teleconferencing is a system that joins three or
more participants in different locations through a
telephone system (Kuramoto and Dean, 1993).
Telephone groups have been used for group
support (Macintosh, 1981; Wildsoet et al., 1996),
education (Macintosh 1981; Wildsoet et al., 1996),
group interviews, organizational tasks, contacts
before and after face-to-face meetings (Schopler
2 Helen Tolhurst and Sarah Dean
et al., 1998), and in market research (Garigliano,
1998). Teleconference focus groups are similar to
face-to-face focus groups, but there are speci� c
advantages and disadvantages of using them. Parti-
cipants are recruited in the same way as they would
be for face-to-face focus groups either by letter or
telephone, and the focus groups are conducted in
much the same way in terms of using an interview
schedule if they are semistructured, and of having a
facilitator to conduct the focus group. Many issues
pertaining to audioconferencing are equally
applicable to videoconferencing.
Guidelines for the use of focus groups
as a health research method
The guidelines for the use of teleconference focus
groups, are similar to those for face-to-face focus
groups, in relation to size, structure, length of time,
homogeneity and con� dentiality.

Guidelines for the design of focus groups for
general practice research, which can be applied to
teleconference focus groups include:
· Structured with a de� ned set of focus group
questions (Morgan, 1992).
· Less than one and a half hours long because of
problems with fatigue, loss of interest, poor
validity (Cho et al., 1995).
· The group should be homogenous in terms of at
least one research factor (Cho et al., 1995)
with homogenous composition for more effec-
tive communication, and heterogeneous compo-
sition for generating richer, more varied data.
Examples of research factors for which focus
groups may be homogeneous are gender, age
group, and geographical area.
· Participants must give informed consent and
be aware before the session begins that a video-
tape or audiotape recording will be made. In-
dividual contributions must remain con� dential
and anonymous (Schattner et al., 1993).
· Members should be asked to ensure the privacy
of other’s communications as in face-to-face
focus groups but may need to be particularly
aware of this if on the telephone in the home
(Appleby et al., 1999).
· Group structure and size are two important
factors (Morgan, 1992). The usual size of focus
groups is six to 10 participants (Cho et al.,

1995; Schattner et al., 1993). Small groups,
which generate more intense and detailed
discussions are recommended for research
which delves deeply into detailed experiences
and feelings of individuals. Larger groups,
which produce a brief, steady � ow of infor-
mation, are recommended for research, which
requires generation of a number of ideas.
(Morgan, 1992).
Advantages of teleconferencing for
focus groups
Cost effective in terms of time and travel:
· Teleconferencing enables practitioners to par-
ticipate in research focus groups when time and
� nancial costs make this dif� cult. The time
commitment and inconvenience of travel have
been identi� ed as barriers to participation of GPs
in focus groups (Schattner et al., 1993) Incon-
venience can be minimized by teleconferencing.
Examples of research where teleconferencing
was used to facilitate the recruitment of GPs,
located in rural and remote areas, are studies
conducted by Tudiver et al. in Canada (2001)
and Tolhurst et al. in Australia (1997, 1999).
In our studies of female rural GPs (Tolhurst
et al., 1997) and violence against rural GPs
(Tolhurst et al., 1999), GPs from remote areas
of Australia, over 1000 kms apart, were able to
participate, without the cost and time commit-

ment that would have been required for a face-
to-face focus group.
Increased level of anonymity when discussing
sensitive issues:
· Conducting groups by teleconference has
anonymity and facilitates discussion of sensi-
tive issues (Schopler et al., 1998; White and
Thomson, 1995), an example being a study con-
ducted by White et al. in New Zealand on sexual
relationships between doctors and patients
(White et al., 1994). This study used tele-
conferencing to provide anonymity to the
participants. The researchers explained, ‘focus
groups are usually less suitable for exploring
highly sensitive or personally threatening issues,
personal interviews or anonymous mailings
being preferred’ (White et al., 1994: 391). The
Using teleconferencing to enable general practitioner
participation in focus groups 3
focus group adaptation in this study used the
strengths of the focus group method and
maintained anonymity. The participants in this
study were able to freely disclose personal
experiences, beliefs, and values within the group
without the threat of being identi� ed.
Overcoming con� icting responsibilities of
participants:
· Teleconferencing overcomes problems with

participants’ con� icting responsibilities, such as
family responsibilities. For example, in our
study of female rural GPs the use of teleconfer-
encing enabled participants, with child care
responsibilities, to remain at home, while partici-
pating in the focus group (Tolhurst, 1997).
Limitations of the use of
teleconferencing for focus groups
Lack of nonverbal cues:
· A constraint in using teleconferencing is the lack
of nonverbal cues (White and Thomson, 1995),
which may be important in the direction, which
the discussion takes, and in the level of group
participation. For example, during a tele-
conference the facilitator is unable to observe
nonverbal signs of participants’ agreement or
dissatisfaction in relation to other participants’
contribution, or distress when sensitive issues
are being discussed.
The need for the appropriate technology:
· Most potential participants have access to a
telephone and most telephone service providers
can provide teleconferencing facilities. However
technological support is critical to the imple-
mentation and maintenance of group activity
(Schopler et al., 1998).
Possible bias in sampling:
· It is possible that GPs who do not have
experience in the use of teleconferencing might

be uncomfortable with this method and less
likely to participate in teleconference focus
groups although no research has been under-
taken exploring this issue.
Discussion
These examples show that teleconferencing
for focus groups is a useful method for research
on general practitioners. The guidelines outlined
above for face-to-face focus groups can easily be
adhered to when conducting groups by tele-
conference. Teleconferencing can be cost effective
in general practice research. Its use has the poten-
tial to encourage the participation of GPs who
otherwise may not participate because of geo-
graphical location or commitments at home, by
minimizing the time commitment of the GPs, and
providing a high degree of anonymity to parti-
cipants, when discussing sensitive issues.
The use of teleconferencing in general practice
research demonstrates how new technology in-
corporated with older research methodologies
creates new research methods.
References
Appleby, N.J., Dunt, D., Southern, D.M. and Young, D. 1999:
General practice integration in Australia. Primary health
services provider and consumer perceptions of barriers and
solutions. Australian Family Physician 28(8), 858–63.
Cho, T.K., Davis, A., Sullivan, S. and Fisher, J. 1995: A review

of � ve existing guidelines for planning focus groups in GP
research. Australian Family Physician 24(2), 184–86.
Clapper, D.L. and Mussey, A.P. 1996: Electronic focus groups:
a framework for explanation. Information and Management 30,
43–50.
Forde, R. and Evensen, A.R. 1991. Research activities and
research interest among female general practitioners. Why so
few when so many are interested? Tidsskrift for Den Norske
Laegeforening 111(9), 1136–38.
Garigliano, J. 1998: The next best thing to being there. Folio:
the
Magazine for Magazine Management 27(7), 21.
Gray, R.W., Woodward, N.J. and Carter, Y.H. 2001. Barriers to
the development of collaborative research in general practice: a
qualitative study. British Journal of General Practice 51(464),
221–22.
Jowett, S.M., Macleod, J., Wilson, S. and Hobbs, F.D. 2000.
Research in primary care: extent of involvement and perceived
determinants among practitioners from one English region.
British Journal of General Practice 50(454): 387–89.
Krueger, R.A. 1988: Focus Groups. London: Sage, 41–8.
Kuramoto, A.M. and Dean, J.L. 1993: Audiographics
teleconferencing: a method of distance learning. Journal of
Nursing Staff Development 13(1), 13–17.
MacIntosh, J.A. 1993: Focus groups in distance nursing
education. Journal of Advanced Nursing 18(12), 1981–85.

4 Helen Tolhurst and Sarah Dean
Morgan, D.L. 1992: Designing focus group research. In Stewart,
M.,
Tudiver, F., Bass, M.J., Dunn, E.V. and Norton, P.G., editors,
Tools for Primary Care Research. London: Sage, 180–93.
Schattner, P., Schmerling, A. and Murphy, B. 1993: Focus
groups: a useful research method in general practice. Medical
Journal of Australia 158, 623–25.
Schopler, J.H., Abell, M.D. and Galinsky, M.J. 1998:
Technology-based groups: a review and conceptual framework
for practice. Social Work 43(3): 254–68.
Tolhurst, H., Bell, P., Baker, L., Talbot, J. and Cleasby, L.
1997: Educational and Support Needs of Female Rural
General Practitioners. Discipline of General Practice. Faculty
of Medicine and Health Sciences, University of Newcastle,
Newcastle, Australia.
Tolhurst, H., Talbot, J., Bell, P., Baker, L., Murray, G.,
McMillan, J., Sutton, A., Treloar, C. and Harris, G. 1999:
After Hours Medical Care and Personal Safety Needs of Rural
General Practitioners: Final report to the General Practice
Evaluation Program. Canberra: Department of Health and
Aged Care, ACT.
Tudiver, F., Brown, J., Medved, W., Herbert, C., Guibert, P.,
Haggerty, J., Goel, V., Smith, J., Obeirne, M., Katz, A.,
Molin, P., Ciampi, A. and Williams, J.I. 2001: Making
decisions about cancer screening when the guidelines are

unclear or con� icting. Journal of Family Practice 50(8),
674–82.
Wildsoet, C., Wood, J. and Parke, J. 1996: Audio-tele-
conferencing as a medium for distance learning: an application
for continuing education in optometry. Australian Journal of
Rural Health 4(1), 18–27.
White, G.E., Coverdale, J.A., Thomson, A.N. 1994: Can one be
a
good doctor and have a sexual relationship with one’s patient?
Family Practice 11(4), 389–93.
White, G.E. and Thomson, A.N. 1995: Anonymized focus
groups
as a research tool for health professionals. Qualitative Health
Research 5(2), 256–62.
Reproduced with permission of the copyright owner. Further
reproduction prohibited without permission.
1279
QUALITATIVE RESEARCH METHODS: A COMPARISON
BETWEEN FOCUS-GROUP AND IN-DEPTH INTERVIEW
Zaharia Rodica Milena
Bucharest University of Eonomics, Romania, Faculty of
International Business and Economics,
[email protected], Tel.:+40 21 319 19 90, tel. +40 0722179201

Grundey Dainora
Vilnius University, Kaunas Faculty of Humanities,
[email protected], Tel.: +370 37 425 462,
Fax: +370 37 423 222
Stancu Alin
Bucharest University of Economics, Faculty of Marketing,
[email protected], Tel.:+40 21 319 19 90
Qualitative research methods tend to be used more and more in
academic research. The cost for these
methods is quite low and the results may be very interesting and
useful for many fields of study. However,
the utility and the characteristic of qualitative research methods
differ from subject to subject and from
discipline to discipline. This paper comes close to a comparison
of two qualitative research methods
(focus-group and in-depth interview) used in investigating the
opinion of academics, analyzing by
comparison the results founded in a research conducted in the
Bucharest University of Economics using
focus group and in-depth interviews. The conclusions of the
study reveal that apart of the limits states in
the literature, there are other elements that can contribute to
obtaining unrealistic results.
Key words: Qualitative research methods, focus group, in-depth
interview, academic research
Introduction
Rapid social change and the diversity of the world have

contributed on a large scale to the diversification
of research methods. Limits of quantitative research methods
have determined orientation to the qualitative
instruments which are more reliable in certain circumstances.
There are many virtues of qualitative
research that determine a lot of researchers to manifest
preference for these kinds of methods: are the
correct choice of appropriate methods and theories, the
recognition and analysis of different perspectives,
the researchers’ reflections on their research as part of the
process of knowledge production, and the
variety of approaches and methods (Flick, 2002:4). Qualitative
research explain how it may be useful for
exploring “why” rather “how many”.
There are various types of instruments used to collect data for
qualitative research. Focus groups and in-
depth interviews are among the most utilize instruments that
researchers are using in collecting their data.
Focus group implies a group discussion in order to identify
perceptions, thoughts and impressions of a
selected group of people regarding a specific topic of
investigations (Kairuz, Crump and O’Brien, 2007).
Discussion should be perceived by the participants as no-
threatening and free to express any kind of
opinion, no matter if this opinion is shared or not by the other
participants. Focus groups generate valuable
information, especially when the participants represent small
groups of interest, ignored by the quantitative
research or when the area of investigation.
The in-depth interview is a technique designed to elicit a vivid
picture of the participant’s perspective on
the research topic. During in-depth interviews, the person being
interviewed is considered the expert and

the interviewer is considered the student. The researcher’s
interviewing techniques are motivated by the
desire to learn everything the participant can share about the
research topic.
In depth interview is an effective qualitative method for getting
people to talk about their personal feelings,
opinions, and experiences. It is also an opportunity to gain
insight into how people interpret and order the
world. We can accomplish this by being attentive to the causal
explanations.
The results obtained through these two qualitative methods
varied (among others) according to the subject
investigated. As a general rule, in depth interviews, the
participants are more confident, more relaxed and
they feel more encouraged to express the deepest thoughts about
a certain subject. In focus group the
1280
participants act according to their personality; it is the risk that
in some situation those with a week
personality to follow those with a stronger personality. There
are also some situations when the issue
discussed is incommode and the participants are not confident
in expressing their real opinions. They are
more preoccupied by the image that the other participants will
build up on them that to express what they
really think about that subject.
The case study
This study tries to identify the limits of using focus group
comparing with in-depth interview in identifying

the opinions of academics regarding some aspects of academic
research. The aspects evaluated through
these two qualitative methods were related to the connection
between academic research and business
environment and to the performance of the academic research.
Both issues are quite very important for Romanian academics
and universities management. The
performance of academic research (through the results of the
research process) is perceived as almost
similar with the performance of the university.
Cooperation between academic research and business
environment is considered to be one of the proofs
that academic research is in the benefit of society, a
confirmation of the market utility of universities.
Academic research is considered to be efficient and social
valuable if it offers solution to real problems that
companies confront on. Students seem that are not willing to
sacrifice functional expertise in favor of
generalist expertise. (Schelfhaudt and Crittenden, 2005)
From universities point of view, collaboration between business
environment and universities is an
important source of casuistic for the teaching process. It is also
an important source of financial resources
for universities, and many times sponsorship activity is
determined by the implication of business
environment in the academic activity. There are cases when
industries developed in a perfect harmony with
universities (Silicon Valley).
From the students’ point of view, those universities with strong
connection in business environment are
very attractive. It is a proof of the fact that they will have better
opportunities to find a job and a

confirmation that higher education does not offer only
theoretical knowledge, but also practical skills and
abilities.
In Romania it is still believed that universities are more
oriented to theoretical approach than to practical
direction. Universities considered that business environment
does not have positive reception for academic
research. Business environment is not convinced that
universities have necessary abilities to find correct
answers for their problems.
Methodology
The purpose of this paper is to present the differences between
focus group and in depth interview using
these qualitative methods on a subject regarding the academics’
evaluation of the performance of academic
research and the relation between business environment and
academic research. Both focus group and in-
depth interviews were conducted in the same university,
following the same conversation guide. Some of
the academics interviewed by in depth interview method were
present in the focus group. The structure of
focus group was similar with that of in-depth interviews.
Academics were grouped in three categories,
according to age and involvement in research. Group A, below
30, group B between 30 and 50 (with two
sub-groups, B1 with little involvement in research and B2 with
involvement in research) and group C, over
50 (also with 2 sub-groups: C1, with little involvement in
research and C2 with involvement in research).
The structure of the conversation guide was around three major
themes: what do you consider about the
present evaluation system of academic research, why the
performance of academic research in Romania is

considered to be low and how do you appreciate the relation
between academic research and business
environment. In order to discuss the comparison between these
two qualitative methods were selected two
questions: the first one, what do you consider about the present
evaluation system of academic research,
and the last one, how do you appreciate the relation between
academic research and business environment.
The reason of selecting these two questions is given by the fact
that the first two question offer similar
information from this paper’s goal point of view.
Also, literature review was used in order to determine the place
of this study among other researches. At
the international level it is a constant preoccupation in
analyzing both the virtues and the limits of focus
1281
group and in-depth interview and some of these studies were
consulted in order to sustain the conclusions
of the present paper.
Major findings
What do you consider about the present evaluation system of
academic research?
On this question, the answers gained in focus group and in in-
depth interview were quite different. If both
in-depth interviews and focus group all of the participants
complained about the present evaluation system,
the motives of complains were different.
The participants’ attitude in focus group was rather favorable to

the research aspects. Nobody mentioned in
the focus group pecuniary aspects. All critics expressed by the
participants in focus group were related to
the relativity of the criteria established by the Ministry “why
they are asking for publication in ISI journals,
when we don’t have such journals in Romania” (X12), to the
high pressure that exist on the professors
“students are asking for a good professor, evaluation system
asks for a good researcher. It is quite difficult
to be good on both” (X3, X9, X11). All participants in focus
groups agreed that publication should be a
criterion of evaluation, but not in this way. A lot of vague
aspects were revealed, as “originality of the
research”, or “the contribution to the field”, criteria which are
difficult to evaluate and nobody explained
what it mean.
In in-depth interviews the critics were very acid. The attitude
against research activity was evident. “When
I was hired, nobody told me that I supposed to be a researcher. I
was thinking that I will be a professor”
(X9, X6). “Which is the difference between research institute
and universities? We should educate people,
not to research” (X12, X10). Financial aspects were also among
those reasons invoke the most. “They
should first look to our wages and then they should ask us to
have similar performances as the western
academics” (X2, X7, X5).
The differences between the attitude expressed in focus group
and in in-depth interviews have different
reasons. One reason is given to the fact that all academics
acknowledge that research is an important part of
the academic activity and they don’t want to admit “in public”
that they do not agree with it. The
complaints are related to the way of establishing criteria not

with the criteria. In in-dept interview, they felt
more confident and they expressed their concerning and their
rejection regarding an activity which is
considered too difficult or unnecessary (for some) from their
point of view.
Another reason is given by the differences between generations.
Those over 50 are more reserved than
those under 30. But those over 50 have more official authority,
so they abstain to express negative opinion
in order to preserve their image.
How do you appreciate the relation between academic research
and business environment?
This issue is one on which focus group and in-depth interviews
were convergent. Both in focus-group and
in in-depth interviews a certain defensive attitude was revealed.
The academics consider that business
environment does not offer credit to academic research. “They
are not interested in what can we offer.
Managers are suspicious; they consider us a sort of spy (X1)”.
‘If you know someone in a company, than
you have chances to develop some relation with them, otherwise
is almost impossible”(X2). “Let’s be
realistic. What can we offer? Why should they be interested in
our researchers’ abilities?” (X7) “They
don’t need research; they definitely don’t need academic
research”(X5).
Other opinions consider business environments hostile to
collaboration with academic environment. “Our
students have many difficulties in finding companies for
internship and those which accept students ignore
them and do not allowed students to be involved in their current
job. “All the time, we receive the same
answer from the students when we ask them: what have you

done in the company? I answer to the phone; I
typed something on computer, things like these….(X6)”
From in-depth interviews, many additional comments appeared
in link with the relation between academic
environment and business environment. Almost all of them were
in the same spirit of malfunction relation,
which has roots both in the communist regime mentality and in
the perception that universities are mostly
oriented towards theory, not towards practice. “In communism,
the collaboration between so called
‘business’ and universities were compulsory and nobody could
say no, therefore, it was a formal
cooperation, especially in the economic field. Maybe for
engineering it was different but for us, it was
1282
totally formal, without any kind of practical results.”(X20,X24,
X2) “Companies do not appreciate
academic environment because they consider that universities
offer only pure theory. Many students are
working today and they are saying that what we are teaching
them doesn’t apply in practice”(X31).
Why on this question the results were similar? Why the both
methods lead to the same
conclusion, that business environment is hostile and is not
interested in the research that could
be provided by academics?
One explanation states in the fact that the collaboration between
business environment and academics is a
difficult one. Every academics confronted with this difficult
cooperation between universities and business

environment, mostly through students’ internship activity. As
one of the interviewed academics said, it is
somehow frustrating to ask the students all the time the same
question: “What have you gained from your
internship and to receive the same answer over and over again:
“Almost nothing because they didn’t
allowed me to involve in their business”. It is a real situation
given by the insufficient maturity of business
environment, on one side, and the struggle of academics to
prove their business utility on the other side.
We have to accept that Romania is still learning market
economy and all the actors are in the middle of a
radical transformation process. Business environment in
Romania, in general, gives no real attention to the
research activity. Many things are done “per se”. Romanian
companies are to poor and multinationals
prefer to involve specialized companies in their researches, not
academic environment. This is not a secret,
and academics are aware of this aspect. Therefore, the same
opinions are reflected by the focus-groups and
in-depth interviews.
Another explanation could be offered by the question itself.
Apart from the other two themes, this one is
about “somebody else”. It is a subject that concerns all of us,
but it involves another “entity”, an abstract
one, “business environment”. In focus group nobody was
worried that his/her opinion is misunderstood or
that the expressed opinions will have any kind of negative
consequences. Furthermore, as all of them
expressed the same thoughts, there was no fear of a “bad” image
perceived by the other participants.
Another motive could be given by the fact that the cooperation
with business environment is not necessary
assimilated with the personal academic performance. If an

academic does not publish, this could be
interpreted as a lack of skill or a low performance, but not being
involved in cooperation with business
environment could be the results of many external factors,
which does not necessary should be related with
personal abilities or personal performance.
Conclusions
The use of qualitative research methods is considered to be a
solution in investigating “Why?” especially
when the need to generalize the results is not necessary.
Qualitative research methods are also preferable
when the investigation is oriented to determine motivation,
perceptions or believes.
Focus-group and in-depth interview are among the most used
methods. There are some advantages offered
by these instruments: low costs and valuable information, which
are difficult to obtain from a quantitative
research.
The inconvenient of these methods are determine by their limits
in generalization the results. Also the
results can be altered if the instruments are used in an improper
manner. This case study shows that
interviews are also especially appropriate for addressing
sensitive topics that people might be reluctant to
discuss in a group. The question regarding the present
evaluation system of academic research was a
sensitive one and the results demonstrated that on question like
this in-depth interviews are more indicated.
On the second question the results were similar using both
methods because the subject discussed was
suitable for a group discussion too.
References

1. Aaker D.A; Kumar, V.; Day, G. (2001), Marketing Research,
Seventh Edition, John Wiley&Sons,
Inc. New York
2. Burns, A.C; Bush, R.F (2002), Marketing Research, Third
Edition, prentice Hall, New Jersey
3. Flick, U (2002), An introduction to Qualitative research,
SAGE Publication, London
1283
4. Kairuz, T, Crump K, and O’Brien, A. (2007), “Tools for data
collection and analysis”,The
Pharmaceutical Journal, Vol. 278, pp. 371-377, (referred on
02/04/2008), available from
www.pjonline.com
5. Kvale S. (1996), Interviews: An Introduction to Qualitative
Research Interviewing. London: Sage
Publications
6. Qualitative Research Methods: A Data Collector’s Field
Guide, (referred on 12/04/2008) available
from
ttp://www.fhi.org/nr/rdonlyres/eprvylcljxmbssmuizj3rcnycde2pn
lowtoa7nxedbjl23jqjpitifimwv6v
2tmsmyocepirpjpfrd/interviews1.pdf
7. Rubin HJ, Rubin IS. (1995), Qualitative Interviewing: The
Art of Hearing Data. London: Sage
Publications

8. Schelfhaudt, K; Crittenden, V.L. (2005), “Specialist or
generalist: Views from academia and
industry”, Journal of Business Research, 58, pp. 946– 954,
(referred on 02/03/2008), available
from Elsevier
9. Simon-Kumar, R. (2005), "Is Qualitative Research also
Quality Research? Debating the limits of
Critical Scholarship" Paper presented at the annual meeting of
the International Studies
Association, Hilton Hawaiian Village, Honolulu, Hawaii,
(referred on 22/03/2008), available from
http://www.allacademic.com/meta/p72159_index.html
Copyright of Annals of the University of Oradea, Economic
Science Series is the property of Annals of the
University of Oradea, Economic Science Series and its content
may not be copied or emailed to multiple sites or
posted to a listserv without the copyright holder's express
written permission. However, users may print,
download, or email articles for individual use.
4 NURSERESEARCHER 2010, 18, 1

commentary
The three themed papers in this edition identify particular
issues in the ethi-
cal conduct of qualitative research. Some concerns are common
to all three
papers and other concerns raise queries related to the conduct of
the authors’
own research. This commentary will focus on three of these
issues: gaining
access and recruitment, informed consent and the relationship
between the
researcher and participant.
Ethical challenges in the
conduct of qualitative research
Desley Hegney and Tuck Wai Chan highlight the issues of
particular relevance to qualitative studies
NURSERESEARCHER 2010, 18, 1 5
Accessing the research site and recruiting participants raise
many ethical
questions that researchers need to take into consideration before
starting
their research. Although the focus of the research might differ,
the issues are

much the same for research by qualitative researchers as they
are for those
involved in quantitative research. Houghton et al (2010) and
Walls et al
(2010) describe how they required assistance from clinical staff
to distribute
information about their research as part of their recruitment
processes. Three
distinct steps are suggested:
Clinical staff can ask patients if they are interested in
participating in research,
and if they are they are given written information. If, after
reading this informa-
tion, the patients are still interested in participating in the
research they contact
someone from the research team. Potential participants can then
choose to opt
in or opt out and are also able to withdraw at any time.
Removing the researcher from the consent process until the
potential par-
ticipant is willing to meet them helps to ensure that individuals
do not feel
coerced into taking part in the research. In some instances,

however, it is not
possible to use this process to recruit participants. For example,
Houghton et al’s
(2010) research involved the observation of students who were
providing care
to patients and, therefore, involved healthcare staff, patients
and visitors to the
research area. The authors discuss their process for obtaining
informed consent
to collect data in a public space. They note that visitors to the
clinical area can
also become part of the research and that there can be difficulty
in ‘deciding
from whom to obtain informed consent and by what means’. The
strategy used
to overcome this was to place posters in public spaces to inform
visitors and to
provide information sheets for the ward staff. What the authors
do not consider
is what might happen if a visitor does not consent to participate
in the research.
Does this mean they are unable to visit their relative or does it
mean that the
research must be suspended?

A controversial alternative might be for research participants to
opt out rather
than to opt in to some forms of research. That is, if people do
not indicate their
desire not to be involved in the research then they are
considered to have con-
sented to be in the research. How participants had received
sufficient informa-
tion would, of course, need to be demonstrated to and then
approved by the
relevant research ethics committee (REC). Houghton et al
(2010) consider how
6 NURSERESEARCHER 2010, 18, 1
commentary
a ‘potentially exploitative relationship’ can arise between the
researcher and the
research participant and link this to the ethical principles of
‘autonomy, benefi-
cence and justice’. In the conduct of their research, and in all
human research, the
principle of justice is the most important. Are the students
being treated as mere

objects of the researcher or as a resource with little or no
consideration of their
wellbeing? A similar question could be asked when conducting
sensitive research
as described by McGarry (2010). Given the sensitive nature of
the research, are
the participants the right ones to be recruited or are they simply
a group that is
easy to access?
Houghton et al (2010) also discuss the principles of beneficence
and non-
maleficence, noting that the researchers have to ‘calculate the
risk-benefit
ratio’. They state that the ‘benefit of the interview does not
outweigh the
harm of distress’. This statement acknowledges that all research
carries the
risk of some harm; it is just the degree of harm that has to be
balanced. They
note that if the interviewee becomes distressed then the
researcher ‘may
(our emphasis) refer participants to appropriate professional
intervention’. It

could, however, be argued that researchers have an obligation to
take action.
Consideration should also be given to the fact that the distress
constitutes an
adverse event which must be reported to the REC. In any
research protocol,
harm should be anticipated and processes put in place to manage
it.
Houghton et al (2010) also highlight the complex ‘dual role of
the clinician
and researcher’. They use the example of a ‘potentially
dangerous’ situation
where the researcher observes poor care. They argue that here
the researcher,
as a nurse, should intervene. There are arguments in some
research disci-
plines that interfering in this way might not be appropriate, but
when the
researcher is a nurse, with a professional obligation that takes
precedence
over their role as a researcher, they have no choice but to
always consider the
care of the patient first. Houghton et al (2010) demonstrate the
complexity

of this relationship by describing research in which a nurse
researcher was
observing a nursing student delivering care to a patient. They
say that if
researchers observe harm they are obliged to intervene. It could
be argued
that the obligation not to cause harm in this situation rests with
the student.
Further, the nurse-researcher has no obligation beyond reporting
the situation
to another member of staff. Researchers need to be aware of
which role they
NURSERESEARCHER 2010, 18, 1 7
are working in and that they have multiple roles (nurse, teacher,
researcher)
and must consider how these roles might clash with or affect
their research.
These three papers have examined some interesting ethical
considerations
but they also suggest that the ethical conduct of qualitative
research (in the
social behavioural research ethics paradigm) is somehow
different to that of

quantitative research. While many RECs may believe that there
is a greater
risk of harm from poorly designed social science research, this
is not neces-
sarily so. Ethical principles should equally apply to all research
regardless of
the paradigm being used. Many of the issues considered apply
as equally to
quantitative research as they do to qualitative research n
Desley Hegney is director of research, Alice Lee Centre for
Nursing Studies,
National University of Singapore and Faculty of Health
Sciences, the
University of Queensland
Tuck Wai Chan is associate director, Institutional Review
Board,
National University of Singapore
Houghton CE, Casey D, Shaw D, Murphy K (2010) Ethical
challenges in qualitative research:
examples from practice. Nurse Researcher. 18, 1, 15-25.
McGarry J (2010) Exploring the effect of conducting sensitive
research.
Nurse Researcher. 18, 1, 8-14.

Walls P, Parahoo K, Fleming P, McCaughan E (2010) Issues and
considerations when
researching sensitive issues with men: examples from a study of
men and sexual health.
Nurse Researcher. 18, 1, 26-34.
references
Reproduced with permission of the copyright owner. Further
reproduction prohibited without permission.
Integrating Multiple
Qualitative Research
Methods (or Avoiding the
Precariousness of a One-
Legged Stool)*
Amy L. Hall
The George Washington University
Ray C. Rist
The World Bank
ABSTRACT
This article suggests that market research is enhanced when
multiple qualitative methods are combined in a triangulated
approach to examining marketing questions. The article begins
with
a case study that illustrates how a qualitative study can, by
itself, be
used as a basis for making marketing decisions, when

methodological triangulation is employed to guard against
unreliable or invalid results. Following the case study is a
presentation of the three legs of the qualitative stool—
interviews
(including focus groups), observations, and document analysis.
Each
method is described, with a discussion of inherent strengths and
weaknesses. The article ends with an argument for increasing
the
use of integrated, triangulated qualitative studies in the field of
market research. © 1999 John Wiley & Sons, Inc.
*The views expressed here are those of the authors, and no
endorsement by the World Bank Group
is intended or should be inferred.
Psychology & Marketing Vol. 16(4):291-304 (July 1999)
' © 1999 John Wiley & Sons, Inc. CCC 0742-6046/99/040291-
14
291
A CASE STUDYi
In the early 1990s, a leading bicycle helmet manufacturer,
BikeSafe,
found itself losing market share in the bicycle helmet industry.
BikeSafe
had long prided itself on a no-nonsense approach to bicycle
safety. The
company produced a nigged helmet, shaped for maximum
protection
and formed of white plastic for greater visibility. BikeSafe sold
helmets

in sizes suitable for adults and older children. BikeSafe enjoyed
a solid
reputation in the industry for proAdding a safe product at an
affordable
price, but now their competitors were doing better while their
business
was stagnant. What had changed? The CEO of BikeSafe called
on his
marketing department to provide some answers.
The market researchers at BikeSafe did what many market
research-
ers do: They organized several focus groups. One focus group
consisted
of customers who had previously purchased BikeSafe helmets.
The sec-
ond group was made up of a dozen local high school students
and a third
group included members of a biking club. The first group,
previous cus-
tomers, reported that they were quite satisfied with their bicycle
hel-
mets and would consider buying another BikeSafe product in
the future.
The group of students yielded little information, reporting that
bicycle
helmets were not "cool" and that they preferred not to wear
helmets at
all. The final group, the biking club members, generally focused
on
safety factors and seemed impressed by the safety rating of
BikeSafe's
helmets. Their discussion, however, quickly deteriorated into a
conver-
sation about the group's upcoming ride. In the end, the
reseai'chers

knew only a little more than when they started, and clearly not
enough.
They did learn that the members of the focus group that had
previously
purchased BikeSafe helmets were generally satisfied, but that
did not
address the question of why they were not attracting as many
new buy-
ers. What went wrong? Why did the focus groups not jdeld
better re-
sults?
The focus groups conducted by the BikeSafe researchers were
subject
to some common weaknesses of this qualitative method. One
researcher
observed that the group consisting of previous purchasers
included
mostly retired people. The researchers realized too late that
because the
focus group was held during a workday morning, they excluded
most
working-age people. The high school group probably faced the
common
problem of peer pressure. Once a single, loud student voiced the
opinion
that it was not acceptable to wear a bicycle helmet, no other
student
was willing to go against the tide. Finally, the researcher
conducting
the biking club's focus group was inexperienced with the
technique and
was not able to keep the discussion focused. Although the focus
groups
failed to provide answers to BikeSafe's problems, they did yield
the im-

'The case study has been created by the authors. It does not
refer to actual persons or an existing
company.
292 HALL AND RIST
portant information that the purchasers participating in the first
focus
group were generally satisfied. The researchers decided to focus
next on
how to reach more purchasers.
The researchers decided to reach into the qualitative tool kit for
an-
other technique. They decided to do some document analysis.
BikeSafe
included a survey on their warranty cards that approximately
50% of
purchasers returned. They began by examining these surveys.
What
became immediately apparent was that for the last few years,
most of
their helmets had been purchased by older consumers. They
were sell-
ing very few helmets to young adults and teenagers.
Additionally, the
factor most commonly cited for deciding to buy their product as
opposed
to another brand was the safety reputation of BikeSafe's
helmets. The
researchers concluded that the company should continue to
emphasize
the safety features of their helmets, but needed to find a way to

recap-
ture younger consumers.
Reviewing advertisements in several biking and sports
magazines
from the previous 6 months revealed some additional answers.
The
BikeSafe researchers noticed that most helmet ads of their
competitors
were aimed at the young consumer. The ads often showed young
adults
engaged in mountain biking and other adventure sports.
Previously,
BikeSafe had exclusively targeted the biking community and
casual bi-
cycle riders. Examination of their competitors' ads revealed that
they
were missing those consumers who purchased a helmet to wear
while
in-line skating, skateboarding, and playing other dangerous
sports.
They also noticed the snazzy colors and sleek designs of their
competi-
tors' helmets. The BikeSafe researchers had a new idea about
the au-
dience their own ads should be targeting. But before investing
in a new
advertising campaign, they wanted to gather additional
information. An
important thing the ads could not reveal was how popular these
other
sports had become and how many new potential consumers this
repre-
sented. The researchers pulled out that last qualitative tool—
observa-
tion.

The BikeSafe researchers decided to get a first-hand look at
how pop-
ular sports such as in-line skating and skateboarding had
become. They
chose several parks in the area and on three consecutive
weekends made
observations about helmet use. They paid particular attention to
how
many people rode bikes and what types of bikes were ridden,
and how
many people were skating or skateboarding. Of all these they
noted the
approximate ages of the people, whether or not they were
wearing a
helmet, and if a helmet was worn, what it looked like. At the
end of the
three weeks, they concluded that their initial impressions from
studjdng
the ads had been correct; more people were engaging in in-line
skating
and skateboarding, and the age range of helmet wearers seen
most fre-
quently was approximately 20-40 years old. Something they
concluded
from their observations that had not been revealed by either of
the other
two techniques was that more small children were wearing
bicycle hel-
INTEGRATING METHODS 293
mets. BikeSafe had never targeted small children as consumers;
they

did not even make hicycle helmets to fit small children, hecause
children
rarely rode hicycles that were hig or fast enough to really cause
injury.
Yet it was ohvious from their ohservations that many small chil-
dren riding bicycles with their parents were indeed wearing
safety
helmets.
The researchers finished up their project hy going hack to
interviews.
They returned to their focus group lists and invited some of the
people
from the focus groups in for individual interviews. They made
sure to
schedule some interviews on weekends so that recent purchasers
who
worked during the week would he included. The interviewees
supported
their conclusions that they should expand their marketing to
include all
age groups and should design a marketing campaign selling
their hel-
mets as sports helmets, suitable for a variety of sports, rather
than
simply bicycle helmets. Interviewees also supported the
conclusion
that consumers want safety with style—colorful helmets with
sleek
designs.
BikeSafe learned from their research. Within 2 years, BikeSafe
had
regained their market share of the helmet industry. They had
intro-
duced a new line of helmets that were designed with safety and

style in
mind. The helmets were produced in a greater variety of sizes to
fit ages
"one to ninety-nine," as their new ads stated. An advertising
campaign
had combined an emphasis on style with the company's already
strong
reputation for safety. The ads showed people in several age
ranges wear-
ing BikeSafe helmets while cycling, skating, and skateboarding.
There
were special ads focused on teenagers.
The company profited from their research. The researchers
learned
as well. They realized that focus groups alone cannot provide a
full,
accurate picture of a market situation. Like a one-legged stool,
infor-
mation gleaned from the initial focus groups would have led to
shaky
conclusions; in their case, there simply was not enough
information for
strong conclusions. By adding other research techniques, like
adding
legs to a stool, they firmed up their conclusions and were on
much surer
ground when they presented their results to BikeSafe's CEO.
The com-
pany benefited greatly as a result.
Market researchers have long relied on focus groups as the
mainstay
of their qualitative techniques. Occasionally other techniques
might be
used as well, but it is still rare that market researchers rely on a

qual-
itative study that combines all three of the legs of qualitative
research
into a triangulated approach. As the researchers from BikeSafe
learned,
when the three techniques are combined, they can provide valid,
reliable
conclusions upon which to base marketing decisions. The
balance of this
article aims to (a) demonstrate that there is an array of
qualitative
methods that can be applied to marketing questions and
problems, and
(b) outline strengths and weaknesses of each method.
294 HALL AND RIST
THE QUALITATIVE TOOL KIT
Every good craftsman likes having good tools with which to
work. This
article presumes that marketing researchers are no different, and
sug-
gests another look at the many tools available in the qualitative
meth-
ods tool kit, ones that may not be currently used to tbeir fullest.
Mar-
keting researchers have long been using a few select qualitative
tools,
namely, focus groups and some observation methods. But these
are
largely used in an exploratory fashion and are not often
integrated into
a multimethod approach to discovery.

For anyone using a new tool, or one with which they are only
slightly
familiar, it is important to become acquainted with any
guidelines per-
taining to that particular tool. True, rudimentary tools need only
rudi-
mentary instructions; but more sophisticated tools need more
careful
discussion of their use and abuse. No less the case with the
careful
application of qualitative metbods to marketing research.
Furtber, it is
important to understand a central reality of qualitative
researcb—its
strength lies in tbe concurrent use of multiple tools. Tbus, it is
not ap-
propriate to presume that if one knows how to use one or more
tools,
one knows bow to conduct qualitative researcb. It is the
interactive and
simultaneous use of metbods (and an analysis that takes account
of tbis
interaction) that is unique to qualitative researcb. Stated
differently,
the key to good qualitative research is metbodological
triangulation.
Triangulation is like using a three-legged stool. Remove one
leg, and
tbe stool is mucb less reliable. It is likely to wobble and
collapse under
one's weight. Remove a second leg and tbe stool becomes very
precarious
indeed. So it is witb qualitative research. Tbe three legs of
qualitative
research methods are interviewing, observation, and document

anal-
ysis.
With tbe use of a multimetbod approach tbat combines these
three
major data-collection strategies employed in qualitative
research, one
bas a basis for drawing conclusions witb strong validity.
Relying on only
one or two of tbe methods leaves one subject to questionable
validity.
LeCompte and Preissle (1993) note tbat tbe researcher can
"pinpoint
the accuracy of conclusions drawn by triangulating with several
sources
of data" (p. 48). Tbe underlying assumption here is
straightforward—
using multiple data sources can deepen our understanding and
bence
is advantageous in comparison to using a single method.
The conventional use of the term qualitative methods in
marketing
has come to imply tbe use of small-group interviews, generally
focus
groups. Tbe emphasis is on only one leg of the stool. Limiting
oneself to
focus groups misses the array of other metbods tbat are
contained in
the qualitative metbods tool kit. Marketing researcb will benefit
from
an expanded use of qualitative methods in a multimethod
approach to
learning about market demand, brand loyalty, consumers'
preferences

and needs, consumer behavior, and why it is that what
consumers say
is often not what they do.
A study that exemplifies the use of these three techniques in
con-
junction with one another was recently published in the Harvard
Busi-
ness Review. In "Making Differences Matter: A New Paradigm
for Man-
aging Diversity," Thomas and Ely (1996) examined diversity in
the
workplace and based their analyses on "interviews, surveys,
archival
data, and observation" (p. 85). Their use of multiple methods
facilitated
the exploration of why diversity is so difficult to manage, why
it is sel-
dom honestly discussed, why the disconnects between values
and be-
haviors caused dysfunctional performances in the organizations,
and
why official pronouncements within the organizations were
ignored if
not outright mocked. The basis for their development of a new
approach
to managing diversity was grounded in the multiple qualitative
data
sources they employed. It is hard to imagine that a sensitive
topic like
managing diversity could be fully understood with just
interviews, or

just the review of official documents in the organizations, or
just watch-
ing behaviors in the cafeteria or in the boardrooms.
THE LOGIC AND RATIONALE OF TRIANGULATION
The term triangulation is taken from land surveying, where the
sur-
veyor uses the sightings of two landmarks to locate his own, the
third,
position. The logic of triangulation is that
no single method ever adequately solves the problem of rival
causal
factors . . . Because each method reveals different aspects of
empir-
ical reality, multiple methods of observations must be
employed. This
is termed triangulation. I now offer as a final methodological
rule the
principle that multiple methods should be used in every
investigation.
(Patton, 1990, p. 187)
Denzin (1978) identifies four types of triangulation: data
triangula-
tion, investigator triangulation, theory triangulation, and
methodolog-
ical triangulation. Data triangulation refers to the gathering of
data at
different points in time and from different sources. Investigator
trian-
gulation is the use of multiple researchers to study the same
research
question or the same setting, presuming that different
researchers will

bring different perspectives, thinking, and analysis to the table,
thus
strengthening the final assessment. Triangulating theory stresses
that
the research should examine the phenomenon from different
theoretical
vantage points to see which would be the most robust in helping
to
clarify and explain what has been studied. Methodological
triangulation
refers to the use of multiple methods to gain the most complete
and
detailed data possible on the phenomenon.
296 HALL AND RIST
Any strategy of triangulation that adds a system of
theoretical/meth-
odological checks and halances to a study lends strength to that
study.
As Patton (1990) observed, "studies that use only one method
are more
vulnerable to errors linked to that particular method . . . than
studies
that use multiple methods in which different types of data
provide cross-
data validity checks" (p. 188). It is our view that a key means to
strengthen market research is to expand the range of
methodologies
used for the gathering of both factual and perceptual
information, and
our focus here is on methodological triangulation.
BALANCING STRENGTHS AND WEAKNESSES

Each of the three major types of data collection (interviews,
observation,
and document analysis) has unique strengths and weaknesses. It
is in
the combination of these strengths and in the compensation for
the
weaknesses that the intellectual and methodological power of
qualita-
tive research becomes apparent. In returning to the roots of
triangula-
tion, the use of multiple methods gives a more precise and
accurate fix
on the research question than does any one method by itself.
A brief summary of the strengths and weaknesses of the three
key
methodological approaches suggests why triangulation is so
central.
First, the use of focus groups, perhaps the central interviewing
tech-
nique of the market research community, "allows people to
discuss their
true feelings, anxieties, and frustrations as well as the depth of
their
convictions in their own words" (Zikmund, 1982, p. 123).
Accurate and
truthful responses in these sensitive areas are hard to gather in
any
research effort, and trjdng to gain them from paper and pencil
tests is
woefully inadequate. With regard to observation, Aaker, Kumar,
and
Day (1995) suggested that "observation may be the least
expensive and
most accurate method of collecting purely behavioral data such

as in-
store traffic patterns or traffic passing a certain point on a
highway
system" (p. 189). Observation also allows the researcher to
personally
see and verify that a particular behavior or interchange did or
did not
actually happen—independent of the respondent's perception of
that
event. The third tool, document analysis, can be used to
"analyze written
material into meaningful units, using carefully applied rules"
(Aaker et
al., p. 190). What individuals say is not always what they do.
Thus there
is a need to turn to this additional source of information—the
written
account—which may come in the form of memos, letters,
personal bi-
ographies, diaries, legal briefs, annual corporate reports, or
other doc-
uments.
With the strengths of these three methodological approaches,
how-
ever, come weaknesses. For example, a shortcoming of focus
groups is
that "without a sensitive and effective moderator, a self-
appointed par-
ticipant may dominate the session" (Zikmund, 1982, p. 127).
Relying

only on interviewing leaves the researcher vulnerable to
selective recall,
self-delusion, perceptual distortions, memory loss from the
respondent,
and subjectivity in the researcher's recording and interpreting of
the
data. Relying only on the second tool, observation, means that
the ac-
tor's understanding and motivation for behavior may not be
understood
by the observer. Inference in this circumstance as to why
someone did
as they did is risky and often just plain wrong. Document
analysis is
also subject to error, as it relies solely on the researcher's
interpretation
of what is in the document being analyzed. Conducting research
when
one's only source of information is historical documents (pity
the poor
historian) involves all the problems of selective deposit of the
material
(not all that is know is written down, let alone then saved),
selective
survival (remember the 18-minute gap in the Nixon White
House tapes),
and selective retrieval (not all that survives is found.)
INTERVIEWS
Interviews can be divided into three basic categories: the
individual in-
terview, the small-group or focus-group interview, and the
large-group
interview. Each of these techniques has strengths and liabilities,
ben-

efits and costs. The emphasis here will be on the first and
second of the
three types, those of individual and small-group or focus-group
inter-
viewing, because these are the most applicable to consumer
research.
The Individual In-Depth Interview
Aaker et al. (1995) described individual in-depth interviews as
"inter-
views that are conducted face to face with the respondent in
which the
subject matter of the interview is explored in detail" (p. 176).
Calder
(1994) noted several advantages of the individual interview,
including
those related to the amount of in-depth information obtained
and the
moderator's control of interview timing and related variables.
Another
advantage of the individual interview is that there is not the
potential
problem of group conformity. People may be more spontaneous
in an
individual interview. There is also the benefit that comes from a
face-
to-face interaction; up-close observation of body language, tone
of voice,
reaction to distractions, reaction to the interview setting, and
personal
appearance are all contextual factors that can assist the
interviewer in
analyzing the interview data. One-on-one interaction may also
sensitize
the interviewer to the existence of resistance to questions, and

efforts
to be evasive. There is also the ability to follow up and probe
incomplete,
vague, or ambiguous responses.
The primary weaknesses of the individual interview technique
are
logistical in nature. Individual interviews are both time
consuming and
expensive. A group interview of 20 consumers may require one
moder-
298 HALL AND RIST
ator and perhaps 2 hours of time. Individual interviews of the
same 20
people would require considerably more resources. In some
cases, the
information obtained from tbe individual interview may be
worth the
expense, but often the same information can be obtained
through the
less expensive foeus-group method.
A second shortcoming of the individual interview is that often
the
consumer is more comfortable expressing his or her opinions in
a sup-
portive group environment than in an individual interview.
Individual
interviews can create a feeling of being isolated or singled out
by the
interviewer. Another possible shortcoming stems from the
possibility of

the interviewer and interviewee being in conflict or disliking
one an-
other. In such cases, it is not likely that the person being
interviewed
will participate in an open and cooperative fashion.
The Focus Group
The focus group is probably the most widely used qualitative
technique
in marketing research. In fact, "qualitative research is
epitomized in
marketing practice by the focus group interview" (Calder, 1994,
p. 50).
The focus or small-group interview is essentially a data-
gathering tech-
nique where the interviewer/moderator focuses the attention of
a group
of 7-12 persons on a deflned set of topics with the intent being
for the
members of the group to discuss the issues among themselves.
The goal
is to learn the views and values of those involved on the topics
pre-
sented. In general, a group is selected and a moderator begins a
discus-
sion, allowing the group participants to pick up the flow and,
hopefully,
express themselves as freely as possible. The focus-group
process can
alter slightly depending on the speciflc nature of the group.
Focus groups share a number of strengths, including ones
related to
cost and time. Generally speaking, focus groups are clearly
lower on

both counts in comparison to individual interviews.
Additionally, being
in a group compels the individuals to focus on the topic at hand
if they
are to communicate appropriately in the situation. Another
strength of
focus-group analysis is that the voice of a number of persons on
a topic
may lend more credibility to a point of view than that which
flows from
an equal number of individual interviews. On a related note, the
inter-
actions in the group can 3deld interesting insights as well. A
report based
on a focus group of the phenomenological approach is also
likely to de-
scribe the group discussion without engaging in extensive
technical or
scientiflc jargon. One listing of the advantages of focus groups
included:
sjTiergism, snowballing, stimulation, security, spontaneity,
serendipity,
specialization, scientiflc scrutiny, structure, and speed
(Zikmund, 1982,
pp. 124-126). And although they do not start with the letter s,
the mat-
ters of cost, time, logistics, and coverage could well be added to
Zik-
mund's list.
There are, however, three important disadvantages to the focus-
group

methodology. First, the quality of the effort rests heavily on the
shoul-
ders of the moderator. Second, focus groups present a sampling
problem.
How to select 7-12 people, according to what criteria, with what
avail-
ability, with what previous experience in focus groups, and with
what
relations to others in tbe group are all matters that can cause tbe
dy-
namics of tbe group to become dysfunctional and tbe data less
tban
useful. Tbird, tbere is no ability to generalize from tbe material,
nor is
it easy to quickly summarize tbe discussions. A 90-minute focus
group
can easily generate 30 pages of typed notes. Working tbrougb
all tbis
material to reacb tbe core conclusions is difficult and time
consuming.
In addition, using tbe focus group as tbe single source of data
can lead
to erroneous conclusions.
Clearly, botb focus-group interviews and individual interviews
bave
strengtbs and weaknesses tbat tbe researcber must weigb against
tbe
expected results. Wbetber one or botb interview tecbniques are
used in
a study, tbe researcber sbould consider bow to combine
interviews witb
metbods from tbe otber two categories presented bere:
observation and
document analysis.

OBSERVATION
A second major type of qualitative researcb metbodology is
observation.
Zikmund (1982) describes two types of observation
tecbniques—unob-
trusive observation and visible observation. Wben one is
performing
unobtrusive observation, tbe intent is to collect data on bebavior
or in-
teractions witbout tbe subject's knowledge. Tbis approacb migbt
be
most appropriate wben tbe bebavior being observed would make
tbe
subject self-conscious, tbus leading tbe subject to cbange bis or
ber be-
bavior. It is also appropriate in tbose situations wbere tbere is
no viable
alternative. In tbe visible-observation case, tbe subject is aware
of tbe
observation taking place. Anotber way to cbaracterize
observations is
participant or nonparticipant. Participant observation requires
tbe ob-
server to get close enougb to tbose being observed to acbieve a
level of
comfort witb tbem, tbus allowing unimpeded observations and
tbe re-
cording of information. For example, a researcber migbt join a
consumer
rigbts organization to get an insider's look at bow participants
feel and
interact. In nonparticipant observation, tbe researcber remains
apart
from tbe subject(s) being studied, in a manner similar to

unobtrusive
observation. Witb all of tbese metbods, tbe researcber observes
ongoing
bebavior of tbe subject(s), interpreting meaning from wbat is
observed.
Zikmund (1982) suggests six types of bebavior can be observed.
First,
tbe researcber is able to witness pbysical actions. Do sboppers
stop to
look at sale ads if tbey are posted at tbe entrance of a store?
Verbal and
expressive bebavior can also be noted. How do sboppers interact
witb
store personnel wben a sale item is out of stock? Spatial
relations and
300 HALL AND RIST
locations can provide important information about consumers,
as can
temporal patterns. Do posted sale ads get more attention if
posted out-
side the store, or if more signs are posted in aisles? How much
time do
shoppers spend examining ads if they are posted outdoors, as
opposed
to inside the store? The sixth area of observation includes
verbal re-
cords, or the contents of advertisements. In this article, the
latter ob-
servational technique is addressed under document analysis.
As with interviewing, observation has both strengths and

weaknesses
associated with it. Observation can be a relatively inexpensive
yet ac-
curate means of gathering purely behavioral data. A major
strength of
the observation method is that because the data from the
observations
are recorded as they occur, there is less distortion than in
interviews,
where an important strategy is to encourage recall in the person
being
interviewed. In an interview, the researcher generally has to
rely on the
memory and honesty of the respondent. In a group interview,
this can
be even more critical if a respondent fears the reaction of others
in the
group. Observation, especially unobtrusive observation, negates
this
concern. The technique is not without weaknesses, however.
There are several disadvantages associated with observation
tech-
niques in general. First, behavior may take place out of sight. A
dissat-
isfied customer, for example, may not express much emotion
while in
the store, a public area, but may become very angry upon
arriving home.
Because observers can seldom consistently be in the right place
at the
right time, the observational picture on any phenomenon is
always in-
complete. Third, the researcher must accurately record the data,
with-
out injecting subjectivity into the recording. This can be a

difficult task.
One strategy for guarding against this in an emotionally charged
area
is for researchers to put all their own reactions, emotions, and
feelings
into a separate section at the end of the observational record.
Fourth,
the researcher must accurately interpret the meaning of the
observa-
tions. There are multiple interpretations often possible from
even a sim-
ple gesture or sigh. Getting the interpretation right is not easy.
Partic-
ipant observation carries with it the possible disadvantage that
subjects
may change their behavior as a result of the observation. An
ethical
consideration that itself could be the subject of another article
concerns
consent in observational research. Observations and interviews
are nat-
urally complementary as they allow the researcher to explore
the inter-
relation of words and deeds. Adding a third technique,
document anal-
ysis, allows for triangulation and thus still stronger grounds for
valid
interpretation.
DOCUMENT ANALYSIS
"Human behavior often leaves traces, and the study of those
traces can
tell us a lot" (Bernard, 1995, p. 332). Key among these are the
records.

documents, letters, newspapers, television programming, books,
diaries,
and e-mail messages that trace peoples' lives. This study of the
traces
we leave behind is termed document analysis. Document
analysis aims
to take information stored in such sources as noted above and
abstract
from it key themes, strategies, values, messages, and the like.
Coding
schemes are frequently used to systematically record and then
analyze
the collected information.
Document analysis can provide a wealth of information to
marketing
researchers. Individuals keep records, businesses keep records,
orga-
nizations keep records, and governments keep records. Maxine
Margolis
used advertisements in the Ladies Home Journal to trace "the
trans-
formation of the middle-class homemaker from an employer of
servants
to a direct user of household products" (Bernard, p. 341), surely
a topic
of interest to marketers. Document analysis, like observation,
can be
inexpensive. It is also unobtrusive, so it is not subject to
selective mem-
ory or social bias. But not all documents are equal. Some have
more

credibility and trustworthiness than do others.
Four categories of contemporary (as opposed to historical)
documents
can be suggested, moving from higher to lower levels of
reliability. They
are (a) contemporary records, (b) contemporary individual
accounts, (c)
reconstructed materials, and (d) reports contained in secondary
docu-
ments. For market researchers, the difficulty comes in that the
two cat-
egories most often used [(b) and (d)] are vulnerable to
distortion and
misunderstanding. But as noted, in conjunction with other forms
of
qualitative data discussed earlier, the use of documents is an
integral
part of a more comprehensive and thorough understanding of
the mo-
tivations and beliefs of individuals.
In the first category are contemporary records, such as
government
legislation and regulations, business and legal papers, office
memoran-
dums, minutes of meetings, and notes to the file. Materials in
this cat-
egory are almost always dated, and will state who wrote the
document.
They are often written very close to the actual happening of an
event.
Contemporary records are often written by one or more
participants, for
the purpose of keeping track of important issues, ideas, or
agreements,

so that they may be referred to and used by those same
participants in
later actions. Many materials in this class are reviewed for their
accu-
racy by all concerned parties. This process of verification may
be quite
formal, as in the reading and signing of legal documents, the
approval
of minutes of meetings, or in the review of important office
memoranda
before they are issued.
The second class of materials are contemporary individual
accounts,
such as what one finds in letters, diaries, personal notes from
meetings
or discussions, or notes in one's personal files. Materials here
are gen-
erally as contemporary as those in class one, but they are
prepared by
only one individual without the review, consultation, or
approval of oth-
302 HALL AND RIST
ers. Because of the personal, even potentially biased nature of
this ma-
terial, the credibility is less than class one.
The third class of materials are those that are constructed after
the
event and separated from that event by some period of time.
Recon-
structing a complete and sequenced description of the actions,

conver-
sations, asides, proposals, and so on of even two persons in a
conver-
sation is difficult; trying to capture a large group discussion or
the
activities of a board meeting is even more so. There is also the
matter
that with time, memory recedes and becomes less precise. The
old say-
ing that "We remember 10 percent of what we read, 20 percent
of what
we hear, and 30 percent of what we see" may not be precise in
the per-
centages, but the implications of reduced recollection are real
for the
trustworthiness of any document prepared some time after the
fact.
The fourth class of documents are those that contain elements of
con-
temporary reports (and not contemporary records as found in
class one).
Reports might be those from a state or the national census
bureau.
Newspaper accounts, surveys and opinion polls, and
investigative ac-
counts also fall into this category. Material in this category may
be gen-
erated on the one hand by statistical sampling (surveys or polls)
and on
the other by highly individualistic requirements (investigation
or news-
paper story). In either case, it is hard to know what meaning to
make
of the information, for it is aggregated, summarized, and
sometimes

reduced to statistical figures, leaving interpretation to a
probability
analysis.
It is evident from this discussion that document analysis does
have
pitfalls, based on the quality of the document being analyzed.
But there
is another domain of problems that also needs to be considered.
Specif-
ically, document analysis invariably involves the coding of the
material
being examined. Who makes up the code, and how closely it is
applied
to the material, can be problematic. Coding and analysis are
also subject
to cultural variability, which can invalidate findings if not
considered
(Bernard, 1995). Thus, use of documents in qualitative market
research
requires careful attention to the source of the document, the
intent in
using the document, and the coding scheme designed to classify
the
information within it.
BRINGING TOGETHER INTERVIEWS, OBSERVATIONS,
AND DOCUMENT ANALYSIS
The market researcher who employs multiple qualitative
research
methods is the researcher who is taking out an insurance policy
on the
accuracy of the answer to the research question. The basic
premise of
this article is not that more research is inherently good. We are

not
discussing moral precepts. Rather, the proposition is that a
coordinated.
triangulated strategy emplo3dng different approaches to the
study of a
research question can heighten the certainty with which that
question
is answered. The design and execution of research studies
inevitably
involve trade-offs among different constraints, for example,
scope, cost,
time, specificity, level of desired accuracy, and so on. The
emphasis here
is that when researchers tend to put all their qualitative
methodological
eggs in one basket (e.g., focus groups), there are risks. But
accuracy and
reliability can be enhanced by expanding the range of
qualitative re-
search methods used in any study. These methods exist and are
widely
employed in other sectors of qualitative research. Market
researchers
also have the choice to use these methods or not.
REFERENCES
Aaker, D. A., Kumar, V., & Day, G. S. (1995). Marketing
research (5th ed.).
New York: John Wiley & Sons.

Bernard, H. R. (1995). Research methods in anthropology:
Qualitative and
quantitative approaches (2nd ed.). Walnut Creek, CA: AltaMira
Press.
Calder, B. J. (1994). Qualitative marketing research. In R. P.
Bagozzi (Ed.),
Principles of marketing research (pp. 50-72). Cambridge, MA:
Basil Black-
well Ltd.
Denzin, N. K. (1978). The research act: A theoretical
introduction to sociological
methods (2nd ed.). New York: McGraw-Hill Book Company.
Denzin, N. K., & Lincoln, Y. S. (Eds.) (1994). Handbook of
qualitative research.
Thousand Oaks, CA: Sage.
Eisner, E. W. (1991). The enlightened eye: Qualitative inquiry
and the en-
hancement of educational practice. New York: Macmillan.
Fontana, A., & Frey, J. H. (1994). Interviewing: The art of
science. In N. K.
Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative
research (pp. 3 6 1 -
376). Thousand Oaks, CA: Sage.
LeCompte, M. D., & Preissle, J. (1993). Ethnography and
qualitative design in
educational research (2nd ed.). New York: Academic Press.
Patton, M. Q. (1990). Qualitative evaluation and research
methods (2nd ed.).
Newbury Park, CA: Sage Publications.

Thomas, D. A., & Ely, R. J. (1996, September-October) Making
differences
matter: A new paradigm for managing diversity. Harvard
Business Review,
74, 79-90.
Zikmund, W. G. (1982). Exploring marketing research (4th ed.).
Orlando, FL:
Dryden Press.
Correspondence regarding this article should be sent to: Ray C.
Rist, The World
Bank, 1818 H. St. NW, Washington, DC 20433
([email protected]).
304 HALL AND RIST

MBA 5652, Research Methods 1 Course Learning Outcomes.docx

Recommended

Recommended

More Related Content

Similar to MBA 5652, Research Methods 1 Course Learning Outcomes.docx

Similar to MBA 5652, Research Methods 1 Course Learning Outcomes.docx (20)

More from aryan532920

More from aryan532920 (20)

Recently uploaded

Recently uploaded (20)

MBA 5652, Research Methods 1 Course Learning Outcomes.docx