592019 1 Case Study KY LOGISTICS & DISTRIBUTION COM.docx

5/9/2019 1
Case Study: KY LOGISTICS & DISTRIBUTION COMPANY
(KYLOD)
Overview
KYLOD is a regional transportation and distribution company in
operation for over 40 years. The
company serves major cities in the US’s Midwest region. They
are headquartered in Chicago, Illinois and
have a staff of 700 employees including truck drivers. There are
six distribution terminals (Chicago IL,
Milwaukee WI, Green Bay Wi, Fort Wayne IN, Indianapolis IN
and Grand Rapids MI) for consolidating
freight, and 200 delivery vehicles including 40 tractor/semi-
trailer units, 80 box trucks and 80 panel
vans.
The company operates in a highly competitive business
environment. Growth has been stagnant
because of a slow economy. Kevin, the president of the
company, would like to see growth at 6% per
year. He would also like to see expenses cut by 6% to help fund

new initiatives. Current revenue is about
$42 million a year with profit running at 5%.
To familiarize yourself with commonly-used shipping terms in
the freight industry, visit this site and refer
to it as you read the case study and assignments:
http://www.shipnorthamerica.com/htmfiles/glossary/gloss_shipt
erms.html
Current Business Operations
KYLOD operates 24 hours a day, 7 days a week. Sales
personnel (10 people, two per terminal) visit
prospective customers to outline company capability, services
provided and costs. When a customer
decides to use KYLOD they call the dispatch office with
shipment information. Usually they FAX a copy of
the bill(s) of lading to a terminal with information such as
origin, destination, product description,
weight and number of packages.
A dispatcher at a terminal makes a list of freight pickups and
sends a truck to get the freight. To do this
they use the routing system to determine the sequence of
pickups by zip code. They use local maps
within a zip code to map out the specific order of pickups since
there may be several in a zip code area.

They have a performance goal of 98% of freight picked up
within 24 hours of availability.
A driver follows the dispatch order for pickups. Many of the
drivers complain that the pickup order is not
efficient. When they pick up an order they sign for receipt and
either load the freight or guide the
customer’s forklift operators to arrange it properly in the truck.
After freight is picked up it is brought to the terminal where it
is unloaded and sorted by destination. A
dispatcher then prepares a delivery ticket (again using the
routing system) that is used to load a truck in
the proper sequence for delivery. Some trucks take freight from
one terminal to another while others
make local deliveries. About half of a terminal’s space is used
on any given night. Dispatchers have a
goal to turn freight around in the terminal overnight for next
day delivery.
When freight is sent out for delivery, the driver follows the
delivery ticket order. Often, they are held up
at a delivery destination by traffic or by lack of available
unloading space. This can cause the driver to be
http://www.shipnorthamerica.com/htmfiles/glossary/gloss_shipt
erms.html

5/9/2019 2
late trying to make the day’s deliveries. Sometimes they get to a
destination and the facility is closed
and they bring the freight back to the terminal for delivery the
next day. It is unloaded and re-sorted by
destination. The dispatchers then add it to the next day’s
delivery tickets.
The major freight volumes are between Chicago, Milwaukee and
Indianapolis (about 70% of total
volume). Trucks run at about 70% of capacity between terminals
overall. Local delivery volume is
heaviest in Chicago, followed by Indianapolis and then
Milwaukee. Local delivery trucks operate at
about 80% full while pickups fill about half of the vehicles
space. Some customers pick up and/or drop
freight at a terminal with their own equipment.
Truck drivers communicate with the dispatchers using two-way
commercial radios. Some also carry
personal cell phones and use them if the radio is out of range. A
few drivers also carry GPS devices to
help locate addresses. In general, the drivers are content with
the company. Pay and benefits are good

and they get overtime pay when deliveries run late. Complaints
are few and mostly center around either
the sequence of pickup and delivery of shipments or vehicle
maintenance.
The fleet is maintained at the main Chicago maintenance shop
and at a smaller shop in Naperville.
Either one can handle minor maintenance and preventative
work. Only Chicago can perform major
engine and transmission work. Overall the fleet is in good
operating condition. All vehicles are on a
preventative maintenance schedule which places them out of
service two days a month, usually on
weekends. Maintenance scheduling is a challenge because it can
interfere with the steady flow of
shipments both between terminals and for local delivery. There
are no “extra” vehicles in the fleet.
Administration
The company management team consists of the President, Vice
President of Operations, Chief Financial
Officer (CFO), Chief Information Officer (CIO), Sales Manager,
and a Fleet Manager who oversees
maintenance and safety. They meet weekly to discuss
opportunities and issues and to plan for the
future. Except for the CIO, the management team has been in

place for many years
The president of the company just hired its first Chief
Information Officer (CIO), Jenny, after the previous
IT Director retired. She comes from a nearby manufacturer who
is also a major customer. At that
company she was Deputy CIO and primarily responsible for
network operations and security.
At a recent meeting the management team decided to change the
strategic plan for the business to
meet growth and cost goals. They highlighted three new
strategies they want to employ to increase
profitability and grow the business. First, they desire to provide
warehousing services for customers who
want to reduce delivery time to their customers by having
product available locally. Second, they want
to improve the percent of loaded miles in their fleet to reduce
costs by coordinating the pickup and
delivery of freight at the same time in the same geographic area.
Third, they want to track the
whereabouts of freight both in the terminals and on the trucks to
provide customers with accurate
delivery dates and times.

5/9/2019 3
In addition, the management team wants to ensure that the
company remains in compliance with all
applicable federal and state regulations. The ones they are most
concerned about are: (1) the Sarbanes
Oxley financial audit and reporting requirements; (2) a new
federal requirement to conduct a vehicle
safety check every 10,000 miles; and (3) a Federal Motor
Carrier Safety Administration (FMCSA)
reporting requirement on the number hours per day for each
driver (or max per week, etc.). The CFO
has been charged with the overall project. He has asked Jenny
to help with this effort by modernizing
information systems to support the new strategies. She has
decided her first step is to update the IT
strategic plan to link to the new strategies in the corporate plan.
Second, she wants to engage her
customers in a proactive way to first, identify and prioritize IT
projects that will help meet the new goals,
and then develop a set of requirements for each project. Third,
she wants to decide on the best
approach to modernize the information systems that will meet
requirements at a reasonable cost, and

for this she will need to make some changes to the IT
organization.
Technology
KYLOD is using a mix of older technology products for finance
and accounting, route optimization,
freight tracking and fleet maintenance. There are several
projects already in the IT portfolio competing
for resources. The CIO sees a major challenge in balancing
available funding, IT staff workload and
project prioritization. The project nearest completion is the
adoption of the Accurate Financial Reporting
System to replace the aging finance and accounting system. It
will be completed in six months. There are
two other projects under way, one for management reporting
and one for a mobile application that
sales staff can use to show potential customers information on
the fleet, distribution services available
and freight rates, including a comparison to the competition.
The route optimization and freight tracking system is very
important to the operations manager and
dispatchers. The current system allows the input of freight
origin and destination information. This is
taken from a bill of lading which contains a plethora of specific
information. When the dispatchers enter

the origins and destinations into the system they are grouped by
zip code. The dispatchers then decide
which zip codes will be loaded in a truck and in what sequence
for delivery. This takes several hours at
night to accomplish and must be done as quickly as possible so
trucks can be loaded and sent out in the
morning for delivery. Arranging shipment sequence within a zip
code is done by locating each address
on a map and entering it into the system in the best order.
Pickups are handled in a similar manner.
The fleet maintenance system contains information on each
vehicle in the fleet. It includes all vehicle
specifications, a summary of all repairs, a preventive
maintenance schedule and an inventory of parts on
hand. This information is entered by accounting clerks,
mechanics, purchasing clerks and anyone else
who has time to do data entry. It is not as time consuming as the
routing system but it contains
information critical to fleet reliability. The greatest challenge is
scheduling preventative maintenance
since it requires vehicles to be down for two days. The
dispatchers do not want the equipment taken out
of service because it causes planning headaches. The
relationship between dispatchers and

maintenance personnel is strained.
5/9/2019 4
IT Organization
When Jenny was hired as CIO last month she took a close look
at the current staffing. The IT staff
consists of 22 people, seven of whom are programmers. The
programmers are charged with all systems
development and integration work for the company. They have
three projects in their current portfolio.
Their skill sets include SQL, .Net and C+ programming, and
Web design.
There are six helpdesk personnel who support the six
distribution terminals (one at each terminal). The
remaining staff includes two network engineers, a financial
systems specialist (an expert in Accurate
Financial Reporting), a computer security expert, two shift
supervisors (who supervise the programmers,
network engineers, financial systems specialist and computer
security expert) and the CIO and her two

personal assistants.
The IT staff supports multiple locations. At the Chicago
headquarters/terminal there are 15 servers
(they contain all software and data; one stores a backup copy of
the data) and 30 PCs for accounting,
marketing, IT, administration and management. The terminal
operations office has five PCs for
dispatchers, one for the maintenance office, one for parts and
one for drivers in the driver lounge. The
other four terminals have 10 PCs each and connect to
headquarters by a virtual private network (VPN).
IT Portfolio
Accurate Financial Reporting System- This new system will
replace the current finance and accounting
system. It is an off-the-shelf product that requires the owner to
make modifications to interface with
other systems they may own. Two programmers are working on
the project. One is setting up the
database and loading the software on servers. The other is
learning about the system to write an
interface with the routing system. A representative of Accurate
will train the accounting staff in its use.
This will take about two weeks.

Management Reporting System- Senior management wanted to
know financial information daily. Two
programmers have been working on a system to compile the
data in a format they can use. They plan to
extract information from Accurate Financials when it is ready
but for now have focused on the current
system. They will be done in two months.
Mobile Marketing App- The marketing manager asked for an
app that sales staff could use to show
potential customers information. This would include things like
fleet photos and specifications; pictures
of the six terminals and information about the distribution
services KYLOD can provide; and a
comparison of their costs using sample shipments with rates
from competitors compared to KYLOD
costs. A programmer and the web designer are working on the
project. It will take two more months to
complete.
The current design and development process is best described
by the way it worked in the selection and
integration of Accurate Financials. The CFO asked the (former)
CIO to develop a new finance and
accounting system. The CIO interviewed large, respected

companies and, after comparing their
5/9/2019 5
capability to the current system, chose Accurate Financial
Reporting. Two programmers were assigned
and an Accurate Financial Reporting specialist was hired to
work between IT and the finance office. The
CIO receives progress reports every two weeks.
Situation
When Jenny was hired, she toured each terminal to see the IT
setup and understand local business
operations. It was important to her to know just how each
person used the systems. She spent time with
bookkeepers and accountants, dispatchers, drivers and terminal
management. Since she came from one
of KYLOD’s customers she knew that customers could offer
insight into business improvements that
would be good for both companies. She visited one large
customer in each of the terminal’s area of
service to get feedback on how operations between them and
KYLOD could be improved. Her goal was
to see how she could translate what she learned into systems

improvements.
Interestingly the most complaints came from bookkeepers and
accountants. They said the system was
slow and data entry was tedious because accuracy was very
important. If they entered wrong
information, it could cause incorrect billing (rates are based on
weight and size), improper loading (the
wrong zip code could mean sending freight in the wrong
direction unless a dispatcher caught the error),
and more. They estimated current accuracy at about 95% but
they had no way of knowing for sure.
Further, they complained about financial reporting and their
ability to meet compliance requirements.
Reporting was mostly a manual process and data they needed
from the system was not easily accessed.
Most of them had resorted to keeping small ledgers at their desk
to track information they knew they
would need for reporting.
The dispatchers explained that routing wasn’t all that hard, just
time consuming. The routing system
grouped all the shipments by zip code. They would take all the
shipments in a zip code and look at the
weight and size (how much cubic space each one needed in a
truck), plot them on a map and then put

them in delivery sequence. They thought most trucks left the
loading dock full and that that the drivers
made adjustments in delivery sequence when needed. Pickups
were a bit more challenging. Sometimes
they sent a truck out just to pick up freight and bring it back to
the terminal. Other times they contacted
a driver to ask them to stop at a customer to pick up a shipment
while they were making deliveries.
Since they didn’t know exactly how much space was available
on the truck this was a hit or miss
situation. Drivers were left to decide if they could make it work.
Drivers were the most outspoken, probably because no one ever
asked for their opinion. They were also
the happiest of employees (this might explain why they were
non-union). They liked being able to make
decisions on the go and they knew the customers very well. In
fact, they could call some of them if they
were running late and the customer would stay open so they
could deliver or pick up a shipment. They
seemed to have favorite customers and often spent extra time
with them talking about common
interests. Generally, they were good ambassadors for the
company.

Terminal managers were under constant pressure. Their main
goal was to get shipments into and out of
the terminal as quickly as possible. Delivery times were
measured and part of their performance plan.
They knew the company had established three new strategies
because they were explained in an email
5/9/2019 6
they just got. Jenny asked how they might provide warehousing
services. Most felt they had extra space
and could take on some storage but keeping track of the
shipments might be a problem. They had to do
this manually and the bookkeepers were the ones to keep the
records. They felt more bookkeepers
would be needed but they didn’t know how many.
Jenny also met with the maintenance and safety staff at the
Naperville terminal. The maintenance folks
had a large workload and complained that they had a hard time
getting equipment in the shop for
preventative work. They did not know when equipment would
be available until the last minute so
scheduling was always a scramble because they needed to make
sure mechanics were available to do

the work. They had a lot of complaints about shifting work
hours and the effect it had on their personal
lives.
The safety manager expressed concerns over driver hours of
service. There are federal regulations that
limit drivers to 10 hours of driving at a time. Then they need to
take an eight-hour break. The problem
was tracking the driver’s hours to make sure they stayed within
the law. Dispatchers tried to help with
this when they scheduled pickups and deliveries but there was
no easy way to do it and the results were
often based on best guess. The safety manager who was
ultimately responsible for compliance had
drivers turn in their hours each day but this was always after the
fact.
Jenny’s customer visits were eye-opening. Most of the
customers had automated inventory systems and
could easily track products from raw material to finished goods.
They knew exactly what they would ship
and when, usually several days ahead of time. Some customers
however needed near instantaneous
shipping. They wanted same-day pickup in a lot of cases and
fast delivery. In most cases, they were all

able to produce electronic documents such as the bill of lading
and email or FAX it to KYLOD.
During her interview for the CIO position, Jenny was told that
the previous IT Director had left a good
foundation and that the staff seemed sufficient in number and
appeared to be very capable. However,
since KYLOD is developing its strategies for the future, the
staff must be able to support the business
strategies as well as the IT strategies that Jenny would develop.
One of the first things Jenny did was to
interview each member of her staff. She discovered that the
roles and responsibilities tended to overlap
and that morale among her staff was very low. Jenny also
interviewed the senior leadership of KYLOD
and learned that her staff was not meeting their expectations for
service. The help desk was perceived
as being only somewhat competent and took much too long to
respond to problems. Application
developers were very slow in delivering systems, and when the
systems were finally delivered, they did
not reflect what the customers needed or wanted. Network
outages occurred too often from the users’
perspective. Finally, the Chief Financial Officer told Jenny that
the IT costs need to be reduced.

Jenny knew she had many challenges. She was determined to
identify essential projects and then
prioritize them for management review. The outcomes would
affect almost every aspect of the
business. Her IT portfolio was about to grow and her
organization will need to change to meet the
challenges.
Reproduced with permission of the copyright owner. Further
reproduction prohibited without permission.
Mixed Methods Research: A Research Paradigm Whose Time
Has Come
R Burke Johnson;Onwuegbuzie, Anthony J
Educational Researcher; Oct 2004; 33, 7; ProQuest Central
pg. 14

5/28/2019 Research and Evaluation in Education and
Psychology: Integrating Diversity With Quantitative,
Qualitative, and Mixed Methods
https://ncuone.ncu.edu/d2l/le/content/122307/viewContent/1252
337/View?ou=122307 1/12
T
CHAPTER 10
Mixed Methods Research
Some evaluators employ randomized controlled trials (RCTs) as
the gold standard of evidence-based
practice (EBP). Critics of RCT designs argue that RCTs do not
include the complexity of program
participants’ experiences. . . . I argue that weaving in a
subjectivist methodology and shifting

methodological perspectives and methods into RCT-based
evaluations prior to, during, and after the
RCT design serves to enhance the credibility and social justice
[of] RCT praxis.
—Hess-Biber, 2013, p. 49
In This Chapter
• Mixed methods research is defined, and typical characteristics
are discussed.
• The importance of mixed methods in educational and
psychological research is explored.
• The philosophical assumptions of this approach are explained,
along with the methodological implications
of those assumptions.
• Specific designs for mixed methods research are presented,
along with examples of these approaches.
• Questions for critically analyzing mixed methods research are
provided.
Youngs and Piggot-Irvine (2012) conducted a pragmatic mixed
methods evaluation of a program designed
to prepare people who aspire to become school principals in
New Zealand. They collected national-level
quantitative data from questionnaires and qualitative data from
document analysis, semistructured phone
interviews, and focus groups, as well as data from regional case
studies.
Ungar and Liebenberg (2011) wondered what resilience looked
like in youth who thrive despite living with
adversity. They used a transformative mixed methods approach,

rooted in local understandings of the concept
in 11 countries, to develop an instrument to capture cross-
cultural aspects of resilience and to test the quality
of that instrument.
wo sample studies that used mixed methods are included in
Chapter 1. Berliner, Barrat, Fong,
and Shirk’s (2008) study of school dropouts and reenrollment,
summarized in Sample Study
1.5, illustrates research that used mixed methods within the
pragmatic paradigm. Boddy (2009)
study of the effect of a mentoring program for poor women in
Sample Study 1.4 is an evaluation study
that used mixed methods in the transformative paradigm. The
vocabulary and methodological designs
and their implications are discussed in this chapter.
Definition and Characteristics
Mixed methods designs include both qualitative and quantitative
features in the design, data collection,
and analysis (Teddlie & Tashakkori, 2009).1 In the first issue of
Journal of Mixed Methods Research,
Tashakkori and Creswell (2007) define mixed methods as
“research in which the investigator collects
and analyzes data, integrates the findings, and draws inferences
using both qualitative and quantitative
approaches or methods in a single study or program of inquiry”
(p. 4). Hence, mixed methods can refer
to the use of both quantitative and qualitative methods to
answer research questions in a single study, as
well as those studies that are part of a larger research program
and are designed as complementary to
p. 304

p. 303
00000001583532 - Research and Evaluation in Education and
Psychology:
tative, Q
ualitative, and M
ixed M
ethods
05/28/2019 - RS000000000000000
Integrating D
iversity W
ith Q
uantit
https://platform.virdocs.com/rscontent/epub/196223/OEBPS/ch0
010.xlink.html?#sp8874077
337/View?ou=122307 2/12
provide information related to several research questions, each
answered with a different
methodological approach. While mixed methods have an
intuitive appeal, they also demand that the
researcher be expert in both approaches to research or work
with a team that has such expertise. Issues
related to the design of studies, as well as to ensuring the

quality of a mixed methods approach, are
explored in this chapter.
Teddlie and Tashakkori (2010) described a truly mixed
approach methodology as methodologically
eclectic, meaning that the researcher selects and synergistically
integrates “the most appropriate
techniques from a myriad of QUAL, QUAN, and mixed methods
to more thoroughly investigate a
phenomenon of interest” (p. 8). The intent may be to seek a
common understanding through
triangulating data from multiple methods or to use multiple
lenses simultaneously to achieve
alternative perspectives that are not reduced to a single
understanding. Educational and psychological
researchers mix methods to a varying degree at various points in
their research, although they may not
use mixed methods at every stage of their studies. Researchers
can insert multiple mixed options into
their work at various points in the research process, including
the definition of purpose, overall design,
methods, sampling, data recording, analysis, and interpretation.
Importance in Educational and Psychological Research
Mixed methods have particular value when a researcher is
trying to solve a problem that is present in a
complex educational or social context (Teddlie & Tashakkori,
2009). Because mixed methods designs
incorporate techniques from both the quantitative and
qualitative research traditions, they can be used
to answer questions that could not be answered in any other
way. Many researchers have used mixed
methods because it seemed intuitively obvious to them that this
would enrich their ability to draw
conclusions about the problem under study. Onwuegbuzie and
Combs (2010) cite Green et al.’s (1989)

typology of mixed methods research purposes to provide more
nuanced reasons to use mixed methods:
triangulation (i.e., compare findings from the qualitative data
with the quantitative results);
complementarity (i.e. seek elaboration, illustration,
enhancement, and clarification of the findings
from one analytical strand [e.g., qualitative] with results from
the other analytical strand [e.g.,
quantitative]);
development (i.e., use the results from one analytical strand to
help inform the other analytical
strand);
initiation (i.e., discover paradoxes and contradictions that
emerge when findings from the two
analytical strands are compared that might lead to a reframing
of the research question); and
expansion (i.e., expand breadth and range of a study by using
multiple analytical strands for
different study phases. (p. 411, bold added)
When the purpose of the research is complex, the researcher
often needs to ask multiple questions,
which frequently necessitates the use of mixed methods. Mixed
methods have the potential to
contribute to addressing multiple purposes and thus to meeting
the needs of multiple audiences for the
results. Youngs and Piggot-Irvine (2012) answered several
questions in this mixed methods evaluation
of a school principal preparation program. The questions
focused on the quality of the program design,
delivery, recruitment and retention, and outcomes. They
described their desire to answer multiple
questions using the most appropriate type of data needed to
answer those questions,

Hesse-Biber (2013) and White (2013) both argue that mixed
methods designs can enhance research
that uses randomized control trials (RCTs) described in Chapter
4 of this book. Hesse-Biber (2013)
argues for the inclusion of qualitative methodologies before,
during, and after the RCT in order to
enhance the appropriateness of the intervention for the targeted
population, explore the dynamics
associated with changes in outcomes, and increase the
credibility of the results. White (2013) claims
that RCTs are the most appropriate design for establishing cause
and effect in the sense of determining
the effectiveness of an intervention. However, he encourages
researchers and evaluators to use
qualitative approaches in conjunction with the RCT to answer
questions related to the quality of the
intervention design, the targeting of audiences, barriers to
participation, and adoption of the
p. 305
Psychology:
tative, Q
ualitative, and M
ixed M
ethods
05/28/2019 - RS000000000000000
Integrating D
iversity W
ith Q
uantit

010.xlink.html?#sp8874080
337/View?ou=122307 3/12
intervention by intended beneficiaries.
EXTENDING YOUR THINKING
Mixed Methods Advantages and Disadvantages
1. In your opinion, what are the advantages and disadvantages
of using mixed methods in psychological
or educational research?
2. Talk to potential users of research information or think of
yourself as a consumer. What are the
strengths and weaknesses of a mixed methods approach?
Philosophical Assumptions: Pragmatism, Mixing Paradigms, and
Transformation
On the basis of a review of social research that claimed to use
mixed methods, Greene and Caracelli
(2003) concluded that inquiry decisions are rarely, if ever,
consciously rooted in philosophical
assumptions or beliefs. Rather, researchers based their choice of
mixed methods on the nature of the
phenomena being investigated, the contexts in which the study
is conducted, or the funding agencies’

requirements. Nevertheless, they did not conclude that
paradigms and their associated philosophical
assumptions were irrelevant—merely unexamined. They further
suggested that by attending too little to
philosophical ideas and traditions, many mixed method
inquirers are insufficiently reflective and their
practice insufficiently unproblematized. Examining the
philosophical assumptions underlying research,
mixed methods or not, can offer a better understanding of the
complex social world in which educators
and psychologists operate. (See a list of options concerning the
relationship between paradigmatic
stance and choice of mixed methods in Box 10.1.)
BOX 10.1 Possible Reactions to Paradigms
• Incompatibility Thesis: Mixed methods are impossible due to
the incompatibility of the paradigms
underlying the methods.
• Complementary Strengths: Adhere to the tenets of rigor as
defined within each paradigm.
• Multiple Paradigms Drive Mixed Methods: Mixed methods
can be approached from a pragmatic or
transformative paradigm.
• Dialectical Pragmatism: The research question is the dictator
of method; qualitative and quantitative
methods are taken seriously and results of both methods are
synthesized for each study.
• Transformative Mixed Methods: Mixed methods enhance the
ability of researchers to capture the
complexity of issues of human rights and social transformation.

• Dialectical Thesis: All paradigms have something to offer;
research will advance through the deliberate
and critical dialogues that occur among scholars who situate
themselves in different paradigms (Greene,
2007).
• A-Paradigmatic: Ignore philosophical assumptions (at your
own risk).
SOURCE: Greene (2007); Mertens (2009); Teddlie and
Tashakkori (2009).
Teddlie and Tashakkori (2009) propose the use of pragmatism
as one philosophical orientation to
guide mixed methods researchers. Simply put, pragmatists
consider the research question to be more
important than either the method they use or the worldview that
is supposed to underlie the method.
These researchers use the criterion “what works?” to determine
which method to use to answer a
p. 306
Psychology:
tative, Q
ualitative, and M
ixed M
ethods
05/28/2019 - RS000000000000000
Integrating D
iversity W
ith Q

uantit
010.xlink.html?#sp8874086:box10.1
337/View?ou=122307 4/12
ese esea c e s use t e c te o w at wo s? to dete e w c et od to
use to a swe a
specific research question. Thus, the pragmatic orientation
rejects the either/or mentality suggested by
a choice between the postpositive and the constructivist
paradigms. Here is how Teddlie and
Tashakkori (2009) describe the pragmatic researcher:
Pragmatists decide what they want to study based on what is
important within their personal value
systems. They then study the topic in a way that is congruent
with their value system, including
units of analysis and variables that they feel are most likely to
yield interesting responses. . . . This
description of pragmatists’ behaviors is consistent with the way
that many researchers actually
conduct their studies, especially research that has important
social consequences. (pp. 90–91)
This pragmatic basis can be contrasted with that of the
transformative paradigm in which the
emphasis is on the inclusion of values and viewpoints,

especially of marginalized groups, as the driving
force for all aspects of the research. Mertens (2009) questions
the notion of the centrality of the
researcher’s values in the pragmatic paradigm. She raises
questions in terms of which values, whose
values, and the role of the researcher within the context of
values. While Teddlie and Tashakkori
(2009) claim that practicality should serve as the value basis for
a researcher’s choices, researchers
within the transformative paradigm would ask, Practical for
what? What are the intentions of the
researcher, client, or policymakers? Should the
researcher/evaluator be guided by a higher social
purpose than what is found to be useful to those in power?
Mertens (2009) noted that a mixed methods design could also fit
within a transformative framework
if it was designed to reflect, in both perspective and outcomes, a
dedication to social change at levels
ranging from the personal to the political. Furthermore, it is
possible to conduct almost any mixed
methods study with a transformative or advocacy purpose.
Paradigms and Mixed Methods
What are the philosophical assumptions that underlie the mixed
methods approach? How can you integrate
the assumptions of the three major paradigms (postpositivist,
constructivist, and transformative) to guide
your thinking in the use of mixed methods? What are the
assumptions of the pragmatic paradigm and their
implications for educational and psychological research?
Mixed Methods Design Options
The specific mixed methods approaches are defined by the

ordering of the application of the
quantitative and qualitative methods (simultaneously or
sequentially), as well as at what point the
mixing of methods occurs. Qualitative and quantitative data
collection can occur in parallel form or
sequential form.
• Parallel form: Concurrent mixed methods/model designs in
which two types of data are collected
and analyzed.
• Sequential form: One type of data provides a basis for
collection of another type of data.
Creswell (2009) uses these two basic design types to organize
possible variations in mixed methods
designs, although he uses the term concurrent in place of
parallel. I agree with Teddlie and Tashakkori
(2009) that parallel is a more inclusive term than concurrent
because the latter term implies the two
methods must be used at the same time rather than the inference
from the term parallel, which is that
they occur in proximity to each other. Creswell (2009) describes
possible twists, such as parallel
(concurrent) designs that are explanatory (qualitative data are
collected after quantitative data to try to
explain the quantitative results) or exploratory (qualitative data
collection is followed by quantitative
data collection to explore a phenomenon), and sequential
designs that are triangulated (use of both
quantitative and qualitative data to see if there is convergence
in findings) or embedded (one type of
p. 308
p. 307

Psychology:
tative, Q
ualitative, and M
ixed M
ethods
05/28/2019 - RS000000000000000
Integrating D
iversity W
ith Q
uantit
337/View?ou=122307 5/12
quantitative and qualitative data to see if there is convergence
in findings) or embedded (one type of
data plays a dominant role while the other type of data plays a
supportive role).
Mixed methods can also involve the conversion of qualitative
data to a quantitative form or vice
versa; Teddlie and Tashakkori (2009) call this a conversion
design. They also add another design called
multilevel design in which quantitative data are collected at one

level of an organization (e.g., student
level) and qualitative data are collected at another level (e.g.,
administrator level). The mixed methods
community also talks about designation of approaches as
dominant or less dominant (or supportive),
depending on the extent to which the overall study uses either
quantitative or qualitative approaches—
to the same extent or one or the other to a greater or lesser
extent (Teddlie & Tashakkori, 2009).
Mertens (2009) and Mertens and Wilson (2012) added a
transformative cyclical design in which
different methods are used at different times during the life
cycle of a study, with the data collected at
each stage informing decisions about the data needed in the next
stage.
As I believe that paradigmatic stances are the beginning point
for decision making in research, I
separate the main design options into the two paradigms that are
salient in the mixed methods research
community: pragmatic and transformative. Table 10.1 provides
a summary of four design options for a
mixed methods approach.2 Following a discussion of these
approaches, the transformative cyclical
design is discussed.
Pragmatic Parallel Mixed Methods Design
The pragmatic parallel mixed methods design is one in which
qualitative and quantitative data are
collected and analyzed to answer a single study’s research
questions (Onwuegbuzie & Teddlie, 2002).
The final inferences are based on both data analysis results. The
two types of data are collected
independently at the same time or with a short time lag. (If the
research is designed with two relatively
independent phases, one with qualitative questions and data

collection and analysis techniques and the
other with quantitative questions and data collection and
analysis techniques, then Creswell, 2009, calls
this an embedded design. The inferences made on the basis of
the results of each strand are pulled
together to form meta-inferences at the end of the study.)
Youngs and Piggot-Irvine (2012) provide a detailed example of
a pragmatic mixed methods design
in their evaluation of a preparation program for people who
aspired to become school principals
(hereafter called aspirants). Here is how they described their
approach and methods: “Our pragmatic
position was informed by the intent of our contract with the
Ministry of Education. . . . We took a ‘what
works’ approach” (p. 188). At the national level, they used three
quantitative questionnaires with all
participants; analyzed documents, including regional provider
plans, curricula outlines, and
communications with participants; and conducted
semistructured interviews and focus groups with the
regional providers and facilitators.
The inclusion of two regional case studies into the design meant
that we created the conditions for
multilevel triangulation to occur so that the national data
generated through aspirant
questionnaires and regional provider interviews could be
compared and contrasted with specific
examples from the two regions. This enabled us to report not
just on the more generalized national
findings of the evaluation research but also provide examples of
regional practice and
contextualized aspirant perceptions of their development and
learning. (p. 191)

Table 10.1 Mixed Methods Design Options and Examples
p. 309
Psychology:
tative, Q
ualitative, and M
ixed M
ethods
05/28/2019 - RS000000000000000
Integrating D
iversity W
ith Q
uantit
010.xlink.html?#sp8874086:tab10.1
337/View?ou=122307 6/12
Pragmatic Sequential Mixed Methods Design
In the pragmatic sequential mixed methods design, one type of
data (e.g., quantitative) provides a basis
for the collection of another type of data (e.g., qualitative)
(Creswell, 2009; Teddlie & Tashakkori,

2009). It answers one type of question by collecting and
analyzing two types of data. Inferences are
based on the analysis of both types of data. A sequential design
is one in which the conclusions that are
made on the basis of the first strand lead to formulation of
questions, data collection, and data analysis
for the next strand. The final inferences are based on the results
of both strands of the study. In some
cases, the second strand/phase of the study is used to confirm or
disconfirm the inferences of the first
strand or to provide further explanation for unexpected findings
in the first strand. This approach can
be used to generate hypotheses to be explored in more depth or
to develop surveys that use correct
language for the population.
Berliner et al. (2008) provide an example of a sequential mixed
methods design in their study of high
school dropouts. The researchers had access to a longitudinal
student-level database that contained
information about the date that dropouts left school and if they
reenrolled in the same district. They
used data from a cohort of students who started high school in
2001/2002 and were expected to
graduate in the 2005/2006 school year. They conducted
quantitative statistical analyses on dates of
withdrawal and return, graduation rates, student courses, and
demographic characteristics, as well as
students’ reasons for dropping out. The quantitative data
revealed that dropping out is not necessarily a
permanent decision. While only 45% of the students graduated
within the expected 4 years, of those
who did drop out, almost one third of the students reenrolled
during the 5 years of the study, and 18%
of those graduated from high school in 2005/2006. (Twenty
percent of the students transferred to other

school districts, and researchers did not have access to their
school outcomes.) The authors describe
their sequential mixed methods design as follows:
Drawing on the preliminary analysis of the district dataset,
interviews were conducted to clarify,
affirm, or challenge the study findings and to explore state and
district policies and practices that
affect reenrollment and students’ experiences dropping out and
reenrolling in San Bernardino City
Unified School District high schools. In fall 2007, interview
data were collected from 20 district
contacts during a weeklong, in-person site visit. (p. ii)
Christ (2007, p. 228) provides another example of sequential
design that began with a longitudinal
quantitative survey of the supports needed by college students
with disabilities. The researcher
compared the results of a national survey that was conducted in
1999 and 2001 to see if changes
occurred over this 2-year period. The results were used to raise
additional questions, frame criteria for
purposeful selection of postsecondary institutions, and guide
development of interview questions at the
selected sites. Christ then conducted semistructured interviews
at three postsecondary sites that were
considered by experts in the field to be exemplary. The
resulting data were analyzed to determine
p. 310
Psychology:
tative, Q
ualitative, and M

ixed M
ethods
05/28/2019 - RS000000000000000
Integrating D
iversity W
ith Q
uantit
337/View?ou=122307 7/12
critical themes related to effectively supporting college students
with disabilities. These data were then
used as a basis for designing a third stage of the study: an
intensive case study of a single institution
that was facing a severe budget cut.
Transformative Parallel Mixed Methods Design
The transformative parallel mixed methods design is based on
the use of both quantitative and
qualitative methods in a study that has a goal of social change
at levels ranging from the personal to the
political (Mertens, 2009; Mertens & Wilson, 2012). The design
gives primacy to the value-based and
action-oriented dimensions of different inquiry traditions
(Greene, 2007).

The Shared Reading Project (SRP) provides an example of a
transformative mixed methods design
that used parallel qualitative and quantitative data collection
and analysis strategies (Delk &
Weidekamp, 2001; Mertens, Delk, & Weidekamp, 2003). The
SRP was designed to provide hearing
parents and caregivers with visually based strategies to read
books to their deaf and hard-of-hearing
children from birth through age 8. The SRP was designed to
meet the needs of five groups of
traditionally underserved deaf and hard-of-hearing students,
including members of diverse
ethnocultural groups, those who have secondary disabilities,
people who live in rural areas, people who
come from homes in which a spoken language other than
English is used, and/or people who are lower-
achieving academically. Tutors, most of whom are deaf, visited
the families in their homes to teach
them signing and reading strategies and answer their questions
about the process of teaching their
children to read.
The SRP collected both quantitative and qualitative data to
address multiple evaluation questions.
The quantitative measures included closed-ended surveys and
logs that the families kept that indicated
such things as the demographic characteristics and the number
of times families used the books that the
tutors brought them between visits. The qualitative data were
collected through in-person, on-site
interviews with the families, tutors, and site coordinators.
Foreign language interpreters were used in
homes where the spoken language was other than English. Deaf
interviewers conducted the interviews
with the deaf tutors; these interviews were videotaped and then
transcribed. Participants were asked to

describe such things as their role in the project, their emotional
experiences during the project, and
successes and challenges.
The quantitative and qualitative data were analyzed with a
specific attempt to identify the impact of
the project on the traditionally underserved groups. Quantitative
data indicated that 104 of the 116
children belonged to at least one of the traditionally
underserved groups. The quantitative and
qualitative data were disaggregated in the analysis in order to
provide a picture of the unique successes
and challenges faced by members of these groups in teaching
their deaf and hard-of-hearing children to
read. Because the focus was on traditionally underserved
groups, the results were shared with the
people in their communities to determine lessons learned and
options for additional action. Because of
the demonstrable results in terms of improved literacy, several
of the sites have moved to
institutionalize the SRP, incorporating it into their regular
school program. In some sites, the skill of the
deaf tutors has been increasingly recognized and they have been
given responsibilities to work in the
classrooms as well as at the students’ homes. One site has made
a decision to expand the program to
include Hmong and Latino families of children enrolled in the
school.
Transformative Sequential Mixed Methods Design
The transformative sequential mixed methods approach shares
the transformative goal described in the
previous section, as well as the methodological sequencing of
the pragmatic sequential option
(Mertens, 2009).

Meadow-Orlans et al. (2003) provide an example of a
transformative sequential mixed methods
design in their study of parents’ experiences with their young
deaf and hard-of-hearing children. The
study proceeded in three phases: first, a quantitative national
survey; then, individual parent interviews;
and third, focus group interviews. The initial phase of the study
was designed not only to provide a
broad picture of parents’ early experiences but also to
investigate differences in experiences based on
such characteristics as race or ethnicity, parent hearing status,
presence of additional disabilities beyond
hearing loss, level of parent education, and the socioeconomic
status of the family. The National Parent
p. 311
Psychology:
tative, Q
ualitative, and M
ixed M
ethods
05/28/2019 - RS000000000000000
Integrating D
iversity W
ith Q
uantit

337/View?ou=122307 8/12
g , p , y
Project survey was designed to obtain information directly from
families by eliciting descriptions of
their experiences, evaluations of effectiveness of services, and
recommendations for improvement.
Parents provided important insights throughout the project from
the design of the survey and interview
protocol to the analyses and implications of the findings.
The quantitative data analysis indicated great diversity in the
characteristics of the families and their
responses to the survey questions. For example, one third of the
participating parents had children with
a disability in addition to a hearing loss, and many of these
families encountered difficulties obtaining
needed services. Overall, families generally expressed
satisfaction with the services their children
received; however, some families, particularly non-White
families, were less satisfied with their
services.
These findings led to the qualitative portion of the study, which
used in-depth telephone interviews
and a purposive sampling strategy to investigate in more depth
the experiences of families with
children with co-occurring disabilities, parents who were deaf
or hard of hearing, parents of children
with cochlear implants, and families of color. In the final phase,
three focus groups of parents were
conducted in large urban areas with the intention of expanding

the number of Hispanic and African
American families represented in the sample. In replying to
very open-ended questions, participants
provided insights into their experiences, feelings, and concerns
during the time of suspicion and
confirmation of their child’s hearing loss; discussed the people
or services that were most helpful and
needs that were not addressed by early intervention services;
described their communication decision-
making process; and offered the advice they had for
professionals and for other parents.
The data were analyzed and reported within the larger national
context, while still preserving the
subgroup analyses of traditionally underrepresented groups.
Special attention was given to
interrogating bodies of knowledge that have become
institutionalized as established concepts and
practices that were determined by groups that traditionally have
power in our society. Groups in power
(generally professionals) largely determine for those with less
power (parents) what are accepted
practices. These “cultural facts” are accepted without challenge
until the voices of individuals affected
have the opportunity to articulate their experiences and express
their perspectives. Thus, the parents’
comments served as a basis to give insights not only to other
parents who might be starting down the
road of life with their deaf or hard-of-hearing child but also to
professionals who serve this population.
Transformative Cyclical Mixed Methods Design
Bledsoe (2008) provides an example of a transformative cyclical
mixed methods design in her
evaluation of an obesity reduction program in a school system
in a disadvantaged neighborhood in

New Jersey (see Figure 10.1). She entered the context after
program developers had decided that the
reason students were obese was because of poor self-concept.
Rather than accepting that version of
reality, she asked them if they had checked with the students to
understand their version of reality—
that is, their understanding of the meaning of obesity and
reasons for reducing their weight. As the
program developers had not yet done that, she collected
qualitative data to do just that. What did
obesity mean to the students? Did they overeat because they had
poor self-concepts? Her qualitative
data revealed that the students did not have poor self-concepts.
Indeed, they felt that being “big” was
not such a bad thing. However, they did express concern about
the health consequences associated with
obesity (e.g., heart disease, diabetes) and they wanted to know
how to avoid the likelihood that they
would be afflicted with those conditions. They wanted to eat
healthy foods and exercise more, but they
saw barriers such as lack of access to fresh fruits and vegetables
and a lack of safe places to get
exercise. Thus, a more culturally responsive program was
developed that included food fairs to
demonstrate how to cook favorite foods in a more healthy way
and dance contests in which the number
of steps taken were counted by a pedometer. Students also used
geographic information systems (GIS)
to map food outlets, kept food diaries, and had physical
measurements of weight and blood tests for
diabetes. In this way, the cyclical use of data, both qualitative
and quantitative, allowed for the design,
implementation, and successful outcomes in this project.
Figure 10.1 Obesity Reduction Program Transformative
Cyclical Mixed Methods Design

p. 313
p. 312
Psychology:
tative, Q
ualitative, and M
ixed M
ethods
05/28/2019 - RS000000000000000
Integrating D
iversity W
ith Q
uantit
010.xlink.html?#sp8874086:fig10.1
337/View?ou=122307 9/12
SOURCE: Bledsoe (2008).
Mixed Methods Design Options

Explain the main design options for mixed methods research.
Give an example of how each of these could
be applied in educational and psychological research.
Select a research problem in education or psychology. Find a
research study related to the problem you
selected that uses a mixed methods approach. Use the questions
for critically analyzing mixed methods
research to identify its strengths and weaknesses. What
improvements would you suggest for that study?
Questions for Critically Analyzing Mixed Methods Research
Teddlie and Tashakkori (2009) suggest that there is a unique
character to mixed methods that
transcends a simplistic combination of methods. Therefore, they
suggest the term inference quality to
refer to issues that would be termed internal validity in
quantitative terms or trustworthiness in
qualitative terms. Greene (2007) also discusses this concept as
warranting the quality of inferences in
mixed methods research. Based on the dialectical stance with
regard to mixed methods research, she
offers the following considerations for quality:
• Focuses on the available data support for the inferences,
using data of multiple and diverse kinds;
• Could include criteria or stances from different
methodological traditions;
• Considers warrants for inquiry inferences a matter of
persuasive argument, in addition to a matter
of fulfilling established criteria; and
• Attends to the nature and extent of the better understanding
that is reached with this mixed

methods design. (p. 169)
Table 10 2 provides a framework for considering criteria to
determine quality in mixed methods
p. 314
Psychology:
tative, Q
ualitative, and M
ixed M
ethods
05/28/2019 - RS000000000000000
Integrating D
iversity W
ith Q
uantit
337/View?ou=122307 10/12
Table 10.2 provides a framework for considering criteria to
determine quality in mixed methods

research.
One approach to critically analyzing a mixed methods study
would be to use the criteria that are
presented in previous chapters on specific quantitative methods
for the quantitative portion of the study
and those in the chapters on qualitative methods for the
qualitative portion. This is not a bad strategy
and could be a useful way to assess the quality of the individual
parts of the study. This allows
researchers to begin their critique of the quality of a mixed
methods design by looking at the integrity
of the methods as they are derived from the assumptions of each
paradigm. So using the questions in
the appropriate methodology chapter to critically analyze the
methods for particular approaches in a
project is appropriate.
However, tensions can arise because conflicting demands can be
present when more than one
paradigm is operationalized. For example, if the researcher is
working inductively within the
qualitative part of the study, the sample is small and
purposively selected, and therefore would not meet
the criteria for a quantitative sample that needs to be larger and
have been randomly selected. If a
quantitative component is being used, then a separate,
randomized sample might be added for that
portion. Small samples do raise issues of potential bias.
Researchers should also be aware of the issues
related to the use of a qualitative or quantitative data collection
method to supplement the findings
from the main study. For example, a quantitative study might
include one or two focus groups to add a
qualitative dimension to the main study. The researcher in such
circumstances needs to acknowledge

the limitations of the qualitative data, in that they cannot stand
alone. They are intelligible and
interpretable only when they are linked to the data from the
main quantitative part of the study.
Table 10.2 Rigor: Criteria for Judging Quality in Quantitative,
Qualitative, and Mixed
Methods Research
When looking at the conclusions of a mixed methods design, it
is possible that the results from both
methods will agree with each other and thus confirm the
conclusions reached. It is also possible that
they will not agree with each other. Then, the researcher must
explore plausible explanations for the
disagreement. It may be due to the difference in approach or
because of changes in the context over
time that could influence performance on the dependent
measures.
The selection of different methods in a study may be necessary
to accommodate differences based on
disability, culture, language, reading or writing levels, gender,
class, and race or ethnicity for specific
p. 315
Psychology:
tative, Q
ualitative, and M
ixed M
ethods
05/28/2019 - RS000000000000000

Integrating D
iversity W
ith Q
uantit
337/View?ou=122307 11/12
y, , g g , g g , g , , y p
subgroups in a population (Mertens, 2009). Practically,
individuals with different types of disabilities
may be better able to provide accurate information about
themselves if a qualitative or quantitative
method is used. For example, in the Delk and Weidekamp
(2001) study, telephone surveys were not
appropriate for deaf individuals without adequate English
writing skills. Other human dimensions, such
as race or ethnicity, also need to be considered in the design of
a study. As Stanfield (2011) noted, an
atmosphere of distrust has developed between researchers and
many members of racial or ethnic
minority communities because of historical events such as the
Tuskegee experiments, in which Black
men with syphilis were left untreated so that researchers could
study the progress of the disease. Mixed
methods may be necessary to provide an opportunity to build a

sense of trust between the researcher
and the community. A researcher could define the problem to be
studied through a qualitative phase of
interacting with members of the community, using observation
and interviewing. If trust can be
developed, then it might be appropriate to introduce a
quantitative phase to the project.
These issues and concerns then give rise to the following
questions that can be used to critically
analyze mixed methods research studies:
1. What are the multiple purposes and questions that justify the
use of a mixed methods design?
2. Has the researcher matched the purposes and questions to
appropriate methods?
3. To what extent has the researcher adhered to the criteria that
define quality for the quantitative
portion of the study?
4. To what extent has the researcher adhered to the criteria that
define quality for the qualitative
portion of the study?
5. How has the researcher addressed the tension between
potentially conflicting demands of
paradigms in the design and implementation of the study?
6. Has the researcher appropriately acknowledged the
limitations associated with data that were
collected to supplement the main data collection of the study?
7. How has the researcher integrated the results from the mixed
methods? If necessary, how has the
researcher explained conflicting findings that resulted from

different methods?
8. What evidence is there that the researcher developed the
design to be responsive to the practical
and cultural needs of specific subgroups on the basis of such
dimensions as disability, culture,
language, reading levels, gender, class, and race or ethnicity?
Rigor in Mixed Methods Research
Design a study using a mixed methods design. Explain how your
design respects the integrity of the
qualitative and quantitative methods. Explain how you would
address the criteria implicit in the questions
for critically analyzing mixed methods research.
Using one or more of the studies from the quantitative or
qualitative chapters, rethink the study using
mixed methods. How would the study change? What would you
gain? What would you lose?
Summary of Chapter 10: Mixed Methods Research
In some ways, mixed methods is not a new way of doing
research, as many researchers have engaged
in the collection of both quantitative and qualitative data.
However, the research community now
reflects an increased interest in mixed methods and how to
explore more systematically the advantages
and disadvantages of their use. Both pragmatic and
transformative paradigms are used as a basis for
hi l i d i h l f i d h d di h k h i
p. 316

Psychology:
tative, Q
ualitative, and M
ixed M
ethods
05/28/2019 - RS000000000000000
Integrating D
iversity W
ith Q
uantit
337/View?ou=122307 12/12
this exploration and to examine those elements of mixed
methods studies that make them unique
compared to studies that use a single method. Designs for mixed
methods include consideration of the
temporal relation between the use of each type of method
(quantitative and qualitative) and the
philosophical belief systems that underlie the research
decisions. Criteria for critically analyzing the
strengths and weaknesses of mixed methods research include the
criteria for the individual approaches
(e.g., case studies or surveys), as well as consideration of the

rationale for and implementation of the
mixed methods themselves.
Notes
1. Multiple methods studies are considered those that use more
than one method, but the methods chosen are
singularly either quantitative or qualitative. Teddlie and
Tashakkori (2009) label this monomethod “multi-strand
research.”
2. In Teddlie and Tashakkori (2010), several typologies of
mixed methods designs are presented in the various
chapters. They note that the formal field of mixed methods
research is in its adolescence, and therefore it is to be
expected that various typologies would be present in the
literature. Main points of definition appear to be the
temporal relation between the quantitative and qualitative data
collections (parallel vs. sequential), the
philosophical paradigm underlying the work (e.g., pragmatic or
transformative), the priority given to either
qualitative or quantitative approaches (Creswell, 2009 described
this as dominant, subdominant relations), and the
purpose of the research (confirmatory or exploratory). If design
options were presented for each possible
combination of these dimensions, then there would be a large
number of possible designs for mixed methods. For
the sake of parsimony, I elected to include five major categories
for the major types of mixed methods designs.
p. 317
Psychology:
tative, Q

ualitative, and M
ixed M
ethods
A L A N B RY M A N
University of Leicester
A B S T R A C T This article seeks to move beyond typologies
of the ways
in which quantitative and qualitative research are integrated to
an
examination of the ways that they are combined in practice. The
article is based on a content analysis of 232 social science
articles in
which the two were combined. An examination of the research
methods and research designs employed suggests that on the
quantitative side structured interview and questionnaire
research
within a cross-sectional design tends to predominate, while on
the
qualitative side the semi-structured interview within a
cross-sectional design tends to predominate. An examination of
the
rationales that are given for employing a mixed-methods
research
approach and the ways it is used in practice indicates that the
two
do not always correspond. The implications of this finding for
how
we think about mixed-methods research are outlined.
K E Y W O R D S : qualitative research, quantitative research,

mixed-methods
research, multi-strategy research, typologies
There can be little doubt that research that involves the
integration of quanti-
tative and qualitative research has become increasingly common
in recent
years. While some writers express unease about the ‘whatever
works’ position
that underpins it (e.g. Buchanan, 1992; Pawson and Tilly,
1997), so far as
research practice is concerned, combining quantitative and
qualitative
research has become unexceptional and unremarkable in recent
years. Indeed,
for some writers it has come to be seen as a distinctive research
approach in its
own right that warrants comparison with each of quantitative
and qualitative
research. In this sense, we end up with three distinct approaches
to research:
quantitative; qualitative; and what is variously called multi-
methods
(Brannen, 1992), multi-strategy (Bryman, 2004), mixed methods
(Creswell,
A R T I C L E 97
DOI: 10.1177/1468794106058877
Integrating quantitative and qualitative
research: how is it done?
Q
R

Qualitative Research
Copyright © 2006
SAGE Publications
(London,
Thousand Oaks, CA
and New Delhi)
vol. 6(1) 97–113.
2003; Tashakkori and Teddlie, 2003), or mixed methodology
(Tashakkori and
Teddlie, 1998) research. In the field of evaluation research, and
indeed in
several other applied fields, the case for a multi-strategy
research approach
seems to have acquired especially strong support (Tashakkori
and Teddlie,
2003).
Typologies of mixed-methods research
The discussion of the integration of quantitative and qualitative
research has
increasingly been taken over by a formalized approach which is
especially
apparent in the discussion and proliferation of typologies of
integration. This
has been a particular emphasis among North American
contributors to the
field. Creswell et al. (2003) argue that giving types of mixed-
methods research
names has certain advantages. It conveys a sense of the rigour
of the research
and provides guidance to others about what researchers intend
to do or have

done (for example, funding bodies and journal editors). To that
extent, the
typologies of mixed-methods or multi-strategy research can be
helpful to
researchers and writers in clarifying the nature of their
intentions or of their
accomplishments.
However, the variety and range of typologies has reached the
point where
these exercises have become almost too refined, bearing in mind
that the range
of concrete examples of multi-strategy research is not great.
Indeed, most of
the typologies have been constructed in largely theoretical
terms and have not
been apparently influenced in a systematic way by examples of
multi-strategy
research. To a large extent, they are exercises in logically
possible types of inte-
gration, rather than being built up out of examples.
However, the dimensions out of which the typologies are
constructed are
instructive, in that they draw attention to the different aspects
of multi-
strategy research:
1. Are the quantitative and qualitative data collected
simultaneously or
sequentially? (Morgan, 1998; Morse, 1991).
2. Which has priority – the quantitative or the qualitative data?
(Morgan,
1998; Morse, 1991).

3. What is the function of the integration – for example,
triangulation,
explanation, or exploration? (Creswell, 2003; Creswell et al.,
2003;
Greene et al., 1989).
4. At what stage(s) in the research process does multi-strategy
research
occur? (Tashakkori and Teddlie, 1998). It may be at stages of
research
question formulation, data collection, data analysis, or data
interpret-
ation.
5. Is there more than one data strand? (Tashakkori and Teddlie,
2003). With
a multi-strand study, there is more than one research method
and hence
source of data. With a mono-strand study, there is one research
method
Qualitative Research 6(1)98
and hence one source of data. However, whether a mono-strand
study can
genuinely be regarded as a form of mixing methods is
debatable.
A further issue with the use of these typologies is that they
imply some
forward commitment to a type of design, much like a decision to
employ an
experimental research design entails a commitment to
uncovering data of a

particular type. However, as some authors observe (e.g.
Erzberger and Kelle,
2003), the outcomes of multi-strategy research are not always
predictable.
While a decision about design issues may be made in advance
and for good
reasons, when the data are generated, surprising findings or
unrealized poten-
tial in the data may suggest unanticipated consequences of
combining them.
What do we know about mixing quantitative and qualitative
research?
The exercise of specifying typologies co-exists with unease
among some
authors about what we actually know about the ways in which
quantitative
and qualitative research are combined in practice. For example,
it has been
suggested that there are relatively few guidelines about ‘how,
when and why
different research methods might be combined’ (Bryman, 1988:
155).
Maxwell has suggested that ‘the theoretical debate about
combining methods
has prevented us from seeing the different ways in which
researchers are
actually combining methods’ (Maxwell, 1990: 507, cited in
Maxwell and
Loomis, 2003: 251). He and Loomis have argued further that:
Uncovering the actual integration of qualitative and quantitative
approaches in
any particular study is a considerably more complex
undertaking than simply

classifying the study into a particular category on the basis of a
few broad dimen-
sions or characteristics. (Maxwell and Loomis, 2003: 256)
Remarks such as these suggest that the formalization of
approaches to multi-
strategy research through typologies has moved too far ahead of
a systematic
appreciation of how quantitative and qualitative research are
combined in
practice. The writers who adopt a formalized strategy use many
examples to
illustrate their ‘types’ but we have relatively little
understanding of the
prevalence of different combinations, though there are some
exceptions to this
statement (e.g. Greene et al., 1989; Niglas, 2004).
An investigation of articles combining quantitative and
qualitative research
With these kinds of consideration in mind, an investigation was
undertaken of
the ways that quantitative and qualitative research are combined
in published
journal articles. The findings reported in this article derive from
only one phase
of this research project, albeit a major component of it –
namely, a content
Bryman: Integrating quantitative and qualitative research 99
analysis of articles based on multi-strategy research. Journal
articles do not

encapsulate all possible contexts in which projects reporting
multi-strategy
research might be found. Conference papers and books are other
possible sites.
However, journal articles are a major form of reporting findings
and have the
advantage that, in most cases, the peer review process provides
a quality
control mechanism. By contrast, conference papers and books
are sometimes
not peer reviewed.
The approach to gleaning a sample was to search the Social
Sciences
Citation Index (SSCI) for articles in which relevant key words
or phrases such
as ‘quantitative’ and ‘qualitative’, or ‘multi(-)method’, or
‘mixed method’, or
‘triangulation’ appeared in the title, key words, or abstract. This
means that the
sample comprises articles which to some degree foreground the
fact that the
study is based on both quantitative and qualitative research.
Searches using
other kinds of key words, such as ‘survey’ and ‘ethnography/ic’,
produce a far
larger sample of articles than could be dealt with within the
purview of this
investigation. In conducting the search, the emphasis was on
uncovering
articles in five fields: sociology; social psychology; human,
social and cultural
geography; management and organizational behaviour; and
media and
cultural studies. The analysis was restricted to the 10-year
period of

1994–2003. The fact that the findings are based on a large
corpus of articles
suggests that the sample is unlikely to be overly atypical,
although claims of
representativeness would be impossible to sustain. Judgments
about whether
articles fell within the purview of the investigation, in terms of
whether they
could be regarded as deriving from the five fields, were made
on the basis of the
journal title or information supplied in abstracts. In this way, a
total of 232
articles was generated and content analyzed.
What was and was not an example of the combination of
quantitative and
qualitative research was occasionally problematic. The most
notable of these
occasions had to do with cases in which the researcher claimed
to have used a
qualitative approach or to be using qualitative data, but in fact
the ‘qualitative
data’ were based on a quantitative analysis of unstructured data
– for example,
of responses to open questions. Articles in which this occurred
and where such
data were the only source of the qualitative component were not
included in
the sample, because it is very debatable whether they can be
regarded as indica-
tive of a qualitative approach. This kind of quantification of
qualitative data is
more properly regarded as indicative of a quantitative research
approach.
Indeed, in some articles that were included in the sample, this
kind of process

was depicted by authors as indicative of a quantitative research
approach
rather than a qualitative one. There is clearly some confusion
concerning
whether the quantification of qualitative, unstructured data is
indicative of a
quantitative or a qualitative research approach. For the purposes
of sample
selection, it was taken to be the former, regardless of authors’
claims. However,
this was not a very common occurrence; although a log was not
kept of these
cases, they number no more than five or six articles.
The sample is likely to be biased in the sense that by no means
all authors of
articles reporting multi-strategy research foreground the fact
that the findings
reported derive from a combination of quantitative and
qualitative research,
or do not do so in terms of the key words that drove the online
search strategy.
An alternative search strategy is to select a sample of journals
and to search
for articles exhibiting multi-strategy research. This tactic was
employed by
Niglas (2004) in her investigation of multi-strategy research in
education. Her
sample of 145 articles derived from 1156 articles in 15 journals.
This is a very
good way of generating a sample but, in the context of a study
that is meant

to cover five fields of study, it is difficult to implement and
also results in a lot
of redundancy because a large number of articles have to be
read in order to
establish whether they are based on both quantitative and
(only 12.5 percent of articles read for Niglas’s study were
relevant to the main
focus of her investigation). Moreover, foregrounding that a
study is based on
multi-strategy research is interesting because it implies that the
fact that the
different sources of data were employed is important and
significant to the
author(s) concerned. Since a major focus of the research was the
kinds of
purposes to which multi-strategy research is put, the online
search strategy
that was used for the study reported here was very relevant,
because we might
anticipate that researchers who choose to emphasize this aspect
of their
studies will have given greater consideration to the issues
involved in combin-
ing quantitative and qualitative research. In this sense, the
articles from which
the findings derive constitute a purposive sample. A further
issue that suggests
some advantages to the sampling approach taken for this article
is that it allows
articles in a wide variety of journals to be uncovered. Thus,
while it is certainly
possible to trawl through sociology journals for instances of
multi-strategy
research articles in sociology, such a process risks neglecting
many relevant

sociology articles appearing in specialist journals.
Several writers have pointed out that quantitative and
can be combined at different stages of the research process:
formulation of
research questions; sampling; data collection; and data analysis.
Articles for
this study were chosen in terms of data collection and data
analysis and then
content analyzed in relation to these aspects of the research
process. Issues of
sampling did materialize in the study, as the findings below will
indicate. Data
collection and analysis were emphasized because these are
arguably defining
features of quantitative and qualitative research. Moreover,
multi-strategy
research articles nearly always entail the collection and analysis
of both quan-
titative and qualitative data (Niglas, 2004).
B A C KG RO U N D F I N D I N G S
First, a small number of background features of the articles
analyzed thus far
will be mentioned. When the primary discipline of each article
is examined, we
find that the major contributing discipline is sociology with 36
percent of all
articles. This is followed by social psychology (27%);
management and

organizational behaviour (23%); geography (8%); and media
and cultural
studies (7%). These findings strongly suggest that multi-
strategy research is
more commonly practised in some disciplines than others.
A further interesting background characteristic is the nation of
the insti-
tutional affiliation of the author or first author of each article.
North America
is the major contributor with 49 percent of all articles; the UK
comes second
with 27 percent; followed by Europe and Australia (8% and
7%); Middle East
(4%); with Asia, Africa and Latin America contributing 3
percent between
them. These figures are obviously significantly affected by the
fact that only
English language publications were sought and read.
R E S E A RC H M E T H O D S A N D R E S E A RC H D E
S I G N S U S E D
The first issue to be addressed in this article is: what research
methods and
research designs were employed in the articles? Each article
was coded in terms
of the research methods that were employed. Some of the
research methods
are perhaps better thought of as methods of data analysis, but
they are
frequently portrayed as research methods because of their
distinctive
approaches to sampling or capturing data (for example, content
analysis,
discourse analysis and conversation analysis). Table 1 presents
the main

methods used. The analysis presented derives from a multiple
response analysis
using SPSS.
A striking feature is that a small number of methods account for
the vast
majority of all methods employed. Survey methods and
qualitative interviews
account for the vast majority of methods employed in the
articles. If we
aggregate self-administered questionnaire, structured interview
and
questionnaire/structured interview (a category used when it was
unclear
T A B L E 1 . Research methods employed
Number of articles using
Self-administered questionnaire 121
Structured interview 52
Structured observation 3
Content analysis 18
Quantification of qualitative interview questions 15
Questionnaire/structured interview 18
Semi-structured interview 159
Participant observation/ethnography 14
Unstructured interview 6
Qualitative analysis of documents 28
Answers to open questions in questionnaire 48
Focus groups 33
Language-based analysis 5
Other method 55

how survey instruments were administered), 82.4 percent of all
articles
coded used a survey instrument. If we aggregate semi-structured
interview
and unstructured interview, we find that data for 71.1 percent of
articles
derived from either of these two ways of conducting qualitative
interviews.
Further, 57.3 percent of all articles are based on a combination
of a survey
instrument and qualitative interviewing. In other words, one
combination of
research methods predominates in this data set – that is, one in
which data
are collected by either structured interview or questionnaire on
the quantita-
tive side along with either a semi-structured or unstructured
interview on the
qualitative side.
A further feature is that with 6.5 percent of articles, the
quantitative data
derive from an individual qualitative or focus group interview
and that, in 20.7
percent of articles, the qualitative data derive from open
questions in a struc-
tured interview or self-administered questionnaire. In the
former case, the
quantitative data derive from a research instrument associated
with qualitative
data collection while, in the second case, the qualitative data
derive from a
research instrument associated with quantitative data collection.
In other

words, for around 27 percent of articles, the collection of
quantitative and
qualitative data was not based on the administration of separate
research
instruments.
This finding is interesting because some methodologists might
argue that a
combination of quantitative and qualitative data based on the
administration
of one research instrument does not represent a true integration
of quantita-
tive and qualitative research because one will tend to be
subordinate to the
other. Thus, when multi-strategy research derives from the
administration of
a semi-structured interview, some of whose questions are
quantified, an
argument might be levelled that this does not represent a
genuine form of
quantitative research since the data have not been gathered in
line with its
underlying principles. Similarly, it might be argued that asking
a small number
of open questions in the course of a structured interview does
not really
provide an instance of multi-strategy research because the
qualitative data
have been collected in the course of administering a research
instrument that
has been devised in terms of survey principles. Moreover, such
a situation
requires a modification of approach to answering questions on
the part of
respondents in the course of responding to a research
instrument. However,

articles adopting an approach in which quantitative and
qualitative data
derived from the same research instrument were included.
When we turn to research designs, the aim of the analysis was
to code
articles in terms of the design employed for the quantitative
data and the
design employed for the qualitative data. In a small number of
cases (4),
because of the complexity of the data, a third research design
was coded.
Table 2 presents the data on this aspect of the investigation
using a classifi-
cation that follows Bryman’s (2004) categorization of research
designs. In this
classification, a study is treated as a case study if it involves
just a single case.
If it was a multiple case study, involving two or more cases, it
was treated as a
comparative design. Again, the analysis derived from a multiple
response
analysis using SPSS.
As one might expect from the findings in Table 2, the bulk of
the studies
employed a cross-sectional design for the collection of both the
quantitative
and the qualitative data. Experimental and quasi-experimental
designs barely
figure in the findings. Employing a cross-sectional design for

the collection of
both quantitative and qualitative data is by far the most common
design combi-
nation (62.9% of all articles). When we put the data relating to
research
methods and research designs together, we find that 41.8
percent of all articles
included both a survey instrument and personal qualitative
interviewing
within a cross-sectional design for the collection of both sets of
data. Some-
times, although rarely, this format will have been accompanied
by other
sources of data.
J U S T I F I C AT I O N S F O R C O M B I N I N G Q UA N
T I TAT I V E A N D
Q UA L I TAT I V E R E S E A RC H
A major focus of the content analysis was on the rationales
proffered for
combining quantitative and qualitative research. This aspect of
the investi-
gation was approached in several ways. First, the rationale
given by authors for
combining the two approaches to data collection and/or analysis
was coded. For
this exercise, the reasons that were given before the findings
were presented
were typically examined. Then, the ways in which quantitative
and qualitative
research were actually combined were coded. In doing so, the
coding reflected
authors’ reflections on what they felt had been gleaned from
combining quan-
titative and qualitative research, and any ways in which the two
were combined

which were not reflected in authors’ accounts. The purpose of
discriminating
between these two ways of thinking about the justification for
multi-strategy
research was that authors’ accounts of why they intended to
combine quanti-
tative and qualitative research might differ from how they
actually combined
T A B L E 2 . Research designs employed
Number of articles using
Cross-sectional design 1 169
Cross-sectional design 2 148
Case study 1 24
Case study 2 16
Longitudinal 1 28
Longitudinal 2 19
Experimental 1 (includes quasi-experimental) 9
Experimental 2 (includes quasi-experimental) 5
Comparative 1 (includes multiple case study) 30
Comparative 2 (includes multiple case study) 19
them in practice. If a difference was sometimes found between
the two accounts
(that is, between the rationale and practice), this would be
interesting because
the scientific paper is often perceived among sociologists of
science as an ex post
facto reconstruction that rationalizes and injects coherence into
the different

elements of the research process (e.g. Gilbert and Mulkay,
1984).
In coding the justifications for combining quantitative and
qualitative
research, two different schemes were employed. First, the
influential scheme
devised in the context of evaluation research by Greene et al.
(1989) was used.
This scheme isolates five justifications for combining
quantitative and quali-
tative research:
1. Triangulation: convergence, corroboration, correspondence or
results
from different methods. In coding triangulation, the emphasis
was placed
on seeking corroboration between quantitative and qualitative
data.
2. Complementarity: ‘seeks elaboration, enhancement,
illustration, clarifi-
cation of the results from one method with the results from
another’
(Greene et al., 1989: 259).
3. Development: ‘seeks to use the results from one method to
help develop or
inform the other method, where development is broadly
construed to
include sampling and implementation, as well as measurement
decisions’
(Greene et al., 1989: 259).
4. Initiation: ‘seeks the discovery of paradox and contradiction,
new perspec-

tives of [sic] frameworks, the recasting of questions or results
from one
method with questions or results from the other method’
(Greene et al.,
1989: 259).
5. Expansion: ‘seeks to extend the breadth and range of enquiry
by using
different methods for different inquiry components’ (Greene et
al., 1989:
259).
This scheme has been quite influential and was employed by
Niglas (2004)
in her examination of education research articles. In their
analysis of evalu-
ation research articles, Greene et al. (1989) coded each article
in terms of a
primary and a secondary rationale, a procedure that was also
employed by
Niglas (2004). An advantage of the Greene et al. scheme is its
parsimony, in
that it boils down the possible reasons for conducting multi-
strategy research
to just five reasons, although the authors’ analysis revealed that
initiation was
uncommon. A disadvantage is that it only allows two rationales
to be coded
(primary and secondary). Accordingly, a more detailed but
considerably less
parsimonious scheme was devised. It was based on an extensive
review of the
kinds of reasons that are frequently given in both
methodological writings and
research articles for combining quantitative and qualitative
research. The

scheme provided for the following rationales:
a) Triangulation or greater validity – refers to the traditional
view that quan-
titative and qualitative research might be combined to
triangulate
findings in order that they may be mutually corroborated. If the
term was
used as a synonym for integrating quantitative and qualitative
research,
it was not coded as triangulation.
b) Offset – refers to the suggestion that the research methods
associated with
both quantitative and qualitative research have their own
strengths and
weaknesses so that combining them allows the researcher to
offset their
weaknesses to draw on the strengths of both.
c) Completeness – refers to the notion that the researcher can
bring together
a more comprehensive account of the area of enquiry in which
he or she
is interested if both quantitative and qualitative research are
employed.
d) Process – quantitative research provides an account of
structures in social
life but qualitative research provides sense of process.

e) Different research questions – this is the argument that
quantitative and
qualitative research can each answer different research
questions but this
item was coded only if authors explicitly stated that they were
doing this.
f) Explanation – one is used to help explain findings generated
by the other.
g) Unexpected results – refers to the suggestion that
quantitative and quali-
tative research can be fruitfully combined when one generates
surprising
results that can be understood by employing the other.
h) Instrument development – refers to contexts in which
is employed to develop questionnaire and scale items – for
example, so that
better wording or more comprehensive closed answers can be
generated.
i) Sampling – refers to situations in which one approach is used
to facilitate
the sampling of respondents or cases.
j) Credibility – refers to suggestions that employing both
approaches
enhances the integrity of findings.
k) Context – refers to cases in which the combination is
rationalized in terms
of qualitative research providing contextual understanding
coupled with
either generalizable, externally valid findings or broad

relationships
among variables uncovered through a survey.
l) Illustration – refers to the use of qualitative data to illustrate
quantitative
findings, often referred to as putting ‘meat on the bones’ of
‘dry’ quanti-
tative findings.
m) Utility or improving the usefulness of findings – refers to a
suggestion,
which is more likely to be prominent among articles with an
applied focus,
that combining the two approaches will be more useful to
practitioners
and others.
n) Confirm and discover – this entails using qualitative data to
generate
hypotheses and using quantitative research to test them within a
single
project.
o) Diversity of views – this includes two slightly different
rationales – namely,
combining researchers’ and participants’ perspectives through
quantitative and qualitative research respectively, and
uncovering
relationships between variables through quantitative research
while also
revealing meanings among research participants through

qualitative
research.
p) Enhancement or building upon quantitative/qualitative
findings – this
entails a reference to making more of or augmenting either
quantitative
or qualitative findings by gathering data using a qualitative or
quantita-
tive research approach.
q) Other/unclear.
r) Not stated.
This classification includes a larger number of categories than
other
schemes and as such is meant to capture in finer detail the range
of reasons
that are given for conducting multi-strategy research. There are
clearly
symmetries between the Greene et al. scheme and the more fine-
grained
approach just outlined. For example, ‘development of a research
instrument’
and ‘for sampling/case study selection reasons’ correspond to
‘development’,
while ‘to enhance or build upon quantitative/qualitative
findings’ corresponds
to ‘complementarity’.
Table 3 shows the distribution of articles in terms of just the
primary
rationale using the Greene et al. scheme (see column for
‘Rationale’). In just
over a quarter of all articles, no rationale was provided.
Complementarity and

expansion were the most frequently cited primary rationales
with 29 percent
and 25 percent of all articles mentioning each of them as a
primary rationale.
Triangulation and development were less commonly mentioned,
while initia-
tion was extremely uncommon. The latter was also the case in
Greene et al.
(1989) but was even more the case in the data reported here.
Turning to the
actual uses of the integration of quantitative and qualitative
research, Table 3
(column for ‘Practice’) provides the primary use in terms of the
Greene et al.
scheme. All the frequencies are greater because the category of
‘not stated’
virtually disappears. Most striking is that nearly half of all
articles can be
subsumed into the complementarity category. In terms of the
Greene et al.
scheme, this is by far the most prominent primary approach to
the integration
of quantitative and qualitative research.
T A B L E 3 . Uses of multi-strategy research – Greene et al.
scheme
Category Rationale % Practice
Triangulation 7.8 12.5
Complementarity 28.9 44.8
Development 10.3 8.6
Initiation 0.4 1.3
Expansion 25.4 31.5

Not stated 27.2 1.3
Note: All percentages are based on 232 cases
When the data for the ‘practice’ column in Table 3 using the
Greene et al.
scheme are contrasted with comparable data from Greene et al.
(1989) and
Niglas (2004), we find that the pattern is closer to Niglas’s
examination of
educational articles than to that of Greene et al. Similarly to
Niglas, comple-
mentarity is the most common use of multi-strategy research,
followed by
expansion. In the examination of evaluation research articles by
Greene et al.,
it was the other way around in that expansion was more
common that
complementarity. In the case of all three studies, then, these two
uses of multi-
strategy research were the most common forms. ‘Development’
as a use occurs
with noticeably greater frequency in educational articles than in
the present
study and that of Greene et al. These findings suggest that there
are slightly
different uses being made of multi-strategy research when we
compare general
social research (present study), evaluation research and
educational research,
though the differences are not great.
When the articles are examined in terms of the more detailed
scheme

592019 1 Case Study KY LOGISTICS & DISTRIBUTION COM.docx

592019 1 Case Study KY LOGISTICS & DISTRIBUTION COM.docx

Recommended

Recommended

More Related Content

Similar to 592019 1 Case Study KY LOGISTICS & DISTRIBUTION COM.docx

Similar to 592019 1 Case Study KY LOGISTICS & DISTRIBUTION COM.docx (20)

More from troutmanboris

More from troutmanboris (20)

Recently uploaded

Recently uploaded (20)

592019 1 Case Study KY LOGISTICS & DISTRIBUTION COM.docx