Larry Culp, the brand manager for BigHoney cereal, requests $500,000 for a new advertising campaign highlighting updated packaging that improves freshness. The CFO, Mark Weinberg, questions if the campaign is necessary and if the same results could be achieved for less. Culp can design marketing experiments to test the effectiveness and optimal budget for the campaign by tracking sales of test and control groups exposed to different advertisements.

1. Business Case #2
DESIGNING MARKETING EXPERIMENTS
Introduction
In his quarterly budget presentation, Larry Culp, brand manager
for BigHoney cereal, requests
funds for an advertising campaign highlighting new packaging
that retains freshness better and
longer. In response, Mark Weinberg, BigHoney CFO, asks Culp,
“Can you convince me that sales
of BigHoney will be hurt if you do not advertise?” As a follow-
up, he asks, “You have requested
$500,000 for a national campaign? Is that the right amount? Can
you get the same result for
$250,000?” How can Culp convince Weinberg?
Culp’s challenge is typical for marketing managers who need to
invest money in the marketing
mix with the expectation that sales will increase in the future.
Attributing an increase in sales to a
specific marketing action is a major challenge, because the
effect on a brand of any single

2. marketing activity is difficult to isolate; it consists of several
levers being pulled at the same time,
including price promotion, new product introductions,
competitive actions, television advertising,
PR events, and seasonality. Also, sales resulting from such
inputs take time; a television
advertisement is unlikely to compel a viewer to immediately
jump off the couch, run to the store,
and buy a soda. Isolating the influence of a specific event on
consumer behavior can be a daunting
task.
One way to isolate such influences is through experiments. In
our example, Culp could, on a
smaller scale, measure the effect of his proposed campaign on
the brand’s sales. Return on
investment (ROI)—and a prospective budget—could then be
estimated by projecting any
identified lift to a national scale. But how does one design an
experiment that would provide
accurate results?
Establishing Causality
Four key rules determine a causal relationship between two
variables or factors. Let us consider

3. Culp’s challenge. The marketing campaign may be considered
effective if:
the sales figures.
subsequent time periods.
competitive action) affecting the
sales of BigHoney.
To clearly ascertain whether the targeted sales increase could be
achieved without spending
marketing dollars or whether any marketing spend is warranted,
it becomes very important for
Culp to establish causality.
After-Only Experiment
An experiment provides a mechanism to manipulate one or more
input factors while controlling
all other factors and observe changes in an output of interest,
such as sales or brand awareness.
Business Case #2

4. A very basic experiment that can be designed by Culp is
illustrated in Figure 1. Culp recruits 1,000
participating customers, half of whom—the test group—are
exposed to the new advertisement
highlighting the new packaging technology; the other half—the
control group— are exposed to
the old advertisements. If cereal purchases by all 1,000
customers are tracked, the difference in
sales between the test and control groups will indicate the
magnitude of the potential sales lift
provided by the new advertising campaign. Such an
experimental design is called after-only
because we measure the sales of BigHoney among the
participants of the experiment only after
they are exposed to the advertisement.
Figure 1. After-only experiment design.
The after-only design satisfies two of the four conditions for
causality: sales increase in the short
term and in subsequent periods. It cannot indicate whether the
increase might have occurred
without the new advertisement, nor whether preference
differences existed prior to the experiment.
The underlying issue here is the extent to which the participants

5. in the test and control groups are
similar in terms of the factors relevant to the experiment. The
more similar the factors are, the
more the two pools of subjects are exposed to the same external
environment— store promotions,
competitive reactions, even the same weather—and then the
causal inferences are more reliable,
because the only difference would be the advertisement
campaign to which each was exposed.
Therefore, the experimenter could, with confidence, attribute a
causal relationship between the
marketing input and product sales.
Test and Control Group Participants
For the experimenter, deciding how to distribute customers
between the test group and the control
group is critical. There are two primary ways to select control
groups: randomization and attribute
matching. Randomization involves allocating participants
randomly between the test group and
the control group. With a big enough sample size,
randomization will help improve the similarity
between the test and control groups. Consider that Culp is using
an e-mail advertisement campaign

6. Business Case #2
and has at his disposal a list of 1,000 BigHoney customers. In a
random assignment, he would
assign every other customer to the test group and the rest of the
customers to the control group.
Randomization creates fairly homogeneous test and control
groups because it removes all sou rces
of extraneous variation, which are not controllable by the
experimenter. The chance that the test
and control groups end up being different even with random
assignment decreases as the sample
size increases. For most practical marketing applications, a
sample size needs to exceed at least
100 participants for a reliable random assignment process.
Attribute matching is used when the available sample size is not
large enough to permit random
assignment. Participants are assigned based on certain known
attributes such as demography,
geography, or annual income. If Culp were testing the effect of
a TV advertisement campaign, he
would be better off choosing cities that are similar in key
demographic or psychographic attributes

7. critical to BigHoney’s sales.
Before-After Experiment
A before-after design (Figure 2) requires an experimenter to
measure the output of interest both
before and after the participants have been exposed to the
inputs.
In this before-after design for an e-mail advertising campaign,
Culp would randomly divide the
1,000 participants into test and control groups, as with the after
-only design. Both groups would
be exposed to the old advertising campaign, and sales in the
respective campaigns would be
the test and control groups when they
are exposed to the old campaign. The test group is then exposed
to the new advertising campaign,
whereas the control group is still exposed to the old campaign.
The difference in sales between the
sales due to the new advertising
differences between the test and

8. control groups. This before-after design, along with random or
matched assignment of participants,
provides a belt-and-suspenders approach for controlling for all
external differences between test
and control groups.
Figure 2. Before-after experiment design.
Business Case #2
If BigHoney cereal is sold to retailers for $1.59, and the cost of
goods is 99 cents, the unit
contribution equals $0.60. The lift of 100 units in the
experiment translates to $60. If the cost of a
single e-mail sent to a customer is $0.10, the cost of e-mails to
500 test-group customers is $50.
The experiment suggests that the e-mail campaign provides an
ROI of 20%. This ROI estimate
can be used by Culp to plan national campaigns. Alternatively,
if 500 e-mails provide $10 in
contribution, the contribution from a single e-mail is $0.02. If
the target lift from a national
campaign is $100,000, the experiment suggests that Culp would
require 5 million e-mails to attain

9. the target lift.
Field Experiments
When experiments are conducted within a natural setting, they
are termed field experiments. In
some industries, field experiments are a part of everyday
business. Retail outfits regularly use
catalogs or e-mails to conduct massive field experiments that
assess consu mer price sensitivity and
optimal catalog design. One advantage of field experiments is
that subjects are seldom aware that
they are part of an experiment, so the collected data is more
likely to represent the realities
prevalent in the marketplace. The disadvantage is that it is very
difficult to control extraneous
variables or to manipulate inputs precisely.
Business Case #2
Field experiments are also very transparent to the competition,
and competitive reaction could
cloud the results. If, during Culp’s experiment, his competition,
BigSugar, launches a promotion,
the results could be overly pessimistic. But field experiments

10. are still preferred because they allow
the marketer to test a campaign with customers in a natural
setting, increasing the accuracy of any
prediction. In general, it is easier to experiment with pricing,
product, or promotion decisions than
with place or channel management decisions.
If Culp also wants to determine the best price for his new
packaging, he can create different test
conditions where advertising, promotions, and coupons are all
the same, and the only difference is
price. As part of the experiment, Culp could introduce
BigHoney with its new packaging in three
cities (selected because they are similar in factors that affect
BigHoney’s sales). The only
difference is that the products are priced differently in each
city: $1.59, $1.89, and $2.15. The sales
figures are then tracked in the three cities over time. The city
where the product is priced at $1.59
could be expected to have higher sales volume—the question
will be how much higher? The
experiment results, based as before on the original cost of goods
sold being 99 cents, are given in
Table 1:

11. Table 1. BigHoney sales data.
Given this data, Culp is better off introducing the product at the
$1.59 price point, because it
provides the highest profit.
Web Experiments
Because they can be executed quickly and cheaply, web
experiments have gained a significant
edge over traditional offline field experiments. Consider the
difference between TV and e-mail
advertising campaigns for Culp. With TV advertising, Culp has
to buy spots in different channels
in the test markets with significant lead time. Once the video is
shot, it is difficult, time-consuming,
and very expensive to change it. Furthermore, the cost of the
experiment increases rapidly with
each new version that Culp would like to test. The e-mail
advertisement, on the other hand, can be
created much more quickly and at a much lower cost, making it
easier and less expensive for Culp
to test different versions.
The faster execution and lower cost of web experiments allows
marketers to easily test the

12. simultaneous influence of multiple inputs. When Culp wants to
test three differ ent campaigns,
“Lasts Longer,” “Tastes Better,” and the current campaign,
“Good for You,” each at three different
price points ($1.59, $1.89, and $2.15), he can create the full
factorial design shown in Table 2.
Table 2. Full factorial design.
Business Case #2
Each cell in Table 2 represents a combination of advertisement
copy and price point that is tested
in the experiment. Since we are testing three types of copy and
three price points, the total
possibilities that need to be tested are 3 × 3 = 9. The profit from
each combination is provided
within each cell.
A TV advertisement campaign would have required Culp to
recruit nine different cities for the
experiment (one for each cell), as well as retailers in each city
willing to manipulate the prices, a
very expensive and time-consuming process. If BigHoney is
sold direct to consumers through a

13. website, then Culp can randomize the e-mails sent to consumers
to match one of the nine cells in
Table 2. The e-mail open rates, click-throughs to the website,
and the subsequent purchases when
consumers visit the website can be tracked for each consumer.
This provides Culp with a much
stronger sense of the effectiveness of the campaign because the
same consumer is tracked from
exposure to purchase.
The full factorial design also allows Culp to test combinations
of the advertisement message and
price point. In Table 2, we see that the “Tastes Better” message
with a price point of $2.15 provides
the highest profit, followed by the “Lasts Longer” campaign at
the $1.59 price point. W e also see
that when the price is maintained at the current level of $1.89,
the “Lasts Longer” campaign
provides the highest profit. At least in this case, had Culp tested
only the advertisement campaigns
and not the different prices, he would have wrongly concluded
that the “Lasts Longer” copy
provided the highest profits.
Natural Experiments

14. In a natural experiment, a marketer observes the effect of
certain naturally occurring incidents on
customer behavior and other factors, such as sales volume.
Recognizing such occurrences allows
companies to learn about their customers at no or little
additional expense. A classic example is
Amazon collecting sales tax data from California residents.
Analyzing the effect of a newly levied
tax on sales volume will give Amazon an opportunity to
discover how a sales tax affects online
retailing. Amazon could compare sales before and after the
sales tax introduction for customers
who lived on either side of the state’s border. The only change
would be the newly introduced
taxation of online purchases, which affects consumers only on
one side of the border.
The most important part of identifying and analyzing natural
experiments is to find test and control
groups created by some external factor. Many marketers resort
to geographic segmentation for
natural experiments, but it will not always be a distinguishing
characteristic. For example, when
the Ford Motor Company introduced an employee pricing
promotion, there was no natural

15. geographic separation; all customers were offered the same
deal. Instead, marketers compared
sales in the weeks immediately before and after the program
was introduced.
Business Case #2
Ford Motor Company discovered that the jump in sales levels
was accompanied by a sharp increase
in prices. Customers presumed that they were getting a good
deal, but the prices on many models
were actually lower before the promotion than at the time of the
employee discount prices. But
customers responded to the promotion despite the prices, not
because of the prices. The program
led to many happy customers, even though they were paying
higher prices.
Challenges
The accuracy of data obtained from a marketing experiment
increases with the experiment’s
duration. For example, experiments that have shorter durations
might not adequately a ccount for
the carry-over effects of marketing interventions; however,
marketing decisions are mostly

16. sensitive to time, highlighting the tension between quick versus
accurate decisions. The longer the
gap between the field experiment and the full campaign, the less
accurate the prediction from the
field experiment. Yet the time required to obtain buy-in for the
field experiment results could delay
the timing of the full campaign and thereby the relevance of the
field experiment. If an experiment
involves salespeople, the mere knowledge of being in an
experiment could change their behavior
(the demand effect), leading to biased conclusions.
Experiments provide a bridge between new ideas and
management decisions. Digital marketing
has popularized experimentation in the marketing community.
Organizations can succeed if they
develop a system to learn from experiments and strive toward
continuous improvement.

17. Business Case #2
Questionnaire
Q1.
Mark Weinberg is quite correct in asking if the budget of
$500,000 for an advertising campaign
could be streamlined, what systems are in place that would
prove beyond doubt that a cheaper
campaign would have less impact on sales?
Q2.
As stated in the case that not launching the campaign does not
affect sales then how much increase
in sales can be projected if a budgeted campaign of half a
million dollars is given the go ahead and
would it be cost effective?
Q3.
If the test group of 1000 people are in full knowledge of the
product design changes would this
give a true account of the alterations in buying behavior or
would it merely suggest that because

18. of the trial behaviors would revert back to the original state
when it was completed?
Q4.
When considering the description of figure 2 and the cost of e-
mails against the uplift in sales can
it truly be said that such a test of a pool of 1000 customers is
going to make such an impact on
overall sales?
Q5. (True/False)
The inception of web advertising proves to be a much more cost
effective than that of television
ads, campaigns can be altered with minimal cost and can be
directly targeted to existing customers
by the use of emails, and with the use of a "read" receipt the
data can be collected more precisely.
Q6.
Advertising is such a vital tool in the promotion of any product,
especially in such a market as
food sales, that constant updating of campaigns are required. So
easily can one product have sales
reduced by a more intensive campaign by a rival?
Q7.

19. When producing an advertising campaign is the duration of the
campaign an exacting factor, as in,
will the length of the run be exponentially more beneficial is
extra sales or will the growth plateau
before the end, thus increasing the overall cost without
additional sales?
Business Case #2
Q8.
In a normal situation would a reduction in the cost of the unit
be more effective in additional sales,
for instance by $0.10, than the cost of running an expensive ad
campaign?
Q9. (True/False)
It has been shown, table 1 that a reduced number of sales due to
additional price per unit makes no
difference to overall profit, in fact it reduces in comparison to
larger sales by the lower priced item.
Therefore the sensible approach would be to create a
recommended retail price across all cities.
Q10.

20. There is to be a campaign which promotes a new design on the
cereal packaging which projects
an increase in sales, however would it not be more effective if
this was coupled with a slightly
improved product?
Veterans in the Criminal Justice System
Syllabus - Spring 2017
Required Reading:
- Winners and Losers: Battles, Retreats, Gains, Losses and
Ruins from the Vietnam War,
Gloria Emerson, WW. Norton and CO. (Reissue Edition, 2014)
- Lethal Warriors: When the New Band of Brothers Came Home,
David Philipps,
Palgrave/Macmillan)
- What Have We Done: The Moral Injury of Our Longest Wars,
David Wood, Little Brown
and CO. There will be several articles posted on Moodle
throughout this course. Following

21. the posting of these articles, they will become required reading.
THE COURSE: The subject matter of this course – veterans
entangled in the criminal justice
system – has been, for the most part, absent in the academic
arena. In fact, the subject matter of
this course has only rarely been openly addressed by the
criminal justice system. To place this
topic in perspective, relative to criminal justice, in July 2007,
your instructor conducted a study of
the Marion County Jail. Veterans comprised just over 5% of the
jail population of the Marion
County Jail (Approximately 1,000 veterans booked into the
Marion County jail per year). In
December 2011, this instructor conducted another study of the
Marion County Jail, and veterans
accounted for 10% of the jail population (Over 2,000 veterans
booked into that facility per year).
That reflects a 100% increase in 3 ½ years. In 2007, this
instructor conducted a homeless study in
Marion and Polk counties. Of the 360-homeless people
interviewed, over 30% were veterans, and
about 30% of those veterans were products of the Iraq and
Afghanistan wars. Among the Iraq and

22. Afghanistan veterans in that study, 6 were females. The females
were married and most had
children. They were homeless because they could not adjust to
the transformation of
soldier/Marine to mother and spouse. The number of veterans,
and in some cases active duty
personnel, becoming entangled in the criminal justice system
has increased significantly over the
past few years. However, most criminal justice agencies do not
keep track of the number of
veterans processed through the system. Many veterans are
apprehensive, in part because of shame
as well as concern about losing benefits, about revealing their
veteran status when arrested or
detained. Altercations in the public arena between law
enforcement and veterans – particularly
combat veterans – are increasing. Most law enforcement
agencies fail to even provide training to
officers in respect to veteran confrontation. I have been told by
many law enforcement
administrators that they do not want any training provided.
Most defense attorneys, prosecutors,
and judges are not veterans, and therefore struggle with their

23. own capacity to understand the social
realities of veterans in the community and in the criminal
justice system. Although Post 2
Traumatic Stress Disorder is a topic reserved for the discipline
of psychology, it is imperative that
students have some understanding of that mental issue relative
to many veterans. Due to its
relevance to this topic, war-related PTSD will be addressed in
this class as a source of information
– not as a guide to diagnosing PTSD. To adequately address the
topic of veterans entangled in the
criminal justice system it is imperative to understand “what” a
veteran is – particularly if that
veteran has been exposed to combat. Therefore, much of your
required reading is focused on that
aspect of the subject. Another area of concentration will be the
processes associated with
transforming the “civilian” into the military, continued
development of those processes during the
period the individual is serving in the military, and the
transition experienced by veterans back
into the civilian culture. These topics are sociologically related
to the term: The Military Total
Institution. The Military Total Institution is the formal structure

24. and maintenance of military
culture. Drawing from the assumption that people are products
of their social environments, the
military culture significantly influences the reintegration
process for veterans. Those veterans who
have trouble reintegrating back into the civilian culture are
more likely to become entangled in the
criminal justice system – not because they are more inclined to
engage in criminal behavior but
because they “don’t fit.” Because this is a criminal justice
course focusing on veterans, it is
imperative that students are presented with recent research data
pertaining to veterans who have
and have not been processed through the criminal justice
system. One purpose of using this
approach – veterans not in the criminal justice system versus
veterans entangled in the criminal
justice system – is to compare characteristics, experiences,
types of behavior, etc. and conduct
analysis and develop possible preventive measures and
alternative approaches for the criminal
justice system when addressing veteran issues. The research
data provided for students in this class

25. are derived from a recent study conducted by the instructor that
focused on Iraq and Afghanistan
veterans located in 16 U.S. states from 2008-2010. These data,
in part, are included in peer-
reviewed publications, appellate processes, professional legal
training for attorneys and in court
testimony in criminal cases involving Iraq and Afghanistan
veterans. Additionally, data collected
from veteran criminal defendants will also be included
throughout this course. Topics related to
Moral Injury and Traumatic Brain Injury will also be covered in
this course. There will be many
films presented throughout this course. These films are not for
entertainment purposes. Instead,
they will be presented for educational purposes and will often
be the focal point for class
discussions afterwards. The content of these films, along with
your assigned readings, will be
expected to be included in your required paper. Periodically,
there may be guest speakers brought
into the classroom. Following the presentation by any guest
speakers, students will be REQUIRED
to ask questions and participate in discussions. Ultimately, this
course will be a collaboration of

26. history, philosophy, psychology, anthropology, sociology,
political science, and criminal justice,
with the primary purpose to make students culturally competent
in the topic of veterans entangled
in criminal justice. Periodically, there will be articles,
PowerPoints, etc. posted on Moodle. These
postings will assist student in their preparation for completing
the course paper.
ALL students will be expected to participate in classroom
discussions and other activities. This
includes questions, answers/responses. All students must be
prepared to participate in open
discussions. Students are expected to appreciate and respect the
right of all students to express
opinions. Dialogues and arguments must be civil, constructive,
and based on information validated
through social science research. Attendance will be taken at the
end of each class period – each

27. class period is valued at 10 points. 2 Each student is required to
write a Veteran Re-Acculturation
Paper. This paper must highlight the overall content of the
course. This paper must also include
preventive options that would reduce the likelihood of veterans
becoming entangled in criminal
justice and a plan that would improve the criminal justice
system’s handling of veterans. The plan
must be based on class material and discussions. Students may
use other research that can support
his or her plan. The paper must include a definition of
“veteran,” the unusual nature and
experiences of many veterans, and an explanation of why or
why not veterans should be considered
different at various stages of the criminal justice system
process. Make certain that you cite the
assigned reading material germane to this course, and feel free
to use other social science
references. YOUR PLAN MUST BE SUBMITTED BY JUNE 9,
2017. IT MUST BE IN WORD
FORMAT AND SENT AS A SINGLE ATTACHMENT TO
PROFESSOR. YOUR RE-
ACCULTURATION MUST BE BETWEEN 8-10 PAGES

28. (DOUBLE-SPECED). 4 Below are
lists of reading assignments. Apr 3 – Apr 17: Winners and
Losers: Battles, Retreats, Gains, Losses
and Ruins from the Vietnam War Apr 18- May 15: Lethal
Warriors May 16 – Jun 5: What Have
We Done: The Moral Injury of Our Longest Wars
a
Highlight