Field Experiments, Obesity, Bing vs. Google

1. Field Experiments
Data Driven Decision Making (D3M)
Vishal Singh
Stern School of Business
New York University
D3M

2. Example 1
Bing it ON
D3M

3. Bing it ON
Context
Microsoft's "Bing It On" campaign purports to show that users prefer the company's
search engine to Google's in a majority of blind tests. Recently, Ian Ayres (faculty at
Yale Law) ran a blind test at BingItOn.com with 1,000 people recruited through
Amazon's Mechanical Turk. The paper concludes that Bing's claims are misleading
and are based on search words provided by the company. This in turn warrants legal
scrutiny under the Lanham Act on false advertising (you can find the unpublished
working paper on his web page).
Data
In the file “Bing_it_on.csv” you are provided the data used in this study (it may be
useful to visit the "Bing It On" web page to understand the experiment). There are
approximately 900 participants in the experiment that were randomly assigned to one
of the 3 groups based on what search words to use (variable: “Search Type”):
1: Popular searches (based on 2012 most popular google search words)
2: Bing suggested search words
3: User-generated search words
The key variable of interest is “Preference” coded as 1-Bing Wins, 2-Tie, and 3-Google
wins. Data also contains an additional variable “Gender” (1=Male, 2=Female) that you
can ignore.
Objective
Analyze the relationship between “Search Type” and “Preference”.

6. Will a Fat Tax Work?
Quasi-Experiment

7. Why Intervene?
• Dire consequences of obesity to Individual
– Increased risks of type 2 diabetes, hypertension,
cardiovascular diseases, cancer, gallbladder disease,
osteoarthritis, disabilities, psychosocial problems
...Estimated 112,000 deaths every year
Externalities: Significant economic
implications costing $150 billion p.a.
– Medical costs: half on Medicare and Medicaid
– Additional productivity loss
Vishal Singh, Stern School of Business, NYU 7

8. How to Intervene?
 Disclosure & Education
 Limiting choices (zoning and prohibition)
 Marketing regulation (Limiting messages)
 Surveillance (data provision)
 Taxation
– “Fat Taxes” or “Junk Food Tax”
– Already in place in many states
– Soda tax (mean rate 5.2%) in 33 US states
– A sugar based tax has been proposed
o Bans/Regulations
Vishal Singh, Stern School of Business, NYU 8

9. Problems with “Twinkie” Tax
o Ideological
Highly Regressive
Will it Work?
o Will it get Implemented?
o Strong Industry Opposition

10. Previous Evidence
o Field Work
Econometric/data problems
Focus on Sales Tax
Industry Funded
Experimental Work 
Lab/Cafeteria/Vending Machines
Small non-representative samples

11. This Paper: Quasi Natural Experiment
$2.91 $2.91 $2.91 $2.90
$2.87
$2.73
$2.71
$2.60
$2.40
$2.45
$2.50
$2.55
$2.60
$2.65
$2.70
$2.75
$2.80
$2.85
$2.90
$2.95
Whole milk 2% milk 1% milk Skim milk
Uniform Price Non-Uniform Price
Depending on where you live and what supermarket chain you patronize, you see one of these patterns.
Milk Pricing in the US

12. Milk Pricing in the US
Vishal Singh, Stern School of Business, NYU 12
Non Flat Pricing
Primarily Non-Flat
Mixed
Primarily Flat
Flat Pricing
No Data Available
Southeast FMMO
Pennsylvania: Large milk
producer. State
regulations.
Uniform/Non-Uniform price
structure is consistent across
stores within a chain, even in
mixed states.
Upper Midwest FMMO: Wisconsin is
2nd largest producer
Central FMMO
Northeast FMMO
MidEast
FMMO
DATA
 1800 + supermarkets
 6 Years weekly data
 UPC level sales,
price, promotion etc.
 Counties represent
approximately 50% of
the population

14. a) Comparison of Demographic Profile between Flat and NonFlat Stores
Flat stores Non-Flat stores
Mean
Std
Dev Mean
Std
Dev p-value
Low income 18% 38% 21% 41% 0.08
High income 19% 39% 20% 40% 0.60
% Poverty 2% 1% 2% 1% 0.22
% Children 4% 1% 4% 1% 0.62
% College 39% 49% 41% 49% 0.58
% White 78% 19% 77% 19% 0.49
% Elderly 12% 4% 12% 5% 0.32
Population density 0.12 0.31 0.13 0.18 0.52
(b) (1) Regression of (Price Whole/ Price 2%) milk and (2) Variance Decomposition
(1) (2)
Estimate Std Error
% of explained variation
accounted for by:
Intercept 1.0393 (0.006)
Median Income -0.0017 (0.002) 0.06%
% HH Kids -0.0003 (0.001) 0.00%
% College -0.0005 (0.002) 0.01%
% White -0.0014 (0.001) 0.09%
Population Density -0.0003 (0.001) 0.00%
Wage 0.0028 (0.002) 0.14%
All retailers within 5 miles -0.0002 (0.001) 0.00%
Discount retailers within 10 miles -0.0021 (0.001) 0.18%
Marketing Order Fixed Effects Included 15.44%
Chain Fixed Effects Included 84.07%
R square 0.658
Is Pricing Structure Exogenous?

16. Does it Change Behavior?

17. No Unobserved Differences in Taste

18. Correlation of Shares & SEC

19. Large Response to Small Price Changes

20. Recommendations
• Small price gaps that are reflected at the
point of purchase
– Mitigates the regressive nature of taxes
$1.05
$.95
$2.05
$1.95
NOTE: Approximately half of the total grocery sales are on promotion

22. Altering the Food Subsidies