Economic Impact of Mixed Content Warnings on Consumer Behavior

Economic Impact of
Mixed Content Warnings
on Consumer Behavior
Sponsored by Ghostery
Independently conducted by Ponemon Institute
April 2015

Ponemon Institute©: Private & Confidential Report Page 2
Economic Impact of Mixed Content Warnings on Consumer Behavior
Ponemon Institute, April 2015
What is a mixed content warning? When a user visits a page served over HTTPS, his or her connection
with the web server is encrypted with TLS and, hence, safeguarded from sniffers and man-in-the-middle
attacks. If the HTTPS page includes content retrieved through regular, clear text HTTP, then the connection
is only partially encrypted. The unencrypted content is accessible to sniffers and can be modified by man-in-
the-middle attackers. Therefore, the connection is no longer safeguarded. When a webpage exhibits this
behavior, it is called a mixed content page. Depending on the browser type, this results in a visual icon or
pop-up that attempts to warn the visitor.
Description of the project:
Ghostery engaged Ponemon Institute to independently determine the economic impact of mixed content
warnings. Specifically, we designed and fielded an experimental study that tests consumer reactions to
mixed content warnings when browsing secure e-commerce sites.
We utilized scientific sampling methods to recruit a representative sample of adult-aged consumers (a.k.a.
respondents) located in the United States. Table 1 summarizes our survey response. We achieved a final
sample of 1,577 qualified respondents or a 3.4 response rate. This experiment was conducted over a two-
week period ending in March 2015.
1
Table 1. Survey response Freq
Total sampling frame (US consumers) 46,559
Total returns 1,732
Rejected or screened surveys 155
Final sample 1,577
Response rate 3.4%
Key takeaways:
Most respondents (52 percent) have a basic understanding of what a mixed content warning means.
Respondents who view the standard warning on Internet Explorer have the highest continuance level or
lowest attrition rate.
Respondents who view the standard warning on Chrome have the lowest continuance level or highest
attrition rate.
Prior to participation in this research, most respondents (69 percent) can recall seeing mixed content
warnings either frequently or very frequently. Only 14 percent say they saw a mixed content warning for
the first time.
The main reason for leaving a website after viewing the mixed content warning is concern about the
pop-up message displayed on the checkout page.
Consumer attrition resulting from mixed content warnings on web pages is estimated to cost the top 100
Internet retailers in the United States $310 million per annum.
1
Respondents were compensated with a $5 dollar gift certificate or participation in a lottery.
10 East 39th St-8th Floor New York, NY 10016 | 917.262.2530 | ghosteryenterprise.com 1

Sample characteristics:
Following are four charts that show the basic characteristics of individuals who participated in this study. Pie
Charts 1 and 2 have a sample distribution of 1,577 respondents by gender and age, respectively.
Pie Chart 1: Gender Pie Chart 2: Age Range
Pie Charts 3 and 4 show the sample distribution by household income and education level, respectively.
Pie Chart 3: Household income Pie Chart 4: Education level
808
769
Female Male
80
292
411
320
196
137
141
Below 18 18 to 25 26 to 35 36 to 45
46 to 55 56 to 65 Above 65
190
277
590
328
61
58
40 33
Less than $25,000 $25,000 to $40,000
$40,001 to $60,000 $60,001 to $80,000
$80,001 to $100,000 $100,001 to $150,000
$150,001 to $250,000 More than $250,000
282
341
462
356
121 15
High School Vocational
College (no degree) College (degree)
Post Graduate Doctorate
10 East 39th St-8th Floor New York, NY 10016 | 917.262.2530 | ghosteryenterprise.com10 East 39th St-8th Floor New York, NY 10016 | 917.262.2530 | ghosteryenterprise.com 2

Experimental design
Utilizing a survey instrument, we asked respondents to make a decision about continuing or discontinuing an
online activity that displayed a mixed content warning. Respondents were randomly assigned to one of two
website activities, described as follows:
• Booking a car rental on a Hertz registration site (n1 = 781)
• Buying a pair of sneakers on a Sneakerhead checkout page (n2 = 796)
Task 1: Respondents were asked if they would continue an online activity such as booking a car or buying
sneakers after viewing a “clean” webpage – that is, an HTTPS webpage that does not contain a mixed
content warning. This reading served as our baseline.
Task 2: Respondents were asked if they would continue an online activity such as booking a car or buying
sneakers after viewing a “dirty” webpage – that is, an HTTPS webpage that contains one of three mixed
content warnings. Here we used the standard warning displayed in Chrome, Internet Explorer or Foxfire.
Dependent Variable: Our primary measure is each respondent’s attrition or churn decision after completing
Task 2 versus his or her baseline result in Task 2. This aggregated attrition rate is used to extrapolate the
total economic impact of mixed content warnings for online merchants (retailers). The following table
summarizes our research design
Table 2: Experimental design
Context Task1 Task 2 Difference
Hertz A C X1 = C – A
Sneakerhead B D X2 = D – B
Guiding hypotheses
X1 Likelihood of attrition > 0
X2 Likelihood of attrition > 0
Experimental findings
Figure 1 summarizes the respondents’ decision to continue or discontinue an online activity. Both Hertz and
Sneakerhead results show a very low attrition for the baseline task and a very high attrition rate after seeing
mixed content warnings. The aggregated attrition rate is 57 percent.
Figure 1. Would you continue to book a car or buy sneakers online?
Percentage Yes response
90% 88% 89%
31% 33% 32%
59%
55% 57%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Hertz Sneakerhead Combined
Task 1 (baseline) Task 2 (post-experiment) Attrition rate

Figure 2 summarizes the respondents’ decision to continue an online activity after seeing mixed content
warnings in Chrome, Foxfire or Internet Explorer. As shown, respondents who view the standard warning on
Internet Explorer have the highest continuance level or lowest attrition rate. In contrast, respondents who
view the standard warning on Chrome have the lowest continuance level or highest attrition rate.
Figure 2. Would you continue after seeing a mixed content warning?
Figure 3 summarizes the respondents’ self-reported level of understanding about mixed content warnings.
These results suggest a majority of respondents (52 percent) believe they have a basic understanding about
these messages.
Figure 3. Do you understand what the mixed content warning means?
25%
33%
41%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Chrome Foxfire Internet Explorer
52%
48%
0%
10%
20%
30%
40%
50%
60%
Yes No

Figure 4 shows 69 (29+40) percent of respondents say they recall seeing mixed content warnings either
frequently or very frequently prior to their participation in this research. Only 14 percent say they never saw
a mixed content warning for this experimental study.
Figure 4. Do your recall seeing a mixed content warning when browsing a website?
Figure 5 list the reasons why respondents decided to churn after viewing the mixed content warning. These
results show an overwhelming majority of respondents (94 percent) were motivated to stop because of the
standard warning.
Figure 5. Why did you decide to discontinue after viewing the mixed content warning?
More than one response permitted
29%
40%
17%
14%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Yes, frequently Yes, sometimes Yes, rarely No
1%
3%
6%
8%
94%
0% 20% 40% 60% 80% 100%
Other
I don’t like providing my personal information on
websites
I don’t like the form used to capture [reservation
or payment and billing] details
I don’t like [booking a car or buying sneakers]
online
I’m concerned about the pop-up message
displayed on the checkout page

Determining economic impact
Based on the above analysis, we extrapolated the economic impact of consumer attrition resulting from
mixed content warnings. We assume our guiding hypotheses X1 and X2 are validated and, hence, utilize
the calculated attrition rate of 57 percent. Following are key factors utilized in this analysis:
Targeted population: Top 100 Internet retailers headquartered in the United States
Rate of mixed content messages: Non-secure calls provided by Ghostery Ghostrank data. This rate is the
average compiled over three months for 18 top 100 U.S. online retailers.
Online revenue and unique visitors: Determined from the Top 500 Internet Retailer Database (FY 2014
data points).
Our analysis is contained in the following three tables. Table 3 lists 18 companies, all containing
frequencies of non-secure calls (e.g., mixed content). This information is derived from the Ghostery
Ghostrank data over a three-month period. All 18 companies are Top 100 U.S. Retailers.
The first step is the collection of conversion rates (e.g., percent of visitors who make a purchase) for the list
of 18 retailers. Conversion rates ranged from a low of 1.0 percent to a high of 8.9 percent.
The second step is the calculation of a percentage represented by those visitors who saw non-secure calls
and then churned or discontinued the web session. Hence, we multiply non-secure calls times 57 percent.
This calculation is the basis from which we later determine economic impact.
Table 3. Determining consumer attrition after mixed content warning
Retailers
Top 100
retailer rank
Percent of
visitors who
make a
purchase*
Percent of
visitors who
saw non-
secure calls**
Percent of
visitors who
saw non-
secure calls
and churned***
amazon.com 1 4.00% 1.96% 1.12%
apple.com 2 4.00% 1.86% 1.06%
bestbuy.com 15 1.30% 2.51% 1.43%
crateandbarrel.com 77 2.12% 13.30% 7.58%
etsy.com 30 2.30% 0.84% 0.48%
gap.com 19 3.50% 13.93% 7.94%
homedepot.com 16 1.30% 6.39% 3.64%
lowes.com 36 1.30% 3.53% 2.01%
macys.com 8 4.00% 1.16% 0.66%
netflix.com 7 NA 1.76% 1.00%
nordstrom.com 24 3.20% 4.30% 2.45%
overstock.com 31 2.50% 3.50% 1.99%
sears.com 5 1.00% 7.66% 4.36%
staples.com 3 8.90% 8.01% 4.57%
target.com 18 1.60% 14.58% 8.31%
toysrus.com 34 3.00% 16.97% 9.67%
walgreens.com 43 2.30% 28.03% 15.97%
walmart.com 4 3.31% 22.18% 12.64%
Average 20.72 2.92% 8.47% 4.83%
* Internet Retailer’s Top 500 Database
** Ghostrank data three-month average
***Derived from the Ponemon experiment
NA Conversion rate was not available for Netflix

Drawing from the Internet Retailer’s Top 500 database, we obtain annual web sales for 18 retailers. We
simply divide annual sales by 12 to determine monthly sales.
Our third step is to calculate monthly web sales under the condition of zero attrition or no mixed content.
Following is our gross-up formula:
Monthly web sales ÷ (1 – [attrition X conversion])
Zero attrition would happen if mixed content was eliminated and, hence, mixed content warnings are no
longer needed.
Our fourth and final step is to calculate the difference between monthly web sales and grossed-up web sales
for 18 companies. As shown in Table 4, the total monthly difference or “net gain” for 18 Internet retailers is
$14,176,781.
Table 4. Calculation of net gain for 18 Internet retailers
Retailers
FY 2014
Annual web
sales
($billions)*
Monthly web
sales
($millions)
Monthly web
sales
assuming zero
attrition
($millions) Net gain ($)
amazon.com 79.50 6,625 6,628 2,960,124
apple.com 20.60 1,717 1,717 729,590
bestbuy.com 3.54 295 295 54,925
crateandbarrel.com 0.51 42 42 68,019
etsy.com 1.93 161 161 17,632
gap.com 2.50 208 209 580,399
homedepot.com 3.76 314 314 148,438
lowes.com 1.27 105 105 27,597
macys.com 5.40 450 450 118,947
netflix.com 5.50 458 458 NA
nordstrom.com 2.50 208 208 163,367
overstock.com 1.50 125 125 62,229
sears.com 5.70 475 475 207,359
staples.com 11.23 936 940 3,820,383
target.com 2.99 249 249 331,851
toysrus.com 1.20 100 100 291,023
walgreens.com 1.13 94 94 345,724
walmart.com 12.14 1,011 1,016 4,249,173
Total 162.88 13,573.67 13,587.51 $14,176,781
* Internet Retailer’s Top 500 Database
** Ghostrank data three-month average
***Derived from the Ponemon experiment
NA Conversion rate was not available for Netflix
Table 5 contains the extrapolated economic impact of mixed content warnings on consumers; Internet
behaviors. As shown, the estimated annual value for the 18 top retailers is over $170 million. We then
gross up this value using a ratio based on total sales for 18 and the top 100. This produces a total annual
estimated net gain of $310 million.
Table 5. Key measures of economic impact
Monthly net gain for 18 retailers $14,176,781
Annual net gain for 18 retailers $170,121,377
Gross-up ratio* 55%
Extrapolated value for Top 100 $309,674,297
*Ratio = Total sales for 18 retailers ÷ total sales for top 100 retailers

Appendix: Experimental Results
The following tables provide the results of our experimental study on mixed content warnings.
Frequencies
Displayed = clean home page 781 796 1577
Displayed = clean checkout page (no popup) 781 796 1577
Q1. Would you continue [to book a car or buy
sneakers online] after seeing this page? Hertz Sneakerhead Combined
Yes 699 703 1402
No 82 93 175
Total 781 796 1577
Q2. If yes, please rate the likelihood that you
would continue checkout? Use the following 10-
point scale from 1 = not likely to 10 = very likely Hertz Sneakerhead Combined
1 or 2 54 68 122
3 or 4 168 156 324
5 or 6 176 185 361
7 or 8 165 175 340
9 or 10 136 119 255
Total 699 703 1402
Q3. If no, why? Hertz Sneakerhead Combined
online 44 56 100
I don’t like the form used to capture [reservation or
payment and billing] details 31 29 60
websites 40 39 79
Other (please specify) 2 5 7
Displayed = dirty page ( randomly assigned one of
three browser/message type) 781 796 1577
Q4. Would you continue to [book a car or buy
sneakers] online after seeing this page? Hertz Sneakerhead Combined
Yes 246 263 509
No 535 533 1068
Total 781 796 1577
Q4. Would you continue to [book a car or buy
sneakers] online after seeing this page? Hertz Sneakerhead Combined
Yes, Chrome 65 63 128
Yes, Firefox 80 90 170
Yes Internet Explorer 101 110 211
Total 246 263 509

Q5. If yes, please rate the likelihood that you
would continue [booking a reservation or buying
sneakers]? Use the following 10-point scale from 1
= not likely to 10 = very likely. Hertz Sneakerhead Combined
1 or 2 90 86 176
3 or 4 78 85 163
5 or 6 61 74 135
7 or 8 12 15 27
9 or 10 5 3 8
Total 246 263 509
Q6. If no, why? Hertz Sneakerhead Combined
online 45 39 84
I don’t like the form used to capture [reservation or
payment and billing] details 30 39 69
websites 17 18 35
I’m concerned about the pop-up message
displayed on the checkout page 506 499 1005
Other (please specify) 3 4 7
Q7. Do you understand what the pop-up message
actually means? Hertz Sneakerhead Combined
Yes 405 416 821
No 376 380 756
Total 781 796 1577
Q8. Do your recall seeing a mixed content warning
when browsing a website like the one viewed
before? Hertz Sneakerhead Combined
Yes, frequently 225 240 465
Yes, sometimes 309 314 623
Yes, rarely 129 138 267
No 118 104 222
Total 781 796 1577
Q9. How important is security of the websites you
browse or shop? Use the following 10-point scale
from 1 = not important to 10 = very important. Hertz Sneakerhead Combined
1 or 2 12 15 27
3 or 4 40 41 81
5 or 6 157 163 320
7 or 8 235 238 473
9 or 10 337 339 676
Total 781 796 1577
Q10. How important are the privacy commitments
of the websites you browse or shop? Use the
following 10-point scale from 1 = not important to
10 = very important. Hertz Sneakerhead Combined
1 or 2 30 35 65
3 or 4 65 63 128
5 or 6 221 219 440
7 or 8 240 240 480
9 or 10 225 239 464
Total 781 796 1577

Demographics
D1. Gender: Hertz Sneakerhead Combined
Female 403 405 808
Male 378 391 769
Total 781 796 1577
D2. Age range: Hertz Sneakerhead Combined
Below 18 38 42 80
18 to 25 142 150 292
26 to 35 201 210 411
36 to 45 167 153 320
46 to 55 96 100 196
56 to 65 67 70 137
Above 65 70 71 141
Total 781 796 1577
D3. Highest level of education: Hertz Sneakerhead Combined
High School 139 143 282
Vocational 168 173 341
College (attended, no degree) 231 231 462
College (4 year degree) 176 180 356
Post Graduate 59 62 121
Doctorate 8 7 15
Total 781 796 1577
D4. Household income: Hertz Sneakerhead Combined
Less than $25,000 93 97 190
$25,000 to $40,000 140 137 277
$40,001 to $60,000 287 303 590
$60,001 to $80,000 168 160 328
$80,001 to $100,000 29 32 61
$100,001 to $150,000 29 29 58
$150,001 to $250,000 19 21 40
More than $250,000 16 17 33
Total 781 796 1577
Please contact research@ponemon.org or call us at 800.877.3118 if you have any questions.
Ponemon Institute
Advancing Responsible Information Management
Ponemon Institute is dedicated to independent research and education that advances responsible
information and privacy management practices within business and government. Our mission is to conduct
high quality, empirical studies on critical issues affecting the management and security of sensitive
information about people and organizations.
As a member of the Council of American Survey Research Organizations (CASRO), we uphold strict
data confidentiality, privacy and ethical research standards. We do not collect any personally identifiable
information from individuals (or company identifiable information in our business research). Furthermore, we
have strict quality standards to ensure that subjects are not asked extraneous, irrelevant or improper
questions.

Economic Impact of Mixed Content Warnings on Consumer Behavior

Recommended

Recommended

More Related Content

Similar to Economic Impact of Mixed Content Warnings on Consumer Behavior

Similar to Economic Impact of Mixed Content Warnings on Consumer Behavior (20)

More from Ghostery, Inc.

More from Ghostery, Inc. (11)

Recently uploaded

Recently uploaded (20)

Economic Impact of Mixed Content Warnings on Consumer Behavior