This document summarizes a study on the effects of ad quality and content relevance on perceptions of content stream quality. The study found that while high quality ads on their own can be engaging, putting multiple high quality ads together in a content stream, including ones directly related to the content, can negatively impact perceptions of the overall stream quality. Specifically, conditions with no ads or unrelated high quality ads were rated significantly higher than conditions with related or annoying ads. The implications are that ad-supported platforms need to carefully evaluate ad quality both individually and within the context of surrounding content.

1. Effects of
ad quality & content-relevance
on perceived content stream quality
Henriette Cramer
Yahoo Personalization & Platforms
Sunnyvale, CA, USA

2. An illustration:
Ads that individually have been rated as high quality
can have negative effects in context.

4. bad ad
1. ad that is annoying to the user
(Goldstein & McAfee, 2013)
(+ decreases product engagement)

5. ad ad
ad
ad
Display advertising: banners, pop-ups

6. content content
content
ad
contentad
contentcontent
content
content
Native advertising
search ads, in-stream ads, sponsored posts
ad
content
content
content
content

7. How do native ad quality & context-relevance
affect content perceptions?

8. Research shows …
Noticeable ads get more engagement (Goldfarb and Tucker, 2011)
Overly noticeable, annoying ads cost both user & site (Goldstein & Mcafee, 2013)
Relevance increases ad effectiveness (Goldfarb & Tucker 2013, Buscher et al., 2010, Yan et al. ’09)
Personal relevance can lead to more pleasant UX (De Sa et al, 2013)
Too personally relevant is too close for comfort (Agarwal et al., 2013)

9. Two-step study
Mechanical Turk (US)
1.
ads rated as ‘good’ or ‘bad’
2.
Take good/bad un/related ads
Rate stream quality.

10. Step 1:
45 native ads x 10 ratings per ad = 450 ratings
98 participants
Likert-type scales (1-5) + Why? for 1-3.
1. … how annoying would you say that this ad is?
2. … how would you rate the design of this ad?
3. … rate the trustworthiness of this ad?
4. how familiar are you with the brand advertised?

11. Step 2
Ads in context
4 conditions,
Good Ads: 2 ads rated as not annoying & not
untrustworthy
• 1. 2 good ads non-related to content
• 2. 2 good ads, 1 content-related: a credit card
ad near credit-related headline.
• 3. ’Bad Ads: 2 bad ads rated as annoying &
untrustworthy
• 4. No Ads: only articles
237 participants
Likert-type scales (1-7) + Why?
1. How credible do you think this news site is?
2. How would you rate the quality of this collection
of news links?

12. Limitations
1 ad format
1 type of content-relatedness
1 specific credit-related ad
not looking at millions of actual users
… go forth and do a better study!
CC Steve Hersh
flic.kr/p/7YqHge
should've, could've, would've

13. People distinguish between
credibility of stories vs. overall site quality
Quality
•No ads > both good and bad ads (MW, p<.01)
•Good non-related > good content related (MW, p <.01
• High quality ads in context can have negative effects
• Interesting: Having no ads is better than good, but too
content-related ads, but not sig different from good,
unrelated ads
Step 2
Quality Results
sig dig for credibility (H=9.083, df=3, p=.028) &
quality (H=14.997, df=3, p=.002)
Mann-Whitney 1-tailed comparisons between
conditions, with Holm-Bonferroni adjustment: target
p value/significance n-rank of pair +1.

14. Our (too) content related ad.

15. Branding goes both ways
“I think it's credible because [companyX] is one of the
advertisers and [companyX] is a very reputable
company…”-GCR

16. Ads were mentioned in ‘why’ people
perceived the stream as high or low quality
whether they were there or not.

17. Two positives can be a negative
Extends Goldfarb & Tucker, 2013:
…combining multiple effective ad strategies can decrease their
effectiveness when each alone prime user concerns.

18. Ads that individually have been
rated as high quality can have
negative effects in context.
Implications for ad-supported services:
• Clearly distinguish ads & content
• Evaluate ads for quality, then evaluate for
quality within-context
• Offer in-context feedback opportunities
Ads affect experiences, research them!
henriettecramer.com
@hsmcramer