Media Campaign Performance Update

Media Campaigns
March 31, 2019 Performance Update
All of Us, the All of Us logo, and “The Future of Health Begins with You”
are service marks of the U.S. Department of Health and Human Services.

As we approach the first anniversary of the paid
media campaign, the question that must be asked is
whether we are realizing an ROI.

To answer this primary and fundamental question,
we must first start with understanding our campaign
objective, goals and guiding strategic direction.

Campaign Objective & Goals
What It Is Not
An independent marketing channel
to convert unassisted enrollment
independently and in isolation either
on the first website visit or returning
visit.
Our Media Objective
The objective is to fill the
top of the funnel for website
traffic across population
segments via creating
program awareness and
interest.
Awareness + Interest
Website Visits
Our Media Goals
1. The primary goal is to
create quality program
clicks to the national
website as an initial
touchpoint to assist in the
enrollment experience
journey.
2. The secondary goal is to
reinforce initial awareness
and return site visits via
added impressions and
frequency throughout the
personal consideration and
enrollment process.
1
information collection
3
enrollment
2
consideration

Strategically, we can not expect paid
media as a marketing channel to convert
on the first click even though it does
occur frequently.

Our conversion path is truly a multi-channel, multi-
touchpoint user experience journey requiring multiple
impressions across all marketing channels.

Paid Media Can (And Does) Have A Delayed Impact
Lag Time (in Days) for
Conversion After First Site Visit
Percentage of Paid Media
Assisted Conversions
1 7.4%
2 6.4%
3 6.4%
4 3.1%
5 2.3%
6 2.3%
7 2.0%
8 3.1%
9 3.1%
10 3.1%
11 2.0%
12-30 59.6%
With each program impression after initial awareness and click
from our paid media efforts, users are collecting and consuming
program information upon each returning site visit.
There is a distinct trending pattern that the majority of
users are converting farther out from the first paid media
impression and click.
This reveals that personal consideration for participation is
being taken seriously before a person commits to create an
account and go deeper into the enrollment path.
As the user experience journey unfolds across a multi-channel,
multi-touchpoint model, each new impression and each new
piece of content consumed helps reinforce consideration for
participating in the program.

What Does The Multi-Channel, Multi-Touchpoint Journey Path Look Like?

As a single touchpoint in a multi-touchpoint user
journey, paid media has a specific role and purpose.
Aligning to our campaign objective, that role is to
create program awareness and reinforce interest
throughout the consideration process.

Where Does Paid Media Fit In The User Experience Journey Path?
No touchpoint is a silo. Each channel and touchpoint (in combinations) are dependent on each other to deliver a holistic user journey experience.
Content Sharing
Organic Search
Website Content
Partner
Touchpoints
News/Influencer
3rd Party Sites
Call Center
Partner
Engagement
Website Consent
& Full Enrollment Path
Community Forum
IM/Chat
Biosamples
FAQ
Knowledge Base
Call Center
Testimonials
Social Networks
Newsletter
Email Retargeting
CTAs
Landing Page
Email
Social Engagements
News Feed Content
Return Visits
Email Subscribe
Social Media Follow
PR
Special Events
& Journey Bus Tour
Word of Mouth
Social Media &
Groups/Forums
Partner
Engagement
D
i
g
i
t
a
l
T
o
u
c
h
p
o
i
n
t
s
O
f
f
l
i
n
e
T
o
u
c
h
p
o
i
n
t
s
Display +
Social Ads
Other Ad Vehicles
Radio, TV, Print,
Outdoor Ads
Impression Reach & Frequency

It should also be noted that paid media and its value
is more than a single marketing channel.
Paid media supports a holistic integrated marketing
effort which reinforces the experience and message
delivery for greater program influence and impact.

Where Does Paid Media Fit In An Integrated Digital Marketing Framework?
Online Brand
Conversation
• PARTICIPATION
• MENTIONS
• SHARES
• REPOSTS
• UGC
• ADVOCACY
EARNED
MEDIA
• WEB PROPERTIES
• LANDING PAGES
• PORTALS/MICROSITES
• MOBILE APP
• EMAIL
• SOCIAL MEDIA
Brand Experience
OWNED
MEDIA
Brand Awareness
• BANNER ADS
• PAY-PER-CLICK
• DISPLAY ADS
• NATIVE ADS
• RETARGETING
• PAID INFLUENCERS
• SYNDICATED CONTENT
PROMOTION
• RICH MEDIA ADS
• SOCIAL MEDIA ADS
PAID
MEDIA
SEO & brand content drive earned
media (sharing) & traffic
More exposure to web
properties can be gained
with SEO and PPC
Sharing & engagement
is enhanced with paid promotion
awareness >> consideration >> trust >> decision >> conversion >> validation >> consent >> enrollment >> retention >> advocacy
When leveraged together
owned, earned and paid media
create a comprehensive and
integrated digital strategy.

Paid media is also not a channel which is activated
and then forgotten. Campaigns require continuous
tracking, measurement, analysis and optimization.
Optimizing against the right KPIs is paramount to our
success to maximize ROI and benefit.

Campaign KPIs & Optimization
Media
Performance
Core
KPIs
2
consideration
CVR
(conversion rate)
Quality of Funnel Lead
2
3
enrollment
CPA
(cost per acquisition)
Attributed Assisted
Conversion efficiency
3
Media Optimization
The underlying optimization
goal to our campaign is the
quality of the click linked to
program conversions.
It’s not about quantity of
clicks, but rather quality.
Although clicks are truly
important and fill the top of
the funnel, CTR, CVR and
CPA in combination delivers
the KPI mix to optimize
campaign ROI and benefit.
1
information collection
CTR
(click-through rate)
Top of Funnel
Impact
1
Website Visit Clicks
Optimization is continuously applied and aligned with
CTR, CVR and CPA to reap the greatest ROI.

15
Why Do We Track, Measure & Analyze KPIs?
Our national campaign is optimized for conversion as opposed to clicks for maximum ROI. This is a strategic and purposeful direction.
Optimization Performance Metrics
The national paid media campaign performance is
tracked, measured and strategically optimized
against three core performance KPI metrics.
 CTR (click-through rate): measures
campaign efficiencies as a baseline for
creating awareness, interest and site traffic.
 CVR (conversion rate): measures quality of
the click for program signup from first to last
website visit engagement (via delayed
response tracking).
 CPA (cost per acquisition): measures
overall efficiency of media campaign budget
spend for conversion signups compared
against all active national and partner
marketing channels and programs.
Guiding Baseline Metrics
Beyond the Core KPIs, additional campaign performance metrics are used as
cross tabulation validation points to ensure maximum campaign efficiencies.
 Impressions: measures reach and frequency of media buy against
targeted audiences for creating mass awareness.
 Clicks: measures top of funnel impact via measurable and attributed
referral site traffic.
 CPC (cost per click): measures channel placement budget spend
efficiencies and impact on overall budget spend.
 Conversions (accounts): measures ASSISTED impact on program
signups (count) cross-tabulated against other marketing channels as well
as CPAs.
 Enrollments (consents): measures the quality of the initial conversion
and the total impact (count) on the next step for full enrollment.
 Quality Ratio: measures the ‘quality ratio’ of the account conversion
measured against journey path advancement (account to consent
completion process) and as cross-tabulated against other marketing
channels.

Let’s take a look at the Campaign landscape.
Three campaigns are currently active.
The National campaign, the HPO Co-Branded
campaign and the Houston A/B regional test
campaign.

KEY CAMPAIGN EVENTS
Three concurrent and active media campaign efforts are underway: 1) National Media, 2) Local
Co-Branded Media, and 3) Houston Regional A/B Test Market Media.
Launch Optimization
Pixels
Removed
Phase 1
A/B Testing
Optimization
Phase 2
A/B Testing
May
2018
Oct
2018
Aug
2018
Jan
2019
Nov
2018
Mar
2019
National & HPO Co-branded
Media Campaigns start on
May 7, 2018 in support of
national and local HPO
cobranded efforts.
Algorithmic learning and
optimization continually
applied lowering CPA to $104
with goal to reach sub $50.
Houston A/B test market
campaign begins running
concurrently with National
Display campaign without
tracking pixels.
Phase 1 A/B testing completed.
Headline winners identified. Phase 2
begins to test headline + image
creative.
NIH Security Team deems
tracking pixels security risk
Removes pixels from site.
Unable to optimize to CPA.
Campaigns optimized to
CTR. Predictive modelling
represents low CPA
performance but can’t be
verified without tracking
pixels.
$512
CPA
$340
CPA
$104
CPA
0.11%
CTR
$73
Est. CPA
0.12%
CTR
$71
Est. CPA
$59
Est. CPA
* CPA references are real costs based on Display Banners ads. Estimated CPA is based on median average for predictive model using an average conversion rate calculation from national campaign when pixel tracking was in place.
0.12%
CTR
Facebook campaign stopped as
pixels are removed from site.
Facebook campaign
restarted

Co-Branded Temporarily
Suspended
AARP & Telemundo
Channels Added
Houston A/B Test Phase 2
Houston A/B Test Phase 1
18
Campaign Milestone Timeline
Campaign Launch (Phase 1) National & Co-Branded
Channels
Removed
Twitter,
Digital
Audio,
Native,
Email
Newsletters
May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr
Facebook & Instagram
Suspended
Tracking Pixels Removed
Facebook & Instagram Reactivated
Urban One Channel Added
Campaign (Phase 2) National & Co-Branded
Programmatic Display, Facebook, Instagram Core Channel Placements
1,085
Account Conversions
(Jul. 31)
3,053
Account Conversions
(Oct.11)
29M Impressions 128M Impressions 211M Impressions 443M Impressions
0.74% CVR 2.19% CVR 0.02% CTR
0.16% CTR 0.09% CTR 0.11% CTR
3,053 Account Conversions Created @ 74% Quality Ratio

National Media Campaign
Lifetime Period: May 7, 2018 – Mar. 31, 2019
Display
Twitter
Facebook
Audio & Native
Instagram
Email Newsletters

Based upon our core performance metrics, how is
the National Paid Media campaign performing?

21
Milestone: Impressions Delivered Impact on Awareness
(Lifetime Period: May 7, 2018 – Mar. 31, 2019)
The national paid media campaign is creating AWARENESS at scale across all 50 states.
MILESTONE
Number of Lifetime Impressions Delivered to Date

22
The national paid media campaign is creating REACH at scale across all 50 states.
MILESTONE
Count of Approximate Monthly Unique People Reached
(5:1 lifetime impression frequency ratio)
Percentage of U.S Internet Users Reached with At Least One Impression
(312 million Internet Users in the United States*)
* Statista, December 2017

23
Milestone: Clicks Created Impact on Website Traffic
The nationalpaid media campaign is creating INTEREST and WEBSITEVISITSat scale across all 50 states.
MILESTONE
Number of Lifetime Clicks (Website Visits) Delivered to Date

MILESTONE
24
Milestone: Awareness & Interest Impact on Website Traffic
954,265 Total Website Visits . 406,623 Paid Media Website Visits
The national paid media campaign is creating a continuous stream of NEW WEBSITE VISITS at scale.
Attribution percentage for the national paid media campaign click impact on website traffic.
(Statistical probability: +5% variance)
Our objective for leveraging paid media to create program awareness and interest is being realized. We are consistently
filling the top of the funnel with monthly new and returning website traffic which is a critical step for advancing the program.

What has been the Net impact?
Based upon our most recent time period when
tracking pixels were in place, all KPIs were performing
extremely well getting better with
each passing month.

26
Milestone: National Campaign KPI Metric Performance
(Optimized Last Pixel Tracking Period: Sep. 12 – Oct. 11, 2018)
MILESTONE
Lifetime Campaign Performance Comparison (May 7 – Oct 11, 2018)
0.10% $272.23 1.69%
Display Only Performance Averages Comparison (Sep. 12 – Oct. 11, 2018)
0.02% $116.05 14.03%

But our national story isn’t done...

During our pixel activation period, trending patterns
for key campaign performance metrics had indeed
been moving in all the right directions.
Limitations now exist for measuring and optimizing
performance since pixel removal. However, predictive
models suggest trending patterns are continuing.

As a result, our story gets even better.
When applying our model using two scenarios,
we can predict a performance span with confidence
for how the current campaign is performing.

30
Predictive Modeling: National Conservative Scenario
(Forecasted Period: Oct. 12, 2018 – Mar. 31, 2019)
Conservative
MODELING
Past Actual 30-Day Tracking KPI Comparison Performance (Sep. 12, 2018 – Oct. 11, 2018)
0.05% $144.49 5.28%

31
Predictive Modeling: National Best Case Scenario
(Forecasted Period: Oct. 12, 2018 – Mar. 31, 2019)
Best Case
MODELING
Past Actual 30-Day Tracking KPI Comparison Performance (Sep. 12, 2018 – Oct. 11, 2018)
0.05% $144.49 5.28%

So with a high degree of confidence…
The National Media Campaign
could realistically have a CPA somewhere in the range
of $72 to $35.

Co-Branded Campaign
Lifetime Period: May 7, 2018 – Mar 31, 2019
Display Facebook Instagram

34
(Lifetime Period: May 7, 2018 – Mar 31, 2019)
The co-brandedpaid media campaign is creating AWARENESS at scale in support of all HPO regions.
MILESTONE

35
The co-branded paid media campaign is creating REACH at scale in support of all HPO regions.
MILESTONE
Percentage of U.S Internet Users Reached with At Least One Impression
(312 million Internet Users in the United States*)
* Statista, December 2017

36
MILESTONE
Theco-brandedpaidmediacampaigniscreatingINTERESTandWEBSITEVISITSatscale insupportofallHPOregions.

MILESTONE
37
Milestone: Awareness & Interest Impact on Website Traffic
The co-brandedpaid media campaign is creating a continuousstream ofNEW WEBSITEVISITSat scale.
Attribution percentage for the co-branded paid media campaign click impact on website traffic.
(Statistical probability: +5% variance)
Our objective for leveraging paid media to create program awareness and interest is being realized. We are consistently
filling the top of the funnel with monthly new and returning website traffic which is a critical step for advancing the program.

What has been the Net impact?
The Co-Branded Local Campaign based on attribution
under performs compared to the National campaign.

This is to be expected.
The HPO Journey path is unique. Most HPO
conversions are assisted and do not follow the direct
digital path.

They create initial awareness and interest to
reinforce the user experience journey with their own
HPO provider.
However, campaign impressions and clicks
do serve a valuable purpose and role.

Fewer conversions and lesser performing
KPIs when compared to the national campaign is
absolutely to be expected.
Therefore, we can not dismiss the
co-branded efforts even if conversions can not be
directly attributed to the media campaign.

42
Milestone: Co-Branded Campaign KPI Metric Performance
(Optimized Last Pixel Tracking Period: Sep. 12 – Oct. 11, 2018)
729 Account Conversions Created – 76% Quality Ratio
MILESTONE
Lifetime Campaign Performance Comparison (May 7 – Oct 11, 2018)
0.14% $306.78 0.89%
Display Only Performance Averages Comparison (Sep. 12 – Oct. 11, 2018)
0.05% $320.25 1.66%

Key Insights & Learnings
44
Overall, both the National and Co-Branded campaigns have shown an ROI realizing original objectives and goals. Since
the tracking pixel was removed, the campaign remains at risk for not being able to demonstrate continuing and improving
value. Key takeaways are as follows:
----------------
1. As a channel for creating site traffic, paid media surpasses all other channels. It’s uncontestable.
2. There is a documented lag time (delayed response) for paid media conversion attribution. We can not under value this media
characteristic and it should be reinforced in all campaign analyses.
3. The CPA during the tracking period continually was improving and as a cost efficiency metric, it demonstrates budget economies were
being realized. When considering CPAs across all other national and partner touchpoints, paid media is likely the most efficient budget
spend in the mix. Further analysis is required for verification and validation.
4. We, as a group, must not buy into any false narrative that paid media has a high bounce rate and is inefficient. To the contrary, paid
media is actually a very efficient budget spend to garner program awareness and interest.
5. Benchmarking our efforts against published benchmarks is difficult. Google is a main publisher of such benchmarks and reports
represent the GDN network which is based on search. We must understand that this is an unfair comparison. Furthermore, industries do
not align to our special niche. The best scenario is we continue to research comparable case studies and publish findings as available to
benchmark campaign effectiveness.
----------------

Houston Media Campaign
Test Period: Nov. 20, 2018 – Mar 31, 2019
Headline A/B Test
Phase 1
Display & Facebook
Nov. 20 – Mar. 5
Image & Headline A/B Test
Phase 2
Display & Facebook
Mar. 6 – Mar. 31
Regional Market A/B Testing

The Houston Regional Campaign is designed to test
headlines and creative images within a defined and
controlled metropolitan statistical area.
The performance insights are intended to help steer
future strategic execution as a replicable model.

48
Houston A/B Testing Campaign Milestone Timeline
Campaign Launch (Phase 1) Headline A/B Testing
Urban-One Channel Added
Nov Dec Jan Feb Mar Apr
IRB Reporting
AARP Channel Added
Local Channels Added
Campaign (Phase 2) Headline + Image A/B Testing
Programmatic Display, Facebook, Instagram Channel Placements
7M Impressions
8K Clicks 0.11% CTR
48M Impressions
51K Clicks 0.11% CTR
92M Impressions
100K Clicks 0.11% CTR
Telemundo Channel Added
Click Optimization Activated
IRB Reporting IRB Reporting

49
(Campaign Test Period: Nov. 20, 2018 – Mar. 31, 2019)
The Houston paid media campaign is creating awareness at scale across the Houston area.
MILESTONE

50
The Houston campaign is creating REACH at scale in support of the A/B testing.
MILESTONE
Percentage of Houston Population Reached with At Least One Impression
(Houston MSA population: 6.4 million)

51
The Houston paid media campaign is creating interest at scale across the Houston area.
MILESTONE

52
The Houston paid media campaign is creating an augmented stream of new website visits at scale.
MILESTONE
Milestone: Awareness & Interest Impact on Total Website Traffic
Attribution percentage (+5% variance) for the Houstonpaid media campaignimpact on all websitetraffic.
Our test pilot objective for leveraging regional paid media to create program awareness and interest is being realized.
We are augmenting filling the top of the funnel by creating a steady stream of monthly new and return website traffic.

Overall, what has been the Net impact
on campaign performance?
Based upon our adjusted KPI metrics
without tracking pixels, our primary KPI becomes CTR
which is performing very well for creating program
awareness.

54
Milestone: Lifetime Campaign KPI Metric Performance
MILESTONE

What are the results of the A/B testing?
Two testing periods have been created.
Phase 1 for testing headlines only is complete.
Phase 2 is in progress designed to further test
headlines with added image testing.

Test Phase 1
Nov. 20 – Mar. 5, 2019
Creative Theme & Headline Testing Results

Without tracking pixels in place, optimization and
performance measurement has been primarily based
on CTR and CPC.
We have used these metrics as the leading
performance indicators for measuring results.

58
Houston Phase 1 Test Methodology
HeadlinesweretestedbasedonCreativeThemeandtargetedtospecificaudiencegroupsacrosstwochannelplacements(DisplayandFacebook).

59
Creative Theme Phase 1 Test Final Results
Creative Theme Winners: 1) Health - 2) Be Heard (Asian-American) - 3) Be Heard (African-American).
The Health creative theme by far resonated the most with target audiences and had the most influence and impact.

Testing Review Based on a Single Variable
CTR =
Clicks
Impressions
Headline Testing

Headline 1 Headline 2
H1 - It starts with YOU. It ends
with All of Us.
H2 - It starts with YOU. It gets better
with All of Us.
Test Group 1: Creative Theme (Altruistic) – Contextual Targeting
Differences do exist among channel placement.
Which headline within the Test Group performed best for Phase 1 A/B testing?
Review Period: Campaign to Date (Nov. 20 – Mar. 5, 2019) n = 11,973,559
Display Channel
No clear winner with a slight edge
for CPC for Headline 2.
Facebook Channel
Strong preference for Headline 2.

H1 – Introducing the next
breakthrough in medicine. You.
H2 – Every body tells a story…
what’s yours?
Test Group 2: Creative Theme (Be Heard) – Aggregate Targeting
Display Channel
No clear winner.
Facebook Channel

what’s yours?
Test Group 2: Creative Theme (Be Heard) – Elderly 65+ Targeting
Display Channel
Facebook Channel

what’s yours?
Test Group 2: Creative Theme (Be Heard) – African-American Targeting
Display Channel
Based on CTR, marginal winner
with a slight edge for Headline 1.
Facebook Channel
Based on CTR, marginal winner with a
slight edge for Headline 2.

what’s yours?
Test Group 2: Creative Theme (Be Heard) – Asian-American Targeting
Display Channel
Facebook Channel
Marginal winner with a slight edge for
Headline 2.

what’s yours?
Test Group 2: Creative Theme (Be Heard) – Hispanic-Latino Targeting
Display Channel
Facebook Channel
Marginal winner with a slight edge for
Headline 1.

what’s yours?
Test Group 2: Creative Theme (Be Heard) – LGBTQI Targeting
Display Channel
Facebook Channel

H1 - We invite you to be one in a
million to help speed up medical
breakthroughs.
H2 - We want you to be one in a million
to help speed up medical
breakthroughs.
Test Group 2: Creative Theme (Breakthrough) – Contextual Targeting
Display Channel
Facebook Channel
Slight preference for Headline 1 albeit
impressions delivered nullify results.

H1 - She has her mom's eyes and
her grandpa's heart disease.
H2 - Every woman in my family has
had asthma. I want to be the last.
Test Group 2: Creative Theme (Generational) – Contextual Targeting
Review Period: Campaign to Date (Nov. 20 – Mar. 5, 2019) n = 875,079
Display Channel
Facebook Channel
Strong preference for Headline 1 albeit
impressions delivered nullify results.

H1 - Does cilantro taste like soap? H2 - Can you curl your tongue?
Test Group 2: Creative Theme (Health) – Contextual Targeting
Display Channel
Clear winner for Headline 2.
Facebook Channel
Clear winner for Headline 2.

H1 - Accelerated medical research…
fueled by you.
H2 - Medical research hasn't always
seen you. We're changing that.
Test Group 2: Creative Theme (Personalized/Speed) – Contextual Targeting
Display Channel
Facebook Channel
Clear winner and strong preference for
Headline 1.

Phase 1 Test Insights & Learnings
73
Test results revealed the selection of headlines in large had little appeal differentiation. Key takeaways follow:
----------------
1. The inability to validate results without a tracking conversion point hindered the A/B testing and the final conclusions and insights.
Without a conversion point, CTR and CPC were the leading performance indicators, but does not reveal actual test impact on program
enrollments.
2. Headline tests need to be single variable tests and rely on banner text ads as the delivery ad format without image. Images may skew or
sway results based on not being able to glean useful test results in Phase 1 testing.
3. Creative theme and headline testing may have been misaligned and overly engineered when targeting ethnic audiences. More
consideration is needed for how we apply targeting to our testing in the future to ensure creative and copy are universally validated
across all target audiences.
4. Headlines that speak directly to the person do seem to fair better. Requires additional testing.
5. The use of ‘You’ in headline text creates enhanced appeal. Requires additional testing.
6. Action phrases such as ‘we want you’ instead of ‘we invite you’ work best. Requires additional testing.
7. Pithy, emotive headline themes and copy (Health) that is light without being overly complex or medical does well to create curiosity and
clicks, but true click intent for actual interest versus simple curiosity and the impact on a website bounces can not be validated without a
tracking pixel in place. We may be able to overcome this by activating a pixel for the Join Now button.
----------------

Test Phase 2
Mar. 6 – Mar. 31, 2019
Headline & Image Testing Results

Phase 2 testing commenced on March 6, 2019.
There are two variations of tests underway.
Within this Test Phase, we are continuing to test
headlines with one audience group and images with
another group to gather deeper insights.

HEADLINE TEST
76
Houston Phase 2 Headline Test Methodology
HeadlinetestsarealignedbyCreative Themeandtargetedtospecificaudiencegroupsacrosstwochannelplacements(DisplayandFacebook).

HEADLINE TEST
77
Houston Phase 2 Headline Test Methodology
HeadlinetestsarealignedbyCreative Themeandtargetedtospecificaudiencegroupsacrosstwochannelplacements(DisplayandFacebook).

78
Houston Phase 2 Image Test Methodology
Image tests are aligned by Creative Theme and targeted to specific audience groups across two channel placements (Display and Facebook).
IMAGE TEST

Image A Image B
H - It starts with YOU. It gets better
with All of Us.
Test Group 1B: Image Test (Altruistic) – Contextual Targeting
Which image within the Test Group performed best for Phase 2 A/B testing?
Review Period: Campaign to Date (Mar. 6, 2019 – Mar. 31, 2019) n = 594,930
Display Channel
Image A preference
Local Channels
Image B preference
Facebook Channel
Image B preference

H1 – There’s a gap in medical
Research that only you can fill.
what’s yours?
Test Group 2B: Headline Test (Be Heard) – African-American Targeting
Display Channel
Headline 2 preference
Facebook Channel
UrbanOne Channel
Local Channels

what’s yours?
Test Group 2B: Headline Test (Be Heard) – Asian-American Targeting
Review Period: Campaign to Date (Mar. 6, 2019 – Mar. 31, 2019) n = 2,105,439
Display Channel
Facebook Channel

what’s yours?
Test Group 2B: Headline Test (Be Heard) – Hispanic Targeting
Display Channel
Facebook Channel
Telemundo-Univision Channel
Local Channels

what’s yours?
Test Group 2B: Headline Test (Be Heard) – Elderly 65+ Targeting
Display Channel
Facebook Channel
AARP Channel
No preference yet
Local Channels

what’s yours?
Test Group 2B: Headline Test (Be Heard) – LGBTQI Targeting
Display Channel
Facebook Channel

H1 – The most diverse medical
research program. Ever.
H2 – We want you to be one in a million
to help speed up medical
breakthroughs.
Test Group 2B: Headline Test (Breakthrough) – Contextual Targeting
Display Channel
Facebook Channel
AARP Channel
No preference yet
Local Channels
No preference yet

H1 – You brought them into this world.
Now help make it a healthier one.
H2 – Every woman in my family has
had asthma. I want to be the last.
Test Group 2B: Headline Test (Generational) – Contextual Targeting
Display Channel
Facebook Channel
Local Channels

H1 – Are you left-handed? H2 – Can you curl your tongue?
Test Group 2B: Headline Test (Health) – Contextual Targeting
Display Channel
Facebook Channel
Local Channels

Image A Image B
H1 – Accelerated medical research…
fueled by you.
H2 – Accelerated medical research…
fueled by you.
Test Group 2B: Image Test (Personalized/Speed) – Contextual Targeting
Display Channel
Image B preference
Facebook Channel
Image A preference
Local Channels
Image B preference

Phase 2 Test Insights & Learnings
89
Test results are early 3-weeks into testing. Limited takeaways so far include:
----------------
1. Image tests should leverage demographics to truly test personal resonance and appeal based upon self-identification. This requires
more imagery aligned to gender and age.
----------------

Closing Takeaways
91
1. We’re facing a Catch 22 situation where we need to optimize the campaign based on the current tracking mechanisms in place. Do we
optimize to conversions as we have done early in the campaign when tracking was in place, or do we shift more to click optimization as
we are moving to as of March. The stop gap solution is to activate a tracking pixel for intent and the Join Now button click. This will allow
us to optimize to that event while maintaining integrity in our optimization direction.
2. We must convince decision makers to adjust the Bounce Rate as a remedy to eliminate the false narrative associated with Paid Media.
3. We must reconsider best practices for our A/B testing process and frame it so we can ascertain better insights and learnings. It may not
be aligned with Scripps’ over technical data analysis approach, but it should use industry media best practices by leveraging single
variable testing.
4. Market saturation is a concern. Going forward we need to be cognizant of over saturating a market over a short period of time creating
diminishing returns. A better approach for A/B testing may be to get in and get out within a defined time period and move testing in
cycles to new markets for greater sustained learnings.
5. Via our paid media analysis and the documentation of its success, the weak links in the national marketing as added touchpoints in the
pre-enrollment journey do become social media and email. More integration and more planning is required to have a holistic integrated
marketing approach.
6. We must benchmark our performance. Frequent research and updated findings need to be published to reinforce paid media’s value
along with researching CPAs across the consortium for all enrollment journey paths.
7. We must continually educate all stakeholders for the value of paid media at every opportunity with a defined and adopted reporting
structure.

BENCHMARKING
Q. Looking ahead how should we
benchmark our media performance?

Of note, industry benchmarks referenced on the following slides for display
advertising are taken from the Google Display Network, which leverages search
and are misaligned for programmatic buys.
Industries selected for campaign comparison are not similar
enough to our own program, but may be loosely used as one
benchmark source with this understanding.

Benchmarking a research program like ours falls into a special niche category
and is difficult to compare against published industry benchmarks. Furthermore,
we should understand there are distinct performance nuances between a display
impression for search vs. programmatic.
When benchmarking, a better approach may be to research clinical trial
RECRUITMENT digital marketing case studies to extrapolate key
performance campaign metrics for a more accurate and comparable
representation of our own performance.

Optimization
 As noted, the national and Houston campaigns have historically been optimized for Conversion, and CPA with an emphasis on quality of
click vs. quantity of clicks.
 Of further note, there is a difference between optimizing to consumers and marketing at the enterprise level for securing an expected and
desired tangible (conversion) outcome. For a campaign like ours, it is imperative we optimize for conversions as opposed to clicks and
build consensus that this direction reaps the largest program benefit.
 CTR and CPC are good performance metrics for search, but not so good for Programmatic. They can be used as cross-tabulated support
metrics, but should not be used in isolation or a leading KPIs.
 Finding comparable program case studies should be the benchmark measure vs. industry benchmarks. More research is required, but
case studies already discovered reveal our campaign is performing very well.
Benchmarking Considerations

Search vs. Programmatic Display
 Benchmarking should always align apples to apples and benchmark specific campaign media channels, i.e., Display (Programmatic),
Facebook, and other channels we may test.
 The nuances associated between search and programmatic display is of special consideration. Search advertising borrows from the
inherent utility of the search engine where consumers are presented with ads that match their query and are designed to quickly direct
them to specific products and/or services.
 Online display advertising (programmatic) is a different story where people are online to visit sites for information, entertainment and
engagement and not to click on ads to send them elsewhere. Therefore, when ads are served across this landscape, target audiences will
only click if the advertisement resonates with the individual at a deeper level. Thus, the quality of click for programmatic becomes much
more valued than either search or Facebook ad clicks.
 We should understand industry benchmark metrics for search will always be represented as higher than those for Programmatic. When
benchmarking using industry benchmarks, it is critical we use informational sources where possible to break out the differentiation, and
find the most appropriate use for them.

Case Study Comparisons
 A number of comparable programs have been identified, and which we can extrapolate relevant KPI performance benchmarks to cross-
compare against our own program. We believe this is most accurate benchmark to compare performance. If adopted as the most
representative approach to benchmarking, we will continue to update these case studies in repeatable cycles to continuously measure,
optimize and report on our performance.
Industry Benchmark Comparisons
 As noted, there may be risks for misinterpreting public domain industry benchmarks without more deeply exploring the type of
organizations included in the benchmarks and for how the campaigns behind the benchmarks have been optimized (e.g., clicks vs.
conversion). As an example, the health and/or health & beauty category, a commonly found published benchmark, is not an appropriate
comparative benchmark as it may represent medical services and/or consumer product companies who have typically optimized
campaigns based on clicks.
 If industry benchmarks are used, the closest comparable industry benchmarks to our own program would be unique based on the source
of the benchmark. Yet, they are still not equal in comparison.

Baseline Metric Nuances
 CTR: A high click-through rate doesn’t tell us anything about the quality of website visitors. For instance, a campaign might be getting a
0.3% CTR, more than 3x the average click-through rate for display ads. Still, if deeper analysis were applied, and the bounce rate for the
campaign is 95%, something’s not quite right if a large percentage of clicks just leave. So, BOUNCE RATE is an important consideration
in combination with CTR. (Note: please see bounce rate discussion slides.)
 CPC: There’s a silent rule of thumb associated with this metric: a high CPC means low performance. Like with CTR, that’s not
necessarily true. Even if you pay more for each click that you get, if the clicks are relevant, they’re ultimately worth the price.
 Conversions: Conversions represent the desired end-result user action for the media campaign. Conversions are an insightful way to
measure our performance because that’s what we are really advertising. Because we want to increase program signups, we don’t simply
want clicks (click-through conversions) and we don’t simply want low CPCs. We want to measure view-through conversions. The benefit
of the latter is that we are able to attribute the ad impression to a conversion post initial website visit and when the person returns. This is
critical and it becomes a CORE KPI.
 CVR: Conversion rate represents the percentage of people that complete account signup from those who clicked our media ads and
visited the website. This is a CORE KPI.
 CPA: While conversions and conversion rate are important, the final end-all performance measurement is CPA. To calculate the actual
return on investment for our budget spend, we need to calculate CPA and then cross compare it against other marketing activities. CPA
is not equal across channels. Therefore, continued optimization for CTR cross tabulated against CVR and CPA is the best optimization
direction for our campaign.

100
What Case Studies May Be Most Relevant for Benchmarking?
https://www.wordstream.com/blog/ws/2016/02/29/google-adwords-industry-benchmarks
Patient Recruitment & Clinical Trial Digital Campaign Case Studies KPI #1 KPI #2 KPI #3
Case Study
Resource
Case Study
Comparisons
Clinical Trial Recruitment Situation Objective Budget Spend
Ad Campaign
Impressions
Generated
Website Visits
Generated
Impressions to
Recruitment
Ratio
Enrollments CTR CVR CPA
Wondros All of Us National
Media
Medical Research Campaign Objective: create
consistent monthly awareness
and interest to drive traffic to the
national website to support
enrollment (account) signups.
Site traffic &
enrollments
$846,000 195M 216K 64,655 3,016 0.11% 4.30% $136
Wondros Pitt + Me Campaign
Performance
Registry & Medical Study
Interest Enrollments
Campaign Objective: capture
interest to drive traffic to the
website for registry and study
signups.
Site traffic &
enrollments
$86,000 20M 48K 23,753 842 0.24% 1.75% $102
GoPraxis.com Case Study #1 -
Campaign: Tardive
Dyskinesia Study
Central Nervous System:
Kinect - Tardive Dyskinesia
Study. Treatment options that
would help reduce the
severity of TD symptoms.
Campaign objective: capture
website for study signups.
Site traffic &
enrollments
$368,808 70M 65K 551,181 127 0.09% 0.20% $2,904
MDConnect Case Study #2 v1
(Rollout) - Campaign:
Chronic Pain Study
Clincal Trial Patient
Recruitment: Chronic Pain
Campaign objective: leverage
advertising (print, TV, radio,
digital) to drive interested 'patient
referral' website traffic to site
across national metro and rural
footprint in support of 23 pain
treatment locations.
Site traffic &
prescreen
survey
completion
$160,110 unknown unknown n/a 593 n/a n/a $270
MDConnect Case Study #2 v2
(Relaunch) - Campaign:
Chronic Pain Study
Clinical Trial Patient
Recruitment: Chronic Pain
search marketing and digital
advertising (display, contextual,
retargeting & Facebook) to drive
down costs of traditional media
($270 average/referral) and to
drive patient 'referral' to the
clinical trial website.
Site traffic &
enrollments
$57,639 n/a n/a n/a 420 n/a n/a $137
30 days before pixel removal

101
What Case Studies May Be Most Relevant for Benchmarking?
Patient Recruitment & Clinical Trial Digital Campaign Case Studies KPI #1 KPI #2 KPI #3
Case Study
Resource
Case Study
Comparisons
Clinical Trial Recruitment Situation Objective Budget Spend
Ad Campaign
Impressions
Generated
Website Visits
Generated
Impressions to
Recruitment
Ratio
Enrollments CTR CVR CPA
Wondros All of Us National
Media
Medical Research Campaign Objective: create
consistent monthly awareness
and interest to drive traffic to the
national website to support
enrollment (account) signups.
Site traffic &
enrollments
$846,000 195M 216K 64,655 3,016 0.11% 4.30% $136
Clariness Case Study #3 -
Campaign: Lupus
Study
Recruitment: Lupus
Campaign objective: drive traffic
to a trial-specific landing page set
up for study registration and a
pre-screening questionnaire.
Site traffic &
enrollments
n/a n/a 1MM n/a n/a n/a n/a $750
Campaign Multiple
Sclerosis Study
Cinical Trial Patient
Recruitment: Multiple
Sclerosis treatment options
to manage and reduce
relapses
online search, display and
Facebook advertising to drive
website traffic and audiences to
study website.
Site traffic &
enrollments
$52,700 10M 23K -- n/a 0.23% n/a $521
Campaign Tardive
Dyskinesia Study #2
Cinical Trial Patient
Recruitment: Central Nevous
System: Schizophrenia or
Schizoaffective Disorder
treatment options to reduce
severity of TD symptoms
online digital media to capture
Site traffic &
enrollments
$79,502 70M 35.5K 94,488 127 0.05% 0.36% $626
Campaign Tardive
Dyskinesia Study #2
Recruitment (Part 2): Central
Nervous System:
Schizophrenia or
Schizoaffective Disorder
treatment options to reduce
severity of TD symptoms
online digital media to capture
Site traffic &
enrollments
$63,226 12M 43.5K 118,812 101 0.36% 0.23% $626
30 days before pixel removal

102
What Industries or Market Sectors are Available for Benchmarking?
Display Benchmark Comparisons Facebook Benchmark Comparisons
Source Report Date Industry CTR CPC CVR CPA Report Date Industry CTR CPC CVR CPA
All of Us National
Media Campaign
Lifetime
(May—Oct 2018)
Medical Research 0.04% $6.60 3.37% $236.23
Lifetime
(May—Oct 2018)
All of Us National
Media Campaign
30-Day Recent
(Sep—Oct 2018)
30-Day Recent
(Sep—Oct 2018)
All of Us Houston
Media Campaign
Lifetime
(Nov—Feb 2019)
Medical Research 0.05% $5.16 n/a n/a
Lifetime
(Nov—Feb 2019)
Medical Research 0.61% $1.96 n/a n/a
Programmatic
(varied)
2018 All 0.05% TBD TBD TBD -- -- -- -- -- --
Wordstream
Google (GDN)
Dec 2018
Health & Medical 0.59% $0.63 0.82% $72.58
FB Benchmarks
Feb 2019
Healthcare 0.70% $1.32 11.0% $44.66
Wordstream
Google (GDN)
Dec 2018
Advocacy 0.59% $0.62 1.00% $70.69
FB Benchmarks
Feb 2019
Consumer Services 0.62% $3.08 9.96% $31.11
Wordstream
Google (GDN)
Dec 2018
All Industries 0.46% $0.63 0.77% $75.51
FB Benchmarks
Feb 2019
All Industries 0.90% $1.72 9.21% $18.68
https://www.wordstream.com/blog/ws/2017/02/28/facebook-advertising-benchmarks
Note: When reviewing benchmarks, All of Us national & houston media campaigns optimized for conversions and CPA (not clicks).

Media Campaign Performance Update

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Media Campaign Performance Update

Similar to Media Campaign Performance Update (20)

More from Daniel McKean

More from Daniel McKean (20)

Recently uploaded

Recently uploaded (20)

Media Campaign Performance Update