The document discusses how political campaigns and advocacy groups can objectively evaluate different voter outreach tactics through randomized controlled experiments in order to maximize their chances of success. It provides an example of how a campaign client tested the effectiveness of tele-town halls through a randomized experiment involving different groups of voters. The results showed tele-town halls increased voter support for the candidate but the effect varied based on the number of times voters participated, allowing the campaign to determine the most cost-effective strategy.

3. • Campaigns and issue advocacy efforts exist for one reason: to maximize their candidates/issues odds of success
• They achieve this objective by pulling one of two levers:
- Turnout: Turning out those who would support your candidate/issue
- Persuasion: influencing those who will turn out to support your candidate/issue
• They attempt to pull these levers by paying for voter contacts across a range of mediums, from robo-calls to
volunteer phone bank phone calls to television ads
• The way voter contacts are deployed is subjective:
- Voter Contact Copy: Be it mail designs, TV ads, or phone scripts, typically, creative, is decided by “experienced hands” sitting
around a table or getting on a phone call, and debating until a consensus is reached
 Problem: Just because a group of experienced individuals thinks it will work does not mean it always will
- Medium selection: the mediums a campaign or advocacy issue chooses to communicate through are often influenced by
the “experienced hands” on a campaign, who typically own a company selling voter communications via one medium
 Problem: Perverse incentives often guide recommendations
- Medium mix: The % of spend going to each medium of voter contact is decided again by these experienced hands
 Problem: Perverse incentives often guide large % of spend to the the “experienced hands” who have closest
relationship with candidate or key decision maker

4. • How do we select voter contact objectively while maximizing the candidate/issue’s chance of success?
• When you have lots of voter contact options, and little more than subjective (and potentially self-interested)
opinions to guide decisions, you need objective measurement to get the best return on investment.
• It is true that measuring political voter contacts isn’t easy, but 0ptimus has developed deep experience in
measuring the effectiveness of voter contact across all contact mediums
- 0ptimus staff has conducted over 500+ voter contact measurements, from robo calls to TV ads.
• As it turns out, any voter contact can be tested… yes we said any. You can know if a given piece of voter
contact is going to work, and at what cost per vote, prior to deploying large quantities of resources.
• Why you should care: if you invest in objective measurement you can credibly do what campaigns/advocacy
efforts are supposed to do: maximize the candidate or issue’s odds of success at the ballot box

6. Maximize return on
investment
Maximize return on
donor dollar
Maximize patient
outcomes
Owners guess what
is good/not for
their business
Donors give to
causes based on
emotion and
pitches
Witch-doctors rub
random salves in
wounds
Accounting
Tracking against key
outcome metrics
(e.g. benefit/$)
Randomized
controlled
experimentation
Exponential
increase decade-
over-decade of
global output
Efficient use of
donor dollars: each
$ spent does more
good
“Western”
medicine,
antibiotics, field of
nutrition, expanded
lifespans
Field Goal The fix OutcomeThe flawed way

7. Field Goal The fix Outcome
Maximize return on
investment
Maximize return on
donor dollar
Maximize patient
outcomes
Owners guess what
is good/not for
their business
Donors give to
causes based on
emotion and
pitches
Witch-doctors rub
random salves in
wounds
Accounting
Tracking against key
outcome metrics
(e.g. benefit/$)
Randomized
controlled
experimentation
Exponential
increase decade-
over-decade of
global output
Efficient use of
donor dollars: each
$ spent does more
good
“Western”
medicine,
antibiotics, field of
nutrition, expanded
lifespans
->Let’s walk through measurement in medicine…
The flawed way

8. • “Medicine” has existed for a very long time, likely starting
with tribal holy-men doing enthusiastic dances to appease
the gods, evolving to Medieval doctors drawing on notions of
poisons that needed to be extracted, to what we recognize
today as modern western medicine
• In the past, doctors would try treatments they thought could
cure patients. If the patient lived, they associated the
medicine with the outcome, if the patient died, they
determined the treatment to be ineffective.
-The problem: these observations were anecdotal, and
did not “control” for other factors around the patient
E.G. what if treatment was truly ineffective, and
patient recovered for reasons other than the
treatment?
E.G. what if treatment was truly effective, but while
effectively stopping the bleeding, it could not cure the
infection caused by the wound?
• This had some adverse consequences….

9. -Patients likely suffering from viruses or infections were treated with
bloodletting, the purposeful draining of blood in times of sickness by cuts
or leeches. Modern medicine has reveled this to be counterproductive,
despite it being the common advice of the “experienced hands” of the
time
-To knock patients out before painful medieval surgery, a potion called
“Dwale” was often given. It would indeed often knock patients out.
Unfortunately, its ingredients also caused deaths that were likely
associated with the surgery (instead of the potion!). The “experienced
hands” thus kept administering the potion- after all, it did knock the
patient out.
-People suffering from seizures, migraines, or medical disorders had holes
drilled in their head in the practice known as “Trepanation”. Modern
medicine has revealed this to be counterproductive, despite the opinion
of many “experienced hands” of the time.
-In recent history (1900s) people suffering from Ashtma were given
“medical cigarettes” or the fumes of combustive powders to inhale.
Modern medicine has, you guessed it, reveled this to be extremely
harmful, despite the opinion of the “experienced hands” of the time.

11. ->When your doctor has an
idea for a “treatment” he
tests it with an experiment.
Let’s say he believes Aspirin
can prevent or reduce the
harmful effects of massive
heart attacks in at-risk
patients…

12. Test Population
->The first thing a doctor
does is find a large group
(say a couple thousand) of
“at-risk patients for heart
attacks”

13. Test Population
Control +
treatment groups
->He then randomly splits
the group into two groups,
and checks to ensure the
groups are balanced for pre-
existing factors that could
effect the outcome of the test.
For example: age, sex,
ethnicity, prior heart issues,
proximity of patient to
pollution sources, etc.

14. Test Population
Control +
treatment groups
Treatment and
control issued
->One group is given the
medicine the doctor wants to
test, another is given a
sugar pill

15. Test Population
Control +
treatment groups
Treatment and
control issued
Results measured15% heart
attack deaths
->At the end of the study
period, the doctor examines
health outcomes. If he sees
an effect, he knows it to be
that of the medicine, because
he controlled for all outside
factors
10% heart
attack deaths

16. Test Population
Control +
treatment groups
Treatment and
control issued
Results measured
0
No contact
30% voted 35% voted
15% heart
attack deaths
10% heart
attack deaths

18. • Øptimus client: Incumbent, very well-known candidate in state-wide
TX primary contest
• Campaign is considering using tele-town halls to communicate with
voters
-Pro:
A personal form of contact, promotes feel of interaction with candidate
A longer impression than a TV ad, or a mail piece read
-Con:
Somewhat expensive
Takes a lot of candidate/staff time to pull off
• Question: do tele-town halls persuade voters to vote for the
candidate? What is the cost per vote of utilizing this technique? If one
town-hall touch is good, are multiple town-hall touches better?

20. 50k Texas Republican primary voters (verified by the vote file), once an
individual at targeted household spent 1 min in a town hall, not called for
subsequent town halls (each target voter could enter into a town hall once)
50k Republican primary voters, left untouched, no call attempts made
50k Texas Republican primary voters (verified by the vote file), targeted for
contact via town hall. Each targeted voter is dialed and given an opportunity
to join a town hall on three separate nights over a 1.5 week time period (thus
a targeted voter could enter a town hall up to three times)

21. • Each 60 min tele-town hall featured
candidate for statewide office
-Intro by comms director (1 min)
-Q&A interaction between candidate for
remainder of call (remainder of time)

22. • Verified identity of targeted voter (“Hi is
this Jane?”)
• Asked verified voter (not their spouses,
kids, etc) who they planned to vote for
in the upcoming statewide TX election
-Offered names of all candidates, in
randomized order
• Aggregated results…

23. 10.0
20.0
30.0%
Control
19.3%
Townhalls
Treatment #1
(re-dialed regardless
of past participation)
22.8%
Townhalls
Treatment #2
(only re-dialed if did not
enter previous townhall)
22.6%

24. Group Effect
Test
Population
size
Voter minds
moved
Cost for
treatment
Cost per
vote
3.5%
3.3%
50k
voters
50k
voters
1,750
1,650
$20.1k
$20.1k
$11.48
per vote
$12.18
per vote

26. • Campaigns and issue advocacy efforts exist for one reason: to maximize their
candidates/issues odds of success
• Measurement allows decision makers to rationally achieve that objective
 Each form of voter contact can be measured for efficacy (out of 100 voters, X will move after
being touched by this form of voter contact), and then efficiency (and each moved voter will
cost us $Y.YY)
 When multiple forms of voter contact are measured to establish their efficacy and efficiency,
design makers can objectively choose between voter contact options
• No two campaigns are alike (don’t think that the takeaway here is to go do tele-town
halls!), a piece of creative that works for Candidate A likely will not work for Candidate
B; a tactic that works on the PR campaign for Defense Firm Q likely will not work for
Agricultural Firm R.
• Every credible business must invest in measurement (accounting), every credible
medical researcher must invest in measurement (experimental trials), and every
credible candidacy/advocacy effort must invest in measurement (voter contact testing)

27. Optimus is a consultancy that uses data-driven
analysis to drive superior results for our political,
issue advocacy, and non-profit clients
• Experience across campaigns
ranging from city council to
President
• Experience working with issue
advocacy and non-profit clients,
small ($500k) to large ($50M+)
• Partners have backgrounds prior to
0ptimus based in the real world: from
infantry leadership to top-tier
management consulting
• Wide range of services based on using
data, measurement and analysis to
drive processes that generate higher
ROI for the donor dollar
• 15-man staff with backgrounds
ranging from prior AMD executive
to PhD in behavioral psychology
• True “consulting” model: analytical
teams that travel to client sites (as
needed) to perform work shoulder-to-
shoulder with clients

29. - Experience in House, Senate, Gubernatorial, and
Presidential races building custom tailored data
solutions & analytics to needs of campaign
- Finance degree, American
- Free time: Xbox, gourmet pancake making
- Prior analytical experience ranging from state-wide &
issue advocacy campaigns to fortune 500 case work at
Bain & Company
- Engineering degree, University of Texas at Austin
- Free time: Texas Longhorns, Dallas Cowboys, Rye
( Yea, Republicans can be smart with data too….)
- Our services:
 End-to-end bespoke data infrastructure for
campaigns and data-driven ROI-oriented talent
 Small-to-large scale quantitative analysis (controlled
tests, A/B tests, ROI analysis, benchmarking)
 In-campaign testing/experimentation support
(know if you are getting bang for your buck)
 Vote modeling and opinion shift monitoring
 Scenario based analytics
- Real expertise:
 Experience across small (ballot initiative) to large
(presidential & targeted senate seat) data-driven
campaigns for candidates and 501c organizations
 Experience integrating campaign data systems across
political, field, comms, finance, and digital
 Over 500+ randomized control experiments run in field
in live campaigns ( turnout + persuasion )
 Supported and double checked by Ph.Ds (our work hits
academic quality) and other smarties when needed
 Oh, and we are also experts at Buckhunter, Ninja
Turtles Arcade, and many more high-tech cutting edge
platforms… so bring it.
- 16 years working in campaigns from State Legislature to
President
- Founder of RNC's Field School and contributor
curriculum's for multiple candidate/campaign schools
- Marine Corps veteran
- Free time: obsessive sports parent/coach to 3 boys