My AAPOR 2019 presentation on how and why we expanded the ATP. Additional information: https://www.pewresearch.org/methods/2019/02/27/growing-and-improving-pew-research-centers-american-trends-panel/

1. Using Address-Based Sampling
to Recruit to Pew Research
Center’s American Trends Panel
Nick Bertoni
2019 AAPOR 74th Annual Conference
Panel Manager

2. May 17, 2019 2
American Trends Panel Overview
• National probability-based sample of adults who take surveys for us
• 100% online survey administration
• Non-internet households are provided tablets and data plan
• We conducted 1 to 2 surveys per month, each 15-min long
• Created in 2014
• $5 to $20 for each survey
• Primary platform for Center domestic data collection

3. May 17, 2019 3
Recruiting to the ATP
Previous recruitments
• RDD telephone
• 2014-2015 piggy back on end of politics survey
• 2017 standalone recruitment survey
2018 recruitment
• Address-Based Sampling (ABS)
• Push-to-web design
• Experiment for recruiting non-internet households

4. May 17, 2019 4
Why the switch from RDD to ABS?
Reduce limitations of declining RDD response rates

5. ABS RECRUITMENT METHODOLOGY
May 17, 2019 5

6. May 17, 2019 6
ABS Sample Definition
Stratified random sample from the USPS Delivery Sequence File (DSF)
targeting
the following stratum:
• Hispanic 18-24
• Hispanic 25+
• Black-AA 18-24
• Black-AA 25+
• Other 18-24 (non-Hispanic/non-Black)
• All else 25+ (non-Hispanic/non-Black)

7. May 17, 2019 7
Contact Protocol
The basics
• 9x12 invitation letter
• Two $1 bill cash pre-incentive, promise of $10 post-incentive
• Oversized color reminder postcard
• 9x12 reminder letter

8. May 17, 2019 8
Invitation Letter

9. May 17, 2019 9
Postcard Reminder

10. May 17, 2019 10
Reminder Letter

11. May 17, 2019 11
Actual Invitation Letter

12. May 17, 2019 12
Recruitment Materials
Methodological twists
• Bilingual materials only with Hispanic stratum
• Within-household selection method: Next birthday
• Experiment for non-internet households

13. May 17, 2019 13
Recruitment Survey Questionnaire Design
• Mobile optimized
• Automatic language selection
• Core demographics
• No polarizing or controversial content
• Panel invitation

14. May 17, 2019 14
Recruitment Results
• Field period: August 8 – October 31, 2018

15. May 17, 2019 15
Recruitment Results
• Field period: August 8 – October 31, 2018
• 9,341 respondents to the survey

16. May 17, 2019 16
Recruitment Results
• Field period: August 8 – October 31, 2018
• 9,341 respondents to the survey (AAPOR RR3 = 11.7%)

17. May 17, 2019 17
Recruitment Results
• Field period: August 8 – October 31, 2018
• 9,341 respondents to the survey (AAPOR RR3 = 11.7%)
• 8,778 survey respondents joined the panel

18. May 17, 2019 18
Recruitment Results
• Field period: August 8 – October 31, 2018
• 9,341 respondents to the survey (AAPOR RR3 = 11.7%)
• 8,778 survey respondents joined the panel (94%)

19. May 17, 2019 19
Recruitment Results

20. COMPARING NEW PANELISTS TO THE CURRENT
PANEL
May 17, 2019 20

21. May 17, 2019 21
Profile Survey Respondent Comparison
Key Demographics
9 9
10
15
19
11
12
17
18
22
0
5
10
15
20
25
Black non-Hispanic Hispanic 18-29 years old HS or less
education
<$30,000 income
Current panelists
N=4,792
New panelists
N=8,778
%
Data are unweighted

22. May 17, 2019 22
Profile Survey Respondent Comparison
Party ID /Ideology / Voter Registration
27
38
35 33
89
26
36 38
29
81
0
10
20
30
40
50
60
70
80
90
100
Republican Democrat Ind/Other/Ref NET Conservative Registered to vote
Current panelists
N=4,792
New panelists
N=8,778
%
Data are unweighted

23. May 17, 2019 23
Profile Survey Respondent Comparison
FOLNEWS: Would you say you follow the news…
88
63
25
9
3
84
56
28
11
5
0
10
20
30
40
50
60
70
80
90
100
NET All or
most/Some
All or most of the
time
Some of the time Only now and then Hardly ever
Current panelists
N=4,792
New panelists
N=8,778
%
Data are unweighted

24. May 17, 2019 24
Profile Survey Respondent Comparison
FOLGOV: Follow what’s going on in gov’t and public affairs…
87
61
26
9
4
80
53
27
13
7
0
10
20
30
40
50
60
70
80
90
100
NET Most/some of
the time
Most of the time Some of the time Only now and then Hardly at all
Current panelists
N=4,792
New panelists
N=8,778
%
Data are unweighted

25. RECRUITING NON-INTERNET HOUSEHOLDS
May 17, 2019 25

26. May 17, 2019 26
How do you get non-internet households
into a 100% online panel?

27. May 17, 2019 27
Non-Internet Household Recruitment Experiment
• Sample was randomly assigned to control group or treatment group
• Control group received standard recruitment protocol
• Treatment group received additional instructions and a return postcard

28. May 17, 2019 28
Non-Internet Household Return Postcard

29. May 17, 2019 29
Household Internet Screener

30. May 17, 2019 30
Non-Internet Household Experiment Results
818 returned the postcard
461 actually had the
internet
264 could not be reached
93 were eligible for the
tablet
55 took the tablet and ran
38 joined the panel

31. May 17, 2019 31
Non-Internet Household Recruitment Summary
• 38 tablet panelists came from the treatment group
• 27 tablet panelists came from the control group
• 65 total tablet panelists were added…
• …out of 8,778

32. IN CONCLUSION…
May 17, 2019 32

33. May 17, 2019 33
Summary of Key Findings
What went well
• Response rate
• Panel agreement rate
• Key demos: 18-29 years old, HS or less education, non-white, non-
registered, <$30k income, follow news or government less often
What did NOT go well
• Non-internet households
• Less-acculturated Hispanics
• Spanish language panelists

34. May 17, 2019 34
Contact Information
Nick Bertoni
Panel Manager
nbertoni@pewresearch.org