2. Founded in 1997.
Located in Upstate NY.
Extensive expertise and knowledge in survey design,
administration, analysis and data submission.
National survey work includes B2B, B2C, government,
medical, telecommunications and utility projects.
MetrixMatrix.com (888) 865-0065 2
3. MetrixMatrix.com (888) 865-0065 3
In 2012, Metrix Matrix implemented the SmartID method by which
outbound caller ID area codes were matched to target respondents’ area
codes.
This SmartID method was expected to improve efficiency and survey
completion rates.
In a 2014 poster presentation to AAPOR, preliminary research revealed
that projects conducted with SmartID were less likely to result in a
completed survey.
4. MetrixMatrix.com (888) 865-0065 4
Can this method increase response rates within an
individual project when respondents receive calls that are
associated with local (i.e., familiar) area codes?
And…
Does the use of SmartID reduce cost(s) by improving
performance?
Looking more deeply into the before-and-after effects of
SmartID…
5. MetrixMatrix.com (888) 865-0065 5
Utility Authority within the State of
Oklahoma
Residential customers were asked about
satisfaction with the utility service/pricing
Telephone survey, 38 questions
Wave 1: 9/5/2014-10/7/2014
SmartID was not used
Records were attempted up to 3 times
Wave 2: 10/8/2014- 11/5/2014
SmartID was used in the 405 area code only
Records with 3 attempts (from Wave 1) were re-called, with no
more than 2 additional attempts
New phone records were added
6. MetrixMatrix.com (888) 865-0065 6
Region Name Area Code
Target
Sample
Completed
Surveys
(Wave 1)
% Completed
Surveys /Target
(Wave 1)
South West 580 350 319 91%
South Central 405, 580 350 255 73%
Central
405, 580,
918/539
450 188 42%
North
580,
918/549
700 549 78%
TOTAL: 1,850 1,311 71%
7. MetrixMatrix.com (888) 865-0065 7
20%
36%
45%
60%
55% 57%
100%
51%
100%
70%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
City A City B City C City D City E City F City G City H City I City J
Midpoint Sample Assessment: Central Region Completes, by City
After Wave 1, completed surveys for three cities within the Central Region were still less
than 50% complete based on their target sample size. All three cities were also within the
405 area code.
8. MetrixMatrix.com (888) 865-0065 8
(Contact Made)
24%
6%
63%
7%
34%
11%
52%
4%
38%
9%
50%
2%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Contact Fail Completed Survey Declined Survey Problem Record
Wave 1 Record Status, by Area Code
405
580
918/
539
(9%)
Records within the 405 area code were more likely to result in a contact, when compared to
records in the remaining area codes.
However, when contacts were made, calls made to the 405 area code were less likely to result in a
completed a survey (9%) when compared to the 580 area code (17%) and the 918/539 area code
(15%).
% Complete/
Contact Made
(17%) (15%)
9. MetrixMatrix.com (888) 865-0065 9
(Contact Made)
24%
6%
63%
7%
55%
4%
38%
4%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Contact Fail Completed Survey Declined Survey Problem Record
Wave 1 and Wave 2 Final Record Status for SmartID
in the 405 Area Code
Wave 1
Wave 2
After implementing the SmartID method (Wave 2), records were more likely to result in a failed
contact.
When a contact was made, SmartID calls were no more likely to generate a completed survey when
compared to Wave 1.
% Complete/
Contact Made
(10%)(9%)
10. MetrixMatrix.com (888) 865-0065 10
(Contact Made)
40%
5%
51%
4%
58%
3%
35%
4%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Contact Fail Completed survey Declined Survey Problem Record
Wave 2, Record Type with SmartID
Wave 2: Re-
loaded Records
Wave 2: New
Records
In Wave 2, 15% of the SmartID calls were re-loaded sample records from Wave 1 and 81% were new-
purchase records.
Overall, new-purchase records were more likely to result in a failed contact.
Re-loaded sample records were more likely to result in an overall contact.
% Complete/
Contact Made
(8%)(9%)
11. MetrixMatrix.com (888) 865-0065 11
Outcome
Minutes
/Call
Unit Cost
Unit
Count
Total Project
Cost
Wave 1 Dials 0.75 $ 0.19 3,861
Contact Made –
Completed Survey
10 $ 2.50 116 $1,606.09
Contact Made –
Declined Survey
2 $ 0.50 1,165
Cost per Completed Survey: $13.85
Wave 2 Dials 0.75 $ 0.19 19,825
Contact Made –
Completed Survey
10 $ 2.50 264 $5,750.25
Contact Made –
Declined Survey
2 $ 0.50 2,647
Cost per Completed Survey: $21.78
12. MetrixMatrix.com (888) 865-0065 12
The overall cost per completed survey increased by 57.3% from Wave 1 ($13.85) to Wave 2
($21.78)
Calling effort increased in Wave 2 with more dials needed to complete a survey.
Meanwhile, the cost remained nearly equivalent for declined and completed surveys when
individuals picked up the phone.
Wave 1 Wave 2
Dollars
Cost per Completed Survey, Wave 1 and Wave 2
Dials Declined Surveys Completed Surveys
13. MetrixMatrix.com (888) 865-0065 13
Overall, the use of SmartID did not have the anticipated benefits on
completion rates and production costs.
Calls made with SmartID were more likely to generate a failed
contact.
Records that were re-called in Wave 2 using SmartID were more
likely to result in a contact.
When a contact was made, SmartID calls were no more likely to
result in a completed survey.
The overall cost per completed survey increased by 57.3% after
using the SmartID method (due to increased effort).
However, SmartID may help improve survey coverage…
14. 14MetrixMatrix.com (888) 865-0065
Jim Antonevich, President
Metrix Matrix
Chris Horn, Chief Information Officer
Metrix Matrix
Thank you to those who also contributed to this research:
Ann Rogers Lane, Director of Operations
Metrix Matrix
