Presentation for Pew Research Center's Methodological Workshop on Public Opinion Research in Sub-Saharan Africa

1. November 2017
USING CAPI TO ASSESS AND
IMPROVE DATA QUALITY

2. OVERVIEW
2
1
2
4
3
5
Introduction
Quality control overview
Process oriented checks
Audio checks
Geolocation checks

3. INTRODUCTION
3

4. ORB International is a small business
with a US and a UK office
We have 20+ researchers who manage
social and political research around the
world, with a particular focus on Africa.
4
INTRODUCTION

5. A few years ago we started running all
projects on tablets, using CAPI
technology, which enabled greater
quality control enforcement
Our Quality Control system is not static –
it improves with every project
CAPI with SurveyToGo enables us to
use a series of ‘flags’ along with the
following variables to ensure quality:
• Interviewer ID
• Timer and Time Stamp features
• Contact data
5

6. QUALITY CONTROL
OVERVIEW
6

7. Audio checks Geolocation checks Process oriented
checks
7
QUALITY CONTROL IS COMPRISED OF MULTIPLE CHECKS

8. 8
Eliminate fraud Validate sampling
processes
Improve interviewing
quality
THESE CHECKS ENABLE US TO

9. AUDIO QUALITY CONTROL
12
VALIDATION FEEDBACK

10. Clearly-stated question
Audible response from interviewee
Tone and delivery of interviewer and
respondent are reflective of a valid interaction
Ambient noise is unique from other interviews
Ambient noise is commensurate with daytime
sounds that would be expected in the
interviewing environment
ACCEPTABLE AUDIO
RECORDING

11. Interviewer takes liberties with question
verbiage
No response from interviewee and no time for
a response
Suspicious tone and delivery of interviewer /
respondent
Interviewer moves to next question even if
respondent does not give a coherent
response
SUSPICIOUS AUDIO
RECORDING

12. 16
Are the pings in/ within 3 miles
of the correct sampling unit?
Does the geolocation data
indicate a valid interview?
Interview data passed into a GPS QC web
app daily for visualization and verification
GEOLOCATION QC

13. 17
”
Good
“

14. 18
”
Within a half mile of the border
of Township SL and Garissa
town.
“

15. Short time between
interviews
Recording captured
at odd time of day
Interviewer conducts multiple
interviews at the same time
20
ISSUES THAT MUST BE FOLLOWED UP IN COMBINED AUDIO AND GPS QC
AUDIO, GPS – FIRST LINE OF INQUIRY FOR FRAUD, FOLLOW UP