Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

6. Mathematica Presentation

2,061 views

Published on

Presentation for Pew Research Center's Methodological Workshop on Public Opinion Research in Sub-Saharan Africa

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

6. Mathematica Presentation

  1. 1. Using CAPI to assess interviewer skills during training: Presentation at the Pew Research Center Workshop on data collection in Sub-Saharan Africa, Washington, D.C. advice from Kenya, Ghana, Uganda and Senegal Sarah M. Hughes November 30, 2017
  2. 2. 22 “Collecting good data is one of the most powerful tools to end extreme poverty.” Jim Yong Kim 2015 source: World Bank, 2015
  3. 3. 33 Outline of the presentation • Why focus on interviewer training? • Introducing 3 training tools – Round robin – Mock script – Mock dataset • Assessing interviewers • Pros and cons of the method
  4. 4. 44 Outline of the presentation • Why focus on interviewer training? • Introducing 3 training tools – Round robin – Mock script – Mock dataset • Assessing interviewers • Pros and cons of the method
  5. 5. 55 Uneven statistical capacity • Limited training for statistical capacity • Use of paper questionnaires • Few private organizations with high-quality survey methodology capabilities • Pay and employment structure for field teams AfDB ranking statistical capacity for collecting agriculture data African Development Bank, 2014
  6. 6. 66 Outline of the presentation • Why focus on interviewer training? • Introducing 3 training tools – Round robin – Mock script – Mock dataset • Assessing interviewers • Pros and cons of the method
  7. 7. 77 Interviewer 1 Iboughtit Interviewer 2 Interviewer 3 Howdidyou obtainthisplot? Trainer / respondent Round Robin: interviewers have tablets
  8. 8. 88 Outline of the presentation • Why focus on interviewer training? • Introducing 3 training tools – Round robin – Mock script – Mock dataset • Assessing interviewers • Pros and cons of the method
  9. 9. 99 Using the mock script SIMPLE MOCK: "MA DIOP" TRAINER: INSTRUCT INTERVIEWERS TO SELECT RESPONDENT MA DIOP FROM THEIR TRAINING SAMPLE CASES Q # VARNAME QUESTION TEXT TRAINER'S VERBATIM RESPONSE EXPECTED INTERVIEWER ACTION STOP AND EXPLAIN: TRAINER NOTES B.10 HHNUMCF I would like to confirm that the total number of people living in your household is %textfill%. Well, there's me, my husband, our two kids and my mother in law. PROBE: So that would be how many? (or) So, that would be five? Respondents sometimes have a hard time adding up numbers and you might make a mistake recording the number. So, remember to probe and to repeat numbers Interviewer uses CAPI tablet to pose the question
  10. 10. 1010 Using the mock script SIMPLE MOCK: "MA DIOP" TRAINER: INSTRUCT INTERVIEWERS TO SELECT RESPONDENT MA DIOP FROM THEIR TRAINING SAMPLE CASES Q # VARNAME QUESTION TEXT TRAINER'S VERBATIM RESPONSE EXPECTED INTERVIEWER ACTION STOP AND EXPLAIN: TRAINER NOTES B.10 HHNUMCF I would like to confirm that the total number of people living in your household is %textfill%. Well, there's me, my husband, our two kids and my mother in law. PROBE: So that would be how many? (or) So, that would be five? Respondents sometimes have a hard time adding up numbers and you might make a mistake recording the number. So, remember to probe and to repeat numbers Trainer reads aloud exactly as written
  11. 11. 1111 Using the mock script SIMPLE MOCK: "MA DIOP" TRAINER: INSTRUCT INTERVIEWERS TO SELECT RESPONDENT MA DIOP FROM THEIR TRAINING SAMPLE CASES Q # VARNAME QUESTION TEXT TRAINER'S VERBATIM RESPONSE EXPECTED INTERVIEWER ACTION STOP AND EXPLAIN: TRAINER NOTES B.10 HHNUMCF I would like to confirm that the total number of people living in your household is %textfill%. Well, there's me, my husband, our two kids and my mother in law. PROBE: So that would be how many? (or) So, that would be five? Respondents sometimes have a hard time adding up numbers and you might make a mistake recording the number. So, remember to probe and to repeat numbers Trainers observe how interviewer probes
  12. 12. 1212 Using the mock script SIMPLE MOCK: "MA DIOP" TRAINER: INSTRUCT INTERVIEWERS TO SELECT RESPONDENT MA DIOP FROM THEIR TRAINING SAMPLE CASES Q # VARNAME QUESTION TEXT TRAINER'S VERBATIM RESPONSE EXPECTED INTERVIEWER ACTION STOP AND EXPLAIN: TRAINER NOTES B.10 HHNUMCF I would like to confirm that the total number of people living in your household is %textfill%. Well, there's me, my husband, our two kids and my mother in law. PROBE: So that would be how many? (or) So, that would be five? Respondents sometimes have a hard time adding up numbers and you might make a mistake recording the number. So, remember to probe and to repeat numbers. Trainer pauses and explains to improve interviewer behavior and understanding
  13. 13. 1313 Mock script: only for the trainer! SIMPLE MOCK: "MA DIOP" TRAINER: INSTRUCT INTERVIEWERS TO SELECT RESPONDENT MA DIOP FROM THEIR TRAINING SAMPLE CASES Q # VARNAME QUESTION TEXT TRAINER'S VERBATIM RESPONSE EXPECTED INTERVIEWER ACTION STOP AND EXPLAIN: TRAINER NOTES B.10 HHNUMCF I would like to confirm that the total number of people living in your household is %textfill%. Well, there's me, my husband, our two kids and my mother in law. PROBE: So that would be how many? (or) So, that would be five? Respondents sometimes have a hard time adding up numbers and you might make a mistake recording the number. So, remember to probe and to repeat numbers G.15 PLOTOWN How did you obtain this plot? 1-Purchase , 2- Inherit , 3-Rent for cash or kind , 4-Sharecrop , 5- Allocated free, 6-Other, specify I bought it Select 1-Purchase Be sure to read and know the response options so you select from the precoded responses. G.16 SEEDS$ How much did you spend on seeds for this plot? 5,000 CFA REPEAT RESPONSE, Enter 5,000 It is easy to make mistakes when entering numbers on a tablet-read the number you have entered out loud to the respondent to make sure you have entered it correctly L.1 PHONE1 What phone number should we use to contact you for follow up in 2-3 months? 77-328-44-91 REPEAT RESPONSE, enter 77-328-44-91 We check phone numbers for accuracy and follow up, so be sure this number is correct.
  14. 14. 1414 Interviewer 1 Iboughtit Interviewer 2 Interviewer 3 Howdidyou obtainthisplot? Trainer / respondent/ Ma Diop Round Robin: interviewers have tablets
  15. 15. 1515 Outline of the presentation • Why focus on interviewer training? • Introducing 3 training tools – Mock script – Round robin – Mock dataset • Assessing interviewers • Pros and cons of the method
  16. 16. 1616 Create excel version of data Filter columns
  17. 17. 1717 Identify errors Identify values that do not match the Mock script
  18. 18. 1818 Highlight for ease of presentation Highlight wrong values
  19. 19. 1919 Identify data quality errors by interviewer This interviewer has made multiple errors
  20. 20. 2020 Outline of the presentation • Why focus on interviewer training? • Introducing 3 training tools – Mock script – Round robin – Mock dataset • Assessing interviewers • Pros and cons of the method
  21. 21. 2121 Interviewer assessment • Expect errors during training! • Mock script with data review allows assessment of: – reading – listening – recording • Does NOT assess ability to select correct respondent or to gain cooperation • Does NOT replace supervisor observation or back-checks
  22. 22. 2222 Outline of the presentation • Why focus on interviewer training? • Introducing 3 training tools – Mock script – Round robin – Mock dataset • Assessing interviewers • Pros and cons of the method
  23. 23. 2323 Pros and cons of the method Pros Cons Identify trainees for additional training before hiring Time consuming for material development Trains on content, basic interviewer skills (active listening, probing, correctly entering numeric values), tablet/CAPI function Requires new approach to classroom training for local team Forces local team to become very familiar with questionnaire, programming Requires Training of Trainers Adds objectivity to interviewer assessment
  24. 24. 2424 Better training = better data “Data are the lifeblood of decision-making and the raw material for accountability. Without high-quality data providing the right information on the right things at the right time; designing, monitoring and evaluating effective policies becomes almost impossible.” United Nations, A World That Counts, 2014
  25. 25. 2525 For More Information • Sarah Hughes – SHughes@mathematica-mpr.com

×