Presentation by Rahija and Mtui at the 2016 AAPOR session on Monitoring data quality during fieldwork in international CAPI surveys. This was not presented at AAPOR although the speakers were part of the program.

1. Report on Tablet Assisted Personal Interview (TAPI)
Implementation by the Ministry of Livestock and
Fisheries Development (MLFD) in Tanzania
Michael Rahija, Research Officer, GSARS
Dr. Niwael Mtui, Principle Veterinarian, MLFD
1

2. Overview
• Background: Study and justification of using
TAPI
• Survey Implementation
• Monitoring reports and communication with
field staff
• Overview of costs
• Challenges encountered in the field
• Lessons Learned and Conclusion
2

3. Background
3

4. Background – Study Rationale
FAO, the Tanzania Ministry of Agriculture,
Livestock and Fisheries and the Tanzania National
Bureau of Statistics have been collaborating since
2011 to:
- Improve the livestock component of the
agricultural statistics system
- Facilitate the use of official statistics / statistics
methods for formulating / implementing
evidence-based policies
4

5. Background – Study Rationale
- Tanzania currently has one of the largest datasets
on livestock at household (farm) level throughout
Africa via the National Panel Survey)
- Ministry of Agriculture analysed the data and found
that many farmers relied on livestock, but did not
access extension services.
- Accordingly, a policy increasing access to
extension services would benefit farmers and
enhance food security.
5

6. Background – Study Rationale
MALF agreed to invest resources to improve the
system of livestock extension.
HOW?
6

7. Background – Study Rationale
MALF agreed to invest resources to improve the
system of livestock extension.
1. A survey of extension officers (change agents)
2. An experiment to test alternative options to
improve their on the job performance
7

8. Background – Study Rationale
1. A survey of extension officers (change agents)
2. An experiment to test alternative options to
improve their on the job performance (Jan 16)
TAPI
8

9. • Timelineness and cost
• Data Quality
• Capacity development
• Choice of Survey Solutions
Background – TAPI Rationale
9

10. Implementation
10

11. • Headquarters at MALF and PI from
London School of Economics (LSE)
–Survey design - Translation
–Survey management
• Enumerators and field staff
–Basic tablet operations
–Questionnaire training directly on tablet
–Use of quizzes
Implementation - training
11

12. • 5 teams of 3 enumerators and 1 supervisor
• Allocation of enumerators
– Based on quiz performance
– Tech savviness
• Assignment of Field Supervisors
– Based on understanding of questionnaire
– Tech savviness
• Headquarters – MALF, Global Strategy, LSE
Implementation – Teams
12

13. • Target population: Village Ward livestock
officers, and District Livestock Officers
• Goal: complete enumeration in Morogoro,
Iringa, and Dodoma
• Final sample: 63 DLOs and 415 VLOs
Implementation – Sample
13

14. • Questionnaire (~150 questions for VLOs):
– Personal info
– Livestock service provision and transport
– Farmer interaction
– Hierarchy
– Other income sources
– Personality measures
– Knowledge of rule, regulations, and policies
– Incentives
– The game
Implementation – Q
14

15. Implementation – Field work
Interview locations
15

16. Implementation – Field work
Progress by day
16

17. Implementation – Field work
Duration
17

18. Monitoring
& Communication
18

19. • Daily export of paradata
• Rmarkdown for automated tabulations
– Interviews by team and enumerator
– Duration of interviews
– Average interviews per enumerator
– Assignments
• Daily export of microdata and automated
checks
Monitoring
19

20. • After report generation, mgmt at the MALF
called Supervisors
• HQ called tablet of enumerators re:
specific interviews
• Comments inside of questionnaire used to
alert enumerators of various problems
Communication
20

21. • Information management:
– Email, 2 sim cards, IDs through CAPI software
Communication
internal.id devicename email interviewname enumerator AIRTEL card tigo
MinistryLF1 MinistryLF1 MinistryLF1@gmail.com lss_inter1 ### 0687333122 0675343970
MinistryLF2 MinistryLF2 MinistryLF2@gmail.com lss_inter2 ### 0687332727 0675343971
MinistryLF3 MinistryLF3 MinistryLF3@gmail.com lss_inter3 ### 0687327582 0675433985
21

22. Data analysis
• September 2015 data
cleaning (limited)
• October 2015: MALF
report available
• Nov 9th 2015: report
presented to the
Permanent Secretary
22

23. Overview
of Costs
23

24. Overview of costs of using TAPI
Item Units Total (TZS) Total (USD 17/9/15)
Samsung Galaxy 4 22 20,592,000 9,950.00
Battery chargers 22 50,000 512.00
Tablet covers 22 20,000 204.00
1 month data Tigo 22 25,000 256
1 month data Airtel 22 25,000 256
Server (Amazon – cloud) 0 0 0
Programming cost 0 0 0
Total - 23,232,000 10,820
24

25. Challenges encountered
in the field
25

26. • In some remote areas, it was difficult to
find a signal.
• Solution 1: Enumerators waited to
synchronize until a signal was found.
• Solution 2: Back-up paper questionnares
were provided.
Challenge - Connectivity
26

27. • Seemingly at random, SIM cards would
become de-configured and the tablets
wouldn’t synchronize.
• Solution 1: The second SIM card was
used.
• Solution 2: Field Supervisors were trained
and provided instructions for manual re-
configuration.
Challenge – De-configuration
27

28. • During the first week of data collection,
enumerators recommended some
improvements to questionnaire.
• Solution 1: HQ team made changes and
conducted trouble-shooting at night. Then
field supervisors were instructed to tell
enumerators to re-synchronize. Not
Recommended!
Challenge – Questionnaire
28

29. • The initial delivery of tablets was missing 6
tablets.
• Solution 1: Anyway, four tablets were
meant to be back-ups.
• Solution 2: Paper questionnaires provided
to some enumerators
• Solution 3: NBS loaned the project 2
tablets.
Challenge – Procurement
29

30. Lessons Learned,
Software recommendations,
and Conclusion
30

31. • Cost – justified in terms of data quality,
time b/t collection and analysis, and cost.
• Monitoring – Daily monitoring reports were
valuable to enhance team performance.
• Connectivity – Giving enumerators 2 SIM
cards minimized these problems.
• Training – Supervisors should be trained
on basic tablet maintenance.
Lessons Learned
31

32. • Training – One should train enumerators
on the questionnaire directly in the tablet.
• Training – Survey Solutions can be used
by people with limited or no programming
experience.
Lessons Learned
32

33. • Offline survey designer
• Feature to toggle between languages
• Automated generation of PAPI
questionnaire needs to be greatly
improved
• More documentation on enabling and
validation conditions.
Recommendations
33

34. • MALF consider implementation of CAPI to
be a success.
• Preliminary results were made available
and circulated within 1.5 months.
• The expected benefits of using Survey
Solutions were realized.
• Experience to be replicated / improved
through other FAO’s policy-assistance
initiatives
Conclusion
34

35. Thank You
• For more questions,
email:
• Michael Rahija 35