This document discusses the importance of pilot testing questions for a census. It outlines key reasons to conduct pilot testing, such as ensuring respondent understanding, question clarity, and effectiveness of operational procedures. Pilot testing allows identifying needed changes to questions, instructions or processes. It also confirms that the questions will provide the intended data. The document stresses that an effective pilot test requires an adequate sample size, both urban and rural, as well as sufficient resources for development, training, data collection and analysis. Input from participants is requested to determine what advice or areas of difficulty they foresee in pilot testing.

1. Pilot testing
Why and how
Ken Black
Regional workshop on promoting disability data collection through the 2010 population and housing censuses – Bangkok, Thailand 8-10 April 2008
Key reasons to pilot test
• Questions able to be administered in a ‘real
world environment’ by interviewers or self-
reported, depending on method used, and
can then assess:
– Respondent understanding of question meaning
– Appropriateness of response categories
– Question clarity
Regional workshop on promoting disability data collection through the 2010 population and housing censuses – Bangkok, Thailand 8-10 April 2008
1

2. Key reasons to pilot test (cont.)
• Correct sequencing of respondents if needed
• Adequacy of training procedures
• Adequacy of instructions
• Effectiveness of data capture
• Edits – field (if electronic collection) and
post field
Regional workshop on promoting disability data collection through the 2010 population and housing censuses – Bangkok, Thailand 8-10 April 2008
Key reasons to pilot test (cont.)
• Interviewer field notes of problems/issues
arising
• Interviewer debriefing opportunities –
preferably group face-to-face
• Form receipt and follow-up procedures
• Processing instructions and procedures
• Mock-ups of data output for user liaison
Regional workshop on promoting disability data collection through the 2010 population and housing censuses – Bangkok, Thailand 8-10 April 2008
2

3. Desired results of pilot testing
• Need for change identified in question
design, wording or instructions
• Effectiveness of operational procedures
tested
• Do the questions asked do what was
intended, and are they likely to provide the
output data required?
Regional workshop on promoting disability data collection through the 2010 population and housing censuses – Bangkok, Thailand 8-10 April 2008
Desired results of pilot testing
(cont.)
In the end – a better standard product for the
final enumeration, and assurance of more
meaningful and accurate data
More likely to meet data users needs!
Regional workshop on promoting disability data collection through the 2010 population and housing censuses – Bangkok, Thailand 8-10 April 2008
3

4. Essential elements
• Adequate sample size selected to test all
facets of the disability questions and
processes against a broad spectrum of the
population, including coverage of people
with a disability – both urban and rural
minimum 1,000 households
Regional workshop on promoting disability data collection through the 2010 population and housing censuses – Bangkok, Thailand 8-10 April 2008
Essential elements (cont.)
Adequate resources available for:
– development
– training
– enumeration
– data capture
– editing and processing
– analysis
– final report
Regional workshop on promoting disability data collection through the 2010 population and housing censuses – Bangkok, Thailand 8-10 April 2008
4

5. Your needs
Each of you will have different strengths
and weaknesses – I need your input here to
identify what advice you need, and where
you see the most difficulty in conducting an
effective pilot test as outlined
Regional workshop on promoting disability data collection through the 2010 population and housing censuses – Bangkok, Thailand 8-10 April 2008
5