2. How to do (or not to do) . . . 323
section focuses on the generalizability of our approach to Pre-testing the survey questions
testing questions for surveys and introduces the reader to a
Pre-testing is the main chance for researchers to gauge the
range of literature for further reading.
meaning attributed to survey questions ‘before it is too late’,
i.e. before a substantial investment is made in the wrong
Context of research questions or in questions where the researcher cannot be
sure about what is being asked. It is therefore imperative that
The overall aim of our research is to develop a culturally
the researchers undertake these interviews, rather than
relevant generic measure of ‘health’ to measure the impact of
assigning the task to field assistants, because they know the
interventions designed to reduce disease and/or improve
aim of the research. The pre-testing involved a number of
health in Kenya. We have been working amongst the Kamba
steps including:
of Makueni District in Eastern Kenya and the Maragoli of
Vihiga District in Western Kenya. The conceptualization and
• establishing the intended referential and connotative
operationalization of ‘health’ emerged from extensive
meaning of each question;
periods of participant observation by anthropologists, in
• agreeing a set of criteria to judge the appropriateness of
addition to qualitative research by others (see Fox-Rushby et
survey questions;
al. 1997; Fox-Rushby, for the KENQOL Group, 2000). The
• selecting the methods for judging appropriateness and
definition of health adopted focused on the subjective per-
undertaking research;
ceptions of the positive and negative aspects of health that
• reviewing questions for inclusion, revising (the question or
comprised ‘contentment’, ‘cleanliness’, corporeal capacity’,
intended meaning) or exclusion.
‘co-operation’ and ‘completeness’ of an intending individual
within the intentional worlds of Makueni and Vihiga (Fox-
Rushby, for the KENQOL Group, 2000). Step 1: Establishing the intended referential and
connotative meaning of each question
The KENQOL survey questionnaire was developed to
It is important to ascertain whether respondents are inter-
access those aspects of health that a member of the com-
preting the question as intended to ensure appropriate
munity could respond to in public4 to an interviewer who was
interpretations are placed on the data. Therefore we pre-
not necessarily known to them personally. This paper focuses
pared descriptions of the intended referential and connota-
on our work amongst the Kamba and reflects the develop-
tive meaning for each of the survey questions drawn up.
ment of the household survey in the Kamba language.5 The
There were several advantages to this: it served as an account-
aim of this part of our research was to investigate: how
able approach to assessing content validity; it facilitated
respondents understood the KENQOL survey questions;
clarification of our own intentions in asking questions; it
the types of information retrieved and over what time
ensured we assessed the same hypotheses as there was less
period; the types of socially acceptable answers that might
variation in our own interpretation of the meaning of the
be given and why; in addition to evaluating whether they
original questions (either over time or between researchers);
performed the task the way the researcher expected. We
and it was expected to be helpful in the future assessment of
tested a total of 150 potential health measurement survey
the conceptual equivalence of the Kamba language version
questions.
with other language versions (see Herdman et al. 1997, 1998).
Box 1 provides an example of the type of description we
The field site was located in Makueni District in Eastern
attached to a question.
Kenya. It is approximately halfway between Nairobi and
Mombasa, just north of the main trunk road. The Kamba
population (part of the Central Bantu group) is the majority
ethnic group in this region (99%) and traditionally they are
pastoralists. Currently 50% of the population are aged
0–14 years but infant mortality rates are high, at 96 per 1000
live births in 1995 (Kenya Government 2000). Rainfall is Box 1. Sample meaning for a KENQOL survey question
scarce in the area, with an annual precipitation of less than
500 mm and this, combined with high daytime temperatures,
leads to a hot, dry environment. Agriculture is unreliable and Do you feel that there are some people who could sabo-
food deficits are often experienced. Health facilities are rela- tage the progress of people in your neighbourhood at the
tively scarce, although they can be reached by matatu from moment?
the main Nairobi/Mombasa highway; the walk to the road can
take up to 2 hours. We want to find out an individual’s current views about
whether they feel that their ability to improve them-
A day’s bus journey is required to reach hospital facilities in selves and their lives (collectively as a community)
Nairobi. The major diseases are malaria, respiratory tract may be adversely affected by people within the com-
infections, diarrhoeal diseases, skin ulcers, intestinal worms munity in some way. It is possible this may include a
and urinary tract infections (Kenya Government 2000). range of ideas such as violence and witchcraft and we
Incomes are low compared with many other areas in Kenya, leave it up to the respondent to decide if they are being
with many people living below the poverty line and involved threatened.
in a daily struggle for survival and development.
3. 324 How to do (or not to do) . . .
Table 1. Criteria selected for judging appropriateness of survey questions and investigative method employed
1 Criteria 2 Method
Expanded Targeted Group Expert
interviews methods methods reviews
1 No negative questions, and therefore no double negative answers √ √
2 There are not two questions in one √ √
3 Level of language is not too high/old fashioned/unusual in any sense √ √
4 The question is simple and grammatically correct √
5 The question is free from jargon √
6 Singular and plural ‘you’ is clear √ √ √
7 The same idea is not contained in another question √ √
8 The question is sensitive to measure change over time √ √
9 Captures current views √ √
10 Reflects local issues √ √
11 Meaning and interpretation of question is clear √ √ √
12 The question makes sense to everyone √ √
13 A ‘yes’ is clear and unambiguous in meaning √ √
14 A ‘no’ is clear and unambiguous in meaning √ √
15 Time period is clear √ √ √
16 Conception(s) of health it represents is specified √ √
17 How the question relates to: time of year; head of household; individual/
homestead/wider family/community √ √ √
18 Understanding of how it relates to severity of HRQL √ √
Step 2: Criteria to judge the appropriateness of survey organize the expanded interviews we divided the set of survey
questions questions into groups of between three and five questions. In
each interview6 the respondent was first asked the KENQOL
Having developed the survey question and agreed on the
survey question and asked to respond either ‘yes’, ‘no’ or
intended meaning, we developed a series of criteria for evalu-
‘don’t know’ as they chose. The probes used were divided into
ating the appropriateness of the survey questions. This facili-
three types and used in separate interviews:
tated an agreed and standardized way of making decisions
about which questions to include and enabled different
(1) Comprehension of ‘key phrases’ within the question. We
researchers to bring their own judgements together in a
asked respondents to talk about the kinds of things they
comparable way. Column 1 of Table 1 outlines the range of
thought about when answering the question. Examples of
criteria we adopted. It can be seen that the issues focus largely
key phrases included the time frame the respondent
on the style of language, attributed meaning, clarity of the
considered and who respondents included when the
question and consistency of interpretations.
word ‘friends’ or ‘neighbours’ was used (see Box 2 for an
example).
(2) Applicability of the questions: to people of different ages,
Step 3: Methods for judging appropriateness of survey
gender and socioeconomic groups (see Box 3 for an
questions
There are a wide variety of techniques available to
researchers wanting to test survey questions. We adopted
four principal types (described in Forsyth and Lessler 1991): Box 2. Comprehension of key phrases in the question
expanded interview techniques, targeted methods, group
methods and expert evaluation. Others approaches are
addressed in the discussion. Table 1 (column 2) indicates Do you feel safe at the moment?
which technique(s) addresses which type of issue and this
section outlines each method. Of these techniques, the The question intended to access concerns about per-
expanded interviews took most of our time. sonal safety and safety of belongings. The probe ques-
tions asked respondents to give examples for why
someone (a) will feel safe and (b) may not feel safe. It
Expanded interviews
emerged that the question not only addressed notions
Expanded interviews begin by one or more of the intended of personal safety and safety of belongings, but also
survey questions being asked and answered by the respon- having enough food, being properly taken care of and
dent, followed by a set of probe questions to investigate the having money for school fees.
thought processes involved in answering the question(s). To
4. How to do (or not to do) . . . 325
Box 3. Applicability of questions for respondents of different ages
We asked respondents who in the family can and cannot be asked the question, ‘At the moment are you always worry-
ing about getting food?’ The text below demonstrates some of our findings with respect to this question.
Interview with an elderly man:
Interviewer: Can children worry about getting food?
Respondent: No, since it is their father’s fault they are fed.
Interviewer: Could a young unmarried man answer this question?
Respondent: A young unmarried man isn’t responsible for that. You will find that in most cases he comes at night and
gets in his room and sleeps. Maybe he has passed to his friends’ home and had something to eat. The old mamas can’t
worry since they are just looked after by their children they brought up.
Interviewer: Is there anyone who might say no to this question?
Respondent: Old and young people.
example); by level of sensitivity of the question; and you ‘clean’? – meaning in a hygienic and moral sense),
social desirability of responses (see Box 4). but further probing highlights some potential problems.
Box 4 has a particularly interesting response because it (3) The meaning of a ‘yes’ or ‘no’ response needed to be clear
also shows how sensitive the questioning on sensitivity and as intended (see Box 5).
needs to be. It might at first appear that there are no prob-
lems with the question ‘Ivindani yii nutonya kwiyikalya The respondents for the expanded interviews were volun-
wi mutheu undu utonya kwenda?’ (At the moment, are teers willing to expend more time than the data collection
Box 4. Addressing the sensitivity of the question
We asked respondents directly about the sensitivity of questions. We needed to know whether a question was so sensi-
tive that people might purposefully provide the wrong answer. Alternatively they may provide the right answer, but this
makes the respondent or their family uncomfortable in some way. The transcript below is a translation from an inter-
view that addressed sensitivity.
Interviewer: At the moment, are you ‘clean’ [hygienically and morally]?
Respondent: Yes
Interviewer: When we ask this question in the interview, we want to be sure that we are not embarrassing the person we
are asking. Do you think this question might cause embarrassment if it is asked in an interview?
Respondent: The question is good and cannot cause embarrassment in an interview.
Interviewer: Do you think people will feel they can answer this question truthfully in an interview?
Respondent: Not all people can answer truthfully. This is because if one feels unclean or has done wrong, he wouldn’t
want to be known to have wronged.
Interviewer: If someone answers ‘no’ to this question, how will it reflect on the family of that person?
Respondent: One who answers no would be thought to be unclean, but not the whole family.
5. 326 How to do (or not to do) . . .
Box 5. Testing the meaning of a ‘yes’ or ‘no’ response
By undertaking a relatively simple interview we could quickly establish whether a ‘yes’ or ‘no’ response could be inter-
preted clearly and as intended. This particular question was dropped because it became clear that we could not be sure
that a ‘no’ meant worse ‘health’.
Interviewer: Do you get enough information about current events in Kenya?
Respondent: No.
Interviewer: If someone answers ‘no’ to this question, is he/she in a better or worse position than someone who answers
‘yes?
Respondent: The one who answers ‘yes’ is in a better position.
Interviewer: Why is this?
Respondent: Information about current events in Kenya can assist people in this area by helping them develop, such as
knowing different methods of farming, knowing about disease and outbreaks and ways of combating them, and also
improvement of educational facilities.
Interviewer: What other reasons are there that someone could answer ‘yes’?
Respondent: Those who own radios and are literate. People can get access to this information from radios, newspapers,
meetings and friends.
Interviewer: What other reasons are there that someone could answer ‘no’?
Respondent: Those who don’t have radios and don’t read newspapers. Everyone doesn’t want to have access to infor-
mation about Kenya. Those who do are mostly interested in politics and current affairs. Those who aren’t are the old,
illiterate and those not interested in politics.
itself in helping us learn about the questions. They were poor; and those with harmonious relationships compared
selected purposively to include equal proportions of males with those in disputes.
and females, as well as being roughly representative of the • As the questionnaire was intended to be able to measure
different generations in the community. In total each ques- change we asked eight people for one of two types of hypo-
tion was the focus of between four and eight interviews with thetical considerations of the questionnaire. For example,
a range of people from the local population. members of a youth-based water and agriculture project
were asked to imagine how they might have answered the
questions before and after the project, and other key
Targeted methods
informants were asked how they might have completed
Targeted methods focus on a particular component of the the questionnaire during and after malaria. The task was
response process, to gain a detailed insight into the respon- followed up with a discussion of the answers most likely to
dent’s comprehension and preparation of a response and change and why.
to evaluate whether they performed the task the way the • To address which conception of health people felt the
researcher intended. We used three types of targeted question represented, as well as helping further investi-
methods: gations into the meaning and interpretation of the ques-
tion, respondents were asked to sort the Kikamba
• People with known (and opposite) characteristics of inter- questions into groups, either under the Kikamba words
est were asked to complete specific questions to assess how used to translate the English word ‘health’7 that each ques-
likely it was that the questionnaire would be interpreted in tion was most closely related to, or to devise their own
the same way by people with particular characteristics. This categories and group them. Each question was placed on a
was important in being able to interpret any future differ- separate card for ease of sorting. The task was followed up
ences/similarities across groups. Therefore we contacted with an interview about the meanings of the Kikamba
(through the help of our field assistants): people with and words used to translate the English word ‘health’ and
without sickness/illness; the relatively wealthy and very reasons for groupings.
6. How to do (or not to do) . . . 327
Box 6. Group discussion on ‘people in your homestead’
Many of the KENQOL questions ask respondents about people in their homestead,i for example,
• Do people in your homestead listen to your advice/suggestions at the moment?
• When you are sick, do people in your homestead help you in getting treatment?
• At the moment is it easy for people in your homestead to find transport to go for treatment when they are sick?
To check who people thought about when answering these questions, we provided several groups of respondents with
an example family (see below) and initiated a discussion about who different people in that family would think about
when they were asked specific questions.
Example family: Headed by a married male (mzee) and female (mama), who had three sons. The first son is married
to one wife with a child and they live just outside the compound. The second son is married with two wives (polygy-
nous marriage) and both wives have children. The third son lives away in Nairobi.
The discussion revealed that people would make reference to different people in their response. In the example above,
the mzee would consider seven adults, whereas the first-born son thinks of his wife and his wife thinks only of her
husband. To complicate matters further still, we discovered that a person could alter the group of people they thought
about depending on the question that was being asked, even if each question used identical phrases for ‘people in your
homestead’. We noted all the instances in which this occurred.
i The phrase musyi kwaku/kwenyu for homestead was used after finding that the word family was interpreted very
differently across people.
Group methods questions and issues 1–6 in Table 1. The local field assistants
also reviewed the written Kikamba questions and gave their
Group methods also focus on a particular component of the
views on issues 1–6 as well as 13–17. They were very helpful
response process, but a group of people are addressed, rather
in their criticisms and suggestions, and once we had been able
than an individual, to encourage greater discussion around
to take down their advice, it was also easier for them to inter-
the issues involved. There were a number of key issues within
view others (as opposed to discussing their own ideas with
the survey questions that were repeated often, such as time
interviewees).
frames, the use of singular or plural ‘you’ and terms such as
‘neighbours’, ‘friends’ and ‘family’. Individual interviews
were very helpful in understanding interpretations but did Step 4: Reviewing questions for inclusion, revising the
not give us the opportunity to see opposing suggestions question or intended meaning, or dropping questions
debated. We therefore held group discussions on focused
The field transcripts and results from each research method
issues. The group discussions lasted 1–2 hours, and were led
were collated and summarized by the authors for each survey
by two moderators. Each interview was recorded and later
question. Reviews of questions were supported by a dis-
transcribed. Each group involved people with similar ages
cussion between the authors and field assistants. Following
and gender, and the sum of groups captured the range of ages
these discussions, decisions were made concerning the action
and gender in the area. Box 6 gives an example of our
required for each question including:
approach to finding out how people interpreted the word
‘homestead’.
• acceptance of the original question and meaning;
• acceptance of the original question, with a change to the
meaning;
Expert evaluation
• changing the question (slightly) but keeping the meaning
Expert evaluation involved no interaction with potential the same;
survey respondents, but rather those who could offer particu- • dropping the question;
lar guidance on specific issues. The ‘experts’ included two • writing a new question (and new intended meaning).
groups of people: external advisors including Kikamba Bible
translators and Kamba university lecturers on sociology; and Any changes were tested through additional expanded
the field assistants living locally in Makueni who had assisted interviews and any new questions also went through various
the KENQOL research group for several years. The external translation checks. Prior to accepting any new or changed
language advisors reviewed the written survey questions and questions all data on the question were once again collated
focused particularly on their interpretation of the meaning of and reviewed prior to being taken forward to piloting.
7. 328 How to do (or not to do) . . .
Piloting into other languages. There has been some use of qualitative
techniques to assess the content validity of original (e.g.
Moser and Kalton (1992) refer to piloting as the ‘dress
Saxena et al. 1998), but more usually the adapted versions of,
rehearsal’. Questions are placed together as it is expected that
generic HRQL instruments (e.g. Chwalow et al. 1992; Lam et
they will appear on the final questionnaire and the dynamics of
al. 1994), as well as the applicability of generic instruments to
the survey as a whole are investigated. The KENQOL group
specific disease groups (e.g. Tazaki et al. 1998). These investi-
undertook piloting in two stages. The first focused on ques-
gations tend to be relatively unstructured, and can be
tions from each sub-section (n = 6) of the survey. The second
described as cursory investigations. In general there is
involved straight runs through the entire survey. One of the
too much emphasis placed on quantitative assessments of
authors was present at each of these interviews, and a field
reliability, validity and sensitivity. The quantitative assess-
assistant conducted the interview in Kikamba. At both stages
ment is a rather closed approach, that ignores the respon-
of piloting, the interviewer noted down the response to each
dent’s own interpretation of questions.
question (yes, no, don’t know, no response). They also noted
any other information the respondent gave, such as asking for
We therefore adopted a qualitative approach to the assess-
clarification of the question, or the respondent providing
ment of validity, reliability and sensitivity. This made both the
additional information after responding ‘yes’, ‘no’ or ‘don’t
researcher’s and the respondent’s interpretation of questions
know’. In addition, the authors noted any body language,
more open and accountable, and ensured that the final survey
pauses before a response, emotional reactions to questions etc.
questions were more reliable and valid and able to detect
At the end of the pilot survey interviews, each respondent was
change as revisions were made. Earlier we defined a reliable
asked for their opinion about the questions, including:
measure as one which yields the same results in repeated
trials (Carmine and Zeller 1979). A question may be un-
• what they thought about the questions in general;
reliable due to bad wording, causing a person to understand
• whether any of the questions seemed to be strange or
the question differently on different occasions (De Vaus
unusual;
1995). This paper has described procedures for assessing the
• their opinion on the order of questions;
reliability of questions during the pre-testing stage, for
• how appropriate the response categories were;
example identifying questions that cannot elicit a double
• whether the correct procedures are used to address:
negative for a response, do not contain high level or old fash-
(a) married and unmarried people,
ioned language, and are free from jargon. A valid measure is
(b) old and young people;
one which measures what is intended (Carmine and Zeller
• whether any questions should not be asked in the survey;
1979), and methods described in this paper which helped to
• to point out any questions they do not want to answer, or
address this included ensuring that the questions capture
think they should not be asked;
current views, reflect local issues and the conception of health
• any problems they thought we might encounter asking
each question represents is clear. Similarly, during pre-testing
people the questions:
the sensitivity to questions over time was investigated. De
(a) alone,
Vaus (1995) explains that instead of this procedure some
(b) with others listening (e.g. young people were asked
people include a large number of indicators and at a later
whether they might have inhibitions answering ques-
stage eliminate those that do not seem to be working.
tions in front of parents);
However, he explains: ‘this seems to be the wrong way of
• whether any questions appeared to be asking the same
doing things since we will end up by defining the concept in
thing;
terms of the indicators which ‘worked’. If this is done then the
• their opinion on what changes we could make to improve
indicators may not represent the concepts of the theory we
this survey in terms of:
set out to test and as such the research can end up having
(a) introduction,
little relevance to the original research question’ (De Vaus
(b) questions;
1995: 54).
• which questions might be difficult for someone to talk
about in front of an interviewer who comes from the same
However, we recognize that the process was labour intensive
area as the respondent;
and took a considerable amount of time (around 6 months of
• whether it matters who interviews people;
researcher and 12 months of field assistant’s time). In our case
• who would be a good interviewer in terms of age, gender
we judged this to be an important part of the process and
and any other characteristics.
outcome of our research. First, we are intending that this
measure becomes used across many different diseases as
Based on feedback during piloting, several additional
an outcome measure used to judge the effectiveness and
revisions were made to the KENQOL questions. Each
efficiency of health-improving interventions. It is therefore
change was followed up by further piloting (and pre-testing
important to us that the people whose health we are trying to
where necessary) until the final version of the KENQOL
measure (and improve) are integrated into this process, and
survey was agreed.
it is often the case that generic HRQL instruments are devel-
oped and used for years following their development. Sec-
ondly, it provided valuable comparative data for explaining
Discussion
any similarities and dissonances in measured health in Vihiga
We have been concerned about the procedures used in the district, Kenya. Related to this is the possibility that others
development of HRQL measures as well as their translations would want to translate the instrument for use elsewhere, and
8. How to do (or not to do) . . . 329
we wanted to improve the basis from which judgements are required information through household survey, readers
made about the equivalence of future translations. should consider some of the literature from demographic
studies; some examples include Caldwell and Igun (1997),
There are other techniques that are available to researchers Ewbank (1981), Bledsoe and Pison (1994). We found cogni-
to test their survey questions that we did not use. The tive research assessment a particularly inspiring approach as
KENQOL researchers adopted some cognitive assessment it captures approaches given to a variety of psychological
survey methodologies, but there are others that may be more processes involved in information retrieval. Some of this
appropriate for other research groups (Jobe and Mingay literature has already been cited above, but readers might
1989): also like to consider Lessler and Tourangeu (1989) and the
proceedings from a seminar on the Cognitive Aspects of
• Concurrent think-aloud interviews: respondents think Survey Methodology (Sirken et al. 1999). Researchers in the
aloud when answering questions, and responses are field of marketing have also been considering these issues for
probed. some time, consider for example Gendall and Hoek (1990)
• Retrospective think-aloud interviews: respondents answer and Kumar et al. (1999).
all question first and then are asked how they arrived at
their answers.
• Confidence ratings: respondents relate the degree of confi- Conclusions/recommendations
dence they have in the accuracy of their answers. The investigations described in this paper should not be seen
• Paraphrasing: respondents repeat the questions in their as an unusual step in the development and testing of survey
own words. questionnaires – particularly those that will be used repeat-
• Response latency: measurement of the elapsed time edly. The validity and reliability (and possibly sensitivity) of
between the questions and respondent’s answers. any survey is important to establish prior to accepting results
and conclusions. Each of these ideas can begin to be explored,
The KENQOL group also adopted some methods recognized and improved, through the use of qualitative methods in pre-
as participatory or rapid rural appraisal (PRA and RRA) testing and piloting questionnaires. Indeed, evidence that a
approaches, for example the use of anthropologists and local survey tool has ‘content validity’ is required, rather than
leaders as key informants (Chambers 1983). Other useful simply relying on unsubstantiated assertions by researchers,
RRA and PRA approaches might include a well-being (or and only once this has been demonstrated is it appropriate to
HRQL) ranking exercise of local households with local adopt quantitative assessment methods for further analysis.
participants, and linking this to likely responses to questions Finally, the knowledge gained about the interpretation of
and analysis of difference based on responses to questions survey questions will also help in judging the equivalence
(see Chambers 1997). of any future translations of the survey questionnaire.
Whilst the approach we have taken to the qualitative assess-
ment of meaning attached to the KENQOL survey questions Endnotes
is unique amongst generic HRQL research, many of the ideas 1 Validity determines that an instrument is measuring what was
were drawn from a variety of sources, including survey intended. It concerns the crucial relationship between concept and
research (e.g. Moser and Kalton 1992; Fowler 1993), trans- indicator. The objective is to ensure that the indicator represents the
lation studies (e.g. Larson 1984; Barnwell 1992), the cognitive intended (and only the intended) concept (Carmines and Zeller
sciences (e.g. Jobe 1990; Forsyth and Lessler 1991) as well as 1979).
2 Reliability concerns the extent to which an experiment, test
the health measurement literature (e.g. Guillemin et al. 1993; and any measuring procedure yields the same results on repeated
Herdman et al. 1997, 1998). De Vaus (1995) is particularly trials (Carmines and Zeller 1979).
useful because of the links made between investigating ques- 3 The ability of a measure to detect change.
tionnaire interpretation and the reliability and validity of a 4 Because most household interviews happen outside the home
questionnaire. This gives an indication of the applicability of with many people around. To ask to interview people privately in a
our approach to the design of survey questions more gener- household survey tends to arouse a lot of suspicion.
5 Our household survey eventually consisted of 112 health
ally. As Tourangeau (1984) states ‘[in] the respondent’s task
measurement questions and 50 socioeconomic and demographic
. . . there is considerable room for error. Respondents may questions. We achieved a total of 550 interviews – a response rate of
misunderstand the question or the response categories; they 81.3% and a mean item response rate of 98.1% (SD 3.6).
may forget or misremember the crucial information; they may 6 All interviews were undertaken in Kikamba, the local lan-
misjudge the information they do recall; and they may mis- guage of the Kamba population we were working amongst. The
report their answer’ (Tourangeau 1984: 73–74). Kikamba versions of the KENQOL questions were used for all
survey testing, although the English versions are reported here. The
pre-test interviews were written up in full in English to facilitate
Finally, to help researchers give greater attention to investi- participation by the non-Kikamba speaking researchers.
gating whether respondents interpret questions in ways 7 wailu, wianie, utheu, useo, wiwa nesa, uima.
researchers expect (or hope), we give a brief introduction to
a variety of different types of source texts. Most social
research texts address questionnaire design, but few provide References
detail on investigations into question meanings. One text that Barnwell K. 1992. Bible Translation: An introductory course in trans-
is useful in this context is Oppenheim (2000). For an excellent lation principles (3rd edition). Dallas, TX: Summer Institute of
insight into the many pitfalls associated with eliciting the Linguistics International.
9. 330 How to do (or not to do) . . .
Bledsoe C, Pison G. 1994. Introduction. In: Bledsoe C, Pison G (eds). Lessler JT, Tourangeau R. 1989. Vital health statistics. Series 6.1.
Nuptiality in Sub-Saharan Africa: Contemporary anthropo- Washington, DC: National Center for Health Statistics.
logical and demographic perspectives. Oxford: Clarendon Press, Moser CA, Kalton G. 1992. Survey methods in social investigation
pp. 1–22. (2nd edition). Aldershot, UK: Gower.
Bowden A, Fox-Rushby J. 2000. A critical review of the adaptation Oppenheim AN. 2000. Questionnaire design, interviewing and
and use of generic HRQL measures amongst populations attitude measurement. Buckingham, UK: Open University
outside of North America, western/southern/northern Europe, Press.
Australia and New Zealand. Quality of Life Research 9: 313. Saxena S, Chandiramani K, Bhargava R. 1998. WHOQOL-Hindi: A
Caldwell JC, Igun AA. 1971. An experiment with census-type age questionnaire for assessing the quality of life in health care set-
enumeration in Nigeria. Population Studies 25: 287–302. tings in India. National Medical Journal of India 11: 160–5.
Carmines EG, Zeller RA. 1979. Reliability and validity assessment. Sirken M, Jabine T, Willis G, Martin E, Tucker C. 1999. A New
London: Sage Publications. Agenda for Interdisciplinary Survey Research Methods: Pro-
Chambers R. 1983. Rural development: putting the last first. Harlow, ceedings of the CASM II Seminar. Hyattsville, MD: National
UK: Longman. Center for Health Statistics. [http://www.cdc.gov/nchs/data/
Chambers R. 1997. Whose reality counts? Putting the first last. casm2pro.pdf, 2nd August 2001]
London: Intermediate Technology. Tazaki M, Nakane Y, Endo T et al. 1998. Results of a qualitative and
Chwalow AJ, Lurie A, Bean K et al. 1992. A French version of the field study using the WHOQOL instrument for cancer patients.
Sickness Impact Profile (SIP): stages in the cross-cultural vali- Japanese Journal of Clinical Oncology 28: 134–41.
dation of a generic quality of life scale. Fundamental Clinical Tourangeau R. 1984. Cognitive sciences and survey methods. In:
Pharmacology 6: 319–26. Jabine TB (ed). Cognitive aspects of survey methodology: build-
De Vaus. 1995. Surveys in social research (4th edition). London: ing a bridge between disciplines. Washington, DC: National
Routledge. Academy Press, pp. 73–100.
Ewbank DC. 1981. Age misreporting and age-selective undernumer-
ation: sources, patterns, and consequences for demographic
analysis. Committee on Population and Demography. Report Acknowledgements
No. 4. Washington, DC: National Academy Press.
Forsyth BH, Lessler JT. 1991. Cognitive laboratory methods: a tax- We thank the following organizations for the funding of this survey:
onomy. In: Biemer PP, Groves SM, Lyberg LE, Mathiowetz UNDP/World Bank/WHO Special Programme for Research and
NA, Sudan S (eds). Measurement errors in surveys. New York: Training in Tropical Disease, the UK Economic and Social Research
John Wiley, pp. 393–418. Council (Ref. H52427005994) and the Health Economics and
Fowler FJ. 1993. Survey research methods (2nd edition). London: Financing Programme (funded by the Department for International
Sage Publications. Development). We also thank all our field assistants, especially
Fox-Rushby J, Parker M. 1995. Culture and the measurement of Justus Mutunga, Jeremiah Nyamai and Alex Muvea, for their help,
health related quality of life. European Review of Applied Psy- guidance and enthusiasm. Finally we thank all the people who
chology 45: 257–63. participated in the preparation for this survey and made us welcome
Fox-Rushby J, Johnson K, Mwanzo I et al. 1997. Creating an instru- in their homes.
ment to assess lay perceptions of HRQL: options and impli-
cations. Quality of Life Research 6: 633.
Fox-Rushby J, for the KENQOL Group. 2000. Operationalising con- Biographies
ceptions of ‘health’ amongst the Wakamba and Maragoli of Annabel Bowden, Ph.D. (Health Geography), was a Research Fellow
Kenya: the basis of the KENQOL instrument. Quality of Life at the London School of Hygiene and Tropical Medicine (LSHTM)
Research 9: 316. at the time of this study. She was involved with the KENQOL project
Gendall P, Hoek J. 1990. A question of wording. Marketing Bulletin for 5 years and was based in Kenya for 1 year during the household
1: 25–36. survey. She also has a wider interest in the applicability of
Grosh M, Glewwe P. 2000. Designing household survey question- adapted/translated versions of generic health-related quality of life
naires for developing countries: lessons from 15 years of the measures. She is now based with the Lewin Group (UK).
Living Standard Measurement Survey (Volume 1). Washington,
DC: World Bank.
Julia Fox-Rushby, Ph.D. (Economics), is a Senior Lecturer at the
Guillemin F, Bombardier C, Beaton D. 1993. Cross-cultural adap-
LSHTM. She has written numerous academic papers on the cost-
tation of health-related quality of life measures: literature
effectiveness of health interventions across the world, specializing
review and proposed guidelines. Journal of Clinical Epidemiol-
particularly in maternal and child health, malaria and, more recently,
ogy 46: 1417–32.
vaccine preventable disease. She has also been involved over the past
Herdman M, Fox-Rushby JA, Badia X. 1997. ‘Equivalence’ and the
15 years in developing a number of non-disease specific measures of
translation and adaptation of health-related quality of life ques-
health-related quality of life as a member of the EuroQol group,
tionnaires. Quality of Life Research 6: 237–47.
advisor to the WHOQOL group and as principal investigator of the
Herdman M, Fox-Rushby JA, Badia X. 1998. A model of equival-
KENQOL group.
ence in the cultural adaptation of HRQL measures: a univer-
salist approach. Quality of Life Research 7: 323–35.
Jobe JB. 1990. Research on questionnaire design: perspectives from Lilian Nyandieka, MA (Sociology), is a Research Officer at the
other disciplines. American Journal of Epidemiology 132: 824. Center for Public Health Research, a department of the Kenya
Jobe JB, Mingay DJ. 1989. Cognitive research improves question- Medical Research Institute (PO Box 20752, Nairobi, Kenya). She has
naires. American Journal Public Health 79: 1053–5. been involved in the KENQOL project for 3 years and is also
Kenya Government. 2000. Makueni District Development Report researching social and economic aspects of malaria.
1997–2001. Nairobi: Office of the Vice President and Ministry of
Planning and National Development. John Wanjau, BA (Anthropology), is an Assistant Research Officer
Kumar V, Aaker DA, Day GS. 1999. Essentials of marketing at the Center for Public Health Research, Kenya Medical Research
research. Chichester, UK: John Wiley. Institute. He has been involved in the KENQOL project for 3 years.
Lam CL, Van Weel C, Lauder IJ. 1994. Can the Dartmouth COOP/
WONCA charts be used to assess the functional status of Correspondence: Julia Fox-Rushby Ph.D., Health Policy Unit,
Chinese patients? Family Practice 11: 85–94. London School of Hygiene and Tropical Medicine, Keppel Street,
Larson ML. 1984. Meaning based translation: a guide to cross-lan- London WC1E 7HT, UK. Tel: +44 (0) 20 7927 2267, Fax: +44 (0) 20
guage equivalence. Lanham, MD: University Press of America 7637 5391, Email: Julia.Fox-Rushby@lshtm.ac.uk
