Questionnaire design and validation

Karim Elghanam
Faculty of Medicine, Menoufia
University
k.elghanam@med.menofia.edu.eg
September 15, 2015
Questionnaire
Development
& Validation

2
What’s the Q.?
Is a formalized set of questions to
obtain certain information from
certain respondents.
Questionnaire Development

3
Developing a good questionnaire
is NOT EASY!
• It takes time, time, and more time.
• You may end up writing multiple
drafts.
• It helps to work with others when
developing a questionnaire.Questionnaire Development

Some Definitions Before
Starting
4 Questionnaire Development

5
Types of Q.?
1- Self-
administrated
• Computer assisted
and Web-based
• Paper and pencil 2- Interviewer-
administrated
• Face to face
• Telephone

6
• Self-administered
questionnaire
Advantages:
 Cheap and easy to administer
 Preserves confidentiality
 Completed at any time
 No influence by interviewer
 Allow for instant data coding

7
• Self-administered
questionnaire
Disadvantages:
 Low response rate
 Questions can be
misunderstood
 No control by interviewer

8
• Interview-administered Q.
Advantages:
 Participation of illiterate
people
 Clarification of ambiguity
 Quick answers

9
• Interview-administered Q.
Disadvantages:
 Interviewer bias
 Needs more resources
 Only short questionnaires possible
especially on telephone
 Difficult for sensitive issues

1- Formulations of idea
(Conceptualization)
Clear Idea
Clear objectives ( primary and
secondary objectives )
You will be Strict to them.

Home Message
“Ask what you want to know,
not something else”
(Bradburn, 2004)

2- Literature Review
Asking others for their own
questionnaires !

3- First draft
Ideas Roadmap and the begin to
change theses idea to questions
Idea Question/s

First Draft

4- Condensed Literature
Review..!
Another draft more specific than the
first one

Another Draft

5- Detect .. ?
1. Study design
2. Population
3. Sample size
4. Type of the Q. will you use ?

6- Design
Questionnaire

20
Types of questions ?
1- Open-ended question
2- Closed-ended question
3- Contingency questions.

21
1- Closed questions
You ask the respondent to choose, among a
possible number of answers, the response
that most closely represents his/her
viewpoint.

22
Advantages of closed question
• The respondent is restricted to a certain
set of responses,
• They are easy to answer ( no hesitations
),
• Less time consuming,
• Easy to code, and

23
 You will force the respondent to choose
between specific answers … BIAS
 You offer answers that otherwise would not
have come to his mind,
 They don’t allow for creativity and
generating ideas,
Disadvantages of closed
question

24
They do not permit the respondent to qualify
the chosen response or express a more
complex or subtle meaning,
There is may a tendency for the respondent
to tick systematically either the first or last
answer,
May Select what may be considered as the
most socially desirable response,
question

25
To answer all items in a list in the same
way
question

26
2- Open-ended questions
Free-response questions are not followed
by any choices and the respondent must
answer by supplying a response, usually by
entering a number, a word, or a short text.

27
Advantages of open-ended
 They allow respondents to express their
ideas spontaneously in their own
language.
 They are less likely to guide the answer
than closed questions .
 They can add new information when there
is very little existing information available
about a topic.

28
 They may be difficult to answer
 They require effort and time from the
respondent,
 They require the development of a system of
coded categories with which to classify the
responses..!
 Can’t be coded !
Disadvantages of open-
ended

29
 They require the respondent to have some
degree of writing ability
Disadvantages of open-
ended

30
3- Contingency questions
Is a special case of a closed-ended
question because it applies only to a
subgroup of respondents…!

Types of closed questions
?
Scales ?

6- Design
Questionnaire

Wording of the question
Logic flow of the questions
Simple language
Mother language is the best
Use concise and clear words
And Avoid …..

34
Avoid

35
1- Double-barreled questions:
Single questions that ask for two things and
therefore require two answers.
1. Do you have your own table or your own room to do your
homework?
2. Do you think it is good idea for children to study geography
and history in primary school?

36
2- Double negatives
Either in questions or answers
1.All the following aren’t used in the TTT of … Except
? :D

37
3- Avoid overlapping response categories
Under 20 1
20-30 2
30-40 3
40-50 4
50-60 5
60 … or more
How old are
you?
Under 20 1
21-30 2
31-40 3
41-50 4
51-60 5
61 … or more

38
4- Leading questions
Would you say that you are not in favour of school
on Saturday morning??

39
5- Overload the respondent’s memory
Would you say that you are not in favour of school
on Saturday morning??

40
6- Long questions
Generally, it is recommended to hold
questions to 25 words or less.
If a longer sentence is used then it should be
broken up so that there will be several shorter
sentences.

41
Avoid
7- Hypothetical questions
• Prediction of behavior and future
• “Would you use this resource in your class
if it were available..?”
• People are generally poor predictors.

A new
Draft

44
1. Informal tests
2.Experts review
3.Small-scale tests (pre test)
4.Functional or technical testing
5.Large-scale pilots and trials
Testing

A new
Draft

47
1. …..
2. Experts review
Testing

A new
Draft

49
Pre-pilot
Focusing
group
Pre-testing the
questionnaire

50
Aims
Pre-testing the
questionnaire

51
a. Is each item producing the kind of
information needed?
b. What role is the item going to play in the
proposed analysis?
c. Are the questions meaningful to the
respondents?
d. Are respondents easily able to understand
the items?
e. Can respondents use the response format
for each item?

52
g. Was the question order logical and did the
interview flow smoothly?
h. Did some parts of the questionnaire arouse
suspicion?
I. Did other parts of the questionnaire seem
repetitive or boring?
j. Were interviewers able to read the questions
without difficulty?
k. Were respondents able to follow all
instructions?
l. Was the questionnaire too long?

53
Select a sample similar in socio-economic
background and geographic location to the one
that will be used in the main study.
This sample will not be included in the final
survey.
 Not to be a convenience sample.
Character Of this group:

A new
Draft

55
1. ……
2.……
3. ……
4.Functional or technical testing
Testing

A new
Draft

57
1. …….
2. …….
3. …….
4. …….
5.Large-scale pilots and trials
Testing

A new
Draft

Reliability and validity
59
Validity: Concerns the degree to which a
question measures what it was intended to
measure and not something else.
• Content (or face) validity
• Empirical (or predictive) validity
• Concurrent validity

60
Reliability: concerns the consistency of a
measure. That is, the tendency to obtain the same
results if the measure was to be repeated by using the
same subjects under the same conditions.
Kappa coefficient !

61
Covering letters and
introductory paragraphs
 You must explain the objectives of the
survey,
 In an interview, one of the tasks of the
interviewer is to persuade the respondent to
co-operate.
 In a self-administered questionnaire, the
covering letter is the only instrument forQuestionnaire Development

62
 Identify the organization conducting the study.
 Explain the purpose of the study.
 Assure the respondent that information provided will
be managed in a strictly confidential manner.
Covering letter Items

63
 Explain WHY it is important that the respondent
should complete the questionnaire.
 Brief detail on how the respondent was selected
(for example, ‘Your name was randomly selected ....’).
 Expression of appreciation for the respondent’s help.

64
Estimate of questionnaire completion time.
Provide the name and contact numbers of
the Principal Researcher(PI).

 k.elghanam@med.menofia.ed
u.eg
 01068950053
 FB: Karim Elghanam
Contact Me65
Hope Happiness finds you when you don’t Expect to find it
.

Questionnaire design and validation

More Related Content

What's hot

Similar to Questionnaire design and validation

Recently uploaded

Questionnaire design and validation