2. Introduction
Data collection is perhaps the most
important component of Monitoring and
Evaluation
Without good data we can not tell if or not
the project is beneficial as expected
Mechanisms must be put in place to
collect adequate, timely and relevant data
on a continuous basis
2
3. 3
Qualities of good data
Adequate
Timely
Relevant
Consistent
Reliable
4. 4
Data Collection methods
Questionnaire
Observation
Laboratory processes
Desk review (books, records, other
documents)
5. 5
Questionnaire
Quality and adequacy of data collected
depends on the nature of questionnaire
used
The questionnaire must address all the
variables under study
Length and complication (language) of
the questionnaire can affect data
collection
7. 7
Open-ended Questionnaires
Allow respondent to answer the question in his or her
own words
Example: What benefits have you drawn from the
project?
Open-ended questions allow the interviewer to ask or
probe the respondent further if he/she feels that
clarification of a point, or additional information, is
needed.
Good for exploratory research.
Disadvantage are the difficulties of analyzing the data
and in categorizing and summarizing answers because
of the unique responses.
Also, there is the possibility of interviewer bias and bias
caused by the different education levels of the
respondents
8. 8
Closed-ended Questionnaires
Respondent answers the question in
in very specific pre-determined choice
of words
Example: Would you study participte
in the project if you were to make the
choice a 2nd time? (tick yes or no)
They are easy to statistically analyse
10. 10
Key questions when designing a
questionnaire
What should the respondents be asked?
How should each question be phrased?
In what sequence should the questions be arranged?
What is the best questionnaire layout for the research
problem in question?
What communication medium should be utilized
(personal interview, telephone interview etc.)
Should the questionnaire be pretested?
How should the questionnaire be pretested?
Does the questionnaire require a revision?
11. 11
Relevance and Accuracy
Questionnaires must collect data that is both accurate
and relevant
Relevance means:
1. No unnecessary information is obained from the
questionnaire
2. All the information that is needed for the purpose of
the evaluation is collected
3. No important information is omitted
Accuracy means that the questions are worded in a
manner which ensures the collection of correct
information from respondents
12. 12
Guidelines for phrasing
questionnaires
Avoidance of Complexity / Use Simple Language
Avoidance of leading and Loaded questions
Counterbiasing Statements (leading respondent with
an introductory remark before the question
Avoidance of Ambiguity
Avoidance of Double-Barreled Questions –
Questions which adress two or more issues
simultaneously
Avoidance of Assumptions
Avoidance of Burdensome (and Memory Taxing)
Questions
13. 13
Sequencing Questions
The order in which questions are put in a
questionnaire may significantly affect the
response rate
Usually, interviewers prefer to ask general
questions from respondents before moving
on to specific questions (funnel technique)
14. 14
Key Factors to consider in
questionnaire layout
Don‘t overcrowd the questionnaires
Keep questionnaires as brief as possible
Ensure that the title and subtitles of the
questionnaire and questionnaire sections are
carefully phrased and captures the
respondents attention
Use appealing language
15. 15
Pretesting Questionnaires
Pretesting is a very useful method for determining whether
respondents have any difficulty understanding the
questionnaire and whether the questions are ambiguous or
can lead possibly to biased answers
Pretesting ensures that costly errors in questionnaires
which are given to a large number of respondents is
avoided
The respondents involved in a pretest should be similar in
essence to the target respondents of the research
16. 16
Pretesting provide answers to important
questions for the, such as:
Can the questionnaire format be followed by the
interviewers?
Is the questionnaires ambiguous?
Does the questionnaire flow naturally and
conversationally?
Can respondents answer the questions easily?
Which alternative forms of questions work best?
18. Introduction
A sample is a subset of a larger population of
objects individuals, households, businesses,
organizations and so forth
Sampling enables interviewers to make
estimates of some unknown characteristics of
the population in question
A finite group is called population whereas a
non-finite (infinite) group is called universe
A census is a investigation of all the individual
elements of a population
20. Reasons for Sampling
Budget and time Constraints (in case of large populations)
High degree of accuracy and reliability (if sample is
representative of population)
Sampling may sometimes produce more accurate results
than taking a census as in the latter, there are more risks
for making interviewer and other errors due to the high
volume of persons contacted and the number of census
takers, some of whom may not be well-trained
21. The Sampling Process
Define the Target
Population
Select a
Sampling Frame
Determine if a probability
or non-probability sampling
method will be chosen
Plan procedure for
selecting sampling units
Determine sample size
Select actual sampling units
Conduct fieldwork
1
2
3
4
5
6
7
22. Defining the Target Population
The target population is that complete group
whose relevant characteristics are to be
determined through the sampling
A target population may be, for example, all
beneficiaries of a project during the life of the
project
23. The Sampling Frame
The sampling frame is a list of all those population
elements that will be used in the sample
An example a of sampling frames is the list of direct
beneficiaries of a project
Often, the list does not include the entire population. The
discrepancy is often a source of error associated with the
selection of the sample (sampling frame error)
24. Sampling Units
The sampling unit is a single element – or
group of elements – subject to selection in
a sample. Examples:
Every project beneficiary whose name begins
with “F”
All project beneficiaries under 18 years of age
25. Probability and
Non-Probability Sampling
Probability Sampling – Every element in the
population under study has a non-zero
probability of selection to a sample, and every
member of the population has an equal
probability of being selected
Non-Probability Sampling – An arbitrary means
of selecting sampling units based on subjective
considerations, such as personal judgment or
convenience.
27. Convenience Sampling
This is a sampling technique which selects those
sampling units most conveniently available at a
certain point in, or over a period, of time
Major advantages of convenience sampling is that it
is quick, convenient and economical; a major
disadvantage is that the sample may not be
representative
Convenience sampling is best used for the purpose
of exploratory research and supplemented
subsequently with probability sampling
28. Judgment (purposive) Sampling
A sampling technique in which the interviewer
selects the sample based on judgment about
some appropriate characteristic of the sample
members
29. Quota Sampling
A sampling technique in which the interviewer ensures
that certain characteristics of a population are
represented in the sample to an extent which he or she
desires
Example: An interviewer wants to determine through
interview, the demand for Product X in a district which
is very diverse in terms of its ethnic composition. If the
sample size is to consist of 100 units, the number of
individuals from each ethnic group interviewed should
correspond to the group’s percentage composition of
the total population of that district
30. Contd.
Quota Sampling has several advantages and
disadvantages:
Advantages include the speed of data collection, less
cost, the element of convenience, and
representativeness (if the subgroups in the sample
are selected properly)
Disadvantages include the element of subjectivity
(convenience sampling rather than probability-based
which leads to improper selection of sampling units)
31. Snowball Sampling
A sampling technique in which individuals or organizations are
selected first by probability methods, and then additional
respondents are identified based on information provided by the first
group of respondents
Example: 20 project beneficiaries are identified which, in turn,
identify a number of other project beneficiaries
The advantage of snowball sampling is that smaller sample sizes
and costs are necessary; a major disadvantage is that the
second group of respondents suggested by the first group may
be very similar and not representative of the population with that
characteristic
33. Simple Random Sampling
This is a technique which ensures that each
element in the population has an equal chance
of being selected for the sample
Example: Choosing raffle tickets from a drum,
computer-generated selections, random-digit
telephone dialing
The major advantage of simple random sampling is
its simplicity
34. Systematic Sampling
This is a technique in which an initial starting point is
selected by a random process, after which every nth
number on the list is selected to constitute part of the
sample
Example: From a list of 1500 name entries, a name on the list is
randomly selected and then (say) every 25th name thereafter.
The sampling interval in this case would equal 25.
For systematic sampling to work best, the list should be random
in nature and not have some underlying systematic pattern
35. Stratified Sampling
A technique which in which simple random sub samples are drawn
from within different strata that share some common characteristic
Example: project members are divided into two groups
(beneficiaries and staff) and from each group, respondents are
selected for a sample using simple random sampling in each of
the two groups, whereby the size of the sample for each group is
determined by that group’s overall strength
Stratified Sampling has the advantage of giving more
representative samples and less random sampling error.
However, it is more complex and information on the strata may
be difficult to obtain
37. Issues in Sample Design and Selection
Accuracy – Samples should be
representative of the target population
Resources – Time, money and individual
or institutional capacity are very important
considerations due to the limitation on
them. Often, these resources must be
“traded” against accuracy
38. Issues in Sample Design and Selection
Availability of Information – Often information on sample
participants in the form of lists, directories etc. is
unavailable which makes some sampling techniques
(e.g. systematic sampling) impossible to undertake
Geographical Considerations – The number and
dispersion of population elements may determine the
sampling technique used (e.g. cluster sampling)
40. Methods of Data Analysis
1. Quantitative analysis
2. Qualitative analysis
Both methods can be applied manually or by
computer software (e.g. SPSS or MS Excel)
40
41. Quantitative Analysis
Analyses done through use of statistical
processes that may include derivation of
percentages, ratios, correlations and
trends among others as may be required
Statistics give a more accurate picture of
the situation at hand
Statistics must be selected carefully- they
should be able to give expected results
41
42. Qualitative Analysis
Not all data can be analyzed quantitatively
Some data is too qualitative and so has to
be analyzed by applying perceptional
meaning
Thematic analysis is one of the most
popular methods of qualitative analyses
In this method data is classified into
themes based on its relatedness
42
43. Choice of Method
Depends on:
1. Nature of data to be analyzed
2. Nature of expected findings
3. Expected use of findings
4. Audience of the findings
A mixed method is preferred in most cases
43
