SEO Master Class - Steve Wiideman, Wiideman Consulting Group
Research methods presentation
1. Why Do I Need to Know About Different
Methods?
As a future practitioner…
To be able to intelligently participate in
research projects, evaluations, and
studies undertaken by your institution.
As an educated citizen ...
To understand the difference between
scientifically acquired knowledge and
other kinds of information.
1
2. What’s the Difference Between “Method” and
“Methodology”?
Method Methodology
•Techniques for
gathering evidence
•The various ways of
proceeding in gathering
information
•The underlying theory
and analysis of how
research does or
should proceed, often
influenced by discipline
2
3. What is research?
• We ask questions all the time
• Research is a formal way of going about
asking questions
• Uses methodologies
• Many different kinds (e.g. market
research, media research and social
research)
• Basic research methods can be learned
easily
3
4. Basic Research Methods
• Quantitative research (e.g. survey)
• Qualitative research (e.g. face-to-
face interviews; focus groups; site
visits)
• Case studies
• Participatory research
4
5. Surveys
• Think clearly about questions (need
to constrain answers as much as
possible)
• Make sure results will answer your
research question
• Can use Internet for conducting
surveys if need to cover wide
geographic reach
5
6. Quantitative Research
• Involves information or data in the form
of numbers
• Allows us to measure or to quantify
things
• Respondents don’t necessarily give
numbers as answers - answers are
analysed as numbers
• Good example of quantitative research is
the survey 6
7. Qualitative Research
• Helps us flesh out the story and develop a
deeper understanding of a topic
• Often contrasted to quantitative research
• Together they give us the ‘bigger picture’
• Good examples of qualitative research are
face-to-face interviews, focus groups and
site visits
7
8. Face-to-face interviews
• Must prepare questions
• Good idea to record your interviews
• Interviews take up time, so plan for an
hour or less (roughly 10 questions)
• Stick to your questions, but be flexible if
relevant or interesting issues arise during
the interview
8
9. Focus groups
• Take time to arrange, so prepare in
advance (use an intermediary to help
you if you can)
• Who will be in your focus group? (e.g.
age, gender)
• Size of focus group (8-10 is typical)
• Consider whether or not to have
separate focus groups for different ages
or genders (e.g. discussing sex and
sexuality) 9
10. Site visits and observation
• Site visits involve visiting an organization,
community project etc
• Consider using a guide
• Observation is when you visit a location
and observe what is going on, drawing
your own conclusions
• Both facilitate making your research
more relevant and concrete
10
11. Case Studies
• Method of capturing and presenting
concrete details of real or fictional
situations in a structured way
• Good for comparative analysis
11
12. Participatory Research
• Allows participation of community being
researched in research process (e.g.
developing research question; choosing
methodology; analysing results)
• Good way to ensure research does not
simply reinforce prejudices and
presumptions of researcher
• Good for raising awareness in
community and developing appropriate
action plans 12
13. Planning your research:
Key questions
• What do you want to know?
• How do you find out what you want to
know?
• Where can you get the information?
• Who do you need to ask?
• When does your research need to be
done?
• Why? (Getting the answer)
13
14. Objectives of presentation
• Definition of sampling
• Why do we use samples?
• Concept of representativeness
• Main methods of sampling
• Sampling error
• Sample size calculation
14
15. Definition of sampling
Procedure by which some members
of a given population are selected as
representatives of the entire population
15
16. Definition of sampling terms
• Sampling unit
– Subject under observation on which information
is collected
• Sampling fraction
– Ratio between the sample size and the population
size
• Sampling frame
– Any list of all the sampling units in the population
• Sampling scheme
– Method of selecting sampling units from sampling
frame
16
17. Why do we use samples ?
Get information from large populations
–At minimal cost
–At maximum speed
–At increased accuracy
–Using enhanced tools
17
19. What we need to know
• Concepts
– Representativeness
– Sampling methods
– Choice of the right design
• Calculations
– Sampling error
– Design effect
– Sample size
19
21. Representativeness
• Person
• Demographic characteristics (age, sex…)
• Exposure/susceptibility
• Place (ex : urban vs. rural)
• Time
• Seasonality
• Day of the week
• Time of the day
Ensure representativeness before starting,
confirm once completed !!!!!!
21
23. Non probability samples
• Quotas
• Sample reflects population structure
• Time/resources constraints
• Convenience samples (purposive units)
• Biased
• Best or worst scenario
Probability of being chosen : unknown
23
24. Probability samples
• Random sampling
• Each subject has a known probability of
being chosen
• Reduces possibility of selection bias
• Allows application of statistical theory to
results
24
25. Sampling error
• No sample is the exact mirror image of
the population
• Magnitude of error can be measured in
probability samples
• Expressed by standard error
– of mean, proportion, differences, etc
• Function of
– amount of variability in measuring factor of
interest
– sample size
25
26. Methods used in probability samples
• Simple random sampling
• Systematic sampling
• Stratified sampling
• Multistage sampling
• Cluster sampling
26
27. Quality of an estimate
Precision &
validity
No precision
Random error !
Precision but
no validity
Systematic
error (Bias) !
27
28. Simple Random Sampling
• Principle
–Equal chance of drawing each unit
• Procedure
–Number all units
–Randomly draw units
28
29. Simple random sampling
• Advantages
–Simple
–Sampling error easily measured
• Disadvantages
–Need complete list of units
–Does not always achieve best
representativeness
–Units may be scattered
29
30. Example: evaluate the prevalence of tooth
decay among the 1200 children attending a
school
• List of children attending the school
• Children numerated from 1 to 1200
• Sample size = 100 children
• Random sampling of 100 numbers between 1
and 1200
How to randomly select?
Simple random sampling
30
33. Systematic sampling
• N = 1200, and n = 60
⇒ sampling fraction = 1200/60 = 20
• List persons from 1 to 1200
• Randomly select a number between 1 and
20 (ex : 8)
⇒ 1st
person selected = the 8th
on the
list
⇒ 2nd
person = 8 + 20 = the 28th
etc .....
33
37. Stratified Sampling
• Principle :
–Classify population into internally
homogeneous subgroups (strata)
–Draw sample in each strata
–Combine results of all strata
37
38. Stratified Sampling
• Advantages
– More precise if variable associated with
strata
– All subgroups represented, allowing
separate conclusions about each of
them
• Disadvantages
– Sampling error difficult to measure
– Loss of precision if very small numbers
sampled in individual strata
38
39. Example: Stratified sampling
• Determine vaccination coverage in a
country
• One sample drawn in each region
• Estimates calculated for each stratum
• Each stratum weighted to obtain
estimate for country (average)
39
40. Multiple Stage Sampling
Principle
• = consecutive samplings
• example :
sampling unit = household
– 1rst
stage : drawing areas or blocks
– 2nd
stage : drawing buildings, houses
– 3rd
stage : drawing households
40
41. Cluster Sampling
• Principle
–Random sample of groups
(“clusters”) of units
–In selected clusters, all units or
proportion (sample) of units included
41
43. Cluster Sampling
• Advantages
– Simple as complete list of sampling units
within population not required
– Less travel/resources required
• Disadvantages
– Imprecise if clusters homogeneous and
therefore sample variation greater than
population variation (large design effect)
– Sampling error difficult to measure
43
44. Cluster Sampling
To evaluate vaccination coverage:
• Without list of persons
• Total population of villages
• Randomly choose 30 clusters
• 30 cluster of 7 children each= 210 children
44
45. Drawing the Clusters
You need :
– Map of the region
– Distribution of population (by villages or area)
– Age distribution (population 12-23 m :3%)
1600
220
3200
400
800
200
1200
200
1600
400
53000
7300
106000
13000
26500
6600
40000
6600
53000
13200
A
B
C
D
E
F
G
H
I
J
12-23Pop.Village
45
46. Distribution of the clusters
A
B
C
D
E
F
G
H
I
J
1600
220
3200
400
800
200
1200
200
1600
400
1600
1820
5020
5420
6220
6420
7620
7820
9420
9820
Total population = 9820
Compute cumulated population
46
47. Distribution of the clusters
Then compute sampling fraction :
K = = 327
Draw a random number (between 1
and 327)
Example: 62
Start from the village including “62”
and draw the clusters adding the
sampling fraction
9820
30
A
B
C
D
E
F
G
H
I
J
1600
1820
5020
5420
6220
6420
7620
7820
9420
9820
I I I I
I
I I I I I I I I I I
I
I I
I
I I I I
I
I I I I I
I
47
48. Drawing households and children
On the spot
Go to the center of the village , choose direction
(random)
Number the houses in this direction
Ex: 21
Draw random number (between 1 and 21) to
identify the first house to visit
From this house progress until finding the 7
children ( itinerary rules fixed beforehand)
48
49. Selecting a sampling method
• Population to be studied
– Size/geographical distribution
– Heterogeneity with respect to variable
• Level of precision required
• Resources available
• Importance of having a precise estimate
of the sampling error
49
50. Steps in estimating sample size
• Identify major study variable
• Determine type of estimate (%, mean, ratio,...)
• Indicate expected frequency of factor of interest
• Decide on desired precision of the estimate
• Decide on acceptable risk that estimate will fall
outside its real population value
• Adjust for estimated design effect
• Adjust for expected response rate
• (Adjust for population size? In case of small size
population only)
50
51. Place of sampling
in descriptive surveys
• Define objectives
• Define resources available
• Identify study population
• Identify variables to study
• Define precision required
• Establish plan of analysis (questionnaire)
• Create sampling frame
• Select sample
• Pilot data collection
• Collect data
• Analyse data
• Communicate results
• Use results
51
53. • If in doubt…
Call a statistician !!!!
Thank You
53
Editor's Notes
précision = si on répète les mesures, on obtient des estimations proches (mesurée par la variance de l’échantillon)
validité = capacité à estimer la vraie valeur du paramètre dans la population
pour chaque individu,
probabilité égale
d'être désigné dans l'échantillon
il y a 1200 individus
donc tout chiffre entre 1 et 1200 doit pouvoir être tiré
on décide de prendre les 4 derniers chiffres de chaque série
à partir d'un chiffre pris au hasard, on descend jusqu'à rencontrer un nombre (à 4 chiffres) compris entre 1 et 1200 : ce nombre est alors retenu pour l'échantillon
on continue jusqu'à obtenir 60 enfants.
