Module 3 Research Methods

1. SURVEYS &
QUESTIONNAIRES
MODULE 3

2. MODULE LEARNING OUTCOMES
After completion of Module 3, you should:
1. Be able to identify methods of probability sampling and nonprobability sampling.
2. Understand the essential steps in designing a survey that reflects postpositivist
philosophical assumptions.
3. Understand generalizability, validity, and reliability and know the difference.

3. SAMPLING METHODS
For more information about sampling for quantitative studies, refer to the Additional
Readings in Module 3. Sampling for qualitative studies will be reviewed at a later time.

4. Sampling is the selection of cases or materials
for study from a larger population or variety of
possibilities. A population is the group in which
the researcher is ultimately interested; thus a
sample is a subgroup drawn from the
population.
There are a number of requirements for a
sample: it should be a minimized
representation of the population, the elements
of such sample need to be defined, and the
population should be clear and empirically
defined.

5. PROBABILITY SAMPLING
Probability sampling is a method that enables researchers to specify for each case in the
population the probability of its inclusion in the sample.
The purpose of probability sampling is to select a sample that is as representative as
possible of the population. The sample is selected in such a way as to allow the use of the
principles of probability to evaluate the generalizations made from the sample of the
population.
A probability sample enables the researcher to estimate the extent to which the findings
bases on one sample are likely to differ from what would be found by studying the entire
population.
Frankfort-Nachmias, Social Statistics for a Diverse Society 8e. SAGE Publications, 2018.

6. PROBABILITY SAMPLING: SIMPLE RANDOM SAMPLE
A simple random sample is a sample designed in such a way as to ensure that (a) every member of the
population has an equal chance of being chosen and (b) every combination of N members has an equal
chance of being chosen.
N usually refers to a population size, while n refers to a sample size.
Example:
Suppose that you are a hospital administrator and you want to conduct a cost-containment study by
examining patients’ records. You have a total of 300 patients’ records and want to draw a random sample
of 10. You would follow these steps:
1. Number the patient accounts starting at 001 and ending at 300
2. You can use an online randomizer or close your eyes and point at random record numbers
You will now have a list of ten random records that you can use as you simple random sample and N=300,
n=10.
Frankfort-Nachmias, Social Statistics for a Diverse Society 8e. SAGE Publications, 2018.

7. PROBABILITY SAMPLING: SYSTEMATIC RANDOM SAMPLE
The systematic random sampling is a methods of sampling in which every Kth (K is the ratio
obtained by dividing the population size by the desired sample size) in the total population
is chosen for inclusion in the sample after the first member of the sample is selected at
random from among the first K member in the population.
Example:
Suppose we have a population of 15,000 commuting students and our sample is limited to
500. If K= population size/sample size, then K= 15,000/500. So, K= 30. We will first choose
any student at random from among the first 30 students, then we will select every 30th
student after that until we reach 500 which is our desired sample size.
Frankfort-Nachmias, Social Statistics for a Diverse Society 8e. SAGE Publications, 2018.

8. PROBABILITY SAMPLING: STRATIFIED RANDOM SAMPLE
A stratified random sample is a method of sampling obtained by (a) dividing the population subgroups
based on one or more variables central to the analysis and (b) then drawing a simple random sample from
each of the subgroups.
Example:
Suppose that our population of interest consists of 1,000 individuals, with 700 whites, 200 blacks, and 100
Latinos. We have divided the population into subgroups (black, whites, and Latinos). We can now, choose
to do either of the following:
1. Proportionate stratified sample. The size of the sample selected from each subgroup is
proportional to the size of that subgroup in the entire population. This ensures that the
representation of the subgroup is variable.
2. Disproportionate stratified sample. The size of the sample selected from each subgroup is
deliberately disproportional to the size of that subgroup in the population. This is useful
when we want to compare subgroups to each other and when the size of some subgroups in
the population is relatively small.
Frankfort-Nachmias, Social Statistics for a Diverse Society 8e. SAGE Publications, 2018.

9. NON-PROBABILITY SAMPLING
It is not always possible or desirable to draw a random sample. Nevertheless, sampling
should be as systematic as possible.
Sampling in qualitative research is more oriented towards the purposive selection of cases,
which in the end will give you insights into features of the population. Purposive sampling
is a method of non-probability sampling.
For more information about sampling for quantitative studies, refer to the Additional Readings in Module
3. Sampling for qualitative studies will be reviewed at a later time.

10. DETERMINING THE NUMBER OF PARTICIPANTS
The following are some variables in which the number of participants depends:
1. If there are time limitations or a short deadline, small number of participants
might be acceptable.
2. Different types of research vary on norms. Some experiments might be
published with as little as 30 participants if the results have important
implications for the profession or for theory development. Qualitative studies
that explore issues in depth are characterized by having a relatively small
number of participants, sometimes a dozen or less. In contrast, surveys will
require hundreds of participants for the study to be judged as reliable.
3. Some participants might be extremely hard to locate so the standards for
sample size might be lowered.
4. Sometimes smaller and more diverse samples add more to the study than a
large sample that only represents a narrow segment of the population.

11. DESIGNING A SURVEY

12. A survey design provides a quantitative or numeric description of trends, attitudes, or
opinions of a population by studying a sample of that population. From sample results, the
researcher generalizes or draws inferences to the population.
Creswell, Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 2014

13. COMPONENTS OF A SURVEY METHOD PLAN
In writing a survey methods section, follow this format:
1. The Survey Design. Identify the purpose of the survey, why the survey is the preferred type of data collection
procedure for the study, whether the survey will be cross-sectional or longitudinal, and the form of data
collection (mail, phone, Internet, in-person)
2. The Population and Sample. Identify the population in the study, whether the sampling design for the
population is single-stage or multi-stage, the selection process for the individuals, whether the study will
involve stratification of the population before selecting the sample, discuss the procedures for selecting the
sample from available lists, indicate the number of people in the sample and the procedures used to
compute this number.
3. Instrumentation. Name the survey instrument used to collect data, describe the validity of the instrument,
mention reliability of scores, include sample items from the instrument, indicate the major content sections in
the instrument, discuss plans for pilot testing or field-testing the survey and provide rationale of these plans,
and identify steps for administering the survey and following up to ensure a high response rate.
4. Variables in the Study. Restate the variables, research questions or hypotheses.
5. Data Analysis and Interpretation. Present the steps involved in analyzing the data.
Creswell, Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 2014

14. QUANTITATIVE DATA ANALYSIS: DESCRIPTIVE STATISTICS
Descriptive statistics are the numbers or information we use to summarize raw data; they focus on
describing the features of the data in the sample.
Some examples of the data that can be explained using descriptive statistics include gender, age
distribution, zip code distribution, etc.
Descriptive statistics are the first step in a quantitative study; the second step would be to complete the
inferential analysis.
Descriptive statistics analyses include
○ Frequencies
○ Measures of central tendency
○ Measures of dispersion

15. QUANTITATIVE DATA ANALYSIS: FREQUENCIES
To report frequencies, you need
● The total number of cases (Ex. 390 total responses)
● The possible answers (Ex. vanilla, chocolate, strawberry) for the specific question (Ex. what is your
favorite type of ice cream?)
You will then calculate
● The number of cases per possible answer
○ Vanilla= 117
○ Chocolate= 78
○ Strawberry= 195
● Percentages
○ Vanilla= 117/390
○ Chocolate= 78/390
○ Strawberry= 195/390
Presenting information in percentages helps people understand information quicker. Saying “50% of the
respondents like strawberry ice cream the best” is easier to understand than “195 respondents out of 390
like strawberry ice cream the best”

16. QUANTITATIVE DATA ANALYSIS: MEASURES OF CENTRAL TENDENCY
Central tendency helps you describe in which way the data cluster.
The measures for central tendency are:
1. Mean (average)- represents the sum of all values in a dataset divided by the total
number of the values
2. Median- The middle value in a dataset that is arranged in ascending order (from the
smallest value to the largest value)
3. Mode- defines the most frequently occurring value in a dataset

17. QUANTITATIVE DATA ANALYSIS: MEASURES OF DISPERSION
● Dispersion in statistics helps you describe how spread out a set of data is
● Important measures for dispersion are
○ Standard deviation (sd)- it tells you how spread out the numbers are from the
mean
○ Variance- squared value of a standard deviation
● Standard deviation helps you answer the following questions:
○ Are all your scores close to the average?
○ Are lots of scores way above or way below the average score?
● A small standard deviation means that the values in the data are close to the mean of
the data set; a large standard deviation means that the values in the data set are
farther away from the mean

18. QUANTITATIVE DATA ANALYSIS: INFERENTIAL STATISTICS
Inferential statistics is about providing an explanation to the issue being investigated.
The goal is to show relationships and patterns in the sample that has been studied to make
inferences about the basic population it was drawn from.
Examples of inferential statistics analyses include:
● T-test- when two data sets are compared for their differences
● Chi-square-when two data sets are compared to see if they are related

19. GENERALIZABILITY, VALIDITY, AND
RELIABILITY
For more information about validity and reliability for quantitative studies, refer to the
Additional Readings in Module 3.

20. GENERALIZABILITY
The degree to which the results derived from a sample can be transferred to the population.
In quantitative research, the extent to which results can be generalized may be checked in two ways- by
assessing external validity, on e would (a) assure that the results found for the sample are valid for the
population and also (b) test how far they can be transferred to other, comparable populations.
Various sampling procedures can be used to ensure generalizability. One procedure is to use random
sample, in which every element in the population has the same chance to be an element in the sample.
This procedure enables the exclusion of any biases resulting from the disproportionally weighted
distribution of features in the sample compared to the population. Thus, the sample is representative for
the population.

21. VALIDITY
A standard criterion in standardized/quantitative research, for which you will check, for example, whether
confounding influences affected the relations under study (internal validity) or how the results are transferable to
instances beyond the current research situation (external validity).
Validity is assessed for both research designs and for measurement of instruments.
Research Designs
In the case of research designs, the focus will be on evaluating the results by checking the internal validity of a
research design. Internal validity characterizes how far the results of a study can be analyzed unambiguously. The
general question of internal validity is: how far can we transfer results beyond the situations and persons for which
they are produced, to situations and persons outside of the research?
Instruments
The issue of validity of a research instrument can be summarized in the question: does the method measure what it
is supposed to measure? To answer this question, you can apply various forms of validity checks: (a) content validity,
(b) criterion validity, (c) construct validity.
For more information about validity and reliability for quantitative studies, refer to the Additional Readings in Module 3.

22. RELIABILITY
A standard criterion in standardized/quantitative research which is based on repeated application of a test
to assess whether the results are the same in both cases. The reliability indicates the degree of exactness
in measurement (precision) of an instrument.
The reliability of a measurement can be assessed in different ways:
1. Retest reliability
2. Parallel test reliability
3. Split-half reliability
4. Inter-coder reliability
For more information about validity and reliability for quantitative studies, refer to the Additional Readings in Module 3.