Methodology

1. Methodology
Tips for Constructing Instruments

2. Matching methods to research paradigm
Method Quantitative Qualitative
Written
Instrument
Standardized Instrument
(validity and reliability
established)
Constructed survey or
interview guide (Use some
questions from other surveys;
construct new questions). May
determine validity and
reliability after administration
No predeveloped questions in survey
or interview guide
Predeveloped questions – may be
administered in no established order;
questions may be added depending
on circumstances.
Predeveloped questions – established
order
Observation Structured observation –
predetermined items are
counted in terms of frequency,
duration, occurrence,
permanent product, etc.
Open-ended observations – trying to
identify themes and patters
Content
analysis
Structured counting of a pre-
determined phenomena in
terms of frequency, etc.
Open-ended analysis; themes and
patterns are identified.

3. Quantitative/Descriptive Studies
involve specific Research Designs
• Designs structure how the interviews are
conducted.
• Allow different groups and time periods to
be compared.
• Minimize threats to internal and external
validity.

4. Components of Designs
• Sampling (Random; Other probability
sampling or nonprobability – such as
convenience, purposive, or snowball)
• Groups – one group; two, or multiple
group comparisons.
• Time intervals – pre and post tests or
multiple observations.

5. Common design types
• Experimental Design (pre-post; random sample,
two or more groups compared.
• Quasi Experimental with nonequivalent
comparisons groups- pre and post.
• Quasi Experiments with post-test only.
• One group - pre and post test only.
• One group – multiple time periods measured
(time series or longitudinal studies)
• One group – one measure – usually called
cross-sectional survey.

6. Random designs help ensure
internal/external validity
• Internal validity – attribute of people included or
way the research was conducted may limit our
ability to establish a real connection between the
independent and dependent variables.
• External validity – can we generalize results to
other people, settings, and times.
(Note for some types of studies – qualitative
research and needs assessments and some
types of evaluations – we do not need
generalizability. We are only interested in finding
things out about a particular situation).

7. Exceptions to Experimental/Quasi-
experimental Designs: One group studies
• Case studies (combination of methods
looking at one individual, group, family,
organization, or community)
• Survey research
• Consumer satisfaction surveys
• Needs Assessments

8. Use these alternative methods
• For descriptive studies
• When you do not have an intervention
• When you want to look at a specific
situation or group and you do not intend to
generalize.
• When it would be difficult or unethical to
create an experimental or control group.
• When you do not have time or money to
conduct an experiment.

9. Threats to Internal Validity Include:
• History
• Maturation
• Testing
• Instrumentation
• Regression to the mean
• Selection
• Interaction between Selection and the
other threats to Internal Validity.

10. Qualitative Research
• Usually one group
• Comparisons may be made after data
analysis – decision to compare made
based on obvious patterns in the data.
• Sampling – nonprobability – purposive,
convenience, snowball
• Uses observation, interviews, and content
analysis.

11. Focus groups may also be used to conduct qualitative
research. Focus groups:
• Are used to assess people’s perceptions about programs or
services.
• Are used to generate recommendations for changing programs and
services.
• Are often used in conjunction with needs assessments or to
generate ideas or explore themes that can be used in descriptive
studies.
• Are used to answer questions about how or why programs work.
• May be used in conjunction with other research methods.
• Include six to eight participants who are selected using purposive
sampling methods.
• Utilize a semi-structured interview guide with 6-8 questions.
• Require that a facilitator ask each of the questions and solicit
responses from members.
• Are used to generate a common response or a consensus from
group members.

12. In any study:
• A mixture of methods may be used.
• Researchers may seek to increase reliability and
validity by using both quantitative and qualitative
approaches. This is called triangulation!
• Methods used should match the research question:
a) What happens/what is the outcome? quantitative)
b) Why does something happen? How does it happen?
(qualitative)
c) A description of what exists (Descriptive)

13. Descriptive studies
Generally involve surveys, standardized
interviews, or the use of pre-existing
(secondary data).
• Include needs assessments.
• Can include both qualitative and
quantitative data collection. In some
cases, one instrument can include a mix of
qualitative and quantitative data collection
methods.

14. Survey and Interview Construction
Tips
• Only measure one concept per question.
• Don’t write the question to solicit a specific response
• Response categories (if used) should be mutually
exclusive and exhaustive
• Questions should be tactful and should not invoke a
social stigma.
• May be filter questions – respondents can be directed to
successive questions based on one response.
• Demographic questions are generally put at the
beginning or the end of the instrument.
• Be aware that respondents do not like to answer some
types of questions (age, income, organization budgets,
etc.)

15. Example of closed-ended
(response category) questions
1. What is your gender
1. Male 2. Female
2. What is your ethnic identification?
1. African American
2. American Indian
3. Asian/Pacific Islander
4. European American
5. Mexican American/Latino
6. Multicultural or multi-racial
7. Other ____________________

16. More examples
3. What is your age?
1. 18 or under
2. 19-29
3. 30-49
4. 50 – 64
5. Over 65
4. How satisfied are you with Social Work 292?
1. Very satisfied
2. Satisfied
3. Neutral
4. Unsatisfied
5. Very Unsatisfied

17. Sample question: How would you assess
your own practice ability in the following
areas:
High Medium Low
Write Case
Notes
1 2 3
Write Reports 1 2 3
Use
Computers
1 2 3
Apply
Theories
1 2 3
Evaluate
Practice
1 2 3

18. 1
.
Can you describe how you first became aware of your deafness
2
.
How do you see yourself today, in terms of your deafness?
3
.
What does your deafness mean to you?
4
.
Can you describe any particularly difficult or traumatic experiences
in your life related to your deafness?
5
.
Can you describe how you fit into deaf culture?
6
.
To what extent do you consider yourself active in both the deaf
world and the hearing world?

19. Each type of question corresponds to a level of
measurement. These levels of measurement will
determine what type of data analysis we use:
• All qualitative research is nominal.
• All categorical data in which there is no
difference in value among the categories is
nominal.
• All categorical data in which there is an implied
ranking (for example, high-medium-low) is
ordinal.
• Any question that asks respondents to rank
order responses is also ordinal.
• Any question in which the response is a number
or can be interpreted as a number with equal
values among the data points is ratio.

20. For example, ratio data can
include:
• What is your age? _______
• Five point likert scales (1=Very likely, 2 =
likely, etc. Usually likert scales are
illustrated on the survey instrument as 1
2 3 4 5. Respondents are asked to circle
the best response.
• Scores on standardized instruments are
also interpreted as ratio data.