Descriptive (1)

Educational Research:Educational Research:
Descriptive ResearchDescriptive Research
EDU 8603
Educational Research
Richard M. Jacobs, OSA, Ph.D.

Research...Research...
The systematic application of a
family of methods employed to
provide trustworthy information
about problems
…an ongoing process based on many accumulated
understandings and explanations that, when taken
together, lead to generalizations about problems
and the development of theories

Quantitative research methods...Quantitative research methods...
…collect and analyze numerical datadata
obtained from formal instrumentsinstruments

Quantitative methods...Quantitative methods...
descriptive research (“survey research”)
correlational research
causal-comparative research (“ex post facto research”)
experimental research

descriptive researchdescriptive research (“survey research”)
…collects data in order to answer
questions about the current status of
the subject or topic of study
…uses formal instruments to study
preferences, attitudes, practices,
concerns, or interests of a samplesample

The basic steps of descriptiveThe basic steps of descriptive
research...research...
An orderly scientific and disciplined
process, involving:
selecting an appropriate sample of
participants
collecting valid and reliable data
reporting conclusions
recognizing and identifying a topic to
be studied

Classifications of descriptiveClassifications of descriptive
research studies...research studies...
cross-sectional surveycross-sectional survey…
…involves collecting data from
selected individuals in a single
time period however long it
takes to collect data from
participants

longitudinal surveylongitudinal survey…
…involves collecting data at two or more
instances in order to measure changes
over time

self-report surveyself-report survey…
…requires individuals to respond to a
series of statements or questions
about themselves

observation studyobservation study…
…the researcher obtains data by
watching participants in a situation

Types of longitudinal surveys...Types of longitudinal surveys...
trend survey
cohort survey
panel survey
follow-up survey

trend surveytrend survey
…a study where a sample is taken from
the general population in order to
collect data over time
…involves different groups and
different samples over time

cohort surveycohort survey
…a study where a specific population is
examined by sampling different groups
within the population over time
…involves the same group but different
samples from that group over time

panel surveypanel survey
…a study where the same group and the
same sample are examined over time

follow-up surveyfollow-up survey
…a study undertaken after a panel survey
and seeks to examine subsequent
development or change

Conducting a questionnaire study…Conducting a questionnaire study…
1. state the problem
2. select participants
3. construct the questionnaire
4. prepare cover letter
5. pretest questionnaire
6. follow up activities
7. analyze/report results

1. state the problem…
…topic must be of sufficient significance to motivate
potential respondents and justify the research effort in
the first place
…define topic in terms of specific
objectives indicating the kind of
information needed

2. select participants…
…use an appropriate sampling
technique
…participants must be able to provide the
desired information sought and willing
to provide it to the researcher

3. construct the instrument…
…should be attractive, brief, and easy
to fill out
…there are a variety of data collection methods that can
be used, including: questionnaires, interviews,
observations, and telephone calls

types of self-report items
…scaled
…ranked
…checklist
…free response

…include a brief statement describing the
study and its purpose at the top of the
instrument
…provide standardized directions
…organize items leaving sufficient
white space on instrument
Designing an instrument…Designing an instrument…

…place similar items together
…ask general, non-threatening items
first, moving into more specific, self-
revealing items

some do’s and don’t’s…
…construct items according to a set of
predetermined guidelines
…include only items relating directly
to the purpose of the study
…avoid jargon or any term or concept that
might mean different things to different
people

…each item should deal with a single
concept and be worded as clearly as
possible
…items should indicate a point of
reference beyond the self in order to
judge the self
…avoid ambiguous terms like several
or usually

…avoid touchy matters in items which
respondents might not respond to
honestly or at all
…avoid leading questions which suggest
that one response may be more
appropriate than another
…each item must be able to stand on
its own

…don’t jam items together
…don’t put the most important questions
at the end of the instrument

4. prepare the cover letter…
…a brief, neat, explanation of the
significance of the study and what is
being asked of the respondent and why
…addressed to the specific, potential
respondent

…an endorsement adds credibility
…guarantee anonymityanonymity or
confidentialityconfidentiality
…include a specific deadline date by
which the completed instrument
(“questionnaire”) is to be returned

…individually sign each letter
…include a self-addressed, stamped,
return envelope

5. pretest the instrument…
…conduct a pilot studypilot study to gather information about
deficiencies and suggestions for improving the
instrument
…provides greater content validity

6. follow-up activities…
…because first mailing results tend to be low (30% - 50%
response rate), researchers need strategies to
increase the response rate

initial follow-up strategies to increase response rate up to 20%…
…send out reminder postcard
…mail a second packet with a new, positively worded
cover letter and another self-addressed, stamped,
return envelope

secondary follow-up strategies to increase response rate up to 10%…
…telephone nonrespondents to
encourage them to respond

dealing with nonrespondents
…study whether nonrespondents differ from
respondents in some systematic way by selecting a
sample from the nonrespondents
…telephone sample, aggregate data,
include in report

dealing with nonresponses
…study the items to determine the
problem with the item
…include description of the nonresponses
and the determination of the reason in
the report

7. analyze/report results…
…discriminant item analysisitem analysis includes giving the response rate for each
item as well as the total sample size and the overall percentage of
returns, since not all respondents will answer questions

…group items into clusters that address the same issue
and develop total scores across an item cluster in
order to avoid “information overload”

Conducting an interview study…Conducting an interview study…
3. construct the interview guide
4. communicate professionally and record responses accurately
5. pretest interview procedure

1. state the problem…
…topic must be of sufficient significance to motivate
individuals to participate and justify the research effort
in the first place
…define topic in terms of specific
objectives indicating the kind of
information needed

3. construct the interview instrument (“instrument guideinstrument guide”)…
…indicates the question to be asked, the
order, and how much additional
prompting or probing is permitted
…the goal is to obtain standardized,
comparable data from each interviewee

4. communicate professionally and record responses accurately…
…effective interviewing requires training and experience
to avoid interviewer bias and interviewer error

…record responses manually on the
interview guide
…(requires interviewee permission) use a
tape recorder or VCR to verify accuracy
of responses

5. pretest interview procedure…
…use a small group from the same population or a similar population to the one
being studied to validate the interview instrument and the interviewer’s
communication and recording skills

6. analyze/report results…
…item analysis includes giving the response rate for each item, as well
as the total sample size and the overall percentage of returns, since
not every respondent will answer all questions

…also include in-depth data to open-ended interview
items to portray responses in more accurate and
honest terms

Conducting an observational study…Conducting an observational study…
3. define the observational variables
4. record observations
5. assess observer reliability
6. reduce observer bias

3. define the observational variables…
…operationalize the variables to be
observed in terms of specific behaviors
that can be quantified
…ensure that the observations can be quantified in a
way so that all observers will count the behavioral
activities in exactly the same way

…simplify procedure for recording
observations by developing a codingcoding
procedureprocedure

4. record observations…
…record behavior at the time it occurs
…alternate observation periods and
recording periods

5. assess observer reliability…
…use at least two observers who
make independent observations
…compute interobserver reliabilityinterobserver reliability

6. reduce observer bias…
…train observers until a satisfactory level
of agreement is achieved (at least 80%)
…monitor observers

types of observer bias
…response set
…halo effect
…knowledge of participants

response setresponse set
…the tendency of an observer to rate the majority of
observees as above average, average, or below
average regardless of the observees’ actual behavior

halo effecthalo effect
…the tendency of an observer’s initial positive or
negative impressions of the observee to affect
subsequent observations

self-fulfilling prophecyself-fulfilling prophecy
…the tendency of an observer’s knowledge of the
observees or the purposes of the study to affect the
observation(s)

Meta-Analysis...Meta-Analysis...
a statistical approach to
summarizing the results of many
studies that have investigated
basically the same problem
…provides a numerical coefficient
expressing the “average” result
of the studies

…requiring the researcher to find, describe, classify, and code the
research studies to be included meta-analytic review, and for
measuring and analyzing the study findings

…each study’s results are translated into an effect sizeeffect size (ES) which
is a numerical expression of the strength or magnitude of a
reported relationship, be it causal or not

Xe – Xc
ES = 
SDc
Where Xe = the mean score of the experimental group
Xc = the mean score of the control group
SDc = the standard deviation of the control group

Strength or magnitude of an effectStrength or magnitude of an effect
size (size (∆∆)…)…
-.80 +.80
strong negative strong positive
control group
performed better
than experimental
group
experimental
group performed
better than
control group
0.00
both groups
performed
similarly

Mini-Quiz…Mini-Quiz…
True or false…
…in a descriptive research study, the researcher
predetermines what variables will be surveyed before
selecting or observing the research participants
True

…one of the most difficult activities on the part of
questionnaire researchers is to write or select
questions that are clear and unambiguous
True

…longitudinal surveys are useful for
studying the dynamics of a topic or
issue over time
True

…one of the problems with longitudinal
studies is that the samples tend to
shrink as time goes by
True

…one of the problems with cross-sectional
studies is selecting samples that truly represent
the population at a particular level or ability
True

…an external review of an instrument
provides the researcher greater
assurance of it its content validity
True

…if the responses from respondents and nonrespondents are essentially the same,
the researcher may assume that the response group is representative of the whole
sample and that the survey results are generalizable
True

…if the responses from respondents and nonrespondents are different,
the generalizability across both the respondent and nonrespondent
groups is not present and must be discussed in the report
True

…analyzing clusters of instrument items
related to the same issue improves the
reliability of the scores
True

…bias can affect the validity of the
interpretations in observational
studies
True

…although there are no hard and fast rules, it is generally agreed than an effect size in the
twenties indicates a treatment that produces a relatively small effect, whereas an effect
size in the eighties indicates a powerful treatment
True

Fill in the blank…
…a study requiring individuals to
respond to a series of statements or
questions about themselves
self-report study

…a study in which individuals are not directly asked for
information but data is obtained as the researcher
watches participants engage in a situation
observation study

…items on a survey that are answered
by circling a letter, checking a list, or
numbering preferences
close-ended items

…the researcher guarantees that no
one, including the researcher, knows
who completed each questionnaire
anonymity

…the researcher guarantees that no one, except the
researcher, knows who completed each questionnaire
and promises not to divulge that information
confidentiality

…the oral, in person administration of
an instrument to each member of a
sample
interview

…determining the current status of a phenomenon not
through an instrument but as the researcher watches the
participants engage in the activity being studied
observational study

…an observation in which the
researcher watches behavior as it
normally unfolds
naturalistic observation

…an observation in which the researcher
creates a situation to be observed and tells
participants what activities to engage in
simulation observation

…a coefficient found by dividing the total
number of agreed observations by the total
number of agreed and disagreed observations
inter-observer reliability

…a means by which observers record
observations at the time it occurs by
simplifying the recording process
coding

…a statistical approach that summarizes the
results of many studies having investigate the
same problem
meta-analysis

This module has focused on...This module has focused on...
…which involves collecting data in
order to test hypotheses or to
answer questions about the
opinions of people about some
topic or issue
descriptive research

The next module will focus on...The next module will focus on...
...which involves collecting data in
order to determine whether, and to
what degree, a relationship exists
between two or more quantifiable
variables
correlational research

Descriptive (1)

More Related Content

What's hot

Similar to Descriptive (1)

Recently uploaded

Descriptive (1)