The document provides an overview of research methods for a final exam review. It discusses the exam format, guidelines for constructing surveys, different question types, ordering questions, self-administered questionnaires, and analyzing data. It also covers field research, agency records, secondary data analysis, content analysis, evaluation research, experimental designs, and interpreting findings. The exam will include multiple choice and short answer questions covering these topics on conducting research and analyzing data.

1. MONDAY 13TH MAY 2013
Research Methods
Finals Exam Review

2. The Exam
 Is worth 15% of your grade
 Will include:
- 40 multiple choice questions
- 3 short answer questions

3. How to ask Questions
 Open-ended
 Respondent is asked to provide his or her own answer
 Produces narrative data
 Closed-ended
 Respondent selects an answer from a list
 Choices should be exhaustive and mutually exclusive
 Produces data that can be readily processed

4. Guidelines for Asking Questions
 Make items clear – avoid ambiguous questions; do
not ask “double-barreled” questions
 Short items are best – respondents like to read and
answer a question quickly
 Avoid negative items – leads to misinterpretation
 Avoid biased items and terms – do not ask questions
that encourage a certain answer
 Avoid terms that have bad associations

5. Questionnaire Construction
 General questionnaire format – critical, must be laid
out properly
 Matrix questions – same set of answer categories
used by multiple questions
 Contingency questions – relevant only to some
respondents – answered only based on their previous
response

6. Ordering Questions
 Ordering may affect the answers given
 Estimate the effect of question order
 Perhaps devise more than one version
 Depending on whether it is in-person, or via mail,
internet etc.
 Begin with most interesting questions
 End with less interesting, demographic data

7. Self-Administered Questionnaires
 Can be home-delivered
 Researcher delivers questionnaire to home of respondent,
explains the study, and then comes back later
 Mailed (sent and returned) survey is most common
 Researchers must reduce the trouble it takes to return a
questionnaire

8. Warning Mailings and Cover Letters
 Used to increase response rates
 Warning mailings – “address correction requested”
card sent out to determine incorrect addresses and to
“warn” residents to expect questionnaire in mail
 Cover letters – detail why survey is being conducted,
why respondent was selected, why is it important to
complete questionnaire
 Follow-ups to mail surveys increase response rates

9. Survey Research
 Surveys are best suited for studies that have
individuals as their units of analysis
 Strengths:
 Surveys tend to be high on reliability and generalizability, but
validity can often be a weak point
 Surveys useful in describing characteristics of large population

10. Survey Research
 Weaknesses
 Standardized questionnaire items often represent the
least common denominator in assessing people‟s
attitudes, orientations, circumstances, and experiences
 Surveys often appear superficial in their coverage of
complex topics
 Difficult populations are hard to contact through
customary sampling methods
 Generally weaker on validity and stronger on reliability
 Subject to recall error
 Social desirability may be a problem

11. What is Field Research?
 Field research is usually associated with qualitative
data
 It encompasses two different methods of obtaining
data:
 Direct observation
 Asking questions
 Often no precisely defined hypotheses to be tested
 Used to make sense out of an ongoing process
 Questions related to field research tend to be „How?‟
and „Why?‟

12. Characteristics of Qualitative Research
 Context is critical
 In-depth and detailed
 Researcher immerses self in data
 Bias is recognized
 Inductive rather than deductive (bottom up)
 Data are interpreted rather than analyzed

13. Roles of the Observer in Field Research
 Full participant (e.g., a participant in a
demonstration against stop and frisk)
 Participant as observer (e.g., Perrone‟s research on
drug use in New York dance clubs)
 Observer as participant (e.g., observational studies of
the police)
 Complete observer (e.g., research in a courtroom
setting, or setting that is open to the public)

14. Recording Observations
 Note-taking, tape recording when interviewing and
when making observations (dictation device)
 Videotaping or photographs can make records of
“before” and “after” some physical design change
 Field notes – observations are recorded as written
notes, often in a field journal; first take sketchy notes
and then rewrite your notes in detail
 Structured observations – observers mark closed-
ended forms, which produce numeric measures

15. Strengths and Weaknesses of Field Research
Strengths:
 Provides great depth of understanding
 Flexibility (no need to prepare much in advance)
 More appropriate to measure behavior than surveys
 High on validity
Weaknesses:
 Low on reliability – often very personal
 Generalizability – personal nature may produce
findings that may not be replicated by another
 Precise probability samples can‟t normally be drawn

16. Agency Records
 Published Statistics – government organizations
routinely collect and publish compilations of data
(e.g., NCVS, Census Bureau, BJS; often available in
libraries and online
 Nonpublic Agency Records – agencies produce data
not routinely released (e.g., police departments,
courthouses, correctional facilities)
 New Data Collected by Agency Staff – collected for
specific research purposes; less costly than collecting
the data yourself and more control
 Agency records are often used for descriptive studies

17. Problems with Reliability and Validity
 Virtually all criminal justice record keeping is a social
process – “social production of data”
 Records reflect decisions made by CJ personnel as well as actual
behavior by juveniles and adults
 Discretion factors in keeping records
 Criminal justice organizations are often more interested in
keeping track of individual cases than in examining
patterns
 Potential for clerical errors due to volume of data
 Users of data series collected over time must be especially
careful
 Imperative that you understand how the data were collected

18. Secondary Data Analysis
 Sources – websites (BJS, NCVS, ICPSR, NACJD),
libraries
 Advantages
 Cheaper
 Faster
 Benefit from work of skilled researchers
 Disadvantages
 Data may not be appropriate to your research question
 Not useful for evaluation studies (which are designed to answer
specific questions about specific programs)
 Threats to validity

19. Content Analysis
 Systematic study of messages – can be applied to
virtually any form of communication
 Decide on operational definitions of key variables
 Decide what to watch, read, listen to & time frame of
documents
 Analyze collected data
 Well-suited to answer “Who says what, to whom,
why, how, and with what effect?”

20. Content Analysis
 Systematic study of messages – can be applied to
virtually any form of communication
 Decide on operational definitions of key variables
 Decide what to watch, read, listen to & time frame of
documents
 Analyze collected data
 Well-suited to answer “Who says what, to whom,
why, how, and with what effect?”
 Examines content and meaning

21. Content Analysis
 Essentially a coding operation
 Communications need to be coded according to some
conceptual framework
 Choice between depth & specificity of understanding:
 Manifest content – visible, surface content – similar to
using closed-ended survey questions
 Latent content – underlying meaning

22. Evaluation Research
 It is gaining in popularity among researchers that
really want to make a difference
 Federal requirements for program evaluations often
accompany the implementation of new programs
 Funding is available for program evaluations
 Can utilize a number of different research designs
 Links the intended actions and goals of criminal
justice policy to empirical evidence that supports
them having the desired effects

23. Conditions Requisite for Randomized Experiments
 Staff must accept random assignment and agree to
minimize exceptions to randomization
 Case flow must produce enough subjects in
experimental and control groups for statistical tests
 Experimental interventions must be consistently
applied to experimental and withheld from control
group
 Need equivalence prior to intervention, and ability to
detect differences in outcome measures after
intervention

24. The Policy Process
 Begins with a demand for a new course of action or
opposition to an existing policy
 Policy makers consider their ultimate goals and
means of achieving those goals
 Resources are allocated (considerations include;
personnel, equipment, supplies etc.)
 What are the policy outputs (i.e. what is actually
produced?)
 What is the impact of the policy output?

25. Interpreting Data
 Empirical research is a logical rather than a
mathematical operation
 Statistics – branch of math appropriate to research
 Descriptive statistics – used to summarize and
describe data in manageable forms
 Inferential statistics – assist in forming conclusions
from our observations; usually about a population
based on studying a sample

26. Types of Analysis
 Univariate analysis – describing single variables (e.g.
# of males; average age; place of birth etc.)
 Bivariate analysis – describes the associations that
connect one variable with another
 Multivariate analysis – examines relationships
among three or more variables

27. Data Analysis
 Central tendency - mean, median, mode
 Means are susceptible to extreme values. A few very large, or a
few very small numbers can change the mean dramatically
 Because of this, it is important to examine measures of
dispersion
 Simplest measure of dispersion is the range – the distance
from the highest to the lowest value (e.g. 13 to 19 years)
 Standard deviation – the average amount of variability in a set
of scores (i.e. the average amount each individual observation
varies from the mean)
 The larger the standard deviation, the larger the average
distance each data point is from the mean of the distribution

28. Rates
 Fundamental descriptive statistics in criminal justice
research
 Used to standardize some measure for comparative purposes
Total Murders in Four States, 2004
Total Murders Total Population
California 2,407 35,894,000
Florida 946 17,397,000
Louisiana 574 4,516,000
Pennsylvania 650 12,406,000

29. Short Answer Questions
1. Compare and contrast the strengths and
weaknesses of field research and survey
research. Give specific attention to the topics of
validity, reliability and generalizability
2. Discuss the advantages and disadvantages of
secondary data and give an example of each
3. Identify the difference between the various roles
of the observer in field research. Give an
example of a research scenario that would be
appropriate for each role

30. Final Class
 Final Exam
 Article critiques due