PA 550: Research Methods Week 1 - 1/3 Andrew (Andy) Pattison S What is Research? Why Take a Class in Research Methods? Evidence-based Argumentation Induction: learning from the world by observation The Scientific Method Mendel & pea pods Modern medicine, Biochemistry, Physics Social Science Research Behavior of people: economics, sociology, anthropology Public Administration & Public Policy Do people use more public transit when unemployment rates go up? Make predictions based on evidence, as opposed to assumptions Ask questions Surveys, interviews? How many people? Which people? The challenge to the public manager School principal, fire chief, emergency manager Types of Research Descriptive Research: Who? What? When? How many? Example: How much homelessness exists in Ventura County? Explanatory Research: Why? How? Example: Why is there so much homelessness in Ventura County? Causation & Correlation Causal Inference: X causes Y Example: Will a proposed policy (X) actually reduce homelessness in Ventura County (Y)? Conditions of Causality: Temporal Order: X comes before Y Correlation: There is a pattern Theory: There is an argument as to why X COULD cause Y No spuriousness: No other factors (Z) are the actual cause Direct “Positive” Correlation As X increases, Y increases Inverse “Negative” Correlation As X increases, Y decreases Car Correlation Examples PA 550: Research Methods Week 1 - 3/3 Andrew (Andy) Pattison A Broad Outline for the Class Foundations of research design Data collection approaches Interviews, field research, surveys Data analysis approaches Quantitative and non-quantitative approaches to analysis Post-analysis issues Management of projects and presenting results Basic Elements and Concepts in Research Clarifying the Purpose of Research and Evaluation Projects Conceptualization and Operationalization Project Objectives Theory and Prior Work Understanding What to Study and What to Measure Unit of Analysis The entities (objects or events) being examined by a researcher Variables A characteristic of a unit of analysis that is not constant but instead varies across individual observed cases Research Questions and Hypotheses Asking Research Questions (RQ) A project’s central RQ helps guide a series of more specific sub-questions The nature of a core RQ relates to the basic purpose of the project: Descriptive purpose produces “what” or “how” questions Explanatory purpose produces causality questions Writing research Hypotheses “A proposition indicating how two or more factors are related to one another, and stated in a way that it can be tested with empirical evidence” Empirical, Generalizable, Plausible, Precise Approaches or Strategies Qualitative Quantitative Mixed Methods Ethics and the Research Process Protection of Human Subjects The case of the Tuskegee Syphilis Study The Belmont Report Institutional Review Boards Chapter 4: Conducting Preli.

1. PA 550:
Research Methods
Week 1 - 1/3
Andrew (Andy) Pattison
S
What is Research?
Why Take a Class in Research Methods?
Evidence-based Argumentation
Induction: learning from the world by observation
The Scientific Method
Mendel & pea pods
Modern medicine, Biochemistry, Physics
Social Science Research
Behavior of people: economics, sociology, anthropology
Public Administration & Public Policy
Do people use more public transit when unemployment rates go
up?
Make predictions based on evidence, as opposed to assumptions
Ask questions
Surveys, interviews?
How many people?
Which people?
The challenge to the public manager
School principal, fire chief, emergency manager
Types of Research

2. Descriptive Research: Who? What? When? How many?
Example:
How much homelessness exists in Ventura County?
Explanatory Research: Why? How?
Example:
Why is there so much homelessness in Ventura County?
Causation & Correlation
Causal Inference: X causes Y
Example:
Will a proposed policy (X) actually reduce homelessness in
Ventura County (Y)?
Conditions of Causality:
Temporal Order: X comes before Y
Correlation: There is a pattern
Theory: There is an argument as to why X COULD cause Y
No spuriousness: No other factors (Z) are the actual cause
Direct “Positive” Correlation
As X increases, Y increases
Inverse “Negative” Correlation
As X increases, Y decreases
Car Correlation Examples

3. PA 550:
Research Methods
Week 1 - 3/3
Andrew (Andy) Pattison
A Broad Outline for the Class
Foundations of research design
Data collection approaches
Interviews, field research, surveys
Data analysis approaches
Quantitative and non-quantitative approaches to analysis
Post-analysis issues
Management of projects and presenting results
Basic Elements and Concepts in Research
Clarifying the Purpose of Research and Evaluation Projects
Conceptualization and Operationalization
Project Objectives
Theory and Prior Work
Understanding What to Study and What to Measure
Unit of Analysis
The entities (objects or events) being examined by a researcher
Variables
A characteristic of a unit of analysis that is not constant but
instead varies across individual observed cases

4. Research Questions and Hypotheses
Asking Research Questions (RQ)
A project’s central RQ helps guide a series of more specific
sub-questions
The nature of a core RQ relates to the basic purpose of the
project:
Descriptive purpose produces “what” or “how” questions
Explanatory purpose produces causality questions
Writing research Hypotheses
“A proposition indicating how two or more factors are related to
one another, and stated in a way that it can be tested with
empirical evidence”
Empirical, Generalizable, Plausible, Precise
Approaches or Strategies
Qualitative
Quantitative
Mixed Methods
Ethics and the Research Process
Protection of Human Subjects
The case of the Tuskegee Syphilis Study
The Belmont Report
Institutional Review Boards

5. Chapter 4: Conducting Preliminary Research
Overview:
Starting Points in a Research Project
Getting Started with a Literature Review
Gathering Background Information
Data Gathering—Preview
Ethical Considerations
Chapter Learning Objectives
Understand the initial tasks of research and evaluation,
Writing a literature review
Creating an annotated bibliography
Understand the techniques used in obtaining general background
information
Conducting preliminary interviews with key informants or
subject matter experts
Identify and understand basic concepts of types of data
gathering techniques
Getting Started with a Literature Review
Purpose of a Literature Review
Substantive introduction
Explanation of importance
Context-setting
Annotated Bibliography
A comprehensive listing of major research articles, reports,
books, and other similar sources of information about a specific
topic
Source Materials

6. Gathering Background Information
Conducting Background Interviews with Subject Matter Experts
(SMEs)
Persons with specialized knowledge, expertise, and experience
within a particular policy domain or program area
The Value of SME Interviews
Identify core policy issues and challenges
Identify core research challenges
Assess critical questions
Data Gathering—Preview
Field Research
Surveys
Existing Data Sets
Creating an Original Data Set
Ethical Considerations
The importance of confidentiality
The importance of transparency
Understanding the need to balance competing values in the
research process
Chapter 5: Research Design
Overview:
Research Design
Basic Concepts for Research Design
Issues of Validity and Inference
Research Design Types

7. Ethics and Research Design
Chapter 5 Learning Objectives
Understand core concepts related to the design of an evaluative
project
Understand the different types of research design
Experimental
Quasi-experimental
Identify key issues of validity
Across a range of research design types
Identify ethical issues with the research design choices
Research Design
Basic challenge:
Develop an understanding of exactly how to approach the data
collection process when attempting to evaluate or assess some
program or policy
Research design:
Refers to the general process by which data gathering efforts
are structured and defined; that is, what is to be studied and
how, what variables are to be included in the study, how they
are measured in relation to one another, and how those data are
gathered
Basic Concepts for Research Design
Experiment:
An activity where a researcher controls or manipulates the
conditions under which some sort of subject is examined in
order to observe and measure a specific cause-and-effect
relationship
Treatment:

8. A variable or condition that the researcher introduces into the
experiment in order to see whether it has an effect on the
subjects
Control group:
Those subjects in the experiment that do not receive the
treatment
Issues of Validity and Inference
Internal validity:
Did you actually measure (accurately and validly) the concepts
and variables you intended to?
Did you account for, or rule out, competing explanations?
Understanding the nature of internal validity, and threats to it,
are critical to drawing causal inferences
Threats to Internal Validity
Threat of history
Threat of maturation
Threat of testing
Threat of instrumentation
Threat of regression to the mean
Threat of selection bias
Threat of mortality
Threats to External Validity
External validity:
Can the results can be generalized to other settings beyond the
specific sample gathered for the purposes of conducting a given
study?

9. Threats to external validity:
Involve interaction of the nature of the study (and its treatment)
with specific aspects of conducting an experiment (i.e., an
interaction between treatment and testing)
Research Design Types
True experiments
Quasi-experiments
Nonexperiments
Ethics and Research Design
Issues of potential bias and conflicts of interest in the conduct
of a research or evaluation study is of paramount concern
Issues of subject selection and subject inclusion
American Evaluation Association and the guiding principles for
the conduct of evaluation research
Chapter 6: Measurement Chapter Overview
Measurement in Research
Understanding the Basics of Measurement
Levels of Measurement
Determining the Level of Measurement
Issues of Measurement Error
Assessing Measurement Reliability and Validity

10. Chapter Learning Objectives
Understand the basic issues of measurement in social science
research
Understand the core concept of levels of measurement
Learn to determine the appropriate level of measurement for a
variable
Understand and identify the nature and sources of measurement
error
Understanding the Basics of Measurement
Operationalization
Translating a concept of interest into a form that can be
measured
Types of Variables
Qualitative vs. Quantitative measures
Dependent vs. Independent variables
Levels of Measurement
Nominal Variables
Race/ethnicity, colors
Ordinal Variables
Agree / disagree scales, ranked orders, variables with discrete
categories
Interval (ratio) Variables
Temperature, age, income, any continuous variable
Determining the Level of Measurement
Challenges in recognizing which is the actual level of
measurement for a given variable
Challenges in knowing which is the appropriate level of
measurement to use for a variable in a given research situation

11. Issues of Measurement Error
Sources of measurement error:
Systematic measurement error: occurs when factors create error
in the measuring process itself or errors influencing the concept
being measured
Random measurement error: an error that is not directly related
to the process of measuring a concept
Assessing Measurement Reliability and Validity
Validity assessment approach
Face validity
Content validity
Reliability assessment approach
Test—Retest
Intercoder checks