Download to read offline
![ Writing a Policy Memo
Writing a Research Proposal
Acknowledgements
Introduction
Before beginning your paper, you need to decide how you plan to design the study.
The research design refers to the overall strategy that you choose to integrate the different
components of the study in a coherent and logical way, thereby, ensuring you will
effectively address the research problem; it constitutes the blueprint for the collection,
measurement, and analysis of data. Note that your research problem determines the type of
design you should use, not the other way around!
De Vaus, D. A. Research Design in Social Research. London: SAGE, 2001; Trochim, William M.K.
Research Methods Knowledge Base. 2006.
General Structure and Writing Style
The function of a research design is to ensure that the evidence obtained enables you
to effectively address the research problem logically and as unambiguously as
possible. In social sciences research, obtaining information relevant to the research
problem generally entails specifying the type of evidence needed to test a theory, to
evaluate a program, or to accurately describe and assess meaning related to an observable
phenomenon.
With this in mind, a common mistake made by researchers is that they begin their
investigations far too early, before they have thought critically about what information is
required to address the research problem. Without attending to these design issues
beforehand, the overall research problem will not be adequately addressed and any
conclusions drawn will run the risk of being weak and unconvincing. As a consequence,
the overall validity of the study will be undermined.
The length and complexity of describing research designs in your paper can vary
considerably, but any well-developed design will achieve the following:
1. Identify the research problem clearly and justify its selection, particularly in
relation to any valid alternative designs that could have been used,
2. Review and synthesize previously published literature associated with the research
problem,
3. Clearly and explicitly specify hypotheses [i.e., research questions] central to the
problem,
4. Effectively describe the data which will be necessary for an adequate testing of the
hypotheses and explain how such data will be obtained, and](https://image.slidesharecdn.com/researchmethodology-170425115822/75/Research-methodology-2-2048.jpg)
![4. Action research studies often have direct and obvious relevance to improving
practice and advocating for change.
5. There are no hidden controls or preemption of direction by the researcher.
What these studies don't tell you?
1. It is harder to do than conducting conventional research because the researcher
takes on responsibilities of advocating for change as well as for researching the
topic.
2. Action research is much harder to write up because it is less likely that you can use
a standard format to report your findings effectively [i.e., data is often in the form
of stories or observation].
3. Personal over-involvement of the researcher may bias research results.
4. The cyclic nature of action research to achieve its twin outcomes of action (e.g.
change) and research (e.g. understanding) is time-consuming and complex to
conduct.
5. Advocating for change requires buy-in from participants.
Coghlan, David and Mary Brydon-Miller. The Sage Encyclopedia of Action Research. Thousand Oaks,
CA: Sage, 2014; Efron, Sara Efrat and Ruth Ravid. Action Research in Education:A Practical Guide. New
York: Guilford, 2013; Gall, Meredith. Educational Research:An Introduction.Chapter18, Action Research.
8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007; Gorard, Stephen. Research Design: Creating Robust
Approaches for the Social Sciences. Thousand Oaks, CA: Sage, 2013; Kemmis, Stephen and Robin
McTaggart. “Participatory Action Research.” In Handbook of Qualitative Research. Norman Denzin and
Yvonna S. Lincoln, eds. 2nd ed. (Thousand Oaks, CA: SAGE, 2000), pp. 567-605; McNiff, Jean. Writing
and Doing Action Research. London: Sage, 2014; Reason, Peter and Hilary Bradbury. Handbook of Action
Research: Participative Inquiry and Practice. Thousand Oaks, CA: SAGE, 2001.
Case Study Design
Definition and Purpose
A case study is an in-depth study of a particular research problem rather than a sweeping
statistical survey or comprehesive comparative inquiry. It is often used to narrow down a
very broad field of research into one or a few easily researchable examples. The case study
research design is also useful for testing whether a specific theory and model actually
applies to phenomena in the real world. It is a useful design when not much is known about
an issue or phenomenon.
What do these studies tell you?
1. Approach excels at bringing us to an understanding of a complex issue through
detailed contextual analysis of a limited number of events or conditions and their
relationships.](https://image.slidesharecdn.com/researchmethodology-170425115822/75/Research-methodology-4-2048.jpg)
![ Empirical association -- a valid conclusion is based on finding an association
between the independent variable and the dependent variable.
Appropriate time order -- to conclude that causation was involved, one must see
that cases were exposed to variation in the independent variable before variation in
the dependent variable.
Nonspuriousness -- a relationship between two variables that is not due to variation
in a third variable.
What do these studies tell you?
1. Causality research designs assist researchers in understanding why the world works
the way it does through the process of proving a causal link between variables and
by the process of eliminating other possibilities.
2. Replication is possible.
3. There is greater confidence the study has internal validity due to the systematic
subject selection and equity of groups being compared.
What these studies don't tell you?
1. Not all relationships are casual! The possibility always exists that, by sheer
coincidence, two unrelated events appear to be related [e.g., Punxatawney Phil
could accurately predict the duration of Winter for five consecutive years but, the
fact remains, he's just a big, furry rodent].
2. Conclusions about causal relationships are difficult to determine due to a variety of
extraneous and confounding variables that exist in a social environment. This
means causality can only be inferred, never proven.
3. If two variables are correlated, the cause must come before the effect. However,
even though two variables might be causally related, it can sometimes be difficult
to determine which variable comes first and, therefore, to establish which variable
is the actual cause and which is the actual effect.
Beach, Derek and Rasmus Brun Pedersen. Causal Case Study Methods: Foundations and Guidelines for
Comparing, Matching,and Tracing. Ann Arbor, MI: University of Michigan Press, 2016; Bachman, Ronet.
The Practice of Research in Criminology and Criminal Justice.Chapter 5, Causation and Research Designs.
3rd ed. Thousand Oaks, CA: Pine Forge Press, 2007; Brewer, Ernest W. and Jennifer Kubn. “Causal-
Comparative Design.” In Encyclopedia of Research Design. Neil J. Salkind, editor. (Thousand Oaks, CA:
Sage, 2010), pp. 125-132; Causal Research Design: Experimentation. Anonymous SlideShare Presentation;
Gall, Meredith. Educational Research: An Introduction. Chapter 11, Nonexperimental Research:
Correlational Designs. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007; Trochim, William M.K.
Research Methods Knowledge Base. 2006.
Cohort Design
Definition and Purpose](https://image.slidesharecdn.com/researchmethodology-170425115822/75/Research-methodology-6-2048.jpg)
![Often used in the medical sciences, but also found in the applied social sciences, a cohort
study generally refers to a study conducted over a period of time involving members of a
population which the subject or representative member comes from, and who are united by
some commonality or similarity. Using a quantitative framework, a cohort study makes
note of statistical occurrence within a specialized subgroup, united by same or similar
characteristics that are relevant to the research problem being investigated, rather than
studying statistical occurrence within the general population. Using a qualitative
framework, cohort studies generally gather data using methods of observation. Cohorts can
be either "open" or "closed."
Open Cohort Studies [dynamic populations, such as the population of Los Angeles]
involve a population that is defined just by the state of being a part of the study in
question (and being monitored for the outcome). Date of entry and exit from the
study is individually defined, therefore, the size of the study population is not
constant. In open cohort studies, researchers can only calculate rate based data, such
as, incidence rates and variants thereof.
Closed Cohort Studies [static populations, such as patients entered into a clinical
trial] involve participants who enter into the study at one defining point in time and
where it is presumed that no new participants can enter the cohort. Given this, the
number of study participants remains constant (or can only decrease).
What do these studies tell you?
1. The use of cohorts is often mandatory because a randomized control study may be
unethical. For example, you cannot deliberately expose people to asbestos, you can
only study its effects on those who have already been exposed. Research that
measures risk factors often relies upon cohort designs.
2. Because cohort studies measure potential causes before the outcome has occurred,
they can demonstrate that these “causes” preceded the outcome, thereby avoiding
the debate as to which is the cause and which is the effect.
3. Cohort analysis is highly flexible and can provide insight into effects over time and
related to a variety of different types of changes [e.g., social, cultural, political,
economic, etc.].
4. Either original data or secondary data can be used in this design.
What these studies don't tell you?
1. In cases where a comparative analysis of two cohorts is made [e.g., studying the
effects of one group exposed to asbestos and one that has not], a researcher cannot
control for all other factors that might differ between the two groups. These factors
are known as confounding variables.
2. Cohort studies can end up taking a long time to complete if the researcher must wait
for the conditions of interest to develop within the group. This also increases the
chance that key variables change during the course of the study, potentially
impacting the validity of the findings.](https://image.slidesharecdn.com/researchmethodology-170425115822/75/Research-methodology-7-2048.jpg)
![1. Finding people, subjects, or phenomena to study that are very similar except in one
specific variable can be difficult.
2. Results are static and time bound and, therefore, give no indication of a sequence
of events or reveal historical or temporal contexts.
3. Studies cannot be utilized to establish cause and effect relationships.
4. This design only provides a snapshot of analysis so there is always the possibility
that a study could have differing results if another time-frame had been chosen.
5. There is no follow up to the findings.
Bethlehem, Jelke. "7: Cross-sectional Research." In Research Methodology in the Social, Behavioural and
Life Sciences. Herman J Adèr and Gideon J Mellenbergh, editors. (London, England: Sage, 1999), pp. 110-
43; Bourque, Linda B. “Cross-Sectional Design.” In The SAGE Encyclopedia of Social Science Research
Methods.Michael S. Lewis-Beck, Alan Bryman, and Tim Futing Liao. (Thousand Oaks, CA: 2004), pp. 230-
231; Hall, John. “Cross-Sectional Survey Design.” In Encyclopedia of Survey Research Methods. Paul J.
Lavrakas, ed. (Thousand Oaks, CA: Sage, 2008), pp. 173-174; Helen Barratt, Maria Kirwan. Cross-Sectional
Studies: Design,Application, Strengths and Weaknesses ofCross-SectionalStudies.Healthknowledge, 2009.
Cross-Sectional Study. Wikipedia.
Descriptive Design
Definition and Purpose
Descriptive research designs help provide answers to the questions of who, what, when,
where, and how associated with a particular research problem; a descriptive study cannot
conclusively ascertain answers to why. Descriptive research is used to obtain information
concerning the current status of the phenomena and to describe "what exists" with respect
to variables or conditions in a situation.
What do these studies tell you?
1. The subject is being observed in a completely natural and unchanged natural
environment. True experiments, whilst giving analyzable data, often adversely
influence the normal behavior of the subject [a.k.a., the Heisenberg effect whereby
measurements of certain systems cannot be made without affecting the systems].
2. Descriptive research is often used as a pre-cursor to more quantitative research
designs with the general overview giving some valuable pointers as to what
variables are worth testing quantitatively.
3. If the limitations are understood, they can be a useful tool in developing a more
focused study.
4. Descriptive studies can yield rich data that lead to important recommendations in
practice.
5. Appoach collects a large amount of data for detailed analysis.
What these studies don't tell you?](https://image.slidesharecdn.com/researchmethodology-170425115822/75/Research-methodology-9-2048.jpg)
![1. The results from a descriptive research cannot be used to discover a definitive
answer or to disprove a hypothesis.
2. Because descriptive designs often utilize observational methods [as opposed to
quantitative methods], the results cannot be replicated.
3. The descriptive function of research is heavily dependent on instrumentation for
measurement and observation.
Anastas,Jeane W. Research Design for Social Work and the Human Services. Chapter 5, Flexible Methods:
Descriptive Research. 2nd ed. New York: Columbia University Press, 1999; Given, Lisa M. "Descriptive
Research." In Encyclopedia of Measurement and Statistics.Neil J. Salkind and Kristin Rasmussen, editors.
(Thousand Oaks, CA: Sage, 2007), pp. 251-254; McNabb, Connie. Descriptive Research Methodologies.
Powerpoint Presentation; Shuttleworth, Martyn. Descriptive Research Design, September 26, 2008.
Explorable.com website.
Experimental Design
Definition and Purpose
A blueprint of the procedure that enables the researcher to maintain control over all factors
that may affect the result of an experiment. In doing this, the researcher attempts to
determine or predict what may occur. Experimental research is often used where there is
time priority in a causal relationship (cause precedes effect), there is consistency in a causal
relationship (a cause will always lead to the same effect), and the magnitude of the
correlation is great. The classic experimental design specifies an experimental group and a
control group. The independent variable is administered to the experimental group and not
to the control group, and both groups are measured on the same dependent variable.
Subsequent experimental designs have used more groups and more measurements over
longer periods. True experiments must have control, randomization, and manipulation.
What do these studies tell you?
1. Experimental research allows the researcher to control the situation. In so doing, it
allows researchers to answer the question, “What causes something to occur?”
2. Permits the researcher to identify cause and effect relationships between variables
and to distinguish placebo effects from treatment effects.
3. Experimental research designs support the ability to limit alternative explanations
and to infer direct causal relationships in the study.
4. Approach provides the highest level of evidence for single studies.
What these studies don't tell you?
1. The design is artificial, and results may not generalize well to the real world.
2. The artificial settings of experiments may alter the behaviors or responses of
participants.
3. Experimental designs can be costly if special equipment or facilities are needed.](https://image.slidesharecdn.com/researchmethodology-170425115822/75/Research-methodology-10-2048.jpg)
![4. Exploratory research is often used to generate formal hypotheses and develop more
precise research problems.
5. In the policy arena or applied to practice, exploratory studies help establish research
priorities and where resources should be allocated.
What these studies don't tell you?
1. Exploratory research generally utilizes small sample sizes and, thus, findings are
typically not generalizable to the population at large.
2. The exploratory nature of the research inhibits an ability to make definitive
conclusions about the findings. They provide insight but not definitive conclusions.
3. The research process underpinning exploratory studies is flexible but often
unstructured, leading to only tentative results that have limited value to decision-
makers.
4. Design lacks rigorous standards applied to methods of data gathering and analysis
because one of the areas for exploration could be to determine what method or
methodologies could best fit the research problem.
Cuthill, Michael. “Exploratory Research: Citizen Participation, Local Government, and Sustainable
Development in Australia.” Sustainable Development 10 (2002): 79-89; Streb, Christoph K. "Exploratory
Case Study." In Encyclopedia ofCase Study Research. Albert J. Mills, Gabrielle Durepos and Eiden Wiebe,
editors. (Thousand Oaks, CA: Sage, 2010), pp. 372-374; Taylor, P. J., G. Catalano, and D.R.F. Walker.
“Exploratory Analysis of the World City Network.” Urban Studies 39 (December 2002): 2377-2394;
Exploratory Research. Wikipedia.
Historical Design
Definition and Purpose
The purpose of a historical research design is to collect, verify, and synthesize evidence
from the past to establish facts that defend or refute a hypothesis. It uses secondary sources
and a variety of primary documentary evidence, such as, diaries, official records, reports,
archives, and non-textual information [maps, pictures, audio and visual recordings]. The
limitation is that the sources must be both authentic and valid.
What do these studies tell you?
1. The historical research design is unobtrusive; the act of research does not affect the
results of the study.
2. The historical approach is well suited for trend analysis.
3. Historical records can add important contextual background required to more fully
understand and interpret a research problem.
4. There is often no possibility of researcher-subject interaction that could affect the
findings.](https://image.slidesharecdn.com/researchmethodology-170425115822/75/Research-methodology-12-2048.jpg)
![2. Enables survey researchers to get close to the kinds of causal explanations usually
attainable only with experiments.
3. The design permits the measurement of differences or change in a variable from
one period to another [i.e., the description of patterns of change over time].
4. Longitudinal studies facilitate the prediction of future outcomes based upon earlier
factors.
What these studies don't tell you?
1. The data collection method may change over time.
2. Maintaining the integrity of the original sample can be difficult over an extended
period of time.
3. It can be difficult to show more than one variable at a time.
4. This design often needs qualitative research data to explain fluctuations in the
results.
5. A longitudinal research design assumes present trends will continue unchanged.
6. It can take a long period of time to gather results.
7. There is a need to have a large sample size and accurate sampling to reach
representativness.
Anastas,Jeane W. Research Design for Social Work and the Human Services. Chapter 6, Flexible Methods:
Relational and Longitudinal Research. 2nd ed. New York: Columbia University Press, 1999; Forgues,
Bernard, and Isabelle Vandangeon-Derumez. "Longitudinal Analyses." In Doing Management Research.
Raymond-Alain Thiétart and Samantha Wauchope, editors. (London, England: Sage, 2001), pp. 332-351;
Kalaian, Sema A. and Rafa M. Kasim. "Longitudinal Studies." In Encyclopedia ofSurvey Research Methods.
Paul J. Lavrakas, ed. (Thousand Oaks, CA: Sage, 2008), pp. 440-441; Menard, Scott, editor. Longitudinal
Research. Thousand Oaks, CA: Sage, 2002; Ployhart, Robert E. and Robert J. Vandenberg. "Longitudinal
Research: The Theory, Design, and Analysis of Change.” Journal of Management 36 (January 2010): 94-
120; Longitudinal Study. Wikipedia.
Meta-Analysis Design
Definition and Purpose
Meta-analysis is an analytical methodology designed to systematically evaluate and
summarize the results from a number of individual studies, thereby, increasing the overall
sample size and the ability of the researcher to study effects of interest. The purpose is to
not simply summarize existing knowledge, but to develop a new understanding of a
research problem using synoptic reasoning. The main objectives of meta-analysis include
analyzing differences in the results among studies and increasing the precision by which
effects are estimated. A well-designed meta-analysis depends upon strict adherence to the
criteria used for selecting studies and the availability of information in each study to
properly analyze their findings. Lack of information can severely limit the type of analyzes
and conclusions that can be reached. In addition, the more dissimilarity there is in the
results among individual studies [heterogeneity], the more difficult it is to justify](https://image.slidesharecdn.com/researchmethodology-170425115822/75/Research-methodology-14-2048.jpg)
![Walker, Esteban, Adrian V. Hernandez, and Micheal W. Kattan. "Meta-Analysis: It's Strengths and
Limitations." Cleveland Clinic Journal of Medicine 75 (June 2008): 431-439.
Mixed-Method Design
Definition and Purpose
Mixed methods research represents more of an approach to examining a research problem
than a methodology. Mixed method is characterized by a focus on research problems that
require, 1) an examination of real-life contextual understandings, multi-level perspectives,
and cultural influences; 2) an intentional application of rigorous quantitative research
assessing magnitude and frequency of constructs and rigorous qualitative research
exploring the meaning and understanding of the constructs; and, 3) an objective of drawing
on the strengths of quantitative and qualitative data gathering techniques to formulate a
holistic interpretive framework for generating possible solutions or new understandings of
the problem. Tashakkori and Creswell (2007) and other proponents of mixed methods
argue that the design encompasses more than simply combining qualitative and quantitative
methods but, rather, reflects a new "third way" epistemological paradigm that occupies the
conceptual space between positivism and interpretivism.
What do these studies tell you?
1. Narrative and non-textual information can add meaning to numeric data, while
numeric data can add precision to narrative and non-textual information.
2. Can utilize existing data while at the same time generating and testing a grounded
theory approach to describe and explain the phenomenon under study.
3. A broader, more complex research problem can be investigated because the
researcher is not constrained by using only one method.
4. The strengths of one method can be used to overcome the inherent weaknesses of
another method.
5. Can provide stronger, more robust evidence to support a conclusion or set of
recommendations.
6. May generate new knowledge new insights or uncover hidden insights, patterns, or
relationships that a single methodological approach might not reveal.
7. Produces more complete knowledge and understanding of the research problem that
can be used to increase the generalizability of findings applied to theory or practice.
What these studies don't tell you?
1. A researcher must be proficient in understanding how to apply multiple methods to
investigating a research problem as well as be proficient in optimizing how to
design a study that coherently melds them together.
2. Can increase the likelihood of conflicting results or ambiguous findings that inhibit
drawing a valid conclusion or setting forth a recommended course of action [e.g.,
sample interview responses do not support existing statistical data].](https://image.slidesharecdn.com/researchmethodology-170425115822/75/Research-methodology-16-2048.jpg)
![3. Because the research design can be very complex, reporting the findings requires a
well-organized narrative, clear writing style, and precise word choice.
4. Design invites collaboration among experts. However, merging different
investigative approaches and writing styles requires more attention to the overall
research process than studies conducted using only one methodological paradigm.
5. Concurrent merging of quantitative and qualitative research requires greater
attention to having adequate sample sizes, using comparable samples, and applying
a consistent unit of analysis. For sequential designs where one phase of qualitative
research builds on the quantitative phase or vice versa, decisions about what results
from the first phase to use in the next phase, the choice of samples and estimating
reasonable sample sizes for both phases, and the interpretation of results from both
phases can be difficult.
6. Due to multiple forms of data being collected and analyzed, this design requires
extensive time and resources to carry out the multiple steps involved in data
gathering and interpretation.
Burch, Patricia and Carolyn J. Heinrich. Mixed Methods for Policy Research and Program Evaluation. Thousand Oaks, CA: Sage, 2016;
Creswell, John w. et al. Best Practices for Mixed Methods Research in the Health Sciences. Bethesda, MD: Office of Behavioral and Social
Sciences Research, National Institutes of Health, 2010Creswell, John W. Research Design: Qualitative, Quantitative, and Mixed Methods
Approaches. 4th edition. Thousand Oaks, CA: Sage Publications, 2014; Domínguez, Silvia, editor. Mixed Methods Social Networks Research.
Cambridge, UK: Cambridge University Press, 2014; Hesse-Biber, Sharlene Nagy. Mixed Methods Research: Merging Theory with Practice.
New York: Guilford Press, 2010; Niglas, Katrin. “ How the Novice Researcher Can Make Sense of Mixed Methods Designs.” International
Journal of Multiple Research Approaches 3 (2009): 34-46; Onwuegbuzie, Anthony J. and Nancy L. Leech. “ Linking Research Questions to
Mixed Methods Data Analysis Procedures.” The Qualitative Report 11 (September 2006): 474-498; Tashakorri, Abbas and John W. Creswell.
“ The New Era of Mixed Methods.” Journal of Mixed Methods Research 1 (January 2007): 3-7; Zhanga, Wanqing. “ Mixed Methods Application
in Health Intervention Research: A Multiple Case Study.” International Journal of Multiple Research Approaches 8 (2014): 24-35.
Observational Design
Definition and Purpose
This type of research design draws a conclusion by comparing subjects against a control
group, in cases where the researcher has no control over the experiment. There are two
general types of observational designs. In direct observations, people know that you are
watching them. Unobtrusive measures involve any method for studying behavior where
individuals do not know they are being observed. An observational study allows a useful
insight into a phenomenon and avoids the ethical and practical difficulties of setting up a
large and cumbersome research project.
What do these studies tell you?
1. Observational studies are usually flexible and do not necessarily need to be
structured around a hypothesis about what you expect to observe [data is emergent
rather than pre-existing].
2. The researcher is able to collect in-depth information about a particular behavior.
3. Can reveal interrelationships among multifaceted dimensions of group interactions.
4. You can generalize your results to real life situations.](https://image.slidesharecdn.com/researchmethodology-170425115822/75/Research-methodology-17-2048.jpg)
![ Epistemology -- the study that explores the nature of knowledge; for example, by
what means does knowledge and understanding depend upon and how can we be
certain of what we know?
Axiology -- the study of values; for example, what values does an individual or
group hold and why? How are values related to interest, desire, will, experience,
and means-to-end? And, what is the difference between a matter of fact and a matter
of value?
What do these studies tell you?
1. Can provide a basis for applying ethical decision-making to practice.
2. Functions as a means of gaining greater self-understanding and self-knowledge
about the purposes of research.
3. Brings clarity to general guiding practices and principles of an individual or group.
4. Philosophy informs methodology.
5. Refine concepts and theories that are invoked in relatively unreflective modes of
thought and discourse.
6. Beyond methodology, philosophy also informs critical thinking about
epistemology and the structure of reality (metaphysics).
7. Offers clarity and definition to the practical and theoretical uses of terms, concepts,
and ideas.
What these studies don't tell you?
1. Limited application to specific research problems [answering the "So What?"
question in social science research].
2. Analysis can be abstract, argumentative, and limited in its practical application to
real-life issues.
3. While a philosophical analysis may render problematic that which was once simple
or taken-for-granted, the writing can be dense and subject to unnecessary jargon,
overstatement, and/or excessive quotation and documentation.
4. There are limitations in the use of metaphor as a vehicle of philosophical analysis.
5. There can be analytical difficulties in moving from philosophy to advocacy and
between abstract thought and application to the phenomenal world.
Burton, Dawn. "Part I, Philosophy of the Social Sciences." In Research Training for Social Scientists.
(London, England: Sage, 2000), pp. 1-5; Chapter 4, Research Methodology and Design. Unisa Institutional
Repository (UnisaIR), University of South Africa; Jarvie, Ian C., and Jesús Zamora-Bonilla, editors. The
SAGE Handbook of the Philosophy of Social Sciences. London: Sage, 2011; Labaree, Robert V. and Ross
Scimeca. “The Philosophical Problem of Truth in Librarianship.” The Library Quarterly 78 (January 2008):
43-70; Maykut,Pamela S. Beginning Qualitative Research:A Philosophic and Practical Guide.Washington,
DC: Falmer Press, 1994; McLaughlin, Hugh. "The Philosophy of Social Research." In Understanding Social
Work Research. 2nd edition. (London: SAGE Publications Ltd., 2012), pp. 24-47; Stanford Encyclopedia of
Philosophy. Metaphysics Research Lab, CSLI, Stanford University, 2013.](https://image.slidesharecdn.com/researchmethodology-170425115822/75/Research-methodology-19-2048.jpg)
![Sequential Design
Definition and Purpose
Sequential research is that which is carried out in a deliberate, staged approach [i.e. serially]
where one stage will be completed, followed by another, then another, and so on, with the
aim that each stage will build upon the previous one until enough data is gathered over an
interval of time to test your hypothesis. The sample size is not predetermined. After each
sample is analyzed, the researcher can accept the null hypothesis, accept the alternative
hypothesis, or select another pool of subjects and conduct the study once again. This means
the researcher can obtain a limitless number of subjects before making a final decision
whether to accept the null or alternative hypothesis. Using a quantitative framework, a
sequential study generally utilizes sampling techniques to gather data and applying
statistical methods to analze the data. Using a qualitative framework, sequential studies
generally utilize samples of individuals or groups of individuals [cohorts] and use
qualitative methods, such as interviews or observations, to gather information from each
sample.
What do these studies tell you?
1. The researcher has a limitless option when it comes to sample size and the sampling
schedule.
2. Due to the repetitive nature of this research design, minor changes and adjustments
can be done during the initial parts of the study to correct and hone the research
method.
3. This is a useful design for exploratory studies.
4. There is very little effort on the part of the researcher when performing this
technique. It is generally not expensive, time consuming, or workforce intensive.
5. Because the study is conducted serially, the results of one sample are known before
the next sample is taken and analyzed. This provides opportunities for continuous
improvement of sampling and methods of analysis.
What these studies don't tell you?
1. The sampling method is not representative of the entire population. The only
possibility of approaching representativeness is when the researcher chooses to use
a very large sample size significant enough to represent a significant portion of the
entire population. In this case, moving on to study a second or more specific sample
can be difficult.
2. The design cannot be used to create conclusions and interpretations that pertain to
an entire population because the sampling technique is not randomized.
Generalizability from findings is, therefore, limited.
3. Difficult to account for and interpret variation from one sample to another over
time, particularly when using qualitative methods of data collection.](https://image.slidesharecdn.com/researchmethodology-170425115822/75/Research-methodology-20-2048.jpg)
Research methodology
- 1. Skip to maincontent Research Guides 1. University of Southern California 2. Research Guides 3. Organizing Your Social Sciences Research Paper 4. Types of Research Designs Search this Guide Organizing Your Social Sciences Research Paper: Types of Research Designs The purpose of this guide is to provide advice on how to develop and organize a research paper in the social sciences. Purpose of Guide Types of Research Designs Toggle Dropdown 1. Choosing a Research Problem Toggle Dropdown 2. Preparing to Write Toggle Dropdown 3. The Abstract Toggle Dropdown 4. The Introduction Toggle Dropdown 5. The Literature Review Toggle Dropdown 6. The Methodology Toggle Dropdown 7. The Results Toggle Dropdown 8. The Discussion Toggle Dropdown 9. The Conclusion Toggle Dropdown 10. Proofreading Your Paper Toggle Dropdown 11. Citing Sources Toggle Dropdown Annotated Bibliography Giving an Oral Presentation Toggle Dropdown Grading Someone Else's Paper How to Manage Group Projects Toggle Dropdown Writing a Book Review Toggle Dropdown Writing a Case Study Writing a Field Report Toggle Dropdown
- 2. Writing aPolicy Memo Writing a Research Proposal Acknowledgements Introduction Before beginning your paper, you need to decide how you plan to design the study. The research design refers to the overall strategy that you choose to integrate the different components of the study in a coherent and logical way, thereby, ensuring you will effectively address the research problem; it constitutes the blueprint for the collection, measurement, and analysis of data. Note that your research problem determines the type of design you should use, not the other way around! De Vaus, D. A. Research Design in Social Research. London: SAGE, 2001; Trochim, William M.K. Research Methods Knowledge Base. 2006. General Structure and Writing Style The function of a research design is to ensure that the evidence obtained enables you to effectively address the research problem logically and as unambiguously as possible. In social sciences research, obtaining information relevant to the research problem generally entails specifying the type of evidence needed to test a theory, to evaluate a program, or to accurately describe and assess meaning related to an observable phenomenon. With this in mind, a common mistake made by researchers is that they begin their investigations far too early, before they have thought critically about what information is required to address the research problem. Without attending to these design issues beforehand, the overall research problem will not be adequately addressed and any conclusions drawn will run the risk of being weak and unconvincing. As a consequence, the overall validity of the study will be undermined. The length and complexity of describing research designs in your paper can vary considerably, but any well-developed design will achieve the following: 1. Identify the research problem clearly and justify its selection, particularly in relation to any valid alternative designs that could have been used, 2. Review and synthesize previously published literature associated with the research problem, 3. Clearly and explicitly specify hypotheses [i.e., research questions] central to the problem, 4. Effectively describe the data which will be necessary for an adequate testing of the hypotheses and explain how such data will be obtained, and
- 3. 5. Describe themethods of analysis to be applied to the data in determining whether or not the hypotheses are true or false. The organization and structure of the section of your paper devoted to describing the research design will vary depending on the type of design you are using. However, you can get a sense of what to do by reviewing the literature of studies that have utilized the same research design. This can provide an outline to follow for your own paper. NOTE: To search for scholarly resources on specific research designs and methods, use the SAGE ResearchMethods database. The database contains links to more than 175,000 pages of SAGE publisher's book, journal, and reference content on quantitative, qualitative, and mixed research methodologies. Also included is a collection of case studies of social research projects that can be used to help you better understand abstract or complex methodological concepts. De Vaus, D. A. Research Design in Social Research. London: SAGE, 2001; Gorard, Stephen. Research Design: Creating Robust Approachesfor the Social Sciences.Thousand Oaks, CA: Sage, 2013; Leedy, Paul D. and Jeanne Ellis Ormrod. Practical Research: Planning and Design.Tenth edition. Boston,MA: Pearson, 2013; Vogt, W. Paul, Dianna C. Gardner, and Lynne M. Haeffele. When to Use What Research Design. New York: Guilford, 2012. Action Research Design Definition and Purpose The essentials of action research design follow a characteristic cycle whereby initially an exploratory stance is adopted, where an understanding of a problem is developed and plans are made for some form of interventionary strategy. Then the intervention is carried out (the "action" in Action Research) during which time, pertinent observations are collected in various forms. The new interventional strategies are carried out, and this cyclic process repeats, continuing until a sufficient understanding of (or a valid implementation solution for) the problem is achieved. The protocol is iterative or cyclical in nature and is intended to foster deeper understanding of a given situation, starting with conceptualizing and particularizing the problem and moving through several interventions and evaluations. What do these studies tell you? 1. This is a collaborative and adaptive research design that lends itself to use in work or community situations. 2. Design focuses on pragmatic and solution-driven research outcomes rather than testing theories. 3. When practitioners use action research, it has the potential to increase the amount they learn consciously from their experience; the action research cycle can be regarded as a learning cycle.
- 4. 4. Action researchstudies often have direct and obvious relevance to improving practice and advocating for change. 5. There are no hidden controls or preemption of direction by the researcher. What these studies don't tell you? 1. It is harder to do than conducting conventional research because the researcher takes on responsibilities of advocating for change as well as for researching the topic. 2. Action research is much harder to write up because it is less likely that you can use a standard format to report your findings effectively [i.e., data is often in the form of stories or observation]. 3. Personal over-involvement of the researcher may bias research results. 4. The cyclic nature of action research to achieve its twin outcomes of action (e.g. change) and research (e.g. understanding) is time-consuming and complex to conduct. 5. Advocating for change requires buy-in from participants. Coghlan, David and Mary Brydon-Miller. The Sage Encyclopedia of Action Research. Thousand Oaks, CA: Sage, 2014; Efron, Sara Efrat and Ruth Ravid. Action Research in Education:A Practical Guide. New York: Guilford, 2013; Gall, Meredith. Educational Research:An Introduction.Chapter18, Action Research. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007; Gorard, Stephen. Research Design: Creating Robust Approaches for the Social Sciences. Thousand Oaks, CA: Sage, 2013; Kemmis, Stephen and Robin McTaggart. “Participatory Action Research.” In Handbook of Qualitative Research. Norman Denzin and Yvonna S. Lincoln, eds. 2nd ed. (Thousand Oaks, CA: SAGE, 2000), pp. 567-605; McNiff, Jean. Writing and Doing Action Research. London: Sage, 2014; Reason, Peter and Hilary Bradbury. Handbook of Action Research: Participative Inquiry and Practice. Thousand Oaks, CA: SAGE, 2001. Case Study Design Definition and Purpose A case study is an in-depth study of a particular research problem rather than a sweeping statistical survey or comprehesive comparative inquiry. It is often used to narrow down a very broad field of research into one or a few easily researchable examples. The case study research design is also useful for testing whether a specific theory and model actually applies to phenomena in the real world. It is a useful design when not much is known about an issue or phenomenon. What do these studies tell you? 1. Approach excels at bringing us to an understanding of a complex issue through detailed contextual analysis of a limited number of events or conditions and their relationships.
- 5. 2. A researcherusing a case study design can apply a variety of methodologies and rely on a variety of sources to investigate a research problem. 3. Design can extend experience or add strength to what is already known through previous research. 4. Social scientists, in particular, make wide use of this research design to examine contemporary real-life situations and provide the basis for the application of concepts and theories and the extension of methodologies. 5. The design can provide detailed descriptions of specific and rare cases. What these studies don't tell you? 1. A single or small number of cases offers little basis for establishing reliability or to generalize the findings to a wider population of people, places, or things. 2. Intense exposure to the study of a case may bias a researcher's interpretation of the findings. 3. Design does not facilitate assessment of cause and effect relationships. 4. Vital information may be missing, making the case hard to interpret. 5. The case may not be representative or typical of the larger problem being investigated. 6. If the criteria for selecting a case is because it represents a very unusual or unique phenomenon or problem for study, then your intepretation of the findings can only apply to that particular case. Case Studies. Writing@CSU. Colorado State University; Anastas, Jeane W. Research Design for Social Work and the Human Services. Chapter 4, Flexible Methods:Case Study Design. 2nd ed. New York: Columbia University Press, 1999; Gerring, John. “ What Is a Case Study and What Is It Good for?” American Political Science Review 98 (May 2004): 341-354; Greenhalgh, Trisha, editor. Case Study Evaluation: Past, Present and Future Challenges. Bingley, UK: Emerald Group Publishing, 2015; Mills, Albert J., Gabrielle Durepos, and Eiden Wiebe, editors. Encyclopedia of Case Study Research. Thousand Oaks, CA: SAGE Publications, 2010; Stake, Robert E. The Art of Case Study Research. Thousand Oaks, CA: SAGE, 1995; Yin, Robert K. Case Study Research:Design and Theory. Applied Social Research Methods Series, no.5. 3rd ed. Thousand Oaks, CA: SAGE, 2003. Causal Design Definition and Purpose Causality studies may be thought of as understanding a phenomenon in terms of conditional statements in the form, “If X, then Y.” This type of research is used to measure what impact a specific change will have on existing norms and assumptions. Most social scientists seek causal explanations that reflect tests of hypotheses. Causal effect (nomothetic perspective) occurs when variation in one phenomenon, an independent variable, leads to or results, on average, in variation in another phenomenon, the dependent variable. Conditions necessary for determining causality:
- 6. Empirical association-- a valid conclusion is based on finding an association between the independent variable and the dependent variable. Appropriate time order -- to conclude that causation was involved, one must see that cases were exposed to variation in the independent variable before variation in the dependent variable. Nonspuriousness -- a relationship between two variables that is not due to variation in a third variable. What do these studies tell you? 1. Causality research designs assist researchers in understanding why the world works the way it does through the process of proving a causal link between variables and by the process of eliminating other possibilities. 2. Replication is possible. 3. There is greater confidence the study has internal validity due to the systematic subject selection and equity of groups being compared. What these studies don't tell you? 1. Not all relationships are casual! The possibility always exists that, by sheer coincidence, two unrelated events appear to be related [e.g., Punxatawney Phil could accurately predict the duration of Winter for five consecutive years but, the fact remains, he's just a big, furry rodent]. 2. Conclusions about causal relationships are difficult to determine due to a variety of extraneous and confounding variables that exist in a social environment. This means causality can only be inferred, never proven. 3. If two variables are correlated, the cause must come before the effect. However, even though two variables might be causally related, it can sometimes be difficult to determine which variable comes first and, therefore, to establish which variable is the actual cause and which is the actual effect. Beach, Derek and Rasmus Brun Pedersen. Causal Case Study Methods: Foundations and Guidelines for Comparing, Matching,and Tracing. Ann Arbor, MI: University of Michigan Press, 2016; Bachman, Ronet. The Practice of Research in Criminology and Criminal Justice.Chapter 5, Causation and Research Designs. 3rd ed. Thousand Oaks, CA: Pine Forge Press, 2007; Brewer, Ernest W. and Jennifer Kubn. “Causal- Comparative Design.” In Encyclopedia of Research Design. Neil J. Salkind, editor. (Thousand Oaks, CA: Sage, 2010), pp. 125-132; Causal Research Design: Experimentation. Anonymous SlideShare Presentation; Gall, Meredith. Educational Research: An Introduction. Chapter 11, Nonexperimental Research: Correlational Designs. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007; Trochim, William M.K. Research Methods Knowledge Base. 2006. Cohort Design Definition and Purpose
- 7. Often used inthe medical sciences, but also found in the applied social sciences, a cohort study generally refers to a study conducted over a period of time involving members of a population which the subject or representative member comes from, and who are united by some commonality or similarity. Using a quantitative framework, a cohort study makes note of statistical occurrence within a specialized subgroup, united by same or similar characteristics that are relevant to the research problem being investigated, rather than studying statistical occurrence within the general population. Using a qualitative framework, cohort studies generally gather data using methods of observation. Cohorts can be either "open" or "closed." Open Cohort Studies [dynamic populations, such as the population of Los Angeles] involve a population that is defined just by the state of being a part of the study in question (and being monitored for the outcome). Date of entry and exit from the study is individually defined, therefore, the size of the study population is not constant. In open cohort studies, researchers can only calculate rate based data, such as, incidence rates and variants thereof. Closed Cohort Studies [static populations, such as patients entered into a clinical trial] involve participants who enter into the study at one defining point in time and where it is presumed that no new participants can enter the cohort. Given this, the number of study participants remains constant (or can only decrease). What do these studies tell you? 1. The use of cohorts is often mandatory because a randomized control study may be unethical. For example, you cannot deliberately expose people to asbestos, you can only study its effects on those who have already been exposed. Research that measures risk factors often relies upon cohort designs. 2. Because cohort studies measure potential causes before the outcome has occurred, they can demonstrate that these “causes” preceded the outcome, thereby avoiding the debate as to which is the cause and which is the effect. 3. Cohort analysis is highly flexible and can provide insight into effects over time and related to a variety of different types of changes [e.g., social, cultural, political, economic, etc.]. 4. Either original data or secondary data can be used in this design. What these studies don't tell you? 1. In cases where a comparative analysis of two cohorts is made [e.g., studying the effects of one group exposed to asbestos and one that has not], a researcher cannot control for all other factors that might differ between the two groups. These factors are known as confounding variables. 2. Cohort studies can end up taking a long time to complete if the researcher must wait for the conditions of interest to develop within the group. This also increases the chance that key variables change during the course of the study, potentially impacting the validity of the findings.
- 8. 3. Due tothe lack of randominization in the cohort design, its external validity is lower than that of study designs where the researcher randomly assigns participants. Healy P, Devane D. “MethodologicalConsiderations in Cohort Study Designs.” Nurse Researcher 18 (2011): 32-36; Glenn, Norval D, editor. Cohort Analysis. 2nd edition. Thousand Oaks, CA: Sage, 2005; Levin, Kate Ann. Study Design IV: Cohort Studies. Evidence-Based Dentistry 7 (2003): 51–52; Payne, Geoff. “Cohort Study.” In The SAGE Dictionary of Social Research Methods. Victor Jupp, editor. (Thousand Oaks, CA: Sage, 2006), pp. 31-33; Study Design 101. Himmelfarb Health Sciences Library. George Washington University, November 2011; Cohort Study. Wikipedia. Cross-Sectional Design Definition and Purpose Cross-sectional research designs have three distinctive features: no time dimension; a reliance on existing differences rather than change following intervention; and, groups are selected based on existing differences rather than random allocation. The cross-sectional design can only measure differences between or from among a variety of people, subjects, or phenomena rather than a process of change. As such, researchers using this design can only employ a relatively passive approach to making causal inferences based on findings. What do these studies tell you? 1. Cross-sectional studies provide a clear 'snapshot' of the outcome and the characteristics associated with it, at a specific point in time. 2. Unlike an experimental design, where there is an active intervention by the researcher to produce and measure change or to create differences, cross-sectional designs focus on studying and drawing inferences from existing differences between people, subjects, or phenomena. 3. Entails collecting data at and concerning one point in time. While longitudinal studies involve taking multiple measures over an extended period of time, cross- sectional research is focused on finding relationships between variables at one moment in time. 4. Groups identified for study are purposely selected based upon existing differences in the sample rather than seeking random sampling. 5. Cross-section studies are capable of using data from a large number of subjects and, unlike observational studies, is not geographically bound. 6. Can estimate prevalence of an outcome of interest because the sample is usually taken from the whole population. 7. Because cross-sectional designs generally use survey techniques to gather data, they are relatively inexpensive and take up little time to conduct. What these studies don't tell you?
- 9. 1. Finding people,subjects, or phenomena to study that are very similar except in one specific variable can be difficult. 2. Results are static and time bound and, therefore, give no indication of a sequence of events or reveal historical or temporal contexts. 3. Studies cannot be utilized to establish cause and effect relationships. 4. This design only provides a snapshot of analysis so there is always the possibility that a study could have differing results if another time-frame had been chosen. 5. There is no follow up to the findings. Bethlehem, Jelke. "7: Cross-sectional Research." In Research Methodology in the Social, Behavioural and Life Sciences. Herman J Adèr and Gideon J Mellenbergh, editors. (London, England: Sage, 1999), pp. 110- 43; Bourque, Linda B. “Cross-Sectional Design.” In The SAGE Encyclopedia of Social Science Research Methods.Michael S. Lewis-Beck, Alan Bryman, and Tim Futing Liao. (Thousand Oaks, CA: 2004), pp. 230- 231; Hall, John. “Cross-Sectional Survey Design.” In Encyclopedia of Survey Research Methods. Paul J. Lavrakas, ed. (Thousand Oaks, CA: Sage, 2008), pp. 173-174; Helen Barratt, Maria Kirwan. Cross-Sectional Studies: Design,Application, Strengths and Weaknesses ofCross-SectionalStudies.Healthknowledge, 2009. Cross-Sectional Study. Wikipedia. Descriptive Design Definition and Purpose Descriptive research designs help provide answers to the questions of who, what, when, where, and how associated with a particular research problem; a descriptive study cannot conclusively ascertain answers to why. Descriptive research is used to obtain information concerning the current status of the phenomena and to describe "what exists" with respect to variables or conditions in a situation. What do these studies tell you? 1. The subject is being observed in a completely natural and unchanged natural environment. True experiments, whilst giving analyzable data, often adversely influence the normal behavior of the subject [a.k.a., the Heisenberg effect whereby measurements of certain systems cannot be made without affecting the systems]. 2. Descriptive research is often used as a pre-cursor to more quantitative research designs with the general overview giving some valuable pointers as to what variables are worth testing quantitatively. 3. If the limitations are understood, they can be a useful tool in developing a more focused study. 4. Descriptive studies can yield rich data that lead to important recommendations in practice. 5. Appoach collects a large amount of data for detailed analysis. What these studies don't tell you?
- 10. 1. The resultsfrom a descriptive research cannot be used to discover a definitive answer or to disprove a hypothesis. 2. Because descriptive designs often utilize observational methods [as opposed to quantitative methods], the results cannot be replicated. 3. The descriptive function of research is heavily dependent on instrumentation for measurement and observation. Anastas,Jeane W. Research Design for Social Work and the Human Services. Chapter 5, Flexible Methods: Descriptive Research. 2nd ed. New York: Columbia University Press, 1999; Given, Lisa M. "Descriptive Research." In Encyclopedia of Measurement and Statistics.Neil J. Salkind and Kristin Rasmussen, editors. (Thousand Oaks, CA: Sage, 2007), pp. 251-254; McNabb, Connie. Descriptive Research Methodologies. Powerpoint Presentation; Shuttleworth, Martyn. Descriptive Research Design, September 26, 2008. Explorable.com website. Experimental Design Definition and Purpose A blueprint of the procedure that enables the researcher to maintain control over all factors that may affect the result of an experiment. In doing this, the researcher attempts to determine or predict what may occur. Experimental research is often used where there is time priority in a causal relationship (cause precedes effect), there is consistency in a causal relationship (a cause will always lead to the same effect), and the magnitude of the correlation is great. The classic experimental design specifies an experimental group and a control group. The independent variable is administered to the experimental group and not to the control group, and both groups are measured on the same dependent variable. Subsequent experimental designs have used more groups and more measurements over longer periods. True experiments must have control, randomization, and manipulation. What do these studies tell you? 1. Experimental research allows the researcher to control the situation. In so doing, it allows researchers to answer the question, “What causes something to occur?” 2. Permits the researcher to identify cause and effect relationships between variables and to distinguish placebo effects from treatment effects. 3. Experimental research designs support the ability to limit alternative explanations and to infer direct causal relationships in the study. 4. Approach provides the highest level of evidence for single studies. What these studies don't tell you? 1. The design is artificial, and results may not generalize well to the real world. 2. The artificial settings of experiments may alter the behaviors or responses of participants. 3. Experimental designs can be costly if special equipment or facilities are needed.
- 11. 4. Some researchproblems cannot be studied using an experiment because of ethical or technical reasons. 5. Difficult to apply ethnographic and other qualitative methods to experimentally designed studies. Anastas,Jeane W. Research Design for Social Work and the Human Services. Chapter 7, Flexible Methods: Experimental Research. 2nd ed. New York: Columbia University Press, 1999; Chapter 2: Research Design, Experimental Designs. School of Psychology, University of New England, 2000; Chow, Siu L. "Experimental Design." In Encyclopedia of Research Design. Neil J. Salkind, editor. (Thousand Oaks, CA: Sage, 2010), pp. 448-453; "Experimental Design." In Social Research Methods.Nicholas Walliman, editor. (London, England: Sage, 2006), pp, 101-110; Experimental Research. Research Methods by Dummies. Department of Psychology.California State University, Fresno, 2006; Kirk, Roger E. Experimental Design: Procedures for the Behavioral Sciences. 4th edition. Thousand Oaks, CA: Sage, 2013; Trochim, William M.K. Experimental Design. Research Methods Knowledge Base. 2006; Rasool, Shafqat. Experimental Research. Slideshare presentation. Exploratory Design Definition and Purpose An exploratory design is conducted about a research problem when there are few or no earlier studies to refer to or rely upon to predict an outcome. The focus is on gaining insights and familiarity for later investigation or undertaken when research problems are in a preliminary stage of investigation. Exploratory designs are often used to establish an understanding of how best to proceed in studying an issue or what methodology would effectively apply to gathering information about the issue. The goals of exploratory research are intended to produce the following possible insights: Familiarity with basic details, settings, and concerns. Well grounded picture of the situation being developed. Generation of new ideas and assumptions. Development of tentative theories or hypotheses. Determination about whether a study is feasible in the future. Issues get refined for more systematic investigation and formulation of new research questions. Direction for future research and techniques get developed. What do these studies tell you? 1. Design is a useful approach for gaining background information on a particular topic. 2. Exploratory research is flexible and can address research questions of all types (what, why, how). 3. Provides an opportunity to define new terms and clarify existing concepts.
- 12. 4. Exploratory researchis often used to generate formal hypotheses and develop more precise research problems. 5. In the policy arena or applied to practice, exploratory studies help establish research priorities and where resources should be allocated. What these studies don't tell you? 1. Exploratory research generally utilizes small sample sizes and, thus, findings are typically not generalizable to the population at large. 2. The exploratory nature of the research inhibits an ability to make definitive conclusions about the findings. They provide insight but not definitive conclusions. 3. The research process underpinning exploratory studies is flexible but often unstructured, leading to only tentative results that have limited value to decision- makers. 4. Design lacks rigorous standards applied to methods of data gathering and analysis because one of the areas for exploration could be to determine what method or methodologies could best fit the research problem. Cuthill, Michael. “Exploratory Research: Citizen Participation, Local Government, and Sustainable Development in Australia.” Sustainable Development 10 (2002): 79-89; Streb, Christoph K. "Exploratory Case Study." In Encyclopedia ofCase Study Research. Albert J. Mills, Gabrielle Durepos and Eiden Wiebe, editors. (Thousand Oaks, CA: Sage, 2010), pp. 372-374; Taylor, P. J., G. Catalano, and D.R.F. Walker. “Exploratory Analysis of the World City Network.” Urban Studies 39 (December 2002): 2377-2394; Exploratory Research. Wikipedia. Historical Design Definition and Purpose The purpose of a historical research design is to collect, verify, and synthesize evidence from the past to establish facts that defend or refute a hypothesis. It uses secondary sources and a variety of primary documentary evidence, such as, diaries, official records, reports, archives, and non-textual information [maps, pictures, audio and visual recordings]. The limitation is that the sources must be both authentic and valid. What do these studies tell you? 1. The historical research design is unobtrusive; the act of research does not affect the results of the study. 2. The historical approach is well suited for trend analysis. 3. Historical records can add important contextual background required to more fully understand and interpret a research problem. 4. There is often no possibility of researcher-subject interaction that could affect the findings.
- 13. 5. Historical sourcescan be used over and over to study different research problems or to replicate a previous study. What these studies don't tell you? 1. The ability to fulfill the aims of your research are directly related to the amount and quality of documentation available to understand the research problem. 2. Since historical research relies on data from the past, there is no way to manipulate it to control for contemporary contexts. 3. Interpreting historical sources can be very time consuming. 4. The sources of historical materials must be archived consistently to ensure access. This may especially challenging for digital or online-only sources. 5. Original authors bring their own perspectives and biases to the interpretation of past events and these biases are more difficult to ascertain in historical resources. 6. Due to the lack of control over external variables, historical research is very weak with regard to the demands of internal validity. 7. It is rare that the entirety of historical documentation needed to fully address a research problem is available for interpretation, therefore, gaps need to be acknowledged. Howell, Martha C. and Walter Prevenier. From Reliable Sources: An Introduction to Historical Methods. Ithaca, NY: Cornell University Press, 2001; Lundy, Karen Saucier. "Historical Research." In The Sage Encyclopedia of Qualitative Research Methods. Lisa M. Given, editor. (Thousand Oaks, CA: Sage, 2008), pp. 396-400; Marius, Richard. and Melvin E. Page. A Short Guide to Writing about History. 9th edition. Boston, MA: Pearson, 2015; Savitt, Ronald. “Historical Research in Marketing.” Journal of Marketing 44 (Autumn, 1980): 52-58; Gall, Meredith. Educational Research: An Introduction. Chapter 16, Historical Research. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007. Longitudinal Design Definition and Purpose A longitudinal study follows the same sample over time and makes repeated observations. For example, with longitudinal surveys, the same group of people is interviewed at regular intervals, enabling researchers to track changes over time and to relate them to variables that might explain why the changes occur. Longitudinal research designs describe patterns of change and help establish the direction and magnitude of causal relationships. Measurements are taken on each variable over two or more distinct time periods. This allows the researcher to measure change in variables over time. It is a type of observational study sometimes referred to as a panel study. What do these studies tell you? 1. Longitudinal data facilitate the analysis of the duration of a particular phenomenon.
- 14. 2. Enables surveyresearchers to get close to the kinds of causal explanations usually attainable only with experiments. 3. The design permits the measurement of differences or change in a variable from one period to another [i.e., the description of patterns of change over time]. 4. Longitudinal studies facilitate the prediction of future outcomes based upon earlier factors. What these studies don't tell you? 1. The data collection method may change over time. 2. Maintaining the integrity of the original sample can be difficult over an extended period of time. 3. It can be difficult to show more than one variable at a time. 4. This design often needs qualitative research data to explain fluctuations in the results. 5. A longitudinal research design assumes present trends will continue unchanged. 6. It can take a long period of time to gather results. 7. There is a need to have a large sample size and accurate sampling to reach representativness. Anastas,Jeane W. Research Design for Social Work and the Human Services. Chapter 6, Flexible Methods: Relational and Longitudinal Research. 2nd ed. New York: Columbia University Press, 1999; Forgues, Bernard, and Isabelle Vandangeon-Derumez. "Longitudinal Analyses." In Doing Management Research. Raymond-Alain Thiétart and Samantha Wauchope, editors. (London, England: Sage, 2001), pp. 332-351; Kalaian, Sema A. and Rafa M. Kasim. "Longitudinal Studies." In Encyclopedia ofSurvey Research Methods. Paul J. Lavrakas, ed. (Thousand Oaks, CA: Sage, 2008), pp. 440-441; Menard, Scott, editor. Longitudinal Research. Thousand Oaks, CA: Sage, 2002; Ployhart, Robert E. and Robert J. Vandenberg. "Longitudinal Research: The Theory, Design, and Analysis of Change.” Journal of Management 36 (January 2010): 94- 120; Longitudinal Study. Wikipedia. Meta-Analysis Design Definition and Purpose Meta-analysis is an analytical methodology designed to systematically evaluate and summarize the results from a number of individual studies, thereby, increasing the overall sample size and the ability of the researcher to study effects of interest. The purpose is to not simply summarize existing knowledge, but to develop a new understanding of a research problem using synoptic reasoning. The main objectives of meta-analysis include analyzing differences in the results among studies and increasing the precision by which effects are estimated. A well-designed meta-analysis depends upon strict adherence to the criteria used for selecting studies and the availability of information in each study to properly analyze their findings. Lack of information can severely limit the type of analyzes and conclusions that can be reached. In addition, the more dissimilarity there is in the results among individual studies [heterogeneity], the more difficult it is to justify
- 15. interpretations that governa valid synopsis of results. A meta-analysis needs to fulfill the following requirements to ensure the validity of your findings: Clearly defined description of objectives, including precise definitions of the variables and outcomes that are being evaluated; A well-reasoned and well-documented justification for identification and selection of the studies; Assessment and explicit acknowledgment of any researcher bias in the identification and selection of those studies; Description and evaluation of the degree of heterogeneity among the sample size of studies reviewed; and, Justification of the techniques used to evaluate the studies. What do these studies tell you? 1. Can be an effective strategy for determining gaps in the literature. 2. Provides a means of reviewing research published about a particular topic over an extended period of time and from a variety of sources. 3. Is useful in clarifying what policy or programmatic actions can be justified on the basis of analyzing research results from multiple studies. 4. Provides a method for overcoming small sample sizes in individual studies that previously may have had little relationship to each other. 5. Can be used to generate new hypotheses or highlight research problems for future studies. What these studies don't tell you? 1. Small violations in defining the criteria used for content analysis can lead to difficult to interpret and/or meaningless findings. 2. A large sample size can yield reliable, but not necessarily valid, results. 3. A lack of uniformity regarding, for example, the type of literature reviewed, how methods are applied, and how findings are measured within the sample of studies you are analyzing, can make the process of synthesis difficult to perform. 4. Depending on the sample size, the process of reviewing and synthesizing multiple studies can be very time consuming. Beck, Lewis W. "The Synoptic Method." The Journal of Philosophy 36 (1939): 337-345; Cooper, Harris, Larry V. Hedges, and Jeffrey C. Valentine, eds. The Handbook of Research Synthesis and Meta-Analysis. 2nd edition. New York: Russell Sage Foundation,2009; Guzzo, Richard A., Susan E. Jackson and Raymond A. Katzell. “Meta-Analysis Analysis.” In Research in Organizational Behavior,Volume 9. (Greenwich, CT: JAI Press, 1987), pp 407-442; Lipsey, Mark W. and David B. Wilson. Practical Meta-Analysis. Thousand Oaks, CA: Sage Publications, 2001; Study Design 101. Meta-Analysis. The Himmelfarb Health Sciences Library, George Washington University; Timulak, Ladislav. “Qualitative Meta-Analysis.” In The SAGE Handbook of Qualitative Data Analysis. Uwe Flick, editor. (Los Angeles, CA: Sage, 2013), pp. 481-495;
- 16. Walker, Esteban, AdrianV. Hernandez, and Micheal W. Kattan. "Meta-Analysis: It's Strengths and Limitations." Cleveland Clinic Journal of Medicine 75 (June 2008): 431-439. Mixed-Method Design Definition and Purpose Mixed methods research represents more of an approach to examining a research problem than a methodology. Mixed method is characterized by a focus on research problems that require, 1) an examination of real-life contextual understandings, multi-level perspectives, and cultural influences; 2) an intentional application of rigorous quantitative research assessing magnitude and frequency of constructs and rigorous qualitative research exploring the meaning and understanding of the constructs; and, 3) an objective of drawing on the strengths of quantitative and qualitative data gathering techniques to formulate a holistic interpretive framework for generating possible solutions or new understandings of the problem. Tashakkori and Creswell (2007) and other proponents of mixed methods argue that the design encompasses more than simply combining qualitative and quantitative methods but, rather, reflects a new "third way" epistemological paradigm that occupies the conceptual space between positivism and interpretivism. What do these studies tell you? 1. Narrative and non-textual information can add meaning to numeric data, while numeric data can add precision to narrative and non-textual information. 2. Can utilize existing data while at the same time generating and testing a grounded theory approach to describe and explain the phenomenon under study. 3. A broader, more complex research problem can be investigated because the researcher is not constrained by using only one method. 4. The strengths of one method can be used to overcome the inherent weaknesses of another method. 5. Can provide stronger, more robust evidence to support a conclusion or set of recommendations. 6. May generate new knowledge new insights or uncover hidden insights, patterns, or relationships that a single methodological approach might not reveal. 7. Produces more complete knowledge and understanding of the research problem that can be used to increase the generalizability of findings applied to theory or practice. What these studies don't tell you? 1. A researcher must be proficient in understanding how to apply multiple methods to investigating a research problem as well as be proficient in optimizing how to design a study that coherently melds them together. 2. Can increase the likelihood of conflicting results or ambiguous findings that inhibit drawing a valid conclusion or setting forth a recommended course of action [e.g., sample interview responses do not support existing statistical data].
- 17. 3. Because theresearch design can be very complex, reporting the findings requires a well-organized narrative, clear writing style, and precise word choice. 4. Design invites collaboration among experts. However, merging different investigative approaches and writing styles requires more attention to the overall research process than studies conducted using only one methodological paradigm. 5. Concurrent merging of quantitative and qualitative research requires greater attention to having adequate sample sizes, using comparable samples, and applying a consistent unit of analysis. For sequential designs where one phase of qualitative research builds on the quantitative phase or vice versa, decisions about what results from the first phase to use in the next phase, the choice of samples and estimating reasonable sample sizes for both phases, and the interpretation of results from both phases can be difficult. 6. Due to multiple forms of data being collected and analyzed, this design requires extensive time and resources to carry out the multiple steps involved in data gathering and interpretation. Burch, Patricia and Carolyn J. Heinrich. Mixed Methods for Policy Research and Program Evaluation. Thousand Oaks, CA: Sage, 2016; Creswell, John w. et al. Best Practices for Mixed Methods Research in the Health Sciences. Bethesda, MD: Office of Behavioral and Social Sciences Research, National Institutes of Health, 2010Creswell, John W. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. 4th edition. Thousand Oaks, CA: Sage Publications, 2014; Domínguez, Silvia, editor. Mixed Methods Social Networks Research. Cambridge, UK: Cambridge University Press, 2014; Hesse-Biber, Sharlene Nagy. Mixed Methods Research: Merging Theory with Practice. New York: Guilford Press, 2010; Niglas, Katrin. “ How the Novice Researcher Can Make Sense of Mixed Methods Designs.” International Journal of Multiple Research Approaches 3 (2009): 34-46; Onwuegbuzie, Anthony J. and Nancy L. Leech. “ Linking Research Questions to Mixed Methods Data Analysis Procedures.” The Qualitative Report 11 (September 2006): 474-498; Tashakorri, Abbas and John W. Creswell. “ The New Era of Mixed Methods.” Journal of Mixed Methods Research 1 (January 2007): 3-7; Zhanga, Wanqing. “ Mixed Methods Application in Health Intervention Research: A Multiple Case Study.” International Journal of Multiple Research Approaches 8 (2014): 24-35. Observational Design Definition and Purpose This type of research design draws a conclusion by comparing subjects against a control group, in cases where the researcher has no control over the experiment. There are two general types of observational designs. In direct observations, people know that you are watching them. Unobtrusive measures involve any method for studying behavior where individuals do not know they are being observed. An observational study allows a useful insight into a phenomenon and avoids the ethical and practical difficulties of setting up a large and cumbersome research project. What do these studies tell you? 1. Observational studies are usually flexible and do not necessarily need to be structured around a hypothesis about what you expect to observe [data is emergent rather than pre-existing]. 2. The researcher is able to collect in-depth information about a particular behavior. 3. Can reveal interrelationships among multifaceted dimensions of group interactions. 4. You can generalize your results to real life situations.
- 18. 5. Observational researchis useful for discovering what variables may be important before applying other methods like experiments. 6. Observation research designs account for the complexity of group behaviors. What these studies don't tell you? 1. Reliability of data is low because seeing behaviors occur over and over again may be a time consuming task and are difficult to replicate. 2. In observational research, findings may only reflect a unique sample population and, thus, cannot be generalized to other groups. 3. There can be problems with bias as the researcher may only "see what they want to see." 4. There is no possibility to determine "cause and effect" relationships since nothing is manipulated. 5. Sources or subjects may not all be equally credible. 6. Any group that is knowingly studied is altered to some degree by the presence of the researcher, therefore, potentially skewing any data collected. Atkinson, Paul and Martyn Hammersley. “Ethnography and Participant Observation.” In Handbook of Qualitative Research. Norman K. Denzin and Yvonna S. Lincoln, eds. (Thousand Oaks, CA: Sage, 1994), pp. 248-261; Observational Research. Research Methods by Dummies. Department of Psychology. California State University, Fresno, 2006; Patton Michael Quinn. Qualitiative Research and Evaluation Methods. Chapter 6, Fieldwork Strategies and Observational Methods. 3rd ed. Thousand Oaks, CA: Sage, 2002; Payne, Geoff and Judy Payne. "Observation." In Key Concepts in Social Research. The SAGE Key Concepts series. (London, England: Sage, 2004), pp. 158-162; Rosenbaum, Paul R. Design of Observational Studies. New York: Springer, 2010;Williams, J. Patrick. "Nonparticipant Observation." In The Sage Encyclopedia of Qualitative Research Methods. Lisa M. Given, editor.(Thousand Oaks, CA: Sage, 2008), pp. 562-563. Philosophical Design Definition and Purpose Understood more as an broad approach to examining a research problem than a methodological design, philosophical analysis and argumentation is intended to challenge deeply embedded, often intractable, assumptions underpinning an area of study. This approach uses the tools of argumentation derived from philosophical traditions, concepts, models, and theories to critically explore and challenge, for example, the relevance of logic and evidence in academic debates, to analyze arguments about fundamental issues, or to discuss the root of existing discourse about a research problem. These overarching tools of analysis can be framed in three ways: Ontology -- the study that describes the nature of reality; for example, what is real and what is not, what is fundamental and what is derivative?
- 19. Epistemology --the study that explores the nature of knowledge; for example, by what means does knowledge and understanding depend upon and how can we be certain of what we know? Axiology -- the study of values; for example, what values does an individual or group hold and why? How are values related to interest, desire, will, experience, and means-to-end? And, what is the difference between a matter of fact and a matter of value? What do these studies tell you? 1. Can provide a basis for applying ethical decision-making to practice. 2. Functions as a means of gaining greater self-understanding and self-knowledge about the purposes of research. 3. Brings clarity to general guiding practices and principles of an individual or group. 4. Philosophy informs methodology. 5. Refine concepts and theories that are invoked in relatively unreflective modes of thought and discourse. 6. Beyond methodology, philosophy also informs critical thinking about epistemology and the structure of reality (metaphysics). 7. Offers clarity and definition to the practical and theoretical uses of terms, concepts, and ideas. What these studies don't tell you? 1. Limited application to specific research problems [answering the "So What?" question in social science research]. 2. Analysis can be abstract, argumentative, and limited in its practical application to real-life issues. 3. While a philosophical analysis may render problematic that which was once simple or taken-for-granted, the writing can be dense and subject to unnecessary jargon, overstatement, and/or excessive quotation and documentation. 4. There are limitations in the use of metaphor as a vehicle of philosophical analysis. 5. There can be analytical difficulties in moving from philosophy to advocacy and between abstract thought and application to the phenomenal world. Burton, Dawn. "Part I, Philosophy of the Social Sciences." In Research Training for Social Scientists. (London, England: Sage, 2000), pp. 1-5; Chapter 4, Research Methodology and Design. Unisa Institutional Repository (UnisaIR), University of South Africa; Jarvie, Ian C., and Jesús Zamora-Bonilla, editors. The SAGE Handbook of the Philosophy of Social Sciences. London: Sage, 2011; Labaree, Robert V. and Ross Scimeca. “The Philosophical Problem of Truth in Librarianship.” The Library Quarterly 78 (January 2008): 43-70; Maykut,Pamela S. Beginning Qualitative Research:A Philosophic and Practical Guide.Washington, DC: Falmer Press, 1994; McLaughlin, Hugh. "The Philosophy of Social Research." In Understanding Social Work Research. 2nd edition. (London: SAGE Publications Ltd., 2012), pp. 24-47; Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, CSLI, Stanford University, 2013.
- 20. Sequential Design Definition andPurpose Sequential research is that which is carried out in a deliberate, staged approach [i.e. serially] where one stage will be completed, followed by another, then another, and so on, with the aim that each stage will build upon the previous one until enough data is gathered over an interval of time to test your hypothesis. The sample size is not predetermined. After each sample is analyzed, the researcher can accept the null hypothesis, accept the alternative hypothesis, or select another pool of subjects and conduct the study once again. This means the researcher can obtain a limitless number of subjects before making a final decision whether to accept the null or alternative hypothesis. Using a quantitative framework, a sequential study generally utilizes sampling techniques to gather data and applying statistical methods to analze the data. Using a qualitative framework, sequential studies generally utilize samples of individuals or groups of individuals [cohorts] and use qualitative methods, such as interviews or observations, to gather information from each sample. What do these studies tell you? 1. The researcher has a limitless option when it comes to sample size and the sampling schedule. 2. Due to the repetitive nature of this research design, minor changes and adjustments can be done during the initial parts of the study to correct and hone the research method. 3. This is a useful design for exploratory studies. 4. There is very little effort on the part of the researcher when performing this technique. It is generally not expensive, time consuming, or workforce intensive. 5. Because the study is conducted serially, the results of one sample are known before the next sample is taken and analyzed. This provides opportunities for continuous improvement of sampling and methods of analysis. What these studies don't tell you? 1. The sampling method is not representative of the entire population. The only possibility of approaching representativeness is when the researcher chooses to use a very large sample size significant enough to represent a significant portion of the entire population. In this case, moving on to study a second or more specific sample can be difficult. 2. The design cannot be used to create conclusions and interpretations that pertain to an entire population because the sampling technique is not randomized. Generalizability from findings is, therefore, limited. 3. Difficult to account for and interpret variation from one sample to another over time, particularly when using qualitative methods of data collection.
- 21. Betensky, Rebecca. HarvardUniversity, Course Lecture Note slides; Bovaird, James A. and Kevin A. Kupzyk. "SequentialDesign." In Encyclopedia ofResearch Design. Neil J. Salkind, editor. (Thousand Oaks, CA: Sage, 2010), pp. 1347-1352; Cresswell, John W. Et al. “Advanced Mixed-Methods Research Designs.” In Handbook of Mixed Methodsin Social and Behavioral Research. Abbas Tashakkori and Charles Teddle, eds. (Thousand Oaks, CA: Sage, 2003), pp. 209-240; Henry, Gary T. "Sequential Sampling." In The SAGE Encyclopedia of Social Science Research Methods. Michael S. Lewis-Beck, Alan Bryman and Tim Futing Liao, editors. (Thousand Oaks, CA: Sage, 2004), pp. 1027-1028; Nataliya V. Ivankova. “Using Mixed- Methods Sequential Explanatory Design: From Theory to Practice.” Field Methods 18 (February 2006): 3- 20; Bovaird, James A. and Kevin A. Kupzyk. “Sequential Design.” In Encyclopedia of Research Design. Neil J. Salkind, ed. Thousand Oaks, CA: Sage, 2010; Sequential Analysis. Wikipedia. Last Updated: Apr 21, 2017 12:56 PM URL: http://libguides.usc.edu/writingguide Print Page Login to LibApps Subjects: General Reference & Research Help Tags: citation, writing_support © University of Southern California Contact us
- 22. Home / Introductionto Educational Research / Research design and methods Research design and methods The main features of the study design and methods you select follow the question that has been posed. For example, you may be using a survey by questionnaire or interview; a case study of one or more person or organisation; a trial, which may be randomly controlled or use matched or waiting-list controls. It may be ethnographic, requiring time in the field to look at relationships, culture or communication, or action research, where practical problems are considered and where the feedback, changes and subsequent evaluation of change are all part of the research – rather like audit in its cyclical design. Action research is a common mode of educational research. It appeals to those who like the idea of change and the feeling that the research they do actually contributes to it. It joins together both research and implementation, and is a much messier, more participative research method than most; so people who start it should not be those who demand precision or decimal points in their answers, and they should enjoy being involved with the teams and individuals who actually put the changes into practice (Somekh, 1995). Research may also be cross-sectional or longitudinal; depending on what is the right way to answer the question and what resources are available. For example, cross-sectional designs can provide answers to the immediate success of a programme if success is seen in terms of, for instance, course satisfaction. They can also be used to see, for example, if different people are more or less satisfied depending on their learning preference, or to consider any other differences between them or between course presenters, or an interaction between both. But if you want to assess what really changed as a result of your course, then it has to be longitudinal. Longitudinal design is expensive, not only because you have to have the researchers available over the length of the study, but also because you might have to spend funds on tracking your subjects over time. However, the benefits of longitudinal cohort studies are that you can answer more complicated questions. For example, if you were studying job satisfaction in doctors using a cross-sectional design, you would probably find that around a quarter were pretty dissatisfied. If you had a large enough sample, you could break it down into specialties and see that one particular group (e.g. surgeons) were very satisfied, while another (e.g. psychiatrists) were very dissatisfied. You would be likely to conclude that the work role of psychiatrists has more factors likely to produce dissatisfaction than the role of surgeons does. However, a longitudinal study might have assessed these doctors as students or house officers and looked at personality or at satisfaction then, and would probably have found that many of those who are disgruntled now were disgruntled then. This then allows us to see how much of the dissatisfaction is due to the person and how much is due to the job, which in turn means we may not need to continue to plough resources into the job alone, but also into individual intervention or selection issues.
- 23. Your research questionand available resources will determine over what period you should assess and how many assessments you will need to make. Although it will vary according to the intervention made, where clinical interventions are the subject of research, patients are usually assessed during a baseline period, immediately after the intervention, a few weeks or months later, and after a year or two. Ideally, educational interventions would not differ much from this, but a minimum would be before, immediately after and 6–12 months later. Sampling Selecting Subjects for Survey Research… The main purpose of survey research is to describe the characteristics of a population by collecting data from a sample. POPULATION–> The population is the group consisting of all people to whom we (as researchers) wish to apply our findings. If we were interested in the reading level of 3rd graders in Connecticut, the population would be all third graders in Connecticut. The data (information) we collect from populations are called PARAMETERS and are said to be DESCRIPTIVE. We label the number of subjects (observations) in a population with an upper case N (N=300). The first step in sampling is to define the population (3rd graders in Connecticut). (The actual population to whom the researcher wishes to apply his or her findings is called the TARGET population. Often the TARGET population is not available, and the research must use an ACCESSIBLE POPULATIONS. In this case, the researcher can only apply (generalize) his or her findings to that group. SAMPLE–> Subsets of people are usually used to conduct studies. These subsets are called samples. The samples are used to represent the population from which they were drawn. The data we collect from samples are called STATISTICS and are said to be INFERENTIAL (because we are making inferences about the POPULATION with data collected from the SAMPLE). We
- 24. label the numberof subjects (observations) in a sample with a lower case n (n=25). Statistics are used to effectively communicate numerical information to other people. In statistics we are… …Looking at RELATIONSHIPS among (between) characteristics (i.e., salary & job satisfaction; food consumption & energy) — Correlation Research (which we study in a different unit) is an example of research involving relationships. …Looking at DIFFERENCES between (among) groups (i.e., males & females; experiment & control) — Experimental Research (which we study in a different unit) is an example of research that looks at differences. …Looking to DESCRIBE the characteristics of the population from data collected from a sample — Survey Research. The two major types of surveys are cross-sectional survey and longitudinal survey (trend, cohort, and panel studies). Inferential statistics are used to determine how likely it is that characteristics exhibited by a sample of people are an accurate description of those characteristics exhibited by the population of people from which the sample was drawn. The term statistically significant (which we will study later, so don’t worry about it for this unit) (p < .05) is used merely as a way of indicating the chances are at least 95 out of 100 that the findings obtained from the sample of people who participated in the study are similar to what the findings would be if one were actually able to carry out the study with the entire population. The first step in selecting a sample is to define the population to which one wishes to generalize the results of a study. Unfortunately one may not be able to collect data from his or her TARGET POPULATION. In this case, an ACCESSIBLE
- 25. POPULATION is used.If the latter is used, care must be taken not to generalize beyond the ACCESSIBLE POPULATION. -The sample is drawn from the population -Data is collected from the sample -Statistics are used to determine how likely the sample results are reflective of the population A number of different strategies can be used to select a sample. Each of the strategies has strengths and weaknesses. There are times when the research results from the sample cannot be applied to the population because threats to external validity exist with the study. The most important aspect of sampling is that the sample represent the population. *CHOOSING A SAMPLE* SIMPLE RANDOM SAMPLING – Each subject in the population has an equal chance of being selected STRATIFIED RANDOM SAMPLING – A representative number of subjects from various subgroups TWO STAGE CLUSTER RANDOM SAMPLING – Samples chosen from pre-existing groups SYSTEMATIC SAMPLING – Selection of every nth (i.e., 5th) subject in the population CONVENIENCE SAMPLING – Subjects are easily accessible PURPOSIVE SAMPLING – Subjects are selected because of some characteristic
- 26. Sample Size Larger Samplesare needed when… a large number of uncontrolled variables are interacting unpredictably the total sample is to be divided into several subsamples (the researcher is interested in also studying subgroups within the sample) the population is made up of a wide range of variables and characteristics differences in the results (effect size) are expected to be small high attrition of subjects is expected The number of subjects you select (use the sample size calculator to determine this) will influence how confident you can be that your results depict the population from which the sample was drawn. Del Siegle, Ph.D. Neag School of Education – University of Connecticut del.siegle@uconn.edu www.delsiegle.com
- 27. Types of samples Thebest sampling is probability sampling, because it increases the likelihood of obtaining samples that are representative of the population. Probability sampling (Representative samples) Probability samples are selected in such a way as to be representative of the population. They provide the most valid or credible results because they reflect the characteristics of the population from which they are selected (e.g., residents of a particular community, students at an elementary school, etc.). There are two types of probability samples: random and stratified. Random sample The term random has a very precise meaning. Each individual in the population of interest has an equal likelihood of selection. This is a very strict meaning -- you can't just collect responses on the street and have a random sample. The assumption of an equal chance of selection means that sources such as a telephone book or voter registration lists are not adequate for providing a random sample of a community. In both these cases there will be a number of residents whose names are not listed. Telephone surveys get around this problem by random-digit dialing -- but that assumes that everyone in the population has a telephone. The key to random selection is that there is no bias involved in the selection of the sample. Any variation between the sample characteristics and the population characteristics is only a matter of chance. Stratified sample A stratified sample is a mini-reproduction of the population. Before sampling, the population is divided into characteristics of importance for the research. For example, by gender, social class, education level, religion, etc. Then the population is randomly sampled within each category or stratum. If 38% of the population is college-educated, then 38% of the sample is randomly selected from the college-educated population. Stratified samples are as good as or better than random samples, but they require a fairly detailed advance knowledge of the population characteristics, and therefore are more difficult to construct. How to Construct a probability (representative) sample Nonprobability samples (Non-representative samples)
- 28. As they arenot truly representative, non-probability samples are less desirable than probability samples. However, a researcher may not be able to obtain a random or stratified sample, or it may be too expensive. A researcher may not care about generalizing to a larger population. The validity of non-probability samples can be increased by trying to approximate random selection, and by eliminating as many sources of bias as possible. Quota sample The defining characteristic of a quota sample is that the researcher deliberately sets the proportions of levels or strata within the sample. This is generally done to insure the inclusion of a particular segment of the population. The proportions may or may not differ dramatically from the actual proportion in the population. The researcher sets a quota, independent of population characteristics. Two of each species Example: A researcher is interested in the attitudes of members of different religions towards the death penalty. In Iowa a random sample might miss Muslims (because there are not many in that state). To be sure of their inclusion, a researcher could set a quota of 3% Muslim for the sample. However, the sample will no longer be representative of the actual proportions in the population. This may limit generalizing to the state population. But the quota will guarantee that the views of Muslims are represented in the survey. Purposive sample A purposive sample is a non-representative subset of some larger population, and is constructed to serve a very specific need or purpose. A researcher may have a specific group in mind, such as high level business executives. It may not be possible to specify the population -- they would not all be known, and access will be difficult. The researcher will attempt to zero in on the target group, interviewing whomever is available. A subset of a purposive sample is a snowball sample -- so named because one picks up the sample along the way, analogous to a snowball accumulating snow. A snowball sample is achieved by asking a participant to suggest someone else who might be willing or appropriate for the study. Snowball samples are particularly useful in hard-to-track populations, such as truants, drug users, etc.
- 29. Convenience sample A conveniencesample is a matter of taking what you can get. It is an accidental sample. Although selection may be unguided, it probably is not random, using the correct definition of everyone in the population having an equal chance of being selected. Volunteers would constitute a convenience sample. Non-probability samples are limited with regard to generalization. Because they do not truly represent a population, we cannot make valid inferences about the larger group from which they are drawn. Validity can be increased by approximating random selection as much as possible, and making every attempt to avoid introducing bias into sample selection. Examples of nonprobability samples Self-test #1: Sample types Self-test #2: Using the random numbers table Continue on to Sample size Sampling Brooke is a psychologist who is interested in studying how much stress college students face during finals. She works at a university, so she is planning to send out a survey around finals time and ask some students to rank on a scale of 1 to 5 how stressed out they are. But which students should she survey? All of the students at the university? Only the students in the psychology department? Only freshmen? There are a lot of possibilities for Brooke's sample. The sample of a study is simply the participants in a study. In Brooke's case, her sample will be the students who fill out her survey. Sampling is the process whereby a researcher chooses her sample. This might seem pretty straightforward: just get some people together, right? But how does Brooke do that? Should she just stand on a corner and start asking people to take her survey? Should she send out an email to every college student in the world? Where does she even begin?
- 30. Because sampling isn'tas straightforward as it initially seems, there is a set process to help researchers choose a good sample. Let's look closer at the process and importance of sampling. Process So Brooke wants to choose a group of college students to take part in her study. To select her sample, she goes through the basic steps of sampling. 1. Identify the population of interest. A population is the group of people that you want to make assumptions about. For example, Brooke wants to know how much stress college students experience during finals. Her population is every college student in the world because that's who she's interested in. Of course, there's no way that Brooke can feasibly study every college student in the world, so she moves on to the next step. 2. Specify a sampling frame. A sampling frame is the group of people from which you will draw your sample. For example, Brooke might decide that her sampling frame is every student at the university where she works. Notice that a sampling frame is not as large as the population, but it's still a pretty big group of people. Brooke still won't be able to study every single student at her university, but that's a good place from which to draw her sample. 3. Specify a sampling method. There are basically two ways to choose a sample from a sampling frame: randomly or non-randomly. There are benefits to both. Basically, if your sampling frame is approximately the same demographic makeup as your population, you probably want to randomly select your sample, perhaps by flipping a coin or drawing names out of a hat. But what if your sampling frame does not really represent your population? For example, what if the school where Brooke works has a lot more men than women and a lot more whites than minority races? In the population of every college student in the world, there might be more of a balance, but Brooke's sampling frame (her school) doesn't really represent that well. In that case, she might want to non-randomly select her sample in order to get a demographic makeup that is closer to that of her population. 4. Determine the sample size. In general, larger samples are better, but they also require more time and effort to manage. If Brooke ends up having to go through 1,000 surveys, it will take her more time than if she only has to go through 10 surveys. But the results of her study will be stronger with 1,000 surveys, so she (like all researchers) has to make choices and find a balance between what will give her good data and what is practical. 5. Implement the plan. Once you know your population, sampling frame, sampling method, and sample size, you can use all that information to choose your sample. Importance
- 31. As you cansee, choosing a sample is a complicated process. You might be wondering why it has to be that complicated. Why bother going through all those steps? Why not just go to a class and pull some students out and have them fill out the survey? Why is sampling so important to research? To answer those questions, let's look at an example of an actual study that was done in the mid-1970s. A researcher mailed out surveys to a bunch of married women and asked them questions about their marriage. Only 4% of people responded, and of those who did, 98% said they were dissatisfied in their marriage, and 75% said they had or were having an extramarital affair.