The document defines experimental research and provides details on key aspects of experimental design. It discusses that experimental research involves testing ideas to determine their effect on outcomes. Random assignment of participants to groups is a critical characteristic, as it helps control for extraneous variables. Various types of experimental designs are described, including between-group designs like true experiments, quasi-experiments, and factorial designs, and within-group designs like time series experiments, repeated measures experiments, and single subject experiments. Threats to validity like internal, external, and statistical conclusion validity are also outlined. The document provides guidance on how to properly conduct an experiment and evaluate its quality.
Characteristics of a good researcher - am i a researcher?Dr. Mazlan Abbas
Presentation to IIUM - Industry Talk
March 15, 2013 @ 3.00pm
Auditorium B, E2-Level 2,
Kulliyyah of Engineering
International Islamic University Malaysia (IIUM),
Gombak, Malaysia
Thematic analysis in qualitative research Explained with ExampleSufi Nouman Riaz
https://youtu.be/QNP4KkNFzu4
Thematic analysis is a technique of data analysis while conducting a qualitative study. Thematic analysis is the most recognized, adapted, and used approach to analyze qualitative data.
This video is made as per the illustrations and procedures explained in the Braun and Clarke (2006) research article on Thematic Analysis.
Have you just conducted a qualitative study involving:
Interviews
Focus Groups
Observations
Document or artifact analysis
Journal notes or reflections?
How to use this type of data?
Just as there are numerous statistical tests to run for quantitative data, there are just as many options for qualitative data analysis.
THEMATIC APPROACH
Most common forms of analysis in qualitative research. It emphasizes Pinpointing, Examining, Recording
Patterns (or "themes") within data.
Themes are patterns across data sets that are important to the description of a phenomenon and are associated to a specific research question.
Themes become categories for analysis
6 Phases of Coding
(Thematic Analysis)
1-Familiarization with data
2-Generating initial codes
3-Searching for themes among codes
4-Reviewing themes
5-Defining and naming themes
6-Producing the final report
There are three major methods of research. the method which enquirers regarding past is called as historical method. In this presentation, historical method is described in brief.
Characteristics of a good researcher - am i a researcher?Dr. Mazlan Abbas
Presentation to IIUM - Industry Talk
March 15, 2013 @ 3.00pm
Auditorium B, E2-Level 2,
Kulliyyah of Engineering
International Islamic University Malaysia (IIUM),
Gombak, Malaysia
Thematic analysis in qualitative research Explained with ExampleSufi Nouman Riaz
https://youtu.be/QNP4KkNFzu4
Thematic analysis is a technique of data analysis while conducting a qualitative study. Thematic analysis is the most recognized, adapted, and used approach to analyze qualitative data.
This video is made as per the illustrations and procedures explained in the Braun and Clarke (2006) research article on Thematic Analysis.
Have you just conducted a qualitative study involving:
Interviews
Focus Groups
Observations
Document or artifact analysis
Journal notes or reflections?
How to use this type of data?
Just as there are numerous statistical tests to run for quantitative data, there are just as many options for qualitative data analysis.
THEMATIC APPROACH
Most common forms of analysis in qualitative research. It emphasizes Pinpointing, Examining, Recording
Patterns (or "themes") within data.
Themes are patterns across data sets that are important to the description of a phenomenon and are associated to a specific research question.
Themes become categories for analysis
6 Phases of Coding
(Thematic Analysis)
1-Familiarization with data
2-Generating initial codes
3-Searching for themes among codes
4-Reviewing themes
5-Defining and naming themes
6-Producing the final report
There are three major methods of research. the method which enquirers regarding past is called as historical method. In this presentation, historical method is described in brief.
This presentation is for educational purpose only. I do not own the rights to written material or pictures or illustrations used.
This is being uploaded for students who are in search of, or trying to understand how a quasi-experimental research design should look like.
This slide set provides an introduction to narrative inquiry in education research. Narrative and narratology provides a new model for a critical language in education
This presentation is for educational purpose only. I do not own the rights to written material or pictures or illustrations used.
This is being uploaded for students who are in search of, or trying to understand how a quasi-experimental research design should look like.
This slide set provides an introduction to narrative inquiry in education research. Narrative and narratology provides a new model for a critical language in education
All the concepts related to research design are covered in this PPT Presentation.Research Design being an integral and crucial part of Research majorly deals with Parametric and non-parametric test, Type 1 and type 2 error, level of significance etc.It helps in ascertaining which research technique is used in which situation.
Welcome to TechSoup New Member Orientation and Q&A (May 2024).pdfTechSoup
In this webinar you will learn how your organization can access TechSoup's wide variety of product discount and donation programs. From hardware to software, we'll give you a tour of the tools available to help your nonprofit with productivity, collaboration, financial management, donor tracking, security, and more.
Ethnobotany and Ethnopharmacology:
Ethnobotany in herbal drug evaluation,
Impact of Ethnobotany in traditional medicine,
New development in herbals,
Bio-prospecting tools for drug discovery,
Role of Ethnopharmacology in drug evaluation,
Reverse Pharmacology.
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
How to Make a Field invisible in Odoo 17Celine George
It is possible to hide or invisible some fields in odoo. Commonly using “invisible” attribute in the field definition to invisible the fields. This slide will show how to make a field invisible in odoo 17.
Synthetic Fiber Construction in lab .pptxPavel ( NSTU)
Synthetic fiber production is a fascinating and complex field that blends chemistry, engineering, and environmental science. By understanding these aspects, students can gain a comprehensive view of synthetic fiber production, its impact on society and the environment, and the potential for future innovations. Synthetic fibers play a crucial role in modern society, impacting various aspects of daily life, industry, and the environment. ynthetic fibers are integral to modern life, offering a range of benefits from cost-effectiveness and versatility to innovative applications and performance characteristics. While they pose environmental challenges, ongoing research and development aim to create more sustainable and eco-friendly alternatives. Understanding the importance of synthetic fibers helps in appreciating their role in the economy, industry, and daily life, while also emphasizing the need for sustainable practices and innovation.
2. DEFINITION
What is it?
•The testing of an idea (or practice or procedure)
•Determine whether it affects any outcome or dependent variable
•It is the best of the quantitative designs to use to establish probable
cause and effect.
When do
you do it?
•When there is a need to establish possible cause and effect between
independent and dependent variables.
When did it
begin?
•In the late 19th to early 20th century
•1916 : the idea of randomly selecting individuals to groups started
•1925 : firmly established procedure of comparing groups
•1963: major experimental designs identified
•1979: the types of designs are elaborated
•2002: discussions about major experimental designs are refined
3. Before you even think of doing an experiment, you must take note of the
key characteristics first!
1. RANDOM ASSIGNMENT
KEY CHARACTERISTICS OF EXPERIMENT
• The process of assigning individuals at random to groups or to different groups in an
experiment.
• The random assignment of individuals to groups differentiates a rigorous, “true”
experiment from an adequate, but less-than-rigorous “quasi-experiment”.
• Biasness distributed equally among the groups.
• You can control for extraneous characteristics (e.g., student ability, attention span).
• The experimental term for this process is “equating” the groups “ = randomly assigns
individuals to groups and equally distributes any variability of individuals
• BUT, personal factors that participants bring to an experiment can never be totally
controlled—some bias / error will always affect the outcome of a study.
4. Cont..
1. RANDOM ASSIGNMENT
KEY CHARACTERISTICS OF EXPERIMENT
• Random assignment vs random selection
• Impossible to randomly select for
experimental because the participants
are usually those who are available or
volunteer to take part.
• Random assignment is more
sophisticated.
Random selection refers to
how sample are selected from the
population.
Random assignment refers to how
the participants are assigned to the
treatment or control group using
a random procedure.
5. 2. CONTROL OVER EXTRANEOUS VARIABLES
KEY CHARACTERISTICS OF EXPERIMENT
• Extraneous factors are any influences in the selection of
participants/procedures/design etc that are likely to affect the outcome.
• All experiments have some random error that you cannot control, but you can
try to control extraneous factors as much as possible.
• Random assignment is a decision made by the investigator before the
experiment begins.
• Other control procedures before and during the experiment:
• Pretests
• Covariates
• Matching of participants
• Homogeneous samples
• Blocking variables.
6. A measure on some attribute or characteristic that you assess for participants in an
experiment before they receive a treatment.
A measure on some attribute or characteristic that is assessed for participants in an
experiment after a treatment.
Variables that the researcher controls for using statistics and that relate to the
dependent variable but that do not relate to the independent variable.
Process of identifying personal characteristics that influence the outcome
assigning the characteristics equally to the experimental and control groups.
(gender, pretest scores, abilities etc)
People who vary little in their personal characteristics (only junior/senior students,
racial group etc). The more similar they are, the more controlled it is.
A variable the researcher controls before the experiment starts by dividing (or
“blocking”) the participants into subgroups (or categories). For example : gender –
M&F
Pretest
Posttest
(after)
Covariates
Matching of
Participants
Homogeneous
Samples
Blocking
Variables
7. After you select your participants, you can now:
3. MANIPULATE TREATMENT CONDITIONS.
In experimental treatment, the researcher physically
intervenes to alter the conditions experienced by the
experimental unit (e.g., a reward for good spelling
performance or a special type of classroom instruction, such
as small group discussion).
Independent is the treatment given to the experimental group
Needs to have 2 or more levels. Eg: math training theoretical (1),
math theoretical + practice (2), math theoretical + practice + extra
class (3).
Dependent is the measurable variable
Treatment variables = Independent & Dependent
KEY CHARACTERISTICS OF EXPERIMENT
8. 4. GROUP COMPARISONS
A group comparison is the process of a researcher obtaining
scores on the dependent variable and comparing the means
and variance both within the group and between the groups.
KEY CHARACTERISTICS OF EXPERIMENT
9. 5. THREATS TO VALIDITY
How do you know the inferences you
made from your outcome are valid? Are
they really true and correct?
Threats to validity refer to specific
reasons for why we can be wrong when
we make an inference in an experiment
because of covariance, causation
constructs, or whether the causal
relationship holds over variations in
persons, setting, treatments, and
outcomes ( Shadish, Cook, & Campbell,
2002 ).
KEY CHARACTERISTICS OF
EXPERIMENT
10. 5. THREATS TO VALIDITY (cont…)
• Refers to the appropriate use of statistics
• Eg: violating statistical assumptions, restricted range on a variable, etc to infer
whether the presumed independent and dependent variables covary in the
experiment.
Statistical Conclusion Validity
• The validity of inferences about the constructs (or variables) in the study.
Construct Validity
• The validity of inferences drawn about the cause and effect relationship between
the independent and dependent variables.
Internal Validity
• The validity of the cause-and-effect relationship being generalizable to other
persons, settings, treatment variables, and measures.
External Validity
11. 5. THREATS TO VALIDITY (cont…)
Threats to internal validity
Participant
History
Maturation
Regression
Selection
Mortality
Interactions with
selection
Treatment
Diffusion of
treatments
Compensatory
equalization
Compensatory rivalry
Resentful
demoralization
Procedure
Testing
Instrumentation
12. 5. THREATS TO VALIDITY (cont…)
Threats to external validity
•Inability to generalize beyond the groups in the experiment
•Eg: other racial, social, geographical, age, gender etc.
•So, increase generalizability by making participation as
convenient as possible for all individuals in a population.
Interaction of
selection and
treatment
•The inability to generalize from the setting where the
experiment occurred to another setting.
•Eg: private high schools different from public high schools
•So, analyze the effect of a treatment for each type of setting
Interaction of
setting and
treatment
•When the researcher tries to generalize findings to past and
future situations.
•So, replicate the study at a later time rather than trying to
generalize results to other times.
Interaction of
history and
treatment
13. ◆ Between Group Designs
• True experiments (pre- and posttest, posttest only)
• Quasi-experiments (pre- and posttest, posttest only)
• Factorial designs
◆ Within Group or Individual Designs
• Time series experiments (interrupted, equivalent)
• Repeated measures experiments
• Single subject experiments
TYPES OF EXPERIMENTAL DESIGN
14. ◆ Between Group Designs
It is a strong experimental design - random assignment.
Experimental group receives treatment, but not control group.
Then, compile average (or mean) scores on a posttest.
When collecting pretest scores, you may compare net scores
(the differences between the pre- and posttests).
You may relate the pretest scores for the control and
experimental groups to see if they are statistically similar
Then, compare the two posttest group scores.
As you randomly assign individuals to the groups, most of the
threats to internal validity do not arise.
When true experiments include only a posttest, it reduces the
threats of testing, instrumentation, and regression because
you don’t use a pretest.
• True experiments (pre- and posttest, posttest only)
15. ◆ Between Group Designs
NO random assignment of participants to groups.
WHY? This is because you cannot artificially create groups for the
experiment.
Eg: studying new math program requires using existing fourth-grade
classes (one as experimental and one as control group). Randomly
assigning students to the two groups would disrupt classroom
learning.
Has more threats to internal validity than the true experiment:
(threats to participants, treatments and procedure are possible)
While the quasi-experimental design has the advantage of utilizing
existing groups in educational settings, it introduces many threats
that you need to address in the design of the experiment
• Quasi experiments (pre- and posttest, posttest only)
16. ◆ Between Group Designs
A study of two or more categorical, independent variables; each
examined at two or more levels (Vogt, 2005).
For example, in the math training program, you wish to combine the
influence of all the types of instruction and their motivation to learn
math
Assume that you have a reason to believe that motivation is
important but its “interaction” or combination with type of
instruction is unknown.
So, you need factorial design.
Thus, “motivation” is a blocking variable and you make random
assignment of each “block” (high, medium, and low) to each
treatment instructional group.
So, if you have 2 independent type of math training and motivation
(3 levels) – you will have a “2 x 3” factorial design. This will then go
against the result of the posttest.
• Factorial Design
17. The study of one
group, over time,
with multiple pretest
and posttest.
Does not require
large numbers of
participants.
Requires only one
group for the study.
There two important
variations:
Interrupted
Equivalent
• Time series experiments (interrupted, equivalent)
Interrupted time series design
•Study one group, obtain multiple pretest over
time, administering an intervention
•Then, measure outcomes (posttests) several
times
•Data analysis: pretests vs posttests or
posttest-only scores and using the pretests as
covariates
Equivalent time series design
•Alternate a treatment/intervention with a
posttest measure.
•Data analysis: comparing posttest measures.
◆ Within Group or Individual Designs
18. All participants in a single group participate in all experimental
treatments, with each group becoming its own control.
It compares a group’s performance under one experimental
treatment with its performance under another experimental
treatment.
The experimenter decides on multiple treatments (as in factorial
designs) but administers each one by one to only one group.
• Repeated measures
◆ Within Group or Individual Designs
19. Assume that you want to learn about the behavior of single individuals
and not groups.
It is the study of single individuals:
Observe them over a period of time
Administer an intervention.
Another observation after the intervention to see if treatment affects outcome.
• Single Subject
◆ Within Group or Individual Designs
A/B SINGLE DESIGN
(A): Administering an
intervention
(B): Observing and
measuring the behavior
after the intervention
20. Multiple Baseline Design:
each participant receives
an experimental treatment
at a different time (hence,
multiple baselines exist)
WHY? So that treatment
diffusion will not occur
among participants.
Choose this design when
the treatment (e.g., skill or
strategy being taught)
cannot be reversed and
doing so would be unethical
or injurious to participants.
• Single Subject (CONT…)
◆ Within Group or Individual Designs
• An alternating treatment design
is a single-subject design in
which the researcher examines
the relative effects of two or
more interventions and
determines which intervention is
the more effective treatment on
the outcome.
21. 1. Decide if an
experiment can solve
your RQ.
• Do you need to know if
there is a cause-effect
relationship?
2. Form hypotheses to
test the C/E
relationship
3. Select an
experimental unit &
identify participants
•Unit – individual, groups
•How to choose participants?
•How to assign to groups?
4. Select experimental
treatment & introduce
•Should have little intrusion
•Based on small pilot test
•Good intervention can
predict change
5. Choose a type of
experimental design
6. Conduct the
experiment
• Do pretests
• Introduce treatment
• Observe & posttest
7. Organize &
analyze data
8. Develop an
experimental
research report
HOW TO CONDUCT AN EXPERIMENT?
22. For a good experiment, here are some criteria:
The experiment has a powerful intervention.
The treatment groups are few in number.
Participants will gain from the intervention.
The researcher derives the number of participants per group in some
systematic way.
An adequate number of participants were used in the study.
The researcher uses measures and observations that are valid,
reliable, and sensitive.
The researcher controls for extraneous factors that might influence
the outcome.
The researcher addresses threats to internal and external validity.
EVALUATE THE QUALITY