TSL3133 Topic 4 Educational Research Procedure

Lecturer: Yee Bee Choo
IPGKTHO
Topic 4

Educational Research Procedure
1. Choosing a research problem
2. Determining the purpose statement
3. Determining the research objective
4. Determining the research problems
5. Determining the research hypotheses
6. Reviewing the literature
7. Planning the research design
8. Determining the sampling procedure
9. Developing research instrument
10. Determining the validity and reliability of research instrument
11. Determining the data collection procedure
12. Collecting data
13. Analysing and interpreting data
14. Discussing and reporting research findings

1. Choosing a Research Problem
• A research problem is an educational issue,
controversy, or concern that guides the need
for conducting a study.
• Examples of good research problems for
educational are:
1. The disruptions caused by at-risk students in
classrooms
2. The increase in violence on college campuses
3. The lack of parental involvement in schools
for students with challenging behaviours
(Creswell, 2012)

• 5 ways to assess whether you should research a
problem:
1. Study the problem if your study will fill a gap or
void in the existing literature
2. Study the problem if your study replicates a past
study but examines different participants and
different research sites
3. Study the problem if your study extends past
research or examines the topic more thoroughly
4. Study the problem if your study gives voice to
people silenced, not heard, or rejected in society
5. Study the problem if your study informs practice
(Creswell, 2012)

How to justify the need
to study the research
problem?
Personal experiences
Used in qualitative
research, such as
narrative research
Personal or others’
experiences in the
workplace
Used in both
quantitative and
qualitative research,
such as action research
Suggestions made by
other researchers as
reported in the
literature
Used in both quantitative
and qualitative research,
such as experiments,
surveys, correlational
studies, grounded theory
studies
(Creswell, 2012)

Flow of ideas to a “Statement of Problem” Section (Creswell, 2012)
Topic Research
Problem
Justifications
for the
Research
Problem
Deficiencies
in the
Evidences
Relating the
Discussion to
Audiences
Subject area A concern
or issue, a
problem
Evidence from
the literature
or practical
experience
Evidence that
is missing
Audiences that will
profit from the
study
Example
Parents’ role in
promoting
access to
college by
underrepresen
ted students
Need for
better
access for
students of
colour
Past literature
has
documented
poor
attendance
We need to
evaluate how
parents can
promote
access
Parents can better
assess their role
Counselors can
better involve
parents
Colleges can better
work with parents

2. Determining the Purpose Statement
• The purpose statement is a statement that
advances the overall direction or focus for the
study.
• It describes the intent of the study, the
objectives, and the major idea of a study.
• This idea builds on a need (the problem) and is
refined into specific questions, the research
questions.
• It often appears as the last sentence of an
introduction which begins with the phrase “The
purpose of this study is…”

3. Determining the Research Objective
• A research objective is a statement that
specifies goals that the researcher plans to
achieve in a study.

4. Determining the Research Questions
• Research questions are questions that narrow
the purpose statement and research
objectives to specific questions that the
researcher seeks to answer.

Feasible —enable you to collect data.
Clear (i.e. most people agree as to what the key terms in the question
mean).
Significant (i.e. it is worth investigating because it will contribute important
knowledge about the human condition)
Ethical (i.e. it will not involve physical or psychological harm or damage to
human beings, or to the natural or social environment of which they are a
part)
Indicates a relationship of some sort (i.e. two or more qualities are
suggested as being connected or related to each other in some way).
A research questions should be:

Examples of research questions with appropriate methodology
1. Does client-centered therapy produce more satisfaction in
clients than does traditional therapy?
(experimental research)
2. How do parents feel about the school counselling
programme?
(survey research)
3. What are the cultural influences that affect the professional
development of athletic trainers in a high school setting?
(ethnographic research)
4. How do nontraditional students experience clinical education
with people who are younger than themselves?
(phenomenological research)

Examples of research questions with appropriate methodology
5. Do teachers behave differently toward students of different
genders?
(causal-comparative research)
6. How can we predict which students might have trouble
learning certain kinds of subject matter?
(correlational research)
7. What was the process of implementing (or not-implementing)
the protocols (from the perspective of dentists, practice staff,
and patients)? How did this process vary?
(grounded theory research)
8. Are the descriptions of people in social studies in textbooks
biased?
(content analysis research)

Differences among the Topic, Research Problem,
Purpose and Research Questions
• A research topic is the broad subject matter
addressed by the study.
• A research problem is a general educational
issue, concern or controversy addressed in
research that narrows the topic.
• A purpose is the major intent or objective of
the study used to address the problem.
• Research questions narrow the purpose into
specific questions that the researcher would
like answered or addressed in the study.

Differences among the Topic, Research Problem,
Purpose and Research Questions
Topic
Research Problem
Purpose of
Statement
Research
Questions
General
Specific
Distance Learning
Lack of students in
distance education
classes
To study why students do
not attend distance
education classes at a
community college
Does the use of Website
technology in the
classroom deter students
from enrolling in a
distance education class?
(Creswell, 2012)

Which research question suggests relationship?
1. To what extent are manipulatives used to teach mathematics in the
primary schools in Batu Pahat District?
2. Do the principals in our district favor mainstreaming gifted students?
3. Are people with eating disorders more likely to have a history of sexual
abuse in early childhood?
4. Do students who are taught English in a language laboratory become
more fluent in the language than students who are taught English in a
traditional classroom setting?

Question 1 and 2 do not suggest a
relationship.
Question 1 asks for no more than a description regarding the
current usage of manipulative materials in a particular school
district.
Similarly, question 2 asks only for a survey of administrative
opinions. Investigations of such questions may be extremely
useful in their own right, but they do not extend our
knowledge as to why such conditions exist.
Discussion

Question 3 and 4 indicate a relationship.
Question 3 seeks to investigate a possible relationship between eating
disorders and sexual abuse. If a history of sexual abuse is related to eating
disorders, this suggests (although it does not prove) that such abuse may be
a cause of such disorders. It also suggests that counselling which addresses
patient history may be helpful.
Question 4 seeks to investigate a possible relationship between the type of
language instruction and fluency in the language taught. If the language
laboratory method is shown to be more effective than classroom instruction
by individual teachers, this has clear implications for improving language
learning.
Discussion

A variable is any
characteristic that is not
always the same—that is,
any characteristic that
varies.
Examples of variables
include gender, eye
color, achievement,
motivation, and
running speed.
18
Relationship and Variable

“Are younger students more anxious in
Mathematics courses than older
students?”
Answer:
The variables are age and level of
anxiety in Mathematics courses.
19
What are the variables in this research question?

Measured/ Quantitative Variables
• exist in some degree rather than all or none
• are measured along a continuum from “less” to “more”
• assigned numbers to different individuals or objects
• An example would be height.
Categorical/ Qualitative Variables
• not vary in degree, amount, or quantity, but are qualitatively
different
• e.g. eye color, gender, religious preference, occupation, position
on a baseball team, and most kinds of “treatments” or
“methods”
20
Quantitative Vs Qualitative Variables

A categorical variable is usually an independent or predictor variable
that contains values indicating membership in one of several
possible categories. E.g., gender (male or female), marital status
(married, single, divorced, widowed). The categories are often
assigned numerical values used as labels, e.g., 0 = male; 1 = female.
Synonym for nominal variable.
A continuous variable is a variable that is not restricted to particular
values (other than limited by the accuracy of the measuring
instrument). E.g., reaction time, neuroticism, IQ. Equal size intervals
on different parts of the scale are assumed, if not demonstrated.
Synonym for interval, rating or scaled variable.
21
Categorical and Continuous Variables

Independent Variables
• are those the investigator chooses to study (and often
manipulate) in order to assess their possible effect(s) on
one or more other variables
• are those the investigator chooses to study (and often
manipulate) in order to assess their possible effect(s) on
one or more other variables
Dependent Variable
• is the variable which the independent variable is
presumed to affect
• All outcome variables are dependent variables. 22
Independent Vs Dependent Variables

“Will students who are taught by a team of three
teachers learn more than students taught by one
individual teacher?”
What are the independent and dependent variables in this question?
Answer:
Do you see that the independent variable is
the number of teachers, and the dependent
variable is the amount of learning? Notice that
there are two conditions (sometimes called
levels) of the independent variable—three
teachers and one teacher. Also notice that the
dependent variable is not “learning” but
“amount of learning”. Can you see why?

The Family of Variables in Quantitative Studies
Independent
Variables
-Treatment
-Measured
Intervening
Variables
Dependent
Variables
Moderating
Variables
Confounding
Variables
Control
Variables
(Creswell, 2012)

Four Types of Independent Variables (Creswell, 2012)
Measured
Variable
Control Variable Treatment Variable Moderating Variable
Definition An independent
variable that is
measured in a
study
A special type of
independent that is
of secondary interest
and is neutralised
through statistical or
design procedures
An independent variable
manipulated by the researcher
A special type of independent
variable that is of secondary
interest and combines with
another independent variable to
influence the dependent
variable
Type of
Variable
Measure
ment
A categorical
continuous
variable that is
measured or
observed in a
study
A variable not directly
measured but
controlled through
statistical or research
design procedures
A categorical variable actively
manipulated by the researcher
and composed of two or more
groups
A categorical or continuous
variable measured or observed
as it interacts with other
variables
Use in Experiments,
surveys
Experiments,
correlational studies
Experiments Experiments
Examples Age of a child,
performance on
a test, attitudes
assessed on a
survey
Often demographic
variables such as age,
gender, race,
socioeconomic level
Classroom learning: one group
receives standard lecture and
one group receives discussion;
researcher assigns students to
groups and thus manipulates
group membership
Demographic variables such as
age, gender, race, socioeconomic
level, a measured variable such
as performance or attitude, or a
manipulated variable such as
classroom instruction

Intervening Variables (sometimes called mediating
variables) are variables that stand between
independent and dependent variables.
Example: Parents transmit their social status to their
children directly, but they also do so indirectly,
through education: viz.
Parent’s status ➛ child’s education ➛ child’s status
Intervening Variables

Confounding variables (sometimes called spurious variables) are
variables that researcher cannot directly measured because their effects
cannot be easily separated from those of other variables.
Example: If one elementary reading teacher used a phonics textbook in
her class and another instructor used a whole language textbook in his
class, and students in the two classes were given achievement tests to
see how well they read, the independent variables (teacher
effectiveness and textbooks) would be confounded. There is no way to
determine if differences in reading between the two classes were
caused by either or both of the independent variables.
Confounding Variables

Extraneous Variables are
independent variables that have
not been controlled
Constants are potential variables
that are not allowed to change
Extraneous Variables and Constants

Identify the independent, dependent and extraneous variables
in the statement below.
“Effects of different teaching strategies on students’
achievement.”
Dependent variable
Achievement
Independent variable
Teaching strategies
Extraneous variable
Teacher
Time of day
Student ability

There is a relationship between children gains in
social studies achievement and their self concept
Identify the independent, dependent and
extraneous variables in the statement below.
IV: self-concept
DV: social studies achievement
EV: Intelligence may as well affect on the social achievement. But
it is not related to the study undertaken by the researcher so
it is an extraneous variable.

5. Determining the Research Hypotheses
• Hypotheses are statements in quantitative
research in which the investigator makes a
prediction about the outcome of a relationship
among attributes or characteristics.
• They are based on results from past research and
literature where researchers have found certain
results and can now offer predictions as to what
other researchers will find when they repeat the
study with new people or at new sites.
(Creswell, 2012)

• A hypothesis is, simply put, a prediction of some
sort regarding the possible outcomes of a study.
• A research question is often restated as a
hypothesis.
For example:
• Question: “Do individuals who see themselves as
socially attractive want their romantic partners
also to be socially attractive?”
• Hypothesis: “Individuals who see themselves as
socially attractive will want their romantic
partners also to be (as judged by others) socially
attractive.”

• A Directional Hypothesis is one that indicates
the specific direction (e.g. higher, lower, more,
less) that a researcher expects to emerge in a
relationship.
• Nondirectional Hypothesis does not make a
specific prediction about what direction the
outcome of a study will take.
Directional and Nondirectional Hypotheses

• A null hypothesis makes predictions that of all possible
researchers might study, there is no relationship
between independent and dependent variables or no
difference between groups of an independent variable
or a dependent variable.
• An alternative hypothesis is any hypothesis other than
the null.
• If the null hypothesis is not accepted, then the
alternative hypothesis is used.
Null and Alternative Hypotheses

Examples
Research question Research hypothesis Null hypothesis
Is there a relationship
between teachers’
attitudes toward the
curriculum & student
achievement?
Is there a difference in
achievement between
students who are given
detailed written
comments on their
work compared to
grades only?
There is a positive
correlation between
teachers’ attitudes
toward the curriculum
& student
achievement.
Students receiving
detailed written
comments on their
work will show higher
achievement than
grades only
There is no correlation
between teachers’
attitudes toward the
curriculum & student
achievement.
There is no difference
in achievement
between students
receiving detailed
written comments on
their work compared to
grades only

Example:
There is no difference between at-risk and non-at-risk
students in terms of student achievement on Math test
scores for Year Three students in the primary schools in
Batu Pahat district.
Identify if this is a Null Hypotheses or Alternative Hypotheses.
Also, identify the independent and dependent variables.
This is a Null Hypotheses.
IV: at-risk students (members and non-members)
DV: student achievement test scores

6. Reviewing the Literature
• A literature review is a written summary of
journal articles, books, and other documents that
describes the past and current state of
information on the topic of your research study.
• This review servers the purpose of providing a
need for a study and demonstrating that other
studies have not addressed the same topic in
exactly the same way.
• It also indicates to audiences that the researcher
is knowledgeable about studies related to topic.

Sources of Literature Review
General References: the sources a researcher
refers to first.
Secondary Sources: publications in which
authors describe the work of others.
Primary Sources: publications in which
investigators report the results of their
studies.

5 steps in conducting a literature review
1.
• Identify key terms to use in your search for literature.
2.
• Locate literature about a topic by consulting several types of
materials and databases, including those available at an academic
library and on the Internet.
3.
• Critically evaluate and select the literature for your review.
4.
• Organise the literature you have selected by abstracting or taking
notes on the literature and developing a visual diagram of it.
5.
• Write a literature review that reports summaries of the literature for
inclusion in your research report.

7. Planning the Research Design
• A research design is a plan or strategy for
conducting the research.
• It spells out the basic strategies that researchers
adopt to develop evidence that is accurate and
interpretive.
• It is the master plan specifying the methods and
procedures for collecting and analysing the
needed information.
• It deals with matters such as selecting
participants for the research and preparing for
data collection.

8. Determining the Sampling Procedure
Subject: an individual who participates in a research study or
is someone from whom data are collected. (McMillan, 1992:
68)
Population: a group of elements or cases, whether individuals,
objects, or events, that conform to specific criteria and to
which we intend to generalize the results of the research.
(McMillan, 1992: 69)
Sample: the group of elements, or a single element, from
which data are obtained.
Subjects and Sampling

Examples of Population
• All of the high school principals in the United
States.
• All of the elementary school counselors in
the state of California.
• All of the students attending Central High
School in Omaha, Nebraska, during the
academic year 1987-1988.
• All of the students in Mrs. Browns’ third-
grade class at Wharton Elementary School.

Examples of Sample
• A researcher is interested in studying the effects of diet on
the attention span of third-grade students in a large city.
There are 1500 third graders attending the elementary
schools in the city. The researcher selects 150 of these third
graders, 30 each in five different schools, to study.
• The principal of an elementary school district wants to
investigate the effectiveness of a new U.S. history textbook
being used by some of teachers in her district. Out of a total
22 teachers who are using the text, she selects 6, comparing
the achievement of students in the classes of these 6
teachers with those of another 6 teachers who are not using
the text.

Probability
Sampling
Nonprobability
Sampling
Simple Random Sampling
Stratified Random Sampling
Random Cluster Sampling
Two Stage Random sampling
Convenience Sampling
Systematic Sampling
Purposive Sampling
Sampling Procedure

Simple Random Sampling (SRS)
• In SRS every member of the population has an
equal and independent chance of being
selected for the sample.
• Example:" We interviewed a sample of 41
mothers of eight graders from one middle
school. These mothers were randomly
selected from a list of 129 mothers provided
by the principal of the school.” (Baker and
Stevenson, 1986, p.157).

Population
Sample
A
C
B
D
F
G
I
H
K
J
M
L
E
O
N
Q
P
S
R
U
T
ZV
W
YX
N
D
P
H
L
Y
Simple Random

Stratified Random Sampling
• Stratified sampling is a process whereby
certain subgroups, or strata, are selected for
the sample in the same proportion as they
exist in the population.
• Example: ”From a pool of all children who
returned a parental permission form (more
than 80% return rate) 24 first graders (10
girls, 14 boys; mean age, 6 years, 6 months),
and 24 third graders (13 girls, 11 boys; mean
age, 8 years, 8 months) were randomly
selected.” (Clements and Nastasi, 1988, p.93)

Population
Sample
Stratified Random
A B C D E
25%
F G H I J
K L M N O
50%
P Q R S T
25%
B D
25%
F M O J
50%
P S
25%

Cluster Random Sampling
When it is not possible to select a sample of
individuals from a population--for example, a
list of all members of the population of interest
is not available—cluster sampling is used. It
involves the random selection of naturally
occurring groups or areas and then the selection
of individual elements from the chosen groups
or areas.

Population
Sample
QR
AB
Cluster Random
STU
NOP
CD
EFG
LM
JK
HI
EFG
CDQR

Two-Stage Random Sampling
It is often useful to combine cluster sampling
with individual sampling. Rather than randomly
selecting 200 students from a population of
3000 ninth graders located in 100 classes, the
researcher might decide to select 25 classes
randomly from the population of 100 classes
and then randomly select 8 students from each
class.

Population
Sample
QR
AB
Two-Stage Random
STU
NOP
CD
EFG
LM
JK
HI
STU
LMCD
C,L,T
Sample of clusters
Sample of individuals

Convenience Sampling
• A convenience sample is a group of
individuals who (conveniently) are available
for study.
• Example:" A high school counselor interviews
all of the students who come to her for
counseling about their carrier plans.”

Population
Sample
A
C
B
D
F
G
H
K
J
M
E
O
N
P
SR
U
T
Z
V
W
L
Q
X
I
Y
Convenience
Q
X
I
Y
L
Easily Accessible

Purposive Sampling
• In purposive sampling the researcher selects
particular elements from the population that
will be representative or informative about
the topic.
• Purposive sampling is different from
convenience sampling in that the researcher
does not simply study whoever is available,
but uses his or her judgment to select the
sample for a specific purpose.

Example of Purposive Sampling
• “Introductory psychology students (N=210)
volunteered to take the Dogmatism Scale
(Form E) for experimental credit. From the
upper and lower quartiles on the Dogmatism
Scale, 44 high and 44 low dogmatic subjects
were selected for the experiment.” (Rickards
and Slife, 1987, pp.636-637)

Population
Sample
A
C
B
D
F
G
I
H
K
J
M
L
E
O
N
Q
P
S
R
U
T
ZV
W
YX
N
B
V
L
F
Purposive

Systematic Sampling
• In systematic sampling every nth element is
selected from a list of all elements in the
population.

A C D E
F H I J
K M N O
P R S T
Population
Sample
B G L
Q
Systematic B
G
L
Q

RESEARCH DESIGN
Nonexperimental Research Experimental Research
Weak Experimental Designs:
• The One-Shot Case Study Design
• The One-Group Pretest-Posttest Design
• The Static-Group Comparison Design
Descriptive Studies
Relationship Studies
e.g. Simple Correlational
Studies, and Prediction Studies
Causal-Comparative Studies
True Experimental Design
•The Randomized Posttest-Only Control
Group Design
•The Randomized Pretest-Posttest Control
Group Design
•The Randomized Solomon Four-Group Design
Quasi-Experimental Design
• The Matching Only Posttest-Only Control
Group Design
• The Matching Only Pretest-Posttest Control
Group Design
True Experimental Designs in Suter (1998)

Weak Experimental Designs
• These designs are referred to as
“weak” because they do not have
built-in controls for threats to internal
validity.
• Any researcher who uses one of these
designs has difficulty assessing the
effectiveness of the independent
variable.

1. The One-Shot Case Study: a single group is
exposed to a treatment or event, and a
dependent is subsequently observed
(measured) in order to assess the effect of the
treatment.
X O
treatment Observation
(dependent variable)

2. The One-Group Pretest-Posttest Design: a
single group is measured or observed, not
only after being exposed to a treatment of
some sort, but also before.
O X O
treatment Posttest
Pretest

3. The Static-Group Comparison Design: Two
already existing, or intact, are used.
Comparisons are made between groups
receiving different treatments.
X1 O
Note:
------ : already formed, not randomly assigned
X1 and X2: different treatments
Os : placed vertically to each other, occurs at the same time
X2 O

True Experimental Designs
Subjects are randomly assigned
to treatment groups for
controlling the subject
characteristics threat to internal
validity.

1. The Randomized Posttest-Only Control Group Design:
involves two groups, one receives the experimental
treatment while the other does not.
Treatment Group R X1 O
Control Group R X2 O
R: random assignment
X1 = T = Treatment
X2 = No treatment
O = test

2. The Randomized Pretest-Posttest Control Group Design:
both groups are measured twice, the first measurement
serves as the pretest, the second as the posttest.
Treatment Group R O X1 O
Control Group R O X2 O

3. The Randomized Solomon Four-Group Design:
involves random assignment of subjects to four groups, with two of the groups being
pretested and two not. One of the pretested groups and one of the unpretested groups
is exposed to the experimental treatment. All four groups are then posttested.
Treatment Group R O X1 O
Control Group R O X2 O
Treatment Group R X1 O
Control Group R X2 O

3. The Randomized Matched Control Group Design:
It is similar to the randomized posttest control group design, but it is distinguished by
the use of matching prior to random assignment. This design is used if the sample size is
too small (perhaps less than 40 per group) to reasonably assure group comparability
after random assignment. Subjects are first rank ordered on a variable closely related to
the posttest. Then one of the two highest – forming matched pair – is randomly
assigned to T or C, with the remaining one being assigned to the other. The next highest
matched pair is similarly assigned, and this until the lowest two matched subjects are
assigned randomly.
Treatment Group M R X1 O
Control Group M R X2 O

Quasi-Experimental Designs
• Do not include the use of random
assignment.
• Researchers who employ these
design rely instead on other
techniques to control (or at least
reduce) threats to internal validity.

A. The Matching Only Design:
• The researcher still matches the subjects in the
experimental and control groups on certain variables,
but he/she has no assurance that they are equivalent
on others since subjects are not randomly assigned to
groups.
• The two groups are intact (they are already existed
before the intervention) and so are probably not
comparable.

1. The Matching Only Posttest-Only Control
Group Design
Treatment Group M X1 O
Control Group M X2 O
M = Matched

2. The Matching Only Pretest-Posttest Control
Group Design
Treatment Group O M X1 O
Control Group O M X2 O

B. Counterbalanced Designs:
• Represent another technique for equating experimental and control groups.
• Each group is exposed to all treatments, however many there are, but in a different order. Any
number of treatments may be involved.
• Researchers determine the effectiveness of the various treatments simply by comparing the
average scores for all groups on the posttest for each treatment.
Example: A Three-Treatment Counterbalanced Design
Group One X1 O X2 O X3 O
Group Two X2 O X3 O X1 O
Group Three X3 O X1 O X2 O

C. Time-Series Designs:
involves repeated measurements or
observations over a period of time both
before and after treatment.
O1 O2 O3 O4 X O5 O6 O7 O8

D. Factorial Design:
• extend the number of relationships that may be examined in an experimental
study
• allows a researcher to study the interaction of an independent variable with
one or more other variables, sometimes called moderator variables
Treatment Group R O X1 Y1 O
Control Group R O X2 Y1 O
Treatment Group R O X1 Y2 O
Control Group R O X2 Y2 O

9. Developing the Research Instrument
• An instrument is a device or procedure for
systematically collecting information.
Common types of instruments include tests,
questionnaires, rating scales, checklists, and
observation forms.
• Instrumentation refers not only to the
instrument itself but also to the conditions
under which it is used, when it is to be used,
and by whom it is to be used.
Instrument Vs Instrumentation

10. Determining the Validity and Reliability of the Instrument
Validity
Validity refers to
the extent to
which an
instrument gives
us the information
we want.
Validity is a judgment of
the appropriateness of a
measure for the specific
inferences or decisions
that result from the
scores generated by the
measure.

Content-
related
evidence
• refers to the nature of the content included within the
instrument, and the specifications the researcher used
to formulate the content
Criterion-related
evidence
• refers to the relationship between scores obtained using
the instrument and scores obtained using one or more
other instruments or measures (often called criteria)
Construct-
related
evidence
• refers to the nature of psychological construct or
characteristic being measured by the instrument
Types of Evidence for Judging Validity

Reliability
Reliability refers to the
consistency of scores or
answers—how consistent they
are for each individual from
one administration of an
instrument to another, and
from one set of items to
another.

Validity and Reliability Coefficient
A validity
coefficient
• expresses the relationship which exists
between scores of the same individuals on
two different instruments
A reliability
coefficient
• expresses a relationship between scores of
the same individuals on the same instrument
at two different times, or between two forms
of the same instrument

Methods of
Estimating Reliability
Require two Administrations Require One Administration
The Test-Retest Method
The Equivalent Forms Method
Internal Consistency Methods
The Kuder-Richardson Approaches
Split-Half Testing
KR20 KR21

Threats to Internal Validity
Mortality
Subject Characteristics
Location
Instrumentation
Testing
History
Maturation
Implementation
Regression
Attitude of Subject

11. Determining the Data Collection Procedure
• According to Creswell (2009), four steps related to the
data collection process include
(a) the determination of the purpose of the research;
(b) the format e.g. quantitative survey or qualitative
interviews;
(c) the sample - i.e. who should be included in the study,
and the sample size – i.e. how many should be
included; and
(d) whether the survey should be cross-sectional – i.e. the
data collected at one point in time, or longitudinal,
wherein data is collected over time as in quantitative
studies.

12. Collecting Data
5 Steps in Collecting Data
1.
• Identify and select participants
2.
• Obtain permissions from the participants.
3.
• Decide on what types of data to collect.
4.
• Locate, modify or develop instruments to collect the data
5.
• Describe the procedures to collect the data.

12. Collecting Data
Assign numbers to different values of a variable
Measurement
The Purpose of Measurement
To provide information about the variables that
are being studied.
• In an experiment, the dependent variable is
measured.
• In correlational research each variable is
measured.
• In practice, the variable is defined by how it is
measured (operational definition), not by how it
is labeled or defined by the researcher.

12. Collecting Data
Scales of Measurement
1. Nominal – numbers assigned to categories eg.
gender, race, location
2. Ordinal – numbers rank-ordered eg. achievement
levels

3. Interval – equal intervals between numbers
4. Ratio – a true zero point, use to compare & interpret
scores
12. Collecting Data
Scales of Measurement

13. Analysing and Interpreting Data
Quantitative Data
• Assign numeric scores to each response option on instrument.
• Determine whether single-item or different scores will be used in the
analysis.
• Select a computer software programme to analyse the data
• Enter the data into a computer file by building a data grid consisting
of variables and their values.
1. Preparing Data for
Analysis
• Decide whether to use descriptive analysis or inferential analysis.
• Descriptive analysis is used for measures of central tendency, the
spread of the scores, and the relative ranking of the scores.
• Inferential analysis is used when a researcher studies a sample and
draws inferences from the sample to a population.
2. Analysing the Data
• Present results in tables, figures and a detailed discussion of the results.
3. Reporting the Results
• Summarise the detailed results in general statements.
• Provide explanations for the findings based on prior prediction made
in the literature or in theories.
• Conclude with limitations and give suggestions for future research.
4. Interpreting the
results

Descriptive Statistics
1. Frequency – how often each score is
obtained
2. Mode – score that occurs most frequently
3. Median – score in the middle of the
distribution
4. Mean – arithmetic average of all scores
5. Standard deviation –measure of variability
indicating the average distance of scores from
the mean

Descriptive Statistics

Inferential Statistics
1. Parametric – assumptions met/ not met
about data where population is normally
distributed, more ohm to detect significant
differences eg. t-test (compare 2 means),
ANOVA (compare 2 or more means), correlation
2. Nonparametric - assumptions cannot be met
about data where population is normally
distributed eg. chi square

Alpha level (p level)
• In statistical analysis we are looking to see if there is
any significance in the results. The acceptance or
rejection of a hypothesis is based upon a level of
significance – the alpha (a) level
• This is usually set at the 5% (0.05) a level, followed
in popularity by the 1% (0.01) a level
• We usually designate these as p, i.e. p =0.05 or p =
0.01

94
• There are many statistical tests that we can
use to analyse our data, and which particular
one we use to analyse our data depends upon
what we are looking for, and what data we
collected (and how we collected it).

t-test
• The t-test assesses whether the means of two
groups are statistically different from each other.
This analysis is appropriate whenever you want to
compare the means of two groups.

Pearson Correlation
• We use the Pearson's correlation in order to find a
correlation between at least two continuous
variables. The value for such a correlation lies
between 0.00 (no correlation) and 1.00 (perfect
correlation).

ANOVA (Analysis of Variance)
• ANOVA is one of a number of tests (ANCOVA -
analysis of covariance - and MANOVA - multivariate
analysis of variance) that are used to
describe/compare the relationship among a number
of groups.

Chi-square test
• There are two different types of chi-square tests -
but both involve categorical data (Pallant 2001).
• One type of chi-square test compares the frequency
count of what is expected in theory against what is
actually observed.
• The second type of chi-square test is known as a chi-
square test with two variables or the chi-square test
for independence.

Selecting your statistical test
When it comes to the selection of the appropriate test for your
research in order to determine the p-value, you need to base the
selection of four major factors, namely:
• The level of data (nominal, ordinal, ratio, or interval).
• The number of groups/samples in your research study (one, two,
or more).
• Were the data collected from independent groups/samples or from
related groups? Remember that independent groups are two or
more separated groups of participants, whilst related groups are
often the same group, but at a different time in the study, e.g. pre-
and post-testing, or even a different environment.
• The characteristics of the data (i.e. the distribution of the data).

Common Statistical Tests
The t Test
The F Test
(ANOVA)
Test for r
Chi-square Test
To compare two means
To test two or more means
To test the significance of a
correlation coefficient
To test for relationships
involving frequency data in the
form of tallies or percentages
100

Statistical Analysis
• There is a computer package for statistical analysis known as
SPSS (Statistical Package for Social Sciences)
• SPSS is one of a number of computer packages that can do
just about any calculation that you want, using any statistical
test.

Qualitative Data
• Organise the data, transcribe interviews and type field notes.
1. Preparing Data
for Analysis
• Code the data by reducing a text or image database to descriptions and
themes of people, places, or events.
• Examine the text database line by line, ask oneself what the participants is
saying and then assign a code label to the text segment.
• Themes are developed from codes to present a broader abstraction. These
themes may be layered or organised to tell a story, or may be
interconnected to portary the complexity of the phenomenon.
2. Analysing the
Data
• Present results figures, diagrams, comparison tables, and demographic tables.
• Report findings in narrative discussions comprising many forms, such as a
chronology, questions or commentary about any changes that the participants
experience.
3. Reporting the
Results
• Advance personal views, make comparisons between the findings and the
literature, and suggesting limitations and future research.
4. Interpreting
the results
• Validation procedures such as member checking, triangulation and auditing are
used.
• The intent of validation is to have participants, external reviewers or the data
sources themselves provideevidence of the accuracy of the information.
5. Validate the
Accuracy of the
Findings

14. Discussing and Reporting Research Findings
• A research report is a completed study that
reports an investigation or exploration of a
problem, identifies questions to be addressed,
includes the collection and analysis of data, and
advances an interpretation of the data.
• Researchers write their reports with their
audiences in mind.
• The audiences differ for research reports such as
dissertations and the theses, proposals, journal
articles, conference papers, policy or school
reports.
• Reports vary in purpose, length and format.

Tutorial 4
• Do a summary for the articles you have found
in the table below.
No Article Author Issue /
Problems
Suggestions Reserach
Objectives/
Questions
Sample Research Method/
Instruments
Research Findings/
Discussion of
Findings
1 Using a poetry
wiki: How can
the medium
support pre-
service
teachers of
English in their
profesional
learning about
writing poetry
and teaching
poetry writing
in a digital
age?
-Sue Dymoke,
University of
Leicester, Uk
-Janette
Hughes,
University of
Ontario Institute
of Technology,
Canada
(December,
2009)
-Teachers used
limited range of
poetry in the
classroom and
provide few
opportunities,
esp for primary
children, to read
or write poems
which directly
connect with
their own
experiences.
- Teachers
have
very
limited
experien
ce of
writing
poetry
themselv
es
An online wiki community
was developed to build
collaborative knowledge
about poetry
Laurillard et al, 2000
-affordances of a
multimodal, wiki
environment offered the
teachers for learning about
poetry writing and question
the impact that these
affordances have had both
on the teachers’
collaborations and the
poetry they wrote.
Knobel & Lankshear, 2007
-a sense of ‘relatedness’
for people participate in
text-making in the
collaborative process
1. How the
teachers shaped
themselves as
writers and
intervened in
each other’s work
in progress within
a digital thrid
space?
2. How the wiki
had supported
their professional
learning about
the teaching of
poetry writing?
3. To what extent
they had
identified its
potential as a
pedagogical tool
for their own
future use?
2 groups of 56
pre-service
English and
Language Arts
postgraduate
teachers (aged
22-42) who
were based at
Higher
Education
Institutions
(HEIs) in the UK
and Canada.
Qualitative
-Analysis of
teachers’ writing
and discussion
-Analysis of draft
poems created
through the medium
of the wiki and the
digital dialogue
which has evolved
through drafting and
communication
about this writing.
-seminar notes,
digital artefacts
created and edited
by the teachers,
comments and
written reflections,
post-course surveys
-The teachers felt they
grew in confidence
through participation in
writing poetry and to
reflect on themselves
as writers
-The experiences
could enhance their
own creativity,
criticality and emerging
classroon craft.
-Some developed their
technological skills
indigital and
multimodal
communication
through use of written,
visual and sound
modes.
-“it provided
encouragement and
facilitates a workshop
environment”

References
• Creswell, J. W. (2009). Research design.
Thousand Oaks, CA : Sage Publications, Inc.
• Creswell, J.W. (2012). Educational research:
planning, conducting and evaluating
quantitative and qualitative research. (4th Ed.).
Boston, MA: Pearson Education Inc.

TSL3133 Topic 4 Educational Research Procedure

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to TSL3133 Topic 4 Educational Research Procedure

Similar to TSL3133 Topic 4 Educational Research Procedure (20)

More from Yee Bee Choo

More from Yee Bee Choo (20)

Recently uploaded

Recently uploaded (20)

TSL3133 Topic 4 Educational Research Procedure