RESEARCH IN EDUCATION
M. ASFAH RAHMAN
FBS UNM
E-mail: asfah_rahman@yahoo.com
Research: Scientific Approach
“... the activity of solving problems which leads to new
knowledge using method of inquiry which are
currently accepted as adequate by scholars in the
field” (Helmstadter, 1970:5)
“... a systematic approach to searching for answers to
questions” (Hatch & Lazaraton,1991:9)
“... is the formal, systematic application of the scientific
method to the study of problems; educational
research is the formal, systematic application of the
scientific method to the study of educational
problems.”
TYPES OF RESEARCH
• Penelitian Eksploratif
– Memperdalam pengetahuan/pemahaman tentang
gejala/fenomena tertentu;
– Menurunkan hipotesis
• Penelitian Deskriptif
– Menggambarkan sifat, keadaan, individu, gejala atau
kelompok dalam bentuk hasil analisis statistik dasar
(deskriptif): tabel frekuensi, tabel silang, grafik, nilai rata-
rata, median. Modus dan varians.
• Penelitian Eksplanatoris (Menerangkan)
– Menguji hipotesis tentang ada-tidaknya hubungan sebab
akibat antar-variabel
TYPES OF RESEARCH
Luas Penerapan
Murni
Terapan
Layanan
Tindakan
Deskriptif Perpustakaan
Prediktif Lapangan
Diagnostik Laboratorium
Tingkatan Hasil Tingkat Kendali
Jenis-Jenis Penelitian
KUALITATIF KUANTITATIF
Introspeksi Obsevasi Praeksperimen Eksperimen
non-partisipan
Observasi Deskriptif Eksperimen
partisipan Semu
SINTETIK ANALITIK
HEURISTIK DEDUKTIF
Didorong oleh data Dirorong oleh hipotesis
Tanpa prakonsepsi Membuat prediksi
Menghasilkan hipotesis Menguji hipotesis
Produk: deskripsi/hipotesis Produk: teori
TYPES OF RESEARCH
BY PURPOSE
 Basic vs applied
research
 Evaluation research
 Research &
Development
 Action research
BY METHOD
Historical research
Descriptive research
Correlational research
Causal-comparative &
experimental research
Types of research …
Basic Research
Semata-mata untuk pengembangan dan penyempurnaan
teori;
Applied Research
Menerapkan, menguji teori, dan mengevaluasi
kemanfaatannya dalam pemecahan masalah;
Evaluation Research
Menfasilitasi pengambilan keputusan tentang manfaat
dan nilai dari dua atau lebih program tindakan
alternatif;
Types of research …
Research & Development
Mengembangkan produk, model yang efektif dan
inovatif;
Penelitian Tindakan
Memecahkan masalah yang dihadapi kini, di sini,
yang menerapkan pendekatan ilmiah;
“…proses yang diupayakan oleh praktisi untuk
mengkaji secara ilmiah masalah sendiri untuk
membimbing, memperbaiki, dan mengevaluasi
keputusan dan tindakan mereka.”
Guidelines for Classification
Cause-effect Relationship?
No
Yes
Relationship?
Prediction?
Independent Variable
Manipulated?
Yes No
Yes
No
Experimental
Causal
Comparative
Descriptive
Correlational
Groupings of Research Methods Found in Five Leading Texts
Ary et al. Best & Kahn Borg & Gall Gay Travers
RESEARCH STEPS
Selection and definition of problem
Execution of research procedures
Analysis of data
Drawing and stating conclusion
Define the Problem
-from education assessment
-from a review of whether initial
education/instructional objectives
were met
-from exploratory research to
clarify problem areas or as a
precursor to a full-scale survey
Specify Research Process
Formulate the research objectives or
hypotheses
Formulate the Research Proposal
-devise the research plan
-estimate time and costs
Data Search
Specify information requirements.
Explore available resources from individuals &
organisations.
Search for information from secondary sources
(published & on-line) and primary sources.
The Research Design
Create a research design: descriptive, diagnostic, predictive.
Choose an appropriate data collection method – survey,
observation, experimentation.
Sampling: decide on sampling technique
-probability or non-probability.
Data collection & processing
Data analysis – interpretation of findings
Research conclusion: evaluating & presenting results.
Problem Solving
Will the research
outcomes help to
solve the problem?
Cost/benefit
analysis
Justify costs of
the research
undertaking &
establish the
benefits to the
client.
Research Problem
• Is researchable
• Has theoretical and practical significance
• Appropriate:
our current level of research skills
available resources
time and other restriction
Problem Statement
• Indicates all variables of interest, and
specific relationship between those
variables
• Defines all variables (directly or
operationally)
* Background: justification for the study in
terms of significance of the problem
Hypothesis
• A tentative explanation for certain behavior, phenomenon,
or events that have occurred or will occur
• Not to “prove” but to support or not to support
Characteristics of Good Hypothesis:
• Consistent with previous research
• Tentative, reasonable explanation for the occurrence of
certain behavior, phenomenon, or events
• Clear and concise expected relationship (or difference)
between two variables
• Testable
Hypothesis
• Inductive: generalization based on observation;
• Deductive: derived from theory, supports, expands,
contradicts a given theory;
• Research hyp.: declarative
• Statistical hyp.: stated in null form;
• Nondirectional hyp.: relationship or difference exists;
• Directional hyp.: indicates the nature of the relationship or
difference;
Ss who get X do better on Z than
Ss who do not get X (or get some other X)
Research Plan: Components
Introduction
- Statement of the problem
- A review of related literature
- A Statement of the hypothesis
Method
- Subjects (population: characteristics,
size from which sample will be
selected)
- Instruments
- Materials/apparatus
- Design: (basic structure of study,
variables involved, the number of
groups, wehther randomly formed,
pretest, if any)
Procedure
- Steps to follow from beginning to end;
- Technique used in selecting sample
- Administration of pretest and posttest, if any.
Data Analysis
- Statistical technique to be used to analyzed
study data depending on:
* how groups are formed (random
assignment, matching, using existing groups);
* How many different treatment groups
involved;
* How many independent variables involved;
* The kinds of data collected (nominal,
ordinal, interval, ratio)
Time Schedule
Budget
Sampling
• Population
• The group to which
research results to be
generalizable
* Target/theoretical population
* Accessible / available
population
• Sample
a number of individuals
selected to represent the
population from which they are
selected
Representative:
homogeneous
heterogenous
Sampling
• Selecting sample
 Identifying population;
 Determination of required
sample size;
 Selection of sample;
Representative
Generalizable
• Sampling technique
Random sampling
Stratified sampling
Cluster sampling
Systematic sampling
Sample Size: Krejcie & Morgan Table
N S N S N S N S N S
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95
10
14
19
24
28
32
36
40
44
48
52
56
59
63
66
70
73
76
100
110
120
130
140
150
160
170
180
190
200
210
220
230
240
250
260
270
80
86
92
97
103
108
113
118
123
127
132
136
140
144
148
152
155
159
280
290
300
320
340
360
380
400
420
440
460
480
500
550
600
650
700
750
162
165
169
175
181
186
191
196
201
205
210
214
217
226
234
242
248
254
800
850
900
1000
1100
1200
1300
1400
1500
1600
1700
1800
1900
2000
2200
2400
2600
2800
260
265
269
274
278
285
291
197
302
306
310
313
317
320
322
327
331
335
2800
3000
3500
4000
4500
5000
6000
7000
8000
9000
10000
15000
20000
30000
40000
50000
75000
100000
338
341
346
351
354
357
361
364
367
368
370
375
377
379
380
381
382
384
Measurement Instruments
Validity
Content Validity
Construct validity
Concurrent validity
Predictive validity
Reliability
Test-retest reliability
Equivalent-forms rel
Split-half reliability
Interscorer Reliability
Intrascorer reliability
• Types of Measurements
Achievement test
Non-projective instr.
Personality inventory
Attitude scales
Tests of creativity
Interest inventory
Learning style inv.
Aptitude test
- Standardized instrument
- Self-developed instrument
- Naturally available data (e.g. GPA)
VALIDITY
Validity
The degree to which a test
measures what it is supposed to
measure;
* for a particular group
* for a particular purpose
Content Validity
• Measures intended content area
• Item validity: test items
represent measurement in the
intended content area;
• Sampling validity: how well the
test samples the total content
area;
• By expert judgement
Construct Validity
• The degree to which a test
measures an intended
hypothetical construct;
• Nonobservable trait;
• Involving testing hypothesis
deduced from theory
concerning the construct;
Concurrent Validity
The degree to which the scores on a
test are related to the scores on a
another;
Determining relationship between
scores on the test and the scores on
some established test or criterion;
VALIDITY …
• Predictive Validity
The degree to which a test can predict how well an
individual will do in the future situation;
Establishing the relationship between scores on
the test and some measure of success in the
situation of interest;
Test  predictor
Behavior predicted  criterion
RELIABILITY
The degree to which a test
consistently measures
whatever it is supposed to
measure;
Expressed numerically, a
coefficient;
High coefficient
High reliability
Test-retest Reliability
The degree to which scores are
consistent over time;
The same test administered to
the same group at different
occasions;
Correlation of two sets of
scores  coefficient of
stability;
Problem:Time interval
too short: remember responses;
too long: increase ability,
maturation;
RELIABILITY …
Equivalent-form Reliability
Two tests identical in every
way except for the actual items
included;
Establishing relationship
between scores resulting from
administering two different
forms of the same test to the
same group at the same time;
Correlation coefficient of
equivalence
Split-half Reliability
Establishing relationship
between the scores on two
equivalent halves of a test
administered to a group at one
time;
Called: internal consistency
reliability;
Apply Spearman-Brown
prophecy formula:
2r split half
rtotal test = -----------------
1 + r split half
RELIABILITY …
Rational Equivalence Reliability
Establishing internal consistency by determining how all items on a
test relate to all other items and to the total test;
Determined by applying Kuder-Richardson formula (KR-20 or KR-
21);
Require items scored dichotomously: correct or incorrect (1 or 0);
KR-21 formula:
(K)(SD2) – X (K-X)
r total test = -------------------------
(SD2) (K-1)
Where: K = the number of items in the test
SD = the standard deviation of the scores
X = the mean of the scores
Standard Error of Measurement
An estimate of how often we can expect errors of a given
size;
Small SE-m indicates high reliability;
Large SE-m indicates low reliability;
SE-m allows ua to estimate how much difference there
probably is between a person obtained score and true
score; the size of this difference being a function of the
reliability of the rest;
SE-m = SD 1 – r
Where SEm = standard error of measurement
SD = standard deviation of the test scores
r = the reliability coefficient
Descriptive Method
1. Self-report research
- Survey research
- Developmental studies
- Follow-up studies
- Sociometric studies
2. Observational research
- Nonparticipant observation (naturalistic obs., simulation obs., the case
study, content analysis);
- Participant observation
- Ethnography
Statement of the problem
Selection of subjects
Construction of the Questionnaire
Validation of the questionnaire
(interview guide);
Pretesting the questionnaire
Analysis of results
Defining observational variables
Recording of observation
Assessing observer reliability
Training Observers
Monitoring observers
Correlational Method
Collecting data to determine whether, and to what degree, a relationship
exists between two or more variables
Degree of relationship: correlation coefficient (between +1.00 and –1.00)
Corr. Coefficient: negative or positive
- Problem selection
- Sample and instrument selection
- Design and procedure
- Data analysis and interpretation
Relationship studies
Prediction studies
Causal-comparative Method
CC or ex-post facto, research: attempt to determine the cause, or reason, for
existing differences in behavior or status of groups of individuals; to
identify cause-effect relationship (tentative);
Starting with an EFFECT, then seeking POSSIBLE CAUSES;
(Variation: CAUSE first then investigating EFFECT)
Independent variable (alleged cause) is not manipulated, already occurred
- Design and procedure
- Control procedures (lack of randomization, manipulation, control)
- Matching, comparing homogeneous groups/subgroups, ancova
- Data analysis and interpretation
Descriptive and inferential statistics
t-test (two groups), analysis of variance (more than two groups),
Experimental Method
Manipulates at one independent variable, controls other variables, and
observes the effect on one or more dependent variables;
IV = experimental variable, treatment, cause (activity/characteristics
believed to make a difference)
DV = criterion variable, effect, posttest
- Selection and definition of Problem
- Selection of subjects and measuring instrument
- Selection of design
- Execution of procedures
- Data analysis and formulation of conclusions
Threats to Experimental Validity
Uncontrolled extraneous variables
Internal Validity
condition that observed differences on the DV are a direct result of
manipulation of the IV, not some other variables;
(History, maturation, testing, instrumentation, statistical regression,
differential selection of subjects, mortality)
External Validity
condition that results are generalizable, or applicable to groups and
environments outside of the experimental setting;
(pretest-posttest interaction, selection-treatment interaction, specificity of
variables, reactive arrangements, multiple treatment interference,
contamination and experimenter bias)
Control of extraneous variables
Randomization
Matching
Comparing homogeneous groups/subgroups
Using subjects as their own control
Analysis of covariance
Group Designs
Pre-experimental designs
The one-shot case study X O
The one group pretest-posttest design: O X O
The static group comparison: X O X1 O
--------- ----------
O X2 O
True experimental designs
The pretest-posttest control group design R O X1 O
R O X2 O
The posttest only control group design R X1 O
R X2 O
The Solomon four group design R O X O
R O O
R X O
R O
Quasi-experimental designs
Non-equivalent control group design O X O
--------
O O
Time series design O O O O X O O O O
Counterbalanced design X1 O X2 O X3 O
X3 O X1 O X2 O
X2 O X3 O X1 O
Group Designs
Factorial designs
Group 1 Group 2
Group 4
Group 3
Type of intstruction
Programmed Traditional
IQ
High
Low
A model for stating hypotheses for an experimental or
causal-comparative study.
• If X is the independent variable, Y is the
dependent variable, and S is the Subject, we can
state our research hypothesis as Ss who get X do
better on Y than subjects who do not get X (or
get some other X).
• Here is an example: In our study we will
hypothesize that first grade girls will show better
reading comprehension than first grade boys.
Now lets state that as a research hypothesis. It
would be a causal-comparative research study.
• Research Hypothesis: Girls will achieve higher reading
comprehension test scores than boys at the end of the
first grade.
• Operational Variables: Reading comprehension will be
measured by the Iowa Tests of Educational
Development, Reading Comprehension, administered at
the end of the year.
• Statistical Hypotheses: When we get ready to analyze
our data we might also wish to state statistical
hypotheses for our problem. The statistical hypotheses
consist of the null hypothesis (H0) and the alternative
hypothesis (H1). If we let stand for the mean of the girls
and stand for the mean of the boys, our null and
alternative hypotheses would be:
• In other words the null hypothesis states that there is no
difference between the two means on the reading
comprehension test scores, while the alternative
hypothesis states that the girls mean score on the reading
comprehension test will significantly exceed that of the
boys. Generally we use a statistical test (e.g. the t-test) to
decide if the girls score significantly higher than the
boys. We will discuss statistical hypotheses and the use
of inferential statistics in lessons 9 through 13 when we
look at the design, proceedures, and data analysis for
each of the types of quantitative research types.

RESEARCH IN EDUCATION

  • 1.
    RESEARCH IN EDUCATION M.ASFAH RAHMAN FBS UNM E-mail: asfah_rahman@yahoo.com
  • 2.
    Research: Scientific Approach “...the activity of solving problems which leads to new knowledge using method of inquiry which are currently accepted as adequate by scholars in the field” (Helmstadter, 1970:5) “... a systematic approach to searching for answers to questions” (Hatch & Lazaraton,1991:9) “... is the formal, systematic application of the scientific method to the study of problems; educational research is the formal, systematic application of the scientific method to the study of educational problems.”
  • 3.
    TYPES OF RESEARCH •Penelitian Eksploratif – Memperdalam pengetahuan/pemahaman tentang gejala/fenomena tertentu; – Menurunkan hipotesis • Penelitian Deskriptif – Menggambarkan sifat, keadaan, individu, gejala atau kelompok dalam bentuk hasil analisis statistik dasar (deskriptif): tabel frekuensi, tabel silang, grafik, nilai rata- rata, median. Modus dan varians. • Penelitian Eksplanatoris (Menerangkan) – Menguji hipotesis tentang ada-tidaknya hubungan sebab akibat antar-variabel
  • 4.
    TYPES OF RESEARCH LuasPenerapan Murni Terapan Layanan Tindakan Deskriptif Perpustakaan Prediktif Lapangan Diagnostik Laboratorium Tingkatan Hasil Tingkat Kendali
  • 5.
    Jenis-Jenis Penelitian KUALITATIF KUANTITATIF IntrospeksiObsevasi Praeksperimen Eksperimen non-partisipan Observasi Deskriptif Eksperimen partisipan Semu SINTETIK ANALITIK HEURISTIK DEDUKTIF Didorong oleh data Dirorong oleh hipotesis Tanpa prakonsepsi Membuat prediksi Menghasilkan hipotesis Menguji hipotesis Produk: deskripsi/hipotesis Produk: teori
  • 6.
    TYPES OF RESEARCH BYPURPOSE  Basic vs applied research  Evaluation research  Research & Development  Action research BY METHOD Historical research Descriptive research Correlational research Causal-comparative & experimental research
  • 7.
    Types of research… Basic Research Semata-mata untuk pengembangan dan penyempurnaan teori; Applied Research Menerapkan, menguji teori, dan mengevaluasi kemanfaatannya dalam pemecahan masalah; Evaluation Research Menfasilitasi pengambilan keputusan tentang manfaat dan nilai dari dua atau lebih program tindakan alternatif;
  • 8.
    Types of research… Research & Development Mengembangkan produk, model yang efektif dan inovatif; Penelitian Tindakan Memecahkan masalah yang dihadapi kini, di sini, yang menerapkan pendekatan ilmiah; “…proses yang diupayakan oleh praktisi untuk mengkaji secara ilmiah masalah sendiri untuk membimbing, memperbaiki, dan mengevaluasi keputusan dan tindakan mereka.”
  • 9.
    Guidelines for Classification Cause-effectRelationship? No Yes Relationship? Prediction? Independent Variable Manipulated? Yes No Yes No Experimental Causal Comparative Descriptive Correlational
  • 10.
    Groupings of ResearchMethods Found in Five Leading Texts Ary et al. Best & Kahn Borg & Gall Gay Travers
  • 11.
    RESEARCH STEPS Selection anddefinition of problem Execution of research procedures Analysis of data Drawing and stating conclusion
  • 12.
    Define the Problem -fromeducation assessment -from a review of whether initial education/instructional objectives were met -from exploratory research to clarify problem areas or as a precursor to a full-scale survey Specify Research Process Formulate the research objectives or hypotheses Formulate the Research Proposal -devise the research plan -estimate time and costs Data Search Specify information requirements. Explore available resources from individuals & organisations. Search for information from secondary sources (published & on-line) and primary sources. The Research Design Create a research design: descriptive, diagnostic, predictive. Choose an appropriate data collection method – survey, observation, experimentation. Sampling: decide on sampling technique -probability or non-probability. Data collection & processing Data analysis – interpretation of findings Research conclusion: evaluating & presenting results. Problem Solving Will the research outcomes help to solve the problem? Cost/benefit analysis Justify costs of the research undertaking & establish the benefits to the client.
  • 13.
    Research Problem • Isresearchable • Has theoretical and practical significance • Appropriate: our current level of research skills available resources time and other restriction
  • 14.
    Problem Statement • Indicatesall variables of interest, and specific relationship between those variables • Defines all variables (directly or operationally) * Background: justification for the study in terms of significance of the problem
  • 15.
    Hypothesis • A tentativeexplanation for certain behavior, phenomenon, or events that have occurred or will occur • Not to “prove” but to support or not to support Characteristics of Good Hypothesis: • Consistent with previous research • Tentative, reasonable explanation for the occurrence of certain behavior, phenomenon, or events • Clear and concise expected relationship (or difference) between two variables • Testable
  • 16.
    Hypothesis • Inductive: generalizationbased on observation; • Deductive: derived from theory, supports, expands, contradicts a given theory; • Research hyp.: declarative • Statistical hyp.: stated in null form; • Nondirectional hyp.: relationship or difference exists; • Directional hyp.: indicates the nature of the relationship or difference; Ss who get X do better on Z than Ss who do not get X (or get some other X)
  • 17.
    Research Plan: Components Introduction -Statement of the problem - A review of related literature - A Statement of the hypothesis Method - Subjects (population: characteristics, size from which sample will be selected) - Instruments - Materials/apparatus - Design: (basic structure of study, variables involved, the number of groups, wehther randomly formed, pretest, if any) Procedure - Steps to follow from beginning to end; - Technique used in selecting sample - Administration of pretest and posttest, if any. Data Analysis - Statistical technique to be used to analyzed study data depending on: * how groups are formed (random assignment, matching, using existing groups); * How many different treatment groups involved; * How many independent variables involved; * The kinds of data collected (nominal, ordinal, interval, ratio) Time Schedule Budget
  • 18.
    Sampling • Population • Thegroup to which research results to be generalizable * Target/theoretical population * Accessible / available population • Sample a number of individuals selected to represent the population from which they are selected Representative: homogeneous heterogenous
  • 19.
    Sampling • Selecting sample Identifying population;  Determination of required sample size;  Selection of sample; Representative Generalizable • Sampling technique Random sampling Stratified sampling Cluster sampling Systematic sampling
  • 20.
    Sample Size: Krejcie& Morgan Table N S N S N S N S N S 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 10 14 19 24 28 32 36 40 44 48 52 56 59 63 66 70 73 76 100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 250 260 270 80 86 92 97 103 108 113 118 123 127 132 136 140 144 148 152 155 159 280 290 300 320 340 360 380 400 420 440 460 480 500 550 600 650 700 750 162 165 169 175 181 186 191 196 201 205 210 214 217 226 234 242 248 254 800 850 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 2200 2400 2600 2800 260 265 269 274 278 285 291 197 302 306 310 313 317 320 322 327 331 335 2800 3000 3500 4000 4500 5000 6000 7000 8000 9000 10000 15000 20000 30000 40000 50000 75000 100000 338 341 346 351 354 357 361 364 367 368 370 375 377 379 380 381 382 384
  • 21.
    Measurement Instruments Validity Content Validity Constructvalidity Concurrent validity Predictive validity Reliability Test-retest reliability Equivalent-forms rel Split-half reliability Interscorer Reliability Intrascorer reliability • Types of Measurements Achievement test Non-projective instr. Personality inventory Attitude scales Tests of creativity Interest inventory Learning style inv. Aptitude test - Standardized instrument - Self-developed instrument - Naturally available data (e.g. GPA)
  • 22.
    VALIDITY Validity The degree towhich a test measures what it is supposed to measure; * for a particular group * for a particular purpose Content Validity • Measures intended content area • Item validity: test items represent measurement in the intended content area; • Sampling validity: how well the test samples the total content area; • By expert judgement Construct Validity • The degree to which a test measures an intended hypothetical construct; • Nonobservable trait; • Involving testing hypothesis deduced from theory concerning the construct; Concurrent Validity The degree to which the scores on a test are related to the scores on a another; Determining relationship between scores on the test and the scores on some established test or criterion;
  • 23.
    VALIDITY … • PredictiveValidity The degree to which a test can predict how well an individual will do in the future situation; Establishing the relationship between scores on the test and some measure of success in the situation of interest; Test  predictor Behavior predicted  criterion
  • 24.
    RELIABILITY The degree towhich a test consistently measures whatever it is supposed to measure; Expressed numerically, a coefficient; High coefficient High reliability Test-retest Reliability The degree to which scores are consistent over time; The same test administered to the same group at different occasions; Correlation of two sets of scores  coefficient of stability; Problem:Time interval too short: remember responses; too long: increase ability, maturation;
  • 25.
    RELIABILITY … Equivalent-form Reliability Twotests identical in every way except for the actual items included; Establishing relationship between scores resulting from administering two different forms of the same test to the same group at the same time; Correlation coefficient of equivalence Split-half Reliability Establishing relationship between the scores on two equivalent halves of a test administered to a group at one time; Called: internal consistency reliability; Apply Spearman-Brown prophecy formula: 2r split half rtotal test = ----------------- 1 + r split half
  • 26.
    RELIABILITY … Rational EquivalenceReliability Establishing internal consistency by determining how all items on a test relate to all other items and to the total test; Determined by applying Kuder-Richardson formula (KR-20 or KR- 21); Require items scored dichotomously: correct or incorrect (1 or 0); KR-21 formula: (K)(SD2) – X (K-X) r total test = ------------------------- (SD2) (K-1) Where: K = the number of items in the test SD = the standard deviation of the scores X = the mean of the scores
  • 27.
    Standard Error ofMeasurement An estimate of how often we can expect errors of a given size; Small SE-m indicates high reliability; Large SE-m indicates low reliability; SE-m allows ua to estimate how much difference there probably is between a person obtained score and true score; the size of this difference being a function of the reliability of the rest; SE-m = SD 1 – r Where SEm = standard error of measurement SD = standard deviation of the test scores r = the reliability coefficient
  • 28.
    Descriptive Method 1. Self-reportresearch - Survey research - Developmental studies - Follow-up studies - Sociometric studies 2. Observational research - Nonparticipant observation (naturalistic obs., simulation obs., the case study, content analysis); - Participant observation - Ethnography Statement of the problem Selection of subjects Construction of the Questionnaire Validation of the questionnaire (interview guide); Pretesting the questionnaire Analysis of results Defining observational variables Recording of observation Assessing observer reliability Training Observers Monitoring observers
  • 29.
    Correlational Method Collecting datato determine whether, and to what degree, a relationship exists between two or more variables Degree of relationship: correlation coefficient (between +1.00 and –1.00) Corr. Coefficient: negative or positive - Problem selection - Sample and instrument selection - Design and procedure - Data analysis and interpretation Relationship studies Prediction studies
  • 30.
    Causal-comparative Method CC orex-post facto, research: attempt to determine the cause, or reason, for existing differences in behavior or status of groups of individuals; to identify cause-effect relationship (tentative); Starting with an EFFECT, then seeking POSSIBLE CAUSES; (Variation: CAUSE first then investigating EFFECT) Independent variable (alleged cause) is not manipulated, already occurred - Design and procedure - Control procedures (lack of randomization, manipulation, control) - Matching, comparing homogeneous groups/subgroups, ancova - Data analysis and interpretation Descriptive and inferential statistics t-test (two groups), analysis of variance (more than two groups),
  • 31.
    Experimental Method Manipulates atone independent variable, controls other variables, and observes the effect on one or more dependent variables; IV = experimental variable, treatment, cause (activity/characteristics believed to make a difference) DV = criterion variable, effect, posttest - Selection and definition of Problem - Selection of subjects and measuring instrument - Selection of design - Execution of procedures - Data analysis and formulation of conclusions
  • 32.
    Threats to ExperimentalValidity Uncontrolled extraneous variables Internal Validity condition that observed differences on the DV are a direct result of manipulation of the IV, not some other variables; (History, maturation, testing, instrumentation, statistical regression, differential selection of subjects, mortality) External Validity condition that results are generalizable, or applicable to groups and environments outside of the experimental setting; (pretest-posttest interaction, selection-treatment interaction, specificity of variables, reactive arrangements, multiple treatment interference, contamination and experimenter bias) Control of extraneous variables Randomization Matching Comparing homogeneous groups/subgroups Using subjects as their own control Analysis of covariance
  • 33.
    Group Designs Pre-experimental designs Theone-shot case study X O The one group pretest-posttest design: O X O The static group comparison: X O X1 O --------- ---------- O X2 O True experimental designs The pretest-posttest control group design R O X1 O R O X2 O The posttest only control group design R X1 O R X2 O The Solomon four group design R O X O R O O R X O R O Quasi-experimental designs Non-equivalent control group design O X O -------- O O Time series design O O O O X O O O O Counterbalanced design X1 O X2 O X3 O X3 O X1 O X2 O X2 O X3 O X1 O
  • 34.
    Group Designs Factorial designs Group1 Group 2 Group 4 Group 3 Type of intstruction Programmed Traditional IQ High Low
  • 35.
    A model forstating hypotheses for an experimental or causal-comparative study. • If X is the independent variable, Y is the dependent variable, and S is the Subject, we can state our research hypothesis as Ss who get X do better on Y than subjects who do not get X (or get some other X). • Here is an example: In our study we will hypothesize that first grade girls will show better reading comprehension than first grade boys. Now lets state that as a research hypothesis. It would be a causal-comparative research study.
  • 36.
    • Research Hypothesis:Girls will achieve higher reading comprehension test scores than boys at the end of the first grade. • Operational Variables: Reading comprehension will be measured by the Iowa Tests of Educational Development, Reading Comprehension, administered at the end of the year. • Statistical Hypotheses: When we get ready to analyze our data we might also wish to state statistical hypotheses for our problem. The statistical hypotheses consist of the null hypothesis (H0) and the alternative hypothesis (H1). If we let stand for the mean of the girls and stand for the mean of the boys, our null and alternative hypotheses would be:
  • 37.
    • In otherwords the null hypothesis states that there is no difference between the two means on the reading comprehension test scores, while the alternative hypothesis states that the girls mean score on the reading comprehension test will significantly exceed that of the boys. Generally we use a statistical test (e.g. the t-test) to decide if the girls score significantly higher than the boys. We will discuss statistical hypotheses and the use of inferential statistics in lessons 9 through 13 when we look at the design, proceedures, and data analysis for each of the types of quantitative research types.