Quantitative analysis in language research

Carlo Magno, PhD
crlmgn@yahoo.com

 State research questions and hypothesis
anchored on a language theory
 Decide on the statistical analysis to be used
given research cases.
 Create an outline of a research that will be
conducted using quantitativeAnalysis

BASIC
 Read the prevailing
literature
 Test the theory
 Restate the theory
APPLIED
 Observe the immediate
need
 Address the need
 Solution to the problem

Research
Question
Read Reviews
Find
theory/model
Gaps? What’s new

Undergraduate
Statistics
Masters Statistics Doctorate
Statistics
•Computation and
Interpretation of
data
•Descriptive and
Inferential
•Review on
Computation and
Interpretation
•Descriptive and
Inferential statistics
•Statistical Literacy
– understanding
statistics as used in
journal articles
•Using statistics to
test theories
generated.
•Multivariate data
analysis

 Considerations in the selection of statistics to
use.
 List of statistics
 Examples in using the statistics

 Increasing Reading Comprehension and
EngagementThrough Concept-Oriented
Reading Instruction. By: Guthrie, JohnT.,
Wigfield,Allan, Barbosa, Pedro, Perencevich,
KathleenC.,Taboada, Ana, Davis, Marcia H.,
Scafiddi, NicoleT.,Tonks, Stephen, Journal of
Educational Psychology, 00220663, 2004,Vol.
96, Issue 3.

 Based on an engagement perspective of reading development, we
investigated the extent to which an instructional framework of
combining motivation support and strategy instruction (Concept-
Oriented Reading Instruction—CORI) influenced reading outcomes
for third-grade children. In CORI, five motivational practices were
integrated with six cognitive strategies for reading
comprehension. In the first study, we compared this framework to
an instructional framework emphasizing Strategy Instruction (SI),
but not including motivation support. In the second study, we
compared CORI to SI and to a traditional instruction group (TI), and
used additional measures of major constructs. In both studies,
class-level analyses showed that students in CORI classrooms were
higher than SI and/orTI students on measures of reading
comprehension, reading motivation, and reading strategies.

 What was the aim of the study?
 What is the independent variable in the first study?
 What is the dependent variable it the first study?
 How many groups were used in the first study?
 How many levels of IV was used in the first study?
 How was the DV measured?
 How was the data analyzed?What statistics was
used?
 Why do you think this is the appropriate analysis?
 What is the difference between study 1 and 2?Would
the analysis change?

 When we analyzed the use of the statistics in
the study by Guthrie et al., what information
did we determine first?

Variables Involved
•Independent
•Dependent
How many groups?
•Design
•Comparison
•Correlating
•Effect
Levels of data of the
variables (IV and DV)

 Case 1: A study compared males and females. More
specifically, the study wanted to determine who is
higher in verbal ability between the two groups. A
test on verbal ability is given for the two groups and
the mean scores were compared.
 Case 2: The effect of Project-Based Learning (PBL) on
the grades of students was studied among college
students. It was hypothesized that students will
achieve more in the PBL as compared to a group who
received pure lecture.The grades of the students were
compared at the end of the term.

 Case 3:Writing anxiety, writing metacognition, and
topic knowledge was used to predict students writing
proficiency. Students essays were scored which
served as indicator for their writing proficiency.
Scales were used to determine writing anxiety, writing
metacognition, and topic knowledge.
 Case 4: Neophyte and experienced principals,
coordinators, and directors were compared on their
degree of transformational leadership. A scale
measuring transformational leadership was
administered to the administrators across 200 school
in NCR.

 Case 4: Filipino and Korean high school students were
compared on their oral proficiency (TOEFL), vocabulary,
and reading comprehension in English (English test).
 Case 5: The effect of case study method on students
critical thinking was studied. The Watson Glaser Critical
Thinking Appraisal (WGCTA) was administered as a
pretest then the case study method was implemented for
the rest of the term.Towards the end of the term, the
WGCTA was administered again.
 Case 6: The frequencies of SV agreement errors were
counted among high school students in the public and
private.The comparison was also done among high and
low ability students in these two schools.

A B C D
Type of school
Ethnicity
Gender
Socio-economic
status
Favorite movie
from like to least
like
Ranking of best
science fiction
stories
Perceived highest
to lowest reputable
universities in
terms of research
EnglishAbility
Math ability
Achievement in
Science
Motivation
Stress
Self-esteem
Self-efficacy
temperature
Height of children
Weight of first
graders
Length of travel
Width of the table
Brightness of light

Nominal
Ordinal
Interval
Ratio

 Three important properties:
 Magnitude--property of “moreness”. Higher score
refers to more of something.
 Equal intervals--is the difference between any two
adjacent numbers referring to the same amount of
difference on the attribute?
 Absolute zero--does the scale have a zero point that
refers to having none of that attribute?

Levels of Data
Nominal Scales - there must be distinct classes but these classes
have no quantitative properties. Therefore, no comparison can be made
in terms of one category being higher than the other.
For example - there are two classes for the variable gender -- males and
females. There are no quantitative properties for this variable or these
classes and, therefore, gender is a nominal variable.
Other Examples:
country of origin
biological sex (male or female)
animal or non-animal
married vs. single

 Sometimes numbers are used to designate category
membership
 Example:
Country of Origin
1 = United States 3 = Canada
2 = Mexico 4 = Other
 However, in this case, it is important to keep in mind
that the numbers do not have intrinsic meaning

Levels of Data
Ordinal Data - there are distinct classes but these
classes have a natural ordering or ranking. The
differences can be ordered on the basis of magnitude.
For example - final position of horses in a
thoroughbred race is an ordinal variable. The horses
finish first, second, third, fourth, and so on. The
difference between first and second is not necessarily
equivalent to the difference between second and third,
or between third and fourth.
20

 Does not assume that the intervals between numbers are equal
Example:
finishing place in a race (first place, second place)
1 hour 2 hours 3 hours 4 hours 5 hours 6 hours 7 hours 8 hours
1st place 2nd place 3rd place 4th place
21

Levels of Data
Interval Scales - it is possible to compare differences in magnitude,
but importantly the zero point does not have a natural meaning. It
captures the properties of nominal and ordinal scales -- used by most
psychological tests.
Designates an equal-interval ordering - The distance between, for
example, a 1 and a 2 is the same as the distance between a 4 and a 5
Example - Celsius temperature is an interval variable. It is meaningful to
say that 25 degrees Celsius is 3 degrees hotter than 22 degrees Celsius,
and that 17 degrees Celsius is the same amount hotter (3 degrees) than 14
degrees Celsius. Notice, however, that 0 degrees Celsius does not have a
natural meaning. That is, 0 degrees Celsius does not mean the absence
of heat!
22

Levels of Data
Ratio Scales - captures the properties of the other types of
scales, but also contains a true zero, which represents the
absence of the quality being measured.
For example - heart beats per minute has a very natural zero
point. Zero means no heart beats. Weight (in grams) is also a
ratio variable. Again, the zero value is meaningful, zero grams
means the absence of weight.
Example:
the number of intimate relationships a person has had
0 quite literally means none
a person who has had 4 relationships has had twice as
many as someone who has had 2 23

Levels of Data
• Each of these scales have different properties (i.e.,
difference, magnitude, equal intervals, or a true zero point)
and allows for different interpretations.
• The scales are listed in hierarchical order. Nominal scales
have the fewest measurement properties and ratio having the
most properties including the properties of all the scales
beneath it on the hierarchy.
• The goal is to be able to identify the type of measurement
scale, and to understand proper use and interpretation of the
scale.

 Nominal scales--qualitative, not quantitative
distinction (no absolute zero, not equal intervals,
not magnitude)
 Ordinal scales--ranking individuals (magnitude, but
not equal intervals or absolute zero)
 Interval scales--scales that have magnitude and
equal intervals but not absolute zero
 Ratio scales--have magnitude, equal intervals, and
absolute zero (so can compute ratios)

Test Your Knowledge:
A professor is interested in the relationship between the number
of times students are absent from class and the letter grade that
students receive on the final exam. He records the number of
absences for each student, as well as the letter grade
(A,B,C,D,F) each student earns on the final exam. In this
example, what is the measurement scale for number of
absences?
a) Nominal b) Ordinal c) Interval d) Ratio

In the previous example, what is the measurement scale of
letter grade on the final exam?

A researcher is interested in studying the effect of room
temperature in degrees Fahrenheit on productivity of automobile
assembly workers. She controls the temperature of the three
manufacturing facilities, such that employees in one facility work
in a room temperature of 60 degrees, employees in another
facility work in a room temperature of 65 degrees, and the last
group works in a room temperature of 70 degrees. The
productivity of each group is indicated by the number of
automobiles produced each day. In this example, what is the
measurement scale of room temperature?
a) Nominal b) Ordinal c) Interval d)Ratio

In the previous example, what is the level of data of productivity?

Select the highest appropriate level of data:
Bicycle models:
1= Road
2 =Touring
3 = Mountain
4 = Hybrid
5 = Comfort
6 = Cruiser

Educational Level:
1 = Some High school
2 =High school Diploma
3 = Undergraduate Degree
4 = Masters Degree
5 = Doctorate Degree

Number of questions asked during a class lecture

Select the highest level of data:
Categories on a Likert-type scale measuring attitudes:
1 = Strongly Disagree
2 = Disagree
3 = Neutral
4 = Agree
5 = Strongly Agree

 Case 1: A study compared males and females on their
verbal ability. More specifically, the study wanted to
determine who is higher in verbal ability between the
two groups. A test on verbal ability is given for the
two groups and the mean scores were compared.
 Case 2: The effect of Project-Based Learning (PBL) on
the grades of students was studied among college
students. It was hypothesized that students will
achieve more in the PBL as compared to a group who
received pure lecture.The grades of the students were
compared at the end of the term.

 Case 4: Filipino and Korean high school students were
compared on their oral proficiency (TOEFL), vocabulary,
and reading comprehension in English (English test).
 Case 5: The effect of case study method on students
critical thinking was studied. The Watson Glaser Critical
Thinking Appraisal (WGCTA) was administered as a
pretest then the case study method was implemented for
the rest of the term.Towards the end of the term, the
WGCTA was administered again.
 Case 6: The frequencies of SV agreement errors in an
essay were counted among high school students in the
public and private.The comparison was also done among
high and low ability students in these two schools.

Parametric Non-Parametric
•Enables researchers to
make assumptions about
the population
•Large sample size is
requires (N>30)
•Used for interval and ratio
scales
•Difficult to make
assumptions about the
population
•Large sample size is not a
requirement
•Used for nominal and
ordinal scales

Design Parametric Non-Parametric
One sample
-the mean of one sample is compared with
a standard
No. of comparisons:
nominal
DV: interval/ratio
One sample, categories
are nominal/ordinal
One sample repeated measures (dependent
groups)
-One sample is studies but more measured
twice (2 set of data)
- e. g. pre and post test design
No. of comparisons:
nominal
DV: interval/ratio
No. of comparisons:
nominal
DV: nominal/ordinal
Two independent groups
-studying two distinct samples/groups
Groups/IV: nominal
DV: interval/ratio
Groups/IV: nominal
DV: nominal/ordinal
Comparing multiple groups (independent
or dependent groups)
Groups/IV: nominal
DV: interval/ratio
Groups/IV: nominal
DV: nominal
Relating one variable to another

Design Parametric Non-Parametric
One sample
-the mean of one sample is compared
with a standard
z-test
t-test
One-way chi-square
Kolmogorov smirnov
One sample repeated measures
(dependent groups)
-One sample is studies but more
measured twice (2 set of data)
- e. g. pre and post test design
t-test for 2 dependent
samples
McNemar change test
Wilcoxon signed ranks test
Two independent groups
-studying two distinct samples/groups
t-test for 2 independent
samples
Two-way chi-square
Mann Whitney U test
Comparing multiple groups
(independent or dependent groups)
Analysis ofVariance
(ANOVA)
1 IV, 1 DV: one way
ANOVA
2 IV, 1 DV: two way
ANOVA
1 more IV, 2 or more DV:
MANOVA
Kruskal wallis test
Relating one variable to another Pearson r Spearman rho

 Case A:
 The number of students were counted
categorized for those who prefer to take the
science and humanities track. Males and
females were counted for each track as well.
The researcher wanted to compare the
number of students categorized by gender
and tracks.

 Case B
 The attitude towards learning a foreign
language were determined using a 10
item questionnaire using a Lickert scale.
The Filipinos, Chinese, and Japanese
stduents were compared on their
attitude towards learning a foreign
langauge.

 Case C
 Students were grouped for those whose
parents are native speakers (L1) of English
and those whose English is L2.These two
groups were requested to answer the an
English Language Exposure scale with 10
items (4 point scale). Students with parents
who speaks English in L1 and L2 were
compared on their scores on the English
Language Exposure.

 Case D
 Students were asked to rank how well they
speak their local dialect. Students who
studied grade school and high school in their
province (where the dialect is spoken) and
those that did not were compared on their
rankings.

 Case E
 Students were asked to watch newscasterA
then followed by newscaster B. The students
were asked to rate the English proficiency of
both newscasterA and B on a scale of 1 (not
proficient) to 7 (very proficient). The ratings
of newscasterA and B were compared.

 Case F
 Students who passed and failed in a grammar test
were counted.Then students were given a special
grammar class.The students were given a similar test
again and those who passed and failed were counted.
Those who initially passed then failed after the
hypnosis were compared to those who initially failed
and then passed after the special grammar class.
Before the special grammar class
Pass Fail
After Pass 29
hypnosis Fail 15

 Case G
 There were 30 students who took a
reading comprehension test (mean and
SD were obtained).Their performance
were compared with the mean score
obtained from the test manual.

 Case H
 Males and females are classified into those
with high and low verbal ability.These groups
were compared on their self-efficacy (6
items, 4 point scale) and self-regulation (52
items, 4 point scale). Males with high and low
ability and females with high and low ability
were compared on the two scales.

 Case I
 Students answered a scale measuring their
language learning strategies composed of
cognitive, affective, social, and metacogntive
strategies. At the end of the term, their
grades in English were obtained. It is
hypothesized that cognitive, affective, social,
and metacogntive strategies will predict
students English grades.

 Case J
 Students level of proficiency in writing an
essay (rate using a rubric) and students
knowledge of content (using a test) were
determined. The researcher hypothesized
that when students knowledge of content
increases, their proficiency in writing an essay
also increases.

 It was hypothesized in a study that students
ability in school is related to perfectionism.
College students were tested using the OTIS
Lenon School AblityTest (OLSAT) and the
perfectionism scale by Frost was administered
to the same group.
 How many variables are studied?
 What are the levels of measurement of the
variables?
 What is the purpose of the study?
 What statistics will be used?

OLSAT (X) Perfectionism (Y)
100 99
95 98
90 94
85 87
82 84
80 81
75 78
70 73
65 68
50 60

Scatterplot: X vs. Y
Y = 14.379 + .85633 * X
Correlation: r = .98966
40 50 60 70 80 90 100 110
X
55
60
65
70
75
80
85
90
95
100
105
Y
95% confidence

 There is a straight line relationship between
variables X andY
 When X increases,Y also increases-positive
relationship
 When X increases,Y decreases or vice versa –
negative relationship

 Pearson Product-Moment correlation – (r)
used for interval/ratio sets of variables
 Spearman Rank-order correlation – two sets
of data are ordinal
 Phi coefficient – each of the variables is a
dichotomy

Writing proficiency Errors in Grammar
100 35
95 40
90 45
85 50
75 55
70 60
65 64
60 70
55 76
50 80

Scatterplot: Y vs. X
X = 139.94 - 1.138 * Y
Correlation: r = -.9959
30 40 50 60 70 80 90
Y
40
50
60
70
80
90
100
110
X
95% confidence

 Positive relationship – as one variable
increases the other variable also increases
 Ex. academic grades and intelligence
 Negative relationship – as one variable
increases, the other decreases or vice
versa
 Ex. procrastination and motivation
 Absence of relationship between variables
– denoted by .00
Show computation in statistica

 A correlation coefficient is computed for a
bivariate distribution using a statistical
formula
Correlation CoefficientValue Interpretation
0.80 – 1.00 Very strong relationship
0.6 – 0.79 Strong relationship
0.40 – 0.59 Substantial/marked relationship
0.2 – 0.39 Low relationship
0.00 – 0.19 Negligible relationship

 How much ofY’s is explained/accounted for
by X
 Proportion explained
 Square of the correlation coefficient value

 Students ranked their degree of importance on
learning a foreign language and working overseas.
Learning a foreign language Working overseas
14 13
11 12
10 9
10 8
14 10
13 14

Struggling Independent
Have books at
home
30 20
No books at home 10 40
Reading Level
Availability
of books

 7 Filipino college students have taken theTest for
English as a Second Language (TESL).The researcher
wanted to determine if their scores are far from the
standard norm among speakers of ESL. The standard
norm in the manual is 40.5 with a standard error of
4.54.
42
45
46
45
43
46
47

Errors found F Expected
frequency
Poor sentence
construction
26 21.11
Wrong choice of
word
32 21.11
Faulty parallelism 12 21.11
Wrong case 14 21.11
Wrong punctuation 46 21.11
Fragment 8 21.11
Wrong article 16 21.11
Run-on sentence 27 21.11
Wrong verb 9 21.11
Total=190

fo fe
Asst. Instructor
25 15.6
Instructor
10 15.6
Ass. Prof
31 15.6
Prof
7 15.6
Full Prof 5 15.6
ft/∑ fo = 78

 A study investigated whether the effect of
project-based learning in an English class would
develop students deep approach to learning
English.The students were first given a pre test
using the learning process questionnaire (LPQ)
that measures deep approach to learning. The
students are exposed to situations in English
they were asked to respond thorugh speaking
and writing. After the instruction, the LPQ was
again administered to the same 10 students.

LPQ pre test LPQ post test
24 2
28 30
32 37
18 22
24 29
36 40
40 38
37 41
24 29
20 28

 One group of students were asked to rank the
English proficiency of a person speaking with
an English accent. In another occasion, the
same students watched another speaker with
a Filipino accent. Is there a difference in the 2
sets of rankings?

Student No. English
Accent
Filipino
Accent
1 12 12
2 14 16
3 15 14
4 12 11
5 16 14
6 15 18
7 13 16
8 10 11

 An experiment was conducted to determine whether
word work strategy can be a intervention to help
studnets become readers.A reading test was given
and students who are readers and non readers were
identified.The students have undergone word work
strategy and after session they were again given an
identical reading test.The students who are readers
and non readers were again identified.
Before word work strategy
Readers Non readers
After word
work strategy
Non readers 7 10
Readers 15 20

 The effect of exposure to a scientific report on
students technical writing skill was investigated
among 30 senior high school STEM students.
The 15 participants in the experiment group
were given a model of a good scientific paper
before they wrote their own investigative
project.The other 15 participants in the control
group were just given guidelines how to write
without an example. After the procedure, both
groups submitted their report for the
investigative report.The report was rated using
a rubric.

Experimental group Control group
9 4
14 9
11 3
9 6
12 4
13 2
14 5
9 5
11 4
13 5
11 6
13 6
12 4
14 7
15 8
∑x1 = 180 ∑x2= 78
1 = 12 2 = 5.2

 In the study, 8 students who attended an English remedial class
and 7 who immediately attended a regular class in English were
asked to rank their confidence in speaking English using a ranking
scale.Test whether they differ in their rankings.
Attended a
remedial class
Went to a
regular English
class
40 10
37 75
35 40
37 32
51 25
38 62
42 5
49

 Three corpora was studied: Philippine English,
Singaporean English, and Malaysian English.
Three modals were counted in each of the
corpora.
Corpus Modals
could would should Total
Philippine
English
3 7 1 11
Singapore
English
2 3 6 11
Malaysian
English
1 2 5 8
Total 6 12 12 30

 In an experiment, the effect of three reading
comprehension techniques were investigated on the
reading comprehension of literature students. The
techniques has three levels: questioning, context
clue, and guided practice. These techniques were
used as a teaching strategy in a lesson in a literatire
class for three sections. Each of the strategy was
used for each class. One section did not receive any
strategy which served as the control group. After
undergoing the strategy, the students were tested
where they answered a series of items about a text
read.

Control Questioning Guided
practice
Context clue
8 14 19 15
9 13 18 15
6 12 19 14
7 15 19 15
2 15 17 13
4 14 18 14
4 13 18 13

English
Proficiency
Achievement
Effect of Achievement andType of school on self-efficacy
Low Achievers HighAchievers
Type of school
Public
school
Private
School

low achiever high achiever
Public
10 15
9 16
5 17
6 15
5 16
Private
15 19
14 20
14 19
13 18
15 18

 Public and Private schools were compared on
their self-monitoring and goal-setting
Self-monitoring goal-setting
Public 10 9
10 8
9 7
7 7
8 5
7 5
6 4
Private 18 17
19 19
18 19
17 18
17 18
18 17
18 18

 Goal-setting, self-evaluation, seeking
assistance, and environmental structuring
were used to predict English grades.

Regression Summary for DependentVariable: GRades (data for multiple regression)
R= .68021315 R²= .46268992 Adjusted R²= .46163998 F(4,2047)=440.68 p
b*
Std.Err. -
of b*
b
Std.Err. -
of b
t(2047) p-value
Intercept 0.45 0.06 6.89 0.00
goal_set 0.19** 0.02 0.17 0.01 8.80 0.00
self_ev 0.21** 0.02 0.23 0.02 8.88 0.00
seek_ast 0.15** 0.02 0.17 0.02 6.74 0.00
env_stru
c
0.28** 0.02 0.26 0.02 13.32 0.00

 Work with a team
 Make an outline of a study that will make use of
quantitative analysis
 State the purpose of the study (research question)
 Possible hypothesis (if there is)
 Framework that supports the study
 Research Design
 Participants
 Instruments
 Procedure
 Data Analysis

Quantitative analysis in language research

More Related Content

What's hot

Viewers also liked

Similar to Quantitative analysis in language research

More from Carlo Magno

Recently uploaded

Quantitative analysis in language research