its about methodology

1. ADVANCED RESEARCH METHODOLOGY
Prof. Fuad Abdul Hamied, Ph.D.
Prof. Dr. Sri Setyarini, M.A.

4. • The most common instrument for collecting quantitative data is the
test. Tests have several types: language test or psychological test
(aptitude or personal batteries)
• A frequent method of collecting quantitative data is through
conducting a survey using some sort of questionnaire.

5.  How many people do I need to
include in my study?
 How large should my sample be?
 What sort of people shall I select?
Sampling decisions
affect the necessary
arrangements,
timing and
scheduling of the
project as well as
the costs.

6. Sample: The group of participants whom the researcher actually examines in
investigation.
Population: The group of people whom the study is about.
- The sample is a subset of the population that is representative of the whole
population.
- The strength of the conclusions we can draw from the results obtained from
a selected small group depends on how accurately the particular sample
represents the larger population.

7. 1. Probability Sampling
- Random Sampling: selecting members of population on a
completely random basis
- Stratified random Sampling: combination of randomization and
categorization
- Systematic Sampling: selecting every nth member of the target
group
- Cluster Sampling: random selection of larger groupings or units of
population (especially when the population is widely dispread)

8. 2. None-probability Sampling
o Quota Sampling and Dimensional Sampling:
Quota Sampling is similar to proportional stratified random sampling
without the ‘random’ element. In Dimensional Sampling at least one
representative of every combination of the various parameters is
included in sample.
o Snowball Sampling
a few people having criteria are asked to identify further members.
o Convenience or Opportunity Sampling
members are selected if they meet certain pratical criteria.

9.  Rule of thumb: a range of between 1% to 10% of the population, with
a minimum of 100 participants.
 Statistical consideration: sample should have a normal distribution.
 Sample composition: identify any distinct sub-groups that may
behave differently in advance.
 Safety margin: leave a decent margin for unforeseen or unplanned
circumstances.
 Reverse approach: first approximate the expected magnitude of
results then determine the sample size.

10.  The actual composition of sample is not only the function of some
systematic selection but also of factors related to respondents' own
willingness to participate.
 The problem can arise, for example when:
- Researcher invite volunteers to take part in study
- The design allows for high degree of dropout
- Participants are free to choose to be in study or not

11. Main methodological
issues:
a) How to sample the
participants
b) How to design and
administer the
research tool
Positive point of questionnaires:
 Easy to construct
 Extremely Versatile
 Capable of gathering a large
amount of information
quickly in a form that is
readily processible

12. It is used in at least 2 broad senses:
1. Interview schedules/guides
2. Self–administered pencil-and-paper questionnaire
“any written instruments that present respondents with a series of questions
to which they are to react either by writing out or selecting their answers”
Questionnaires can yield 3 types of data:
 Factual questions: finding certain facts about the respondents such as
demographic characteristic, occupation, residential location, marital and socio-
economical status, educational level, etc.
 Behavioral question: finding out what the respondents are doing, have done in
the past, habits, lifestyles, focusing on action and personal history.
 Attitudinal question: finding out what people think, covering attitudes, opinions,
beliefs, interest and values.

13. o A test takes a sample of the respondent`s behavior/knowledge for the purpose
of evaluating the individual`s more general underlying
competence/abilities/skills. It measures how well some one can do something.
o Questionnaire items do not have good/bad answers. They elicit information in a
non-evaluative manner, without gauging their performance against a set of
criteria.

14.  Multi-item scales: a cluster of several differently
worded items that focus on the same target.
 Item wording in general has a substantial impact on the
responses.

15.  There`s no production of free writing; respondents
choose one of the given alternatives.
 Most professional questionnaires are made up of
‘closed-ended’ items.

16.  There`s no production of free writing; respondents
choose one of the given alternatives.
 Most professional questionnaires are made up of
‘closed-ended’ items.

17.  There`s no production of free writing; respondents choose one of the
given alternatives.
 Most professional questionnaires are made up of ‘closed-ended’
items.
 Common closed-ended items:
- Likert scale: consisting of a characteristic statement. Respondents are
asked to indicate the extent to which they ’agree’ or ’disagree’.
- Semantic differential scales: by using it researchers can avoid writing
statements.
- Numerical rating scale: ’Giving so many marks out of so many’
- True-false items: Problems: simplifying things too much; resulting
highly reduced and distorted
- Multiple-Choice items: When asking about personal information, such
as level of education
- Rank order: Ordering items by assigning a number to them

18.  They permit greater freedom of expression.
 We use the when we do not know the range of possible answers.
 They can work well if they are not completely open but contain
guidance.
 Four question types in open-ended questions:
1) Specific open questions: asking about concrete pies of information
2) Clarification questions: can be attached to answers with special
importance or after the “Other” in a multiple-choice item
3) Sentence completion: completing an unfinished sentence
4) Short-answer questions: more than a phrase less than a paragraph

19. Source: https://jcu.pressbooks.pub/intro-res-/chapter/3-3-methods-of-quantitative-data-collection/

20. o In social research by mail
o In educational research by hand
o In applied linguistics by group administration
People in general do not mind answering the questions as long as they
think that the survey is related to a worthy cause and that their opinion
matters.

21.  Collecting a huge amount of information in less than an
hour
 Straightforward and fast data processing
 Useable with a variety of people in variety of situations
targeting variety of topics
 Offering anonymity if needed

22.  Easy to produce unreliable and invalid data
 Need of simple and straightforward items
 Unsuitable for probing deeply into an issue
 Usually resulting in superficial data
 Including possible respondent literacy problems
 Including social desirability bias

23.  Experimental study can establish unambiguous cause-effect
relationships.
 First take a group of learners and do something special to them while
measuring their progress. Then compare the data with another group
that is similar to first group, but it did not receive special treatment.
 First group is “treatment” or “experimental group” and the Second
one is “control group”

24.  They are similar to experiments except they do not use
random assignment to create the comparisons
 Two ways of improving its design:
1) Avoiding student's self-selection to be in treatment group
2) Minimizing pre-test differences between the two groups

25.  Matching participants in the treatment and control
groups:
first determine particular individual difference
variables then identify participants in two comparison
groups with similar parameters
 Using analysis of covariance(ANCOVA):
it offers statistical method for adjusting the post-test
scores for any pre-test differences; we can statistically
screen the unwanted effects out of the outcome
measure

26. Two ways of analyzing data obtained with a ‘pre-test-
post-test control group design’:
1. ANOVA: computing ‘gain scores’ separately(by
subtracting the pre-test scores from post-test)
then comparing with T-test or ‘analysis of
variance’
2. ANCOVA: comparing the post-test scores by
controlling from the pre-test scores

27. Strengths
- Best method of establishing cause-effect relationship and
evaluating educational innovations
- ‘pre-test-post-test control group design’ can control the
threats to its validity
Weaknesses
- High price for implementation
- We may end up with artificial framework in laboratory
conditions which reduce the external validity

28. Strengths
We do not have to worry about external validity
Weaknesses
‘selection bias’: inequality of the initial treatment and
control group

29.  A web-based study offers some tempting benefits:
- Reduced costs
- Convenience of administration
- Automatic coding
- High level of anonymity(high level of honesty)
- International access
- Access to specialized population (small, scattered, or specialized)
 Limitation of this approach:
- Technical issues
- Sampling issues

30.  Internet users have different computers, systems, browsers,
and monitors, so the actual stimulus received may differ from
what the investigator has intended. For this reason, we can
employ a single HTML page and improve the user interface.
 Participation is limited by technical issues like connection
speed and quality of installed software but given the speed of
progress these restrictions are likely to be temporary.

31.  There's a lack of control over who will eventually
participate in the study
 The actual sample that completes the web-based
survey or experiment may be much more
heterogeneous than in traditional research.

32. Quantitative data collection refers to the
collection of numerical data that can be analyzed
using statistical methods. The data collected
through quantitative methods is typically in the
form of numbers, such as response frequencies,
means, and standard deviations, and can be
analyzed using statistical software.

37. Purposes

38. Sample selection time

39. General Use

41. 





45. 



46. 



54. Dornyei, Z. (2007). Research methods in applied linguistics. Oxford university press.