This file is used in my elt research class at UKI Jakarta

1. SURVEY RESEARCH
for ELT

2. Defining Surveys
• Survey research designs are procedures in quantitative research in
which investigators collect quantitative, numbered data using
questionnaires (e.g., mailed questionnaires) or interviews
(e.g., one-on-one interviews) and statistically analyze the data to
describe the attitudes, opinions, behaviors, or characteristics of the
surveyed population and to test research questions or hypotheses.
(Creswell, 2008).
• Survey Research is the systematic gathering of information from
respondents for the purpose of understanding and/or predicting
some aspects of the behavior of the population of interest.
• Language surveys are any studies that gather data on the
characteristics and views of informants about the nature of
language or language learning through the use of oral interviews or
written questionnaires (Brown, 2001),

3. Goals of Surveys
• To describe trends, such as high school students’
preferences in having native speaker teachers or non-
native ones.
• To determine individual opinions about policy
issues, such as whether English should be taught in
primary schools.
• To identify important beliefs and attitudes of
individuals, such as college students’ perceptions on
the use of blogs to develop writing skills.
• To get information necessary to evaluate programs in
schools, such as the success of using multimedia
laboratory in English teaching.

4. Three Types of Information Surveys Can Provide
• Factual information, such as the characteristics of individual teachers
and learners (e.g., students’ age, gender, ethnicity, language
background, proficiency level, etc.).
• Behavioral information, i.e. the one that describe what students or
teachers have done or regularly do in terms of their language
teaching and learning. Such data are frequently collected on
language learning strategy questionnaires in which students are
asked, for example, to report how often they look up unfamiliar words
in a dictionary or make an outline before they write an essay.
• Attitudinal information, which depict the opinions, beliefs, or interests
of teachers or learners. These questions are often used in needs
analysis research when researchers want to gather information on
such topics as what learning goals students have or what skill areas
they are most interested in.

5. KEY CHARACTERISTICS OF SURVEY RESEARCH
• Sampling from a population
• Collecting data through questionnaires or
interviews
• Designing instruments for data collection
• Obtaining a high response rate

6. Types of Survey: CROSS-SECTIONAL
A cross-sectional study is one that produces a ‘snapshot’ of
a population at a particular point in time. The researcher
collects data at one point in time in order to:
(1) to examine current attitudes, beliefs, opinions, or
practices;
(2) to compare two or more educational groups (students
with students, students with teachers, students with
parents) in terms of attitudes, beliefs, opinions, or
practices;
(3) to measure community needs of educational services as
they relate to programs, courses, or school facilities
projects;
(4) to evaluate a program, such as a survey that provides
useful information to decision makers.

7. Types of Survey: LONGITUDINAL
• The researcher collects data to study individuals over time. This
design is differentiated into:
(1) Trend study, involves identifying a population and examining
changes within that population over time, e.g., Gallup Poll, which
is used during elections to monitor trends in the population of
voters from the primary to the final election;
(2) cohort study, in which a researcher identifies a subpopulation
based on some specific characteristic and then studies that
subpopulation over time. For instance, a group of 18-year-old
students is studied in the year 2001. Five years later (in 2006), a
group of 23-year-olds is studied. (They may or may not be the
same individuals studied in 2001.) Five years after that (in
2011), a group of 28-year-olds is studied.

8. CROSS-SECTIONAL vs. LONGITUDINAL SURVEY

9. Designing Surveys

10. Questionnaire Design
Tips for designing questionnaire:
1) Consider whether a survey instrument is available to
measure your variables
2) Consider modifying an existing instrument
3) design your own instrument by following 3 steps:
 Write different types of questions, including
personal, attitudinal, and behavioral questions; sensitive
questions; and closed-and open-ended questions.
 Use strategies for good question construction, i.e. using
clear language, making sure the answer options do not
overlap, and posing questions that are applicable to all
participants.
 Perform a pilot test of the questions, and make revision
based on obtained feedback.

11. Guidelines for Designing Questionnaire
1. Keep the questionnaire sufficiently short (30 minutes maximum).
2. Avoid jargon. Seek simplicity but avoid being condescending.
3. Keep questions short, as long and complex questions are difficult to understand.
4. Split double-barrelled questions, e.g. How long have you studied English and
been in receipt of Government support? into two questions.
5. Avoid leading questions which encourage a particular answer, e.g. Do you agree
that your supervisor is supportive?
6. Avoid negatively framed questions or statements which are difficult to
understand, particularly when asked to agree or disagree.
7. Make sure respondents have the knowledge, otherwise you may get false
answers.
8. Check terms are suitable for the context in which they are used, as meanings
may vary for different age groups, religions, cultures etc.
9. Ensure the frame of reference for each question is clear e.g. when asking for the
frequency of an event, specify the time period.
10. Avoid creating opinions. Respondents do not necessarily hold opinions on
topics. Allow a no opinion alternative.
11. Use personal wording if you want the respondents to express their feelings.

12. A good questionnaire …
• is complete, i.e. gets all the data you need;
• is short, i.e. doesn't abuse the respondents’ time or concentration;
• asks only relevant questions;
• gives clear instructions;
• has precise, unambiguous and understandable questions;
• has objective questions, i.e. doesn't suggest answers;
• starts with general questions;
• has appropriate questions;
• puts sensitive questions at the end; is complete, i.e. gets all the data you need;
• is short, i.e. doesn't abuse the respondents7 time or concentration;
• asks only relevant questions;
• 0 gives clear instructions;
• has precise, unambiguous and understandable questions;
• has objective questions, i.e. doesn't suggest answers;
• starts with general questions;
• has appropriate questions;
• puts sensitive questions at the end;

13. Types of Sampling Methods
Sampling Techniques
Non-probability Probability
Sampling Techniques Sampling Techniques
Convenience Judgmental Quota Snowball
Sampling Sampling Sampling Sampling
Simple Random Systematic Stratified Cluster Other Sampling
Sampling Sampling Sampling Sampling Techniques

14. Questionnaire
Telephone
Surveys Mail
Web/Email
Face-to-Face
Interviews

15. Data Analysis Techniques
• DESCRIPTIVE. Deals with the question of ‘what’
things are like, not ‘why’ they are that way, and
includes means, standard deviations, frequency
counts, graphs, and charts.
• ANALYTICAL. Seeks to explain relationships,
causes or consequences, and include bivariate
and multivariate analyses such as correlations,
cross-tabulations and regressions
• CONTEXTUAL. Narrows down the context by
reinterpreting the data for subgroups. E.g. EFL
students vs. ESL students, academic vs non-
academic employees.

16. Instrument Validity
• Construct validity: Does the questionnaire really measure the
construct being examined?
• Criterion-related validity. Does the instrument accurately
predicts (predictive validity) or diagnoses (concurrent validity)
some particular variable (criterion).
• Content validity. Does the contents of the questionnaire really
measure the variable being measured ? To achieve
this, compare your questionnaire to existing related
instruments. If none exists, gather expert opinion on each
question on the instrument to determine whether or not it
actually tests what it is supposed to.

17. Measures for Assuring a Survey’s Reliability
• Giving the same survey on two occasions to the
same individuals and checking the consistency of
the same response to the same item.
• Having the same individuals taking two forms of a
survey.
• check the internal consistency of responses in a
survey, i.e. seeing how consistently the same
respondents answer similar questions formulated
in different forms

18. References
McKay, S. L. (2006). Researching second language classrooms.
New Jersey: Lawrence Erlbaum Associates, Inc., Publishers
Burns. A. (2010). Doing action research in english language
teaching: A guide for practitioners. New York: Routledge:
Creswell, J. W. 2008. Educational research:
Planning, conducting, and evaluating quantitative and
qualitative research. New Jersey: Pearson
Denscombe, M. (2010). The good research guide for small-scale
social research projects. New York: McGraw-Hill
Goddard, W & Melville, S. (2006). Research methodology: An
introduction. Lansdowne: Juta & Co, Ltd.
Ross, Kenneth N. (ed.). (2005). Educational research: Some
basic concepts and terminology. Paris: International Institute
for Educational Planning/ UNESCO.