Fundamentals of Research in Education course content

1. Research Design:
Survey
Lecturer: Yee Bee Choo
IPGKTHO
Topic 1d

2. What is Survey
• Survey design (also descriptive design) seeks to
describe trends in a large population of
individuals.
• Survey designs are procedures in quantitative
research in which researchers administer a
survey or questionnaire to a small group of
people (called the sample) to identify trends in
attitudes, opinions, behaviours, characteristics,
or interests of a large group of people (called the
population) (Creswell, 2012).

3. What is a Survey
• Survey designs differ from experimental research
in that they do not involve a treatment given to
participants by the researcher.
• Because survey researchers do not
experimentally manipulate the conditions, they
cannot explain cause and effect as well as
experimental researchers can.

4. When do Use Survey
• To describe trends, such as community interests in school
bond issues or state or national trends about mandatory
student uniform policies.
• To determine individual opinions about policy issues,
such as whether students need a choice of schools to
attend.
• To identify individual beliefs and attitudes, such as
college students’ beliefs about what constitutes abusive
behaviors in dating relationships.
• To study individual over time, such as to follow up with
graduates 5, 10, or 15 years after college to learn about
their present careers.
• To evaluate programmes in schools, such as the success
of a robotics program in science education.

5. Types of Survey
Survey
Research
Cross-sectional
collect data about
current attitudes,
opinions, or beliefs at just
one point in time
Longitudinal
collects data to study
individuals over time
at different points in
time

6. Types of Survey
(Creswell, 2012, p.378)

7. Types of Survey
Longitudinal or cross-sectional?
Cross-sectional
National assessment

8. Types of Survey
Longitudinal or cross-sectional?
• A study examines how adolescents with learning
disabilities made the transition from vocational–technical
schools to work.
• Two groups of high school seniors: one with learning
disabilities and one without learning disabilities were
surveyed at graduation and at 6-, 12-, and 24-month
intervals after graduation to learn about their
occupational and living experiences.
• The surveys were sent to seniors who graduated in 1986
and 1987.
Longitudinal
Panel Studies

9. Types of Survey
Longitudinal or cross-sectional?
• One study compared 98 rural and urban secondary
school teachers from 11 school systems in Georgia and
North Carolina in terms of their sources of stress and
symptoms of burnout.
Cross-secctional
Group comparisons

10. Types of Survey
Longitudinal or cross-sectional?
• A survey is conducted to identify the needs and priorities
for human service programmes by the Hispanic in the
Tampa Bay, Florida area.
Cross-secctional
Community needs

11. Types of Survey
Longitudinal or cross-sectional?
• A study was conducted on a group of 18-year-olds in the
year 2001. Five years later (in 2006), a group of 23-year-
olds is studied. (They may or may not be the same
individuals studied in 2001.) Five years after that (in
2011), a group of 28-year-olds is studied. While the
individuals studied each time might be different, they
must have been 18 years old in the year 2001 to qualify
as representatives of the group.
Longitudinal
Cohort studies

12. Characteristics of Survey
1. Sampling from a population
2. Collecting data through questionnaires or
interviews
3. Designing instruments for data collection
4. Obtaining a high response rate

13. Characteristics of Survey
1. Sampling from a population
• Survey researchers typically select and study a
sample from a population and generalise results
from the sample to the population.
• In survey research, it is important to select as
large a sample as possible so that the sample will
exhibit similar characteristics to the target
population.

14. Characteristics of Survey
(Creswell, 2012, p.381)

15. Characteristics of Survey
1. Sampling from a population
• It is also possible to study the entire population
when it is small (e.g., members of literacy
councils in a state) and can be easily identified.
• This survey is called a census study, permits
conclusions to be drawn about the entire
population.
• Therefore, random sampling, hypothesis testing,
and the use of inferential statistics are not
necessary but simple report of descriptive
statistics about the entire population.

16. Characteristics of Survey
2. Collecting data through questionnaires or interviews
• The data is collected by examining who
completes or records the data on the
instrument: the participants (called respondents
or interviewees) or the researcher.

17. Characteristics of Survey
(Creswell, 2012, p.383)

18. Characteristics of Survey
2. Collecting data through questionnaires or interviews
Quantitative Vs Qualitative interview
• In quantitative survey interviews, a structured or
semistructured interview consisting of mostly
closed-ended questions which provides response
options to interviewees is used to record their
responses.
• In qualitative survey interviews, an interviewer asks
open-ended questions without response options and
listens to and records the comments of the
interviewee.

19. Activity
Suppose you wish to study teachers’ views or
opinions of allowing pupils to bring handphones to
school, which data will you use to collect the data?
Justify your answers.
(a) a mailed questionnaire
(b) an electronic questionnaire
(c) one-on-one interviews
(d) focus group interviews
(e) telephone interviews

20. Characteristics of Survey
3. Designing instruments for data collection
Steps in designing a survey:
a. Write different types of questions: Personal,
attitudinal, and behavioral questions; sensitive
questions; and closed- and open-ended questions.
b. Use strategies for good question construction:
Using clear language, making sure the answer
options do not overlap, and posing questions that
are applicable to all participants.
c. Perform a pilot test of the questions:
Administering the instrument to a small number of
individuals and making changes based on their
feedback.

21. Characteristics of Survey
3. Designing instruments for data collection
a. Different Types of Questions:
1. Personal (also called demographic or background information)
e.g. age _____, gender (M/F), race (M/C/I)
2. Attitudinal (obtaining individual attitudes or opinions)
e.g. How much do you agree or disagree with this statement:
Most days I am enthusiastic about being a student.
Strongly agree, Agree, Disagree, Strongly disagree
3. Behavioral (solicit information about the actual behavior of
individuals)
e.g. Did you take a semester off during any of your 4 years of
college?
________ Yes
________ No

22. Characteristics of Survey
3. Designing instruments for data collection
a. Different Types of Questions:
4. Sensitive (need to use with care)
e.g. Have you ever used marijuana?
You might ask: In past surveys, many men have
reported that at some point in their lives they have
used marijuana. This could have happened before
adolescence, during adolescence, or as an adult.
Have you ever smoked marijuana?
X X X

23. Characteristics of Survey
3. Designing instruments for data collection
a. Different Types of Questions:
5. Closed-ended
• The researcher poses a question and provides preset response options for
the participant.
e.g. What is your most important reason for coming to teacher education?
_______ To get a better job
_______ To get a high salary
_______ To fulfill my parent’s dream
_______ To fulfill my interests
6. Open-ended
• The researcher do not provide the response options and allow the
participants to provide their own responses to questions.
e.g. What is your most important reason for coming to teacher education?
_________________________________________________________
_ ________________________________________________________

24. Characteristics of Survey
3. Designing instruments for data collection
a. Different Types of Questions:
5. Semi-closed-ended
• The researcher asks a closed-ended question and then ask for
additional responses in an open-ended question
e.g. What is your most important reason for coming to teacher
education?
_______ To get a better job
_______ To get a high salary
_______ To fulfill my parent’s dream
_______ To fulfill my interests
_______ Other (Please comment) ______________________

25. Characteristics of Survey
3. Designing instruments for data collection
b. Question Construction

26. Characteristics of Survey
3. Designing instruments for data collection
b. Question Construction

27. Characteristics of Survey
3. Designing instruments for data collection
b. Question Construction

28. Characteristics of Survey
3. Designing instruments for data collection
c. Pilot Testing the Questions
• A pilot test of a questionnaire or interview survey is a procedure in
which a researcher makes changes in an instrument based on
feedback from a small number of individuals who complete and
evaluate the instrument.
• The participants in the pilot test provide written comments directly
on the survey (such as poorly worded questions, responses that do
not make sense, or if it takes an excessive amount of time to
complete the instrument, etc.), and the researcher modifies or
changes the survey to reflect those concerns.
• Because the pilot group provides feedback on the questionnaire,
they are excluded from the final sample for the study.

29. Characteristics of Survey
4. Response Rate
• When using interviews, the response rate is high
because individuals interviewed typically
consent to the interview in advance.
• However, when questionnaires are used, the
number of responses returned (through mail or
electronically) will vary.

30. Advantages of Survey
1. Relatively easy to administer.
2. Can be developed in less time.
3. Cost-effective, but cost depends on survey mode (paper
or online).
4. Can be administered remotely via online, mobile
devices, mail, email, kiosk, or telephone.
5. Capable of collecting data from a large number
of respondents.
6. A broad range of data can be collected (e.g., attitudes,
opinions, beliefs, values, behavior, factual).

31. Disadvantages of Survey
1. A large sample of the population must be studied for
more accurate results.
2. Respondents may not feel encouraged to provide
accurate, honest answers.
3. Data errors due to question non-responses may exist.
4. Survey question answer options could lead to unclear
data because certain answer options (e.g. “somewhat
agree”) may be interpreted differently by respondents.
5. Sometimes, low response rate (for the respondents to
return questionnaire survey).

32. Steps in Conducting Survey
1. Decide if a survey is the best design to use.
2. Identify the research questions or hypotheses
3. Identify the population, the sampling frame and the
sample
4. Determine the survey design and data collection
procedures
5. Develop or locate an instrument
6. Administer the instrument
7. Analyse the data to address research questions or
hypotheses
8. Write the report

33. A Good Survey
1. To reduce coverage error, have a good sampling frame
list on which to select individuals.
2. To reduce sampling error, select as large a sample from
the population as possible.
3. To reduce measurement error, use a good instrument,
with clear, unambiguous questions and response
options.
4. To reduce nonresponse error, use rigorous
administration procedures to achieve as large a return
rate as possible.

34. Let’s do a Survey
Topic
• Reflective Practice and Effective Teaching Among Pre-
service Teachers
Background of Study
• Reflective practice, a common practice by teachers and
pre-service teachers, has gained the status of an integral
element of teacher pedagogy.
Statement of Problem
• In reflective teaching discourse, while much attention
has been paid to the what of teachers’ reflective practice
and to its how in terms of methodological principles and
guidelines, language teachers’ perception of their level of
reflection has remained relatively unexplored.

35. Let’s do a Survey
Purpose of Study
• To examine the English language teachers’ perception of their level of
reflection
Research Objectives
• To examine the level of reflection perceived by the pre-service teachers
in terms of practical, cognitive, learner, meta-cognitive, and critical
• To examine the difference of level of reflection and the gender of the
pre-service teachers
Research Question
• What level of reflection do the pre-service teachers perceive
themselves?
• Is there a difference in the level of reflection and the gender of the pre-
service teachers?
Research Hypothesis
• There is a difference in the level of reflection and the gender of the pre-
service teachers.

36. Let’s do a Survey
Scope of Study
• A group of sixteen pre-service teachers at IPGKTHO who are
majoring in TESL and they had the teaching experience during
the first phase of practicum.
Limitations of Study
• The number of respondents is limited.
• The gender of the respondents is not equal.
Significance of Study
• It raises awareness among the language teachers of the
importance of reflection in effective language teaching in the
ESL context.
• In particular, it might raise awareness among teachers with
regard to their professional development through reflecting
on those aspects of their instruction which are directly related
to effective teaching.

37. Let’s do a Survey
Operational Definitions
1. Reflective Practice
• Dewey (1933) defined reflective practice as the "active,
persistent, and careful consideration of any belief or
supposed form of knowledge in the light of the grounds
that support it and further conclusions to which it tends”.
• Therefore, reflective practice in this study involves the
acquisition of certain cognitive skills (knowledge) and
attitudes which enable the practitioner to have reflective
thinking, self-awareness and self-concepts.
• It can be categorised into five levels: practical, cognitive,
learner, meta-cognitive and critical.

38. Let’s do a Survey
Operational Definitions
2. Effective teaching
• There is a proactive role of reflective teachers who teach
effectively as compared with the passive role of teachers
in traditional teaching practice (Osterman & Kottkamp,
1993).
• Therefore, effective teaching involves changing the
learners’ behaviour and performance with the aim of
making them active class participants.

39. Tutorial 1d (Pair Work)
• Read the article from http://ueab.ac.ke/BIRJ/wp-
content/uploads/2017/09/Truman-63-76.pdf
• Based on the article, identify the following:
• Title
• Author
• Problem/Issue
• Suggestion(s) to overcome the problem/issue
• Research objectives/questions
• Research hypotheses
• Sample
• Research methodology
• Research instruments
• Findings/ Discussion of findings
• You can type or handwrite your answer.
• Write your answers in not more than two sentences.

40. Main Reference
• Creswell, J.W. (2012). Educational research:
planning, conducting and evaluating quantitative
and qualitative research. (4th Ed.). Boston, MA:
Pearson Education Inc.