1
Research Seminar for
Educational Sciences
Prof. Dr. Chang Zhu
Department of Educational Sciences
Major sections of a research
proposal
2
Title
Summary
Research problem
Literature review
Research questions
Research methods
Expects results &
significance
Planning
2
Literature review
• Theoretical basis/ conceptual framework
• What research has been done in the
area?
• What has been found in previous
studies?
3
A good research idea
• Choose something you are interested in, think it
fascinating, relevant for your future work,…
– You’ll need that personal motivation to pursue the studies.
• Relevant, in need and ‘hot’:
– useful to someone, such as policy makers
– more motivating to work on
– help in a broader research project
– other sources
3
Research questions: why
important?
• Developing a good research question is the
most important part of the research process.
• The question should be narrow and fits the
research objectives.
Research questions: why
important?
• The most difficult part, spend time on it!
• Risk if not well defined:
– collect data that will not be used;
– don’t collect data that would be useful.
• Good research questions limit the scope of the thesis
• A good research questions structure the thesis:
– define relevant literature,
– develop good hypotheses,
– define data needed, ….
• It helps the reader – you tell them what you are going
to answer. It simply makes it more interesting!
4
Central research questions and
specific research questions
• Ask one or two central research questions
• Followed by a number of specific research
questions
Types of research
questions
• Description
• Relationship
• Descriptive-Comparative
• Causality/impact or affect
• Causality-Comparative
• ………
5
Types of research
questions
• Research questions in qualitative research
• Research questions in quantitative research
Types of research questions
• Questions of Description
• What are the characteristics of…?
• What/Which factors influence…..?
?
Qualitative? Quantitative?
6
Types of research questions
• Questions of Description
• What/Which factors influence
international student decision in
choosing VUB for their studies?
Types of research questions
• Questions of relationship
• Is there an association between … and ….?
• Is (variable A) related to (variable B)?
7
Types of research questions
• Questions of relationship
• Does critical thinking ability relate to
student achievement?
• To what extent does critical thinking ability
relate to student achievement?
Types of research questions
• Descriptive-Comparative Questions
• What differences exist….?
• Is group A differ from Group B?
• Are there differences between…..?
• Are there differences between female and
male students with regard to….?
8
Types of research questions
• Causality/impact Questions
• Does ….(variable A) affect….(variable B)?
• Does …..have an impact on…?
• To what extent does ….(variable A)
affect….(variable B)?
Types of research questions
• Causality-Comparative
• Does ….(variable A) have a higher
impact on (variable Y) compared to the
impact of….(variable B) on … (variable
Y)?
9
good research questions
• Not purely descriptive
• Not just What/Which questions
• But also How/To what extent….questions
• Relationship questions
• Comparative questions
• Causality/impact questions
• …
good research questions
• Be clear about the research variables
• Reflect in your research questions
10
Variables
• A variable: any entity that can take on
different values.
• A variable is a label or name that
represents a concept or characteristic that
varies
19
Variables
• E.g. age, gender, grade, school, study domain,
diploma, income, country…
use of ICT, exam score, performance,
academic achievement, attitude, emotion,
motivation, perception, satisfaction, well-
being, self-esteem, critical thinking, ….
20
11
Independent variable
• The one you will vary to see how it may affect
the dependent variable
• Independent Variable – something that is
changed by the scientist/researcher
– What is tested
– What is manipulated
• In experimental research:
• Manipulated Variable
Independent variable
• In experimental research:
• Keep all variables constant, except for the
independent variable.
• A Experimental Group will be the group with the
Independent Variable:
• A Control Group will be the group without the
Independent Variable
12
Dependent variable
• The One You Measure as a result of the
changing Independent variable
• A variable that might be affected by the
change in the independent variable
– What is observed
– What is measured
– The data collected during the investigation
Construct
• What is a construct?
A construct is a (theoretical) concept that is
inferred from commonalities among observed
phenomena and that is presumed to explain or
shed light on the meaning of those phenomena.
• A variable is a quantitative expression of a
construct.
24
13
Construct
• Constructs, Factors, Scales, Sub-scales
• Measured through a number of observed items
• E.g. Teacher professional identity (including
Self-efficacy, job satisfaction, motivation and
commitment)
25
• Characterized by complete random assignment
of groups or subjects
• Groups are independent
• Usually employs strong control
Experimental design
14
• Groups or subjects not randomly assigned
– e.g., sample of convenience
• May not have a comparison group
Quasi-Experimental design
• Single factor (one-way)
– Studies one independent variable
• Multi-factor
– Studies multiple independent variables
• May have several levels
– Examples:
• Two-way (e.g., 2 x 2)
• Three-way (e.g., 2 x 2 x 2)
• Time-series
Types of experimental design
15
• Pretest-posttest (one-group)
• Pretest-posttest (control group)
• Posttest-only (control group)
Single factor design
• Quasi-experimental
• One set of measures taken before and after
treatment or intervention
• Compare pretest and posttest scores
• Analysis
– paired t test
• Weakness
– No comparison or
control group
Single factor design
16
• Two-way factorial
– e.g., 2 x 3
• Three-way factorial
– e.g., 2 x 2 x 3
Experimental Designs
– Quasi-experimental
– True experimental
– Factorial
17
Experimental Designs
• Notation
– R indicates random selection or random
assignment
– O indicates an observation
• Test
• Observation score
• Scale score
– X indicates a treatment
– A, B, C, ... indicates a group
Quasi-Experimental Designs
• Threats to internal validity
– Selection is the major concern
– Controls for statistical regression
– Likely to control for most other threats provided
the groups are not significantly different from one
another
18
True Experimental Designs
• Important
– Random assignment
• Participants are placed into groups using a
random procedure
• This ensures equivalency of the groups
– Random selection of subjects
• Participants are chosen from a population using
random procedures
• This ensures generalizability to the population
from which the participants were selected (i.e.,
external validity)
True Experimental Designs
• Types
– Randomized posttest only experimental control
groups
• R A X O R B O
– Randomized posttest only multiple treatment
groups
• R A X1 O R B X2 O
– Randomized pretest-posttest experimental control
groups
• R A O X O R B O O
19
Factorial Designs
• Research designs containing two or
more independent variables
– A study of the effects of two instructional
strategies on male and female students’
math achievement
– A study of two counseling approaches on
middle and secondary students’ self-
esteem
Hypotheses in quantitative
research
• Hypotheses about comparison
• Null hypothesis
• There is no significant difference between….
• Directional hypothesis
• Students in Group A will have higher achievement
than students in Group B…
20
Hypotheses in quantitative
research
• Hypotheses about relationships
• There is no relationship between student
computer use skills and their performance in e-
learning
• Student achievement motivation is related to their
anxiety for failure. (nondirectional)
• Students from higher SES families have higher
media literacy than students from lower SES
families. (directional)
The hypotheses
• The hypothesis is based on known facts,
but takes it a step further, often by
combining facts and thereby creating
(possible) links.
• The hypothesis posits the expected
relationship between some variables.
• Two requirements:
– Must fit known facts
– Be testable
21
Do all theses need to have
hypotheses?
• No, but most of them would be better if the
research is organized around them
• What’s the difference between research
question and hypotheses?
– Hypotheses often more specific, and each research
question can have several hypotheses.
Suggestion
A good approach and balance between the three
first components of the research:
– one grand research idea
– a few (3-5) research questions
– some hypotheses, at least one, maximum 3-5 linked
to each research question. But in total not more than
6-8 hypotheses for a master thesis.
22
How to make a good research
question?
• Read – discuss – think!
• Get into your creative mood!
Some useful note
• Research should not be embarked upon with the
idea that the empirical evidence will prove a
general “truth”, but it can increase our
knowledge & understanding on a certain issue.
• Research can only demonstrate the utility of an
idea within a specific context.
23
Some useful note
• Researchers must be prepared for unexpected results.
Unexpected results represent the possibility of
learning something new about the phenomenon under
study.
• Finding unexpected results also shows that there is a
good reason for conducting research.
Additional readings
• Creswell (2008). Research questions and hypotheses
24
Assignment
• Read Theme 5
• Self-test/quiz: theme 1-5
Research topics
• Research group
• Or individual work
• Thesis Info session
• 3 Dec. 12-14h

Research seminar lecture_4_research_questions

  • 1.
    1 Research Seminar for EducationalSciences Prof. Dr. Chang Zhu Department of Educational Sciences Major sections of a research proposal 2 Title Summary Research problem Literature review Research questions Research methods Expects results & significance Planning
  • 2.
    2 Literature review • Theoreticalbasis/ conceptual framework • What research has been done in the area? • What has been found in previous studies? 3 A good research idea • Choose something you are interested in, think it fascinating, relevant for your future work,… – You’ll need that personal motivation to pursue the studies. • Relevant, in need and ‘hot’: – useful to someone, such as policy makers – more motivating to work on – help in a broader research project – other sources
  • 3.
    3 Research questions: why important? •Developing a good research question is the most important part of the research process. • The question should be narrow and fits the research objectives. Research questions: why important? • The most difficult part, spend time on it! • Risk if not well defined: – collect data that will not be used; – don’t collect data that would be useful. • Good research questions limit the scope of the thesis • A good research questions structure the thesis: – define relevant literature, – develop good hypotheses, – define data needed, …. • It helps the reader – you tell them what you are going to answer. It simply makes it more interesting!
  • 4.
    4 Central research questionsand specific research questions • Ask one or two central research questions • Followed by a number of specific research questions Types of research questions • Description • Relationship • Descriptive-Comparative • Causality/impact or affect • Causality-Comparative • ………
  • 5.
    5 Types of research questions •Research questions in qualitative research • Research questions in quantitative research Types of research questions • Questions of Description • What are the characteristics of…? • What/Which factors influence…..? ? Qualitative? Quantitative?
  • 6.
    6 Types of researchquestions • Questions of Description • What/Which factors influence international student decision in choosing VUB for their studies? Types of research questions • Questions of relationship • Is there an association between … and ….? • Is (variable A) related to (variable B)?
  • 7.
    7 Types of researchquestions • Questions of relationship • Does critical thinking ability relate to student achievement? • To what extent does critical thinking ability relate to student achievement? Types of research questions • Descriptive-Comparative Questions • What differences exist….? • Is group A differ from Group B? • Are there differences between…..? • Are there differences between female and male students with regard to….?
  • 8.
    8 Types of researchquestions • Causality/impact Questions • Does ….(variable A) affect….(variable B)? • Does …..have an impact on…? • To what extent does ….(variable A) affect….(variable B)? Types of research questions • Causality-Comparative • Does ….(variable A) have a higher impact on (variable Y) compared to the impact of….(variable B) on … (variable Y)?
  • 9.
    9 good research questions •Not purely descriptive • Not just What/Which questions • But also How/To what extent….questions • Relationship questions • Comparative questions • Causality/impact questions • … good research questions • Be clear about the research variables • Reflect in your research questions
  • 10.
    10 Variables • A variable:any entity that can take on different values. • A variable is a label or name that represents a concept or characteristic that varies 19 Variables • E.g. age, gender, grade, school, study domain, diploma, income, country… use of ICT, exam score, performance, academic achievement, attitude, emotion, motivation, perception, satisfaction, well- being, self-esteem, critical thinking, …. 20
  • 11.
    11 Independent variable • Theone you will vary to see how it may affect the dependent variable • Independent Variable – something that is changed by the scientist/researcher – What is tested – What is manipulated • In experimental research: • Manipulated Variable Independent variable • In experimental research: • Keep all variables constant, except for the independent variable. • A Experimental Group will be the group with the Independent Variable: • A Control Group will be the group without the Independent Variable
  • 12.
    12 Dependent variable • TheOne You Measure as a result of the changing Independent variable • A variable that might be affected by the change in the independent variable – What is observed – What is measured – The data collected during the investigation Construct • What is a construct? A construct is a (theoretical) concept that is inferred from commonalities among observed phenomena and that is presumed to explain or shed light on the meaning of those phenomena. • A variable is a quantitative expression of a construct. 24
  • 13.
    13 Construct • Constructs, Factors,Scales, Sub-scales • Measured through a number of observed items • E.g. Teacher professional identity (including Self-efficacy, job satisfaction, motivation and commitment) 25 • Characterized by complete random assignment of groups or subjects • Groups are independent • Usually employs strong control Experimental design
  • 14.
    14 • Groups orsubjects not randomly assigned – e.g., sample of convenience • May not have a comparison group Quasi-Experimental design • Single factor (one-way) – Studies one independent variable • Multi-factor – Studies multiple independent variables • May have several levels – Examples: • Two-way (e.g., 2 x 2) • Three-way (e.g., 2 x 2 x 2) • Time-series Types of experimental design
  • 15.
    15 • Pretest-posttest (one-group) •Pretest-posttest (control group) • Posttest-only (control group) Single factor design • Quasi-experimental • One set of measures taken before and after treatment or intervention • Compare pretest and posttest scores • Analysis – paired t test • Weakness – No comparison or control group Single factor design
  • 16.
    16 • Two-way factorial –e.g., 2 x 3 • Three-way factorial – e.g., 2 x 2 x 3 Experimental Designs – Quasi-experimental – True experimental – Factorial
  • 17.
    17 Experimental Designs • Notation –R indicates random selection or random assignment – O indicates an observation • Test • Observation score • Scale score – X indicates a treatment – A, B, C, ... indicates a group Quasi-Experimental Designs • Threats to internal validity – Selection is the major concern – Controls for statistical regression – Likely to control for most other threats provided the groups are not significantly different from one another
  • 18.
    18 True Experimental Designs •Important – Random assignment • Participants are placed into groups using a random procedure • This ensures equivalency of the groups – Random selection of subjects • Participants are chosen from a population using random procedures • This ensures generalizability to the population from which the participants were selected (i.e., external validity) True Experimental Designs • Types – Randomized posttest only experimental control groups • R A X O R B O – Randomized posttest only multiple treatment groups • R A X1 O R B X2 O – Randomized pretest-posttest experimental control groups • R A O X O R B O O
  • 19.
    19 Factorial Designs • Researchdesigns containing two or more independent variables – A study of the effects of two instructional strategies on male and female students’ math achievement – A study of two counseling approaches on middle and secondary students’ self- esteem Hypotheses in quantitative research • Hypotheses about comparison • Null hypothesis • There is no significant difference between…. • Directional hypothesis • Students in Group A will have higher achievement than students in Group B…
  • 20.
    20 Hypotheses in quantitative research •Hypotheses about relationships • There is no relationship between student computer use skills and their performance in e- learning • Student achievement motivation is related to their anxiety for failure. (nondirectional) • Students from higher SES families have higher media literacy than students from lower SES families. (directional) The hypotheses • The hypothesis is based on known facts, but takes it a step further, often by combining facts and thereby creating (possible) links. • The hypothesis posits the expected relationship between some variables. • Two requirements: – Must fit known facts – Be testable
  • 21.
    21 Do all thesesneed to have hypotheses? • No, but most of them would be better if the research is organized around them • What’s the difference between research question and hypotheses? – Hypotheses often more specific, and each research question can have several hypotheses. Suggestion A good approach and balance between the three first components of the research: – one grand research idea – a few (3-5) research questions – some hypotheses, at least one, maximum 3-5 linked to each research question. But in total not more than 6-8 hypotheses for a master thesis.
  • 22.
    22 How to makea good research question? • Read – discuss – think! • Get into your creative mood! Some useful note • Research should not be embarked upon with the idea that the empirical evidence will prove a general “truth”, but it can increase our knowledge & understanding on a certain issue. • Research can only demonstrate the utility of an idea within a specific context.
  • 23.
    23 Some useful note •Researchers must be prepared for unexpected results. Unexpected results represent the possibility of learning something new about the phenomenon under study. • Finding unexpected results also shows that there is a good reason for conducting research. Additional readings • Creswell (2008). Research questions and hypotheses
  • 24.
    24 Assignment • Read Theme5 • Self-test/quiz: theme 1-5 Research topics • Research group • Or individual work • Thesis Info session • 3 Dec. 12-14h