Basic research questions
Devesh Roy
IFPRI-ICAR training
Important Components of Empirical Research
(Chaturvedi 2012)
2
• Problem statement, research questions, purposes,
benefits
• Theory, assumptions, background literature
• Variables and hypotheses
• Operational definitions and measurement
• Research design and methodology
• Instrumentation, sampling
• Data analysis
• Conclusions, interpretations, recommendations
Focus on world of extension (Donald Davis)
• Before that things about research questions generically
• What Makes for a Successful Paper and Seminar?
• The introduction to a paper or seminar, and the overall structure of the
paper or seminar, can be usefully thought of as efforts to win over a
skeptical referee.
• Imagine being asked the following set of questions:
• What is the question that you want to answer?
• If it takes you more than a few sentences to answer this question, then you likely
have not thought about it hard enough.
• There is an old story of (Ben Franklin?) apologizing for writing a long letter because
he didn’t have time to write a short one. This is the right spirit.
Next question
• Why should we care?
• The kind of answer that one provides to this depends very much on
the problem you consider.
• The kinds that you should think about
• There is a real-world problem that is quantifiably important, potentially with
policy implications, for which this will provide insight.
• There is a significant literature that has addressed this problem, so that the
profession has demonstrated its interest.
• Existing literature inadequate or incomplete in various respects.
• In selecting a topic, remember Summers’ Law: “It takes as much time to
answer a minor question as an important one.”
About the problem to answer
• What do you have to say about the problem that is new?
• Again, if you cannot answer this concisely -- a few sentences at most -
- then probably you have not thought about it hard enough.
• Often it is very helpful to note a couple or a few papers that are
closely related, and be able to say what is distinctive about your
contribution relative to those mentioned.
Another elements of forming a research?
• Why should we believe you?
• You should be able again to describe in just a few sentences what
experiments you are going to perform (empirical or theoretical), and what the
major conclusions will be.
• How convinced should I be?
• No piece of work is the a final word on a problem. You should be clearly able
to acknowledge that there are problems that remain unresolved (lack of data,
future work, etc.).
• In what way should I change my view of the world due to your work?
• This is the bottom line. What have we really learned? You must
have an answer.
Moving to particular
• Do I know the field and its literature well?
• What are the important research questions in my field?
• What areas need further exploration?
• Could my study fill a gap? Lead to greater understanding?
• Has a great deal of research already been conducted in this topic
area?
• Has this study been done before? If so, is there room for
improvement?
• Is the timing right for this question to be answered?
• If you are proposing a service program, is the target community
interested?
• Most importantly, will my study have a significant impact on the field?
Practical marker of a research question
• A well-thought-out and focused research question leads directly into
your hypotheses. What predictions would you make about the
phenomenon you are examining?
• Hypotheses are more specific predictions about the nature and
direction of the relationship between two variables.
• Strong hypotheses:
• Give insight into a research question;
• Are testable and measurable by the proposed experiments;
• Rule of thumb -Normally, no more than three primary hypotheses
should be proposed for a research study.
Elements of hypothesis
• Provide a rationale for your hypotheses—where did they come from, and why are
they strong?
• Provide alternative possibilities for the hypotheses that could be tested—why did
you choose the ones you did over others?
• If you have good hypotheses, they will lead into your Specific Aims. Specific
aims are the steps you are going to take to test your hypotheses and what you
want to accomplish :
• Your objectives are measurable and highly focused;
• Each hypothesis is matched with a specific aim.
• The aims are feasible, given the time and money.
• An example of a specific aim would be “Conduct a rigorous empirical evaluation
of the farmer training schools comparing outcome and process measures from
two groups—those with exposure to the FFS and those without.”
10
Carrying Out an Empirical
Project
11
Choosing a Topic
• Start with a general area or set of questions
• Make sure you are interested in the topic
• Use search services such as (EconLit for economics – what is the
equivalent in this discipline) to investigate past work on this topic
• Narrow down your topic to a specific question or issue to be
investigated
• Work through the theoretical/conceptual issue
12
Choosing Data
• Want data that includes measures of the things that your
theoretical/conceptual model imply are important
• Investigate what type of data sets have been used in the past
literature
• Search for what other data sets are available – Again what secondary
datasets are available?
• Consider collecting your own data – in the world of extension that is
quite likely and natural
13
Using the Data
• Create variables appropriate for analysis
• For example, create dummy variables from categorical variables,
caste, gender, occupational groups
• Generally expected to have an abundance of categorical variables
• Check the data for missing values, errors, outliers, etc.
• Recode as necessary, be sure to report what you did. Logging the
work done is very important
14
Estimating a Model
• Start with a model that is clearly based in theory or driven by
conceptual framework
• Test for significance of other variables that are theoretically less clear
• Test for functional form misspecification
• Consider reasonable interactions, quadratics, logs, etc.
15
Estimating a Model (continued)
• Don’t lose sight of theory or some conceptual framework and the
ceteris paribus interpretation – you need to be careful about including
variables that greatly alter the interpretation
• Be careful about putting functions of y on the right hand side – affects
interpretation
16
Estimating a Model (continued)
• Once you have a well-specified model, need to worry about the
standard errors
• Test for heteroskedasticity
• Correct if necessary
• Test for serial correlation if there is a time component
• Correct if necessary
17
Other Problems
• Often you have to worry about endogeneity of the key explanatory
variable
• Endogeneity could arise from omitted variables that are not
observed in the data
• Endogeneity could arise because the model is really part of a
simultaneous equation
• Endogeneity could arise due to measurement error
18
Other Problems (continued)
• If you have panel data, can consider a fixed effects model (or first
differences)
• Problem with FE is that need good variation over time
• Can instead try to find a perfect instrument and perform 2SLS
• Problem with IV is finding a good instrument
19
Interpreting Your Results
• Keep theory in mind when interpreting results
• Be careful to keep ceteris paribus in mind
• Keep in mind potential problems with your estimates – be cautious
drawing conclusions
• Can get an idea of the direction of bias due to omitted variables,
measurement error or simultaneity
20
Further Issues
• Some problems are just too hard to easily solve with available data
• May be able to approach the problem in several ways, but something
wrong with each one
• Provide enough information for a reader to decide whether they find
your results convincing or not

ICAR-IFPRI - Basic Research Questions lecture 1 - Devesh Roy, IFPRI

  • 1.
    Basic research questions DeveshRoy IFPRI-ICAR training
  • 2.
    Important Components ofEmpirical Research (Chaturvedi 2012) 2 • Problem statement, research questions, purposes, benefits • Theory, assumptions, background literature • Variables and hypotheses • Operational definitions and measurement • Research design and methodology • Instrumentation, sampling • Data analysis • Conclusions, interpretations, recommendations
  • 3.
    Focus on worldof extension (Donald Davis) • Before that things about research questions generically • What Makes for a Successful Paper and Seminar? • The introduction to a paper or seminar, and the overall structure of the paper or seminar, can be usefully thought of as efforts to win over a skeptical referee. • Imagine being asked the following set of questions: • What is the question that you want to answer? • If it takes you more than a few sentences to answer this question, then you likely have not thought about it hard enough. • There is an old story of (Ben Franklin?) apologizing for writing a long letter because he didn’t have time to write a short one. This is the right spirit.
  • 4.
    Next question • Whyshould we care? • The kind of answer that one provides to this depends very much on the problem you consider. • The kinds that you should think about • There is a real-world problem that is quantifiably important, potentially with policy implications, for which this will provide insight. • There is a significant literature that has addressed this problem, so that the profession has demonstrated its interest. • Existing literature inadequate or incomplete in various respects. • In selecting a topic, remember Summers’ Law: “It takes as much time to answer a minor question as an important one.”
  • 5.
    About the problemto answer • What do you have to say about the problem that is new? • Again, if you cannot answer this concisely -- a few sentences at most - - then probably you have not thought about it hard enough. • Often it is very helpful to note a couple or a few papers that are closely related, and be able to say what is distinctive about your contribution relative to those mentioned.
  • 6.
    Another elements offorming a research? • Why should we believe you? • You should be able again to describe in just a few sentences what experiments you are going to perform (empirical or theoretical), and what the major conclusions will be. • How convinced should I be? • No piece of work is the a final word on a problem. You should be clearly able to acknowledge that there are problems that remain unresolved (lack of data, future work, etc.). • In what way should I change my view of the world due to your work? • This is the bottom line. What have we really learned? You must have an answer.
  • 7.
    Moving to particular •Do I know the field and its literature well? • What are the important research questions in my field? • What areas need further exploration? • Could my study fill a gap? Lead to greater understanding? • Has a great deal of research already been conducted in this topic area? • Has this study been done before? If so, is there room for improvement? • Is the timing right for this question to be answered? • If you are proposing a service program, is the target community interested? • Most importantly, will my study have a significant impact on the field?
  • 8.
    Practical marker ofa research question • A well-thought-out and focused research question leads directly into your hypotheses. What predictions would you make about the phenomenon you are examining? • Hypotheses are more specific predictions about the nature and direction of the relationship between two variables. • Strong hypotheses: • Give insight into a research question; • Are testable and measurable by the proposed experiments; • Rule of thumb -Normally, no more than three primary hypotheses should be proposed for a research study.
  • 9.
    Elements of hypothesis •Provide a rationale for your hypotheses—where did they come from, and why are they strong? • Provide alternative possibilities for the hypotheses that could be tested—why did you choose the ones you did over others? • If you have good hypotheses, they will lead into your Specific Aims. Specific aims are the steps you are going to take to test your hypotheses and what you want to accomplish : • Your objectives are measurable and highly focused; • Each hypothesis is matched with a specific aim. • The aims are feasible, given the time and money. • An example of a specific aim would be “Conduct a rigorous empirical evaluation of the farmer training schools comparing outcome and process measures from two groups—those with exposure to the FFS and those without.”
  • 10.
    10 Carrying Out anEmpirical Project
  • 11.
    11 Choosing a Topic •Start with a general area or set of questions • Make sure you are interested in the topic • Use search services such as (EconLit for economics – what is the equivalent in this discipline) to investigate past work on this topic • Narrow down your topic to a specific question or issue to be investigated • Work through the theoretical/conceptual issue
  • 12.
    12 Choosing Data • Wantdata that includes measures of the things that your theoretical/conceptual model imply are important • Investigate what type of data sets have been used in the past literature • Search for what other data sets are available – Again what secondary datasets are available? • Consider collecting your own data – in the world of extension that is quite likely and natural
  • 13.
    13 Using the Data •Create variables appropriate for analysis • For example, create dummy variables from categorical variables, caste, gender, occupational groups • Generally expected to have an abundance of categorical variables • Check the data for missing values, errors, outliers, etc. • Recode as necessary, be sure to report what you did. Logging the work done is very important
  • 14.
    14 Estimating a Model •Start with a model that is clearly based in theory or driven by conceptual framework • Test for significance of other variables that are theoretically less clear • Test for functional form misspecification • Consider reasonable interactions, quadratics, logs, etc.
  • 15.
    15 Estimating a Model(continued) • Don’t lose sight of theory or some conceptual framework and the ceteris paribus interpretation – you need to be careful about including variables that greatly alter the interpretation • Be careful about putting functions of y on the right hand side – affects interpretation
  • 16.
    16 Estimating a Model(continued) • Once you have a well-specified model, need to worry about the standard errors • Test for heteroskedasticity • Correct if necessary • Test for serial correlation if there is a time component • Correct if necessary
  • 17.
    17 Other Problems • Oftenyou have to worry about endogeneity of the key explanatory variable • Endogeneity could arise from omitted variables that are not observed in the data • Endogeneity could arise because the model is really part of a simultaneous equation • Endogeneity could arise due to measurement error
  • 18.
    18 Other Problems (continued) •If you have panel data, can consider a fixed effects model (or first differences) • Problem with FE is that need good variation over time • Can instead try to find a perfect instrument and perform 2SLS • Problem with IV is finding a good instrument
  • 19.
    19 Interpreting Your Results •Keep theory in mind when interpreting results • Be careful to keep ceteris paribus in mind • Keep in mind potential problems with your estimates – be cautious drawing conclusions • Can get an idea of the direction of bias due to omitted variables, measurement error or simultaneity
  • 20.
    20 Further Issues • Someproblems are just too hard to easily solve with available data • May be able to approach the problem in several ways, but something wrong with each one • Provide enough information for a reader to decide whether they find your results convincing or not