Empirical research methods poster


Published on

Published in: Education
1 Comment
  • would be great to download it...
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Empirical research methods poster

  1. 1. Empirical Research Methods Poster Check for the latest version at: http://researchmethods.itposter.net Define research question How facts become theories Common research process Select research method Analyze quantitative data Confidence Start empirical Statistics is a mathematical science pertaining to the collection, analysis, interpretation or Can be pretty sure your Research is a systematic explanation, and presentation of data. in theory is conclusions are valid ... Relationships increased Empirical research is a research process for answering Experiment Case study Survey But are you getting at the Why Explanatory Event between Laws are Prediction is research approach in which questions to solve Statistical methods can be used to summarize or describe a collection of data; this is called correct issues? research problems and create new How constructs formed confirmed empirical observations (data) descriptive statistics. In addition, patterns in the data may be modeled in a way that accounts for Level of control Purpose are identified Theory is are collected to answer knowledge. Establishes causal Investigate a typical Investigate information collected randomness and uncertainty in the observations, and then used to draw inferences about the process How many rejected research question. relationships, confirm »case« in realistic from a group of people, projects, or population being studied; this is called inferential statistics. Both descriptive and inferential How much Descriptive Research Question (RQ) is what theories. representative conditions. organizations or literature. statistics comprise applied statistics. The goal of the theory has to you are trying to find out by When research Event Predictions Research is undertaking the research Theory is from theory performed be defined here. Based on the Who can be research question an empirical Define research process. Control Requires high control Requires medium control Requires low control formed that drawn, which Theory is strategy has to be chosen. question Descriptive statistics Where Exploratory explains laws form NOT modified A clear and precise RQ guides Control on who is using Change to be assessed Technology change is theory development, research Descriptive statistics are used to describe the basic features of the data in a study. They provide appropriate What research hypotheses which technology, (e.g., new technology) implemented across a large Can be pretty sure you are Event Prediction is It is used to find out what is design, data collection and data when, where, and number of projects. simple summaries about the sample and the measures. Together with simple graphics analysis, is wide-ranging When NOT confirmed already known about a question analysis. they form the basis of virtually every quantitative analysis of data. getting at the correct Level of access Relationships Confidence in before trying to answer it. under which conditions throughout the Description of results, issues … between Laws are is possible. development process. influence factors, But can you draw valid constructs formed theory is reduced Review Types of research questions are: Measures of central tendency Theory is modified To investigate self Assessment in a typical differences and conclusions? are identified A good literature review is an What?, Where?, Who?, When?, A measure of central tendency is a single number that is used to represent the average score in important part of any research. Literature How Much?, How many?, Why?, standing tasks from situation required. commonalities is needed. Event which results can be the distribution. How? Mode – the most common score in a frequency distribution obtained immediately. Research question examples: The goal of literature review is Median – the middlemost score in a distribution to demonstrate a familiarity Can establish causal Can be incorporated in Can use existing experience. Mean – the common average What are the key success factors of object-oriented frameworks? with a body of knowledge and Does the proposed software improvement increases the efficiency of its users? relationships. normal development Can confirm an effect How does software development methodology and team size influence developers productivity? Create theoretical model establish credibility. Additional, to show the path to prior research and how a current Create theoretical Theoretical model is used to conceptualize the problem Can confirm theories. activities. Already scaled up to generalizes to many projects/organizations. Measures of variability A single number which describes how much the data vary in the distribution. research is related to it. model stated in research question. It life size if performed on Allow to use standard Range – The difference between the highest and lower score in a distribution. Research question: »How does software development methodology and team size is commonly represented with real projects. statistical techniques. Variance – The average of the squared deviations from the mean. causal model. Standard deviation – the square root of the variance, a measure of variability in the Pro's Can determine whether Enable research in the influences developers productivity?« same units as the scores being described. expected effects apply large. in studied context. Consider threats to the research Theoretical model is based on research question and represents set of concepts and Design research Other research methods Easy to plan. Help answer why and Applicable to real world projects in practice. Generalization usually Correlation and regression Determine associations between two variables. Correlation – The strength of the relationship between two variables. relationships between them! Select Practitioner oriented methods how questions. easier. Threats to the research are related to operationalization and ... Regression – Predicting the value of one variable from another based on the research Delphi method Can provide qualitative Good for early exploratory correlation. measurement issues: Action research Insights. method analysis. Operationalization issues – The validity of the operationalization Levels Independent Dependent Measures Laboratory oriented methods Measurement issues – Reliability, validity, sensitivity (see below) (observed variables) Mathematical modeling Application in With little or no May rely on different variables variables (observed variables) Computer simulation industrial context replication they may projects/organizations Inferential statistics (latent variables) (latent variables) Laboratory experiment requires give inaccurate results. keeping comparable data. Technology oriented methods Compromises. OSSD Proof of technical concept Difficult to interpret and No control over variables Inferential statistics or statistical induction comprises the use of statistics to make inferences Software Design Design case Design generalize (e.g., due to methods. Literature based methods concerning some unknown aspect of a population. Lines of code (LOC) experiment study survey confounding factors). Can at most confirm Con's RUP development Literature review H1 Developer per developer per day Statistical analysis association but not causality. methodology Conceptual study XP productivity usually not possible. Can be biased due to Sampling distribution – the distribution of means of samples from a population. Valid but Valid and Few agreed standards differences between Sampling distribution has three important properties: Reliable but on procedures for respondents and It has the same mean as the population distribution. not reliable reliable not valid H2 undertaking case nonrespondents. Development It has smaller standard deviation as the population distribution. Measurement relationship Perform research on defined The objective of this activity studies. Questionnaire design may be As the sample size becomes larger, the shape of the distribution approaches a normal Number of team size – associate latent variables Reliability threats – refers to the question whether the research can be repeated sample Perform is to run the study according tricky (validity, reliability). distribution, regardless of the shape of the population from which the samples are drawn. developers with their measures to the study plan. with the same results. research collection General sable population – Process and product Process and product Questionnaires. Stability reliability – Does the measurement vary over time? Hypothesis testing - is the use of statistics to determine the probability that a given hypothesis Data Causal relationships (H1,H2) – define population to which you want to measurement. measurement. Interviews. Representative reliability – Does the measurement give the same answer when cause-effect relationship between Questionnaires. is true. The usual process of hypothesis testing consists of four steps. ultimately generalize results. Questionnaires. Project measurement. applied to all groups? latent variables (theoretical Formulate the null hypothesis H0 (the hypothesis that is of no scientific interest) and the Equivalence reliability – When there are many measures of the same construct, do All other variables which Interviews Literature survey. alternative hypothesis Ha (statistical term for the research hypothesis). propositions). Can be tested only by Accessible population – they all give the same answer? are not the focus of Data is collected with a Identify a test statistic that can be used to assess the truth of the null hypothesis. evaluating relationships between population that you can actually research are irrelevant research instrument, for Parametric and Compare case study results Comparing different populations Compute the P-value, which is the probability that a test statistic at least as significant as observed variables (hypotheses)! gain access. Analysis variables. example questionnaire. nonparametric to a representative among respondents, association the one observed would be obtained assuming that the null hypothesis were true. The types Validity threats Collect data statistics, compare central comparison baseline: sister and trend analysis, consistency of Face validity – Research community »good feel«. Actual sample – the sample smaller the P-value, the stronger the evidence against the null hypothesis. Content validity – Are all aspect of the conceptual variable included in the Variables types actually used in research. tendencies of treatments, groups. project, company baseline, project subset with no scores. Compare the p-value to an acceptable significance value alpha (sometimes called an alpha measurement? value). If p<=alpha, that the observed effect is statistically significant, the null hypothesis is Hypothesis testing change. ruled out, and the alternative hypothesis is valid. Criterion validity – validity is measured against some other standard or measure for Dependent Developer Hypotheses are tested by comparing LOC the conceptual variable. efficiency Conclusion validity Internal validity Internal validity predictions with observed data Represent Statistical errors threats Predictive validity – The measure is known to predict future behavior that is related the »effect« Mean time Analyze data Major Observations that confirm a prediction do Internal validity Construct validity Experimental validity or H0 is TRUE H0 is FALSE to the conceptual variable. Software not establish the truth of a hypothesis between failure Construct validity External validity reliability Construct validity – A measure is found to give correct predictions in multiple reliability Wrong decision – Deductive testing of hypotheses look for Qualitative Construct validity Accept H0 Correct decision unrelated research processes. This confirm both the theories and the construct validity Quantitative External validity Experimental validity or The null Type II error disconfirming evidence to falsify Chose data Independent of the measure. Requirements {OSSD, RUP, data reliability External validity hypothesis hypotheses change XP} analysis data Wrong decision – Conclusion validity – is concerned with the relationship between the treatment and Represent (H0) Reject H0 Type I error Correct decision the outcome of the research 8choice of sample size, choice of statistical tests). the »cause« Experimental validity – (see reliability) Development Number of Measurement issues team size developers Depends on data Use Statistical significance – the probability that an experimental result happened by chance. Sources of invalidity Internal – Is concerned with the validity within the given environment and the Reliability - does the measurement give the same results under the same conditions (consistency)? Latent Observed Use qualitative data analysis and the goal of the study. quantitative data analysis Select statistical test Repeated Here is the Alpha is the probability reliability of results. It relates to validity of research process design, controls and ANOVA distribution of Validity - does the measurement method Describe abstract of rejecting the Logistic Regression measures. ANOVA values of Z when Discriminant of actually provide information about the theoretical concepts. Define ways of measuring Nom Int hypothesis tested when External – Is the question of how general the findings are. Can you carry over the LoM the hypothesis conceptual variable? They cannot be latent variables. Level of that hypothesis is true. Nominal Inte research results into actual environment? Level of for tested is true. Sensitivity - how much does the directly measured. Each latent variable may The objective of this measurement measurement change with the changes on have multiple empirical measurement for IV 3+ DV? Ord (mean Z = 0) rval activity is to analyze How Here we have Linear t Sensitivity In the conceptual variable? indicators. the collected data in of DV? many levels Friedman regression set alpha = 0.05 How much does the measurement change with the change of the conceptual variable? order to answer the does the IV operationalized study Draw Consider Consider reliability, have Paired goal (research validity and sensitivity! The IV is the ? Int -3 -2 -1 0 1 2 3 question). conclusions threats 3+ 2 (related) t- Values of Zx