SlideShare a Scribd company logo
1 of 60
Download to read offline
Daniel Méndez
TUM, Germany
www.mendezfe.org
Building and Evaluating Theories 

in Software Engineering
@mendezfe
Software Engineering Research Methods
Dagstuhl, April 2019
Simplified view on epistemological setting
Principle ways of working
Methods and strategies
Fundamental theories
Philosophy of science
Simplified view on epistemological setting
Principle ways of working
Methods and strategies
Fundamental theories
Philosophy of science Epistemology
Empiricism
Statistics
Hypothesis testing
Controlled Experimentation
Example!
Setting: Empirical Software Engineering
Principle ways of working
Methods and strategies
Fundamental theories
Philosophy of science
Typically in scope of
Empirical Software 

Engineering
Setting: Empirical Software Engineering
Principle ways of working
Methods and strategies
Fundamental theories
Philosophy of science
Theory building 

and evaluation
Methods and 

strategies
… supported by…
Analogy: Theoretical and 

Experimental Physics
Goals of this talk
Principle ways of working
Methods and strategies
Fundamental theories
Philosophy of science
Theory building 

and evaluation
Methods and 

strategies
… supported by…
Analogy: Theoretical and 

Experimental Physics
Basic understanding of
• Notion of theories in our field
• (Philosophy of science) context of
research methods and strategies
for theory building and evaluation
Ground rule
Whenever you have questions / remarks,
don’t ask , but
share them with the whole group.
Theory Building in Software Engineering
‣Theories in a Nutshell
… where we will briefly talk about the general notion of theories
‣State of Evidence in Software Engineering
… where we will see why theory building is so important to our field
‣Research Methods in Software Engineering
… where we will put research methods in a larger (philosophical) picture
‣Qualitative “vs” quantitative research
… where we will discuss different research approaches
Theory Building in Software Engineering
‣Theories in a Nutshell
… where we will briefly talk about the general notion of theories
‣State of Evidence in Software Engineering
… where we will see why theory building is so important to our field
‣Research Methods in Software Engineering
… where we will put research methods in a larger (philosophical) picture
‣Qualitative “vs” quantitative research
… where we will discuss different research approaches
What is a theory?
What do you think?
Theories (generally speaking)
Examples:
• Vaccinations lead to autism
• Global warming is invented to harm the industry
• Earth is flat
• …
A theory is a belief that there is a pattern in phenomena.
Further examples: Twitter profiles by right-wing populists
Theories (generally speaking)
Examples:
• Vaccinations lead to autism
• Global warming is invented to harm the industry
• Earth is flat
• …
A theory is a belief that there is a pattern in phenomena.
Are these theories scientific?
Further examples: Twitter profiles by right-wing populists
» Speculations based on imagination or opinions that often cannot be refuted 

(i.e. logical fallacies)
Scientific Theories
1. Tests
• Experiments, simulations, …
• Replications
2. Criticism
• Peer reviews / acceptance in the community
• Corroborations / extensions with further theories
A scientific theory is a belief that there is a pattern in phenomena while
having survived
1. tests against experiences
2. criticism by critical peers
Scientific Theories
1. Tests
• Experiments, simulations, …
• Replications
2. Criticism
• Peer reviews / acceptance in the community
• Corroborations / extensions with further theories
A scientific theory is a belief that there is a pattern in phenomena while
having survived
1. tests against experiences
2. criticism by critical peers
Note: Addresses so-called Demarcation
Problem to distinguishing science from non-
science (as per introduction by K. Popper)
“There is no universally agreed
upon definition of the concept
of an empirically-based theory
[in Software Engineering], nor is
there any uniform terminology
for describing theories.”
Source: Sjøberg, D., Dybå, T., Anda, B., Hannay, J. Building Theories in Software Engineering, 2010.
Approach by characteristics
Scientific Theories have…
Quality criteria:
• Testability
• Empirical support / (high) level of confidence
• Explanatory power
• Usefulness to researchers and / or practitioners
• …
Adopted from: Sjøberg, D., Dybå, T., Anda, B., Hannay, J. Building Theories in Software Engineering, 2010.
Analytical Explanatory Predictive Explanatory &
Predictive
Scope Descriptions and
conceptualisation, 

including
taxonomies,
classifications, and
ontologies

- What is?
Identification of
phenomena by
identifying causes,
mechanisms or
reasons

- Why is?
Prediction of what
will happen in the
future 

- What will happen?
Prediction of what will
happen in the future
and explanation

- What will happen
and why?

Further information: https://goo.gl/SQQwxt
A purpose:
Scientific Theories have…
Quality criteria:
• Testability
• Empirical support / (high) level of confidence
• Explanatory power
• Usefulness to researchers and / or practitioners
• …
Adopted from: Sjøberg, D., Dybå, T., Anda, B., Hannay, J. Building Theories in Software Engineering, 2010.
Analytical Explanatory Predictive Explanatory &
Predictive
Scope Descriptions and
conceptualisation, 

including
taxonomies,
classifications, and
ontologies

- What is?
Identification of
phenomena by
identifying causes,
mechanisms or
reasons

- Why is?
Prediction of what
will happen in the
future 

- What will happen?
Prediction of what will
happen in the future
and explanation

- What will happen
and why?

Further information: https://goo.gl/SQQwxt
A purpose:
Note: Many theories in software engineering are
so-called “design [science] theories”, i.e. scientific
theories about artefacts in a context.
[Artefact specification] X [Context assumptions] → [Effects]
Scientific Theories have…
Quality criteria:
• Testability
• Empirical support / (high) level of confidence
• Explanatory power
• Usefulness to researchers and / or practitioners
• …
Adopted from: Sjøberg, D., Dybå, T., Anda, B., Hannay, J. Building Theories in Software Engineering, 2010.
Analytical Explanatory Predictive Explanatory &
Predictive
Scope Descriptions and
conceptualisation, 

including
taxonomies,
classifications, and
ontologies

- What is?
Identification of
phenomena by
identifying causes,
mechanisms or
reasons

- Why is?
Prediction of what
will happen in the
future 

- What will happen?
Prediction of what will
happen in the future
and explanation

- What will happen
and why?

Note: Laws “versus” theories
A law is a purely descriptive theory about
phenomena (without explanations), i.e. an
analytical theory.
Further information: https://goo.gl/SQQwxt
A purpose:
Note: Many theories in software engineering are
so-called “design [science] theories”, i.e. scientific
theories about artefacts in a context.
[Artefact specification] X [Context assumptions] → [Effects]
Exemplary framework for describing theories
in Software Engineering
•Constructs: What are the basic elements?

(Actors, technologies, activities, system entities,
context factors)
•Propositions: How do the constructs interact?
•Explanations: Why are the propositions as specified?
•Scope: What is the universe of discourse in which the
theory is applicable?
Source: Sjøberg, D., Dybå, T., Anda, B., Hannay, J. Building Theories in Software Engineering, 2010.
Exemplary framework for describing theories
in Software Engineering
•Constructs: What are the basic elements?

(Actors, technologies, activities, system entities,
context factors)
•Propositions: How do the constructs interact?
•Explanations: Why are the propositions as specified?
•Scope: What is the universe of discourse in which the
theory is applicable?
Source: Sjøberg, D., Dybå, T., Anda, B., Hannay, J. Building Theories in Software Engineering, 2010.
Example
Source: Wagner, Mendez Fernandez et al. Status Quo in Requirements Engineering: A Theory and a Global Family of Surveys, TOSEM 2018.
Req Elicitation Technique
Interview
Scenario
Prototyping
Facilitated Meetings
Observation
Req Documentation Technique
Structured req list
Domain/business process model
Use case model
Goal model
Data model
Non‐functional req
Textual
Semi‐formal
Formal
Technology
Req Test Alignment Approach
Req review by tester
Coverage by tests
Acceptance criteria
Test derivation from models
Req Change Approach
Product backlog update
Change requests
Trace management 
Impact analysis
Activity
Req Elicitation
Req Documentation
Req Change Management
Req Test Alignment
P 1‐5
P 6‐13
P 14‐20
P 21‐24
Actor
Req Engineer
Test Engineer
Req Standard Application
Practice
Control
Tailoring
Req Eng Process Standard
P 25‐28
Req Standard Defintion
Compliance
Development
Tool support
Quality assurance
Project management
Knowledge transfer
Process complexity
Communication demand
Willigness to change
Possibility of standardisation
 
Req Improvement Means
Continuous improvement
Strengths/weaknesses
Own business unit/role
Req Eng Improvement 
P 29-44
P 45-49
Proposition:
“Structured requirements lists are documented
textually in free form or textually with constraints.”
Explanation:
“Free-form and constraint textual requirements
are sufficient for many contexts such as in agile
projects where they only act as reminders for
further conversations.”
Theories and hypotheses
Empirical Approaches
Theory / Theories
(Tentative) Hypothesis
Falsification / 

Corroboration
Theory (Pattern)

Building
Hypothesis

Building
Hypothesis
• “[…] a statement that proposes a possible
explanation to some phenomenon or
event” (L. Given, 2008)
• Grounded in theory, testable and
falsifiable
• Often quantified and written as a
conditional statement
Scientific theory
• “[…] based on hypotheses tested and
verified multiple times by detached
researchers” (J. Bortz and N. Döring,
2003)
If cause/assumption (independent variables)
then (=>) consequence (dependent variables)
Note: We don’t “test theories”, but
their consequences via hypotheses 

(i.e. testable propositions)
From real world phenomena to theories 

(and back) - The Empirical Lifecycle
Empirical Approaches
Theory / Theories
(Tentative) Hypothesis
Falsification / 

Corroboration
Theory (Pattern)

Building
Units of Analysis
Sampling Frame
Sampling
Hypothesis

Building
Empirical Inquiries
Induction
Inference of a 

general rule 

from a particular
case/result 

(observation)
Abduction
(Creative) Synthesis of an 

explanatory case from a general rule 

and a particular result (observation)
Deduction
Application of a general rule 

to a particular case, 

inferring a specific result
Source: Mendez and Passoth. Empirical Software 

Engineering: from Discipline to Interdiscipline, 2018.
Theory Building in Software Engineering
‣Theories in a Nutshell
… where we will briefly talk about the general notion of theories
‣State of Evidence in Software Engineering
… where we will see why theory building is so important to our field
‣Research Methods in Software Engineering
… where we will put research methods in a larger (philosophical) picture
‣Qualitative “vs” quantitative research
… where we will discuss different research approaches
Example: Goal-oriented RE (“GORE”)
Papers published [1]: 966
[2] Mavin, et al. Does Goal-Oriented Requirements Engineering Achieve its Goal?, 2017
[3] Mendez et al. Naming the Pain in Requirements Engineering Initiative - www.re-survey.org
[1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016
Papers including a case study [1]: 131
Studies involving practitioners [2]: 20
Practitioners actually using GORE [3]: ~ 5%
Example: Goal-oriented RE (“GORE”)
Papers published [1]: 966
[2] Mavin, et al. Does Goal-Oriented Requirements Engineering Achieve its Goal?, 2017
[3] Mendez et al. Naming the Pain in Requirements Engineering Initiative - www.re-survey.org
[1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016
Papers including a case study [1]: 131
Studies involving practitioners [2]: 20
Practitioners actually using GORE [3]: ~ 5%
For comparison:
Icelanders believing in elves [4]:
[4] https://www.nationalgeographic.com/travel/destinations/europe/iceland/believes-elves-exist-mythology/
54%
“[…] judging a theory by assessing the number,
faith, and vocal energy of its supporters […]
basic political credo of contemporary religious
maniacs”
— Imre Lakatos, 1970
Current state of evidence in SE is weak
* Addressing the situation in the quantum mechanics research community, an analogy
Current state of evidence in SE is weak
Source (levels of evidence): Wohlin. An Evidence Profile for Software Engineering Research and Practice, 2013.
In favour /
corroboration
Against /
refutation
Strong evidence
Evidence
Circumstantial
evidence
Third-party claim
First or second party claim
Strong evidence
Circumstantial
evidence
Third-party claim
Evidence
First or second party claim
+
-
In most cases, 

we are here
Available studies often…
• … remain isolated
• … don’t report negative results
• … strengthen confidence on 

own hopes (and don’t 

report anything around)
• … discuss little (if at all) 

relation to existing evidence
SE largely dominated by conventional wisdom
“Leprechauns”: Folklore turned into facts
• Emerge from times where claims by
authorities were treated as “facts”
• Reasons manifold:
– Lack of empirical awareness
– Neglecting particularities 

of practical contexts
– Neglecting relation to existing
evidence
– No proper citations (one side the
medal, over-conclusions, etc.)
– Lack of data
– …
Consequences
» Practical relevance and impact?
» Potential for transfer into practice and adoption?
The essence
Theory building and theory evaluation 

are crucial in SE research
» Reason about the discipline and (e.g. social)
phenomena involved
» Recognise and understand limits and effects of
artefacts (technologies, techniques, processes,
models, etc.) in their contexts
Theory Building in Software Engineering
‣Theories in a Nutshell
… where we will briefly talk about the general notion of theories
‣State of Evidence in Software Engineering
… where we will see why theory building is so important to our field
‣Research Methods in Software Engineering
… where we will put research methods in a larger (philosophical)
picture
‣Qualitative “vs” quantitative research
… where we will discuss different research approaches
Recap: The Empirical Lifecycle
Source: Mendez and Passoth. Empirical Software 

Engineering: from Discipline to Interdiscipline, 2018.
(Empirical) methods
•Each method…
– …has a specific purpose
– …relies on a specific data type
•Purposes
– Exploratory
– Descriptive
– Explanatory
– Improving
•Data Types
– Qualitative
– Quantitative
(Empirical) methods
•Each method…
– …has a specific purpose
– …relies on a specific data type
•Purposes
– Exploratory
– Descriptive
– Explanatory
– Improving
•Data Types
– Qualitative
– Quantitative
“Grounded Theory”
Qualitative Data
Descriptive,
Exploratory, or
Explanatory
Example!
(Empirical) methods - where do they belong?
Ethnographic studies,
Folklore gathering
Case studies /

Field studies
(Confirmatory)
Formal analysis /
logical reasoning
Survey research
Case studies /

Field studies
(Exploratory)
(Empirical) methods - where do they belong?
Ethnographic studies,
Folklore gathering
Case studies /

Field studies
(Confirmatory)
Formal analysis /
logical reasoning
Survey research
Case studies /

Field studies
(Exploratory)
Which research methods to use 

in which situations?
Empirical process: an abstract view
Planning and Definition
Method and Strategy
Selection
Design and (Method)
Execution
Conclusion Drawing
Packaging and
Reporting
•Identify and outline problem (area)
•Determine research objectives and questions
•Select type of study and method(s)
•Identify necessary environment 

(including units of analysis)
•Design and validate study protocol (and validity
procedures)
•Collect & analyse data following detailed processes
•Interpret results
•Reflect upon potential threats to validity
•Package and (ideally) disclose data
•Report on results in tune with audience
Empirical process: an abstract view
Planning and Definition
Method and Strategy
Selection
Design and (Method)
Execution
Conclusion Drawing
Packaging and
Reporting
Scope of detailed empirical methods
The selection of proper research 

methods is a recognised problem 

in the community
There is no such thing as a 

universal way of scientific practice!
Method selection depends on many 

non-trivial questions
• What is the purpose of the study?
– Exploratory? Descriptive? Explanatory? Improving?
• What is the nature of the study?
– Inductive? Deductive?
• What is the relation to existing evidence?
– Building a new theory? “Testing” existing theory?
• What is the nature of the questions we ask?
– What-questions? Why-questions?
• What is the nature of the environment?
– Controlled environments? Realistic environments?
• What is the necessary sample?
– Population source?
– Units of analysis?
Criteria for 

selecting methods
Criteria for 

environment selection

(and sampling)
Non-exclusive andnon-sequential
Not trivial, but possible: By using checklists
More advanced
• Chapter 16 + Appendix
• http://bit.ly/checklists-design_science
Good starting point
How to achieve scientific progress?
In step-wise iterations, with multiple methods 

(aka “research programme”)
Progress via multi-study approaches
Large-scale evaluation
5
e.g. Field Study or longitudinal study
1
2
3 4 5
Validation of new technology
in artificial setting
3
e.g. Controlled Experiment
Evaluation of new technology
in realistic setting
4
e.g. Case Study
Replication
Replication
Proposal new / adaptation
existing technology
2
Problem analysis
1
e.g. Systematic Mapping Study

or Survey
e.g. RE Improvement 

Approach
Postulate I
Every study has a specific scope of validity only
Source: Sjøberg, D., Dybå,T.,Anda, B., Hannay, J. Building Theories in Software Engineering, 2010.
Postulate II
Scope of validity ≠ Degree of reality
Simulation
Field Study 

Research
Case Study 

Research
Survey
Research
Artificial environment
Realistic environment
Scope of validity*Controlled (lab)
Experiment
Replications
Replications
* Extremely simplified view to orient discussions
Postulate III
Case studies and experiments complement
each other in scaling up to practice
Based on: Wieringa R. Empirical Research Methods for TechnologyValidation: Scaling Up to Practice, 2013.
Lab credibility
Street credibility
Simple model
Realistic case
Small sample Large sample
Focus of 

case
studies
Focus of field studies 

and replications
Scaling up to practice
Focus of 

(lab) experiments
Similarity to 

population units
Sample size
Theory Building in Software Engineering
‣Theories in a Nutshell
… where we will briefly talk about the general notion of theories
‣State of Evidence in Software Engineering
… where we will see why theory building is so important to our field
‣Research Methods in Software Engineering
… where we will put research methods in a larger (philosophical) picture
‣Qualitative “vs” quantitative research
… where we will discuss different research approaches
What is the difference between qualitative
and quantitative research?
What do you think?
Warning: EmSE emerges from natural science,
thus, qualitative methods are often confronted
with prejudice “I prefer working with real data 

[not with qualitative data]”
— Anonymous ISERN member
“In contrast [to previous qualitative studies], this study attempts
to obtain more scientific evidence in the form of objective,
quantitative data.”
— Anonymous author
“With all due respect, DO NOT make such ridiculous claims that only quantitative
studies are "scientific" and “objective" There is NO more objectivity in numbers
than there is in qualitative data.
”
— Anonymous reviewer
… response
The essence
Qualitative studies focus primarily on the kind of evidence
that will enable you to understand the meaning [and purpose,
reasoning, etc] of what is going on.
Quantitative studies focus primarily on the kind of evidence
that will enable you to understand what is going on.
Qualitative and quantitative methods have complementary
purposes, strengths, and limitations in theory building.
Two complementary approaches (+1)
•Quantitative research
– Describing events and finding causes to predict similar events in the future
– (Typically) focus on what, how much, or how many
– (Typically) in numerical forms
– (Typically) descriptive purpose
•Qualitative research
– Understanding meaning [and purpose, reasoning, etc.] of a phenomenon for
those involved
– (Typically) focus on why/meaning, and how people interpret their experiences
– (Typically) in variety of non-numerical forms, like texts, diagrams, etc.
– (Typically) exploratory or explanatory purpose
•Mix-method research
Adopted from: Da Silva. Tutorial given at the Ibero-American Conference on Software Engineering, 2018 (Bogota, Colombia)
Two complementary approaches (+1)
Quantitative data Qualitative data
“Why?”-Questions“What?”-Questions
Descriptive & predictive purposes Explanatory & exploratory purposes
Two complementary approaches (+1)
Quantitative data Qualitative data
Case study research
Survey research
(Quasi-) Controlled 

experiments
Action 

research
Ethnographic
studies
“Why?”-Questions“What?”-Questions
Descriptive & predictive purposes Explanatory & exploratory purposes
Comparison: qualitative “vs” quantitative
Qualitative Research Quantitative Research
Goals • Understanding, reasoning,
explanations, descriptions, meaning
(to subjects), discovery
• Hypothesis generation (typically)
• Description, control, prediction
• Hypothesis testing (typically)
Design
characteristics
• Flexible, evolving, emergent
• Inductive, constant comparative
• Predetermined, structured / fixed
• Deductive, statistical
Samples • Small, non-random (sometimes even
opportunistic), purposeful,
theoretical
• Large(r), random, representative
Data collection • Researcher often primary instrument
• Interviews, observations, document
analysis, …
• Inanimate instruments (tests,
surveys, questionnaires, etc.)
Findings • Comprehensive, holistic, rich
descriptions
• Precise and statistical
Adopted from: Da Silva. Tutorial given at the Ibero-American Conference on Software Engineering, 2018 (Bogota, Colombia)
* For all, you can add a “in tendency”
Researchers’ necessary skills
•Being a careful observer with
high tolerance to ambiguity and
existing belief systems
•Ability to ask “proper”, context-
specific questions without
actively distorting results
•Thinking inductively
•Patience (and charm)
•Comfort with observing 

(and writing :-)
Adopted from: Da Silva. Tutorial given at the Ibero-American Conference on Software Engineering, 2018 (Bogota, Colombia)
Challenges in conducting qualitative studies
• Cases/subjects need to offer rich insights
(challenging in industry contexts) —>
Case selection often opportunistic
• Data collection demands skills beyond
statistics (e.g. social skills)
• Data analysis is often a (pragmatic and)
creative task.
• Data disclosure particularly important
(and challenging for ethical and legal
reasons)
• Communication and publication often
rendered difficult by prejudice
Source: Mendez, Wagner. Case Studies in Industry: What we have Learnt, 2016
Further reading: Selected Papers
Good starting point Terminological demarcation
and key characteristics 

of methods
Epistemological setting
Theory Building in Software Engineering
‣Theories in a Nutshell
… where we will briefly talk about the general notion of theories
‣State of Evidence in Software Engineering
… where we will see why theory building is so important to our field
‣Research Methods in Software Engineering
… where we will put research methods in a larger (philosophical) picture
‣Qualitative “vs” quantitative research
… where we will discuss different research approaches
Theory Building in Software Engineering
‣ Theories are the backbone of our
discipline allowing us to move forward
from paradigmatic stage of an
engineering discipline to a scientific one.
‣ Every research method has its place in a
larger picture.
Thank you!
‣ Qualitative and quantitative research has
complementary purposes, strengths, and
limitations in building and evaluating
theories.

More Related Content

What's hot

Empirical Software Engineering
Empirical Software EngineeringEmpirical Software Engineering
Empirical Software EngineeringRahimLotfi
 
Theory Building in RE - The NaPiRE Initiative
Theory Building in RE - The NaPiRE InitiativeTheory Building in RE - The NaPiRE Initiative
Theory Building in RE - The NaPiRE InitiativeDaniel Mendez
 
Surveys in Software Engineering
Surveys in Software EngineeringSurveys in Software Engineering
Surveys in Software EngineeringDaniel Mendez
 
RESEARCH in software engineering
RESEARCH in software engineeringRESEARCH in software engineering
RESEARCH in software engineeringIvano Malavolta
 
Qualitative Studies in Software Engineering - Interviews, Observation, Ground...
Qualitative Studies in Software Engineering - Interviews, Observation, Ground...Qualitative Studies in Software Engineering - Interviews, Observation, Ground...
Qualitative Studies in Software Engineering - Interviews, Observation, Ground...alessio_ferrari
 
Systematic Literature Reviews and Systematic Mapping Studies
Systematic Literature Reviews and Systematic Mapping StudiesSystematic Literature Reviews and Systematic Mapping Studies
Systematic Literature Reviews and Systematic Mapping Studiesalessio_ferrari
 
Empirical Software Engineering for Software Environments - University of Cali...
Empirical Software Engineering for Software Environments - University of Cali...Empirical Software Engineering for Software Environments - University of Cali...
Empirical Software Engineering for Software Environments - University of Cali...Marco Aurelio Gerosa
 
empirical software engineering, v2.0
empirical software engineering, v2.0empirical software engineering, v2.0
empirical software engineering, v2.0CS, NcState
 
Improving Requirements Engineering by Artefact Orientation
Improving Requirements Engineering by Artefact OrientationImproving Requirements Engineering by Artefact Orientation
Improving Requirements Engineering by Artefact OrientationDaniel Mendez
 
Empirical Methods in Software Engineering - an Overview
Empirical Methods in Software Engineering - an OverviewEmpirical Methods in Software Engineering - an Overview
Empirical Methods in Software Engineering - an Overviewalessio_ferrari
 
An Exploratory Study on Technology Transfer in Software Engineering
An Exploratory Study on Technology Transfer in Software EngineeringAn Exploratory Study on Technology Transfer in Software Engineering
An Exploratory Study on Technology Transfer in Software EngineeringDaniel Mendez
 
Survey Research In Empirical Software Engineering
Survey Research In Empirical Software EngineeringSurvey Research In Empirical Software Engineering
Survey Research In Empirical Software Engineeringalessio_ferrari
 
Case Study Research in Software Engineering
Case Study Research in Software EngineeringCase Study Research in Software Engineering
Case Study Research in Software Engineeringalessio_ferrari
 
Open Assessment In Metal Risks
Open Assessment In Metal RisksOpen Assessment In Metal Risks
Open Assessment In Metal Risksguest4d312e0
 
Master Beginners Workshop - September 2019
Master Beginners Workshop - September 2019Master Beginners Workshop - September 2019
Master Beginners Workshop - September 2019Miguel Pardal
 
Why and How to Get a PhD? (In software engineering)
Why and How to Get a PhD? (In software engineering)Why and How to Get a PhD? (In software engineering)
Why and How to Get a PhD? (In software engineering)Lionel Briand
 
Why and How to get a PhD (in Software Engineering)
Why and How to get a PhD (in Software Engineering)Why and How to get a PhD (in Software Engineering)
Why and How to get a PhD (in Software Engineering)Lionel Briand
 
How To Research
How To ResearchHow To Research
How To ResearchFengyi
 
Proposal Templet 10page
Proposal Templet 10pageProposal Templet 10page
Proposal Templet 10pageCho Li Hao
 
Not Losing Sight of the Essential: Enjoying your Career in Research
Not Losing Sight of the Essential: Enjoying your Career in ResearchNot Losing Sight of the Essential: Enjoying your Career in Research
Not Losing Sight of the Essential: Enjoying your Career in ResearchLionel Briand
 

What's hot (20)

Empirical Software Engineering
Empirical Software EngineeringEmpirical Software Engineering
Empirical Software Engineering
 
Theory Building in RE - The NaPiRE Initiative
Theory Building in RE - The NaPiRE InitiativeTheory Building in RE - The NaPiRE Initiative
Theory Building in RE - The NaPiRE Initiative
 
Surveys in Software Engineering
Surveys in Software EngineeringSurveys in Software Engineering
Surveys in Software Engineering
 
RESEARCH in software engineering
RESEARCH in software engineeringRESEARCH in software engineering
RESEARCH in software engineering
 
Qualitative Studies in Software Engineering - Interviews, Observation, Ground...
Qualitative Studies in Software Engineering - Interviews, Observation, Ground...Qualitative Studies in Software Engineering - Interviews, Observation, Ground...
Qualitative Studies in Software Engineering - Interviews, Observation, Ground...
 
Systematic Literature Reviews and Systematic Mapping Studies
Systematic Literature Reviews and Systematic Mapping StudiesSystematic Literature Reviews and Systematic Mapping Studies
Systematic Literature Reviews and Systematic Mapping Studies
 
Empirical Software Engineering for Software Environments - University of Cali...
Empirical Software Engineering for Software Environments - University of Cali...Empirical Software Engineering for Software Environments - University of Cali...
Empirical Software Engineering for Software Environments - University of Cali...
 
empirical software engineering, v2.0
empirical software engineering, v2.0empirical software engineering, v2.0
empirical software engineering, v2.0
 
Improving Requirements Engineering by Artefact Orientation
Improving Requirements Engineering by Artefact OrientationImproving Requirements Engineering by Artefact Orientation
Improving Requirements Engineering by Artefact Orientation
 
Empirical Methods in Software Engineering - an Overview
Empirical Methods in Software Engineering - an OverviewEmpirical Methods in Software Engineering - an Overview
Empirical Methods in Software Engineering - an Overview
 
An Exploratory Study on Technology Transfer in Software Engineering
An Exploratory Study on Technology Transfer in Software EngineeringAn Exploratory Study on Technology Transfer in Software Engineering
An Exploratory Study on Technology Transfer in Software Engineering
 
Survey Research In Empirical Software Engineering
Survey Research In Empirical Software EngineeringSurvey Research In Empirical Software Engineering
Survey Research In Empirical Software Engineering
 
Case Study Research in Software Engineering
Case Study Research in Software EngineeringCase Study Research in Software Engineering
Case Study Research in Software Engineering
 
Open Assessment In Metal Risks
Open Assessment In Metal RisksOpen Assessment In Metal Risks
Open Assessment In Metal Risks
 
Master Beginners Workshop - September 2019
Master Beginners Workshop - September 2019Master Beginners Workshop - September 2019
Master Beginners Workshop - September 2019
 
Why and How to Get a PhD? (In software engineering)
Why and How to Get a PhD? (In software engineering)Why and How to Get a PhD? (In software engineering)
Why and How to Get a PhD? (In software engineering)
 
Why and How to get a PhD (in Software Engineering)
Why and How to get a PhD (in Software Engineering)Why and How to get a PhD (in Software Engineering)
Why and How to get a PhD (in Software Engineering)
 
How To Research
How To ResearchHow To Research
How To Research
 
Proposal Templet 10page
Proposal Templet 10pageProposal Templet 10page
Proposal Templet 10page
 
Not Losing Sight of the Essential: Enjoying your Career in Research
Not Losing Sight of the Essential: Enjoying your Career in ResearchNot Losing Sight of the Essential: Enjoying your Career in Research
Not Losing Sight of the Essential: Enjoying your Career in Research
 

Similar to Building and Evaluating Theories 
 in Software Engineering

An Introduction into Philosophy of Science for Software Engineers
An Introduction into Philosophy of Science for Software Engineers An Introduction into Philosophy of Science for Software Engineers
An Introduction into Philosophy of Science for Software Engineers Daniel Mendez
 
Ch 1 research introduciton
Ch 1 research introducitonCh 1 research introduciton
Ch 1 research introducitonTemtim assefa
 
chapter-3.pptx
chapter-3.pptxchapter-3.pptx
chapter-3.pptxAsmaRauf5
 
How to Read Academic Papers
How to Read Academic PapersHow to Read Academic Papers
How to Read Academic PapersJia-Bin Huang
 
Research methodoligies in architecture
Research methodoligies in architectureResearch methodoligies in architecture
Research methodoligies in architectureSamanth kumar
 
Scientific software engineering methods and their validity
Scientific software engineering methods and their validityScientific software engineering methods and their validity
Scientific software engineering methods and their validityDaniel Mendez
 
Chapter 5 theory and methodology
Chapter 5 theory and methodology Chapter 5 theory and methodology
Chapter 5 theory and methodology grainne
 
Introductionresearchmethodology 1
Introductionresearchmethodology 1Introductionresearchmethodology 1
Introductionresearchmethodology 1JO GALLEGOS
 
Grounded Theory: an Introduction (updated Jan 2011)
Grounded Theory: an Introduction (updated Jan 2011)Grounded Theory: an Introduction (updated Jan 2011)
Grounded Theory: an Introduction (updated Jan 2011)Hora Tjitra
 
Lecture 3 Computer Science Research SEM1 22_23 (1).pptx
Lecture 3 Computer Science Research SEM1 22_23 (1).pptxLecture 3 Computer Science Research SEM1 22_23 (1).pptx
Lecture 3 Computer Science Research SEM1 22_23 (1).pptxNabilaHassan13
 
Research methodology (Philosophies and paradigms) in Arabic
Research methodology (Philosophies and paradigms) in ArabicResearch methodology (Philosophies and paradigms) in Arabic
Research methodology (Philosophies and paradigms) in ArabicAmgad Badewi
 
The dissertation proposal2
The dissertation proposal2The dissertation proposal2
The dissertation proposal2wxg2525
 
IT3010 Lecture-7 Research Paradigms
IT3010 Lecture-7 Research ParadigmsIT3010 Lecture-7 Research Paradigms
IT3010 Lecture-7 Research ParadigmsBabakFarshchian
 
Research Methodology UNIT 2.pptx
Research Methodology UNIT 2.pptxResearch Methodology UNIT 2.pptx
Research Methodology UNIT 2.pptxPallawiBulakh1
 

Similar to Building and Evaluating Theories 
 in Software Engineering (20)

An Introduction into Philosophy of Science for Software Engineers
An Introduction into Philosophy of Science for Software Engineers An Introduction into Philosophy of Science for Software Engineers
An Introduction into Philosophy of Science for Software Engineers
 
Ch 1 research introduciton
Ch 1 research introducitonCh 1 research introduciton
Ch 1 research introduciton
 
chapter-3.pptx
chapter-3.pptxchapter-3.pptx
chapter-3.pptx
 
How to Read Academic Papers
How to Read Academic PapersHow to Read Academic Papers
How to Read Academic Papers
 
Tesol proposoal writingworkshop
Tesol proposoal writingworkshopTesol proposoal writingworkshop
Tesol proposoal writingworkshop
 
Research methodoligies in architecture
Research methodoligies in architectureResearch methodoligies in architecture
Research methodoligies in architecture
 
Scientific software engineering methods and their validity
Scientific software engineering methods and their validityScientific software engineering methods and their validity
Scientific software engineering methods and their validity
 
Chapter 5 theory and methodology
Chapter 5 theory and methodology Chapter 5 theory and methodology
Chapter 5 theory and methodology
 
Introductionresearchmethodology 1
Introductionresearchmethodology 1Introductionresearchmethodology 1
Introductionresearchmethodology 1
 
Grounded Theory: an Introduction (updated Jan 2011)
Grounded Theory: an Introduction (updated Jan 2011)Grounded Theory: an Introduction (updated Jan 2011)
Grounded Theory: an Introduction (updated Jan 2011)
 
Defense questions, Prof. Venter
Defense questions, Prof. VenterDefense questions, Prof. Venter
Defense questions, Prof. Venter
 
Lecture 3 Computer Science Research SEM1 22_23 (1).pptx
Lecture 3 Computer Science Research SEM1 22_23 (1).pptxLecture 3 Computer Science Research SEM1 22_23 (1).pptx
Lecture 3 Computer Science Research SEM1 22_23 (1).pptx
 
Research in home science
Research in home scienceResearch in home science
Research in home science
 
Lecture-2.pdf
Lecture-2.pdfLecture-2.pdf
Lecture-2.pdf
 
Research methodology (Philosophies and paradigms) in Arabic
Research methodology (Philosophies and paradigms) in ArabicResearch methodology (Philosophies and paradigms) in Arabic
Research methodology (Philosophies and paradigms) in Arabic
 
The dissertation proposal2
The dissertation proposal2The dissertation proposal2
The dissertation proposal2
 
IT3010 Lecture-7 Research Paradigms
IT3010 Lecture-7 Research ParadigmsIT3010 Lecture-7 Research Paradigms
IT3010 Lecture-7 Research Paradigms
 
Study design: from questions to projects
Study design: from questions to projectsStudy design: from questions to projects
Study design: from questions to projects
 
classJan11.ppt
classJan11.pptclassJan11.ppt
classJan11.ppt
 
Research Methodology UNIT 2.pptx
Research Methodology UNIT 2.pptxResearch Methodology UNIT 2.pptx
Research Methodology UNIT 2.pptx
 

More from Daniel Mendez

Where do we stand in Requirements Engineering Improvement Today? First Result...
Where do we stand in Requirements Engineering Improvement Today? First Result...Where do we stand in Requirements Engineering Improvement Today? First Result...
Where do we stand in Requirements Engineering Improvement Today? First Result...Daniel Mendez
 
Naming the Pain in Requirements Engineering - Design of a Global Family of Su...
Naming the Pain in Requirements Engineering - Design of a Global Family of Su...Naming the Pain in Requirements Engineering - Design of a Global Family of Su...
Naming the Pain in Requirements Engineering - Design of a Global Family of Su...Daniel Mendez
 
Who cares about Software Process Modelling? A First Investigation about the P...
Who cares about Software Process Modelling? A First Investigation about the P...Who cares about Software Process Modelling? A First Investigation about the P...
Who cares about Software Process Modelling? A First Investigation about the P...Daniel Mendez
 
On the Distinction of Functional and Quality Requirements in Practice
On the Distinction of Functional and Quality Requirements in PracticeOn the Distinction of Functional and Quality Requirements in Practice
On the Distinction of Functional and Quality Requirements in PracticeDaniel Mendez
 
Software Engineering Excellence - The key to mastering the Digital Transforma...
Software Engineering Excellence - The key to mastering the Digital Transforma...Software Engineering Excellence - The key to mastering the Digital Transforma...
Software Engineering Excellence - The key to mastering the Digital Transforma...Daniel Mendez
 
Case Studies in Industry - What We Have Learnt
Case Studies in Industry - What We Have LearntCase Studies in Industry - What We Have Learnt
Case Studies in Industry - What We Have LearntDaniel Mendez
 
A Case Study on Artefact-based RE Improvement in Practice
A Case Study on Artefact-based RE Improvement in PracticeA Case Study on Artefact-based RE Improvement in Practice
A Case Study on Artefact-based RE Improvement in PracticeDaniel Mendez
 
Artefact-based Requirements Engineering Improvement - Learning to Walk in Pra...
Artefact-based Requirements Engineering Improvement - Learning to Walk in Pra...Artefact-based Requirements Engineering Improvement - Learning to Walk in Pra...
Artefact-based Requirements Engineering Improvement - Learning to Walk in Pra...Daniel Mendez
 
Case studies in industry - fundamentals and lessons learnt
Case studies in industry - fundamentals and lessons learntCase studies in industry - fundamentals and lessons learnt
Case studies in industry - fundamentals and lessons learntDaniel Mendez
 
In Quest for Requirements Engineering Oracles: Dependent Variables and Measur...
In Quest for Requirements Engineering Oracles: Dependent Variables and Measur...In Quest for Requirements Engineering Oracles: Dependent Variables and Measur...
In Quest for Requirements Engineering Oracles: Dependent Variables and Measur...Daniel Mendez
 

More from Daniel Mendez (10)

Where do we stand in Requirements Engineering Improvement Today? First Result...
Where do we stand in Requirements Engineering Improvement Today? First Result...Where do we stand in Requirements Engineering Improvement Today? First Result...
Where do we stand in Requirements Engineering Improvement Today? First Result...
 
Naming the Pain in Requirements Engineering - Design of a Global Family of Su...
Naming the Pain in Requirements Engineering - Design of a Global Family of Su...Naming the Pain in Requirements Engineering - Design of a Global Family of Su...
Naming the Pain in Requirements Engineering - Design of a Global Family of Su...
 
Who cares about Software Process Modelling? A First Investigation about the P...
Who cares about Software Process Modelling? A First Investigation about the P...Who cares about Software Process Modelling? A First Investigation about the P...
Who cares about Software Process Modelling? A First Investigation about the P...
 
On the Distinction of Functional and Quality Requirements in Practice
On the Distinction of Functional and Quality Requirements in PracticeOn the Distinction of Functional and Quality Requirements in Practice
On the Distinction of Functional and Quality Requirements in Practice
 
Software Engineering Excellence - The key to mastering the Digital Transforma...
Software Engineering Excellence - The key to mastering the Digital Transforma...Software Engineering Excellence - The key to mastering the Digital Transforma...
Software Engineering Excellence - The key to mastering the Digital Transforma...
 
Case Studies in Industry - What We Have Learnt
Case Studies in Industry - What We Have LearntCase Studies in Industry - What We Have Learnt
Case Studies in Industry - What We Have Learnt
 
A Case Study on Artefact-based RE Improvement in Practice
A Case Study on Artefact-based RE Improvement in PracticeA Case Study on Artefact-based RE Improvement in Practice
A Case Study on Artefact-based RE Improvement in Practice
 
Artefact-based Requirements Engineering Improvement - Learning to Walk in Pra...
Artefact-based Requirements Engineering Improvement - Learning to Walk in Pra...Artefact-based Requirements Engineering Improvement - Learning to Walk in Pra...
Artefact-based Requirements Engineering Improvement - Learning to Walk in Pra...
 
Case studies in industry - fundamentals and lessons learnt
Case studies in industry - fundamentals and lessons learntCase studies in industry - fundamentals and lessons learnt
Case studies in industry - fundamentals and lessons learnt
 
In Quest for Requirements Engineering Oracles: Dependent Variables and Measur...
In Quest for Requirements Engineering Oracles: Dependent Variables and Measur...In Quest for Requirements Engineering Oracles: Dependent Variables and Measur...
In Quest for Requirements Engineering Oracles: Dependent Variables and Measur...
 

Recently uploaded

Extensive Pollution of Uranus and Neptune’s Atmospheres by Upsweep of Icy Mat...
Extensive Pollution of Uranus and Neptune’s Atmospheres by Upsweep of Icy Mat...Extensive Pollution of Uranus and Neptune’s Atmospheres by Upsweep of Icy Mat...
Extensive Pollution of Uranus and Neptune’s Atmospheres by Upsweep of Icy Mat...Sérgio Sacani
 
Jet reorientation in central galaxies of clusters and groups: insights from V...
Jet reorientation in central galaxies of clusters and groups: insights from V...Jet reorientation in central galaxies of clusters and groups: insights from V...
Jet reorientation in central galaxies of clusters and groups: insights from V...Sérgio Sacani
 
GBSN - Microbiology Lab (Microbiology Lab Safety Procedures)
GBSN -  Microbiology Lab (Microbiology Lab Safety Procedures)GBSN -  Microbiology Lab (Microbiology Lab Safety Procedures)
GBSN - Microbiology Lab (Microbiology Lab Safety Procedures)Areesha Ahmad
 
GBSN - Microbiology (Unit 7) Microbiology in Everyday Life
GBSN - Microbiology (Unit 7) Microbiology in Everyday LifeGBSN - Microbiology (Unit 7) Microbiology in Everyday Life
GBSN - Microbiology (Unit 7) Microbiology in Everyday LifeAreesha Ahmad
 
GBSN - Microbiology (Unit 6) Human and Microbial interaction
GBSN - Microbiology (Unit 6) Human and Microbial interactionGBSN - Microbiology (Unit 6) Human and Microbial interaction
GBSN - Microbiology (Unit 6) Human and Microbial interactionAreesha Ahmad
 
mixotrophy in cyanobacteria: a dual nutritional strategy
mixotrophy in cyanobacteria: a dual nutritional strategymixotrophy in cyanobacteria: a dual nutritional strategy
mixotrophy in cyanobacteria: a dual nutritional strategyMansiBishnoi1
 
IISc Bangalore M.E./M.Tech. courses and fees 2024
IISc Bangalore M.E./M.Tech. courses and fees 2024IISc Bangalore M.E./M.Tech. courses and fees 2024
IISc Bangalore M.E./M.Tech. courses and fees 2024SciAstra
 
Exomoons & Exorings with the Habitable Worlds Observatory I: On the Detection...
Exomoons & Exorings with the Habitable Worlds Observatory I: On the Detection...Exomoons & Exorings with the Habitable Worlds Observatory I: On the Detection...
Exomoons & Exorings with the Habitable Worlds Observatory I: On the Detection...Sérgio Sacani
 
Tuberculosis (TB)-Notes.pdf microbiology notes
Tuberculosis (TB)-Notes.pdf microbiology notesTuberculosis (TB)-Notes.pdf microbiology notes
Tuberculosis (TB)-Notes.pdf microbiology notesjyothisaisri
 
EU START PROJECT. START-Newsletter_Issue_4.pdf
EU START PROJECT. START-Newsletter_Issue_4.pdfEU START PROJECT. START-Newsletter_Issue_4.pdf
EU START PROJECT. START-Newsletter_Issue_4.pdfStart Project
 
Film Coated Tablet and Film Coating raw materials.pdf
Film Coated Tablet and Film Coating raw materials.pdfFilm Coated Tablet and Film Coating raw materials.pdf
Film Coated Tablet and Film Coating raw materials.pdfPharmatech-rx
 
WASP-69b’s Escaping Envelope Is Confined to a Tail Extending at Least 7 Rp
WASP-69b’s Escaping Envelope Is Confined to a Tail Extending at Least 7 RpWASP-69b’s Escaping Envelope Is Confined to a Tail Extending at Least 7 Rp
WASP-69b’s Escaping Envelope Is Confined to a Tail Extending at Least 7 RpSérgio Sacani
 
NuGOweek 2024 programme final FLYER short.pdf
NuGOweek 2024 programme final FLYER short.pdfNuGOweek 2024 programme final FLYER short.pdf
NuGOweek 2024 programme final FLYER short.pdfpablovgd
 
Biochemistry and Biomolecules - Science - 9th Grade by Slidesgo.pptx
Biochemistry and Biomolecules - Science - 9th Grade by Slidesgo.pptxBiochemistry and Biomolecules - Science - 9th Grade by Slidesgo.pptx
Biochemistry and Biomolecules - Science - 9th Grade by Slidesgo.pptxjayabahari688
 
MIP Award presentation at the IEEE International Conference on Software Analy...
MIP Award presentation at the IEEE International Conference on Software Analy...MIP Award presentation at the IEEE International Conference on Software Analy...
MIP Award presentation at the IEEE International Conference on Software Analy...Annibale Panichella
 
Detectability of Solar Panels as a Technosignature
Detectability of Solar Panels as a TechnosignatureDetectability of Solar Panels as a Technosignature
Detectability of Solar Panels as a TechnosignatureSérgio Sacani
 
Erythropoiesis- Dr.E. Muralinath-C Kalyan
Erythropoiesis- Dr.E. Muralinath-C KalyanErythropoiesis- Dr.E. Muralinath-C Kalyan
Erythropoiesis- Dr.E. Muralinath-C Kalyanmuralinath2
 
Mining Activity and Investment Opportunity in Myanmar.pptx
Mining Activity and Investment Opportunity in Myanmar.pptxMining Activity and Investment Opportunity in Myanmar.pptx
Mining Activity and Investment Opportunity in Myanmar.pptxKyawThanTint
 
GBSN - Microbiology Lab (Compound Microscope)
GBSN - Microbiology Lab (Compound Microscope)GBSN - Microbiology Lab (Compound Microscope)
GBSN - Microbiology Lab (Compound Microscope)Areesha Ahmad
 

Recently uploaded (20)

Extensive Pollution of Uranus and Neptune’s Atmospheres by Upsweep of Icy Mat...
Extensive Pollution of Uranus and Neptune’s Atmospheres by Upsweep of Icy Mat...Extensive Pollution of Uranus and Neptune’s Atmospheres by Upsweep of Icy Mat...
Extensive Pollution of Uranus and Neptune’s Atmospheres by Upsweep of Icy Mat...
 
Jet reorientation in central galaxies of clusters and groups: insights from V...
Jet reorientation in central galaxies of clusters and groups: insights from V...Jet reorientation in central galaxies of clusters and groups: insights from V...
Jet reorientation in central galaxies of clusters and groups: insights from V...
 
GBSN - Microbiology Lab (Microbiology Lab Safety Procedures)
GBSN -  Microbiology Lab (Microbiology Lab Safety Procedures)GBSN -  Microbiology Lab (Microbiology Lab Safety Procedures)
GBSN - Microbiology Lab (Microbiology Lab Safety Procedures)
 
GBSN - Microbiology (Unit 7) Microbiology in Everyday Life
GBSN - Microbiology (Unit 7) Microbiology in Everyday LifeGBSN - Microbiology (Unit 7) Microbiology in Everyday Life
GBSN - Microbiology (Unit 7) Microbiology in Everyday Life
 
GBSN - Microbiology (Unit 6) Human and Microbial interaction
GBSN - Microbiology (Unit 6) Human and Microbial interactionGBSN - Microbiology (Unit 6) Human and Microbial interaction
GBSN - Microbiology (Unit 6) Human and Microbial interaction
 
mixotrophy in cyanobacteria: a dual nutritional strategy
mixotrophy in cyanobacteria: a dual nutritional strategymixotrophy in cyanobacteria: a dual nutritional strategy
mixotrophy in cyanobacteria: a dual nutritional strategy
 
IISc Bangalore M.E./M.Tech. courses and fees 2024
IISc Bangalore M.E./M.Tech. courses and fees 2024IISc Bangalore M.E./M.Tech. courses and fees 2024
IISc Bangalore M.E./M.Tech. courses and fees 2024
 
Exomoons & Exorings with the Habitable Worlds Observatory I: On the Detection...
Exomoons & Exorings with the Habitable Worlds Observatory I: On the Detection...Exomoons & Exorings with the Habitable Worlds Observatory I: On the Detection...
Exomoons & Exorings with the Habitable Worlds Observatory I: On the Detection...
 
Tuberculosis (TB)-Notes.pdf microbiology notes
Tuberculosis (TB)-Notes.pdf microbiology notesTuberculosis (TB)-Notes.pdf microbiology notes
Tuberculosis (TB)-Notes.pdf microbiology notes
 
EU START PROJECT. START-Newsletter_Issue_4.pdf
EU START PROJECT. START-Newsletter_Issue_4.pdfEU START PROJECT. START-Newsletter_Issue_4.pdf
EU START PROJECT. START-Newsletter_Issue_4.pdf
 
Chemistry Data Delivery from the US-EPA Center for Computational Toxicology a...
Chemistry Data Delivery from the US-EPA Center for Computational Toxicology a...Chemistry Data Delivery from the US-EPA Center for Computational Toxicology a...
Chemistry Data Delivery from the US-EPA Center for Computational Toxicology a...
 
Film Coated Tablet and Film Coating raw materials.pdf
Film Coated Tablet and Film Coating raw materials.pdfFilm Coated Tablet and Film Coating raw materials.pdf
Film Coated Tablet and Film Coating raw materials.pdf
 
WASP-69b’s Escaping Envelope Is Confined to a Tail Extending at Least 7 Rp
WASP-69b’s Escaping Envelope Is Confined to a Tail Extending at Least 7 RpWASP-69b’s Escaping Envelope Is Confined to a Tail Extending at Least 7 Rp
WASP-69b’s Escaping Envelope Is Confined to a Tail Extending at Least 7 Rp
 
NuGOweek 2024 programme final FLYER short.pdf
NuGOweek 2024 programme final FLYER short.pdfNuGOweek 2024 programme final FLYER short.pdf
NuGOweek 2024 programme final FLYER short.pdf
 
Biochemistry and Biomolecules - Science - 9th Grade by Slidesgo.pptx
Biochemistry and Biomolecules - Science - 9th Grade by Slidesgo.pptxBiochemistry and Biomolecules - Science - 9th Grade by Slidesgo.pptx
Biochemistry and Biomolecules - Science - 9th Grade by Slidesgo.pptx
 
MIP Award presentation at the IEEE International Conference on Software Analy...
MIP Award presentation at the IEEE International Conference on Software Analy...MIP Award presentation at the IEEE International Conference on Software Analy...
MIP Award presentation at the IEEE International Conference on Software Analy...
 
Detectability of Solar Panels as a Technosignature
Detectability of Solar Panels as a TechnosignatureDetectability of Solar Panels as a Technosignature
Detectability of Solar Panels as a Technosignature
 
Erythropoiesis- Dr.E. Muralinath-C Kalyan
Erythropoiesis- Dr.E. Muralinath-C KalyanErythropoiesis- Dr.E. Muralinath-C Kalyan
Erythropoiesis- Dr.E. Muralinath-C Kalyan
 
Mining Activity and Investment Opportunity in Myanmar.pptx
Mining Activity and Investment Opportunity in Myanmar.pptxMining Activity and Investment Opportunity in Myanmar.pptx
Mining Activity and Investment Opportunity in Myanmar.pptx
 
GBSN - Microbiology Lab (Compound Microscope)
GBSN - Microbiology Lab (Compound Microscope)GBSN - Microbiology Lab (Compound Microscope)
GBSN - Microbiology Lab (Compound Microscope)
 

Building and Evaluating Theories 
 in Software Engineering

  • 1. Daniel Méndez TUM, Germany www.mendezfe.org Building and Evaluating Theories 
 in Software Engineering @mendezfe Software Engineering Research Methods Dagstuhl, April 2019
  • 2. Simplified view on epistemological setting Principle ways of working Methods and strategies Fundamental theories Philosophy of science
  • 3. Simplified view on epistemological setting Principle ways of working Methods and strategies Fundamental theories Philosophy of science Epistemology Empiricism Statistics Hypothesis testing Controlled Experimentation Example!
  • 4. Setting: Empirical Software Engineering Principle ways of working Methods and strategies Fundamental theories Philosophy of science Typically in scope of Empirical Software 
 Engineering
  • 5. Setting: Empirical Software Engineering Principle ways of working Methods and strategies Fundamental theories Philosophy of science Theory building 
 and evaluation Methods and 
 strategies … supported by… Analogy: Theoretical and 
 Experimental Physics
  • 6. Goals of this talk Principle ways of working Methods and strategies Fundamental theories Philosophy of science Theory building 
 and evaluation Methods and 
 strategies … supported by… Analogy: Theoretical and 
 Experimental Physics Basic understanding of • Notion of theories in our field • (Philosophy of science) context of research methods and strategies for theory building and evaluation
  • 7. Ground rule Whenever you have questions / remarks, don’t ask , but share them with the whole group.
  • 8. Theory Building in Software Engineering ‣Theories in a Nutshell … where we will briefly talk about the general notion of theories ‣State of Evidence in Software Engineering … where we will see why theory building is so important to our field ‣Research Methods in Software Engineering … where we will put research methods in a larger (philosophical) picture ‣Qualitative “vs” quantitative research … where we will discuss different research approaches
  • 9. Theory Building in Software Engineering ‣Theories in a Nutshell … where we will briefly talk about the general notion of theories ‣State of Evidence in Software Engineering … where we will see why theory building is so important to our field ‣Research Methods in Software Engineering … where we will put research methods in a larger (philosophical) picture ‣Qualitative “vs” quantitative research … where we will discuss different research approaches
  • 10. What is a theory? What do you think?
  • 11. Theories (generally speaking) Examples: • Vaccinations lead to autism • Global warming is invented to harm the industry • Earth is flat • … A theory is a belief that there is a pattern in phenomena. Further examples: Twitter profiles by right-wing populists
  • 12. Theories (generally speaking) Examples: • Vaccinations lead to autism • Global warming is invented to harm the industry • Earth is flat • … A theory is a belief that there is a pattern in phenomena. Are these theories scientific? Further examples: Twitter profiles by right-wing populists » Speculations based on imagination or opinions that often cannot be refuted 
 (i.e. logical fallacies)
  • 13. Scientific Theories 1. Tests • Experiments, simulations, … • Replications 2. Criticism • Peer reviews / acceptance in the community • Corroborations / extensions with further theories A scientific theory is a belief that there is a pattern in phenomena while having survived 1. tests against experiences 2. criticism by critical peers
  • 14. Scientific Theories 1. Tests • Experiments, simulations, … • Replications 2. Criticism • Peer reviews / acceptance in the community • Corroborations / extensions with further theories A scientific theory is a belief that there is a pattern in phenomena while having survived 1. tests against experiences 2. criticism by critical peers Note: Addresses so-called Demarcation Problem to distinguishing science from non- science (as per introduction by K. Popper)
  • 15. “There is no universally agreed upon definition of the concept of an empirically-based theory [in Software Engineering], nor is there any uniform terminology for describing theories.” Source: Sjøberg, D., Dybå, T., Anda, B., Hannay, J. Building Theories in Software Engineering, 2010. Approach by characteristics
  • 16. Scientific Theories have… Quality criteria: • Testability • Empirical support / (high) level of confidence • Explanatory power • Usefulness to researchers and / or practitioners • … Adopted from: Sjøberg, D., Dybå, T., Anda, B., Hannay, J. Building Theories in Software Engineering, 2010. Analytical Explanatory Predictive Explanatory & Predictive Scope Descriptions and conceptualisation, 
 including taxonomies, classifications, and ontologies
 - What is? Identification of phenomena by identifying causes, mechanisms or reasons
 - Why is? Prediction of what will happen in the future 
 - What will happen? Prediction of what will happen in the future and explanation
 - What will happen and why?
 Further information: https://goo.gl/SQQwxt A purpose:
  • 17. Scientific Theories have… Quality criteria: • Testability • Empirical support / (high) level of confidence • Explanatory power • Usefulness to researchers and / or practitioners • … Adopted from: Sjøberg, D., Dybå, T., Anda, B., Hannay, J. Building Theories in Software Engineering, 2010. Analytical Explanatory Predictive Explanatory & Predictive Scope Descriptions and conceptualisation, 
 including taxonomies, classifications, and ontologies
 - What is? Identification of phenomena by identifying causes, mechanisms or reasons
 - Why is? Prediction of what will happen in the future 
 - What will happen? Prediction of what will happen in the future and explanation
 - What will happen and why?
 Further information: https://goo.gl/SQQwxt A purpose: Note: Many theories in software engineering are so-called “design [science] theories”, i.e. scientific theories about artefacts in a context. [Artefact specification] X [Context assumptions] → [Effects]
  • 18. Scientific Theories have… Quality criteria: • Testability • Empirical support / (high) level of confidence • Explanatory power • Usefulness to researchers and / or practitioners • … Adopted from: Sjøberg, D., Dybå, T., Anda, B., Hannay, J. Building Theories in Software Engineering, 2010. Analytical Explanatory Predictive Explanatory & Predictive Scope Descriptions and conceptualisation, 
 including taxonomies, classifications, and ontologies
 - What is? Identification of phenomena by identifying causes, mechanisms or reasons
 - Why is? Prediction of what will happen in the future 
 - What will happen? Prediction of what will happen in the future and explanation
 - What will happen and why?
 Note: Laws “versus” theories A law is a purely descriptive theory about phenomena (without explanations), i.e. an analytical theory. Further information: https://goo.gl/SQQwxt A purpose: Note: Many theories in software engineering are so-called “design [science] theories”, i.e. scientific theories about artefacts in a context. [Artefact specification] X [Context assumptions] → [Effects]
  • 19. Exemplary framework for describing theories in Software Engineering •Constructs: What are the basic elements?
 (Actors, technologies, activities, system entities, context factors) •Propositions: How do the constructs interact? •Explanations: Why are the propositions as specified? •Scope: What is the universe of discourse in which the theory is applicable? Source: Sjøberg, D., Dybå, T., Anda, B., Hannay, J. Building Theories in Software Engineering, 2010.
  • 20. Exemplary framework for describing theories in Software Engineering •Constructs: What are the basic elements?
 (Actors, technologies, activities, system entities, context factors) •Propositions: How do the constructs interact? •Explanations: Why are the propositions as specified? •Scope: What is the universe of discourse in which the theory is applicable? Source: Sjøberg, D., Dybå, T., Anda, B., Hannay, J. Building Theories in Software Engineering, 2010. Example Source: Wagner, Mendez Fernandez et al. Status Quo in Requirements Engineering: A Theory and a Global Family of Surveys, TOSEM 2018. Req Elicitation Technique Interview Scenario Prototyping Facilitated Meetings Observation Req Documentation Technique Structured req list Domain/business process model Use case model Goal model Data model Non‐functional req Textual Semi‐formal Formal Technology Req Test Alignment Approach Req review by tester Coverage by tests Acceptance criteria Test derivation from models Req Change Approach Product backlog update Change requests Trace management  Impact analysis Activity Req Elicitation Req Documentation Req Change Management Req Test Alignment P 1‐5 P 6‐13 P 14‐20 P 21‐24 Actor Req Engineer Test Engineer Req Standard Application Practice Control Tailoring Req Eng Process Standard P 25‐28 Req Standard Defintion Compliance Development Tool support Quality assurance Project management Knowledge transfer Process complexity Communication demand Willigness to change Possibility of standardisation   Req Improvement Means Continuous improvement Strengths/weaknesses Own business unit/role Req Eng Improvement  P 29-44 P 45-49 Proposition: “Structured requirements lists are documented textually in free form or textually with constraints.” Explanation: “Free-form and constraint textual requirements are sufficient for many contexts such as in agile projects where they only act as reminders for further conversations.”
  • 21. Theories and hypotheses Empirical Approaches Theory / Theories (Tentative) Hypothesis Falsification / 
 Corroboration Theory (Pattern)
 Building Hypothesis
 Building Hypothesis • “[…] a statement that proposes a possible explanation to some phenomenon or event” (L. Given, 2008) • Grounded in theory, testable and falsifiable • Often quantified and written as a conditional statement Scientific theory • “[…] based on hypotheses tested and verified multiple times by detached researchers” (J. Bortz and N. Döring, 2003) If cause/assumption (independent variables) then (=>) consequence (dependent variables) Note: We don’t “test theories”, but their consequences via hypotheses 
 (i.e. testable propositions)
  • 22. From real world phenomena to theories 
 (and back) - The Empirical Lifecycle Empirical Approaches Theory / Theories (Tentative) Hypothesis Falsification / 
 Corroboration Theory (Pattern)
 Building Units of Analysis Sampling Frame Sampling Hypothesis
 Building Empirical Inquiries Induction Inference of a 
 general rule 
 from a particular case/result 
 (observation) Abduction (Creative) Synthesis of an 
 explanatory case from a general rule 
 and a particular result (observation) Deduction Application of a general rule 
 to a particular case, 
 inferring a specific result Source: Mendez and Passoth. Empirical Software 
 Engineering: from Discipline to Interdiscipline, 2018.
  • 23. Theory Building in Software Engineering ‣Theories in a Nutshell … where we will briefly talk about the general notion of theories ‣State of Evidence in Software Engineering … where we will see why theory building is so important to our field ‣Research Methods in Software Engineering … where we will put research methods in a larger (philosophical) picture ‣Qualitative “vs” quantitative research … where we will discuss different research approaches
  • 24. Example: Goal-oriented RE (“GORE”) Papers published [1]: 966 [2] Mavin, et al. Does Goal-Oriented Requirements Engineering Achieve its Goal?, 2017 [3] Mendez et al. Naming the Pain in Requirements Engineering Initiative - www.re-survey.org [1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016 Papers including a case study [1]: 131 Studies involving practitioners [2]: 20 Practitioners actually using GORE [3]: ~ 5%
  • 25. Example: Goal-oriented RE (“GORE”) Papers published [1]: 966 [2] Mavin, et al. Does Goal-Oriented Requirements Engineering Achieve its Goal?, 2017 [3] Mendez et al. Naming the Pain in Requirements Engineering Initiative - www.re-survey.org [1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016 Papers including a case study [1]: 131 Studies involving practitioners [2]: 20 Practitioners actually using GORE [3]: ~ 5% For comparison: Icelanders believing in elves [4]: [4] https://www.nationalgeographic.com/travel/destinations/europe/iceland/believes-elves-exist-mythology/ 54%
  • 26. “[…] judging a theory by assessing the number, faith, and vocal energy of its supporters […] basic political credo of contemporary religious maniacs” — Imre Lakatos, 1970 Current state of evidence in SE is weak * Addressing the situation in the quantum mechanics research community, an analogy
  • 27. Current state of evidence in SE is weak Source (levels of evidence): Wohlin. An Evidence Profile for Software Engineering Research and Practice, 2013. In favour / corroboration Against / refutation Strong evidence Evidence Circumstantial evidence Third-party claim First or second party claim Strong evidence Circumstantial evidence Third-party claim Evidence First or second party claim + - In most cases, 
 we are here Available studies often… • … remain isolated • … don’t report negative results • … strengthen confidence on 
 own hopes (and don’t 
 report anything around) • … discuss little (if at all) 
 relation to existing evidence
  • 28. SE largely dominated by conventional wisdom “Leprechauns”: Folklore turned into facts • Emerge from times where claims by authorities were treated as “facts” • Reasons manifold: – Lack of empirical awareness – Neglecting particularities 
 of practical contexts – Neglecting relation to existing evidence – No proper citations (one side the medal, over-conclusions, etc.) – Lack of data – …
  • 29. Consequences » Practical relevance and impact? » Potential for transfer into practice and adoption?
  • 30. The essence Theory building and theory evaluation 
 are crucial in SE research » Reason about the discipline and (e.g. social) phenomena involved » Recognise and understand limits and effects of artefacts (technologies, techniques, processes, models, etc.) in their contexts
  • 31. Theory Building in Software Engineering ‣Theories in a Nutshell … where we will briefly talk about the general notion of theories ‣State of Evidence in Software Engineering … where we will see why theory building is so important to our field ‣Research Methods in Software Engineering … where we will put research methods in a larger (philosophical) picture ‣Qualitative “vs” quantitative research … where we will discuss different research approaches
  • 32. Recap: The Empirical Lifecycle Source: Mendez and Passoth. Empirical Software 
 Engineering: from Discipline to Interdiscipline, 2018.
  • 33. (Empirical) methods •Each method… – …has a specific purpose – …relies on a specific data type •Purposes – Exploratory – Descriptive – Explanatory – Improving •Data Types – Qualitative – Quantitative
  • 34. (Empirical) methods •Each method… – …has a specific purpose – …relies on a specific data type •Purposes – Exploratory – Descriptive – Explanatory – Improving •Data Types – Qualitative – Quantitative “Grounded Theory” Qualitative Data Descriptive, Exploratory, or Explanatory Example!
  • 35. (Empirical) methods - where do they belong? Ethnographic studies, Folklore gathering Case studies /
 Field studies (Confirmatory) Formal analysis / logical reasoning Survey research Case studies /
 Field studies (Exploratory)
  • 36. (Empirical) methods - where do they belong? Ethnographic studies, Folklore gathering Case studies /
 Field studies (Confirmatory) Formal analysis / logical reasoning Survey research Case studies /
 Field studies (Exploratory)
  • 37. Which research methods to use 
 in which situations?
  • 38. Empirical process: an abstract view Planning and Definition Method and Strategy Selection Design and (Method) Execution Conclusion Drawing Packaging and Reporting •Identify and outline problem (area) •Determine research objectives and questions •Select type of study and method(s) •Identify necessary environment 
 (including units of analysis) •Design and validate study protocol (and validity procedures) •Collect & analyse data following detailed processes •Interpret results •Reflect upon potential threats to validity •Package and (ideally) disclose data •Report on results in tune with audience
  • 39. Empirical process: an abstract view Planning and Definition Method and Strategy Selection Design and (Method) Execution Conclusion Drawing Packaging and Reporting Scope of detailed empirical methods The selection of proper research 
 methods is a recognised problem 
 in the community
  • 40. There is no such thing as a 
 universal way of scientific practice!
  • 41. Method selection depends on many 
 non-trivial questions • What is the purpose of the study? – Exploratory? Descriptive? Explanatory? Improving? • What is the nature of the study? – Inductive? Deductive? • What is the relation to existing evidence? – Building a new theory? “Testing” existing theory? • What is the nature of the questions we ask? – What-questions? Why-questions? • What is the nature of the environment? – Controlled environments? Realistic environments? • What is the necessary sample? – Population source? – Units of analysis? Criteria for 
 selecting methods Criteria for 
 environment selection
 (and sampling) Non-exclusive andnon-sequential
  • 42. Not trivial, but possible: By using checklists More advanced • Chapter 16 + Appendix • http://bit.ly/checklists-design_science Good starting point
  • 43. How to achieve scientific progress? In step-wise iterations, with multiple methods 
 (aka “research programme”)
  • 44. Progress via multi-study approaches Large-scale evaluation 5 e.g. Field Study or longitudinal study 1 2 3 4 5 Validation of new technology in artificial setting 3 e.g. Controlled Experiment Evaluation of new technology in realistic setting 4 e.g. Case Study Replication Replication Proposal new / adaptation existing technology 2 Problem analysis 1 e.g. Systematic Mapping Study
 or Survey e.g. RE Improvement 
 Approach
  • 45. Postulate I Every study has a specific scope of validity only Source: Sjøberg, D., Dybå,T.,Anda, B., Hannay, J. Building Theories in Software Engineering, 2010.
  • 46. Postulate II Scope of validity ≠ Degree of reality Simulation Field Study 
 Research Case Study 
 Research Survey Research Artificial environment Realistic environment Scope of validity*Controlled (lab) Experiment Replications Replications * Extremely simplified view to orient discussions
  • 47. Postulate III Case studies and experiments complement each other in scaling up to practice Based on: Wieringa R. Empirical Research Methods for TechnologyValidation: Scaling Up to Practice, 2013. Lab credibility Street credibility Simple model Realistic case Small sample Large sample Focus of 
 case studies Focus of field studies 
 and replications Scaling up to practice Focus of 
 (lab) experiments Similarity to 
 population units Sample size
  • 48. Theory Building in Software Engineering ‣Theories in a Nutshell … where we will briefly talk about the general notion of theories ‣State of Evidence in Software Engineering … where we will see why theory building is so important to our field ‣Research Methods in Software Engineering … where we will put research methods in a larger (philosophical) picture ‣Qualitative “vs” quantitative research … where we will discuss different research approaches
  • 49. What is the difference between qualitative and quantitative research? What do you think?
  • 50. Warning: EmSE emerges from natural science, thus, qualitative methods are often confronted with prejudice “I prefer working with real data 
 [not with qualitative data]” — Anonymous ISERN member “In contrast [to previous qualitative studies], this study attempts to obtain more scientific evidence in the form of objective, quantitative data.” — Anonymous author “With all due respect, DO NOT make such ridiculous claims that only quantitative studies are "scientific" and “objective" There is NO more objectivity in numbers than there is in qualitative data. ” — Anonymous reviewer … response
  • 51. The essence Qualitative studies focus primarily on the kind of evidence that will enable you to understand the meaning [and purpose, reasoning, etc] of what is going on. Quantitative studies focus primarily on the kind of evidence that will enable you to understand what is going on. Qualitative and quantitative methods have complementary purposes, strengths, and limitations in theory building.
  • 52. Two complementary approaches (+1) •Quantitative research – Describing events and finding causes to predict similar events in the future – (Typically) focus on what, how much, or how many – (Typically) in numerical forms – (Typically) descriptive purpose •Qualitative research – Understanding meaning [and purpose, reasoning, etc.] of a phenomenon for those involved – (Typically) focus on why/meaning, and how people interpret their experiences – (Typically) in variety of non-numerical forms, like texts, diagrams, etc. – (Typically) exploratory or explanatory purpose •Mix-method research Adopted from: Da Silva. Tutorial given at the Ibero-American Conference on Software Engineering, 2018 (Bogota, Colombia)
  • 53. Two complementary approaches (+1) Quantitative data Qualitative data “Why?”-Questions“What?”-Questions Descriptive & predictive purposes Explanatory & exploratory purposes
  • 54. Two complementary approaches (+1) Quantitative data Qualitative data Case study research Survey research (Quasi-) Controlled 
 experiments Action 
 research Ethnographic studies “Why?”-Questions“What?”-Questions Descriptive & predictive purposes Explanatory & exploratory purposes
  • 55. Comparison: qualitative “vs” quantitative Qualitative Research Quantitative Research Goals • Understanding, reasoning, explanations, descriptions, meaning (to subjects), discovery • Hypothesis generation (typically) • Description, control, prediction • Hypothesis testing (typically) Design characteristics • Flexible, evolving, emergent • Inductive, constant comparative • Predetermined, structured / fixed • Deductive, statistical Samples • Small, non-random (sometimes even opportunistic), purposeful, theoretical • Large(r), random, representative Data collection • Researcher often primary instrument • Interviews, observations, document analysis, … • Inanimate instruments (tests, surveys, questionnaires, etc.) Findings • Comprehensive, holistic, rich descriptions • Precise and statistical Adopted from: Da Silva. Tutorial given at the Ibero-American Conference on Software Engineering, 2018 (Bogota, Colombia) * For all, you can add a “in tendency”
  • 56. Researchers’ necessary skills •Being a careful observer with high tolerance to ambiguity and existing belief systems •Ability to ask “proper”, context- specific questions without actively distorting results •Thinking inductively •Patience (and charm) •Comfort with observing 
 (and writing :-) Adopted from: Da Silva. Tutorial given at the Ibero-American Conference on Software Engineering, 2018 (Bogota, Colombia)
  • 57. Challenges in conducting qualitative studies • Cases/subjects need to offer rich insights (challenging in industry contexts) —> Case selection often opportunistic • Data collection demands skills beyond statistics (e.g. social skills) • Data analysis is often a (pragmatic and) creative task. • Data disclosure particularly important (and challenging for ethical and legal reasons) • Communication and publication often rendered difficult by prejudice Source: Mendez, Wagner. Case Studies in Industry: What we have Learnt, 2016
  • 58. Further reading: Selected Papers Good starting point Terminological demarcation and key characteristics 
 of methods Epistemological setting
  • 59. Theory Building in Software Engineering ‣Theories in a Nutshell … where we will briefly talk about the general notion of theories ‣State of Evidence in Software Engineering … where we will see why theory building is so important to our field ‣Research Methods in Software Engineering … where we will put research methods in a larger (philosophical) picture ‣Qualitative “vs” quantitative research … where we will discuss different research approaches
  • 60. Theory Building in Software Engineering ‣ Theories are the backbone of our discipline allowing us to move forward from paradigmatic stage of an engineering discipline to a scientific one. ‣ Every research method has its place in a larger picture. Thank you! ‣ Qualitative and quantitative research has complementary purposes, strengths, and limitations in building and evaluating theories.