The document discusses the importance of strong theories for empirical software engineering research. It argues that theories are needed to provide standardized conceptualizations, allow predictions for new situations, and provide a framework for combining individual studies. Empirical research should be based on hypotheses derived from theoretical predictions and clarify how findings relate to and potentially revise the underlying theory. The goal is to move beyond isolated studies and build larger, integrated theories that define a "calculus" or framework for making predictions across different contexts. Stronger theories are needed to improve the rigor and reproducibility of empirical software engineering research.

1. www.sse.uni-hildesheim.de
Software Systems Engineering
University of Hildesheim
schmid@sse.uni-hildesheim.de
Klaus Schmid
On the importance of strong theories for progress
in empirical software engineering research
If you want better empirical research, value your theory

2. 1
Klaus Schmid, University of Hildesheim
21.05.2022
If you want better empirical research, value your theory
Why this paper?
• A few words about me
• Focus on „engineering“ methods to support companies
 Lack of possibility of dealing with new situations
• Motivations from philosophy of science
 Looking at „best“ sciences
 Physics background
→ Some positions on the question: are we doing it right?
Goal
• Motivate you to read 6 pages
• Make you agree with the motivation
• Illustrate some of the ideas
• Hopefully give you some different perspectives

3. 1
Klaus Schmid, University of Hildesheim
21.05.2022
If you want better empirical research, value your theory
How does (SE) science work?
Theory
(Tentative)
Hypothesis
Empirical Approach
Empirical
Inquiry
Theoretical
Prediction
Confirmation/
Improvement
of theory
Falsification
/ Support
Constraints
on experiment
design
Everyone agrees on this ✓
But what does the relation of
theory and empirical approach
really imply
physics inside

4. 1
Klaus Schmid, University of Hildesheim
21.05.2022
If you want better empirical research, value your theory
Observations
 Theory is about establishing strong & standardized
conceptualizations
 What does a „developer“ do?
 When is s/he „experienced“?
 Cover both the empirical work AND the domain to which the
theory can be applied
→ A theory has a well-defined range of applicability
 Clear preconditions / boundaries
 „External validity“ fallacy
The purpose of scientific theories is to allow to make predictions
(qualitative/quantitative) for new situations (and explain them)

5. 1
Klaus Schmid, University of Hildesheim
21.05.2022
If you want better empirical research, value your theory
Theory as a foundation for empiricsm
 Any theory is better than no theory
 What to look at / take into account?
 Identify range of results
 Empiricial work should always clarify the input
and output relative to the underlying theory
→ Only way to move from isolated studies (theoretical singletons)
to theories in this sense of the word
 Hardly any method paper in last year‘s EASE fits this!
Theory
(Tentative)
Hypothesis
Empirical Approach
Empirical
Inquiry
Theoretical
Prediction
Confirmation/
Improvement
of theory
Falsification
/ Support
Constraints
on experiment
design
↯

6. 1
Klaus Schmid, University of Hildesheim
21.05.2022
If you want better empirical research, value your theory
Operations on theories (Calculus)
 Theory: defines „a calculus“ (not analysis)
 How to make predictions using the theory (in new situations)
 How to combine very different individual studies
 How to reconcile new evidence with existing information
→ Central to identify „hidden concepts“
 Identifying such concepts was core to the advance of other
sciences, why not SE?
Theory
(Tentative)
Hypothesis
Empirical Approach
Empirical
Inquiry
Theoretical
Prediction
Confirmation/
Improvement
of theory
Falsification
/ Support
Constraints
on experiment
design
How to combine new evidence with existing knowledge?

7. 1
Klaus Schmid, University of Hildesheim
21.05.2022
If you want better empirical research, value your theory
Propositions
1. A scientific theory should allow to make
predictions for new situations.
2. A theory should provide a rich set of concepts
sufficient to characterize all relevant phenomena
to which it applies.
3. A scientific theory needs to define ways to apply
its content to new situations.
4. A scientific theory should have a clearly defined
range of applicability.
5. Any theory is better than no theory as it provides
a crystallization point for insights.
6. An empirical study (except for pure theory
finding) should be started based on a hypothesis,
which should be derived from predictions based
on an initial theory.
7. Empirical work should always clarify the input
and the output relation relative to an underlying
theory. On the input side, it should clarify how it uses
existing theories, e.g., to derive predictions, while on the
output side, it should highlight the need for revisions of
the theory (in an as limited way as possible).
8. We need to stop doing theoretical singletons
and move towards larger, integrated theories.
9. Any theory should define a relevant form of
underlying software engineering calculus.
10.Having an underlying SE calculus helps to
address the many-variable-problem in SE
validation.
11.Introducing hidden concepts that help to
organize the theory and connects subtheories is
central to creating powerful scientific
frameworks.
12.Theory construction needs to focus on creating
partial theories interlinked through a common
SE calculus.

8. 1
Klaus Schmid, University of Hildesheim
21.05.2022
If you want better empirical research, value your theory
Models of scientific research
Medical research (epidemology mostly) Reproducible: 36%
Reproducible: 62%
On average: 50% efffect size