Empirical Methods in Software Engineering - an Overviewalessio_ferrari
A first introductory lecture on empirical methods in software engineering. It includes:
1) Motivation for empirical software engineering studies
2) How to define research questions
3) Measures and data collection methods
4) Formulating theories in software engineering
5) Software engineering research strategies
Find the videos at: https://www.youtube.com/playlist?list=PLSKM4VZcJjV-P3fFJYMu2OhlTjEr9Bjl0
Empirical Methods in Software Engineering - an Overviewalessio_ferrari
A first introductory lecture on empirical methods in software engineering. It includes:
1) Motivation for empirical software engineering studies
2) How to define research questions
3) Measures and data collection methods
4) Formulating theories in software engineering
5) Software engineering research strategies
Find the videos at: https://www.youtube.com/playlist?list=PLSKM4VZcJjV-P3fFJYMu2OhlTjEr9Bjl0
Systematic Literature Reviews and Systematic Mapping Studiesalessio_ferrari
Lecture slides on Systematic Literature Reviews and Systematic Mapping Studies in software engineering. It describes the different steps, discusses differences between the two methods, and gives guidelines on how to conduct these types of study.
14th slide set of CECS 542
Non-functional requirements, namely the subcategories of quality requirements, process requirements, and system constraints
Complete course: http://foss2serve.org/index.php/Requirements_Engineering,_CSU_Long_Beach,_Penzenstadler
Use of mathematical models for specification and validation in formal methodsGamentortc
Formal methods are system design techniques that use rigorously specified mathematical models to build software and hardware systems. In contrast to other design systems, formal methods use mathematical proof as a complement to system testing in order to ensure correct behavior.
Presented on Oct 28, 2014 at the Greater Atlanta Chapter IIBA.
People seek to make connections of items to make sense of them in a larger context. As children (or adults), we connect the dots to form a picture of something recognizable. As a business analyst, we connect requirements and other analysis outputs to get the bigger picture of an initiative and to check the completeness of our work.
We will explore how IIBA® has defined requirement traceability, how traceability works, and the benefits of the practice to the current project and future analysis.
This presentation is about a lecture I gave within the "Software systems and services" immigration course at the Gran Sasso Science Institute, L'Aquila (Italy): http://cs.gssi.infn.it/.
http://www.ivanomalavolta.com
Systematic Literature Reviews and Systematic Mapping Studiesalessio_ferrari
Lecture slides on Systematic Literature Reviews and Systematic Mapping Studies in software engineering. It describes the different steps, discusses differences between the two methods, and gives guidelines on how to conduct these types of study.
14th slide set of CECS 542
Non-functional requirements, namely the subcategories of quality requirements, process requirements, and system constraints
Complete course: http://foss2serve.org/index.php/Requirements_Engineering,_CSU_Long_Beach,_Penzenstadler
Use of mathematical models for specification and validation in formal methodsGamentortc
Formal methods are system design techniques that use rigorously specified mathematical models to build software and hardware systems. In contrast to other design systems, formal methods use mathematical proof as a complement to system testing in order to ensure correct behavior.
Presented on Oct 28, 2014 at the Greater Atlanta Chapter IIBA.
People seek to make connections of items to make sense of them in a larger context. As children (or adults), we connect the dots to form a picture of something recognizable. As a business analyst, we connect requirements and other analysis outputs to get the bigger picture of an initiative and to check the completeness of our work.
We will explore how IIBA® has defined requirement traceability, how traceability works, and the benefits of the practice to the current project and future analysis.
This presentation is about a lecture I gave within the "Software systems and services" immigration course at the Gran Sasso Science Institute, L'Aquila (Italy): http://cs.gssi.infn.it/.
http://www.ivanomalavolta.com
Introductory talk given to PhD students starting research at NUS PhD open day 2020. Covers research in Computer Science, and some experience in research on trustworthy software systems.
Starting - kickoff notes for PhD Candidates of the ABC Program, at Politecnico di Milano, Italy (A=Architecture, B=Built Environment, C=Construction Engineering)
In this lecture you will learn about the importance of research questions, how they related to research problems, the properties of good research questions, and the differences between quantitative and qualitative research questions.
Opponent's questions in the public examination of Marcus Duveskog's doctoral dissertation, School of Computing, University of Eastern Finland, January 29, 2015.
Interface Design - an overview on recent findings in HCI research and examples of interfaces created by WebFoo Interface Division.
This slideshow was presented by our Creative Director, Mihai Varga, at a guest lecture at Surrey University in March 2014.
Field Studies as evaluation method for socio-technical interventions in Techn...Viktoria Pammer-Schindler
Field studies as evaluation method for socio-technical interventions in Technology-Enhanced Learning.
Much research in TEL is design work – i.e., the research team designs an intervention that is intended to support learning. This intervention needs to be evaluated to show the extent to which this goal has been reached; and to gain additional insights that are sought for. Field studies are one main type of evaluations. They are challenging to set up; and in case of a bad study design cannot be easily repeated due to the effort and cost of running a field study. The goal of this lecture and workshop is
To provide a blueprint for field studies as evaluation method for socio-technical interventions in technology enhanced learning
To present a hierarchical principle of evaluating learning interventions– based on Kirkpatrick & Kirkpatrick: Usage/observable activities – Learning – Impact on task/work performance – Impact on organization (in workplace learning/applicable to settings in which individual learning impacts a wider social entity)
To have students plan a field study for their own PhD in rough lines individually
To discuss their plans with peers and the lecturer, as well as other senior researchers who may be present – i.e., students will get feedback on their own plan
The blueprint for field studies is to evaluate in a hierarchy of research questions/evaluation level: First, one assesses the observable (learning) activities that are carried out – in particular how and whether participants adhered to the prescribed intervention; this helps understand the success of the intervention and it is possible to identify problems. Second, one assesses concrete learning outcomes – insights that are generated. Thirdly, one assesses a change in behaviour, and fourthly a change in performance. In parallel, a mix of qualitative and quantitative methods should be used – this allows on the one hand statistical comparison (pre/post; between groups). On the other hand, one can get in depth explanatory insights.
Theory Building in RE - The NaPiRE InitiativeDaniel Mendez
Talk I gave on the "Naming the Pain in Requirements Engineering" initiative (www.re-survey.org) at the Seminar on Forty Years of Requirements Engineering – Looking Forward and Looking Back (RE@40) in Kappel am Albis, Switzerland
Theories in Empirical Software EngineeringDaniel Mendez
Slides from the International Advanced School on Empirical Software Engineering 2015, held as part of the Empirical Software Engineering International Week in Beijing. The slides are posted with the permission of the main organiser Roel Wieringa.
Hybrid optimization of pumped hydro system and solar- Engr. Abdul-Azeez.pdffxintegritypublishin
Advancements in technology unveil a myriad of electrical and electronic breakthroughs geared towards efficiently harnessing limited resources to meet human energy demands. The optimization of hybrid solar PV panels and pumped hydro energy supply systems plays a pivotal role in utilizing natural resources effectively. This initiative not only benefits humanity but also fosters environmental sustainability. The study investigated the design optimization of these hybrid systems, focusing on understanding solar radiation patterns, identifying geographical influences on solar radiation, formulating a mathematical model for system optimization, and determining the optimal configuration of PV panels and pumped hydro storage. Through a comparative analysis approach and eight weeks of data collection, the study addressed key research questions related to solar radiation patterns and optimal system design. The findings highlighted regions with heightened solar radiation levels, showcasing substantial potential for power generation and emphasizing the system's efficiency. Optimizing system design significantly boosted power generation, promoted renewable energy utilization, and enhanced energy storage capacity. The study underscored the benefits of optimizing hybrid solar PV panels and pumped hydro energy supply systems for sustainable energy usage. Optimizing the design of solar PV panels and pumped hydro energy supply systems as examined across diverse climatic conditions in a developing country, not only enhances power generation but also improves the integration of renewable energy sources and boosts energy storage capacities, particularly beneficial for less economically prosperous regions. Additionally, the study provides valuable insights for advancing energy research in economically viable areas. Recommendations included conducting site-specific assessments, utilizing advanced modeling tools, implementing regular maintenance protocols, and enhancing communication among system components.
Welcome to WIPAC Monthly the magazine brought to you by the LinkedIn Group Water Industry Process Automation & Control.
In this month's edition, along with this month's industry news to celebrate the 13 years since the group was created we have articles including
A case study of the used of Advanced Process Control at the Wastewater Treatment works at Lleida in Spain
A look back on an article on smart wastewater networks in order to see how the industry has measured up in the interim around the adoption of Digital Transformation in the Water Industry.
We have compiled the most important slides from each speaker's presentation. This year’s compilation, available for free, captures the key insights and contributions shared during the DfMAy 2024 conference.
Using recycled concrete aggregates (RCA) for pavements is crucial to achieving sustainability. Implementing RCA for new pavement can minimize carbon footprint, conserve natural resources, reduce harmful emissions, and lower life cycle costs. Compared to natural aggregate (NA), RCA pavement has fewer comprehensive studies and sustainability assessments.
Cosmetic shop management system project report.pdfKamal Acharya
Buying new cosmetic products is difficult. It can even be scary for those who have sensitive skin and are prone to skin trouble. The information needed to alleviate this problem is on the back of each product, but it's thought to interpret those ingredient lists unless you have a background in chemistry.
Instead of buying and hoping for the best, we can use data science to help us predict which products may be good fits for us. It includes various function programs to do the above mentioned tasks.
Data file handling has been effectively used in the program.
The automated cosmetic shop management system should deal with the automation of general workflow and administration process of the shop. The main processes of the system focus on customer's request where the system is able to search the most appropriate products and deliver it to the customers. It should help the employees to quickly identify the list of cosmetic product that have reached the minimum quantity and also keep a track of expired date for each cosmetic product. It should help the employees to find the rack number in which the product is placed.It is also Faster and more efficient way.
Forklift Classes Overview by Intella PartsIntella Parts
Discover the different forklift classes and their specific applications. Learn how to choose the right forklift for your needs to ensure safety, efficiency, and compliance in your operations.
For more technical information, visit our website https://intellaparts.com
CW RADAR, FMCW RADAR, FMCW ALTIMETER, AND THEIR PARAMETERSveerababupersonal22
It consists of cw radar and fmcw radar ,range measurement,if amplifier and fmcw altimeterThe CW radar operates using continuous wave transmission, while the FMCW radar employs frequency-modulated continuous wave technology. Range measurement is a crucial aspect of radar systems, providing information about the distance to a target. The IF amplifier plays a key role in signal processing, amplifying intermediate frequency signals for further analysis. The FMCW altimeter utilizes frequency-modulated continuous wave technology to accurately measure altitude above a reference point.
NUMERICAL SIMULATIONS OF HEAT AND MASS TRANSFER IN CONDENSING HEAT EXCHANGERS...ssuser7dcef0
Power plants release a large amount of water vapor into the
atmosphere through the stack. The flue gas can be a potential
source for obtaining much needed cooling water for a power
plant. If a power plant could recover and reuse a portion of this
moisture, it could reduce its total cooling water intake
requirement. One of the most practical way to recover water
from flue gas is to use a condensing heat exchanger. The power
plant could also recover latent heat due to condensation as well
as sensible heat due to lowering the flue gas exit temperature.
Additionally, harmful acids released from the stack can be
reduced in a condensing heat exchanger by acid condensation. reduced in a condensing heat exchanger by acid condensation.
Condensation of vapors in flue gas is a complicated
phenomenon since heat and mass transfer of water vapor and
various acids simultaneously occur in the presence of noncondensable
gases such as nitrogen and oxygen. Design of a
condenser depends on the knowledge and understanding of the
heat and mass transfer processes. A computer program for
numerical simulations of water (H2O) and sulfuric acid (H2SO4)
condensation in a flue gas condensing heat exchanger was
developed using MATLAB. Governing equations based on
mass and energy balances for the system were derived to
predict variables such as flue gas exit temperature, cooling
water outlet temperature, mole fraction and condensation rates
of water and sulfuric acid vapors. The equations were solved
using an iterative solution technique with calculations of heat
and mass transfer coefficients and physical properties.
Saudi Arabia stands as a titan in the global energy landscape, renowned for its abundant oil and gas resources. It's the largest exporter of petroleum and holds some of the world's most significant reserves. Let's delve into the top 10 oil and gas projects shaping Saudi Arabia's energy future in 2024.
Empirical Software Engineering - What is it and why do we need it?
1. Empirical Software Engineering
What is it and why do we need it?
Blekinge Institute of Technology, Sweden
fortiss GmbH, Germany
www.mendezfe.org
mendezfe
Daniel Méndez
2. About me // Daniel Mendez
Head of research division
Requirements Engineering
daniel.mendez@bth.se
www.mendezfe.org
📍Munich
📍 Karlskrona
Full Professor in
Software Engineering
Research areas / interests
▪ (Empirical Software Engineering)
▪ Requirements engineering, software quality
management, and processes improvement
▪ Strong focus on Academia-Industry
Collaborations and Technology Transfer
Research Networks and Initiatives
▪ Naming the Pain in Requirements Engineering
(www.napire.org)
▪ IREB Research working group
▪ International Software Engineering Research
Network (ISERN)
3. Ground rules
As we have to use ,
please feel free to interrupt me any time as I
might not see you raising your hand.
Whenever you have questions / remarks,
please don’t ask , but
share them with the whole group.
(1)
(2)
4. What could be wrong with such a study?
Research Question: Which car has the best driving performance?
H_0: There is no difference.
20 people without a driving licence participate.
They are taught to drive in a lecture of 2 hours.
Results: The BMW is significantly better than the Audi (p<0.01)
Adopted from: Dag I.K. Sjøberg, Keynote at the International Conference on Product-Focused SW Process Improvement 2016, Trondheim, Norway.
Image soruces: Manufacturer websites
Empirical research is
more than simplyapplying statistical
equations
5. Goal of the lecture
What is this little thing called “Empirical Software Engineering”?
What we will discuss
• In a nutshell:
• Broader perspective on Software Engineering (SE) as a scientific discipline
• A few principles, concepts, and terms in Empirical SE
• Why we need empiricism in Software Engineering research
• What the perspectives are for the research community and for you
Focus:
What & Why
Basis for…
• Course on research methods
• Master Thesis projects
• The time afterwards
Focus:
How
6. Outline
• What is Empirical Software Engineering?
• Why do we need Empirical Software Engineering?
• What are the perspectives in Empirical Software Engineering?
7. Outline
• What is Empirical Software Engineering?
• Why do we need Empirical Software Engineering?
• What are the perspectives in Empirical Software Engineering?
8. Let’s start step by step….
What is science?
-- What do you think? --
9. “Science” wasn’t built in a day…
384-322 BC 1561-1626 1694-1778 1724-1804 1896-1980 1902-1994… … … …
Aristoteles Bacon Voltaire Kant Piaget Popper
• Search for laws and
reasoning for phenomena
• Understanding the nature
of phenomena
• Progress of
knowledge of
nature (reality)
• Draw benefits from
knowledge growth
• Emancipation from
gods and beliefs
• System of
Epistemology (theory
of knowledge)
• Era of constructivism
• Falsification as
demarcation
criterion
• Birth of null
hypothesis testing
Science is understood as the human undertaking for the search of
knowledge (through systematic application of scientific methods)
àNeeds to be considered in a historical context
àIncreased understanding of scientific practice (and what science eventually is)
10. Scientific practices and research methods have
changed over time, the role of empiricism* not
Today
* Gaining knowledge through sensory experiences
384-322 BC
Le Petit Prince (1943) Large Hadron Collider
11. Scientific knowledge and practice
(Necessary) postulates for scientific practice:
• There are certain rules, principles, and norms for scientific practices
• Rationalism: Reasoning by argument / logical inference / mathematical proof
• Empiricism: Reasoning by sensory experiences (case studies, experiments,…)
• There is nothing absolute about truth
• There is a scientific community to judge about the quality of empirical studies
• Although empirical observations may be faulty, it is possible (in the long run)
to make reliable observations and to falsify incorrect statements about reality
Scientific knowledge is the portrait of
our understanding of reality (via scientific theories).
12. What is Software Engineering research?
Is it scientific?
-- What do you think? --
13. Different purposes in science
• Gaining and validating new
insights
• Often theoretical character
• Typically addressed by natural
and social sciences
Basic Science
• Applying scientific methods to
practical ends
• Often practical (& pragmatic)
character
• Typically addressed by
engineering disciplines
In software engineering research, we
• apply scientific methods to practical ends (treating design science problems)
• treat insight-oriented questions, thus, we are an insight-oriented science, too
Applied Science
14. Different purposes in science
Fundamental / basic research Applied research
Image Sources: Wikipedia (l), Apple (r)
Basic Science Applied Science
* Graph theory
(Königsberg Bridge problem)
15. What is Software Engineering research?
Is it scientific?
-- Yes, Software Engineering research is scientific! --
16. Software Engineering research
• Software engineering is about development (not production),
inherently complex, and human-centric
• Only empirical research allows us to advance our knowledge:
• Reason about the discipline and (e.g. social) phenomena involved
• Recognise and understand limits and effects of artefacts in their contexts
(e.g. by evaluating technologies, techniques, processes, models, etc.)
àExemplary questions:
• There exist over 200 documented requirements elicitation techniques
• Which one(s) work in my context?
• To which extent? Under which conditions?
• There is a new method for requirements elicitation
• What are the strengths and limitations?
17. Empirical Software Engineering
Practitioners “versus” Researchers
• Researchers usually concerned with
understanding the nature of artefacts and
their relationship in the context
• What is the effect?
• Why is it so?
• Practitioners usually concerned with
improving their engineering practices and
outcomes, using available knowledge
• What is the problem?
• What is the best solution?
The ultimate goal of Empirical Software Engineering is theory
building and evaluation to advance our body of knowledge.
In our field, theoretical and
practical relevance have often
a special symbiotic relation
18. But what is a Scientific Theory?
-- What do you think? --
19. Theories (generally speaking)
Examples (following this general notion of theory):
• “Vaccinations lead to autism”
• “Global warming is a hoax by ecologists to harm the industry”
• “Earth is flat”
• “Wearing face masks does more harm than the effects of COVID-19”
• …
A theory is a belief that there is a pattern in phenomena.
Are these theories scientific?
No: Speculations based on imagination, hopes and fears, and resulting in
opinions that often cannot be refuted (i.e. logical fallacies)
20. Scientific Theories
1. Tests
• Experiments, simulations, …
• Replications
2. Criticism
• Peer reviews / acceptance in the community
• Corroborations / extensions with further theories
A scientific theory is a belief that there is a pattern in phenomena while
having survived
1. tests against sensory experiences
2. criticism by critical peers
Note: Addresses so-called
Demarcation Problem to
distinguish science from non-
science (as per introduction by K.
Popper)
In scope of empirical research methods
(but out of scope for today)
21. Scientific Theories have…
… a purpose:
… quality criteria:
• Testability
• Empirical support / (high) level of confidence
• Explanatory power
• Usefulness to researchers and / or practitioners
• …
Analytical Explanatory Predictive Explanatory &
Predictive
Scope Descriptions and
conceptualisation,
including
taxonomies,
classifications, and
ontologies
- What is?
Identification of
phenomena by
identifying causes,
mechanisms or
reasons
- Why is?
Prediction of what
will happen in the
future
- What will happen?
Prediction of what
will happen in the
future and
explanation
- What will happen
and why?
Note: “Law” versus “Theory”
A law is a descriptive theory
without explanations (i.e. an
analytical theory)
Adapted from: Sjøberg, D., Dybå, T., Anda, B., Hannay, J. Building Theories in Software Engineering, 2010.
Further information on: https://goo.gl/SQQwxt
22. Exemplary framework for describing theories
in Software Engineering
• Constructs: What are the basic elements?
(Actors, technologies, activities, system entities, context factors)
• Propositions: How do the constructs interact?
• Explanations: Why are the propositions as specified?
• Scope: What is the universe of discourse in which the theory is
applicable?
Source (framewrk): Sjøberg, D., Dybå, T., Anda, B., Hannay, J. Building Theories in Software Engineering, 2010.
23. Exemplary framework for describing theories
in Software Engineering
• Constructs: What are the basic elements?
(Actors, technologies, activities, system entities, context factors)
• Propositions: How do the constructs interact?
• Explanations: Why are the propositions as specified?
• Scope: What is the universe of discourse in which the theory is
applicable?
Source (example): Wagner, Mendez et al. Status Quo in Requirements Engineering: A Theory and a Global Family of Surveys, TOSEM 2018.
Source (framewrk): Sjøberg, D., Dybå, T., Anda, B., Hannay, J. Building Theories in Software Engineering, 2010.
Example
Proposition:
“Structured requirements lists are documented
textually in free form or textually with constraints.”
Explanation and Scope:
“Free-form and constraint textual requirements are
sufficient for many contexts such as in agile projects
where they only act as reminders for further
conversations.”
24. Theories and hypotheses
Scientific theory
• “[…] based on hypotheses tested and verified
multiple times by detached researchers” (J. Bortz
and N. Döring, 2003)
Hypothesis
• “[…] a statement that proposes a possible
explanation to some phenomenon or event” (L.
Given, 2008)
• Grounded in theory, testable and falsifiable
• Often quantified and written as a conditional
statementEmpirical Approaches
Theory / Theories
(Tentative) Hypothesis
Falsification /
Corroboration
Theory (Pattern)
Building
Hypothesis
Building
If cause/assumption (independent variables)
then (=>) consequence (dependent variables)
Note: We don’t “test theories”, but
their consequences via hypotheses
(i.e. testable propositions)
25. From real world phenomena to theories and back:
The empirical life cycle
Empirical Approaches
Theory / Theories
(Tentative) Hypothesis
Falsification /
Corroboration
Theory (Pattern)
Building
Units of Analysis
Sampling Frame
Sampling
Hypothesis
Building
Empirical Inquiries
Induction
Inference of a
general rule
from a particular
case/result
(observation)
Abduction
(Creative) Synthesis of an
explanatory case from a general rule
and a particular result (observation)
Deduction
Application of a general rule
to a particular case,
inferring a specific result
Source: Mendez and Passoth. Empirical Software
Engineering: from Discipline to Interdiscipline, 2018.
26. From real world phenomena to theories and back:
The empirical life cycle
Empirical Approaches
Theory / Theories
(Tentative) Hypothesis
Falsification /
Corroboration
Theory (Pattern)
Building
Units of Analysis
Sampling Frame
Sampling
Hypothesis
Building
Empirical Inquiries
Induction
Inference of a
general rule
from a particular
case/result
(observation)
Abduction
(Creative) Synthesis of an
explanatory case from a general rule
and a particular result (observation)
Deduction
Application of a general rule
to a particular case,
inferring a specific result
Source: Mendez and Passoth. Empirical Software
Engineering: from Discipline to Interdiscipline, 2018.
Further reading and outlook
• Epistemological setting of Empirical Software Engineering
• Theory building and evaluation
• Challenges in Empirical Software Engineering
Preprint: https://arxiv.org/abs/1805.08302
27. Outline
• What is Empirical Software Engineering?
• Why do we need Empirical Software Engineering?
• What are the perspectives in Empirical Software Engineering?
28. Importance of
Empirical Software Engineering
We need a reliable body of knowledge about
• our discipline and (e.g. social) phenomena involved, and
• limits and effects of artefacts (technologies, techniques,
processes, models, etc.) in their practical contexts.
Building and evaluating theories is crucial
for Software Engineering research and practice
Recap
29. What are exemplary scientific
Software Engineering Theories?
-- Which ones do you know? --
30. Scientific Theories in Software Engineering
Image Source: https://www.worldwildlife.org/habitats/deserts
Disclaimer: Symbolic statement, might be slightly over exaggerated
Transferred verbatim from other
disciplines (i.e. not adopted)
Example: Theory of Gatekeeping
Isolated and vague (i.e. universal)
Example: “Frontloading efforts
decreases overall development costs”
Not backed by evidence (i.e. non-
scientific conventional wisdom)
Example: “GoTo statements are harmful”
31. Current state of evidence in Software Engineering
“[…] judging a theory by assessing the number,
faith, and vocal energy of its supporters […] basic
political credo of contemporary religious maniacs”
— Imre Lakatos, 1970
* Addressing the situation in the quantum mechanics research community, an analogy
32. Example: Goal-oriented RE
[1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016
Papers published [1]: 966
Papers including a case study [1]: 131
Studies involving practitioners [2]: 20
Practitioners actually using GORE [3]: ~ 5%
[3] Mendez et al. Naming the Pain in Requirements Engineering Initiative – www.napire.org
[2] Mavin, et al. Does Goal-Oriented Requirements Engineering Achieve its Goal?, 2017
33. Example: Goal-oriented RE
[1] Horkoff et al. Goal-Oriented Requirements Engineering: A Systematic Literature Map, 2016
Papers published [1]: 966
Papers including a case study [1]: 131
Studies involving practitioners [2]: 20
Practitioners actually using GORE [3]: ~ 5%
[3] Mendez et al. Naming the Pain in Requirements Engineering Initiative – www.napire.org
[2] Mavin, et al. Does Goal-Oriented Requirements Engineering Achieve its Goal?, 2017
34. Current state of evidence in SE
Available studies often…
• … remain isolated
• … discuss little (to no) relation
to existing evidence
• … strengthen confidence on own hopes
(and don’t report anything around)
• … don’t report negative results
Source (levels of evidence): Wohlin. An Evidence Profile for Software Engineering Research and Practice, 2013.
In most cases, we
are here
35. Conventional Wisdom in SE
“Leprechauns”: Folklore turned into facts
• Emerge from times where claims by
authorities were treated as “facts”
• Reasons manifold:
• Lack of empirical awareness
• Neglecting particularities
of practical contexts
• Neglecting relation to existing evidence
• No proper citations
(one side of the medal, over-conclusions, etc.)
• Lack of data
• …
36. Exemplary “leprechaun”:
Go To statements considered harmful
[1] Edsger Dijkstra . Go To Statement Considered Harmful. Communications of the ACM, 1968.
• Public exchange based on reasoning by argument (rationalist arguments)...
[4] Nagappan et al. An empirical study of goto in C code from GitHub repositories, 2015.
[2] Frank Rubin. ”GOTO Considered Harmful" Considered Harmful. Communications of the ACM, 1969.
[3] Donald Moore et al. " 'GOTO Considered Harmful' Considered Harmful" Considered Harmful?" Communications of the ACM, 1987.
1968 1969 1987
• … finally tackled by one empirical study nearly 50 years later.
37. Exemplary “leprechaun”:
Go To statements considered harmful
[1] Edsger Dijkstra . Go To Statement Considered Harmful. Communications of the ACM, 1968.
• Public exchange based on reasoning by argument (rationalist arguments)...
[4] Nagappan et al. An empirical study of goto in C code from GitHub repositories, 2015.
[2] Frank Rubin. ”GOTO Considered Harmful" Considered Harmful. Communications of the ACM, 1969.
[3] Donald Moore et al. " 'GOTO Considered Harmful' Considered Harmful" Considered Harmful?" Communications of the ACM, 1987.
2015
• … finally tackled by one empirical study nearly 50 years later.
“We conclude that developers
limit themselves to using goto
appropriately, [not] like Dijkstra
feared, [thus] goto does not
appear to be harmful in practice.”
38. Key Takeaway
• The current state of evidence in
Software Engineering is still weak
• Practical relevance and impact?
• Potential for transfer into practice and
adoption?
• But there is hope…
• Importance of empirical research
recognised
• Growth of a strong research
community over last two decades
39. Outline
• What is Empirical Software Engineering?
• Why do we need Empirical Software Engineering?
• What are the perspectives in Empirical Software Engineering?
41. Goals and perspectives in
Empirical Software Engineering
1. Provide tools and methods for empirical research
2. Establish strong Software Engineering theories
3. Eradicate conventional wisdom (“leprechauns”)
Various settings
• Industry settings
• Research initiatives from the community
• Publicly funded research projects
42. Example 1:
Industry setting
(as part of a Academia-industry collaboration)
Challenge: Role and Relevance of RE to Business Success unclear
Goal: “Find the proper Problem before solving it properly”
Exemplary question: “How does Customer Satisfaction depend on RE?”
43. How does Customer Satisfaction depend on RE?
• Interviews of different roles in different project settings
• Root cause analysis of customer satisfaction to phenomena in RE
44. How does Customer Satisfaction depend on RE?
• Interviews of different roles in different project settings
• Root cause analysis of customer satisfaction to phenomena in RE
45. How does Customer Satisfaction depend on RE?
• Interviews of different roles in different project settings
• Root cause analysis of customer satisfaction to phenomena in RE
46. … effective product and portfolio management, feature sizing, and project organisation
(feature planning, prioritisation, and sizing, resource and expertise planning, technology prioritisations)
… clear (dev.) process interfaces, responsibilities, and liability in distributed environments
… (regulatory) compliance: safety, security, and usability
… effective risk management and identification of moving target (and wicked problems)
(basis for “good-enough RE” and potential infusion of new RE techniques such as Design Thinking)
… increased stakeholder involvement and participation (accountability but also motivation)
… (…)
RE is a recognised basis for…
47. Example 2:
Research initiatives
Background: Requirements Engineering research often dominated by
conventional wisdom
Challenge: What research topics are of high practical relevance?
Exemplary initiatives in context of IREB working group (“IREB Research”)
NaPiRE
What are practices and problems in practical
Requirements Engineering environments?
How do practitioners perceive the relevance of
contributions by the RE research community?
How do practitioners perceive the relevance of
RE standards?
48. Naming the Pain in Requirements Engineering
Objectives: Build theory on RE practices used in industry and
on problems practitioners experience
Research method: Large-scale survey research
NaPiRE
www.napire.org
Practices Problems, causes, and effects
50. See yourself and play with the interactive data visualisation:
www.napire.org
Interactive data visualisation
51. See yourself and play with the interactive data visualisation:
www.napire.org
Filter data by various variables
52. See yourself and play with the interactive data visualisation:
www.napire.org
53. Quiz
What is the most frequently stated
problem companies face in RE when
using a rather agile software
development process model?
?
Unprecise / Unmeasurable requirements
Incomplete or hidden requirements
Communication problems
?
?
See yourself and play with the interactive data visualisation:
www.napire.org
* Prize
Ericsson Space sweater J
54. Quiz
What is the most frequently stated
problem companies face in RE when
using a rather agile software
development process model?
Unprecise / Unmeasurable requirements
Incomplete or hidden requirements
Communication problems
See yourself and play with the interactive data visualisation:
www.napire.org
* Prize
Ericsson Space sweater J
55. Example 3:
Publicly funded research projects
https://rethought.se
Background: Empirical software engineering is advancing already, but it
needs to integrate multiple competences:
• Data-centric automation
• Value orientation
• Human-centricity
Goal: Extend empirical SE to solve next generation SE problems.
57. Outline
• What is Empirical Software Engineering?
• Why do we need Empirical Software Engineering?
• What are the perspectives in Empirical Software Engineering?
58. Key Takeaways
Empirical research is important to turn
software engineering into a scientific discipline
The state of evidence in Software
Engineering is still weak
The growth of a community of empirical
researchers and practitioners is promising
We are now entering the next generation of
empirical Software Engineering research
• Many opportunities:
Courses, master theses, and
beyond• Interested in knowing
more? The doors to our
offices are always open