Survey on Software Defect Prediction (PhD Qualifying Examination Presentation)
Paderborn
1. Challenges of self-adaptive software
the fading boundary between
development time and run time
9th International Heinz Nixdorf Symposium
Carlo Ghezzi
Politecnico di Milano
Deep-SE Group @ DEI-PoliMI
1
2. The vision
• World fully populated by computationally rich
devices offering services (disappearing computer)
– appliances, sensors/actuators, ... “things”
• Cyber-physical systems
• Mobility
• Situation-aware computing
– new “services” built dynamically in a situation-dependent
manner
• Continuously running systems
– need to evolve while they offer service
2
3. The challenge
• Change and flexibility adversary of
dependability
• Can they be reconciled through sound design
methods?
3
4. The machine and the world
Domain
properties
(assumptions)
Goals Specification
Requirements
P. Zave and M. Jackson. Four dark corners of requirements engineering.
ACM Trans. Softw. Eng. Methodol., 6(1):1–30, 1997. 4
5. Dependability arguments
Assume that a rigorous (formal) representation is given for
– R = requirements
– S = specification
– D = domain assumptions
if S and D are all satisfied and consistent, it is necessary to
prove
– S, D |= R
5
6. Change comes into play
• Changes in goals/requirements
• Changes in domain assumptions
– Usage context
• request profiles
– Physical context
• space, time, …
– Computational context
• external components/services (multiple ownership)
‣ systems increasingly built out of parts that are developed, maintained,
and even operated by independent parties
‣ no single stakeholder oversees all parts, which may change
independently
‣ yet by assembling the whole one commits to achieving certain goals
6
7. Changes may affect dependability
• Changes may concern
• R
• D
• We can decompose D into Df and Dc
– Df is the fixed/stable part
– Dc is the changeable part
We need to detect changes to Dc
and make changes to S to keep satisfying R
(self-)adaptation (vs. evolution)
7
8. Change revisited
• Change recognized as a crucial problem since the 1970’s
(see work by M. Lehman)
• Traditionally managed off-line: software maintenance
• What is new here
– the unprecedented degree of change
– the request that software responds to changes while
the system is running (continuously running systems),
possibly in a self-managed manner
8
9. A paradigm change
• Conventional separation between development time
and run time is blurring
• Models + requirements need to be kept + updated at
run time (systems need to be reflective)
• Continuous verification must be performed to detect
the need for adaptation
F(S=OK)
Env
R. Calinescu, C. Ghezzi, M. Kwiatkokwska, R. Mirandola, “Self-adaptive software
9
needs quantitative verification at runtime”, Comm. ACM, Sept. 2012
10. Zoom-in
A framework for (self) adaptation
• [ICSE 2009] I. Epifani, C. Ghezzi, R. Mirandola, G. Tamburrelli, "Model
Evolution by Run-Time Parameter Adaptation”
• [RE 2009] C. Ghezzi, G. Tamburrelli, "Reasoning on Non Functional
Requirements for Integrated Services”
• [FSE 2010] I. Epifani, C. Ghezzi, G. Tamburrelli, "Change-Point
Detection for Black-Box Services”
10
11. Specific focus
• Non-functional requirements
– reliability, performance, energy consumption, cost, …
• Quantitatively stated in probabilistic terms
• Dc decomposed into Du , Ds
– Du = usage profile
– Ds = S1 ∧ .... ∧ Sn Si assumption on i-th service
?
Hard to estimate at
? ? ?
System under
design time +
development ?
very likely to change
?
?
at run time
?
11
12. Our approach in a nutshell
Reqs + Formalization
0
Learning & update
Initial
Assumptions 1 E
Implementation Monitoring
Execution
12
13. Models
• Different models provide different viewpoints from
which a system can be analyzed
• Focus on non-functional properties and quantitative
ways to deal with uncertainty
• Use of Markov models
– DTMCs for reliability
– CTMCs for performance
– Reward DTMCs for energy/cost
13
14. Quantitative modelling
• Operational model: state machine with probabilities on
transitions (DTMC)
• Success/failure states as final
• Requirements expressed as properties in PCTL (probabilistic
extension of CTL)
• typically: reachability properties
• P>0.8 [◊(system state = success)]
• Excellent tools exist to perform verification via model
checking
– PRISM (Kwiatkowska et al.)
• http://www.prismmodelchecher.org/
– MRMC (Katoen at al.)
• http://www.mrmc-tool.org/trac/
14
15. The approach in in action:
e-commerce service composition
Users
classified as BigSpender
or SmallSpender based on their
usage profile.
3 probabilistic requirements:
R1: “Probability of success is > 0.8”
R2: “Probability of a ExpShipping failure for a user recognized as
BigSpender < 0.035”
R3: “Probability of an authentication failure is less then < 0.06”
15
17. DTMC model
Property check via model checking
R1: “Probability of success is > 0.8” 0.84
R2: “Probability of a ExpShipping failure for a user recognized as
BigSpender < 0.035” 0.031
R3: “Probability of an authentication failure is less then < 0.06” 0.056
17
18. What happens at run time?
• We monitor the actual behavior
• A statistical (Bayesian) approach estimates the updated DTMC
matrix (posterior) given run time traces and prior transitions
• Boils down to the following updating rule
A-priori Knowledge A-posteriori Knowledge
18
19. Model @ run time
• A detected violation of a requirements does not mean that the
violation actually occurred in reality (i.e., experienced by user)
• Model analysis has a predictive nature
0.067
Predicted
failure!
Probability of a ExpShipping failure for a user recognized a BigSpender < 0.035
19
20. Reflective run time environments
• Traditionally software engineering has been
mostly concerned with development time
• The result is code that simply needs to be run
(Self-)adaptive software requires much more
- must be able to reason at run time about itself and
the environment
✓ models
✓ goals and requirements
✓ strategies
must be available at runtime
20
21. Run-time agility, incrementality
• Agility taken to extremes
- time boundaries shrink
✓ constrained by real-time requirements
• Verification approaches must be re-visited
- they must be incremental
Given S system (model), P property to verify for S
Change = new pair S’, P’
Incremental verification reuses part of the proof of
S against P to verify S’ against P’
21
22. Incrementality by parameterization
g m
d i
• Requires anticipation of changing parameters
ra
• The model is partly numeric and partly symbolic
a
p
• Evaluation of the verification condition requires
m st r
partial evaluation (mixed numerical/symbolic
o ir
m k f ate
processing)
g o
n o
i C p l
• Result is a formula (polynomial for reachability on
r k
DTMCs)
-u
o
• Evaluation at run time substitutes actual values to
W arm
symbolic parameters
W
[ICSE 2011] A. Filieri, C. Ghezzi, G. Tamburrelli, “Run-Time Efficient Probabilistic Model Checking”
[FAC 2012] A Filieri, C. Ghezzi, G. Tamburrelli,, “A formal approach to adaptive software: continuous
assurance of non-functional requirements”, Formal Aspects of Computing, vol. 4, n. 2, 2012
22
23. Working mom paradigm
Design-Time
Partial
(offline)
0
evaluation
1 E
Parameter
Run-Time
(online)
values
Analyzable properties: reliability, costs (e.g., energy consumption)
23
24. Back to example
Returning Search Buy ExpShipping FailedExpSh
1
1 1 4 7 0.5 10 x
0.05 13 1
0.2
0.3 0.95
0.35 1-x
CheckOut
Login Profiler FailedLg FailedChk Logout Success
1
0.97 1 1
0 2 5 8 0.1 11 0.9 14 0.97 16
0.03
0.65 0.95 0.03
0.25
1
1
3 1 6 9 0.6 12 0.05 15
0.15
NewCustomer Search Buy NrmShipping FailedNrmSh
Design time evaluation has very high complexity
Run time evaluation is extremely efficient
24
25. Zoom-in
Control-theory based
self adaptation
[ASE 2011] A. Filieri, C. Ghezzi, A. Leva, M. Maggio, “Self-Adaptive Software Meets
Control Theory: A Preliminary Approach Supporting Reliability Requirements”
[SEAMS 2012] A. Filieri, C. Ghezzi, A. Leva, M. Maggio, "Reliability-driven dynamic
binding via feedback control"
25
26. First approach
• Tune software through its model via feedback control loop
• Formally prove controller’s capabilities (error reduction,
convergence, ...)
Dynamic
Parametric Dynamic controllable
behavioral controllable model
model model
Control the model, control the software
Controller
26
27. Reliability-driven
Dynamic-binding
C0
p 1-p PI Controllers
C1 C2
Abstract interface p 1-p p 1-p
Concrete
implementations
[SEAMS2012]
27
28. Zoom-in
Dynamic software
update
[ESEC/FSE 2011] X, Ma, L, Baresi, C, Ghezzi,V. Panzica La Manna, and J. Lu,
“Version-consistent Dynamic Reconfiguration of Component-based Distributed
Systems”
[SEAMS 2012] C. Ghezzi, J. Greenyer,V. Panzica La Manna, "Synthesizing
Dynamically Updating Controllers from Changes in Scenario-based Specifications"
28
29. Goal
• Support run-time changes to the structure and
behavior of a running software
Low$
Timeliness$ Safety$
Disruption$
• Two approaches:
• Module level
• Model level
29
31. A
Speci(ication-‐oriented
Perspective
Speci5ication
Requirements:
The
set
of
runs
(events
Requirements
Environment sequences)
allowed
in
Assumptions the
system.
“Something
that
the
system
must
(not)
do”
Assumptions:
The
set
of
runs
we
assume
can
happen
in
the
system.
“Something
that
the
Environment environment
will
(not)
11
32. A
Speci(ication-‐oriented
Perspective
Speci5ication NEW
Speci5ication
Environment Environment
Requirements Requirements
Assumptions Assumptions
Environment
12
33. A
Speci(ication-‐oriented
Perspective
Speci5ication NEW
Speci5ication
Environment Environment
Requirements Requirements
Assumptions Assumptions
Environment
13
34. A
Speci(ication-‐oriented
Perspective
Speci5ication NEW
Speci5ication
Environment Environment
Requirements Requirements
Assumptions Assumptions
Challenges:
1)
In
which
state
a
running
system
can
be
safely
updated
to
satisfy
the
new
speci5ication?
Environment
14
35. A
Speci(ication-‐oriented
Perspective
Speci5ication NEW
Speci5ication
Environment Environment
Requirements Requirements
Assumptions Assumptions
Challenges:
1)
In
which
state
a
running
system
can
be
safely
updated
to
satisfy
the
new
speci5ication?
2)
How
the
dynamically
updating
system
can
be
automatically
derived?
Environment
15
36. Contribution
Challenges: 1.General
formal
de5inition
of
1)
In
which
state
a
correct
dynamic
updates
running
system
can
be
from
changes
in
the
safely
updated
to
satisfy
speci5ication.
the
new
speci5ication?
2)
How
the
dynamically
2.Approach
for
automatically
updating
system
can
be
synthesizing
a
dynamically
automatically
derived? updating
controller
from
changes
in
scenario-‐based
speci5ications.
17
37. Conclusions and future work
• (Self-)adaptation is needed
• It requires a paradigm shift
• Run-time environments must become semantically
rich
• Run-time reasoning must be supported, not just
execution
• Continuous change and the quest for incrementality
• Opportunities for interdisciplinary
research
• Challenges from real scenarios
37
