Slides of the SAM 2022 talk titled "History-Aware Explanations: Towards Enabling Human-in-the-Loop in Self-Adaptive Systems".

1. History-Aware Explanations: Towards
Enabling Human-in-the-Loop in
Self-Adaptive Systems
J.M. Parra-Ullauri, A. García-Domínguez, N. Bencomo, L.H. García-Paucar
SAM 2022, 24 October 2022

2. Introduction

3. Explainability for trustworthy self-adaptive systems
Software working in difficult environments
• Fixed behaviour cannot handle complex and uncertain situations
• Instead, a self-adaptive system changes its behaviour to meet its
goals as needed
• Consider self-driving cars, complex cloud deployments, data/power
networks...
Emergent behaviour needs to be explained
• Lack of trust on SAS is hindering their adoption
• Trust can be gained by allowing users to understand why the SAS
made its decisions, and to influence the decisions as desired
1

4. Are humans integrated in decision-making loops?
• Many SAS follow feedback loops: MAPE-K is a common
architecture
• How does the human get involved there?
• Can the human observe the loop?
• Can the human pitch in with their own input, e.g. driving preferences
for a self-driving car, or what to clean for a robot vaccuum? 2

5. Context: roadmap for history-aware self-adaptive systems
• This work is part of our
roadmap for history-aware SAS
• Level 1: explain decisions after
the fact
• Level 2: explain behaviour on
the fly
• Level 3 (this paper): external
agent (human) uses the
explanations to influence the
system via “effectors”
(adaptation controls)
3

6. Proposal

7. Extending MAPE-K: explanatory and feedback layer
• We propose adding an layer to
MAPE-K to integrate the
human
• Filter: collect relevant history
of the system
• Explain: use history to describe
system behaviour
• Feedback: human uses relatable
“effectors” to influence
behaviour
4

8. Extending MAPE-K: the Filter component
Log
timesliceID: EString
Agent
name: EString
Decision
name: EString
Observation
description: EString
probability: EDouble
Action
name: EString
NFR
name: EString
0..*
0..*
0..*
0..*
0..*
decisions
0..1
0..*
observations
0..1
0..*
actionTaken
0..1
observation
0..1
• The Filter component collects
information from the Monitor,
Analyze, and Plan stages: for
instance, raw sensor / decision
logs
• This information is reshaped
according to a trace
metamodel, divided into a
algorithm-independent half and
an algorithm-centric half
• Model versions are indexed by
Hawk into a temporal graph DB
5

9. Extending MAPE-K: the Explain component
Explanation construction: done in this paper
• Query the TGDB for the info to create explanations
• Time-aware EOL dialect in Hawk for formulating questions
Explanation presentation: done in this paper
• Plots (e.g. time series of key performance metrics)
• Yes/no answers (e.g. “was X always/never true?”)
• Examples of matches of a given situation
Explanation reception: future work
• Collect info on how the user reacted to the explanations
• Track what the user knows and how they perceive the system
6

10. Extending MAPE-K: the Feedback component
Abstracting away influences into “effectors”
• Users should not have to be familiar with the underlying algorithm
• The system should include effectors to allow the user to influence
the system, expressed in their terms
• User input should be recorded in system history (for accountability)
Possible effectors at Plan/Execute stages
• A SAS manages tradeoffs between competing goals: users can
influence the relative priority of those goals (e.g. performance vs
efficiency)
• Users can suggest specific actions to the SAS at the Execute stage,
triggering a reconfiguration to meet its new preference
7

11. Case study

12. Case study: Remote Data Mirroring (RDM)
• SAS manages data servers and
network links
• Two actions: switch between
minimal/redundant topologies
• Handles
cost/reliability/performance
tradeoffs, while meeting SLAs
• SLA satisfaction partially
observable over monitoring
variables (RBC, TTW, ANL)
• Uses Requirements-aware
Model POMDP for
decision-making
8

13. RDM: Filter
Filter component collects into a temporal graph DB:
• Initial stakeholder preferences about the NFRs and SLAs
• Adaptation strategies selected by SAS based on preferences, and
their impact on the observed satisfaction levels
• Situations detected at runtime, where initial preferences may drive
SAS to unsuitable adaptation strategies
9

14. RDM: Explain (construction)
1 var result : Sequence;
2 var nfrs = NFRBelief.latest.all;
3 /∗ ... ∗/
4 for (nfr in nfrs) {
5 var currentNFR = nfr.latest;
6 result.add(Sequence {
7 currentNFR.eContainer.eContainer.timesliceID,
8 currentNFR.nfr.name,
9 currentNFR.satisfied,
10 currentNFR.estimatedProbability,
11 currentNFR.eContainer.actionTaken.name,
12 aveMEC, aveMR, aveMP
13 });
14 }
15 return result;
• An EOL query is run after each
timeslice
• For each NFR, we know:
• Timeslice ID
• Name of NFR
• Considered satisfied? (Y/N)
• Satisfaction level
• Taken action (topology)
• Average MEC/MR/MP
satisfaction over the history
of the system
10

15. RDM: Explain (presentation)
• Results are fed to a custom GUI, with historic/current values
• User can track satisfaction levels over time
11

16. RDM: Feedback
• +/- buttons allows for changing relative weights for Plan stage
• Simple description: “make the algorithm focus less/more on this”
• Interactions are recorded, and algorithm still tries to meet all SLAs 12

17. RDM: example - slices 1–323
Initially, the system is working as expected by the user.
13

18. RDM: example - slices 324–645
System suffers connectivity issues, but relative weights of
reliability/cost/performance keep it on the minimal spanning topology.
14

19. RDM: example - use of effector
User decides to put more focus onto reliability, clicking on “+” under MR:
GUI runs an EOL query, and shows a dialog with a quick summary of the
current situation before asking for confirmation.
User confirms the action, and MR weight is increased.
15

20. RDM: example - slices 646+
System switches to RT after putting more weight on reliability, which
does impact cost/performance but stays within SLAs.
16

21. RDM: example - impact of change
Before update of
preferences
After update of
preferences
• Before the preferences were
updated, average satisfaction of
MR was below SLA threshold
• After the update, MR
satisfaction improves at the
expense of the others, but all
SLAs are still met
17

22. What we have done so far
Extension to MAPE-K
• We proposed involving humans in the MAPE-K feedback loop, by
adding an explanatory & feedback layer
• Layer made up of Filter, Explain, and Feedback components
Implementation of E&F layer
• Filter: reshape to trace model + index into temporal graph DB
• Explain: query temporal graph + generate plots/answers
• Feedback: effectors for users to influence Plan/Execute
Case study: RDM
• Applied E&F layer to the RDM SAS
• Custom GUI with system-specific effectors
• Simulated scenario of preference readjustment

23. What’s next?
Explanation receptions
• Explanations currently targeted SAS developers
• SAS users will need a different style of explanations
• Follow-up study on explanation efficacy and appropriateness
(Opportunity-Willigness-Capability), and effectors’ impact on
trustworthiness
Further lines of work
• Currently ongoing: non-human consumers of explanations (e.g.
external system optimising AI/ML hyper-parameters)
• Additional case studies on other SAS
• Other explanations besides factual ones, e.g. formulating hypotheses
and producing evidence supporting/rejecting them
• Distributed SAS (→ distributed trace models)

24. Thank you!
@antoniogado / a.garcia-dominguez@york.ac.uk
j.parra-ullauri@aston.ac.uk