Confidence Monitoring and Composition for Dynamic Assurance of Learning-Enabled Cyber-Physical Systems

•

0 likes•57 views

A presentation at the Hot Topics Day "Application of DARPA Assured Autonomy Program Technologies to Autonomous Learning-Enabled Real-Time Systems" at RTSS 2020 http://2020.rtss.org/workshop-darpa

Technology

Confidence Monitoring and Composition for
Dynamic Assurance of Learning-Enabled CPS
Ivan Ruchkin, Matthew Cleaveland, Oleg Sokolsky, Insup Lee
University of Pennsylvania
DARPA Hot Topics Day
IEEE Real-Time Systems Symposium
December 1, 2020

2
Assurance confidence
Req: no obstacle collisions
Req: no pipeline loss

3
Assurance confidence
Detection confidence
Detection confidence
Dynamics confidence
Req: no obstacle collisions
Req: no pipeline loss

4
Problem
●
Assurance confidence monitoring
– Compute confidence in the guarantees of safety reqs
– Given confidence measures from run-time monitors
Challenge:
Safety reqs ←?→ run-time monitors

5
Problem
●
Assurance confidence monitoring
– Compute confidence in the guarantees of safety reqs
– Given confidence measures from run-time monitors
●
Challenge:
– Guarantees ←?→ run-time monitors

21
Assumption effect analysis
Assumption required for
R1: no collisions
Mode1:
obstacle detected
Mode2:
obstacle not detected
A1: “Reported distance is
within 1m of true distance”
A2: “No false-negative
obstacle detections”
A3: “The obstacle is >10m
away”
A4: “Vehicle follows the
given dynamics equations”

22
Assumption effect analysis
Assumption required for
R1: no collisions
Mode1:
obstacle detected
Mode2:
obstacle not detected
A1: “Reported distance is
within 1m of true distance” Y N
A2: “No false-negative
obstacle detections”
A3: “The obstacle is >10m
away”
A4: “Vehicle follows the
given dynamics equations”

23
Assumption effect analysis
Assumption required for
R1: no collisions
Mode1:
obstacle detected
Mode2:
obstacle not detected
A1: “Reported distance is
within 1m of true distance” Y N
A2: “No false-negative
obstacle detections” N Y
A3: “The obstacle is >10m
away”
A4: “Vehicle follows the
given dynamics equations”

24
Assumption effect analysis
Assumption required for
R1: no collisions
Mode1:
obstacle detected
Mode2:
obstacle not detected
A1: “Reported distance is
within 1m of true distance” Y N
A2: “No false-negative
obstacle detections” N Y
A3: “The obstacle is >10m
away” Y Y
A4: “Vehicle follows the
given dynamics equations” Y Y

25
Assumption effect analysis
Assumption required for
R1: no collisions
Mode1:
obstacle detected
Mode2:
obstacle not detected
A1: “Reported distance is
within 1m of true distance” Y N
A2: “No false-negative
obstacle detections” N Y
A3: “The obstacle is >10m
away” Y Y
A4: “Vehicle follows the
given dynamics equations” Y Y
Composition logic: (Mode1 → A1 A3 A4) (Mode2 → A2 A3 A4)∧ A3 ∧ A4) ∧ (Mode2 → A2 ∧ A3 ∧ A4) ∧ A3 ∧ A4) ∧ (Mode2 → A2 ∧ A3 ∧ A4) ∧ A3 ∧ A4) ∧ (Mode2 → A2 ∧ A3 ∧ A4) ∧ A3 ∧ A4) ∧ (Mode2 → A2 ∧ A3 ∧ A4) ∧ A3 ∧ A4) ∧ (Mode2 → A2 ∧ A3 ∧ A4)

30
●
Random variables:
– Reported distance (RD)
– True distance (TD)
An assumption is an assertion over variables:
A1: | OD – TD | ≤ 1m
2: OD = ∞ → TD = ∞
Goal: compute probabilistic queries over assns given observations
E.g., P( f(A1, A2) | OD ), where f is a given Boolean function
Probabilistic modeling of assumptions

31
●
Random variables:
– Reported distance (RD)
– True distance (TD)
●
An assumption is an assertion over variables:
– A1: | RD – TD | ≤ 1m (bounded error)
– A2: RD = ∞ → TD = ∞ (no false negatives)
Goal: compute probabilistic queries over assns given observations
E.g., P( f(A1, A2) | OD ), where f is a given Boolean function
Probabilistic modeling of assumptions

32
●
Random variables:
– Reported distance (RD)
– True distance (TD)
●
An assumption is an assertion over variables:
– A1: | RD – TD | ≤ 1m (bounded error)
– A2: RD = ∞ → TD = ∞ (no false negatives)
●
Goal: compute probabilistic queries over assns given observations
– P( f(A1, A2) | RD ), where f is a given Boolean function
Probabilistic modeling of assumptions

37
Assumption monitoring with ReCal/CAM
P(A1) = 0.78
P(A2) = 0.97
P(A1 V A2) = 0.98

41
Using our prior composition technique based on symbolic manipulation:
Ruchkin, Sokolsky, Weimer, Hedaoo, Lee. Compositional Probabilistic Analysis of
Temporal Properties over Stochastic Detectors, EMSOFT 2020.

42
Future work
●
Application to several UUV monitors
●
Guarantees of confidence accuracy/conservatism
●
Second-order assumptions (of our assumption models)

43
Summary
●
Assurance confidence requires closing a gap:
– Safety guarantees ←?→ run-time monitors
●
We address this problem in four steps:
– Offline verification
– Assumption effect analysis
– Assumption monitoring
– Composition of monitors

Presented at Formal Methods in Outer Space Essays Dedicated to Klaus Havelund on the Occasion of His 65th Birthday. Link to the paper: https://link.springer.com/chapter/10.1007/978-3-030-87348-6_8 Abstract: Design-time approaches to safety assurance for autonomous systems are limited because they must rely on assumptions about the behaviors of learned components in previously unseen environments. These assumptions may be violated at run time, thus invalidating the guarantees produced at design time. To overcome this limitation, we propose to complement design-time assurance with run-time monitoring that calculates the confidence that those assumptions are satisfied and, therefore, design-time guarantees continue to hold. As the first step in our vision, we elicit the logical relationship between assumption violations and safety violations. Then, we develop a probabilistic confidence monitor for each design-time assumption. Finally, we compose these assumption monitors based on their logical relation to safety violations, producing a system-wide assurance monitor. Our vision is illustrated with a case study of an autonomous underwater vehicle that performs pipeline inspection.

Confidence Composition (CoCo) for Dynamic Assurance of Learning-Enabled Auton...

Ivan Ruchkin

JacobSiegler_Research_2015Jacob Siegler

Dynamic Music Emotion Recognition Using State-Space Models

multimediaeval

This paper describes the temporal music emotion recognition system developed at the University of Aizu for the Emotion in Music task of the MediaEval 2014 benchmark evaluation campaign. The arousal-valence trajectory prediction is cast as a time series filtering task and is modeled by a state-space models. These models include standard linear model (Kalman filter) as well as novel non-linear, non-parametric Gaussian Processes based dynamic system. The music signal was parametrized using standard features extracted with the Marsyas toolkit. Based on the preliminary results obtained from small random validation set, clear advantage of any feature or model could not be observed. http://ceur-ws.org/Vol-1263/mediaeval2014_submission_28.pdf

SERENE 2014 Workshop: Paper "Verification and Validation of a Pressure Contro...

SERENEWorkshop

Anderson-Darling Test and ROC Curve

Kavi

The Anderson–Darling test is a statistical test of whether a given sample of data is drawn from a given probability distribution. In its basic form, the test assumes that there are no parameters to be estimated in the distribution being tested, in which case the test and its set of critical values is distribution-free. This is an implementation of Research paper titled - Spectrum Sensing in Cognitive Radio Using Goodness of Fit Testing by Wang, Yang, Zhao and Zhang

Tachometers.pptx

PMTIndustry

Compositional Probabilistic Analysis of Temporal Properties over Stochastic D...

Ivan Ruchkin

Authors: Ivan Ruchkin, Oleg Sokolsky, James Weimer, Tushar Hedaoo, and Insup Lee Abstract: Run-time monitoring is a vital part of safety-critical systems. However, early-stage assurance of monitoring quality is currently limited: it relies either on complex models that might be inaccurate in unknown ways, or on data that would only be available once the system has been built. To address this issue, we propose a compositional framework for modeling and analysis of noisy monitoring systems. Our novel 3-value detector model uses probability spaces to represent atomic (non-composite) detectors, and it composes them into a temporal logic-based monitor. The error rates of these monitors are estimated by our analysis engine, which combines symbolic probability algebra, independence inference, and estimation from labeled detection data. Our evaluation on an autonomous underwater vehicle found that our framework produces accurate estimates of error rates while using only detector traces, without any monitor traces. Furthermore, when data is scarce, our approach shows higher accuracy than non-compositional data-driven estimates from monitor traces. Thus, this work enables accurate evaluation of logical monitors in early design stages before deploying them. Original paper: http://dx.doi.org/10.1109/TCAD.2020.3... Supplementary materials: https://www.researchgate.net/publication/342993188_Supplementary_Materials_for_Compositional_Probabilistic_Analysis_of_Temporal_Properties_over_Stochastic_Detectors Video presentation: https://www.youtube.com/watch?v=q-9aHAQwd9Q Source code and data: https://github.com/bisc/prob-comp-asst

Presented at the following seminar series in Spring 2022: - University of Arizona, SIE Department - San Diego State University, CS Department - University of Kansas, EECS Department - Tulane University, CS Department - Auburn University, CSSE Department - University of Hawaii, ICS Department - Virginia Tech, ISE Department - Santa Clara University, CS Department - University of Kentucky, CS Department - Indiana University - Purdue University Indianapolis, CIS Department - Michigan State University, CSE Department - University of Florida, ECE Department - Florida Atlantic University, EECS Department Abstract: From autonomous vehicles to smart grids, cyber-physical systems (CPS) play an increasingly important role in today's society. Often, CPS operate autonomously in highly critical settings, and thus it is imperative to engineer these systems to be safe and trustworthy. However, it is particularly difficult to do so due to CPS heterogeneity -- the high diversity of components and models used in these systems. This heterogeneity substantially contributes to fragmented, incoherent assurance as well as to inconsistencies between different models of the system. This talk will present two complementary techniques for overcoming CPS heterogeneity: confidence composition and model integration. The former technique combines heterogeneous confidence monitors to produce calibrated estimates of the run-time probability of safety in CPS with machine learning components. The latter technique discovers inconsistencies between heterogeneous CPS models using a logic-based specification language and a verification algorithm. The application of these techniques will be demonstrated on an unmanned underwater vehicle and a power-aware service robot. These techniques serve as stepping stones towards the vision of engineering autonomous systems that are aware of their own limitations.

tracking.ppt

BangalirecipeLaboni

Verify-then-Monitor: Calibration Guarantees for Safety Confidence

Ivan Ruchkin

Presented at the Sixth International Workshop on Design Automation for Cyber-Physical Systems (DACPS), co-located with the Design Automation Conference (DAC) 2023. Abstract: Autonomous cyber-physical systems (CPS) are increasingly deployed in complex and safety-critical environments. To help CPS interact with such environments, learning-enabled components, such as neural networks, often implement perception and control functions. Unfortunately, the complexity of the environments and learning components is a major challenge to ensuring the safety of CPS. An emerging assurance paradigm prescribes verifying as much of the CPS as possible at design time - and then monitoring the probability of safety at run time in case of unexpected situations. How can we guarantee that the monitor produces a probability that is well-calibrated to the true chance of safety? This talk will overview our recent answers in two settings. The first combines Bayesian filtering with probabilistic model checking of Markov decision processes. The second is based on confidence monitoring of assumptions behind closed-loop neural-network verification.

Generalized Property-Directed Reachability for Hybrid Systems (presented in V...

Kohei Suenaga

Illumination Independent Marker Tracking using Cross-Ratio Invariance

Vincent Agnus

VRST2014 presentation Marker tracking is used in numerous applications. Depending on the context and its constraints, tracking accuracy can be a crucial component of the application. In this paper, we firstly highlight that the tracking accuracy depends on the illumination, which is usually not controlled in most applications. Particularly, we show how corner detection can shift of several pixels when light power or background context change, even if the camera and the marker are static in the scene. Then, we propose a method, based on the cross ratio invariance, that allows to re-estimate the corner extraction so that the cross ratio of the marker model corresponds to the one computed from the extracted corners in the image. Finally, we show on real data that our approach improves the tracking accuracy, particularly along the camera depth axis, up to several millimeters, depending on the marker depth.

motor_2.ppt

VikramJit13

"Bridge condition evaluation using LDVs installed on a vehicle" presented at ...

TRUSS ITN

In this paper, a structural health monitoring approach is proposed involving Laser Doppler Vibrometers (LDVs) installed on a vehicle. Relative velocities are measured to obtain the Rate of Instantaneous Curvature of the velocity (RIC). Standard deflection curvature is shown to be sensitive to local damage. Instantaneous Curvature (IC) is likewise sensitive but calculated using measurements provided from a vehicle. RIC is obtained using the first derivative of IC with respect to time.A damage indicator obtained from RIC, the Difference Ratio, is tested both in noise-free and noisy conditions.

Implementation of the fully adaptive radar framework: Practical limitations

Luis Úbeda Medina

Protection of digital watermarking based on SVD against false positive detect...

wassila belferdi

The singular value decomposition (SVD) is drawing the attention as a new transform technique for robust digital watermarking. However; several SVD based watermarking algorithms have a serious limitation causing the false positive detection vulnerability. In this paper, we propose watermarking system to counterattack the weakness above mentioned; the proposed technique use the sharing secret principle to calculate secret keys to be embedded in the singular vectors of the watermark as a control parameter, this step invariably ensures that watermark extraction from the watermarked image, using a modified matrix, is not possible, even if it was attacked.

Tracking[1]

mervebayrak

icacis2012.pptx

WslaBlf

icacis2012.pdf

WslaBlf

Extended Kalman observer based sensor fault detection

IJECEIAES

This article discusses the Kalman observer based fault detection approach. The calculation of the residues can detect faults, but if there are noises, uncertainties become very important. To reduce the influence of these noises, a calculation of the instantaneous energy of the residues gave a better precision. The Kalman observer was used to estimate system performance and eliminate unknown noise and external disturbances. Instantaneous Power Calculation (IPCFD) based fault detection can detect potential sensor faults in hybrid systems. The effectiveness of the proposed approach is illustrated by the main application.

Confidence Composition for Monitors of Verification Assumptions

Ivan Ruchkin

Presented at 13th ACM/IEEE Intl. Conf. on Cyber-Physical Systems, part of CPS-IoT Week, on May 4, 2022. Presentation video: https://youtu.be/nnhcUhih-vQ Abstract: Closed-loop verification of cyber-physical systems with neural network controllers offers strong safety guarantees under certain assumptions. It is, however, difficult to determine whether these guarantees apply at run time because verification assumptions may be violated. To predict safety violations in a verified system, we propose a three-step confidence composition (CoCo) framework for monitoring verification assumptions. First, we represent the sufficient condition for verified safety with a propositional logical formula over assumptions. Second, we build calibrated confidence monitors that evaluate the probability that each assumption holds. Third, we obtain the confidence in the verification guarantees by composing the assumption monitors using a composition function suitable for the logical formula. Our CoCo framework provides theoretical bounds on the calibration and conservatism of compositional monitors. Two case studies show that compositional monitors are calibrated better than their constituents and successfully predict safety violations.

reportmurali vnv

DSD-INT 2020 Radar rainfall estimation and nowcasting

Deltares

Adaptive relaying

Surabhi Vasudev

Signal Processing Algorithms for MIMO Radar

sansam77

"Pavement condition measurement at high speed using a TSD" presented at ESREL...

TRUSS ITN

Abstract: The aim of this paper is to present the latest developments in the use of an instrumented vehicle called the Traffic Speed Deflectometer (TSD). A large axle load is applied to the pavement under the TSD. The deflection caused by this axle load is measured using several Doppler lasers. In the first step, the velocity of the deflection of the pavement is measured which can be shown to be proportional to the slope of the deformed profile. The pavement deflection is calculated in the second step using an integration model. A Winkler model is used to simulate the pavement behaviour under the axle load and the TSD is represented as a half-car model. The TSD is shown to be an effective tool for pavement damage detection.

Language-Enhanced Latent Representations for Out-of-Distribution Detection in...

Ivan Ruchkin

Presented by Zhenjiang Mao at the Robot Trust for Symbiotic Societies (RTSS) Workshop, ICRA 2024. Out-of-distribution (OOD) detection is essential in autonomous driving, to determine when learning-based components encounter unexpected inputs. Traditional detectors typically use encoder models with fixed settings, thus lacking effective human interaction capabilities. With the rise of large foundation models, multimodal inputs offer the possibility of taking human language as a latent representation, thus enabling language-defined OOD detection. In this paper, we use the cosine similarity of image and text representations encoded by the multimodal model CLIP as a new representation to improve the transparency and controllability of latent encodings used for visual anomaly detection. We compare our approach with existing pre-trained encoders that can only produce latent representations that are meaningless from the user's standpoint. Our experiments on realistic driving data show that the language-based latent representation performs better than the traditional representation of the vision encoder and helps improve the detection performance when combined with standard representations.

Poster: Zero-shot Safety Prediction for Autonomous Robots with Foundation Wo...

Ivan Ruchkin

This poster was presented by Zhenjiang Mao at ICRA 2024. Related paper: https://arxiv.org/abs/2404.00462 Abstract: A world model creates a surrogate world to train a controller and predict safety violations by learning the internal dynamic model of systems. However, the existing world models rely solely on statistical learning of how observations change in response to actions, lacking precise quantification of how accurate the surrogate dynamics are, which poses a significant challenge in safety-critical systems. To address this challenge, we propose foundation world models that embed observations into meaningful and causally latent representations. This enables the surrogate dynamics to directly predict causal future states by leveraging a training-free large language model. In two common benchmarks, this novel model outperforms standard world models in the safety prediction task and has a performance comparable to supervised learning despite not using any data. We evaluate its performance with a more specialized and system-relevant metric by comparing estimated states instead of aggregating observation-wide error.

Similar to Confidence Monitoring and Composition for Dynamic Assurance of Learning-Enabled Cyber-Physical Systems

Conen 442 module1b: Traffic Studies

Wael ElDessouki

Overcoming Heterogeneity in Autonomous Cyber-Physical Systems

Ivan Ruchkin

tracking.ppt

BangalirecipeLaboni

Verify-then-Monitor: Calibration Guarantees for Safety Confidence

Ivan Ruchkin

Generalized Property-Directed Reachability for Hybrid Systems (presented in V...

Kohei Suenaga

Illumination Independent Marker Tracking using Cross-Ratio Invariance

Vincent Agnus

motor_2.ppt

VikramJit13

"Bridge condition evaluation using LDVs installed on a vehicle" presented at ...

TRUSS ITN

Implementation of the fully adaptive radar framework: Practical limitations

Luis Úbeda Medina

Protection of digital watermarking based on SVD against false positive detect...

Tracking[1]

icacis2012.pptx

icacis2012.pdf

Extended Kalman observer based sensor fault detection

IJECEIAES

Confidence Composition for Monitors of Verification Assumptions

Ivan Ruchkin

reportmurali vnv

DSD-INT 2020 Radar rainfall estimation and nowcasting

Deltares

Adaptive relaying

Surabhi Vasudev

Signal Processing Algorithms for MIMO Radar

sansam77

"Pavement condition measurement at high speed using a TSD" presented at ESREL...

TRUSS ITN

Similar to Confidence Monitoring and Composition for Dynamic Assurance of Learning-Enabled Cyber-Physical Systems (20)

Conen 442 module1b: Traffic Studies

Overcoming Heterogeneity in Autonomous Cyber-Physical Systems

tracking.ppt

Verify-then-Monitor: Calibration Guarantees for Safety Confidence

Generalized Property-Directed Reachability for Hybrid Systems (presented in V...

Illumination Independent Marker Tracking using Cross-Ratio Invariance

motor_2.ppt

"Bridge condition evaluation using LDVs installed on a vehicle" presented at ...

Implementation of the fully adaptive radar framework: Practical limitations

Protection of digital watermarking based on SVD against false positive detect...

Tracking[1]

icacis2012.pptx

icacis2012.pdf

Extended Kalman observer based sensor fault detection

Confidence Composition for Monitors of Verification Assumptions

report

DSD-INT 2020 Radar rainfall estimation and nowcasting

Adaptive relaying

Signal Processing Algorithms for MIMO Radar

"Pavement condition measurement at high speed using a TSD" presented at ESREL...

More from Ivan Ruchkin

Language-Enhanced Latent Representations for Out-of-Distribution Detection in...

Ivan Ruchkin

Poster: Zero-shot Safety Prediction for Autonomous Robots with Foundation Wo...

Ivan Ruchkin

Curating Naturally Adversarial Datasets for Learning-Enabled Medical Cyber-Ph...

Ivan Ruchkin

Presented by Sydney Pugh at the 15th ACM/IEEE International Conference on Cyber-Physical Systems. Abstract: Deep learning models have shown promising predictive accuracy for time-series healthcare applications. However, ensuring the robustness of these models is vital for building trustworthy AI systems. Existing research predominantly focuses on robustness to synthetic adversarial examples, crafted by adding imperceptible perturbations to clean input data. However, these synthetic adversarial examples do not accurately reflect the most challenging real-world scenarios, especially in the context of healthcare data. Consequently, robustness to synthetic adversarial examples may not necessarily translate to robustness against naturally occurring adversarial examples, which is highly desirable for trustworthy AI. We propose a method to curate datasets comprised of natural adversarial examples to evaluate model robustness. The method relies on probabilistic labels obtained from automated weakly-supervised labeling that combines noisy and cheap-to-obtain labeling heuristics. Based on these labels, our method adversarially orders the input data and uses this ordering to construct a sequence of increasingly adversarial datasets. Our evaluation on six medical case studies and three non-medical case studies demonstrates the efficacy and statistical validity of our approach to generating naturally adversarial datasets.

Repairing Learning-Enabled Controllers While Preserving What Works

Ivan Ruchkin

Presented at the 15th ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS 2024). Abstract: Learning-enabled controllers have been adopted in various cyber-physical systems (CPS). When a learning-enabled controller fails to accomplish its task from a set of initial states, researchers leverage repair algorithms to fine-tune the controller's parameters. However, existing repair techniques do not preserve previously correct behaviors. Specifically, when modifying the parameters to repair trajectories from a subset of initial states, another subset may be compromised. Therefore, the repair may break previously correct scenarios, introducing new risks that may not be accounted for. Due to this issue, repairing the entire initial state space may be hard or even infeasible. As a response, we formulate the Repair with Preservation (RwP) problem, which calls for preserving the already-correct scenarios during repair. To tackle this problem, we design the Incremental Simulated Annealing Repair (ISAR) algorithm, which leverages simulated annealing on a barriered energy function to safeguard the already-correct initial states while repairing as many additional ones as possible. Moreover, formal verification is utilized to guarantee the repair results. Case studies on an Unmanned Underwater Vehicle (UUV) and OpenAI Gym Mountain Car (MC) show that ISAR not only preserves correct behaviors from previously verified initial state regions, but also repairs 81.4% and 23.5% of broken state spaces in the two benchmarks. Moreover, the average signal temporal logic (STL) robustnesses of the ISAR repaired controllers are larger than those of the controllers repaired using baseline methods.

Poster: Conservative Safety Monitors of Stochastic Dynamical Systems

Ivan Ruchkin

Poster for the paper presented at the NASA Formal Methods Symposium (NFM) by Matthew Cleaveland on May 16, 2023. Abstract: Generating accurate runtime safety estimates for autonomous systems is vital to ensuring their continued proliferation. However, exhaustive reasoning about future behaviors is generally too complex to do at runtime. To provide scalable and formal safety estimates, we propose a method for leveraging design-time model checking results at runtime. Specifically, we model the system as a probabilistic automaton (PA) and compute bounded-time reachability probabilities over the states of the PA at design time. At runtime, we combine distributions of state estimates with the model checking results to produce a bounded time safety estimate. We argue that our approach produces well-calibrated safety probabilities, assuming the estimated state distributions are well-calibrated. We evaluate our approach on simulated water tanks.

Poster: How Safe Am I Given What I See? Calibrated Prediction of Safety Chanc...

Ivan Ruchkin

This poster was presented at the University of Florida AI Days 2023. It is based on this paper: https://arxiv.org/abs/2308.12252 Title: How Safe Am I Given What I See? Calibrated Prediction of Safety Chances for Image-Controlled Autonomy Abstract: End-to-end learning has emerged as a major paradigm for developing autonomous systems. Unfortunately, with its performance and convenience comes an even greater challenge of safety assurance. A key factor of this challenge is the absence of the notion of a low-dimensional and interpretable dynamical state, around which traditional assurance methods revolve. Focusing on the online safety prediction problem, this paper proposes a configurable family of learning pipelines based on generative world models, which do not require low-dimensional states. To implement these pipelines, we overcome the challenges of learning safety-informed latent representations and missing safety labels under prediction-induced distribution shift. These pipelines come with statistical calibration guarantees on their safety chance predictions based on conformal prediction. We perform an extensive evaluation of the proposed learning pipelines on two case studies of image-controlled systems: a racing car and a cartpole.

Causal Repair of Learning-Enabled Cyber-physical Systems

Ivan Ruchkin

Presented by Pengyuan (Eric) Lu at the International Conference on Assured Autonomy 2023. Abstract: Models of actual causality leverage domain knowledge to generate convincing diagnoses of events that caused an outcome. It is promising to apply these models to diagnose and repair run-time property violations in cyber-physical systems (CPS) with learning-enabled components (LEC). However, given the high diversity and complexity of LECs, it is challenging to encode domain knowledge (e.g., the CPS dynamics) in a scalable actual causality model that could generate useful repair suggestions. In this paper, we focus causal diagnosis on the input/output behaviors of LECs. Specifically, we aim to identify which subset of I/O behaviors of the LEC is an actual cause for a property violation. An important by-product is a counterfactual version of the LEC that repairs the run-time property by fixing the identified problematic behaviors. Based on this insights, we design a two-step diagnostic pipeline: (1) construct and Halpern-Pearl causality model that reflects the dependency of property outcome on the component's I/O behaviors, and (2) perform a search for an actual cause and corresponding repair on the model. We prove that our pipeline has the following guarantee: if an actual cause is found, the system is guaranteed to be repaired; otherwise, we have high probabilistic confidence that the LEC under analysis did not cause the property violation. We demonstrate that our approach successfully repairs learned controllers on a standard OpenAI Gym benchmark.

Conservative Safety Monitors of Stochastic Dynamical Systems

Ivan Ruchkin

Presented at the NASA Formal Methods Symposium (NFM) by Matthew Cleaveland on May 16, 2023. Abstract: Generating accurate runtime safety estimates for autonomous systems is vital to ensuring their continued proliferation. However, exhaustive reasoning about future behaviors is generally too complex to do at runtime. To provide scalable and formal safety estimates, we propose a method for leveraging design-time model checking results at runtime. Specifically, we model the system as a probabilistic automaton (PA) and compute bounded-time reachability probabilities over the states of the PA at design time. At runtime, we combine distributions of state estimates with the model checking results to produce a bounded time safety estimate. We argue that our approach produces well-calibrated safety probabilities, assuming the estimated state distributions are well-calibrated. We evaluate our approach on simulated water tanks.

High-Confidence Data Programming for Evaluating Suppression of Physiological ...

Ivan Ruchkin

Presented by Sydney Pugh at the IEEE/ACM international conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE) 2021. Abstract: False alarms generated by physiological monitors can overwhelm clinical caretakers with a variety of alarms. The resulting alarm fatigue can be mitigated with alarm suppression. Before being deployed, such suppression mechanisms need to be evaluated through a costly observational study, which would determine and label the truly suppressible alarms. This paper proposes a lightweight method for evaluating alarm suppression without access to the true alarm labels. The method is based on the data programming paradigm, which combines noisy and cheap-to-obtain labeling heuristics into probabilistic labels. Based on these labels, the method estimates the sensitivity/specificity of a suppression mechanism and describes the likely outcomes of an observational study in the form of confidence bounds. We evaluate the proposed method in a case study of low SpO2 alarms using a dataset collected at Children's Hospital of Philadelphia and show that our method provides tight and accurate bounds that significantly outperform the naive comparative method.

Data Generation with PROSPECT: a Probability Specification Tool

Ivan Ruchkin

Presented at the Winter Simulation Conference 2021. Abstract: Stochastic simulations of complex systems often rely on sampling dependent discrete random variables. Currently, their users are limited in expressing their intention about how these variables are distributed and related to each other over time. This limitation leads the users to program complex and error-prone sampling algorithms. This paper introduces a way to specify, declaratively and precisely, a temporal distribution over discrete variables. Our tool PROSPECT infers and samples this distribution by solving a system of polynomial equations. The evaluation on three simulation scenarios shows that the declarative specifications are easier to write, 3x more succinct than imperative sampling programs, and are processed correctly by PROSPECT.

On the Role of Assumptions in Engineering Smart Systems

Ivan Ruchkin

A position talk given at the Smart Designing of Smart Systems workshop: https://sdpsnet.org/sdps-2020/ws8.html Engineers necessarily make assumptions during design and implementation of complex systems. These assumptions often set the expectations of one component towards the environment and other components. Assumptions are typically seen as implicit weak points of the system: should they be violated at run time, the system is likely to fall short of its required performance and safety. Although some recent work makes the assumptions explicit, they are still seen as liabilities. This talk will take a complementary perspective and explore how engineering assumptions enable intelligent behaviour in systems and design processes -- and how their violations can be managed at design time and run time.

Overview of Epidemic Models for COVID-19

Ivan Ruchkin

Thesis Defense: Integration of Modeling Methods for Cyber-Physical Systems

Ivan Ruchkin

A slide deck from my PhD thesis defense. The video of the defense talk can be seen here: https://scs.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=aebd3567-e42b-4281-94a7-a98f011d1268 Abstract: "Cyber-physical systems (CPS) incorporate digital (cyber) and mechanical (physical) elements that interact in complex ways. Many safety-critical CPS, such as autonomous vehicles and drones, are becoming increasingly widespread and hence demand rigorous quality assurance. To this end, CPS engineering relies on modeling methods, which use models to represent the system and design-time analyses to interpret/change the models. Coming from diverse scientific and engineering fields, these modeling methods are difficult to combine, or integrate, due to implicit relations and dependencies between them. CPS failures can lead to substantial damage or loss of life, and are often due to two key integration challenges: (i) inconsistencies between models — contradictions in models that do not add up to a cohesive design, and (ii) incorrect interactions of analyses — out-of-order executions in mismatched contexts, leading to erroneous analysis outputs. This thesis presents a novel approach to detect and prevent integration issues between CPS modeling methods during the design phase. To detect inconsistencies between models, the approach allows engineers to specify integration properties — quantified logical statements that relate various elements of multiple models — in the Integration Property Language (IPL). IPL statements describe verifiable conditions that are equivalent to an absence of inconsistencies. To interface with the models, IPL relies on integration abstractions — simplified representations of models for integration purposes. Two abstractions are proposed in this thesis: views (annotated component-and-connector models, inspired by software architecture) and behavioral properties (expressions in model-specific languages, such as the linear temporal logic). Combining these abstractions enables engineers to mix model structure and behavior in IPL statements. To ensure correct interactions of analyses, I introduce analysis contracts — a lightweight specification that captures inputs, outputs, assumptions, and guarantees for each analysis, in terms of the integration abstractions. Given these contracts, an analysis execution platform performs analyses in the order of their dependencies, and only in contexts that guarantee correct outputs. My approach to integration was validated on four case studies of CPS modeling methods in different systems: energy-aware planning in a mobile robot, collision avoidance in a mobile robot, thread/battery scheduling in a quadrotor, and reliable/secure sensing in an autonomous vehicle. The validation has shown that the approach supports expressive integration properties, which can be soundly checked within practical constraints, all while being customizable to diverse models, analyses, and domains."

Towards a Formal Framework for Hybrid Planning in Self-Adaptation

Ivan Ruchkin

A presentation from SEAMS 2017 on formalization of hybrid planning. Lead author: https://www.cs.cmu.edu/~ashutosp/ Full paper: http://www.cs.cmu.edu/~iruchkin/docs/pandey17-towards.pdf Abstract: "Decision-making approaches in self-adaptation face a fundamental trade-off between quality and timeliness of adaptation plans. Due to this trade-off, designers often have to make an offline compromise between finding adaptation plans quickly and finding closer-to-optimal plans that demand longer computation times. Recent work has proposed that hybrid planning can resolve this trade-off dynamically, achieving higher utility than either fast or slow approaches individually. The promise of hybrid planning is to combine multiple decision-making approaches at run time to produce adaptation plans of the high quality within given time constraints. However, the diversity of decision-making approaches makes the problem of hybrid planning complex and multi-faceted. This paper advances the theory of hybrid planning by formalizing the central concepts and four sub-problems of hybrid planning. This formalization can serve as a foundation for creating and evaluating hybrid planners in the future."

IPL: An Integration Property Language for Multi-Model Cyber-Physical Systems

Ivan Ruchkin

Our talk from the 22nd International Symposium on Formal Methods. Full paper: http://www.cs.cmu.edu/~iruchkin/docs/ruchkin18-ipl.pdf Abstract: "Design and verification of modern systems requires diverse models, which often come from a variety of disciplines, and it is challenging to manage their heterogeneity -- especially in the case of cyber-physical systems. To check consistency between models, recent approaches map these models to flexible static abstractions, such as architectural views. This model integration approach, however, comes at a cost of reduced expressiveness because complex behaviors of the models are abstracted away. As a result, it may be impossible to automatically verify important behavioral properties across multiple models, leaving systems vulnerable to subtle bugs. This paper introduces the Integration Property Language (IPL) that improves integration expressiveness using modular verification of properties that depend on detailed behavioral semantics while retaining the ability for static system-wide reasoning. We prove that the verification algorithm is sound and analyze its termination conditions. Furthermore, we perform a case study on a mobile robot to demonstrate IPL is practically useful and evaluate its performance. "

Inconsistencies in Models of Adaptive Service Robots

Ivan Ruchkin

Consider an adaptive robot that completes movement tasks in an uncertain environment with limited resources. A promising way to design such robots is model-based engineering -- using multiple explicit models (e.g., of the architecture, power, and motion) to make choices based on the current state estimates and future state predictions. Ideally, these multiple models should be integrated; that is, they should cooperate seamlessly to support the adaptation goals. However, inconsistencies between models threaten their proper integration and, consequently, the intended adaptive behavior. We will describe several examples of inconsistencies that arise between the models of power, motion, configuration, and physical environment for a mobile service robot. The talk will discuss the causes and impacts of these inconsistencies, as well as the preliminary ways to detect and correct them.

Challenges in Physical Modeling for Adaptation of Cyber-Physical Systems

Ivan Ruchkin

The initial version of slides is due to Selva Samuel. Abstract: "Cyber-physical systems (CPSs) mix software, hardware, and physical aspects with equal importance. Typically, the use of models of such systems during run time has concentrated only on managing and controlling the cyber (software) aspects. However, to fully realize the goals of a CPS, physical models too have to be treated as first-class models. This approach gives rise to three main challenges: (a) identifying and integrating physical and software models with different characteristics and semantics; (b) obtaining instances of physical models at a suitable level of abstraction for adaptation; and (c) using and adapting physical models to control CPSs. In this position paper, we elaborate on these three challenges and describe our vision of making physical models first-class entities in adaptation. We illustrate this vision in the context of power adaptation for a service robotic system."

Hardware Power Modeling for Turtlebot

Ivan Ruchkin

Integration Beyond Components and Models: Research Challenges and Directions

Ivan Ruchkin

A talk given at ACVI 2016. Abstract: Recent research in embedded and cyber-physical systems has developed theories and tools for integration of heterogeneous components and models. These efforts, although important, are insufficient for high-quality and error-free systems integration since inconsistencies between system elements may stem from factors not directly represented in models (e.g., analysis tools and expert disagreements). Therefore, we need to broaden our perspective on integration, and devise approaches in three novel directions of integration: modeling methods, data sets, and humans. This paper summarizes the latest advances, and discusses those directions and associated challenges in integration for cyber-physical systems.

Eliminating Inter-Domain Vulnerabilities in Cyber-Physical Systems: An Analys...

Ivan Ruchkin

Presented at the 1st ACM Workshop on Cyber-Physical Systems Security and Privacy (CPS-SPC). Abstract: Designing secure cyber-physical systems (CPS) is a particularly difficult task since security vulnerabilities stem not only from traditional cybersecurity concerns, but also physical ones. Many of the standard methods for CPS design make strong and unverified assumptions about the trustworthiness of physical devices, such as sensors. When these assumptions are violated, subtle inter-domain vulnerabilities are introduced into the system model. In this paper we use formal specification of analysis contracts to expose security assumptions and guarantees of analyses from reliability, control, and sensor security domains. We show that this specification allows us to determine where these assumptions are violated, opening the door to malicious attacks. We demonstrate how this approach can help discover and prevent vulnerabilities using a self-driving car example.

More from Ivan Ruchkin (20)

Language-Enhanced Latent Representations for Out-of-Distribution Detection in...

Poster: Zero-shot Safety Prediction for Autonomous Robots with Foundation Wo...

Curating Naturally Adversarial Datasets for Learning-Enabled Medical Cyber-Ph...

Repairing Learning-Enabled Controllers While Preserving What Works

Poster: Conservative Safety Monitors of Stochastic Dynamical Systems

Poster: How Safe Am I Given What I See? Calibrated Prediction of Safety Chanc...

Causal Repair of Learning-Enabled Cyber-physical Systems

Conservative Safety Monitors of Stochastic Dynamical Systems

High-Confidence Data Programming for Evaluating Suppression of Physiological ...

Data Generation with PROSPECT: a Probability Specification Tool

On the Role of Assumptions in Engineering Smart Systems

Overview of Epidemic Models for COVID-19

Thesis Defense: Integration of Modeling Methods for Cyber-Physical Systems

Towards a Formal Framework for Hybrid Planning in Self-Adaptation

IPL: An Integration Property Language for Multi-Model Cyber-Physical Systems

Inconsistencies in Models of Adaptive Service Robots

Challenges in Physical Modeling for Adaptation of Cyber-Physical Systems

Hardware Power Modeling for Turtlebot

Integration Beyond Components and Models: Research Challenges and Directions

Eliminating Inter-Domain Vulnerabilities in Cyber-Physical Systems: An Analys...

Recently uploaded

Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx

nkrafacyberclub

How world-class product teams are winning in the AI era by CEO and Founder, P...

Product School

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

FIDO Alliance

Transcript: Selling digital books in 2024: Insights from industry leaders - T...

BookNet Canada

The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more. Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/ Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.

Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...

Ramesh Iyer

In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.

PCI PIN Basics Webinar from the Controlcase Team

ControlCase

SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf

Peter Spielvogel

Building better applications for business users with SAP Fiori. • What is SAP Fiori and why it matters to you • How a better user experience drives measurable business benefits • How to get started with SAP Fiori today • How SAP Fiori elements accelerates application development • How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities • How SAP Fiori paves the way for using AI in SAP apps

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf

FIDO Alliance

Free Complete Python - A step towards Data Science

RinaMondal9

FIDO Alliance Osaka Seminar: Overview.pdf

FIDO Alliance

UiPath Test Automation using UiPath Test Suite series, part 3

DianaGray10

Monitoring Java Application Security with JDK Tools and JFR Events

Ana-Maria Mihalceanu

State of ICS and IoT Cyber Threat Landscape Report 2024 preview

Prayukth K V

The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development. The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers: State of global ICS asset and network exposure Sectoral targets and attacks as well as the cost of ransom Global APT activity, AI usage, actor and tactic profiles, and implications Rise in volumes of AI-powered cyberattacks Major cyber events in 2024 Malware and malicious payload trends Cyberattack types and targets Vulnerability exploit attempts on CVEs Attacks on counties – USA Expansion of bot farms – how, where, and why In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East Why are attacks on smart factories rising? Cyber risk predictions Axis of attacks – Europe Systemic attacks in the Middle East Download the full report from here: https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/

PHP Frameworks: I want to break free (IPC Berlin 2024)

Ralf Eggert

In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development. This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.

Key Trends Shaping the Future of Infrastructure.pdf

Cheryl Hung

Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf

91mobiles

The Future of Platform Engineering

Jemma Hussein Allen

FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf

FIDO Alliance

Assure Contact Center Experiences for Your Customers With ThousandEyes

ThousandEyes

Essentials of Automations: Optimizing FME Workflows with Parameters

Safe Software

Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place. Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects. Here’s what you’ll gain: - Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows. - Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy. - Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency. - Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity. We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic. Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.

Recently uploaded (20)

Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx

How world-class product teams are winning in the AI era by CEO and Founder, P...

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

Transcript: Selling digital books in 2024: Insights from industry leaders - T...

Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...

PCI PIN Basics Webinar from the Controlcase Team

SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf

Free Complete Python - A step towards Data Science

FIDO Alliance Osaka Seminar: Overview.pdf

UiPath Test Automation using UiPath Test Suite series, part 3

Monitoring Java Application Security with JDK Tools and JFR Events

State of ICS and IoT Cyber Threat Landscape Report 2024 preview

PHP Frameworks: I want to break free (IPC Berlin 2024)

Key Trends Shaping the Future of Infrastructure.pdf

Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf

The Future of Platform Engineering

FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf

Assure Contact Center Experiences for Your Customers With ThousandEyes

Essentials of Automations: Optimizing FME Workflows with Parameters

Confidence Monitoring and Composition for Dynamic Assurance of Learning-Enabled Cyber-Physical Systems

1. Confidence Monitoring and Composition for Dynamic Assurance of Learning-Enabled CPS Ivan Ruchkin, Matthew Cleaveland, Oleg Sokolsky, Insup Lee University of Pennsylvania DARPA Hot Topics Day IEEE Real-Time Systems Symposium December 1, 2020

2. 2 Assurance confidence Req: no obstacle collisions Req: no pipeline loss

3. 3 Assurance confidence Detection confidence Detection confidence Dynamics confidence Req: no obstacle collisions Req: no pipeline loss

4. 4 Problem ● Assurance confidence monitoring – Compute confidence in the guarantees of safety reqs – Given confidence measures from run-time monitors Challenge: Safety reqs ←?→ run-time monitors

5. 5 Problem ● Assurance confidence monitoring – Compute confidence in the guarantees of safety reqs – Given confidence measures from run-time monitors ● Challenge: – Guarantees ←?→ run-time monitors

6. 7

7. 8

8. 9

9. 10

10. 11

11. 12

12. 13

13. 14

14. 15

15. 16

16. 17

17. 18

18. 19

19. 20 Assumption effect analysis

20. 21 Assumption effect analysis Assumption required for R1: no collisions Mode1: obstacle detected Mode2: obstacle not detected A1: “Reported distance is within 1m of true distance” A2: “No false-negative obstacle detections” A3: “The obstacle is >10m away” A4: “Vehicle follows the given dynamics equations”

21. 22 Assumption effect analysis Assumption required for R1: no collisions Mode1: obstacle detected Mode2: obstacle not detected A1: “Reported distance is within 1m of true distance” Y N A2: “No false-negative obstacle detections” A3: “The obstacle is >10m away” A4: “Vehicle follows the given dynamics equations”

22. 23 Assumption effect analysis Assumption required for R1: no collisions Mode1: obstacle detected Mode2: obstacle not detected A1: “Reported distance is within 1m of true distance” Y N A2: “No false-negative obstacle detections” N Y A3: “The obstacle is >10m away” A4: “Vehicle follows the given dynamics equations”

23. 24 Assumption effect analysis Assumption required for R1: no collisions Mode1: obstacle detected Mode2: obstacle not detected A1: “Reported distance is within 1m of true distance” Y N A2: “No false-negative obstacle detections” N Y A3: “The obstacle is >10m away” Y Y A4: “Vehicle follows the given dynamics equations” Y Y

24. 25 Assumption effect analysis Assumption required for R1: no collisions Mode1: obstacle detected Mode2: obstacle not detected A1: “Reported distance is within 1m of true distance” Y N A2: “No false-negative obstacle detections” N Y A3: “The obstacle is >10m away” Y Y A4: “Vehicle follows the given dynamics equations” Y Y Composition logic: (Mode1 → A1 A3 A4) (Mode2 → A2 A3 A4)∧ A3 ∧ A4) ∧ (Mode2 → A2 ∧ A3 ∧ A4) ∧ A3 ∧ A4) ∧ (Mode2 → A2 ∧ A3 ∧ A4) ∧ A3 ∧ A4) ∧ (Mode2 → A2 ∧ A3 ∧ A4) ∧ A3 ∧ A4) ∧ (Mode2 → A2 ∧ A3 ∧ A4) ∧ A3 ∧ A4) ∧ (Mode2 → A2 ∧ A3 ∧ A4)

25. 26

26. 27

27. 28

28. 29

29. 30 ● Random variables: – Reported distance (RD) – True distance (TD) An assumption is an assertion over variables: A1: | OD – TD | ≤ 1m 2: OD = ∞ → TD = ∞ Goal: compute probabilistic queries over assns given observations E.g., P( f(A1, A2) | OD ), where f is a given Boolean function Probabilistic modeling of assumptions

30. 31 ● Random variables: – Reported distance (RD) – True distance (TD) ● An assumption is an assertion over variables: – A1: | RD – TD | ≤ 1m (bounded error) – A2: RD = ∞ → TD = ∞ (no false negatives) Goal: compute probabilistic queries over assns given observations E.g., P( f(A1, A2) | OD ), where f is a given Boolean function Probabilistic modeling of assumptions

31. 32 ● Random variables: – Reported distance (RD) – True distance (TD) ● An assumption is an assertion over variables: – A1: | RD – TD | ≤ 1m (bounded error) – A2: RD = ∞ → TD = ∞ (no false negatives) ● Goal: compute probabilistic queries over assns given observations – P( f(A1, A2) | RD ), where f is a given Boolean function Probabilistic modeling of assumptions

32. 33 Assumption monitoring with ReCal/CAM

33. 34 Assumption monitoring with ReCal/CAM

34. 35 Assumption monitoring with ReCal/CAM

35. 36 Assumption monitoring with ReCal/CAM

36. 37 Assumption monitoring with ReCal/CAM P(A1) = 0.78 P(A2) = 0.97 P(A1 V A2) = 0.98

37. 38

38. 39

39. 40

40. 41 Using our prior composition technique based on symbolic manipulation: Ruchkin, Sokolsky, Weimer, Hedaoo, Lee. Compositional Probabilistic Analysis of Temporal Properties over Stochastic Detectors, EMSOFT 2020.

41. 42 Future work ● Application to several UUV monitors ● Guarantees of confidence accuracy/conservatism ● Second-order assumptions (of our assumption models)

42. 43 Summary ● Assurance confidence requires closing a gap: – Safety guarantees ←?→ run-time monitors ● We address this problem in four steps: – Offline verification – Assumption effect analysis – Assumption monitoring – Composition of monitors

Confidence Monitoring and Composition for Dynamic Assurance of Learning-Enabled Cyber-Physical Systems

Recommended

Recommended

More Related Content

Similar to Confidence Monitoring and Composition for Dynamic Assurance of Learning-Enabled Cyber-Physical Systems

Similar to Confidence Monitoring and Composition for Dynamic Assurance of Learning-Enabled Cyber-Physical Systems (20)

More from Ivan Ruchkin

More from Ivan Ruchkin (20)

Recently uploaded

Recently uploaded (20)

Confidence Monitoring and Composition for Dynamic Assurance of Learning-Enabled Cyber-Physical Systems