This poster was presented by Zhenjiang Mao at ICRA 2024. Related paper: https://arxiv.org/abs/2404.00462 Abstract: A world model creates a surrogate world to train a controller and predict safety violations by learning the internal dynamic model of systems. However, the existing world models rely solely on statistical learning of how observations change in response to actions, lacking precise quantification of how accurate the surrogate dynamics are, which poses a significant challenge in safety-critical systems. To address this challenge, we propose foundation world models that embed observations into meaningful and causally latent representations. This enables the surrogate dynamics to directly predict causal future states by leveraging a training-free large language model. In two common benchmarks, this novel model outperforms standard world models in the safety prediction task and has a performance comparable to supervised learning despite not using any data. We evaluate its performance with a more specialized and system-relevant metric by comparing estimated states instead of aggregating observation-wide error.

1. ■ Low-quality observation prediction by world model W
given current observation yt
and action ut
■ Fine-tuning of
foundation models
■ Additional dynamical
states like speed and
angular velocity
■ Apply multimodal
foundation models
■ Case studies on
physical robots
■ Unknown states and dynamical models
Preprint: Mao, Z.; Dai, S.; Geng, Y and
Ruchkin, I. "Zero-shot Safety Prediction for
Autonomous Robots with Foundation
World Models" arXiv (2024).
Zhenjiang Mao, Siqi Dai, Yuang Geng, Ivan Ruchkin
Trustworthy Engineered Autonomy (TEA) Lab
Department of Electrical and Computer Engineering, University of Florida
Zero-shot Safety Prediction for Autonomous Robots
with Foundation World Models
APPROACH
FUTURE WORK
RESULTS: LUNAR LANDER
CHALLENGES & CONTRIBUTIONS
PROBLEM
REFERENCES
─ yt+1
= W (yt
,ut
) ≠ yt+1
^
■ How to predict/evaluate the observation at object level?
─ MSE( yt+1
,yt+1
)
^
Challenges:
Contributions:
■ A training-free world model combines foundation models
and overcomes distribution shift in existing world models
■ A segmentation-based metric to measure the learned
surrogate dynamics by comparing object-level error
■ An interpretable latent representation that can evaluate
safety directly and improve safety predictions
■ Need to train specialized safety evaluator NN: y→ {T, F}
■ Out-of-distribution (OOD) shift in predictions
World models: Ha, D.; and Schmidhuber,
J. 2018. Recurrent World Models
Facilitate Policy Evolution. In Advances in
Neural Information Processing Systems,
volume 31.
Safety prediction: Mao, Z.; Sobolewski, C.;
and Ruchkin, I. "How Safe Am I Given
What I See? Calibrated Prediction of
Safety Chances for Image-Controlled
Autonomy." In Learning for Dynamics and
Control Conference 2024.
Image Rebuilding:
Move the objects from previous position wt
into the
predicted position wt+1
Coarse evaluation:
Object duplication
Object loss
Ground truth
Ground truth
─ Object Centroid Distance:
CD = ||( w i
t+1
)C
-( w i
t+1
)C
||
Fine-grained evaluation:
Observation: y Segmented
objects: w 1
-w 4
^
F1 score and False Positive Rate (FPR) of safety prediction
SSIM and MSE of predicted observation
10 20 30 40 50 60
Prediction Horizon
10 20 30 40 50 60
Prediction Horizon
10 20 30 40 50 60
Prediction Horizon
10 20 30 40 50 60
Prediction Horizon
10 20 30 40 50 60
Prediction Horizon
10 20 30 40 50 60
Prediction Horizon
1.0
0.8
0.6
0.4
0.2
0.0
Standard World Model Segmentation + Supervised World Model Foundation World Model
VAE + MD-LSTM SAM + LSTM SAM + GPT3.5
SAM + Gemma
Dashed lines indicate the use of supervised training data.
1.00
0.95
0.90
0.85
0.80
0.75
0.70
0.200
0.175
0.150
0.125
0.100
0.075
0.050
0.025
0.000
5
4
3
2
1
0
7
6
5
4
3
2
1
0
Before After Before After
^
1.0
0.8
0.6
0.4
0.2
0.0
Horizontal and vertical position error