This talk was given at the Cologne AI and Machine Learning Meetup on April 13, 2023 (https://www.meetup.com/de-DE/cologne-ai-and-machine-learning-meetup/events/291513393/) by Philipp Wendland, PhD Student at Koblenz University of Applied Sciences, Group of Prof. Dr. Maik Kschischo: Neural ODEs - A state-of-the-art Deep Learning approach to process time series data
Neural ODEs are a hybrid deep learning approach based on modelling the dynamic of hidden layers of a neural network in a continuous fashion as an Ordinary Differential Equation (ODE). Due to its continuous nature and promising performance Neural ODEs are a state-of-the-art approach to process (unevenly sampled) multivariate time-series data. Further promising and succesful applications of the Neural ODEs are image classifications, density estimation with continuous normalizing flows and the creation of multi-state survival models. In this talk we want to introduce the general framework of Neural ODEs with a particular focus on its applications to patient data. We will present our extension the so-called Multimodel Neural ODEs to generate highly realistic synthetic patient data based on both static and continuous covariates.
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
Neural ODEs - A state-of-the-art Deep Learning approach to process time series data
1. Motivation Methodological Background MultiNODEs Discussion References
Neural ODEs
A state-of-the-art Deep Learning approach to process time
series data
Philipp Wendland
Prof. Dr. Maik Kschischo
13.04.2023
1 16
2. Motivation Methodological Background MultiNODEs Discussion References
Applications
Processing of time-series data
Image classifications
Density estimation with continuous normalizing flows
2 16
3. Motivation Methodological Background MultiNODEs Discussion References
Applications
Time Series data
Generative model
Predictions
Classifications
Survival analysis
4. Motivation Methodological Background MultiNODEs Discussion References
Ordinary Differential Equations
Dynamical Systems
ẋ(t) = f(x(t))
y(t) = h(x(t))
x(t0) = xt0
All ClipArts from Pixabay.com
6. Motivation Methodological Background MultiNODEs Discussion References
Neural ODEs (Chen et al., 2019)
Hidden layers are a continous
time dynamical system
Neural ODE
dx(t)
dt = fθ (x(t), t)
(p.1, fig.1, Chen et al., 2019)
7. Motivation Methodological Background MultiNODEs Discussion References
What makes Neural ODEs so special?
Data-driven modelling of continuous latent dynamics
Processing of time series data
Technically: Usage of ODE solvers
Constant memory cost at training time
Adaptive computation
7 16
14. Motivation Methodological Background MultiNODEs Discussion References
Conclusion
Neural ODEs are a powerful tool
Especially for processing time-series data
Neural ODEs can be used for many different tasks
Implementations in Pytorch, Jax and Tensorflow
Torchdyn package
https://github.com/DiffEqML/torchdyn
14 16
15. Motivation Methodological Background MultiNODEs Discussion References
Acknowledgements
I would like to thank all the coauthors of the
MultiNODE publication!
Colin Birkenbihl (Shared First Co-Author)
Marc Gomez-Freixa
Meemansa Sood
Prof. Dr. Maik Kschischo
Prof. Dr. Holger Fröhlich
15 16
16. Motivation Methodological Background MultiNODEs Discussion References
Chen, Ricky T. Q. et al. (Dec. 13, 2019). Neural Ordinary
Differential Equations. arXiv:1806.07366. type: article.
arXiv. arXiv: 1806.07366[cs,stat]. url:
http://arxiv.org/abs/1806.07366 (visited on
10/07/2022).
Wendland, Philipp et al. (Aug. 20, 2022). “Generation of
realistic synthetic data using Multimodal Neural Ordinary
Differential Equations”. In: npj Digital Medicine 5.1, p. 122.
issn: 2398-6352. doi: 10.1038/s41746-022-00666-x. url:
https://www.nature.com/articles/s41746-022-00666-x
(visited on 11/21/2022).
16 16