SlideShare a Scribd company logo
1 of 11
Download to read offline
Time Series
Foundation Models
Time Series Foundation Models – Nathaniel Shimoni © 2024
• What is a foundational time series model?
• All pre-mentioned characteristics:
• Granularity
• Channels
• Domain
• Cross dependencies (time related, channel related, both, externalities)
• Pretraining task
• Forecasting (short / long term), imputation, reconstruction & AE,
• Data availability
• Large institutes
• Extendibility
Understanding Time Series Foundation Models
Time Series Foundation Models – Nathaniel Shimoni ©
Background - PatchTST
Time Series Foundation Models – Nathaniel Shimoni ©
Taxonomy of TSFMs
Time Series Foundation Models – Nathaniel Shimoni ©
https://arxiv.org/pdf/2403.14735.pdf
Taxonomy of TSFMs
Time Series Foundation Models – Nathaniel Shimoni ©
https://arxiv.org/pdf/2403.14735.pdf
Chronos – a Univariate General Forecaster
Time Series Foundation Models – Nathaniel Shimoni ©
1. AWS team paper
2. Using quantiles as
tokenizers
3. Based on Google’s
T5 model
architecture with
similar training as in
language
TimesFM (Google)
Time Series Foundation Models – Nathaniel Shimoni ©
1. Created a decoder-only
transformer model for time-
series forecasting, which is
better suited than large
language models like GPT-3
and Llama-2.
2. Trained on large corpus
containing Google trends,
Wikipedia page views and
synthetic data.
3. Surpassed statistics-based
methods on complex data.
4. Analyzed forecasting models
and demonstrated
generalization through diverse
dataset evaluations.
MOIRAI - Salesforce
Time Series Foundation Models – Nathaniel Shimoni ©
1. Introduced MOIRAI, a universal
time series forecasting
Transformer.
2. Addressed challenges with a
novel Transformer
3. Leveraged LOTSA dataset for
pre-training, with 27B
observations across 13
domains.
4. Demonstrated competitive or
superior performance
compared to state-of-the-art
baselines.
Moment – a Family of TSFM
Time Series Foundation Models – Nathaniel Shimoni ©
1. Developed MOMENT, a
transformer model pre-trained
on masked time-series
prediction.
2. Evaluated MOMENT's
effectiveness across various
tasks, demonstrating superior
performance in anomaly
detection and imputation.
3. Investigated MOMENT's ability
to capture time-series
characteristics without explicit
training, providing insights into
its interpretability and
generalization.
Other significant improvements - iTransformer
Time Series Foundation Models – Nathaniel Shimoni ©
• Multi-Target Training
• Multi-Modal Models
• Segregation – Task-Specific foundation models
Data-Specific foundation models
Future Directions
Time Series Foundation Models – Nathaniel Shimoni ©

More Related Content

Similar to Time Series Foundation Models - current state and future directions

04 distance learning standards-scorm specification
04 distance learning standards-scorm specification04 distance learning standards-scorm specification
04 distance learning standards-scorm specification
宥均 林
 
Tufts University shares its experience selecting and implementing rSmart Saka...
Tufts University shares its experience selecting and implementing rSmart Saka...Tufts University shares its experience selecting and implementing rSmart Saka...
Tufts University shares its experience selecting and implementing rSmart Saka...
rSmart
 

Similar to Time Series Foundation Models - current state and future directions (20)

An adaptative framework for tracking Web–based Learning Environments
An adaptative framework for tracking Web–based Learning EnvironmentsAn adaptative framework for tracking Web–based Learning Environments
An adaptative framework for tracking Web–based Learning Environments
 
Open Source TIMES Excel Reader
Open Source TIMES Excel ReaderOpen Source TIMES Excel Reader
Open Source TIMES Excel Reader
 
Exploiting Big Data in Time Series Forecasting: A Cross-Sectional Approach
Exploiting Big Data in Time Series Forecasting: A Cross-Sectional ApproachExploiting Big Data in Time Series Forecasting: A Cross-Sectional Approach
Exploiting Big Data in Time Series Forecasting: A Cross-Sectional Approach
 
Soa
SoaSoa
Soa
 
AWS Forcecast: DeepAR Predictor Time-series
AWS Forcecast: DeepAR Predictor Time-series AWS Forcecast: DeepAR Predictor Time-series
AWS Forcecast: DeepAR Predictor Time-series
 
Chatbots are coming!
Chatbots are coming!Chatbots are coming!
Chatbots are coming!
 
Sesa
SesaSesa
Sesa
 
04 distance learning standards-scorm specification
04 distance learning standards-scorm specification04 distance learning standards-scorm specification
04 distance learning standards-scorm specification
 
Novel Ensemble Tree for Fast Prediction on Data Streams
Novel Ensemble Tree for Fast Prediction on Data StreamsNovel Ensemble Tree for Fast Prediction on Data Streams
Novel Ensemble Tree for Fast Prediction on Data Streams
 
Predicting Stock Market Price Using Support Vector Regression
Predicting Stock Market Price Using Support Vector RegressionPredicting Stock Market Price Using Support Vector Regression
Predicting Stock Market Price Using Support Vector Regression
 
Arcomem training twitter-dynamics_beginner
Arcomem training twitter-dynamics_beginnerArcomem training twitter-dynamics_beginner
Arcomem training twitter-dynamics_beginner
 
Discovering the New SuccessFactors LMS Admin Features
Discovering the New SuccessFactors LMS Admin FeaturesDiscovering the New SuccessFactors LMS Admin Features
Discovering the New SuccessFactors LMS Admin Features
 
Duraspace Hot Topics Series 6: Metadata and Repository Services
Duraspace Hot Topics Series 6: Metadata and Repository ServicesDuraspace Hot Topics Series 6: Metadata and Repository Services
Duraspace Hot Topics Series 6: Metadata and Repository Services
 
WWW2014: Long Time No See: The Probability of Reusing Tags as a Function of F...
WWW2014: Long Time No See: The Probability of Reusing Tags as a Function of F...WWW2014: Long Time No See: The Probability of Reusing Tags as a Function of F...
WWW2014: Long Time No See: The Probability of Reusing Tags as a Function of F...
 
Attention Please! A Hybrid Resource Recommender Mimicking Attention-Interpret...
Attention Please! A Hybrid Resource Recommender Mimicking Attention-Interpret...Attention Please! A Hybrid Resource Recommender Mimicking Attention-Interpret...
Attention Please! A Hybrid Resource Recommender Mimicking Attention-Interpret...
 
A Web-scale Study of the Adoption and Evolution of the schema.org Vocabulary ...
A Web-scale Study of the Adoption and Evolution of the schema.org Vocabulary ...A Web-scale Study of the Adoption and Evolution of the schema.org Vocabulary ...
A Web-scale Study of the Adoption and Evolution of the schema.org Vocabulary ...
 
Nasscom ml ops webinar
Nasscom ml ops webinarNasscom ml ops webinar
Nasscom ml ops webinar
 
Production Monitoring Platform
Production Monitoring PlatformProduction Monitoring Platform
Production Monitoring Platform
 
Tufts University shares its experience selecting and implementing rSmart Saka...
Tufts University shares its experience selecting and implementing rSmart Saka...Tufts University shares its experience selecting and implementing rSmart Saka...
Tufts University shares its experience selecting and implementing rSmart Saka...
 
Cloud-Computing-Course-Description-and-Syllabus-Spring2020.pdf
Cloud-Computing-Course-Description-and-Syllabus-Spring2020.pdfCloud-Computing-Course-Description-and-Syllabus-Spring2020.pdf
Cloud-Computing-Course-Description-and-Syllabus-Spring2020.pdf
 

More from Nathaniel Shimoni

More from Nathaniel Shimoni (6)

Current and future challenges in data science
Current and future challenges in data scienceCurrent and future challenges in data science
Current and future challenges in data science
 
ML whitepaper v0.2
ML whitepaper v0.2ML whitepaper v0.2
ML whitepaper v0.2
 
Machine learning basic course with KNIME analytics platform
Machine learning basic course with KNIME analytics platformMachine learning basic course with KNIME analytics platform
Machine learning basic course with KNIME analytics platform
 
My path to data science
My path to data scienceMy path to data science
My path to data science
 
Introduction to competitive data science
Introduction to competitive data scienceIntroduction to competitive data science
Introduction to competitive data science
 
Starting data science with kaggle.com
Starting data science with kaggle.comStarting data science with kaggle.com
Starting data science with kaggle.com
 

Recently uploaded

Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
UK Journal
 

Recently uploaded (20)

PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptxBT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 
TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024
 
AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara Laskowska
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxWSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
 

Time Series Foundation Models - current state and future directions

  • 1. Time Series Foundation Models Time Series Foundation Models – Nathaniel Shimoni © 2024
  • 2. • What is a foundational time series model? • All pre-mentioned characteristics: • Granularity • Channels • Domain • Cross dependencies (time related, channel related, both, externalities) • Pretraining task • Forecasting (short / long term), imputation, reconstruction & AE, • Data availability • Large institutes • Extendibility Understanding Time Series Foundation Models Time Series Foundation Models – Nathaniel Shimoni ©
  • 3. Background - PatchTST Time Series Foundation Models – Nathaniel Shimoni ©
  • 4. Taxonomy of TSFMs Time Series Foundation Models – Nathaniel Shimoni © https://arxiv.org/pdf/2403.14735.pdf
  • 5. Taxonomy of TSFMs Time Series Foundation Models – Nathaniel Shimoni © https://arxiv.org/pdf/2403.14735.pdf
  • 6. Chronos – a Univariate General Forecaster Time Series Foundation Models – Nathaniel Shimoni © 1. AWS team paper 2. Using quantiles as tokenizers 3. Based on Google’s T5 model architecture with similar training as in language
  • 7. TimesFM (Google) Time Series Foundation Models – Nathaniel Shimoni © 1. Created a decoder-only transformer model for time- series forecasting, which is better suited than large language models like GPT-3 and Llama-2. 2. Trained on large corpus containing Google trends, Wikipedia page views and synthetic data. 3. Surpassed statistics-based methods on complex data. 4. Analyzed forecasting models and demonstrated generalization through diverse dataset evaluations.
  • 8. MOIRAI - Salesforce Time Series Foundation Models – Nathaniel Shimoni © 1. Introduced MOIRAI, a universal time series forecasting Transformer. 2. Addressed challenges with a novel Transformer 3. Leveraged LOTSA dataset for pre-training, with 27B observations across 13 domains. 4. Demonstrated competitive or superior performance compared to state-of-the-art baselines.
  • 9. Moment – a Family of TSFM Time Series Foundation Models – Nathaniel Shimoni © 1. Developed MOMENT, a transformer model pre-trained on masked time-series prediction. 2. Evaluated MOMENT's effectiveness across various tasks, demonstrating superior performance in anomaly detection and imputation. 3. Investigated MOMENT's ability to capture time-series characteristics without explicit training, providing insights into its interpretability and generalization.
  • 10. Other significant improvements - iTransformer Time Series Foundation Models – Nathaniel Shimoni ©
  • 11. • Multi-Target Training • Multi-Modal Models • Segregation – Task-Specific foundation models Data-Specific foundation models Future Directions Time Series Foundation Models – Nathaniel Shimoni ©