SlideShare a Scribd company logo
1 of 12
2018 IBM Systems
Technical University
August / 2018
São Paulo
Time Series Predictions
With Tensorflow
Paulo Queiroz
Systems Consultant
IBM Systems Lab Services
Session Objectives
— Concepts of time series predictions
— Preparing and understanding the data
– Trends / Seasons
– Stationary data
– Transformations
— Statistical approach & Linear Models
— Neural Network approach
– LSTM
© Copyright IBM Corporation 20182
Time series processing
Different of normal data series estimations, time series might require a different approach when
predicting or explaining the data movements
●
If I want to predict jacket sales, does the month matter ?
●
How many beers should I expect to sell in the morning ?
Time Series ( TS ) introduce the concept of seasonal trends,
which might make usual analisis methods less efficient
© Copyright IBM Corporation 20183
If a sample TS dataset is loaded some information becomes proeminent:
●
The data has a positive trend
●
It‘s high seasonal
Does this variations affect the prediction ?
How to handle these effects ?
© Copyright IBM Corporation 20184
Time series processing
Statistical models are made to analyse only stationary ( no trend no season ) data
This happens mainly due the way that regressive models works, which relies on always finding the
best fit straight line to the points.
●
If the trend is negative the prediction will be negative as the line progression is negative
●
Between the seasons the trend changes which will affect the line
The key point when the data isn‘t stationary is identify the trends and work into make it stationary, by
removing the noise ( it‘s a pain )
© Copyright IBM Corporation 20185
Stationary data
Make the data Stationary
There are a few tricks to make the data stationary and isolate non-stationary data as noise.
They mainly goes around logarithm transformation of the numbers and shift the differentiation
© Copyright IBM Corporation 20186
After the differentiation, we can assume with near 90% accuracy that the residual data is
stationary
Make Predictions !
Within the linear approach, when a prediction is wanted, the most complete model is called ARIMA ( Auto Regressive Integrated
Moving averages ).
To simplify, ARIMA models can be seen as a composition of linear regressions with tweaks.
While a simple linear regression looks like y = mx+b, our ARIMA looks like: ŷt = μ + 1 yt-1 +…+ p yt-p - θ1et-1 -…- θqet-qϕ ϕ
Before begin predicting… it‘s important to train ARIMA with the data, and the trained data will look like this:
© Copyright IBM Corporation 20187
Re-insert Trends
The key objective of ARIMA was predict the behavior within regular the seasons, however it didn‘t take into account the trend,
therefore the trend must be inserted back
© Copyright IBM Corporation 20188
Code so far
Python comes with tons of modules that help construct these predictions…
© Copyright IBM Corporation 20189
But even with all these tools the results weren‘t so good :S
Would another approach produce better results ?
Neural Network Approach
Another approach is use neural networks and feed as much as data as possible to it, in order to explain the behavior.
When using neural network is required to remove season and trends, as the network learn this behavior.
© Copyright IBM Corporation 201810
Neural Network Approach
Due that, the code base is much simpler, and only feature normalization is required due the network convergency
© Copyright IBM Corporation 201811
Thank you!
Paulo Queiroz
Systems Consultant
pqueiroz@br.ibm.com
+55 0800 701 4262
ibm.com
Please complete the Session
Evaluation!
© Copyright IBM Corporation 201812

More Related Content

Similar to Time Series Predictions With Tensorflow

Jinxing_LIN_S224266_Poster
Jinxing_LIN_S224266_PosterJinxing_LIN_S224266_Poster
Jinxing_LIN_S224266_Posterjinxing lin
 
Machinelearning: The next step in manufacturing performance
Machinelearning: The next step in manufacturing performance Machinelearning: The next step in manufacturing performance
Machinelearning: The next step in manufacturing performance Blackberry&Cross
 
Automatic algorithms for time series forecasting
Automatic algorithms for time series forecastingAutomatic algorithms for time series forecasting
Automatic algorithms for time series forecastingRob Hyndman
 
Big data meets evm (submitted)
Big data meets evm (submitted)Big data meets evm (submitted)
Big data meets evm (submitted)Glen Alleman
 
STOCK PRICE PREDICTION USING ML TECHNIQUES
STOCK PRICE PREDICTION USING ML TECHNIQUESSTOCK PRICE PREDICTION USING ML TECHNIQUES
STOCK PRICE PREDICTION USING ML TECHNIQUESIRJET Journal
 
Machine Learning Algorithms | Machine Learning Tutorial | Data Science Algori...
Machine Learning Algorithms | Machine Learning Tutorial | Data Science Algori...Machine Learning Algorithms | Machine Learning Tutorial | Data Science Algori...
Machine Learning Algorithms | Machine Learning Tutorial | Data Science Algori...Simplilearn
 
Lec 2 Human Resource Analytics
Lec 2 Human Resource AnalyticsLec 2 Human Resource Analytics
Lec 2 Human Resource Analyticspal83111
 
March 2, 2018 - Machine Learning for Production Forecasting
March 2, 2018 - Machine Learning for Production ForecastingMarch 2, 2018 - Machine Learning for Production Forecasting
March 2, 2018 - Machine Learning for Production ForecastingDavid Fulford
 
SPC (Statistical Process Control) concepts in forecasting
SPC (Statistical Process Control) concepts in forecastingSPC (Statistical Process Control) concepts in forecasting
SPC (Statistical Process Control) concepts in forecastingCharles Novak
 
Applied Data Science Part 3: Getting dirty; data preparation and feature crea...
Applied Data Science Part 3: Getting dirty; data preparation and feature crea...Applied Data Science Part 3: Getting dirty; data preparation and feature crea...
Applied Data Science Part 3: Getting dirty; data preparation and feature crea...Dataiku
 
Demand Planning Leadership Exchange: SAP APO DP Statistical Forecast Optimiza...
Demand Planning Leadership Exchange: SAP APO DP Statistical Forecast Optimiza...Demand Planning Leadership Exchange: SAP APO DP Statistical Forecast Optimiza...
Demand Planning Leadership Exchange: SAP APO DP Statistical Forecast Optimiza...Plan4Demand
 
Xpanse-Manufacturing-2023.pdf
Xpanse-Manufacturing-2023.pdfXpanse-Manufacturing-2023.pdf
Xpanse-Manufacturing-2023.pdfNiallWalsh25
 
Machine Learning in the Financial Industry
Machine Learning in the Financial IndustryMachine Learning in the Financial Industry
Machine Learning in the Financial IndustrySubrat Panda, PhD
 
The 4 Machine Learning Models Imperative for Business Transformation
The 4 Machine Learning Models Imperative for Business TransformationThe 4 Machine Learning Models Imperative for Business Transformation
The 4 Machine Learning Models Imperative for Business TransformationRocketSource
 
Case Study Sap Establishing A Research Center Over China
Case Study Sap Establishing  A Research Center Over ChinaCase Study Sap Establishing  A Research Center Over China
Case Study Sap Establishing A Research Center Over ChinaLakeisha Jones
 

Similar to Time Series Predictions With Tensorflow (20)

Jinxing_LIN_S224266_Poster
Jinxing_LIN_S224266_PosterJinxing_LIN_S224266_Poster
Jinxing_LIN_S224266_Poster
 
Machinelearning: The next step in manufacturing performance
Machinelearning: The next step in manufacturing performance Machinelearning: The next step in manufacturing performance
Machinelearning: The next step in manufacturing performance
 
Automatic algorithms for time series forecasting
Automatic algorithms for time series forecastingAutomatic algorithms for time series forecasting
Automatic algorithms for time series forecasting
 
Analytics
AnalyticsAnalytics
Analytics
 
Big data meets evm (submitted)
Big data meets evm (submitted)Big data meets evm (submitted)
Big data meets evm (submitted)
 
STOCK PRICE PREDICTION USING ML TECHNIQUES
STOCK PRICE PREDICTION USING ML TECHNIQUESSTOCK PRICE PREDICTION USING ML TECHNIQUES
STOCK PRICE PREDICTION USING ML TECHNIQUES
 
Machine Learning Algorithms | Machine Learning Tutorial | Data Science Algori...
Machine Learning Algorithms | Machine Learning Tutorial | Data Science Algori...Machine Learning Algorithms | Machine Learning Tutorial | Data Science Algori...
Machine Learning Algorithms | Machine Learning Tutorial | Data Science Algori...
 
A New Sales Forecasting Model for International Restaurants
A New Sales Forecasting Model for International RestaurantsA New Sales Forecasting Model for International Restaurants
A New Sales Forecasting Model for International Restaurants
 
Lec 2 Human Resource Analytics
Lec 2 Human Resource AnalyticsLec 2 Human Resource Analytics
Lec 2 Human Resource Analytics
 
March 2, 2018 - Machine Learning for Production Forecasting
March 2, 2018 - Machine Learning for Production ForecastingMarch 2, 2018 - Machine Learning for Production Forecasting
March 2, 2018 - Machine Learning for Production Forecasting
 
Minkiewicz - Lessons Learned from the ISBSG Database
Minkiewicz - Lessons Learned from the ISBSG DatabaseMinkiewicz - Lessons Learned from the ISBSG Database
Minkiewicz - Lessons Learned from the ISBSG Database
 
SPC (Statistical Process Control) concepts in forecasting
SPC (Statistical Process Control) concepts in forecastingSPC (Statistical Process Control) concepts in forecasting
SPC (Statistical Process Control) concepts in forecasting
 
Applied Data Science Part 3: Getting dirty; data preparation and feature crea...
Applied Data Science Part 3: Getting dirty; data preparation and feature crea...Applied Data Science Part 3: Getting dirty; data preparation and feature crea...
Applied Data Science Part 3: Getting dirty; data preparation and feature crea...
 
Predictive analytics roadshow
Predictive analytics roadshowPredictive analytics roadshow
Predictive analytics roadshow
 
Demand Planning Leadership Exchange: SAP APO DP Statistical Forecast Optimiza...
Demand Planning Leadership Exchange: SAP APO DP Statistical Forecast Optimiza...Demand Planning Leadership Exchange: SAP APO DP Statistical Forecast Optimiza...
Demand Planning Leadership Exchange: SAP APO DP Statistical Forecast Optimiza...
 
Practical Machine Learning at Work
Practical Machine Learning at WorkPractical Machine Learning at Work
Practical Machine Learning at Work
 
Xpanse-Manufacturing-2023.pdf
Xpanse-Manufacturing-2023.pdfXpanse-Manufacturing-2023.pdf
Xpanse-Manufacturing-2023.pdf
 
Machine Learning in the Financial Industry
Machine Learning in the Financial IndustryMachine Learning in the Financial Industry
Machine Learning in the Financial Industry
 
The 4 Machine Learning Models Imperative for Business Transformation
The 4 Machine Learning Models Imperative for Business TransformationThe 4 Machine Learning Models Imperative for Business Transformation
The 4 Machine Learning Models Imperative for Business Transformation
 
Case Study Sap Establishing A Research Center Over China
Case Study Sap Establishing  A Research Center Over ChinaCase Study Sap Establishing  A Research Center Over China
Case Study Sap Establishing A Research Center Over China
 

Recently uploaded

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CVKhem
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessPixlogix Infotech
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 

Recently uploaded (20)

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 

Time Series Predictions With Tensorflow

  • 1. 2018 IBM Systems Technical University August / 2018 São Paulo Time Series Predictions With Tensorflow Paulo Queiroz Systems Consultant IBM Systems Lab Services
  • 2. Session Objectives — Concepts of time series predictions — Preparing and understanding the data – Trends / Seasons – Stationary data – Transformations — Statistical approach & Linear Models — Neural Network approach – LSTM © Copyright IBM Corporation 20182
  • 3. Time series processing Different of normal data series estimations, time series might require a different approach when predicting or explaining the data movements ● If I want to predict jacket sales, does the month matter ? ● How many beers should I expect to sell in the morning ? Time Series ( TS ) introduce the concept of seasonal trends, which might make usual analisis methods less efficient © Copyright IBM Corporation 20183
  • 4. If a sample TS dataset is loaded some information becomes proeminent: ● The data has a positive trend ● It‘s high seasonal Does this variations affect the prediction ? How to handle these effects ? © Copyright IBM Corporation 20184 Time series processing
  • 5. Statistical models are made to analyse only stationary ( no trend no season ) data This happens mainly due the way that regressive models works, which relies on always finding the best fit straight line to the points. ● If the trend is negative the prediction will be negative as the line progression is negative ● Between the seasons the trend changes which will affect the line The key point when the data isn‘t stationary is identify the trends and work into make it stationary, by removing the noise ( it‘s a pain ) © Copyright IBM Corporation 20185 Stationary data
  • 6. Make the data Stationary There are a few tricks to make the data stationary and isolate non-stationary data as noise. They mainly goes around logarithm transformation of the numbers and shift the differentiation © Copyright IBM Corporation 20186 After the differentiation, we can assume with near 90% accuracy that the residual data is stationary
  • 7. Make Predictions ! Within the linear approach, when a prediction is wanted, the most complete model is called ARIMA ( Auto Regressive Integrated Moving averages ). To simplify, ARIMA models can be seen as a composition of linear regressions with tweaks. While a simple linear regression looks like y = mx+b, our ARIMA looks like: ŷt = μ + 1 yt-1 +…+ p yt-p - θ1et-1 -…- θqet-qϕ ϕ Before begin predicting… it‘s important to train ARIMA with the data, and the trained data will look like this: © Copyright IBM Corporation 20187
  • 8. Re-insert Trends The key objective of ARIMA was predict the behavior within regular the seasons, however it didn‘t take into account the trend, therefore the trend must be inserted back © Copyright IBM Corporation 20188
  • 9. Code so far Python comes with tons of modules that help construct these predictions… © Copyright IBM Corporation 20189 But even with all these tools the results weren‘t so good :S Would another approach produce better results ?
  • 10. Neural Network Approach Another approach is use neural networks and feed as much as data as possible to it, in order to explain the behavior. When using neural network is required to remove season and trends, as the network learn this behavior. © Copyright IBM Corporation 201810
  • 11. Neural Network Approach Due that, the code base is much simpler, and only feature normalization is required due the network convergency © Copyright IBM Corporation 201811
  • 12. Thank you! Paulo Queiroz Systems Consultant pqueiroz@br.ibm.com +55 0800 701 4262 ibm.com Please complete the Session Evaluation! © Copyright IBM Corporation 201812