SlideShare a Scribd company logo
1 of 37
Download to read offline
Valencian Summer School in Machine Learning
3rd edition
September 14-15, 2017
BigML, Inc 2Time Series / Deepnets
Time Series Analysis
BigML, Inc 3Time Series / Deepnets
Beyond Supervision
• Traditional machine learning data is assumed to
be IID
• Independent (points have no information about each
other’s class) and
• Identically distributed (come from the same distribution)
• But what if you want to predict just the next value
in a sequence? Is all lost?
• Applications
• Predicting battery life from change-discharge cycles
• Predicting sales for the next day/week/month
BigML, Inc 4Time Series / Deepnets
Machine Learning Data
Color Mass Type
red 11 pen
green 45 apple
red 53 apple
yellow 0 pen
blue 2 pen
green 422 pineapple
yellow 555 pineapple
blue 7 pen
Discovering patterns within data:
• Color = “red” Mass < 100
• Type = “pineapple” Color ≠ “blue”
• Color = “blue” PPAP = “pen”
BigML, Inc 5Time Series / Deepnets
Machine Learning Data
Color Mass Type
red 53 apple
blue 2 pen
red 11 pen
blue 7 pen
green 45 apple
yellow 555 pineapple
green 422 pineapple
yellow 0 pen
Patterns valid despite reshuffling
• Color = “red” Mass < 100
• Type = “pineapple” Color ≠ “blue”
• Color = “blue” PPAP = “pen”
BigML, Inc 6Time Series / Deepnets
Time Series Data
Year Pineapple Harvest
1986 50,74
1987 22,03
1988 50,69
1989 40,38
1990 29,80
1991 9,90
1992 73,93
1993 22,95
1994 139,09
1995 115,17
1996 193,88
1997 175,31
1998 223,41
1999 295,03
2000 450,53
Pineapple Harvest
Tons
0
125
250
375
500
Year
1986 1988 1990 1992 1994 1996 1998 2000
Trend
BigML, Inc 7Time Series / Deepnets
Time Series Data
Year Pineapple Harvest
1986 139,09
1987 175,31
1988 9,91
1989 22,95
1990 450,53
1991 73,93
1992 40,38
1993 22,03
1994 295,03
1995 50,74
1996 29,8
1997 223,41
1998 115,17
1999 193,88
2000 50,69
Pineapple Harvest
Tons
0
125
250
375
500
Year
1986 1988 1990 1992 1994 1996 1998 2000
Patterns invalid after shuffling
BigML, Inc 8Time Series / Deepnets
Prediction
Use the data from the past to predict the future
BigML, Inc 9Time Series / Deepnets
Exponential Smoothing
BigML, Inc 10Time Series / Deepnets
Exponential Smoothing
Weight 0
0,05
0,1
0,15
0,2
Lag
1 3 5 7 9 11 13
BigML, Inc 11Time Series / Deepnets
Trendy
0
12,5
25
37,5
50
Time
Apr May Jun Jul
y
0
50
100
150
200
Time
Apr May Jun Jul
Additive Multiplicative
BigML, Inc 12Time Series / Deepnets
Seasonalityy
0
30
60
90
120
Time
1 4 7 10 13 16 19
y
0
35
70
105
140
Time
1 4 7 10 13 16 19
Additive Multiplicative
BigML, Inc 13Time Series / Deepnets
Errory
0
150
300
450
600
Time
1 4 7 10 13 16 19
y
0
125
250
375
500
Time
1 4 7 10 13 16 19
Additive Multiplicative
BigML, Inc 14Time Series / Deepnets
Model Types
None Additive Multiplicative
None A,N,N M,N,N A,N,A M,N,A A,N,M M,N,M
Additive A,A,N M,A,N A,A,A M,A,A A,A,M M,A,M
Additive + Damped A,Ad,N M,Ad,N A,Ad,A M,Ad,A A,Ad,M M,Ad,M
Multiplicative A,M,N M,M,N A,M,A M,M,A A,M,M M,M,M
Multiplicative + Damped A,Md,N M,Md,N A,Md,A M,Md,A A,Md,M M,Md,M
M,N,A
Multiplicative Error
No Trend
Additive Seasonality
BigML, Inc 15Time Series / Deepnets
Evaluating Model Fit
• AIC: Akaike Information Criterion; tries to trade off
accuracy and model complexity
• AICc: Like the AIC, but with a sample size
correction
• BIC: Bayesian Information Criterion; like the AIC
but penalizes large numbers of parameters more
harshly
• R-squared: Raw performance, the number of
model parameters isn’t considered
BigML, Inc 16Time Series / Deepnets
Linear Splitting
Year Pineapple Harvest
1986 139,09
1987 175,31
1988 9,91
1989 22,95
1990 450,53
1991 73,93
1992 40,38
1993 22,03
1994 295,03
1995 115,17
Random Split
Year Pineapple Harvest
1986 139,09
1987 175,31
1988 9,91
1989 22,95
1990 450,53
1991 73,93
1992 40,38
1993 22,03
1994 295,03
1995 115,17
Linear Split
BigML, Inc 17Time Series / Deepnets
Deep Neural Networks
BigML, Inc 18Time Series / Deepnets
BigML Deepnets
• Not Done Yet!
• I’m the tech lead, so I’m the reason we don’t have a demo for
this (sorry).
• Check out our next release webinar!
• Let’s Still Have a Chat
• Deep learning is regarded in the media as some sort of strange
robot messiah, destined to either save or destroy us all
• What’s good about deep learning and why is it so popular
now?
• How much is hype and what are some of the major issues with
it?
BigML, Inc 19Time Series / Deepnets
Going Further
• Trees
• Pro: Massive representational power that expands as the data
gets larger; efficient search through this space
• Con: Difficult to represent smooth functions and functions of
many variables
• Ensembles mitigate some of these difficulties
• Logistic Regression
• Pro: Some smooth, multivariate, functions are not a problem;
fast optimization of chosen
• Con: Parametric - If decision boundary is nonlinear, tough luck
• Can these be mitigated?
BigML, Inc 20Time Series / Deepnets
LR Level Up
Outputs
Inputs
BigML, Inc 21Time Series / Deepnets
LR Level Up
wi
Class 1, logistic(w, b)
BigML, Inc 22Time Series / Deepnets
LR Level Up
Outputs
Inputs
Hidden
layer
BigML, Inc 23Time Series / Deepnets
LR Level Up
Class 1, logistic(w, b)
Hidden unit 1,
logistic(w, b)
BigML, Inc 24Time Series / Deepnets
LR Level Up
Class 1, logistic(w, b)
Hidden unit 1,
logistic(w, b)
n nodes ?
BigML, Inc 25Time Series / Deepnets
LR Level Up
Class 1, logistic(w, b)
Hidden unit 1,
logistic(w, b)
n
hidden
layers?
BigML, Inc 26Time Series / Deepnets
LR Level Up
Class 1, logistic(w, b)
Hidden unit 1,
logistic(w, b)
BigML, Inc 27Time Series / Deepnets
Why?
• This isn’t new. Why the sudden interest?
• Scale
• Massive parameter space <=> Massive data
• Abundance of compute power + GPUs
• Frameworks for computational graph composition
(TensorFlow, Theano, Torch, Caffe)
• “Compiles” the network architecture into a highly
optimized set of commands that run quickly and with
maximum parallelism
• Symbolically differentiates the objective for gradient
descent
BigML, Inc 28Time Series / Deepnets
Deep Networks
• Like Trees / Ensembles, we have arbitrary
representational power by modifying the structure
• Like logistic regression, smooth, multivariate
objectives aren’t a problem (provided we have the
right structure)
• So what have we lost?
BigML, Inc 29Time Series / Deepnets
Deep Network Cons
• Efficiency
• The right structure for given data is not easily found,
and most structures are bad
• Solution: Try a bunch of them, and be clever about
how you do it
• Interpretability
• We’ve gotten quite far away from the interpretability of
trees
• Solution: Use sampling and tree induction to create
decision tree-like explanations for predictions
BigML, Inc 30Time Series / Deepnets
Bayesian Parameter Optimization
Model and
EvaluateStructure 1
Structure 2
Structure 3
Structure 4
Structure 5
Structure 6
BigML, Inc 31Time Series / Deepnets
Bayesian Parameter Optimization
Model and
EvaluateStructure 1
Structure 2
Structure 3
Structure 4
Structure 5
Structure 6
0.75
BigML, Inc 32Time Series / Deepnets
Bayesian Parameter Optimization
Model and
EvaluateStructure 1
Structure 2
Structure 3
Structure 4
Structure 5
Structure 6
0.75
0.48
BigML, Inc 33Time Series / Deepnets
Bayesian Parameter Optimization
Model and
EvaluateStructure 1
Structure 2
Structure 3
Structure 4
Structure 5
Structure 6
0.75
0.48
0.91
BigML, Inc 34Time Series / Deepnets
Bayesian Parameter Optimization
Structure 1
Structure 2
Structure 3
Structure 4
Structure 5
Structure 6
0.75
0.48
0.91
Model!
Structure -> performance
Model and
Evaluate
BigML, Inc 35Time Series / Deepnets
Bayesian Parameter Optimization
Structure 1
Structure 2
Structure 3
Structure 4
Structure 5
Structure 6
0.75
0.48
0.91
Model!
Structure -> performance
Model and
Evaluate
BigML, Inc 36Time Series / Deepnets
Should I use it?
• Things that make deep learning less useful:
• Small data (where that could still be thousands of instances)
• Problems where you could benefit by iterating quickly (better
features always beats better models)
• Problems that are easy, or for which top-of-the-line
performance isn’t absolutely critical
• Remember deep learning is just another sort
of classifier
“…deep learning has existed in the neural network community for over 20 years. Recent advances are
driven by some relatively minor improvements in algorithms and models and by the availability of large
data sets and much more powerful collections of computers.” — Stuart Russell
https://people.eecs.berkeley.edu/~russell/research/future/
VSSML17 L6. Time Series and Deepnets

More Related Content

What's hot

What's hot (20)

BSSML16 L1. Introduction, Models, and Evaluations
BSSML16 L1. Introduction, Models, and EvaluationsBSSML16 L1. Introduction, Models, and Evaluations
BSSML16 L1. Introduction, Models, and Evaluations
 
BSSML17 - Logistic Regressions
BSSML17 - Logistic RegressionsBSSML17 - Logistic Regressions
BSSML17 - Logistic Regressions
 
BSSML17 - Ensembles
BSSML17 - EnsemblesBSSML17 - Ensembles
BSSML17 - Ensembles
 
VSSML16 LR1. Summary Day 1
VSSML16 LR1. Summary Day 1VSSML16 LR1. Summary Day 1
VSSML16 LR1. Summary Day 1
 
BigML Education - Feature Engineering with Flatline
BigML Education - Feature Engineering with FlatlineBigML Education - Feature Engineering with Flatline
BigML Education - Feature Engineering with Flatline
 
BSSML17 - Introduction, Models, Evaluations
BSSML17 - Introduction, Models, EvaluationsBSSML17 - Introduction, Models, Evaluations
BSSML17 - Introduction, Models, Evaluations
 
BSSML16 L3. Clusters and Anomaly Detection
BSSML16 L3. Clusters and Anomaly DetectionBSSML16 L3. Clusters and Anomaly Detection
BSSML16 L3. Clusters and Anomaly Detection
 
BSSML16 L4. Association Discovery and Topic Modeling
BSSML16 L4. Association Discovery and Topic ModelingBSSML16 L4. Association Discovery and Topic Modeling
BSSML16 L4. Association Discovery and Topic Modeling
 
VSSML17 L3. Clusters and Anomaly Detection
VSSML17 L3. Clusters and Anomaly DetectionVSSML17 L3. Clusters and Anomaly Detection
VSSML17 L3. Clusters and Anomaly Detection
 
BSSML17 - Time Series
BSSML17 - Time SeriesBSSML17 - Time Series
BSSML17 - Time Series
 
VSSML16 L3. Clusters and Anomaly Detection
VSSML16 L3. Clusters and Anomaly DetectionVSSML16 L3. Clusters and Anomaly Detection
VSSML16 L3. Clusters and Anomaly Detection
 
BSSML17 - Feature Engineering
BSSML17 - Feature EngineeringBSSML17 - Feature Engineering
BSSML17 - Feature Engineering
 
BSSML17 - Basic Data Transformations
BSSML17 - Basic Data TransformationsBSSML17 - Basic Data Transformations
BSSML17 - Basic Data Transformations
 
BSSML17 - Clusters
BSSML17 - ClustersBSSML17 - Clusters
BSSML17 - Clusters
 
Feature Engineering
Feature Engineering Feature Engineering
Feature Engineering
 
DutchMLSchool. ML: A Technical Perspective
DutchMLSchool. ML: A Technical PerspectiveDutchMLSchool. ML: A Technical Perspective
DutchMLSchool. ML: A Technical Perspective
 
VSSML18. Feature Engineering
VSSML18. Feature EngineeringVSSML18. Feature Engineering
VSSML18. Feature Engineering
 
BSSML17 - Anomaly Detection
BSSML17 - Anomaly DetectionBSSML17 - Anomaly Detection
BSSML17 - Anomaly Detection
 
MLSD18. Feature Engineering
MLSD18. Feature EngineeringMLSD18. Feature Engineering
MLSD18. Feature Engineering
 
L5. Data Transformation and Feature Engineering
L5. Data Transformation and Feature EngineeringL5. Data Transformation and Feature Engineering
L5. Data Transformation and Feature Engineering
 

Similar to VSSML17 L6. Time Series and Deepnets

Dataiku productive application to production - pap is may 2015
Dataiku    productive application to production - pap is may 2015 Dataiku    productive application to production - pap is may 2015
Dataiku productive application to production - pap is may 2015
Dataiku
 

Similar to VSSML17 L6. Time Series and Deepnets (20)

VSSML18. Deepnets and Time Series
VSSML18. Deepnets and Time SeriesVSSML18. Deepnets and Time Series
VSSML18. Deepnets and Time Series
 
DutchMLSchool. Logistic Regression, Deepnets, Time Series
DutchMLSchool. Logistic Regression, Deepnets, Time SeriesDutchMLSchool. Logistic Regression, Deepnets, Time Series
DutchMLSchool. Logistic Regression, Deepnets, Time Series
 
MLSEV. Logistic Regression, Deepnets, and Time Series
MLSEV. Logistic Regression, Deepnets, and Time Series MLSEV. Logistic Regression, Deepnets, and Time Series
MLSEV. Logistic Regression, Deepnets, and Time Series
 
MLSD18. Ensembles, Logistic Regression, Deepnets
MLSD18. Ensembles, Logistic Regression, DeepnetsMLSD18. Ensembles, Logistic Regression, Deepnets
MLSD18. Ensembles, Logistic Regression, Deepnets
 
MLSEV. Automating Decision Making
MLSEV. Automating Decision MakingMLSEV. Automating Decision Making
MLSEV. Automating Decision Making
 
Web UI, Algorithms, and Feature Engineering
Web UI, Algorithms, and Feature Engineering Web UI, Algorithms, and Feature Engineering
Web UI, Algorithms, and Feature Engineering
 
BigML Education - Deepnets
BigML Education - DeepnetsBigML Education - Deepnets
BigML Education - Deepnets
 
DutchMLSchool. Automating Decision Making
DutchMLSchool. Automating Decision MakingDutchMLSchool. Automating Decision Making
DutchMLSchool. Automating Decision Making
 
DutchMLSchool. Introduction to Machine Learning with the BigML Platform
DutchMLSchool. Introduction to Machine Learning with the BigML PlatformDutchMLSchool. Introduction to Machine Learning with the BigML Platform
DutchMLSchool. Introduction to Machine Learning with the BigML Platform
 
BigML Release: PCA
BigML Release: PCABigML Release: PCA
BigML Release: PCA
 
VSSML18. Introduction to Machine Learning and the BigML Platform
VSSML18. Introduction to Machine Learning and the BigML PlatformVSSML18. Introduction to Machine Learning and the BigML Platform
VSSML18. Introduction to Machine Learning and the BigML Platform
 
Dataiku productive application to production - pap is may 2015
Dataiku    productive application to production - pap is may 2015 Dataiku    productive application to production - pap is may 2015
Dataiku productive application to production - pap is may 2015
 
VSSML18. Ensembles and Logistic Regressions
VSSML18. Ensembles and Logistic RegressionsVSSML18. Ensembles and Logistic Regressions
VSSML18. Ensembles and Logistic Regressions
 
DutchMLSchool 2022 - End-to-End ML
DutchMLSchool 2022 - End-to-End MLDutchMLSchool 2022 - End-to-End ML
DutchMLSchool 2022 - End-to-End ML
 
“Performance” - Dallas Oracle Users Group 2019-01-29 presentation
“Performance” - Dallas Oracle Users Group 2019-01-29 presentation“Performance” - Dallas Oracle Users Group 2019-01-29 presentation
“Performance” - Dallas Oracle Users Group 2019-01-29 presentation
 
Big Data Spain 2018: How to build Weighted XGBoost ML model for Imbalance dat...
Big Data Spain 2018: How to build Weighted XGBoost ML model for Imbalance dat...Big Data Spain 2018: How to build Weighted XGBoost ML model for Imbalance dat...
Big Data Spain 2018: How to build Weighted XGBoost ML model for Imbalance dat...
 
Troubleshooting Deep Neural Networks - Full Stack Deep Learning
Troubleshooting Deep Neural Networks - Full Stack Deep LearningTroubleshooting Deep Neural Networks - Full Stack Deep Learning
Troubleshooting Deep Neural Networks - Full Stack Deep Learning
 
BigML Release: Image Processing
BigML Release: Image ProcessingBigML Release: Image Processing
BigML Release: Image Processing
 
VSSML18. OptiML and Fusions
VSSML18. OptiML and FusionsVSSML18. OptiML and Fusions
VSSML18. OptiML and Fusions
 
The Past, Present, and Future of Machine Learning APIs
The Past, Present, and Future of Machine Learning APIsThe Past, Present, and Future of Machine Learning APIs
The Past, Present, and Future of Machine Learning APIs
 

More from BigML, Inc

More from BigML, Inc (20)

Digital Transformation and Process Optimization in Manufacturing
Digital Transformation and Process Optimization in ManufacturingDigital Transformation and Process Optimization in Manufacturing
Digital Transformation and Process Optimization in Manufacturing
 
DutchMLSchool 2022 - Automation
DutchMLSchool 2022 - AutomationDutchMLSchool 2022 - Automation
DutchMLSchool 2022 - Automation
 
DutchMLSchool 2022 - ML for AML Compliance
DutchMLSchool 2022 - ML for AML ComplianceDutchMLSchool 2022 - ML for AML Compliance
DutchMLSchool 2022 - ML for AML Compliance
 
DutchMLSchool 2022 - Multi Perspective Anomalies
DutchMLSchool 2022 - Multi Perspective AnomaliesDutchMLSchool 2022 - Multi Perspective Anomalies
DutchMLSchool 2022 - Multi Perspective Anomalies
 
DutchMLSchool 2022 - My First Anomaly Detector
DutchMLSchool 2022 - My First Anomaly Detector DutchMLSchool 2022 - My First Anomaly Detector
DutchMLSchool 2022 - My First Anomaly Detector
 
DutchMLSchool 2022 - Anomaly Detection
DutchMLSchool 2022 - Anomaly DetectionDutchMLSchool 2022 - Anomaly Detection
DutchMLSchool 2022 - Anomaly Detection
 
DutchMLSchool 2022 - History and Developments in ML
DutchMLSchool 2022 - History and Developments in MLDutchMLSchool 2022 - History and Developments in ML
DutchMLSchool 2022 - History and Developments in ML
 
DutchMLSchool 2022 - A Data-Driven Company
DutchMLSchool 2022 - A Data-Driven CompanyDutchMLSchool 2022 - A Data-Driven Company
DutchMLSchool 2022 - A Data-Driven Company
 
DutchMLSchool 2022 - ML in the Legal Sector
DutchMLSchool 2022 - ML in the Legal SectorDutchMLSchool 2022 - ML in the Legal Sector
DutchMLSchool 2022 - ML in the Legal Sector
 
DutchMLSchool 2022 - Smart Safe Stadiums
DutchMLSchool 2022 - Smart Safe StadiumsDutchMLSchool 2022 - Smart Safe Stadiums
DutchMLSchool 2022 - Smart Safe Stadiums
 
DutchMLSchool 2022 - Process Optimization in Manufacturing Plants
DutchMLSchool 2022 - Process Optimization in Manufacturing PlantsDutchMLSchool 2022 - Process Optimization in Manufacturing Plants
DutchMLSchool 2022 - Process Optimization in Manufacturing Plants
 
DutchMLSchool 2022 - Anomaly Detection at Scale
DutchMLSchool 2022 - Anomaly Detection at ScaleDutchMLSchool 2022 - Anomaly Detection at Scale
DutchMLSchool 2022 - Anomaly Detection at Scale
 
DutchMLSchool 2022 - Citizen Development in AI
DutchMLSchool 2022 - Citizen Development in AIDutchMLSchool 2022 - Citizen Development in AI
DutchMLSchool 2022 - Citizen Development in AI
 
Democratizing Object Detection
Democratizing Object DetectionDemocratizing Object Detection
Democratizing Object Detection
 
Machine Learning in Retail: Know Your Customers' Customer. See Your Future
Machine Learning in Retail: Know Your Customers' Customer. See Your FutureMachine Learning in Retail: Know Your Customers' Customer. See Your Future
Machine Learning in Retail: Know Your Customers' Customer. See Your Future
 
Machine Learning in Retail: ML in the Retail Sector
Machine Learning in Retail: ML in the Retail SectorMachine Learning in Retail: ML in the Retail Sector
Machine Learning in Retail: ML in the Retail Sector
 
ML in GRC: Machine Learning in Legal Automation, How to Trust a Lawyerbot
ML in GRC: Machine Learning in Legal Automation, How to Trust a LawyerbotML in GRC: Machine Learning in Legal Automation, How to Trust a Lawyerbot
ML in GRC: Machine Learning in Legal Automation, How to Trust a Lawyerbot
 
ML in GRC: Supporting Human Decision Making for Regulatory Adherence with Mac...
ML in GRC: Supporting Human Decision Making for Regulatory Adherence with Mac...ML in GRC: Supporting Human Decision Making for Regulatory Adherence with Mac...
ML in GRC: Supporting Human Decision Making for Regulatory Adherence with Mac...
 
ML in GRC: Cybersecurity versus Governance, Risk Management, and Compliance
ML in GRC: Cybersecurity versus Governance, Risk Management, and ComplianceML in GRC: Cybersecurity versus Governance, Risk Management, and Compliance
ML in GRC: Cybersecurity versus Governance, Risk Management, and Compliance
 
Intelligent Mobility: Machine Learning in the Mobility Industry
Intelligent Mobility: Machine Learning in the Mobility IndustryIntelligent Mobility: Machine Learning in the Mobility Industry
Intelligent Mobility: Machine Learning in the Mobility Industry
 

Recently uploaded

obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
yulianti213969
 
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotecAbortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh +966572737505 get cytotec
 
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
pwgnohujw
 
edited gordis ebook sixth edition david d.pdf
edited gordis ebook sixth edition david d.pdfedited gordis ebook sixth edition david d.pdf
edited gordis ebook sixth edition david d.pdf
great91
 
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
Amil baba
 
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
ju0dztxtn
 
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Valters Lauzums
 
Displacement, Velocity, Acceleration, and Second Derivatives
Displacement, Velocity, Acceleration, and Second DerivativesDisplacement, Velocity, Acceleration, and Second Derivatives
Displacement, Velocity, Acceleration, and Second Derivatives
23050636
 

Recently uploaded (20)

obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
 
Sensing the Future: Anomaly Detection and Event Prediction in Sensor Networks
Sensing the Future: Anomaly Detection and Event Prediction in Sensor NetworksSensing the Future: Anomaly Detection and Event Prediction in Sensor Networks
Sensing the Future: Anomaly Detection and Event Prediction in Sensor Networks
 
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotecAbortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
 
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
 
edited gordis ebook sixth edition david d.pdf
edited gordis ebook sixth edition david d.pdfedited gordis ebook sixth edition david d.pdf
edited gordis ebook sixth edition david d.pdf
 
SCI8-Q4-MOD11.pdfwrwujrrjfaajerjrajrrarj
SCI8-Q4-MOD11.pdfwrwujrrjfaajerjrajrrarjSCI8-Q4-MOD11.pdfwrwujrrjfaajerjrajrrarj
SCI8-Q4-MOD11.pdfwrwujrrjfaajerjrajrrarj
 
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
 
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
 
The Significance of Transliteration Enhancing
The Significance of Transliteration EnhancingThe Significance of Transliteration Enhancing
The Significance of Transliteration Enhancing
 
Aggregations - The Elasticsearch "GROUP BY"
Aggregations - The Elasticsearch "GROUP BY"Aggregations - The Elasticsearch "GROUP BY"
Aggregations - The Elasticsearch "GROUP BY"
 
MATERI MANAJEMEN OF PENYAKIT TETANUS.ppt
MATERI  MANAJEMEN OF PENYAKIT TETANUS.pptMATERI  MANAJEMEN OF PENYAKIT TETANUS.ppt
MATERI MANAJEMEN OF PENYAKIT TETANUS.ppt
 
Data Analysis Project Presentation : NYC Shooting Cluster Analysis
Data Analysis Project Presentation : NYC Shooting Cluster AnalysisData Analysis Project Presentation : NYC Shooting Cluster Analysis
Data Analysis Project Presentation : NYC Shooting Cluster Analysis
 
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
 
Predictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting TechniquesPredictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting Techniques
 
Seven tools of quality control.slideshare
Seven tools of quality control.slideshareSeven tools of quality control.slideshare
Seven tools of quality control.slideshare
 
What is Insertion Sort. Its basic information
What is Insertion Sort. Its basic informationWhat is Insertion Sort. Its basic information
What is Insertion Sort. Its basic information
 
Displacement, Velocity, Acceleration, and Second Derivatives
Displacement, Velocity, Acceleration, and Second DerivativesDisplacement, Velocity, Acceleration, and Second Derivatives
Displacement, Velocity, Acceleration, and Second Derivatives
 
Bios of leading Astrologers & Researchers
Bios of leading Astrologers & ResearchersBios of leading Astrologers & Researchers
Bios of leading Astrologers & Researchers
 
NOAM AAUG Adobe Summit 2024: Summit Slam Dunks
NOAM AAUG Adobe Summit 2024: Summit Slam DunksNOAM AAUG Adobe Summit 2024: Summit Slam Dunks
NOAM AAUG Adobe Summit 2024: Summit Slam Dunks
 
社内勉強会資料_Object Recognition as Next Token Prediction
社内勉強会資料_Object Recognition as Next Token Prediction社内勉強会資料_Object Recognition as Next Token Prediction
社内勉強会資料_Object Recognition as Next Token Prediction
 

VSSML17 L6. Time Series and Deepnets

  • 1. Valencian Summer School in Machine Learning 3rd edition September 14-15, 2017
  • 2. BigML, Inc 2Time Series / Deepnets Time Series Analysis
  • 3. BigML, Inc 3Time Series / Deepnets Beyond Supervision • Traditional machine learning data is assumed to be IID • Independent (points have no information about each other’s class) and • Identically distributed (come from the same distribution) • But what if you want to predict just the next value in a sequence? Is all lost? • Applications • Predicting battery life from change-discharge cycles • Predicting sales for the next day/week/month
  • 4. BigML, Inc 4Time Series / Deepnets Machine Learning Data Color Mass Type red 11 pen green 45 apple red 53 apple yellow 0 pen blue 2 pen green 422 pineapple yellow 555 pineapple blue 7 pen Discovering patterns within data: • Color = “red” Mass < 100 • Type = “pineapple” Color ≠ “blue” • Color = “blue” PPAP = “pen”
  • 5. BigML, Inc 5Time Series / Deepnets Machine Learning Data Color Mass Type red 53 apple blue 2 pen red 11 pen blue 7 pen green 45 apple yellow 555 pineapple green 422 pineapple yellow 0 pen Patterns valid despite reshuffling • Color = “red” Mass < 100 • Type = “pineapple” Color ≠ “blue” • Color = “blue” PPAP = “pen”
  • 6. BigML, Inc 6Time Series / Deepnets Time Series Data Year Pineapple Harvest 1986 50,74 1987 22,03 1988 50,69 1989 40,38 1990 29,80 1991 9,90 1992 73,93 1993 22,95 1994 139,09 1995 115,17 1996 193,88 1997 175,31 1998 223,41 1999 295,03 2000 450,53 Pineapple Harvest Tons 0 125 250 375 500 Year 1986 1988 1990 1992 1994 1996 1998 2000 Trend
  • 7. BigML, Inc 7Time Series / Deepnets Time Series Data Year Pineapple Harvest 1986 139,09 1987 175,31 1988 9,91 1989 22,95 1990 450,53 1991 73,93 1992 40,38 1993 22,03 1994 295,03 1995 50,74 1996 29,8 1997 223,41 1998 115,17 1999 193,88 2000 50,69 Pineapple Harvest Tons 0 125 250 375 500 Year 1986 1988 1990 1992 1994 1996 1998 2000 Patterns invalid after shuffling
  • 8. BigML, Inc 8Time Series / Deepnets Prediction Use the data from the past to predict the future
  • 9. BigML, Inc 9Time Series / Deepnets Exponential Smoothing
  • 10. BigML, Inc 10Time Series / Deepnets Exponential Smoothing Weight 0 0,05 0,1 0,15 0,2 Lag 1 3 5 7 9 11 13
  • 11. BigML, Inc 11Time Series / Deepnets Trendy 0 12,5 25 37,5 50 Time Apr May Jun Jul y 0 50 100 150 200 Time Apr May Jun Jul Additive Multiplicative
  • 12. BigML, Inc 12Time Series / Deepnets Seasonalityy 0 30 60 90 120 Time 1 4 7 10 13 16 19 y 0 35 70 105 140 Time 1 4 7 10 13 16 19 Additive Multiplicative
  • 13. BigML, Inc 13Time Series / Deepnets Errory 0 150 300 450 600 Time 1 4 7 10 13 16 19 y 0 125 250 375 500 Time 1 4 7 10 13 16 19 Additive Multiplicative
  • 14. BigML, Inc 14Time Series / Deepnets Model Types None Additive Multiplicative None A,N,N M,N,N A,N,A M,N,A A,N,M M,N,M Additive A,A,N M,A,N A,A,A M,A,A A,A,M M,A,M Additive + Damped A,Ad,N M,Ad,N A,Ad,A M,Ad,A A,Ad,M M,Ad,M Multiplicative A,M,N M,M,N A,M,A M,M,A A,M,M M,M,M Multiplicative + Damped A,Md,N M,Md,N A,Md,A M,Md,A A,Md,M M,Md,M M,N,A Multiplicative Error No Trend Additive Seasonality
  • 15. BigML, Inc 15Time Series / Deepnets Evaluating Model Fit • AIC: Akaike Information Criterion; tries to trade off accuracy and model complexity • AICc: Like the AIC, but with a sample size correction • BIC: Bayesian Information Criterion; like the AIC but penalizes large numbers of parameters more harshly • R-squared: Raw performance, the number of model parameters isn’t considered
  • 16. BigML, Inc 16Time Series / Deepnets Linear Splitting Year Pineapple Harvest 1986 139,09 1987 175,31 1988 9,91 1989 22,95 1990 450,53 1991 73,93 1992 40,38 1993 22,03 1994 295,03 1995 115,17 Random Split Year Pineapple Harvest 1986 139,09 1987 175,31 1988 9,91 1989 22,95 1990 450,53 1991 73,93 1992 40,38 1993 22,03 1994 295,03 1995 115,17 Linear Split
  • 17. BigML, Inc 17Time Series / Deepnets Deep Neural Networks
  • 18. BigML, Inc 18Time Series / Deepnets BigML Deepnets • Not Done Yet! • I’m the tech lead, so I’m the reason we don’t have a demo for this (sorry). • Check out our next release webinar! • Let’s Still Have a Chat • Deep learning is regarded in the media as some sort of strange robot messiah, destined to either save or destroy us all • What’s good about deep learning and why is it so popular now? • How much is hype and what are some of the major issues with it?
  • 19. BigML, Inc 19Time Series / Deepnets Going Further • Trees • Pro: Massive representational power that expands as the data gets larger; efficient search through this space • Con: Difficult to represent smooth functions and functions of many variables • Ensembles mitigate some of these difficulties • Logistic Regression • Pro: Some smooth, multivariate, functions are not a problem; fast optimization of chosen • Con: Parametric - If decision boundary is nonlinear, tough luck • Can these be mitigated?
  • 20. BigML, Inc 20Time Series / Deepnets LR Level Up Outputs Inputs
  • 21. BigML, Inc 21Time Series / Deepnets LR Level Up wi Class 1, logistic(w, b)
  • 22. BigML, Inc 22Time Series / Deepnets LR Level Up Outputs Inputs Hidden layer
  • 23. BigML, Inc 23Time Series / Deepnets LR Level Up Class 1, logistic(w, b) Hidden unit 1, logistic(w, b)
  • 24. BigML, Inc 24Time Series / Deepnets LR Level Up Class 1, logistic(w, b) Hidden unit 1, logistic(w, b) n nodes ?
  • 25. BigML, Inc 25Time Series / Deepnets LR Level Up Class 1, logistic(w, b) Hidden unit 1, logistic(w, b) n hidden layers?
  • 26. BigML, Inc 26Time Series / Deepnets LR Level Up Class 1, logistic(w, b) Hidden unit 1, logistic(w, b)
  • 27. BigML, Inc 27Time Series / Deepnets Why? • This isn’t new. Why the sudden interest? • Scale • Massive parameter space <=> Massive data • Abundance of compute power + GPUs • Frameworks for computational graph composition (TensorFlow, Theano, Torch, Caffe) • “Compiles” the network architecture into a highly optimized set of commands that run quickly and with maximum parallelism • Symbolically differentiates the objective for gradient descent
  • 28. BigML, Inc 28Time Series / Deepnets Deep Networks • Like Trees / Ensembles, we have arbitrary representational power by modifying the structure • Like logistic regression, smooth, multivariate objectives aren’t a problem (provided we have the right structure) • So what have we lost?
  • 29. BigML, Inc 29Time Series / Deepnets Deep Network Cons • Efficiency • The right structure for given data is not easily found, and most structures are bad • Solution: Try a bunch of them, and be clever about how you do it • Interpretability • We’ve gotten quite far away from the interpretability of trees • Solution: Use sampling and tree induction to create decision tree-like explanations for predictions
  • 30. BigML, Inc 30Time Series / Deepnets Bayesian Parameter Optimization Model and EvaluateStructure 1 Structure 2 Structure 3 Structure 4 Structure 5 Structure 6
  • 31. BigML, Inc 31Time Series / Deepnets Bayesian Parameter Optimization Model and EvaluateStructure 1 Structure 2 Structure 3 Structure 4 Structure 5 Structure 6 0.75
  • 32. BigML, Inc 32Time Series / Deepnets Bayesian Parameter Optimization Model and EvaluateStructure 1 Structure 2 Structure 3 Structure 4 Structure 5 Structure 6 0.75 0.48
  • 33. BigML, Inc 33Time Series / Deepnets Bayesian Parameter Optimization Model and EvaluateStructure 1 Structure 2 Structure 3 Structure 4 Structure 5 Structure 6 0.75 0.48 0.91
  • 34. BigML, Inc 34Time Series / Deepnets Bayesian Parameter Optimization Structure 1 Structure 2 Structure 3 Structure 4 Structure 5 Structure 6 0.75 0.48 0.91 Model! Structure -> performance Model and Evaluate
  • 35. BigML, Inc 35Time Series / Deepnets Bayesian Parameter Optimization Structure 1 Structure 2 Structure 3 Structure 4 Structure 5 Structure 6 0.75 0.48 0.91 Model! Structure -> performance Model and Evaluate
  • 36. BigML, Inc 36Time Series / Deepnets Should I use it? • Things that make deep learning less useful: • Small data (where that could still be thousands of instances) • Problems where you could benefit by iterating quickly (better features always beats better models) • Problems that are easy, or for which top-of-the-line performance isn’t absolutely critical • Remember deep learning is just another sort of classifier “…deep learning has existed in the neural network community for over 20 years. Recent advances are driven by some relatively minor improvements in algorithms and models and by the availability of large data sets and much more powerful collections of computers.” — Stuart Russell https://people.eecs.berkeley.edu/~russell/research/future/