ShiftTree: model based time series classifier (ECML/PKDD 2011 presentation)

ShiftTree: model based time series classifier (ECML/PKDD 2011 presentation)
SHIFTTREE: AN INTERPRETABLE MODEL-BASED APPROACH FOR TIME SERIES CLASSIFICATION Balázs Hidasi - hidasi@tmit.bme.hu Csaba Gáspár-Papanek - gaspar@tmit.bme.hu Budapest University of Technology and Economics, Hungary ECML/PKDD, 6TH SEPTEMBER 2011, ATHENS
OUTLINE  General concept  Task & adaptation  Labeling & learning  Evaluation  Forest methods  Conclusion 2/17
GENERAL CONCEPT  Not limited to time series  Time series  Semi-structured data  Graphs  Two questions:  „Where to look at?”  „What to observe?”  Operator families  EyeShifter Operator(s) (ESO) F(x)  ConditionBuilder Operators (CBO)  Dynamic attributes  Model: sequence of simple operatos / rules 3/17
TIME SERIES CLASSIFICATION  Time series Learning Model  With one observed variable  Evenly sampled  The algorithm has no such limitations  Standard classification task  Focusing on accuracy Model Labeling  Goals  Satisfying accuracy  Model-based instead of memory-based 4/17  Interpretable models
EXPLAINING THE GOALS  Why model-based?  Labeling is much faster   Can learn more general properties of the classes   Needs more training samples   What is interpretability good for?  Getting user trust   Helping in understanding the data  5/17
CONCEPT APPLIED TO TIME SERIES  Dynamic attributes of time series  EyeShifter Operator (ESO): „Where to look at?”  Moves a cursor along the time axis  To a specified point of the series  E.g.: „To the next local minimum”, „100 steps ahead”, etc.  Note: result of operator sequence depens on the order  ConditionBuilder Operator (CBO): „What to observe?” F(x)  Computes the dynamic attribute  E.g.: „Value at the given time”, „Length of the jump”, etc.  Decision tree as model Weighted average  Works well with the dynamic attribute concept 6/17  Operator sequence from root to leaf
LABELING Level 0 ESO: To global max CBO: Value at time Threshold: 1,20775 Level 1 Level 1 ESO: Step forward 100 ESO: To next local max CBO: Value at time CBO: Value at time Threshold : 0,390093 Threshold : -0,700739 Level 2 Level 2 Level 2 Level 2 7/17 Leaf Leaf Leaf Leaf
LABELING Dynamic attribute value: 0,836848 Level 0 ESO: To global max CBO: Value at time Threshold: 1,20775 Level 1 Level 1 ESO: Step forward 100 ESO: To next local max CBO: Value at time CBO: Value at time Threshold : 0,390093 Threshold : -0,700739 Level 2 Level 2 Level 2 Level 2 7/17 Leaf Leaf Leaf Leaf
LEARNING (1/3)  Operator pool definition: ESOs, CBOs  Root node  Cursor starts at the beginning  May be overridden 8/17
F(x) LEARNING (2/3)  Available dynamic attributes (CBO pool)  At/around reachable positions  Selecting a…  Attribute  Threshold value Minimize weighted child node entropy 9/17
LEARNING (3/3)  Best split selected  Cursor is moved by ESO  Reachable positions will change  Train set split  Child nodes continue the learning process  Stop: homogenous nodes 10/17
INTERPRETABILITY  Depends on the operators  Helps undepstanding the data  E.g.: CBF dataset  Z-normalized, 3 class: Cylinder, Bell, Funnel  Distinguish cylinder from bell and funnel by global maxima  The data is z-normalized (standardized)  Distinguish bell from funnel by stepping back 25 steps + noise 11/17 filtering through weighted average  On which side is the peak
PERFORMANCE OF THE ALGORITHM  Databases  UCR: mostly smaller data sets (20)  Ford: larger data sets (2)  No optimizations in the experiments  Performance  Better on larger data sets (model-based) 100.00% 94.97% 96.52% 92.94% 90.00% 81.29% 83.19% 80.00% 74.67% 73.33% 69.11% 70.00% 64.93% 60.00% 50.00% 40.00% 30.00% 20.00% 10.00% 12/17 0.00% Smaller UCR sets (15) Larger UCR sets (5) Ford (largest) sets (2) 1NN Eucledian 1NN DTW ShiftTree
ADVANTAGES AND DRAWBACKS  Advantages  Advantages of being model based (fast labeling, etc)  Interpretable (depens on operators)  Can be used independently of application domain  Expert’s knowledge can be built in through operators  Disadvantages  Optimization of the operators is not trivial  Learning may take a long time when too many operators used  Disadvantages of the model based methods (larger training set required, etc) 13/17
SHIFTFOREST: COMBINING MULTIPLE MODELS  Combining models enhances accuracy  Boosting  Some issues on smaller datasets  The „XV” method  Splitting training set (random splits)  Building model on the first part, evaluating on the second  Model weight: the measured accuracy 14/17
SHIFTFOREST RESULTS  Accuracy increases  Interpretability is often lost  May the intersection of the trees can be interpreted 100.00% 96.52%97.77% 92.94%94.97% 88.87% 90.00% 85.93% 81.29% 83.19% 80.00% 74.67% 73.33% 69.11% 70.00% 64.93% 60.00% 50.00% 40.00% 30.00% 20.00% 10.00% 0.00% Smaller UCR sets (15) Larger UCR sets (5) Ford (largest) sets (2) 15/17 1NN Eucledian 1NN DTW ShiftTree ShiftForest
CONCLUSION  Novel model-based algorithm  Uses cursors and operators  Cursor movement  Attribute computation  Sequence of simple operators as the model  Decision tree  Base of a whole algorithm family 16/17
THANK YOU FOR YOUR ATTENTION! Balázs Hidasi Csaba Gáspár-Papanek (hidasi@tmit.bme.hu) (gaspar@tmit.bme.hu) 17/17 For more ShiftTree related research visit my side: http://www.hidasi.eu
TIME OF MODEL BUILDING, SCALABILITY  O(N*log(N)) per node (ordering)  Time of model building  Depends on tree structure 200  Acceptable 180 160 140  Scales linearly 120 Time (s)  Experiments 100 80 60 40 20 0 Size of training set Min of 20 Max of 20 Mean of 20
LEARNING THE MODEL Defined operators EyeShifter – Cursor position ESONextMax ESONextMax To the next local maxima ESO ESONext100 ESONext100 Step forward 100 ESOMax ESOMax To the global maxima CBO CBSimple CBOSimple Value at cursor position ShiftTree F(x) ConditionBuilder - Ordering Best split in node (so far) ESO: ESONextMax ESONext100 ESOMax - CBO: CBOSimple - Dynamic attribute selection M Threshold: -0,739969 -0,332858 -0,700739 0,390093 1,02775 - Best threshold by current dynamic attribute: -0,332858 -0,739969 -0,700739 0,390093 0,369887 -1,16155 1,02775 0,77183 2,48622 - Score of the best split by current dyn. attribute : 0,805777 0,985078 0,983634 0,600131 1,32624 Inf 0 Best score in the node (so far): 0,985078 0,983634 0,805777 Inf 0 Best ordering: 27
LEARNING THE MODEL Defined operators EyeShifter – Cursor position ESONextMax ESONextMax To the next local maxima ESO ESONext100 ESONext100 Step forward 100 ESOMax ESOMax To the global maxima CBO CBSimple CBOSimple Value at cursor position ShiftTree F(x) ConditionBuilder - Ordering Best split in node (so far) ESO: ESONextMax ESONext100 ESOMax - CBO: CBOSimple - Dynamic attribute selection M Threshold: -0,739969 -0,332858 -0,700739 0,390093 1,02775 - Best threshold by current dynamic attribute: -0,739969 -0,700739 0,390093 0,369887 -1,16155 1,02775 0,77183 2,48622 - Score of the best split by current dyn. attribute : 0,805777 0,983634 0,600131 1,32624 Inf 0 Best score in the node (so far): 0,985078 0,983634 0,805777 Inf 0 Best ordering: 34
LEARNING THE MODEL Defined operators EyeShifter – Cursor position ESONextMax ESONextMax To the next local maxima ESO ESONext100 ESONext100 Step forward 100 ESOMax ESOMax To the global maxima CBO CBSimple CBOSimple Value at cursor position ShiftTree F(x) ConditionBuilder - Ordering Best split in node (so far) ESO: ESONextMax ESONext100 ESOMax - CBO: CBOSimple - Dynamic attribute selection M Threshold: -0,739969 -0,332858 -0,700739 0,390093 1,02775 - Best threshold by current dynamic attribute: -0,739969 -0,700739 0,390093 0,369887 1,02775 0,77183 2,48622 - Score of the best split by current dyn. attribute : 0,805777 0,983634 0,600131 Inf 0 Best score in the node (so far): 0,983634 0,805777 Inf 0 Best ordering: 37
LEARNING THE MODEL Defined operators EyeShifter – Cursor position ESONextMax ESONextMax To the next local maxima ESO ESONext100 ESONext100 Step forward 100 ESOMax ESOMax To the global maxima CBO CBSimple CBOSimple Value at cursor position ShiftTree F(x) ConditionBuilder - Ordering Best split in node (so far) ESO: ESONextMax ESONext100 ESOMax - CBO: CBOSimple - Dynamic attribute selection M Threshold: -0,739969 -0,700739 0,390093 1,02775 - Best threshold by current dynamic attribute: -0,739969 -0,700739 0,390093 0,369887 1,02775 0,77183 2,48622 - Score of the best split by current dyn. attribute : 0,805777 0,983634 0,600131 Inf 0 Best score in the node (so far): 0,983634 0,805777 Inf 0 Best ordering: 38
LEARNING THE MODEL Defined operators EyeShifter – Cursor position ESONextMax ESONextMax To the next local maxima ESO ESONext100 ESONext100 Step forward 100 ESOMax ESOMax To the global maxima CBO CBSimple CBOSimple Value at cursor position ShiftTree F(x) ConditionBuilder - Ordering Best split in node (so far) ESO: ESONextMax ESONext100 - CBO: CBOSimple - Dynamic attribute selection M Threshold: -0,739969 -0,700739 0,390093 - Best threshold by current dynamic attribute: -0,739969 -0,700739 0,390093 0,369887 1,02775 0,77183 2,48622 - Score of the best split by current dyn. attribute : 0,805777 0,983634 0,600131 Inf 0 Best score in the node (so far): 0,983634 0,805777 Inf 0 Best ordering: 39
LEARNING THE MODEL Defined operators EyeShifter – Cursor position ESONextMax ESONextMax To the next local maxima ESO ESONext100 ESONext100 Step forward 100 ESOMax ESOMax To the global maxima CBO CBSimple CBOSimple Value at cursor position ShiftTree F(x) ConditionBuilder - Ordering Best split in node (so far) ESO: ESONextMax ESONext100 - CBO: CBOSimple - Dynamic attribute selection M Threshold: -0,739969 -0,700739 0,390093 - Best threshold by current dynamic attribute: -0,739969 -0,700739 0,390093 0,369887 0,77183 2,48622 - Score of the best split by current dyn. attribute : 0,805777 0,600131 Inf 0 Best score in the node (so far): 0,805777 Inf 0 Best ordering: 40
LEARNING THE MODEL Defined operators EyeShifter – Cursor position ESONextMax ESONextMax To the next local maxima ESO ESONext100 ESONext100 Step forward 100 ESOMax ESOMax To the global maxima CBO CBSimple CBOSimple Value at cursor position ShiftTree F(x) ConditionBuilder - Ordering Best split in node (so far) ESO: ESONextMax ESONext100 - CBO: - CBOSimple Dynamic attribute selection M Threshold: -0,739969 -0,700739 0,390093 - Best threshold by current dynamic attribute: -0,739969 -0,700739 0,390093 0,369887 0,77183 2,48622 - Score of the best split by current dyn. attribute : 0,805777 0,600131 Inf 0 Best score in the node (so far): 0,805777 Inf 0 Best ordering: 44
LEARNING THE MODEL Defined operators EyeShifter – Cursor position ESONextMax ESONextMax To the next local maxima ESO ESONext100 ESONext100 Step forward 100 ESOMax ESOMax To the global maxima CBO CBSimple CBOSimple Value at cursor position ShiftTree F(x) ConditionBuilder - Ordering Best split in node (so far) ESO: ESONextMax ESONext100 - CBO: - CBOSimple Dynamic attribute selection M Threshold: -0,739969 -0,700739 0,390093 - Best threshold by current dynamic attribute: -0,700739 0,390093 0,369887 0,77183 2,48622 - Score of the best split by current dyn. attribute : 0,600131 Inf 0 Best score in the node (so far): Inf 0 Best ordering: 47
LEARNING THE MODEL Defined operators EyeShifter – Cursor position ESONextMax ESONextMax To the next local maxima ESO ESONext100 ESONext100 Step forward 100 ESOMax ESOMax To the global maxima CBO CBSimple CBOSimple Value at cursor position ShiftTree F(x) ConditionBuilder - Ordering Best split in node (so far) ESO: ESONextMax ESONext100 - CBO: - CBOSimple Dynamic attribute selection M Threshold: -0,700739 0,390093 - Best threshold by current dynamic attribute: -0,700739 0,390093 0,369887 0,77183 2,48622 - Score of the best split by current dyn. attribute : 0,600131 Inf 0 Best score in the node (so far): Inf 0 Best ordering: 48
LEARNING THE MODEL Defined operators EyeShifter – Cursor position ESONextMax ESONextMax To the next local maxima ESO ESONext100 ESONext100 Step forward 100 ESOMax ESOMax To the global maxima CBO CBSimple CBOSimple Value at cursor position ShiftTree F(x) ConditionBuilder - Ordering Best split in node (so far) ESO: ESONextMax - CBO: - CBOSimple Dynamic attribute selection M Threshold: -0,700739 - Best threshold by current dynamic attribute: -0,700739 0,390093 0,369887 0,77183 2,48622 - Score of the best split by current dyn. attribute : 0,600131 Inf 0 Best score in the node (so far): Inf 0 Best ordering: 49
LEARNING THE MODEL Defined operators EyeShifter – Cursor position ESONextMax ESONextMax To the next local maxima ESO ESONext100 ESONext100 Step forward 100 ESOMax ESOMax To the global maxima CBO CBSimple CBOSimple Value at cursor position ShiftTree F(x) ConditionBuilder - Ordering Best split in node (so far) ESO: ESONextMax - CBO: - CBOSimple Dynamic attribute selection M Threshold: -0,700739 - Best threshold by current dynamic attribute: -0,700739 0,369887 2,48622 - Score of the best split by current dyn. attribute : Inf 0 Best score in the node (so far): Inf 0 Best ordering: 50
LEARNING THE MODEL Defined operators EyeShifter – Cursor position ESONextMax ESONextMax To the next local maxima ESO ESONext100 ESONext100 Step forward 100 ESOMax ESOMax To the global maxima CBO CBSimple CBOSimple Value at cursor position ShiftTree F(x) ConditionBuilder - Ordering Best split in node (so far) ESO: ESONextMax - CBO: - CBOSimple Dynamic attribute selection M Threshold: -0,700739 - Best threshold by current dynamic attribute: -0,700739 0,369887 2,48622 Score of the best split by current dyn. attribute : 0 Best score in the node (so far): 0 Best ordering: 53
LEARNING THE MODEL Defined operators EyeShifter – Cursor position ESONextMax ESONextMax To the next local maxima ESO ESONext100 ESONext100 Step forward 100 ESOMax ESOMax To the global maxima CBO CBSimple CBOSimple Value at cursor position ShiftTree F(x) ConditionBuilder - Ordering Best split in node (so far) ESO: ESONextMax - CBO: - CBOSimple Dynamic attribute selection M Threshold: -0,700739 - Best threshold by current dynamic attribute: -0,700739 0,369887 2,48622 Score of the best split by current dyn. attribute : 0 Best score in the node (so far): 0 Best ordering: 54
LEARNING THE MODEL Defined operators EyeShifter – Cursor position ESONextMax ESONextMax To the next local maxima ESO ESONext100 ESONext100 Step forward 100 ESOMax ESOMax To the global maxima CBO CBSimple CBOSimple Value at cursor position ShiftTree F(x) ConditionBuilder - Ordering Best split in node (so far) ESO: - CBO: - Dynamic attribute selection M Threshold: - Best threshold by current dynamic attribute: -0,700739 0,369887 2,48622 Score of the best split by current dyn. attribute : 0 Best score in the node (so far): 0 Best ordering: 55

Check these out next

ShiftTree: model based time series classifier (ECML/PKDD 2011 presentation)

Recommended

More Related Content

Viewers also liked(6)

More from Balázs Hidasi(15)

Recently uploaded(20)

ShiftTree: model based time series classifier (ECML/PKDD 2011 presentation)