This document discusses streaming and parallel decision trees in Flink. It motivates the need for a classifier system that can learn from streaming data and classify both the streaming training data and new streaming data. It describes the architecture of keeping the classifier model fresh as new data streams in, allowing classification during the learning process in real-time. It also outlines decision tree algorithms and their implementation using Flink streaming.

1. StreamingStreamingStreamingStreaming&ParallelParallelParallelParallel
Decision TreeDecision TreeDecision TreeDecision Treein FlinkFlinkFlinkFlink
1 2 3 4
1 2 3 4 anwar.rizal @anrizal

2. MotivationMotivationMotivationMotivation
Motivation
Architecture
Decision Trees
Implementation
Conclusion

3. MotivationMotivationMotivationMotivation
Need a classifier system on streaming data
The data used for learning come as
a stream
So are the data to be classified

4. MotivationMotivationMotivationMotivation
$90 $90 $120 $90 $90 $150 $200
$90 $75 $90 $90 $90 $90 $90
$120 $90 Sold out Sold out $75 $90 $90
$120 $90 $90 $90 $100 $90 $120
(predicted) to increase zero to two days
(predicted) to increase this week
(predicted) to increase next week

5. MotivationMotivationMotivationMotivation
FRA – NYC
FRA - LON
FRA - MEX
Need attention
Revenue decrease
Need attention
passenger decrease
Need attention
revenue decrease,
cost increase

6. MotivationMotivationMotivationMotivation
Need a classifier system on streaming data
The data used for learning come as
a stream
So are the data to be classified

7. MotivationMotivationMotivationMotivation
The classifier is kept fresh
No need for separate batch learning/evaluation
The feedback is taken into account in real time, regularly
The classifier can be introspected
Transparent model structure
(e.g. know the tree, information gain for each split
point)
Known expected performance (accuracy, precision, recall,
AUC)
Seamless support for workflow of machine
learning
Data preprocessing: up/down sampling, imputations, …
Feature selections
Model evaluation, cross validation,
MUST

8. MotivationMotivationMotivationMotivation
The classifier is immediately available
The classifier can already predict during learning
When learning phase is terminated, it starts another cycle of
learning
The classifier has a meta-learning capability
The classifier has several models different parameters
It is possible to learn about the learning capability of the
models
NICE TO HAVE

9. MotivationMotivationMotivationMotivation
Learning Learning &
Classifying
End of
learning
New cycle of
learning
Cycle of
Learning, Classifying during Learning, End
of Learning, Classifying, New Learning

10. MotivationMotivationMotivationMotivation
Classifying Application
Stream Learner
Labeled
points
Classifier Predicted
points
Unlabeled
points

11. MotivationMotivationMotivationMotivation
Decision Tree Algorithms Outline
Implementation using Flink Streaming
Further Discussions

12. Thanks!
Credit to: Yiqing Yan (Eurecom) & Tianshu Yang (Telecom Bretagne), Amadeus Interns