Nowadays, every device connected to the Internet generates an ever-growing stream of data (formally, unbounded). Machine Learning on unbounded data streams is a grand challenge due to its resource constraints. In fact, standard machine learning techniques are not able to deal with data whose statistics is subject to gradual or sudden changes without any warning. Massive Online Analysis (MOA) is the collective name, as well as a software library, for new learners that are able to manage data streams. We present a research study on streaming rebalancing. Indeed, data streams can be imbalanced as static data, but there is not a method to rebalance them incrementally, one element at a time. For this reason we propose a new streaming approach able to rebalance data streams online. Our new methodology is evaluated against some synthetically generated datasets using prequential evaluation in order to demonstrate that it outperforms the existing approaches.

1. Inductive
Stream
Reasoning
Incremental
Rebalancing Learning on
Evolving Data Streams
Alessio Bernardo
& Emanuele Della Valle
& Albert Bifet
Politecnico di Milano
University of Waikato

2. Inductive
Stream
Reasoning
Agenda
10/06/2019
2Alessio Bernardo & Emanuele Della Valle & Albert Bifet
 Big data trend and Stream Reasoning
 Related work and state of the art
 Paper contributions
 Conclusion and future work

3. Inductive
Stream
Reasoning
Introduction
Big Data Trend
10/06/2019
3Alessio Bernardo & Emanuele Della Valle & Albert Bifet
https://www.amazingsupport.co.uk/2013/03/22/464

4. Inductive
Stream
Reasoning
Introduction
Stream reasoning
10/06/2019
4Alessio Bernardo & Emanuele Della Valle & Albert Bifet
Traditional Analytics Stream Reasoning based analytics
D. Dell’Aglio, E. Della Valle, F. van Harmelen, A. Bernstein: Stream reasoning: A survey and
outlook. Data Science, 1 (1-2), pp. 59-83, 2017, ISSN: 2451-8484.
E. Della Valle, S. Ceri, F. van Harmelen, D. Fensel It's a Streaming World! Reasoning upon
Rapidly Changing Information. IEEE Intelligent Systems 24(6): 83-89 (2009)

5. Inductive
Stream
Reasoning
Introduction
Standard Machine Learning Workflow
10/06/2019
5Alessio Bernardo & Emanuele Della Valle & Albert Bifet

6. Inductive
Stream
Reasoning
Introduction
The Concept Drift problem
610/06/2019 Alessio Bernardo & Emanuele Della Valle & Albert Bifet

7. Inductive
Stream
Reasoning
Introduction
Concept Drift and Streaming Machine Learning
7
• Hoeffding
Adaptive Tree
• Adaptive
Random Forest
• Temporally
Augmented
Classifier
A. Bifet, R. Gavaldà, G. Holmes, B. Pfahringer: Machine Learning for Data Streams: with
Practical Examples in MOA. The MIT Press (March 2, 2018)
10/06/2019 Alessio Bernardo & Emanuele Della Valle & Albert Bifet

8. Inductive
Stream
Reasoning
Introduction
An early attempt in Stream Reasoning
• How can we determining the optimal size of the window?
• What’s the correct way to perform evaluation?
D.F. Barbieri, D. Braga, S. Ceri, E. Della Valle, Y. Huang, V. Tresp,
A. Rettinger, H. Wermser: Deductive and Inductive Stream
Reasoning for Semantic Social Media Analytics.
IEEE Intelligent Systems 25(6): 32-41 (2010)
810/06/2019 Alessio Bernardo & Emanuele Della Valle & Albert Bifet

9. Inductive
Stream
Reasoning
Introduction
Adaptive Sliding Window (ADWIN)
9
Bifet, A. and Gavaldà, R., 2009, August. Adaptive learning from evolving data streams. In
International Symposium on Intelligent Data Analysis (pp. 249-260). Springer
10/06/2019 Alessio Bernardo & Emanuele Della Valle & Albert Bifet

10. Inductive
Stream
Reasoning
Introduction
Prequential Evaluation
10
Gama, J., Sebastião, R. and Rodrigues, P.P.: Issues in evaluation of stream learning algorithms.
In KDD (pp. 329-338). ACM, 2009, June.
10/06/2019 Alessio Bernardo & Emanuele Della Valle & Albert Bifet

11. Inductive
Stream
Reasoning
Introduction
K-Statistic
11
Bifet, A., de Francisci Morales, G., Read, J., Holmes, G., & Pfahringer, B.: Efficient online
evaluation of big data stream classifiers. In KDD (pp. 59-68). ACM, 2015, August.
10/06/2019 Alessio Bernardo & Emanuele Della Valle & Albert Bifet
Prequential evaluation
accuracy of our model
Probability that the chance
classifier correctly predicts
the label

12. Inductive
Stream
Reasoning
Introduction
Dataset Imbalance
1210/06/2019 Alessio Bernardo & Emanuele Della Valle & Albert Bifet

13. Inductive
Stream
Reasoning
Introduction
SMOTE
13
Assumption: availability of a finite static batch of data
Chawla, Nitesh V., et al. SMOTE: synthetic minority over-sampling technique. Journal of artificial
intelligence research 16 (2002): 321-357.
10/06/2019 Alessio Bernardo & Emanuele Della Valle & Albert Bifet

14. Inductive
Stream
Reasoning
Contributions
• Research on Incremental Rebalancing
Learning on Evolving Data Streams
• RebalanceStream
• RebalanceStream+
1410/06/2019 Alessio Bernardo & Emanuele Della Valle & Albert Bifet

15. Inductive
Stream
Reasoning
MOA
Standard Algorithm Implementation
1510/06/2019 Alessio Bernardo & Emanuele Della Valle & Albert Bifet

16. Inductive
Stream
Reasoning
Dataset Used
1610/06/2019 Alessio Bernardo & Emanuele Della Valle & Albert Bifet
https://moa.cms.waikato.ac.nz/

17. Inductive
Stream
Reasoning
RebalanceStream Algorithm
1710/06/2019 Alessio Bernardo & Emanuele Della Valle & Albert Bifet
Sources: https://github.com/alessiobernardo/RebalanceDataStream

18. Inductive
Stream
Reasoning
Prequential Evaluation Results
of RebalanceStream Algorithm
18
Balancelevel
10/06/2019 Alessio Bernardo & Emanuele Della Valle & Albert Bifet

19. Inductive
Stream
Reasoning
RebalanceStream+
1910/06/2019 Alessio Bernardo & Emanuele Della Valle & Albert Bifet
Sources: https://github.com/alessiobernardo/RebalanceDataStream

20. Inductive
Stream
Reasoning
Prequential Evaluation Results
of RebalanceStream+ Algorithm
20
Balancelevel
10/06/2019 Alessio Bernardo & Emanuele Della Valle & Albert Bifet

21. Inductive
Stream
Reasoning
Conclusions
21
 RebalanceStream method outperforms Base one
except in case of high level imbalance
 RebalanceStream+ method outperforms all the
others, as expected
10/06/2019 Alessio Bernardo & Emanuele Della Valle & Albert Bifet

22. Inductive
Stream
Reasoning
Future Work
Alessio Bernardo & Emanuele Della Valle & Albert Bifet 22
 Understanding and finding a solution in case of high level
imbalance
 Evaluating methods against real-word data streams
 Comparing methods in terms of computing time
 On the long term, add deductive reasoning
10/06/2019

23. Inductive
Stream
Reasoning
Thank you!
Any Question?
Alessio Bernardo
@alebernardo94
alessio.bernardo@mail.polimi.it