We propose a strategy for the problem of Portfolio Diversification in Financial Economics. This task can be seen as a clustering process performed with the exploitation of Non-Negative Matrix Factorization techniques.

1. CLUSTERING UNDERLYING STOCK
TRENDS VIA NON-NEGATIVE MATRIX
FACTORIZATION
MIDAS WORKSHOP @ ECML-PKDD 2016
Andrea Pazienza, Sabrina Francesca Pellegrino,
Stefano Ferilli, Floriana Esposito
19/09/2016 - Riva del Garda, Italy

2. Overview
1. Introduction
2. Clustering with NMFs
3. Experiments on Financial Data
4. Conclusions

3. INTRODUCTION

4. Introduction
In Market Trading, the trader needs to predict future stock prices
to determine a self-financing trading strategy that maximizes
the portfolio return.
Problem: creating and managing successful portfolios of
financial assets is a difficult practice.
Solution: Portfolio Diversification to attempt to minimize the
risk for a given amount of return.
This problem can be seen as a clustering process:
group data (e.g., stocks) into subgroups of similar behavior
(e.g., the same market trend).

5. Motivation
With K-Means it is not possible to establish the effectiveness
and coherence of the clusters when dealing with stock data1:
it tends to find spherical clusters: centroid-based clustering
does not handle the noise;
need to introduce weighted Euclidean distance instead of
standard Euclidean distance to re-evaluate centroid-based
clusters.
Proposal: Non-negative Matrix Factorization (NMF) to cluster
underlying stock trends.
1F. Cai, N. Le-Khac, and M. Kechadi. Clustering Approaches for Financial
Data Analysis: A Survey, Proceedings of DMIN 2012. pp. 1-7. 2012.

6. CLUSTERING WITH NMFS

7. Problem formulation
Market made up of m stocks S1, S2, . . . , Sm stored as a row
vector whose entries are n daily closing prices.
Suppose there are k latent bases, W1, W2, . . . , Wk; each Wj is a
n-dimensional row vector, thought as a Brownian motion.
Express each stock as linear combination of these bases with a
non negative real number Hij indicating the association degree
of the i-th stock with the basis Wj.
Using a matrix notation,
S+ H+ W±,
where S ∈ Rm×n
+ , H ∈ Rm×k
+ and W ∈ Rk×n
± .

8. NMFs
Standard definition:
S H W,
where S ∈ Rm×n
+ , H ∈ Rm×k
+ and W ∈ Rk×n
+ , and k ≤ m.
Role of k: force representation for data to capture
underlying regularities in the data
Matrices H and W are found by solving the optimization
problem
min
H≥0,
W≥0
S − H W 2
F,
where · F is the Frobenius norm.

9. Convex NMF
Convex NMF (C-NMF) allows the data matrix S to have mixed
signs. It minimizes
min
Hi 1 1,
H≥0
S − S H W 2
F,
Advantage of the convex constraint imposed on H:
interpreting the rows of H as weighted sums of certain
data points so that rows can be interpreted as centroids.

10. Convex-Hull NMF
Convex-Hull NMF (CH-NMF) is a fast technique and scales
extremely well.
The task now is to solve the following optimization problem
min S − S H W 2
F,
subject to the convexity constraints
Hi 1 1, H ≥ 0,
Wj 1 1, W ≥ 0.
This optimization problem is equivalent to projecting the
solution in the convex hull of S.
Advantage: new opportunities for data interpretation.

11. EXPERIMENTS ON FINANCIAL DATA

12. Experiments
Data gathered:
NASDAQ Stock Market
28 stocks belonging to 8 different sectors
10 years of closing prices (2518 working days)
Clustering methods applied:
NMF
C-NMF
CH-NMF
K-Means

13. Experiments
Tried different numbers of clusters:
all methods were run for each k ∈ {3, 4, . . . , 8}.
For each k, clustering evaluated in terms of:
1. plots of reconstruction of matrix S by matrix multiplication H W
2. plots of trend matrix W
3. analysis of colormaps for matrix H
4. Analysis of convergence iterations, Frobenius error and number
of attracted clusters for each method
5. Qualitative grouping of recurrent subgroups of stocks for each
method

14. Stock prices data trends for k 4

15. Figure: Trends for NMF

16. Figure: Trends for C-NMF

17. Figure: Trends for CH-NMF

18. Table: Numerical results for NMF
NMF
k # iter error # clusters
3 1528 33.7703 3
4 2355 27.1966 4
5 3358 21.0838 5
6 2523 16.9987 4
7 5000 14.6706 6
8 5000 13.5482 7

19. Table: Numerical results for C-NMF
C-NMF
k # iter error # clusters
3 500 45.7185 2
4 500 42.4148 2
5 500 40.2502 2
6 500 33.5761 2
7 500 38.6675 2
8 500 32.1786 2

20. Table: Numerical results for CH-NMF
CH-NMF
k # iter error # clusters
3 1 47.5844 2
4 1 43.9824 4
5 1 38.6585 4
6 1 56.4050 4
7 1 32.5755 5
8 1 46.2535 5

21. Figure: Colormap for NMF

22. Figure: Colormap for C-NMF

23. Figure: Colormap for CH-NMF

24. Table: NMF Clusters
k 3 k 4 k 5 k 6 k 7 k 8
3 4 5 6 3 5 8 10 12 14 15 3 4 5 6 3 4 5 6 11 15 17 18
7 8 9 10 11 15 16 16 17 18 7 8 10 11 7 8 14 16 20 21 22 23
11 14 15 16 19 20 21 19 20 23 13 14 15 16 19 20 24 24 25 26 28
17 19 20 21 22 23 24 24 25 26 17 18 19 20
22 23 24 25 25 26 28 21 22 24 25
26 28 28
12 13 18 6 7 12 13 4 5 8 10 23 26 12 15 23 2 4 5 8
14 17 18 21 22 28 26 9 10
1 2 27 27 7 11 1 2 9 27 21 28 27
1 2 4 9 3 6 13 12 10 13 17 16
18 22 25
1 2 9 27 27 3 13 14
1 2 9 11 1 6 7 12
19

25. Table: C-NMF Clusters
k 3 k 4 k 5 k 6 k 7 k 8
1 2 12 1 2 9 12 1 2 9 12 1 2 27 1 2 6 9 1 2 9 12
27 27 27 12 23 27 27
3 4 5 6 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 7 8 3 4 5 6 7
7 8 9 10 8 10 11 8 10 11 8 9 10 11 10 11 13 8 10 11
11 13 14 13 14 15 13 14 15 12 13 14 15 14 15 16 13 14 15
15 16 17 16 17 18 16 17 18 16 17 18 19 17 18 19 16 17 18
18 19 20 19 20 21 19 20 21 20 21 22 23 20 21 22 19 20 21
21 22 23 22 23 24 22 23 24 24 25 26 28 24 25 26 22 23 24
24 25 26 25 26 28 25 26 28 28 25 26 28
28

26. Table: CH-NMF Clusters
k 3 k 4 k 5 k 6 k 7 k 8
1 2 9 12 27 1 2 9 27 1 2 9 27 1 2 9 12 27 1 2 9 2 27
3 4 5 6 7 3 4 5 7 8 3 5 7 8 3 4 5 7 8 3 4 6 7 4 8 10
8 9 10 11 10 11 14 10 11 18 11 14 15 16 13 14 17 11 13 15
13 14 15 15 16 17 21 22 24 17 20 22 23 20 23 24 16 17 18
16 17 18 18 19 20 25 24 25 26 28 25 26 20 21 22
19 20 21 21 22 23 23 24 25
22 23 24 24 25 26 26 28
25 26 28 28
6 13 4 6 12 14 6 13 5 8 10 11 1 5 9 12
15 16 17 15 16 18 14 19
19 20 23 19 21 22
26 28 28
12 13 10 18 19 21 12 6 7
27 3

27. Table: K-Means Clusters
k 3 k 4 k 5 k 6 k 7 k 8
3 4 5 6 7 8 3 4 5 6 7 3 4 7 8 3 4 7 8 3 4 7 8 5 6 15 18
10 11 13 15 8 10 11 13 10 16 17 10 11 10 11 20 23 24
16 18 20 23 15 16 18 20 19 21 25 16 25 16 25 26 28
24 25 26 28 23 24 26 28 26
1 9 12 14 1 9 12 14 5 6 11 13 5 6 13 5 6 15 9 14 17
17 19 21 22 17 19 21 22 15 18 20 23 15 18 20 18 20 23 19 21 22
25 24 26 28 23 24 28 24 26 28
2 27 2 1 9 12 9 14 17 9 14 17 4 7 8 10
14 22 19 21 22 19 21 22 11 16 25
27 27 27 1 12 27
2 2 27 2
1 12 2 1 12
13 3
13

28. CONCLUSIONS

29. Conclusions
Portfolio diversification is the financial process of allocating
capital in a way that reduces the exposure to risk by investing in
a variety of assets (i.e., stocks).
This equals to clustering stocks having similar trend.
K-Means is not effective on this task. Hence, we applied NMF.
Adding convexity constraints in the transformation improves
the exploitation of similar stock trends.
In particular
CH-NMF is a very fast and scalable convex NMF technique
that compares favorably for large data sets, both in terms of
speed and reconstruction quality

30. Conclusions
Extensive experimental evaluation on real world NASDAQ
stock data show that, compared to K-Means, NMF techniques:
better point out the clustering properties,
yield very low error in Frobenius norm,
high efficiency in terms of convergence time.
Future works:
use more datasets from different markets
investigate further decomposition techniques to improve
the effectiveness of clustering stock data
impose other penalty constraints in order to achieve a
better portfolio diversification strategy