Reading group PPT

1. Robust Unsupervised Feature Selection on
Networked Data
Reading Group: Manqing
Paper resource: SDM 2016

2. Introduction
Networked data: encodes pairwise relations among instances in a network.
Features:
1. Structural interactions
2. High-dimensional features
Challenges:
1. The curse of dimensionality.
2. Memory storage requirements and computational costs for data analytics.
3. The existence of irrelevant, redundant and noisy features.

3. Introduction
Why feature selection?
Feature selection, as a data preprocessing step has shown to be effective in
preparing high-dimensional data for many data mining tasks such as
sentiment analysis and node classification.
How feature selection?
According to the availability of labels, feature selection methods consist of
supervised methods and unsupervised methods.

4. Introduction
1. Supervised
Minimum redundancy feature selection from microarray gene expression data. (C. Ding et al.
2005)
Efficient and robust feature selectionvia joint 2,1-norm minimization. (F. Nie et al. 2010)
Regression shrinkage and selection via the lasso. (R. Tibshirani 1996)
1. Unsupervised
Unsupervised feature selection for multi-cluster data. (D. Cai et al. 2010)
Laplacian score for feature selection. (X. He et al. 2005)
Unsupervised feature selection using nonnegative spectral analysis. (Z. Li et al. 2012)

5. Introduction
As it is easy to amass substantial amounts of unlabeled data while label information is costly to
obtain, unsupervised feature selection has received increasingly attention in the past few years.
And it exploit different criteria to define the relevance of features such as: data similarity, local
discriminative information, and data reconstruction error.
Challenges for using current model to analyze network data:
1. In networks, data instances are not independent and identically distributed but inherently
interconnected with each other. Meanwhile, they are often associated with some content
features.
2. In addition to noisy features in the content space, link information is prevalent and also
contains a lot of noise.

6. Problem Statement A robust unsupervised
feature selection framework
NetFS:
First, to capture the inherent
interactions among
networked instances,
introduce the concept of
latent representations to
uncover some hidden
attributes encoded in the
network structure.
Second, reduce the negative
effects from noisy links:
embedding the latent
representation learning into
the feature selection phase.
Specifically, content
information can help
mitigate noisy links.

7. Problem Statement
denote a set of n
linked instances in the network.
represent the network
structure of
For undirected network,
For directed network,
Feature set:
Content information:

8. Robust Unsupervised Feature Selection for Networked Data - NetFS
Modeling link information with latent representation.
Embedding latent representation learning into feature selection.
Optimization solution.
Convergence analysis.

9. 1. Modeling link information with latent representation
By a symmetric nonnegative matrix factorization model (SymNMF).
Mathematically, it decomposes the adjacency matrix A into a product of a
nonnegative matrix U and its transpose U′ in a low-dimensional latent space:

10. 2. Embedding latent Representation Learning into Feature selection
As latent factors encode some hidden attributes of instances, they should be related to some
features (or attributes) of networked instances.
Therefore, we take U as a constraint to model the content information through a multivariate linear
regression model:

11. 2. Embedding latent Representation Learning into Feature selection

12. 3. Optimization solution
Equation (3.4) is not convex in both U and W.
Besides, is not smooth.
And based on paper: Efficient and robust feature selection via joint 2,1-norms minimization, it
adopted alternating optimization scheme.
Thus, when U is fixed, the objective funciton is convex.
Therefore, we take the derivative of with respect to W to be zero, then we have:

13. 3. Optimization solution

14. 4. Convergence Analysis
Goal: to prove

15. Experiments
Datasets:BlogCatalog Flickr Epinions

16. Experiments
Datasets
Experimental settings
Quality of Selected Features by Netfs
Effect of Parameters

17. Experimental Settings
Using NMI (normalized mutual information) and ACC (accuracy) to see the clustering performance.
[Notes: Let C and C’ denote the clustering results from ground truth class labels and the predicted
cluster labels, respectively.
The definition:
The mutual information between two clusters C and C’ is:
NMI:
ACC:
So the higher the ACC and NMI values are, the better the feature selection performance is.

18. Experimental settings

19. Effect of parameters
alpha controls the sparsity of the model while beta balances the latent
representation learning and feature selection phase.

20. Future Work
First, in this work, the authors use social media as a test bed to evaluate the
proposed NetFS framework, we also would like to validate the proposed
framework on other kinds of networks such as gene networks, citation
networks.
Second, real-world networks are usually not static but evolve over time such
that both the network structure and the content information may change.
Future works can include how to perform feature selection on dynamic
networks in the future.

21. Thank you