This document summarizes work on estimating sparse inverse covariance matrices using graphical lasso. It discusses how graphical lasso uses an L1 regularization and coordinate descent algorithms to efficiently estimate sparse inverse covariance matrices. The new GLASSO R package was developed that is 30-4000x faster than existing methods for estimating sparse graphical models on large datasets with thousands of nodes and parameters. Future work aims to apply this approach to even larger datasets where the number of parameters exceeds the number of samples.

1. Sparse inverse covariance
estimation with graphical lasso
Ayush Singh
CCIS, Northeastern University

2. Motivation
● All matrices based on gaussian dist. have mean and covariance matrix
● Covariance is zero if elements are independent
● Inverse covariance matrix would have zeros making its sparse sometimes
● Density functions not suitable for Sparse matrices
● Estimate covariance and apply L1 to reduce to zero

3. Current Work
● L1 has been proposed for estimation of sparse undirected graphical model
● Meinshausen and Bühlmann (MB) estimate a sparse graphical model by
fitting a lasso model to each variable, using the others as predictors.
● MB approach consistently estimates nonzero in Inverse covariance matrix
● R Packages: COVSEL based on work of Banerjee and others
● Exact maximization of the L1 -penalized log-likelihood is another way
● Some used Interior-point optimization methods after proving convexity
● Banerjee and others proves MB approach does not yield the maximum
likelihood estimator.

4. Intuition
● Instead use blockwise coordinate descent algorithms to solve lasso problem
● Graphical Lasso : L1 Regularization using Coordinate Descent
● Blockwise: Partition keeping target always last and estimate based on r.v.
● Update original matrix from results of previous step
● Repeat above until convergence: change in estimate , t = 0.001
● Resulting matrix is sparse so computations are cheap

5. Results
● Birth of a new R package: GLASSO
● 30-4000x faster vs COVSEL and 2-10x slower vs MB approach on a sample 1k
node and 500k params graph
● The computation time depends strongly on the value of ρ

6. Future
● Allow application to large datasets N with thousands parameters p
● Can even be used on datasets with p > N
● Sensitive to ρ, should be chosen empirically and with due diligence
Thanks for your attention!