When they are applied to spatial data, the prediction accuracy happens to be low due to the disregarded spatial dependencies among the samples. The SAR model solution of the general case requires a maximum likelihood estimation procedure involving a large number of matrix inversion and determinant computations over very large matrices for real-world applications.
Computing eigenvalues is very hard. So, we claim that there are quicker but approximate methods to solve SAR model
The dotted boxed term is the term added to linear regression to get spatial auto-regression.
There is only one parallel implementation of SAR model based on the estimation of maximum likelihood method by eigenvalue computation from [Li, 1996].
The random number generator can generate very long sequence of normal random numbers with desired mean and standard deviation. Such algorithms are rarely found
Short-cut comes from doing two matrix-vector multiplications instead of 2 matrix-matrix multiplications. D^(-1/2) is a vector instead of a diagonal matrix. W_tilda is symmetric and has got the same eigenvalues as W.
Please go back to SLIDE #10 for the second term in the log-likelihood function
Please refer to slide #10 for the funtion to optimize
Back-up Slide
Back-up slide
Back-up slide
Back-up slide
Transcript
1.
AHPCRC SPATIAL DATA-MINING TUTORIAL on Scalable Parallel Formulations of Spatial Auto-Regression (SAR) Models for Mining Regular Grid Geospatial Data Shashi Shekhar, Barış M. Kazar, David J. Lilja EECS Department @ University of Minnesota Army High Performance Computing Research Center (AHPCRC) Minnesota Supercomputing Institute (MSI) 05.14.2003
This study is the first study that offers the only available parallel SAR formulation and evaluates its scalability
All of the eigenvalues of any type of dense neighborhood (square) matrix can be computed in parallel
Scalable parallel formulations of spatial auto-regression (SAR) models for 1-D and 2-D location prediction problems for planar surface partitionings using the eigenvalue computation
Hand-parallelized EISPACK, pre-parallelized LAPACK-based NAG linear algebra libraries and shared-memory parallel programming, i.e. OpenMP are used
There are a number of sequential algorithms computing SAR model, most of which are based on the estimation of maximum likelihood method that solves for the spatial autoregression parameter ( ) and regression coefficients ( ).
As the problem size gets bigger, the sequential methods are incapable of solving this problem due to
extensive number of computations and
large memory requirement .
The new parallel formulation proposed in this study will outperform the previous parallel implementation in terms of:
The logarithm of the maximum likelihood function is called
log-likelihood function
The ML estimates of the SAR parameters:
The function to optimize:
11.
System Diagram B Golden Section Search to find that minimizes ML function A Compute Eigenvalues Pre-processing Step C Compute and given the best estimate using least squares Calculate the ML function Eigenvalues of W The Symmetric Eigenvalue-equivalent Neighborhood Matrix
Input: p =4; q =4; n = pq =16; neighborhood type (4- Neighbors)
Output: The binary (non-row-normalized) 16-by-16 C matrix; the row-sum in a 16-by-1 column vector D_onehalf; the row-normalized 16-by-16 neighborhood matrix W
The neighborhood matrix, W is formed by using the following neighborhood relationship ((i.j) is the current pixel):
Matlab programs use eig function which finds all of the eigenvalues of non-symmetric dense matrix
Fortran programs use the tred2 and tql1 EISPACK subroutines, which is the most efficient to find eigenvalues
There are two sub-steps:
Convert dense symmetric matrix to tridiagonal matrix
Find all eigenvalues of the tridiagonal matrix
21.
2.1 Convert symmetric matrix to tridiagonal matrix
Input: n ,
Output: Diagonal elements of the resulting tri-diagonal matrix in 16-by-1 column vector d , the sub-diagonal elements of the resulting tri-diagonal matrix in 16-by-1 column vector e
This is Householder Transformation which is only used in fortran programs
This step is the most-time consuming part (%99 of the total execution time)
22.
2.2 Find All Eigenvalues of the Tridiagonal Matrix
Input: Diagonal elements of the resulting tri-diagonal matrix in 16-by-1 column vector d , the sub-diagonal elements of the resulting tri-diagonal matrix in 16-by-1 column vector e
Output: All of the eigenvalues of the neighborhood matrix W
This is QL transformation which is only used in fortran programs
Fortran programs use this subroutine to perform K-optimization where constant term K =[ I - x (( x T x ) -1 ) x T ] which is 16-by-16 for n=16
The second term in log-likelihood expression
= y T ( I -rho W ) T [ I - x (( x T x ) -1 ) x T ] T [ I - x *(( x T x ) -1 )* x T ] ( I -rho W ) y
= y T ( I -rho* W )'* K T K ( I -rho W ) y
= ( K ( I -rho W ) y )T * ( K ( I -rho W ) y )
= ( Ky - rho KWy ) T * ( Ky - rho KWy )
which saves many matrix-vector multiplications
Matlab programs directly calculate all (constant and non-constant) terms in the log-likelihood function over and over again so do not need this operation
Those terms are not expensive to calculate in Matlab but expensive in fortran
Prof. Mark Bull’s “Expert Programmer vs Parallelizing Compiler” Scientific Programming 1996 paper
The loop 240 & loop 280 are the major bottlenecks and parallelized most of the code as will be shown
The data distribution on both loops should be similar to benefit from value re-use
Loop 280 cannot benefit from block-wise partitioning, it should use interleaved scheduling for load balance. Thus, both loops use interleaved scheduling
Parallelizing initialization phase imitates manual data distribution, page-placement & page-migration utilities of SGI Origin machines
The variable “etemp” enables reduction operation on the variable “e” that is updated by different processors
[ICP03] Baris Kazar, Shashi Shekhar, and David J. Lilja, "Parallel Formulation of Spatial Auto-Regression", submitted to ICPP 2003 [under review]
[IEE02] S. Shekhar, P. Schrater, R. Vatsavai, W. Wu, and S. Chawla, Spatial Contextual Classification and Prediction Models for Mining Geospatial Data , IEEE Transactions on Multimedia (special issue on Multimedia Dataabses) , 2002
Be the first to comment