Circuit analysis with matlab computing and simulink-modeling
JasmineAlstonThesis
1. Numerical and Algebraic Techniques for Computing Matrix Determinant
By
Jasmine Renee Alston
A thesis submitted to the Faculty of the College of Graduate Studies of Virginia
State University in partial fulfillment of the requirements for the degree of Masters
of Science in Mathematics in the College of Engineering and Technology
Virginia
2014
Approved by:
Chair/Advisor:
Dr. Mohammad Tabanjeh
Committee Member:
Dr. Raymond Fletcher
Committee Member:
Dr. Tariq Qazi
2. ABSTRACT
Numerical and Algebraic Techniques for Computing Matrix Determinant
By: Jasmine Renee Alston
(Under the direction of Dr. Mohammad Tabanjeh, Raymond Fletcher, Tariq Qazi)
The computation of the sign of a matrix determinant is a well-known problem.
This problem finds its application in computational geometry, such as computation
of convex hulls, Voronoi diagrams, and testing whether the line intervals of a given
family have nonempty common intersection. In these applications, we need to deter-
mine whether det(A) > 0, det A < 0, or (det A) vanishes for an n × n matrix A.
In this thesis, we review, examine, and combine several numerical and algebraic
techniques in order to achieve our main goal of decreasing the bit-precision for com-
puting det(A) or its sign. The resulting algorithms enable us to obtain the solution
of the problem in fewer arithmetic operations. Among the techniques we research
are the triangular and orthogonal algorithms, the Chinese reminder theorem, p-adic
Hensel lifting, and the computation of determinant modulo a prime p. Finally, we
conclude that the numerical algorithms for computing different factorization of the
input matrix A seems to be the most effective algorithms for computing the sign of
det(A) provided that | det(A)| is large enough relative to the computer precision.
ii
3. BIOGRAPHICAL SKETCH OF THE AUTHOR
Jasmine Renee Alston, the daughter of Larry Alston / John T. Williams Jr. and
Michelle Alston-Williams, was born on August 16, 1989 in Newark, New Jersey. The
middle child of an older brother, Justin Alston and a younger sister, Janntae Alston.
After moving to North Carolina, Jasmine received her Bachelors degree in Mathe-
matics at East Carolina University in 2011. Shortly after Jasmine went on to graduate
school at Virginia State University. She has tutored many mathematics courses in
Richmond Public School System including Math 8, Algebra I, and Algebra II. This
has lead her to become a research assistant at Virginia State University, focusing on
the areas of Numerical Computer Algebra.
Jasmine future endeavors include furthering her studies in Mathematics and teach-
ing on the collegiate level. She desires one day to run her own company where math-
ematical models can be put to use to assist in medical research.
iii
4. ACKNOWLEDGEMENTS
I would like to take the opportunity to thank the efforts of those who assisted me
in the completion of my thesis. First and foremost, God, who gave me the strength
and ability to the endure this task and give me complete understanding of the work.
Next, I would like to thank my advisor, Dr. Mohammad Tabanjeh, for the guidance
and encouragement to make it through the process. For the wisdom and time he
has shared to make sure I completed every detail. I would also like to thank the
faculty and staff of the VSU Mathematics and Computer Science Department for
their support and encouragement, and the HBCU MS Graduate Assistantship for
providing financial and academic support.
I should not forget to acknowledge my fellow classmates. For the countless hours
of laughter to help relieve some of the stress and pressure, and the critical feedback
that was needed in the completion of my studies. Additionally I would like to thank
Mr.Eugene Evans. For the numerous nights spent on editing and assisting me in
completion of this thesis, and for the support and love shown on this journey.
Last but not least, I would like to thank my family. For the encouragement, strong
support, and faith in me that they have shown. For the multitude of prayers that
were sent up for the endurance needed to complete my research.
iv
5. TABLE OF CONTENTS
LIST OF TABLES................................................................................................... vii
LITERATURE OF REVIEW................................................................................... 1
INTRODUCTION.................................................................................................... 3
CHAPTER 1: MATRIX DETERMINANT - PRELIMINARY................................ 5
1.1 Definitions and Basic Facts ........................................................................... 5
1.2 Properties of Determinant.............................................................................. 6
1.3 Vector Norms ................................................................................................ 9
1.4 Matrix Norms .............................................................................................. 10
1.5 The Properties of Matrix Norm................................................................... 10
1.6 The Effect of Elementary Row Operations on Determinants........................ 12
CHAPTER 2: NUMERICAL COMPUTATIONS OF MATRIX
DETERMINANT ............................................................................................. 14
2.1 The LU - Factorization................................................................................ 14
2.2 The Permutation Matrices ........................................................................... 15
2.3 The Permuted LU - Factorization ............................................................... 16
2.4 Gaussian Elimination with Partial Pivoting ................................................ 18
2.5 Gaussian Elimination with Complete Pivoting ........................................... 19
2.6 The QR - Factorization ................................................................................ 21
2.7 Modified Gram - Schmidt Algorithm .......................................................... 22
2.8 Householder Algorithm ................................................................................ 25
2.9 Givens Algorithm ........................................................................................ 29
v
6. CHAPTER 3: ALGEBRAIC COMPUTATIONS OF MATRIX
DETERMINANT ............................................................................................. 32
3.1 Determinant Computation Based on the Chinese Remainder Theorem ..... 32
3.2 Modular Determinant .................................................................................. 35
3.3 Modular Determinant of a fixed prime......................................................... 36
3.4 Matrix Singularity ....................................................................................... 36
CHAPTER 4: ALTERNATIVE APPROACH FOR COMPUTING MATRIX
INVERSES AND DETERMINANTS ............................................................... 39
4.1 Newton-Hensel’s lifting (p-adic) ................................................................. 39
4.2 Newtons iteration and p-adic lifting of matrix inverses............................... 40
4.3 p-adic lifting for the computation of matrix determinant............................ 42
4.4 Faster p-adic lifting of the determinant ....................................................... 47
CHAPTER 5: NUMERICAL IMPLEMENTATION OF MATRIX
DETERMINANT ............................................................................................. 50
5.1 Numerical experiment of the solution based on Gaussian elimination.......... 50
CHAPTER 6: COMPUTATIONAL COMPLEXITY AND PRECISION.............. 56
CHAPTER 7: DISCUSSION.................................................................................. 59
CHAPTER 8: CONCLUSION................................................................................ 61
APPENDIX: MATLAB CODE............................................................................... 63
BIBLIOGRAPHY .................................................................................................. 66
vi
7. LIST OF TABLES
Table 1: Average estimated relative error on random matrices............................. 55
vii
8. LITERATURE OF REVIEW
In computational geometry, one needs to test the sign of the determinant of an
n × n matrix. Finding the determinant by the definition is a difficult task, especially
if the size of the matrix large. Therefore, we propose several numerical and algebraic
techniques for evaluating the determinant of a matrix or its sign and we compare the
arithmetic operations of these algorithms as well as the bit precision. These methods
include Gaussian elimination (triangular factorization) of the input matrix and the
QR (orthogonal factorization) such as the QR based on modified Gram-Schmidt,
Householder, and the Givens algorithms. On the other hand, we show some other
effective techniques, namely, algebraic techniques including p-adic lifting, the Chinese
remainder theorem, and modular determinant of a fixed prime. Those techniques have
been proposed for fast and exact determinant computation.
The most powerful general technique computes the determinant of an integer
matrix using modular arithmetic [20]. The matrix determinant is computed modulo
several primes, and the complete determinant is reconstructed from the residues.
Bronnimann, Emiris, and Pan in [20] have recently improved the reconstruction step
by avoiding the use of multiprecision arithmetic and reduced the computation to
single or double precision. These algorithms seem to be among the best ones for
this problem. The algorithm of [18] complement ones of [20] by computing det A
numerically. The correctness of the output is certified unless the algorithm establishes
a relatively small upper bound on | det A|. Clarkson [6] proposes determinant sign
algorithm based on Gram-Schmidt orthogonalization. The orthogonal preconditions
the matrix for safe determinant sign computation by Gaussian elimination for larger
matrices.
9. The algorithm requires arithmetic at a higher precision than machine precision. The
algorithm of [3] competes with one of [6] for n ≤ 4 but does not work well for larger
n.
Pan and Yu [21] instead solved a much simpler task of estimating the minimum
distance N =
1
||A−1||
from A to a singular matrix. This gave them a desired range
for the invariance of the sign of det A. They improved the algorithms and estimates
of [18].
While adaptive precision has been proposed and used, the resulting code and
time complexity are still larger than one might hope. Clarkson [6] takes the approach
of exact evaluation of the determinant and its sign with low relative error. One
limitation of this approach is that the matrix entries are required to be integers
rather than rational numbers. The algorithm yields a QR factorization of A and is
an adaptation of the modified Gram-Schmidt procedure.
2
10. INTRODUCTION
Most decisions in geometric algorithms are based on evaluation of the determinant
and testing the signs of such determinants. For example, deciding if a point belongs
to a give half-space or simply the arrangement of lines and line segments. By such
numerical test, a combinatorial structure is defined. If the tests are incorrect, the
resulting structure may be different from a sensible result, and programs for com-
puting the structures may crash. Therefore, it is crucial to have reliable answers to
such test. Several approaches have been proposed on how to obviate the shortcom-
ings. The common objective is to produce robust algorithms whose answer is a small
perturbation of the correct answer. There are basically two categories of approaches
to this objective. The most common one is the redesign of algorithms to use limited
precision arithmetic and still guarantee sensible answers. The other uses exact inte-
ger computations, but the implementation of this approach has a large performance
penalty.
Our approach falls in the exact integer arithmetic category and our objective
is to use as few bits as possible to evaluate the determinant of a matrix (without
blowing up the arithmetic complexity bounds). I did not restrict my research to
a single algorithm but considered a broad range of various techniques of numerical
linear algebra and rational algebraic computations (with no error). I chose these
techniques based on the joining together of the articles I research. Some articles used
only numerical techniques were others just focused on one major algorithm. I decided
to combine the algorithms that were already studied and add the Newton-Hensel’s
lifting. I went further into detail on how to compute each algorithm for a matrix A.
We studied the problem of computing the determinant of a matrix or testing the
11. singularity for a matrix (det A = 0 or not). We demonstrated some effective ap-
proaches of combining algebraic and numerical techniques. High speed of these com-
putations is ensured as they are performed numerically, with a fixed (single or double
precision). The major problem, is to certify that the output is correct in the presence
of rounding errors. In particular, numerical algorithms for computing various fac-
torizations for a matrix A include the QR and its triangular PLU, LU, and PLUP1
factorizations, seem to be the most effective for computing the sign of det(A) provided
that | det A| is large enough relatively to computer precision.
4
12. CHAPTER 1
MATRIX DETERMINANT-PRELIMINARY
1.1 Definitions and Basic Facts.
Definition 1.1.1. For an n × n matrix A, the determinant of A is given by
det(A) = |A| =
n
j=1
aijCij = ai1Ci1 + ai2Ci2 + . . . + ainCin, (1.1.1)
or
det(A) = |A| =
n
i=1
aijCij = a1jC1j + a2jC2j + . . . + anjCnj. (1.1.2)
Cij are the cofactors of the element aij defined below. (1.1.1) is the i-th row expansion
and (1.1.2) is the j-th column expansion.
Definition 1.1.2. If A is a square matrix, then the minor Mij of the element aij is
the determinant of the matrix obtained by deleting the i-th row and j-th column of
A. The cofactor Cij is given by
Cij = (−1)i+j
Mij.
Remark 1.1.1. The determinant of a matrix of order 1 is simply defined as the entry
of the matrix. For instance, if A = [−2], then det(A) = −2.
Definition 1.1.3. For a 2 × 2 matrix A =
a11 a12
a21 a22
, the determinant is given
by det(A) = |A| = a11a22 − a21a12.
13. Example 1.1.1: Let A =
0 4 2
6 −2 4
8 0 2
. Now, let us compute the cofactors:
C11 = (−1)2
M11 =
−2 4
0 2
= −4,
C12 = (−1)3
M12 = −1
6 4
8 2
= 20,
C13 = (−1)4
M13 =
6 −2
8 0
= 16.
Then, by definition 1.1.1, |A| = a11C11 + a12C12 + a13C13
= 0(−4) + 4(20) + 2(16) = 112.
1.2 Properties of Determinant
1) The determinant of the product of two matrices is the product of determinants,
that is, det(AB) = det(A)det(B).
Example 1.2.1: Let A =
2 −4 4
0 6 4
2 0 2
, and B =
4 0 2
0 −2 −4
6 2 −4
. The matrix
product AB is,
AB =
2 −4 4
0 6 4
2 0 2
4 0 2
0 −2 −4
6 2 −4
=
32 16 4
24 −4 −40
20 4 −4
.
Now, since |A| = −56, |B| = 88, and |AB| = −4928, then we have
|AB| = |A||B| = (−56)(88) = −4928.
Therefore, the determinant of the matrix product is equal to the product of the
6
14. determinants.
Remark 1.2.1. Note that the sum of the determinants of two matrices does not
equal the determinant of their sum. That is, in general, |A| + |B| = |A + B|.
Example 1.2.2: If A =
1 2
5 3
and B =
0 3
−2 −1
.
Then |A| = −7 and |B| = 6, which implies that |A| + |B| = −7 + 6 = −1.
However, A + B =
1 5
3 2
and then |A + B| = 2 − 15 = −13.
Hence |A + B| = |A| + |B|.
2) The determinant of the transpose is the same as the determinant of the matrix.
That is, |AT
| = |A|.
Example 1.2.3: Let A =
6 2 −4
4 0 0
−8 −2 10
.
If we expand the determinant along the second row, then we have
|A| = 4(−1)3
2 −4
−2 10
= (−4)(20 − 8) = (−4)(12) = −48.
Now, AT
=
6 4 −8
2 0 −2
−4 0 10
, its determinant can be found by expanding cofactors
along the second column.
|AT
| = 4(−1)3
2 −2
−4 10
= (−4)(20 − 8) = (−4)(12) = −48.
Hence, |A| = |AT
|.
7
15. 3) For an n × n matrix A, |cA| = cn
|A|, where c is a real number.
Example 1.2.4: Let A =
20 −40 80
60 0 100
−40 −60 20
.
Since A = 20
1 −2 4
3 0 5
−2 −3 1
and
1 −2 4
3 0 5
−2 −3 1
= 5,
we can conclude that, |A| = (20)3
1 −2 4
3 0 5
−2 −3 1
= (20)3
(5) = 40, 000.
Definition 1.2.1. If A is a triangular matrix of order n, then its determinant is the
product of the entries on the main diagonal. That is,
det(A) = |A| = a11 · a22 · . . . · ann.
Example 1.2.5: Let A =
4 0 0
8 −4 0
−10 12 2
, then |A| = (4)(−4)(2) = −32.
Definition 1.2.2. The matrix L is called a special lower triangular matrix, where
”lower triangular” means that all the entries above the main diagonal are zero, while
”special” indicates that all the entries on the main diagonal are equal to 1.
Definition 1.2.3. A square matrix is called upper triangular if it has all zero entries
below its main diagonal.While a square matrix is called lower triangular if it has all
zero entries above its main diagonal.
Theorem 1.2.1. A square matrix A is invertible (nonsingular) if and only if
det|A| = 0.
Proof: Assume that A is invertible. Then AA−1
= I, and |AA−1
| = |A||A−1
| = |I| =
1. This implies that neither determinant on the left is zero, thus |A| = 0.
8
16. On the other hand, assume that |A| = 0, then by elimination, we can find a matrix
B in reduced row-echelon form, that is row-equivalent to A. Because B is in reduced
row-echelon form, it must be the identity matrix I or it must have at least one row
that consists entirely of zeros. But if B has a row of all zeros, then |B| = 0, which
would imply that |A| = 0. Because we assumed that |A| = 0, we can conclude that
B = I. A is therefore, row-equivalent to the identity matrix, hence A is invertible.
Example 1.2.6: Let A =
0 4 −2
3 −4 2
6 4 2
, then |A| = 192 = 0.
Hence, A is nonsingular, that is, its inverse exists.
1.3 Vector Norms
Definition 1.3.1. A vector norm on Rn
(or Cn
) is a function || · || : Rn
−→ R
that assigns a real-valued length to each vector, and must satisfy the following three
conditions. For all vectors x and y, and for all scalars α ∈ R(or C),
1) ||x|| ≥ 0, and ||x|| = 0 only if x =0,
2) ||x + y|| ≤ ||x|| + ||y||,
3) ||αx|| = |α|||x||.
The most important class of vector norms is the p-norms. We defined the 1-norm,
2-norm, ∞-norm, and p-norm as follows:
1-norm: ||x||1 =
n
i=1
|xi|,
2-norm: ||x||2 = (
n
i=1
|xi|2
)
1
2 ,
∞-norm: ||x||∞ = max
1≤i≤n
|xi|,
p-norm: ||x||p = (
n
i=1
|xi|p
)
1
p , p ≥ 1.
9
17. Example 1.3.1: If x =
1
2
−1
, then
||x||1 = 4, ||x||2 =
√
6, and ||x||∞ = 2.
Lemma 1.3.1. Let || · ||α and || · ||β be two norms on Rn
(or Cn
). There exists
constants c1, c2 ≥ 0 such that, for all x ∈ Rn
(Cn
),
c1||x||α ≤ ||x||β ≤ c2||x||α.
We also say that norms || · ||α and || · ||β are equivalent with respect to constants c1
and c2.
Lemma 1.3.2. For x ∈ Rn
(or Cn
), we have
1) ||x||2 ≤ ||x||1 ≤
√
n||x||2,
2) ||x||∞ ≤ ||x||2 ≤
√
n||x||∞,
3) ||x||∞ ≤ ||x||1 ≤ n||x||∞.
1.4 Matrix Norms
The definition of a matrix norm is equivalent to the definition of a vector norm. In
particular, || · || : Rm×n
−→ R is a matrix norm if the following three properties hold
for A ∈ Rm×n
(or Cm×n
) :
1) ||A|| ≥ 0,
2) ||A + B|| ≤ ||A|| + ||B||,
3) ||αA|| = |α|||A|| for a scalar α ∈ R (or C)
1.5 The Properties of Matrix Norm
For a matrix A of size m×n, we define the 1-norm, ∞-norm, Frobenius norm, p-norm,
and 2-norm as follows:
10
18. 1) 1-norm: ||A||1 = max
1≤j≤n
m
i=1
|aij|.
2) ∞-norm: ||A||∞ = max
1≤i≤m
n
j=1
|aij|.
3) Frobenius norm: ||A||F =
m
i=1
n
j=1
|aij|2
4) p-norms: ||A||p = sup
x=0
||Ax||p
||x||p
= sup
x=0
||A(
x
||x||p
)||p = max
||x||p=1
||Ax||p.
5) 2-norm:||A||2 = max
||x||2=0
||Ax||2
||x||2
= λmax(AT A),
where λmax denotes the largest eigenvalue.
Example 1.5.1:
Let A =
3 5
4 0
, then AT
A =
3 4
5 0
3 5
4 0
=
25 15
15 25
has eigenvalues λ1 = 40, and λ2 = 10.
||A||1 = max(|3| + |4|, |5| + |0|) = max(7, 5) = 7
||A||∞ = max(|3| + |5|, |4| + |0|) = max = (8, 4) = 8
||A||F = |3|2 + |5|2 + |4|2 =
√
50 ≈ 7.0711
||A||2 =
√
λ1 =
√
40 ≈ 6.3246
6) If A ∈ Rm×n
, 1√
n
||A||∞ ≤ ||A||2 ≤
√
m||A||∞.
For A =
3 5
4 0
, we have
5.6569 ≈ 8√
2
= 1√
2
(8) ≤ ||A||2 ≤
√
2(8) = 8
√
2 ≈ 11.3137,
where ||A||2 = 6.3246
7) If A ∈ Rm×n
, 1√
m
||A||1 ≤ ||A||2 ≤
√
n||A||1. Using the matrix A =
3 5
4 0
,
4.9497 ≈ 7√
2
= 1√
2
(7) ≤ ||A||2 ≤ (
√
2)(7) ≈ 9.8994,
where ||A||2 = 6.3246
8) If A ∈ Rm×n
, 1
n
||A||∞ ≤ ||A||1 ≤ n||A||∞, using the matrix above we can easily
11
19. verify the inequality.
4 = 1
2
(8) ≤ ||A||1 ≤ 2(8) = 16 where ||A||1 = 7
Theorem 1.5.1. If A ∈ Rm×n
, then there exists a unit (2-norm) n-dim vector z such
that AT
Az = µ2
z where µ = ||A||2.
Theorem 1.5.2. If A ∈ Rm×n
, then ||A||2 ≤ ||A||1||A||∞
Proof: If z = 0 is such that AT
Az = µ2
z with µ = ||A||2, then µ2
||z||1 = ||AT
Az||1 ≤
||AT
||1||A||1||z||1 = ||A||∞||A||1||z||1.
9) If A ∈ Rm×n
, then ||A||2
2 ≤ ||A||1||A||∞
From Theorem 1.5.2, we have ||A||2
2 ≤ ||A||1||A||∞.
1.6 The Effect of Elementary Row Operations on Determinants
Theorem 1.6.1. Let A and B be two square matrices, then
1) If B is obtained from A by interchanging two rows of A, then |B| = −|A|.
2) If B if obtained from A by adding a multiple of a row of A to another row, then
|B| = |A|.
3) If B is obtained from A by multiplying a row by a nonzero constant c, then
|B| = c|A|.
Example 1.6.1: Let A =
4 −6 20
2 4 −4
0 2 −6
.
If we interchange the first two rows of A, we get
|A| =
4 −6 20
2 4 −4
0 2 −6
= −
2 4 −4
4 −6 20
0 2 −6
= −|B|.
Now, if we add −2 times the first row of B to the second row to produce a new second
12
20. row, then
|A| = −
2 4 −4
4 −6 20
0 2 −6
= −
2 4 −4
0 −14 28
0 2 −6
.
Let us factor −14 out of the second row of the last matrix:
|A| = −
2 4 −4
0 −14 28
0 2 −6
= 14
2 4 −4
0 1 −2
0 2 −6
.
Finally, if we add −2 times the second row to the third row to produce a new third
row, we have
|A| = 14
2 4 −4
0 1 −2
0 2 −6
= 14
2 4 −4
0 1 −2
0 0 −2
.
Therefore, |A| = 14(2)(1)(−2) = −56, since the last matrix is an upper triangular.
13
21. CHAPTER 2
NUMERICAL COMPUTATIONS OF MATRIX DETERMINANT
2.1 The LU-Factorization
We find the LU Factorization of a square matrix A by applying Gaussian Elimi-
nation algorithm to reduce A to its upper triangular form U. The entries of L can be
filled in during the computation process with the negatives of the multiples used in
the elementary operations. L is a special lower triangular matrix, which means that
the diagonal entries are all 1. Then A = LU. For an n × n matrix A, the following
row operations can be used to reduce A into upper triangular:
1) Interchange any two rows.
2) Multiply a row by a nonzero constant.
3) Add a multiple of a row to another row.
Algorithm 2.1.1
Input: An n × n matrix A
Output: det A
Computations:
1) Compute the matrices L, U satisfying the equation A = LU.
2) Compute det L, det U.
3) Compute and output det A =(det L)(det U).
Example 2.1.1: Let A =
4 2 2
8 10 4
4 −4 0
.
22. Step 1: R2 + (−2)R1 → R2 :
4 2 2
0 6 0
4 −4 0
Step 2: R3 + (−1)R1 → R3 :
4 2 2
0 6 0
0 −6 −2
Step 3: R3 + R2 → R3 :
4 2 2
0 6 0
0 0 −2
= U.
This is the upper triangular matrix U whose determinant is |U| = −48, and hence
the corresponding lower triangular matrix L is: L =
1 0 0
2 1 0
1 −1 1
,
whose entries are the negatives of the multiples we used during the elimination pro-
cess. Clearly, |L| = 1. Now,
A = LU =
1 0 0
2 1 0
1 −1 1
4 2 2
0 6 0
0 0 −2
=
4 2 2
8 10 4
4 −4 0
,
and det A = (det L)(det U) = (−48)(1) = −48.
2.2 The Permutation Matrices
A Permutation matrix P is a matrix obtained from the identity matrix I by any
combination of row (or column) interchanges.
Definition 2.2.1. A matrix P is a permutation matrix if and only if each row of P
contains all 0 entries except for a single 1, and in addition, each column of P also
contains all 0 entries except for a single 1.
15
23. For example, by interchanging rows 1 and 2 of the 3 × 3 identity matrix, we produce
the following permutation matrix:
P =
0 1 0
1 0 0
0 0 1
Example 2.2.1: The result PA of multiplying any 3-rowed matrix A on the left by
the above permutation matrix P is the same as interchanging the first two rows of A.
For instance,
PA =
0 1 0
1 0 0
0 0 1
2 4 6
8 10 12
14 16 18
=
8 10 12
2 4 6
14 16 18
.
Lemma 2.2.1. If P is a permutation matrix, then (det P)=1 or −1.
Lemma 2.2.2. (det I)=1, where I is an identity matrix.
2.3 The Permuted LU-Factorization
Definition 2.3.1. If A is a nonsingular matrix, then there is a permutation matrix
P such that A admits an LU factorization, that is,
PA = LU,
where P is a permutation matrix, L is a special lower triangular, and U is upper
triangular with the pivots on the main diagonal.
Example 2.3.1: Let A =
10 −7 0
−3 2 6
5 −1 5
.
16
25. 2.4 Gaussian Elimination with Partial Pivoting
1) Find the entry in the left column with the largest absolute value. The entry is
called the pivot.
2) Perform a row interchange, so that the pivot is in the first row.
3) Divide first row by the pivot.
4) Use elementary row operations to reduce the remaining entries in the first column
to zero.
Algorithm 2.4.1
Input: An n × n matrix A.
Output: det A.
Computations:
1) Compute the matrices P, L, U satisfying the equation A = PLU.
2) Compute det P, det L, and det U.
3) Compute and output det A = (det P)(det L)(det U).
Remark 2.4.1. One can easily output the sign of det (A) from Algorithm 2.4.1. We
only need to compute the sign of det P, det L, and det U at stage 2 and multiply
these signs.
Example 2.4.1: Let A =
1 2 −3
2 4 0
3 3 −6
.
Step 1: Pivot element is a31 = 3, so interchange R1 ↔ R3 :
3 3 −6
2 4 0
1 2 −3
Step 2: R2 + (−2
3
)R1 → R2 , R3 + (−1
3
)R1 → R3 :
3 3 −6
0 2 4
0 1 −1
18
26. Step 4: R3 + (−1
2
)R2 → R3 :
3 3 −6
0 2 4
0 0 −3
= U,
clearly |U| = −18.
The matrix U is an upper triangular matrix, and hence the corresponding lower tri-
angular matrix L is :
L =
1 0 0
2
3
1 0
1
3
1
2
1
, |L| = 1. P =
0 0 1
0 1 0
1 0 0
, |P| = −1.
Therefore, A = PLU =
0 0 1
0 1 0
1 0 0
1 0 0
2
3
1 0
1
3
1
2
1
3 3 −6
0 2 4
0 0 −3
and hence det A = (det P)(det L) (det U) = (−1)(1)(−18) = 18.
2.5 Gaussian Elimination with Complete Pivoting
1) Perform row or column interchanges so that there are no zeros in the pivot posi-
tions.
2) Find the entry with the largest absolute value. The entry is called the pivot.
3) Perform row or column interchange once again, so that the pivot is in the first row.
4) Use elementary row operations to reduce the remaining entries in the first column
to zero
Algorithm 2.5.1
Input: An n × n matrix A.
Output: det(A).
Computations:
1) Compute the matrices P, L, U and P1 satisfying the equation A = PLUP1.
19
27. 2) Compute det P, det L, det U, det P1.
3) Compute and output det(A) =(det P)(det L)(det U)(det P1).
Remark 2.5.1. One can easily output the sign of det A from the above algorithm.
We only need to compute the sign of det P, det L, det U, and det P1 at stage 2 and
multiply these signs.
Example 2.5.1: Let A =
0 2 −3
2 2 1
2 4 4
.
Step 1: Interchange C1 ↔ C2 :
2 0 −3
2 2 1
4 2 4
.
Step 2: Pivot element a31 = 4, so interchange R1 ↔ R3 :
4 2 4
2 2 1
2 0 −3
.
Step 3: R2 + (−1
2
)R1 → R2, R3 + (−1
2
)R1 → R3 :
4 2 4
0 1 −1
0 −1 −5
.
Step 4: Get largest absolute element in pivot position, a22.
Interchange C2 ↔ C3, R2 ↔ R3 :
4 4 2
0 −5 −1
0 −1 1
.
Step 5: R3 + (−1
5
)R2 → R3 :
4 4 2
0 −5 −1
0 0 6
5
= U, |U| = −24.
This is the upper triangular matrix U and hence the corresponding lower triangular
20
28. matrix L is:
1 0 0
1
2
1 0
1
2
1
5
1
, |L| = 1.
The permutation matrices are as follows: P =
0 0 1
1 0 0
0 1 0
, (P is the row inter-
changes), |P| = 1. P1 =
0 0 1
1 0 0
0 1 0
, (P1 is the column interchanges), |P1| = 1.
Therefore, det A = (det P)(det L)(det U)(det P1) = (1)(1)(−24)(1) = −24.
2.6 The QR-Factorization
Definition 2.6.1. A square matrix Q is called orthogonal matrix if QT
Q = I. Equiv-
alently Q−1
= QT
, where QT
is the transpose of Q.
Lemma 2.6.1. If Q is an orthogonal matrix, then det(Q) = ±1.
Proof: Since Q is orthogonal, then QT
Q = I. Now |QT
Q| = |I| = 1 and thus
|Q||QT
| = 1, but |QT
| = |Q| which implies that 1 = |Q||Q| = |Q|2
. Hence |Q| = ±1.
Example 2.6.1: Let Q =
0 1 0
−1 0 0
0 0 −1
. Then Q−1
= QT
=
0 −1 0
1 0 0
0 0 −1
,
since QT
Q = I. det(Q) = −1.
Example 2.6.2: Let Q =
12
13
5
13
−5
13
12
13
. Then Q−1
= QT
=
12
13
−5
13
5
13
12
13
, since
QT
Q = I. det(Q) = 1.
Definition 2.6.2. Let A be an n × n matrix with linearly independent columns.
Then A can be written as A = QR, where Q is orthogonal and R is an upper trian-
21
29. gular matrix with positive diagonal entries.
Definition 2.6.3. A complex square matrix U is called unitary if it satisfies U∗
U = I,
where U∗
= U
T
denotes the Hermitian transpose in which one first take complex con-
jugates of all entries and then take the transpose.
Remark 2.6.1. We can use the QR decomposition to find the determinant of a
square matrix or the absolute value of the determinant. Suppose a matrix is decom-
posed as A = QR, then det(A) = det(Q) det(R). Since Q is unitary, det(Q) = ±1.
Thus det(A) = ± det(R) = ±
i
rii, where rii are the entries on the main diagonal
of R. In addition, | det(A)| = | det(R)| since | det(Q)| = 1. Here, | · | refers to the
absolute value.
Example 2.6.3: Let U =
1√
2
i√
2
i√
2
1√
2
.
U =
1√
2
−i√
2
−i√
2
1√
2
, U∗
= U
T
=
1√
2
−i√
2
−i√
2
1√
2
.
Now, U∗
U = U
T
U =
1√
2
−i√
2
−i√
2
1√
2
1√
2
i√
2
i√
2
1√
2
=
1 0
0 1
= I.
Hence, U−1
= U∗
= U
T
=
1√
2
−i√
2
−i√
2
1√
2
.
2.7 Modified Gram-Schmidt Algorithm
Theorem 2.7.1. Let A be an m × n matrix. Suppose that A has full column rank.
Then there exists a unique m × n orthogonal matrix Q and a unique n × n upper
triangular matrix R with positive diagonals rii > 0 such that A = QR.
Gram-Schmidt Process
A = [a1, . . . , an]
22
32. r23 =< aT
3 , q2 >=
0
1
1
T
1√
6
2√
6
− 1√
6
= 1√
6
r33 = ||a3||2 − r2
13 − r2
23 = 2 − 1
2
− 1
6
= 2√
3
q3 = a3−r13q1−r23q2
r33
=
√
3
2
(
0
1
1
− 1√
2
1√
2
0
1√
2
− 1√
6
1√
6
2√
6
− 1√
6
) =
− 1√
3
1√
3
1√
3
.
Thus, R = QT
A =
r11 r12 r13
0 r22 r23
0 0 r33
=
√
2 1√
2
1√
2
0 3
2
1√
6
0 0 2√
3
.
Therefore, det(A) =(det Q)
i
ri,i = (1)(2) = 2.
2.8 Householder Algorithm
Definition 2.8.1. The Householder transformation (or reflection) is a matrix of the
form H = I − 2vvT
where vT
· v = 1, or ||v||2 = 1, which serves to introduce zeros
into the lower triangle of a matrix.
Lemma 2.8.1. H is symmetric and orthogonal. That is, HT
= H and HT
· H = I.
Proof: H = I − 2vvT
, and since IT
= I and (vvT
)T
= vTT
vT
= vvT
, then
HT
= IT
− (2vvT
)T
= I − 2vvT
= H. Hence, H is symmetric. On the other hand,
HT
H = (I − 2vvT
)(I − 2vvT
) = I − 4vvT
+ 4v(vT
v)vT
= I − 4vvT
+ 4vvT
= I,
since vT
v = 1. Therefore, H is orthogonal, that is, H−1
= HT
.
Remark 2.8.1. H is called a reflection matrix since Hy is a reflection of the vector
y in the plane through 0 perpendicular to v.
In general, for any given m-dimensional vector x = (x1, . . . , xm)T
, it is easy to find
a Householder reflection matrix H = I − 2vvT
to zero out all but the first entry of
25
33. x. Hx = [α, 0, . . . , 0]T
= α · e1, where e1 = (1, 0, · · · , 0)T
. To do this, we first write
Hx = (I − 2vvT
)x = x − 2v(vT
x) = αe1 so that x − αe1 = 2v(vT
x) and hence
v = 1
2(vT x)
(x − αe1); that is, v is a linear combination of x and e1.
Since ||Hx||2 = |α| = ||x||2, v must be parallel to the vector u = x ||x||2e1 and
hence v = u
||u||2
. Here, either choice of the + or − sign yields a vector v satisfying
Hx = αe1 as long as u = 0. We will use α = sign(eT
1 x) and write the vector u as
u =
x1 sign(x1) · ||x||2
x2
...
xn
with v = u
||u||2
and we write v = House(x).
Remark 2.8.2. In practice, and to save the work of computing the vector v, we can
store the vector u instead of v and use the formula H = I − ( 2
||u||2
2
)uuT
instead of
H = I − 2vvT
, [9].
Example 2.8.1: Let A =
2 2 4
2 0 −4
−2 4 6
.
In the first phase of Householder’s method, we introduce the elementary reflection
matrix that maps the first column x1 of the matrix A to a basis vector αe1. This
results in a matrix H1A with zeros in the left column (except for the first row).
Choose the vector x1 to be the first column of A, so that x1 = a1 =
2
2
−2
, then
||x1||2 =
√
12.
The first Householder vector u1 = x1 + ||x1||2e1
26
35. Now, we have R = H2A1 =
1 0 0
0 −.0716 .9974
0 .9974 .0717
−3.4640 1.1547 3.4641
0 −.3094 −4.1961
0 4.3094 6.1961
=
−3.4647 1.1549 3.4647
0 4.3205 6.4807
0 0 −3.7412
which an upper triangular matrix. Therefore, the
orthogonal matrix is obtained by multiplying the reflection matrices H1 and H2.
Q = H1H2 =
−.5773 −.5773 .5773
−.5773 .7887 .2113
.5773 .2113 .7887
1 0 0
0 −.0716 .9974
0 .9974 .0717
=
−.5774 .6172 −.5345
−.5774 .1543 .8018
.5774 .7715 .2673
.
Therfore, det A = (det Q) ri,i = (1)(56) = 56
Remark 2.8.3. For a matrix Q of A = QR computed by the Householder algorithm,
we have det|Q| = (−1)n−1
, where | · | refers to absolute value of the elements of Q.
28
36. 2.9 Givens Algorithm
Definition 2.9.1. A matrix G(θ) =
c −s 0
s c 0
0 0 1
with the choice of c = cos(θ) and s =
sin(θ) for some rotation θ can be used to zero an element ai,j. The conditions
sa + cb = 0 and c2
+ s2
= 1 give the following equations:
cos(θ) = c =
a
√
a2 + b2
, sin(θ) = s =
−b
√
a2 + b2
,
for a given 2-dim vector (a11, a21)T
= (a, b)T
. The elements can be zeroed column by
column from the bottom up. Each rotation zeros an element in the subdiagonal of
the matrix forming R to make it an upper triangular matrix. The composition of all
Givens rotations form the orthogonal matrix Q which computes the QR decomposi-
tion.
Example 2.9.1: Let A =
12 10 0
10 2 8
0 8 6
. First we need to form a rotation matrix
that will bring the above matrix A to an upper triangular. In order to form this
desired matrix, we must zero elements a21 and a32. We form this matrix using the
Givens rotational method:
c =
a
√
a2 + b2
=
12
√
244
≈ .7682 and s =
−b
√
a2 + b2
=
−10
√
244
≈ −.6402 for the 2-dim
vector (a, b) = (12, 10)T
.
G1 =
c −s 0
s c 0
0 0 1
=
12√
244
10√
244
0
− 10√
244
12√
244
0
0 0 1
.
29
37. This yields the matrix: G1A = A1 =
15.6205 8.9624 5.1216
0 −4.8656 6.1456
0 8 6
.
We now zero out the element a32 with a rotation matrix G2 =
1 0 0
0 c −s
0 s c
=
1 0 0
0 −4.8656√
87.6740
8√
87.6740
0 −8√
87.6740
−4.8656√
87.6740
, where c =
a
√
a2 + b2
=
−4.8656
√
87.6740
≈ −.5196,
and s =
−b
√
a2 + b2
=
−8
√
87.6740
≈ −.8544 for the 2-dim vector (a, b)T
= (−4.8656, 8)T
.
We have the following matrix multiplication:
A2 = G2A1 =
1 0 0
0 −.5196 .8544
0 −.8544 −.5196
15.6205 8.9624 5.1216
0 −4.8656 6.1456
0 8 6
=
15.6205 8.9624 5.1216
0 9.3633 1.9331
0 0 −8.3684
.
This gives us our upper triangular matrix R = A2 =
15.6205 8.9624 5.1216
0 9.3633 1.9331
0 0 −8.3684
.
Q is now formed by the product of the Given Rotation matrices Q = G1G2.
30
39. CHAPTER 3
ALGEBRAIC COMPUTATIONS OF MATRIX DETERMINANT
3.1 Determinant Computation Based on the Chinese Remainder Theorem
Definition 3.1.1. a is said to be congruent to b modulo m if a − b is divisible by
m ≥ 0. Or in other words, we say that a ≡ b(mod m) when the remainders obtained
in dividing a and b by m are the same.
Example 3.1.1:
1) 200 ≡ 2(mod 9) because 9 | (200 − 2) = 198.
2) −2 ≡ 1(mod 3) because 3 | (−2 − 1) = −3.
3) 13 ≡ 5(mod 9) because 9 | (13 − 5) = 8.
Definition 3.1.2. The greatest common divisor of two integers a and b, which are
not both 0, is the largest integer that divides both a and b. It is usually written as
gcd(a, b). The gcd of any two integers can be found by the Euclidean Algorithm.
Theorem 3.1.1. (The Euclidean Algorithm)
Let r0 = a and r1 = b be two integers such that a ≥ b > 0. If the division algo-
rithm is successively applied to obtain rj = rj+1qj+1 + rj+2, with 0 < rj+2 < rj+1 for
j = 0, 1, 2, . . . , n−2 and rn+1 = 0, then the gcd(a, b) = rn, the last nonzero remainder.
Example 3.1.2: gcd(578, 442)
587 = 442 · 1 + 136
442 = 136 · 3 + 34
136 = 34 · 4 + 0
Therefore, gcd(578, 442) = 34.
40. Theorem 3.1.2. [18] (Chinese Remainder Theorem)
Let m1, m2, . . . , mk be distinct pairwise relatively prime integers such that
m1 > m2 > . . . > mk > 1. (3.1.1)
Let
M = m1m2 . . . mk, (3.1.2)
and let D denotes an integer satisfying the inequality
0 ≤ D < M. (3.1.3)
Let
ri = D mod mi. (3.1.4)
Mi =
M
mi
, vi ≡ Mi (mod mi), yivi ≡ 1 (mod mi), ∀ i = 1, . . . , k. (3.1.5)
Then D is a unique integer satisfying (3.1.3) and (3.1.4). In addition,
D =
k
i=1
(Miriyi) (mod M). (3.1.6)
Lemma 3.1.1. [18] For any pair of integers d and M > 2|d|, we have
d = (d mod M), if (d mod M) <
M
2
, d = −M + (d mod M) otherwise. (3.1.7)
If our input matrix A is filled with integers and we know an upper bound d+
≥ |detA|,
we may choose any integer M exceeding 2d+
and then compute (det A) mod M.
Hence, det A can be recovered by using (3.1.7).
33
41. Algorithm 3.1.1 Computation of (det A)mod M based on the Chinese remainder
theorem.
Input: an integer matrix A, an algorithm that computes (det A) mod m for any fixed
integer m > 1, k integers m1, . . . , mk satisfying (3.1.1) and are pairwise relatively
prime, and M = m1m2 . . . mk.
Output: (det A)mod M.
Computations:
1) Compute ri = (det A) mod mi, i = 1, . . . k.
2) Compute the integers Mi, vi, and yi, ∀i = 1, . . . , k.
3) Compute and output D as in (3.1.6).
Example 3.1.3: Let A =
4 5
6 9
.
Using the first three prime numbers as moduli we have,
ri =(det A) mod mi
r1 = (detA)mod 5 ⇒ r1 = 1
r2 = (detA)mod 3 ⇒ r2 = 0
r3 = (detA)mod 2 ⇒ r3 = 0
M = m1 · m2 · m3 = (5)(3)(2) = 30.
M1 ≡ M
m1
= 30
5
= 6, M2 ≡ M
m2
= 30
3
= 10, M3 ≡ M
m3
= 30
2
= 15,
vi ≡ Mi mod mi :
v1 ≡ 6 mod 5 = 1
v2 ≡ 10 mod 3 = 1
v3 ≡ 15 mod 2 = 1.
Then, we compute the values of yi from the equation yivi ≡ 1 mod mi, for i = 1, . . . , k
by the Euclidean algorithm as follows:
y1v1 ≡ 1 mod m1 ⇒ y1(1) ≡ 1 mod 5 ⇒ y1 = 1
34
42. y2v2 ≡ 1 mod m2 ⇒ y2(1) ≡ 1 mod 3 ⇒ y2 = 1
y3v3 ≡ 1 mod m3 ⇒ y3(1) ≡ 1 mod 2 ⇒ y3 = 1.
Finally, compute and output D = det(A)mod M, that is, D = det (A)( mod 30).
D =
3
i=1
(Miriyi)(mod M) = M1r1y1+M2r2y2+M3r3y3 = (6·1·1)+(10·0·1)+(30·0·1) =
6(mod 30) ≡ 6. Therefore, this shows that 6 is the determinant of A.
3.2 Modular Determinant
For a matrix A ∈ Zn×n
, we choose a prime p that is bigger than 2| det(A)|, and then
perform Gaussian elimination on A (mod p) ∈ Zn×n
p to calculate det(A)(mod p).
Remark 3.2.1. Calculation of det(A) (mod p) is the same as Gaussian elimination
over Q, except that when dividing by a pivot element a we have to calculate its in-
verse modulo p by the Euclidean Algorithm.
Definition 3.2.1. Hadamard’s Inequality is an upper bound on the determinant
that is defined by
|detA| ≤ n
n
2 an
= (a
√
n)n
where a = max
1≤i,j≤n
|aij| ∈ Z+
(the set of all nonnegative integers) is the maximal abso-
lute value of an entry of A.
Example 3.2.1: Let A =
8 10
12 −11
. Applying Gaussian Elimination to the
matrix A to obtain the following matrix:
8 10
0 −26
. Its determinant det (A) =
(−26)(8) = −208. The Hadamard bound |detA| ≤ 21
122
= 288 is reasonably close to
|detA| = 208. For a modular computation, we choose a prime p > 2·208, say p = 417
and perform Gaussian Elimination modulo p.
35
43. det(A)(mod 417) = det
8 10
0 391
(mod 417) = 3128 (mod 417) = 209 ≡ −208 in
Z417.
3.3 Modular Determinant of a fixed prime
Let p denote a fixed prime. Then (detA) mod p can be computed as we compute
modulo p of the PLU factorization of A. This can be achieved through Gaussian
elimination with partial pivoting. If during the elimination process all entries of the
pivot column equal 0 mod p, then we output det A = 0 mod p. If we need to compute
(detA) mod p for the updated matrix A, we rely on the QR factorization of A, such
that A = (QR) mod p and QT
Q = D mod p, where D is a diagonal matrix, R is
a unit upper triangular matrix. This factorization can be computed by the Givens
algorithm.
Remark 3.3.1. If the entries of the matrix A are much smaller than p, then we do
not need to reduce modulo p the results at the initial steps of Gaussian elimination.
That is, the computation can be performed in exact rational arithmetics using lower
precision. One may apply the algorithm of [1] and [2] at these steps to keep the
precision low.
3.4 Matrix Singularity
An n×n matrix A is called nonsingular or invertible if there exists an n×n matrix B
such that AB = In = BA. Any matrix B with the above property is called an inverse
of A. If A does not have an inverse, then A is called singular. In the case where we
apply the Chinese remainder theorem, a matrix is singular if det A = 0 (mod M),
and nonsingular otherwise. If ri = (det A) mod mi = 0 for some i, then, det A = 0
and the matrix is nonsingular. If ri = 0(mod mi) for all i, then (detA) mod M = 0.
36
44. The latest equations imply that det A = 0 and the matrix is singular.
Algorithm 3.4.1 (Singularity Test)
Input: an integer matrix A, an algorithm that computes (det A) mod m for any
fixed integer m > 1, k integers m1, . . . , mk satisfying the inequality m1 > m2 > . . . >
mk > 1 and are pairwise relatively prime, and M = m1m2 . . . mk where M > |det A|.
Output: Singular if det A = 0 mod M, Nonsingular otherwise.
Computations:
1) Compute ri = (detA) mod mi, ∀i = 1, . . . , k.
2) Output Singular if ri = 0 for i = 1, . . . , k; otherwise, output Nonsingular.
Example 3.4.1: The matrix A =
3 2
1 9
transforms into
3 2
0 25
3
after ap-
plying Gaussian elimination.
Using the first three prime numbers as moduli we have,
ri = (det A) mod mi
r1 = (det A) mod 5 ⇒ r1 = 0
r2 = (det A) mod 3 ⇒ r2 = 1
r3= (det A) mod 2 ⇒ r3 = 1
Since ri = 0 for some i, then the matrix A is nonsingular. Moreover, M = m1m2m3
= (5)(3)(2) = 30,
det(A)(mod M) = det
3 2
0 25
3
(mod 30) ≡ 25 mod 30 = 0, and hence det A = 0.
Example 3.4.2: Let A =
5 7
0 0
.
r1 = (det A) mod 5 ⇒ r1 = 0
r2 = (det A) mod 3 ⇒ r2 = 0
r3 = (det A) mod 2 ⇒ r3 = 0
37
45. Now, since ri = 0 for all i, then the matrix A is singular. In addition, det(A)(mod M) =
det
5 7
0 0
(mod 30) ≡ 0, and hence det A = 0.
38
46. CHAPTER 4
ALTERNATIVE APPROACH FOR COMPUTING MATRIX
INVERSES AND DETERMINANTS
4.1 Newton-Hensel’s lifting (p-adic)
In this chapter, we present an alternative approach for computing matrix deter-
minant to one we discussed in chapter 3. The main goal behind this technique is to
decrease the precision of algebraic computations with no rounding errors. It relies
on Newton’s-Hensel’s lifting (p-adic lifting) and uses order n4
log n ops. However, we
will also show how to modify this technique to use order of n3
ops by employing the
faster p-adic lifting.
Theorem 4.1.1. [16] (Hensel Lemma) Suppose that f(x) is a polynomial with inte-
gral coefficients. k is an integer such that k ≥ 2, and p is a prime. Suppose further
that r is solution of the congruence f(x) ≡ 0 (mod pk−1
). Then,
(i) if f (r) ≡ 0(mod p), then there is a unique integer t, 0 ≤ t < p, such that
f(r + tpk−1
) ≡ 0(mod pk
), given by
t ≡ −f (r)( f(r)
pk−1 )(mod p),
where f (r) is an integer chosen so that f (r)f (r) ≡ 1 (mod p);
(ii) if f (r) ≡ 0 (mod p) and f(r) ≡ 0 (mod pk
), then f(r + tpk−1
) ≡ 0(mod pk
) for
all integers t;
(iii) if f (r) ≡ 0 (mod p) and f(r) ≡ 0 (mod pk
), then f(x) ≡ 0 (mod pk
) has no
solutions with x ≡ r (mod pk−1
).
Remark 4.1.1. Notice that, in cases (ii) and (iii), such a solution either lifts to p
incongruent solutions modulo pk
or to none at all. Whereas, in case (i), a solution to
f(x) ≡ 0 (mod pk−1
) lifts to a unique solution of f(x) ≡ 0 (mod pk
).
47. Corollary 4.1.1. Suppose that r is a solution of the polynomial congruence f(x) ≡
0(mod p), where p is a prime. If f (r) ≡ 0(mod p), then there is a unique solution rk
modulo pk
, k ≥ 2, such that r1 = r and
rk = rk−1 − f(rk−1)f (r),
where f (r) is an inverse of f (r) modulo p.
Example 4.1.1: x3
+ x2
+ 29 ≡ 0(mod 25). Let f(x) = x3
+ x2
+ 29. Because
f (x) = 3x2
+ 2x and f (3) = 33 = 0 (mod 5), Hensel’s lemma says that there is a
unique solution modulo 25 of the form x = 3+5t where t ≡ −f (3)f(3)
5
(mod 5). Since
f (3) = 3 = 2, we have, t ≡ (−2)65
5
(mod 5) = (−2)(13)(mod 5) = −26(mod 5) ≡
4(mod 5). Hence, we can conclude that x = 3 + 5(4) = 23 is the unique solution of
f(x) ≡ 0(mod 25).
4.2 Newton’s iteration and p-adic lifting of matrix inverses
Recall that Newton’s iteration involves computing successive approximations to
the solutions of f(r) = 0 given a suitable initial approximation r0 to the exact solution.
Then, subsequent approximations are computed using
ri+1 = ri −
f(ri)
f (ri)
(4.2.1)
where f is the derivative of f.
Newton’s iteration (4.2.1) can be extended to the case where r, ri, and f(ri) are
vectors or matrices. In particular, we can define f(S) = A − S−1
for two matrices
A and S. Given an initial approximation to the inverse, say, S0, Newton’s iteration
rapidly improves the initial approximation to the inverse of a nonsingular n×n matrix
A, [5]:
Si+1 = Si(2I − ASi) = 2Si − SiASi = (2I − SiA)Si, i ≥ 0 (4.2.2)
40
48. Algorithm 4.2.1. (p-adic lifting of matrix inverses)
Input: an n × n matrix A, an integer p > 1, the matrix S0 = A−1
mod p, and a
natural number h.
Output: the matrix A−1
mod pH
, H = 2h
.
Computations: Recursively compute the matrix Sj given by the equation, Sj =
Sj−1(2I − ASj−1) mod pJ
, where J = 2j
and j = 1, . . . , h. Finally, outputs Sh.
Note that, Sj = A−1
mod pJ
, J = 2j
, ∀j. This follows from the equation I − ASj =
(I − ASj−1)2
mod pJ
. The above algorithm uses order of n3
ops performed modulo
pJ
at the j-th stage, that is, a total of hn3
ops, with precision of at most J log2 p bits.
Example 4.2.1: Let A =
−1 1 0
−1 0 1
6 −2 −3
.
We can first find a given approximation to the inverse using the equation,
S0 = A−1
mod p, for p > 1.
However, it is not necessary to find the inverse. One can start with the identity or a
matrix that is closely related to A−1
.Therefore we will begin with our initial approx-
imation:
1 2 1
2 2 1
1 3 1
.
Then, S0 =
1 2 1
2 2 1
1 3 1
mod 21
=
1 0 1
0 0 1
1 1 1
.
41
50. by the following equation [5, p.327]:
det A =
1
n
k=1(A−1
k )k,k
. (4.3.1)
Here, Ak denotes the k × k leading principal submatrix of A, (north-western subma-
trix) so that An = A, for k = 1, · · · , n. In general, (B)k,k denotes the (k, k)-th entry
of a matrix B. To apply formula (4.3.1), we must have the inverses modulo p of Ak
available for all k and for a fixed prime integer p. The existence of such inverses of
those submatrices is called strong nonsingularity of the matrix A modulo p. (See
definition 4.3.1). So, here we are assuming that the matrix A is strongly nonsingular
for a choice of p. This can be actually guaranteed with a random choice of p in a
fixed interval and under a certain probability that the matrix is strongly nonsingular
modulo p [18, Appendix A]. Finally, Algorithm 4.2.1 can be extended to lifting det A
(See Algorithm 4.3.1).
Definition 4.3.1. A matrix A is called strongly nonsingular matrix if A is nonsin-
gular and all k × k leading principle submatrices are nonsingular.
Example 4.3.1: Let A =
2 4 6
8 11 12
14 15 18
.
The 1 × 1 leading principle submatrix of A is A1 = 2 ; det(A1) = 2.
The 2 × 2 leading principle submatrix of A is A2 =
2 4
8 11
; det(A2) = −10.
The 3×3 leading principle submatrix of A is A3 =
2 4 6
8 11 12
14 15 18
; det(A3) = −72.
Since all k × k leading principle submatrices are nonsingular, and det (A) = 0, then
43
51. the matrix A is strongly nonsingular.
Algorithm 4.3.1 (p-adic lifting of matrix determinant)
Input: an integer p > 1, an n × n matrix A, the matrices S0,k = A−1
k mod p, and a
natural number h.
Output: det A mod p2H
, H = 2h
.
Computations:
1) Apply Algorithm 4.2.1 to all pairs of matrices Ak and S0,k, (replacing the matrices
A and S0 in the algorithm), so as to compute the matrices Sh,k = A−1
k mod pH
, for
k = 1, . . . , n.
2) Compute the value
(
1
det A
)mod p2H
=
n
k=1
[Sh,k(2I − AkSh,k)]k,k mod p2H
(4.3.2)
3) Compute and output the value (det A) mod p2H
, as the reciprocal of (
1
det A
)
modulo p2H
.
Example 4.3.2: Let A =
1 17 18
1 18 19
5 16 20
. Then, A1 = 1 , A2 =
1 17
1 18
, and
A3 =
1 17 18
1 18 19
5 16 20
. Then compute the matrices, S0,k = A−1
k mod p.
S0,1 = A−1
1 mod 3 = 1 mod 3 = 1 ,
S0,2 = A−1
2 mod 3 =
18 −17
−1 1
mod 3 =
0 1
2 1
,
44
53. = 1
1,1
·
−144 −17
−1216 −161
2,2
·
−2243 −2783 −2510
−2100 −2603 −2348
−2113 −2580 −2269
3,3
= 1 · −161 · −2269 = 365, 309 mod 81 ≡ 80 ≡ −1(mod 81).
Finally, we output the value of (detA)mod 32·2·1
as the reciprocal of (
1
det A
) mod 34
which is -1.
Due to the computations of all k × k leading principal submatrices Ak of A, which
required order of n4
ops, algorithm 4.3.1 is not an efficient way of computing ma-
trix determinant. Therefore, in the next section, we will present an alternative faster
technique to computing matrix determinant that uses only O(n3
) based on the divide-
and-conquer algorithm.
Remark 4.3.1.We estimate H and p in order to satisfy the bound p2H
> 2|detA|.
Due to Hadamard’s bound,
|detA| ≤ (a
√
n)n
, a = maxi,j|ai,j|,
which holds for any matrix A = (ai,j), it suffices to choose p that satisfies the following
inequality:
2(a
√
n)n
< p2H
. By taking the logp of both sides of the above inequality, we have
logp 2(a
√
n)n
< logp(p2H
)
n logp(a
√
n) + logp 2 < 2H
Therefore we obtain H that satisfies the following inequality:
2H > n logp(a
√
n) + logp 2.
46
54. 4.4 Faster p-adic lifting of the determinant
As we discussed in the previous section, we will assume that A is strongly non-
singular. Then, block Gauss-Jordan elimination can be applied to the block 2 × 2
matrix A =
B C
E G
as follows:
Step 1: R2 + (−EB−1
)R1 → R2 :
B C
0 S
, where S = G − EB−1
C. (The Schur
complement of B in A).
The corresponding elementary matrix is: E1 =
I 0
−EB−1
I
, E−1
1 =
I 0
EB−1
I
.
Step 2: (S−1
)R2 → R2 :
B C
0 I
.
The corresponding elementary matrix is: E2 =
I 0
0 S−1
, E−1
2 =
I 0
0 S
.
Step 3: (B−1
)R1 → R1 :
I B−1
C
0 I
.
The corresponding elementary matrix is: E3 =
B−1
0
0 I
, E−1
3 =
B 0
0 I
.
Step 4: R1 + (−B−1
C)R2 → R1 :
I 0
0 I
.
The corresponding elementary matrix is: E4 =
I −B−1
C
0 I
, E−1
4 =
I B−1
.C
0 I
.
Then we have the following equation: E4E3E2E1A = I, and hence A = E−1
1 E−1
E−1
4 ,
where E = E3E2. Also, A−1
= E4EE1. Now, we have the well-known block factor-
ization of A and A−1
:
47
55. A =
I O
EB−1
I
B O
O S
I B−1
C
O I
, where S = G − EB−1
C.
A−1
=
I −B−1
C
O I
B−1
O
O S−1
I O
−EB−1
I
.
Sine we assumed that A is strongly nonsingular modulo p, then this implies also
B and S are strongly nonsingular modulo p. The following is the divide-and-conquer
algorithm for computing det A.
Algorithm 4.4.1 Computing the determinant of a strongly nonsingular ma-
trix
Input: an n × n strongly nonsingular matrix A
Output: det A
Computations:
1) Partition the input matrix A according to the block factorization above, where B is
an
n
2
×
n
2
matrix. Compute B−1
and S using the matrix equation S = G−EB−1
C.
2) Compute det B and det S.
3) Compute and output det A = (det B) det S.
Example 4.4.1: Let A =
5 11 7 16
13 1 11 20
4 8 9 2
1 19 12 3
. We partition the matrix as follows:
B =
5 11
13 1
, C =
7 16
11 20
, E =
4 8
1 19
, G =
9 2
12 3
.
B−1
=
−1
138
11
138
13
138
−5
138
, S =
9 2
12 3
−
4 8
1 19
−1
138
11
138
13
138
−5
138
7 16
11 20
=
48
57. CHAPTER 5
NUMERICAL IMPLEMENTATION OF MATRIX DETERMINANT
5.1 Numerical experiment of the solution based on Gaussian elimination
Algorithm 2.5.1 computes the triangular matrices L = L + EL and U = U + EU ,
the permutation matrices P and P1, where EL and EU are the perturbations of L
and U. Usually, we can assume that P = P and P1 = P1 since the rounding error is
small. However, in general we do not need to assume this.
Definition 5.1.1. A peturbation is a mathematical method that gives approximate
solutions to problems that can not be solved exactly.
Definition 5.1.2.
A = P−1
AP−1
1 , A = A + EA = LU, (5.1.1)
and let e , a, l, and u denote the maximum absolute value of the entries of the matrices
EA , A, L, and U.
Here, we assume floating point arithmetic and we round to β bits. Then the upper
bound on the magnitude of the relative rounding errors is = 2−β
, where is the
machine epsilon. So our goal is to estimate e .
Theorem 5.1.1. [19, p. 181] For a matrix A = (ai,j), let |A| denote the matrix
(|ai,j|). Then under (5.1.1),
|EA | ≤ (|A | + |L||U|) ,
and
e ≤ e+
= (a + nlu) . (5.1.2)
58. Theorem 5.1.2. From section 1.5, we recall some of the matrix norm properties.
In particular, (
1
n
)||A||q ≤ max
i,j
|ai,j| ≤ ||A||q, for q = 1, 2, ∞. Then, we obtain the
following bound: e ≤ ||EA||∞. Also, we recall the following result:
e ≤ e+ = ||EA||∞ ≤ n2
a+
, (5.1.3)
where a+
denotes the maximum absolute value of all entries of the matrices A(i)
,
computed by Gaussian elimination, which reduces A to the upper triangular form.
By using the following results, we can estimate the magnitude of the perturbation
of det A and det A caused by the perturbation of A . Here we assume that P =
P1 = I, A = A and write,
e = e , ed = det(A + EA) − detA (5.1.4)
Fact 5.1.1. |detA| ≤ ||A||n
q , q = 1, 2, ∞.
Fact 5.1.2. ||B||q ≤ ||A||q, q = 1, ∞, if|B| ≤ |A|; ||B||q ≤ ||A||q, q = 1, 2, ∞, if B is a
submatrix of A.
Fact 5.1.3. ||A + EA||q ≤ ||A||q + ne, for q = 1, 2, ∞; ||A + EA||2
2 ≤ ||A||2
2 + ne.
Lemma 5.1.1.
|ed| ≤ (||A||q + ne)n−1
n2
e, for q = 1, 2, ∞.
51
59. Combining Lemma 5.1.1. with the bound (5.1.2.) enables us to extend Algorithm
2.5.1 as follows.
Algorithm 5.1.1
Input: an n × n real matrix A and a positive .
Output: a pair of real numbers d− and d+ such that
d− = det U − e+
d ≤ det A ≤ d+ = det U + e+
d .
Computations:
1) Apply Algorithm 2.5.1 in floating point arithmetic with unit roundoff bound by .
Let U = U + EU denote the computed upper triangular matrix, where EU is due to
the accumulation of rounding errors and approximating U of A from the factorization
A = PLU or A = PLUP1.
2) Compute the upper bound e+ where e+ = ||EA||∞ of (5.1.3).
3) Substitute e+ for e and min(||A||1, ||A||∞, (||A||1||A||∞)
1
2 ) for ||A||q in Lemma
(5.1.1) and obtain an upper bound e+
d on |ed|.
4) Output the values det U − e+
d and det U + e+
d .
Example 5.1.1: Let A =
0 5
3
−31
6
19
4
3
2
16
5
17
5
21
4
12
9
.
Step 1: Interchange C1 ↔ C2 :
5
3
0 −31
6
3
2
19
4
16
5
21
4
17
5
12
9
.
Step 2: Pivot element a31 = 21
4
, so interchange R1 ↔ R3 :
21
4
17
5
12
9
3
2
19
4
16
5
5
3
0 −31
6
.
52
61. = max(.00012, .00000, .00003)
e+ = .00012.
We obtain an upper bound e+
d on |ed| by Lemma 5.1.1,
|ed| ≤ (min(||A||1, ||A||∞, (||A||1||A||∞)
1
2 ) + ne+)n−1
n2
e+.
||A||1 = max(|0|+|4.750|+|3.400|, |1.667|+|1.500|+|5.250|, |5.167|+|3.200|+|1.333|)
= max(8.15, 8.417, 9.70)
||A||1 = 9.70
||A||∞ = max(|0|+|1.667|+|5.167|, |4.750|+|1.500|+|3.200|, |3.400|+|5.250|+|1.333|)
= max(6.834, 9.450, 9.983)
||A||∞ = 9.983
(||A||1||A||∞)
1
2 = (9.700)(9.983) = 9.840
Therefore, |ed| ≤ (min (9.700, 9.983, 9.840) + 3(.00012))2
· 32
(.00012) = (9.700 +
.00036)2
· .00108 = .101625.
On the other hand, |ed| = |det (A+EA) - det A| = |−94.9172−(−94.9182)| = .00100
where, A + EA =
0 1.667 −5.167
4.750 1.500 3.200
3.400 5.250 1.333
+
0 + .00006 + .00006
.00000 + .00000 + .00000
.00000 + .00000 + .00003
=
.00000 1.6671 −5.1669
4.7500 1.5000 3.2000
3.4000 5.2500 1.3330
.
det(A + EA) = −94.9172, and det(A) = −94.9182.
So we can choose an upper bound e+
d > |ed| = .00100, and output the values
det U − e+
d and det U + e+
d . Choose e+
d = .101625. Then det U − e+
d = −94.8942 −
.101625 = −94.9958, and det U + e+
d = −94.8942 + .101625 = −94.7926.
Therefore, the bound placed on det A is
−94.9958 ≤ det A ≤ −94.7926.
54
62. Table 1: Average estimated relative error on random matrices.
Matrix Size PLUP1
2 6.1731e-14
3 2.5467e-13
4 4.0959e-12
5 9.8509e-11
6 1.0134e-09
7 2.2630e-08
8 1.9591e-07
9 2.5812e-06
10 2.9829e-05
Remark 5.1.1. We may obtain the sign of det A as the sign of det U provided that
e+
d = |detU|.
Otherwise, additional computations are needed. They can be simplified by the bound,
|detA| ≤ 2e+
d .
Table 1, above, presents the results of my experiment. 1000 random matrices of each
size (from 2 ≤ n ≤ 10) have been tested for the PLUP1 factorization. The relative
errors min(1, |e+
d |) if det A = 0, min(1, |
e+
d
det A
|) otherwise, have been evaluated and
the average values have been presented in the table.
The algorithms have been implemented in MatLab. All numerical operations have
been performed with double precision floating point arithmetic. The system pseudo-
random number generator function randi() have been used to generate input matri-
ces. See Appendix.
55
63. CHAPTER 6
COMPUTATIONAL COMPLEXITY AND PRECISION
A t-digit, base β floating-point number has the form f = ±.d1d2 . . . dt × βt
,
d1 = 0, β, , di are integers and 0 ≤ di ≤ β − 1. For internal machine representation,
β = 2 (binary representation), and for paper and pencil we use β = 10. The value of
t is called the precision, and the exponent can vary with the choice of hardware and
software. Let F be the set of floating-point numbers, which is a finite set, that is, no
real number can be found in F. fl is a way of approximating a real number. Given
a real number x, the floating-point approximation fl(x) is defined to be the nearest
element in F to x.
Example 6.1.1: In 2-digit, base-10 floating-point arithmetic, fl(
3
80
) = fl(0.0375) =
fl(.375 × 10−1
) = .38 × 10−1
= 0.038
The complexity of a function explains what happens as the size of the problem
being solved gets larger. We express complexity by using ”big O” notation. Arith-
metic operations propagate rounding errors. When floating point arithmetic is used,
such large row multipliers tend to propagate rounding errors. This type of error
propagation can be lessened by appropriate row interchanges that produce smaller
multipliers.
Definition 6.1.1. f(n) = O(g(n)) as n → ∞ iff there exists a positive real number
c and a real number n0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0. We say that ”f(n)
is big-O of g(n)”.
Remark 6.1.1. As n increases, f(n) grows no faster than g(n). In other words, g(n)
is an asymptotic upper bound on f(n).
64. We could have come to the same conclusion from the following lemma.
Lemma 6.1.1. f(n) ∈ O(g(n)) iff
lim
n→∞
f(n)
g(n)
= c ≥ 0.
Example 6.1.2:
1
2
n2
− 3n ∈ O(n2
).
Since lim
n→∞
1
2
n2
− 3n
n2
=
1
2
, then by the above lemma 1
2
n2
− 3n ∈ O(n2
).
The computational complexity of Gauss elimination with partial pivoting is O(n3
).
Even for the following factorizations: A = LU, A = PLU, and A = PLUP1, which
rely on Gaussian elimination required O(n3
) ops. Whereas the QR, that is, A = QR
required (4
3
)n3
+ O(n2
).
In particular, and in the QR decomposition, the Householder algorithm computes
Q as a product of n − 1 matrices of Householder transformations, therefore stage 3
of algorithm 2.7.1 uses only n − 1 multiplications. Stage 1, where we compute the
matrix Q involves (4
3
)n3
+O(n2
) multiplications [11]. With the Givens algorithm, this
bound turns into 2n3
+ O(n2
) ops. This shows that the Householder algorithm has
a minor advantage. Alternatively the Gram-Schmidt algorithm requires 4n3
+ O(n2
)
arithmetic operations.
The computation modulo a fixed prime of the determinant of a matrix can be ob-
tained by Gaussian elimination with partial pivoting involving at most (2
3
)n3
+O(n2
)
ops modulo p and at most (n−1)n
2
comparisons of integers modulo p.
For the recovery of det A from det A modulo M, we need O(kn3
) arithmetic
operations at the first computation, O((k log m1) log log m1) operations at the second
stage of computations, and O(klog2
k) operations at the last stage. The computations
are performed with the precision of log2 m1 bits at stage 1, and log2 M bits at
stage 2 and 3. These computations require fewer arithmetic operations but with a
higher precision. The approach based on the Chinese remainder theorem decreases
57
65. the precision of computations at the cost of increasing the arithmetic cost.
Algebraic computations such as p-adic (Newton-Hensel’s) lifting decrease the pre-
cision with no rounding errors. This approach is applied to computing det A by using
n4
log n arithmetic operations, and clearly inferior to that of the recovery of det A
for larger n. In addition, it has a disadvantage by involving a greater amount of
arithmetic operations. In particular the p-adic lifting of matrix inverses uses a total
of O(hn3
) arithmetic operations with a precision of J log2 p bits. When we extend
p-adic lifting of matrix inverses to p-adic lifting of (det A) mod p, it requires a com-
putational cost of hn4
arithmetic operations performed at stage 1 with a precision of
Hlog2 p bits. At stage 2 we use O(n3
) arithmetic operations. To avoid inverting
all k × k leading principle submatrices, we perform block Gauss-Jordan elimination.
This technique involves O(n3
) ops.
58
66. CHAPTER 7
DISCUSSION
We compare the results of the numerical experiment performed based on algo-
rithm 2.5.1. We presented the results of our experiment. 1000 random matrices of
each size, from 2 to 10 have been tested. The relative errors have been evaluated
in each case, and their average values have been presented in Table 1. The numer-
ical algorithms (QR, LU, PLU, PLUP1) based on Gaussian elimination seem to be
the most effective for the solution to the problem of determinants being applied in
computational geometry. We verified this output is correct by the accumulation of
rounding errors in our computations. This enables us to obtain both the product of
d of the diagonal entries L and U, and a positive e such that det A is bracketed in
the interval d − e ≤ det A ≤ d + e. If e < |d|, then d and det A have the same sign
and our main problem of the sign computation is solved.
All of the numerical algorithms involve an elimination process. Gauss elimination
is a well known technique which requires O(n3
) arithmetic operations to reduce the
matrix. The orthogonal factorizations require 4
3
n3
+ O(n3
) ops. Although algebraic
algorithms based on the Chinese remainder theorem and Newton-Hensel’s p-adic lift-
ing reduces the precision, it is at the cost of increases the arithmetic operations.
Therefore, our conclusion of the numerical algorithms seemed to be the most effective
was correct.
Clarkson [3] was able to bound the precision of computations more effectively in
the case where n ≤ 3. He produced an algorithm that is based on the QR factoriza-
tion, which is a numerical algorithm. Although he just focused on just one algorithm,
this paper also supports my conclusion of the numerical algorithms seeming to be the
67. best fit. [21] went about the results in a different way. This paper certifies the sign
of det A without estimating the roundoff error of computing det A. Instead, they
estimated the minimum distance N from the matrix LU to a singular matrix. Each
article I research had a variation of methods used and different paths to the outcome,
the conclusion was inevitably still the same. Numerical computations are the best fit
to the problem of geometric applications.
60
68. CHAPTER 8
CONCLUSION
In this thesis, we researched various techniques for computing det A or its sign.
We studied numerical and rational algebraic (no errors) algorithms for computing
different factorizations of the input matrix A. Among those, the orthogonal that
includes the QR factorizations having an arithmetic count of 4
3
n3
+ O(n2
) and the
triangular that includes PLUP1, PLU, and LU factorizations having an arithmetic
count of O(n3
).
The numerical algorithms seem to be the most effective ones for computing the sign
of det(A). The sign of det A can be immediately read off from the signs of the
diagonal entries of the computed triangular matrices, that is, from either R or L
and U, respectively. We verified the output is correct by estimating the output error
caused by the accumulation of the rounding errors of our floating point computation
with finite precision.
The well known method of finding the determinant of a matrix is cofactor expan-
sion. In this method one either risks losing the correct answer due to rounding errors
or one needs to double the input precision. When we calculate the determinant of an
n×n matrix using cofactor expansion we must find n of (n−1)×(n−1) determinants.
So roughly Cn ≈ nCn−1, where Cn is the complexity of finding an n × n determinant.
To aviod involving higher precision values we perform elementary transformation on
a matrix A. Gauss elimination, with or without pivoting is one way to perform these
transformations. Gaussian elimination with partial pivoting computes the factoriza-
tion A = PLU and A = PLUP1 (if P1 = I) in (n−1)n
2
ops (arithmetic operations),
but uses (2n−1)(n−1)n
6
ops for A = PLUP1. Therefore, there is a small advantage in
69. using partial pivoting rather than complete pivoting.
For small values of n the cofactor method has an advantage over Gauss elimina-
tion, but as n grows, the computational cost, O(n!), gets big very quickly and the
cofactor method becomes impractical. The algorithms have been implemented and
compared based on the number of arithmetic operation each algorithm uses. After
carefully evaluation we have concluded that it is in fact true that the numerical al-
gorithms, PL, PLU, PLU1 have fewer arithmetic operations. The data we computed
for the average relative error, verifies that this conclusion is correct. This is an ideal
example of effective combination of symbolic and numerical techniques.
The algebraic methods based on Chinese remaindering and p-adic lifting for find-
ing the determinant can be extended to any algorithm that outputs det A or its
sign. For example, this can be done to the straightforward algorithm of computing
the determinant based on equation (1.1.1) or (1.1.2). Although these methods use a
low precision of computing with no rounding errors, it is at the cost of increases the
arithmetic operation count.
Therefore, the main idea is to choose an algorithm for computing det A at lower
bit-precision and lower arithmetic complexity. Such a decrease is highly important
for applications to computational geometry. By combining several numerical and
algebraic techniques for computing the determinant, we have obtained and verified
a solution to this problem. We conclude that the numerical algorithm seemed to be
the best fit.
62
70. APPENDIX
MATLAB CODE
GECP calculates Gauss elimination with complete pivoting. (G)aussian (E)limination
(C)omplete (P)ivoting
Input: An n × n matrix
Output:
L = Lower triangular matrix with ones as diagonals
U = Upper triangular matrix
P and P1 permutations matrices so that PAP1 = LU
function [L, U, P, P1] = gecp(A)
format double;
[n, n] = size(A);
p = 1 : n;
p1 = 1 : n;
for k = 1 : n − 1
[maxc, rowindices] = max( abs(A(k : n, k : n)));
[maxm, colindex] = max(maxc);
row = rowindices(colindex)+k − 1;
col = colindex +k − 1;
A([k,row], : ) =A( [row, k], : );
A(:, [k, col] ) =A( :, [col, k]);
p([k,row] ) =p( [row, k]); p1([k, col] ) =p1( [col, k] );
ifA(k, k) == 0
break
end
63
71. A(k + 1 : n, k) = A(k + 1 : n, k)/A(k, k);
i = k + 1 : n;
A(i, i) = A(i, i) − A(i, k) ∗ A(k, i);
end
L = tril(A, −1) + eye(n);
U = triu(A);
P = eye(n);
P = P(p, :);
P1 = eye(n);
P1 = P1(:,p1);
Input: Random n × n matrix
Output: Average estimated relative error: min(1, |e+
d |) if det A = 0, min(1, |
e+
d
det A
|)
otherwise.
data = zeros(1,10000);
for loopIdx = 1:10000;
r=randi([-32687,32687],n, n);
d=det(r);
q = norm(r, 1);
p = norm(r,inf);
o = (q · p)
1
2 ;
M = [q, p, o];
J=min(M);
e = n2
· max(max(r)) · 2−64
;
M1 = (J + ne)n−1
· (n2
e);
64
72. if d == 0
data(loopIdx) = min(1, (abs(M1)));
else
data(loopIdx) = min(1, (abs(M1 / det(r))));
end
65
73. BIBLIOGRAPHY
[1] J. Edmonds, Systems of distinct represenatives and linear algebra, J. Res. Nat.
Bur. Standards, Sect. B 71(4), 241-245, (1967).
[2] E. H. Bareiss, Sylvester’s identity and mulitstep integer-preserving Gaussian elim-
ination. Math. of Comp., 22, 565-578, (1969).
[3] F. Avnaim, J. Boissonnat, O. Devillers, F.P. Preparata, M. Yvinec, Evaluating
Signs of Determinants Using Single-Precision Arithmetic. Algorithmica, 17, 111-
132, 1997.
[4] G. Barker,& H. Schneider, Matrices & Linear Algebra. New York, NY: Dover
Publishing, Inc., (1968).
[5] D. Bini,& V. Y. Pan, Polynomial and Matrix Computations. Fundamental Algo-
rithms, Vol 1., Birkh¨auser, (1994).
[6] K. L. Clarkson, Safe and effective determinant evaluation. 33rd Annual IEEE
Symp. on Foundations of Computer Science, 387-395, (1992).
[7] D. Falvo, & R. Larson, Elementary Linear Algebra. Sixth Edition. Belmont, CA:
Brooks/Cole Publishing, (2009).
[8] Fox, L. An Introduction to Numerical Linear Algebra. Oxford: Clarendon Press.
[9] G. H. Golub, & C. F. Van Loan, Matrix Computations. Third Edition. John
Hopkins University Press, Baltimore, Maryland, (1996).
[10] T. Muir, The Theory of Determinants in Historical Order of Development, 1,
(1693-1841)
[11] V. Y. Pan, How to Multiply Matrices Faster. Lecture Notes in Computer Science.
Volume 179, (1984).
[12] V. Y. Pan, Fast and efficient parallel algorithms for the exact inversion of the
integer matrices. 5th Conference on Foundation of Software Technology and The-
oretical Computer Science, 206, 504-521, (1985).
[13] V. Y. Pan, Computing the determinant and the characteristic polynomial of a
matrix via solving linear systems of equations. Information Processing Letters,
28(2),71-75, (1988).
[14] V. Y. Pan, Simple Multivariate Polynomial Multiplication. J. Symbolic Compu-
tation, 18, 183-186, (1990).
[15] H. Rose, Linear Algebra: A Pure Mathematical Approach. Bertlesmann Springer
Publishing, (2002).
66
74. [16] K. Rosen, Elementary Number Theory and its Applications. 6th Edition. Addison
Wesley, (2011).
[17] I.Z. Emiris, V. Y. Pan, Improved Algorithms for Computing Determinants and
Resultants, Journal of Complexity, 21, 1,43-71, (2005).
[18] V. Pan, Y. Yu, C. Stewart, Algebraic and Numerical Techniques for Computation
of Matrix Determinants, Computers Math. Applications, 34, 1,43-70, (1997).
[19] S. D. Conte and C. de Boor, Elementary Numerical Analysis: An Algorithm
Approach, McGraw-Hill, New York, (1980).
[20] H. Bronnimann, I.Z. Emiris, V.Y. Pan, S. Pion Sign Determination in Residue
Number System, Theoretical Computer Science, (1998).
[21] V.Y, Pan, Y. Yu, Certified Numerical Computation of the Sign of a Matrix
Determinant.
67
