1. Some Multivariate Methods Used by Ecologists
Peter Chapman
Wokingham U3A
4 October 2012

2. Contents
• Assumptions
• Introduction
• Notation
• Refresher on matrices
• Eigen vectors and eigen values
• Principal component analysis
• Principal coordinate analysis
• Correspondence analysis
• Redundancy analysis and canonical correspondence analysis
• R Software

3. Assumptions

4.  1 2 3
2
I am assuming that you are familiar with the following :
A sum of several numbers ............. , 1 to
The mean of several numbers
1 1
Variance ( ) , or
( 1)
n
i
i
n
i
i
n
i
i
x x x x i n
x
x
n
x x
n n
     
 
 

 


 
2
( )
1
Covariance ( )
( 1)
n
i
i
n
i i
i
x x
x x y y
n

  




5. Introduction

6. Before 1980 multivariate research tended to be theoretical. Often this
involved working on distribution theory for hypothetical but unrealistic
situations. Very rarely did anyone carry out a multivariate analysis.
Since 1980 or so, with growth of computing power, people started using
multivariate methods. This has led to further development of the old
methods, plus introduction of a lot of new methods.
Starting in the mid 1980s a lot of biologically trained people who were
also computer literate started appearing in the work place. These
people, who rarely had any formal training in mathematics or statistics
were able to run multivariate software and get results. This led to a lot
of nonsense, including some published nonsense.

7. 1 2 3 4 5 6 7
1 3 8
2 0 9
3 13 4
4 5 5
8 19
23
77 7 3
SpeciesSites
1 2 3 4 5 6 7
1 45.8 78.6
2 32.8 98.5
3 56.1 45.0
4
77 78.3
Counts
Environmental
Sites
e.g. Soil, Climate, etc.
Real Numbers
Typical Example of Ecology Data
In pesticide research “site”
could refer to different
chemicals, or different rates
of the same chemical

9. Example data sets: Although this is a talk about methods used in
ecology, I shall be using data from other sources.
This is because it is easier to understand what is going on if data is
more familiar.
Also, because during my search of the web I found it quite difficult to
find suitable ecological examples.
I have also tended to use data with small numbers of dimensions

10. Notation

11. is an matrix :
I will call it an n by p matrix - which means n rows and p columns
Always in blue, and always a captital if p >1
And using the matric"style"in "Mathtype w hit
n pY
dimensions in subscripts
also represents matrix
showing generic cell member,
with subscripts but not dimensions....................not used very much
ijy  
11 12 13 1
21 22 23 2
31 32 33 3
1 2 3
. . .
. . .
. . .
. . . . . . . is another way of representing a matrix
. . . . . . .
. . . . . . .
. . .
p
p
p
n n n np
y y y y
y y y y
y y y y
y y y y
 
 
 
 
 
 
 
 
 
 
 
If matrix has only one row or only one column i will use lower case blue,e.g. n×1u

12. Refresher on Matrices

13. Generic Matrix
Square Matrix
Row Matrix Column Matrix
Vector
11 12 13 1
21 22 23 2
31 32 33 3
1 2 3
. . .
. . .
. . .
. . . . . . .
. . . . . . .
. . . . . . .
. . .
p
p
p
ij
n n n np
y y y y
y y y y
y y y y
y
y y y y

 
 
 
 
 
     
 
 
 
 
 
n pY
11 12 13
21 22 23
31 32 33
ij
y y y
y y y y
y y y

 
      
 
 
3 3Y
 11 12 11 3 1. . . . . . ny y y y pY
11
21
31
1
.
.
.
.
n
y
y
y
y

 
 
 
 
 
 
 
 
 
 
 
 
 
n 1Y

14. Matrix Multiplication
  3 3 3 2 2 3Q B C
11 12 13 11 12
11 12 13
21 22 23 21 22
21 22 23
31 32 33 31 32
q q q b b
c c c
q q q b b
c c c
q q q b b
   
    
     
    
   
11 11 11 12 21
12 11 12 12 22
13 11 13 12 23
33 31` 13 32 23
.
q b c b c
q b c b c
q b c b c
q b c b c
 
 
 
 
 32 32 32Q B C
11 12 11 12 11 12 11 11 12 12
21 22 21 22 21 22 21 21 22 21
31 32 31 32 31 32 31 31 32 33
q q b b c c b c b c
q q b b c c b c b c
q q b b c c b c b c
        
       
           
               
Matrix Addition

15. Diagonal (Square) Matrix
11
22
33
44
55
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
ij
a
a
b a
a
a
 
 
 
      
 
 
 
5×5B
Identity Matrix
1 0 0 0 0
0 1 0 0 0
0 0 1 0 0
0 0 0 1 0
0 0 0 0 1
 
 
 
 
 
 
 
 
5×5I
n×n n×n n×n n×nI C C I

16. Norm: Normalisation
1
2
3
.
.
.
.
n
b
b
b
b
 
 
 
 
 
 
 
 
 
 
 
 
 
nb
3
4
 
 
 
4
3
 
 
 
4
3
2 2 2
21 ....... nb b b   b
2 2
3 4 5  b
Length or Norm
/ / 5  b b b b b
Normalised
/b b b

17. Transpose of a Matrix
11 12 13 14
21 22 23 24
31 32 33 34
41 42 43 44
51 52 53 54
11 21 31 41 51
12 22 32 42 52
13 23 33 43 53
14 24 34 44 54
the transpose of is
y y y y
y y y y
y y y y
y y y y
y y y y
y y y y y
y y y y y
y y y y y
y y y y y
 
 
 
 
 
 
 

 

 




 
5×4
5×4 4×5
Y
Y Y
( ) =
( ) =
 
  






n×p p×q q×p p×n
n×p p×q q×m m×q q×p p×n
A B B A
A B C C B A

18. Scalar Product
 1 2 3 1
2
3
. .
.
.
n
n
b c b b b b c
c
c
c
      
 
 
 
 
 
 
  
 
1 n n 1b c
1 1 2 2 3 3 .......... n nbc b c b c b c   
If b and c are orthogonal then
  length of length of cos b c
cos cos90 0  
 1 2 3 1
2
3
. .
.
.
n
n
b c b b b b b
b
b
b
    
 
 
 
 
 
 
 


 
1×n n×1b b
1 1 2 2 3 3 .......... n nbb b b b b b c   
0  1 n n 1b cso
  length of length of cos0 b b
 
2
length of b
1 if is normalised b

19. Determinant
11 12
11 22 12 21
21 22
b b
b b b b
b b
 
   
 
B
11 12 13
21 22 23
31 32 33
22 23 21 23 21 221 1 1 2 1 3
11 12 13
32 33 31 33 31 32
( 1) ( 1) ( 1)
b b b
b b b
b b b
b b b b b b
b b b
b b b b b b
  
 
 
  
 
 
     
          
     
B
Scalar

20. Rank of a Square Matrix
1 1 1
3 0 2
4 1 3
  
 
 
 
 
(-2*Col 1) = col 2 + (3*col3)
row 1 = row 2 – row 3
Only two linearly independent
(orthogonal) rows so rank = 2.
2 1 4
2 1 4
2 1 4
 
 
 
  
(-2*col1) = (4*col2) = col 3
row 1 = row 2 = row 3
Rank = 1
Order of a square matrix is number of rows/columns
Rank of a square matrix is the number of linearly independent rows/columns
A square matrix whose rank is less than its order has a determinant of zero.
If a square matrix has a non-zero determinant it has full rank = number of rows or columns
A full rank square matrix is called non-singular

21. Inverse of a Square Matrix
If is non singular then  -1 -1
B BB B B I
is called theinverse of-1
B B
1 1
1 0
3 1
 
 
  
  
32B
1 3 1
2 5 1
 
   
 
23 23 32 22 32 23 33C C B I B C I
4 15 4
7 25 6
 
   
 
23 23 32 22 32 23 33C C B I B C I
C is not unique
http://www.mathwords.com/i/inverse_of_a_matrix.htm
http://mathworld.wolfram.com/MatrixInverse.html
http://www.purplemath.com/modules/mtrxinvr.htm

22. Association Matrices

23. Association Matrices Q-Mode
11 1
1
p
n np
y y
y y
 
 
  
 
 
npY
 11 12 13 1. . . . . . py y y y
 21 22 23 2. . . . . . py y y y
Row 1
Row 2
       
2 2 2 2
12 21 11 21 11 21 11 21 11 21.......a a y y y y y y y y         
Euclidean Distance
Correlation
  
   
1 1. 2 2.
1
12 21
2 2
1 1. 2 2.
p
j j
j
j j
y y y y
a a
y y y y

 
 
          

11 12 13 1
21 22 23 2
31 32 33 3
1 2 3
. . .
. . .
. . .
. . . . . . .
. . . . . . .
. . . . . . .
. . .
n
n
n
n
i
n n
n j
n nn
a a a a
a a a a
a a a a
a
a a a a

 
 
 
 
 
     
 
 
 
 
 
A

24. 11 1
1
p
n np
y y
y y
 
 
  
 
 
npY
11
21
31
1
.
.
.
.
n
y
y
y
y
 
 
 
 
 
 
 
 
 
 
 
 
 
12
22
32
2
.
.
.
.
n
y
y
y
y
 
 
 
 
 
 
 
 
 
 
 
 
 
11 12 13 1
21 22 23 2
31 32 33 3
1 2 3
. . .
. . .
. . .
. . . . . . .
. . . . . . .
. . . . . . .
. . .
p
p
p
ij
p p p pp
a a a a
a a a a
a a a a
a
a a a a
 
 
 
 
 
     
 
 
 
 
 
nnA
Compute
12 21a a
Association Matrices R-Mode

25. Summary: Q and R Mode
Descriptors Objects
Objects
e.g. Sites
Descriptors
e.g. Species
n pY n nA
p pA
R mode
association
matrix
Q mode
association
matrix
Original
data
matrix

26. Eigen Values and Eigen Vectors

27. Ecological data sets usually include a large number of variables that are associated to one
another (e.g. linearly correlated). The basic idea underlying several methods of data analysis is
to reduce this large number of inter-correlated variables to a smaller number of composite, but
linearly independent variables, each explaining a different fraction of the observed variation.
One of the main goals of numerical data analysis is to generate a small number of variables,
each explaining a large proportion of the variation, and to ascertain that these new variables
explain different aspects of the phenomena under study.
Eigen analysis is a key tool in helping us achieve this aim.

28. Eigen Values and Eigenvectors
For any square matrix the following relationship always exists
Where the colums of are orthonormal

n×n
-1
n×n n×n n×n n×n n×n n×n n×n
n×n
A
A U Λ U U Λ U
U
11 12 13
21 22 23
31 32 33
. . .
. . .
. . .
. . . . . .
. . . . . .
ij
a a a
a a a
a a a a
 
 
 
    
 
 
 
 
n×nA
1
2
3
0 0 . .
0 0 . .
0 0 . .
. . . . .
. . . . .



 
 
 
 
 
 
 
 
n×nΛ
11 12 13 1
21 22 23 2
31 32 33 3
1 2 3
. .
. .
. .
. . . . . .
. .
n
n
ij n
n n n nn
u u u u
u u u u
u u u u u
u u u u
 
 
 
      
 
 
 
n×nU
Matrix is known as the canonical form ofn×n n×nΛ A
Any square matrix
Eigenvalues
Some may be zero
Some may be equal
Lagrange multipliers
Columns are eigenvectors
Columns orthonormal

29. Eigen Values and Eigenvectors
We compute eigen values of square matrix by solving n equations:
i
i iAu u i = 1 to n
i i iAu λ u 0
( )i i A I u 0
n*n matrix n*1 vector
n*1 vector
0i A Iare found by solving
i a polynomial of degree n
iu are then easily found characteristic equation
n×nA

30. Singular Value Decomposition

31. Singular Value Decomposition (SVD)
Any matrix can be factorised as follows:
Columns of are the left singular vectors of
is a diagonal matrix containing (non -negative) singular value




n×p
n×p n×p p×p p×p
n×p n×n n×p p×p
n×n n×p
n×p
Y
Y V W U
Y V W U
V Y
W s
Columns of are the right singular vectors ofp×p n×pU Y
Lack of consistency in literature

32. SVD can be applied to any m × n matrix.
Eigenvalue decomposition can only be applied to certain classes of square matrices.
Nevertheless, the two decompositions are related.
Given an SVD of M, as described above, the following holds:
)(
   
   
p×n n×p p×p p×n n×n n×n n×p p×p
p×p p×n n×p p×p p×p p×p p×p
Y Y U W V V W U
U W W U U Λ U
SVD
Transpose
of SVD
   n×p p×n n×n n×p p×n n×n n×n n×n n×nY Y V (W W )V V Λ Valso
eigenvectors eigenvalues

33. Principal Component Analysis
Karl Pearson, 1901
Harold Hotelling, 1933, 1936

34. 11
21
1
We have a matrix with variance-covariance matrix
We transform as follows
Now consider the first column of
:
.
or .
.
.
n
z
z
z

 











n×p p p
n×p n×p n×p p×p
n×p n×1 n×p n×1
Y S
Y Z Y U
Z z Y u
11 12 1 11 11 11 12 21 1 1
21 22 2 21 21 11 22 21 2 1
1 1 1 11 2 21
. . .....
. . .....
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . .....
p p p
p p p
n np p n n
y y y u y u y u y u
y y y u y u y u y u
y y u y u y u y
     
         
   
   
    
   
   
   
            1np pu
 
 
 
 
 
 
 
 
 
 
 

35.  
 
2
1 .
1
2
1 11 2 21 1 .1 11 .2 21 . 1
1
2
1 .1 11 2 .2 21 21 1
1
2 2
1 .1 11 1 .1 2 .2 11 21
1
( )
1
..... ( ..... )
1
( ) ( ) ..... ( )
1
( ) ( )( ) ..............
n
i p
n
i i ip p p p
n
i i ip ip p
i i i
Var z z
n
y u y u y u y u y u y u
n
y y u y y u u y y u
n
y y u y y y y u u
n
 
       
        
     



n×1z
1
..........
and so on
n

  


1×p p×p p×1u S u

36. 1
1
Now we need to find that maximises var[ ]subject to =1
Let be a Lagrange multiplier
Then maximise = ( -1)

 

 
p×1 p×1 1×p p×1
1×p p×p p×1 1×p p×1
u z u u
u S u u u
1
1
1
0
( ) 0
We find first by solving the polynomial



 


 
p×p p×1 p×1
p×p p×p p×1
p
S u (1)u
u (2S I
S
)
1
1
1
1
0 (degree p)
We then substitute into to find
From we can show that var[ ] , subject to =1
is the first eig
(
e
1)







 
×p p×p
p×1
1×p p×p p×1 p×1 1×p p×1
I
u
u S u z u
(2)
u
n value of and is the first eigen vectorp×p p×1S u

37. To compute 2nd,3rd, and higher eigen values and eigen vectors is an identical process
Additional constraints are needed
Notably ( ) = 0 equivalent to Cov( ) = 0
where and are d
,
i
1×p p×1 p×1 p×1
p×1 p×1
u v u v
u v fferent eigen vectors

38. Burglary Violence Robbery
Bath 3 5.7 0.4
Birmingham 7.2 11.6 4.1
Brighton 4.5 7.6 1.5
Bristol 9.8 10.4 2.9
Cambridge 7.6 7.6 1.6
Canterbury 2.7 6.3 0.5
Cardiff 6 8.6 1.1
Coventry 7.4 11.2 2.2
Lancaster 2.5 7.7 0.4
Leeds 12 7.5 1.7
Leicester 8.6 11.7 2.6
Liverpool 9.1 7.7 2.3
Manchester 13.2 11.5 5.1
Newcastle 5.2 8.6 0.9
Nottingham 11.7 15.1 4.3
Oxford 5.2 8.3 1.7
Sheffield 6.9 7.5 1.2
Southampton 4.9 14.6 1.8
Swansea 4.7 7.3 0.4
York 4.4 6.7 0.5
Bournemouth 5.2 12.9 0.9
Plymouth 4 10.7 0.7
Norwich 3.9 8.9 1.1
Lincoln 6 9.8 0.9
Objects
Source: http://www.thecompleteuniversityguide.co.uk/preparing-to-go/staying-safe-and-secure/how-safe-is-your-city/
Descriptors

39. Robbery
Violence
Violence
Burglary
Burglary
Robbery

40. 3.0 5.7 6.5 9.4 3.49 3.70
7.2 11.6 6.5 9.4 0.71 2.20
4.5 7.6. . . 1.99. 1.80
. . . . . .
=
. . . . . .
. . . . . .
. . . . . .
6.0 9.8 6.5 9.4 0.49 0.40
     
   
   
     
   
    
   
   
   
   
           
24×2 24×2 24×2 24×2Y Y Y Y-

 
 
 
 
 
 
 
 
 
  

Burglary Violence
Let’s start with two variables

41. 8.799 3.0361
3.036 6.5051n
             
24×2 24×2 24×2 24×2 24×2S Y Y Y - Y
R-Mode
08.799 3.036
0 1 2
03.036 6.505
k
k
k
k or



  
     


  
24×2 24×2S I
Eigen Values = 10.898 = 4.407
1 2
-0.8226264 0.5685823
-0.5685823 -0.8226264
 
  
 
2×2UEigen Vectors
Variance-covariance matrix

42. 3.49 3.70
0.71 2.20
1.99. 1.80
. . -0.8226264 0.5685823
. . -0.5685823 -0.8226264
. .
. .
0.49 0
(
.
)
40
  
 
 
  
 
         
 
 
 
   
24×2 24×2 24×2 2×2Z Y Y U
1 0.8226264 0.5685823z Burglary Violence    
Accounting for 10.898*100/(10.898+4.407) = 71.2 % of variance
2 0.5685823 0.8226264z Burglary Violence   
Accounting for 28.8% of Variance

43. Violence
2 0.5685823 0.8226264Z Burglary Violence   
Plots of 1st and 2nd Components
Robbery

44. Violence
Plots of 1st and 2nd Components
Robbery
1Z
2Z

2 2 2 2
1 2
2 2 2 2
1 2
1 1 1 1
1 2
1 1 1 1
variance( ) variance( ) variance( ) variance( )
n n n n
Robbery Violence Z Z
Robbery Violence Z Z
n n n n
Robbery Violence Z Z
  
  
  
   
The PCA rotation
maximises variance of Z1
relative to Z2
It also maximises
relative to
1
2

45. Now three variables
2 3
3.0 5.7 0.4 6.5 9.4 1.7 3.49 3.70 1
7.2 11.6 4.1 6.5 9.4 1.7
4.5 7.6. 1.5 6.5 9.4 1.7
. . . . . .
=
. . . . . .
. . . . . .
. . . . . .
6.0 9.8 0.9 6.5 9.4 1.7
     
   
   
   
   
    
   
   
   
   
      
   
4× 24×3 24×3 24×3Y Y Y Y-
.3
0.71 2.20 2.4
1.99. 1.80 0.2
. . .
. . .
. . .
. . .
0.49 0.40 0.8
 
 
 
   
 
 
 
 
 
 
    
Burglary Violence Robbery

46. 8.799 3.036 3.121
1
3.036 6.505 2.005
1
3.121 2.005 1.689
n
 
           

 


24×3 24×3 24×3 24×3 24×3S Y Y Y Y
8.799 3.036 3.121 0 0
3.036 6.505 2.005 0 0 0 1,2, 3
3.121 2.005 1.689 0 0
k
k k
k
k

 

   
   
      
   
   
24×3 24×3S I
Eigen Values = 12.205 = 4.411 = 0.378
0.7788 0.5562 0.2900
0.5318 0.8307 0.1648
0.3325 0.0259 0.9427
 
 
   
 
 
3×3UEigen Vectors
1 2 3

47. 3.49 3.70 1.3
0.71 2.20 2.4
1.99. 1.80 0.2
0.7788 0.5562 0.2900
. . .
0.5318 0.8307 0.1648
. . .
0.3325 0.0259 0.9427
. . .
. . .
0.49 0.40 0.
( )
8
   
 
 
   
  
        
    
 
 
    
24×3 24×3 24×3 3×3Z Y Y U

48. 1 0.7788 0.5318 0.3325z Burglary Violence Robbery     
Accounting for 12.205*100/(12.205+4.411+0.378) = 71.8 % of variance
2 0.5562 0.8307 0.0259z Burglary Violence Robbery     
Accounting for 26.0% of Variance
3 0.2900 0.1648 0.9427z Burglary Violence Robbery      
Accounting for 2.2% of Variance

49. -6
-4
-2
0
2
4
6
-6 -4 -2 0 2 4 6 8 10
High
Crime
High Burglary/ Lower Violence
Low
Crime
High Violence/ Lower Burglary
Bath
Canterbury
Lancaster
Leeds
Liverpool
Nottingham
Manchester
Bournemouth
Southampton
2Z
1Z

50. Violence
2 0.5685823 0.8226264Z Burglary Violence   
Plots of 1st and 2nd Components
Robbery

51. Bi-Plot
Ordination in 2 dimensions
Original 3-dimensions
Projection of 3 dimensions into 2 dimensions

52. Bi-Plot






















54. Both the direction and length of the vectors can be interpreted. So, for
these data, where the vectors represent judges, and the points cars, a
group of vectors pointing in the same direction correspond to a group
of judges who have the same preference opinions about the
automobiles

55. In a biplot, the length of the lines approximates the variances of the variables.
The longer the line, the higher is the variance.
The angle between the lines, or, to be more precise, the cosine of the angle
between the lines, approximates the correlation between the variables they
represent. The closer the angle is to 90, or 270 degrees, the smaller the
correlation. An angle of 0 or 180 degrees reflects a correlation of 1 or −1,
respectively.
Taken from
The Stata Journal (2005) 5, Number 2, pp. 208–223
Data inspection using biplots.
Ulrich Kohler, Wissenschaftszentrum Berlin
kohler@wz-berlin.de,Magdalena Luniak, Wissenschaftszentrum Berlin
luniak@wz-berlin.de
http://onlinelibrary.wiley.com/doi/10.1002/9780470238004.app2/pdf

56. Hardcover: 476 pages
Publisher: Wiley-Blackwell (24 Dec 2010)
Language: English
ISBN-10: 0470012552
ISBN-13: 978-0470012550

57. Bi-Plot
Ordination in 2 dimensions
Original 3-dimensions
Projection of 3 dimensions into 2 dimensions

58. US Agricultural Exports: 1990 to 2010
Country
Value (million dollars)
1990 1995 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
Canada 4,214 5,796 7,060 7,643 8,124 8,662 9,315 9,733 10,618 11,951 14,062 16,253 15,725 16,856
Mexico 2,560 3,522 5,625 6,410 7,407 7,238 7,891 8,510 9,429 10,881 12,692 15,508 12,932 14,575
Caribbean 1,015 1,281 1,493 1,408 1,400 1,518 1,590 1,850 1,913 2,114 2,575 3,592 3,082 3,192
Central America 483 871 1,100 1,121 1,234 1,252 1,338 1,429 1,589 1,832 2,363 3,106 2,553 2,923
Brazil 177 522 212 264 221 329 384 279 228 287 411 677 386 575
Colombia 120 464 441 417 453 521 513 594 680 868 1,223 1,675 907 832
Japan 8,142 11,149 8,893 9,292 8,884 8,384 8,906 8,147 7,931 8,390 10,159 13,223 11,072 11,819
Korea, South 2,650 3,742 2,449 2,546 2,588 2,673 2,886 2,489 2,233 2,851 3,528 5,561 3,917 5,308
Taiwan 3 1,663 2,591 1,945 1,996 2,010 1,966 2,025 2,065 2,301 2,477 3,097 3,419 2,988 3,190
China 3 4 818 2,633 854 1,716 1,939 2,068 5,017 5,542 5,233 6,711 8,314 12,115 13,109 17,522
Hong Kong 702 1,503 1,209 1,264 1,228 1,091 1,114 913 872 977 1,168 1,715 2,008 2,808
India 108 194 145 210 353 274 317 257 295 365 475 489 691 755
Indonesia 275 816 531 668 907 810 996 925 958 1,102 1,542 2,195 1,796 2,246
Philippines 381 765 783 901 794 776 626 695 798 888 1,112 1,734 1,294 1,634
Thailand 275 590 409 493 570 611 684 686 675 703 885 1,063 1,046 1,152
Australia 226 339 320 317 292 339 412 410 463 520 662 826 840 928
European Union 5 7,474 8,789 6,858 6,515 6,676 6,398 6,736 6,953 7,052 7,408 8,754 10,080 7,445 8,894
USSR 2,248 1,233 839 670 1,060 695 740 1,112 1,227 1,025 1,665 2,304 1,736 1,454
Saudi Arabia 482 517 447 477 429 343 332 364 350 426 710 890 694 840
Turkey 226 516 502 658 571 675 921 944 1,062 1,030 1,496 1,696 1,499 2,112
Egypt 687 1,309 966 1,050 1,022 863 967 935 819 1,022 1,801 2,050 1,354 2,092
South Africa 81 267 173 134 99 148 149 169 146 126 291 393 162 292
Oceania 343 506 486 490 473 512 621 601 742 760 963 1,189 1,282 1,394

59. (1) 55418724 (2) 26490533 (3) 13560738 (4) 8083174 (5) 6362753 (6) 3097378.91
(7) 1717442 (8) 1214768 (9) 344449 (10) 137884 (11) 67213 (12) 49836 (13) 11222 (14) 6340
Eigenvalues

61. Principal Coordinate Analysis
Gower, 1966

62. Start with a matrix with n rows (objects) and p columns (descriptors)
Compute the Euclidean distance (between rows) matrix [ ]
Transform into new matrix = [a ], such that :
hi
hi
d
n×p
n×n
n×n n×n
Y
D
D A
21
= -
2
Then compute centred matrix [ ]: . . ..
Finally, scale eigen vectors so that
If eigen vectors are arranged as columns. Rows of the
hi hi
hi hi hi h i
k
a d
a a a a 

    

n×p
k k
Δ
u u
resulting table are the
coordinates of the objects in the space of principal coordinates.

63. Oxford Newcastle Plymouth Dover Glasgow Bristol
Oxford 0 258 187 144 363 72
Newcastle 258 0 408 368 145 297
Plymouth 187 408 0 287 484 118
Dover 144 368 287 0 498 194
Glasgow 363 145 484 498 0 372
Bristol 72 297 118 194 372 0
Distances Between British Cities
Q-Mode

64. 0 258 187 144 363 72
258 0 408 368 145 297
187 408 0 287 484 118
144 144 287 0 498 194
363 363 484 498 0 372
72 72 118 194 372 0
 
 
 
 
  
 
 
  
 
6×6D
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] 0.1473475 0.1128741 0.04538458 0.4082483 0.81339727 0.3677045
[2,] -0.4335791 0.2304177 0.65857972 0.4082483 -0.30873244 0.2514103
[3,] 0.3970476 -0.6344915 0.35761362 0.4082483 -0.03729046 -0.3792480
[4,] 0.3754683 0.6761005 -0.15553511 0.4082483 -0.16408006 -0.4291055
[5,] -0.6752523 -0.1782828 -0.42405981 0.4082483 0.13883744 -0.3827275
[6,] 0.1889680 -0.2066181 -0.48198299 0.4082483 -0.44213175 0.5719662
Distances computed from Google
maps – road distances are not
straight – so ordination will have 3 or
more dimensions but should be
dominated by 2-dimensions
1.931479e+05 = 19314.97
4.733022e+04 = 4733.022
3.820062e+03 = 382.0062
4.820322e-11
-4.952561e+02
-6.316731e+03
Eigenvalues
Can be ignored
Eigen vectors

65. Ordination Plot: First Two Eigen Vectors

66. Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Angouleme 4.2 4.9 7.9 10.4 13.6 17.0 18.7 18.4 16.1 11.7 7.6 4.9
Angers 4.6 5.4 8.9 11.3 14.5 17.2 19.5 19.4 16.9 12.5 8.1 5.3
Besancon 1.1 2.2 6.4 9.7 13.6 16.9 18.7 18.3 15.5 10.4 5.7 2.0
Biarritz 7.6 8.0 10.8 12.0 14.7 17.8 19.7 19.9 18.5 14.8 10.9 8.2
Bordeaux 5.6 6.6 10.3 12.8 15.8 19.3 20.9 21.0 18.6 13.8 9.1 6.2
Brest 6.1 5.8 7.8 9.2 11.6 14.4 15.6 16.0 14.7 12.0 9.0 7.0
Cler-Ferr 2.6 3.7 7.5 10.3 13.8 17.3 19.4 19.1 16.2 11.2 6.6 3.6
Dijon 1.3 2.6 6.9 10.4 14.3 17.7 19.6 19.0 15.9 10.5 5.7 2.1
Grenoble 1.5 3.2 7.7 10.6 14.5 17.8 20.1 19.5 16.7 11.4 6.5 2.3
Lille 2.4 2.9 6.0 8.9 12.4 15.3 17.1 17.1 14.7 10.4 6.1 3.5
Limoges 3.1 3.9 7.4 9.9 13.3 16.8 18.4 17.8 15.3 10.7 6.7 3.8
Lyon 2.1 3.3 7.7 10.9 14.9 18.5 20.7 20.1 16.9 11.4 6.7 3.1
Marseille 5.5 6.6 10.0 13.0 16.8 20.8 23.3 22.8 19.9 15.0 10.2 6.9
Montpellier 5.6 6.7 9.9 12.8 16.2 20.1 22.7 22.3 19.3 14.6 10.0 6.5
Nancy 0.8 1.6 5.5 9.2 13.3 16.5 18.3 17.7 14.7 9.4 5.2 1.8
Nantes 5.0 5 3 8.4 10.8 13.9 17.2 18.8 18.6 16.4 12.2 8.2 5.5
Nice 7.5 8.5 10.8 13.3 16.7 20.1 22.7 22.5 20.3 16.0 11.5 8.2
Nimes 5.7 6.8 10.1 13.0 16.6 20.8 23.6 22.9 19.7 14.6 9.8 6.5
Orleans 2.7 3.6 6.9 9.8 13.4 16.6 18.4 18.2 15.6 10.9 6.6 3.6
Paris 3.4 4.1 7.6 10.7 14.3 17.5 19.1 18.7 16.0 11.4 7.1 4.3
Perpignan 7.5 8.4 11.3 13.9 17.1 21.1 23.8 23.3 20.5 15.9 11.5 8.6
Reims 1.9 2.8 6.2 9.4 13.3 16.4 18.3 17.9 15.1 10.3 6.1 3.0
Rennes 4.8 5.3 7.9 10.1 13.1 16.2 17.9 17.8 15.7 11.6 7.8 5.4
Rouen 3.4 3.9 6.8 9.5 12.9 15.7 17.6 17.2 15.0 11.0 6.8 4.3
St-Quent 2.0 2.9 6.3 9.2 12.7 15.6 17.4 17.4 15.0 10.5 6.1 3.1
Strasbourg 0.4 1.5 5.6 9.8 14.0 17.2 19.0 18.3 15.1 9.5 4.9 1.3
Toulon 8.6 9.1 11.2 13.4 16.6 20.2 22.6 22.4 20.5 16.5 12.6 9.7
Toulouse 4.7 5.6 9.2 11.6 14.9 18.7 20.9 20.9 18.3 13.3 8.6 5.5
Tours 3.5 4.4 7.7 10.6 13.9 17.4 19.1 18.7 16.2 11.7 7.2 4.3
Vichy 2.4 3.4 7.1 9.9 13.6 17.1 19.3 18.8 16.0 11.0 6.6 3.4
Monthly Average Temperatures: 30 French Cities

68. Eigen values
1113.8
170.4
4.6
2.7
1.2
0.6
0.5
0.3
0.2
0.1
0.1
0
In practice only 2 dimensions in these data
30 in total

69. Mediterranean
Coast
Brittany Coast
Southwest
South and West of Paris
North and East of Paris – Belgian Border
South East of Paris

70. North and East of Paris –
Belgian Border
Brittany Coast
Mediterranean Coast
South East of Paris
South and West of Paris

71. Correspondence Analysis
Many people stretching back to 1933

72. 11 12 13
21 22 23
31 32 33
41 42 43
O O O
O O O
O O O
O O O
 
 
 
 
 
 
 
11 12 13
21 22 23
31 32 33
41 42 43
E E E
E E E
E E E
E E E
 
 
 
 
 
 
Matrix of
observed
counts
Matrix of
expectations
ij ij ijij
j i ji
ij ij
i jij ij ij
i j i j i j
O O OO
E O
O O O
    
   
      
   
   
  

  
Under hypothesis that
row and column are
independent
2
2 2
( 1) ( 1) 3 2
( )ij ij
n p
i j ij
O E
E
    

  and
follows a chi-squared
distribution

73. 10 50 90
20 60 100
30 70 110
40 80 120
 
 
 
 
 
 
19.23 50.00 80.77
23.08 60.00 96.92
26.92 70.00 113.08
30.77 80.00 129.23
X-squared = 9.8576,
df = 3*2=6, p-value = 0.1308
non-significant
rows and columns independent
10 50 120
20 60 110
30 70 100
40 80 90
 
 
 
 
 
 
12.82 33.33 53.85
14.10 36.67 59.23
15.38 40.00 64.62
16.67 43.33 70.00
X-squared = 36.7282,
df = 3*26, p-value = 1.989e-06
significant result
Rows, columns not independent
ijE
ijO

74. Marsh Lotus Open
Swamp Swamp Water
Purple swamphen 798 78 25
Yellow-vented bulbul 690 101 129
Pink-necked green pigeon 614 150 90
Peaceful dove 462 101 84
Spotted dove 386 56 67
Pacific swallow 208 39 85
White-breasted waterhen 200 38 25
Baya weaver 173 7 52
Common myna 166 17 51
Purple heron 164 52 22
Yellow bittern 162 42 11
Jungle myna 154 15 117
White-throated kingfisher 128 51 42
Scaly-breasted munia 125 36 49
Relative abundance of bird species recorded
at three habitats of Paya IndahWetland
Reserve, Peninsular Malaysia.
Chi-squared = 505.9142,
df = 13*2=26, p-value < 2.2e-16
International Journal of Zoology
Volume 2011, Article ID 758573, 17
pages doi:10.1155/2011/758573
Bird Species Abundance and Their
Correlationship with Microclimate
and Habitat Variables at Natural
Wetland Reserve, Peninsular
Malaysia

75. 2
2
( 1) ( 1)
( )
Earlier we saw that ij ij
n p
i j ij
O E
E
   

 
( )1
Now we start with ij ij
ij
ij ij
i j
O E
O E

 
 
      
  

r×cQ
:Apply SVD to r×c r×c r×r r×c c×cQ Q V W U
orthonormal orthonormal
diagonal
c×c
We know from an earlier discussion that
are the eigen vectors of
are the eigen vectors of
Diagonal elements of W are square roots of eigenvalues, ii iw 


c×c c×r r×c
r×r r×c c×r
U Q Q
V Q Q
We will also need : [ ] ij
ij
ij
i j
O
p
O
 
 
   
  

r×cP

76. Matrices and can be used to plot the positions
of the row and column vectors in two seperate scatter diagrams
c×c r×rU V
Eigen Values (1) 2.506575e-01 (2) 1.436226e-01 (3) 2.032566e-17

77. For joint plots a number of different scalings have been proposed :
For example:
Where is a diagonal matrix in which the diagonals are the
reciprocals of the square roots of the
 -1/2
c×c c×c c×c
-1/2
c×c
X D U
D
column totals of
And
Where is a diagonal matrix in which the diagonals are the
reciprocals of row totals of
And
Finally, plot column1of against c
r r


r×c
-1
r×c r×r r×c c×c
-1
×
r×c
c×c c×c c×c
r×c
P
F D P X
D
P
G X W
F olumn2 of :on same graph
plot column1of against column 2 of
r×c
c×c c×c
F
G G

80. H+ H- A+ A- "+" "-" GD
1 Manchester City 55 12 38 17 93 29 64
2 Manchester United 52 19 37 14 89 33 56
3 Arsenal 39 17 35 32 74 49 25
4 Tottenham Hotspur 39 17 27 24 66 41 25
5 Newcastle United 29 17 27 34 56 51 5
6 Chelsea 41 24 24 22 65 46 19
7 Everton 28 15 22 25 50 40 10
8 Liverpool 24 16 23 24 47 40 7
9 Fulham 36 26 12 25 48 51 -3
10 West Bromwich Albion 21 22 24 30 45 52 -7
11 Swansea City 27 18 17 33 44 51 -7
12 Norwich City 28 30 24 36 52 66 -14
13 Sunderland 26 17 19 29 45 46 -1
14 Stoke City 25 20 11 33 36 53 -17
15 Wigan Athletic 22 27 20 35 42 62 -20
16 Aston Villa 20 25 17 28 37 53 -16
17 Queens Park Rangers 24 25 19 41 43 66 -23
18 Bolton Wanderers 23 39 23 38 46 77 -31
19 Blackburn Rovers 26 33 22 45 48 78 -30
20
Wolverhampton
Wndrs
19 43 21 39
40 82 -42
Premier League Final Table 2011-2012
Data used

81. Principal inertias (eigenvalues):
1 2 3
Value 0.053538 0.008691 0.00678
Percentage 77.58% 12.59% 9.82%

82. Redundancy Analysis and Canonical
Correspondence Analysis
In this context canonical means that we have two matrices or, alternatively,
two sets of descriptors for one set of objects
Rao, 1964 Ter Braak, 1986

83. 1 2 3 4 5 6 7
1 3 8
2 0 9
3 13 4
4 5 5
8 19
23
77 7 3
SpeciesSites
1 2 3 4 5 6 7
1 45.8 78.6
2 32.8 98.5
3 56.1 45.0
4
77 78.3
Counts
Environmental
Sites
e.g. Soil, Climate, etc.
Real Numbers
Typical Example of Ecology Data

84. Indirect Comparison: The matrix of explanatory variables, , does not
intervene in the in the calculation that produces the ordination of .
Correlation or regression of the ordination vectors on are carried
out first and the ordination is carried out on a modified .
In a direct comparison the matrix X intervenes in the calculation , forcing
the ordination vectors, , to be maximally related to combinations of the
columns of .
In mathematics more generally, a canonical form is the simplest and most
comprehensive form to which certain functions , relations, or expressions
can be reduced without loss of generality. For example, the canonical form
of a covariance matrix is its matrix of eigenvalues
n×pY
n×pX
n×pXn×pY
n×pX
n×pY
n×pY

85. n×pY
n×pY
n×mXn×1y
n×mX
Simple ordination
Principal Components
Correspondence Analysis
Ordination of under constrained
Multiple regression
n×1y n×pX
Ordination of under constrained
Redundancy Analysis
Canonical Correspondence Analysis
n×pXn×pY

86. 1 1 1 1 1 1
1 1
1 1 1 1 1 1
1 1 1
, , , ,
, , , ,
, , , ,
, , , ,
. . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . .
. . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . .
p m
p p p p p m
p m
p m m m m
y y y y y x y x
y y y y y x y x
x y x y x x x x
x y x y x x x x
S S S S
S S S S
S S S S
S S S S








 






Y+XS







   
    
   




 




YY YX YY YX
XY XX YX XX
S S S S
S S S S
Variance covariance matrix of
n×pY
Variance covariance matrix of n×pX
Covariances amongst descriptors of
X and Y
Covariances amongst descriptors of
X and Y

87.  
 
In principal component analysis the eigen analysis equation was :
In redundancy analysis it is
0
:
0
k
k


 
 
YY k
-1 T
YX XX YX k
S I u
S S S I u

88. An Aside on Multiple Linear Regression
 2
0 1 1 2 2 .... ,y b b x b x e e N      
1 0 1 11 2 12
2 0 1 21 2 22
3 0 1 31 2 32
0 1 1 2 2
.......
.......
.......
.
.
.......n n n
y b b x b x
y b b x b x
y b b x b x
y b b x b x
   
   
   
   
1 11 12
2 21 22 0
3 31 32 1
3
1 2
1 . .
1 . .
1 . .
. . . . . .
. . . . . . .
. . . . . . .
1 . .n n n
y x x
y x x b
y x x b
b
y x x
   
   
    
    
    
    
    
     
    
   
   
ˆ
ˆ
ˆ
ˆ
ˆ
 
   
 
 
-1 -1
-1
-1
y = Xb
X y = (X X)b
(X X) X y = (X X) (X X)b
(X X) X y = Ib
(X X) X y = bLeast squares solution
Coefficients to be
estimated Data

89. The Algebra of Redundancy Analysis
Centre both response matrix and matrix of independent variables
by subtracting the column means from the column values / elements
For each column in compute , giving
Fo
.
ˆ ˆ
r
    -1 -1
Y X
Y b (X X) X y B (X X) X Y
ˆ ˆ
ˆ ˆ ˆˆ ,
ˆ ˆ
each column of compute fitted values giving =
[1/ (1 )]
= [1/ (1 )]
[1/ (1 )]
= [1/ (1 )
=
]
n
n
n
n

 

  


   

 
  

-1
XX
Y Y
-1 -1
-1
-1
YX
Y y Xb Y XB X(X X) X Y
S X X
S Y Y
Y X(X X) X X(X X) X Y
Y X(X X) X Y
S S
   ˆ ˆSo, perform redundancy analysis by solving : 0k k 

  
XX YX
-1
k YX XX YX kY Y
S
S I u S S S I u

90. Burglary Violence Robbery
Pop
Density
Bath 3 5.7 0.4 5
Birmingham 7.2 11.6 4.1 39
Brighton 4.5 7.6 1.5 31
Bristol 9.8 10.4 2.9 40
Cambridge 7.6 7.6 1.6 31
Canterbury 2.7 6.3 0.5 5
Cardiff 6 8.6 1.1 24
Coventry 7.4 11.2 2.2 32
Lancaster 2.5 7.7 0.4 2
Leeds 12 7.5 1.7 14
Leicester 8.6 11.7 2.6 42
Liverpool 9.1 7.7 2.3 40
Manchester 13.2 11.5 5.1 43
Newcastle 5.2 8.6 0.9 26
Nottingham 11.7 15.1 4.3 41
Oxford 5.2 8.3 1.7 34
Sheffield 6.9 7.5 1.2 15
Southampton 4.9 14.6 1.8 48
Swansea 4.7 7.3 0.4 6
York 4.4 6.7 0.5 7
Bournemouth 5.2 12.9 0.9 36
Plymouth 4 10.7 0.7 32
Norwich 3.9 8.9 1.1 37
Lincoln 6 9.8 0.9 25
Objects
Source: http://www.thecompleteuniversityguide.co.uk/preparing-to-go/staying-safe-and-secure/how-safe-is-your-city/
Descriptors
http://www.guardian.co.uk/news/datablog/interactive/2011/jun/30/uk-population-mapped

91. Population density /ha Population density /ha
Population density /ha
Burglary Violence
Robbery

92. Eigen values (1) 6.612048e+00 (2) 2.664535e-15 (3) -1.110223e-16

93. 1 1 1 2 1 3
2 1 2 2 2 3
3 1 3 2 3 3
1 2 3
ˆ ˆ ˆ
ˆ ˆ ˆ
ˆ ˆ ˆ
. . .
. . .
. . .
ˆ ˆ ˆ
n n n
x b x b x b
x b x b x b
x b x b x b
x b x b x b
 
 
 
 
 
 
 
 
 
 
 
 
11 12 13
21 22 23
31 32 33
1 2 3
. . .
. . .
. . .
n n n
y y y
y y y
y y y
y y y
 
 
 
 
 
 
 
 
 
 
 
 
1 1 1 2 1 311 12 13 1
21 22 23 2 2 1 2 2 2 3
31 32 33 3 3 1 3 2 3 3
1 2 3
1 2 3
1 2 3
ˆ ˆ ˆ
ˆ ˆ ˆ
ˆ ˆ ˆ
ˆ ˆ ˆ. . . . . .
. . . . . .
. . . . . .
ˆ ˆ ˆn n n n
n n n
x b x b x by y y x
y y y x x b x b x b
y y y x x b x b x b
b b b
y y y x x b x b x b
                                                        

Regression model fitted
Matrix used in PCA, Rank = 3 Matrix used in Redundancy Analysis, Rank = 1

94. Eigen Values: (1) 6.706016e+00 (2) 7.439234e-02 (3) 6.071489e-18
Second attempt: fitting a quadratic instead of a straight line

95. 11 12 13
1 2 321 22 23
11 21 3131 32 33
12 22 3241 42 43
11 21 31
12 22 32
1 1 0 1 0
ˆ ˆ ˆ1 1 0 0 1
ˆ ˆ ˆ1 0 1 1 0
ˆ ˆ ˆ1 0 1 0 1
ˆ ˆ ˆ. . . . . . . .
ˆ ˆ ˆ. . . . . . . .
. . . . . . . .
ˆ
y y y
y y y
y y y
y y y
  
  
  
  
  
   
    
    
    
        
    
    
    
   
   

1 11 11 2 21 21 3 31 31
1 11 12 2 21 22 3 31 32
1 12 11 2 22 21 3 32 31
1 12 12 2 22 22 3 32 32
ˆ ˆ ˆˆ ˆ ˆˆ ˆ
ˆ ˆ ˆˆ ˆ ˆˆ ˆ ˆ
ˆ ˆ ˆˆ ˆ ˆˆ ˆ ˆ
ˆ ˆ ˆˆ ˆ ˆˆ ˆ ˆ
. . .
. . .
. . .
        
        
        
        
      
 
      
 
      
 
      
 
 
 
 
 
1st replicate of a 2
by 2 factorial
Used in
redundancy
analysis

96. R Software
http://www.r-project.org/index.html