Mathematical Foundations for Machine Learning and Data Mining

1. Dr G Madhava Rao
Professor
Department of Mathematics
Malla Reddy University, Hyderabad
Telangana, India.
rao.gmr.madhav@gmail.com
Mobile Number: 9515565088

2. Over view of presentation
• Mathematical Foundation
• Linear Algebra
• Probability and Statistics
• Calculus
• Optimization

3. Mathematical Foundation
• Google Page Ranking : Googol = 1.0 × 10100
Linked Matrix A =
0 0 1 1
1 0 0 0
1
1
1
1
0 1
0 0

4. Mathematical Foundation
Google Page Ranking:

5. Mathematical Foundation
Computed Tomography:

6. Mathematical Foundation
Computed Tomography:

7. Mathematical Foundation

8. Mathematical Foundation

9. Mathematical Foundation
Finding best flat:
Apartment-1 Apartment-2 Apartment-3 Apartment-4 Apartment-5 Apartment-6 Apartment-7 Apartment-8 Apartment-9 Apartment-10
No.of Bed rooms 2 3 4 5 3 2 4 1 2 3
No. of halls 2 1 1 2 2 3 1 2 2 1
Ventilation Excellent Average Average Poor Better Above Average Poor Excellent Excellent Average
Vastu Yes Partially yes Yes No No Yes Partially yes No Yes Partially yes
Distance from near by
Metro station
2 Km 2.5 KM 3 KM 3.5 KM 2.5 KM 4 KM 5 KM 2 Km 2.5 KM 3 KM
Super market 5 KM 2 Km 2.5 KM 3 KM 2 Km 2.5 KM 3 KM 3.5 KM 2.5 KM 4 KM

10. Mathematical Foundation
Many Machine Models require the data to be present in numerical
format. Machine Learning Model involves performing various
mathematical operations under the hood. Like Humans need oxygen,
mathematics needs numbers.

11. Mathematical Foundation
Pattern Recognition: A biometric is a unique, measurable
characteristic of a human being that can be used to automatically
recognize an individual or verify an individual’s identity.

12. Linear Algebra
• Vectors in Rn
• Vector Spaces
• Subspaces of Vector Spaces
• Spanning Sets and Linear Independence
• Basis and Dimension
An ordered n-tuple : a sequence of n real numbers

13. Linear Algebra
 Rn
-space : The set of all ordered n-tuples
R1
-space = set of all real numbers
(R1
-space can be represented geometrically by the x-axis)
n = 1
n = 2 R2
-space = set of all ordered pair of real numbers )
,
( 2
1 x
x
(R2
-space can be represented geometrically by the xy-plane)
n = 3 R3
-space = set of all ordered triple of real numbers )
,
,
( 3
2
1 x
x
x
(R
3
-space can be represented geometrically by the xyz-space)
n = 4 R4
-space = set of all ordered quadruple of real numbers )
,
,
,
( 4
3
2
1 x
x
x
x

14. Linear Algebra

15. Linear Algebra

Vector addition (the sum of u and v):
Scalar multiplication (the scalar multiple of u by c):
   
1 2 1 2
, , , , , , ,
n n
u u u v v v
 
u v (two vectors in Rn
)
Equality:
if and only if
v
u  1 1 2 2
, , , n n
u v u v u v
  
 
n
n v
u
v
u
v
u 



 ,
,
, 2
2
1
1 
v
u
 
n
cu
cu
cu
c ,
,
, 2
1 

u

16. Linear Algebra
 Note: The sum of two vectors and the scalar multiple
of a vector in Rn
are called the standard operations in
Rn
 Difference between u and v:
 Zero vector:
1 1 2 2 3 3
( 1) ( , , ,..., )
n n
u v u v u v u v
        
u v u v
)
0
...,
,
0
,
0
(

0

17. Linear Algebra

18. Linear Algebra

19. Linear Algebra

20. Linear Algebra

21. Linear Algebra

22. Linear Algebra

23. Subspaces of Vector Spaces
• Subspace :
( , , ) :
V   a vector space
:
W
W V
 

 
a nonempty subset
( , , ) :
W   The nonempty subset W is called a subspace if W is a vector space
under the operations of addition and scalar multiplication defined in
V
 Trivial subspace :
Every vector space V has at least two subspaces
(1) Zero vector space {0} is a subspace of V
(2) V is a subspace of V
※ Any subspaces other than these two are call proper (or nontrivial) subspaces
(It satisfies the ten axioms)

24. Subspaces of Vector Spaces
The intersection of two subspaces is a subspace
If and are both subspaces of a vector space ,
then the intersection of and (denoted by )
is also a subspace of
V W U
V W V W
U


25. Ex: A subspace of M2×2
Let W be the set of all 2×2 symmetric matrices, then
W is a subspace of the vector space M2×2, with the standard
operations of matrix addition and scalar multiplication
2 2
First, we knon that , the set of all 2 2 symmetric matrices,
is an nonempty subset of the vector space
W
M 

Sol:
)
( 2
1
2
1
2
1
2
1 A
A
A
A
A
A
W
A
W,
A T
T
T








, ( )T T
c R A W cA cA cA
    
2 2
is a subspace of
W M 

1 2
( )
A A W
 
( )
cA W

The definition of a symmetric matrix A is that AT = A
Second,

26. 1 0
0 1
A B W
 
  
 
 
2 2 2
is not a subspace of
W M 

 Ex: The set of singular matrices is not a subspace of M2×2
Let W be the set of singular (noninvertible) matrices of order 2. Then W is not a
subspace of M2×2 with the standard matrix operations
1 0 0 0
,
0 0 0 1
A W B W
   
   
   
   
Sol:
(W is not closed under vector addition)

27.  Ex : The set of first-quadrant vectors is not a subspace of R
2
Show that , with the standard
operations, is not a subspace of R
2
Sol:
Let (1, 1) W
 
u
2
is not a subspace of
W R

}
0
and
0
:
)
,
{( 2
1
2
1 

 x
x
x
x
W
       W






 1
,
1
1
,
1
1
1 u

(W is not closed under scalar multiplication)

28. Spanning Sets and Linear Independence
1 1 2 2 ,
k k
c c c
   
u v v v
1 2
A vector in a vector space is called a linear combination of
the vectors in if can be written in the form
k
V
, , , V
u
v v v u
Linear combination:
1 2
where , , , are real-number scalars
k
c c c

29. If S={v1, v2,…, vk} is a set of vectors in a vector space V, then the span
of S is the set of all linear combinations of the vectors in S,
 The span of a set: span(S)
 
1 1 2 2
span( )
(the set of all linear combinations of vectors in )
k k i
S c c c c R
S
     
v v v
 Definition of a spanning set of a vector space:
If every vector in a given vector space V can be written as a linear
combination of vectors in a set S, then S is called a spanning set of the
vector space V

30. 3
3
1 2 3
1 2 3
(a) The set {(1,0,0),(0,1,0),(0,0,1)} spans because any vector
( , , ) in can be written as
(1,0,0) (0,1,0) (0,0,1)
S R
u u u R
u u u


  
u
u
2
2
2
2
2
(b) The set {1, , } spans because any polynomial function
( ) in can be written as
( ) (1) ( ) ( )
S x x P
p x a bx cx P
p x a b x c x

  
  

31. 1 1 2 2
For k k
c c c
   
v v v 0
1 2
1 2
(1) If the equation has only the trivial solution ( 0)
then (or , , , ) is called
(2) If the equation has a nontrivial solution (i.e., not all zeros),
the
   
v v v linearly independent
k
k
c c c
S
1 2
1
n (or , , , ) is called (The name of
linear dependence is from the fact that in this case, there exist a
which can be represented by the linear combination of {
v v v linearly dependent
v
v
k
i
S
2 1
1
, , , ,
, } in which the coefficients are not all zero.


v v
v v
i
i k
 
1 2
, , , : a set of vectors in a vector space
k
S V
 v v v
Definitions of Linear Independence (L.I.) and
Linear Dependence (L.D.):

32. Basis and Dimension
Basis :
V: a vector space
(1)
(2)







S spans V (i.e., span(S) = V)
S is linearly independent
Spanning
Sets
Bases
Linearly
Independent
Sets
S is called a basis for V
S ={v1, v2, …, vn} V
(For any , = has a solution (det( ) 0),
see Ex 5 on Slide 4.44)
i i
V c A A
  

u v x u
(For = , there is only the trivial solution (det( ) 0),
 
 v x 0
i i
c A A
V

33. (1) The standard basis for R3:
{i, j, k} i = (1, 0, 0), j = (0, 1, 0), k = (0, 0, 1)
(2) The standard basis for R
n
:
{e1, e2, …, en} e1=(1,0,…,0), e2=(0,1,…,0),…, en=(0,0,…,1)
Ex: For R4, {(1,0,0,0), (0,1,0,0), (0,0,1,0), (0,0,0,1)}
(3) The standard basis matrix space:






























1
0
0
0
,
0
1
0
0
,
0
0
1
0
,
0
0
0
1
(4) The standard basis for Pn(x): {1, x, x2, …, xn}

34. {

35. {

36. {

37. {

38. {

39. {

40. {

41. References
• Korchevskii. E. M and Marochnik. L.S
"Magnetohydrodynamic version of movement of
blood", Biophysics, Vol. 10, 1965, pp. 411–413.

42. Any queries