mathematics for machine learning matrices.pptx

Matrices
Introduction
National Institute of Electronics & Information Technology
Ministry of Electronics & Information Technology (MeitY), Government of India
Gorakhpur Center

Matrices - Introduction
Matrix algebra has at least two advantages:
•Reduces complicated systems of equations to simple expressions
•Adaptable to systematic method of mathematical treatment and well
suited to computers
Definition:
A matrix is a set or group of numbers arranged in a square or rectangular array
enclosed by two brackets
 
1
1  





 0
3
2
4






d
c
b
a
Gorakhpur Center

Properties:
•A specified number of rows and a specified number of columns
•Two numbers (rows x columns) describe the dimensions or size of
the matrix.
Examples:
3x3 matrix
2x4 matrix
1x2 matrix











3
3
3
5
1
4
4
2
1





 
2
3
3
3
0
1
0
1
 
1
1 
Gorakhpur Center

A matrix is denoted by a bold capital letter and the elements within the
matrix are denoted by lower case letters
e.g. matrix [A] with elements aij














mn
ij
m
m
n
ij
in
ij
a
a
a
a
a
a
a
a
a
a
a
a
2
1
2
22
21
12
11
...
...




i goes from 1 to m
j goes from 1 to n
Amxn= mAn
=
Gorakhpur Center

Matrices - Operations
ADDITION AND SUBTRACTION OF MATRICES
The sum or difference of two matrices, A and B of the
same size yields a matrix C of the same size
ij
ij
ij b
a
c 

Matrices of different sizes cannot be added or
subtracted
Gorakhpur Center

MULTIPLICATION OF MATRICES
The product of two matrices is another matrix
Two matrices A and B must be conformable for
multiplication to be possible
i.e. the number of columns of A must equal the number of
rows of B
Example.
A x B = C
(1x3) (3x1) (1x1)
Gorakhpur Center
























22
21
12
11
32
31
22
21
12
11
23
22
21
13
12
11
c
c
c
c
b
b
b
b
b
b
a
a
a
a
a
a
22
32
23
22
22
12
21
21
31
23
21
22
11
21
12
32
13
22
12
12
11
11
31
13
21
12
11
11
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
c
b
a
b
a
b
a
c
b
a
b
a
b
a
c
b
a
b
a
b
a
c
b
a
b
a
b
a
























Successive multiplication of row i of A with column j
of B – row by column multiplication
Gorakhpur Center












































)
3
7
(
)
2
2
(
)
8
4
(
)
5
7
(
)
6
2
(
)
4
4
(
)
3
3
(
)
2
2
(
)
8
1
(
)
5
3
(
)
6
2
(
)
4
1
(
3
5
2
6
8
4
7
2
4
3
2
1







57
63
21
31
Remember also:
IA = A






1
0
0
1






57
63
21
31







57
63
21
31
Gorakhpur Center

Assuming that matrices A, B and C are conformable for the operations
indicated, the following are true:
1. AI = IA = A
2. A(BC) = (AB)C = ABC - (associative law)
3. A(B+C) = AB + AC - (first distributive law)
4. (A+B)C = AC + BC - (second distributive law)
Caution!
1. AB not generally equal to BA, BA may not be conformable
2. If AB = 0, neither A nor B necessarily = 0
3. If AB = AC, B not necessarily = C
Gorakhpur Center

AB not generally equal to BA, BA may not be conformable






















































0
10
6
23
0
5
2
1
2
0
4
3
20
15
8
3
2
0
4
3
0
5
2
1
2
0
4
3
0
5
2
1
ST
TS
S
T
Gorakhpur Center

If AB = 0, neither A nor B necessarily = 0





















0
0
0
0
3
2
3
2
0
0
1
1
Gorakhpur Center

TRANSPOSE OF A MATRIX
If :








1
3
5
7
4
2
3
2 A
A












1
7
3
4
5
2
3
2
T
T
A
A
Then transpose of A, denoted AT
is:
T
ji
ij a
a  For all i and j
Gorakhpur Center

Properties of transposed matrices:
1. (A+B)T
= AT
+ BT
2. (AB)T
= BT
AT
3. (kA)T
= kAT
4. (AT
)T
= A
Gorakhpur Center

SYMMETRIC MATRICES
A Square matrix is symmetric if it is equal to
its transpose:
A = AT














d
b
b
a
A
d
b
b
a
A
T
Gorakhpur Center

INVERSE OF A MATRIX
Consider a scalar k. The inverse is the reciprocal or division
of 1 by the scalar.
Example:
k=7 the inverse of k or k-1
= 1/k = 1/7
Division of matrices is not defined since there may be AB =
AC while B = C
Instead matrix inversion is used.
The inverse of a square matrix, A, if it exists, is the unique
matrix A-1
where:
AA-1
= A-1
A = I
Gorakhpur Center

Example:


















3
2
1
1
1
2
1
3
1
2
2
A
A
A










































1
0
0
1
3
2
1
1
1
2
1
3
1
0
0
1
1
2
1
3
3
2
1
1
Because:
Gorakhpur Center

Properties of the inverse:
1
1
1
1
1
1
1
1
1
1
)
(
)
(
)
(
)
(
)
(













A
k
kA
A
A
A
A
A
B
AB
T
T
A square matrix that has an inverse is called a nonsingular matrix
A matrix that does not have an inverse is called a singular matrix
Square matrices have inverses except when the determinant is zero
When the determinant of a matrix is zero the matrix is singular
Gorakhpur Center

DETERMINANT OF A MATRIX
To compute the inverse of a matrix, the determinant is
required
Each square matrix A has a unit scalar value called the
determinant of A, denoted by det A or |A|
5
6
2
1
5
6
2
1








A
A
If
then
Gorakhpur Center

If A = [A] is a single element (1x1), then the determinant is
defined as the value of the element
Then |A| =det A = a11
If A is (n x n), its determinant may be defined in terms of
order (n-1) or less.
Gorakhpur Center

MINORS
If A is an n x n matrix and one row and one column are
deleted, the resulting matrix is an (n-1) x (n-1) submatrix of
A.
The determinant of such a submatrix is called a minor of A
and is designated by mij , where i and j correspond to the
deleted
row and column, respectively.
mij is the minor of the element aij in A.
Gorakhpur Center












33
32
31
23
22
21
13
12
11
a
a
a
a
a
a
a
a
a
A
Each element in A has a minor
Delete first row and column from A .
The determinant of the remaining 2 x 2 submatrix is the minor
of a11
eg.
33
32
23
22
11
a
a
a
a
m 
Gorakhpur Center

Therefore the minor of a12 is:
And the minor for a13 is:
33
31
23
21
12
a
a
a
a
m 
32
31
22
21
13
a
a
a
a
m 
Gorakhpur Center

COFACTORS
The cofactor Cij of an element aij is defined as:
ij
j
i
ij m
C 

 )
1
(
When the sum of a row number i and column j is even, cij = mij
and when i+j is odd, cij =-mij
13
13
3
1
13
12
12
2
1
12
11
11
1
1
11
)
1
(
)
3
,
1
(
)
1
(
)
2
,
1
(
)
1
(
)
1
,
1
(
m
m
j
i
c
m
m
j
i
c
m
m
j
i
c





















Gorakhpur Center

Therefore the 2 x 2 matrix :







22
21
12
11
a
a
a
a
A
Has cofactors :
22
22
11
11 a
a
m
c 


And:
21
21
12
12 a
a
m
c 





And the determinant of A is:
21
12
22
11
12
12
11
11 a
a
a
a
c
a
c
a
A 



Gorakhpur Center

Example 1:







2
1
1
3
A
5
)
1
)(
1
(
)
2
)(
3
( 


A
Gorakhpur Center

For a 3 x 3 matrix:











33
32
31
23
22
21
13
12
11
a
a
a
a
a
a
a
a
a
A
The cofactors of the first row are:
31
22
32
21
32
31
22
21
13
31
23
33
21
33
31
23
21
12
32
23
33
22
33
32
23
22
11
)
(
a
a
a
a
a
a
a
a
c
a
a
a
a
a
a
a
a
c
a
a
a
a
a
a
a
a
c











Gorakhpur Center

The determinant of a matrix A is:
21
12
22
11
12
12
11
11 a
a
a
a
c
a
c
a
A 



Which by substituting for the cofactors in this case is:
)
(
)
(
)
( 31
22
32
21
13
31
23
33
21
12
32
23
33
22
11 a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
A 





Gorakhpur Center

ADJOINT MATRICES
A cofactor matrix C of a matrix A is the square matrix of the
same order as A in which each element aij is replaced by its
cofactor cij .
Example:








4
3
2
1
A








1
2
3
4
C
If
The cofactor C of A is
Gorakhpur Center

The adjoint matrix of A, denoted by adj A, is the transpose of its
cofactor matrix T
C
adjA 
It can be shown that:
A(adj A) = (adjA) A = |A| I
Example:





 














1
3
2
4
10
)
3
)(
2
(
)
4
)(
1
(
4
3
2
1
T
C
adjA
A
A
Gorakhpur Center

USING THE ADJOINT MATRIX IN MATRIX
INVERSION
A
adjA
A 
 1
Since
AA-1
= A-1
A = I
and
A(adj A) = (adjA) A = |A| I
then
Gorakhpur Center

Linear Equations
Gorakhpur Center

Linear Equations
Linear equations are common and important for survey problems
Matrices can be used to express these linear equations and aid in the
computation of unknown values
Example
n equations in n unknowns, the aij are numerical coefficients, the bi
are constants and the xj are unknowns
n
n
nn
n
n
n
n
n
n
b
x
a
x
a
x
a
b
x
a
x
a
x
a
b
x
a
x
a
x
a
















2
2
1
1
2
2
2
22
1
21
1
1
2
12
1
11
Gorakhpur Center

Linear Equations
The equations may be expressed in the form
AX = B
where
,
, 2
1
1
1
2
22
21
1
12
11


























n
nn
n
n
n
n
x
x
x
X
a
a
a
a
a
a
a
a
a
A







and













n
b
b
b
B

2
1
n x n n x 1 n x 1
Number of unknowns = number of equations = n
Gorakhpur Center

Linear Equations
If the determinant is nonzero, the equation can be solved to produce n
numerical values for x that satisfy all the simultaneous equations
To solve, premultiply both sides of the equation by A-1
which exists because |
A| = 0
A-1
AX = A-1
B
Now since
A-1
A = I
We get
X = A-1
B
So if the inverse of the coefficient matrix is found, the unknowns, X would
be determined
Gorakhpur Center

Linear Equations
Example
3
2
1
2
2
3
3
2
1
2
1
3
2
1








x
x
x
x
x
x
x
x
The equations can be expressed as

































3
1
2
1
2
1
0
1
2
1
1
3
3
2
1
x
x
x
Gorakhpur Center

Linear Equations
When A-1
is computed the equation becomes






































 
7
3
2
3
1
2
5
.
2
5
.
3
5
.
1
0
.
1
0
.
2
0
.
1
5
.
0
5
.
0
5
.
0
1
B
A
X
Therefore
7
,
3
,
2
3
2
1





x
x
x
Gorakhpur Center

Linear Equations
The values for the unknowns should be checked by
substitution back into the initial equations
3
2
1
2
2
3
3
2
1
2
1
3
2
1








x
x
x
x
x
x
x
x
3
)
7
(
)
3
(
2
)
2
(
1
)
3
(
)
2
(
2
2
)
7
(
)
3
(
)
2
(
3
















7
,
3
,
2
3
2
1





x
x
x
Gorakhpur Center

Introduction to Statistics and
Probability
Gorakhpur Center

STATISTICS
• It is the science of collecting, organizing, analyzing and interpreting
data.
• There are two types of Statistics:
Inferential Statistics : It is about using sample data from a dataset and
making inferences and conclusions using probability theory.
Descriptive Statistics: It is used to summarize and represent the data in
an accurate way using charts, tables and graphs.
For example, you might stand in a mall and ask a sample of 100
people if they like shopping at Sears. You could make a bar chart of
yes or no answers (that would be descriptive statistics) or you could
use your research (and inferential statistics) to reason that around
75%-80% of population.
Gorakhpur Center

DESCRIPTIVE STATISTICS
The following measures are used to represent the data set :
Descriptiv
e Statistics
Measure of
Position
Measure
of
Spread
Measure
of Shape
Gorakhpur Center

MEASURE OF POSITION
• Also known as measure of Central Tendency.
• A measure of central tendency is a single value that attempts to describe
a set of data by identifying the central position within that set of data.
• There are three measures of central tendencies: Mean, Median and
Mode.
Gorakhpur Center

Median: It is a point that divides the data into two equal halves while being less
susceptible to outliers compare to mean.
For ungrouped data: middle data point of an ordered data set.
For grouped data :
Where,
• L = lower limit of median class
• n = number of observations
• cf = cumulative frequency of class preceding the median class
• f = frequency of median class
• w = class size
Mean: It is a point where mass of distribution of data balances.
Gorakhpur Center

Mode: It refers to the data item that occurs most frequently in a
given data set.
Mode for ungrouped data: Most frequent observation in the data.
Mode for grouped data:
Gorakhpur Center

Example for ungrouped data:
Question:
Gorakhpur Center

MEASURE OF DISPERSION
• It refers to how the data deviates from the position measure i.e.
gives an indication of the amount of variation in the process.
• Dispersion of the data set can be described by:
Range: It is the difference between highest and the lowest values.
Standard Deviation: It is the measurement of average distance
between each quantity and mean i.e. how data is spread out from
mean. Higher the standard deviation, more is the data spread from
mean.
Gorakhpur Center

In normal distribution, when data is unimodal, z-scores is used to
calculate the probability of a score occurring within a standard normal
distribution and helps to compare two scores from different samples.
Below, calculating the probability of randomly obtaining a score from
the distribution.
68% probability for -1 and +1 standard deviation from mean.
Similarly, 95% for -1.96 and +1.96 standard deviation.
Gorakhpur Center

MEASURE OF SHAPE
• It is used to characterize the location and variability of data set.
• Two common statistics that measure the shape of the data are:
Skewness and Kurtosis
Skewness : It is the horizontal displacement of the normal curve
about the mean position. Skewness for a normal distribution is zero.
Gorakhpur Center

CORRELATION ANALYSIS
It is a statistical technique that can show whether and how strongly
pairs of variables are related.
If correlation coefficient (r) is
• Positive, then both variables are directly proportional.
• Zero, there is no relation between them.
• Negative, then both variables are inversely proportional
Gorakhpur Center

REGRESSION ANALYSIS
The statistical technique of estimating the unknown value of one
variable(i.e. dependent variable) from the known value of other variable
(i.e. independent variable) is called regression analysis.
The regression equation of X on Y is: X = a +bY
The regression equation of Y on X is: Y = a +bX
Dependent Variable: The single variable which we wish to estimate/predict
by regression model.
Independent Variable: The known variable(s) used to predict/estimate the
value of dependent variable.
X is dependent, Y
is independent
Y is depedent, X
is independent
Gorakhpur Center

PROBABILITY
Probability is a numerical description of how likely an event is to
occur or how likely it is that a proposition is true.
Tossing a coin: When a coin is tossed, there are two
possible outcomes: Heads (H) or Tails (T).Thus,
probability of the coin landing H is ½ and the
probability of the coin landing T is ½.
Rolling a die: When a single die is thrown, there are six
possible outcomes: 1, 2, 3, 4, 5, 6.The probability of
any one of them is 1/6.
Some examples are:
Gorakhpur Center

TERMINOLOGY
Experiment: A process by which an outcome is obtained.
Sample space: The set S of all possible outcomes of an experiment. i.e. the
sample space for a dice roll is {1, 2, 3, 4, 5, 6}
Event: Any subset E of the sample space i.e.
Let,
E1 = An even number is rolled.
E2 = A number less than three is rolled.
Outcome: Result of a single trial.
Equally likely outcomes: Two outcomes of a random experiment are said to be
equally likely, if upon performing the experiment a (very) large number of times,
the relative occurrences of the two outcomes turn out to be equal.
Trial: Performing a random experiment.
Gorakhpur Center

EVENTS
Simple Events : If the event E has only single element of a sample
space, it is called as a simple event. Eg: if S = {56 , 78 , 96 , 54 , 89}
and E = {78} then E is a simple event.
Compound Events: Any event consists of more than one element
of the sample space. Eg: if S = {56 ,78 ,96 ,54 ,89}, E1 = {56 ,54 },
E2 = {78 ,56 ,89 } then, E1 and E2 represent two compound events.
Independent Events and Dependent Events:
If the occurrence of any event is completely unaffected by the
occurrence of any other event, such events are Independent Events.
Probability of two independent event is given by,
Gorakhpur Center

The events which are affected by other events are Dependent Events.
Probability of dependent event is given by,
Exhaustive Events: A set of events is called exhaustive if all the events together
consume the entire sample space. Eg: A and B are sets of mutually exclusive events,
Mutually Exclusive Events: If the occurrence of one event
excludes the occurrence of another event i.e. no two events can occur
simultaneously.
Where,
S = sample space
Gorakhpur Center

Addition Theorem
Theorem 1: If A and B are two mutually exclusive events, then
P(A B) = P(A) + P(B)
∪
Where,
n = Total number of exhaustive cases
n1= Number of cases favorable to A.
n2= Number of cases favorable to B.
Theorem2: If A and B are two events that are not mutually exclusive, then
P(A B) = P( A ) + P( B ) - P ( A B )
∪ ∩
Where,
P (A B) = Probability of events favorable to both A and B
∩
Gorakhpur Center

Multiplication Theorem
If A and B are two independent events, then the probability that both
will occur is equal to the product of their individual probabilities.
Example:
The probability of appointing a lecturer who is B.Com, MBA, and PhD,
with probabilities 1/20, 1/25 and 1/40 is given by:
Using multiplicative theorem for independent events,
Gorakhpur Center

Conditional Probability
The conditional probability of an event B is the probability that the event
will occur given the knowledge that an event A has already occurred. It is
representated as P( B | A).
P(A | B) = P(A B) P(B)
∩ ⁄
Where A and B are two dependent events.
Gorakhpur Center

Total Probability Theorem
Given n mutually exclusive events A1, A2, … Ak such that their
probabilities sum is unity and their union is the event space E, then
Ai Aj = NULL, for all i not equal to j
∩
A1 U A2 U ... U Ak = E
Then Total Probability Theorem or Law of Total Probability is:
where B is an arbitrary event, and P(B/Ai) is the conditional probability of
B assuming A already occurred.
Gorakhpur Center

Proof of Total Probability Theorem :
As intersection and Union are Distributive. Therefore,
B = (B A1) U (B A2) U
∩ ∩ ….... U (B AK)
∩
Since all these partitions are disjoint. So, we have,
P (B A1) = P (B A1) U P(B A2) U ….... U P (B AK)
∩ ∩ ∩ ∩
This is, addition theorem of probabilities for union of disjoint events.
Using Conditional Probability:
P (B / A) = P(B A) / P(A)
∩
We know,
A1 U A2 U A3 U ….. U AK =
E(Total)
Then, for any event B, we have,
B = B E
∩
B = B (A1 U A2 U A3 U
∩ … U
AK)
Gorakhpur Center

As the events are said to be independent here,
P(A B) = P(A) * P(B)
∩
where P(B|A) is the conditional probability which gives the probability
of occurrence of event B when event A has already occurred. Hence,
P( B Ai ) = P( B | Ai ).P( Ai ) ; i = 1,2,3
∩ . . . k
Applying this rule above:
This is Law of Total Probability.
It is used for evaluation of denominator in Bayes’ Theorem.
Gorakhpur Center

BAYES’ THEOREM
It is a mathematical formula for
determining conditional probability.
In above formula, the posterior probability is equal to the conditional
probability of event B given A multiplied by the prior probability of A,
all divided by the prior probability of B.
Science itself is a special case of Bayes’ theorem
because we are revising a prior
probability( hypothesis) in the light of observation
or experience that confirms our
hypothesis( experimental evidence) to develop a
posterior probability( conclusion)
Gorakhpur Center

Example Bayes’ Theorem:
Gorakhpur Center

Gorakhpur Center

BINOMIAL DISTRIBUTION OF PROBABILITY
A binomial distribution is the probability of a SUCCESS or FAILURE
outcome in an experiment or survey that is repeated multiple times.
Criteria for binomial distribution:
• The number of observations or trials is fixed
• Each observation or trial is independent.
• The probability of success (tails, heads, fail or pass) is exactly the same from one
trial to another.
Gorakhpur Center

Example:
Q. A coin is tossed 10 times. What is the probability of getting exactly 6 heads?
The number of trials (n) is 10
x = 6
The odds of success (p) (tossing a heads) is 0.5
Odds of failure (q) = 1- p
P(x=6) = 10C6 * 0.5^6 * 0.5^4
= 210 * 0.015625 * 0.0625
= 0.205078125
Gorakhpur Center

POISSON DISTRIBUTION
OF PROBABILITY
The Poisson distribution is the discrete probability distribution of the
number of events occurring in a given time period, given the average
number of times the event occurs over that time period.
When the number of trials in a binomial distribution is very large, and the
probability of success is very small, then np ~ npq (as q ~ 1), therefore it is
possible to change the distribution to a Poisson distribution.
Where,
x = 0,1,2,3….
= mean number of occurrences in the interval
ƛ
e = Euler’s constant
Gorakhpur Center

Example:
Q. Twenty sheets of aluminum alloy were examined for surface flaws.
The frequency of the number of sheets with a given number of flaws
per sheet was as follows
The total number of flaws = (0x4)+(1x3)+(2x5)+(3x2+(4x4)+(5x1)+(6x1) = 46
So the average for 20 sheets ( ) = 46/20 = 2.3
ℳ
Probability = P(X>=3)
= 1 – (P(x0) +P(x1)+P(x2))
Using Poisson distribution formula
= 0.40396
What is the probability of finding a sheet chosen at random which contains 3 or
more surface flaws?
Gorakhpur Center

Continuous Distribution
A probability distribution in which the
random variable X can take on any value
(is continuous) i.e. the probability of X
taking on any one specific value is zero.
Normal Distribution: A continuous random variable x is said to
follow normal distribution, if its probability density function is define
as follow,
Where, ( )= means and ( )= standard deviations.
μ σ
Gorakhpur Center

Chi- Squared Test:
The Chi-Square statistic is commonly used for testing relationships
between categorical variables.
The null hypothesis of the Chi-Square test is that no relationship exists
on the categorical variables in the population. They are independent.
The calculation of the Chi-Square statistic is quite straight-forward and
intuitive.
Where,
fo = The observed frequency ,
fe = The expected frequency if NO relationship existed
between the variables,
χ2
= Degree of freedom.
Gorakhpur Center

mathematics for machine learning matrices.pptx

More Related Content

Similar to mathematics for machine learning matrices.pptx

More from mrsam3062

Recently uploaded

mathematics for machine learning matrices.pptx