Everything you need to know about confusion matrix

1. True Positiv
True Negative
False Positive
False Negativ
confusion
matrix
Almost everything about
#MLvisualguides

2. What is a confusion matrix?
Itisatooltoevaluate
performanceofsupervised
machinelearningalgorithm
whenusedforclassification
problem.

3. Consider....
YoutrainaMLmodelforadataset
thatislabelledonlyforYESorNO
foreachdatarow.
Ex.Loanworthinessofaperson
againsttheirincome,education,
creditscore,maritalstatus,number
ofchildrenandotherfeatures.

4. But remember....
Onlyforthisproblemourclasses
areYESandNO.Ingeneralclasses
canbeanytwodifferentthings
suchasPositiveorNegative,Busor
Truck,SpamorNotSpam,Dogor
Catandsoon.
NOTE:Classesdonothavetobe
necessarilyoppositetoeachother.

5. Now how do you check the performance of
your model on test data?
YES!With the help of confusion matrix.

6. Let us assume your test data contained
100 rows. Then your confusion matrix
might look something like this...
Yes Predicted No
Yes
Actual
No
45 15
10 30
n = 100

7. Ok, so what
does these
numbers mean?

8. Oh before that let us consider ....
YES to be positive label
No to be negative label

9. Yes
Actual
No
True Positives
Yes Predicted No
45 15
10 30
n = 100
When model predicts YES
and actual value is also YES
TP

10. Yes
Actual
No
Yes Predicted No
45 15
10 30
n = 100
False Positives or Type 1 error
When model predicts YES
but actual value is NO
FP

11. Yes
Actual
No
Yes Predicted No
45 15
10 30
n = 100
True Negatives
When model predicts NO
and actual value is also NO
TN

12. Yes
Actual
No
Yes Predicted No
45 15
10 30
n = 100
False Negatives or Type II error
When model predicts NO
but actual value is YES
FN

13. Yes Predicted No
Yes
Actual
No
n = total no. of predictions
FNTP
FP TN
In summary...

14. True Positive
True Negative
False Positive
False Negative
Whatwastheprediction?
Wasthepredictionright?
No more confusion...

15. Ok!
But how does this help
in evaluating model
performance?

16. Classification models can
be evaluated using ...
Accuracy
Precision
Recall
F1 Score
Area Under Curve (AUC)

17. Totalnumberofcorrect/true
predictionsamongallthepredictions.
accuracy = TP + TN
Accuracy
n

18. Measuresclassifiers'completeness.
Positivecasescorrectlypredicted
amongallactualpositivecases.
recall = TP
Recall
TP + FN

19. precision =
Precision
TP + FP
TP
Measuresclassifiers'exactness.
Positivecasescorrectlypredicted
amongallpredictedpositivecases.

20. Judgingthemodelonaccuracy,precisionorrecall
aloneisflawedsincemostoftheMLordatascience
problemsarecontextbased.
Therefore,F1Scoreisusuallyasafemetricto
evaluateamodel.ModelwithhighestF1Scorenot
onlyreturnstruepositivebutalsoincreasesthe
numberoftruepositives.
F1 score =
F1 Score
precision x recall
precision + recall
x 2

21. AUC
Before coming to area under curve, it is
important we cover the following.....
SpecificitySensitivity
ROC curve
(Sensitivity vs 1-Specificity)

22. Sensitivity
ItistheTruePositiveRateorsimply
recall.
Bestvalue=1
Worstvalue=0

23. Specificity
Numberofcorrectnegativepredictions
dividedbythetotalnumberofnegatives
=TN/(TN+FP)
Itisalsocalledtruenegativerate(TNR).
FalsePositive Rate = 1-Specificity

24. Slightly better classifier
AUC
ROC Curve
1 - Specificity (FPR)
Sensitivity(TPR)
Random Classifier (Useless)
0
1---
|
1
More AUC
Better Model

25. Pic source: glassboxmedicine.com

26. Points to remember....
Confusionmatrixisaclassifierevaluationtool.
Classesneednotbeopposites.
Oneclassislabelledpositive,otherislabelled
negative.Justlabelled,doesnotmean
anything.
Accuracy,recallandprecisiondependsonthe
contextandmaynotbetruejudgeofthemodel.
F1ScoreandAUCaremorecomprehensive
evaluationmetrics.

27. made with love
using Canva
By
RajatGupta
DataScientist@SciBiSystems