Machine Learning Classification models: Logistic Regression K-Nearest Neighbors (K-NN) Support Vector Machine Kernel SVM Naïve Bayes Decision Tree Classification Random Forest Classification

1. 3
CLASSIFICATION MODEL

2. • Following are the Machine Learning Classification models:
1. Logistic Regression
2. K-Nearest Neighbors (K-NN)
3. Support Vector Machine
4. Kernel SVM
5. Naïve Bayes
6. Decision Tree Classification
7. Random Forest Classification

4. 1. Logistic Regression

5. 1. Logistic Regression

6. 1. Logistic Regression

7. 1. Logistic Regression

8. 1. Logistic Regression

9. 1. Logistic Regression

10. 1. Logistic Regression

11. 1. Logistic Regression

12. 1. Logistic Regression

13. Logistic Regression IMP Code
from sklearn.linear_model import LogisticRegression
classifier = LogisticRegression(random_state = 0)
classifier.fit(X_train, y_train)

14. 1. Logistic Regression
# Visualising the Training set results
from matplotlib.colors import ListedColormap
X_set, y_set = X_train, y_train
X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() - 1, stop = X_set[:, 0].max() + 1, step = 0.01),
np.arange(start = X_set[:, 1].min() - 1, stop = X_set[:, 1].max() + 1, step = 0.01))
plt.contourf(X1, X2, classifier.predict(np.array([X1.ravel(), X2.ravel()]).T).reshape(X1.shape),
alpha = 0.75, cmap = ListedColormap(('red', 'green')))
plt.xlim(X1.min(), X1.max())
plt.ylim(X2.min(), X2.max())
for i, j in enumerate(np.unique(y_set)):
plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1],
c = ListedColormap(('red', 'green'))(i), label = j)
plt.title('Logistic Regression (Training set)')
plt.xlabel('Age')
plt.ylabel('Estimated Salary')
plt.legend()
plt.show()

15. 2. K-Nearest Neighbor (KNN)

16. KNN Algorithm

18. Euclidean Distance

23. KNN IMP Code
from sklearn.neighbors import KNeighborsClassifier
classifier = KNeighborsClassifier(n_neighbors = 5, metric = 'minkowski', p = 2)
classifier.fit(X_train, y_train)

24. Support Vector Machine (SVM)

25. 3. Support Vector Machine (SVM)

26. 3. Support Vector Machine (SVM)

27. 3. Support Vector Machine (SVM)

28. What is so special about SVM?

29. What is so special about SVM?

30. What is so special about SVM?

31. SVM IMP Code & Confusion Matrix
from sklearn.svm import SVC
classifier = SVC(kernel = 'linear', random_state = 0)
classifier.fit(X_train, y_train)
array([[66, 2],
[ 8, 24]], dtype=int64)

32. Output of SVM

33. 4. Kernel SVM

34. 4. Kernel SVM

36. Mapping to Higher Dimension

37. Mapping to Higher Dimension

38. Mapping to Higher Dimension

39. Mapping to Higher Dimension

40. Mapping to Higher Dimension

41. SVM Kernel Functions

42. What is the Kernel Trick?
The Kernel trick is a very interesting and powerful tool.
It is powerful because it provides a bridge from linearity to non-linearity to any
algorithm that can be expressed solely on terms of dot products between two
vectors. It comes from the fact that, if we first map our input data into a higher-
dimensional space, a linear algorithm operating in this space will behave non-
linearly in the original input space. And, we do not exactly need the exact data
points, but only their inner products to compute our decision boundary.
What it implies is that if we want to transform our existing data into a higher
dimensional data, which in many cases help us classify better, we need not
compute the exact transformation of our data, we just need the inner product of
our data in that higher dimensional space.

43. What is the Kernel Trick?

44. What is the Kernel Trick?

45. Mapping from 1D to 2D

46. Regularization

47. The Gaussian RBF Kernel

48. The Gaussian RBF Kernel

49. The Gaussian RBF Kernel

50. The Gaussian RBF Kernel

51. The Gaussian RBF Kernel

52. The Gaussian RBF Kernel

53. Types of Kernel Functions

54. Types of Kernel Functions in 3D

55. IMP Code for Kernel SVM
from sklearn.svm import SVC
classifier = SVC(kernel = 'rbf', random_state = 0)
classifier.fit(X_train, y_train)
array([[64, 4],
[ 3, 29]], dtype=int64)
93 correctly classified and 7 incorrectly classified.

56. Projection from 3D to 2D using Kernel SVM

57. Projection from 3D to 2D using Kernel SVM

58. Output of Kernel SVM

59. 4. Naïve Bayes Classification

60. Find the Defective Spanner

61. Bayes Theorem

62. Bayes Theorem

63. Bayes Theorem

64. Bayes Theorem

65. Bayes Theorem is Intuitive

66. Bayes Theorem is Intuitive

67. Naïve Bayes Theorem

68. Naïve Bayes Theorem

69. Naïve Bayes Theorem

70. Naïve Bayes Theorem

71. Naïve Bayes Theorem

72. Naïve Bayes Theorem

74. Naïve Bayes Theorem

75. Naïve Bayes Theorem

76. Naïve Bayes Theorem

77. Naïve Bayes Theorem

78. IMP code for Naïve Bayes
from sklearn.naive_bayes import GaussianNB
classifier = GaussianNB()
classifier.fit(X_train, y_train)
Confusion Matrix
array([[65, 3],
[ 7, 25]], dtype=int64)

79. IMP code for Naïve Bayes

80. 5. Decision Tree Classification

81. IMP code for DTC
from sklearn.tree import DecisionTreeClassifier
classifier = DecisionTreeClassifier(criterion = 'entropy', random_state = 0)
classifier.fit(X_train, y_train)
• Confusion Matrix
array([[62, 6],
[ 3, 29]], dtype=int64)
91 correctly classified and 9 incorrectly classified

82. Output of DTC

83. 6. Random Forest Classification : Based on Ensemble Learning

84. IMP Code for Random Forest Classification
from sklearn.ensemble import RandomForestClassifier
classifier = RandomForestClassifier(n_estimators = 10, criterion = 'entropy', random_state = 0)
classifier.fit(X_train, y_train)
Confusion Matrix
array([[63, 5],
[ 4, 28]], dtype=int64)
91 correctly classified and 9 incorrectly classified

85. Out of RFC

86. Classification Model Pros and Cons

87. Evaluating Classification
Models Performance
1. False Positives and False Negatives
2. Confusion Matrix
3. Accuracy Paradox
4. CAP Curve and its Analysis

88. 1. False Positives & False Negatives

89. False Positives & False Negatives
There are two errors that often rear their head when you are learning about
hypothesis testing — false positives and false negatives, technically referred to
as type I error and type II error respectively.
A false positive (type I error) — when you reject a true null hypothesis
A false negative (type II error) — when you accept a false null hypothesis?
A False Positive Rate is an accuracy metric that can be measured on a
subset of machine learning models.

90. False Positives & False Negatives

91. False Positives & False Negatives
In binary prediction/classification terminology, there are four conditions
for any given outcome:
•True Positive: is the correct identification of anomalous data as such,
e.g., classifying as “abnormal” data which is in fact abnormal.
•True Negative: is the correct identification of data as not being
anomalous, i.e. classifying as “normal” data which is in fact normal.
•False Positive: is the incorrect identification of anomalous data as such,
i.e. classifying as “abnormal” data which is in fact normal.
•False Negative: is the incorrect identification of data as not being
anomalous, i.e. classifying as “normal” data which is in fact abnormal.

92. False Positives & False Negatives

93. False Positives & False Negatives
• A true positive is an outcome where the model correctly predicts the positive class. Similarly,
a true negative is an outcome where the model correctly predicts the negative class.
• A false positive is an outcome where the model incorrectly predicts the positive class. And a false
negative is an outcome where the model incorrectly predicts the negative class.

94. False Positives & False Negatives

95. False Positives & False Negatives

96. False Positives & False Negatives

97. False Positives & False Negatives

98. False Positives & False Negatives

99. False Positives & False Negatives

100. False Positives & False Negatives
A false positive is an
outcome where the
model incorrectly
predicts
the positive class.
A false negative is
an outcome where
the model incorrectly
predicts
the negative class.

101. 2. Confusion Matrix

102. Confusion Matrix

103. Confusion Matrix

104. 3. Accuracy Paradox

105. Accuracy Paradox

106. Accuracy Paradox

107. Cumulative Accuracy Profile (CAP)

108. Cumulative Accuracy Profile (CAP)

109. Cumulative Accuracy Profile (CAP)

110. Cumulative Accuracy Profile (CAP)

111. Cumulative Accuracy Profile (CAP)

112. Cumulative Accuracy Profile (CAP)

113. Cumulative Accuracy Profile (CAP)

114. Cumulative Accuracy Profile (CAP)

115. Cumulative Accuracy Profile (CAP)

116. CAP Curve

117. CAP Curve

118. Cumulative Accuracy Profile (CAP)

119. Cumulative Accuracy Profile (CAP)