ML

1. CH 17
GOING A STEP BEYOND USING
SUPPORT VECTOR
MACHINES
10766012陳遠任 Jason

2. Revisiting the Separation
Problem
• Nonseparability of classes
• There is no straight line that traces a precise border
between different examples.
• Other options
• K-Nearest Neighbors : Ch14
• Logistic regression : Ch15
• Transforming the features : Solves the problem by
employing both feature creation
• Decision trees、Neural networks

3. Characteristics of
Support Vector Machines
• Binary and multiclass classification, regression, and
detection of anomalous or novelty data
• Robust handling of overfitting, noisy data, and outliers
• A capability to handle situations with many variables
• Easy and timely handling of up to about 10,000
training examples
• Automatic detection of nonlinearity in data

4. Explaining the Algorithm

5. SVM - Linear
Negative hyperplane
Positive hyperplane
Support Vector

6. Applying Nonlinearity
• Nonlinearly
separable points
requiring feature
transformation
(left) to be fit by a
line (right).
• Make the existing
features onto a
feature space of
higher
dimensionality

7. Applying Nonlinearity
• Problems and limits :
• The number of features increases exponentially, making
computations cumbersome 計算繁複
• The expansion creates many redundant features,
causing overfitting. 創造冗餘特徵
• Difficult to determine becoming linearly or not, requiring
many iterations of expansion and test

8. Kernel functions
• kernel functions project the original features into
a higher dimensional space by combining them
in a nonlinear way
• rely on algebra calculations

9. Discovering the different
kernels
• Linear: Suitable for linear
• No extra parameters
• Radial Basis Function: Suitable for non-linear
• parameters: gamma
• Polynomial: suitable for non-linear
• parameters: gamma, degree, and coef0
• Sigmoid: Binary classification like Logistic Regression
• parameters: gamma and coef0
• Custom-made kernels: Depends upon the kernel

10. Radial Basis Function
• An RBF kernel
that uses
diverse hyper-
parameters to
create unique
SVM solutions.
• The RBF kernel can adapt itself to different
learning strategies
• the error cost is high -> bended hyperplane
• the error cost is low -> smoother curve line

11. Kernels
• The polynomial and sigmoid kernels aren’t as
adaptable as RBF, thus showing more bias
• Most data problems are easily solved using the
RBF
sigmoid polynomial

12. Classifying and Estimating
with SVM
• handwritten recognition task
• the digits dataset (from Scikit-learn)
• nonlinear kernel, using the RBF
• a series of 8-x-8 grayscale pixel images of
handwritten numbers ranging from 0 to 9.