• Like
Data Selection For Support Vector Machine Classifier
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Data Selection For Support Vector Machine Classifier

  • 489 views
Published

 

Published in Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
489
On SlideShare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
35
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Glenn Fung and Olvi L. Mangasarian August 2000 20081021 Kuan-Chi-I
  • 2. Outline
    • Introduction
    • SVM
    • MSVM
    • Comparisons
    • Conclusion
  • 3. Introduction
    • A method for selecting a small set of support vectors which determines a separating plane clsssifier.
    • Useful for applications contain millions of data points.
  • 4. SVM
    • A method for classification.
  • 5. SVM (Linear Separable Case)
  • 6. SVM
    • To find the maximum margin ,equivelent to find minimum ½|| w || 2.
    • We can transfer above problem to a quadratic problem with parameter v > 0.
    • A : a real m×n matrix.
    • e : column vectors of ones in arbitrary dimension.
    • e ′ : transpose of e .
    • y : nonnegitive slack variables.
    • D : m×m diagonal matrix of 1 or -1.
  • 7. SVM
    • Written in individual component natation .
    • A i :row vector of matrix A .
  • 8. SVM
    • x′w = γ +1 bounds the class A + points.
    • x′w = γ +1 bounds the class A - points.
    • γ : the location relative to the origin.
    • w : normal to the bounding planes.
    • The linear separating surface is the plane:
  • 9. SVM (Linearly Inseparable Case)
  • 10. SVM (Inseparable)
    • If the class are inseparable then the two planes bound the two class with a 〝 soft margin”.
  • 11. MSVM (1-Norm SVM)
    • A minimal support vertor machine (MSVM).
    • In order to make use of a faster programming based approach, we reformulate (1) by replacing the 2-norm by a 1-norm as follows:
  • 12. MSVM
    • The mathematical program (7) is easily convert to a linear program as follows:
    • υ : the absolute value | w | of w , and υ i ≧| w i |
  • 13. MSVM
    • If we define nonnegative multipliers u ∈ R m associated with the first set of constraints of the linear program (8), and multipliers (r, s) ∈ R n+n for the second set of constraints of (8), then the dual linear program associated with the linear SVM formulation (8) is the following:
  • 14. MSVM
    • We modify the linear program to generate an SVM with as fewer support vector as possible by addingan error term e ′ y *
    • The term e ′ y * suppresses mis-classified points and results in our minimal support vector machine MSVM:
    • y * :vector x in R n with components ( y * ) i =1 if y i > 0 and 0 otherwise.
    • μ :positive parameter ,chosen by a tuning set .
  • 15. MSVM
    • We approximate e ′ y * here by a smooth concave exponential on the nonnegative real line as was done in the feature selection approach of. For y ≥ 0, the approximation of the step vector y∗ of (9) by the concave exponential, , i = 1, . . . ,m, that is:
  • 16. MSVM
    • The smooth MSVM:
  • 17. MSVM (SLA)
  • 18. Comparison
  • 19. Observations of Comparisons
    • 1. For all test problems MSVM had least number of support vectors.
    • 2. For the Ionosphere problem, the reduction in the num-
    • ber of support vectors of MSVM over SVM| · | 1 is 81%, and
    • the average reduction in the number of support vectors of MSVM over SVM| · | is 65.8%.
    • 3. Tenfold testing set correctness of MSVM was good.
    • 4. Computing times were higher for MSVM than for other classifiers.
  • 20. Conclution
    • We proposed a minimal support vector machine.
    • Useful in classifying very large datasets by using only a fraction of the data.
    • Improves generalization over other classifiers that use a higher number of data points.
    • MSVM requires the solution of a few linear programs to determine a sepaeating surface .