Download as PDF, PPTX
Uploaded bySjoerd Maessen
Machine learning support vector machines
Support Vector Machines are a type of machine learning algorithm that can automatically create a model to classify data based on training examples. The model represents data as points in space and finds the optimal hyperplane that separates categories of data by the widest possible margin. SVMs use kernels to project data into a higher dimensional space to allow for the discovery of more complex patterns that would not be possible in the original input space.
More Related Content
Similar to Machine learning support vector machines
Machine learning support vector machines
- 1. Machine Learning Support VectorMachines
- 2. Sjoerd Maessen • AFOL • Works at E-sites Breda • Stock market enthusiast
- 3. Titanic The chance ofsurvival
- 4. PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked 1 0 3 Braund, Mr. Owen Harris male 22 1 0 A/5 21171 7.25 S 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Thayer) female 38 1 0 PC 17599 712.833 C85 C 3 1 3 Heikkinen, Miss. Laina female 26 0 0 STON/O2. 3101282 7.925 S 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35 1 0 113803 53.1 C123 S 5 0 3 Allen, Mr. William Henry male 35 0 0 373450 8.05 S 6 0 3 Moran, Mr. James male 0 0 330877 84.583 Q 7 0 1 McCarthy, Mr. Timothy J male 54 0 0 17463 518.625 E46 S 8 0 3 Palsson, Master. Gosta Leonard male 2 3 1 349909 21.075 S 9 1 3 Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg) female 27 0 2 347742 111.333 S 10 1 2 Nasser, Mrs. Nicholas (Adele Achem) female 14 1 0 237736 300.708 C 11 1 3 Sandstrom, Miss. Marguerite Rut female 4 1 1 PP 9549 16.7 G6 S 12 1 1 Bonnell, Miss. Elizabeth female 58 0 0 113783 26.55 C103 S 13 0 3 Saundercock, Mr. William Henry male 20 0 0 A/5. 2151 8.05 S 14 0 3 Andersson, Mr. Anders Johan male 39 1 5 347082 31.275 S
- 5.
- 6. “Could you takein account siblings?”
- 7.
- 8. “Oh and thenumber of parents and children aboard…”
- 9.
- 10. “Could you adda ‘simple if’ for age as well?”
- 12. “Great! We arealmost there! But…”
- 14. “Field of studythat gives computers the ability to learn without being explicitly programmed” Arthur Lee Samuel
- 16.
- 18. Alright! Let's become datascientists!
- 20. A comparison Traditional programming Machine learning Input programInput output output programnew input new output
- 21. Classification vs regression Input Output 0.98 68 0 0.76 42 0 1.23 78 1 1.91 109 1 Input Output 0.98 68 0.23 0.76 42 0.15 1.23 78 4.74 1.91 109 7.98
- 22. Support Vector Machine • Automatically creates a “program” or model • Inputs are ‘features’ • Model represents a space • New input fits somewhere
- 26. Support Vectors • Optimalhyperplane • Linear classifier • Maximum margin • Classification
- 27.
- 28.
- 29. The kernel trick Awhole new dimension
- 30.
- 31. Choosing a kernel • No kernel or linear kernel • Gaussian kernel • Polynomial kernel • Sigmoid kernel • Radial basis function kernel • …
- 32. Choosing a kernel Ruleof thumb • N much bigger than M => linear kernel • N small, M intermediate => gaussian kernel N = number of features M = number of training examples
- 33. Spam detection • N= 10000 (bad words, # of urls,…) • M = 250 (sample mails) => linear kernel
- 34. Validation of housingprices • N = 1-1000 (# of rooms, m3, location,…) • M = 100,000 (of transactions) => Gaussian kernel
- 35. Features It’s all aboutpreparation
- 36. Features • Representation ofraw data • The hardest part Raw data Pre- processing Feature scaling Feature extraction
- 37.
- 38. OCR – Pre-processing • De-skew
- 39. OCR – Pre-processing • De-skew • Despeckle
- 40. OCR – Pre-processing • De-skew • Despeckle • Convert to black & white
- 41. OCR – Pre-processing • De-skew • Despeckle • Convert to black & white • Zoning
- 42. OCR – Pre-processing • De-skew • Despeckle • Convert to black & white • Zoning • Character segmentation
- 43. OCR – Featureextraction
- 44. OCR – Featurescaling How to scale the number of black pixels? 22 / 56=> 0.39286 Scale between 0 - 1
- 45.
- 46. OCR – Trainingthe model Training file • Labels • Features Label % Black Pixels X1 % X2 % X3 % … 0 0.33 0.546 0.840 1 0.78 0.123 0.567 0.347 1 0.75 0.512 0.543
- 47. Alice in Wonderland Downthe rabbit hole
- 48. • Avg wordlength • Char frequency • … Basic text features
- 49. • Avg wordlength • Char frequency • … Basic text features
- 50.
- 51.
- 52. Input “Thank you forcontacting us. This is an automated response confirming the receipt of your ticket. Our team will get back to you as soon as possible. When replying, please make sure that the ticket ID is kept in the subject so that we can track your replies.” Testing the unknown
- 53. Input “Thank you forcontacting us. This is an automated response confirming the receipt of your ticket. Our team will get back to you as soon as possible. When replying, please make sure that the ticket ID is kept in the subject so that we can track your replies.” Output This is an English text Testing the unknown
- 54. Input “Hierbij bevestigen wijde ontvangst en verwerking van uw e-mail met ticketnummer PCL-98124-735. Uw vraag wordt opgepakt door één van onze engineers. Wij streven ernaar spoedig een oplossing aan u terug te kunnen koppelen. “ Testing the unknown
- 55. Input “Hierbij bevestigen wijde ontvangst en verwerking van uw e-mail met ticketnummer PCL-98124-735. Uw vraag wordt opgepakt door één van onze engineers. Wij streven ernaar spoedig een oplossing aan u terug te kunnen koppelen. “ Output This is a Dutch text Testing the unknown
- 56. Titanic The chance ofsurvival
- 57. PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked 1 0 3 Braund, Mr. Owen Harris male 22 1 0 A/5 21171 7.25 S 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Thayer) female 38 1 0 PC 17599 712.833 C85 C 3 1 3 Heikkinen, Miss. Laina female 26 0 0 STON/O2. 3101282 7.925 S 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35 1 0 113803 53.1 C123 S 5 0 3 Allen, Mr. William Henry male 35 0 0 373450 8.05 S 6 0 3 Moran, Mr. James male 0 0 330877 84.583 Q 7 0 1 McCarthy, Mr. Timothy J male 54 0 0 17463 518.625 E46 S 8 0 3 Palsson, Master. Gosta Leonard male 2 3 1 349909 21.075 S 9 1 3 Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg) female 27 0 2 347742 111.333 S 10 1 2 Nasser, Mrs. Nicholas (Adele Achem) female 14 1 0 237736 300.708 C 11 1 3 Sandstrom, Miss. Marguerite Rut female 4 1 1 PP 9549 16.7 G6 S 12 1 1 Bonnell, Miss. Elizabeth female 58 0 0 113783 26.55 C103 S 13 0 3 Saundercock, Mr. William Henry male 20 0 0 A/5. 2151 8.05 S 14 0 3 Andersson, Mr. Anders Johan male 39 1 5 347082 31.275 S
- 61. Feature extraction • Title(Mrs, Miss, Mr, Jonkheer, Capt,..) • Passengerclass • Sex • Age • Siblings/spouses • Parent/children • Cabin • Port of embarkation
- 62.
- 63.
- 64.
- 65.
- 66.
- 67.
- 68.
- 69. Common issues • Featurenumbering • Training data <> real world • Overfitting • Feature selection • Multiclass classification
- 70.
- 71.
- 76. Resources • https://www.csie.ntu.edu.tw/~cjlin/libsvm/ • http://php.net/manual/en/book.svm.php • https://www.kaggle.com/ • http://scikit-learn.org/stable/ • https://packagist.org/packages/sjoerdmaessen/machinelearning
- 77.