Q-Metric Based
                                                       Support Vector Machines


                      Dime...
Metrics
                                                                “distance indicators”




                      λ=...
Overview
                                                    frontiers on nonlinear modeling techniques


                ...
Q-Metrics Modeling (QMM)
                                       Computational Intelligence Applications & Impact on State ...
Support Vector Machines
                                 Prior Art and Problem Statement




       Linear Partitioning   ...
The Idea
               Systematic Application of Q-Metrics Modeling to Support Vector Machines



•     A Q-Metric is def...
Metrics
                                                                “distance indicators”




                      λ=...
Implementation
                  Java Applet




2009:01:24       Magdi A. Mohamed   8/23
Experimental Results
               Nonlinear Classification and Regression Cases




   Novel                            ...
Experimental Results
               Nonlinear Classification and Regression Cases




   Novel                            ...
4-Dimensional XOR Data Set
                    Nonlinear Classification Case




2009:01:24                 Magdi A. Moham...
More Experimental Results
                                   Testing Kernel Types Using X-DATA Set


     Type=0          ...
More Experimental Results
                    Testing Over Fitting Using X-DATA Set


                           Novel QMB...
Advantages of QMB-SVM
                                  Characteristics and Promises



1.     computational efficiency
2....
Potential Applications
                                      one vision for suites of techniques that work together




  ...
Novel
             QMB-SVC




2009:01:24   Magdi A. Mohamed   16/23
Conventional
              RBF-SVC




2009:01:24    Magdi A. Mohamed   17/23
Novel
             QMB-SVR




2009:01:24   Magdi A. Mohamed   18/23
Conventional
              RBF-SVR




2009:01:24    Magdi A. Mohamed   19/23
Novel
             QMB-SVC




2009:01:24   Magdi A. Mohamed   20/23
Conventional
              RBF-SVC




2009:01:24    Magdi A. Mohamed   21/23
Novel
             QMB-SVR




2009:01:24   Magdi A. Mohamed   22/23
Conventional
              RBF-SVR




2009:01:24    Magdi A. Mohamed   23/23
Upcoming SlideShare
Loading in...5
×

Q-Metric Based Support Vector Machines

1,577

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,577
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Q-Metric Based Support Vector Machines

  1. 1. Q-Metric Based Support Vector Machines Dimension2 dp=infinity = 1 dλ=-1 = 1 de = dp=2 = 1 y=(y1,y2) --- advances in kernel based machine learning systems --- Dimension1 x=(x1,x2) dλε(-1,0) = 1 dt = dp=1 = 1 dλ=0 = 1 Graph of d(x,y)=1 in 2-D Space Method for Constructing Q-Metric Based Support Vector Classification and Regression Machines 2009:01:24 Magdi A. Mohamed 1/23
  2. 2. Metrics “distance indicators” λ=-1 Max P=INF Euclidian Q-Metrics P-Metrics P=2 Euclidian Mainstream Manhattan Approaches λ=0 P=1 ge Probability era Av Measures Plausibility ion Believe Probability λ=INF ect “confidence quantifiers” ers Int ge Q-Aggregates er a λ=0 Av ion Un λ=-1 −1<λ<0 λ>0 λ=0 Aggregates Q-Measures “connective operators” 2009:01:24 Magdi A. Mohamed 2/23
  3. 3. Overview frontiers on nonlinear modeling techniques Nonlinear Models Motivating Object Image Pattern Applications Tracking Processing Recognition Fundamental Probability Robust Neural Theories Measure Estimation Networks 1940 Weiner Filter (MIT) Feed Forward Networks 1962 Hough Transform Filter 1960 Extended Kalman Filter (NASA) 1965 Morphological Filters (ECOLE) Shared Weight Network (IBM) Approaches Hidden Markov Model 1979 Alternating Sequential Filter (ERIM) Back Propagation Through Time Gaussian Sum Filter Weighted Median Filter Self Organizing Maps Condensation Filter (MS) Order Statistics Filters Dynamic Programming Networks 1993 Particle Filter Stack Filters 1997 Support Vector Machines (ATT) Motivating Automatic Computer Data Analysis & Applications Control Vision Financial Predictions Non-Additive Measures, Fundamental Non-Linear Integrals, Theories and Random Sets 1954 Choquet Capacities/Integral (ADIF) 1975 Sugeno Measure/Integral (TIT) Approaches Order Weighted Average 2000 Generalized Hidden Markov Model (UMC) 2003 Q-Filters (MOT) 2005 Q-Machines (MOT) 2009:01:24 Magdi A. Mohamed 3/23
  4. 4. Q-Metrics Modeling (QMM) Computational Intelligence Applications & Impact on State of the Art Supervised Learning Unsupervised Learning (NP-Hard) Supervised Learning Objective : Unsupervised Learning Objective : Find the set of centers, Q ⊂ P, that minimizes objective criterion Find the form, f, that minimizes objective criterion Φ(f) = ∑ distance ( f(p), t ) Ψ(Q) = ∑ min distance ( p, q ) q∈Q p∈P p∈P Applications Applications • linear/nonlinear optimization • vector quantization & cluster analysis • sequence analysis • automatic feature extraction • decision making • visualization & dimensionality reduction • compression (lossy & loss-free) Impact on Existing Machine Learning Paradigms 1. Feed-Forward Artificial Neural Nets • automated data labeling & data cleaning tools 2. Genetic Computing • data mining & knowledge discovery 3. Tree Classifiers • continuous adaptation (automatic tuning & 4. Dynamic Programming & Reinforcement Learning customization) 5. Hidden Markov Models Impact on Existing Machine Learning Paradigms 6. Nearest Prototype Classification 1. Crisp and Soft Clustering Algorithms 7. Crisp and Soft K-Nearest Neighbor Algorithms 2. Self Organizing Maps 8. Discriminant Analysis 3. Adaptive Resonance Theory 9. Support Vector Machines 2009:01:24 Magdi A. Mohamed 4/23
  5. 5. Support Vector Machines Prior Art and Problem Statement Linear Partitioning Nonlinear Partitioning Several Kernel Functions • Original Theory developed by Vapnik & Chervonenkis (VC Dimension) in 1974 • Boser, Guyon & Vapnik (AT&T) issued first patent (US5649068(A)) in July 15, 1997 • Several Kernel functions K(x,x’) exist (linear and nonlinear) • Kernel functions are defined using weighted Euclidean Distance (P-Metrics, P=2) • Fixing P=2, and other parameters (such as γ) causes critical limitations 2009:01:24 Magdi A. Mohamed 5/23
  6. 6. The Idea Systematic Application of Q-Metrics Modeling to Support Vector Machines • A Q-Metric is defined for computing distances in a Q-Metric Based Support Vector Machine (QMB-SVM) network using a variable parameter λ, that is bounded between the real values -1 and 0 resulting in an efficient distance function covering feasible range of potential metrics. The Q-Metric is constructed by computing a polynomial in the variable parameter λ. The parameter λ can then be automatically optimized to discover the ideal functionalities of the Q-Metric, based on the data to be analyzed. • The mathematical programming (training) task is formulated as an optimization problem where the QMB-SVM network parameters are adjusted to minimize an overall risk criterion quantified using Q-Metrics Modeling. 2009:01:24 Magdi A. Mohamed 6/23
  7. 7. Metrics “distance indicators” λ=-1 Max P=INF Euclidian Q-Metrics P-Metrics P=2 Q-Metrics QMB-SVM Manhattan Space λ=0 P=1 Measures Plausibility ion Believe Probability λ=INF ect “confidence quantifiers” ers Int ge ge Probability era Q-Aggregates er a λ=0 Av Av ion Un λ=-1 −1<λ<0 λ>0 λ=0 Aggregates Q-Measures “connective operators” 2009:01:24 Magdi A. Mohamed 7/23
  8. 8. Implementation Java Applet 2009:01:24 Magdi A. Mohamed 8/23
  9. 9. Experimental Results Nonlinear Classification and Regression Cases Novel Novel QMB-SVC QMB-SVR Conventional Conventional RBF-SVR RBF-SVC 2009:01:24 Magdi A. Mohamed 9/23
  10. 10. Experimental Results Nonlinear Classification and Regression Cases Novel Novel QMB-SVC QMB-SVR Conventional Conventional RBF-SVR RBF-SVC 2009:01:24 Magdi A. Mohamed 10/23
  11. 11. 4-Dimensional XOR Data Set Nonlinear Classification Case 2009:01:24 Magdi A. Mohamed 11/23
  12. 12. More Experimental Results Testing Kernel Types Using X-DATA Set Type=0 Type=1 Type=2 Type=3 Type=4 B P/ B P/ B P/ B P/ B P/ B 21 15 B 36 00 B 36 00 B 07 29 B 36 00 P/ 21 15 P/ 13 23 P/ 08 28 P/ 27 09 P/ 00 36 Acc 50.0% Acc 81.9% Acc 88.9% Acc 22.2% Acc 100% Conventional Conventional Conventional Conventional Novel Linear Polynomial RBF Sigmoid QMB-RBF 2009:01:24 Magdi A. Mohamed 12/23
  13. 13. More Experimental Results Testing Over Fitting Using X-DATA Set Novel QMB-SVC λ=-1.00 λ=-0.75 λ=-0.50 λ=-0.25 λ=0.00 Conventional RBF-SVC γ=0.5 γ=11 γ=111 γ=1111 γ=11111 2009:01:24 Magdi A. Mohamed 13/23
  14. 14. Advantages of QMB-SVM Characteristics and Promises 1. computational efficiency 2. numerical stability 3. per unit calculations simplify implementations (software and hardware) 4. suitability for massive parallel implementations 5. automatic discovery of multiple metric spaces 6. consistent handling of “curse of dimensionality” concerns 7. improvement over existing kernel functions 8. usability for both classification and regression applications 9. ease of use “A hypothesis or theory is clear, decisive, and positive, but it is believed by no one but the man who create it. Experimental findings, on the other hand, are messy, inexact things, which are believed by everyone except the man who did the work.” - Harlow Shapley 2009:01:24 Magdi A. Mohamed 14/23
  15. 15. Potential Applications one vision for suites of techniques that work together INPUTS EVENTS CLASSIFIER SENSOR SIGNALS SIGNALS ACTION CODES FEATURES SIGNAL DATA SIGNAL SENSOR SENSOR PRE- PROCESSING/ POST- FUSION PROCESSING ANALYSIS PROCESSING ACTIONS SENSOR SIGNALS ACTION CODES DECISIONS OUTPUTS EVENTS INPUTS FEATURES SIGNALS SENSOR CLASSIFIER SIGNAL DATA SIGNAL SENSOR CLASSIFER DECISION SENSOR PRE- PROCESSING/ POST- DISPLAY FUSION FUSION CONTROL PROCESSING ANALYSIS PROCESSING SENSOR SIGNALS SIGNALS ACTION CODES EVENTS INPUTS FEATURES SENSOR CLASSIFIER SIGNAL DATA SIGNAL SENSOR SENSOR PRE- PROCESSING/ POST- FUSION PROCESSING ANALYSIS PROCESSING SENSOR Q-AGGREGATES Q-FILTERS Q-METRICS Q-FILTERS Q-AGGREGATES Q-METRICS 2009:01:24 Magdi A. Mohamed 15/23
  16. 16. Novel QMB-SVC 2009:01:24 Magdi A. Mohamed 16/23
  17. 17. Conventional RBF-SVC 2009:01:24 Magdi A. Mohamed 17/23
  18. 18. Novel QMB-SVR 2009:01:24 Magdi A. Mohamed 18/23
  19. 19. Conventional RBF-SVR 2009:01:24 Magdi A. Mohamed 19/23
  20. 20. Novel QMB-SVC 2009:01:24 Magdi A. Mohamed 20/23
  21. 21. Conventional RBF-SVC 2009:01:24 Magdi A. Mohamed 21/23
  22. 22. Novel QMB-SVR 2009:01:24 Magdi A. Mohamed 22/23
  23. 23. Conventional RBF-SVR 2009:01:24 Magdi A. Mohamed 23/23

×