The document summarizes research on making support vector machines (SVMs) more robust to adversarial label noise. It discusses how adversaries can intentionally flip labels in training data to undermine SVMs. The researchers propose a label noise robust SVM that learns from an expected kernel matrix to be less sensitive to label flips. Experiments on several datasets show their approach maintains higher accuracy than standard SVMs when the training data contains adversarial or random label noise. In conclusions, they discuss further investigating the properties and parameter selection for their kernel correction method.

1. The 3rd Asian Conference on Machine Learning,
ACML2011 Taoyuan, Taiwan, November, 13-15, 2011
Support vector machines under
adversarial label noise
Battista Biggio1, Blaine Nelson2, Pavel Laskov2
(1) Pattern Recognition and Applications Group
PRA
group Department of Electrical and Electronic Engineering (DIEE)
University of Cagliari, Italy
(2) Cognitive Systems Group
Reactive WilhelmUniversity of Institute for Computer Science
Schickard
Security University of Tuebingen, Germany
Cagliari

2. Outline
• Adversarial classification
• Our work
– Attacking SVMs
– Label Noise robust SVM
• Experiments
• Conclusions
University of
Cagliari
13-11-2011 SVMs under adv ersarial label noise - B. Nelson - ACML2011 2

3. Adversarial classification
• Pattern recognition in security applications
– spam filtering, intrusion detection, biometrics
x2 legitimate
f(x) malicious
x1
University of
Cagliari
13-11-2011 SVMs under adv ersarial label noise - B. Nelson - ACML2011 3

4. Adversarial classification
• Pattern recognition in security applications
– spam filtering, intrusion detection, biometrics
• Malicious adversaries aim to mislead the system
x2 legitimate
f(x) malicious
Buy viagra!
Buy vi4gr@!
x1
University of
Cagliari
13-11-2011 SVMs under adv ersarial label noise - B. Nelson - ACML2011 4

5. Open issues
1. Vulnerability identification
2. Security evaluation of pattern classifiers
3. Design of secure pattern classifiers
University of
Cagliari
13-11-2011 SVMs under adv ersarial label noise - B. Nelson - ACML2011 5

6. A taxonomy of potential attacks
against machine learning systems
Causative (TR) / Exploratory (TS)
Influence M. Barreno, B. Nelson, A. Joseph, and J.
Tygar. The security of machine learning.
Machine Learning, 81:121–148, 2010.
Security violation Specificity
Integrity (FN) / Availability (FP+FN) Targeted / Indiscriminate
University of
Cagliari
13-11-2011 SVMs under adv ersarial label noise - B. Nelson - ACML2011 6

7. Attacking SVMs
adversarial label flips
• Support vector machines • Label flips
f (x;w,b) = sign(w! x + b) !{"1, +1} – Max. classification error
• Heuristic strategy
1 – Flip labels of samples
min # !Q# " 1! # n which are farthest from
# 2
the hyperplane (high loss)
s. t. 0 $ # i $ C, i = 1,…, n,
n – Correlated label flips
%# y i i = 0, where Q = K ! yy! .
i =1
n
Solution is sparse! w = " ! i yi xi
i =1
University of
Cagliari
13-11-2011 SVMs under adv ersarial label noise - B. Nelson - ACML2011 7

8. Label Noise (LN) robust SVMs
• Label flip yi ' = !yi " yi ' = yi (1 ! 2# i ), # i ${0,1}
• Kernel matrix becomes
Qij = yi y j K(xi , x j )(1 ! 2" i )(1 ! 2" j )
• To be less sensitive to label flips, we learn an SVM using the
expected kernel matrix
– random noise (ε iid r.v.)
% yi y j K(xi , x j )(1 " 4# 2 ), if i $ j,
'
E ! [Qij ] = &
' yi y j K(xi , x j ),
( otherwise,
where # 2 = µ (1 " µ ).
University of
Cagliari
13-11-2011 SVMs under adv ersarial label noise - B. Nelson - ACML2011 8

9. Label Noise (LN) robust SVMs
• Pros
– Kernel correction
– Convex QP problem
• Cons
– Parameter selection µ
– Heuristic approach (not guaranteed to be optimal)
University of
Cagliari
13-11-2011 SVMs under adv ersarial label noise - B. Nelson - ACML2011 9

10. A simple example
SVM LN-robust SVM
• Weights are more spread among training points
• Solution is less sparse (but more robust)
University of
Cagliari
13-11-2011 SVMs under adv ersarial label noise - B. Nelson - ACML2011 10

11. Experiments
• SVM with linear kernel
– similar results with RBF kernel
• 7 UCI data sets
– australian, breast-cancer, diabetes, fourclass, heart,
ionosphere, sonar
• Attack strategies
– adversarial label flips
– random label flips
• Classification error is evaluated on a (untainted) testing
set w.r.t. the percentage of flipped labels in training data
University of
Cagliari
13-11-2011 SVMs under adv ersarial label noise - B. Nelson - ACML2011 11

12. Experimental results
adversarial label flips random label flips
University of
Cagliari
13-11-2011 SVMs under adv ersarial label noise - B. Nelson - ACML2011 12

13. Conclusions and future work
• Accuracy vs robustness trade-off
– Guidelines for parameter selection (µ)
• Investigation of properties of the proposed
kernel correction
– Weight equalization
– Modified loss function
University of
Cagliari
13-11-2011 SVMs under adv ersarial label noise - B. Nelson - ACML2011 13

14. Thank you!
?
Battista Biggio battista.biggio@diee.unica.it
Blaine Nelson blaine.nelson@wsii.uni-tuebingen.de
Pavel Laskov pavel.laskov@uni-tuebingen.de
University of
Cagliari

15. Backup slides
University of
Cagliari

16. Results: adversarial label flips
University of
Cagliari
13-11-2011 SVMs under adv ersarial label noise - B. Nelson - ACML2011 16

17. Results: random label flips
University of
Cagliari
13-11-2011 SVMs under adv ersarial label noise - B. Nelson - ACML2011 17