1. Poisoning attacks against
support vector machines
Battista Biggio (1), Blaine Nelson (2), Pavel Laskov (2)
(1) Pattern Recognition and Applications Group
Department of Electrical and Electronic Engineering (DIEE)
University of Cagliari, Italy
(2) Cognitive Systems Group
Wilhelm Schickard Institute for Computer Science
University of Tuebingen, Germany

2. Machine learning in adversarial settings
• Machine learning in computer security
– spam filtering, network intrusion detection, malware detection, biometrics
• Malicious adversaries aim to mislead the system
IDS Tr
inbound traffic
Network
outbound traffic
June 28th, 2012 Poisoning attacks against SVMs - ICML 2012 - B. Biggio 2

3. Machine learning in adversarial settings
• Machine learning in computer security
– spam filtering, network intrusion detection, malware detection, biometrics
• Malicious adversaries aim to mislead the system
IDS Tr
inbound traffic
Network
poisoning attack
outbound traffic
June 28th, 2012 Poisoning attacks against SVMs - ICML 2012 - B. Biggio 3

4. Poisoning attack against SVMs
Problem setting
• Goal. To maximize the classification error (DoS attack)
by injecting an attack point xc into the training set
• Main assumption. Perfect knowledge / worst-case scenario
classification error = 0.022 classification error = 0.039
xc
June 28th, 2012 Poisoning attacks against SVMs - ICML 2012 - B. Biggio 4

5. Poisoning attack against SVMs
Problem setting
• Goal. To maximize the classification error (DoS attack)
by injecting an attack point xc into the training set
• Main assumption. Perfect knowledge / worst-case scenario
classification error = 0.022 classification error as a function of xc
xc
June 28th, 2012 Poisoning attacks against SVMs - ICML 2012 - B. Biggio 5

6. Our approach
• To maximize the hinge loss on a validation set hinge loss: max(0,-g)
max L(xc ) = " (1 ! yk fxc (xk ))+
xc
k 1
!gk (xc ) yf(x)
1
• Gradient ascent xc = xc + t " #L(xc )
!
dgk
!L(xc ) = " # dx
k: gk <0 c
dgk % d$ j ( db dQkc
= # ' Qkj + yk + $ c , where Q = yyT ! K
dxc j & dxc *) dxc dxc
How does the SVM solution change during a single update of xc?
June 28th, 2012 Poisoning attacks against SVMs - ICML 2012 - B. Biggio 6

7. A trick from incremental SVM
• Assumption. No structural change occurs during a single update of xc
– Karush-Kuhn-Tucker conditions must hold before and after the update
yi f (xi ) ! 1 = 0, 0 < " i < C
d! i
S: margin vectors = 0, i "R # E
dxc
gi
dgi
R: reserve vectors gi > 0, ! i = 0 = 0, i "S
dxc
dh
h = $ y j! j = 0 % =0
j dxc
" db %
$ dx ' " 0 (1 " 0 %
yT
% $ '
$ c '=$ s
' $ dQsc '
E: error vectors gi < 0, ! i = C $ d! s ' # ys Qss &
$ dx ' $ dxc '
# &
# c&
June 28th, 2012 Poisoning attacks against SVMs - ICML 2012 - B. Biggio 7

8. Our approach
dgk " d! j % db dQkc
= ) $ Qkj ' + yk dx + dx ! c
dxc j (S # dxc & c c
dgk $ dQsc dQkc '
!L(xc ) = " # = # & Mk + ) *c
k: gk <0 dxc k: gk <0 % dxc dxc (
The gradient now only depends on the derivative of the kernel function!
1
+.
"1
( 0)
M k = " -Qks Qss " ,, T + yk, T / , + = ys Qss ys and , = Qss ys
T "1 "1
June 28th, 2012 Poisoning attacks against SVMs - ICML 2012 - B. Biggio 8

9. Poisoning attack algorithm
Linear kernel
(0)
xc
xc
(0)
xc
dQkc d
= yk yc K(xk , xc ) = yk yc ! xk
dxc dxc xc
June 28th, 2012 Poisoning attacks against SVMs - ICML 2012 - B. Biggio 9

10. Poisoning attack algorithm
RBF kernel
(0)
xc xc
dQkc
= yk yc ! K(xk , xc ) ! " ! (xk # xc ) (0) xc
dxc xc
June 28th, 2012 Poisoning attacks against SVMs - ICML 2012 - B. Biggio 10

11. Experiments on the MNIST digit data
Single-point attack
• Linear SVM; 784 features; TR: 100; VAL: 500; TS: about 2000
(0)
xc xc
June 28th, 2012 Poisoning attacks against SVMs - ICML 2012 - B. Biggio 11

12. Experiments on the MNIST digit data
Multiple-point attack
• Linear SVM; 784 features; TR: 100; VAL: 500; TS: about 2000
June 28th, 2012 Poisoning attacks against SVMs - ICML 2012 - B. Biggio 12

13. Conclusions and future work
• SVM may be very vulnerable to poisoning (worst-case scenario)
• What if we assume more realistic scenarios?
– Effectiveness with surrogate data
• How to improve robustness to poisoning?
• Find us at the poster session (#12)
– 17:40, Informatics Forum (IF)
Thanks for your attention!
June 28th, 2012 Poisoning attacks against SVMs - ICML 2012 - B. Biggio 13