13. 最適化問題
X X
max L(xc ) = [1 yk fxc (xk )]+ = gk (xc )
xc
k k
• Validation Setにおける損失を最大化
• fx (·) : 悪性データ込みで学習されたSVM
c
• 非凸な最適化問題 0
xc = xc + t · u
• 解法：Gradient Ascent u / rL(xc )
• SVMの更新と悪性データの更新を繰り返す
• ステップ幅を適切に設定すれば局所最適解に収束
12年7月28日土曜日 11
14. アルゴリズムの概要
Poisoning Attacks against SVMs
Algorithm 1 Poisoning attack against SVM 3.1. Artificial data
Input: Dtr , the training data; Dval , the validation
(0)
data; yc , the class label of the attack point; xc , the
初期点は，既存データ
We first consider a two-dimensional da
model in which each class follows a G
initial attack point; t, the step size. のラベルをflipして作成
bution with mean and covariance mat
Output: xc , the final attack point. µ = [ 1.5, 0], µ+ = [1.5, 0], ⌃ =
1: {↵i , b} learn an SVM on Dtr . The points from the negative distributio
2: k 0. the label 1 (shown as red in the subse
3: repeat and otherwise +1 (shown as blue). Th
(p)
4: Re-compute the SVM solution on Dtr [{xc , yc } the validation sets, Dtr and Dval (consis
using incremental SVM (e.g., Cauwenberghs & SVMの更新
500 points per class, respectively) are ra
Poggio, 2001). This step requires {↵i , b}. from this distribution.
5: Compute @L on Dval according to Eq. (10).
6:
@u
Set u to a unit vector aligned with @L . 勾配を算出
In the experiment presented below, the r
(p) (p 1)
@u attacking class. To this end, a random
7: k k⇣ 1 and xc⇣
+ ⌘ xc⌘ + tu blue class is selected and its label is fli
(p) (p 1)
8: until L xc L xc <✏ 悪性データの更新
as the starting point for our method.
9: return: xc = xc
(p) ascent method is then used to refine t
til its termination condition is satisfied.
from [Biggio+ 12] trajectory is traced as the black line in F
12年7月28日土曜日 the linear kernel (upper two plots) and 12
15. SVMの更新
• Incremental SVM [Cauwenberghs+ NIPS00]
• １つずつデータを追加しながらSVMを学習
• 全データの役割が不変な範囲の最適化問題を反復的に解く
• Reserve Point / Support Vector / Error Vector
• 条件を破らないと収束しない場
W W W
合，データの役割を変更 gi=0
gi>0 gi<0
• 各最適化時には，サポートベク αi=0
xi
C αi
xi
C αi=C
ターのパラメータのみが更新 xi
• データが追加されるたびに，全 support vector error vector
パラメータが収束するまで最適 Figure 1: Soft-margin classification SVM training.
from [Cauwenberghs+ NIPS00]
化すれば，SVMの最適解に収束
coefficients are obtained by minimizing a convex quadratic objective func
constraints [12]
12年7月28日土曜日 13
16. 最適化問題の勾配計算
• Incremental SVMのアイデアを用いる
Poisoning Attacks against SVMs
product rule: • 更新時に各データの役割が変動しない仮定を置く
inverted matrix are independent of x , we obtain:
c
@b • サポートベクターのみに着目すれば良い
@↵
=
1
↵ (Qc ss
1
)·
@Q > sc
+ yk , (3) @u ⇣ @u
@u
• 更新式はカーネル関数に依存
@b
=
1
↵ c·
@Q
>
.
sc
(9)
@u ⇣ @u
• 厳密な計算には，条件を破らないステップ幅の導出が必要
Substituting (9) into (3) and further into (1), we obtain
@Qkc @b
il. ,
@u @u
.
• 本研究では定数ステップ幅で値を更新，計算をサボる
the desired gradient used for optimizing our attack:
@L X⇢
m
@Qsc @Qkc
= Mk + ↵c , (10)
@u @u @u
can be further re- k=1
taken in direction
where
VM solution. This
ate condition using 1
Mk = (Qks (Qss1 T
) + yk T
).
enberghs & Poggio, ⇣
from [Biggio+ 12]
oint in the training
12年7月28日土曜日 14
19. 手書き文字認識
実験設定
MNIST
実験データ
( 7 vs. 1; 9 vs. 8; 4 vs. 0)
線形カーネル
SVM
C=1
training set 100
validation set 500
12年7月28日土曜日 17
20. 手書き文字認識
実験結果 (7 vs. 1)
Poisoning Attacks against SVMs
Before attack (7 vs 1) After attack (7 vs 1) classification error
0.4
validation error
5 5
0.3 testing error
10 10
15 15 0.2
20 20
0.1
25 25
0
5 10 15 20 25 5 10 15 20 25 0 200 400
number of iterations
ラベルは1
from [Biggio+ 12]
Before attack (9 vs 8) After attack (9 vs 8) classification error
0.4
validation error
5 5
0.3 testing error
10 10
15
12年7月28日土曜日 15 0.2 18
21. Before attack (7 vs 1) After attack (7 vs 1) classification error
0.4
5
10
手書き文字認識 5
10
0.3
validation error
testing error
0.2
実験結果 (8 vs. 9)
15 15
20 20
0.1
25 25
0
5 10 15 20 25 5 10 15 20 25 0 200 400
number of iterations
Before attack (9 vs 8) After attack (9 vs 8) classification error
0.4
validation error
5 5
0.3 testing error
10 10
15 15 0.2
20 20
0.1
25 25
0
5 10 15 20 25 5 10 15 20 25 0 200 400
number of iterations
ラベルは8
from [Biggio+ 12]
Before attack (4 vs 0) After attack (4 vs 0) classification error
0.4
validation error
5 5
0.3 testing error
10 10
15
12年7月28日土曜日 15 0.2 19
22. Before attack (9 vs 8) After attack (9 vs 8) classification error
0.4
5
10
手書き文字認識 5
10
0.3
validation error
testing error
実験結果 (4 vs. 0)
15 15 0.2
20 20
0.1
25 25
0
5 10 15 20 25 5 10 15 20 25 0 200 400
number of iterations
Before attack (4 vs 0) After attack (4 vs 0) classification error
0.4
validation error
5 5
0.3 testing error
10 10
15 15 0.2
20 20
0.1
25 25
0
5 10 15 20 25 5 10 15 20 25 0 200 400
number of iterations
ラベルは0
from [Biggio+ 12]
Modifications to the initial (mislabeled) attack point performed by the proposed attack strategy, for
d two-class problems from the MNIST data set. The increase in validation and testing errors across
is also reported.
12年7月28日土曜日 20
24. 0.25
0.2
0.15
複数データ実験
0.1
0.05
Poisoning Attacks against SVMs
0
0 2 4 6 8
ing manyattack points in training steps. It would be int
% of tiny gradient data
to investigate a more accurate and e cient co
tion of the largest possible step that does not a
classification error (7 vs 1) classification error (9 vs 8)
0.4 0.4
structure of the optimal solution.
validation error validation error
0.35 testing error 0.35 Anothererror
testing direction for research is the simultaneo
•
mization of multi-point attacks, which we succ
悪性データを一個ずつ追加で入れていった場合の性能推移 0.3 0.3
0.25
approached with sequential single-point attac
0.25
first question is how to optimally perturb a s
0.2
•
0.2
the training data; that is, instead of individua
決定境界に近いデータを初期点に置くと，悪性データ 0.15 0.15 mizing each attack point, one could derive sim
0.1 0.1 ous steps for every attack point to better optim
がreserve pointに陥るため，そこで更新がストップ
Poisoning Attacks against SVMs
0.05 0.05 overall e↵ect. The second question is how to
the best subset of points to use as a startin
0 0
ing many attack points in trainingsteps. It would be interestingof attack points in training datathe latter is a subs
for the attack. Generally,
0 2 0 2 4 6 8
% of
tiny 4gradient data 6 8
%
to investigate a more accurate and e cient computa- tion problem but heuristics may allow for impr
tion of the largest possible step that does not alter the proximations. Regardless, we demonstrate th
classification error (7 vs 1) structureclassificationoptimal8)solution.
of the error (9 vs non-optimal multi-point attack strategies sign
classification error (4 vs 0)
0.4 0.4 0.4
validation error validation error
degrade the SVM’s performance.
validation error
0.35 testing error 0.35 Another direction for research is the simultaneous opti- error
testing error 0.35 testing
mization of multi-point attacks, which we0.3 An important practical limitation of the p
successfully
0.3 0.3
approached with sequential single-point 0.25 method is the assumption that the attacker
attacks. The
first question is how to optimally perturb a subset of of the injected points. Such assu
the labels
0.25 0.25
0.2 0.2 0.2 may not hold when the labels are only assi
the training data; that is, instead of individually opti-
0.15 0.15 trusted sources such as humans. For instance,
0.15 mizing each attack point, one could derive simultane-
filter uses its users’ labeling of messages as its
0.1 0.1 ous steps for every attack point to better optimize their0.1
truth. Thus, although an attacker can send a
0.05 0.05 overall e↵ect. The second question is how to choose 0.05
messages, he cannot guarantee that they will h
the best subset of points to use as a starting point 0
0 0
0 0 labels2 necessary for his attack. This imposes
4 6 8
0 2 4 6 8 for the%attack.4 Generally, the latter is a subset selec-of attack points in training data
2 6 8
% of attack points in training data of attack points in training data ditional requirement that the attack data mus
%
tion problem but heuristics may allow for improved ap-
certain side constraints to fool the labeling orac
from [Biggio+ 12]
proximations. Regardless, we demonstrate Results of the multi-point, multi-run experiments
Figure 3. that even is needed to understand these poten
ther work
classification error (9 vs 8) non-optimal multi-point0)
classification error (4 vs attack strategies MNIST data set. In each plot, we show the clas-
on the significantly
0.4 0.4 constraints and to incorporate them into atta
validation error
degrade the SVM’s performance. sification errors due to poisoning as a function of the per-
validation error
0.35 centage of training contamination would bethe incorporate t
0.35 testing error
testing error The final extension for both to validation
0.3
An important practical limitation of solid line) and inverse sets (black dashed problem; that
(red the proposedtesting feature-mapping line). The
world
0.3
12年7月28日土曜日 method is the assumption that the attacker controls of7 finding real-worldmiddle is data t
topmost plotproblem
is for the vs.1 classifier, the attack for 22
0.25
