The document discusses the development of 1.5-class multi-class classifiers (MCSs) aimed at enhancing security against evasion attacks in machine learning applications, particularly in spam filtering and PDF malware detection. It details the methodology for evaluating classifier security and experimental outcomes demonstrating the effectiveness of MCSs in retaining high accuracy and security against such attacks. Future work aims to formalize the trade-offs between security and accuracy and address robustness to poisoning attacks.

1. Pa#ern
Recogni-on
and
Applica-ons
Lab
University
of
Cagliari,
Italy
Department
of
Electrical
and
Electronic
Engineering
1.5-class MCSs for Secure Learning against
Evasion Attacks at Test Time
Ba#sta
Biggio1,
Igino
Corona1,
Zhi-­‐min
He2,
Patrick
P.K.
Chan2,
Giorgio
Giacinto1,
Daniel
Yeung2,
Fabio
Roli1
(1)
Dept.
Of
Electrical
and
Electronic
Engineering,
University
of
Cagliari,
Italy
(2)
School
of
Computer
Science
and
Eng.,
South
China
University
of
Technology,
China
Guenzburg,
Germany,
Jun
29
-­‐
Jul
1,
2015
MCS
2015

2.
http://pralab.diee.unica.it
Machine Learning in Adversarial Settings
• Pattern recognition in security applications
– spam filtering, malware detection, biometrics
• Attackers manipulate data to evade detection at test time
2
legitimate
malicious
x1
x2
f(x)
…cheap…
…che4p…
a(x)

3.
http://pralab.diee.unica.it
a(x)
Simplified Risk Analysis under Attack
• Malicious data distribution is not stationary (TR/TS)
3
f
Rts ( f )− Rtr ( f ) = Ex,y l(y, f (a(x)))−l(y, f (x)){ }
x
p(x, y)

4.
http://pralab.diee.unica.it
a(x)
Simplified Risk Analysis under Attack
• Malicious data distribution is not stationary (TR/TS)
4
Rts ( f )− Rts ( f *
) = Ex,y l(y, f (a(x)))−l(y, f *
(a(x))){ }
x
p(x, y)
f *
Be+er
enclosing
legi4mate
data
in
feature
space
may
improve
classifier
security
…
at
the
expense
of
more
false
alarms

5.
http://pralab.diee.unica.it
1.5-class Classification
The Rationale Behind
5
2−class classification
−5 0 5
−5
0
5
1−class classification (legitimate)
−5 0 5
−5
0
5
• 2-­‐class
classifica-on
is
usually
more
accurate
in
the
absence
of
a#ack
• …
but
poten-ally
more
vulnerable
under
a#ack
(not
enclosing
legi-mate
data)
1.5C classification (MCS)
−5 0 5
−5
0
5
1.5-­‐class
classifica4on
aims
at
retaining
high
accuracy
and
security
under
a+ack

6.
http://pralab.diee.unica.it
Secure 1.5-class Classification with MCSs
• Heuristic approach to 1.5-class classification
• Base classifiers
– 2-class classifier: good accuracy in the absence of attacks
– 1-class classifiers: detect anomalous patterns (no support in TR)
• Combiner
– 1-class classifier on legitimate data to improve classifier security
6
data
1C Classifier
(malicious)
Feature
Extraction
malicious
1C Classifier
(legitimate)
2C Classifier
1C Classifier
(legitimate)
legitimate
x
g1(x)
g2(x)
g3(x)
g(x) ≥ t
g(x)
true
false

7.
http://pralab.diee.unica.it
Classifier Security against Evasion Attacks
7
• How to evaluate classifier security
against evasion attacks?
• Attack strategy:
• Non-linear, constrained optimization
– Gradient descent: approximate
solution for smooth functions
• Gradients of g(x) can be analytically
computed in many cases
– SVMs, Neural networks
−2−1.5−1−0.500.51
x
f (x) = sign g(x)( )=
+1, malicious
−1, legitimate
"
#
$
%$
min
x'
g(x')
s.t. d(x, x') ≤ dmax
x '
B. Biggio et al., Evasion attacks against machine learning at test time, ECML-PKDD 2013

8.
http://pralab.diee.unica.it
Computing Descent Directions
Support vector machines
1.5-class MCS
g(x) = αi yik(x,
i
∑ xi )+ b, ∇g(x) = αi yi∇k(x, xi )
i
∑
RBF kernel gradient: ∇k(x,xi
) = −2γ exp −γ || x − xi
||2
{ }(x − xi
)
8
1C Classifier
(malicious)
e
on
malicious
1C Classifier
(legitimate)
2C Classifier
1C Classifier
(legitimate)
legitimate
x
g1(x)
g2(x)
g3(x)
g(x) ≥ t
g(x)
true
false
z(x) = g1
(x), g2
(x), g3
(x)!
"
#
$
T
∇g(x) = −2γ αi
exp −γ z(x)− z(xi
)
2
{ }i
∑ z(x)− z(xi
)( )
Τ δz
δx
B. Biggio et al., Evasion attacks against machine learning at test time, ECML-PKDD 2013

9.
http://pralab.diee.unica.it
Bounding the Adversary’s Knowledge
Limited-knowledge attacks
• Only feature representation and learning algorithm are known
• Surrogate data sampled from the same distribution as the
classifier’s training data
• Classifier’s feedback to label surrogate data
PD(X,Y)data
Surrogate
training data
f(x)
Send queries
Get labels
Learn
surrogate
classifier
f’(x)
9
B. Biggio et al., Evasion attacks against machine learning at test time, ECML-PKDD 2013

10.
http://pralab.diee.unica.it
Experimental Analysis
• Two case studies: spam and PDF malware detection
– Perfect-knowledge (PK) and limited-knowledge (LK) attacks
• Spam data (TREC ’07)
– 25,220 ham and 50,199 spam emails
• we used the first 5,000 emails in chronological order
– 2-class linear SVM, 1-class RBF SVMs
• PDF data
– 2,000 samples collected from the web and public malware
databases (e.g., Contagio)
– 2-class RBF SVM, 1-class RBF SVMs
• Experimental setup
– 50% TR/TS splits, 20% TR for surrogate learning
– 5-fold cross-validation to tune
10
C,γ ∈ 2−10
,2−9
,...,2+10
{ }

11.
http://pralab.diee.unica.it
Spam Filtering
• Features: presence/absence of words
• Attacks: bad word obfuscation / good word insertion
• Attack strategy:
11
Start 2007
with a bang!
Make WBFS
YOUR
PORTFOLIO’s
first winner
of the year
...
start
bang
portfolio
winner
year
...
university
campus
1
1
1
1
1
...
0
0
x
x’
St4rt 2007
with a b4ng!
Make WBFS
YOUR
PORTFOLIO’s
first winner
of the year
... campus
start
bang
portfolio
winner
year
...
university
campus
0
0
1
1
1
...
0
1
min
x'
g(x')
s.t. d(x, x') ≤ dmax
L1-­‐distance
counts
the
number
of
modified
words
in
each
spam

12.
http://pralab.diee.unica.it
Experiments on PDF Malware Detection
• PDF: hierarchy of interconnected objects (keyword/value pairs)
• Attack strategy
– adding up to dmax objects to the PDF
– removing objects may
compromise the PDF file
(and embedded malware code)!
/Type
2
/Page
1
/Encoding
1
…
13
0
obj
<<
/Kids
[
1
0
R
11
0
R
]
/Type
/Page
...
>>
end
obj
17
0
obj
<<
/Type
/Encoding
/Differences
[
0
/C0032
]
>>
endobj
Features:
keyword
count
min
x'
g(x')
s.t. d(x, x') ≤ dmax
x ≤ x'
12

13.
http://pralab.diee.unica.it
Experimental Results
13
0 5 10 15 20 25 30
0
0.2
0.4
0.6
0.8
1
maximum number of modified words
AUC
1%
(PK)
2C SVM
1C SVM (L)
1C SVM (M)
1.5C MCS
0 5 10 15 20 25 30
0
0.2
0.4
0.6
0.8
1
maximum number of modified words
AUC
1%
(LK)
2C SVM
1C SVM (L)
1C SVM (M)
1.5C MCS
Spam
filtering
0 5 10 15 20 25 30
0
0.2
0.4
0.6
0.8
1
maximum number of added keywords
AUC
1%
(PK)
2C SVM
1C SVM (L)
1C SVM (M)
1.5C MCS
0 5 10 15 20 25 30
0
0.2
0.4
0.6
0.8
1
maximum number of added keywords
AUC
1%
(LK)
2C SVM
1C SVM (L)
1C SVM (M)
1.5C MCS
PDF
Malware
Detec-on

14.
http://pralab.diee.unica.it
Conclusions and Future Work
• 1.5-class MCSs
– to improve classifier security under attack (enclosing legitimate data)
– to retain good accuracy in the absence of attack
• General approach
– Suitable for any learning/classification algorithm (in principle)
– No specific assumption on adversarial data manipulation
• Future work
– Formal characterization of trade-off between security and accuracy
– Robustness to poisoning attacks (training data contamination)
14

15.
http://pralab.diee.unica.it
?
Any questions
Thanks
for
your
a#en-on!
15