SlideShare a Scribd company logo
1 of 25
Download to read offline
A A D 5D 7
D 6 0 CD : A
5 : / CD C
. J 4 5 K : 45 :J / 5 5 5J
J 6 5 1 C D K : 1 C D
DDAC 5 : 56C
) - -1 21 ( 2 0
- - - 21
• . . . .
. . .
Figure 4: DeepXplore workflow.
•
•
•
•
•
•
• - -, -, - - - . -
, - - -: . :
• -, : ,- - , , : . - :
, .-: : :-
• - / - . , / - ,-:- : , .:
, - :-
• :- : - :- :-, :- : . , ..- - :
- - : ,-:- : :- : . ,- -,
-
: - - . : - - -
:: ,- - -: : , . - . : .
- -:
• - . . . :.
• . . . : . . . :
• . .
• (: A
• , . : . . .
. : :. : : : . A .
D . . . . A :A : . . . :
• ) . : . . . : . .
D (. : . : . . :
• , , / ,
,
, ,
, / , , , ? ,
• / ,? ,
, , , ( , ) ,
- , , ,
,
• , 1 = ? : :2 1 ? ?1 1 =: 2:= ?1 ? ? ? 1
1? ?:? .1= :2 1 =: ? 1 (
1=1
• D =1 1 =: :2 (
• , D =1 ?1 ? ?
• : ? ? 1 : ? ? 1 :2 1 =:
2:= ?
• ? : 1 ? =1 :
• . 1 ? 1
• )1 1= ?1 ? ? ? =1 1 1 =: : 1= 1
a function that returns the output value of neuron n in
NN for a given test input x. Let t represent the thres
r considering a neuron to be activated. Neuron cove
n be defined as follows.
NCov(T, x) = |{n|8x2T,out(n,x)>t}|
|N|
To demonstrate how neuron coverage is calculate
actice, consider the DNN showed in Figure 3b. The ne
verage (with threshold 0) for the input picture of the
r shown in Figure 3b will be 5/8 = 0.625.
G : .1 G
2 -
• , , , ,
• : , : : , , , : , , ,
.
• , ,
• , , , : , , , , , ,, : ,
(a) DNNs produce same output (b) DNNs produce different output
neuron coverage and gradient for DNNs. Next, we desc
how the testing problem can be formulated as a joint
mization problem. Finally, we provide the algorithm for
optimization using gradients.
5.1 Definitions
Neuron coverage. We define neuron coverage of a set o
inputs as the ratio of the number of unique activated neu
for all test inputs in the set and the total number of neuro
(a) DNNs produce same output (b) DNNs produce different output
Figure 5: Inputs inducing different behaviors in two similar DNNs.
Figure 6: Gradient ascent starting from a seed input and gradually
finding the difference-inducing test inputs.
neu
how
miz
opt
5.1
Ne
inp
for
the
is h
are
are
is a
DN
for
can
pra
cov
car
Figure 4: DeepXplore workflow.
•
•
•
•
•
•
Let Fk(x)[c] be the class probability that Fk predic
be c. We randomly select one neural network Fj (Algo
line 6) and maximize the following objective function
obj1(x) = ⌃i6=jFi(x)[c] 1 · Fj(x)[c]
where 1 is a parameter to balance the objective
between the DNNs Fk6=j that maintain the same class o
as before and the DNN Fj that produce different class o
As Fk21..n are all differentiable, Equation 2 can be
iteratively computed gradient: @x (Algorithm 1 line
8-14 and procedure COMPUTE_OBJ1).
Maximizing neuron coverage. The second objective is to
generate inputs that maximize neuron coverage. We achieve
this by iteratively picking inactivated neurons and perturbing
the input such that output of that neuron goes above the
neuron coverage threshold. Let us assume that we want
to maximize the output of a neuron n, i.e., we want to
maximize obj2(x) = fn(x) such that fn(x) > t, where
t is the neuron coverage threshold, and we write fn(x) as
the function performed by neuron n that takes x (the input
to DNN) as input and produce the output of neuron n (as
defined in Equation 1). We can again leverage the gradient
24: rest = dnns - d
25: loss1 := 0
26: for dnn in rest do
27: loss1 += dnnc(x) //confidence score of x being in class c
28: loss2 := dc(x) //d’s output confidence score of x being in class c
29: return (loss1 - 1·loss2)
30: procedure COMPUTE_OBJ2(x, dnns, cov_tracker)
31: loss := 0
32: for dnn 2 dnns do
33: select a neuron n inactivated so far using cov_tracker
34: loss += n(x) //the neuron n’s output when x is the dnn’s inpu
35: return loss
Joint optimization. We jointly maximize obj1 and fn d
scribed above and maximize the following function:
objjoint = (⌃i6=jFi(x)[c] 1Fj(x)[c]) + 2 · fn(x)
• 2 2 2 5 2 . . 2 . 2 . 250 0 2.5 ,.5
• 0 2.5 2 2 2. 2 25 , 5 25 . 0
2 0. 2 . . 2 25 ( ) . . . .
Figure 4: DeepXplore workflow.
• 2HKGAHAIN NE I
• .4 BL HAQ L AI LBG Q I 3AL
• 46-
• KALEHAIN AIPEL IHAIN
• 4EIO G KN K LOIIEIC : OINO
• IA 2INAG E 18 01T KL A L QEND LA
• 0, B HAH L
• 2.2 0 07:
Table 1: Details of the DNNs and datasets used to evaluate DeepXplore
Dataset
Dataset
Description
DNN
Description
DNN Name
No. of
Neurons
Architecture
Reported
Acc.
Our
Acc.
MNIST Hand-written digits LeNet variations
MNI_C1 52 LeNet-1, LeCun et al. [26, 28] 98.3% 98.33%
MNI_C2 148 LeNet-4, LeCun et al. [26, 28] 98.9% 98.59%
MNI_C3 268 LeNet-5, LeCun et al. [26, 28] 99.05% 98.96%
Imagenet General images
State-of-the-art
image classifiers
from ILSVRC
IMG_C1 14,888 VGG-16, Simonyan et al. [45] 92.6%⇤⇤ 92.6%⇤⇤
IMG_C2 16,168 VGG-19, Simonyan et al. [45] 92.7%⇤⇤ 92.7%⇤⇤
IMG_C3 94,059 ResNet50, He et al. [22] 96.43%⇤⇤ 96.43%⇤⇤
Driving Driving video frames
Nvidia DAVE
self-driving systems
DRV_C1 1,560 Dave-orig [4], Bojarski et al. [5] N/A 99.91%#
DRV_C2 1,560 Dave-norminit [55] N/A 99.94%#
DRV_C3 844 Dave-dropout [11] N/A 99.96%#
Contagio PDFs
PDF
malware detectors
PDF_C1 402 <200, 200>+ 98.5% 96.15%
PDF_C2 602 <200, 200, 200>+ 98.5% 96.25%
PDF_C3 802 <200, 200, 200, 200>+ 98.5% 96.47%
Drebin Android apps
Android app
malware detectors
APP_C1 402 <200, 200>+, Grosse et al. [21] 98.92% 98.6%
APP_C2 102 <50, 50>+, Grosse et al. [21] 96.79% 96.82%
APP_C3 212 <200, 10>+, Grosse et al. [21] 92.97% 92.66%
** top-5 test accuracy; we exactly match the reported performance as we use the pretrained networks
# we report 1-MSE (Mean Squared Error) as the accuracy because steering angle is a continuous value
+ <x,y,...> denotes three hidden layers with x neurons in first layer, y neurons in second layer and so on
- accuracy using SVM as reported by Šrndic et al. [56]
- - -
• , 0 , ( , 0(0 ,
,1.. 1 . 1 3 1 / ,1.. 1
1/ 1 / .. . 1 3 1 / ,1.. 1 1 1 . 31/ 1 /
3 1 1 /3 33 /3 . 1 3 1 / 2 1 33
3 21 / . 1 /
3 1 3 1 3 1 3 2 /3 . 1 3 1 / .. . ,
3 .
• 0
• ) , 0 1 , 1 0 10, 0 0 , 0 0 , , 10
1 0 31 1 , 1 .. ,
• ) , , 10, 0 , 0 0 10, 0
, 1 1. .13 31 1 . 1 31
/ 3 2 , , 1 ..1 1 1 1
• ,
• 1 1 . 1 1 3 , , 3 3
1 3 ,1 1 ,1.1 3
• . . . . .
.
( ) (
Different lighting conditions:
all: right all: right all: right all: 1 all: 3 all: 5 all: diver all: cheeseburger all: flamingo
DRV_C1: left DRV_C2: left DRV_C3: left MNI_C1: 8 MNI_C2: 5 MNI_C3: 7 IMG_C1: ski IMG_C2: icecream IMG_C3: goldfish
Occlusion with a single small rectangle:
all: right all: right all: left all: 5 all: 7 all: 9 all: cauliflower all: dhole all: hay
all: right all: right all: right all: 1 all: 3 all: 5 all: diver all: cheeseburger all: flamingo
DRV_C1: left DRV_C2: left DRV_C3: left MNI_C1: 8 MNI_C2: 5 MNI_C3: 7 IMG_C1: ski IMG_C2: icecream IMG_C3: goldfish
Occlusion with a single small rectangle:
all: right all: right all: left all: 5 all: 7 all: 9 all: cauliflower all: dhole all: hay
DRV_C1: left DRV_C2: left DRV_C3: right MNI_C1: 3 MNI_C2: 4 MNI_C3: 2 IMG_C1:carbonara IMG_C2:umbrella IMG_C3:stupa
Occlusion with multiple tiny black rectangles:
all: left all: left all: left all: 1 all: 5 all: 7 all: castle all: cock all: groom
( ) (
( ) (
all: right all: right all: left all: 5 all: 7 all: 9 all: cauliflower all: dhole all: hay
DRV_C1: left DRV_C2: left DRV_C3: right MNI_C1: 3 MNI_C2: 4 MNI_C3: 2 IMG_C1:carbonara IMG_C2:umbrella IMG_C3:stupa
Occlusion with multiple tiny black rectangles:
all: left all: left all: left all: 1 all: 5 all: 7 all: castle all: cock all: groom
DRV_C1: right DRV_C2: right DRV_C3: right MNI_C1: 2 MNI_C2: 4 MNI_C3: 4 IMG_C1: beacon IMG_C2: hen IMG_C3: vestment
Figure 7: Odd rows show the seed test inputs and even rows show the difference-inducing test inputs generated by DeepXplore. The left three
columns show inputs for self-driving car, the middle three are for MNIST, and the right three are for ImageNet.
Table 2: The top-3 most in(de)cremented features for generating two
sample malware inputs which PDF classifiers (Contagio) incorrectly
mark as benign.
Table 3: The features added to the manifest file by DeepXplore for
generating two sample malware inputs which Android app classifiers
(Drebin) incorrectly mark as benign.
• 0 .4 . 4 1 ? 0 3 . 4 ?
0 3 ) ? ? 4(
• .
• % . 1 ? ? % 0 1 0 .4 3 . )
? 0 .4 4 . . 3 ? )
? ? .? 0 1 0 .4 ? . 1 ? ? .0 3 0 ? ? 0?
• , 0 .4 ? 1 ? . 1 .0 1 33 ? )
• . .. . .
• ? 0 4 3 . 0 . . .0? .? 1 ? . ?
0 4 3 1 33 ? 0 .
•
• ) , , 4 . % , %% 2,
, 3 , , , 3,4 3
• (3 2 3 2
•
• , 4 4
(a) MNIST (b) ImageNet (c) Driving (d) Contagio
Figure 8: The neuron coverage achieved by the same number of inputs (1% of the original test set) produced by DeepX
testing [18], and random selection from the original test set. The plots show the change in neuron coverage for all three metho
t (defined in Section 5) increases. DeepXplore, on average, covers 34.4% and 33.2% more neurons than random testing and a
Table 6: Total time taken by DeepXplore to achieve 100% neuron
coverage for different DNNs averaged over 10 runs.
C1 C2 C3
MNIST 6.6 s 6.8 s 7.6 s
ImageNet 43.6 s 45.3 s 42.7 s
Driving 11.7 s 12.3 s 9.8 s
Contagio 31.1 s 29.7 s 23.2 s
Drebin 180.2 s 196.4 s 152.9 s
Table 9: The variation in DeepXplore runtime (
generating the first difference-inducing input for
with different 2, a parameter in Equation 3. Hig
cate higher priority for increasing coverage. All n
over 10 runs. The fastest times for each dataset
gray.
2 = 0.5 2 = 1 2 = 2
MNIST 0.19 s 0.22 s 0.23 s
ImageNet 1.55 s 1.6 s 1.62 s
•
• .
•
• . - 5 .5. .. .
• . , . . . 5 1 5 ,1 ,
. . . , .- .
• - . ,. 5 .5 1
5 5 . 1 . , . . - 1. 5 . .
. . .1
• .- . 1. .- .
- - . - - 1 - . . .1
1. . .-
. .
• I- C = F C = = C C = D
) / ( : C
• =C D E E C = ,,
• CC FFF = C : C F E C = = D
= CF :
. C = E = C C C = C
E = C =
D=C
C DC C = F
C D
- C = = = D=C D
C = C
= C DC C
D C D
.
• -
• - - - - - -
• - - - -
• - -
• - - - - -
• - -
• - - - - - - -
- -
• 1:7 B C7 :7 C BBC : : C7 7 9
• :7 C BBC : C7 7 7 C 7 7 C C
6 /C 9C 4 7 . 9 C 7C /.
• C 76 C7 7 C : C 7 7 97 7C C :
7 BC7 4
• : B 6I 6 C9 ) 012 (
• : B 6 B ) (

More Related Content

What's hot

Metaheuristic Tuning of Type-II Fuzzy Inference System for Data Mining
Metaheuristic Tuning of Type-II Fuzzy Inference System for Data MiningMetaheuristic Tuning of Type-II Fuzzy Inference System for Data Mining
Metaheuristic Tuning of Type-II Fuzzy Inference System for Data MiningVarun Ojha
 
Solutions manual for calculus an applied approach brief international metric ...
Solutions manual for calculus an applied approach brief international metric ...Solutions manual for calculus an applied approach brief international metric ...
Solutions manual for calculus an applied approach brief international metric ...Larson612
 
โครงงานคอม
โครงงานคอมโครงงานคอม
โครงงานคอมnoeiinoii
 
Precalculus 1 chapter 2
Precalculus 1 chapter 2Precalculus 1 chapter 2
Precalculus 1 chapter 2oreves
 
Arna Friend Controls II Final
Arna Friend Controls II FinalArna Friend Controls II Final
Arna Friend Controls II FinalArna Friend
 
The International Journal of Engineering and Science (IJES)
The International Journal of Engineering and Science (IJES)The International Journal of Engineering and Science (IJES)
The International Journal of Engineering and Science (IJES)theijes
 
2020 1학기 정기스터디 2주차
2020 1학기 정기스터디 2주차2020 1학기 정기스터디 2주차
2020 1학기 정기스터디 2주차Moonki Choi
 
Calculus 10th edition anton solutions manual
Calculus 10th edition anton solutions manualCalculus 10th edition anton solutions manual
Calculus 10th edition anton solutions manualReece1334
 
Image Cryptography and Steganography
Image Cryptography and SteganographyImage Cryptography and Steganography
Image Cryptography and SteganographyMohammad Amin Amjadi
 
เลขยกกำลังชุด 2
เลขยกกำลังชุด 2เลขยกกำลังชุด 2
เลขยกกำลังชุด 2kanjana2536
 
Mm chap08 -_lossy_compression_algorithms
Mm chap08 -_lossy_compression_algorithmsMm chap08 -_lossy_compression_algorithms
Mm chap08 -_lossy_compression_algorithmsEellekwameowusu
 
Microsoft Word Hw#3
Microsoft Word   Hw#3Microsoft Word   Hw#3
Microsoft Word Hw#3kkkseld
 
Application of recursive perturbation approach for multimodal optimization
Application of recursive perturbation approach for multimodal optimizationApplication of recursive perturbation approach for multimodal optimization
Application of recursive perturbation approach for multimodal optimizationPranamesh Chakraborty
 

What's hot (17)

Metaheuristic Tuning of Type-II Fuzzy Inference System for Data Mining
Metaheuristic Tuning of Type-II Fuzzy Inference System for Data MiningMetaheuristic Tuning of Type-II Fuzzy Inference System for Data Mining
Metaheuristic Tuning of Type-II Fuzzy Inference System for Data Mining
 
Solutions manual for calculus an applied approach brief international metric ...
Solutions manual for calculus an applied approach brief international metric ...Solutions manual for calculus an applied approach brief international metric ...
Solutions manual for calculus an applied approach brief international metric ...
 
Gilat_ch02.pdf
Gilat_ch02.pdfGilat_ch02.pdf
Gilat_ch02.pdf
 
jammer_placement
jammer_placementjammer_placement
jammer_placement
 
โครงงานคอม
โครงงานคอมโครงงานคอม
โครงงานคอม
 
Gilat_ch03.pdf
Gilat_ch03.pdfGilat_ch03.pdf
Gilat_ch03.pdf
 
Precalculus 1 chapter 2
Precalculus 1 chapter 2Precalculus 1 chapter 2
Precalculus 1 chapter 2
 
Arna Friend Controls II Final
Arna Friend Controls II FinalArna Friend Controls II Final
Arna Friend Controls II Final
 
The International Journal of Engineering and Science (IJES)
The International Journal of Engineering and Science (IJES)The International Journal of Engineering and Science (IJES)
The International Journal of Engineering and Science (IJES)
 
2020 1학기 정기스터디 2주차
2020 1학기 정기스터디 2주차2020 1학기 정기스터디 2주차
2020 1학기 정기스터디 2주차
 
Gate-Cs 1993
Gate-Cs 1993Gate-Cs 1993
Gate-Cs 1993
 
Calculus 10th edition anton solutions manual
Calculus 10th edition anton solutions manualCalculus 10th edition anton solutions manual
Calculus 10th edition anton solutions manual
 
Image Cryptography and Steganography
Image Cryptography and SteganographyImage Cryptography and Steganography
Image Cryptography and Steganography
 
เลขยกกำลังชุด 2
เลขยกกำลังชุด 2เลขยกกำลังชุด 2
เลขยกกำลังชุด 2
 
Mm chap08 -_lossy_compression_algorithms
Mm chap08 -_lossy_compression_algorithmsMm chap08 -_lossy_compression_algorithms
Mm chap08 -_lossy_compression_algorithms
 
Microsoft Word Hw#3
Microsoft Word   Hw#3Microsoft Word   Hw#3
Microsoft Word Hw#3
 
Application of recursive perturbation approach for multimodal optimization
Application of recursive perturbation approach for multimodal optimizationApplication of recursive perturbation approach for multimodal optimization
Application of recursive perturbation approach for multimodal optimization
 

Similar to DeepXplore: Automated Whitebox Testing of Deep Learning

03 image transformations_i
03 image transformations_i03 image transformations_i
03 image transformations_iankit_ppt
 
Introduction to Applied Machine Learning
Introduction to Applied Machine LearningIntroduction to Applied Machine Learning
Introduction to Applied Machine LearningSheilaJimenezMorejon
 
Digit recognizer by convolutional neural network
Digit recognizer by convolutional neural networkDigit recognizer by convolutional neural network
Digit recognizer by convolutional neural networkDing Li
 
5707_10_auto-encoder.pptx
5707_10_auto-encoder.pptx5707_10_auto-encoder.pptx
5707_10_auto-encoder.pptxSidoriOne
 
CD504 CGM_Lab Manual_004e08d3838702ed11fc6d03cc82f7be.pdf
CD504 CGM_Lab Manual_004e08d3838702ed11fc6d03cc82f7be.pdfCD504 CGM_Lab Manual_004e08d3838702ed11fc6d03cc82f7be.pdf
CD504 CGM_Lab Manual_004e08d3838702ed11fc6d03cc82f7be.pdfRajJain516913
 
Computer Graphics Unit 1
Computer Graphics Unit 1Computer Graphics Unit 1
Computer Graphics Unit 1aravindangc
 
Image Texture Analysis
Image Texture AnalysisImage Texture Analysis
Image Texture Analysislalitxp
 
Introduction to Neural Networks and Deep Learning
Introduction to Neural Networks and Deep LearningIntroduction to Neural Networks and Deep Learning
Introduction to Neural Networks and Deep LearningVahid Mirjalili
 
ADAPTIVE FUZZY KERNEL CLUSTERING ALGORITHM
ADAPTIVE FUZZY KERNEL CLUSTERING ALGORITHMADAPTIVE FUZZY KERNEL CLUSTERING ALGORITHM
ADAPTIVE FUZZY KERNEL CLUSTERING ALGORITHMWireilla
 
ADAPTIVE FUZZY KERNEL CLUSTERING ALGORITHM
ADAPTIVE FUZZY KERNEL CLUSTERING ALGORITHMADAPTIVE FUZZY KERNEL CLUSTERING ALGORITHM
ADAPTIVE FUZZY KERNEL CLUSTERING ALGORITHMijfls
 
NeuralArt 電腦作畫
NeuralArt 電腦作畫NeuralArt 電腦作畫
NeuralArt 電腦作畫Mark Chang
 
3D Brain Image Segmentation Model using Deep Learning and Hidden Markov Rando...
3D Brain Image Segmentation Model using Deep Learning and Hidden Markov Rando...3D Brain Image Segmentation Model using Deep Learning and Hidden Markov Rando...
3D Brain Image Segmentation Model using Deep Learning and Hidden Markov Rando...EL-Hachemi Guerrout
 
NIPS2017 Few-shot Learning and Graph Convolution
NIPS2017 Few-shot Learning and Graph ConvolutionNIPS2017 Few-shot Learning and Graph Convolution
NIPS2017 Few-shot Learning and Graph ConvolutionKazuki Fujikawa
 
Matlab 3
Matlab 3Matlab 3
Matlab 3asguna
 
Sample Exam Questions on Python for revision
Sample Exam Questions on Python for revisionSample Exam Questions on Python for revision
Sample Exam Questions on Python for revisionafsheenfaiq2
 
K-means Clustering Algorithm with Matlab Source code
K-means Clustering Algorithm with Matlab Source codeK-means Clustering Algorithm with Matlab Source code
K-means Clustering Algorithm with Matlab Source codegokulprasath06
 
A practical Introduction to Machine(s) Learning
A practical Introduction to Machine(s) LearningA practical Introduction to Machine(s) Learning
A practical Introduction to Machine(s) LearningBruno Gonçalves
 
Convolutional Neural Network
Convolutional Neural NetworkConvolutional Neural Network
Convolutional Neural NetworkJun Young Park
 

Similar to DeepXplore: Automated Whitebox Testing of Deep Learning (20)

03 image transformations_i
03 image transformations_i03 image transformations_i
03 image transformations_i
 
Introduction to Applied Machine Learning
Introduction to Applied Machine LearningIntroduction to Applied Machine Learning
Introduction to Applied Machine Learning
 
Digit recognizer by convolutional neural network
Digit recognizer by convolutional neural networkDigit recognizer by convolutional neural network
Digit recognizer by convolutional neural network
 
Eye deep
Eye deepEye deep
Eye deep
 
5707_10_auto-encoder.pptx
5707_10_auto-encoder.pptx5707_10_auto-encoder.pptx
5707_10_auto-encoder.pptx
 
CD504 CGM_Lab Manual_004e08d3838702ed11fc6d03cc82f7be.pdf
CD504 CGM_Lab Manual_004e08d3838702ed11fc6d03cc82f7be.pdfCD504 CGM_Lab Manual_004e08d3838702ed11fc6d03cc82f7be.pdf
CD504 CGM_Lab Manual_004e08d3838702ed11fc6d03cc82f7be.pdf
 
Computer Graphics Unit 1
Computer Graphics Unit 1Computer Graphics Unit 1
Computer Graphics Unit 1
 
Image Texture Analysis
Image Texture AnalysisImage Texture Analysis
Image Texture Analysis
 
Introduction to Neural Networks and Deep Learning
Introduction to Neural Networks and Deep LearningIntroduction to Neural Networks and Deep Learning
Introduction to Neural Networks and Deep Learning
 
ADAPTIVE FUZZY KERNEL CLUSTERING ALGORITHM
ADAPTIVE FUZZY KERNEL CLUSTERING ALGORITHMADAPTIVE FUZZY KERNEL CLUSTERING ALGORITHM
ADAPTIVE FUZZY KERNEL CLUSTERING ALGORITHM
 
ADAPTIVE FUZZY KERNEL CLUSTERING ALGORITHM
ADAPTIVE FUZZY KERNEL CLUSTERING ALGORITHMADAPTIVE FUZZY KERNEL CLUSTERING ALGORITHM
ADAPTIVE FUZZY KERNEL CLUSTERING ALGORITHM
 
NeuralArt 電腦作畫
NeuralArt 電腦作畫NeuralArt 電腦作畫
NeuralArt 電腦作畫
 
3D Brain Image Segmentation Model using Deep Learning and Hidden Markov Rando...
3D Brain Image Segmentation Model using Deep Learning and Hidden Markov Rando...3D Brain Image Segmentation Model using Deep Learning and Hidden Markov Rando...
3D Brain Image Segmentation Model using Deep Learning and Hidden Markov Rando...
 
Cgm Lab Manual
Cgm Lab ManualCgm Lab Manual
Cgm Lab Manual
 
NIPS2017 Few-shot Learning and Graph Convolution
NIPS2017 Few-shot Learning and Graph ConvolutionNIPS2017 Few-shot Learning and Graph Convolution
NIPS2017 Few-shot Learning and Graph Convolution
 
Matlab 3
Matlab 3Matlab 3
Matlab 3
 
Sample Exam Questions on Python for revision
Sample Exam Questions on Python for revisionSample Exam Questions on Python for revision
Sample Exam Questions on Python for revision
 
K-means Clustering Algorithm with Matlab Source code
K-means Clustering Algorithm with Matlab Source codeK-means Clustering Algorithm with Matlab Source code
K-means Clustering Algorithm with Matlab Source code
 
A practical Introduction to Machine(s) Learning
A practical Introduction to Machine(s) LearningA practical Introduction to Machine(s) Learning
A practical Introduction to Machine(s) Learning
 
Convolutional Neural Network
Convolutional Neural NetworkConvolutional Neural Network
Convolutional Neural Network
 

More from Masahiro Sakai

Towards formal verification of neural networks
Towards formal verification of neural networksTowards formal verification of neural networks
Towards formal verification of neural networksMasahiro Sakai
 
関数プログラマから見たPythonと機械学習
関数プログラマから見たPythonと機械学習関数プログラマから見たPythonと機械学習
関数プログラマから見たPythonと機械学習Masahiro Sakai
 
Writing a SAT solver as a hobby project
Writing a SAT solver as a hobby projectWriting a SAT solver as a hobby project
Writing a SAT solver as a hobby projectMasahiro Sakai
 
RClassify: Classifying Race Conditions in Web Applications via Deterministic ...
RClassify: Classifying Race Conditions in Web Applications via Deterministic ...RClassify: Classifying Race Conditions in Web Applications via Deterministic ...
RClassify: Classifying Race Conditions in Web Applications via Deterministic ...Masahiro Sakai
 
SAT/SMT solving in Haskell
SAT/SMT solving in HaskellSAT/SMT solving in Haskell
SAT/SMT solving in HaskellMasahiro Sakai
 
SAT/SMTソルバの仕組み
SAT/SMTソルバの仕組みSAT/SMTソルバの仕組み
SAT/SMTソルバの仕組みMasahiro Sakai
 
Introduction to Max-SAT and Max-SAT Evaluation
Introduction to Max-SAT and Max-SAT EvaluationIntroduction to Max-SAT and Max-SAT Evaluation
Introduction to Max-SAT and Max-SAT EvaluationMasahiro Sakai
 
Aluminum: Principled Scenario Exploration through Minimality
Aluminum: Principled Scenario Exploration through MinimalityAluminum: Principled Scenario Exploration through Minimality
Aluminum: Principled Scenario Exploration through MinimalityMasahiro Sakai
 
代数的実数とCADの実装紹介
代数的実数とCADの実装紹介代数的実数とCADの実装紹介
代数的実数とCADの実装紹介Masahiro Sakai
 
How a CDCL SAT solver works
How a CDCL SAT solver worksHow a CDCL SAT solver works
How a CDCL SAT solver worksMasahiro Sakai
 
萩野服部研究室 スキー合宿 2012 自己紹介(酒井)
萩野服部研究室 スキー合宿 2012 自己紹介(酒井)萩野服部研究室 スキー合宿 2012 自己紹介(酒井)
萩野服部研究室 スキー合宿 2012 自己紹介(酒井)Masahiro Sakai
 
自動定理証明の紹介
自動定理証明の紹介自動定理証明の紹介
自動定理証明の紹介Masahiro Sakai
 
“Adoption and Focus: Practical Linear Types for Imperative Programming”他の紹介@P...
“Adoption and Focus: Practical Linear Types for Imperative Programming”他の紹介@P...“Adoption and Focus: Practical Linear Types for Imperative Programming”他の紹介@P...
“Adoption and Focus: Practical Linear Types for Imperative Programming”他の紹介@P...Masahiro Sakai
 
“Design and Implementation of Generics for the .NET Common Language Runtime”他...
“Design and Implementation of Generics for the .NET Common Language Runtime”他...“Design and Implementation of Generics for the .NET Common Language Runtime”他...
“Design and Implementation of Generics for the .NET Common Language Runtime”他...Masahiro Sakai
 
Relaxed Dependency Analysis
Relaxed Dependency AnalysisRelaxed Dependency Analysis
Relaxed Dependency AnalysisMasahiro Sakai
 
“Symbolic bounds analysis of pointers, array indices, and accessed memory reg...
“Symbolic bounds analysis of pointers, array indices, and accessed memory reg...“Symbolic bounds analysis of pointers, array indices, and accessed memory reg...
“Symbolic bounds analysis of pointers, array indices, and accessed memory reg...Masahiro Sakai
 
自然言語をラムダ式で解釈する体系PTQのHaskell実装
自然言語をラムダ式で解釈する体系PTQのHaskell実装自然言語をラムダ式で解釈する体系PTQのHaskell実装
自然言語をラムダ式で解釈する体系PTQのHaskell実装Masahiro Sakai
 
Whole Program Paths 等の紹介@PLDIr#3
Whole Program Paths 等の紹介@PLDIr#3Whole Program Paths 等の紹介@PLDIr#3
Whole Program Paths 等の紹介@PLDIr#3Masahiro Sakai
 

More from Masahiro Sakai (20)

Towards formal verification of neural networks
Towards formal verification of neural networksTowards formal verification of neural networks
Towards formal verification of neural networks
 
関数プログラマから見たPythonと機械学習
関数プログラマから見たPythonと機械学習関数プログラマから見たPythonと機械学習
関数プログラマから見たPythonと機械学習
 
Writing a SAT solver as a hobby project
Writing a SAT solver as a hobby projectWriting a SAT solver as a hobby project
Writing a SAT solver as a hobby project
 
RClassify: Classifying Race Conditions in Web Applications via Deterministic ...
RClassify: Classifying Race Conditions in Web Applications via Deterministic ...RClassify: Classifying Race Conditions in Web Applications via Deterministic ...
RClassify: Classifying Race Conditions in Web Applications via Deterministic ...
 
ゼロピッチ: MOOC
ゼロピッチ: MOOCゼロピッチ: MOOC
ゼロピッチ: MOOC
 
SAT/SMT solving in Haskell
SAT/SMT solving in HaskellSAT/SMT solving in Haskell
SAT/SMT solving in Haskell
 
SAT/SMTソルバの仕組み
SAT/SMTソルバの仕組みSAT/SMTソルバの仕組み
SAT/SMTソルバの仕組み
 
Introduction to Max-SAT and Max-SAT Evaluation
Introduction to Max-SAT and Max-SAT EvaluationIntroduction to Max-SAT and Max-SAT Evaluation
Introduction to Max-SAT and Max-SAT Evaluation
 
Aluminum: Principled Scenario Exploration through Minimality
Aluminum: Principled Scenario Exploration through MinimalityAluminum: Principled Scenario Exploration through Minimality
Aluminum: Principled Scenario Exploration through Minimality
 
代数的実数とCADの実装紹介
代数的実数とCADの実装紹介代数的実数とCADの実装紹介
代数的実数とCADの実装紹介
 
How a CDCL SAT solver works
How a CDCL SAT solver worksHow a CDCL SAT solver works
How a CDCL SAT solver works
 
Omega test and beyond
Omega test and beyondOmega test and beyond
Omega test and beyond
 
萩野服部研究室 スキー合宿 2012 自己紹介(酒井)
萩野服部研究室 スキー合宿 2012 自己紹介(酒井)萩野服部研究室 スキー合宿 2012 自己紹介(酒井)
萩野服部研究室 スキー合宿 2012 自己紹介(酒井)
 
自動定理証明の紹介
自動定理証明の紹介自動定理証明の紹介
自動定理証明の紹介
 
“Adoption and Focus: Practical Linear Types for Imperative Programming”他の紹介@P...
“Adoption and Focus: Practical Linear Types for Imperative Programming”他の紹介@P...“Adoption and Focus: Practical Linear Types for Imperative Programming”他の紹介@P...
“Adoption and Focus: Practical Linear Types for Imperative Programming”他の紹介@P...
 
“Design and Implementation of Generics for the .NET Common Language Runtime”他...
“Design and Implementation of Generics for the .NET Common Language Runtime”他...“Design and Implementation of Generics for the .NET Common Language Runtime”他...
“Design and Implementation of Generics for the .NET Common Language Runtime”他...
 
Relaxed Dependency Analysis
Relaxed Dependency AnalysisRelaxed Dependency Analysis
Relaxed Dependency Analysis
 
“Symbolic bounds analysis of pointers, array indices, and accessed memory reg...
“Symbolic bounds analysis of pointers, array indices, and accessed memory reg...“Symbolic bounds analysis of pointers, array indices, and accessed memory reg...
“Symbolic bounds analysis of pointers, array indices, and accessed memory reg...
 
自然言語をラムダ式で解釈する体系PTQのHaskell実装
自然言語をラムダ式で解釈する体系PTQのHaskell実装自然言語をラムダ式で解釈する体系PTQのHaskell実装
自然言語をラムダ式で解釈する体系PTQのHaskell実装
 
Whole Program Paths 等の紹介@PLDIr#3
Whole Program Paths 等の紹介@PLDIr#3Whole Program Paths 等の紹介@PLDIr#3
Whole Program Paths 等の紹介@PLDIr#3
 

Recently uploaded

Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demoHarshalMandlekar2
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...AliaaTarek5
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dashnarutouzumaki53779
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 

Recently uploaded (20)

Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demo
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dash
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 

DeepXplore: Automated Whitebox Testing of Deep Learning

  • 1. A A D 5D 7 D 6 0 CD : A 5 : / CD C . J 4 5 K : 45 :J / 5 5 5J J 6 5 1 C D K : 1 C D DDAC 5 : 56C ) - -1 21 ( 2 0 - - - 21
  • 2. • . . . . . . . Figure 4: DeepXplore workflow.
  • 4. • - -, -, - - - . - , - - -: . : • -, : ,- - , , : . - : , .-: : :- • - / - . , / - ,-:- : , .: , - :- • :- : - :- :-, :- : . , ..- - : - - : ,-:- : :- : . ,- -, - : - - . : - - - :: ,- - -: : , . - . : . - -:
  • 5. • - . . . :. • . . . : . . . : • . . • (: A • , . : . . . . : :. : : : . A . D . . . . A :A : . . . : • ) . : . . . : . . D (. : . : . . :
  • 6. • , , / , , , , , / , , , ? , • / ,? , , , , ( , ) , - , , , ,
  • 7. • , 1 = ? : :2 1 ? ?1 1 =: 2:= ?1 ? ? ? 1 1? ?:? .1= :2 1 =: ? 1 ( 1=1 • D =1 1 =: :2 ( • , D =1 ?1 ? ? • : ? ? 1 : ? ? 1 :2 1 =: 2:= ? • ? : 1 ? =1 : • . 1 ? 1 • )1 1= ?1 ? ? ? =1 1 1 =: : 1= 1 a function that returns the output value of neuron n in NN for a given test input x. Let t represent the thres r considering a neuron to be activated. Neuron cove n be defined as follows. NCov(T, x) = |{n|8x2T,out(n,x)>t}| |N| To demonstrate how neuron coverage is calculate actice, consider the DNN showed in Figure 3b. The ne verage (with threshold 0) for the input picture of the r shown in Figure 3b will be 5/8 = 0.625. G : .1 G
  • 8. 2 - • , , , , • : , : : , , , : , , , . • , , • , , , : , , , , , ,, : , (a) DNNs produce same output (b) DNNs produce different output neuron coverage and gradient for DNNs. Next, we desc how the testing problem can be formulated as a joint mization problem. Finally, we provide the algorithm for optimization using gradients. 5.1 Definitions Neuron coverage. We define neuron coverage of a set o inputs as the ratio of the number of unique activated neu for all test inputs in the set and the total number of neuro (a) DNNs produce same output (b) DNNs produce different output Figure 5: Inputs inducing different behaviors in two similar DNNs. Figure 6: Gradient ascent starting from a seed input and gradually finding the difference-inducing test inputs. neu how miz opt 5.1 Ne inp for the is h are are is a DN for can pra cov car
  • 9. Figure 4: DeepXplore workflow.
  • 10. • • • • • • Let Fk(x)[c] be the class probability that Fk predic be c. We randomly select one neural network Fj (Algo line 6) and maximize the following objective function obj1(x) = ⌃i6=jFi(x)[c] 1 · Fj(x)[c] where 1 is a parameter to balance the objective between the DNNs Fk6=j that maintain the same class o as before and the DNN Fj that produce different class o As Fk21..n are all differentiable, Equation 2 can be iteratively computed gradient: @x (Algorithm 1 line 8-14 and procedure COMPUTE_OBJ1). Maximizing neuron coverage. The second objective is to generate inputs that maximize neuron coverage. We achieve this by iteratively picking inactivated neurons and perturbing the input such that output of that neuron goes above the neuron coverage threshold. Let us assume that we want to maximize the output of a neuron n, i.e., we want to maximize obj2(x) = fn(x) such that fn(x) > t, where t is the neuron coverage threshold, and we write fn(x) as the function performed by neuron n that takes x (the input to DNN) as input and produce the output of neuron n (as defined in Equation 1). We can again leverage the gradient 24: rest = dnns - d 25: loss1 := 0 26: for dnn in rest do 27: loss1 += dnnc(x) //confidence score of x being in class c 28: loss2 := dc(x) //d’s output confidence score of x being in class c 29: return (loss1 - 1·loss2) 30: procedure COMPUTE_OBJ2(x, dnns, cov_tracker) 31: loss := 0 32: for dnn 2 dnns do 33: select a neuron n inactivated so far using cov_tracker 34: loss += n(x) //the neuron n’s output when x is the dnn’s inpu 35: return loss Joint optimization. We jointly maximize obj1 and fn d scribed above and maximize the following function: objjoint = (⌃i6=jFi(x)[c] 1Fj(x)[c]) + 2 · fn(x)
  • 11. • 2 2 2 5 2 . . 2 . 2 . 250 0 2.5 ,.5 • 0 2.5 2 2 2. 2 25 , 5 25 . 0 2 0. 2 . . 2 25 ( ) . . . . Figure 4: DeepXplore workflow.
  • 12. • 2HKGAHAIN NE I • .4 BL HAQ L AI LBG Q I 3AL • 46- • KALEHAIN AIPEL IHAIN • 4EIO G KN K LOIIEIC : OINO • IA 2INAG E 18 01T KL A L QEND LA • 0, B HAH L • 2.2 0 07:
  • 13. Table 1: Details of the DNNs and datasets used to evaluate DeepXplore Dataset Dataset Description DNN Description DNN Name No. of Neurons Architecture Reported Acc. Our Acc. MNIST Hand-written digits LeNet variations MNI_C1 52 LeNet-1, LeCun et al. [26, 28] 98.3% 98.33% MNI_C2 148 LeNet-4, LeCun et al. [26, 28] 98.9% 98.59% MNI_C3 268 LeNet-5, LeCun et al. [26, 28] 99.05% 98.96% Imagenet General images State-of-the-art image classifiers from ILSVRC IMG_C1 14,888 VGG-16, Simonyan et al. [45] 92.6%⇤⇤ 92.6%⇤⇤ IMG_C2 16,168 VGG-19, Simonyan et al. [45] 92.7%⇤⇤ 92.7%⇤⇤ IMG_C3 94,059 ResNet50, He et al. [22] 96.43%⇤⇤ 96.43%⇤⇤ Driving Driving video frames Nvidia DAVE self-driving systems DRV_C1 1,560 Dave-orig [4], Bojarski et al. [5] N/A 99.91%# DRV_C2 1,560 Dave-norminit [55] N/A 99.94%# DRV_C3 844 Dave-dropout [11] N/A 99.96%# Contagio PDFs PDF malware detectors PDF_C1 402 <200, 200>+ 98.5% 96.15% PDF_C2 602 <200, 200, 200>+ 98.5% 96.25% PDF_C3 802 <200, 200, 200, 200>+ 98.5% 96.47% Drebin Android apps Android app malware detectors APP_C1 402 <200, 200>+, Grosse et al. [21] 98.92% 98.6% APP_C2 102 <50, 50>+, Grosse et al. [21] 96.79% 96.82% APP_C3 212 <200, 10>+, Grosse et al. [21] 92.97% 92.66% ** top-5 test accuracy; we exactly match the reported performance as we use the pretrained networks # we report 1-MSE (Mean Squared Error) as the accuracy because steering angle is a continuous value + <x,y,...> denotes three hidden layers with x neurons in first layer, y neurons in second layer and so on - accuracy using SVM as reported by Šrndic et al. [56] - - -
  • 14. • , 0 , ( , 0(0 , ,1.. 1 . 1 3 1 / ,1.. 1 1/ 1 / .. . 1 3 1 / ,1.. 1 1 1 . 31/ 1 / 3 1 1 /3 33 /3 . 1 3 1 / 2 1 33 3 21 / . 1 / 3 1 3 1 3 1 3 2 /3 . 1 3 1 / .. . , 3 . • 0 • ) , 0 1 , 1 0 10, 0 0 , 0 0 , , 10 1 0 31 1 , 1 .. , • ) , , 10, 0 , 0 0 10, 0 , 1 1. .13 31 1 . 1 31 / 3 2 , , 1 ..1 1 1 1 • , • 1 1 . 1 1 3 , , 3 3 1 3 ,1 1 ,1.1 3
  • 15. • . . . . . .
  • 16. ( ) ( Different lighting conditions: all: right all: right all: right all: 1 all: 3 all: 5 all: diver all: cheeseburger all: flamingo DRV_C1: left DRV_C2: left DRV_C3: left MNI_C1: 8 MNI_C2: 5 MNI_C3: 7 IMG_C1: ski IMG_C2: icecream IMG_C3: goldfish Occlusion with a single small rectangle: all: right all: right all: left all: 5 all: 7 all: 9 all: cauliflower all: dhole all: hay
  • 17. all: right all: right all: right all: 1 all: 3 all: 5 all: diver all: cheeseburger all: flamingo DRV_C1: left DRV_C2: left DRV_C3: left MNI_C1: 8 MNI_C2: 5 MNI_C3: 7 IMG_C1: ski IMG_C2: icecream IMG_C3: goldfish Occlusion with a single small rectangle: all: right all: right all: left all: 5 all: 7 all: 9 all: cauliflower all: dhole all: hay DRV_C1: left DRV_C2: left DRV_C3: right MNI_C1: 3 MNI_C2: 4 MNI_C3: 2 IMG_C1:carbonara IMG_C2:umbrella IMG_C3:stupa Occlusion with multiple tiny black rectangles: all: left all: left all: left all: 1 all: 5 all: 7 all: castle all: cock all: groom ( ) (
  • 18. ( ) ( all: right all: right all: left all: 5 all: 7 all: 9 all: cauliflower all: dhole all: hay DRV_C1: left DRV_C2: left DRV_C3: right MNI_C1: 3 MNI_C2: 4 MNI_C3: 2 IMG_C1:carbonara IMG_C2:umbrella IMG_C3:stupa Occlusion with multiple tiny black rectangles: all: left all: left all: left all: 1 all: 5 all: 7 all: castle all: cock all: groom DRV_C1: right DRV_C2: right DRV_C3: right MNI_C1: 2 MNI_C2: 4 MNI_C3: 4 IMG_C1: beacon IMG_C2: hen IMG_C3: vestment Figure 7: Odd rows show the seed test inputs and even rows show the difference-inducing test inputs generated by DeepXplore. The left three columns show inputs for self-driving car, the middle three are for MNIST, and the right three are for ImageNet. Table 2: The top-3 most in(de)cremented features for generating two sample malware inputs which PDF classifiers (Contagio) incorrectly mark as benign. Table 3: The features added to the manifest file by DeepXplore for generating two sample malware inputs which Android app classifiers (Drebin) incorrectly mark as benign.
  • 19. • 0 .4 . 4 1 ? 0 3 . 4 ? 0 3 ) ? ? 4( • . • % . 1 ? ? % 0 1 0 .4 3 . ) ? 0 .4 4 . . 3 ? ) ? ? .? 0 1 0 .4 ? . 1 ? ? .0 3 0 ? ? 0? • , 0 .4 ? 1 ? . 1 .0 1 33 ? ) • . .. . . • ? 0 4 3 . 0 . . .0? .? 1 ? . ? 0 4 3 1 33 ? 0 .
  • 20. • • ) , , 4 . % , %% 2, , 3 , , , 3,4 3 • (3 2 3 2 • • , 4 4 (a) MNIST (b) ImageNet (c) Driving (d) Contagio Figure 8: The neuron coverage achieved by the same number of inputs (1% of the original test set) produced by DeepX testing [18], and random selection from the original test set. The plots show the change in neuron coverage for all three metho t (defined in Section 5) increases. DeepXplore, on average, covers 34.4% and 33.2% more neurons than random testing and a Table 6: Total time taken by DeepXplore to achieve 100% neuron coverage for different DNNs averaged over 10 runs. C1 C2 C3 MNIST 6.6 s 6.8 s 7.6 s ImageNet 43.6 s 45.3 s 42.7 s Driving 11.7 s 12.3 s 9.8 s Contagio 31.1 s 29.7 s 23.2 s Drebin 180.2 s 196.4 s 152.9 s Table 9: The variation in DeepXplore runtime ( generating the first difference-inducing input for with different 2, a parameter in Equation 3. Hig cate higher priority for increasing coverage. All n over 10 runs. The fastest times for each dataset gray. 2 = 0.5 2 = 1 2 = 2 MNIST 0.19 s 0.22 s 0.23 s ImageNet 1.55 s 1.6 s 1.62 s
  • 22. • . - 5 .5. .. . • . , . . . 5 1 5 ,1 , . . . , .- . • - . ,. 5 .5 1 5 5 . 1 . , . . - 1. 5 . . . . .1 • .- . 1. .- . - - . - - 1 - . . .1 1. . .-
  • 23. . . • I- C = F C = = C C = D ) / ( : C • =C D E E C = ,, • CC FFF = C : C F E C = = D = CF : . C = E = C C C = C E = C = D=C C DC C = F C D - C = = = D=C D C = C = C DC C D C D
  • 24. . • - • - - - - - - • - - - - • - - • - - - - - • - - • - - - - - - - - -
  • 25. • 1:7 B C7 :7 C BBC : : C7 7 9 • :7 C BBC : C7 7 7 C 7 7 C C 6 /C 9C 4 7 . 9 C 7C /. • C 76 C7 7 C : C 7 7 97 7C C : 7 BC7 4 • : B 6I 6 C9 ) 012 ( • : B 6 B ) (