1 of 25

## What's hot

Metaheuristic Tuning of Type-II Fuzzy Inference System for Data Mining
Metaheuristic Tuning of Type-II Fuzzy Inference System for Data MiningVarun Ojha

Solutions manual for calculus an applied approach brief international metric ...
Solutions manual for calculus an applied approach brief international metric ...Larson612

โครงงานคอม
โครงงานคอมnoeiinoii

Precalculus 1 chapter 2
Precalculus 1 chapter 2oreves

Arna Friend Controls II Final
Arna Friend Controls II FinalArna Friend

The International Journal of Engineering and Science (IJES)
The International Journal of Engineering and Science (IJES)theijes

2020 1학기 정기스터디 2주차
2020 1학기 정기스터디 2주차Moonki Choi

Calculus 10th edition anton solutions manual
Calculus 10th edition anton solutions manualReece1334

Image Cryptography and Steganography

เลขยกกำลังชุด 2
เลขยกกำลังชุด 2kanjana2536

Mm chap08 -_lossy_compression_algorithms
Mm chap08 -_lossy_compression_algorithmsEellekwameowusu

Microsoft Word Hw#3
Microsoft Word Hw#3kkkseld

Application of recursive perturbation approach for multimodal optimization
Application of recursive perturbation approach for multimodal optimizationPranamesh Chakraborty

### What's hot(17)

Metaheuristic Tuning of Type-II Fuzzy Inference System for Data Mining
Metaheuristic Tuning of Type-II Fuzzy Inference System for Data Mining

Solutions manual for calculus an applied approach brief international metric ...
Solutions manual for calculus an applied approach brief international metric ...

Gilat_ch02.pdf
Gilat_ch02.pdf

jammer_placement
jammer_placement

โครงงานคอม
โครงงานคอม

Gilat_ch03.pdf
Gilat_ch03.pdf

Precalculus 1 chapter 2
Precalculus 1 chapter 2

Arna Friend Controls II Final
Arna Friend Controls II Final

The International Journal of Engineering and Science (IJES)
The International Journal of Engineering and Science (IJES)

2020 1학기 정기스터디 2주차
2020 1학기 정기스터디 2주차

Gate-Cs 1993
Gate-Cs 1993

Calculus 10th edition anton solutions manual
Calculus 10th edition anton solutions manual

Image Cryptography and Steganography
Image Cryptography and Steganography

เลขยกกำลังชุด 2
เลขยกกำลังชุด 2

Mm chap08 -_lossy_compression_algorithms
Mm chap08 -_lossy_compression_algorithms

Microsoft Word Hw#3
Microsoft Word Hw#3

Application of recursive perturbation approach for multimodal optimization
Application of recursive perturbation approach for multimodal optimization

## Similar to DeepXplore: Automated Whitebox Testing of Deep Learning

03 image transformations_i
03 image transformations_iankit_ppt

Introduction to Applied Machine Learning
Introduction to Applied Machine LearningSheilaJimenezMorejon

Digit recognizer by convolutional neural network
Digit recognizer by convolutional neural networkDing Li

5707_10_auto-encoder.pptx
5707_10_auto-encoder.pptxSidoriOne

CD504 CGM_Lab Manual_004e08d3838702ed11fc6d03cc82f7be.pdf
CD504 CGM_Lab Manual_004e08d3838702ed11fc6d03cc82f7be.pdfRajJain516913

Computer Graphics Unit 1
Computer Graphics Unit 1aravindangc

Image Texture Analysis
Image Texture Analysislalitxp

Introduction to Neural Networks and Deep Learning
Introduction to Neural Networks and Deep LearningVahid Mirjalili

NeuralArt 電腦作畫
NeuralArt 電腦作畫Mark Chang

3D Brain Image Segmentation Model using Deep Learning and Hidden Markov Rando...
3D Brain Image Segmentation Model using Deep Learning and Hidden Markov Rando...EL-Hachemi Guerrout

NIPS2017 Few-shot Learning and Graph Convolution
NIPS2017 Few-shot Learning and Graph ConvolutionKazuki Fujikawa

Matlab 3
Matlab 3asguna

Sample Exam Questions on Python for revision
Sample Exam Questions on Python for revisionafsheenfaiq2

K-means Clustering Algorithm with Matlab Source code
K-means Clustering Algorithm with Matlab Source codegokulprasath06

A practical Introduction to Machine(s) Learning
A practical Introduction to Machine(s) LearningBruno Gonçalves

Convolutional Neural Network
Convolutional Neural NetworkJun Young Park

### Similar to DeepXplore: Automated Whitebox Testing of Deep Learning(20)

03 image transformations_i
03 image transformations_i

Introduction to Applied Machine Learning
Introduction to Applied Machine Learning

Digit recognizer by convolutional neural network
Digit recognizer by convolutional neural network

Eye deep
Eye deep

5707_10_auto-encoder.pptx
5707_10_auto-encoder.pptx

CD504 CGM_Lab Manual_004e08d3838702ed11fc6d03cc82f7be.pdf
CD504 CGM_Lab Manual_004e08d3838702ed11fc6d03cc82f7be.pdf

Computer Graphics Unit 1
Computer Graphics Unit 1

Image Texture Analysis
Image Texture Analysis

Introduction to Neural Networks and Deep Learning
Introduction to Neural Networks and Deep Learning

NeuralArt 電腦作畫
NeuralArt 電腦作畫

3D Brain Image Segmentation Model using Deep Learning and Hidden Markov Rando...
3D Brain Image Segmentation Model using Deep Learning and Hidden Markov Rando...

Cgm Lab Manual
Cgm Lab Manual

NIPS2017 Few-shot Learning and Graph Convolution
NIPS2017 Few-shot Learning and Graph Convolution

Matlab 3
Matlab 3

Sample Exam Questions on Python for revision
Sample Exam Questions on Python for revision

K-means Clustering Algorithm with Matlab Source code
K-means Clustering Algorithm with Matlab Source code

A practical Introduction to Machine(s) Learning
A practical Introduction to Machine(s) Learning

Convolutional Neural Network
Convolutional Neural Network

## More from Masahiro Sakai

Towards formal verification of neural networks
Towards formal verification of neural networksMasahiro Sakai

Writing a SAT solver as a hobby project
Writing a SAT solver as a hobby projectMasahiro Sakai

RClassify: Classifying Race Conditions in Web Applications via Deterministic ...
RClassify: Classifying Race Conditions in Web Applications via Deterministic ...Masahiro Sakai

SAT/SMTソルバの仕組み
SAT/SMTソルバの仕組みMasahiro Sakai

Introduction to Max-SAT and Max-SAT Evaluation
Introduction to Max-SAT and Max-SAT EvaluationMasahiro Sakai

Aluminum: Principled Scenario Exploration through Minimality
Aluminum: Principled Scenario Exploration through MinimalityMasahiro Sakai

How a CDCL SAT solver works
How a CDCL SAT solver worksMasahiro Sakai

“Adoption and Focus: Practical Linear Types for Imperative Programming”他の紹介＠P...
“Adoption and Focus: Practical Linear Types for Imperative Programming”他の紹介＠P...Masahiro Sakai

“Design and Implementation of Generics for the .NET Common Language Runtime”他...
“Design and Implementation of Generics for the .NET Common Language Runtime”他...Masahiro Sakai

Relaxed Dependency Analysis
Relaxed Dependency AnalysisMasahiro Sakai

“Symbolic bounds analysis of pointers, array indices, and accessed memory reg...
“Symbolic bounds analysis of pointers, array indices, and accessed memory reg...Masahiro Sakai

Whole Program Paths 等の紹介@PLDIr#3
Whole Program Paths 等の紹介@PLDIr#3Masahiro Sakai

### More from Masahiro Sakai(20)

Towards formal verification of neural networks
Towards formal verification of neural networks

Writing a SAT solver as a hobby project
Writing a SAT solver as a hobby project

RClassify: Classifying Race Conditions in Web Applications via Deterministic ...
RClassify: Classifying Race Conditions in Web Applications via Deterministic ...

ゼロピッチ: MOOC
ゼロピッチ: MOOC

SAT/SMTソルバの仕組み
SAT/SMTソルバの仕組み

Introduction to Max-SAT and Max-SAT Evaluation
Introduction to Max-SAT and Max-SAT Evaluation

Aluminum: Principled Scenario Exploration through Minimality
Aluminum: Principled Scenario Exploration through Minimality

How a CDCL SAT solver works
How a CDCL SAT solver works

Omega test and beyond
Omega test and beyond

“Adoption and Focus: Practical Linear Types for Imperative Programming”他の紹介＠P...
“Adoption and Focus: Practical Linear Types for Imperative Programming”他の紹介＠P...

“Design and Implementation of Generics for the .NET Common Language Runtime”他...
“Design and Implementation of Generics for the .NET Common Language Runtime”他...

Relaxed Dependency Analysis
Relaxed Dependency Analysis

“Symbolic bounds analysis of pointers, array indices, and accessed memory reg...
“Symbolic bounds analysis of pointers, array indices, and accessed memory reg...

Whole Program Paths 等の紹介@PLDIr#3
Whole Program Paths 等の紹介@PLDIr#3

Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada

Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoHarshalMandlekar2

DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu

Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair

(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...AliaaTarek5

TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey

What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett

Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dashnarutouzumaki53779

Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3

unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21

Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3

WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan

Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada

"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays

TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc

From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix

Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan

"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays

What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina

New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada

Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024

Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demo

DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms

Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...

(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...

TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024

What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024

Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dash

Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx

unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx

Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf

WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand

Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024

"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan

TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy

From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .

Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers

"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack

What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf

New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024

### DeepXplore: Automated Whitebox Testing of Deep Learning

• 1. A A D 5D 7 D 6 0 CD : A 5 : / CD C . J 4 5 K : 45 :J / 5 5 5J J 6 5 1 C D K : 1 C D DDAC 5 : 56C ) - -1 21 ( 2 0 - - - 21
• 2. • . . . . . . . Figure 4: DeepXplore workﬂow.
• 4. • - -, -, - - - . - , - - -: . : • -, : ,- - , , : . - : , .-: : :- • - / - . , / - ,-:- : , .: , - :- • :- : - :- :-, :- : . , ..- - : - - : ,-:- : :- : . ,- -, - : - - . : - - - :: ,- - -: : , . - . : . - -:
• 5. • - . . . :. • . . . : . . . : • . . • (: A • , . : . . . . : :. : : : . A . D . . . . A :A : . . . : • ) . : . . . : . . D (. : . : . . :
• 6. • , , / , , , , , / , , , ? , • / ,? , , , , ( , ) , - , , , ,
• 7. • , 1 = ? : :2 1 ? ?1 1 =: 2:= ?1 ? ? ? 1 1? ?:? .1= :2 1 =: ? 1 ( 1=1 • D =1 1 =: :2 ( • , D =1 ?1 ? ? • : ? ? 1 : ? ? 1 :2 1 =: 2:= ? • ? : 1 ? =1 : • . 1 ? 1 • )1 1= ?1 ? ? ? =1 1 1 =: : 1= 1 a function that returns the output value of neuron n in NN for a given test input x. Let t represent the thres r considering a neuron to be activated. Neuron cove n be deﬁned as follows. NCov(T, x) = |{n|8x2T,out(n,x)>t}| |N| To demonstrate how neuron coverage is calculate actice, consider the DNN showed in Figure 3b. The ne verage (with threshold 0) for the input picture of the r shown in Figure 3b will be 5/8 = 0.625. G : .1 G
• 8. 2 - • , , , , • : , : : , , , : , , , . • , , • , , , : , , , , , ,, : , (a) DNNs produce same output (b) DNNs produce different output neuron coverage and gradient for DNNs. Next, we desc how the testing problem can be formulated as a joint mization problem. Finally, we provide the algorithm for optimization using gradients. 5.1 Deﬁnitions Neuron coverage. We deﬁne neuron coverage of a set o inputs as the ratio of the number of unique activated neu for all test inputs in the set and the total number of neuro (a) DNNs produce same output (b) DNNs produce different output Figure 5: Inputs inducing different behaviors in two similar DNNs. Figure 6: Gradient ascent starting from a seed input and gradually ﬁnding the difference-inducing test inputs. neu how miz opt 5.1 Ne inp for the is h are are is a DN for can pra cov car
• 9. Figure 4: DeepXplore workﬂow.
• 10. • • • • • • Let Fk(x)[c] be the class probability that Fk predic be c. We randomly select one neural network Fj (Algo line 6) and maximize the following objective function obj1(x) = ⌃i6=jFi(x)[c] 1 · Fj(x)[c] where 1 is a parameter to balance the objective between the DNNs Fk6=j that maintain the same class o as before and the DNN Fj that produce different class o As Fk21..n are all differentiable, Equation 2 can be iteratively computed gradient: @x (Algorithm 1 line 8-14 and procedure COMPUTE_OBJ1). Maximizing neuron coverage. The second objective is to generate inputs that maximize neuron coverage. We achieve this by iteratively picking inactivated neurons and perturbing the input such that output of that neuron goes above the neuron coverage threshold. Let us assume that we want to maximize the output of a neuron n, i.e., we want to maximize obj2(x) = fn(x) such that fn(x) > t, where t is the neuron coverage threshold, and we write fn(x) as the function performed by neuron n that takes x (the input to DNN) as input and produce the output of neuron n (as deﬁned in Equation 1). We can again leverage the gradient 24: rest = dnns - d 25: loss1 := 0 26: for dnn in rest do 27: loss1 += dnnc(x) //conﬁdence score of x being in class c 28: loss2 := dc(x) //d’s output conﬁdence score of x being in class c 29: return (loss1 - 1·loss2) 30: procedure COMPUTE_OBJ2(x, dnns, cov_tracker) 31: loss := 0 32: for dnn 2 dnns do 33: select a neuron n inactivated so far using cov_tracker 34: loss += n(x) //the neuron n’s output when x is the dnn’s inpu 35: return loss Joint optimization. We jointly maximize obj1 and fn d scribed above and maximize the following function: objjoint = (⌃i6=jFi(x)[c] 1Fj(x)[c]) + 2 · fn(x)
• 11. • 2 2 2 5 2 . . 2 . 2 . 250 0 2.5 ,.5 • 0 2.5 2 2 2. 2 25 , 5 25 . 0 2 0. 2 . . 2 25 ( ) . . . . Figure 4: DeepXplore workﬂow.
• 12. • 2HKGAHAIN NE I • .4 BL HAQ L AI LBG Q I 3AL • 46- • KALEHAIN AIPEL IHAIN • 4EIO G KN K LOIIEIC : OINO • IA 2INAG E 18 01T KL A L QEND LA • 0, B HAH L • 2.2 0 07:
• 13. Table 1: Details of the DNNs and datasets used to evaluate DeepXplore Dataset Dataset Description DNN Description DNN Name No. of Neurons Architecture Reported Acc. Our Acc. MNIST Hand-written digits LeNet variations MNI_C1 52 LeNet-1, LeCun et al. [26, 28] 98.3% 98.33% MNI_C2 148 LeNet-4, LeCun et al. [26, 28] 98.9% 98.59% MNI_C3 268 LeNet-5, LeCun et al. [26, 28] 99.05% 98.96% Imagenet General images State-of-the-art image classiﬁers from ILSVRC IMG_C1 14,888 VGG-16, Simonyan et al. [45] 92.6%⇤⇤ 92.6%⇤⇤ IMG_C2 16,168 VGG-19, Simonyan et al. [45] 92.7%⇤⇤ 92.7%⇤⇤ IMG_C3 94,059 ResNet50, He et al. [22] 96.43%⇤⇤ 96.43%⇤⇤ Driving Driving video frames Nvidia DAVE self-driving systems DRV_C1 1,560 Dave-orig [4], Bojarski et al. [5] N/A 99.91%# DRV_C2 1,560 Dave-norminit [55] N/A 99.94%# DRV_C3 844 Dave-dropout [11] N/A 99.96%# Contagio PDFs PDF malware detectors PDF_C1 402 <200, 200>+ 98.5% 96.15% PDF_C2 602 <200, 200, 200>+ 98.5% 96.25% PDF_C3 802 <200, 200, 200, 200>+ 98.5% 96.47% Drebin Android apps Android app malware detectors APP_C1 402 <200, 200>+, Grosse et al. [21] 98.92% 98.6% APP_C2 102 <50, 50>+, Grosse et al. [21] 96.79% 96.82% APP_C3 212 <200, 10>+, Grosse et al. [21] 92.97% 92.66% ** top-5 test accuracy; we exactly match the reported performance as we use the pretrained networks # we report 1-MSE (Mean Squared Error) as the accuracy because steering angle is a continuous value + <x,y,...> denotes three hidden layers with x neurons in ﬁrst layer, y neurons in second layer and so on - accuracy using SVM as reported by Šrndic et al. [56] - - -
• 14. • , 0 , ( , 0(0 , ,1.. 1 . 1 3 1 / ,1.. 1 1/ 1 / .. . 1 3 1 / ,1.. 1 1 1 . 31/ 1 / 3 1 1 /3 33 /3 . 1 3 1 / 2 1 33 3 21 / . 1 / 3 1 3 1 3 1 3 2 /3 . 1 3 1 / .. . , 3 . • 0 • ) , 0 1 , 1 0 10, 0 0 , 0 0 , , 10 1 0 31 1 , 1 .. , • ) , , 10, 0 , 0 0 10, 0 , 1 1. .13 31 1 . 1 31 / 3 2 , , 1 ..1 1 1 1 • , • 1 1 . 1 1 3 , , 3 3 1 3 ,1 1 ,1.1 3
• 15. • . . . . . .
• 16. ( ) ( Different lighting conditions: all: right all: right all: right all: 1 all: 3 all: 5 all: diver all: cheeseburger all: ﬂamingo DRV_C1: left DRV_C2: left DRV_C3: left MNI_C1: 8 MNI_C2: 5 MNI_C3: 7 IMG_C1: ski IMG_C2: icecream IMG_C3: goldﬁsh Occlusion with a single small rectangle: all: right all: right all: left all: 5 all: 7 all: 9 all: cauliﬂower all: dhole all: hay
• 17. all: right all: right all: right all: 1 all: 3 all: 5 all: diver all: cheeseburger all: ﬂamingo DRV_C1: left DRV_C2: left DRV_C3: left MNI_C1: 8 MNI_C2: 5 MNI_C3: 7 IMG_C1: ski IMG_C2: icecream IMG_C3: goldﬁsh Occlusion with a single small rectangle: all: right all: right all: left all: 5 all: 7 all: 9 all: cauliﬂower all: dhole all: hay DRV_C1: left DRV_C2: left DRV_C3: right MNI_C1: 3 MNI_C2: 4 MNI_C3: 2 IMG_C1:carbonara IMG_C2:umbrella IMG_C3:stupa Occlusion with multiple tiny black rectangles: all: left all: left all: left all: 1 all: 5 all: 7 all: castle all: cock all: groom ( ) (
• 18. ( ) ( all: right all: right all: left all: 5 all: 7 all: 9 all: cauliﬂower all: dhole all: hay DRV_C1: left DRV_C2: left DRV_C3: right MNI_C1: 3 MNI_C2: 4 MNI_C3: 2 IMG_C1:carbonara IMG_C2:umbrella IMG_C3:stupa Occlusion with multiple tiny black rectangles: all: left all: left all: left all: 1 all: 5 all: 7 all: castle all: cock all: groom DRV_C1: right DRV_C2: right DRV_C3: right MNI_C1: 2 MNI_C2: 4 MNI_C3: 4 IMG_C1: beacon IMG_C2: hen IMG_C3: vestment Figure 7: Odd rows show the seed test inputs and even rows show the difference-inducing test inputs generated by DeepXplore. The left three columns show inputs for self-driving car, the middle three are for MNIST, and the right three are for ImageNet. Table 2: The top-3 most in(de)cremented features for generating two sample malware inputs which PDF classiﬁers (Contagio) incorrectly mark as benign. Table 3: The features added to the manifest ﬁle by DeepXplore for generating two sample malware inputs which Android app classiﬁers (Drebin) incorrectly mark as benign.
• 19. • 0 .4 . 4 1 ? 0 3 . 4 ? 0 3 ) ? ? 4( • . • % . 1 ? ? % 0 1 0 .4 3 . ) ? 0 .4 4 . . 3 ? ) ? ? .? 0 1 0 .4 ? . 1 ? ? .0 3 0 ? ? 0? • , 0 .4 ? 1 ? . 1 .0 1 33 ? ) • . .. . . • ? 0 4 3 . 0 . . .0? .? 1 ? . ? 0 4 3 1 33 ? 0 .
• 20. • • ) , , 4 . % , %% 2, , 3 , , , 3,4 3 • (3 2 3 2 • • , 4 4 (a) MNIST (b) ImageNet (c) Driving (d) Contagio Figure 8: The neuron coverage achieved by the same number of inputs (1% of the original test set) produced by DeepX testing [18], and random selection from the original test set. The plots show the change in neuron coverage for all three metho t (deﬁned in Section 5) increases. DeepXplore, on average, covers 34.4% and 33.2% more neurons than random testing and a Table 6: Total time taken by DeepXplore to achieve 100% neuron coverage for different DNNs averaged over 10 runs. C1 C2 C3 MNIST 6.6 s 6.8 s 7.6 s ImageNet 43.6 s 45.3 s 42.7 s Driving 11.7 s 12.3 s 9.8 s Contagio 31.1 s 29.7 s 23.2 s Drebin 180.2 s 196.4 s 152.9 s Table 9: The variation in DeepXplore runtime ( generating the ﬁrst difference-inducing input for with different 2, a parameter in Equation 3. Hig cate higher priority for increasing coverage. All n over 10 runs. The fastest times for each dataset gray. 2 = 0.5 2 = 1 2 = 2 MNIST 0.19 s 0.22 s 0.23 s ImageNet 1.55 s 1.6 s 1.62 s
• 22. • . - 5 .5. .. . • . , . . . 5 1 5 ,1 , . . . , .- . • - . ,. 5 .5 1 5 5 . 1 . , . . - 1. 5 . . . . .1 • .- . 1. .- . - - . - - 1 - . . .1 1. . .-
• 23. . . • I- C = F C = = C C = D ) / ( : C • =C D E E C = ,, • CC FFF = C : C F E C = = D = CF : . C = E = C C C = C E = C = D=C C DC C = F C D - C = = = D=C D C = C = C DC C D C D
• 24. . • - • - - - - - - • - - - - • - - • - - - - - • - - • - - - - - - - - -
• 25. • 1:7 B C7 :7 C BBC : : C7 7 9 • :7 C BBC : C7 7 7 C 7 7 C C 6 /C 9C 4 7 . 9 C 7C /. • C 76 C7 7 C : C 7 7 97 7C C : 7 BC7 4 • : B 6I 6 C9 ) 012 ( • : B 6 B ) (
Current LanguageEnglish
Español
Portugues
Français
Deutsche