SlideShare a Scribd company logo
Guaranteeing Deep Neural Network
Outputs in a Feasible Region
Hiroshi Maruyama
PFN Fellow
2
What is “invariance” in ML?
3
Fundamental Limitation of Machine Learning :
It’s Statistics!
Original Distribution
i. i. d.
Training Data Set
Trained Model
Random
Sampling !!
No guarantee of “100% correctness”
4
In deep learning, any point in the Rn is possible as output
Input
Output:A Point in Rn
For any point P in Rn, there is a combination of
the input, training data set, hyper-parameters,
and random-number seed that generates P
• Training Data Set
• Hyper parameters
• Random # seeds
• … and program itself
5
Example: Controlling a Drone
DL ModuleSensor Input
Reference Point
How to Guarantee that the reference point is always in the region?
6
Definition: Feasible Region and Non-Feasible Solutions
Feasible Region
Non-feasible
solutions
We assume the feasible region is convex
DNN
Policy
Filter
Simple Solution: Policy Filter
Remove (no output) Snap to the boarder
Proposed Solution: Transformation of Output Space
Rn → Rn Space
Transformation
Select an interior point (called pivot)
Step 1 (Bounding) : Transform Rn to n-dimensional
hypercube
9
Sigmoid Function
Apply Sigmoid on each dimension
Move the pivot to the origin of the hypercube
Step 2: Shrink / Extend every point towards the origin
10
11
Step 3: Finally move the pivot to the original position
For any combination of the input, training data set, hyper parameters, and
random number seed, the output is guarantted to be feasible
Proposed transformation works for any “star-shaped” space
12
Make this x0 the pivot
Set S is Star-shaped iff there is x0 s.t. for any interior point x,
the line segment xx0 ∈ S
Teacher signals can be given in the transformed space
Original DNN
(parameters to be
trained)
Transformation to
Feasible Region
(fixed parameters)
Back propagation
Rn Space
Input
Feasible Region
Teacher
Signal
loss
14
Thank You
Twitter: @maruyama

More Related Content

Similar to 20181204i mlse 1

convolutional_rbm.ppt
convolutional_rbm.pptconvolutional_rbm.ppt
convolutional_rbm.ppt
AyushSingh398902
 
featurers_Machinelearning___________.pdf
featurers_Machinelearning___________.pdffeaturers_Machinelearning___________.pdf
featurers_Machinelearning___________.pdf
AmirMohamedNabilSale
 
Distributed deep learning_over_spark_20_nov_2014_ver_2.8
Distributed deep learning_over_spark_20_nov_2014_ver_2.8Distributed deep learning_over_spark_20_nov_2014_ver_2.8
Distributed deep learning_over_spark_20_nov_2014_ver_2.8
Vijay Srinivas Agneeswaran, Ph.D
 
기계학습을 이용하여 정적 분석기의 안전성을 선별적으로 조절하는 방법
기계학습을 이용하여 정적 분석기의 안전성을 선별적으로 조절하는 방법기계학습을 이용하여 정적 분석기의 안전성을 선별적으로 조절하는 방법
기계학습을 이용하여 정적 분석기의 안전성을 선별적으로 조절하는 방법
NAVER Engineering
 
V2.0 open power ai virtual university deep learning and ai introduction
V2.0 open power ai virtual university   deep learning and ai introductionV2.0 open power ai virtual university   deep learning and ai introduction
V2.0 open power ai virtual university deep learning and ai introduction
Ganesan Narayanasamy
 
Deep Learning Introduction - WeCloudData
Deep Learning Introduction - WeCloudDataDeep Learning Introduction - WeCloudData
Deep Learning Introduction - WeCloudData
WeCloudData
 
VOICE CONTROLLED WHEELCHAIR using Amharic.pdf
VOICE CONTROLLED WHEELCHAIR using Amharic.pdfVOICE CONTROLLED WHEELCHAIR using Amharic.pdf
VOICE CONTROLLED WHEELCHAIR using Amharic.pdf
Mubarek kebede
 
Muhammad Usman Akhtar | Ph.D Scholar | Wuhan University | School of Co...
Muhammad Usman Akhtar  |  Ph.D Scholar  |  Wuhan  University  |  School of Co...Muhammad Usman Akhtar  |  Ph.D Scholar  |  Wuhan  University  |  School of Co...
Muhammad Usman Akhtar | Ph.D Scholar | Wuhan University | School of Co...
Wuhan University
 
Compiler Design- Machine Independent Optimizations
Compiler Design- Machine Independent OptimizationsCompiler Design- Machine Independent Optimizations
Compiler Design- Machine Independent Optimizations
Jyothishmathi Institute of Technology and Science Karimnagar
 
NLP Classifier Models & Metrics
NLP Classifier Models & MetricsNLP Classifier Models & Metrics
NLP Classifier Models & Metrics
Sanghamitra Deb
 
AI and Deep Learning
AI and Deep Learning AI and Deep Learning
AI and Deep Learning
Subrat Panda, PhD
 
Introduction to deep learning @ Startup.ML by Andres Rodriguez
Introduction to deep learning @ Startup.ML by Andres RodriguezIntroduction to deep learning @ Startup.ML by Andres Rodriguez
Introduction to deep learning @ Startup.ML by Andres Rodriguez
Intel Nervana
 
Thesis Presentation
Thesis PresentationThesis Presentation
Thesis Presentation
Reuben Feinman
 
[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習
[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習
[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習
台灣資料科學年會
 

Similar to 20181204i mlse 1 (14)

convolutional_rbm.ppt
convolutional_rbm.pptconvolutional_rbm.ppt
convolutional_rbm.ppt
 
featurers_Machinelearning___________.pdf
featurers_Machinelearning___________.pdffeaturers_Machinelearning___________.pdf
featurers_Machinelearning___________.pdf
 
Distributed deep learning_over_spark_20_nov_2014_ver_2.8
Distributed deep learning_over_spark_20_nov_2014_ver_2.8Distributed deep learning_over_spark_20_nov_2014_ver_2.8
Distributed deep learning_over_spark_20_nov_2014_ver_2.8
 
기계학습을 이용하여 정적 분석기의 안전성을 선별적으로 조절하는 방법
기계학습을 이용하여 정적 분석기의 안전성을 선별적으로 조절하는 방법기계학습을 이용하여 정적 분석기의 안전성을 선별적으로 조절하는 방법
기계학습을 이용하여 정적 분석기의 안전성을 선별적으로 조절하는 방법
 
V2.0 open power ai virtual university deep learning and ai introduction
V2.0 open power ai virtual university   deep learning and ai introductionV2.0 open power ai virtual university   deep learning and ai introduction
V2.0 open power ai virtual university deep learning and ai introduction
 
Deep Learning Introduction - WeCloudData
Deep Learning Introduction - WeCloudDataDeep Learning Introduction - WeCloudData
Deep Learning Introduction - WeCloudData
 
VOICE CONTROLLED WHEELCHAIR using Amharic.pdf
VOICE CONTROLLED WHEELCHAIR using Amharic.pdfVOICE CONTROLLED WHEELCHAIR using Amharic.pdf
VOICE CONTROLLED WHEELCHAIR using Amharic.pdf
 
Muhammad Usman Akhtar | Ph.D Scholar | Wuhan University | School of Co...
Muhammad Usman Akhtar  |  Ph.D Scholar  |  Wuhan  University  |  School of Co...Muhammad Usman Akhtar  |  Ph.D Scholar  |  Wuhan  University  |  School of Co...
Muhammad Usman Akhtar | Ph.D Scholar | Wuhan University | School of Co...
 
Compiler Design- Machine Independent Optimizations
Compiler Design- Machine Independent OptimizationsCompiler Design- Machine Independent Optimizations
Compiler Design- Machine Independent Optimizations
 
NLP Classifier Models & Metrics
NLP Classifier Models & MetricsNLP Classifier Models & Metrics
NLP Classifier Models & Metrics
 
AI and Deep Learning
AI and Deep Learning AI and Deep Learning
AI and Deep Learning
 
Introduction to deep learning @ Startup.ML by Andres Rodriguez
Introduction to deep learning @ Startup.ML by Andres RodriguezIntroduction to deep learning @ Startup.ML by Andres Rodriguez
Introduction to deep learning @ Startup.ML by Andres Rodriguez
 
Thesis Presentation
Thesis PresentationThesis Presentation
Thesis Presentation
 
[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習
[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習
[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習
 

More from Hiroshi Maruyama

20230925プレジデント社60周年.pdf
20230925プレジデント社60周年.pdf20230925プレジデント社60周年.pdf
20230925プレジデント社60周年.pdf
Hiroshi Maruyama
 
20230912JSSST大会基調講演_丸山.pdf
20230912JSSST大会基調講演_丸山.pdf20230912JSSST大会基調講演_丸山.pdf
20230912JSSST大会基調講演_丸山.pdf
Hiroshi Maruyama
 
20230712Kuramae-Seminar.pdf
20230712Kuramae-Seminar.pdf20230712Kuramae-Seminar.pdf
20230712Kuramae-Seminar.pdf
Hiroshi Maruyama
 
202212APSEC.pptx.pdf
202212APSEC.pptx.pdf202212APSEC.pptx.pdf
202212APSEC.pptx.pdf
Hiroshi Maruyama
 
20210731知財学会研究会
20210731知財学会研究会20210731知財学会研究会
20210731知財学会研究会
Hiroshi Maruyama
 
2021 06-17 ism-symposium
2021 06-17 ism-symposium2021 06-17 ism-symposium
2021 06-17 ism-symposium
Hiroshi Maruyama
 
Jsai
JsaiJsai
20181212 ibm aot
20181212 ibm aot20181212 ibm aot
20181212 ibm aot
Hiroshi Maruyama
 
20181205 sakigake
20181205 sakigake20181205 sakigake
20181205 sakigake
Hiroshi Maruyama
 
20181204i mlse discussions
20181204i mlse discussions20181204i mlse discussions
20181204i mlse discussions
Hiroshi Maruyama
 
20181120 ldp ai
20181120 ldp ai20181120 ldp ai
20181120 ldp ai
Hiroshi Maruyama
 
20181030 fun
20181030 fun20181030 fun
20181030 fun
Hiroshi Maruyama
 
20180719 cocn dist
20180719 cocn dist20180719 cocn dist
20180719 cocn dist
Hiroshi Maruyama
 
20180601 ai discussions
20180601 ai discussions20180601 ai discussions
20180601 ai discussions
Hiroshi Maruyama
 
構造改革徹底推進会合におけるプレゼン
構造改革徹底推進会合におけるプレゼン構造改革徹底推進会合におけるプレゼン
構造改革徹底推進会合におけるプレゼン
Hiroshi Maruyama
 
深層学習よもやま話
深層学習よもやま話深層学習よもやま話
深層学習よもやま話
Hiroshi Maruyama
 

More from Hiroshi Maruyama (16)

20230925プレジデント社60周年.pdf
20230925プレジデント社60周年.pdf20230925プレジデント社60周年.pdf
20230925プレジデント社60周年.pdf
 
20230912JSSST大会基調講演_丸山.pdf
20230912JSSST大会基調講演_丸山.pdf20230912JSSST大会基調講演_丸山.pdf
20230912JSSST大会基調講演_丸山.pdf
 
20230712Kuramae-Seminar.pdf
20230712Kuramae-Seminar.pdf20230712Kuramae-Seminar.pdf
20230712Kuramae-Seminar.pdf
 
202212APSEC.pptx.pdf
202212APSEC.pptx.pdf202212APSEC.pptx.pdf
202212APSEC.pptx.pdf
 
20210731知財学会研究会
20210731知財学会研究会20210731知財学会研究会
20210731知財学会研究会
 
2021 06-17 ism-symposium
2021 06-17 ism-symposium2021 06-17 ism-symposium
2021 06-17 ism-symposium
 
Jsai
JsaiJsai
Jsai
 
20181212 ibm aot
20181212 ibm aot20181212 ibm aot
20181212 ibm aot
 
20181205 sakigake
20181205 sakigake20181205 sakigake
20181205 sakigake
 
20181204i mlse discussions
20181204i mlse discussions20181204i mlse discussions
20181204i mlse discussions
 
20181120 ldp ai
20181120 ldp ai20181120 ldp ai
20181120 ldp ai
 
20181030 fun
20181030 fun20181030 fun
20181030 fun
 
20180719 cocn dist
20180719 cocn dist20180719 cocn dist
20180719 cocn dist
 
20180601 ai discussions
20180601 ai discussions20180601 ai discussions
20180601 ai discussions
 
構造改革徹底推進会合におけるプレゼン
構造改革徹底推進会合におけるプレゼン構造改革徹底推進会合におけるプレゼン
構造改革徹底推進会合におけるプレゼン
 
深層学習よもやま話
深層学習よもやま話深層学習よもやま話
深層学習よもやま話
 

Recently uploaded

Salesforce AI & Einstein Copilot Workshop
Salesforce AI & Einstein Copilot WorkshopSalesforce AI & Einstein Copilot Workshop
Salesforce AI & Einstein Copilot Workshop
CEPTES Software Inc
 
CHAPTER-8 COMPONENTS OF COMPUTER SYSTEM CLASS 9 CBSE
CHAPTER-8 COMPONENTS OF COMPUTER SYSTEM CLASS 9 CBSECHAPTER-8 COMPONENTS OF COMPUTER SYSTEM CLASS 9 CBSE
CHAPTER-8 COMPONENTS OF COMPUTER SYSTEM CLASS 9 CBSE
kumarjarun2010
 
Feature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptxFeature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptx
ssuser1915fe1
 
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptxRPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
SynapseIndia
 
CiscoIconsLibrary cours de réseau VLAN.ppt
CiscoIconsLibrary cours de réseau VLAN.pptCiscoIconsLibrary cours de réseau VLAN.ppt
CiscoIconsLibrary cours de réseau VLAN.ppt
moinahousna
 
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
aslasdfmkhan4750
 
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
digitalxplive
 
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
Neo4j
 
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptxDublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Kunal Gupta
 
find out more about the role of autonomous vehicles in facing global challenges
find out more about the role of autonomous vehicles in facing global challengesfind out more about the role of autonomous vehicles in facing global challenges
find out more about the role of autonomous vehicles in facing global challenges
huseindihon
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
Zilliz
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
SynapseIndia
 
Implementations of Fused Deposition Modeling in real world
Implementations of Fused Deposition Modeling  in real worldImplementations of Fused Deposition Modeling  in real world
Implementations of Fused Deposition Modeling in real world
Emerging Tech
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
alexjohnson7307
 
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
Priyanka Aash
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
Ivanti
 
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptxIntroduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
313mohammedarshad
 
IPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite SolutionIPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite Solution
IPLOOK Networks
 
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Muhammad Ali
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
Jimmy Lai
 

Recently uploaded (20)

Salesforce AI & Einstein Copilot Workshop
Salesforce AI & Einstein Copilot WorkshopSalesforce AI & Einstein Copilot Workshop
Salesforce AI & Einstein Copilot Workshop
 
CHAPTER-8 COMPONENTS OF COMPUTER SYSTEM CLASS 9 CBSE
CHAPTER-8 COMPONENTS OF COMPUTER SYSTEM CLASS 9 CBSECHAPTER-8 COMPONENTS OF COMPUTER SYSTEM CLASS 9 CBSE
CHAPTER-8 COMPONENTS OF COMPUTER SYSTEM CLASS 9 CBSE
 
Feature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptxFeature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptx
 
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptxRPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
 
CiscoIconsLibrary cours de réseau VLAN.ppt
CiscoIconsLibrary cours de réseau VLAN.pptCiscoIconsLibrary cours de réseau VLAN.ppt
CiscoIconsLibrary cours de réseau VLAN.ppt
 
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
 
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
 
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
 
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptxDublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
 
find out more about the role of autonomous vehicles in facing global challenges
find out more about the role of autonomous vehicles in facing global challengesfind out more about the role of autonomous vehicles in facing global challenges
find out more about the role of autonomous vehicles in facing global challenges
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
 
Implementations of Fused Deposition Modeling in real world
Implementations of Fused Deposition Modeling  in real worldImplementations of Fused Deposition Modeling  in real world
Implementations of Fused Deposition Modeling in real world
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
 
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
(CISOPlatform Summit & SACON 2024) Digital Personal Data Protection Act.pdf
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
 
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptxIntroduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
 
IPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite SolutionIPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite Solution
 
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
 

20181204i mlse 1

  • 1. Guaranteeing Deep Neural Network Outputs in a Feasible Region Hiroshi Maruyama PFN Fellow
  • 3. 3 Fundamental Limitation of Machine Learning : It’s Statistics! Original Distribution i. i. d. Training Data Set Trained Model Random Sampling !! No guarantee of “100% correctness”
  • 4. 4 In deep learning, any point in the Rn is possible as output Input Output:A Point in Rn For any point P in Rn, there is a combination of the input, training data set, hyper-parameters, and random-number seed that generates P • Training Data Set • Hyper parameters • Random # seeds • … and program itself
  • 5. 5 Example: Controlling a Drone DL ModuleSensor Input Reference Point How to Guarantee that the reference point is always in the region?
  • 6. 6 Definition: Feasible Region and Non-Feasible Solutions Feasible Region Non-feasible solutions We assume the feasible region is convex
  • 7. DNN Policy Filter Simple Solution: Policy Filter Remove (no output) Snap to the boarder
  • 8. Proposed Solution: Transformation of Output Space Rn → Rn Space Transformation Select an interior point (called pivot)
  • 9. Step 1 (Bounding) : Transform Rn to n-dimensional hypercube 9 Sigmoid Function Apply Sigmoid on each dimension Move the pivot to the origin of the hypercube
  • 10. Step 2: Shrink / Extend every point towards the origin 10
  • 11. 11 Step 3: Finally move the pivot to the original position For any combination of the input, training data set, hyper parameters, and random number seed, the output is guarantted to be feasible
  • 12. Proposed transformation works for any “star-shaped” space 12 Make this x0 the pivot Set S is Star-shaped iff there is x0 s.t. for any interior point x, the line segment xx0 ∈ S
  • 13. Teacher signals can be given in the transformed space Original DNN (parameters to be trained) Transformation to Feasible Region (fixed parameters) Back propagation Rn Space Input Feasible Region Teacher Signal loss