SlideShare a Scribd company logo
1 of 14
Download to read offline
Terry Taewoong Um (terry.t.um@gmail.com)
University of Waterloo
Department of Electrical & Computer Engineering
Terry Taewoong Um
ON CALIBRATION OF MODERN NE
URAL NETWORKS
1
TODAY’S PAPER
Terry Taewoong Um (terry.t.um@gmail.com)
https://vimeo.com/238242536
GOAL
Terry Taewoong Um (terry.t.um@gmail.com)
Softmax output (ෝ𝒑) != Confidence (ෝ𝒒)
MEASURING MISCALIBRATION
Terry Taewoong Um (terry.t.um@gmail.com)
MEASURING MISCALIBRATION
Terry Taewoong Um (terry.t.um@gmail.com)
MEASURING MISCALIBRATION
Terry Taewoong Um (terry.t.um@gmail.com)
What factors cause
misclalibration?
WHAT CAUSES MISCALIBRATION?
Terry Taewoong Um (terry.t.um@gmail.com)
OVERFITTING
Terry Taewoong Um (terry.t.um@gmail.com)
CALIBRATION
Temperature scaling
COMPARISON TO RELATED WORKS
Terry Taewoong Um (terry.t.um@gmail.com)
Histogram Binning
Plot Scaling
Bayesian Binning into Quantiles (BBQ)
FINDINGS & CONCLUSION
Terry Taewoong Um (terry.t.um@gmail.com)
MY THOUGHTS
Terry Taewoong Um (terry.t.um@gmail.com)
TODAY’S PAPER
Terry Taewoong Um (terry.t.um@gmail.com)
https://vimeo.com/238242536

More Related Content

What's hot

Connie zweig & steve wolf o jogo das sombras (doc)(rev)
Connie zweig & steve wolf   o jogo das sombras (doc)(rev)Connie zweig & steve wolf   o jogo das sombras (doc)(rev)
Connie zweig & steve wolf o jogo das sombras (doc)(rev)
Saphira Trilena
 

What's hot (20)

修士論文発表:「非負値行列分解における漸近的Bayes汎化誤差」
修士論文発表:「非負値行列分解における漸近的Bayes汎化誤差」修士論文発表:「非負値行列分解における漸近的Bayes汎化誤差」
修士論文発表:「非負値行列分解における漸近的Bayes汎化誤差」
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
 
Prml6
Prml6Prml6
Prml6
 
Applying Deep Learning with Weak and Noisy labels
Applying Deep Learning with Weak and Noisy labelsApplying Deep Learning with Weak and Noisy labels
Applying Deep Learning with Weak and Noisy labels
 
Tensor Decomposition and its Applications
Tensor Decomposition and its ApplicationsTensor Decomposition and its Applications
Tensor Decomposition and its Applications
 
OpenOpt の線形計画で圧縮センシング
OpenOpt の線形計画で圧縮センシングOpenOpt の線形計画で圧縮センシング
OpenOpt の線形計画で圧縮センシング
 
GANs and Applications
GANs and ApplicationsGANs and Applications
GANs and Applications
 
【DL輪読会】Egocentric Video Task Translation (CVPR 2023 Highlight)
【DL輪読会】Egocentric Video Task Translation (CVPR 2023 Highlight)【DL輪読会】Egocentric Video Task Translation (CVPR 2023 Highlight)
【DL輪読会】Egocentric Video Task Translation (CVPR 2023 Highlight)
 
[DL輪読会]Hindsight Experience Replay
[DL輪読会]Hindsight Experience Replay[DL輪読会]Hindsight Experience Replay
[DL輪読会]Hindsight Experience Replay
 
Bayesian Model-Agnostic Meta-Learning
Bayesian Model-Agnostic Meta-LearningBayesian Model-Agnostic Meta-Learning
Bayesian Model-Agnostic Meta-Learning
 
Differential privacy and applications to location privacy
Differential privacy and applications to location privacyDifferential privacy and applications to location privacy
Differential privacy and applications to location privacy
 
Active learning lecture
Active learning lectureActive learning lecture
Active learning lecture
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
 
Connie zweig & steve wolf o jogo das sombras (doc)(rev)
Connie zweig & steve wolf   o jogo das sombras (doc)(rev)Connie zweig & steve wolf   o jogo das sombras (doc)(rev)
Connie zweig & steve wolf o jogo das sombras (doc)(rev)
 
Noisy Labels と戦う深層学習
Noisy Labels と戦う深層学習Noisy Labels と戦う深層学習
Noisy Labels と戦う深層学習
 
Automatic Mixed Precision の紹介
Automatic Mixed Precision の紹介Automatic Mixed Precision の紹介
Automatic Mixed Precision の紹介
 
[DL輪読会]Recent Advances in Autoencoder-Based Representation Learning
[DL輪読会]Recent Advances in Autoencoder-Based Representation Learning[DL輪読会]Recent Advances in Autoencoder-Based Representation Learning
[DL輪読会]Recent Advances in Autoencoder-Based Representation Learning
 
K meansによるクラスタリングの解説と具体的なクラスタリングの活用方法の紹介
K meansによるクラスタリングの解説と具体的なクラスタリングの活用方法の紹介K meansによるクラスタリングの解説と具体的なクラスタリングの活用方法の紹介
K meansによるクラスタリングの解説と具体的なクラスタリングの活用方法の紹介
 
Differentiable Ray Sampling for Neural 3D Representation
Differentiable Ray Sampling for Neural 3D RepresentationDifferentiable Ray Sampling for Neural 3D Representation
Differentiable Ray Sampling for Neural 3D Representation
 
How good is your prediction a gentle introduction to conformal prediction.
How good is your prediction  a gentle introduction to conformal prediction.How good is your prediction  a gentle introduction to conformal prediction.
How good is your prediction a gentle introduction to conformal prediction.
 

More from Terry Taewoong Um

More from Terry Taewoong Um (18)

#44. KAIST에서 "대학 유죄"를 외치다: ART Lab의 도전
#44. KAIST에서 "대학 유죄"를 외치다: ART Lab의 도전#44. KAIST에서 "대학 유죄"를 외치다: ART Lab의 도전
#44. KAIST에서 "대학 유죄"를 외치다: ART Lab의 도전
 
A brief introduction to OCR (Optical character recognition)
A brief introduction to OCR (Optical character recognition)A brief introduction to OCR (Optical character recognition)
A brief introduction to OCR (Optical character recognition)
 
Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynami...
Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynami...Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynami...
Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynami...
 
인공지능의 사회정의의 편이 될 수 있을까? (인공지능과 법)
인공지능의 사회정의의 편이 될 수 있을까? (인공지능과 법)인공지능의 사회정의의 편이 될 수 있을까? (인공지능과 법)
인공지능의 사회정의의 편이 될 수 있을까? (인공지능과 법)
 
Deep learning (Machine learning) tutorial for beginners
Deep learning (Machine learning) tutorial for beginnersDeep learning (Machine learning) tutorial for beginners
Deep learning (Machine learning) tutorial for beginners
 
Deep Variational Bayes Filters (2017)
Deep Variational Bayes Filters (2017)Deep Variational Bayes Filters (2017)
Deep Variational Bayes Filters (2017)
 
Deep Learning: A Critical Appraisal (2018)
Deep Learning: A Critical Appraisal (2018)Deep Learning: A Critical Appraisal (2018)
Deep Learning: A Critical Appraisal (2018)
 
Understanding Black-box Predictions via Influence Functions (2017)
Understanding Black-box Predictions via Influence Functions (2017)Understanding Black-box Predictions via Influence Functions (2017)
Understanding Black-box Predictions via Influence Functions (2017)
 
Learning with side information through modality hallucination (2016)
Learning with side information through modality hallucination (2016)Learning with side information through modality hallucination (2016)
Learning with side information through modality hallucination (2016)
 
Human Motion Forecasting (Generation) with RNNs
Human Motion Forecasting (Generation) with RNNsHuman Motion Forecasting (Generation) with RNNs
Human Motion Forecasting (Generation) with RNNs
 
Deformable Convolutional Network (2017)
Deformable Convolutional Network (2017)Deformable Convolutional Network (2017)
Deformable Convolutional Network (2017)
 
Introduction to Deep Learning with TensorFlow
Introduction to Deep Learning with TensorFlowIntroduction to Deep Learning with TensorFlow
Introduction to Deep Learning with TensorFlow
 
About Two Motion Planning Papers
About Two Motion Planning PapersAbout Two Motion Planning Papers
About Two Motion Planning Papers
 
기계학습(Machine learning) 입문하기
기계학습(Machine learning) 입문하기기계학습(Machine learning) 입문하기
기계학습(Machine learning) 입문하기
 
Lie Group Formulation for Robot Mechanics
Lie Group Formulation for Robot MechanicsLie Group Formulation for Robot Mechanics
Lie Group Formulation for Robot Mechanics
 
로봇과 인공지능, 그리고 미래의 노동
로봇과 인공지능, 그리고 미래의 노동로봇과 인공지능, 그리고 미래의 노동
로봇과 인공지능, 그리고 미래의 노동
 
Introduction to Machine Learning and Deep Learning
Introduction to Machine Learning and Deep LearningIntroduction to Machine Learning and Deep Learning
Introduction to Machine Learning and Deep Learning
 
Lie Group Formulation for Robot Mechanics
Lie Group Formulation for Robot MechanicsLie Group Formulation for Robot Mechanics
Lie Group Formulation for Robot Mechanics
 

Recently uploaded

Call Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
Call Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort ServiceCall Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
Call Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
9953056974 Low Rate Call Girls In Saket, Delhi NCR
 
Top Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoor
Top Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoorTop Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoor
Top Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoor
dharasingh5698
 
Integrated Test Rig For HTFE-25 - Neometrix
Integrated Test Rig For HTFE-25 - NeometrixIntegrated Test Rig For HTFE-25 - Neometrix
Integrated Test Rig For HTFE-25 - Neometrix
Neometrix_Engineering_Pvt_Ltd
 
Cara Menggugurkan Sperma Yang Masuk Rahim Biyar Tidak Hamil
Cara Menggugurkan Sperma Yang Masuk Rahim Biyar Tidak HamilCara Menggugurkan Sperma Yang Masuk Rahim Biyar Tidak Hamil
Cara Menggugurkan Sperma Yang Masuk Rahim Biyar Tidak Hamil
Cara Menggugurkan Kandungan 087776558899
 
notes on Evolution Of Analytic Scalability.ppt
notes on Evolution Of Analytic Scalability.pptnotes on Evolution Of Analytic Scalability.ppt
notes on Evolution Of Analytic Scalability.ppt
MsecMca
 

Recently uploaded (20)

COST-EFFETIVE and Energy Efficient BUILDINGS ptx
COST-EFFETIVE  and Energy Efficient BUILDINGS ptxCOST-EFFETIVE  and Energy Efficient BUILDINGS ptx
COST-EFFETIVE and Energy Efficient BUILDINGS ptx
 
Introduction to Serverless with AWS Lambda
Introduction to Serverless with AWS LambdaIntroduction to Serverless with AWS Lambda
Introduction to Serverless with AWS Lambda
 
Hazard Identification (HAZID) vs. Hazard and Operability (HAZOP): A Comparati...
Hazard Identification (HAZID) vs. Hazard and Operability (HAZOP): A Comparati...Hazard Identification (HAZID) vs. Hazard and Operability (HAZOP): A Comparati...
Hazard Identification (HAZID) vs. Hazard and Operability (HAZOP): A Comparati...
 
data_management_and _data_science_cheat_sheet.pdf
data_management_and _data_science_cheat_sheet.pdfdata_management_and _data_science_cheat_sheet.pdf
data_management_and _data_science_cheat_sheet.pdf
 
Call Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
Call Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort ServiceCall Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
Call Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
 
Design For Accessibility: Getting it right from the start
Design For Accessibility: Getting it right from the startDesign For Accessibility: Getting it right from the start
Design For Accessibility: Getting it right from the start
 
Employee leave management system project.
Employee leave management system project.Employee leave management system project.
Employee leave management system project.
 
FEA Based Level 3 Assessment of Deformed Tanks with Fluid Induced Loads
FEA Based Level 3 Assessment of Deformed Tanks with Fluid Induced LoadsFEA Based Level 3 Assessment of Deformed Tanks with Fluid Induced Loads
FEA Based Level 3 Assessment of Deformed Tanks with Fluid Induced Loads
 
Top Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoor
Top Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoorTop Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoor
Top Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoor
 
(INDIRA) Call Girl Meerut Call Now 8617697112 Meerut Escorts 24x7
(INDIRA) Call Girl Meerut Call Now 8617697112 Meerut Escorts 24x7(INDIRA) Call Girl Meerut Call Now 8617697112 Meerut Escorts 24x7
(INDIRA) Call Girl Meerut Call Now 8617697112 Meerut Escorts 24x7
 
Integrated Test Rig For HTFE-25 - Neometrix
Integrated Test Rig For HTFE-25 - NeometrixIntegrated Test Rig For HTFE-25 - Neometrix
Integrated Test Rig For HTFE-25 - Neometrix
 
UNIT - IV - Air Compressors and its Performance
UNIT - IV - Air Compressors and its PerformanceUNIT - IV - Air Compressors and its Performance
UNIT - IV - Air Compressors and its Performance
 
Unit 1 - Soil Classification and Compaction.pdf
Unit 1 - Soil Classification and Compaction.pdfUnit 1 - Soil Classification and Compaction.pdf
Unit 1 - Soil Classification and Compaction.pdf
 
Unit 2- Effective stress & Permeability.pdf
Unit 2- Effective stress & Permeability.pdfUnit 2- Effective stress & Permeability.pdf
Unit 2- Effective stress & Permeability.pdf
 
Cara Menggugurkan Sperma Yang Masuk Rahim Biyar Tidak Hamil
Cara Menggugurkan Sperma Yang Masuk Rahim Biyar Tidak HamilCara Menggugurkan Sperma Yang Masuk Rahim Biyar Tidak Hamil
Cara Menggugurkan Sperma Yang Masuk Rahim Biyar Tidak Hamil
 
notes on Evolution Of Analytic Scalability.ppt
notes on Evolution Of Analytic Scalability.pptnotes on Evolution Of Analytic Scalability.ppt
notes on Evolution Of Analytic Scalability.ppt
 
University management System project report..pdf
University management System project report..pdfUniversity management System project report..pdf
University management System project report..pdf
 
Thermal Engineering Unit - I & II . ppt
Thermal Engineering  Unit - I & II . pptThermal Engineering  Unit - I & II . ppt
Thermal Engineering Unit - I & II . ppt
 
Block diagram reduction techniques in control systems.ppt
Block diagram reduction techniques in control systems.pptBlock diagram reduction techniques in control systems.ppt
Block diagram reduction techniques in control systems.ppt
 
Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Boo...
Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Boo...Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Boo...
Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Boo...
 

On Calibration of Modern Neural Networks (2017)