SlideShare a Scribd company logo
1 of 24
Download to read offline
GEX: A flexible method for approximating influence
via Geometric Ensemble (NeurIPS 2023)
Sung-Yub Kim (presenter), Kyungsu Kim, Eunho Yang
Oct. 6th, 2023
TL;DR
• We empirically verify that the proposed method improves the downstream task performance and
relieves the framework dependency of IF (and its approximations).
Overview
Related works
[1]Cook,R.Dennis."Detectionofinfluentialobservationinlinearregression."Technometrics19.1(1977):15-18.
[2] Koh, Pang Wei, and Percy Liang. "Understanding black-box predictions via influence functions." International conference on machine learning. PMLR, 2017.
[3]Pruthi,Garima,etal."Estimatingtrainingdatainfluencebytracinggradientdescent."AdvancesinNeuralInformationProcessingSystems33(2020):19920-19930.
[4]Schioppa,Andrea,etal."Scalingupinfluencefunctions."ProceedingsoftheAAAIConferenceonArtificialIntelligence.Vol.36.No.8.2022.
[5]Sorscher,Ben,etal."Beyondneuralscalinglaws:beatingpowerlawscalingviadatapruning."AdvancesinNeuralInformationProcessingSystems35(2022):19523-19536.
[6]Paul,Mansheej,SuryaGanguli,andGintareKarolinaDziugaite."Deeplearningonadatadiet:Findingimportantexamplesearlyintraining."AdvancesinNeuralInformationProcessingSystems34(2021):
20596-20607.
[7]Song,Jaeyun,Kim,SungYub,andYang,Eunho."RGE:ARepulsiveGraphRectificationforNodeClassificationviaInfluence."InternationalConferenceonMachineLearning.PMLR,2023.
[8]Kim,SungYub,Kim,Kyungsu,Yang,Eunho.“GEX:AflexiblemethodforapproximatinginfluenceviaGeometricEnsemble”.AdvancesinNeuralInformationProcessingSystems(2023)
[9]Song,Jaeyun,etal.“Post-TrainingRecoveryfromInjectedBiaswithSelf-Influence”,Underreview,2023
[10]Daxberger,Erik,etal."Laplaceredux-effortlessbayesiandeeplearning."AdvancesinNeuralInformationProcessingSystems34(2021):20089-20103.
[11]Kong,Zhifeng,andKamalikaChaudhuri."Understandinginstance-basedinterpretabilityofvariationalauto-encoders."AdvancesinNeuralInformationProcessingSystems34(2021):2400-2412.
[12]Terashita,Naoyuki,etal."InfluenceEstimationforGenerativeAdversarialNetworks."InternationalConferenceonLearningRepresentations.2020.
[13]Chen,Zizhang,etal."CharacterizingtheInfluenceofGraphElements."TheEleventhInternationalConferenceonLearningRepresentations.2022.
[14]Grosse,Roger,etal."StudyingLargeLanguageModelGeneralizationwithInfluenceFunctions."arXivpreprintarXiv:2308.03296(2023).
[15]https://medium.com/data-science-in-your-pocket/random-projection-for-dimension-reduction-27d2ec7d40cd
[16]Arnoldi,WalterEdwin."Theprincipleofminimizediterationsinthesolutionofthematrixeigenvalueproblem."Quarterlyofappliedmathematics9.1(1951):17-29.
[17]Garipov,Timur,etal."Losssurfaces,modeconnectivity,andfastensemblingofdnns."Advancesinneuralinformationprocessingsystems31(2018).
[18]Otsu,Nobuyuki."Athresholdselectionmethodfromgray-levelhistograms."IEEEtransactionsonsystems,man,andcybernetics9.1(1979):62-66.
[19]Wei,Jiaheng,etal."LearningwithNoisyLabelsRevisited:AStudyUsingReal-WorldHumanAnnotations."InternationalConferenceonLearningRepresentations.2021.
[20]Kong,Shuming,YanyanShen,andLinpengHuang."Resolvingtrainingbiasesviainfluence-baseddatarelabeling."InternationalConferenceonLearningRepresentations.2021.
[21]He,Kaiming,etal."Deepresiduallearningforimagerecognition."ProceedingsoftheIEEEconferenceoncomputervisionandpatternrecognition.2016.
[22]Vaswani,Ashish,etal."Attentionisallyouneed."Advancesinneuralinformationprocessingsystems30(2017).
[23]Kim,SungYub,etal."Scale-invariantBayesianNeuralNetworkswithConnectivityTangentKernel."TheEleventhInternationalConferenceonLearningRepresentations.2023.
[24]Andriushchenko,Maksym,etal."Amodernlookattherelationshipbetweensharpnessandgeneralization." InternationalConferenceonMachineLearning.PMLR,2023.
[25]Cohen,Jeremy,etal."GradientDescentonNeuralNetworksTypicallyOccursattheEdgeofStability."InternationalConferenceonLearningRepresentations.2020.
[26]Damian,Alex,EshaanNichani,andJasonD.Lee."Self-Stabilization:TheImplicitBiasofGradientDescentattheEdgeofStability."TheEleventhInternationalConferenceonLearningRepresentations.
2022.
[27]Song,Minhak,andChulheeYun."TrajectoryAlignment:UnderstandingtheEdgeofStabilityPhenomenonviaBifurcationTheory."arXivpreprintarXiv:2307.04204(2023).
1. Introduction
Influence Function
Influence Function
<latexit sha1_base64="AJLAeDwPQM3rbP2oKayvxEaWark=">AAACWXicbVBdaxNBFJ1dbY3xo9E++jIYhCg17IraVhEKvvggEj/SFjJpmJ3cTYbOziwzdwvpsH/SB0H8Kz44m+yDth64cDjnXu69JyuVdJgkP6P4xs2t7Vud2907d+/d3+k9eHjsTGUFjIVRxp5m3IGSGsYoUcFpaYEXmYKT7Px9459cgHXS6G+4KmFa8IWWuRQcgzTrlQyXgPzs2eySvnnHKj0PzYB+I1MmNWUFx2WW+S/12aj2zJRgORqreQGe20UhdV2zt/Tj4Ose3Yw9pc8pyy0XnoFSg8u9Vq79p3rW6yfDZA16naQt6ZMWo1nvO5sbURWgUSju3CRNSpyGzSiFgrrLKgclF+d8AZNAm7Pc1K+TqemToMxpbmwojXSt/j3heeHcqshCZ/Olu+o14v+8SYX5wdRLXVYIWmwW5ZWiaGgTM51LCwLVKhAurAy3UrHkIRIM+XZDCOnVl6+T4xfD9PXw1eeX/aPDNo4OeUQekwFJyT45Ih/IiIyJID/I72gr2o5+xVHcibub1jhqZ3bJP4h3/wCCJbTC</latexit>
✓⇤
z := argmin
✓2RP
L(S, ✓)
`(z, ✓)
N
on another instance z!
∈ ℝ"
<latexit sha1_base64="rqko3S5rwHjmaf4C5ABZwHraGWY=">AAACNHicbVDLSgNBEJz1bXxFPXoZDGIiGnbFNwiCF0VBBaNCNi6zk44ZnH0w0yskSz7Kix/iRQQPinj1G5zEHNRYMFBUddNT5cdSaLTtZ6uvf2BwaHhkNDM2PjE5lZ2eudBRojiUeCQjdeUzDVKEUEKBEq5iBSzwJVz6t/tt//IOlBZReI6NGCoBuwlFTXCGRvKyR27AsM6ZTA9bXocjpscnJ618c7m5WKA7u9QFKfPNxWXqYh2QXS95zQJd6ZULXjZnF+0OaC9xuiRHujj1so9uNeJJACFyybQuO3aMlZQpFFxCK+MmGmLGb9kNlA0NWQC6knZCt+iCUaq0FinzQqQd9edGygKtG4FvJtux9F+vLf7nlROsbVVSEcYJQsi/D9USSTGi7QZpVSjgKBuGMK6E+SvldaYYR9NzxpTg/I3cSy5Wi85Gcf1sLbe33a1jhMyReZInDtkke+SAnJIS4eSePJFX8mY9WC/Wu/XxPdpndXdmyS9Yn1/ttah3</latexit>
ILOO(z, z0
) := `(z0
, ✓⇤
z ) `(z0
, ✓⇤
)
<latexit sha1_base64="NhtnBVAUrxqI/4QkdlBFLBS0k0U=">AAACO3icbVBNSxtBGJ7V+tHUatSjl6GhYKWEXfFbhIAXDx5ibFTIxjA7eZMMzs4sM+8KYdn/1Uv/hDcvvXhoEa/enU1yqNoXBh6e5/2Y54kSKSz6/r03Nf1hZnZu/mPp08LnxaXy8sqF1anh0ORaanMVMQtSKGiiQAlXiQEWRxIuo5vjQr+8BWOFVj9wmEA7Zn0leoIzdFSn3AhxAMiuNw6OwlR1XSdgNuZoKBQNY4aDKMoa+XU9z0KdgGGojWIxZMz0Y6HyPDykp+vn3+l47FunXPGr/qjoexBMQIVMqt4p34VdzdMYFHLJrG0FfoJttx4Fl5CXwtRCwvgN60PLweK2bWcj7zn96pgu7WnjnkI6Yv+dyFhs7TCOXGdhxb7VCvJ/WivF3l47EypJERQfH+qlkqKmRZC0KwxwlEMHGDfC/ZXyATOMowux5EII3lp+Dy42q8FOdftsq1Lbn8QxT9bIF7JOArJLauSE1EmTcPKT/CZ/yF/vl/fgPXpP49YpbzKzSl6V9/wCkeuu8g==</latexit>
✓⇤
:= argmin
✓2RP
L(S, ✓)
Since retraining is computationally intractable, [2]
proposed an efficient approximation,
named Influence Function (IF):
This can be interpreted as a two-step approximation of
retraining effect
<latexit sha1_base64="m+8X1kuK18A0q/05NrLtZWw8sZ0=">AAACFXicbVDLSgMxFM3UV62vUZdugkVaoZYZ8Q1CwU3dVbAP6ItMmmlDMw+SjNAO8xNu/BU3LhRxK7jzb8y0s9DWA4HDOfdyc47lMyqkYXxrqYXFpeWV9GpmbX1jc0vf3qkJL+CYVLHHPN6wkCCMuqQqqWSk4XOCHIuRujW8if36A+GCeu69HPmk7aC+S22KkVRSVy+0HCQHGLHwNsqPC+PcIby6hv1uOM5FnZb0fFjuhEdmFCtRV88aRWMCOE/MhGRBgkpX/2r1PBw4xJWYISGapuHLdoi4pJiRKNMKBPERHqI+aSrqIoeIdjhJFcEDpfSg7XH1XAkn6u+NEDlCjBxLTcYZxKwXi/95zUDaF+2Qun4giYunh+yAQenBuCLYo5xgyUaKIMyp+ivEA8QRlqrIjCrBnI08T2rHRfOseHp3ki1dJnWkwR7YB3lggnNQAmVQAVWAwSN4Bq/gTXvSXrR37WM6mtKSnV3wB9rnD7fAneo=</latexit>
I(z, z0
) := g>
z0 H 1
gz
<latexit sha1_base64="fumyYRuOF/qnlPSScjWGdh/lGrM=">AAAC83icbVJda9swFJW9rzb7aLo97kUsbElKG+zCvh4Khb10MGgHS1uIEiMrSiIqy0KSxxKhv7GXPWyMve7P7G3/ZrITQ9rsguH43HOu75GVSs60iaK/QXjr9p2797a2G/cfPHy009x9fK7zQhHaJznP1WWKNeVM0L5hhtNLqSjOUk4v0qt3Zf/iM1Wa5eKTmUs6zPBUsAkj2Hgq2Q22UUqnTFjM2VTsuQb0hTJsZgRz+94lFTbGfjg9dZ3FPly0u5XmBcJSqvwLRJTzUaVSmfVruMQiM6MGj/a8ob0P67fELlwXHmwarqm6CC3nH8Gpd7TdCJlcws76lHJILYe1vt5n3XUysgexqygHj9Zz+ShlEkTFuI6eNFtRL6oKboJ4BVpgVWdJ8w8a56TIqDCEY60HcSTN0GJlGOHUNVChqcTkCk/pwEOBM6qHtvpnDj73zBhOcuUfYWDFrjsszrSeZ6lXllvrm72S/F9vUJjJm6FlQhaGCrL80KTg0OSwvABwzBQlhs89wEQxvyskM6wwMf6aNPwhxDcjb4Lzw178qvfy42Hr+O3qOLbAU/AMdEAMXoNjcALOQB+QQAZfg+/Bj7AIv4U/w19LaRisPE/AtQp//wOR/+cp</latexit>
ILOO(z, z0
) ⇡ `lin
✓⇤ (z0
, ✓⇤
z ) `lin
(z0
, ✓⇤
)
= g>
z0 (✓⇤
z ✓⇤
)
⇡ g>
z0 H 1
gz = I(z, z0
)
Influence Function
Two limitations of IF
Two limitations of IF
2. Identifying distributional bias in IF and its approximations
Distributional bias in IF and its approximations
<latexit sha1_base64="lL3vvGifEpow471EGLQ7F4WhACM=">AAACR3icbVBNaxsxFNQ6Seu4X25z7EXElCRQzG7pRxowGHxJbgnEdsDrLFpZ64hotYv0NmAL/btccs2tf6GXHhJCjtFufGjtDgjNm3kP6U2cC67B9395tbX1jRcv65uNV6/fvH3XfP9hoLNCUdanmcjUWUw0E1yyPnAQ7CxXjKSxYMP4slf6wyumNM/kKcxyNk7JVPKEUwJOiprnYUrgghJhjmxUcQBzqgg9knZ3/nm+s9fADged6goT55jAmp4NdZFGhnYCe+6qaWTmO47RELLc4rIsKxs1W37br4BXSbAgLbTAcdS8DScZLVImgQqi9SjwcxgbooBTwWwjLDTLCb0kUzZyVJKU6bGpcrD4k1MmOMmUOxJwpf49YUiq9SyNXWe5qV72SvF/3qiAZH9suMwLYJI+P5QUAkOGy1DxhCtGQcwcIVRx91dML4jLClz0DRdCsLzyKhl8aQff299Ovra6Pxdx1NFHtI12UYB+oC46RMeojyi6Rr/RHbr3brw/3oP3+Nxa8xYzW+gf1LwnZf2xHw==</latexit>
ITracIn(z, z0
) :=
1
C
C
X
c=1
gc>
z0 gc
z
<latexit sha1_base64="Xjnu88gy+RPwO3H+1aB3E6C0IIg=">AAACVnicbVFNaxsxENWumyZ1mnbbHnsRNSEpBLNb+g2GQC71zQ5xEvA6i1bWOiKSdpFmC7bQn2wv7U/ppVS79qF1OjDozXszSPOUV4IbiOOfQdh5sPNwd+9Rd//xwZOn0bPnl6asNWUTWopSX+fEMMEVmwAHwa4rzYjMBbvK784a/eor04aX6gKWFZtJslC84JSAp7JIppLALSXCDl3WYgB7oQkdqvORO16drI5ed7GPz4P2SAuv2cTZM5eaWmaWDhJ346tFZldHHtEUysrhcXbe5E1T4UZrJJdFvbgft4Hvg2QDemgToyz6ls5LWkumgApizDSJK5hZooFTwVw3rQ2rCL0jCzb1UBHJzMy2tjh86Jk5LkrtUwFu2b8nLJHGLGXuO5vFzbbWkP/TpjUUH2eWq6oGpuj6oqIWGErceIznXDMKYukBoZr7t2J6S7xx4H+i601Itle+Dy7f9JP3/Xfjt73TTxs79tBL9AodowR9QKfoCxqhCaLoO/oVhEEn+BH8DnfC3XVrGGxmXqB/Ioz+AJLfsqw=</latexit>
ITracInRP(z, z0
) :=
1
C
C
X
c=1
gc>
z0 QRQ>
Rgc
z
<latexit sha1_base64="C5MaCNLjqlxsGTzGITmp5Mmztak=">AAACRnicbVBNaxRBEK1Zv5L1I6s5emlcJBF0mRE1MSBEcongIQY3CexMhp6e3k2Tnu6hu0bYbebXefGcmz/Bi4eEkKs9s3vQxAdNP96roqpeVkphMQx/Bp1bt+/cvbe03L3/4OGjld7jJwdWV4bxIdNSm6OMWi6F4kMUKPlRaTgtMskPs9Odxj/8xo0VWn3FacmTgk6UGAtG0UtpL4kLiieMSvepTluO6D4apWUu6vXZy9naiy7x2PrQfpPUzdbq4xh1SYbpPok/+1E5Td1+fexeRXUjzt2msk57/XAQtiA3SbQgfVhgL+2dxblmVcEVMkmtHUVhiYmjBgWTvO7GleUlZad0wkeeKlpwm7g2hpo890pOxtr4p5C06t8djhbWTovMVzaH2uteI/7PG1U43kycUGWFXLH5oHElCWrSZEpyYThDOfWEMiP8roSdUEMZ+uS7PoTo+sk3ycHrQfRu8PbLm/72+0UcS/AUnsE6RLAB27ALezAEBt/hF5zDRfAj+B1cBlfz0k6w6FmFf9CBPyenry4=</latexit>
IArnoldi(z, z0
) := g>
z0 UR⇤ 1
R U>
R gz
Distributional bias in IF and its approximations
<latexit sha1_base64="f47u6XDbCSPi2grflbpZy6fnKhM=">AAAFzHicjVTbbhMxEJ1CAqXcWnjkxaJFaqU0Soq4PSBVKgIqAQqoN6kJkdfrTVb17ka2tzRd5ZUP5I1P4JE/4NjZtuklFSt5PT4z9sw5491goGJjG43fMzduVqq3bs/embt77/6Dh/MLj3ZMlmsht0WmMr0XcCNVnMptG1sl9wZa8iRQcjc42HD+3UOpTZylW3Y4kJ2E99I4igW3gLKFmb/UJkuSjvAOKKKCWqQpowGGoRhoDCslRs+pTk0awVqmd0ANfBpzQPlpFCcFf4AVh59hTsu18rYEroEYWArZVj0awc6BpCTwXkEeRp9gWcy5P8edZmAnQBktUY+6dIyZYW+G3QrvH2Wk87cwVinE6Yk/15yrz1XOUZXyp4VTuDieErX0MGpYfcCe3Nfh2KUe2/CYoD4N/drhoa/+Dx0Cm376ykT0En3Gu12q1UbNHHF9xAfoxzdo/t33hflejTk5ri14lrzKJ8wnu3bo2S17rROsV5EzhB35PLHvOitzueqOvO5byCtLdtd16aS31seP+6BxGsMceU5nPITXvaBN1NsF12X0robhNHiL4bpZYO14Oobu9jGvyZlnWq9zzyUBFpbdnaa4Y/cea13WrP0uDavm2UT/cbMu3oHp2Wqn2kyqdL2iV2Xseb+rlvtv6Bh2WOrr7l0MzdZOb/uVvLvzi416wz/sstEsjUUqn1Z3/lc7zESeyNQKxY3ZbzYGtlNwbWOh5GiunRs54OKA9+Q+zJQn0nQK/zMasWdAQhZlGiO1zKOTOwqeGDNMAkQm3PbNRZ8Dr/Lt5zZ63SnidJBbmYpxoihXzGbM/dlYGGsprBrC4ELHqJWJPtdcWPz/5iBC8yLly8bOWr35sv7i69ri+ptSjll6Qk9xZZv0itbpIz66bRKVzUpWOaoMq1+qtlpUR+PQGzPlnsd07qn+/AdL/z9U</latexit>
Proposition 3.1 (Distributional bias in bilinear self-influence). Let us as-
sume gz follows a P-dimensional stable distribution (e.g., Gaussian, Cauchy, and
Lévy distribution) and M 2 RP ⇥P
is a positive (semi-)definite matrix. Then,
self-influence in the form of IM (z, z) = g>
z Mgz follows a unimodal distribution.
Furthermore, if gz follows a Gaussian distribution, then the self-influence follows
a generalized 2
-distribution.
A simple case study in a modified two-circle dataset
<latexit sha1_base64="tEVn5IcyeMefnGdLs/hWXGoL6PM=">AAACAXicbZDLSsNAFIYn9VbrLepGcBMsgquSSL0sC24UhFawF2hDmEwn7dDJhZkToYS48VXcuFDErW/hzrdxkmahrT8MfPznHOac3404k2Ca31ppaXllda28XtnY3Nre0Xf3OjKMBaFtEvJQ9FwsKWcBbQMDTnuRoNh3Oe26k6us3n2gQrIwuIdpRG0fjwLmMYJBWY5+MPAxjAnmyU3q5AyQ3DabqaNXzZqZy1gEq4AqKtRy9K/BMCSxTwMgHEvZt8wI7AQLYITTtDKIJY0wmeAR7SsMsE+lneQXpMaxcoaGFwr1AjBy9/dEgn0pp76rOrMd5XwtM/+r9WPwLu2EBVEMNCCzj7yYGxAaWRzGkAlKgE8VYCKY2tUgYywwARVaRYVgzZ+8CJ3TmnVeO7urVxv1Io4yOkRH6ARZ6AI10DVqoTYi6BE9o1f0pj1pL9q79jFrLWnFzD76I+3zBwqwlz0=</latexit>
ILOO
<latexit sha1_base64="sRYssw0GitVG1c+smryccc5XV6I=">AAACA3icbVDLSsNAFJ34rPUVdaebwSLUTUnEF4hQdKO7CvYBTSg300k7dPJgZiKUUHDjr7hxoYhbf8Kdf+OkzUJbD1w4nHMv997jxZxJZVnfxtz8wuLScmGluLq2vrFpbm03ZJQIQusk4pFoeSApZyGtK6Y4bcWCQuBx2vQG15nffKBCsii8V8OYugH0QuYzAkpLHXPXCUD1CfD0doSdi7IDPO7DpW1Zhx2zZFWsMfAssXNSQjlqHfPL6UYkCWioCAcp27YVKzcFoRjhdFR0EkljIAPo0bamIQRUuun4hxE+0EoX+5HQFSo8Vn9PpBBIOQw83ZldLKe9TPzPayfKP3dTFsaJoiGZLPITjlWEs0BwlwlKFB9qAkQwfSsmfRBAlI6tqEOwp1+eJY2jin1aObk7LlWv8jgKaA/tozKy0RmqohtUQ3VE0CN6Rq/ozXgyXox342PSOmfkMzvoD4zPH+s/lmg=</latexit>
I (↵ = 100)
<latexit sha1_base64="ZmlcEXoPjl2BxdNi7DFGhbT3CY8=">AAACBHicbVDLSsNAFJ34rPUVddnNYBHqpiTiC0QoutFdBfuAJpSb6aQdOnkwMxFK6MKNv+LGhSJu/Qh3/o2TNgttPTDM4Zx7ufceL+ZMKsv6NhYWl5ZXVgtrxfWNza1tc2e3KaNEENogEY9E2wNJOQtpQzHFaTsWFAKP05Y3vM781gMVkkXhvRrF1A2gHzKfEVBa6polJwA1IMDT2zF2LioO8HgAl1bVsg+7Zln/E+B5YuekjHLUu+aX04tIEtBQEQ5SdmwrVm4KQjHC6bjoJJLGQIbQpx1NQwiodNPJEWN8oJUe9iOhX6jwRP3dkUIg5SjwdGW2spz1MvE/r5Mo/9xNWRgnioZkOshPOFYRzhLBPSYoUXykCRDB9K6YDEAAUTq3og7Bnj15njSPqvZp9eTuuFy7yuMooBLaRxVkozNUQzeojhqIoEf0jF7Rm/FkvBjvxse0dMHIe/bQHxifP17olqA=</latexit>
I (↵ = 0.01)
<latexit sha1_base64="AoSyz/4d860DiF5Wn/pRjBOqz3U=">AAACCnicbVC7SgNBFJ2Nrxhfq5Y2o0GICGFXfDVC0CZ2EcwDsku4O5lNhsw+mJkVwpLaxl+xsVDE1i+w82+cTVJo4oGBM+fcy733eDFnUlnWt5FbWFxaXsmvFtbWNza3zO2dhowSQWidRDwSLQ8k5SykdcUUp61YUAg8Tpve4Cbzmw9USBaF92oYUzeAXsh8RkBpqWPuV0sO8LgPR/gKV/ExnvywE4Dqe356O+qYRatsjYHniT0lRTRFrWN+Od2IJAENFeEgZdu2YuWmIBQjnI4KTiJpDGQAPdrWNISASjcdnzLCh1rpYj8S+oUKj9XfHSkEUg4DT1dmG8pZLxP/89qJ8i/dlIVxomhIJoP8hGMV4SwX3GWCEsWHmgARTO+KSR8EEKXTK+gQ7NmT50njpGyfl8/uTouV62kcebSHDlAJ2egCVVAV1VAdEfSIntErejOejBfj3fiYlOaMac8u+gPj8wcybpi7</latexit>
H(↵) = H + ↵I
3. Resolving distributional bias via Geometric Ensemble
Removing Linearization
<latexit sha1_base64="P2Y6+MzRDhxgW0qX2P+Pb+C/7Uk=">AAAEkXiclVNtTxNBED5oVayKIB/9MpEqraFNS1CBpAZEDURiMOEt6bXN3nau3bC3e+7u8Xbe7/H/+M1/4961RQok4iSXm8w88/bMjhdypk2t9ntiMpe/d//B1MPCo8dPpp/OzD470DJSFPep5FIdeUQjZwL3DTMcj0KFJPA4HnrHm6n/8ASVZlLsmfMQWwHpCeYzSow1dWYnf7oGz4znx3t9lAoDWK7WEyhtSiGQphjw0JwiCtj+DER0YWejXIWiGxDTp4TH20kRmIDm0koLKEnRgGe2Ba2xC0RDwfWwx0RMOOuJ10kBrFwJLl0swsVCOTO/akD2jzO/58Wfkk7shpqBq1kAYSezGxPvbCTDRBx90xyoH5EbAi5y3s5wKogtKWkG00dD2q+zWmm6Mri0K83dQhZGMYMqivX6pjWqTjzkMX5fY36lbyeuUHmSFFwU3ctxT/uo0NL1P92tNe4IrPwLN9LL0IBe56LtGhlCKaPUxo6cxWyvxTF+7VI1WADskJATikDCUMkzZhF2r6HUBhWT6rbtjuUp/R1ptPSvSba2UjYE/LjsYxG22nHFPr4ByeXqGJOdmflatZYJ3FTqQ2XeGcpuZ+aX25U0ClAYyonWzXotNK2YKMMoR7umSGNI6DHpYdOqggSoW3F2UQm8jNLn60tlP2Egs16NiEmg9XngWWQ6lr7uS423+ZqR8VdaMRNhZFDQQSE/4mAkpOcJXabs2fFzqxCqmO0VaJ8oQi3fumBJqF8f+aZysFStv62++bY0v746pGPKee68cEpO3XnnrDtbzq6z79DcdG4518i9z8/lV/Pr+Q8D6OTEMGbOGZP8lz9F7nSe</latexit>
Theorem 4.1 (Connection between IF and LA). I in [28] can be expressed
as
I(z, z0
) = E ⇠pLA
⇥
`lin
✓⇤ (z, ) · `lin
✓⇤ (z0
, )
⇤
where `lin
✓⇤ (z, ) := `lin
✓⇤ (z, ) `lin
✓⇤ (z, ✓⇤
) = g>
z ( ✓⇤
) and pLA is the Laplace
approximated posterior
pLA( ) := N |✓⇤
, H 1
.
<latexit sha1_base64="M+LVLxJYyTlnS/UIE0jcBjaofaM=">AAACknicdVHbatwwEJXdpkm3l2wvb30RXUqSUha7JL0EciMNpJCHFLpJYO2asXa8KyJfkMaBjfEH5Xfy1r+p7N1CLu0BweHMmZF0Ji6UNOR5vx33wcOFR4tLjztPnj57vtx98fLE5KUWOBC5yvVZDAaVzHBAkhSeFRohjRWexuf7Tf30ArWRefaTpgWGKYwzmUgBZKWoexWkQBMBqvpeRy0nqo726tXLD5cra3xzi7diHFcHdVQFhZE8MDLlxU1z3eEWgcKEhjP6DRUBD1Ap20QTJPj1vpnJmwlrPBCjnP7rWvlrm83ScjyhsB91e17fa8HvE39OemyO46h7HYxyUaaYkVBgzND3Cgor0CSFwroTlAYLEOcwxqGlGaRowqqNtObvrDLiSa7tyYi36s2OClJjpmlsnU0O5m6tEf9VG5aUfAkrmRUlYSZmFyWl4pTzZj98JDUKUlNLQGhp38rFBDQIslvs2BD8u1++T04+9v1P/Y0f673dr/M4ltgb9patMp99ZrvskB2zARPOsrPhbDs77mt3091z92dW15n3vGK34B79Abh5w8Q=</latexit>
ILA(z, z0
) := E ⇠pLA
[ `✓⇤ (z, ) · `✓⇤ (z0
, )] .
<latexit sha1_base64="srbKo/FwyKi7kz4Ynx9o8Z1llTs=">AAACLXicbVDLSgMxFM3UV62vqks3wSJU0TIjvkEo6MKlgtVCp5ZMetuGZh4kd4Q69Ifc+CsiuKiIW3/DtFbU6oHAyTnnktzjRVJotO2elRobn5icSk9nZmbn5heyi0tXOowVhxIPZajKHtMgRQAlFCihHClgvifh2muf9P3rW1BahMEldiKo+qwZiIbgDI1Uy566pyCRURekrCUutgDZzUY3f7dJ3UiLdXp0TAfmt7L1df9Kr9eyObtgD0D/EmdIcmSI81r2ya2HPPYhQC6Z1hXHjrCaMIWCS+hm3FhDxHibNaFiaMB80NVksG2XrhmlThuhMidAOlB/TiTM17rjeybpM2zpUa8v/udVYmwcVBMRRDFCwD8fasSSYkj71dG6UMBRdgxhXAnzV8pbTDGOpuCMKcEZXfkvudouOHuF3YudXPFwWEearJBVkicO2SdFckbOSYlwck8eSY+8WA/Ws/VqvX1GU9ZwZpn8gvX+ARRPpYk=</latexit>
`✓⇤ (z, ) := `(z, ) `(z, ✓⇤
)
Utilization of Geometric Ensemble
<latexit sha1_base64="gRUshho/+GpXwFSAGbUgD/oKvMw=">AAAEIXicbVNLbxMxEN4mPEp4tXDkMqKLlJY0SipBOVCpAglVahUF+pS6beR1ZrNWvfZie0uT1f4VLvwVLhxAqDfEn2GyaUtbmIM1+jyPb8afw1QK61qtX1OV6o2bt25P36ndvXf/wcOZ2Uc7VmeG4zbXUpu9kFmUQuG2E07iXmqQJaHE3fDo7fh+9xiNFVptuWGKBwkbKBEJzhxBvdnKcuDwxIVR3jU61VaM4QLqa2itYAqsUINMMiPcECJtQFOxxZQZlqBDI0bYh07HzjdhAx1kFpi1WYLAgFgsOsOIVh8u4sEPXIyOHS5AIBQECXNxGOYfisO8W/jAeCzwGC2M0Ggos6k9SG0t+Bv1zQacp8/DCrT8kpH9mDFDTdAYbZqwFaNqgL/mQ8yIjgOJzDrwu4uddb8svIhigOqYyYw6jSvQAGAzHoOLKd7vrMNr6PpNeJcZamcSbbABIgL/xAdBNRUIlWYOdPSXo2VJKhH8UTnYpt+gYqjGB7WQUn8aB8Va9ictx/iEBnKnDY0X5JD1BARFLxcrnfXn7clOakGIA6FyJsVALRQ1IBv0RoeB02mZsAKBYqFkJdLLAxohHxYFBChlfXRpY0Gm+mhCwzjmV1PKiCKqnzQugqkyEZm8T5S3qB7tuxag6p8z6c3MtZqt0uBfp33mzHln1u3NnAZ9zUkcynFJMtlvt1J3kDPjBJdY1ILMYsr4ERvgPrmKFGMP8lLhBTwjpF9uLtLKQYlezshZYu0wCSlyzNlevxuD/7vbz1z06iAv3xMVnzSKMgmO9EffBfrC0AvJITmM0y8QHHhMcuakZlujJbSvj/yvs7PUbL9svni/NLf65mwd094T76lX99resrfqrXldb9vjlc+Vr5XvlR/VL9Vv1Z/V00loZeos57F3xaq//wD+QVa0</latexit>
Proposition (Hessian singularity for over-parameterized NNs). Let us as-
sume a pre-trained parameter ✓⇤
2 RP
achieves zero training loss L(S, ✓⇤
) = 0
for squared error. Then, H has at least P NK zero-eigenvalues for NNs such
that NK < P. Furthermore, if x is an input of training sample z 2 S, then the
following holds for the eigenvectors {ui}P
i=NK+1
g>
z ui = r>
ŷ `(z, ✓⇤
) r>
✓ f(x, ✓⇤
)ui
| {z }
0
= 0
<latexit sha1_base64="PpgwH7Ake9PCmrRDLUlAeqSlFn8=">AAACEHicbVDJSgNBEO1xjXGLevTSGMR4CTPiBiJEvSh4iGAWyIRQ0+kkTXoWumuEMOQTvPgrXjwo4tWjN//GznLQxAdNP96roqqeF0mh0ba/rZnZufmFxdRSenlldW09s7FZ1mGsGC+xUIaq6oHmUgS8hAIlr0aKg+9JXvG6VwO/8sCVFmFwj72I131oB6IlGKCRGpk91wfsMJDJTb8x5IjJ7UWfumc5F2TUgXM7bzv7jUzW/EPQaeKMSZaMUWxkvtxmyGKfB8gkaF1z7AjrCSgUTPJ+2o01j4B1oc1rhgbgc11Phgf16a5RmrQVKvMCpEP1d0cCvtY93zOVg5X1pDcQ//NqMbZO64kIohh5wEaDWrGkGNJBOrQpFGcoe4YAU8LsSlkHFDA0GaZNCM7kydOkfJB3jvNHd4fZwuU4jhTZJjskRxxyQgrkmhRJiTDySJ7JK3mznqwX6936GJXOWOOeLfIH1ucPChab6g==</latexit>
ILA (↵ = 0.01)
<latexit sha1_base64="JY+U0ZmSLnSoGiMXkCJ3WQ+ilXc=">AAACD3icbZDJSgNBEIZ7XGPcoh69NAYlXsKMuIEIUS8KHiKYBTIh1HQ6SZOehe4aIQx5Ay++ihcPinj16s23sbMcNPGHho+/quiq34uk0Gjb39bM7Nz8wmJqKb28srq2ntnYLOswVoyXWChDVfVAcykCXkKBklcjxcH3JK943atBvfLAlRZhcI+9iNd9aAeiJRigsRqZPdcH7DCQyU2/MWTE5PaiT92znAsy6sC5Y9v7jUzWzttD0WlwxpAlYxUbmS+3GbLY5wEyCVrXHDvCegIKBZO8n3ZjzSNgXWjzmsEAfK7ryfCePt01TpO2QmVegHTo/p5IwNe653umc7CxnqwNzP9qtRhbp/VEBFGMPGCjj1qxpBjSQTi0KRRnKHsGgClhdqWsAwoYmgjTJgRn8uRpKB/kneP80d1htnA5jiNFtskOyRGHnJACuSZFUiKMPJJn8krerCfrxXq3PkatM9Z4Zov8kfX5A5GDm7I=</latexit>
ILA (↵ = 100)
<latexit sha1_base64="s0zEbOYkkBwPLESPadaN5uCYuU0=">AAACCXicbZC7SgNBFIZn4y3G26qlzWAQYhN2xaggQtRGwSKKuUASwuxkkgyZvTBzVgjLtja+io2FIra+gZ1v42SzhSb+MPDxn3OYc34nEFyBZX0bmbn5hcWl7HJuZXVtfcPc3KopP5SUVakvfNlwiGKCe6wKHARrBJIR1xGs7gwvx/X6A5OK+949jALWdknf4z1OCWirY+KWS2BAiYiu407CANHNeYxbp4W7s5K13zHzVtFKhGfBTiGPUlU65ler69PQZR5QQZRq2lYA7YhI4FSwONcKFQsIHZI+a2r0iMtUO0ouifGedrq450v9PMCJ+3siIq5SI9fRneNd1XRtbP5Xa4bQO2lH3AtCYB6dfNQLBQYfj2PBXS4ZBTHSQKjkeldMB0QSCjq8nA7Bnj55FmoHRfuoWLo9zJcv0jiyaAftogKy0TEqoytUQVVE0SN6Rq/ozXgyXox342PSmjHSmW30R8bnDwB/mTo=</latexit>
ILA (R = 50)
<latexit sha1_base64="IF4DsSdOVW+Hs34gt59F9n075OY=">AAAB/XicbVDLSgMxFL3js9bX+Ni5CRahbsqMWBVEKLrRXRX7gM5QMmnahmYeJBmhDsVfceNCEbf+hzv/xkw7C209EDiccy/35HgRZ1JZ1rcxN7+wuLScW8mvrq1vbJpb23UZxoLQGgl5KJoelpSzgNYUU5w2I0Gx73Ha8AZXqd94oEKyMLhXw4i6Pu4FrMsIVlpqm7uOj1WfYJ7cjJBzXry7KFuHbbNglawx0CyxM1KADNW2+eV0QhL7NFCEYylbthUpN8FCMcLpKO/EkkaYDHCPtjQNsE+lm4zTj9CBVjqoGwr9AoXG6u+NBPtSDn1PT6ZZ5bSXiv95rVh1z9yEBVGsaEAmh7oxRypEaRWowwQlig81wUQwnRWRPhaYKF1YXpdgT395ltSPSvZJqXx7XKhcZnXkYA/2oQg2nEIFrqEKNSDwCM/wCm/Gk/FivBsfk9E5I9vZgT8wPn8Ad7eT8A==</latexit>
I (R = 50)
Utilization of Geometric Ensemble
<latexit sha1_base64="pnaUFL7AQVg8XQ793gRTJ7w7D+E=">AAAClXicdVFdT9RAFJ1WFFw/WPDBB18mbgxgzKYliryQIPKhT2Liwibb2tzO3u5OmH5k5tZkafqP/DW88W+YdtcEQU8yyck5596ZuTculDTkedeO+2Dp4aPllcedJ0+fPV/trq2fmbzUAgciV7kexmBQyQwHJEnhsNAIaazwPL743Pjnv1AbmWc/aFZgmMIkk4kUQFaKur+DFGgqQFVf66jlRNXJ0bDevHx3ubHV4RZ7vDXiuDqqoyoojOSBkSkvbhXUdRsNFCY0mtNDVAQ8QKVsEU2R4Ofbpi1vOmzxQIxz+m9q409s3kvLyZTCqNvz+l4Lfp/4C9JjC5xG3atgnIsyxYyEAmNGvldQWIEmKRTWnaA0WIC4gAmOLM0gRRNW7VRr/sYqY57k2p6MeKverqggNWaWxjbZjMHc9RrxX96opGQ3rGRWlISZmF+UlIpTzpsV8bHUKEjNLAGhpX0rF1PQIMgusmOH4N/98n1ytt33d/ofvr/v7R8sxrHCXrHXbJP57CPbZ1/YKRsw4aw7u84n58B96e65h+7xPOo6i5oX7C+4324A8G/EQw==</latexit>
IGEX(z, z0
) = E ⇠pGE
[ `✓⇤ (z, ) · `✓⇤ (z0
, )]
<latexit sha1_base64="M+LVLxJYyTlnS/UIE0jcBjaofaM=">AAACknicdVHbatwwEJXdpkm3l2wvb30RXUqSUha7JL0EciMNpJCHFLpJYO2asXa8KyJfkMaBjfEH5Xfy1r+p7N1CLu0BweHMmZF0Ji6UNOR5vx33wcOFR4tLjztPnj57vtx98fLE5KUWOBC5yvVZDAaVzHBAkhSeFRohjRWexuf7Tf30ArWRefaTpgWGKYwzmUgBZKWoexWkQBMBqvpeRy0nqo726tXLD5cra3xzi7diHFcHdVQFhZE8MDLlxU1z3eEWgcKEhjP6DRUBD1Ap20QTJPj1vpnJmwlrPBCjnP7rWvlrm83ScjyhsB91e17fa8HvE39OemyO46h7HYxyUaaYkVBgzND3Cgor0CSFwroTlAYLEOcwxqGlGaRowqqNtObvrDLiSa7tyYi36s2OClJjpmlsnU0O5m6tEf9VG5aUfAkrmRUlYSZmFyWl4pTzZj98JDUKUlNLQGhp38rFBDQIslvs2BD8u1++T04+9v1P/Y0f673dr/M4ltgb9patMp99ZrvskB2zARPOsrPhbDs77mt3091z92dW15n3vGK34B79Abh5w8Q=</latexit>
ILA(z, z0
) := E ⇠pLA
[ `✓⇤ (z, ) · `✓⇤ (z0
, )] .
Utilization of Geometric Ensemble
Empirical evaluations
Empirical evaluations
4. Discussion points
Conclusion and Future directions
• AsshowninTheorem4.1,IF isclosely relatedto the Laplace Approximation.
• Onthe otherhand,[23] formalizedthe relationshipbetweensharpnessandLaplace ApproximationthroughPAC-Bayes.
• Althoughthere iscriticismthat sharpnessisnot directly relatedto generalization[24],anaccurate understandingof the evolution
(dynamics) of sharpnesswillhelpclarify the relationshipbetweensharpnessandgeneralization.
• Recent worksonEdge-of-Stability (EOS)[25,26,27] wouldbe agoodstartingpoint.
Thank you !
Any Questions ?

More Related Content

Similar to S. Kim, NeurIPS 2023, MLILAB, KAISTAI

Ppt for paper id 696 a review of hybrid data mining algorithm for big data mi...
Ppt for paper id 696 a review of hybrid data mining algorithm for big data mi...Ppt for paper id 696 a review of hybrid data mining algorithm for big data mi...
Ppt for paper id 696 a review of hybrid data mining algorithm for big data mi...Prasanta Paul
 
Motion Prediction Using Depth Information of Human Arm Based on Alexnet
Motion Prediction Using Depth Information of Human Arm Based on AlexnetMotion Prediction Using Depth Information of Human Arm Based on Alexnet
Motion Prediction Using Depth Information of Human Arm Based on Alexnetgerogepatton
 
MOTION PREDICTION USING DEPTH INFORMATION OF HUMAN ARM BASED ON ALEXNET
MOTION PREDICTION USING DEPTH INFORMATION OF HUMAN ARM BASED ON ALEXNETMOTION PREDICTION USING DEPTH INFORMATION OF HUMAN ARM BASED ON ALEXNET
MOTION PREDICTION USING DEPTH INFORMATION OF HUMAN ARM BASED ON ALEXNETgerogepatton
 
Neural Network Applications In Machining: A Review
Neural Network Applications In Machining: A ReviewNeural Network Applications In Machining: A Review
Neural Network Applications In Machining: A ReviewAshish Khetan
 
An Efficient Activity Detection System based on Skeleton Joints Identification
An Efficient Activity Detection System based on Skeleton Joints Identification An Efficient Activity Detection System based on Skeleton Joints Identification
An Efficient Activity Detection System based on Skeleton Joints Identification IJECEIAES
 
Novel framework for optimized digital forensic for mitigating complex image ...
Novel framework for optimized digital forensic  for mitigating complex image ...Novel framework for optimized digital forensic  for mitigating complex image ...
Novel framework for optimized digital forensic for mitigating complex image ...IJECEIAES
 
Development of algorithm for identification of maligant growth in cancer usin...
Development of algorithm for identification of maligant growth in cancer usin...Development of algorithm for identification of maligant growth in cancer usin...
Development of algorithm for identification of maligant growth in cancer usin...IJECEIAES
 
Trends in covolutional neural network in 2020 - International Journal of Arti...
Trends in covolutional neural network in 2020 - International Journal of Arti...Trends in covolutional neural network in 2020 - International Journal of Arti...
Trends in covolutional neural network in 2020 - International Journal of Arti...gerogepatton
 
Fast Feature Pyramids for Object Detection
Fast Feature Pyramids for Object DetectionFast Feature Pyramids for Object Detection
Fast Feature Pyramids for Object Detectionsuthi
 
10 9242 it geo-spatial information for managing ambiguity(edit ty)
10 9242 it  geo-spatial information for managing ambiguity(edit ty)10 9242 it  geo-spatial information for managing ambiguity(edit ty)
10 9242 it geo-spatial information for managing ambiguity(edit ty)IAESIJEECS
 
New Research Articles 2020 June Issue International Journal on Cryptography a...
New Research Articles 2020 June Issue International Journal on Cryptography a...New Research Articles 2020 June Issue International Journal on Cryptography a...
New Research Articles 2020 June Issue International Journal on Cryptography a...ijcisjournal
 
Human Activity Recognition Using Recurrent Neural Network
Human Activity Recognition Using Recurrent Neural NetworkHuman Activity Recognition Using Recurrent Neural Network
Human Activity Recognition Using Recurrent Neural Networkmlaij
 
Can Generative Adversarial Networks Model Imaging Physics?
Can Generative Adversarial Networks Model Imaging Physics?Can Generative Adversarial Networks Model Imaging Physics?
Can Generative Adversarial Networks Model Imaging Physics?Debdoot Sheet
 
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...ijscai
 
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...ijscai
 
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...ijscai
 
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...ijscai
 
A ROBUST JOINT-TRAINING GRAPHNEURALNETWORKS MODEL FOR EVENT DETECTIONWITHSYMM...
A ROBUST JOINT-TRAINING GRAPHNEURALNETWORKS MODEL FOR EVENT DETECTIONWITHSYMM...A ROBUST JOINT-TRAINING GRAPHNEURALNETWORKS MODEL FOR EVENT DETECTIONWITHSYMM...
A ROBUST JOINT-TRAINING GRAPHNEURALNETWORKS MODEL FOR EVENT DETECTIONWITHSYMM...kevig
 
A Robust Joint-Training Graph Neural Networks Model for Event Detection with ...
A Robust Joint-Training Graph Neural Networks Model for Event Detection with ...A Robust Joint-Training Graph Neural Networks Model for Event Detection with ...
A Robust Joint-Training Graph Neural Networks Model for Event Detection with ...kevig
 

Similar to S. Kim, NeurIPS 2023, MLILAB, KAISTAI (20)

Ppt for paper id 696 a review of hybrid data mining algorithm for big data mi...
Ppt for paper id 696 a review of hybrid data mining algorithm for big data mi...Ppt for paper id 696 a review of hybrid data mining algorithm for big data mi...
Ppt for paper id 696 a review of hybrid data mining algorithm for big data mi...
 
Motion Prediction Using Depth Information of Human Arm Based on Alexnet
Motion Prediction Using Depth Information of Human Arm Based on AlexnetMotion Prediction Using Depth Information of Human Arm Based on Alexnet
Motion Prediction Using Depth Information of Human Arm Based on Alexnet
 
MOTION PREDICTION USING DEPTH INFORMATION OF HUMAN ARM BASED ON ALEXNET
MOTION PREDICTION USING DEPTH INFORMATION OF HUMAN ARM BASED ON ALEXNETMOTION PREDICTION USING DEPTH INFORMATION OF HUMAN ARM BASED ON ALEXNET
MOTION PREDICTION USING DEPTH INFORMATION OF HUMAN ARM BASED ON ALEXNET
 
Neural Network Applications In Machining: A Review
Neural Network Applications In Machining: A ReviewNeural Network Applications In Machining: A Review
Neural Network Applications In Machining: A Review
 
An Efficient Activity Detection System based on Skeleton Joints Identification
An Efficient Activity Detection System based on Skeleton Joints Identification An Efficient Activity Detection System based on Skeleton Joints Identification
An Efficient Activity Detection System based on Skeleton Joints Identification
 
Novel framework for optimized digital forensic for mitigating complex image ...
Novel framework for optimized digital forensic  for mitigating complex image ...Novel framework for optimized digital forensic  for mitigating complex image ...
Novel framework for optimized digital forensic for mitigating complex image ...
 
Development of algorithm for identification of maligant growth in cancer usin...
Development of algorithm for identification of maligant growth in cancer usin...Development of algorithm for identification of maligant growth in cancer usin...
Development of algorithm for identification of maligant growth in cancer usin...
 
Trends in covolutional neural network in 2020 - International Journal of Arti...
Trends in covolutional neural network in 2020 - International Journal of Arti...Trends in covolutional neural network in 2020 - International Journal of Arti...
Trends in covolutional neural network in 2020 - International Journal of Arti...
 
Fast Feature Pyramids for Object Detection
Fast Feature Pyramids for Object DetectionFast Feature Pyramids for Object Detection
Fast Feature Pyramids for Object Detection
 
10 9242 it geo-spatial information for managing ambiguity(edit ty)
10 9242 it  geo-spatial information for managing ambiguity(edit ty)10 9242 it  geo-spatial information for managing ambiguity(edit ty)
10 9242 it geo-spatial information for managing ambiguity(edit ty)
 
New Research Articles 2020 June Issue International Journal on Cryptography a...
New Research Articles 2020 June Issue International Journal on Cryptography a...New Research Articles 2020 June Issue International Journal on Cryptography a...
New Research Articles 2020 June Issue International Journal on Cryptography a...
 
Palmprint Identification Using FRIT
Palmprint Identification Using FRITPalmprint Identification Using FRIT
Palmprint Identification Using FRIT
 
Human Activity Recognition Using Recurrent Neural Network
Human Activity Recognition Using Recurrent Neural NetworkHuman Activity Recognition Using Recurrent Neural Network
Human Activity Recognition Using Recurrent Neural Network
 
Can Generative Adversarial Networks Model Imaging Physics?
Can Generative Adversarial Networks Model Imaging Physics?Can Generative Adversarial Networks Model Imaging Physics?
Can Generative Adversarial Networks Model Imaging Physics?
 
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
 
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
 
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
 
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
FUTURISTIC TECHNOLOGY IN ARCHITECTURE & PLANNING - AUGMENTED AND VIRTUAL REAL...
 
A ROBUST JOINT-TRAINING GRAPHNEURALNETWORKS MODEL FOR EVENT DETECTIONWITHSYMM...
A ROBUST JOINT-TRAINING GRAPHNEURALNETWORKS MODEL FOR EVENT DETECTIONWITHSYMM...A ROBUST JOINT-TRAINING GRAPHNEURALNETWORKS MODEL FOR EVENT DETECTIONWITHSYMM...
A ROBUST JOINT-TRAINING GRAPHNEURALNETWORKS MODEL FOR EVENT DETECTIONWITHSYMM...
 
A Robust Joint-Training Graph Neural Networks Model for Event Detection with ...
A Robust Joint-Training Graph Neural Networks Model for Event Detection with ...A Robust Joint-Training Graph Neural Networks Model for Event Detection with ...
A Robust Joint-Training Graph Neural Networks Model for Event Detection with ...
 

More from MLILAB

J. Jeong, AAAI 2024, MLILAB, KAIST AI..
J. Jeong,  AAAI 2024, MLILAB, KAIST AI..J. Jeong,  AAAI 2024, MLILAB, KAIST AI..
J. Jeong, AAAI 2024, MLILAB, KAIST AI..MLILAB
 
J. Yun, NeurIPS 2023, MLILAB, KAISTAI
J. Yun,  NeurIPS 2023,  MLILAB,  KAISTAIJ. Yun,  NeurIPS 2023,  MLILAB,  KAISTAI
J. Yun, NeurIPS 2023, MLILAB, KAISTAIMLILAB
 
C. Kim, INTERSPEECH 2023, MLILAB, KAISTAI
C. Kim, INTERSPEECH 2023, MLILAB, KAISTAIC. Kim, INTERSPEECH 2023, MLILAB, KAISTAI
C. Kim, INTERSPEECH 2023, MLILAB, KAISTAIMLILAB
 
Y. Jung, ICML 2023, MLILAB, KAISTAI
Y. Jung, ICML 2023, MLILAB, KAISTAIY. Jung, ICML 2023, MLILAB, KAISTAI
Y. Jung, ICML 2023, MLILAB, KAISTAIMLILAB
 
J. Song, S. Kim, ICML 2023, MLILAB, KAISTAI
J. Song, S. Kim, ICML 2023, MLILAB, KAISTAIJ. Song, S. Kim, ICML 2023, MLILAB, KAISTAI
J. Song, S. Kim, ICML 2023, MLILAB, KAISTAIMLILAB
 
K. Seo, ICASSP 2023, MLILAB, KAISTAI
K. Seo, ICASSP 2023, MLILAB, KAISTAIK. Seo, ICASSP 2023, MLILAB, KAISTAI
K. Seo, ICASSP 2023, MLILAB, KAISTAIMLILAB
 
G. Kim, CVPR 2023, MLILAB, KAISTAI
G. Kim, CVPR 2023, MLILAB, KAISTAIG. Kim, CVPR 2023, MLILAB, KAISTAI
G. Kim, CVPR 2023, MLILAB, KAISTAIMLILAB
 
S. Kim, ICLR 2023, MLILAB, KAISTAI
S. Kim, ICLR 2023, MLILAB, KAISTAIS. Kim, ICLR 2023, MLILAB, KAISTAI
S. Kim, ICLR 2023, MLILAB, KAISTAIMLILAB
 
Y. Kim, ICLR 2023, MLILAB, KAISTAI
Y. Kim, ICLR 2023, MLILAB, KAISTAIY. Kim, ICLR 2023, MLILAB, KAISTAI
Y. Kim, ICLR 2023, MLILAB, KAISTAIMLILAB
 
J. Yun, AISTATS 2022, MLILAB, KAISTAI
J. Yun, AISTATS 2022, MLILAB, KAISTAIJ. Yun, AISTATS 2022, MLILAB, KAISTAI
J. Yun, AISTATS 2022, MLILAB, KAISTAIMLILAB
 
J. Song, J. Park, ICML 2022, MLILAB, KAISTAI
J. Song, J. Park, ICML 2022, MLILAB, KAISTAIJ. Song, J. Park, ICML 2022, MLILAB, KAISTAI
J. Song, J. Park, ICML 2022, MLILAB, KAISTAIMLILAB
 
J. Park, J. Song, ICLR 2022, MLILAB, KAISTAI
J. Park, J. Song, ICLR 2022, MLILAB, KAISTAIJ. Park, J. Song, ICLR 2022, MLILAB, KAISTAI
J. Park, J. Song, ICLR 2022, MLILAB, KAISTAIMLILAB
 
J. Park, H. Shim, AAAI 2022, MLILAB, KAISTAI
J. Park, H. Shim, AAAI 2022, MLILAB, KAISTAIJ. Park, H. Shim, AAAI 2022, MLILAB, KAISTAI
J. Park, H. Shim, AAAI 2022, MLILAB, KAISTAIMLILAB
 
J. Park, AAAI 2022, MLILAB, KAIST AI
J. Park, AAAI 2022, MLILAB, KAIST AIJ. Park, AAAI 2022, MLILAB, KAIST AI
J. Park, AAAI 2022, MLILAB, KAIST AIMLILAB
 
J. Song, et. al., ASRU 2021, MLILAB, KAIST AI
J. Song, et. al., ASRU 2021, MLILAB, KAIST AIJ. Song, et. al., ASRU 2021, MLILAB, KAIST AI
J. Song, et. al., ASRU 2021, MLILAB, KAIST AIMLILAB
 
J. Song, H. Shim et al., ICASSP 2021, MLILAB, KAIST AI
J. Song, H. Shim et al., ICASSP 2021, MLILAB, KAIST AIJ. Song, H. Shim et al., ICASSP 2021, MLILAB, KAIST AI
J. Song, H. Shim et al., ICASSP 2021, MLILAB, KAIST AIMLILAB
 
T. Yoon, et. al., ICLR 2021, MLILAB, KAIST AI
T. Yoon, et. al., ICLR 2021, MLILAB, KAIST AIT. Yoon, et. al., ICLR 2021, MLILAB, KAIST AI
T. Yoon, et. al., ICLR 2021, MLILAB, KAIST AIMLILAB
 
G. Park, J.-Y. Yang, et. al., NeurIPS 2020, MLILAB, KAIST AI
G. Park, J.-Y. Yang, et. al., NeurIPS 2020, MLILAB, KAIST AIG. Park, J.-Y. Yang, et. al., NeurIPS 2020, MLILAB, KAIST AI
G. Park, J.-Y. Yang, et. al., NeurIPS 2020, MLILAB, KAIST AIMLILAB
 
I. Chung, AAAI 2020, MLILAB, KAIST AI
I. Chung, AAAI 2020, MLILAB, KAIST AII. Chung, AAAI 2020, MLILAB, KAIST AI
I. Chung, AAAI 2020, MLILAB, KAIST AIMLILAB
 
H. Shim, NeurIPS 2018, MLILAB, KAIST AI
H. Shim, NeurIPS 2018, MLILAB, KAIST AIH. Shim, NeurIPS 2018, MLILAB, KAIST AI
H. Shim, NeurIPS 2018, MLILAB, KAIST AIMLILAB
 

More from MLILAB (20)

J. Jeong, AAAI 2024, MLILAB, KAIST AI..
J. Jeong,  AAAI 2024, MLILAB, KAIST AI..J. Jeong,  AAAI 2024, MLILAB, KAIST AI..
J. Jeong, AAAI 2024, MLILAB, KAIST AI..
 
J. Yun, NeurIPS 2023, MLILAB, KAISTAI
J. Yun,  NeurIPS 2023,  MLILAB,  KAISTAIJ. Yun,  NeurIPS 2023,  MLILAB,  KAISTAI
J. Yun, NeurIPS 2023, MLILAB, KAISTAI
 
C. Kim, INTERSPEECH 2023, MLILAB, KAISTAI
C. Kim, INTERSPEECH 2023, MLILAB, KAISTAIC. Kim, INTERSPEECH 2023, MLILAB, KAISTAI
C. Kim, INTERSPEECH 2023, MLILAB, KAISTAI
 
Y. Jung, ICML 2023, MLILAB, KAISTAI
Y. Jung, ICML 2023, MLILAB, KAISTAIY. Jung, ICML 2023, MLILAB, KAISTAI
Y. Jung, ICML 2023, MLILAB, KAISTAI
 
J. Song, S. Kim, ICML 2023, MLILAB, KAISTAI
J. Song, S. Kim, ICML 2023, MLILAB, KAISTAIJ. Song, S. Kim, ICML 2023, MLILAB, KAISTAI
J. Song, S. Kim, ICML 2023, MLILAB, KAISTAI
 
K. Seo, ICASSP 2023, MLILAB, KAISTAI
K. Seo, ICASSP 2023, MLILAB, KAISTAIK. Seo, ICASSP 2023, MLILAB, KAISTAI
K. Seo, ICASSP 2023, MLILAB, KAISTAI
 
G. Kim, CVPR 2023, MLILAB, KAISTAI
G. Kim, CVPR 2023, MLILAB, KAISTAIG. Kim, CVPR 2023, MLILAB, KAISTAI
G. Kim, CVPR 2023, MLILAB, KAISTAI
 
S. Kim, ICLR 2023, MLILAB, KAISTAI
S. Kim, ICLR 2023, MLILAB, KAISTAIS. Kim, ICLR 2023, MLILAB, KAISTAI
S. Kim, ICLR 2023, MLILAB, KAISTAI
 
Y. Kim, ICLR 2023, MLILAB, KAISTAI
Y. Kim, ICLR 2023, MLILAB, KAISTAIY. Kim, ICLR 2023, MLILAB, KAISTAI
Y. Kim, ICLR 2023, MLILAB, KAISTAI
 
J. Yun, AISTATS 2022, MLILAB, KAISTAI
J. Yun, AISTATS 2022, MLILAB, KAISTAIJ. Yun, AISTATS 2022, MLILAB, KAISTAI
J. Yun, AISTATS 2022, MLILAB, KAISTAI
 
J. Song, J. Park, ICML 2022, MLILAB, KAISTAI
J. Song, J. Park, ICML 2022, MLILAB, KAISTAIJ. Song, J. Park, ICML 2022, MLILAB, KAISTAI
J. Song, J. Park, ICML 2022, MLILAB, KAISTAI
 
J. Park, J. Song, ICLR 2022, MLILAB, KAISTAI
J. Park, J. Song, ICLR 2022, MLILAB, KAISTAIJ. Park, J. Song, ICLR 2022, MLILAB, KAISTAI
J. Park, J. Song, ICLR 2022, MLILAB, KAISTAI
 
J. Park, H. Shim, AAAI 2022, MLILAB, KAISTAI
J. Park, H. Shim, AAAI 2022, MLILAB, KAISTAIJ. Park, H. Shim, AAAI 2022, MLILAB, KAISTAI
J. Park, H. Shim, AAAI 2022, MLILAB, KAISTAI
 
J. Park, AAAI 2022, MLILAB, KAIST AI
J. Park, AAAI 2022, MLILAB, KAIST AIJ. Park, AAAI 2022, MLILAB, KAIST AI
J. Park, AAAI 2022, MLILAB, KAIST AI
 
J. Song, et. al., ASRU 2021, MLILAB, KAIST AI
J. Song, et. al., ASRU 2021, MLILAB, KAIST AIJ. Song, et. al., ASRU 2021, MLILAB, KAIST AI
J. Song, et. al., ASRU 2021, MLILAB, KAIST AI
 
J. Song, H. Shim et al., ICASSP 2021, MLILAB, KAIST AI
J. Song, H. Shim et al., ICASSP 2021, MLILAB, KAIST AIJ. Song, H. Shim et al., ICASSP 2021, MLILAB, KAIST AI
J. Song, H. Shim et al., ICASSP 2021, MLILAB, KAIST AI
 
T. Yoon, et. al., ICLR 2021, MLILAB, KAIST AI
T. Yoon, et. al., ICLR 2021, MLILAB, KAIST AIT. Yoon, et. al., ICLR 2021, MLILAB, KAIST AI
T. Yoon, et. al., ICLR 2021, MLILAB, KAIST AI
 
G. Park, J.-Y. Yang, et. al., NeurIPS 2020, MLILAB, KAIST AI
G. Park, J.-Y. Yang, et. al., NeurIPS 2020, MLILAB, KAIST AIG. Park, J.-Y. Yang, et. al., NeurIPS 2020, MLILAB, KAIST AI
G. Park, J.-Y. Yang, et. al., NeurIPS 2020, MLILAB, KAIST AI
 
I. Chung, AAAI 2020, MLILAB, KAIST AI
I. Chung, AAAI 2020, MLILAB, KAIST AII. Chung, AAAI 2020, MLILAB, KAIST AI
I. Chung, AAAI 2020, MLILAB, KAIST AI
 
H. Shim, NeurIPS 2018, MLILAB, KAIST AI
H. Shim, NeurIPS 2018, MLILAB, KAIST AIH. Shim, NeurIPS 2018, MLILAB, KAIST AI
H. Shim, NeurIPS 2018, MLILAB, KAIST AI
 

Recently uploaded

CCS355 Neural Networks & Deep Learning Unit 1 PDF notes with Question bank .pdf
CCS355 Neural Networks & Deep Learning Unit 1 PDF notes with Question bank .pdfCCS355 Neural Networks & Deep Learning Unit 1 PDF notes with Question bank .pdf
CCS355 Neural Networks & Deep Learning Unit 1 PDF notes with Question bank .pdfAsst.prof M.Gokilavani
 
UNIT III ANALOG ELECTRONICS (BASIC ELECTRONICS)
UNIT III ANALOG ELECTRONICS (BASIC ELECTRONICS)UNIT III ANALOG ELECTRONICS (BASIC ELECTRONICS)
UNIT III ANALOG ELECTRONICS (BASIC ELECTRONICS)Dr SOUNDIRARAJ N
 
Solving The Right Triangles PowerPoint 2.ppt
Solving The Right Triangles PowerPoint 2.pptSolving The Right Triangles PowerPoint 2.ppt
Solving The Right Triangles PowerPoint 2.pptJasonTagapanGulla
 
computer application and construction management
computer application and construction managementcomputer application and construction management
computer application and construction managementMariconPadriquez1
 
welding defects observed during the welding
welding defects observed during the weldingwelding defects observed during the welding
welding defects observed during the weldingMuhammadUzairLiaqat
 
Work Experience-Dalton Park.pptxfvvvvvvv
Work Experience-Dalton Park.pptxfvvvvvvvWork Experience-Dalton Park.pptxfvvvvvvv
Work Experience-Dalton Park.pptxfvvvvvvvLewisJB
 
Why does (not) Kafka need fsync: Eliminating tail latency spikes caused by fsync
Why does (not) Kafka need fsync: Eliminating tail latency spikes caused by fsyncWhy does (not) Kafka need fsync: Eliminating tail latency spikes caused by fsync
Why does (not) Kafka need fsync: Eliminating tail latency spikes caused by fsyncssuser2ae721
 
Gurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort service
Gurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort serviceGurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort service
Gurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort servicejennyeacort
 
Past, Present and Future of Generative AI
Past, Present and Future of Generative AIPast, Present and Future of Generative AI
Past, Present and Future of Generative AIabhishek36461
 
Class 1 | NFPA 72 | Overview Fire Alarm System
Class 1 | NFPA 72 | Overview Fire Alarm SystemClass 1 | NFPA 72 | Overview Fire Alarm System
Class 1 | NFPA 72 | Overview Fire Alarm Systemirfanmechengr
 
US Department of Education FAFSA Week of Action
US Department of Education FAFSA Week of ActionUS Department of Education FAFSA Week of Action
US Department of Education FAFSA Week of ActionMebane Rash
 
Vishratwadi & Ghorpadi Bridge Tender documents
Vishratwadi & Ghorpadi Bridge Tender documentsVishratwadi & Ghorpadi Bridge Tender documents
Vishratwadi & Ghorpadi Bridge Tender documentsSachinPawar510423
 
Piping Basic stress analysis by engineering
Piping Basic stress analysis by engineeringPiping Basic stress analysis by engineering
Piping Basic stress analysis by engineeringJuanCarlosMorales19600
 
Earthing details of Electrical Substation
Earthing details of Electrical SubstationEarthing details of Electrical Substation
Earthing details of Electrical Substationstephanwindworld
 
Concrete Mix Design - IS 10262-2019 - .pptx
Concrete Mix Design - IS 10262-2019 - .pptxConcrete Mix Design - IS 10262-2019 - .pptx
Concrete Mix Design - IS 10262-2019 - .pptxKartikeyaDwivedi3
 
Call Us ≽ 8377877756 ≼ Call Girls In Shastri Nagar (Delhi)
Call Us ≽ 8377877756 ≼ Call Girls In Shastri Nagar (Delhi)Call Us ≽ 8377877756 ≼ Call Girls In Shastri Nagar (Delhi)
Call Us ≽ 8377877756 ≼ Call Girls In Shastri Nagar (Delhi)dollysharma2066
 
An experimental study in using natural admixture as an alternative for chemic...
An experimental study in using natural admixture as an alternative for chemic...An experimental study in using natural admixture as an alternative for chemic...
An experimental study in using natural admixture as an alternative for chemic...Chandu841456
 

Recently uploaded (20)

CCS355 Neural Networks & Deep Learning Unit 1 PDF notes with Question bank .pdf
CCS355 Neural Networks & Deep Learning Unit 1 PDF notes with Question bank .pdfCCS355 Neural Networks & Deep Learning Unit 1 PDF notes with Question bank .pdf
CCS355 Neural Networks & Deep Learning Unit 1 PDF notes with Question bank .pdf
 
Design and analysis of solar grass cutter.pdf
Design and analysis of solar grass cutter.pdfDesign and analysis of solar grass cutter.pdf
Design and analysis of solar grass cutter.pdf
 
UNIT III ANALOG ELECTRONICS (BASIC ELECTRONICS)
UNIT III ANALOG ELECTRONICS (BASIC ELECTRONICS)UNIT III ANALOG ELECTRONICS (BASIC ELECTRONICS)
UNIT III ANALOG ELECTRONICS (BASIC ELECTRONICS)
 
Solving The Right Triangles PowerPoint 2.ppt
Solving The Right Triangles PowerPoint 2.pptSolving The Right Triangles PowerPoint 2.ppt
Solving The Right Triangles PowerPoint 2.ppt
 
computer application and construction management
computer application and construction managementcomputer application and construction management
computer application and construction management
 
welding defects observed during the welding
welding defects observed during the weldingwelding defects observed during the welding
welding defects observed during the welding
 
Work Experience-Dalton Park.pptxfvvvvvvv
Work Experience-Dalton Park.pptxfvvvvvvvWork Experience-Dalton Park.pptxfvvvvvvv
Work Experience-Dalton Park.pptxfvvvvvvv
 
Why does (not) Kafka need fsync: Eliminating tail latency spikes caused by fsync
Why does (not) Kafka need fsync: Eliminating tail latency spikes caused by fsyncWhy does (not) Kafka need fsync: Eliminating tail latency spikes caused by fsync
Why does (not) Kafka need fsync: Eliminating tail latency spikes caused by fsync
 
Gurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort service
Gurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort serviceGurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort service
Gurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort service
 
Past, Present and Future of Generative AI
Past, Present and Future of Generative AIPast, Present and Future of Generative AI
Past, Present and Future of Generative AI
 
🔝9953056974🔝!!-YOUNG call girls in Rajendra Nagar Escort rvice Shot 2000 nigh...
🔝9953056974🔝!!-YOUNG call girls in Rajendra Nagar Escort rvice Shot 2000 nigh...🔝9953056974🔝!!-YOUNG call girls in Rajendra Nagar Escort rvice Shot 2000 nigh...
🔝9953056974🔝!!-YOUNG call girls in Rajendra Nagar Escort rvice Shot 2000 nigh...
 
Class 1 | NFPA 72 | Overview Fire Alarm System
Class 1 | NFPA 72 | Overview Fire Alarm SystemClass 1 | NFPA 72 | Overview Fire Alarm System
Class 1 | NFPA 72 | Overview Fire Alarm System
 
US Department of Education FAFSA Week of Action
US Department of Education FAFSA Week of ActionUS Department of Education FAFSA Week of Action
US Department of Education FAFSA Week of Action
 
Vishratwadi & Ghorpadi Bridge Tender documents
Vishratwadi & Ghorpadi Bridge Tender documentsVishratwadi & Ghorpadi Bridge Tender documents
Vishratwadi & Ghorpadi Bridge Tender documents
 
Piping Basic stress analysis by engineering
Piping Basic stress analysis by engineeringPiping Basic stress analysis by engineering
Piping Basic stress analysis by engineering
 
Earthing details of Electrical Substation
Earthing details of Electrical SubstationEarthing details of Electrical Substation
Earthing details of Electrical Substation
 
Concrete Mix Design - IS 10262-2019 - .pptx
Concrete Mix Design - IS 10262-2019 - .pptxConcrete Mix Design - IS 10262-2019 - .pptx
Concrete Mix Design - IS 10262-2019 - .pptx
 
Call Us ≽ 8377877756 ≼ Call Girls In Shastri Nagar (Delhi)
Call Us ≽ 8377877756 ≼ Call Girls In Shastri Nagar (Delhi)Call Us ≽ 8377877756 ≼ Call Girls In Shastri Nagar (Delhi)
Call Us ≽ 8377877756 ≼ Call Girls In Shastri Nagar (Delhi)
 
young call girls in Green Park🔝 9953056974 🔝 escort Service
young call girls in Green Park🔝 9953056974 🔝 escort Serviceyoung call girls in Green Park🔝 9953056974 🔝 escort Service
young call girls in Green Park🔝 9953056974 🔝 escort Service
 
An experimental study in using natural admixture as an alternative for chemic...
An experimental study in using natural admixture as an alternative for chemic...An experimental study in using natural admixture as an alternative for chemic...
An experimental study in using natural admixture as an alternative for chemic...
 

S. Kim, NeurIPS 2023, MLILAB, KAISTAI

  • 1. GEX: A flexible method for approximating influence via Geometric Ensemble (NeurIPS 2023) Sung-Yub Kim (presenter), Kyungsu Kim, Eunho Yang Oct. 6th, 2023
  • 2. TL;DR • We empirically verify that the proposed method improves the downstream task performance and relieves the framework dependency of IF (and its approximations).
  • 4. Related works [1]Cook,R.Dennis."Detectionofinfluentialobservationinlinearregression."Technometrics19.1(1977):15-18. [2] Koh, Pang Wei, and Percy Liang. "Understanding black-box predictions via influence functions." International conference on machine learning. PMLR, 2017. [3]Pruthi,Garima,etal."Estimatingtrainingdatainfluencebytracinggradientdescent."AdvancesinNeuralInformationProcessingSystems33(2020):19920-19930. [4]Schioppa,Andrea,etal."Scalingupinfluencefunctions."ProceedingsoftheAAAIConferenceonArtificialIntelligence.Vol.36.No.8.2022. [5]Sorscher,Ben,etal."Beyondneuralscalinglaws:beatingpowerlawscalingviadatapruning."AdvancesinNeuralInformationProcessingSystems35(2022):19523-19536. [6]Paul,Mansheej,SuryaGanguli,andGintareKarolinaDziugaite."Deeplearningonadatadiet:Findingimportantexamplesearlyintraining."AdvancesinNeuralInformationProcessingSystems34(2021): 20596-20607. [7]Song,Jaeyun,Kim,SungYub,andYang,Eunho."RGE:ARepulsiveGraphRectificationforNodeClassificationviaInfluence."InternationalConferenceonMachineLearning.PMLR,2023. [8]Kim,SungYub,Kim,Kyungsu,Yang,Eunho.“GEX:AflexiblemethodforapproximatinginfluenceviaGeometricEnsemble”.AdvancesinNeuralInformationProcessingSystems(2023) [9]Song,Jaeyun,etal.“Post-TrainingRecoveryfromInjectedBiaswithSelf-Influence”,Underreview,2023 [10]Daxberger,Erik,etal."Laplaceredux-effortlessbayesiandeeplearning."AdvancesinNeuralInformationProcessingSystems34(2021):20089-20103. [11]Kong,Zhifeng,andKamalikaChaudhuri."Understandinginstance-basedinterpretabilityofvariationalauto-encoders."AdvancesinNeuralInformationProcessingSystems34(2021):2400-2412. [12]Terashita,Naoyuki,etal."InfluenceEstimationforGenerativeAdversarialNetworks."InternationalConferenceonLearningRepresentations.2020. [13]Chen,Zizhang,etal."CharacterizingtheInfluenceofGraphElements."TheEleventhInternationalConferenceonLearningRepresentations.2022. [14]Grosse,Roger,etal."StudyingLargeLanguageModelGeneralizationwithInfluenceFunctions."arXivpreprintarXiv:2308.03296(2023). [15]https://medium.com/data-science-in-your-pocket/random-projection-for-dimension-reduction-27d2ec7d40cd [16]Arnoldi,WalterEdwin."Theprincipleofminimizediterationsinthesolutionofthematrixeigenvalueproblem."Quarterlyofappliedmathematics9.1(1951):17-29. [17]Garipov,Timur,etal."Losssurfaces,modeconnectivity,andfastensemblingofdnns."Advancesinneuralinformationprocessingsystems31(2018). [18]Otsu,Nobuyuki."Athresholdselectionmethodfromgray-levelhistograms."IEEEtransactionsonsystems,man,andcybernetics9.1(1979):62-66. [19]Wei,Jiaheng,etal."LearningwithNoisyLabelsRevisited:AStudyUsingReal-WorldHumanAnnotations."InternationalConferenceonLearningRepresentations.2021. [20]Kong,Shuming,YanyanShen,andLinpengHuang."Resolvingtrainingbiasesviainfluence-baseddatarelabeling."InternationalConferenceonLearningRepresentations.2021. [21]He,Kaiming,etal."Deepresiduallearningforimagerecognition."ProceedingsoftheIEEEconferenceoncomputervisionandpatternrecognition.2016. [22]Vaswani,Ashish,etal."Attentionisallyouneed."Advancesinneuralinformationprocessingsystems30(2017). [23]Kim,SungYub,etal."Scale-invariantBayesianNeuralNetworkswithConnectivityTangentKernel."TheEleventhInternationalConferenceonLearningRepresentations.2023. [24]Andriushchenko,Maksym,etal."Amodernlookattherelationshipbetweensharpnessandgeneralization." InternationalConferenceonMachineLearning.PMLR,2023. [25]Cohen,Jeremy,etal."GradientDescentonNeuralNetworksTypicallyOccursattheEdgeofStability."InternationalConferenceonLearningRepresentations.2020. [26]Damian,Alex,EshaanNichani,andJasonD.Lee."Self-Stabilization:TheImplicitBiasofGradientDescentattheEdgeofStability."TheEleventhInternationalConferenceonLearningRepresentations. 2022. [27]Song,Minhak,andChulheeYun."TrajectoryAlignment:UnderstandingtheEdgeofStabilityPhenomenonviaBifurcationTheory."arXivpreprintarXiv:2307.04204(2023).
  • 7. Influence Function <latexit sha1_base64="AJLAeDwPQM3rbP2oKayvxEaWark=">AAACWXicbVBdaxNBFJ1dbY3xo9E++jIYhCg17IraVhEKvvggEj/SFjJpmJ3cTYbOziwzdwvpsH/SB0H8Kz44m+yDth64cDjnXu69JyuVdJgkP6P4xs2t7Vud2907d+/d3+k9eHjsTGUFjIVRxp5m3IGSGsYoUcFpaYEXmYKT7Px9459cgHXS6G+4KmFa8IWWuRQcgzTrlQyXgPzs2eySvnnHKj0PzYB+I1MmNWUFx2WW+S/12aj2zJRgORqreQGe20UhdV2zt/Tj4Ose3Yw9pc8pyy0XnoFSg8u9Vq79p3rW6yfDZA16naQt6ZMWo1nvO5sbURWgUSju3CRNSpyGzSiFgrrLKgclF+d8AZNAm7Pc1K+TqemToMxpbmwojXSt/j3heeHcqshCZ/Olu+o14v+8SYX5wdRLXVYIWmwW5ZWiaGgTM51LCwLVKhAurAy3UrHkIRIM+XZDCOnVl6+T4xfD9PXw1eeX/aPDNo4OeUQekwFJyT45Ih/IiIyJID/I72gr2o5+xVHcibub1jhqZ3bJP4h3/wCCJbTC</latexit> ✓⇤ z := argmin ✓2RP L(S, ✓) `(z, ✓) N on another instance z! ∈ ℝ" <latexit sha1_base64="rqko3S5rwHjmaf4C5ABZwHraGWY=">AAACNHicbVDLSgNBEJz1bXxFPXoZDGIiGnbFNwiCF0VBBaNCNi6zk44ZnH0w0yskSz7Kix/iRQQPinj1G5zEHNRYMFBUddNT5cdSaLTtZ6uvf2BwaHhkNDM2PjE5lZ2eudBRojiUeCQjdeUzDVKEUEKBEq5iBSzwJVz6t/tt//IOlBZReI6NGCoBuwlFTXCGRvKyR27AsM6ZTA9bXocjpscnJ618c7m5WKA7u9QFKfPNxWXqYh2QXS95zQJd6ZULXjZnF+0OaC9xuiRHujj1so9uNeJJACFyybQuO3aMlZQpFFxCK+MmGmLGb9kNlA0NWQC6knZCt+iCUaq0FinzQqQd9edGygKtG4FvJtux9F+vLf7nlROsbVVSEcYJQsi/D9USSTGi7QZpVSjgKBuGMK6E+SvldaYYR9NzxpTg/I3cSy5Wi85Gcf1sLbe33a1jhMyReZInDtkke+SAnJIS4eSePJFX8mY9WC/Wu/XxPdpndXdmyS9Yn1/ttah3</latexit> ILOO(z, z0 ) := `(z0 , ✓⇤ z ) `(z0 , ✓⇤ ) <latexit sha1_base64="NhtnBVAUrxqI/4QkdlBFLBS0k0U=">AAACO3icbVBNSxtBGJ7V+tHUatSjl6GhYKWEXfFbhIAXDx5ibFTIxjA7eZMMzs4sM+8KYdn/1Uv/hDcvvXhoEa/enU1yqNoXBh6e5/2Y54kSKSz6/r03Nf1hZnZu/mPp08LnxaXy8sqF1anh0ORaanMVMQtSKGiiQAlXiQEWRxIuo5vjQr+8BWOFVj9wmEA7Zn0leoIzdFSn3AhxAMiuNw6OwlR1XSdgNuZoKBQNY4aDKMoa+XU9z0KdgGGojWIxZMz0Y6HyPDykp+vn3+l47FunXPGr/qjoexBMQIVMqt4p34VdzdMYFHLJrG0FfoJttx4Fl5CXwtRCwvgN60PLweK2bWcj7zn96pgu7WnjnkI6Yv+dyFhs7TCOXGdhxb7VCvJ/WivF3l47EypJERQfH+qlkqKmRZC0KwxwlEMHGDfC/ZXyATOMowux5EII3lp+Dy42q8FOdftsq1Lbn8QxT9bIF7JOArJLauSE1EmTcPKT/CZ/yF/vl/fgPXpP49YpbzKzSl6V9/wCkeuu8g==</latexit> ✓⇤ := argmin ✓2RP L(S, ✓) Since retraining is computationally intractable, [2] proposed an efficient approximation, named Influence Function (IF): This can be interpreted as a two-step approximation of retraining effect <latexit sha1_base64="m+8X1kuK18A0q/05NrLtZWw8sZ0=">AAACFXicbVDLSgMxFM3UV62vUZdugkVaoZYZ8Q1CwU3dVbAP6ItMmmlDMw+SjNAO8xNu/BU3LhRxK7jzb8y0s9DWA4HDOfdyc47lMyqkYXxrqYXFpeWV9GpmbX1jc0vf3qkJL+CYVLHHPN6wkCCMuqQqqWSk4XOCHIuRujW8if36A+GCeu69HPmk7aC+S22KkVRSVy+0HCQHGLHwNsqPC+PcIby6hv1uOM5FnZb0fFjuhEdmFCtRV88aRWMCOE/MhGRBgkpX/2r1PBw4xJWYISGapuHLdoi4pJiRKNMKBPERHqI+aSrqIoeIdjhJFcEDpfSg7XH1XAkn6u+NEDlCjBxLTcYZxKwXi/95zUDaF+2Qun4giYunh+yAQenBuCLYo5xgyUaKIMyp+ivEA8QRlqrIjCrBnI08T2rHRfOseHp3ki1dJnWkwR7YB3lggnNQAmVQAVWAwSN4Bq/gTXvSXrR37WM6mtKSnV3wB9rnD7fAneo=</latexit> I(z, z0 ) := g> z0 H 1 gz <latexit sha1_base64="fumyYRuOF/qnlPSScjWGdh/lGrM=">AAAC83icbVJda9swFJW9rzb7aLo97kUsbElKG+zCvh4Khb10MGgHS1uIEiMrSiIqy0KSxxKhv7GXPWyMve7P7G3/ZrITQ9rsguH43HOu75GVSs60iaK/QXjr9p2797a2G/cfPHy009x9fK7zQhHaJznP1WWKNeVM0L5hhtNLqSjOUk4v0qt3Zf/iM1Wa5eKTmUs6zPBUsAkj2Hgq2Q22UUqnTFjM2VTsuQb0hTJsZgRz+94lFTbGfjg9dZ3FPly0u5XmBcJSqvwLRJTzUaVSmfVruMQiM6MGj/a8ob0P67fELlwXHmwarqm6CC3nH8Gpd7TdCJlcws76lHJILYe1vt5n3XUysgexqygHj9Zz+ShlEkTFuI6eNFtRL6oKboJ4BVpgVWdJ8w8a56TIqDCEY60HcSTN0GJlGOHUNVChqcTkCk/pwEOBM6qHtvpnDj73zBhOcuUfYWDFrjsszrSeZ6lXllvrm72S/F9vUJjJm6FlQhaGCrL80KTg0OSwvABwzBQlhs89wEQxvyskM6wwMf6aNPwhxDcjb4Lzw178qvfy42Hr+O3qOLbAU/AMdEAMXoNjcALOQB+QQAZfg+/Bj7AIv4U/w19LaRisPE/AtQp//wOR/+cp</latexit> ILOO(z, z0 ) ⇡ `lin ✓⇤ (z0 , ✓⇤ z ) `lin (z0 , ✓⇤ ) = g> z0 (✓⇤ z ✓⇤ ) ⇡ g> z0 H 1 gz = I(z, z0 )
  • 11. 2. Identifying distributional bias in IF and its approximations
  • 12. Distributional bias in IF and its approximations <latexit sha1_base64="lL3vvGifEpow471EGLQ7F4WhACM=">AAACR3icbVBNaxsxFNQ6Seu4X25z7EXElCRQzG7pRxowGHxJbgnEdsDrLFpZ64hotYv0NmAL/btccs2tf6GXHhJCjtFufGjtDgjNm3kP6U2cC67B9395tbX1jRcv65uNV6/fvH3XfP9hoLNCUdanmcjUWUw0E1yyPnAQ7CxXjKSxYMP4slf6wyumNM/kKcxyNk7JVPKEUwJOiprnYUrgghJhjmxUcQBzqgg9knZ3/nm+s9fADged6goT55jAmp4NdZFGhnYCe+6qaWTmO47RELLc4rIsKxs1W37br4BXSbAgLbTAcdS8DScZLVImgQqi9SjwcxgbooBTwWwjLDTLCb0kUzZyVJKU6bGpcrD4k1MmOMmUOxJwpf49YUiq9SyNXWe5qV72SvF/3qiAZH9suMwLYJI+P5QUAkOGy1DxhCtGQcwcIVRx91dML4jLClz0DRdCsLzyKhl8aQff299Ovra6Pxdx1NFHtI12UYB+oC46RMeojyi6Rr/RHbr3brw/3oP3+Nxa8xYzW+gf1LwnZf2xHw==</latexit> ITracIn(z, z0 ) := 1 C C X c=1 gc> z0 gc z <latexit sha1_base64="Xjnu88gy+RPwO3H+1aB3E6C0IIg=">AAACVnicbVFNaxsxENWumyZ1mnbbHnsRNSEpBLNb+g2GQC71zQ5xEvA6i1bWOiKSdpFmC7bQn2wv7U/ppVS79qF1OjDozXszSPOUV4IbiOOfQdh5sPNwd+9Rd//xwZOn0bPnl6asNWUTWopSX+fEMMEVmwAHwa4rzYjMBbvK784a/eor04aX6gKWFZtJslC84JSAp7JIppLALSXCDl3WYgB7oQkdqvORO16drI5ed7GPz4P2SAuv2cTZM5eaWmaWDhJ346tFZldHHtEUysrhcXbe5E1T4UZrJJdFvbgft4Hvg2QDemgToyz6ls5LWkumgApizDSJK5hZooFTwVw3rQ2rCL0jCzb1UBHJzMy2tjh86Jk5LkrtUwFu2b8nLJHGLGXuO5vFzbbWkP/TpjUUH2eWq6oGpuj6oqIWGErceIznXDMKYukBoZr7t2J6S7xx4H+i601Itle+Dy7f9JP3/Xfjt73TTxs79tBL9AodowR9QKfoCxqhCaLoO/oVhEEn+BH8DnfC3XVrGGxmXqB/Ioz+AJLfsqw=</latexit> ITracInRP(z, z0 ) := 1 C C X c=1 gc> z0 QRQ> Rgc z <latexit sha1_base64="C5MaCNLjqlxsGTzGITmp5Mmztak=">AAACRnicbVBNaxRBEK1Zv5L1I6s5emlcJBF0mRE1MSBEcongIQY3CexMhp6e3k2Tnu6hu0bYbebXefGcmz/Bi4eEkKs9s3vQxAdNP96roqpeVkphMQx/Bp1bt+/cvbe03L3/4OGjld7jJwdWV4bxIdNSm6OMWi6F4kMUKPlRaTgtMskPs9Odxj/8xo0VWn3FacmTgk6UGAtG0UtpL4kLiieMSvepTluO6D4apWUu6vXZy9naiy7x2PrQfpPUzdbq4xh1SYbpPok/+1E5Td1+fexeRXUjzt2msk57/XAQtiA3SbQgfVhgL+2dxblmVcEVMkmtHUVhiYmjBgWTvO7GleUlZad0wkeeKlpwm7g2hpo890pOxtr4p5C06t8djhbWTovMVzaH2uteI/7PG1U43kycUGWFXLH5oHElCWrSZEpyYThDOfWEMiP8roSdUEMZ+uS7PoTo+sk3ycHrQfRu8PbLm/72+0UcS/AUnsE6RLAB27ALezAEBt/hF5zDRfAj+B1cBlfz0k6w6FmFf9CBPyenry4=</latexit> IArnoldi(z, z0 ) := g> z0 UR⇤ 1 R U> R gz
  • 13. Distributional bias in IF and its approximations <latexit sha1_base64="f47u6XDbCSPi2grflbpZy6fnKhM=">AAAFzHicjVTbbhMxEJ1CAqXcWnjkxaJFaqU0Soq4PSBVKgIqAQqoN6kJkdfrTVb17ka2tzRd5ZUP5I1P4JE/4NjZtuklFSt5PT4z9sw5491goGJjG43fMzduVqq3bs/embt77/6Dh/MLj3ZMlmsht0WmMr0XcCNVnMptG1sl9wZa8iRQcjc42HD+3UOpTZylW3Y4kJ2E99I4igW3gLKFmb/UJkuSjvAOKKKCWqQpowGGoRhoDCslRs+pTk0awVqmd0ANfBpzQPlpFCcFf4AVh59hTsu18rYEroEYWArZVj0awc6BpCTwXkEeRp9gWcy5P8edZmAnQBktUY+6dIyZYW+G3QrvH2Wk87cwVinE6Yk/15yrz1XOUZXyp4VTuDieErX0MGpYfcCe3Nfh2KUe2/CYoD4N/drhoa/+Dx0Cm376ykT0En3Gu12q1UbNHHF9xAfoxzdo/t33hflejTk5ri14lrzKJ8wnu3bo2S17rROsV5EzhB35PLHvOitzueqOvO5byCtLdtd16aS31seP+6BxGsMceU5nPITXvaBN1NsF12X0robhNHiL4bpZYO14Oobu9jGvyZlnWq9zzyUBFpbdnaa4Y/cea13WrP0uDavm2UT/cbMu3oHp2Wqn2kyqdL2iV2Xseb+rlvtv6Bh2WOrr7l0MzdZOb/uVvLvzi416wz/sstEsjUUqn1Z3/lc7zESeyNQKxY3ZbzYGtlNwbWOh5GiunRs54OKA9+Q+zJQn0nQK/zMasWdAQhZlGiO1zKOTOwqeGDNMAkQm3PbNRZ8Dr/Lt5zZ63SnidJBbmYpxoihXzGbM/dlYGGsprBrC4ELHqJWJPtdcWPz/5iBC8yLly8bOWr35sv7i69ri+ptSjll6Qk9xZZv0itbpIz66bRKVzUpWOaoMq1+qtlpUR+PQGzPlnsd07qn+/AdL/z9U</latexit> Proposition 3.1 (Distributional bias in bilinear self-influence). Let us as- sume gz follows a P-dimensional stable distribution (e.g., Gaussian, Cauchy, and Lévy distribution) and M 2 RP ⇥P is a positive (semi-)definite matrix. Then, self-influence in the form of IM (z, z) = g> z Mgz follows a unimodal distribution. Furthermore, if gz follows a Gaussian distribution, then the self-influence follows a generalized 2 -distribution.
  • 14. A simple case study in a modified two-circle dataset <latexit sha1_base64="tEVn5IcyeMefnGdLs/hWXGoL6PM=">AAACAXicbZDLSsNAFIYn9VbrLepGcBMsgquSSL0sC24UhFawF2hDmEwn7dDJhZkToYS48VXcuFDErW/hzrdxkmahrT8MfPznHOac3404k2Ca31ppaXllda28XtnY3Nre0Xf3OjKMBaFtEvJQ9FwsKWcBbQMDTnuRoNh3Oe26k6us3n2gQrIwuIdpRG0fjwLmMYJBWY5+MPAxjAnmyU3q5AyQ3DabqaNXzZqZy1gEq4AqKtRy9K/BMCSxTwMgHEvZt8wI7AQLYITTtDKIJY0wmeAR7SsMsE+lneQXpMaxcoaGFwr1AjBy9/dEgn0pp76rOrMd5XwtM/+r9WPwLu2EBVEMNCCzj7yYGxAaWRzGkAlKgE8VYCKY2tUgYywwARVaRYVgzZ+8CJ3TmnVeO7urVxv1Io4yOkRH6ARZ6AI10DVqoTYi6BE9o1f0pj1pL9q79jFrLWnFzD76I+3zBwqwlz0=</latexit> ILOO <latexit sha1_base64="sRYssw0GitVG1c+smryccc5XV6I=">AAACA3icbVDLSsNAFJ34rPUVdaebwSLUTUnEF4hQdKO7CvYBTSg300k7dPJgZiKUUHDjr7hxoYhbf8Kdf+OkzUJbD1w4nHMv997jxZxJZVnfxtz8wuLScmGluLq2vrFpbm03ZJQIQusk4pFoeSApZyGtK6Y4bcWCQuBx2vQG15nffKBCsii8V8OYugH0QuYzAkpLHXPXCUD1CfD0doSdi7IDPO7DpW1Zhx2zZFWsMfAssXNSQjlqHfPL6UYkCWioCAcp27YVKzcFoRjhdFR0EkljIAPo0bamIQRUuun4hxE+0EoX+5HQFSo8Vn9PpBBIOQw83ZldLKe9TPzPayfKP3dTFsaJoiGZLPITjlWEs0BwlwlKFB9qAkQwfSsmfRBAlI6tqEOwp1+eJY2jin1aObk7LlWv8jgKaA/tozKy0RmqohtUQ3VE0CN6Rq/ozXgyXox342PSOmfkMzvoD4zPH+s/lmg=</latexit> I (↵ = 100) <latexit sha1_base64="ZmlcEXoPjl2BxdNi7DFGhbT3CY8=">AAACBHicbVDLSsNAFJ34rPUVddnNYBHqpiTiC0QoutFdBfuAJpSb6aQdOnkwMxFK6MKNv+LGhSJu/Qh3/o2TNgttPTDM4Zx7ufceL+ZMKsv6NhYWl5ZXVgtrxfWNza1tc2e3KaNEENogEY9E2wNJOQtpQzHFaTsWFAKP05Y3vM781gMVkkXhvRrF1A2gHzKfEVBa6polJwA1IMDT2zF2LioO8HgAl1bVsg+7Zln/E+B5YuekjHLUu+aX04tIEtBQEQ5SdmwrVm4KQjHC6bjoJJLGQIbQpx1NQwiodNPJEWN8oJUe9iOhX6jwRP3dkUIg5SjwdGW2spz1MvE/r5Mo/9xNWRgnioZkOshPOFYRzhLBPSYoUXykCRDB9K6YDEAAUTq3og7Bnj15njSPqvZp9eTuuFy7yuMooBLaRxVkozNUQzeojhqIoEf0jF7Rm/FkvBjvxse0dMHIe/bQHxifP17olqA=</latexit> I (↵ = 0.01) <latexit sha1_base64="AoSyz/4d860DiF5Wn/pRjBOqz3U=">AAACCnicbVC7SgNBFJ2Nrxhfq5Y2o0GICGFXfDVC0CZ2EcwDsku4O5lNhsw+mJkVwpLaxl+xsVDE1i+w82+cTVJo4oGBM+fcy733eDFnUlnWt5FbWFxaXsmvFtbWNza3zO2dhowSQWidRDwSLQ8k5SykdcUUp61YUAg8Tpve4Cbzmw9USBaF92oYUzeAXsh8RkBpqWPuV0sO8LgPR/gKV/ExnvywE4Dqe356O+qYRatsjYHniT0lRTRFrWN+Od2IJAENFeEgZdu2YuWmIBQjnI4KTiJpDGQAPdrWNISASjcdnzLCh1rpYj8S+oUKj9XfHSkEUg4DT1dmG8pZLxP/89qJ8i/dlIVxomhIJoP8hGMV4SwX3GWCEsWHmgARTO+KSR8EEKXTK+gQ7NmT50njpGyfl8/uTouV62kcebSHDlAJ2egCVVAV1VAdEfSIntErejOejBfj3fiYlOaMac8u+gPj8wcybpi7</latexit> H(↵) = H + ↵I
  • 15. 3. Resolving distributional bias via Geometric Ensemble
  • 16. Removing Linearization <latexit sha1_base64="P2Y6+MzRDhxgW0qX2P+Pb+C/7Uk=">AAAEkXiclVNtTxNBED5oVayKIB/9MpEqraFNS1CBpAZEDURiMOEt6bXN3nau3bC3e+7u8Xbe7/H/+M1/4961RQok4iSXm8w88/bMjhdypk2t9ntiMpe/d//B1MPCo8dPpp/OzD470DJSFPep5FIdeUQjZwL3DTMcj0KFJPA4HnrHm6n/8ASVZlLsmfMQWwHpCeYzSow1dWYnf7oGz4znx3t9lAoDWK7WEyhtSiGQphjw0JwiCtj+DER0YWejXIWiGxDTp4TH20kRmIDm0koLKEnRgGe2Ba2xC0RDwfWwx0RMOOuJ10kBrFwJLl0swsVCOTO/akD2jzO/58Wfkk7shpqBq1kAYSezGxPvbCTDRBx90xyoH5EbAi5y3s5wKogtKWkG00dD2q+zWmm6Mri0K83dQhZGMYMqivX6pjWqTjzkMX5fY36lbyeuUHmSFFwU3ctxT/uo0NL1P92tNe4IrPwLN9LL0IBe56LtGhlCKaPUxo6cxWyvxTF+7VI1WADskJATikDCUMkzZhF2r6HUBhWT6rbtjuUp/R1ptPSvSba2UjYE/LjsYxG22nHFPr4ByeXqGJOdmflatZYJ3FTqQ2XeGcpuZ+aX25U0ClAYyonWzXotNK2YKMMoR7umSGNI6DHpYdOqggSoW3F2UQm8jNLn60tlP2Egs16NiEmg9XngWWQ6lr7uS423+ZqR8VdaMRNhZFDQQSE/4mAkpOcJXabs2fFzqxCqmO0VaJ8oQi3fumBJqF8f+aZysFStv62++bY0v746pGPKee68cEpO3XnnrDtbzq6z79DcdG4518i9z8/lV/Pr+Q8D6OTEMGbOGZP8lz9F7nSe</latexit> Theorem 4.1 (Connection between IF and LA). I in [28] can be expressed as I(z, z0 ) = E ⇠pLA ⇥ `lin ✓⇤ (z, ) · `lin ✓⇤ (z0 , ) ⇤ where `lin ✓⇤ (z, ) := `lin ✓⇤ (z, ) `lin ✓⇤ (z, ✓⇤ ) = g> z ( ✓⇤ ) and pLA is the Laplace approximated posterior pLA( ) := N |✓⇤ , H 1 . <latexit sha1_base64="M+LVLxJYyTlnS/UIE0jcBjaofaM=">AAACknicdVHbatwwEJXdpkm3l2wvb30RXUqSUha7JL0EciMNpJCHFLpJYO2asXa8KyJfkMaBjfEH5Xfy1r+p7N1CLu0BweHMmZF0Ji6UNOR5vx33wcOFR4tLjztPnj57vtx98fLE5KUWOBC5yvVZDAaVzHBAkhSeFRohjRWexuf7Tf30ArWRefaTpgWGKYwzmUgBZKWoexWkQBMBqvpeRy0nqo726tXLD5cra3xzi7diHFcHdVQFhZE8MDLlxU1z3eEWgcKEhjP6DRUBD1Ap20QTJPj1vpnJmwlrPBCjnP7rWvlrm83ScjyhsB91e17fa8HvE39OemyO46h7HYxyUaaYkVBgzND3Cgor0CSFwroTlAYLEOcwxqGlGaRowqqNtObvrDLiSa7tyYi36s2OClJjpmlsnU0O5m6tEf9VG5aUfAkrmRUlYSZmFyWl4pTzZj98JDUKUlNLQGhp38rFBDQIslvs2BD8u1++T04+9v1P/Y0f673dr/M4ltgb9patMp99ZrvskB2zARPOsrPhbDs77mt3091z92dW15n3vGK34B79Abh5w8Q=</latexit> ILA(z, z0 ) := E ⇠pLA [ `✓⇤ (z, ) · `✓⇤ (z0 , )] . <latexit sha1_base64="srbKo/FwyKi7kz4Ynx9o8Z1llTs=">AAACLXicbVDLSgMxFM3UV62vqks3wSJU0TIjvkEo6MKlgtVCp5ZMetuGZh4kd4Q69Ifc+CsiuKiIW3/DtFbU6oHAyTnnktzjRVJotO2elRobn5icSk9nZmbn5heyi0tXOowVhxIPZajKHtMgRQAlFCihHClgvifh2muf9P3rW1BahMEldiKo+qwZiIbgDI1Uy566pyCRURekrCUutgDZzUY3f7dJ3UiLdXp0TAfmt7L1df9Kr9eyObtgD0D/EmdIcmSI81r2ya2HPPYhQC6Z1hXHjrCaMIWCS+hm3FhDxHibNaFiaMB80NVksG2XrhmlThuhMidAOlB/TiTM17rjeybpM2zpUa8v/udVYmwcVBMRRDFCwD8fasSSYkj71dG6UMBRdgxhXAnzV8pbTDGOpuCMKcEZXfkvudouOHuF3YudXPFwWEearJBVkicO2SdFckbOSYlwck8eSY+8WA/Ws/VqvX1GU9ZwZpn8gvX+ARRPpYk=</latexit> `✓⇤ (z, ) := `(z, ) `(z, ✓⇤ )
  • 17. Utilization of Geometric Ensemble <latexit sha1_base64="gRUshho/+GpXwFSAGbUgD/oKvMw=">AAAEIXicbVNLbxMxEN4mPEp4tXDkMqKLlJY0SipBOVCpAglVahUF+pS6beR1ZrNWvfZie0uT1f4VLvwVLhxAqDfEn2GyaUtbmIM1+jyPb8afw1QK61qtX1OV6o2bt25P36ndvXf/wcOZ2Uc7VmeG4zbXUpu9kFmUQuG2E07iXmqQJaHE3fDo7fh+9xiNFVptuWGKBwkbKBEJzhxBvdnKcuDwxIVR3jU61VaM4QLqa2itYAqsUINMMiPcECJtQFOxxZQZlqBDI0bYh07HzjdhAx1kFpi1WYLAgFgsOsOIVh8u4sEPXIyOHS5AIBQECXNxGOYfisO8W/jAeCzwGC2M0Ggos6k9SG0t+Bv1zQacp8/DCrT8kpH9mDFDTdAYbZqwFaNqgL/mQ8yIjgOJzDrwu4uddb8svIhigOqYyYw6jSvQAGAzHoOLKd7vrMNr6PpNeJcZamcSbbABIgL/xAdBNRUIlWYOdPSXo2VJKhH8UTnYpt+gYqjGB7WQUn8aB8Va9ictx/iEBnKnDY0X5JD1BARFLxcrnfXn7clOakGIA6FyJsVALRQ1IBv0RoeB02mZsAKBYqFkJdLLAxohHxYFBChlfXRpY0Gm+mhCwzjmV1PKiCKqnzQugqkyEZm8T5S3qB7tuxag6p8z6c3MtZqt0uBfp33mzHln1u3NnAZ9zUkcynFJMtlvt1J3kDPjBJdY1ILMYsr4ERvgPrmKFGMP8lLhBTwjpF9uLtLKQYlezshZYu0wCSlyzNlevxuD/7vbz1z06iAv3xMVnzSKMgmO9EffBfrC0AvJITmM0y8QHHhMcuakZlujJbSvj/yvs7PUbL9svni/NLf65mwd094T76lX99resrfqrXldb9vjlc+Vr5XvlR/VL9Vv1Z/V00loZeos57F3xaq//wD+QVa0</latexit> Proposition (Hessian singularity for over-parameterized NNs). Let us as- sume a pre-trained parameter ✓⇤ 2 RP achieves zero training loss L(S, ✓⇤ ) = 0 for squared error. Then, H has at least P NK zero-eigenvalues for NNs such that NK < P. Furthermore, if x is an input of training sample z 2 S, then the following holds for the eigenvectors {ui}P i=NK+1 g> z ui = r> ŷ `(z, ✓⇤ ) r> ✓ f(x, ✓⇤ )ui | {z } 0 = 0 <latexit sha1_base64="PpgwH7Ake9PCmrRDLUlAeqSlFn8=">AAACEHicbVDJSgNBEO1xjXGLevTSGMR4CTPiBiJEvSh4iGAWyIRQ0+kkTXoWumuEMOQTvPgrXjwo4tWjN//GznLQxAdNP96roqqeF0mh0ba/rZnZufmFxdRSenlldW09s7FZ1mGsGC+xUIaq6oHmUgS8hAIlr0aKg+9JXvG6VwO/8sCVFmFwj72I131oB6IlGKCRGpk91wfsMJDJTb8x5IjJ7UWfumc5F2TUgXM7bzv7jUzW/EPQaeKMSZaMUWxkvtxmyGKfB8gkaF1z7AjrCSgUTPJ+2o01j4B1oc1rhgbgc11Phgf16a5RmrQVKvMCpEP1d0cCvtY93zOVg5X1pDcQ//NqMbZO64kIohh5wEaDWrGkGNJBOrQpFGcoe4YAU8LsSlkHFDA0GaZNCM7kydOkfJB3jvNHd4fZwuU4jhTZJjskRxxyQgrkmhRJiTDySJ7JK3mznqwX6936GJXOWOOeLfIH1ucPChab6g==</latexit> ILA (↵ = 0.01) <latexit sha1_base64="JY+U0ZmSLnSoGiMXkCJ3WQ+ilXc=">AAACD3icbZDJSgNBEIZ7XGPcoh69NAYlXsKMuIEIUS8KHiKYBTIh1HQ6SZOehe4aIQx5Ay++ihcPinj16s23sbMcNPGHho+/quiq34uk0Gjb39bM7Nz8wmJqKb28srq2ntnYLOswVoyXWChDVfVAcykCXkKBklcjxcH3JK943atBvfLAlRZhcI+9iNd9aAeiJRigsRqZPdcH7DCQyU2/MWTE5PaiT92znAsy6sC5Y9v7jUzWzttD0WlwxpAlYxUbmS+3GbLY5wEyCVrXHDvCegIKBZO8n3ZjzSNgXWjzmsEAfK7ryfCePt01TpO2QmVegHTo/p5IwNe653umc7CxnqwNzP9qtRhbp/VEBFGMPGCjj1qxpBjSQTi0KRRnKHsGgClhdqWsAwoYmgjTJgRn8uRpKB/kneP80d1htnA5jiNFtskOyRGHnJACuSZFUiKMPJJn8krerCfrxXq3PkatM9Z4Zov8kfX5A5GDm7I=</latexit> ILA (↵ = 100) <latexit sha1_base64="s0zEbOYkkBwPLESPadaN5uCYuU0=">AAACCXicbZC7SgNBFIZn4y3G26qlzWAQYhN2xaggQtRGwSKKuUASwuxkkgyZvTBzVgjLtja+io2FIra+gZ1v42SzhSb+MPDxn3OYc34nEFyBZX0bmbn5hcWl7HJuZXVtfcPc3KopP5SUVakvfNlwiGKCe6wKHARrBJIR1xGs7gwvx/X6A5OK+949jALWdknf4z1OCWirY+KWS2BAiYiu407CANHNeYxbp4W7s5K13zHzVtFKhGfBTiGPUlU65ler69PQZR5QQZRq2lYA7YhI4FSwONcKFQsIHZI+a2r0iMtUO0ouifGedrq450v9PMCJ+3siIq5SI9fRneNd1XRtbP5Xa4bQO2lH3AtCYB6dfNQLBQYfj2PBXS4ZBTHSQKjkeldMB0QSCjq8nA7Bnj55FmoHRfuoWLo9zJcv0jiyaAftogKy0TEqoytUQVVE0SN6Rq/ozXgyXox342PSmjHSmW30R8bnDwB/mTo=</latexit> ILA (R = 50) <latexit sha1_base64="IF4DsSdOVW+Hs34gt59F9n075OY=">AAAB/XicbVDLSgMxFL3js9bX+Ni5CRahbsqMWBVEKLrRXRX7gM5QMmnahmYeJBmhDsVfceNCEbf+hzv/xkw7C209EDiccy/35HgRZ1JZ1rcxN7+wuLScW8mvrq1vbJpb23UZxoLQGgl5KJoelpSzgNYUU5w2I0Gx73Ha8AZXqd94oEKyMLhXw4i6Pu4FrMsIVlpqm7uOj1WfYJ7cjJBzXry7KFuHbbNglawx0CyxM1KADNW2+eV0QhL7NFCEYylbthUpN8FCMcLpKO/EkkaYDHCPtjQNsE+lm4zTj9CBVjqoGwr9AoXG6u+NBPtSDn1PT6ZZ5bSXiv95rVh1z9yEBVGsaEAmh7oxRypEaRWowwQlig81wUQwnRWRPhaYKF1YXpdgT395ltSPSvZJqXx7XKhcZnXkYA/2oQg2nEIFrqEKNSDwCM/wCm/Gk/FivBsfk9E5I9vZgT8wPn8Ad7eT8A==</latexit> I (R = 50)
  • 18. Utilization of Geometric Ensemble <latexit sha1_base64="pnaUFL7AQVg8XQ793gRTJ7w7D+E=">AAAClXicdVFdT9RAFJ1WFFw/WPDBB18mbgxgzKYliryQIPKhT2Liwibb2tzO3u5OmH5k5tZkafqP/DW88W+YdtcEQU8yyck5596ZuTculDTkedeO+2Dp4aPllcedJ0+fPV/trq2fmbzUAgciV7kexmBQyQwHJEnhsNAIaazwPL743Pjnv1AbmWc/aFZgmMIkk4kUQFaKur+DFGgqQFVf66jlRNXJ0bDevHx3ubHV4RZ7vDXiuDqqoyoojOSBkSkvbhXUdRsNFCY0mtNDVAQ8QKVsEU2R4Ofbpi1vOmzxQIxz+m9q409s3kvLyZTCqNvz+l4Lfp/4C9JjC5xG3atgnIsyxYyEAmNGvldQWIEmKRTWnaA0WIC4gAmOLM0gRRNW7VRr/sYqY57k2p6MeKverqggNWaWxjbZjMHc9RrxX96opGQ3rGRWlISZmF+UlIpTzpsV8bHUKEjNLAGhpX0rF1PQIMgusmOH4N/98n1ytt33d/ofvr/v7R8sxrHCXrHXbJP57CPbZ1/YKRsw4aw7u84n58B96e65h+7xPOo6i5oX7C+4324A8G/EQw==</latexit> IGEX(z, z0 ) = E ⇠pGE [ `✓⇤ (z, ) · `✓⇤ (z0 , )] <latexit sha1_base64="M+LVLxJYyTlnS/UIE0jcBjaofaM=">AAACknicdVHbatwwEJXdpkm3l2wvb30RXUqSUha7JL0EciMNpJCHFLpJYO2asXa8KyJfkMaBjfEH5Xfy1r+p7N1CLu0BweHMmZF0Ji6UNOR5vx33wcOFR4tLjztPnj57vtx98fLE5KUWOBC5yvVZDAaVzHBAkhSeFRohjRWexuf7Tf30ArWRefaTpgWGKYwzmUgBZKWoexWkQBMBqvpeRy0nqo726tXLD5cra3xzi7diHFcHdVQFhZE8MDLlxU1z3eEWgcKEhjP6DRUBD1Ap20QTJPj1vpnJmwlrPBCjnP7rWvlrm83ScjyhsB91e17fa8HvE39OemyO46h7HYxyUaaYkVBgzND3Cgor0CSFwroTlAYLEOcwxqGlGaRowqqNtObvrDLiSa7tyYi36s2OClJjpmlsnU0O5m6tEf9VG5aUfAkrmRUlYSZmFyWl4pTzZj98JDUKUlNLQGhp38rFBDQIslvs2BD8u1++T04+9v1P/Y0f673dr/M4ltgb9patMp99ZrvskB2zARPOsrPhbDs77mt3091z92dW15n3vGK34B79Abh5w8Q=</latexit> ILA(z, z0 ) := E ⇠pLA [ `✓⇤ (z, ) · `✓⇤ (z0 , )] .
  • 23. Conclusion and Future directions • AsshowninTheorem4.1,IF isclosely relatedto the Laplace Approximation. • Onthe otherhand,[23] formalizedthe relationshipbetweensharpnessandLaplace ApproximationthroughPAC-Bayes. • Althoughthere iscriticismthat sharpnessisnot directly relatedto generalization[24],anaccurate understandingof the evolution (dynamics) of sharpnesswillhelpclarify the relationshipbetweensharpnessandgeneralization. • Recent worksonEdge-of-Stability (EOS)[25,26,27] wouldbe agoodstartingpoint.
  • 24. Thank you ! Any Questions ?