Download to read offline
More Related Content
![ViT (Vision Transformer) Review [CDM]](https://cdn.slidesharecdn.com/ss_thumbnails/vitreviewcdm-201012184226-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DL輪読会]M2Det: A Single-Shot Object Detector based on Multi-Level Feature Pyra...](https://cdn.slidesharecdn.com/ss_thumbnails/20190307m2detkuboshizuma-190308010212-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Noise Distribution Adaptive Self-Supervised Image Denoising using Tweedie Distribution and Score Matching
Noise Distribution Adaptive Self-Supervised Image Denoising using Tweedie Distribution and Score Matching
- 1. Noise Distribution AdaptiveSelf-Supervised Image Denoising using Tweedie Distribution and Score Matching Kwanyoung Kim, Taesung Kwon, Jong Chul Ye Bio Imaging, Signal Processing & Learning Lab @KAIST
- 2. Self-Supervised Image Denoising •Do not require clean images, noisy image pairs, using only single noisy images. • Recently, there have been extensive research in training deep network to denoise images without clean reference.
- 3. Noise2Score Score function Noise2X, SURENoise2Score Kim and Ye, NeurIPS, 2021
- 4. Tweedie’s formula forgeneral exponential family • The closed form solution for the posterior mean: • Bayes optimal solution posterior mean • Probability distribution of exponential family: B Efron, Journal of the American Statistical Association, 2011
- 5. Tweedie’s formula ofexponential family distribution for image denoising As long as we can compute the score function, The optimal denoising can be achieved by using Tweedie’s formula.
- 6. However, Noise2Score stillrequire the information of noise statistics such as Noise model and Noise level. Is it possible to estimate the corresponding the noise model and level given the noisy measurements? In truly Blind denoising problem, the prior knowledge of noise statistics is not given.
- 7. Yes! Kim etal., Noise Distribution Adaptive Self-Supervised Image Denoising using Tweedie Distribution and Score Matching, CVPR 2022
- 8. Comparison with Othermethods
- 9. Tweedie Distribution • Probabilitydistribution of Tweedie family: • Power Variance function: • Tweedie family are characterized by power variance functions : • Tweedie index, ,determine the noise distribution!
- 10. Saddle point approximationof Tweedie family • However, this part cannot be represented in closed form except for special case. • To address this issue, the saddle point approximation was used to lead a simple expression of the density. Peter K Dunn et al. Tweedie family densities: methods of evaluation. In Proceedings of the 16th international workshop on statistical modelling, 2001.
- 11. Noise2Score for TweedieDistribution Apply the above formula Noise2Score We can provide the universal denoising formula that can be used for various form of Tweedie distribution.
- 12. Noise2Score for TweedieDistribution Is this universal closed-form solution converge into specific formula in Noise2Score?
- 13. Noise2Score for TweedieDistribution
- 14. Noise2Score for TweedieDistribution As long as we can estimate the pair of ( 𝝆, ∅ ), We can estimate clean image in case of blind denoising
- 15. How to estimatethe Noise model and Noise level?
- 16. Motivation of EstimatingNoise statistics Tweedie’s formula Tweedie’s formula Perturbed Noise
- 17. Blind Noise modelestimation Gaussian Assumption Poisson Gamma 0 1 2
- 18. Noise level Estimation Knownvalue Unknown value This assumption lead to the equation for only noise level parameter Now, we estimated the noise model, but still require noise level
- 19. Blind Noise levelEstimation
- 20. Case 1. AdditiveGaussian noise • The proposed method require only one more inference steps to find value. (About x30 faster than Noise2Score)
- 21.
- 22. Analysis on NoiseModel Estimation
- 23. Ablation Study onNoise Level Estimation
- 24. Conclusion • We providea general closed-form denoising formula for various classes of noise distribution. • The proposed method can estimate the noise model and the noise level without any prior knowledge. • We show our method results in state-of-the-art performance amongst self-supervised image denoising algorithms.
- 25. Thank you! Jong ChulYe E-mail: jong.ye@kaist.ac.kr Kwanyoung Kim E-mail: cubeyoung@kaist.ac.kr
Editor's Notes
- #2 Hello, every one. I’m kwanyoung kim. I will present our paper, Noise distribution Adaptive Self-supervised Image denoising using Tweedie Distribution and Score Matching,
- #3 Recently, Self-supervised image denoising method have been widely explored. These methods can be trained using only single noisy images..
- #4 Recently, our group propose Noise2Score that have a totally different framework from Noise2X, SURE. Firstly, Noise2Score estimate the score function of data by using AR-DAE loss function. And then, they reconstruct clean image by Tweedie’s formula given the noise model.
- #5 We employed the key point that Tweedie formula could be extended for general exponential family. Given the probability distribution of exponential family, we can obtain the posterior density by using the Bayes’ rule.
- #6 We employed the key point that Tweedie formula could be extended for general exponential family. Given the probability distribution of exponential family, we derive the closed form solution by calculating the posterior mean. This Red box is the key point of Noise2Score. By applying this equation, we can derive the denoising formula for various classes of noise model.
- #7 Noise2Score can be applied any noise distribution which belong to exponential family. As long as we can compute the score function, the optimal denoising can be achieved by using tweedie’s formula.
- #8 Read the slide contents.
- #9 In our method, the training procedure is identical to Noise2Score which to learn the score function of measurement. In the inference stage, given trained neural network, we sequentially estimate the noise model and noise level. After that, the denoised images are obtained by Tweedie’s formula.
- #10 Compared to other methods which utilize the noise statistics, SURE require both noise model and noise level information. Noise2Score and Laine’s method can estimate the noise level but need to know noise distribution. On the other hand, Our method estimate both noise model and noise level information without any prior knowledge.
- #11 We found that The Tweedie distribution can be extend to denoising problem. Tweedie family are characterized by power variance functions. The interesting part of this distribution is that tweedie index rho determine the noise distribution. For example, when rho is 0, the distribution is gaussian, when rho is 1, Poisson. This property is key observation of our method. So, we want to estimate this rho value.
- #12 However, the part including rho cannot be represented in closed form except for special case. To address this issue, the saddle point approximation was used to lead a simple expression of the density. In other words, tweedie density can be defined this equation.
- #13 As I said earlier, Noise2Score provide the key equation given the exponential family density. Here, by combining the saddle point approximation and key idea of Noise2Score, we provide the universal denoising formula that can be used for various form of Tweedie distribution.
- #14 In other words, we propose the proposition 1. I said that this formula is universal formula. Then, is this universal closed-from solution converge into specific formula in Noise2Score?
- #15 Yes, we demonstrated that the universal formula converges to the specific formulae for given parameter pairs as shown in Table 2. In this presentation, we omit the proof of proposition 2.
- #16 In this presentation, we only provide the proof of gaussian case. In this case, alpha can be simplified by this. By pluging this alpha into universal fourmlua, we can obtain this equation. Note that this formula is equivalent to that of Noise2Score.
- #17 To sum up, as long as we can estimate the parameter pair of rho and pi, we can estimate clean image in case of blind denoising.
- #18 Then, How to calculate the noise model and noise level?
- #19 I will introduce the simple motivation of estimating noise statistics. Let y1 be the noisy measurement. Suppose we add a small amount noise to generate y_2. if the injected noise is sufficienty small, we assume that their denoised images by tweedies formula should be simmiar. This motivation lead to the estimation of both noise model and noise level.
- #20 In Noise model estimation, this motivation lead to another assumption. Alpha 1 and alpha2 is same. From this assumption, the key observation is that the noise level parameter pi is cancelled out. That is, the above formula lead to quadratic equation for only noise model parameter rho. By solving the quadratic equation, we can obtain the noise model parameter rho hat. If rho hat is included specific range, we determine the noise distribution.
- #21 Now we know noise distribution, but still require noise level pi to yield denoised image. We can represent the initial motivation like this, The blue line is known value and red line, pi is only unknown value. It implied that we can represent equation for only noise level. parameter.
- #22 So by solving the corresponding equation, we can proposed Proposition 4. the noise level parameter of several noise distribution is given by equation 12.
- #23 We will only provide the proof of gaussian noise case. Given this assumption, we can represent the each denoised formula for gaussian noise case. Subtracting below from above equation, we have this. And finally, we have the following estimate. The interesting part is that our method require only one more inference steps to find value. 30 times faster than Noise2Score.
- #24 To evaluate our method, we performed the synthetic experiments using various methods. Compared to other methods, our method provide the better performance in terms of visual quality.
- #25 According to Tweedie distribution, we expect that rho hat can be distinguished for each noise distribution. From the figure we can observe that the noise model parameters are distinctly distributed in the both case of low noise level(left figure) and of high noise level(right figure). It implies that the proposed method successfully estimate the noise model.
- #26 We analyzed the effect of the proposed noise level estimation by comparing Noise2Score with quality penalty metric. Quailty penalty metric incorrectly estimate the noise level. On the other hand, our method provided more accurate results for all of case.
- #27 To conclude, we provide a ~ Read slide. This is end of my presentation. Thank you for listening
- #28 This is end of our presentation. THANK YOU FOR LISTENING.