Noise Distribution Adaptive Self-Supervised Image Denoising using Tweedie Distribution and Score Matching
1. Noise Distribution Adaptive Self-Supervised
Image Denoising using
Tweedie Distribution and Score Matching
Kwanyoung Kim, Taesung Kwon, Jong Chul Ye
Bio Imaging, Signal Processing & Learning Lab
@KAIST
2. Self-Supervised Image Denoising
• Do not require clean images, noisy image pairs, using only single noisy images.
• Recently, there have been extensive research in training deep network to denoise
images without clean reference.
4. Tweedie’s formula for general exponential family
• The closed form solution for the posterior mean:
• Bayes optimal solution posterior mean
• Probability distribution of exponential family:
B Efron, Journal of the American Statistical Association, 2011
5. Tweedie’s formula of exponential family distribution for
image denoising
As long as we can compute the score function,
The optimal denoising can be achieved by using Tweedie’s formula.
6. However, Noise2Score still require the information of noise statistics such as
Noise model and Noise level.
Is it possible to estimate the corresponding
the noise model and level given the noisy measurements?
In truly Blind denoising problem, the prior knowledge of noise statistics is not
given.
7. Yes! Kim et al., Noise Distribution Adaptive Self-Supervised Image Denoising using
Tweedie Distribution and Score Matching, CVPR 2022
9. Tweedie Distribution
• Probability distribution of Tweedie family:
• Power Variance function:
• Tweedie family are characterized by power variance functions :
• Tweedie index, ,determine the noise distribution!
10. Saddle point approximation of Tweedie family
• However, this part cannot be represented in closed form except for special case.
• To address this issue, the saddle point approximation was used to lead a simple expression of the density.
Peter K Dunn et al. Tweedie family densities: methods of evaluation. In Proceedings of the 16th
international workshop on statistical modelling, 2001.
11. Noise2Score for Tweedie Distribution
Apply the above formula
Noise2Score
We can provide the universal denoising formula that can be used for various form of
Tweedie distribution.
12. Noise2Score for Tweedie Distribution
Is this universal closed-form solution converge into specific formula in
Noise2Score?
18. Noise level Estimation
Known value
Unknown value
This assumption lead to the equation for only noise level parameter
Now, we estimated the noise model, but still require noise level
24. Conclusion
• We provide a general closed-form denoising formula for various classes
of noise distribution.
• The proposed method can estimate the noise model and the noise level
without any prior knowledge.
• We show our method results in state-of-the-art performance amongst
self-supervised image denoising algorithms.
25. Thank you!
Jong Chul Ye
E-mail:
jong.ye@kaist.ac.kr
Kwanyoung Kim
E-mail:
cubeyoung@kaist.ac.kr
Editor's Notes
Hello, every one. I’m kwanyoung kim. I will present our paper, Noise distribution Adaptive Self-supervised Image denoising using Tweedie Distribution and Score Matching,
Recently, Self-supervised image denoising method have been widely explored. These methods can be trained using only single noisy images..
Recently, our group propose Noise2Score that have a totally different framework from Noise2X, SURE.
Firstly, Noise2Score estimate the score function of data by using AR-DAE loss function. And then, they reconstruct clean image by Tweedie’s formula given the noise model.
We employed the key point that Tweedie formula could be extended for general exponential family. Given the probability distribution of exponential family, we can obtain the posterior density by using the Bayes’ rule.
We employed the key point that Tweedie formula could be extended for general exponential family. Given the probability distribution of exponential family,
we derive the closed form solution by calculating the posterior mean. This Red box is the key point of Noise2Score. By applying this equation, we can derive the denoising formula for various classes of noise model.
Noise2Score can be applied any noise distribution which belong to exponential family. As long as we can compute the score function, the optimal denoising can be achieved by using tweedie’s formula.
Read the slide contents.
In our method, the training procedure is identical to Noise2Score which to learn the score function of measurement. In the inference stage, given trained neural network, we sequentially estimate the noise model and noise level. After that, the denoised images are obtained by Tweedie’s formula.
Compared to other methods which utilize the noise statistics, SURE require both noise model and noise level information. Noise2Score and Laine’s method can estimate the noise level but need to know noise distribution. On the other hand, Our method estimate both noise model and noise level information without any prior knowledge.
We found that The Tweedie distribution can be extend to denoising problem.
Tweedie family are characterized by power variance functions. The interesting part of this distribution is that tweedie index rho determine the noise distribution.
For example, when rho is 0, the distribution is gaussian, when rho is 1, Poisson. This property is key observation of our method. So, we want to estimate this rho value.
However, the part including rho cannot be represented in closed form except for special case. To address this issue, the saddle point approximation was used to lead a simple expression of the density. In other words, tweedie density can be defined this equation.
As I said earlier, Noise2Score provide the key equation given the exponential family density. Here, by combining the saddle point approximation and key idea of Noise2Score, we provide the universal denoising formula that can be used for various form of Tweedie distribution.
In other words, we propose the proposition 1. I said that this formula is universal formula. Then, is this universal closed-from solution converge into specific formula in Noise2Score?
Yes, we demonstrated that the universal formula converges to the specific formulae for given parameter pairs as shown in Table 2. In this presentation, we omit the proof of proposition 2.
In this presentation, we only provide the proof of gaussian case.
In this case, alpha can be simplified by this. By pluging this alpha into universal fourmlua, we can obtain this equation. Note that this formula is equivalent to that of Noise2Score.
To sum up, as long as we can estimate the parameter pair of rho and pi, we can estimate clean image in case of blind denoising.
Then, How to calculate the noise model and noise level?
I will introduce the simple motivation of estimating noise statistics.
Let y1 be the noisy measurement. Suppose we add a small amount noise to generate y_2. if the injected noise is sufficienty small, we assume that their denoised images by tweedies formula should be simmiar.
This motivation lead to the estimation of both noise model and noise level.
In Noise model estimation, this motivation lead to another assumption. Alpha 1 and alpha2 is same. From this assumption, the key observation is that the noise level parameter pi is cancelled out. That is, the above formula lead to quadratic equation for only noise model parameter rho. By solving the quadratic equation, we can obtain the noise model parameter rho hat. If rho hat is included specific range, we determine the noise distribution.
Now we know noise distribution, but still require noise level pi to yield denoised image.
We can represent the initial motivation like this, The blue line is known value and red line, pi is only unknown value. It implied that we can represent equation for only noise level. parameter.
So by solving the corresponding equation, we can proposed Proposition 4. the noise level parameter of several noise distribution is given by equation 12.
We will only provide the proof of gaussian noise case.
Given this assumption, we can represent the each denoised formula for gaussian noise case. Subtracting below from above equation, we have this.
And finally, we have the following estimate. The interesting part is that our method require only one more inference steps to find value. 30 times faster than Noise2Score.
To evaluate our method, we performed the synthetic experiments using various methods. Compared to other methods, our method provide the better performance in terms of visual quality.
According to Tweedie distribution, we expect that rho hat can be distinguished for each noise distribution.
From the figure we can observe that the noise model parameters are distinctly distributed in the both case of low noise level(left figure) and of high noise level(right figure). It implies that the proposed method successfully estimate the noise model.
We analyzed the effect of the proposed noise level estimation by comparing Noise2Score with quality penalty metric. Quailty penalty metric incorrectly estimate the noise level. On the other hand, our method provided more accurate results for all of case.
To conclude, we provide a ~
Read slide.
This is end of my presentation. Thank you for listening
This is end of our presentation. THANK YOU FOR LISTENING.
