Demo

750 views
700 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
750
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Demo

  1. 1. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia ThinBlinDe: Blind deconvolution for confocal microscopy Praveen Pankajakshan, Laure Blanc-Féraud, Josiane Zerubia. Ariana joint research group, INRIA/CNRS/UNS, Sophia Antipolis, France. This research work was partially funded by P2R Franco-Israeli project and ANR DIAMOND project. Access to the Applied optics paper is here. October 2010 P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 1 of 42
  2. 2. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Road map of presentation 1 Introduction Optical sectioning fluorescence microscopy Limitations of imaging 2 ThinBlinDe software for deconvolution Breaking the limit-computationally Blind deconvolution 3 Results Phantom data Microspheres Plant samples P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 2 of 42
  3. 3. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Holy grail of biological imaging The holy grail for most cell biologists E. Betzig et al. ‘Imaging intracellular fluorescent proteins at nanometer resolution,’ Sc. Exp., Aug. 2006. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 3 of 42
  4. 4. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Holy grail of biological imaging The holy grail for most cell biologists ◮ dynamically image living cells noninvasively, E. Betzig et al. ‘Imaging intracellular fluorescent proteins at nanometer resolution,’ Sc. Exp., Aug. 2006. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 3 of 42
  5. 5. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Holy grail of biological imaging The holy grail for most cell biologists ◮ dynamically image living cells noninvasively, ◮ to see at the level of an individual protein molecule, and E. Betzig et al. ‘Imaging intracellular fluorescent proteins at nanometer resolution,’ Sc. Exp., Aug. 2006. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 3 of 42
  6. 6. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Holy grail of biological imaging The holy grail for most cell biologists ◮ dynamically image living cells noninvasively, ◮ to see at the level of an individual protein molecule, and ◮ to see how these molecules interact in a cell. E. Betzig et al. ‘Imaging intracellular fluorescent proteins at nanometer resolution,’ Sc. Exp., Aug. 2006. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 3 of 42
  7. 7. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Holy grail of biological imaging The holy grail for most cell biologists ◮ dynamically image living cells noninvasively, ◮ to see at the level of an individual protein molecule, and ◮ to see how these molecules interact in a cell. Eric Betzig (2006) says, “(...) It’s many years away, but you can dream about it.” E. Betzig et al. ‘Imaging intracellular fluorescent proteins at nanometer resolution,’ Sc. Exp., Aug. 2006. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 3 of 42
  8. 8. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Confocal laser scanning microscope Figure 1: Schematic of a confocal laser scanning microscope (Minsky (1988)). A- Laser B- Beam expander C- Dichroic mirror D- Objective E- In-focus plane F- Confocal pinhole G- Photomultiplier tube (PMT) Minsky (1988) ‘Memoir on Inventing the Confocal Scanning Microscope’. Scanning (10): 128 − 138. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 4 of 42
  9. 9. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples 3D imaging by optical sectioning P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 5 of 42
  10. 10. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Point-spread function Figure 2: Experimental PSF determination by imaging point sources. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 6 of 42
  11. 11. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Point-spread function Figure 2: Experimental PSF determination by imaging point sources. Figure 3: 2D Airy disk function. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 6 of 42
  12. 12. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Illustration: Diffraction effect Figure 4: Single GFP if imaged using a widefield microscope. (NA = 1.4, λex = 500nm) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 7 of 42
  13. 13. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Illustration: Effect of circular pinhole Airy Unit (AU); 1AU= 1.22λex/NA P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 8 of 42
  14. 14. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Illustration: Effect of circular pinhole 1AU Airy Unit (AU); 1AU= 1.22λex/NA P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 8 of 42
  15. 15. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Illustration: Effect of circular pinhole 1AU 3AU Airy Unit (AU); 1AU= 1.22λex/NA P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 8 of 42
  16. 16. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Illustration: Effect of circular pinhole 1AU 3AU Fully open Figure 5: Effect of changing confocal pinhole size on the out-of-focus fluorescence, contrast and SNR. Airy Unit (AU); 1AU= 1.22λex/NA P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 8 of 42
  17. 17. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples "The triangle of trade-off" P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 9 of 42
  18. 18. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Illustration: Spherical aberration Figure 6: Schematic of spherical aberration. θi is the azimuthal angle in the lens immersion medium, θs is the angle in the specimen and d (NFP) is the depth under the cover slip and into the specimen. The refractive index of the cover slip is assumed to be either equal to the index of the lens ni or the specimen ns. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 10 of 42
  19. 19. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Illustration: Spherical aberration Figure 7: Spherical aberration causing different focus for paraxial and non-paraxial rays. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 11 of 42
  20. 20. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples The eternal question Figure 8: How to get the best of your microscope? P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 12 of 42
  21. 21. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Problem statement Images from fluorescent optical sectioning microscopes (OSM) are affected by P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 13 of 42
  22. 22. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Problem statement Images from fluorescent optical sectioning microscopes (OSM) are affected by ◮ blurring, due to diffraction limits and aberrations, P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 13 of 42
  23. 23. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Problem statement Images from fluorescent optical sectioning microscopes (OSM) are affected by ◮ blurring, due to diffraction limits and aberrations, ◮ when pinhole size is 1 airy units (AU), 30% of photons collected are from out-of-focus sections. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 13 of 42
  24. 24. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Problem statement Images from fluorescent optical sectioning microscopes (OSM) are affected by ◮ blurring, due to diffraction limits and aberrations, ◮ when pinhole size is 1 airy units (AU), 30% of photons collected are from out-of-focus sections. ◮ shot noise, occurring as detectable statistical fluctuations, P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 13 of 42
  25. 25. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Problem statement Images from fluorescent optical sectioning microscopes (OSM) are affected by ◮ blurring, due to diffraction limits and aberrations, ◮ when pinhole size is 1 airy units (AU), 30% of photons collected are from out-of-focus sections. ◮ shot noise, occurring as detectable statistical fluctuations, ◮ spherical aberrations due to mismatch in medium, P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 13 of 42
  26. 26. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Problem statement Images from fluorescent optical sectioning microscopes (OSM) are affected by ◮ blurring, due to diffraction limits and aberrations, ◮ when pinhole size is 1 airy units (AU), 30% of photons collected are from out-of-focus sections. ◮ shot noise, occurring as detectable statistical fluctuations, ◮ spherical aberrations due to mismatch in medium, ◮ exact point-spread function (PSF) is unknown and varies with experiment. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 13 of 42
  27. 27. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Problem statement Images from fluorescent optical sectioning microscopes (OSM) are affected by ◮ blurring, due to diffraction limits and aberrations, ◮ when pinhole size is 1 airy units (AU), 30% of photons collected are from out-of-focus sections. ◮ shot noise, occurring as detectable statistical fluctuations, ◮ spherical aberrations due to mismatch in medium, ◮ exact point-spread function (PSF) is unknown and varies with experiment. Often only a single observation of the object is given (to avoid photobleaching) for restoration! P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 13 of 42
  28. 28. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Breaking the diffraction barrier Ernst Karl Abbe [1840-1905] S. Hell et al. (1994) ‘Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy’. OL 19 (11): 780 − 782. M. J. Rust et al. (2006) ‘Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM)’. NM 3 (10): 793 − 796. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 14 of 42
  29. 29. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Breaking the diffraction barrier Ernst Karl Abbe [1840-1905] Abbe’s diffraction limit approximation: d = λex 2×no×sinθmax S. Hell et al. (1994) ‘Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy’. OL 19 (11): 780 − 782. M. J. Rust et al. (2006) ‘Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM)’. NM 3 (10): 793 − 796. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 14 of 42
  30. 30. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Breaking the diffraction barrier Ernst Karl Abbe [1840-1905] Abbe’s diffraction limit approximation: d = λex 2×no×sinθmax ◮ Recent fluorescence light microscopy techniques are aimed at surpassing the Abbe resolution limit (Hell, et al.(1994), Betzig, et al. (2006), Rust, et al.(2006)). S. Hell et al. (1994) ‘Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy’. OL 19 (11): 780 − 782. M. J. Rust et al. (2006) ‘Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM)’. NM 3 (10): 793 − 796. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 14 of 42
  31. 31. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Breaking the diffraction barrier Ernst Karl Abbe [1840-1905] Abbe’s diffraction limit approximation: d = λex 2×no×sinθmax ◮ Recent fluorescence light microscopy techniques are aimed at surpassing the Abbe resolution limit (Hell, et al.(1994), Betzig, et al. (2006), Rust, et al.(2006)). ◮ Resulting images are quite similar and differences in the operational details can make these techniques more or less suitable for specific types of biological studies. S. Hell et al. (1994) ‘Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy’. OL 19 (11): 780 − 782. M. J. Rust et al. (2006) ‘Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM)’. NM 3 (10): 793 − 796. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 14 of 42
  32. 32. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Imaging Model Image formation statistics is Poissonian, and γi(x) = P(γ(h ∗o +b)(x)),∀x ∈ Ωs ◮ where ∗ denotes 3D deconvolution, ◮ O(Ωs = {o = oxyz : Ωs ⊂ N3 → R}), P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 15 of 42
  33. 33. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Imaging Model Image formation statistics is Poissonian, and γi(x) = P(γ(h ∗o +b)(x)),∀x ∈ Ωs ◮ where ∗ denotes 3D deconvolution, ◮ O(Ωs = {o = oxyz : Ωs ⊂ N3 → R}), ◮ h : Ωs → R models the PSF, P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 15 of 42
  34. 34. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Imaging Model Image formation statistics is Poissonian, and γi(x) = P(γ(h ∗o +b)(x)),∀x ∈ Ωs ◮ where ∗ denotes 3D deconvolution, ◮ O(Ωs = {o = oxyz : Ωs ⊂ N3 → R}), ◮ h : Ωs → R models the PSF, ◮ b is the low frequency background fluorescence signal, P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 15 of 42
  35. 35. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Imaging Model Image formation statistics is Poissonian, and γi(x) = P(γ(h ∗o +b)(x)),∀x ∈ Ωs ◮ where ∗ denotes 3D deconvolution, ◮ O(Ωs = {o = oxyz : Ωs ⊂ N3 → R}), ◮ h : Ωs → R models the PSF, ◮ b is the low frequency background fluorescence signal, ◮ 1/γ is the photon conversion factor, and γi(x) is the observed photon at the detector. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 15 of 42
  36. 36. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Trends: Deconvolution and CLSM Jan 2004 Jun 2006 Jan 2009 10 20 30 40 50 60 70 80 90 100 Time Normalizednumberofsearches Confocal Microscopy Deconvolution Figure 9: Annual search trends for CLSM and deconvolution between 2004-2009 (Source: Google). 0 20 40 60 80 100 10 20 30 40 50 60 70 80 90 Relative number of searches for "confocal microscopy" Relativenumberofsearchesfor"deconvolution" Figure 10: Scatter plot between the keyword hits “deconvolution” and “confocal microscopy” (Source: Google). Dotted line is the line of LS fit. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 16 of 42
  37. 37. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Trends: scientific interest 1970 1975 1980 1985 1990 1995 2000 2005 0 1 2 3 4 5 6 7 8 9 10 Year NormalizedReferenceVolume Deconvolution Blind Deconvolution Figure 11: Recent scientific trends on deconvolution and blind deconvolution (Data source: ISI web of knowledge). P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 17 of 42
  38. 38. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Breaking the limitations-ThinBlinDe software Figure 12: Observed volume section of a convallaria and restoration using the ThinBlinDe software. c INRA Ariana-INRIA/CNRS/UNS P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 18 of 42
  39. 39. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Deconvolution as Bayesian inference ◮ From the Bayes’ theorem, the posterior probability is Pr(o|i) ∝ Pr(i|o)Pr(o) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 19 of 42
  40. 40. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Deconvolution as Bayesian inference ◮ From the Bayes’ theorem, the posterior probability is Pr(o|i) ∝ Pr(i|o)Pr(o) Pr(i|o) is the likelihood, Pr(o) is the belief of the object. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 19 of 42
  41. 41. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Deconvolution as Bayesian inference ◮ From the Bayes’ theorem, the posterior probability is Pr(o|i) ∝ Pr(i|o)Pr(o) Pr(i|o) is the likelihood, Pr(o) is the belief of the object. ◮ The equivalent energy function is J (o|i) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 19 of 42
  42. 42. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Deconvolution as Bayesian inference ◮ From the Bayes’ theorem, the posterior probability is Pr(o|i) ∝ Pr(i|o)Pr(o) Pr(i|o) is the likelihood, Pr(o) is the belief of the object. ◮ The equivalent energy function is J (o|i) = −log(Pr(o|i)) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 19 of 42
  43. 43. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Deconvolution as Bayesian inference ◮ From the Bayes’ theorem, the posterior probability is Pr(o|i) ∝ Pr(i|o)Pr(o) Pr(i|o) is the likelihood, Pr(o) is the belief of the object. ◮ The equivalent energy function is J (o|i) = −log(Pr(o|i)) = Jobs(i|o)+Jreg(o) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 19 of 42
  44. 44. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Deconvolution as Bayesian inference ◮ From the Bayes’ theorem, the posterior probability is Pr(o|i) ∝ Pr(i|o)Pr(o) Pr(i|o) is the likelihood, Pr(o) is the belief of the object. ◮ The equivalent energy function is J (o|i) = −log(Pr(o|i)) = Jobs(i|o)+Jreg(o) ◮ Maximum a posteriori (MAP) estimate is obtained by P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 19 of 42
  45. 45. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Deconvolution as Bayesian inference ◮ From the Bayes’ theorem, the posterior probability is Pr(o|i) ∝ Pr(i|o)Pr(o) Pr(i|o) is the likelihood, Pr(o) is the belief of the object. ◮ The equivalent energy function is J (o|i) = −log(Pr(o|i)) = Jobs(i|o)+Jreg(o) ◮ Maximum a posteriori (MAP) estimate is obtained by ˆo(x) = arg max o(x)≥0 Pr(o|i) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 19 of 42
  46. 46. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Deconvolution as Bayesian inference ◮ From the Bayes’ theorem, the posterior probability is Pr(o|i) ∝ Pr(i|o)Pr(o) Pr(i|o) is the likelihood, Pr(o) is the belief of the object. ◮ The equivalent energy function is J (o|i) = −log(Pr(o|i)) = Jobs(i|o)+Jreg(o) ◮ Maximum a posteriori (MAP) estimate is obtained by ˆo(x) = arg max o(x)≥0 Pr(o|i) = arg min o(x)≥0 (−log(Pr(o|i))) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 19 of 42
  47. 47. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples State of the art: deconvolution Assumption Method References No Noise Nearest neighbors [Agard 84], Inverse filter [Erhardt et al. 85] Additive Gaussian Reg. lin. least square [Preza et al. 92], white noise Wiener filter [Tommasi et al. 93], JVC [Agard 84, Carrington et al. 90], APEX [Carasso 99], MAP estimate [Levin et al. 09] Poisson noise ML estimate [Holmes 88], MAP estimate [Verveer et al. 99, Dey et al. 06, Pankajakshan et al. 08] P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 20 of 42
  48. 48. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Maximum likelihood object estimation ◮ Likelihood of the observation, i(x), knowing the specimen o(x) is given as P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 21 of 42
  49. 49. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Maximum likelihood object estimation ◮ Likelihood of the observation, i(x), knowing the specimen o(x) is given as Pr(i|o) ∼ (h ∗o +b)(x)i(x) exp(−(h ∗o +b)(x)) i(x)! P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 21 of 42
  50. 50. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Maximum likelihood object estimation ◮ Likelihood of the observation, i(x), knowing the specimen o(x) is given as Pr(i|o) ∼ (h ∗o +b)(x)i(x) exp(−(h ∗o +b)(x)) i(x)! ◮ Find ˆo(x) such that ˆo(x) = arg max o(x)≥0 Pr(i|o) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 21 of 42
  51. 51. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Maximum likelihood object estimation ◮ Likelihood of the observation, i(x), knowing the specimen o(x) is given as Pr(i|o) ∼ (h ∗o +b)(x)i(x) exp(−(h ∗o +b)(x)) i(x)! ◮ Find ˆo(x) such that ˆo(x) = arg max o(x)≥0 Pr(i|o) • Approach: Maximum likelihood expectation maximization (MLEM) algorithm [Richardson 72, Lucy 74, Dempster 77, Celeux 85]. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 21 of 42
  52. 52. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Maximum likelihood object estimation ◮ Likelihood of the observation, i(x), knowing the specimen o(x) is given as Pr(i|o) ∼ (h ∗o +b)(x)i(x) exp(−(h ∗o +b)(x)) i(x)! ◮ Find ˆo(x) such that ˆo(x) = arg max o(x)≥0 Pr(i|o) • Approach: Maximum likelihood expectation maximization (MLEM) algorithm [Richardson 72, Lucy 74, Dempster 77, Celeux 85]. • Assumption: h(x) is available! P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 21 of 42
  53. 53. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Prior as statistical information Gibbsian distribution with total variation (TV) fuction is used as prior on the object [Demoment 1989] Figure 13: Markov random field over a six member neighborhood ηx for a voxel site x ∈ Ωs. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 22 of 42
  54. 54. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Prior as statistical information Gibbsian distribution with total variation (TV) fuction is used as prior on the object [Demoment 1989] Figure 13: Markov random field over a six member neighborhood ηx for a voxel site x ∈ Ωs. Pr(o) = Z−1 λo exp(−λo x∈Ωs |∇o(x)|), P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 22 of 42
  55. 55. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Prior as statistical information Gibbsian distribution with total variation (TV) fuction is used as prior on the object [Demoment 1989] Figure 13: Markov random field over a six member neighborhood ηx for a voxel site x ∈ Ωs. Pr(o) = Z−1 λo exp(−λo x∈Ωs |∇o(x)|), ◮ Zλo = o∈O(Ωs) exp(−λo x∈Ωs |∇o(x)|) is the partition function, P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 22 of 42
  56. 56. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Prior as statistical information Gibbsian distribution with total variation (TV) fuction is used as prior on the object [Demoment 1989] Figure 13: Markov random field over a six member neighborhood ηx for a voxel site x ∈ Ωs. Pr(o) = Z−1 λo exp(−λo x∈Ωs |∇o(x)|), ◮ Zλo = o∈O(Ωs) exp(−λo x∈Ωs |∇o(x)|) is the partition function, ◮ with λo ≥ 0. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 22 of 42
  57. 57. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Maximum a posteriori object estimate ◮ Combining the likelihood and the prior terms Pr(o,h|i) ∝ Pr(i|o,h)Pr(o) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 23 of 42
  58. 58. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Maximum a posteriori object estimate ◮ Combining the likelihood and the prior terms Pr(o,h|i) ∝ Pr(i|o,h)Pr(o) ◮ this posterior probability can be written as Pr(o,h|i) ∝ x∈Ωs ((h ∗o +b)(x))i(x) exp(−(h ∗o)(x)) i(x)! × exp(−λo x∈Ωs |∇o(x)|) o exp(−λo x∈Ωs |∇o(x)|) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 23 of 42
  59. 59. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Gradient vector field X Y 50 100 150 200 250 Lateral direction P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 24 of 42
  60. 60. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Gradient vector field X Y 50 100 150 200 250 X Z 50 100 150 200 250 Lateral direction Axial direction P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 24 of 42
  61. 61. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Gradient vector field X Y 50 100 150 200 250 X Z 50 100 150 200 250 Lateral direction Axial direction Figure 14: The gradient vector field is overlapped over a synthetic object. The field flow is more prominent along the borders and less along the homogeneous interiors. The effect of the noise is negligible. c Ariana-INRIA/CNRS/UNS. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 24 of 42
  62. 62. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Object prior implications −150 −100 −50 0 50 100 150 −25 −20 −15 −10 −5 0 Gradient ∇xi(x) log2probabilitymassfunction 1AU pinhole 2 AU pinhole 5 AU pinhole 10 AU pinhole Figure 15: Distribution of the gradient for an observed Arabidopsis Thaliana plant under different pinhole sizes. As the pinhole size decreases from 10AU to 1AU, the gradient distribution has a longer tail. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 25 of 42
  63. 63. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Incoherent scalar PSF model P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 26 of 42
  64. 64. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Incoherent scalar PSF model ◮ For the case when the acquisition parameters of the experiments are exactly known, deconvolution can be achieved by using theoretically calculated PSFs. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 26 of 42
  65. 65. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Incoherent scalar PSF model ◮ For the case when the acquisition parameters of the experiments are exactly known, deconvolution can be achieved by using theoretically calculated PSFs. ◮ Computation of the incoherent PSF can be reduced to 2Nz 2D Fourier transform of the pupil function hTh(x;λex,λem) = C|hA(x;λex )|×|AR(x,y)∗hA(x,y,z;λem )| P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 26 of 42
  66. 66. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Incoherent scalar PSF model ◮ For the case when the acquisition parameters of the experiments are exactly known, deconvolution can be achieved by using theoretically calculated PSFs. ◮ Computation of the incoherent PSF can be reduced to 2Nz 2D Fourier transform of the pupil function hTh(x;λex,λem) = C|hA(x;λex )|×|AR(x,y)∗hA(x,y,z;λem )| ◮ If P(kx,ky,z) is the 2D complex pupil function and λ is the wavelength, the amplitude PSF is hA(x,y,z;λ) = kx ky P(kx,ky,z)exp(j(kx x +kyy))dkydkx P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 26 of 42
  67. 67. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Numerically computed PSF Figure 16: Numerically calculated PSF for a C-Apochromat water immersion objective. NA 1.2, 63× magnification. c Ariana-INRIA/CNRS/UNS. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 27 of 42
  68. 68. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples State of the art: Blind deconvolution Method References A priori PSF [Boutet de Monvel 2001], identification Marginalization [Jalobeanu 2007, Levin et al. 2009], Joint maximum likelihood [Holmes 1992, Michailovich & Adam 2007], Parametric blind deconvolution [Markham & Conchello 1999]. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 28 of 42
  69. 69. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Blind deconvolution by alternate minimization ◮ When the imaging parameters are not known, it is necessary to estimate o(x) and h(x) from i(x) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 29 of 42
  70. 70. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Blind deconvolution by alternate minimization ◮ When the imaging parameters are not known, it is necessary to estimate o(x) and h(x) from i(x) ◮ Minimizing the cost function w.r.t o(x) ˆo(x) = arg min o(x)≥0 J (o(x)|λo,h(x)) = arg min o(x)≥0 Jobs(i(x)|o(x),h(x))+Jreg(o(x)|λo) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 29 of 42
  71. 71. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Blind deconvolution by alternate minimization ◮ When the imaging parameters are not known, it is necessary to estimate o(x) and h(x) from i(x) ◮ Minimizing the cost function w.r.t o(x) ˆo(x) = arg min o(x)≥0 J (o(x)|λo,h(x)) = arg min o(x)≥0 Jobs(i(x)|o(x),h(x))+Jreg(o(x)|λo) ◮ Minimizing the cost function w.r.t h(x) ˆh(x) = arg min h(x)≥0 J (h(x)|λo,o) = arg min h(x)≥0 Jobs(i(x)|o,h(x))+Jreg(h(x)) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 29 of 42
  72. 72. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Blind deconvolution by PSF parametrization P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 30 of 42
  73. 73. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Blind deconvolution by PSF parametrization ◮ The estimation of the complete PSF from the observation is a difficult problem as the number of unknowns are large. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 30 of 42
  74. 74. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Blind deconvolution by PSF parametrization ◮ The estimation of the complete PSF from the observation is a difficult problem as the number of unknowns are large. ◮ Since the CLSM PSF is theoretically the Bessel function raised to the fourth power, an approximation in the spatial domain is a 3D Gaussian approximation, P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 30 of 42
  75. 75. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Blind deconvolution by PSF parametrization ◮ The estimation of the complete PSF from the observation is a difficult problem as the number of unknowns are large. ◮ Since the CLSM PSF is theoretically the Bessel function raised to the fourth power, an approximation in the spatial domain is a 3D Gaussian approximation, ◮ Diffraction-limited PSF in the LSQ sense (up to 3AU pinhole diameter) h(x;ωh) = (2π)− 3 2 |Σ|− 1 2 exp(− 1 2 (x−µ)T Σ−1 (x−µ)) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 30 of 42
  76. 76. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Blind deconvolution by PSF parametrization ◮ The estimation of the complete PSF from the observation is a difficult problem as the number of unknowns are large. ◮ Since the CLSM PSF is theoretically the Bessel function raised to the fourth power, an approximation in the spatial domain is a 3D Gaussian approximation, ◮ Diffraction-limited PSF in the LSQ sense (up to 3AU pinhole diameter) h(x;ωh) = (2π)− 3 2 |Σ|− 1 2 exp(− 1 2 (x−µ)T Σ−1 (x−µ)) ◮ BD is reduced to estimation of spatial parameters, P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 30 of 42
  77. 77. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Blind deconvolution by PSF parametrization ◮ The estimation of the complete PSF from the observation is a difficult problem as the number of unknowns are large. ◮ Since the CLSM PSF is theoretically the Bessel function raised to the fourth power, an approximation in the spatial domain is a 3D Gaussian approximation, ◮ Diffraction-limited PSF in the LSQ sense (up to 3AU pinhole diameter) h(x;ωh) = (2π)− 3 2 |Σ|− 1 2 exp(− 1 2 (x−µ)T Σ−1 (x−µ)) ◮ BD is reduced to estimation of spatial parameters, ◮ advantages: few parameters and no DFT necessary. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 30 of 42
  78. 78. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Why 3D Gaussian? ◮ Properties of the PSF P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 31 of 42
  79. 79. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Why 3D Gaussian? ◮ Properties of the PSF • positivity: h(x) ≥ 0, ∀x ∈ Ωs P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 31 of 42
  80. 80. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Why 3D Gaussian? ◮ Properties of the PSF • positivity: h(x) ≥ 0, ∀x ∈ Ωs • circular symmetry: h(x,y,z) = h(−x,−y,z) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 31 of 42
  81. 81. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Why 3D Gaussian? ◮ Properties of the PSF • positivity: h(x) ≥ 0, ∀x ∈ Ωs • circular symmetry: h(x,y,z) = h(−x,−y,z) • normalization: x∈Ωs h(x) = 1 P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 31 of 42
  82. 82. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Why 3D Gaussian? ◮ Properties of the PSF • positivity: h(x) ≥ 0, ∀x ∈ Ωs • circular symmetry: h(x,y,z) = h(−x,−y,z) • normalization: x∈Ωs h(x) = 1 ◮ CLSM PSF is theoretically the Bessel function raised to the fourth power. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 31 of 42
  83. 83. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Why 3D Gaussian? ◮ Properties of the PSF • positivity: h(x) ≥ 0, ∀x ∈ Ωs • circular symmetry: h(x,y,z) = h(−x,−y,z) • normalization: x∈Ωs h(x) = 1 ◮ CLSM PSF is theoretically the Bessel function raised to the fourth power. J 0 2 (x) J 0 4 x J0 x x K15 K10 K5 0 5 10 15 K0.5 0.5 1.0 Figure 17: The Bessel function raised to the fourth power loses its side lobes. c Ariana- INRIA/CNRS/UNS. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 31 of 42
  84. 84. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Blind deconvolution by PSF parametrization P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 32 of 42
  85. 85. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Blind deconvolution by PSF parametrization ◮ In the presence of aberrations, spatial approximation leads to a large number of parameters, P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 32 of 42
  86. 86. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Blind deconvolution by PSF parametrization ◮ In the presence of aberrations, spatial approximation leads to a large number of parameters, ◮ a way of handling it is estimating the parameters of the pupil function ωh = d,ni ,ns Pa(λ;x) = Pd(λ;x)exp(jϕ(d,ni ,ns)) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 32 of 42
  87. 87. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Blind deconvolution by PSF parametrization ◮ In the presence of aberrations, spatial approximation leads to a large number of parameters, ◮ a way of handling it is estimating the parameters of the pupil function ωh = d,ni ,ns Pa(λ;x) = Pd(λ;x)exp(jϕ(d,ni ,ns)) ◮ regularization of the PSF is achieved by using a functional form of phase from geometrical optics ϕ(d,ni ,ns) = d(ni cosθi −ns cosθs) ≈ d∆ns secθs P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 32 of 42
  88. 88. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples PSF estimation P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 33 of 42
  89. 89. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples PSF estimation ◮ Deconvolution requires the PSF ˆh(x) or h(x, ˆωh) ωh,MAP = argmax ωh >0 Pr(i(x)|ˆo(x),ωh)Pr(ωh) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 33 of 42
  90. 90. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples PSF estimation ◮ Deconvolution requires the PSF ˆh(x) or h(x, ˆωh) ωh,MAP = argmax ωh >0 Pr(i(x)|ˆo(x),ωh)Pr(ωh) ◮ Pr(ωh) is the parameter prior. We assume the parameters to be uniformly distributed in a set: [ωh,LB,ωh,UB], P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 33 of 42
  91. 91. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples PSF estimation ◮ Deconvolution requires the PSF ˆh(x) or h(x, ˆωh) ωh,MAP = argmax ωh >0 Pr(i(x)|ˆo(x),ωh)Pr(ωh) ◮ Pr(ωh) is the parameter prior. We assume the parameters to be uniformly distributed in a set: [ωh,LB,ωh,UB], ◮ and the cost function is J (o,h(ωh)) = x i(x)log((h(ωh)∗o +b)(x))− x (h(ωh)∗o +b)(x) P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 33 of 42
  92. 92. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples PSF estimation ◮ Deconvolution requires the PSF ˆh(x) or h(x, ˆωh) ωh,MAP = argmax ωh >0 Pr(i(x)|ˆo(x),ωh)Pr(ωh) ◮ Pr(ωh) is the parameter prior. We assume the parameters to be uniformly distributed in a set: [ωh,LB,ωh,UB], ◮ and the cost function is J (o,h(ωh)) = x i(x)log((h(ωh)∗o +b)(x))− x (h(ωh)∗o +b)(x) ◮ The cost function, J (o,h(ωh)), could be minimized by using a gradient-descent type algorithm. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 33 of 42
  93. 93. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Simulation results Phantom object Observation γ = 100, ∆xy = 25nm ∆z = 50nm P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 34 of 42
  94. 94. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Simulation results Sans regularization (Naive MLEM) Proposed approach P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 35 of 42
  95. 95. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Microsphere deconvolution Figure 18: A 15µm microsphere has a thin layer of fluorescence of thickness 500− 700nm. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 36 of 42
  96. 96. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Microsphere deconvolution Observed section BD restored (535.58 nm) Figure 19: Measured thickness of a microsphere, after using our approach, was 535.58nm, and was 260nm with the naive MLEM algorithm. c Ariana-INRIA/CNRS/UNS.P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 37 of 42
  97. 97. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Deconvolution on plant samples Figure 20: Observed section of arabidopsis thaliana in water and observed with a LSM 510 microscope and pinhole 2AU. Lateral sampling is 285.64nm and axial sampling is 845.62nm. c INRA. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 38 of 42
  98. 98. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Deconvolution on plant samples Figure 21: Restoration sans regularization (Naive MLEM). c Ariana-INRIA/CNRS/UNS. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 39 of 42
  99. 99. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Deconvolution on plant samples Figure 22: Restoration using the proposed approach after 10 iterations. c Ariana-INRIA/CNRS/UNS. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 40 of 42
  100. 100. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Deconvolution on plant samples Figure 23: Comparison of observed and restored lateral section of convallaria majalis. Lateral sampling is 53.6nm and axial sampling is 129.4nm. The sample was observed with a Zeiss LSM 510TMmicroscope, and 1.3NA oil immersion objective. c INRA, Ariana-INRIA/CNRS/UNS. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 41 of 42
  101. 101. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Conclusions and discussions P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 42 of 42
  102. 102. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Conclusions and discussions ◮ Developed Bayesian framework for blind deconvolution for thin specimens, P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 42 of 42
  103. 103. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Conclusions and discussions ◮ Developed Bayesian framework for blind deconvolution for thin specimens, ◮ experiments on simulated data, microsphere images and real data show promising results, P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 42 of 42
  104. 104. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Conclusions and discussions ◮ Developed Bayesian framework for blind deconvolution for thin specimens, ◮ experiments on simulated data, microsphere images and real data show promising results, ◮ PSF model chosen initially is a 3D separable Gaussian function for thin specimens, P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 42 of 42
  105. 105. ThinBlinDe P. Panka- jakshan, L. Blanc- Féraud, J. Zerubia Road map Introduction OSM Limitations ThinBlinDe Break the limit Blind deconvolution Results Phantom data Microspheres Plant samples Conclusions and discussions ◮ Developed Bayesian framework for blind deconvolution for thin specimens, ◮ experiments on simulated data, microsphere images and real data show promising results, ◮ PSF model chosen initially is a 3D separable Gaussian function for thin specimens, ◮ Total variation regularization functional sometimes leads to loss in contrast in restoration but there are methods to overcome this. P. Pankajakshan, L. Blanc-Féraud, J. Zerubia October 26, 2010 page 42 of 42

×