【标题】 生物医学中的光学成像仪器及技术介绍 【内容】 生物医学成像仪器在基础生物学和医学研究方面发挥着重要作用,在这其中荧光成像技术和激光技术在诸如神经网络,阿兹海默症抑或是癌症机理的探索过程中不可或缺. 2014 年超分辨率荧光成像技术获得诺贝尔化学奖，2018年脉冲激光技术获得诺贝尔物理学奖, 可见一斑。 本次讲座将分别从光学切片能力,超分辨率技术以及高速成像三个维度介绍“结构光时空聚焦技术”,一种全新的荧光成像技术。其中将从基本的衍射光学，时间和空间的傅立叶变换出发, 伴随当下脉冲激光技术讲解诸如MEMS等器件如何对光进行时间和空间的调制,从而理解新技术的实现和瓶颈. 【时间】6月23号周日上午9h30--13h00 【地点】东京大学农学部FS 7号馆102教室 food sciene地下一楼

1. Study on new mechanisms for improving
performance of two-photon excitation
fluorescence microscopy using temporal focusing
Song, Qiyuan
Kannari Lab, Keio University
Midorikawa Lab, Riken Center for Advanced Photonics
23/06/2019

2. 











2

3. Wide-field Microscopy Confocal Microscopy
共聚焦显微镜
Example: 2D Image of lily pollen grain
Microscope is focused in a mid depth plane
Without sectioning ability, out-of-focus blurred image is remained 3

4. 

4

5. 

z
x
𝑒𝑖
Incoherent imaging case
5

6. k1
-k2
OTFzx PSFzx
 Image spatial frequency is a differential combination of any two possible
emission electric field vectors: 𝑒𝑖 𝑒𝑗
∗
-k3
-k4
-k5
∆𝑘 𝑥
∆𝑘 𝑧
𝐻 𝐻
6
ki:
any possible
emission
Wavevector
ki-kj:
image intensity
spatial frequency
𝑒𝑖 = exp −1𝑘𝑖 𝑟

7.  Definition of sectioning ability
光学切片能力： 从一本书里读一页纸
纸片厚度
𝐼 𝑧 = 𝐻⨂𝛿 𝑧 = 𝐻𝑑𝑥𝑑𝑦; wide-field imaging, 𝐼 𝑧 =constant
⨂ 卷积
In Fourier domain
𝐼 𝑘 𝑧 = 𝐻𝑒−𝑖0𝑥
𝑒−𝑖0𝑦
𝑒−𝑖𝑘 𝑧 𝑧
𝑑𝑥𝑑𝑦𝑑𝑧 = 𝐻 0,0, 𝑘 𝑧 ; wide-field imaging, 𝐼 𝑘 𝑧 = δ(𝑘 𝑧)
It is easy to check whether there is optical sectioning ability by checking 3D OTF
missing cone
7

8. 
x
y
Microscopy
Object: Infinite thin layer
3D image: a bulk image
x
yz
3

9. 𝐼𝑐𝑜𝑛𝑓𝑜𝑐𝑎𝑙 𝑧
= 𝐻𝑐𝑜𝑛𝑓𝑜𝑐𝑎𝑙⨂𝛿 𝑧
= 𝐻2 𝑑𝑥𝑑𝑦
=
1
1 + (𝑧/𝑧 𝑅)2
~|𝑧|−2
𝐻𝑐𝑜𝑛𝑓𝑜𝑐𝑎𝑙 = 𝐻2
Thickness of page image
PSF of confocal
microscopy:
Confocal microscopy is an example for optical sectioning microscopy
8
In 1957, Marvin Minsky invented confocal microscopy

10. Wide-field microscopy OTF Confocal microscopy OTFSelf convolution
The missing cone problem of 3D OTF is solved in confocal microscopy
9

11. 𝐻2𝑝ℎ = 𝐻2
𝐼2𝑝ℎ 𝑧
= 𝐻2 𝑑𝑥𝑑𝑦
=
1
1 + (𝑧/𝑧 𝑅)2
~|𝑧|−2
PSF of TPEF
microscopy:
Thickness of page image
In analogy to confocal microscopy, TPEF microscopy has sectioning ability
10
In 1990, Winfried Denk invented TPEF microscopy

12. 1
2

13. 
Time
Signal
2nd
Line
1st
Line
3rd
Line
4th
Line
Key issue: scanning speed, sampling rate
t
I
11

14. Commercial TPEF Microscopy
∼30fps 512×512pixels
Limitation of Scanning Mirror
Resonant Frequency ∼10kHz
Limitation of sampling rate of
laser source ∼20fps×2k×2k
How fast is TPEF microscopy?
12

15. 



17. 波长整数倍等于腔体长度
的波长都满足激光条件
连续激光器只存在单个波长
脉冲激光器存在很多波长

19. 

Typical values in breast tissue
λ~635 nm:, μs =400cm-1
λ~1,000 nm: μs =50cm-1
(s: scattering)
𝐼 𝑏𝑎𝑙𝑙𝑖𝑠𝑡𝑖𝑐 = 𝐼0exp(−𝜇 𝑠z)

21. 
Frame rate ~KHz - ~MHz
D. Oron et al., Opt. Express, 13, 1468 (2005).
Spatial
Focusing
Temporal
Focusing (TF)
𝐼 𝑇𝑃𝐸 ∝ 1/∆𝑡
13
In 2005, Dan Oron, Guanghao Zhu invented temporal focusing

22. 瞬时频率随时间变化
相位有时间的二次方项或更高阶项
不同颜色的光抵达观测点的时间不同

25. Dispersion – 频谱的位相关系为非线性关系 (位相对频率的多项式展开 含高于二阶以上项)

26. 任何介质都含有分散
物质的折射率对于不同频率不尽相同
例子： 棱镜色散，光纤传输中的色散
脉冲越短 / 频谱越宽
越对分散敏感

27. G. Zhu et al., Optics Express, 13 2153 (2005)
 By using spatial disperser, only at focal
plane the pulse is shortest
 Out-of-focus pulse is chirped
14
光栅

28. Second-Order Phase Modulation
Temporal Focusing—Quadratic Phase
Z=0
Z=Z0
Z=2Z0
Spatial Focusing—Quadratic Wavefront
Z=0 Z=Z0 Z=2Z0
• In order to modulate the pulse duration, we can modulate the
spectrum second order phase.
• If frequency is linear to space coordinate, then we succeed.
x 
G. Zhu et al., Optics
Express, 13 2153
(2005)

29. Li-Chung Cheng et al.,
Opt. Express, 20, 8939 (2012)
200μm×100μm @ 200Hz
Hod Dana et al.,
Nat. Commun., 5, 3997 (2014)
600μm×1000μm×200μm
@ 10 volumes/sec
>1000 developing cells
 Use chirped amplified laser (CPA) with temporal focusing to realize fast TPEF
imaging in TF microsocpy
Brownian motions of 1mm beads
Neuron Networks’ functional imaging
16

30. 时空聚焦显微镜的光学切片能力要比点扫描型差！！！
3.7RZ um
int
2
( )
1
1 ( )
Po Scanning
R
I z
z
z


2
( )
1
1 ( )
TF
R
I z
z
z


Thickness of page image
TF is one-dimensional focusing
system, objective lens is two-
dimensional
TF
17

31. Due to scattering, Strong out-of-focus fluorescence on the surface will decrease
the image contrast in deep imaging. It is important to suppress the out-of-focus
excitation intensity (increasing n).
4
Scattering Excitation
x
z z
Image
Intensity
~1/zn n is important

32. K. H. Kim et al, Opt.
Express, 2007
散射的荧光成为了背景光  限制成像深度
Sample
Scattering light
Ballistic light
Parallel detector
18

33. <50nm lateral resolution, Mats G. L. Gustafsson, PNAS, 2005
Wide-filed microscopy Super-resolution microsocpy
To observe 树突棘 (< 2 mm), 轴突 (0.1 mm – 10 mm), 突触 (20-200 nm), we have
to beak the diffraction limitation ~ 200nm lateral, 500nm axial direction (FWHM of
PSF). 19

34. Measured wavefront

Forward Scattering
M. Schwertner et al, Opt. Express 12, 6540 (2004)
Light field
20

35. 介绍
 光学传递函数，光学切片能力
 时空聚焦技术，高速双光子荧光显微镜
 时空聚焦的缺陷, 可改进之处
二维时空聚焦
 设计想法，理论证明
 增强光学切片能力，实验验证
三维干涉-时空聚焦显微镜
 结构光超分辨显微镜原理
 3维超分辨率，超切片能力验证
 散乱光，背景光去除
三维干涉-时空聚焦显微镜对光学扰乱的耐性
21

36. Pulse front tilting supports self-line scanning
Trainset beam size is comparable to a line focusing microscopy
Pulse front tilting in Temporal focusing
Around focal plane
( ) ( )c     
2
Delay x



( , ) ( , )E t x A t Delay x 
( , ) ( , )exp( ( ))cE x a x ikx     
Fourier Transform
36
15

37. Coupling between Time and space in TF
At Focal Plane Outside Focal Plane
Propagation
(Diffraction)
x
t
FTL
Pulse
Transient
Beam waist
Image
Size
Scanning Time
t
x


x
t
Chirped
Pulse
Broadened
Transient Beam
Image
Size
Scanning Time
t
x
• Fix temporal point at every axial position, it’s spatial focusing
• Fix spatial point at every axial position, it’s temporal focusing

38. 2D tilting of pulse front
Discrete in additional dimension
Transient beam at focal plane
our innovation:
2D Self-Scanning
Scan Line by Line
beam at focal plane
α
Focal Plane
A Pulse in 1D-TF
Transient beam at focal plane
1D-TF:
1D Self-Scanning
Line Scanning
beam at focal plane
Review the case of 1D-TF
α
β
Focal Plane
our innovation: a 2D-tilted
pulse front
23
x
Lateral
direction
y
Lateral
direction
z optical axis
Axial direction

39. 24
x
Lateral
direction
y
Lateral
direction
z optical axis
Axial direction

40. Envelope pulse front is tilted in yz plane to support point scanning
Horizontal Vertical

41. Transient focused beam in y direction enhances the sectioning ability

42. Objective
Lens
2D spatial chirp
Periodic in additional
dimension
750nm-700nm
700nm-650nm
650nm-600nm
600nm-550nm
550nm-500nm
Chirp
Periodic
chirp
Objective
Lens
Chirp
Review the case of
1D-TF
α
Focal
Plane
α
β
Focal
Plane
Our innovation of
2D-TF
25
Period: Free Spectral Range (FSR)

43. Both envelope-pulse and intra-
pulses are manipulated. Thus
focusing power is enhanced.
Scanningless, frame rate as high
as the reparation rate of laser
source, kHz~MHz
26
We first time invent and analyze the 2D-TF,
which will enhance the ‘focusing power’ for TF

44. 44
28

45. Angular
dispersion
θ
Angular Dispersion
Zoom In
Z
X
θ
Wave Front
2 2
( , 0, ) cos( ( ))
( 1 ( ) ) 0.5 ( )
w x z k r kz w
kz w kz kz w
 
 
 
   
   
• Angular dispersion offer us the second order spectrum
phase whose value is proportional to the axial position
( ) ( )cw w w  
27
or Y

46. virtually imaged phased array (VIPA)
FSR=50 GHz
M. Shirasaki, Opt. Lett. 21, 366 (1996)
31

47. 2
2 -
1
( )D TFI z
z
 
  
 
32
Laser source: chirped pulse
amplification (CPA) laser,
50fs, 1 kHz repetition rate
We first time achieve scanning-less
TPEF microscopy with out-of-focus excitation decay
speed of (1/𝑧)2
for TF

48. Setup Axial resolution
comparison
 We first time realize VIPA-based 2D-TF microscopy
 Sectioning resolution is 1.3 mm for objective lens with NA = 1.2 in 2D-TF
compared to 2.2 mm in 1D-TF, 1.7 times improvement by our new 2D-TF idea
 The sectioning resolution of 1.3 mm is world record for scanningless TPEF
microscopy by using TF technology without any post-processing 33

49. Surface
~70 mm
TF 2D-TF
34
Biology phantom: 2-μm diameter yellow green
(505/515) polystyrene fluorescent beads

50. 50 Hz Volume rate, 1kHz frame rate imaging
35
Biology phantom: 1-μm diameter yellow green
(505/515) polystyrene fluorescent beads

51. 


36

52. 介绍
 光学传递函数，光学切片能力
 时空聚焦技术，高速双光子荧光显微镜
 时空聚焦的缺陷, 可改进之处
二维时空聚焦
 设计想法，理论证明
 增强光学切片能力，实验验证
三维干涉-时空聚焦显微镜
 结构光超分辨显微镜原理
 3维超分辨率，超切片能力验证
 散乱光，背景光去除
三维干涉-时空聚焦显微镜对光学扰乱的耐性
37

54. Reference Beat image
High Spatial
Frequency
Low Spatial
Frequency
Mats G. L.
Gustafsson,
J. Microsc. 198,
82 (2000);
Original image
 
Structured illumination
Spatial frequency 𝑘
Optical
Transfer
Function
Image
high spatial
frequency
Interferometry
Spatial frequency
Beat image makes high spatial frequency components
JUMP INSIDE the optical transfer function (OTF)
38
In 1999, Rainer
Heintzmann, Mats G.L.
Gustafsson invented
structured illumination
microscopy for super-
resolution

55. D. Dan, et al. Sci. Rep. 3,1116(2013)
M. A. A. Neil et al. Opt. Lett. 22,1905(1997)
k1 k2
Objective
Lens
∆k=k1-k2
Illumination
Light Field
Excitation Intensity -∆k=k2-k1
DC1=k1-k1 DC2=k2-k2
𝐼𝑒𝑥𝑐 =
1 + cos(∆𝑘𝑥 + 𝜑0)
𝑒 𝑖𝜑0
/2 𝑒−𝑖𝜑0
/2
𝑒 𝑖𝜑0 𝑒−𝑖𝜑0
𝑒 𝑖0 𝑒 𝑖0
39

56. 𝐼 𝑟𝑎𝑤 𝑖𝑚𝑎𝑔𝑒 = 𝐼 𝑠𝑎𝑚𝑝𝑙𝑒 𝐼 𝑒𝑥𝑐 ⨂ℎ
𝐼 𝑟𝑎𝑤 𝑖𝑚𝑎𝑔𝑒 = 𝐼s𝑎𝑚𝑝𝑙𝑒⨂ℎ + 𝐼𝑠𝑎𝑚𝑝𝑙𝑒cos(∆𝑘x + 𝜑0)⨂ℎ
Beat Image
𝐼 𝑟𝑎𝑤 𝑖𝑚𝑎𝑔𝑒 = 𝐼 𝑠𝑎𝑚𝑝𝑙𝑒ℎ +
𝐼 𝑠𝑎𝑚𝑝𝑙𝑒 (𝑘 − ∆𝑘)𝑒 𝑖𝜑0ℎ +
𝐼 𝑠𝑎𝑚𝑝𝑙𝑒(𝑘 + ∆𝑘)𝑒−𝑖𝜑0ℎ
𝐼 𝑆𝐼𝑀𝑖𝑚𝑎𝑔𝑒 = 𝐼 𝑠𝑎𝑚𝑝𝑙𝑒 × {ℎ + ℎ(𝑘 + ∆𝑘) + ℎ(𝑘 − ∆𝑘)}
In Fourier domain
DC Image
0th order
+1st order
Target
Separate each order image
Compensate offset phase
Shift back each order image in spatial frequency domain
-1st order
ℎ: Point Spread Function of Widefield imaging
40

57.  time
Number of
measurement
I
t1 t2 t3
𝜑0
+1st and -1st order component
2𝜋/3 2𝜋/3 2𝜋/3 0th order component
In Fourier domain
j=0th order
j=+1st orderj=-1st order
I1 I2 I3
𝐼𝑗 =
𝑚=1
3
𝐼 𝑚 𝑒−𝑖2𝑗𝑚𝜋/3
41
Number of
measurement

58. 𝐼 𝑆𝐼𝑀𝑖𝑚𝑎𝑔𝑒 = 𝐼 𝑠𝑎𝑚𝑝𝑙𝑒 × {ℎ + ℎ(𝑘 + ∆𝑘) + ℎ(𝑘 − ∆𝑘)}
Effective Optical Transfer Function (OTF)
Effective Point Spread Function (PSF):
ℎ 𝑆𝐼𝑀𝑖𝑚𝑎𝑔𝑒 = ℎ × 𝐼 𝑒𝑥𝑐
Widefield image SIM raw image
SIM raw image Separated images SIM image
SIM image
kx
ky
x
y
R. Heintzmann and M. G. L. Gustafsson, Nat. Photonics 3 362 (2009)
42

59. structured illumination effective Optical Transfer Function
43

60. 44

61. M. G. L. Gustafsson et al, Biophys. J, 94 4957 (2008)
 There is no beat image in z
direction due to z scanning
𝐼 𝑠𝑎𝑚𝑝𝑙𝑒cos(∆𝑘 𝑧 𝑧) ∗ h
 There is no need to shift back
in axial spatial frequency
domain
ℎ 𝑒𝑓𝑓 = ℎ𝐼 𝑒𝑥𝑐
B. Balley et al, Nature, 366 44 (1993)
Standing Wave Fluorescence Microscopy
𝐼 𝑠𝑎𝑚𝑝𝑙𝑒 ∗ {hcos ∆𝑘 𝑧 𝑧 }
k
kz

62. 介绍
 光学传递函数，光学切片能力
 时空聚焦技术，高速双光子荧光显微镜
 时空聚焦的缺陷, 可改进之处
二维时空聚焦
 设计想法，理论证明
 增强光学切片能力，实验验证
三维干涉-时空聚焦显微镜
 结构光超分辨显微镜原理
 3维超分辨率，超切片能力验证
 散乱光，背景光去除
三维干涉-时空聚焦显微镜对光学扰乱的耐性
45

63. Around focal plane
Modulation occurs not only in
lateral direction but also in
axial direction!!!
46We first time invent the idea to use 3D-SIM algorithm for TF
𝜃

64. Missing cone is further filled by axial shift of OTF
Both lateral and axial frequency support is enhanced beyond diffraction
limitation
47

65. 48
We first time invent the
method to implement 3D-
interferometric pattern for TF.
y
z
Measured 3D-
interferometric pattern
CPA fiber laser:
1055 nm,
104 fs,
200 kHz

66. Vertical FringesHorizontal Fringes 45 Degree Fringes 45 Degree Fringes

67. DMD throughput is 54% without pattern, 27% with pattern
Use DMD in TF for patterning
49

68. 0-200 mm depth,
200 nm fluorescent beads
(540/560);
Density: 2.3×1011 beads/mL;
50
First time 3D super-
resolution image in
scanningless TPEF
microscopy

69. TF: axial resolution 820 nm; lateral resolution 300 nm
3D-ITF: axial resolution 660 nm; lateral resolution 180 nm
Spatial resolution improves ~1.6 times in lateral, ~1.2 times in axial direction,
compared to diffraction limited value in TF.
At the first time, 3D super-resolution scanningless TPEF microscopy
51

70. 52
First time 3D super-resolution imaging for scanningless TPEF
microscopy
Imaging depth at（a,b）45 μm and (c, d) 162 μm

71. Searching
modulation
frequency
Searching
offset phase

72. kx
ky
As there is overlapping region in
spatial domain, we can search
modulation frequency from the
image

73. Temporal focusing (TF):1.7 mm
Three-dimensional temporal focusing (3D-ITF): 0.67 mm
Edge of Rhodamine B bulk solution, excitation wavelength 1055 nm
A world record sectioning resolution in scanningless TPEF microscopy

74. 介绍
 光学传递函数，光学切片能力
 时空聚焦技术，高速双光子荧光显微镜
 时空聚焦的缺陷, 可改进之处
二维时空聚焦
 设计想法，理论证明
 增强光学切片能力，实验验证
三维干涉-时空聚焦显微镜
 结构光超分辨显微镜原理
 3维超分辨率，超切片能力验证
 散乱光，背景光去除
三维干涉-时空聚焦显微镜对光学扰乱的耐性
53

75. K. H. Kim et al.,
Opt. Express, 15, 11658 (2007)
The scattered fluorescence becomes background
 limits the imaging depth
Sample
Scattering light
Ballistic light
2D Detector
54

76. Camera
We first time find out that signal and scattered emission is able to be
discriminated by structured illumination in TF 55

77. We first time remove
the scattered
background in TF
200nm-beads
(540/560), 2.3×1011
beads/mL
2mm-beads (535/575),
9×108 beads/mL
56

78. As a result, the imaging
contrast is enhanced by
our new 3D-ITF method.
57

79. Super-resolution is achieved in scattering environments.

80. 



58

81. 介绍
 光学传递函数，光学切片能力
 时空聚焦技术，高速双光子荧光显微镜
 时空聚焦的缺陷, 可改进之处
二维时空聚焦
 设计想法，理论证明
 增强光学切片能力，实验验证
三维干涉-时空聚焦显微镜
 结构光超分辨显微镜原理
 3维超分辨率，超切片能力验证
 散乱光，背景光去除
三维干涉-时空聚焦显微镜对光学扰乱的耐性
59

82. REDUNDANCY(I.E., COPY OF DATA)
We need reliable delivery of
information over unreliable
communication channels.
Simple solution by backups
 In spatial domain
 In time domain
 Even in frequency domain
Redundancy could be applied to
microscopy to construct reliable
image with optical distortions
60

83. Redundancy in 3D-Interferometric TF (3D-ITF),
spectrum domain redundancy
Optical Transfer Function (OTFzy)
61

84. Zernike mode aberration
62

85. 63
OTF PSF
astigmatism

86. SNR=10
SNR>100
3D-ITF shows resistance of spatial resolution to the optical distortions
64
astigmatism
spherical
aberration

87.  With emission wavefront distortion, 3D-ITF could loss sectioning ability like
3D-SIM. Again the redundancy recovers the spatial frequency support
along kz axis so that shows resistance of sectioning ability in 3D-ITF.
 On the other hand, TF lose the sectioning ability due to large amount of
background fluorescence.
Astigmatism Spherical aberration
65

88. 







66

89. Point
scanning
TF 2D-TF 3D-ITF
Excitation
region
Point Wide-field Wide-field Wide-field
Sectioning
ability
~1/z2 ~1/z ~1/z2 ~1/z
Point
scanning
TF 2D-TF 3D-ITF*
Spatial
resolution
Diffraction
limitation
Diffraction
limitation
Diffraction
limitation
Super-
resolution
Scattered
emission
background
No problem problem problem No problem
Optical
distortion
affection
problem problem problem has
resistance
Two photon excitation
Fluorescence image
* 3D-ITF fluorescence image requires post-processing from multiple raw images
8

90. Thanks for your attention!