Transfer Learning of Tissue Photon
Interaction in Optical Coherence
Tomography towards In vivo Histology of
the Oral Mucosa
Debdoot Sheet, Satarupa Banerjee, Sri Phani Krishna Karri, Swarnendu Bag, Anji Anura, Ajoy K. Ray
@ School of Medical Science and Technology, Indian Institute of Technology Kharagpur, India
Amita Giri,
@ Department of Pathology, North Bengal Medical College and Hospital, Darjeeling, India.
Ranjan Rashmi Paul, Mousumi Pal,
@ Oral and Maxillofacial Pathology, Guru Nanak Institute of Dental Sc. and Res., Kolkata, India.
Badal C. Sarkar, Ranjan Ghosh,
@ Oral and Maxillofacial Pathology, North Bengal Dental College and Hospital, Darjeeling, India.
Amin Katouzian, Nassir Navab
@ Chair for Computer Aided Medical Procedures, TU Munich, Germany
Motivation
• Mucosa forms the general internal
lining of the oral cavity protecting it
from harsh external influences.
• Stratified organization
– Stratified squamous epithelium
– Basement membrane
– Lamina propria
• Cancers and Pre-cancers
– Major pathological injury
– Loss of stratified structure
– Dysplasia
• Clinical challenge in management
– Early diagnosis of cancer / pre-cancer
onset
– Patient specific intervention
– In situ investigation of pre-cancer and
cancer progression is challenge
2 May 2014 ISBI 2014 - FrB03.1 [Debdoot Sheet] 2
Where do we stand now?
ISBI 2014 - FrB03.1 [Debdoot Sheet] 3
This Paper
2 May 2014
Text books
R. K. Das (2012), PhD Thesis
A. Barui (2011), PhD Thesis
State of the Art
2 May 2014 ISBI 2014 - FrB03.1 [Debdoot Sheet] 4
• In situ investigation
– Optical Coherence Tomography (OCT)
• Rebol, (2008)
• Jung et al., (2005)
• In situ Histology with OCT
– G. van Soest et al., (2010) –
Cardiovascular OCT
– A. Barui et al., (2011) – Cutaneous
wound beds.
– D. Sheet et al., (2013) – Cutaneous
wounds
• Challenges
– Identify co-located tissue heterogeneity
– Identify and discriminate rete-peg
architecture and inter-digitated
structures
Tissue Photon Interaction
ISBI 2014 - FrB03.1 [Debdoot Sheet] 5
Incident
radiation
Regular
reflection Diffuse
reflection
Scattering
Absorption OCT
B. Saleh, Introduction to Subsurface Imaging, Cambridge, 2011.
2 May 2014
Optical Coherence Tomography
ISBI 2014 - FrB03.1 [Debdoot Sheet] 6
Low time-coherence
light source
Depth scan mirror
Sample
Detector
Source beam
Reference beam
Sample beam
Detector beam
x
z
z
OCT Image
Michelson
interferometer
2 May 2014
Stochastic of TPI in SS-OCT
ISBI 2014 - FrB03.1 [Debdoot Sheet] 7
Source
Ballistic
backscattering
Non-ballistic
backscattering
Reference
Detector
A. F. Fercher, et al, Optical coherence tomography — principles and applications, Rep. Prog. Phys.
66 (2003) 239–303
Epithelium
Sub-epithelium
Speckle intensity
Probability
density
2 May 2014
  






S
S
S
S
I
Ip

exp
1
Framework
2 May 2014 ISBI 2014 - FrB03.1 [Debdoot Sheet] 8
Learn TPI Model
(i) Multiscale estimated speckle statistics
(ii) Attenuation coefficient
Training Image Ground Truth Labels
Test Image
Learn TPI Model
Characterized tissue
  train;,| II, xH
Experiment Design
• Data Collection:
– Multimodal Imaging and
Computing for Theranostics,
School of Medical Science and
Technology, Indian Institute of
Technology Kharagpur
– Imaging: Swept Source OCT
System
– OCS 1300 SS, ThorLabs, NJ,
USA
– In vitro preserved Biopsy
• HE stained
• Cross validation:
– 4 fold cross validation
• Samples
– Normal # 1
– Oral Sub-mucous Fibrosis # 1
– Oral Leukoplakia # 1
– Oral Lichen-planus # 1
• Learning:
– Source task:
• {μ,σ} at 10 scales
• Attenuation coefficient (van Soest
et al., (2010))
– Target task:
• Random forest with 50 binary
decision trees
2 May 2014 ISBI 2014 - FrB03.1 [Debdoot Sheet] 9
In vitro
validation
towards
In vivo
translation
2 May 2014 ISBI 2014 - FrB03.1 [Debdoot Sheet] 10
Area under the ROC Curve
Epithelium = 0.9611
Sub-epithelium = 0.9367
Take Home Message
• Photons interact characteristically with different tissues.
– This is manifested through the stochastic convergence of OCT speckle
intensity.
– Also manifested in the form of optical intensity attenuation.
• The stochastic nature of TPI accounts for uncertainties in
observations.
– Learning of TPI statistical physics overcomes these uncertainties.
• Transfer learning is a good framework for solving stochastic
convergent signal decomposition problems
– Speckle imaging application viz. OCT tissue characterization
– Learn (weak) local uncertainty of signals
– Learn (strong) the uncertainty associated with tissue types
2 May 2014 ISBI 2014 - FrB03.1 [Debdoot Sheet] 11

Transfer Learning of Tissue Photon Interaction in Optical Coherence Tomography towards In vivo Histology of the Oral Mucosa

  • 1.
    Transfer Learning ofTissue Photon Interaction in Optical Coherence Tomography towards In vivo Histology of the Oral Mucosa Debdoot Sheet, Satarupa Banerjee, Sri Phani Krishna Karri, Swarnendu Bag, Anji Anura, Ajoy K. Ray @ School of Medical Science and Technology, Indian Institute of Technology Kharagpur, India Amita Giri, @ Department of Pathology, North Bengal Medical College and Hospital, Darjeeling, India. Ranjan Rashmi Paul, Mousumi Pal, @ Oral and Maxillofacial Pathology, Guru Nanak Institute of Dental Sc. and Res., Kolkata, India. Badal C. Sarkar, Ranjan Ghosh, @ Oral and Maxillofacial Pathology, North Bengal Dental College and Hospital, Darjeeling, India. Amin Katouzian, Nassir Navab @ Chair for Computer Aided Medical Procedures, TU Munich, Germany
  • 2.
    Motivation • Mucosa formsthe general internal lining of the oral cavity protecting it from harsh external influences. • Stratified organization – Stratified squamous epithelium – Basement membrane – Lamina propria • Cancers and Pre-cancers – Major pathological injury – Loss of stratified structure – Dysplasia • Clinical challenge in management – Early diagnosis of cancer / pre-cancer onset – Patient specific intervention – In situ investigation of pre-cancer and cancer progression is challenge 2 May 2014 ISBI 2014 - FrB03.1 [Debdoot Sheet] 2
  • 3.
    Where do westand now? ISBI 2014 - FrB03.1 [Debdoot Sheet] 3 This Paper 2 May 2014 Text books R. K. Das (2012), PhD Thesis A. Barui (2011), PhD Thesis
  • 4.
    State of theArt 2 May 2014 ISBI 2014 - FrB03.1 [Debdoot Sheet] 4 • In situ investigation – Optical Coherence Tomography (OCT) • Rebol, (2008) • Jung et al., (2005) • In situ Histology with OCT – G. van Soest et al., (2010) – Cardiovascular OCT – A. Barui et al., (2011) – Cutaneous wound beds. – D. Sheet et al., (2013) – Cutaneous wounds • Challenges – Identify co-located tissue heterogeneity – Identify and discriminate rete-peg architecture and inter-digitated structures
  • 5.
    Tissue Photon Interaction ISBI2014 - FrB03.1 [Debdoot Sheet] 5 Incident radiation Regular reflection Diffuse reflection Scattering Absorption OCT B. Saleh, Introduction to Subsurface Imaging, Cambridge, 2011. 2 May 2014
  • 6.
    Optical Coherence Tomography ISBI2014 - FrB03.1 [Debdoot Sheet] 6 Low time-coherence light source Depth scan mirror Sample Detector Source beam Reference beam Sample beam Detector beam x z z OCT Image Michelson interferometer 2 May 2014
  • 7.
    Stochastic of TPIin SS-OCT ISBI 2014 - FrB03.1 [Debdoot Sheet] 7 Source Ballistic backscattering Non-ballistic backscattering Reference Detector A. F. Fercher, et al, Optical coherence tomography — principles and applications, Rep. Prog. Phys. 66 (2003) 239–303 Epithelium Sub-epithelium Speckle intensity Probability density 2 May 2014          S S S S I Ip  exp 1
  • 8.
    Framework 2 May 2014ISBI 2014 - FrB03.1 [Debdoot Sheet] 8 Learn TPI Model (i) Multiscale estimated speckle statistics (ii) Attenuation coefficient Training Image Ground Truth Labels Test Image Learn TPI Model Characterized tissue   train;,| II, xH
  • 9.
    Experiment Design • DataCollection: – Multimodal Imaging and Computing for Theranostics, School of Medical Science and Technology, Indian Institute of Technology Kharagpur – Imaging: Swept Source OCT System – OCS 1300 SS, ThorLabs, NJ, USA – In vitro preserved Biopsy • HE stained • Cross validation: – 4 fold cross validation • Samples – Normal # 1 – Oral Sub-mucous Fibrosis # 1 – Oral Leukoplakia # 1 – Oral Lichen-planus # 1 • Learning: – Source task: • {μ,σ} at 10 scales • Attenuation coefficient (van Soest et al., (2010)) – Target task: • Random forest with 50 binary decision trees 2 May 2014 ISBI 2014 - FrB03.1 [Debdoot Sheet] 9
  • 10.
    In vitro validation towards In vivo translation 2May 2014 ISBI 2014 - FrB03.1 [Debdoot Sheet] 10 Area under the ROC Curve Epithelium = 0.9611 Sub-epithelium = 0.9367
  • 11.
    Take Home Message •Photons interact characteristically with different tissues. – This is manifested through the stochastic convergence of OCT speckle intensity. – Also manifested in the form of optical intensity attenuation. • The stochastic nature of TPI accounts for uncertainties in observations. – Learning of TPI statistical physics overcomes these uncertainties. • Transfer learning is a good framework for solving stochastic convergent signal decomposition problems – Speckle imaging application viz. OCT tissue characterization – Learn (weak) local uncertainty of signals – Learn (strong) the uncertainty associated with tissue types 2 May 2014 ISBI 2014 - FrB03.1 [Debdoot Sheet] 11