ICIP 2013 | 2013 IEEE International Conference on Image Processing | September 15 - 18, 2013 | Melbourne, Australia

1. Smoothing Posterior Probabilities
with a Particle Filter of Dirichlet Distribution
for Stabilizing Colorectal NBI Endoscopy Recognition
Tsubasa Hirakawa, Toru Tamaki, Bisser Raytchev, Kazufumi Kaneda,
Tetsushi Koide, Yoko Kominami, Rie Miyaki, Taiji Matsuo,
Shigeto Yoshida, Shinji Tanaka
Hiroshima University, Japan
Sep. 17. 2013

2. Colorectal cancer
• 45,000 people have died from this cancer
each year.
• The 3th leading cause of cancer death in
Japan.
Colorectal tumor must be found as early as possible!
1
0""
20""
40""
60""
80""
100""
stage 1 stage 2 stage 3 stage 4
5 year survival rate of colorectal tumor
Early stage End stage
survivalrate[%]
Time trend of the death by colorectal cancer
0"
10,000"
20,000"
30,000"
40,000"
50,000"
'90" '91" '92" '93" '94" '95" '96" '97" '98" '99" '00" '01" '02" '03" '04" '05" '06" '07" '08" '09"
fatalitiesofcolorectalcancer
year
• 5-year survival rate keeps in high percentage
in early stage.
• Early finding of colorectal tumor causes
complete cure.

3. Endoscopy examination with NBI 2
• Narrow-Banded Imaging (NBI) system
• Enable us to enhance microvessel
structure of polyps.
Normal NBI
Polyp
Type A
Type B
Type C
1
2
3
Microvessels are not observed or extremely opaque.
Fine microvessels are observed around pits, and clear
pits can be observed via the nest of microvessels.
Microvessels comprise an irregular network, pits
observed via the microvessels are slightly non-distinct,
and vessel diameter or distribution is homogeneous.
Microvessels comprise an irregular network, pits
observed via the microvessels are irregular, and
vessel diameter or distribution is heterogeneous.
Pits via the microvessels are invisible, irregular vessel
diameter is thick, or the vessel distribution is
heterogeneous, and a vascular areas are observed.
Narrow-Band Imaging (NBI) magnification findings
Normal
Advanced
Cancer

4. 4
Colorectal tumor classification in magnifying
endoscopic NBI images [Tamaki et al., ACCV2010, MedIA2013]
• Feature: Bag-of-Visual-Words of
densely sampled SIFT
• Classifier: Linear SVM
• Accuracy: 96%
Real-time recognition system [Tamaki et al., MedIA2013]
Extended to recognition
of NBI video
Display posterior probabilities
at each frame.

5. Problem ~Real-time Recognition System~ 5
The output is highly unstable
0
0.5
1
251" 271" 291" 311" 331" 351" 371" 391" 411" 431"
Probability
Frame number
A
B
C
0 20 40 60 80 120100 140 160 180 200
Estimated label
Probability of type A
Probability of type B
Probability of type C3

6. Previous work 1 6
Smoothing of “curves” [Yokota et al., SSII2012]
• No probabilistic interpretation.
• Smoothing requires normalization to ensure that
probabilities sum to 1.
Problem
0 20 40 60 80 100 120 140 160 180 200
0
0.5
1
Time
probability
0
0.5
1
0 20 40 60 80 100 120 140 160 180 200
Probability
Type A
Type B
Type C3
Frame number
• Kalman Filter (x, ẋ and ẍ)
Input
Output

7. Previous work 2 7
Sequence Labeling [Hirakawa et al., EMBC2013]
Type A
Type B
Type C3
Type B_1 (original)
frame number
0 20 40 60 80 100 120 140 160 180 200
Type B_1 (DP_0.99)
frame number
0 20 40 60 80 100 120 140 160 180 200
0
0.5
1
251" 271" 291" 311" 331" 351" 371" 391" 411" 431"
Frame number
A
B
C
0 20 40 60 80 120100 140 160 180 200Type B_1 (original)
frame number
0 20 40 60 80 100 120 140 160 180 200
Type B_1 (DP_0.99)
frame number
0 20 40 60 80 100 120 140 160 180 200
• Map estimation of MRF
• Output is labels assigned to each frame
Labels
applied
MAP
estimation
Output
Input

8. Previous work 2 8
Sequence Labeling [Hirakawa et al., EMBC2013]
Type A
Type B
Type C3
Type B_1 (original)
frame number
0 20 40 60 80 100 120 140 160 180 200
Type B_1 (DP_0.99)
frame number
0 20 40 60 80 100 120 140 160 180 200
0
0.5
1
251" 271" 291" 311" 331" 351" 371" 391" 411" 431"
Frame number
A
B
C
0 20 40 60 80 120100 140 160 180 200Type B_1 (original)
frame number
0 20 40 60 80 100 120 140 160 180 200
Type B_1 (DP_0.99)
frame number
0 20 40 60 80 100 120 140 160 180 200
• Map estimation of MRF
• Output is labels assigned to each frame
Labels
applied
MAP
estimation
Output
Input
• Labels are LESS informative than probabilities.
! We have examined about how we should display the
recognition results.
Problem

9. Motivation 9
! To support decisions by endoscopists
during an endoscopy examination
Visualize temporally smoothed and stabilized
posterior probability curves.
Objective
• Sequential online Bayes filtering
• Introducing the Dirichlet distribution as transition
and likelihood
• Implemented with the Particle filtering.
Probabilistic Approach

10. Sequential Filtering 10
xt = xt
(A)
, xt
(B)
, xt
(C3)
( ), xt
A( )
+ xt
B( )
+ xt
C3( )
=1State vector:
Observation vector: yt = yt
A( )
, yt
B( )
, yt
C3( )
( ), yt
A( )
+ yt
B( )
+ yt
C3( )
=1
We use Dirichlet distribution for state transition and likelihood.
Prediction
p xt y1:t−1( )= p xt xt−1( )∫ p xt−1 y1:t−1( )dxt
Filtering
p xt y1:t( )∝ p yt xt−1( ) p xt y1:t−1( )
State transition
Likelihood
Observation to t-1State of t
Observation to tState of t
※ t : time

11. Dirichlet distribution 11
Dirλ1…K
α1…K[ ]=
Γ αkk=1
K
∑#
$%
&
'(
Γ αk[ ]k=1
K
∏
λk
αk −1
k=1
K
∏
(0.50, 0.50, 0.50)
(0.85, 1.50, 2.00)
(1.00, 1.00, 1.00)
(1.00, 1.76, 2.35)
(4.00, 4.00 ,4.00)
(3.40, 6.00, 8.00)
low
high
α1…K : parameter of distribution

12. Sequential Filtering 12
xt = xt
(A)
, xt
(B)
, xt
(C3)
( ), xt
A( )
+ xt
B( )
+ xt
C3( )
=1State vector:
Observation vector: yt = yt
A( )
, yt
B( )
, yt
C3( )
( ), yt
A( )
+ yt
B( )
+ yt
C3( )
=1
We use Dirichlet distribution for state transition and likelihood.
Prediction
p xt y1:t−1( )= p xt xt−1( )∫ p xt−1 y1:t−1( )dxt
Filtering
p xt y1:t( )∝ p yt xt−1( ) p xt y1:t−1( )
State transition
Likelihood

13. Proposed method ~state transition~ 13
p xt xt−1,θ1( )= Dirxt
α1 θ1, xt−1( )"# $%
• We define the transition as Dirichlet.
! To enforce xt to be close to xt-1.
! With a single parameter θ1 to control the shape of the distribution.
α1 θ1, xt−1( )=θ1xt−1
MAP estimate of xt-1
θ1=1 θ1=100
Should be distributed around xt-1
θ1=?

14. Proposed method ~likelihood~ 14
p yt xt,θ2( )= Dirxt
α2 θ2, yt( )!" #$ α2 θ2, yt( )=θ2 yt + b
• We define the likelihood as Dirichlet.
! To enforce xt to be close to yt.
! With a single parameter θ2 and additional bias (+b)
to control the shape of the distribution.
The value of yt
θ2=100, b=0 θ2=3, b=1
Distribution concentrates too much! Be distributed widely

15. Sequential Filtering 15
xt = xt
(A)
, xt
(B)
, xt
(C3)
( ), xt
A( )
+ xt
B( )
+ xt
C3( )
=1State vector:
Observation vector: yt = yt
A( )
, yt
B( )
, yt
C3( )
( ), yt
A( )
+ yt
B( )
+ yt
C3( )
=1
Prediction
p xt y1:t−1( )= p xt xt−1( )∫ p xt−1 y1:t−1( )dxt
Filtering
p xt y1:t( )∝ p yt xt−1( ) p xt y1:t−1( )
State transition
Likelihood
Implemented with a Particle Filtering

16. Experimental results ~data set~ 16
Learning
• 907 NBI images
(Type A: 359, Type B: 461, Type C3: 87)
• Ensure that the lighting conditions, zooming and
optical magnification were kept as similar as possible
across different images.
• Images were trimmed by medical doctors and
endoscopists.
Test video
• 4 NBI videoendoscopy sequences
(Type A: 2, Type B: 2)
• The length 200 frames, in which polyps
were captured largely enough in each
image.

17. Experimental results 17Type BType A Type C3
Original result
θ1 = 100, θ2 = 1
θ1 = 100, θ2 = 5
θ1 = 500, θ2 = 1
θ1 = 500, θ2 = 5
Type A_2 (original)
0 20 40 60 80 100 120 140 160 180 200
0.01.0
Type A_2 (100,1)
0 20 40 60 80 100 120 140 160 180 200
0.01.0
Type A_2 (100,5)
0 20 40 60 80 100 120 140 160 180 200
0.01.0
Type A_2 (500,1)
0 20 40 60 80 100 120 140 160 180 200
0.01.0
Type A_2 (500,5)
0 20 40 60 80 100 120 140 160 180 200
0.01.0
Type A
MRF labeling
Type A_2 (original)
frame number
0 20 40 60 80 100 120 140 160 180 200
Type A_2 (DP_0.99)
frame number
0 20 40 60 80 100 120 140 160 180 200
Type A_2 (Gibbs_p4=0.9)

18. Experimental results 18Type BType A Type C3
Type B_1 (Original)
frame number
0 20 40 60 80 100 120 140 160 180 200
Type B_1 (MRF)
frame number
0 20 40 60 80 100 120 140 160 180 200
Type B_1 (original)
0 20 40 60 80 100 120 140 160 180 200
0.01.0
Type B_1 (100,1)
0 20 40 60 80 100 120 140 160 180 200
0.01.0
Type B_1 (100,5)
0 20 40 60 80 100 120 140 160 180 200
0.01.0
Type B_1 (500,1)
0 20 40 60 80 100 120 140 160 180 200
0.01.0
Type B_1 (500,5)
0 20 40 60 80 100 120 140 160 180 200
0.01.0
Type B
Original result
θ1 = 100, θ2 = 1
θ1 = 100, θ2 = 5
θ1 = 500, θ2 = 1
θ1 = 500, θ2 = 5
MRF labeling

19. Conclusions
• We have proposed a Particle filter-based smoothing of
posterior probability.
! to visualize the output of NBI videoendoscopy recognition.
19
Future work
• Reduce the effects of optical and motion blurs to make
recognition more stable.
• Implement the filtering considering label changes.
• Parameter selection and learning.
• Quantitative evaluation.