Summarized of two papers 'detection and removal of rain from video, vision and rain' for my project which remove rain and snow from image system.

1. Detection and Removal of Rain from Videos, 2004
Kshitiz Garg and Shree K. Nayar
Vision and Rain, 2006
Kshitiz Garg and Shree K. Nayar
ELECTRICAL AND ELECTRONIC ENGINEERING
CAPSTONE DESIGN
Minjae Ahn
Super Resolution Image Processing Lab

2. Contents
1 Main Idea
2 Algorithm
3 Opinion

3. Main Idea
1st

4. 1-1. Appearance of Rain
1. Dynamics of Rain
• Under a few simple Conditions
• Represent Rain on each pixel in Binary Field
• Derived correlation function with above conditions
• Above Function will be Model Constraints
2. Photometry of a Raindrop
• Models through results of observation
• Derived equation between
‘difference of intensity’ and ‘background intensity’
• Above Equation will be Model Constraints

5. 1-2. Dynamics of Rain
[CONDITIONS]
1. Binary Field Representation
𝑏 𝑟, 𝑡 =
1, 𝑖𝑓 𝑑𝑟𝑜𝑝 𝑝𝑟𝑜𝑗𝑒𝑐𝑡𝑠 𝑡𝑜 𝑙𝑜𝑐𝑎𝑡𝑖𝑜𝑛 𝑟 𝑎𝑡 𝑡𝑖𝑚𝑒 𝑡
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
2. Uniform distribution on both space and time dimensions
→ Wide Sense Stationary in space and time
→ Correlation Function depends only on
Differences in Space and Time
3. Let drop velocity 𝑣
𝑅 𝑏(𝑟1, 𝑡1; 𝑟2, 𝑡2) = 𝑅 𝑏(∆ 𝑟, ∆t) = 𝑅 𝑏( 𝑣 ∆t, ∆t)
It means Two Points by 𝑣 ∆t in space are correlated with time lag ∆t

6. 1-2. Dynamics of Rain (continue)
[Derive Constraint]
1. Our image system, finite pixel 𝑝, finite time 𝑇
𝑅 𝑏(𝑚𝑝, 𝑛𝑇) = 𝑅 𝑏( 𝑣 ∆t, ∆t)
2. As a result,
• Above correlation value will be
• High in the direction 𝑣/| 𝑣|
• Low in the other direction
• It can be used constraints on Detecting Rain

7. 1-3. Photometric of Raindrop
[Observation Results]
1. Irradiance and Intensity has a peak when rain drops are passed.
2. Background Irradiance, Intensity remains steady constant.

8. 1-3. Photometric of Raindrop (continue)
[Derive Constraint]
1. Intensity, (if background is slow, 𝐸 𝑏𝑔 assumed as constant)
𝐼 𝑑 𝑟 =
0
𝜏
𝐸 𝑑 𝑑𝑡 +
𝜏
𝑇
𝐸 𝑏𝑔 𝑑𝑡 =
0
𝜏
𝐸 𝑑 𝑑𝑡 + (𝑇 − 𝜏)𝐸 𝑏𝑔
2. Equation between
difference of intensity and background intensity
∆𝐼 = 𝐼 𝑑 − 𝐼 𝑏𝑔 = −𝛽𝐼 𝑏𝑔 + 𝛼
𝛽 =
𝜏
𝑇
, 𝛼 = 𝜏𝐸 𝑏𝑔
3. Numerical Bounds for 𝛽
0 < 𝛽 < 0.039

9. Algorithm
2nd

10. 2-1. Detecting Algorithm
1. Photometric Constraint
① Compare intensity of n-th Frame 𝐼 𝑛 with 𝐼 𝑛+1 and 𝐼 𝑛−1
② Select as rain when change in intensity is larger than
threshold
∆𝐼 = 𝐼 𝑛+1 − 𝐼 𝑛 = 𝐼 𝑛 − 𝐼 𝑛−1 ≥ 𝑐,
e.g. Threshold c = 3 in gray scale
③ Add constraints with numeric bound of slope beta 𝛽
④ Select as rain with binary field
2. Dynamic Constraint
① Compute correlation on space and time dimension
using above binary field
② The correlation value of same direction of rainfall in
Local patch will be high, the others keep it false-
positive

11. Opinion
3nd

12. 3. Opinion
1. Limitations
• Due to the stationary condition in dynamics of rain,
It can be used in only frames of images, Not in single Image
• Still hard to capture pixels of rain if background varies
rapidly
2. Questions
① How do we compare our models’ performance?
• Is there any metrics for computing reconstruction
score for image?

13. Appreciate your
Listening