1. Research and Implementation
of
Smoke Detection in Video Streams
naveedakram@live.com
Naveed Akram 内维德
School of Computer Science and Engineering ,
Beihang University, Beijing
Slide 1 of 61

2. Agenda
Introduction of Research Work
 Background and Motivation
 Overview of Research Work
 Research and Implementation
 Results / Demo
 Question / Answer

Slide 2 of 61

3. Introduction
An image processing based technique
is proposed to detect fire smoke in
video streams.
 Basic Idea is to use already installed
CCTV cameras for smoke detection
instead of using conventional smoke
detectors.

Slide 3 of 61

4. Background and
Background and Motivation
Fire is one of the biggest disasters for
the human beings.
 In 2009 (only in USA)

◦
◦
◦
◦
estimated 1,348,500 fires
3,010 deaths
17,050 injuries
$12.5 Billion property loss
Slide 4 of 61

5. Background and
Why we need this study

Traditional methods can not work in
some situation and fail to detect fire
smoke.
◦
◦
◦
◦

Some times can not detect at all.
Produces delay and need close proximity
Fail in open places, outdoor, forests
No method to verify false alarms
We are proposing a method that can
overcome these issues.
Slide 5 of 61

6. Background and
Video Based Fire Detection
System
Lower cost
 Faster response
 Large coverage area
 Verification of false alarms

Slide 6 of 61

7. Background and
Challenges




Still evolving technology
Difficult to process due to variability in
smoke density, lighting, diverse
background, interfering non-rigid objects
etc.
Primitive image features such as
intensity, motion, edge, and obscuration
do not characterizes smoke very well in
the videos
Visual pattern of smoke is difficult to
model.
Slide 7 of 61

8. Background and
Related Work

In recent literature a number of
methods for smoke detection in videos
are presented based on
◦ Self-Similarity
◦ Motion and optical flow
◦ Wavelet Transformation
(flickering/∆energy)
◦ Based on Color models
◦ Night Vision fire detection
◦ Feature’s based
Slide 8 of 61

9. OVERVIEW OF
RESEARCH WORK
Slide 9 of 61

10. Overview of Research
Overview of Research Work
PreProcessing
Moving
Target
Detection
Feature
Extraction
Smoke
Detection
Slide 10 of 61

11. Overview of Research
Pre Processing
PreProcessing
Moving
Target
Detection
Feature
Extraction
Smoke
Detection
1. Frame Extraction from Video Stream
2. Color to gray scale conversion
3. Median filtering
Slide 11 of 61

12. Overview of Research
Moving Target Detection
PreProcessing
Moving
Target
Detection
Feature
Extraction
Smoke
Detection
1. Background Subtraction
2. Grayscale to Binary Conversion
3. Contour Extraction
Slide 12 of 61

13. Overview of Research
Feature Detection
PreProcessing
Moving
Target
Detection
Feature
Extraction
Smoke
Detection
1. Calculation of static and dynamic features of
moving target object.
2. Such as Local Wavelet Energy, Growth rate,
Disorder, flickering frequency, Source
Stability etc.
Slide 13 of 61

14. Overview of Research
Smoke Detection
PreProcessing
Moving
Target
Detection
Feature
Extraction
Smoke
Detection
1. Training of Neural Network (Once)
2. Use of Neural Network to decide either
smoke or not
Slide 14 of 61

15. RESEARCH AND
IMPLEMENTATION
Slide 15 of 61

16. Research and
Moving Target Detection
Moving
Target
Detection
Feature
Extraction
Background
Subtraction
Contour
Extraction
Smoke
Detection
Slide 16 of 61

17. Research and
Background Subtraction

Preliminary frame differencing

Dynamics Matrix

Adaptive Background Update
Slide 17 of 61

18. Background Subtraction
Background Update Model
Slide 18 of 61

19. Research and
Background Subtraction
Foreground detection
Start
Current Filtered
Frame, I(k)
Absolute Frame Difference
B(i,j) from Background
update Model
FDi , j ( k )
I i , j (k )
Bi , j ( k )
.F.
Foreground
FG(i,j)=0
If FD(i,j) >T
.T.
Foreground
FG(i,j)=I(i,j)
Link to Next
Process
Slide 19 of 61

20. Research and
Background Subtraction
Results
(a) Original Frames
(b) Foreground Objects Slide
20 of 61

21. Research and
Contour Extraction

Grayscale to Binary Conversion
(Otsu's method )

Dilation and Erosion of Binary Image
Slide 21 of 61

22. Research and
Grayscale->Binary
Conversation
Slide 22 of 61

23. Research and
Erosion and Dilation
Erosio
n
Dilation
Slide 23 of 61

24. Research and
Contour Extraction
Dilation
Contour= DilationErosion
Erosio
n
Slide 24 of 61

25. Research and
Results of Moving Target
Detection
Smoke Detected
Fr 252 Fgnd.Fr 221
GR=4.07 DS =0.01 NOB=2 FLC=47.0 EN=-1.0 S S =1394
Slide 25 of 61

26. Research and
Feature Extraction
Moving
Target
Detection
Feature
Extraction
Smoke
Detection
Dynamic
Features
PreProcessing
Static
Features
Growth Rate
Disorder
Number of
Segments
Source
Stability
Flickering
Frequency
Local
Wavelet
Energy
Slide 26 of 61

27. Research and
Growth Rate

as percentage change in Area of the
current frame with reference to
previous frame
G row thR ate
A( x, y )i
A( x, y )i
A( x, y )i
A( x , y ) i
1
i
2
1
Number of '1's in binary image i
Slide 27 of 61

28. Research and
Growth Rate Results
Slide 28 of 61

29. Research and
Disorder Feature

Smoke has another feature that
makes it distinguish from other
foreground objects that is its rapidly
changing shape. This feature of
smoke is called disorder
Slide 29 of 61

30. Research and
Disorder Feature
Smoke
Video
Human
Movement
Slide 30 of 61

31. Research and
Results of
Disorder Feature
Disorder VS Frame No
3
2.8
Disorder
2.6
Human
Movement
2.4
2.2
2
1.8
1.6
280
285
290
295
300
305
Frame No
310
315
320
325
330
Disorder VS Frame No
6
Disorder
5.5
Smoke
Video
5
4.5
4
3.5
280
285
290
295
300
305
Frame No
310
315
320
325
330
Slide 31 of 61

32. Research and
Number of Segments
While smoke spreads it splits into
different small / large patches.
Sometime these patches may
increase to 8 to 10.
 We used 8-connected pixels algorithm
to calculate number of segments in
current video frame.

Slide 32 of 61

33. Research and
Number of Segments
Slide 33 of 61

34. Research and
Frequent Flickering

A pixel at the edge of a turbulent flame
or boundary of smoke could appear
and disappear several times in one
second of a video sequence. This kind
of temporal periodicity is commonly
known as flickering
Slide 34 of 61

35. Research and
Frequent Flickering

Transition Matrix

Frequency Matrix

Pixels on contour
Slide 35 of 61

36. Research and
Frequent Flickering

Thresholding

Feature Calculation
Slide 36 of 61

37. Research and
Flow Chart
Slide 37 of 61

38. Research and
Frequent Flickering
Thresholding
Subtracting consecutive frames to get transition
Frequency Matrix
AND with Contour
matrices
Slide 38 of 61

39. Results
Smoke
Video
Human
Movement Video
Slide 39 of 61

40. Research and
Local Wavelet Energy
Sharp edges in the background are
sources of high frequency and hence
high wavelet energy
 Fire smoke can smoothen the edges
in an image because of the fuzzy
effect of smoke .
 Hence it decreases local wavelet
energy in the scene

Slide 40 of 61

41. Research and
Local Wavelet Energy

we calculate difference of LWE of
background frame and Current frame
to get this feature
Slide 41 of 61

42. Research and
Local Wavelet Energy
Backgroun
d
Current
Frame
Slide 42 of 61

43. -3
20
Change is Wavelet Energy (eb-e)
x 10
Fire Smoke
Video
Change in Wavellet Energy
15
10
5
0
-5
0
20
40
60
80
100
Video Frames
120
140
160
180
200
Change is Wavelet Energy (eb-e)
0.05
Human Movement
Video
0
Change in Wavellet Energy
Research and
Results
-0.05
-0.1
-0.15
-0.2
-0.25
-0.3
0
20
40
60
80
100
Video Frames
120
140
160
180
200
Slide 43 of 61

44. Research and
Source Stability

Source of fire smoke always remain
near about at same location while in
case of a human movement complete
foreground object moves and there is
not a single emerging (source) path.
Slide 44 of 61

45. Research and
Source Stability
Slide 45 of 61

46. Research and
Results
Slide 46 of 61

47. Source Stability VS Frame No
Source Stability VS Frame No
Fire Smoke
Video
14000
2500
12000
2000
Source Stability
Source Stability
10000
1500
1000
500
0
8000
6000
4000
2000
0
50
100
150
200
250
Frame No
300
350
400
450
0
0
100
Source Stability VS Frame No
200
300
400
500
600
Frame No
Source Stability VS Frame No
700
800
1
600
Human Movement
Video
0.8
500
0.6
0.4
Source Stability
400
Source Stability
Research and
Results Comparison
300
200
0.2
0
-0.2
-0.4
-0.6
100
0
-0.8
-1
0
50
100
150
200
Frame No
250
300
350
0
100
200
300
400
Frame No
500
600
700
Slide 47 of 61

48. Research and
BP Neural Networks
PreProcessing
Moving
Target
Detection
Feature
Extraction
Smoke
Detection
BP Neural Networks are trained using
logsig training function with several
smoke and non-smoke videos.
 Later this trained Network is used for
real time smoke detection.
 MATLAB is use to train the NN

Slide 48 of 61

49. Research and
BP Neural Networks
0.65 & &
1. Growth Rate
2. Disorder
3. Number of Segments
S m oke
0.25 & &
0.64
D anger
0 &&
out
1
0.24
N orm al
4. Frequent Flickering
5. Local Wavelet Energy
6. Source Stability
Slide 49 of 61

50. Research and
Features Weights for NN
Results
Growth Rate
Disorder
Number of Segments
Frequent Flickering
Local Wavelet Energy
Source Stability
Smoke Starts
Smoke
Spreads
Slide 50 of 61

51. Training of the NN
Video
Samples
(8 Videos)
Frame Extraction
Moving Target Detection
Feature Extraction
Save Feature Vector
Samples
Completed
Training of Neural
Networks
Construction of
Network
Slide 51 of 61

52. Research and
Training of the NN
Slide 52 of 61

53. Use of Trained NN
Slide 53 of 61

54. RESULTS
Slide 54 of 61

55. Results
Results of Smoke Detection
Slide 55 of 61

56. Results
NN Results (Smoke Video)
Slide 56 of 61

57. Results
NN Results (non-Smoke
Video)
Slide 57 of 61

58. Results
NN Results
Slide 58 of 61

59. DEMOS
Slide 59 of 61

60. Demos
Smoke Video 1
 Smoke Video 2
 Smoke Video 3
 Smoke Video 4
 Human Video 1
 Human Video 2
 Human Video 3
 Human Video 4

Slide 60 of 61

61. QUESTIONS / ANSWERS
THANKS EVERYONE FOR
YOUR TIME
Slide 61 of 61