This document proposes an approach for abnormal event detection in unseen scenarios using blob-level statistical analysis of video frames. The approach extracts frame-level features from foreground blobs using measures like blob area, count, and distance. These are transformed into temporal features considering speed and order. Event models are built offline using these features and an SVM classifier. During operation, only selected frame and temporal features are extracted in real-time, without additional training or tuning. Experiments on public safety and crowd datasets show the approach can detect events in unseen environments, outperforming optical flow-based methods.
2. Outline
Event Detection for Public Safety
Challenges
Proposed Approach
Experiments
Summary
Q&A
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
2
3. Event Detection for Public Safety
Mob Violence
Crowding
Sudden Group Formation
Sudden Group Deformation
Shooting
Panic Driven Behaviours
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
3
4. Event Detection
time
Specific event (e.g., run) detection vs. abnormality detection
An event persists for a certain duration of time
The duration is variable
The characteristics of the same event is
variable in the same environment
variable from one scene to other
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
4
6. Challenges
Abnormal Event Detection
Supervised
Unsupervised
Manual Labelling,
Prior assumption of well
define event classes
No Event Model
Clustering of observed patterns,
Database of spatiotemporal patches
Semi-supervised
Normal event modelling:
manual labelling,
Abnormal event modelling:
unsupervised adaptation
Explicit Event Model
More Recent Approach
Mixture of Dynamic Bayesian Networks
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
6
7. Challenges
Build Event Model Once
Operate Everywhere
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
7
8. Proposed Approach
Build
•
•
•
•
Targeted Events
Extensive Feature Extraction
Feature Selection/Ranking
Supervised
Operation
•
•
•
•
•
No additional training
No parameter tuning
Selected feature extraction
No feature ranking
Real-time detection
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
8
9. Proposed Approach
f1
f2
Event
f3
Model
.
time
Frame-level Features
fn
Temporal Features
Classifier
Event detection as temporal data classification problem
A distinct set of temporal features can characterise an event
Independent frame-level features extracted using blob statistical
analysis; no object / position specific information, no spatial
association
Frame-level features are transformed into temporal features
considering speed and temporal order
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
9
10. Proposed Approach
Processes
Foreground
Detector
Frame-level
Feature Extractor
Temporal
Feature Extractor
Event
Models
Model Training (offline)
Frame-level
Feature Extraction
(30 features)
Background
Subtraction
Labelled frames
Temporal
Feature Extraction
(270 features)
Feature Ranking
and Selection
Event Model
Training
Foreground blobs
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
10
11. Blob-Statistical Analysis
Frame-level features
Blob Area (BA)
Filling Ratio (FR)
Aspect Ratio (AR)
Bounding Box Area (BBA)
Bounding box Width (BBW)
Bounding box Height (BBH)
Blob Count (BC)
Blob Distance (BD)
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
11
12. Blob Statistical Analysis
Blob Count (BC), Blob Area (BA)
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
12
16. Feature Extraction
Top five features for four different events
Feature ranking using absolute value criteria of two sample t-test, based on
pooled variance estimate.
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
16
17. Proposed Approach
Model Training (offline)
Frame-level
Feature Extraction
(30 features)
Background
Subtraction
Labelled frames
Temporal
Feature Extraction
(270 features)
Feature Ranking
and Selection
Event Model
Training
Foreground blobs
Event Detection in the Operating Environment
Selective
Frame-level
Feature Extraction
Background
Subtraction
Incoming frames
Selective
Temporal
Feature Extraction
Trained
Event Models
Detection
Results
Foreground blobs
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
17
18. Summary and Discussion
Motion based approaches
Key points detection
Point matching in successive frames
Flow vectors: position, direction, speed
Tracking based approaches
Object detection
Object matching in successive frames
Trajectories: object paths
Common characteristics
Inter-frame association
Context specific information
Event models are not generic
Hu et al. (ICPR 2008)
Xiang et al. (IJCV 2006)
Proposed approach
No Inter-frame association
Foreground blob detection
Independent frame-level features =>
Global frame-level descriptor based on
temporal features considering speed
blob statistical analysis, independent
and temporal order
of scene characteristics
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
18
19. Experiments
Model Training
•
•
•
•
•
•
•
Four different events: meet, split, runaway, and fight
CAVIAR dataset with labelled frames
80% of the test frames for model training
100 iterations of 10-fold cross validation
Remaining 20% of the test frames for testing
SVM classifier as event models
Separate model for each event
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
19
20. Experiments
Event Models
Unseen Scenarios in
Known Context
Unseen Scenarios in
Unknown Context
Greenfield
Outdoor
Abnormal Event Detection in Unseen Scenarios
Corridor
December 30, 2013
20
23. Experiments
• Abnormal event detection in unseen scenarios in
unknown context
• University of Minnesota crowd dataset (UMN dataset)
• The Runaway event model
• No additional training or tuning
• Three different sites
Greenfield
Outdoor
Corridor
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
23
29. Experiments
Method
AUC
Proposed Method
0.89
Pure Optical Flow [1]
0.84
[1] R. Mehran, A. Oyama, and M. Shah, “Abnormal crowd behavior detection using social force model,” in Proc. IEEE
Conference on Computer Vision and Pattern Recognition CVPR 2009, Event Detection2009, pp.Scenarios
Abnormal 20–25 June in Unseen 935–942. December 30, 2013
29