Analysing pedestrian dynamics with computer vision techniques - examples

1. Estimating Speeds of Pedestrians in Real-World
Using Computer Vision
Sultan Daud Khan, Fabio Porta, Giuseppe Vizzari and Stefania
Bandini
Complex Systems and Artificial Intelligence Research Center (CSAI)
University of Milano-Bicocca, Italy

2. C&CA @ ACRI 2014 – Sept. 22, 2014
Outline
• Crowd studies: towards integrated analysis and synthesis
• Computer vision and crowd studies
• Velocity estimation in naturalistic conditions
• Crowd flow segmentation and counting
• Identification of sources and sinks, towards pedestrian behaviour
understanding

3. C&CA @ ACRI 2014 – Sept. 22, 2014
Outline
• Crowd studies: towards integrated analysis and synthesis
• Computer vision and crowd studies
• Velocity estimation in naturalistic conditions
• Crowd flow segmentation and counting
• Identification of sources and sinks, towards pedestrian behaviour
understanding

4. C&CA @ ACRI 2014 – Sept. 22, 2014
Motivations of crowd studies
• Large events involving large number of people in relatively small spaces are
periodically held all around the world (sports, expositions, festivals, etc.)
• Public safety in high density crowds is potentially a big issue
• Decision support for designers and crowd managers, both in planning and
management phases, is highly desirable

5. C&CA @ ACRI 2014 – Sept. 22, 2014
Towards integrated analysis and synthesis of
crowd phenomena

6. C&CA @ ACRI 2014 – Sept. 22, 2014
Outline
• Crowd studies: towards integrated analysis and synthesis
• Computer vision and crowd studies
• Velocity estimation in naturalistic conditions
• Crowd flow segmentation and counting
• Identification of sources and sinks, towards pedestrian behaviour
understanding

7. C&CA @ ACRI 2014 – Sept. 22, 2014
Computer vision and crowd studies
• Relevant factors influencing choice of
techniques
• Crowd “types”
• Structured (motion constrained by
environmental structure, crowd management
procedures, other rules)
• Unstructured (little constraints to pedestrian
movements)
• Crowd density
• Initially driven by surveillance and
security (anomalous movements
detection)
• Recent interest in collaborating with
modeling and simulation community
• 1st IEEE Workshop on Modeling, Simulation
and Visual Analysis of Large Crowds – ICCV
2011
• First International Workshop on Pattern
Recognition and Crowd Analysis – ICPR
2012
Junior, Musse, Jung, Crowd Analysis Using Computer
Vision Techniques IEEE Signal Processing Magazine,
2010

8. C&CA @ ACRI 2014 – Sept. 22, 2014
Current activities and results
• Low-medium density
situations
• Velocity estimation in
naturalistic conditions
• High density situations
• Crowd flow segmentation and
counting
• Identification of sources and
sinks, towards pedestrian
behaviour understanding

9. C&CA @ ACRI 2014 – Sept. 22, 2014
Outline
• Crowd studies: towards integrated analysis and synthesis
• Computer vision and crowd studies
• Velocity estimation in naturalistic conditions
• Crowd flow segmentation and counting
• Identification of sources and sinks, towards pedestrian behaviour
understanding

10. C&CA @ ACRI 2014 – Sept. 22, 2014
Initial tracking approach
• Frame enhancement and
foreground segmentation
• Tracking with KLT after corner
detection

11. C&CA @ ACRI 2014 – Sept. 22, 2014
Velocity estimation in side view scenarios
• Pixel to metric coordinates
conversion approach based on the
idea of scaling factors
• Only applicable to limited kind of
scenario
• Requires coordinates of two points
per considered path

12. C&CA @ ACRI 2014 – Sept. 22, 2014
Velocity estimation through homography
• Alternative and more applicable
(not limited to analysis of linear
paths) approach to conversion
between pixel and metric
coordinates
• Requires coordinates of four
points in the analyzed scene

13. C&CA @ ACRI 2014 – Sept. 22, 2014
Tracking with the GMPC approach4 Amir Roshan Zamir, Afshin Dehghan, Mubarak Shah
Human
Detection
GMCP
Tracklet
Generator
GMCP
Trajectory
Generator
Detected HumansInput Video Tracklets Trajectories
Fig. 2. The block diagram of the proposed human tracking method
term. In principle, it’s difficult to model the motion of one person for a long du-
ration without having the knowledge of the destination, structure of the scene,
interactions between people, etc. However, the motion can be modeled suffi-
ciently using constant velocity or acceleration models over a short period of
time. Therefore, the way motion is incorporated into the global data association
process should be di↵erent in short and long terms. This motivated us to employ
the hierarchical approach, i.e. finding tracklets first and then merging them into
full trajectories.
The rest of this section is organized as follows: 2.1 explains the proposed
method for finding tracklets along with an overview of Generalized Minimum
Clique Problem, our global motion-cost model and occlusion handling method.
Merging the tracklets to form global trajectories is explained in 2.2.
Zamir, Dehghan, Shah: GMCP-
Tracker: Global Multi-object
Tracking Using Generalized
Minimum Clique Graphs. ECCV
(2) 2012: 343-356

14. C&CA @ ACRI 2014 – Sept. 22, 2014
Outline
• Crowd studies: towards integrated analysis and synthesis
• Computer vision and crowd studies
• Velocity estimation in naturalistic conditions
• Crowd flow segmentation and counting
• Identification of sources and sinks, towards pedestrian behaviour
understanding

15. C&CA @ ACRI 2014 – Sept. 22, 2014
Crowd flow segmentation and counting

16. C&CA @ ACRI 2014 – Sept. 22, 2014
Outline
• Crowd studies: towards integrated analysis and synthesis
• Computer vision and crowd studies
• Velocity estimation in naturalistic conditions
• Crowd flow segmentation and counting
• Identification of sources and sinks, towards pedestrian
behaviour understanding

17. C&CA @ ACRI 2014 – Sept. 22, 2014
Identification of sources and sinks

18. C&CA @ ACRI 2014 – Sept. 22, 2014
Towards pedestrian behaviour
understanding

19. C&CA @ ACRI 2014 – Sept. 22, 2014
Conclusions
• Modeling and simulation studies
need different types of data for
calibration, validation, but also for
the initial configuration of
plausible simulations
• Computer vision approaches can
offer several solutions to these
requirements and needs
• The jargons, goals, perception of
research challenges is not
necessarily shared...
• ... working together, possibly in
joint projects, can surely improve
the situation and achieved results

20. ありがとうございます。
Giuseppe Vizzari