Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
Crwod_Management.pptx
1. “Engineering Solutions for FIFA World Cup Qatar 2022”
Workshop
Intelligent Crowd Management and Control Systems for FIFA
World Cup Qatar 2022
Presented by: Dr. Yassine Himeur
Professor in Computer Engineering, College of Engineering, Qatar University
2. Crowd counting
People identification
and tracking
Abnormal event
detection
Presentation Plan
II. Introduction
Social distancing
monitoring
III. Principal crowd management tasks
IV. Current challenges
Computation
complexity
Lack of annotated
datasets
Security and privacy
concerns
V. Perspectives and conclusions
I. Team
3. Introduction
Love Parade disaster
(Duisburg, 2010)
Ellis Park Stadium disaster
(Johannesburg, 2001)
The PhilSports Stadium stampede
(Manila, 2006)
21 deaths 21 deaths 73 deaths
Why crowd management in sports venues is important
4. Abnormal event detection
Crowd counting
Facemask detection
Crowd Management Challenges
Social distancing monitoring Security and safety
5. Crowd Management Challenges
• Ensure the safety and smoothness of the
World Cup events due to the inherent
occlusions and density of the crowd
• Rely on deploying cutting-edge technologies,
such as AI, surveillance drones and ICT to
optimize crowd management.
6. Crowd counting
Characteristics
• Capture visitor traffic flows
• Understand occupancy level
• Reduce queuing, at entry points and vending areas
• Executed on AI-based smart cameras or common cameras and
dedicated servers as an edge/cloud solution
Application Environments
• Stadiums.
• Sports events.
• Musical events.
• Massive events.
• Etc.
Dilated and scaled neural networks
7. People identification and tracking
Characteristics
• Consider faces under pose variations using a multi-task CNN
• Manage access control and capacity control
• Identify spectators banned from the facilities
Application Environments
• Stadiums.
• Sports events.
• Musical events.
• Massive events.
• Etc.
8. People identification and tracking
Set
parameters
ResNet-50
Network
training
Masked face dataset
Images
Labels
Pre-processing
+
SVM
Decision tree
EBT
Performance
measurement
Feature extraction Classification
YOLO-v4
YOLO-v4 Tiny
Mask-RCNN
YOLO-v5
Youtube Press
conferences
TV channels ….
Dataset
collection
• Due to Covid-19 pandemic, masks have become an everyday part of our society
• Face recognition has come under scrutiny regarding the accuracy when a
mask is worn
• Maintain a 96% accuracy with individuals wearing masks
and can set up alerts for any persons not wearing a mask
9. Abnormal event detection
Characteristics
• Understand big data
• Providing behavioural analytics
• Avoid annotation of anomalous events in training videos
to reduce the computational cost
Application Environments
• Stadiums.
• Sports events.
• Musical events.
• Massive events.
• Etc.
Goals
• Detect violent spectators
• Prevent hooliganism
10. Social distancing monitoring
Data
collection
Surveillance and monitoring
center
Deep Learning based
analysis
Pedestrian detection
Distance
measurement
Internet
Data center
Characteristics
• Apply AI to detect social distancing
• Create a better fan experience
• Ensure safety is maintained at the same time
Application Environments
• Stadiums.
• Sports events.
• Musical events.
• Massive events.
• Etc.
13. Lack of annotated datasets
Batch sample
𝜃𝑖
𝜏
𝜃𝑖
𝑆
𝜃𝑖
𝑓
𝜃𝑖
𝑏
Neuron linear transformation
Frozen
Update
Target model Predict density map
Target domain
Few shot data
Source domain
Scene 400
Scene 0
𝜃𝑆
𝜃𝜏
1 2
3 Update
4
6
5
Load
Football Supporters Crowd (FSC-Set) Dataset Transfer learning
14. Security and privacy concerns
Benchmarks on diverse CV tasks
Research-oriented
Distributed computing Flexible APIs
FedCV: federated learning frameworks for CV tasks
Personalization
Security/privacy
System efficiency
…
Computer vision applications Federated learning algorithms
CIFAR 10
93%
99%
CIFAR 100
70%
94.5%
ImageNet
80%
88%
Current accuracy in FL
SOTA in centralized ML
How to bridge the research gap for federated learning on Computer vision (CV)?
Can be overcome using:
• Blockchain
• Federated learning
• Edge computing
15. Conclusion
• Fewer Cameras, Higher Definition
• Technology and the Layered Approach
to Stadium Security
• AI is Changing the Way We Secure Stadiums
Editor's Notes
Crowd disasters have taken many human lives. The Love Parade disaster (Duisburg, 2010), the Ellis Park Stadium disaster (Johannesburg, 2001), the PhilSports Stadium stampede (Manila, 2006) are just a few recent examples. Yet, controlling crowds is still an unsolved problem.
Fortunately, smart security cameras based on IoT and drone-based surveillance equipped with AI can help improve both security and operations in sports venues and stadiums. To that end, our ultimate goal, to be achieved in this project, is to develop a system that interacts with the crowd in order to prevent escalation of risk situations into actual disasters.
To achieve that goal, The team directed by Dr. Somaya is actively working on developing intelligent crowd management and control systems, including multiple components for crowd counting, people identification and tracking, abnormal event detection (AED), facemask detection, and social distancing monitoring .
In this respect, Crowd management at the World Cup stadiums and their perimeters is crucial to ensure the safety and smoothness of the World Cup events due to the inherent occlusions and density of the crowd inside and outside the stadiums. FIFA World Cup Qatar 2022 will rely on deploying cutting-edge technologies, such as AI, surveillance drones and ICT to optimize crowd management.
In this direction, Our team has first developed a crowd counting system from drones’ data, which exploits the dilated and scaled neural networks to extract pertinent features and density crowd estimations. Crowd Counting is performed whenever crowds become too dense to detect and localize individual objects with tradition object detectors.
The results can be viewed and analyzed in a dashboard. The data can be exported in many portable formats, enabling users to further analyze the data or use it in presentations and process evaluations.
Our solution can be directly executed on Edge or cloud-based platforms.
The research team's effort has also focused on developing a face recognition system, which considers faces under pose variations using a multitask CNN. This is a robust and agile solution when it comes to managing access control and capacity control and identifying in real time individuals previously banned from the facilities for violent behavior or facilitating the entry of registered users such as VIP members.
On the other hand, because of COVID-19 pandemic, masks have become an everyday part of our society, and therefore, facemask detection has become a challenge.
Also, face recognition has come under scrutiny regarding the accuracy when a mask is worn.
To that end, we developed a new solution that still maintains a 96% accuracy with individuals wearing masks and can set up alerts for any persons not wearing a mask.
Besides, The research team, has developed a novel Abnormal Event Detection scheme, which aims at learning abnormal actions by training both normal and abnormal segments. It enables to avoid the annotation of anomalous events in training video sequences to reduce the computational cost and hence be easily implemented on drones. Concretely, our solution is able to catch instances of fans’ hooliganism and violence.
In addition, the application of artificial intelligence in stadiums and sports venues can help in monitoring social distancing to create a better fan experience, and ensure safety is maintained at the same time.
To that end, we have developed a new solution to monitor social distancing of spectators inside or outside the stadiums using CNN. This helps identify social distancing violations and report them to a surveillance and monitoring center.
Specifically, our approach is based on first detecting spectators or fans using a CNN-based object detector than transforming video frames to a bird’s eye view, where Euclidian distance metric can be used to estimate the distance between detected spectators.
Finally, detected distances are compared with a threshold to detect SD violations.
Despite the progress made by the team, we face different challenges among them is the computational complexity of deep learning models.
That is why, we are trying to develop new light-weight models using knowledge distillation. That means a new student model is developed from the initial complex model with much less complexity while maintaining almost the same performance.
Another issue is the lack of annotated datasets. As you know most deep learning and CNN models are based on supervised learning, so they need large-scale datasets with labels. To overcome this issue, we’ve developed a new Football Supporters Crowd (FSC-Set) dataset which includes 6000 manually annotated images of different types of scenes that contain thousands of people gathering in or around the stadiums.
Also, we are more focusing now on using transfer learning that enables training Deep learning models on synthetic datasets that are easily created with different software then transfer the learned knowledge to real-world datasets.
Other significant issues with most crowd management systems are the security and privacy concerns that raise when processing spectators images.
To close this gap, we plan to use blockchain as it is based on utilizing private and public keys.
Blockchain systems use asymmetric cryptography to secure transactions between users, where each user has a public and private key.
Also, federated learning has been considered as another option, as it helps train algorithms across multiple decentralized edge devices or servers holding local data samples, without exchanging them.
At the end, AI in sports venues and stadiums is not thought of the future, but rather a necessity and goal in today’s post-pandemic world.
A layered approach to video surveillance that combines video management technology with trained staff and access control systems is necessary to help security operators effectively detect and deter suspicious events, avoid crowd disasters and ensure healthy environments both within and outside stadium walls.
