Successfully reported this slideshow.
Your SlideShare is downloading. ×

“COVID-19 Safe Distancing Measures in Public Spaces with Edge AI,” a Presentation from the Government Technology Agency of Singapore

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 22 Ad

“COVID-19 Safe Distancing Measures in Public Spaces with Edge AI,” a Presentation from the Government Technology Agency of Singapore

Download to read offline

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2022/09/covid-19-safe-distancing-measures-in-public-spaces-with-edge-ai-a-presentation-from-the-government-technology-agency-of-singapore/

Ebi Jose, Senior Systems Engineer at GovTech, the Government Technology Agency of Singapore, presents the “COVID-19 Safe Distancing Measures in Public Spaces with Edge AI” tutorial at the May 2022 Embedded Vision Summit.

Whether in indoor environments, such as supermarkets, museums and offices, or outdoor environments, such as parks, maintaining safe social distancing has been a priority during the COVID-19 pandemic. In this talk, Jose presents GovTech’s work developing cloud-connected edge AI solutions that count the number of people present in outdoor and indoor spaces, providing facility operators with real-time information that allows them to manage spaces to enable social distancing.

Jose provide an overview of the system architecture, hardware, algorithms, software and backend monitoring elements of GovTech solution, highlighting differences in how it addresses indoor vs. outdoor spaces. He also presents results from deployments of GovTech’s solution.

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2022/09/covid-19-safe-distancing-measures-in-public-spaces-with-edge-ai-a-presentation-from-the-government-technology-agency-of-singapore/

Ebi Jose, Senior Systems Engineer at GovTech, the Government Technology Agency of Singapore, presents the “COVID-19 Safe Distancing Measures in Public Spaces with Edge AI” tutorial at the May 2022 Embedded Vision Summit.

Whether in indoor environments, such as supermarkets, museums and offices, or outdoor environments, such as parks, maintaining safe social distancing has been a priority during the COVID-19 pandemic. In this talk, Jose presents GovTech’s work developing cloud-connected edge AI solutions that count the number of people present in outdoor and indoor spaces, providing facility operators with real-time information that allows them to manage spaces to enable social distancing.

Jose provide an overview of the system architecture, hardware, algorithms, software and backend monitoring elements of GovTech solution, highlighting differences in how it addresses indoor vs. outdoor spaces. He also presents results from deployments of GovTech’s solution.

Advertisement
Advertisement

More Related Content

More from Edge AI and Vision Alliance (20)

Advertisement

“COVID-19 Safe Distancing Measures in Public Spaces with Edge AI,” a Presentation from the Government Technology Agency of Singapore

  1. 1. COVID-19 Safe Distancing Measures in Public Spaces with Edge AI Ebi Jose, PhD Senior Systems Engineer Government Technology Agency of Singapore
  2. 2. Government Technology Agency of Singapore (GovTech) • Is a statutory board of the Government of Singapore, under the Prime Minister's Office • 5 Capability Centres, conducting 3 key activities TECHNOLOGY EXPERIMENTATION PRODUCT ENGINEERING & DEVELOPMENT TECHNOLOGY ADVISORY & CONSULTANCY ENGINEERING EXPERTISE in: • Sensors & IoT (SIoT) • Cybersecurity • Data Science & AI • Infrastructure • Applications Design, Development, Deployment • Generate key technologies for Whole of Government • Establish key partnerships with industry players and research institutions • Develop core technologies, standards and guidelines • Provide technical consultation. • Develop POCs and products © 2022 Government Technology Agency of Singapore (GovTech) 2
  3. 3. • Whether it’s indoor environments (supermarkets, museums, etc.) or outdoors (parks), crowd management is a priority. • By combining edge AI solutions with cloud connectivity, government agencies are equipped with information they need to manage the crowd effectively. • Crowd counting deployments on the edge in indoor & outdoor environments are presented. Introduction 3 © 2022 Government Technology Agency of Singapore (GovTech) Camera 1 Camera 2 Camera 3 Indoors – Supermarkets, offices, museums Outdoors - Parks
  4. 4. • Current state-of-the-art methods treat crowd counting as: • Detection-and-counting approach • Density map estimation • Where a deep neural network first produces a 2D crowd density map for a given input image, and… • …estimates total size of the crowd by summing the density values across all spatial locations of the density map. Key Approach in Indoor & Outdoor Visitor Counts 4 © 2022 Government Technology Agency of Singapore (GovTech) • For large crowds, the density map estimation approach is more robust than the detection-then-counting approach • as it is less sensitive to occlusion and clutter. • does not need to commit to binarized decisions at an early stage.
  5. 5. People Counting in Indoor Environments - Hardware 5 © 2022 Government Technology Agency of Singapore (GovTech) Nvidia Jetson Nano Wi-Fi Antenna AC-DC Power supply Camera Data Cable Vent Nvidia Jetson Nano Dev Kit Aaeon BOXER-8223AI NVIDIA Jetson Nano One edge device can process up to 2 cameras.
  6. 6. People Counting in Indoor Environments - Software 6 © 2022 Government Technology Agency of Singapore (GovTech) Version 1 Version 2 Model SSD-Mobilenet v2 PeopleNet Pre-trained dataset MS-COCO Nvidia Proprietary Initial Object Classes 80 3 Object class after transfer learning 1 (i.e. person) 1 (i.e. person) Model optimization TensorRT (FP16) TensorRT (FP16) DeepStream No Yes Tracking Simple Online and Realtime Tracking (SORT) https://github.com/abewley/sort NvDCF tracker Accuracy 86% 91% Software Version Model TensorRT FPS CPU Usage GPU Usage RAM Usage (Gb) Version 1 SSD Mobilenet v2 No 17 49% (avg.) 70% (avg.) 2.9 / 4.0 SSD Mobilenet v2 Yes 23 35% (avg.) 50% (avg.) 2.7 / 4.0 Version 2 Nvidia PeopleNet Yes 25 40% (avg.) 50% (avg.) 2.7 / 4.0
  7. 7. People Counting Indoors 7 © 2022 Government Technology Agency of Singapore (GovTech) • Transfer learning with custom dataset can achieve detection accuracy > 91%. • (Almost) a solved problem!
  8. 8. Crowd Counting in Outdoor Environments
  9. 9. • A challenging computer vision problem due to: • Heavy occlusion • Perspective distortion • Scale variation • Diverse crowd distribution Crowd Counting in Outdoors 9 © 2022 Government Technology Agency of Singapore (GovTech)
  10. 10. • Crowd counting algorithms predict a density map from a crowd image. Summation of the density map is the crowd count. • Each training image contains multiple people, each person is annotated by a dot. • MSFANet crowd counting method • Combination of multi-scale-aware modules and dual path decoder • Predicts density maps and attention maps for highlighting crowd regions in input image. • Uses a Gaussian kernel to smooth each annotated dot. • Is trained on L2 pixel-wise loss. • Sensitive to the choice of variance in the Gaussian kernel. • DM-Count crowd counting method • Considers density maps and dot maps as probability distributions. • Loss composed of optimal transport (OT) loss, total variation (TV) pixel-wise loss & counting loss (CL). Crowd Density Methods – MSFANet & DM-Count 10 © 2022 Government Technology Agency of Singapore (GovTech)
  11. 11. Multi-Scale Feature Adaptive Network (M-SFANet) – Data Preprocessing 11 © 2022 Government Technology Agency of Singapore (GovTech) Convolve the head annotation with Gaussian kernel (G) which has fixed standard deviation (σ). Assuming a head annotation at pixel 𝑥𝑖, represented as 𝛿 𝑥 − 𝑥𝑖 , density map, 𝐷 𝑥 = ෍ 𝑖=1 𝑁 𝛿 𝑥 − 𝑥𝑖 ∗ 𝐺𝜎 𝑥 where N = total number of headcounts. attention map, 𝐴(𝑖) = ቊ 0 = 0.001 > 𝐷(𝑖) 1 = 0.001 ≤ 𝐷(𝑖)
  12. 12. M-SFANet Architecture (1) 12 © 2022 Government Technology Agency of Singapore (GovTech) Atrous spatial pyramid pooling (ASPP) with augmented image-level features
  13. 13. M-SFANet Architecture (2) 13 © 2022 Government Technology Agency of Singapore (GovTech) Context-aware module (CAN)
  14. 14. Crowd Counting in Outdoors – DM Count Estimation 14 © 2022 Government Technology Agency of Singapore (GovTech) Input Image Predicted Density Map Annotations (Ground Truth Count = 54) VGG19 model • Almost all previous work converts the sparse point annotation into dense ground truth maps. • Each point is replaced by a gaussian blob. • DM-Count paper shows that imposing Gaussians to annotations hurts generalization performance. How to measure discrepancy for training?
  15. 15. • Counting loss (𝑙𝐶): absolute loss between ground truth and predicted counts • The Optimal Transport loss (𝑙𝑂𝑇): The OT similarity measurements provide valid gradients that can train a network if the source distribution does not overlap with the target distribution. • Total Variation loss (𝑙𝑇𝑉): Improve stability when optimising the OT loss with the Sinkhorn algorithm. DM-Count Loss 15 © 2022 Government Technology Agency of Singapore (GovTech) Total loss, 𝑙 𝒀, ෡ 𝒀 = 𝑙𝐶 𝒀, ෡ 𝒀 + 𝜆1𝑙𝑂𝑇 𝒀, ෡ 𝒀 + 𝜆2| 𝒀 |1𝑙𝑇𝑉(𝒀, ෡ 𝒀) • DM-Count considers density map as distributions. • Use of Optimal Transport (OT) to compute the distance between density maps. Sparse annotations Target distribution Predicted density map Source distribution Distribution distance
  16. 16. People Counting Outdoors - Workflow 16 © 2022 Government Technology Agency of Singapore (GovTech) Training • Google cloud, used 1 K80 GPU • Max epochs – 500 • Learning rate – 1e-05 • Batch size - 10 • Weight decay – 0.0001
  17. 17. People Counting Outdoors - Cloud Backend and UI 17 © 2022 Government Technology Agency of Singapore (GovTech) User Interface Cloud Backend
  18. 18. Crowd Counting Outdoors – Hardware & Software 18 © 2022 Government Technology Agency of Singapore (GovTech) AAEON Boxer-8253AI • Nvidia Jetson Xavier NX • 8GB LPDDR4x • Storage option: 16GB eMMC and 64GB MicroSD • GbE PoE/PSE LAN x 2 • Quectel EP06: LTE-A Cat 6 module with Mini PCIe form factor Pytorch Model on Nvidia Jetson Xavier NX TensorRT (FP-16) on Nvidia Jetson Xavier NX Model Accuracy (%) Latency (sec) DM-Count 92.10 3.467 M-SFANet 88.99 8.841 Model Accuracy (%) Latency (sec) DM-Count 91.08 0.380 M-SFANet 86.46 0.637 Model DM-Count MSFANet Dataset UCF QNRF, Custom UCF QNRF, Custom Framework Pytorch 1.2 Pytorch 1.2 Model optimization TensorRT (FP16) TensorRT (FP16)
  19. 19. Crowd Counting Outdoors – Results 19 2022 Government Technology Agency of Singapore (GovTech)
  20. 20. Crowd Counting in Outdoors – DM-Count Results 20 © 2022 Government Technology Agency of Singapore (GovTech)
  21. 21. • Accurate people counting: • For indoor environments, is almost a solved problem, when there are less/no occlusions. • For outdoor environments, is a challenging problem due to: • Heavy occlusion • Perspective distortion • Scale variation • Diverse crowd distribution • DM-Count performed well even with occlusions, providing accurate counts. Summary 21 © 2022 Government Technology Agency of Singapore (GovTech)
  22. 22. Resources 22 © 2022 Government Technology Agency of Singapore (GovTech) Sensors & IoT Department https://www.tech.gov.sg/capability-centre-siot Government Technology Agency of Singapore (GovTech) https://www.tech.gov.sg M-SFANet for Count Counting (original paper and code) https://arxiv.org/abs/2003.05586 https://github.com/Pongpisit-Thanasutives/Variations-of-SFANet-for-Crowd-Counting Distribution Matching for Count Counting (original paper and code) https://arxiv.org/abs/2009.13077 https://github.com/cvlab-stonybrook/DM-Count

×