DutchMLSchool 2022 - Smart Safe Stadiums
•
1 like•22 views
This document describes a proposed solution using machine learning and artificial intelligence to help create a safer stadium experience. The solution involves two parts: 1) linking access to stadiums to a verified identity through a fan app for preregistration, and 2) using AI/ML to help detect unwanted behaviors or events early. The rest of the document provides more details on the proposed smart video review framework, including using computer vision and audio analysis techniques to help identify issues like flares, flags, banners, chants including monkey chants. The goal is to help reviewers more efficiently identify potential problems but with privacy, ethics and human oversight.
Recommended
Recommended
More Related Content
Similar to DutchMLSchool 2022 - Smart Safe Stadiums
Similar to DutchMLSchool 2022 - Smart Safe Stadiums (20)
More from BigML, Inc
More from BigML, Inc (20)
Recently uploaded
Recently uploaded (20)
DutchMLSchool 2022 - Smart Safe Stadiums
- 1. July 4 - 6, 2022 2 n d E d i t i o n
- 2. BigML, Inc #DutchMLSchool Smart Safe Stadiums A nice and safe stadium experience for everyone Ramon van Ingen CTIO, Siip group 2 Pablo González Machine Learning Engineer, BigML
- 3. BigML, Inc #DutchMLSchool 3 Agenda • Background • Our solution • Smart Video Review Framework
- 5. BigML, Inc #DutchMLSchool 5 “Our football belongs to everyone” • Attack plan to counter racism and discrimination • Applying smart technology • Aligns with a safer, more secure stadium environment for players, fans, and employees Background
- 6. BigML, Inc #DutchMLSchool 6 A nice and safe stadium experience for everyone Our solution
- 7. BigML, Inc #DutchMLSchool 7 2 innovative solutions • Access right to stadium is linked to a real identity through a fan-app (pre-registration) • Early signalling or detection of unwanted behaviour or events using AI/ML Our solution
- 8. BigML, Inc #DutchMLSchool 8 Market potential • All sport stadiums • Festivals, concerts, and large events • Easy integration with existing technology, systems, and processes in stadiums • Easy connection to ticket-selling agencies Our solution Scalability
- 9. BigML, Inc #DutchMLSchool 9 Privacy and Ethics by Design • Pre-registration: Data is securely stored only on individual’s own device. Only necessary data is shared with consent to attend a game • AI: Use of aggregated metadata, no PII or personal characteristics. Non-discriminatory model that explains what prediction is based on. • Segregation of pre-registration and AI: Human in the loop • Guidance Ethics by University of Applied Science Windesheim Our solution
- 10. BigML, Inc #DutchMLSchool 10 Smart Video Review Framework Focus your attention on what’s important ! x2 x3 ! !
- 11. BigML, Inc #DutchMLSchool 11 The framework: fi nd, group, label, review Find interesting events Flags Chants Group events Experts review clusters and assign their own labels 1 2 Find anomalous events Cluster similar events Smoke bombs / Flares Label event groups 3 4 Assign labels to known events Models learn new labels from expert’s feedback Smart Video Review Framework
- 12. BigML, Inc #DutchMLSchool 12 Avigilon Video Security System The hardware Smart Video Review Framework
- 13. BigML, Inc #DutchMLSchool 13 The software Smart Video Review Framework
- 14. BigML, Inc #DutchMLSchool 14 The problem: We are trying to fi nd black swans The Black Swan: The Impact of the Highly Improbable Nassim Nicholas Taleb Smart Video Review Framework
- 15. BigML, Inc #DutchMLSchool 15 The problem: We are trying to fi nd black swans A methodology that combines supervised and unsupervised learning to capture variance in data Our approach • Our objective is not to replace the reviewer, but to make her work easier • Only the reviewer is able to confidently identify real black swans • We will capture the variance in the data trying to maximize the probability of finding a black swan +3 MORE +10 MORE +6 MORE +8 MORE +5 MORE Reviewer’s feedback +2 MORE Feature extraction Clustering Anomaly Detection Supervised Models Anomalous group. It should be reviewed first Smart Video Review Framework
- 16. BigML, Inc #DutchMLSchool 16 Flare/Smoke Bombs Analysis Binary Classification: Smoke/Flare vs not Detection: Flare Detection Smart Video Review Framework
- 17. BigML, Inc #DutchMLSchool 17 Flags/Banners Analysis Detection: Waving-flag and Banner Clustering and Pattern Matching Smart Video Review Framework
- 18. BigML, Inc #DutchMLSchool 18 Chants Analysis raw microphones data 1 3 seconds sound blocks 2 2D spectrograms 3 Filter out high frequencies and obtain chromagram 4 Smart Video Review Framework
- 19. BigML, Inc #DutchMLSchool 19 Chants Analysis raw microphones data 1 3 seconds sound blocks 2 2D spectrograms 3 Filter out high frequencies and obtain chromagram 4 Chromagram Feature Extraction 5 Chants Clustering and Fingerprinting Matching 7 Chant / Noise Classification 6 Chant 1 Chant 2 Chant 3 Smart Video Review Framework
- 20. BigML, Inc #DutchMLSchool 20 Monkey Chants Analysis raw microphones data 1 3 seconds sound blocks 2 2D spectrograms 3 Filter out high frequencies and obtain MEL Spectrogram 4 monkey chant audio blocks non monkey chant audio blocks Monkey Chant: Periodic sound around 400 Hz BigML Image Classification model Smart Video Review Framework