SlideShare a Scribd company logo
PERFORMANCE EVALUATION OF VIDEO ANALYTICS
FOR SURVEILLANCE ON-BOARD TRAINS
Context and Objectives
ACIVS’13, October 30th 2013, Poznan, Poland
Low level Evaluation
Real-time video surveillance is used in public transportation, including on-board metro
trains, to support human operators in control rooms.
Video Content Analytics (VCA) is effective when the performance is such to reduce
false alarms under appropriate acceptability thresholds.
Reducing the causes of false alarms by fine tuning the VCA is particularly important in
on-board applications, that are more prone to disturbances with respect to fixed
installations due to more critical operating conditions (e.g. constrained camera
field of view, frequent light changes, vibrations, occlusions, etc.).
Accurate performance evaluation is essential to decide about:
 Which alarms to activate depending on scenarios
 How to set algorithm parameters in each scenario
Comparison between commercial and open-source implementations is also important
to evaluate the added value provided by the former in terms of achievable
performance.
Black Box Evaluation
Valentina Casola, Mariana Esposito, Francesco Flammini, Nicola Mazzocca, Concetta Pragliola
Contributions
Evaluation of Commercial Off-The-Shelf (COTS) VCA system
Low Level Black Box
Alarm Performance Evaluation
Comparison with Open source
System
Frame and Object Based Metrics
Ground Truth values compared with
Algorithm Result
Computed by specific tool developed in
Matlab
Object Based Metrics
Consider the whole trajectory of each
object in the scene and preserves its
lifetime
Correct Detected Track
(CDT)
Track Detection Failure
(TDF)
False Alarm Track
(FAT)
Track Fragmentation
(TF)
ID Change (IDC)
Temporal Overlap
Spatial Overlap
Performance Evaluation ToolAlgorithm Result
Ground Thruth
Tot Tos
Metrics
(15%)
Frame Based Metrics
Measure the performance on individual
frames of a video stream and do not
consider the preservatio. The blobs are
evaluated in their size and location and
compared with GT.
True Positive (TP)
False Negative (FN):
Fragmentation
ecall
False Positive (FP)
Merging
Precision
0
50
100
150
200
250
300
350
400
Occurrencies
#AR TP FN FP TF IDC
Object-Based Results
COTS
Open-Source
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0,9
1
PerformanceIndex
Accuracy FAR PP FNR
Object-Based Performance
COTS
Open-Source
False Alarm Rate FPTP
FP

FAR
Detection rate
FNTP
TP

DR
Positive Prediction
FPTP
TP

PP
False Negative Rate TPFN
FN

FNR
Fragmentation Index FRAGM
TP
FI
Merge Index MERGE
FPOBJAR 
FM
♦ 1510 objects in 600 frames, 8 GT tracks
♦ COTS system detects less objects but
it is far more reliable regarding false
positives (84 vs 2168)
♦ FA of 6% instead of 41% (the latter
clearly unacceptable).
♦ COTS object Fragmentation is less
then one half w.r.t. Open source.
Metrics COTS Open
TP 1307 1324
FN 203 186
FP 84 2168
Metrics COTS Open
TP 7 8
FN 1 0
FP 9 213
TF 11 17
IDC 0 0
♦ 1510 objects in 600 frames, 8 GT tracks
♦ COTS FP and PP reveal higher reliability
♦ Same consideration for TF (important
for event persistence).
Performance Indices
Test of COTS system on real
vehicle for light railwails and
tramways
False Alarm Index
RealAlarms#
sFalseAlarm#
FAI 
Probability Of Detection
PDTPFN
TP

POD
over it detects camera occlusionblurring with paint or obstacles.
amper it detects the manumission of the camera by moving it.
top it detects still objects in the scene.
Presence it detects objectspeople moving in the scene
rowd it raises an alarm when the scene is overcrowded.
ALARMS

More Related Content

Similar to ACIVS 2013 conference poster presentation

Real-time Bangla License Plate Recognition System for Low Resource Video-base...
Real-time Bangla License Plate Recognition System for Low Resource Video-base...Real-time Bangla License Plate Recognition System for Low Resource Video-base...
Real-time Bangla License Plate Recognition System for Low Resource Video-base...
MD Abdullah Al Nasim
 
Towards Robust and Safe Autonomous Drones
Towards Robust and Safe Autonomous DronesTowards Robust and Safe Autonomous Drones
Towards Robust and Safe Autonomous Drones
SERENEWorkshop
 
IRJET- Robust and Fast Detection of Moving Vechiles in Aerial Videos usin...
IRJET-  	  Robust and Fast Detection of Moving Vechiles in Aerial Videos usin...IRJET-  	  Robust and Fast Detection of Moving Vechiles in Aerial Videos usin...
IRJET- Robust and Fast Detection of Moving Vechiles in Aerial Videos usin...
IRJET Journal
 
ACIVS'12: Evaluating the effects of MJPEG compression on Motion Tracking in m...
ACIVS'12: Evaluating the effects of MJPEG compression on Motion Tracking in m...ACIVS'12: Evaluating the effects of MJPEG compression on Motion Tracking in m...
ACIVS'12: Evaluating the effects of MJPEG compression on Motion Tracking in m...
Francesco Flammini
 
ACIVS'12 Presentation by Francesco Flammini
ACIVS'12 Presentation by Francesco FlamminiACIVS'12 Presentation by Francesco Flammini
ACIVS'12 Presentation by Francesco Flammini
Francesco Flammini
 
niyantra 2014
niyantra 2014niyantra 2014
niyantra 2014
Nitish Goel
 
COIFLET-BASED FUZZY-CLASSIFIER FOR DEFECT DETECTION IN INDUSTRIAL LNG/LPG TANKS
COIFLET-BASED FUZZY-CLASSIFIER FOR DEFECT DETECTION IN INDUSTRIAL LNG/LPG TANKSCOIFLET-BASED FUZZY-CLASSIFIER FOR DEFECT DETECTION IN INDUSTRIAL LNG/LPG TANKS
COIFLET-BASED FUZZY-CLASSIFIER FOR DEFECT DETECTION IN INDUSTRIAL LNG/LPG TANKS
csandit
 
IRJET- Front View Identification of Vehicles by using Machine Learning Te...
IRJET-  	  Front View Identification of Vehicles by using Machine Learning Te...IRJET-  	  Front View Identification of Vehicles by using Machine Learning Te...
IRJET- Front View Identification of Vehicles by using Machine Learning Te...
IRJET Journal
 
IRJET- Features Extraction OCR Algorithm in Indian License Plates
IRJET- Features Extraction OCR Algorithm in Indian License PlatesIRJET- Features Extraction OCR Algorithm in Indian License Plates
IRJET- Features Extraction OCR Algorithm in Indian License Plates
IRJET Journal
 
Camera Encoded Phased Array for Semi-Automated Inspection of Complex Composit...
Camera Encoded Phased Array for Semi-Automated Inspection of Complex Composit...Camera Encoded Phased Array for Semi-Automated Inspection of Complex Composit...
Camera Encoded Phased Array for Semi-Automated Inspection of Complex Composit...
Innerspec Technologies
 
An acoustic approach for multiple fault diagnosis in motorcycles
An acoustic approach for multiple fault diagnosis in motorcyclesAn acoustic approach for multiple fault diagnosis in motorcycles
An acoustic approach for multiple fault diagnosis in motorcycles
Ramesh Wadawadagi
 
AnaVANET: an experiment and visualization tool for vehicular networks
AnaVANET: an experiment and visualization tool for vehicular networksAnaVANET: an experiment and visualization tool for vehicular networks
AnaVANET: an experiment and visualization tool for vehicular networks
Manabu Tsukada
 
CAdViSE or how to find the sweet spots of ABR systems
CAdViSE or how to find the sweet spots of ABR systemsCAdViSE or how to find the sweet spots of ABR systems
CAdViSE or how to find the sweet spots of ABR systems
Minh Nguyen
 
IRJET- Design the Surveillance Algorithm and Motion Detection of Objects for ...
IRJET- Design the Surveillance Algorithm and Motion Detection of Objects for ...IRJET- Design the Surveillance Algorithm and Motion Detection of Objects for ...
IRJET- Design the Surveillance Algorithm and Motion Detection of Objects for ...
IRJET Journal
 
Webinar | HIL-based Wide-area Monitoring, Protection and Control R&D and Testing
Webinar | HIL-based Wide-area Monitoring, Protection and Control R&D and TestingWebinar | HIL-based Wide-area Monitoring, Protection and Control R&D and Testing
Webinar | HIL-based Wide-area Monitoring, Protection and Control R&D and Testing
OPAL-RT TECHNOLOGIES
 
5b50dc69-4ca7-41ee-a9dd-b4e8b220b4fe.pdf
5b50dc69-4ca7-41ee-a9dd-b4e8b220b4fe.pdf5b50dc69-4ca7-41ee-a9dd-b4e8b220b4fe.pdf
5b50dc69-4ca7-41ee-a9dd-b4e8b220b4fe.pdf
SrinidhirkGowda
 
iOLM | iolm | Intelligent Optical Link Mapper | iolm spec sheet | iolm exfo...
iOLM | iolm | Intelligent Optical Link Mapper |  iolm spec sheet |  iolm exfo...iOLM | iolm | Intelligent Optical Link Mapper |  iolm spec sheet |  iolm exfo...
iOLM | iolm | Intelligent Optical Link Mapper | iolm spec sheet | iolm exfo...
INVAS Technologies Pvt. Ltd.
 
IRJET - Military Spy Robot with Intelligentdestruction
IRJET - Military Spy Robot with IntelligentdestructionIRJET - Military Spy Robot with Intelligentdestruction
IRJET - Military Spy Robot with Intelligentdestruction
IRJET Journal
 
OBSERVATIONAL DISCRETE LINES FOR THE DETECTION OF MOVING VEHICLES IN ROAD TRA...
OBSERVATIONAL DISCRETE LINES FOR THE DETECTION OF MOVING VEHICLES IN ROAD TRA...OBSERVATIONAL DISCRETE LINES FOR THE DETECTION OF MOVING VEHICLES IN ROAD TRA...
OBSERVATIONAL DISCRETE LINES FOR THE DETECTION OF MOVING VEHICLES IN ROAD TRA...
ijcseit
 
OBSERVATIONAL DISCRETE LINES FOR THE DETECTION OF MOVING VEHICLES IN ROAD TRA...
OBSERVATIONAL DISCRETE LINES FOR THE DETECTION OF MOVING VEHICLES IN ROAD TRA...OBSERVATIONAL DISCRETE LINES FOR THE DETECTION OF MOVING VEHICLES IN ROAD TRA...
OBSERVATIONAL DISCRETE LINES FOR THE DETECTION OF MOVING VEHICLES IN ROAD TRA...
ijcseit
 

Similar to ACIVS 2013 conference poster presentation (20)

Real-time Bangla License Plate Recognition System for Low Resource Video-base...
Real-time Bangla License Plate Recognition System for Low Resource Video-base...Real-time Bangla License Plate Recognition System for Low Resource Video-base...
Real-time Bangla License Plate Recognition System for Low Resource Video-base...
 
Towards Robust and Safe Autonomous Drones
Towards Robust and Safe Autonomous DronesTowards Robust and Safe Autonomous Drones
Towards Robust and Safe Autonomous Drones
 
IRJET- Robust and Fast Detection of Moving Vechiles in Aerial Videos usin...
IRJET-  	  Robust and Fast Detection of Moving Vechiles in Aerial Videos usin...IRJET-  	  Robust and Fast Detection of Moving Vechiles in Aerial Videos usin...
IRJET- Robust and Fast Detection of Moving Vechiles in Aerial Videos usin...
 
ACIVS'12: Evaluating the effects of MJPEG compression on Motion Tracking in m...
ACIVS'12: Evaluating the effects of MJPEG compression on Motion Tracking in m...ACIVS'12: Evaluating the effects of MJPEG compression on Motion Tracking in m...
ACIVS'12: Evaluating the effects of MJPEG compression on Motion Tracking in m...
 
ACIVS'12 Presentation by Francesco Flammini
ACIVS'12 Presentation by Francesco FlamminiACIVS'12 Presentation by Francesco Flammini
ACIVS'12 Presentation by Francesco Flammini
 
niyantra 2014
niyantra 2014niyantra 2014
niyantra 2014
 
COIFLET-BASED FUZZY-CLASSIFIER FOR DEFECT DETECTION IN INDUSTRIAL LNG/LPG TANKS
COIFLET-BASED FUZZY-CLASSIFIER FOR DEFECT DETECTION IN INDUSTRIAL LNG/LPG TANKSCOIFLET-BASED FUZZY-CLASSIFIER FOR DEFECT DETECTION IN INDUSTRIAL LNG/LPG TANKS
COIFLET-BASED FUZZY-CLASSIFIER FOR DEFECT DETECTION IN INDUSTRIAL LNG/LPG TANKS
 
IRJET- Front View Identification of Vehicles by using Machine Learning Te...
IRJET-  	  Front View Identification of Vehicles by using Machine Learning Te...IRJET-  	  Front View Identification of Vehicles by using Machine Learning Te...
IRJET- Front View Identification of Vehicles by using Machine Learning Te...
 
IRJET- Features Extraction OCR Algorithm in Indian License Plates
IRJET- Features Extraction OCR Algorithm in Indian License PlatesIRJET- Features Extraction OCR Algorithm in Indian License Plates
IRJET- Features Extraction OCR Algorithm in Indian License Plates
 
Camera Encoded Phased Array for Semi-Automated Inspection of Complex Composit...
Camera Encoded Phased Array for Semi-Automated Inspection of Complex Composit...Camera Encoded Phased Array for Semi-Automated Inspection of Complex Composit...
Camera Encoded Phased Array for Semi-Automated Inspection of Complex Composit...
 
An acoustic approach for multiple fault diagnosis in motorcycles
An acoustic approach for multiple fault diagnosis in motorcyclesAn acoustic approach for multiple fault diagnosis in motorcycles
An acoustic approach for multiple fault diagnosis in motorcycles
 
AnaVANET: an experiment and visualization tool for vehicular networks
AnaVANET: an experiment and visualization tool for vehicular networksAnaVANET: an experiment and visualization tool for vehicular networks
AnaVANET: an experiment and visualization tool for vehicular networks
 
CAdViSE or how to find the sweet spots of ABR systems
CAdViSE or how to find the sweet spots of ABR systemsCAdViSE or how to find the sweet spots of ABR systems
CAdViSE or how to find the sweet spots of ABR systems
 
IRJET- Design the Surveillance Algorithm and Motion Detection of Objects for ...
IRJET- Design the Surveillance Algorithm and Motion Detection of Objects for ...IRJET- Design the Surveillance Algorithm and Motion Detection of Objects for ...
IRJET- Design the Surveillance Algorithm and Motion Detection of Objects for ...
 
Webinar | HIL-based Wide-area Monitoring, Protection and Control R&D and Testing
Webinar | HIL-based Wide-area Monitoring, Protection and Control R&D and TestingWebinar | HIL-based Wide-area Monitoring, Protection and Control R&D and Testing
Webinar | HIL-based Wide-area Monitoring, Protection and Control R&D and Testing
 
5b50dc69-4ca7-41ee-a9dd-b4e8b220b4fe.pdf
5b50dc69-4ca7-41ee-a9dd-b4e8b220b4fe.pdf5b50dc69-4ca7-41ee-a9dd-b4e8b220b4fe.pdf
5b50dc69-4ca7-41ee-a9dd-b4e8b220b4fe.pdf
 
iOLM | iolm | Intelligent Optical Link Mapper | iolm spec sheet | iolm exfo...
iOLM | iolm | Intelligent Optical Link Mapper |  iolm spec sheet |  iolm exfo...iOLM | iolm | Intelligent Optical Link Mapper |  iolm spec sheet |  iolm exfo...
iOLM | iolm | Intelligent Optical Link Mapper | iolm spec sheet | iolm exfo...
 
IRJET - Military Spy Robot with Intelligentdestruction
IRJET - Military Spy Robot with IntelligentdestructionIRJET - Military Spy Robot with Intelligentdestruction
IRJET - Military Spy Robot with Intelligentdestruction
 
OBSERVATIONAL DISCRETE LINES FOR THE DETECTION OF MOVING VEHICLES IN ROAD TRA...
OBSERVATIONAL DISCRETE LINES FOR THE DETECTION OF MOVING VEHICLES IN ROAD TRA...OBSERVATIONAL DISCRETE LINES FOR THE DETECTION OF MOVING VEHICLES IN ROAD TRA...
OBSERVATIONAL DISCRETE LINES FOR THE DETECTION OF MOVING VEHICLES IN ROAD TRA...
 
OBSERVATIONAL DISCRETE LINES FOR THE DETECTION OF MOVING VEHICLES IN ROAD TRA...
OBSERVATIONAL DISCRETE LINES FOR THE DETECTION OF MOVING VEHICLES IN ROAD TRA...OBSERVATIONAL DISCRETE LINES FOR THE DETECTION OF MOVING VEHICLES IN ROAD TRA...
OBSERVATIONAL DISCRETE LINES FOR THE DETECTION OF MOVING VEHICLES IN ROAD TRA...
 

More from Francesco Flammini

TRA Visions Senior Researcher Award 2024.pdf
TRA Visions Senior Researcher Award 2024.pdfTRA Visions Senior Researcher Award 2024.pdf
TRA Visions Senior Researcher Award 2024.pdf
Francesco Flammini
 
RAILS - Railway Gazette 2023 (Flammini).pdf
RAILS - Railway Gazette 2023 (Flammini).pdfRAILS - Railway Gazette 2023 (Flammini).pdf
RAILS - Railway Gazette 2023 (Flammini).pdf
Francesco Flammini
 
Award-Fedcsis Conference 2023.pdf
Award-Fedcsis Conference 2023.pdfAward-Fedcsis Conference 2023.pdf
Award-Fedcsis Conference 2023.pdf
Francesco Flammini
 
Internationalization of the Curriculum Conference Certificate
Internationalization of the Curriculum Conference CertificateInternationalization of the Curriculum Conference Certificate
Internationalization of the Curriculum Conference Certificate
Francesco Flammini
 
IEEE SMC TCHS Award Ceremony at IEEE CSR conference 2021
IEEE SMC TCHS Award Ceremony at IEEE CSR conference 2021IEEE SMC TCHS Award Ceremony at IEEE CSR conference 2021
IEEE SMC TCHS Award Ceremony at IEEE CSR conference 2021
Francesco Flammini
 
Appreciation Certificate
Appreciation CertificateAppreciation Certificate
Appreciation Certificate
Francesco Flammini
 
Digital Twins for Trustworthy Autonomy
Digital Twins for Trustworthy AutonomyDigital Twins for Trustworthy Autonomy
Digital Twins for Trustworthy Autonomy
Francesco Flammini
 
IEEE GCAIIoT 2020 certificate
IEEE GCAIIoT 2020 certificateIEEE GCAIIoT 2020 certificate
IEEE GCAIIoT 2020 certificate
Francesco Flammini
 
IEEE Intelligent Transportation Systems Conference 2020 - Low-Power Wide-Area...
IEEE Intelligent Transportation Systems Conference 2020 - Low-Power Wide-Area...IEEE Intelligent Transportation Systems Conference 2020 - Low-Power Wide-Area...
IEEE Intelligent Transportation Systems Conference 2020 - Low-Power Wide-Area...
Francesco Flammini
 
Science of Computer Programming, Certificate of Reviewing
Science of Computer Programming, Certificate of ReviewingScience of Computer Programming, Certificate of Reviewing
Science of Computer Programming, Certificate of Reviewing
Francesco Flammini
 
ACM DSP appointment letter
ACM DSP appointment letter ACM DSP appointment letter
ACM DSP appointment letter
Francesco Flammini
 
“AI techniques in cyber-security applications”. Flammini lnu susec19
“AI techniques in cyber-security applications”. Flammini lnu susec19“AI techniques in cyber-security applications”. Flammini lnu susec19
“AI techniques in cyber-security applications”. Flammini lnu susec19
Francesco Flammini
 
Ansaldo STS Innovation award 2009
Ansaldo STS Innovation award 2009Ansaldo STS Innovation award 2009
Ansaldo STS Innovation award 2009
Francesco Flammini
 
Ansaldo STS Innovation Award 2014 - Francesco Flammini
Ansaldo STS Innovation Award 2014 - Francesco FlamminiAnsaldo STS Innovation Award 2014 - Francesco Flammini
Ansaldo STS Innovation Award 2014 - Francesco Flammini
Francesco Flammini
 
IoT Research & Education at LNU
IoT Research & Education at LNUIoT Research & Education at LNU
IoT Research & Education at LNU
Francesco Flammini
 
Francesco Flammini - talk at DISCORAIL'19
Francesco Flammini - talk at DISCORAIL'19Francesco Flammini - talk at DISCORAIL'19
Francesco Flammini - talk at DISCORAIL'19
Francesco Flammini
 
Towards railway virtual coupling
Towards railway virtual couplingTowards railway virtual coupling
Towards railway virtual coupling
Francesco Flammini
 
Smart-Troubleshooting Symbiotic Autonomous Systems in the Connected Society
Smart-Troubleshooting Symbiotic Autonomous Systems in the Connected SocietySmart-Troubleshooting Symbiotic Autonomous Systems in the Connected Society
Smart-Troubleshooting Symbiotic Autonomous Systems in the Connected Society
Francesco Flammini
 
SuccesfulGrantApplications_Jan 23, 2017_certificate-506560
SuccesfulGrantApplications_Jan 23, 2017_certificate-506560SuccesfulGrantApplications_Jan 23, 2017_certificate-506560
SuccesfulGrantApplications_Jan 23, 2017_certificate-506560
Francesco Flammini
 

More from Francesco Flammini (20)

TRA Visions Senior Researcher Award 2024.pdf
TRA Visions Senior Researcher Award 2024.pdfTRA Visions Senior Researcher Award 2024.pdf
TRA Visions Senior Researcher Award 2024.pdf
 
RAILS - Railway Gazette 2023 (Flammini).pdf
RAILS - Railway Gazette 2023 (Flammini).pdfRAILS - Railway Gazette 2023 (Flammini).pdf
RAILS - Railway Gazette 2023 (Flammini).pdf
 
Award-Fedcsis Conference 2023.pdf
Award-Fedcsis Conference 2023.pdfAward-Fedcsis Conference 2023.pdf
Award-Fedcsis Conference 2023.pdf
 
Internationalization of the Curriculum Conference Certificate
Internationalization of the Curriculum Conference CertificateInternationalization of the Curriculum Conference Certificate
Internationalization of the Curriculum Conference Certificate
 
IEEE SMC TCHS Award Ceremony at IEEE CSR conference 2021
IEEE SMC TCHS Award Ceremony at IEEE CSR conference 2021IEEE SMC TCHS Award Ceremony at IEEE CSR conference 2021
IEEE SMC TCHS Award Ceremony at IEEE CSR conference 2021
 
Appreciation Certificate
Appreciation CertificateAppreciation Certificate
Appreciation Certificate
 
Digital Twins for Trustworthy Autonomy
Digital Twins for Trustworthy AutonomyDigital Twins for Trustworthy Autonomy
Digital Twins for Trustworthy Autonomy
 
IEEE GCAIIoT 2020 certificate
IEEE GCAIIoT 2020 certificateIEEE GCAIIoT 2020 certificate
IEEE GCAIIoT 2020 certificate
 
IEEE Intelligent Transportation Systems Conference 2020 - Low-Power Wide-Area...
IEEE Intelligent Transportation Systems Conference 2020 - Low-Power Wide-Area...IEEE Intelligent Transportation Systems Conference 2020 - Low-Power Wide-Area...
IEEE Intelligent Transportation Systems Conference 2020 - Low-Power Wide-Area...
 
Science of Computer Programming, Certificate of Reviewing
Science of Computer Programming, Certificate of ReviewingScience of Computer Programming, Certificate of Reviewing
Science of Computer Programming, Certificate of Reviewing
 
ACM DSP appointment letter
ACM DSP appointment letter ACM DSP appointment letter
ACM DSP appointment letter
 
“AI techniques in cyber-security applications”. Flammini lnu susec19
“AI techniques in cyber-security applications”. Flammini lnu susec19“AI techniques in cyber-security applications”. Flammini lnu susec19
“AI techniques in cyber-security applications”. Flammini lnu susec19
 
Ansaldo STS Innovation award 2009
Ansaldo STS Innovation award 2009Ansaldo STS Innovation award 2009
Ansaldo STS Innovation award 2009
 
Ansaldo STS Innovation Award 2014 - Francesco Flammini
Ansaldo STS Innovation Award 2014 - Francesco FlamminiAnsaldo STS Innovation Award 2014 - Francesco Flammini
Ansaldo STS Innovation Award 2014 - Francesco Flammini
 
IoT Research & Education at LNU
IoT Research & Education at LNUIoT Research & Education at LNU
IoT Research & Education at LNU
 
Francesco Flammini - talk at DISCORAIL'19
Francesco Flammini - talk at DISCORAIL'19Francesco Flammini - talk at DISCORAIL'19
Francesco Flammini - talk at DISCORAIL'19
 
Towards railway virtual coupling
Towards railway virtual couplingTowards railway virtual coupling
Towards railway virtual coupling
 
Smart-Troubleshooting Symbiotic Autonomous Systems in the Connected Society
Smart-Troubleshooting Symbiotic Autonomous Systems in the Connected SocietySmart-Troubleshooting Symbiotic Autonomous Systems in the Connected Society
Smart-Troubleshooting Symbiotic Autonomous Systems in the Connected Society
 
SuccesfulGrantApplications_Jan 23, 2017_certificate-506560
SuccesfulGrantApplications_Jan 23, 2017_certificate-506560SuccesfulGrantApplications_Jan 23, 2017_certificate-506560
SuccesfulGrantApplications_Jan 23, 2017_certificate-506560
 
Attestato FLAMMINI ISO 14298
Attestato FLAMMINI ISO 14298Attestato FLAMMINI ISO 14298
Attestato FLAMMINI ISO 14298
 

Recently uploaded

BPV-GUI-01-Guide-for-ASME-Review-Teams-(General)-10-10-2023.pdf
BPV-GUI-01-Guide-for-ASME-Review-Teams-(General)-10-10-2023.pdfBPV-GUI-01-Guide-for-ASME-Review-Teams-(General)-10-10-2023.pdf
BPV-GUI-01-Guide-for-ASME-Review-Teams-(General)-10-10-2023.pdf
MIGUELANGEL966976
 
Engineering Drawings Lecture Detail Drawings 2014.pdf
Engineering Drawings Lecture Detail Drawings 2014.pdfEngineering Drawings Lecture Detail Drawings 2014.pdf
Engineering Drawings Lecture Detail Drawings 2014.pdf
abbyasa1014
 
Advanced control scheme of doubly fed induction generator for wind turbine us...
Advanced control scheme of doubly fed induction generator for wind turbine us...Advanced control scheme of doubly fed induction generator for wind turbine us...
Advanced control scheme of doubly fed induction generator for wind turbine us...
IJECEIAES
 
Generative AI leverages algorithms to create various forms of content
Generative AI leverages algorithms to create various forms of contentGenerative AI leverages algorithms to create various forms of content
Generative AI leverages algorithms to create various forms of content
Hitesh Mohapatra
 
Eric Nizeyimana's document 2006 from gicumbi to ttc nyamata handball play
Eric Nizeyimana's document 2006 from gicumbi to ttc nyamata handball playEric Nizeyimana's document 2006 from gicumbi to ttc nyamata handball play
Eric Nizeyimana's document 2006 from gicumbi to ttc nyamata handball play
enizeyimana36
 
Engine Lubrication performance System.pdf
Engine Lubrication performance System.pdfEngine Lubrication performance System.pdf
Engine Lubrication performance System.pdf
mamamaam477
 
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
IJECEIAES
 
Computational Engineering IITH Presentation
Computational Engineering IITH PresentationComputational Engineering IITH Presentation
Computational Engineering IITH Presentation
co23btech11018
 
TIME DIVISION MULTIPLEXING TECHNIQUE FOR COMMUNICATION SYSTEM
TIME DIVISION MULTIPLEXING TECHNIQUE FOR COMMUNICATION SYSTEMTIME DIVISION MULTIPLEXING TECHNIQUE FOR COMMUNICATION SYSTEM
TIME DIVISION MULTIPLEXING TECHNIQUE FOR COMMUNICATION SYSTEM
HODECEDSIET
 
CHINA’S GEO-ECONOMIC OUTREACH IN CENTRAL ASIAN COUNTRIES AND FUTURE PROSPECT
CHINA’S GEO-ECONOMIC OUTREACH IN CENTRAL ASIAN COUNTRIES AND FUTURE PROSPECTCHINA’S GEO-ECONOMIC OUTREACH IN CENTRAL ASIAN COUNTRIES AND FUTURE PROSPECT
CHINA’S GEO-ECONOMIC OUTREACH IN CENTRAL ASIAN COUNTRIES AND FUTURE PROSPECT
jpsjournal1
 
Unit-III-ELECTROCHEMICAL STORAGE DEVICES.ppt
Unit-III-ELECTROCHEMICAL STORAGE DEVICES.pptUnit-III-ELECTROCHEMICAL STORAGE DEVICES.ppt
Unit-III-ELECTROCHEMICAL STORAGE DEVICES.ppt
KrishnaveniKrishnara1
 
A review on techniques and modelling methodologies used for checking electrom...
A review on techniques and modelling methodologies used for checking electrom...A review on techniques and modelling methodologies used for checking electrom...
A review on techniques and modelling methodologies used for checking electrom...
nooriasukmaningtyas
 
Comparative analysis between traditional aquaponics and reconstructed aquapon...
Comparative analysis between traditional aquaponics and reconstructed aquapon...Comparative analysis between traditional aquaponics and reconstructed aquapon...
Comparative analysis between traditional aquaponics and reconstructed aquapon...
bijceesjournal
 
Modelagem de um CSTR com reação endotermica.pdf
Modelagem de um CSTR com reação endotermica.pdfModelagem de um CSTR com reação endotermica.pdf
Modelagem de um CSTR com reação endotermica.pdf
camseq
 
Literature Review Basics and Understanding Reference Management.pptx
Literature Review Basics and Understanding Reference Management.pptxLiterature Review Basics and Understanding Reference Management.pptx
Literature Review Basics and Understanding Reference Management.pptx
Dr Ramhari Poudyal
 
官方认证美国密歇根州立大学毕业证学位证书原版一模一样
官方认证美国密歇根州立大学毕业证学位证书原版一模一样官方认证美国密歇根州立大学毕业证学位证书原版一模一样
官方认证美国密歇根州立大学毕业证学位证书原版一模一样
171ticu
 
Recycled Concrete Aggregate in Construction Part III
Recycled Concrete Aggregate in Construction Part IIIRecycled Concrete Aggregate in Construction Part III
Recycled Concrete Aggregate in Construction Part III
Aditya Rajan Patra
 
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Sinan KOZAK
 
132/33KV substation case study Presentation
132/33KV substation case study Presentation132/33KV substation case study Presentation
132/33KV substation case study Presentation
kandramariana6
 
Understanding Inductive Bias in Machine Learning
Understanding Inductive Bias in Machine LearningUnderstanding Inductive Bias in Machine Learning
Understanding Inductive Bias in Machine Learning
SUTEJAS
 

Recently uploaded (20)

BPV-GUI-01-Guide-for-ASME-Review-Teams-(General)-10-10-2023.pdf
BPV-GUI-01-Guide-for-ASME-Review-Teams-(General)-10-10-2023.pdfBPV-GUI-01-Guide-for-ASME-Review-Teams-(General)-10-10-2023.pdf
BPV-GUI-01-Guide-for-ASME-Review-Teams-(General)-10-10-2023.pdf
 
Engineering Drawings Lecture Detail Drawings 2014.pdf
Engineering Drawings Lecture Detail Drawings 2014.pdfEngineering Drawings Lecture Detail Drawings 2014.pdf
Engineering Drawings Lecture Detail Drawings 2014.pdf
 
Advanced control scheme of doubly fed induction generator for wind turbine us...
Advanced control scheme of doubly fed induction generator for wind turbine us...Advanced control scheme of doubly fed induction generator for wind turbine us...
Advanced control scheme of doubly fed induction generator for wind turbine us...
 
Generative AI leverages algorithms to create various forms of content
Generative AI leverages algorithms to create various forms of contentGenerative AI leverages algorithms to create various forms of content
Generative AI leverages algorithms to create various forms of content
 
Eric Nizeyimana's document 2006 from gicumbi to ttc nyamata handball play
Eric Nizeyimana's document 2006 from gicumbi to ttc nyamata handball playEric Nizeyimana's document 2006 from gicumbi to ttc nyamata handball play
Eric Nizeyimana's document 2006 from gicumbi to ttc nyamata handball play
 
Engine Lubrication performance System.pdf
Engine Lubrication performance System.pdfEngine Lubrication performance System.pdf
Engine Lubrication performance System.pdf
 
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
 
Computational Engineering IITH Presentation
Computational Engineering IITH PresentationComputational Engineering IITH Presentation
Computational Engineering IITH Presentation
 
TIME DIVISION MULTIPLEXING TECHNIQUE FOR COMMUNICATION SYSTEM
TIME DIVISION MULTIPLEXING TECHNIQUE FOR COMMUNICATION SYSTEMTIME DIVISION MULTIPLEXING TECHNIQUE FOR COMMUNICATION SYSTEM
TIME DIVISION MULTIPLEXING TECHNIQUE FOR COMMUNICATION SYSTEM
 
CHINA’S GEO-ECONOMIC OUTREACH IN CENTRAL ASIAN COUNTRIES AND FUTURE PROSPECT
CHINA’S GEO-ECONOMIC OUTREACH IN CENTRAL ASIAN COUNTRIES AND FUTURE PROSPECTCHINA’S GEO-ECONOMIC OUTREACH IN CENTRAL ASIAN COUNTRIES AND FUTURE PROSPECT
CHINA’S GEO-ECONOMIC OUTREACH IN CENTRAL ASIAN COUNTRIES AND FUTURE PROSPECT
 
Unit-III-ELECTROCHEMICAL STORAGE DEVICES.ppt
Unit-III-ELECTROCHEMICAL STORAGE DEVICES.pptUnit-III-ELECTROCHEMICAL STORAGE DEVICES.ppt
Unit-III-ELECTROCHEMICAL STORAGE DEVICES.ppt
 
A review on techniques and modelling methodologies used for checking electrom...
A review on techniques and modelling methodologies used for checking electrom...A review on techniques and modelling methodologies used for checking electrom...
A review on techniques and modelling methodologies used for checking electrom...
 
Comparative analysis between traditional aquaponics and reconstructed aquapon...
Comparative analysis between traditional aquaponics and reconstructed aquapon...Comparative analysis between traditional aquaponics and reconstructed aquapon...
Comparative analysis between traditional aquaponics and reconstructed aquapon...
 
Modelagem de um CSTR com reação endotermica.pdf
Modelagem de um CSTR com reação endotermica.pdfModelagem de um CSTR com reação endotermica.pdf
Modelagem de um CSTR com reação endotermica.pdf
 
Literature Review Basics and Understanding Reference Management.pptx
Literature Review Basics and Understanding Reference Management.pptxLiterature Review Basics and Understanding Reference Management.pptx
Literature Review Basics and Understanding Reference Management.pptx
 
官方认证美国密歇根州立大学毕业证学位证书原版一模一样
官方认证美国密歇根州立大学毕业证学位证书原版一模一样官方认证美国密歇根州立大学毕业证学位证书原版一模一样
官方认证美国密歇根州立大学毕业证学位证书原版一模一样
 
Recycled Concrete Aggregate in Construction Part III
Recycled Concrete Aggregate in Construction Part IIIRecycled Concrete Aggregate in Construction Part III
Recycled Concrete Aggregate in Construction Part III
 
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
 
132/33KV substation case study Presentation
132/33KV substation case study Presentation132/33KV substation case study Presentation
132/33KV substation case study Presentation
 
Understanding Inductive Bias in Machine Learning
Understanding Inductive Bias in Machine LearningUnderstanding Inductive Bias in Machine Learning
Understanding Inductive Bias in Machine Learning
 

ACIVS 2013 conference poster presentation

  • 1. PERFORMANCE EVALUATION OF VIDEO ANALYTICS FOR SURVEILLANCE ON-BOARD TRAINS Context and Objectives ACIVS’13, October 30th 2013, Poznan, Poland Low level Evaluation Real-time video surveillance is used in public transportation, including on-board metro trains, to support human operators in control rooms. Video Content Analytics (VCA) is effective when the performance is such to reduce false alarms under appropriate acceptability thresholds. Reducing the causes of false alarms by fine tuning the VCA is particularly important in on-board applications, that are more prone to disturbances with respect to fixed installations due to more critical operating conditions (e.g. constrained camera field of view, frequent light changes, vibrations, occlusions, etc.). Accurate performance evaluation is essential to decide about:  Which alarms to activate depending on scenarios  How to set algorithm parameters in each scenario Comparison between commercial and open-source implementations is also important to evaluate the added value provided by the former in terms of achievable performance. Black Box Evaluation Valentina Casola, Mariana Esposito, Francesco Flammini, Nicola Mazzocca, Concetta Pragliola Contributions Evaluation of Commercial Off-The-Shelf (COTS) VCA system Low Level Black Box Alarm Performance Evaluation Comparison with Open source System Frame and Object Based Metrics Ground Truth values compared with Algorithm Result Computed by specific tool developed in Matlab Object Based Metrics Consider the whole trajectory of each object in the scene and preserves its lifetime Correct Detected Track (CDT) Track Detection Failure (TDF) False Alarm Track (FAT) Track Fragmentation (TF) ID Change (IDC) Temporal Overlap Spatial Overlap Performance Evaluation ToolAlgorithm Result Ground Thruth Tot Tos Metrics (15%) Frame Based Metrics Measure the performance on individual frames of a video stream and do not consider the preservatio. The blobs are evaluated in their size and location and compared with GT. True Positive (TP) False Negative (FN): Fragmentation ecall False Positive (FP) Merging Precision 0 50 100 150 200 250 300 350 400 Occurrencies #AR TP FN FP TF IDC Object-Based Results COTS Open-Source 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1 PerformanceIndex Accuracy FAR PP FNR Object-Based Performance COTS Open-Source False Alarm Rate FPTP FP  FAR Detection rate FNTP TP  DR Positive Prediction FPTP TP  PP False Negative Rate TPFN FN  FNR Fragmentation Index FRAGM TP FI Merge Index MERGE FPOBJAR  FM ♦ 1510 objects in 600 frames, 8 GT tracks ♦ COTS system detects less objects but it is far more reliable regarding false positives (84 vs 2168) ♦ FA of 6% instead of 41% (the latter clearly unacceptable). ♦ COTS object Fragmentation is less then one half w.r.t. Open source. Metrics COTS Open TP 1307 1324 FN 203 186 FP 84 2168 Metrics COTS Open TP 7 8 FN 1 0 FP 9 213 TF 11 17 IDC 0 0 ♦ 1510 objects in 600 frames, 8 GT tracks ♦ COTS FP and PP reveal higher reliability ♦ Same consideration for TF (important for event persistence). Performance Indices Test of COTS system on real vehicle for light railwails and tramways False Alarm Index RealAlarms# sFalseAlarm# FAI  Probability Of Detection PDTPFN TP  POD over it detects camera occlusionblurring with paint or obstacles. amper it detects the manumission of the camera by moving it. top it detects still objects in the scene. Presence it detects objectspeople moving in the scene rowd it raises an alarm when the scene is overcrowded. ALARMS

Editor's Notes

  1. Recall implementata Precision implementata • Calcola, in ogni frame, il numero di oggetti di GT rilevati correttamente dal sistema e, per ognuno di essi, il numero di bounding-box in cui sono stati frammentati  True Positive (TP)  False Negative (FN)  Fragmentation(FRAGM) • Calcola, in ogni frame, il numero di oggetti di AR che non corrispondono ad alcun oggetto di GT. Per quelli che risultano associati ad oggetti di GT ricava, invece, il numero esatto di questi ultimi  False Positive (FP)  Merging (MERGE)