ViSEvAl is a software platform for visualizing and evaluating video analysis algorithms. It allows loading video data, detecting objects, and comparing detections to ground truths. Metrics are defined using distances, filters, and criteria. ViSEvAl includes graphical and command-line interfaces for configuration, running experiments, and obtaining evaluation results.

1. ViSEvAl
ViSualisation
and
EvAluation
http://team.inria.fr/stars/fr/2012/02/02/viseval-software/

2. Overview

What is evaluation?
− Evaluation process
− Metric definition

ViSEvAl
− Description
− Installation
− Configuration
− Functionalities

3. Evaluation process
General overview

4. Metric definition

Metric = distance + filter + criteria

Distance: associate detected and annotated objects
− Spatial: compare bonding boxes area
− Temporal: compare time intervals

Filter: selects which object to evaluate
− Specific type, distance to the camera, ...

Criteria: how the property of detected and
annotated objects is similar?
− 4 tasks: detection, classification, tracking,
event detection

5. ViSEvAl platform description
ViSEvAlGUI
ViSEvAl:
 Interfaces
 All functionalities
(synchronisation,
display,...) ViSEvAlEvaluation
Core library
Loading Video Frame Metric Distance
Tool
Temporal Metric Event Metric Object Filter Plugin

6. ViSEvAl plugins 1/2

Loading video
− ASF-Videos, Caviar-Images, JPEG-
Images, Kinect-Images (hospital),
OpenCV-Videos (Vanhaeim), PNG-Images

Distance
− Bertozzi, dice coefficient, overlapping

Filter
− CloseTo, FarFrom, Identity, TypeGroup

7. ViSEvAl plugins 2/2

Criteria
− Detection: M1.X

2 criteria (M1.1: area, M1.2: silhouette)
− Classification: M2.X

2 criteria (M2.1: type, M2.2: sub type)
− Tracking: M3.X

6 criteria (M3.1: F2F, M3.2: persistence, M3.4
(tracking time), M3.5: confusion, M3.6, M3.7:
confusion + tracking time, M3.8: frame detection
accuracy)
− Event: M4.X

4 criteria (M4.1, M4.2: begin and end time, M4.3:
common frame, M4.4: common time)

8. ViSEvAl: inputs

A set of XML files

Detection: XML1 file -> sup platform

Recognised event: XML3 file -> sup platform

Ground truth: xgtf file -> Viper tool

Time stamp file for time synchronisation : xml file ->
createTimeStampFile.sh script provided by ViSEvAl

9. ViSEvAl installation

Get the sources
− sup svn repository
− cd sup/evaluation/ViSEvAl/

Run intall.sh at the root of ViSEvAl folder
− Dependence:

Librairies: QT4 (graphical user interface, plugin
management), gl and glu (opengl 3D view), xerces-c
(XML read), opencv (video read)

Tool: xsdcxx (automatically compute C++ classes for
reading XML files)

cd bin/appli; setenv LD_LIBRARY_PATH ../../lib

Run ./ViSEvAlGUI chu.conf

10. ViSEvAl folder organisation

src : appli, plugins (Cdistance, CeventMetric, CframeMetric,
CloadingVideoInterface, CobjectFilter, CTemporalMetric)

include : header files

install.sh, clean.sh

doc : documentation

lib : core library, plugins

scripts : createTimeStampFile.sh makeVideoFile.sh
splitxml1-2-3file.sh

bin : ViSEvAlGUI, ViSEvAlEvaluation

tools : CaviarToViseval, QuasperToViseval

xsd : xml schemas

11. ViSEvAl: configuration file
Configuration file based on Keyword-Parameter
SequenceLoadMethod "JPEG-Images” #"ASF-Videos“
SequenceLocation "0:../../example/CHU/Scenario_02.vid"
TrackingResult "0:../../example/CHU/Scenario_02_Global_XML1.xml"
EventResult "../../example/CHU/Scenario_02_Global_XML3.xml"
GroundTruth "0:../../example/CHU/gt_2011-11-15a_mp.xgtf"
XMLCamera "0:../../example/CHU/jai4.xml"
MetricTemporal "Mono:M3.4:M3.4:DiceCoefficient:0.5:TypeGroup"
MetricEvent "M4.2:M4.2.1:Duration:10

12. ViSEvAl run trace
Loading frame metric:
Mon, 11:15> ./ViSEvAlGUI
M1.1
Load all the plugins
M1.2
------------------------------------
M2.1
Loading video interfaces:
M2.2
ASF-Videos
M3.1
Caviar-Images
------------------------------------
JPEG-Images
Loading temporal metric:
Kinect-Images
M3.2
OpenCV-Videos
M3.4
PNG-Images
M3.5
------------------------------------
M3.6
Loading distance:
M3.7
3DBertozzi
M3.8
3DDiceCoefficient
------------------------------------
3DOverlapping
Loading event metric:
Bertozzi
M4.1
DiceCoefficient
M4.2
Overlapping
M4.3
------------------------------------
M4.4
Loading object filter:
------------------------------------
CloseTo
FarFrom
Identity
TypeGroup

13. ViSEvAl: two tools

ViSEvAlGUI
− Graphical user interface
− Visualise detection and ground truth on the images
− User can easily select parameters (e.g. distance,
threshold,...)
− Frame metrics results are computed in live

ViSEvAlEvaluation
− Generate a .res file containing the results of the metrics
− Frame, temporal and event metrics are computed
− User can evaluate several experiments

Same configuration file for the both tools

14. ViSEvAl: result file (.res)
camera: 0
Tracking result file: /user/bboulay/home/work/svnwork/sup/evaluation/ViSEvAl/example/vanaheim/res.xml1.xml
Fusion result file:
Event result file:
Ground truth file: /user/bboulay/home/work/svnwork/sup/evaluation/ViSEvAl/example/vanaheim/Tornelli-2011-01 28T07_00_01_groups.xgtf
Common frames with ground-truth:
Detection results:
7978 7979 7980 7981 7983 7984 7985
*****
====================================================
Metric M1.1.1
====================================================
Frame;Precision;Sensitivity 0;True Positive;False Positive;False Negative 0;Couples
8004;0.500000;1.000000;1;1;0;(100;170;0.737438)
8005;0.500000;1.000000;1;1;0;(100;170;0.721577)
8006;0.500000;1.000000;1;1;0;(100;170;0.706809)
8007;0.500000;1.000000;1;1;0;(100;170;0.713584)
====================================================
Final Results:
Global results:
Number of True Positives : 1789
Number of False Positives : 1597
Number of False Negatives 0: 2254
Precision (mean by frame) : 0.523071
Sensitivity 0 (mean by frame) : 0.477763
Precision (global) : 0.528352
Sensitivity 0 (global) : 0.442493
---------------------------------------------
Results for GT Object 2
Number of True Positives : 0
Number of False Positives : 0
Number of False Negatives 0: 0
Precision (global) : 0.000000
Sensitivity 0 (global) : 0.000000
---------------------------------------------