Multi-criteria Fusion of Heterogeneous Information for Improving Situation Awareness on Emergency Management Systems

1. Introduction
Objective
Information Fusion
Multi-criteria Fusion of Heterogeneous Information
Conclusions
Multi-criteria Fusion of Heterogeneous Information
for Improving Situation Awareness on Emergency
Management Systems
Valdir Junior, Matheus Ferraroni, Leonardo Botega, Jessica
Oliveira, Caio Saraiva, Elvis Fusco, Márcio Campos
Computing and Information Systems Research Lab - COMPSI
Marilia Euripedes University - UNIVEM
Wirelles Networks and Distributed Interactive Simulation Lab - WINDIS
Computer Departament - Federal University of São Carlos - UFSCar
International Conference on Human-Computer Interaction, 2015
Valdir Junior, Matheus Ferraroni, Leonardo Botega, Jessica Oliveira, Caio Saraiva, Elvis Fusco, Márcio CamposMulti-criteria Fusion of Heterogeneous Information for Impr

2. Introduction
Objective
Information Fusion
Multi-criteria Fusion of Heterogeneous Information
Conclusions
Outline
1 Introduction
2 Objective
3 Information Fusion
4 Multi-criteria Fusion of Heterogeneous Information
Acquisition of Audio and Pre-Processing
Information Fusion from Acquisition
5 Conclusions
Valdir Junior, Matheus Ferraroni, Leonardo Botega, Jessica Oliveira, Caio Saraiva, Elvis Fusco, Márcio CamposMulti-criteria Fusion of Heterogeneous Information for Impr

3. Introduction
Objective
Information Fusion
Multi-criteria Fusion of Heterogeneous Information
Conclusions
Situation Awareness
Situation awareness - SAW is a concept related to the level of
consciousness that an individual or team has to a situation. It is a
dynamic understanding of an operator about what is happening in
the environment and the projection of its status in the near future.
According to Endsley SAW is divided in three levels:
perception of the elements in the environment
comprehension of these elements status in a situation
and the evolution of these elements in a near future
Valdir Junior, Matheus Ferraroni, Leonardo Botega, Jessica Oliveira, Caio Saraiva, Elvis Fusco, Márcio CamposMulti-criteria Fusion of Heterogeneous Information for Impr

4. Introduction
Objective
Information Fusion
Multi-criteria Fusion of Heterogeneous Information
Conclusions
SAW in Emergency Management Systems
Situation awareness systems, specially applications Emergency
Management Systems rely on information quality to provide
specialists a better view of the analyzed scenario for making quality
decisions.
Valdir Junior, Matheus Ferraroni, Leonardo Botega, Jessica Oliveira, Caio Saraiva, Elvis Fusco, Márcio CamposMulti-criteria Fusion of Heterogeneous Information for Impr

5. Introduction
Objective
Information Fusion
Multi-criteria Fusion of Heterogeneous Information
Conclusions
SAW in Emergency Management Systems
In the Emergency domain, there are some elements of awareness
that individual specilialists must posses, such as:
mission goal awareness
systems awareness
resources awareness
team awareness
Valdir Junior, Matheus Ferraroni, Leonardo Botega, Jessica Oliveira, Caio Saraiva, Elvis Fusco, Márcio CamposMulti-criteria Fusion of Heterogeneous Information for Impr

6. Introduction
Objective
Information Fusion
Multi-criteria Fusion of Heterogeneous Information
Conclusions
Data Fusion
Data fusion can be dened as a combination of multiple sources to
obtain improved information.
Data fusion techniques combine data from multiple sensors and
related information from associated databases to achieve improved
accuracy and more specic inferences than could be achieved by
the use of a single sensor alone
Valdir Junior, Matheus Ferraroni, Leonardo Botega, Jessica Oliveira, Caio Saraiva, Elvis Fusco, Márcio CamposMulti-criteria Fusion of Heterogeneous Information for Impr

7. Introduction
Objective
Information Fusion
Multi-criteria Fusion of Heterogeneous Information
Conclusions
The objective of this work is a Data Fusion Framework for
mitigation of information uncertainty in Emergency Management
Systems, using:
multi-criteria fusion
information quality measures
semantic
Valdir Junior, Matheus Ferraroni, Leonardo Botega, Jessica Oliveira, Caio Saraiva, Elvis Fusco, Márcio CamposMulti-criteria Fusion of Heterogeneous Information for Impr

8. Introduction
Objective
Information Fusion
Multi-criteria Fusion of Heterogeneous Information
Conclusions
Information is higher process level than data fusion, in this domain
some semantic information is used:
camera metada
relationship properties among the classes
quality measures
human information
Valdir Junior, Matheus Ferraroni, Leonardo Botega, Jessica Oliveira, Caio Saraiva, Elvis Fusco, Márcio CamposMulti-criteria Fusion of Heterogeneous Information for Impr

9. Introduction
Objective
Information Fusion
Multi-criteria Fusion of Heterogeneous Information
Conclusions
Acquisition of Audio and Pre-Processing
Information Fusion from Acquisition
The Framework
We propose a framework of activities that organize the process of
information fusion the SAW for decision-making systems.
This Framework can be divided into four main processes.
The more important processes for fusion are: acquision and
information fusion
Valdir Junior, Matheus Ferraroni, Leonardo Botega, Jessica Oliveira, Caio Saraiva, Elvis Fusco, Márcio CamposMulti-criteria Fusion of Heterogeneous Information for Impr

10. Introduction
Objective
Information Fusion
Multi-criteria Fusion of Heterogeneous Information
Conclusions
Acquisition of Audio and Pre-Processing
Information Fusion from Acquisition
Figure : Framework for Multi-criteria Fusion of Heterogeneous
Information
Valdir Junior, Matheus Ferraroni, Leonardo Botega, Jessica Oliveira, Caio Saraiva, Elvis Fusco, Márcio CamposMulti-criteria Fusion of Heterogeneous Information for Impr

11. Introduction
Objective
Information Fusion
Multi-criteria Fusion of Heterogeneous Information
Conclusions
Acquisition of Audio and Pre-Processing
Information Fusion from Acquisition
In the proposed framework, there are several data sources, such as:
audio of report call the police
social networking posts
databases
images
In the case study we debate how audiotapes, posts and security
cameras images can be implemented.
All information generated by this stage(objects, attributes and
properties) are submitted to an evaluation, at this stage quality
scores are assigned to information, such score is used as Fusion
criteria.
Valdir Junior, Matheus Ferraroni, Leonardo Botega, Jessica Oliveira, Caio Saraiva, Elvis Fusco, Márcio CamposMulti-criteria Fusion of Heterogeneous Information for Impr

12. Introduction
Objective
Information Fusion
Multi-criteria Fusion of Heterogeneous Information
Conclusions
Acquisition of Audio and Pre-Processing
Information Fusion from Acquisition
Input
The object generated after acquisition and quality assessment
corresponds to objects and attributes level L1 of fusion (according
to the JDL taxonomy)
The input of multi-criteria fusion is composed by that information
from acquisition with quality score and the relationship properties
(semantic) and criteria set by the operator, for example:
which objects to specically associate
which external sources to use
which properties must exist
a new threshold of minimum quality of the information
Valdir Junior, Matheus Ferraroni, Leonardo Botega, Jessica Oliveira, Caio Saraiva, Elvis Fusco, Márcio CamposMulti-criteria Fusion of Heterogeneous Information for Impr

13. Introduction
Objective
Information Fusion
Multi-criteria Fusion of Heterogeneous Information
Conclusions
Acquisition of Audio and Pre-Processing
Information Fusion from Acquisition
Engine
The information fusion engine proposed there are two main
methods:
search for synergic information
multi-criteria association
The result of this process contain all initial information submitted,
with new attributes, properties and new objects
This result is explicitly level L2 of fusion, and this result can be
submitted again to the process of search synergic data.
After this process the resulting information will be submitted to the
quality assessment.
Valdir Junior, Matheus Ferraroni, Leonardo Botega, Jessica Oliveira, Caio Saraiva, Elvis Fusco, Márcio CamposMulti-criteria Fusion of Heterogeneous Information for Impr

14. Introduction
Objective
Information Fusion
Multi-criteria Fusion of Heterogeneous Information
Conclusions
Acquisition of Audio and Pre-Processing
Information Fusion from Acquisition
Phases
This work proposed the fusion in two dierent process:
The primary fusion phase: is the automated process performed
after the acquisition and quality assessment.
The second fusion phase: in this case the process of fusion is
on the demand by the operator, this process is operated via
interface by the expert.
Valdir Junior, Matheus Ferraroni, Leonardo Botega, Jessica Oliveira, Caio Saraiva, Elvis Fusco, Márcio CamposMulti-criteria Fusion of Heterogeneous Information for Impr

15. Introduction
Objective
Information Fusion
Multi-criteria Fusion of Heterogeneous Information
Conclusions
M. R. Endsley
Designing for situation awareness: An approach to
user-centered design.
CRC Press, 2011.
Blasch, E.
High Level Information Fusion (HLIF): Survey of models,
issues, and grand challenges.
IEEE AE Systems magazine, 420, 2012.
Valdir Junior, Matheus Ferraroni, Leonardo Botega, Jessica Oliveira, Caio Saraiva, Elvis Fusco, Márcio CamposMulti-criteria Fusion of Heterogeneous Information for Impr

16. Introduction
Objective
Information Fusion
Multi-criteria Fusion of Heterogeneous Information
Conclusions
Questions
Multi-criteria Fusion of Heterogeneous Information Improving
Situation Awareness on Emergency Management Systems
valdir.a.junior@gmail.com
Computing Science and Information Systems Lab - COMPSI
Marilia Euripedes University - UNIVEM
Wireless Networks and Distributed Interactive Simulations Lab - WINDIS
Computer Department - Federal University of São Carlos - UFSCar
Valdir Junior, Matheus Ferraroni, Leonardo Botega, Jessica Oliveira, Caio Saraiva, Elvis Fusco, Márcio CamposMulti-criteria Fusion of Heterogeneous Information for Impr