Towards a Smart Control Room for Crisis Response Using Visual Perception of Users

Towards a Smart Control Room for Crisis Response Using Visual Perception of Users Joris Ijsselmuiden, Florian van de Camp, Alexander Schick, Michael Voit, Rainer Stiefelhagen {iss, ca, sci, vt, stiefe}@iitb.fraunhofer.de Fraunhofer IITB, Karlsruhe INTRODUCTION Due to ever increasing challenges and complexity, there is a high demand for new human-machine interaction approaches in crisis response scenarios. We aim at building a smart crisis response control room, in which vision-based perception of users will be used to facilitate innovative user interfaces and to support teamwork. Our control room is equipped with several cameras and has a videowall as the main interaction device. Using real-time computer vision, we can track and identify the users in the room, and estimate their head orientations and pointing gestures. In the near future, the room will also be equiped with speech recognition. In order to build a useful smart control room for crisis response, we are currently focusing on situation modeling for such rooms and we are investigating the target crisis response scenarios. Person tracking, head pose estimation, and gesture recognition SYSTEM ARCHITECTURE • All components run in parallel and in real time • We use several computers, with multithreading and GPU programming to obtain sufficient computational power • Our custom-built middleware takes care of network communication Our smart control room laboratory, containing videowall and cameras • A centralized situation model (blackboard) is kept, describing the situation in the room and the objects and users in it GOALS • All perceptual components can read and write in this situation model and a logic • Develop new ways of interacting with computers and support interaction between engine uses it to deduce higher level facts about the situation [6] humans using: tracking, identification, head pose, gestures, speech, and situation/user modeling [1,2] • In the near future, our control room laboratory will be extended using some of the following: speech recognition, standard workstations, a digital situation table [7], • Conduct user studies to find multimodal system setups that improve computer tablet PCs, sound, and synthesized speech supported cooperative work in a crisis response control room • Improve expressive power, ease of use, intuitiveness, speed, reliability, adaptability, and cooperation while reducing physical and mental workload • Create intelligent, context dependant user interfaces through situation modeling and user modeling • Challenges in crisis response control rooms include: team-based operation, limits to mental workload, high cost of failure, time pressure, dense/complex information, and the user acceptance problem PERCEPTION Example interaction with the videowall and a digital situation table in operation • Tracking and identification [3] • Head pose and visual focus of attention [4] This work is supported by the FhG Internal Programs under Grant No. 692 026 • Gestures and bodypose [5] (Fraunhofer Attract). It is a collaboration between the Fraunhofer Institute for • Speech recognition (future work) Information and Data Processing; Business Unit Interactive Analysis and Diagnosis and the University of Karlsruhe (TH); Faculty of Computer Science, in the framework of the five-year Fraunhofer internal project “Computer Vision for Human- Computer Interaction – Interaction in and with attentive rooms”. REFERENCES 1. Project Webpage (2009) www.iitb.fraunhofer.de/?20718 2. Stiefelhagen, Bernardin, Ekenel, Voit (2008) Tracking Identities and Attention in Smart Environments - Contributions and Progress in the CHIL Project IEEE A camera image and its corresponding segmentation and 3D voxel representation International Conference on Face and Gesture Recognition 3. Bernardin, Van de Camp, Stiefelhagen (2007) Automatic Person Detection and INTERACTION Tracking using Fuzzy Controlled Active Cameras IEEE International Conference 1. Identities are obtained through face recognition (in operation) on Computer Vision and Pattern Recognition 2. User models are used to generate personal user interfaces, obeying the user’s 4. Voit, Stiefelhagen (2008) Deducing the Visual Focus of Attention from Head Pose preferences, current tasks, and specialized knowledge (future work) Estimation in Dynamic Multi-view Meeting Scenarios 10th International 3. Using person tracking, interfaces are displayed close to the user (in operation) Conference on Multimodal Interfaces 4. Objects on the videowall are manipulated using pointing gestures and directing 5. Nickel, Stiefelhagen (2007) Visual Recognition of Pointing Gestures for Human- ones visual attention (in operation) Robot Interaction Image and Vision Computing 5. This can be combined with speech recognition and a range of different hand 6. Brdiczka, Crowley, Curín, Kleindienst (2009) Chapter: Situation Modeling, in gestures (future work) Waibel, Stiefelhagen (Eds.) Computers in the Human Interaction Loop 6. Head pose is employed to analyze the interaction of the team, for example to 7. Bader, Meissner, Tschnerney (2008) Digital Map Table with Fovea-Tablett®: determine who has been talking to whom (in operation) Smart Furniture for Emergency Operation Centers 5th International Conference on Information Systems for Crisis Response and Management 7. User-specific information can be displayed on the videowall, at the user’s current focus of attention and we can make people aware of what they haven’t seen yet (future work)

Towards a Smart Control Room for Crisis Response Using Visual Perception of Users

by SerPico75 on May 18, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 7

Statistics

Towards a Smart Control Room for Crisis Response Using Visual Perception of Users — Document Transcript