Your SlideShare is downloading. ×
0
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Phd Thesis Project
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Phd Thesis Project

836

Published on

My PhD Thesis project. Fusion of UWB localization systems with computer vision based people tracking in an ambient intelligence scenario

My PhD Thesis project. Fusion of UWB localization systems with computer vision based people tracking in an ambient intelligence scenario

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
836
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Cooperative and non cooperative
  • We are moving toward an Internet of Things scenario where objects and spaces are becoming “ intelligent ” and rich of information . This scenario is rising new challenges and new opportunities especially in the field of Human Computer Interaction . HCI paradigms are evolving toward “fluid” interfaces that become part of the surrounding environment and that are able to react to human behaviours and that should provide a natural interaction while interacting with complex systems, and avoid cognitive overload while accessing information and services Fusion of this different systems can be used to rethink the human-machine interactive experience developing new kind of interfaces. Integration models can be designed to use this systems as input device itself (i.e. proximity, gesture recognition) or use their outputs in more complex context capture systems and in multi agent systems to realize ambient that are “intelligent” allowing new kind of interaction paradigms between humans and machines.
  • As traditional approaches of usability testing are suitable for testing individual applications or devices in an isolated manner there is the need to design new evaluation methodologies for this new kind of interfaces.
  • Transcript

    • 1. Integration of computer vision and RTL technologies for ambient intelligence and Human-Computer Interaction Massimiliano Dibitonto DRIEI Dottorato di Ricerca in Ingegneria Elettronica e Informatica University of Cagliari, Italy
    • 2. Outline
      • State-of-the-art
        • Wireless Real Time Locating Systems (RTL)
        • Computer Vision Systems (CV)
      • RTLS and CVS “complementarity”
        • Why “fusion” of RTL and CV?
      • Objectives of my PhD Thesis
      • Scenarios
      • Starting point
        • A CV-RFID-based prototype under development at DIEE
      • Expected progress beyond the state of the art
    • 3. Wireless Real Time Locating Systems
      • Realized with radio frequency technologies (to allow mobility) are usually composed by tags (active and passive) , a set of antennas and a control system . They can be terminal or network based.
      • They can be divided into 3 main categories:
      • Time of Arrival
      • Direction (Angle) of Arrival
      • Signal Strenght Based Systems
    • 4. Wireless Real Time Locating Systems
      • Localization of objects/people
      • Possible with a wide range of technologies
      • Different characteristics for different applications
      UWB ACTIVE RFID 1cm 1cm 10 cm 1 m 10 m Outdoor Semi urban Urban Building Room Objects GPS Wi-Fi Passive RFID accuracy ZigBee
    • 5. Computer Vision
      • Computer vision techniques allows to analyze images to get different kind of information.
      • There are a lot of application and a huge literature but t here are two major themes in the computer vision literature:
      • 2D/3D geometry: using vision as a source of metric 2D/3D information
      • Recognition: vision as a source of semantic information
    • 6. CV and RTLS “complementarity”
      • RTLS
      • NLOS
      • Tracking of a tag (not of an embodied characteristic)
      • Unique object identifier
      • CV
      • LOS
      • Tracking of a object/person characteristic
      • Class or Unique object identifier
      Both systems try to aswer to two MAIN common questions: Who/What and Where The main differences are
    • 7. CV and RTLS “complementarity”
      • ...but they can also detect different data regarding the same target
      RTLS CV 2d – 3d position temperature, humidity, pressure, acceleration, etc.. unique ID Orientation, gestures , biometrics, fine grained movements, recognition of different features Other sensors onboard: Fusion models can be developed to harvest the information gained from the two systems
    • 8. CV and RTL/RFID fusion
      • In literature we find different models of integration of RFID and RTLS with CV systems that can be taken as starting point for further integration models:
      • Objects/people recognition Cerrada, C.; Salamanca, S.; Perez, E.; Cerrada, J.A.; Abad, I., "Fusion of 3D Vision Techniques and RFID Technology for Object Recognition in Complex Scenes," Intelligent Signal Processing, 2007. WISP 2007. IEEE International Symposium on , vol., no., pp.1-6, 3-5 Oct. 2007
      • Localization Matthieu Anne, James L. Crowley, Vincent Devin et Gilles Privat. Localisation intra-bâtiment multi-technologies: RFID, Wifi, vision. In Actes de la conférence Ubimob 2005, pages 29–35, Grenoble, France, mai 2005
      • Behaviour analysis Krahnstoever, N.; Rittscher, J.; Tu, P.; Chean, K.; Tomlinson, T., "Activity Recognition using Visual Tracking and RFID," Application of Computer Vision, 2005. WACV/MOTIONS '05 Volume 1. Seventh IEEE Workshops on , vol.1, no., pp.494-500, 5-7 Jan. 2005
    • 9. CV and RTL/RFID fusion: objects/people recognition
      • RFID and CV systems are able to identify object or people but they rely on different identifiers (natural or secondary carrier) and offers different levels of accuracy.
      • Cerrada et al.* uses that improves identification and reduces calculation complexity using an RFID reader to detect the objects and compare the scene with a reduced database .
      *Cerrada, C.; Salamanca, S.; Perez, E.; Cerrada, J.A.; Abad, I., "Fusion of 3D Vision Techniques and RFID Technology for Object Recognition in Complex Scenes," Intelligent Signal Processing, 2007. WISP 2007. IEEE International Symposium on , vol., no., pp.1-6, 3-5 Oct. 2007
    • 10. CV and RTL fusion: Localization
      • A system developer by Matthieu et al.* uses RTLS to determine the region to be investigated by the vision system and then compare the results:
      • Improve localization/tracking performance
      • Solve occlusion/interference problems
      • Solve identity detection problems
      CV RTLS
      • Matthieu Anne, James L. Crowley, Vincent Devin et Gilles Privat. Localisation intra-bâtiment multi-technologies: RFID, Wifi, vision. In Actes de la conférence Ubimob 2005, pages 29–35, Grenoble, France, mai 2005
    • 11. CV and RTL/RFID fusion: Behaviour analysis
      • Data collected by CV and RFID/RTL systems can be used for a better context capture an used with a probabilistic model to capture human behaviour.
      • In a work of Krastoever* a vision system is used to track head and hands movements and an RFID system is used to track objects
      * Krahnstoever, N.; Rittscher, J.; Tu, P.; Chean, K.; Tomlinson, T., "Activity Recognition using Visual Tracking and RFID," Application of Computer Vision, 2005. WACV/MOTIONS '05 Volume 1. Seventh IEEE Workshops on , vol.1, no., pp.494-500, 5-7 Jan. 2005 Frame Grabber Articulated Tracker RFID Reader Custom Pose Estimation Circuitery Activity recognition
    • 12. Objectives Outline Our research will point to:
    • 13. Objectives: integration models
      • At a first stage a model for people localization will be taken into account.
      • Two models will be proposed:
      • A sequential model
      • A parallel model
      • Define integration models that take into account:
      • Characteristic and performance of each system
      • Different scenarios where system can be successfully integrated
      • Improvement of systems performances
    • 14. Proposed integration models Sequential model : the RTL system act as a guide fro the vision based tracking system. The main benefit is a reduction of the computational complexity but on the other hand it doesn’t detect people/object not equipped with a tag.
    • 15. Proposed integration models Parallel model : the two systems works indipendently , then a data fusion system analyze the two outputs . Localization via RTL/RFID system Image analysis – object/person tracking Comparison of the obtained data
    • 16. Proposed integration models
      • It implies also answering to several problems:
      • Choose the RTL system that offers the best accuracy
      • Make the output of the two systems comparable :
        • Space problem: they have two different coordinate systems (image vs plane). An homographic matrix must be used.
        • Time problem: they give output at certain time intervals. Moreover there could be a little difference in the systems time .
      • Choose which system to rely on in ambiguous cases
    • 17. Objectives: performance evaluation
      • It is also necessary to set up evaluation methodologies and indicators to state if the integrated system brings a real advantage and which parameters can be improved.
      • Traditional indicators used for RTL and CV vision system could be adapted.
      • At first, simple indicators can be used to evaluate both the single and the integrated system:
      • - Localization accuracy
      • Localization resolution
      • False positives and false negatives
      • System responsiveness (update, latency)
    • 18. Objectives: new HCI paradigms
      • Fusion of this different systems can be used to rethink the human-machine interactive experience developing new kind of interfaces that are can
      • provide a natural interaction while interacting with complex systems
      • and avoid cognitive overload while accessing information and services.
      • Integration models can be designed to use this systems as input device itself (i.e. proximity, gesture recognition) or use their outputs in more complex context capture systems and in multi agent systems . Thus to realize ambient that are “intelligent” allowing new kind of interaction paradigms between humans and machines.
      Interface adaptation module Multi agent system Context capture system RTLS Wireless Sensors and Actuators Network Computer Vision System
    • 19. Objectives: usability evaluation
      • Along with increased complexity of the system and user workflow in an “intelligent” environment, the usability of the entire system and of all the components must be ensured
      • There is the need to design new evaluation methodologies for this new kind of interfaces but traditional approaches used in HCI and Cognitive Sciences can be taken as starting point as:
      • Usability inquiry and field test with users
      • Cognitive load index (i.e. Nasa TLX )
      • To have a complete evaluation framework, they must be integrated with additional information of application context like:
      • - user position,
      • - user behaviour
      • object position
      • other context data meaningful during the interaction
    • 20. Possible Scenarios – Ambient Intelligence
      • The context captured from the vision system and the user’s position can be used for ambient intelligence systems to make the environment react in an intelligent manner to human but also to other objects(sensors, smart obj... ).
      • In this way interfaces become “fluid” and disappear.
      0100011100101000....
    • 21. PhD Thesis Starting Point: A prototype of CV and RTL fusion at DIEE
      • A prototype is under development using a UWB RTLS and the Intelligent Face Locator (IFL) system with an IP camera.
      • The system can identify and locate people with high time/spatial resolution and take decision in specific cases.
      RTL and RFID services Video module Segnalation module
    • 22. PhD Thesis Starting Point: A prototype of CV and RTL fusion at DIEE
      • The test field is representative of a real scenario due to the presence of obstacles and radio interferences (2,4 GHz), moreover the areas covered by the two system does not perfectly overlap.
      • Test shows differences between IFL and UWB data .
    • 23. Progress beyond SoA
      • The existing literature shows us different authors starting to approach the problems outlined, formulating hypothesis and undertaking experimental researches. This points out a field that is still growing up but that has solid basis with a lot of contribution on RTLS and computer vision systems.
      • Undertaking this research we expected to achieve a progress beyond the state of the art in several of the topics presented :
      • Fusion models between wireless real time locating systems and computer vision systems, especially regarding localization of objects and context capture
      • Evaluation methodologies and indicators for the described system
      • Human Computer Interaction paradigms and Ambient Intelligence applications using the described systems
    • 24. Put together different “worlds”..... ... toward new intelligent scenarios
    • 25.
      • A work that puts together
      Department of Electrical and Electronic Engineering (PRA group) of University of Cagliari with its strong competence on Computer Vision Cattid fron Sapienza University for his competence on Human Computer Interaction and Real Time Locating Systems
    • 26.
      • Thank you for you attention!
      • Q & (hopefully) Answers!

    ×