Border Security and Surveillance System With Smart Cameras  and Motes in a Sensor Web   Sadiye Guler, intuVision, Inc. Tim...
Introduction <ul><li>What :  Smart video surveillance system within a Sensor Web framework, augmented by auxiliary sensors...
Promise of Our Solution: Video & Mote Sensor Collaboration via  Sensor Web  Environment <ul><ul><li>Use of inexpensive  ad...
Components and Functional Approach <ul><li>Inexpensive & Low Power (long endurance) “Motes”  --  scalable & low-cost wirel...
Concept of Operations <ul><li>Auxiliary Sensor Field detects movement. </li></ul><ul><li>Detected movement position relaye...
Why OGC and IEEE Sensor Standards as a Sensor Web Foundation? <ul><li>Increasingly supported and used across government, i...
ASF --  heritage, e2e Worldwide  integrated mote field Relay   (GSM, SAT) SEGMENT ASF SEGMENT Long Haul Radio Switching Co...
Wireless Sensor Network (WSN) Nodes (“Motes”) <ul><li>Mote:  low-power wireless sensor device that constitutes a node in a...
Mote Field, Auxiliary Sensor Field (ASF) <ul><li>Network of motes to detect, minimize false alarms, cue, and direct the sm...
Smart Video Node Prototype <ul><li>Working with IP camera reference design with TI DM6446 DSP </li></ul><ul><li>Single boa...
<ul><li>SensorWeb client receives an incoming sensor observation or alert (e.g., a sensor mote detects motion) </li></ul><...
Video and Mote Sensor Collaboration Example Camera Slews to Location Video Event Detection Mote Sensor Alert
Video and Mote Sensor Collaboration Example ABANDONED OBJECT example Video Event Detection
Video Monitoring for Large Area Surveillance <ul><li>Video Detection and Tracking algorithms need to accommodate: </li></u...
Mote Sensor Directed Video Detection & Tracking  Example Area of interest indicated by mote field (ASF) Detection of peopl...
Video Analysis and Detection <ul><li>Developed video object detection, classification, tracking & </li></ul><ul><li>event ...
Sensor Web Framework Functionality <ul><ul><li>A software framework  with open standards-based suite of web services and e...
Sensor Web (SWE) Encodings & Services <ul><li>Encodings: </li></ul><ul><ul><li>Sensor Model Language (SensorML):   An XML-...
Specific Current Effort (with DHS S&T): Conceptual CBP System  (ASF-SensorWeb-SVN)  -- Data Flow 1 2 3 4 5 6 7 8 9 9 10 Se...
Video Event Alarms on Mobile Device <ul><li>A  light  web server offers access to different forms of output  </li></ul><ul...
Conclusions <ul><li>Collaboration of inexpensive, low-power auxiliary sensor field (ASF) to detect, refine (Pd, FAR), & cu...
<ul><li>Thank you! </li></ul>Sadiye Guler, intuVision [email_address] www.intuvisiontech.com Timothy D. Cole, NGIS [email_...
Upcoming SlideShare
Loading in …5
×

Sophisticated Sensor - Video UNit (SSVU)

1,731 views

Published on

Development of critical technologies, through engineered demo capability, using onboard image analytics with WSN surveillance cueing.

Published in: Business, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,731
On SlideShare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • 07/06/10
  • Sophisticated Sensor - Video UNit (SSVU)

    1. 1. Border Security and Surveillance System With Smart Cameras and Motes in a Sensor Web   Sadiye Guler, intuVision, Inc. Timothy Cole, Northrop Grumman IS Jay Silverstein, Ian Pushee, intuVision Inc. Scott Fairgrieve, Northrop Grumman IS This work is partially supported by DHS S&T Directorate SBIR funding. 2010
    2. 2. Introduction <ul><li>What : Smart video surveillance system within a Sensor Web framework, augmented by auxiliary sensors, to provide efficient real-time situational information as applied to security of our borders and critical infrastructures. </li></ul><ul><li>Why : Issues exist with large-area autonomous video surveillance: </li></ul><ul><ul><li>Expense related with dense video camera coverage of wide areas </li></ul></ul><ul><ul><li>Normally, requiring large volumes video of data that would need: </li></ul></ul><ul><ul><ul><li>collection/storage (power/storage requirements) </li></ul></ul></ul><ul><ul><ul><li>processing/analyses (intelligence, information, awareness) </li></ul></ul></ul><ul><ul><ul><li>timely distribution to be acted upon (bandwidth demand) </li></ul></ul></ul>
    3. 3. Promise of Our Solution: Video & Mote Sensor Collaboration via Sensor Web Environment <ul><ul><li>Use of inexpensive ad hoc wireless sensor nodes (“motes”) to cue and task video cameras to collect and process video only when a mote field detects and verifies activity </li></ul></ul><ul><ul><li>Embedding video-analytics processing within the camera (i.e., Smart Video Node, SVN ) to push processing of video to the sensor edge for efficient use of bandwidth </li></ul></ul><ul><ul><li>Employing a Sensor Web environment to: </li></ul></ul><ul><ul><ul><li>quickly discover available (heterogeneous) sensors </li></ul></ul></ul><ul><ul><ul><li>seamlessly integrate and collect data from disparate sensors </li></ul></ul></ul><ul><ul><ul><li>automatically task various sensors based on events </li></ul></ul></ul><ul><ul><ul><li>receive and disseminate alerts from multiple sensors </li></ul></ul></ul><ul><ul><ul><li>update middleware & obtain sensor status (housekeeping) </li></ul></ul></ul>
    4. 4. Components and Functional Approach <ul><li>Inexpensive & Low Power (long endurance) “Motes” -- scalable & low-cost wireless sensor nodes to cue video sensors </li></ul><ul><li>Smart Video Node (SVN) -- automated extraction of video content, detection of anomalous and pre-determined events for enhanced situational awareness </li></ul><ul><li>Sensor-web Framework -- integrating smart video with various inexpensive mote and unattended (UGS) sensors </li></ul><ul><ul><li>Information transparency between sensor modalities through use of standards (Open Geospatial Consortium, OGC ) </li></ul></ul>
    5. 5. Concept of Operations <ul><li>Auxiliary Sensor Field detects movement. </li></ul><ul><li>Detected movement position relayed to Sensor Web Services. </li></ul><ul><li>An alert is disseminated within the sensor web. </li></ul><ul><li>Sensor Web Client receives the new alert. </li></ul><ul><li>Alert location is highlighted on the Client GUI. </li></ul><ul><li>Client automatically tasks a camera to the location. </li></ul><ul><li>Video from tasked camera processed to detect events. </li></ul><ul><li>If detected, Event Alarm reported to Sensor Web service. </li></ul><ul><li>Detected Event published by the Sensor Web Service. </li></ul>PULSENet Data Base Mote Field Axis Network Camera Relay Positions Positions Standardized Positions/Tracks Observations Alerts Video Event Snapshots Tasking Message Standardized 6 PULSENet Client intuVision Video Event Detection PULSENet Services PULSENet Services (Mobile) 8 3 9 7 1 2 7 9 User Interface SensorWeb Data Base Mote Field Axis Network Camera Relay Positions Positions Standardized Positions/Tracks Observations Alerts Video Event Snapshots Tasking Message Standardized Detected Video Events 6 SensorWeb Client intuVision Video Event Detection SensorWeb Services SensorWeb Services (Mobile) 4 1 Standardized
    6. 6. Why OGC and IEEE Sensor Standards as a Sensor Web Foundation? <ul><li>Increasingly supported and used across government, industry, and academia. </li></ul><ul><ul><li>Strongly supported by NGA used in Empire Challenge </li></ul></ul><ul><ul><li>Currently used in sensor web efforts by NASA, DIA, and other agencies </li></ul></ul><ul><li>Net-Centric – based on XML, web services, and Service Oriented Architecture (SOA) concepts </li></ul><ul><li>Flexible – the standards can be applied to legacy or new sensor systems </li></ul>
    7. 7. ASF -- heritage, e2e Worldwide integrated mote field Relay (GSM, SAT) SEGMENT ASF SEGMENT Long Haul Radio Switching Context MOC Portal Socket MOC SEGMENT C2 Network Infrastructure SEGMENT RELAY Radio NIPRNET CEOSGUI SVN 1 CAMERA SVN 2 CAMERA Mission Programmer Socket Network Comms Manual >10-1000s motes Relay ASF … COP Databases Relay FIELD #2 Relay FIELD #3 Relay FIELD #N
    8. 8. Wireless Sensor Network (WSN) Nodes (“Motes”) <ul><li>Mote: low-power wireless sensor device that constitutes a node in a wireless sensor network (WSN); stresses power management </li></ul><ul><li>Mote consists of a microcontroller, radio, and sensing unit -- e.g., infrared, optical, radar (RF, laser), audio, magnetometer, supports autonomous operation </li></ul><ul><li>Low-power sensors used as scalable, contiguous ad hoc network that provide approximate location information on potential targets-of-interest; network increases detection (Pd) while minimizing false alarm (FAR) </li></ul>MLR_mote (Micro-Laser Radar) PIR_mote (Passive Infrared) Examples of motes used
    9. 9. Mote Field, Auxiliary Sensor Field (ASF) <ul><li>Network of motes to detect, minimize false alarms, cue, and direct the smart video nodes. </li></ul><ul><li>When a mote is powered on, forms a network with neighboring motes within range. </li></ul><ul><li>Node -to- group autonomous behavior enforced, including use of sentry and nearest neighbor verification </li></ul><ul><li>Efficient scalable, multi-hop communications employed </li></ul><ul><li>Long-haul reach back provided by relay transceiver; also forms track files and provides status/housekeeping functions </li></ul>ASF Field RELAY Cue sensor (SVN) In situ SA
    10. 10. Smart Video Node Prototype <ul><li>Working with IP camera reference design with TI DM6446 DSP </li></ul><ul><li>Single board design for video analytics </li></ul><ul><li>Power over IP </li></ul><ul><li>Embedded Video Analytics capability </li></ul>
    11. 11. <ul><li>SensorWeb client receives an incoming sensor observation or alert (e.g., a sensor mote detects motion) </li></ul><ul><li>List of EIP’s are checked to see if this observation triggers an action </li></ul><ul><li>A tasking message is created and sent by the EIP to other sensors for subsequent action (e.g., camera slewed to mote sensor coordinates to view and process video for vehicle activity </li></ul>Sensor Web Event Initiated Procedures (EIP) Sensor Web Framework Sensor Web (PULSENet TM ) SensorWeb Client
    12. 12. Video and Mote Sensor Collaboration Example Camera Slews to Location Video Event Detection Mote Sensor Alert
    13. 13. Video and Mote Sensor Collaboration Example ABANDONED OBJECT example Video Event Detection
    14. 14. Video Monitoring for Large Area Surveillance <ul><li>Video Detection and Tracking algorithms need to accommodate: </li></ul><ul><ul><li>Low frame rates of network cameras </li></ul></ul><ul><ul><li>Occlusions in wooded areas, low contrast environments </li></ul></ul><ul><ul><li>Camera panning and zooming as slewed by other sensor cueing </li></ul></ul><ul><ul><li>Dynamically respond to real-time triggers from different sensor observations </li></ul></ul><ul><ul><li>Detecting and classifying people in the far field </li></ul></ul><ul><li>Use of ASF provides much needed coverage; leverage low-cost averaging of sensor resources by muxing video sensor </li></ul>
    15. 15. Mote Sensor Directed Video Detection & Tracking Example Area of interest indicated by mote field (ASF) Detection of people by Smart Video Node (SVN)
    16. 16. Video Analysis and Detection <ul><li>Developed video object detection, classification, tracking & </li></ul><ul><li>event detection algorithms to best suit Border Patrol environments </li></ul><ul><ul><li>viewing conditions (e.g., medium to high nadir views) </li></ul></ul><ul><ul><li>standoff ranges (e.g., visibility and target distance), environmental effects (e.g., day, night, slight camera jitter due to winds on high mounted tower cameras) </li></ul></ul><ul><ul><li>sensor modalities (e.g., object detection & classification in VIS & IR sensor videos) </li></ul></ul>
    17. 17. Sensor Web Framework Functionality <ul><ul><li>A software framework with open standards-based suite of web services and encodings that enable: </li></ul></ul><ul><ul><ul><li>Remote discovery of compliant (OGC) sensors </li></ul></ul></ul><ul><ul><ul><li>Storage and retrieval of sensor observations </li></ul></ul></ul><ul><ul><ul><li>Subscription to and notification from desired sensor alerts </li></ul></ul></ul><ul><ul><ul><li>Remote sensor tasking based on observed alerts </li></ul></ul></ul><ul><ul><ul><li>Filtering sensor alerts and data based on temporal and geospatial constraints </li></ul></ul></ul>
    18. 18. Sensor Web (SWE) Encodings & Services <ul><li>Encodings: </li></ul><ul><ul><li>Sensor Model Language (SensorML): An XML-based format for completely describing and modeling a sensor </li></ul></ul><ul><ul><li>Observations and Measurements (O & M): An XML-based format for describing sensor observations </li></ul></ul><ul><ul><li>Transducer Markup Language (TML): An XML-based format for streaming sensor descriptions and data in real-time </li></ul></ul><ul><li>Services: </li></ul><ul><ul><li>Sensor Observation Service (SOS): A service for storing and accessing archived and near real-time sensor observations </li></ul></ul><ul><ul><li>Sensor Planning Service (SPS): A service for remote tasking of sensors such as cameras and UAVs </li></ul></ul><ul><ul><li>Sensor Alert Service (SAS): A service for subscribing to and being notified of sensor alerts </li></ul></ul><ul><ul><li>Web Notification Service (WNS): An asynchronous messaging service for long-running processes that works with the SPS and SAS </li></ul></ul>
    19. 19. Specific Current Effort (with DHS S&T): Conceptual CBP System (ASF-SensorWeb-SVN) -- Data Flow 1 2 3 4 5 6 7 8 9 9 10 Sensor Web implemented via PULSENet
    20. 20. Video Event Alarms on Mobile Device <ul><li>A light web server offers access to different forms of output </li></ul><ul><li>Marked-up MJPEG video streams </li></ul><ul><li>RSS feed for alarm event information </li></ul><ul><li>Can receive video events through a mobile Sensor Web Client </li></ul><ul><li>Use of ubiquitous GSM (SATCOM) data communication systems </li></ul>
    21. 21. Conclusions <ul><li>Collaboration of inexpensive, low-power auxiliary sensor field (ASF) to detect, refine (Pd, FAR), & cue video cameras provide a cost effective approach to surveillance video collection, storage, processing and information extraction </li></ul><ul><li>OGC-compliant Smart Video Node enables Video Event Detection in a Sensor Web Environment </li></ul><ul><li>Sensor Web platform provides an efficient means to manage sensor assets, observations and alarms, automatically task sensors and timely sharing of sensor data and extracted knowledge </li></ul>
    22. 22. <ul><li>Thank you! </li></ul>Sadiye Guler, intuVision [email_address] www.intuvisiontech.com Timothy D. Cole, NGIS [email_address] [email_address]

    ×