The document discusses advancements in video analysis and surveillance technology developed by Lawrence Livermore National Laboratory, focusing on achieving high-resolution data processing for anomaly detection over vast areas. It highlights the challenges faced, such as limited storage and poor signal strength, while showcasing innovative solutions for data compression and activity tracking, achieving significant reductions in data size. Potential applications of this technology include law enforcement, disaster response, and large crowd surveillance.

1. Sheila Vaidya
Deputy Program Director, Defense Programs,
Office of Strategic Outcomes
Lawrence Livermore National Laboratory
This work performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.

2. Small
country
LA basin
Large city
1012
1011
1010
109
Lawrence Livermore National Laboratory LLNL-PRES-559252
2
City block
1GB/min 1TB/min 1PB/min
108
107
106
66Mpix 2005 => Army,
AF
DOE
2007
DOE
2004
DOE 4Mpix,
2Hz, 2003
Small city
MIT LL 2Hz,
2009
DARPA 1.8Gpix, 10Hz,
2010
HDTV
m2 = # pixels
LLNL-led wide area surveillance camera development trajectory

3. Lawrence Livermore National Laboratory LLNL-PRES-559252
Constant Hawk ~
200 Mpix/sec
~25 sq. km at ~ 0.5m GSD
~100 Mbps
 Objective:
Video analysis at high resolution over entire field of
regard, for anomaly detection and profiling
 Challenges:
 Limited flight/ground storage
 Poor signal strength
 Minimal platform computing resources
 Low communication bandwidth to ground
 << eyes-on-ground to distill information
State-of-the-art:
Still compression JPEG-2000 5-10 X lossless
Video compression H.264 20-50 X lossy
Analysis algorithms embryonic
In Theater
~12 Mpixel
Can we exploit spatial and temporal coherency in video to assist data
compression and automated analysis?
3

4. Lawrence Livermore National Laboratory LLNL-PRES-559252
4
Innovative video conditioning and stabilization for orders of magnitude data reduction, coupled to
activity detection and tracking
 Compression: 100 –10,000X data reduction over raw collections
 Demonstrated 1000X (TBytes GBytes)
 Scalability: Adaptively assigned computational resources to regions of interest
 Hardware : Ground station cluster embedded on-board processing, optimized for size, weight
and power
 Resolution and Accuracy: Geo-located, super-resolution motion
imagery on “stationary” background
 Output: Areas classified by content, with vehicles/dismount tracks and
track behavior analysis
 Query: Fast real-time interactive search, query, and
playback (TiVo-like)

5. Lawrence Livermore National Laboratory LLNL-PRES-559252
5
Conventional camera stitch Subpixel merge
Static corrections Dynamic, variable shape
corrections
Large residual-from-background with
static-only corrections

6. Lawrence Livermore National Laboratory LLNL-PRES-559252
6

7. 10x20x10 = 2000x background compression
Lawrence Livermore National Laboratory LLNL-PRES-559252
7
θ
θ
Persistent flight
loops:>10x re-use
of background
80 background
frames per quadrant
compress to 4 cubic
control images: 20x
reduction
Standard encoders
compress 10x spatially
<0.1% mover pixels 2000x compression;
Net ~ 1000X data shrinkage: data dependent

8. ROI Selection Vehicles ROI’s: Automatically Generated
Lawrence Livermore National Laboratory LLNL-PRES-559252
8
Analyst Prioritized/ Vehicles – high
Roads – medium
Buildings/Field – low
- Scene segmented

9. Stabilized Imagery Tracks Activity/Events
NW
Threats* 10
Events* 103
Tracks* 106
Geo-regist. Pixels 1013
Lawrence Livermore National Laboratory LLNL-PRES-559252
9
1
100 200 300 400 500 600
50
100
150
200
250
300
350
400
450
Analyst Cueing
Threat Assessment
• Activity classification and scenario discrimination
• Visual template for merging of multi-sensor data
2
100 200 300 400 500 600
50
100
150
200
250
300
350
400
450
20
100 200 300 400 500 600
50
100
150
200
250
300
350
400
450
(state | state ) t 1 t P 
Raw Pixels/ Day 1015
Forensics
Predictive

10. Single Mission View: Handoff Detection
Lawrence Livermore National Laboratory LLNL-PRES-559252
10
Multiple Mission View: Complex Queries
With filters for
• Detecting activity
• Removing spurious tracks
• Height mapping
• Time stamping events
• …

11. Lawrence Livermore National Laboratory LLNL-PRES-559252
11
Pattern-of-life
using traffic
heat-maps

12. Lawrence Livermore National Laboratory LLNL-PRES-559252
12
Wide area video data processing for forensic backtracking and real time anomaly detection
 Orders of magnitude data reduction
 Spatially accurate , high resolution video exploitation and activity analysis
– Mapped onto 3D terrain
 On-board video analysis and streaming
– Merged multi-sensor data product playback at the desktop
 Rapid search architecture with network extraction & pattern of behavior from
– Varying spatial and temporal time domains
 Potential application areas:
− Coastal/Border Surveillance, Law Enforcement, Disaster Imagery, Wideland Firefighting,
Search and Rescue, Large Crowd Events …
Single image Multiple images