Presentation of Personal Work
Yosen Chen
Yosen Chen, Copyright Reserved
Human Machine Interaction
Basic Visual Tracker
Multi-Cue Particle Filter Tracker
Consis...
Implementation Tool
Microsoft Visual C++ with OpenCV Library
IPC (Interprocess Communication) Library
open source @ Car...
Yosen Chen, Copyright Reserved
Human Machine Interaction
Basic Visual Tracker
Multi-Cue Particle Filter Tracker
Consis...
Intelligent Interactive Engine
Image processing
Webcam
Image data
Show message
& animation
Display Central
Server
Webcam
O...
Yosen Chen, Copyright Reserved
Human Machine Interaction
Basic Visual Tracker
Multi-Cue Particle Filter Tracker
Consis...
Basic Visual Tracking Framework
(Color)
Yosen Chen, Copyright Reserved
Camera Real-time
image
Feature Extraction
RGB
to
HS...
Demo Video: Ball Tracking
追球後段
Yosen Chen, Copyright Reserved 8
Peak climbing: MeanShift
Related Works
Varieties of Feature Extraction/Evaluation
Feature evaluation: template matching
Background subtraction
...
Summary
Advantages:
Simple & Fast  low-cost
Each pixel (RGBHSV, backprojection) is processed
independently  good for...
Yosen Chen, Copyright Reserved
Human Machine Interaction
Basic Visual Tracker
Multi-Cue Particle Filter Tracker
Consis...
SIR+SIS Particle Filter Tracking
Yosen Chen, Copyright Reserved
State candidates
(Particle set) Color
Possible region
samp...
Multi-Cue Likelihood Evaluation
Yosen Chen, Copyright Reserved
For each state
candidate
Inner color extraction
Color
Simil...
Demo Video: 4D Bottle Tracking
(x, y, r, θ)
Yosen Chen, Copyright Reserved 14
Candidate group behavior
Bottle tracking vid...
Yosen Chen, Copyright Reserved 15
Interaction
System
Interaction
Triggering
Object
Checking
Tracking
& Interaction
Losing
...
Demo Video:
Interactive Game – Brick Breaker
Yosen Chen, Copyright Reserved
Interaction system: 4 status switching
4 degre...
Summary
High flexibility
State dimension
Motion model
Easy to add new features & tune weights in tracker
for different...
Yosen Chen, Copyright Reserved
Human Machine Interaction
Basic Visual Tracker
Multi-Cue Particle Filter Tracker
Consis...
Related applications:
Visual surveillance
Remote monitoring
Intelligent security system
Introduction
19Yosen Chen, Cop...
Yosen Chen, Copyright Reserved
Human Machine Interaction
Basic Visual Tracker
Multi-Cue Particle Filter Tracker
Consis...
Multi-Human Tracking Flow
Yosen Chen, Copyright Reserved
Prediction Evaluation Resampling
Prediction Evaluation Resampling...
Multi-Cue Likelihood Evaluation
for Head Tracking
Yosen Chen, Copyright Reserved
High color likelihood Low color likelihoo...
Face Detection Implementation Details
Yosen Chen, Copyright Reserved
SIS Face Detection Flow
=======================Pseudo...
Face Detection Demo
Yosen Chen, Copyright Reserved
Skin color backproject
Skin color
blob finding
SIS Face detection
Shape...
People Overlapping Problem
Yosen Chen, Copyright Reserved
When people get close to each other…
State candidates are attrac...
Solutions of People Overlapping
Yosen Chen, Copyright Reserved
Prediction
Evaluation
(Individual
+ joint)
ResamplingJoint ...
Implementation
Yosen Chen, Copyright Reserved
Prediction
Evaluation
(Individual
+ joint)
ResamplingJoint state
check
Depth...
Depth Order Evaluation with Heights
Depth order estimation with height knowledge is accurate
Even when people are not over...
Summary of Multi-Human Tracker
We extend the concept of single-object tracking
to multi-object tracker
We propose a enti...
Yosen Chen, Copyright Reserved
Human Machine Interaction
Basic Visual Tracker
Multi-Cue Particle Filter Tracker
Consis...
Multi-Camera Tracking System
Yosen Chen, Copyright Reserved
Captured image
Local camera #A
multi-head tracker
 1 2,A B
t...
: image observation
: 3D line from image head center
j
rt
j
rt
O
L
 “Are these cameras tracking the same people? ”
 Let ...
 Determine observation correspondence using Multi-Feature Similarity
 Multi-Feature Observation Similarity contains:
He...
 2D Position on image plane  3D head line in real space
 3D ground point in real space  2D position on image plane
obs...
How to Implement 3D-2D Mapping
Yosen Chen, Copyright Reserved
Class: Plane3D
Member functions:
isPointOnPlane
getDistFro...
 Human Body Model Based on Head Tracking
Matching body color
 Human Body is separated into
4 blocks
ground-point
head
g...
Observation Correspondence across Cameras
Object Conf = 3
Object Conf = 2
Matching body color (Region-based body color com...
Body Occlusion Handling
Yosen Chen, Copyright Reserved 38
Prediction
Evaluation
(Individual
+ joint)
ResamplingJoint state...
Labeling Consistency with Confidence Level
Yosen Chen, Copyright Reserved
Observation correspondence process should follow...
Yosen Chen, Copyright Reserved
Human Machine Interaction
Basic Visual Tracker
Multi-Cue Particle Filter Tracker
Consis...
Consistent Labeling with People Re-Entering
Yosen Chen, Copyright Reserved
Captured image
Local camera #A
multi-head track...
Association Processes with Matching Confidence Orders
Hierarchical Association with target databases
 Objects is associ...
Association Process
 Association process:
 To determine if observed object and database are from the same
person
 Featu...
How to Model & Update Database
Adaptive Gaussian Mixture Model
3 color candidates (histogram & weight) to describe 1 bod...
Database Operation Demonstration
Consistent Labeling Using Target Database
Histogram candidatesCamera#2 histogramCamera#1...
Consistent Labeling Using Target Database
Database Demonstration (cont’)
46Yosen Chen, Copyright Reserved
Multi-People Correspondence
Yosen Chen, Copyright Reserved 47
1. Multi-people depth order
2. body occlusion handling
3. Mu...
Multi-People Re-entering
Yosen Chen, Copyright Reserved 48
System Operation Flow:
1. Observation correspondence & Body occ...
Future Works…
Extend to the 3rd Camera
Challenge of communication load, algorithm complexity, and
tracking/labeling stab...
Thank You
BACKUP SLIDES
Yosen Chen, Copyright Reserved 51
Motion Detection : Optical Flow
(Modified from OpenCV Sample Code)
Input gray-scale
image
Optical point
extraction
Optical...
BACKUP SLIDES END
Yosen Chen, Copyright Reserved 53
Upcoming SlideShare
Loading in …5
×

Presentation of Visual Tracking

736 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
736
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
36
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Presentation of Visual Tracking

  1. 1. Presentation of Personal Work Yosen Chen
  2. 2. Yosen Chen, Copyright Reserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 2
  3. 3. Implementation Tool Microsoft Visual C++ with OpenCV Library IPC (Interprocess Communication) Library open source @ Carnegie Mellon University Yosen Chen, Copyright Reserved 3
  4. 4. Yosen Chen, Copyright Reserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 4
  5. 5. Intelligent Interactive Engine Image processing Webcam Image data Show message & animation Display Central Server Webcam Our goal is to construct an interactive engine which utilizes target motion as instruction input to the server. Yosen Chen, Copyright Reserved 5
  6. 6. Yosen Chen, Copyright Reserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 6
  7. 7. Basic Visual Tracking Framework (Color) Yosen Chen, Copyright Reserved Camera Real-time image Feature Extraction RGB to HSV Hue Saturation Value 256-to-16 color Qtz Similarity Evaluation Reference Image Reference 16-color Histogram Hue Saturation Value Reference Color Backprojection Probability distribution figure Probability Peak Search/Climbing Target Information Estimation (Position, size,...) If iterative Reference 16-color Histogram Reference Color Backprojection Input Hue Image Probability distribution figure Color similarity score Iterative Search (Motion Prediction) Previous Current Next Camera View Region of Interest 7
  8. 8. Demo Video: Ball Tracking 追球後段 Yosen Chen, Copyright Reserved 8 Peak climbing: MeanShift
  9. 9. Related Works Varieties of Feature Extraction/Evaluation Feature evaluation: template matching Background subtraction Reference color model: YCrCb, RGB Varieties of Peak Search/Climbing Gravity center Max-Likelihood MeanShift Yosen Chen, Copyright Reserved Cb Cr 9
  10. 10. Summary Advantages: Simple & Fast  low-cost Each pixel (RGBHSV, backprojection) is processed independently  good for parallel processing Drawbacks: Hard to combine with more object features  EX: edge information (object shape, contour)  All object features 2D probability distribution figure Hard to cope with higher-dimension target state  EX: object orientation (rotation angle) Yosen Chen, Copyright Reserved ??suitable 10
  11. 11. Yosen Chen, Copyright Reserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 11
  12. 12. SIR+SIS Particle Filter Tracking Yosen Chen, Copyright Reserved State candidates (Particle set) Color Possible region sampling SIR Real-time State Candidate Prediction Frame Multi-Cue Candidate Evaluation      1 m m t tp X X       1 1 , , , 1,..., mm t t X x y r m N       m t tp O X State Candidate Resampling            , equal weight m m t t t k t X p O X X tO SIS Shape Motion weighting Particles from last candidate set Particles generated from current image Tracking result on Frame tO  m tX Best State Candidate Sequential Importance Sampling (SIS) Particle Filter Sampling Importance Resampling (SIR) Particle Filter Color backproject + filtering + best contour finding Initial State Estimation State Prediction State Evaluation  Find Best Candidate Resampling Last Candidates based on weights State Prediction State Evaluation  Find Best Candidate Last candidate set {state candidate, weight} 12
  13. 13. Multi-Cue Likelihood Evaluation Yosen Chen, Copyright Reserved For each state candidate Inner color extraction Color Similarity Correlation-based comparison Color weight Reference color modelColor Multi-Cue Candidate Evaluation    m t tp O XShape Motion weighting  m tX tO inner outer High continuity Low continuity id maxd ig      , , , mm t t X x y r  Shape Similarity Motion continuity (logic check) outer color extraction Shape distance State difference with last tracking result id  Ih i  Oh i  REFh i Edge strength ig Max matched size 1 null MAX Nr r N        Symmetry 1i N id d   , , ,x y r     Shape weight Motion weight  Overall candidate score    m t tp O X 13
  14. 14. Demo Video: 4D Bottle Tracking (x, y, r, θ) Yosen Chen, Copyright Reserved 14 Candidate group behavior Bottle tracking video Blue: SIS (from current color projection) Green: SIR (from last candidate set) Red: Best candidate as tracking result
  15. 15. Yosen Chen, Copyright Reserved 15 Interaction System Interaction Triggering Object Checking Tracking & Interaction Losing Handling Start Motion detection by Optical flow Image Initialize/Reset all parameters Motion detected? Show Checking Box Record target information and features Target Info is robust? Tracking algorithm Quality index > threshold ? Wait until user puts target in “Losing Box” Is target put back? Visual reality interacting program image Yes No Yes Yes No No Yes No Image
  16. 16. Demo Video: Interactive Game – Brick Breaker Yosen Chen, Copyright Reserved Interaction system: 4 status switching 4 degrees of freedom: location x & y, size, rotation angle 16 Motion Interaction Function Interactive Game Example
  17. 17. Summary High flexibility State dimension Motion model Easy to add new features & tune weights in tracker for different applications and scenarios Trade-off: Feature description vs. Computation time Impoverishment(intensive) vs. Degeneracy(sparse)  Many particles concentrate on high-probability region  Many particles are evaluated at low-probability region  Number of particles ~ Complexity Yosen Chen, Copyright Reserved 17
  18. 18. Yosen Chen, Copyright Reserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 18
  19. 19. Related applications: Visual surveillance Remote monitoring Intelligent security system Introduction 19Yosen Chen, Copyright Reserved
  20. 20. Yosen Chen, Copyright Reserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 20
  21. 21. Multi-Human Tracking Flow Yosen Chen, Copyright Reserved Prediction Evaluation Resampling Prediction Evaluation Resampling Prediction Evaluation Resampling … State joint evaluation SIR Head tracking Multi-human tracking result Real-time captured image Skin color blob finding SIS Face detection Shape & location filter ROI location Sampling Face detector (OpenCV) Losing Track Detector (score, counter) Delete tracker Create new tracker 1. SIS face detector  SIR human trackers 2. Multi-human state joint evaluation 3. Bad evaluated tracker  losing track detector  face detector  delete tracker 21
  22. 22. Multi-Cue Likelihood Evaluation for Head Tracking Yosen Chen, Copyright Reserved High color likelihood Low color likelihood outer inner Skin color mask Hair color mask outer inner Skin color mask Hair color mask   1 , , t x y r   , , t x y r   1 , , t x y r   , , t x y r High continuity Low continuity 1n 2n 6n 9n 13n Shape distance 1n 2n 6n 9n 13n Num of Matched point ig Edge strength 1n 2n 6n 9n 13n symmetry Color •Masked skin color comparison •Masked hair color comparison Texture •Texture SAD similarity Edge •Shape distance •Number of matched point •Edge strength •Contour symmetry fitting Motion •State continuity 22
  23. 23. Face Detection Implementation Details Yosen Chen, Copyright Reserved SIS Face Detection Flow =======================Pseudo code======================== cvCalcBackProject; cvEqualizeHist + cvThreshold; cvResize + cvErode + cvDilate; cvFindContours(LinkedList detect_contour); for( ; detect_contour != 0 ; detect_contour = detect_contour->next ) { if (Area < threshold) continue; Use 8 points average to approximate center (xA, yA); if (distance between all target T and (xA, yA) > threshold) { calculate geometric center (xC, yC), deviation σX, σY of detect_contour; if ((σX > 0.7*σY) && (σY > 0.8*σX) && (σX /sqrt(Area) < 0.3) && (σY /sqrt(Area) < 0.4)) { Draw k random samples (x, y) ~ N((xC, yC), (σX, σY)); where k = (Area/MIN_AREA). if (frontalFaceTest(x, y)->detect() || profileFaceTest(x, y)->detect()) return location (x, y) as start point of new tracker; } } } return NOT_DETECTED; Skin color blob finding SIS Face detection Shape & location filter ROI location Sampling Face detector (OpenCV) Losing Track Detector Create tracker Delete tracker 23
  24. 24. Face Detection Demo Yosen Chen, Copyright Reserved Skin color backproject Skin color blob finding SIS Face detection Shape & location filter ROI location Sampling Face detector (OpenCV) Losing Track Detector Create tracker Delete tracker 24Skin color backproject (down-size & morphology) Last detecting region Blue region: Last detected face Red region: color is matched with skin color. Tracking System GUI panel
  25. 25. People Overlapping Problem Yosen Chen, Copyright Reserved When people get close to each other… State candidates are attracted by other people because trackers don’t know each other!! Prediction Evaluation (individual) Resampling Original Head Trackers 25
  26. 26. Solutions of People Overlapping Yosen Chen, Copyright Reserved Prediction Evaluation (Individual + joint) ResamplingJoint state check Depth order check Body occlusion handling Target information storage Modified Head Trackers for People Overlapping For people re-entering problem 1. Use joint evaluation to avoid candidates merging for people overlapping 2. Solution#1 (left): give higher joint score for candidates further from other people  reduce risk of candidates merging but also lose tracking accuracy 3. Solution#2 (right): kill candidates too close to other people  better tracking accuracy 26 resVideo_FurtherIsGood2.mpg resVideo_CloseThenLowWeight2.mpg
  27. 27. Implementation Yosen Chen, Copyright Reserved Prediction Evaluation (Individual + joint) ResamplingJoint state check Depth order check Modified Head Trackers for People Overlapping Image distance check “if △x2 + △y2 < threshold, …” è group together target1 target2 target3 target4 target5 Grouping (2nd-order tree) root leaf leafleaf Target A candidate 1 (far from B) Target B last best candidate Target A candidate 2 (near B) Pointer to target1 Candidate Evaluation (individual + joint) Individual: (as previous mentioned) Color, shape, motion Joint: (not be attracted by other close targets) Distances to all other targets in the same group Pointer to target2 ……… Depth order: deform 2nd-tree into array, then quick sort by depth features Sort by depth features Depth features: For best candidates of each target Texture changing rate Contour matching score Distance from camera to target ground points Image source 1:Front 2:Rear Candidate A2 has high penalty (since Candidate A2 are too close to B !!) 27
  28. 28. Depth Order Evaluation with Heights Depth order estimation with height knowledge is accurate Even when people are not overlapping. Yosen Chen, Copyright Reserved 28
  29. 29. Summary of Multi-Human Tracker We extend the concept of single-object tracking to multi-object tracker We propose a entire tracking system Face detection  multi-human trackers  detect people leaving Address the solution of people overlapping Robust multi-human tracker is indeed the basis of correspondence process within multi-camera tracking system Yosen Chen, Copyright Reserved 29
  30. 30. Yosen Chen, Copyright Reserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 30
  31. 31. Multi-Camera Tracking System Yosen Chen, Copyright Reserved Captured image Local camera #A multi-head tracker  1 2,A B t tz z2tH 1 A tz 2 A tz 2 B tz 1 B tz Local camera #B multi-head tracker Captured image Central Server Multi-camera multi-observation correspondence Person#1 Person#2  1 2 1,A B t tz ztH Identify the relationship of observations in different cameras Local trackers: multi-human tracking and occlusion handling Central server: observation correspondence across cameras 31
  32. 32. : image observation : 3D line from image head center j rt j rt O L  “Are these cameras tracking the same people? ”  Let be the rth observation tracked by camera at time Problem of Observation Correspondence across Cameras  ,j j j rt rt rtO Lz j t How many people exist in surveillance area? Ground plane Camera j Camera k 1 j tO 2 j tO 3 j tO1 k tO 2 k tO 3 k tO 1 j tL 2 j tL 3 j tL 1 k tL 2 k tL 3 k tL Yosen Chen, Copyright Reserved 32
  33. 33.  Determine observation correspondence using Multi-Feature Similarity  Multi-Feature Observation Similarity contains: Head line intersection verification Body color matching by blocks Confidence level Label history consistency How to Determine Observation Correspondence across Cameras Correspondence Flow Calculate 3D geometric intersection Confidence level Are they from the same person? Camera#A Camera#B Body Color comparison by block Label history consistency 2D Body image block allocation No, if no reasonable intersection No, if color unmatched Yes, 2 observations are from the same person Yosen Chen, Copyright Reserved 33
  34. 34.  2D Position on image plane  3D head line in real space  3D ground point in real space  2D position on image plane observation r Optic center Image plane of camera j Ground plane Optic center observation q Image plane of camera k 3D Geometric Correspondence across Cameras Calibrated camera system Image 2D↔3D coordinate transfer is constructed off-line Head Point height Estimated Ground Point Yosen Chen, Copyright Reserved 34
  35. 35. How to Implement 3D-2D Mapping Yosen Chen, Copyright Reserved Class: Plane3D Member functions: isPointOnPlane getDistFromPoint getProjPointFromPoint Class: line3D Member functions: getPositionFromX,Y,Z isPointOnLine getDistFromPoint getProjPointFromPoint getLinePointOnPlane getMinDistPointFromLine Class: CameraLine3D Member functions: setByImageLocation setBy3DPosition How to verify the mapping accuracy? 3D Optic point (≈ Camera location) Class: CameraLine3D 2D image plane Reference plane in 3D space 3D Normal vector Reference 3D plane center (0,0) derived 35 CAM2 CAM1 Locate 3D points/lines on 2D image planes
  36. 36.  Human Body Model Based on Head Tracking Matching body color  Human Body is separated into 4 blocks ground-point head ground-point head Image boundary Observed ground-point Unobserved ground-point j rtz k qtz Camera kCamera j qtrt 3D Geometric-based Body Image Block Allocation Yosen Chen, Copyright Reserved 36 For some observations, not all the body blocks are unobservable. Here we take the number of observable blocks as confidence level of correspondence result
  37. 37. Observation Correspondence across Cameras Object Conf = 3 Object Conf = 2 Matching body color (Region-based body color comparison) Depth order: Front Depth order: RearBody Occlusion Handling (based on depth order and 3D-2D block mapping ) Yosen Chen, Copyright Reserved 37 Body appearance is available for correspondence process Body appearance is not available (Color turns darker)
  38. 38. Body Occlusion Handling Yosen Chen, Copyright Reserved 38 Prediction Evaluation (Individual + joint) ResamplingJoint state check Depth order check Body occlusion handling Target information storage Modified Head Trackers for People Overlapping
  39. 39. Labeling Consistency with Confidence Level Yosen Chen, Copyright Reserved Observation correspondence process should follow rules stated below Rule1. Claiming “match” for one pair will also imply claiming “collision” for all corresponding residual pairs, with the same confidence level Rule2. Only higher-confidence decisions can enforce changes of lower-confidence ones √ √ √ A B C √ 1 0 0 -4 √ 2 -4 -4 4 √ 3 0 0 -4 √ √ X A B C √ 1 0 0 0 √ 2 -4 -4 0 √ 3 0 0 0 √ √ A B √ 1 -3 0 √ 2 -4 -4 √ 3 3 -3 Camera#1 Camera#2 Conf = 4Conf = -4 1 2 3 A B C Camera#1 Camera#2 Conf = -4 1 2 3 A B Camera#1 Camera#2 Conf = 3 Conf = -4 Conf = -3 1 2 3 A B 39
  40. 40. Yosen Chen, Copyright Reserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 40
  41. 41. Consistent Labeling with People Re-Entering Yosen Chen, Copyright Reserved Captured image Local camera #A multi-head tracker  1 2,A B t tz z2tH 1 A tz 2 A tz 2 B tz 1 B tz Local camera #B multi-head tracker Captured image Central Server Multi-camera multi-observation correspondence Person#1 Person#2  1 2 1,A B t tz ztH Identify the relationship of observations in different cameras 3tX2tX1tX associate with target databases 41
  42. 42. Association Processes with Matching Confidence Orders Hierarchical Association with target databases  Objects is associated to target databases  1 m ht h H   1 n it i X  Consistent Labeling Using Target Database 3tX 2tX 1tX 4tX 1tH i TARL High Low 1 OBJL 1 TARL 2 TARL 3 TARL 4 TARL compare with descendent order : target confidence level (in database) : object confidence level (from observation) i TAR i OBJ L L if association success update related database else create new database PS: database association process is only for objects with conf. level ≧2 Yosen Chen, Copyright Reserved 42
  43. 43. Association Process  Association process:  To determine if observed object and database are from the same person  Features are similar to observation correspondence process  People height  Body color comparison by blocks  Association history consistency Yosen Chen, Copyright Reserved Observation-Database Association Flow: Check height difference Are they from the same person? Body Color comparison by block Association history consistency No, if diff > threshold No, if color unmatched Yes, observations & this database set are from the same person 2tX 1tH From Camera observations People information already stored in databases 43
  44. 44. How to Model & Update Database Adaptive Gaussian Mixture Model 3 color candidates (histogram & weight) to describe 1 body block color Candidate’s histograms & weights are updated based on current color of MATCHED observations Color comparison with observed color (Define MATCHED or not)  Bhattacharyya distance from 3 candidates  3 weighted Gaussian kernels Yosen Chen, Copyright Reserved 44 Color histogram (1-D for demo) Candidate weight Candidate color Adaptive Gaussian Mixture Model Candidate#1 Candidate#3 Candidate#2 Candidate color & weight will be updated based on observations Candidate variance Inspired by [Stauffer., CVPR 2000]
  45. 45. Database Operation Demonstration Consistent Labeling Using Target Database Histogram candidatesCamera#2 histogramCamera#1 histogram Matched with camera#2 Matched with camera#1 Yosen Chen, Copyright Reserved 45
  46. 46. Consistent Labeling Using Target Database Database Demonstration (cont’) 46Yosen Chen, Copyright Reserved
  47. 47. Multi-People Correspondence Yosen Chen, Copyright Reserved 47 1. Multi-people depth order 2. body occlusion handling 3. Multi-camera multi-people correspondence is correct
  48. 48. Multi-People Re-entering Yosen Chen, Copyright Reserved 48 System Operation Flow: 1. Observation correspondence & Body occlusion handling 2. Associate with current databases  (Create then) update new information 3. Return people IDs (show by color) to each tracker of each camera
  49. 49. Future Works… Extend to the 3rd Camera Challenge of communication load, algorithm complexity, and tracking/labeling stability Performance Evaluation of Target Database Comparison with related works More challenges: Outdoors Surveillance (human body detection) Uncalibrated multi-camera system (Training) Motive camera (rotation 3D-2D mapping) Yosen Chen, Copyright Reserved 49
  50. 50. Thank You
  51. 51. BACKUP SLIDES Yosen Chen, Copyright Reserved 51
  52. 52. Motion Detection : Optical Flow (Modified from OpenCV Sample Code) Input gray-scale image Optical point extraction Optical point mapping & filtering Motion detection criteria Reset criteria Interaction trigger Optical flow Resettrigger Previous gray-scale image Store Optical Flow: 1. A technique to create edge points then map them between adjacent images. (edge tracking) 2. Better to analyze group behavior rather than individual (not accurate) 3. Can be utilized as motion detection tool 1. Not enough effective optical points on screen 2. Time out then refresh 1. Average velocity of moving points > threshold 2. Number of moving points > threshold Moving points: points with velocity > 3*average velocity Yosen Chen, Copyright Reserved 52
  53. 53. BACKUP SLIDES END Yosen Chen, Copyright Reserved 53

×