Yu-Sheng (Yosen) Chen, profile picture
Uploaded byYu-Sheng (Yosen) Chen
PDF, PPTX1,826 views

Presentation of Visual Tracking

This presentation discusses computer vision techniques for human tracking and interaction. It begins with an outline of the topics to be covered, including basic visual tracking, multi-cue particle filtering for tracking, multi-human tracking, multi-camera tracking, and handling re-entering people. It then describes implementations of basic color-based tracking, particle filtering with multiple cues, and using particle filtering for human head tracking. Challenges with overlapping people are addressed through joint candidate evaluation and sorting by depth. The multi-camera system correlates tracks across cameras to identify corresponding people. Overall, the presentation explains a complete visual tracking and surveillance system using computer vision algorithms.

Embed presentation

Download as PDF, PPTX
Presentation of Personal Work Yosen Chen
Yosen Chen, Copyright Reserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 2
Implementation Tool Microsoft Visual C++ with OpenCV Library IPC (Interprocess Communication) Library open source @ Carnegie Mellon University Yosen Chen, Copyright Reserved 3
Yosen Chen, Copyright Reserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 4
Intelligent Interactive Engine Image processing Webcam Image data Show message & animation Display Central Server Webcam Our goal is to construct an interactive engine which utilizes target motion as instruction input to the server. Yosen Chen, Copyright Reserved 5
Yosen Chen, Copyright Reserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 6
Basic Visual Tracking Framework (Color) Yosen Chen, Copyright Reserved Camera Real-time image Feature Extraction RGB to HSV Hue Saturation Value 256-to-16 color Qtz Similarity Evaluation Reference Image Reference 16-color Histogram Hue Saturation Value Reference Color Backprojection Probability distribution figure Probability Peak Search/Climbing Target Information Estimation (Position, size,...) If iterative Reference 16-color Histogram Reference Color Backprojection Input Hue Image Probability distribution figure Color similarity score Iterative Search (Motion Prediction) Previous Current Next Camera View Region of Interest 7
Demo Video: Ball Tracking 追球後段 Yosen Chen, Copyright Reserved 8 Peak climbing: MeanShift
Related Works Varieties of Feature Extraction/Evaluation Feature evaluation: template matching Background subtraction Reference color model: YCrCb, RGB Varieties of Peak Search/Climbing Gravity center Max-Likelihood MeanShift Yosen Chen, Copyright Reserved Cb Cr 9
Summary Advantages: Simple & Fast  low-cost Each pixel (RGBHSV, backprojection) is processed independently  good for parallel processing Drawbacks: Hard to combine with more object features  EX: edge information (object shape, contour)  All object features 2D probability distribution figure Hard to cope with higher-dimension target state  EX: object orientation (rotation angle) Yosen Chen, Copyright Reserved ??suitable 10
Yosen Chen, Copyright Reserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 11
SIR+SIS Particle Filter Tracking Yosen Chen, Copyright Reserved State candidates (Particle set) Color Possible region sampling SIR Real-time State Candidate Prediction Frame Multi-Cue Candidate Evaluation      1 m m t tp X X       1 1 , , , 1,..., mm t t X x y r m N       m t tp O X State Candidate Resampling            , equal weight m m t t t k t X p O X X tO SIS Shape Motion weighting Particles from last candidate set Particles generated from current image Tracking result on Frame tO  m tX Best State Candidate Sequential Importance Sampling (SIS) Particle Filter Sampling Importance Resampling (SIR) Particle Filter Color backproject + filtering + best contour finding Initial State Estimation State Prediction State Evaluation  Find Best Candidate Resampling Last Candidates based on weights State Prediction State Evaluation  Find Best Candidate Last candidate set {state candidate, weight} 12
Multi-Cue Likelihood Evaluation Yosen Chen, Copyright Reserved For each state candidate Inner color extraction Color Similarity Correlation-based comparison Color weight Reference color modelColor Multi-Cue Candidate Evaluation    m t tp O XShape Motion weighting  m tX tO inner outer High continuity Low continuity id maxd ig      , , , mm t t X x y r  Shape Similarity Motion continuity (logic check) outer color extraction Shape distance State difference with last tracking result id  Ih i  Oh i  REFh i Edge strength ig Max matched size 1 null MAX Nr r N        Symmetry 1i N id d   , , ,x y r     Shape weight Motion weight  Overall candidate score    m t tp O X 13
Demo Video: 4D Bottle Tracking (x, y, r, θ) Yosen Chen, Copyright Reserved 14 Candidate group behavior Bottle tracking video Blue: SIS (from current color projection) Green: SIR (from last candidate set) Red: Best candidate as tracking result
Yosen Chen, Copyright Reserved 15 Interaction System Interaction Triggering Object Checking Tracking & Interaction Losing Handling Start Motion detection by Optical flow Image Initialize/Reset all parameters Motion detected? Show Checking Box Record target information and features Target Info is robust? Tracking algorithm Quality index > threshold ? Wait until user puts target in “Losing Box” Is target put back? Visual reality interacting program image Yes No Yes Yes No No Yes No Image
Demo Video: Interactive Game – Brick Breaker Yosen Chen, Copyright Reserved Interaction system: 4 status switching 4 degrees of freedom: location x & y, size, rotation angle 16 Motion Interaction Function Interactive Game Example
Summary High flexibility State dimension Motion model Easy to add new features & tune weights in tracker for different applications and scenarios Trade-off: Feature description vs. Computation time Impoverishment(intensive) vs. Degeneracy(sparse)  Many particles concentrate on high-probability region  Many particles are evaluated at low-probability region  Number of particles ～ Complexity Yosen Chen, Copyright Reserved 17
Yosen Chen, Copyright Reserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 18
Related applications: Visual surveillance Remote monitoring Intelligent security system Introduction 19Yosen Chen, Copyright Reserved
Yosen Chen, Copyright Reserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 20
Multi-Human Tracking Flow Yosen Chen, Copyright Reserved Prediction Evaluation Resampling Prediction Evaluation Resampling Prediction Evaluation Resampling … State joint evaluation SIR Head tracking Multi-human tracking result Real-time captured image Skin color blob finding SIS Face detection Shape & location filter ROI location Sampling Face detector (OpenCV) Losing Track Detector (score, counter) Delete tracker Create new tracker 1. SIS face detector  SIR human trackers 2. Multi-human state joint evaluation 3. Bad evaluated tracker  losing track detector  face detector  delete tracker 21
Multi-Cue Likelihood Evaluation for Head Tracking Yosen Chen, Copyright Reserved High color likelihood Low color likelihood outer inner Skin color mask Hair color mask outer inner Skin color mask Hair color mask   1 , , t x y r   , , t x y r   1 , , t x y r   , , t x y r High continuity Low continuity 1n 2n 6n 9n 13n Shape distance 1n 2n 6n 9n 13n Num of Matched point ig Edge strength 1n 2n 6n 9n 13n symmetry Color •Masked skin color comparison •Masked hair color comparison Texture •Texture SAD similarity Edge •Shape distance •Number of matched point •Edge strength •Contour symmetry fitting Motion •State continuity 22
Face Detection Implementation Details Yosen Chen, Copyright Reserved SIS Face Detection Flow =======================Pseudo code======================== cvCalcBackProject; cvEqualizeHist + cvThreshold; cvResize + cvErode + cvDilate; cvFindContours(LinkedList detect_contour); for( ; detect_contour != 0 ; detect_contour = detect_contour->next ) { if (Area < threshold) continue; Use 8 points average to approximate center (xA, yA); if (distance between all target T and (xA, yA) > threshold) { calculate geometric center (xC, yC), deviation σX, σY of detect_contour; if ((σX > 0.7*σY) && (σY > 0.8*σX) && (σX /sqrt(Area) < 0.3) && (σY /sqrt(Area) < 0.4)) { Draw k random samples (x, y) ~ N((xC, yC), (σX, σY)); where k = (Area/MIN_AREA). if (frontalFaceTest(x, y)->detect() || profileFaceTest(x, y)->detect()) return location (x, y) as start point of new tracker; } } } return NOT_DETECTED; Skin color blob finding SIS Face detection Shape & location filter ROI location Sampling Face detector (OpenCV) Losing Track Detector Create tracker Delete tracker 23
Face Detection Demo Yosen Chen, Copyright Reserved Skin color backproject Skin color blob finding SIS Face detection Shape & location filter ROI location Sampling Face detector (OpenCV) Losing Track Detector Create tracker Delete tracker 24Skin color backproject (down-size & morphology) Last detecting region Blue region: Last detected face Red region: color is matched with skin color. Tracking System GUI panel
People Overlapping Problem Yosen Chen, Copyright Reserved When people get close to each other… State candidates are attracted by other people because trackers don’t know each other!! Prediction Evaluation (individual) Resampling Original Head Trackers 25
Solutions of People Overlapping Yosen Chen, Copyright Reserved Prediction Evaluation (Individual + joint) ResamplingJoint state check Depth order check Body occlusion handling Target information storage Modified Head Trackers for People Overlapping For people re-entering problem 1. Use joint evaluation to avoid candidates merging for people overlapping 2. Solution#1 (left): give higher joint score for candidates further from other people  reduce risk of candidates merging but also lose tracking accuracy 3. Solution#2 (right): kill candidates too close to other people  better tracking accuracy 26 resVideo_FurtherIsGood2.mpg resVideo_CloseThenLowWeight2.mpg
Implementation Yosen Chen, Copyright Reserved Prediction Evaluation (Individual + joint) ResamplingJoint state check Depth order check Modified Head Trackers for People Overlapping Image distance check “if △x2 + △y2 < threshold, …” è group together target1 target2 target3 target4 target5 Grouping (2nd-order tree) root leaf leafleaf Target A candidate 1 (far from B) Target B last best candidate Target A candidate 2 (near B) Pointer to target1 Candidate Evaluation (individual + joint) Individual: (as previous mentioned) Color, shape, motion Joint: (not be attracted by other close targets) Distances to all other targets in the same group Pointer to target2 ……… Depth order: deform 2nd-tree into array, then quick sort by depth features Sort by depth features Depth features: For best candidates of each target Texture changing rate Contour matching score Distance from camera to target ground points Image source 1:Front 2:Rear Candidate A2 has high penalty (since Candidate A2 are too close to B !!) 27
Depth Order Evaluation with Heights Depth order estimation with height knowledge is accurate Even when people are not overlapping. Yosen Chen, Copyright Reserved 28
Summary of Multi-Human Tracker We extend the concept of single-object tracking to multi-object tracker We propose a entire tracking system Face detection  multi-human trackers  detect people leaving Address the solution of people overlapping Robust multi-human tracker is indeed the basis of correspondence process within multi-camera tracking system Yosen Chen, Copyright Reserved 29
Yosen Chen, Copyright Reserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 30
Multi-Camera Tracking System Yosen Chen, Copyright Reserved Captured image Local camera #A multi-head tracker  1 2,A B t tz z2tH 1 A tz 2 A tz 2 B tz 1 B tz Local camera #B multi-head tracker Captured image Central Server Multi-camera multi-observation correspondence Person#1 Person#2  1 2 1,A B t tz ztH Identify the relationship of observations in different cameras Local trackers: multi-human tracking and occlusion handling Central server: observation correspondence across cameras 31
: image observation : 3D line from image head center j rt j rt O L  “Are these cameras tracking the same people? ”  Let be the rth observation tracked by camera at time Problem of Observation Correspondence across Cameras  ,j j j rt rt rtO Lz j t How many people exist in surveillance area? Ground plane Camera j Camera k 1 j tO 2 j tO 3 j tO1 k tO 2 k tO 3 k tO 1 j tL 2 j tL 3 j tL 1 k tL 2 k tL 3 k tL Yosen Chen, Copyright Reserved 32
 Determine observation correspondence using Multi-Feature Similarity  Multi-Feature Observation Similarity contains: Head line intersection verification Body color matching by blocks Confidence level Label history consistency How to Determine Observation Correspondence across Cameras Correspondence Flow Calculate 3D geometric intersection Confidence level Are they from the same person? Camera#A Camera#B Body Color comparison by block Label history consistency 2D Body image block allocation No, if no reasonable intersection No, if color unmatched Yes, 2 observations are from the same person Yosen Chen, Copyright Reserved 33
 2D Position on image plane  3D head line in real space  3D ground point in real space  2D position on image plane observation r Optic center Image plane of camera j Ground plane Optic center observation q Image plane of camera k 3D Geometric Correspondence across Cameras Calibrated camera system Image 2D↔3D coordinate transfer is constructed off-line Head Point height Estimated Ground Point Yosen Chen, Copyright Reserved 34
How to Implement 3D-2D Mapping Yosen Chen, Copyright Reserved Class: Plane3D Member functions: isPointOnPlane getDistFromPoint getProjPointFromPoint Class: line3D Member functions: getPositionFromX,Y,Z isPointOnLine getDistFromPoint getProjPointFromPoint getLinePointOnPlane getMinDistPointFromLine Class: CameraLine3D Member functions: setByImageLocation setBy3DPosition How to verify the mapping accuracy? 3D Optic point (≈ Camera location) Class: CameraLine3D 2D image plane Reference plane in 3D space 3D Normal vector Reference 3D plane center (0,0) derived 35 CAM2 CAM1 Locate 3D points/lines on 2D image planes
 Human Body Model Based on Head Tracking Matching body color  Human Body is separated into 4 blocks ground-point head ground-point head Image boundary Observed ground-point Unobserved ground-point j rtz k qtz Camera kCamera j qtrt 3D Geometric-based Body Image Block Allocation Yosen Chen, Copyright Reserved 36 For some observations, not all the body blocks are unobservable. Here we take the number of observable blocks as confidence level of correspondence result
Observation Correspondence across Cameras Object Conf = 3 Object Conf = 2 Matching body color (Region-based body color comparison) Depth order: Front Depth order: RearBody Occlusion Handling (based on depth order and 3D-2D block mapping ) Yosen Chen, Copyright Reserved 37 Body appearance is available for correspondence process Body appearance is not available (Color turns darker)
Body Occlusion Handling Yosen Chen, Copyright Reserved 38 Prediction Evaluation (Individual + joint) ResamplingJoint state check Depth order check Body occlusion handling Target information storage Modified Head Trackers for People Overlapping
Labeling Consistency with Confidence Level Yosen Chen, Copyright Reserved Observation correspondence process should follow rules stated below Rule1. Claiming “match” for one pair will also imply claiming “collision” for all corresponding residual pairs, with the same confidence level Rule2. Only higher-confidence decisions can enforce changes of lower-confidence ones √ √ √ A B C √ 1 0 0 -4 √ 2 -4 -4 4 √ 3 0 0 -4 √ √ X A B C √ 1 0 0 0 √ 2 -4 -4 0 √ 3 0 0 0 √ √ A B √ 1 -3 0 √ 2 -4 -4 √ 3 3 -3 Camera#1 Camera#2 Conf = 4Conf = -4 1 2 3 A B C Camera#1 Camera#2 Conf = -4 1 2 3 A B Camera#1 Camera#2 Conf = 3 Conf = -4 Conf = -3 1 2 3 A B 39
Yosen Chen, Copyright Reserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 40
Consistent Labeling with People Re-Entering Yosen Chen, Copyright Reserved Captured image Local camera #A multi-head tracker  1 2,A B t tz z2tH 1 A tz 2 A tz 2 B tz 1 B tz Local camera #B multi-head tracker Captured image Central Server Multi-camera multi-observation correspondence Person#1 Person#2  1 2 1,A B t tz ztH Identify the relationship of observations in different cameras 3tX2tX1tX associate with target databases 41
Association Processes with Matching Confidence Orders Hierarchical Association with target databases  Objects is associated to target databases  1 m ht h H   1 n it i X  Consistent Labeling Using Target Database 3tX 2tX 1tX 4tX 1tH i TARL High Low 1 OBJL 1 TARL 2 TARL 3 TARL 4 TARL compare with descendent order : target confidence level (in database) : object confidence level (from observation) i TAR i OBJ L L if association success update related database else create new database PS: database association process is only for objects with conf. level ≧2 Yosen Chen, Copyright Reserved 42
Association Process  Association process:  To determine if observed object and database are from the same person  Features are similar to observation correspondence process  People height  Body color comparison by blocks  Association history consistency Yosen Chen, Copyright Reserved Observation-Database Association Flow: Check height difference Are they from the same person? Body Color comparison by block Association history consistency No, if diff > threshold No, if color unmatched Yes, observations & this database set are from the same person 2tX 1tH From Camera observations People information already stored in databases 43
How to Model & Update Database Adaptive Gaussian Mixture Model 3 color candidates (histogram & weight) to describe 1 body block color Candidate’s histograms & weights are updated based on current color of MATCHED observations Color comparison with observed color (Define MATCHED or not)  Bhattacharyya distance from 3 candidates  3 weighted Gaussian kernels Yosen Chen, Copyright Reserved 44 Color histogram (1-D for demo) Candidate weight Candidate color Adaptive Gaussian Mixture Model Candidate#1 Candidate#3 Candidate#2 Candidate color & weight will be updated based on observations Candidate variance Inspired by [Stauffer., CVPR 2000]
Database Operation Demonstration Consistent Labeling Using Target Database Histogram candidatesCamera#2 histogramCamera#1 histogram Matched with camera#2 Matched with camera#1 Yosen Chen, Copyright Reserved 45
Consistent Labeling Using Target Database Database Demonstration (cont’) 46Yosen Chen, Copyright Reserved
Multi-People Correspondence Yosen Chen, Copyright Reserved 47 1. Multi-people depth order 2. body occlusion handling 3. Multi-camera multi-people correspondence is correct
Multi-People Re-entering Yosen Chen, Copyright Reserved 48 System Operation Flow: 1. Observation correspondence & Body occlusion handling 2. Associate with current databases  (Create then) update new information 3. Return people IDs (show by color) to each tracker of each camera
Future Works… Extend to the 3rd Camera Challenge of communication load, algorithm complexity, and tracking/labeling stability Performance Evaluation of Target Database Comparison with related works More challenges: Outdoors Surveillance (human body detection) Uncalibrated multi-camera system (Training) Motive camera (rotation 3D-2D mapping) Yosen Chen, Copyright Reserved 49
Thank You
BACKUP SLIDES Yosen Chen, Copyright Reserved 51
Motion Detection : Optical Flow (Modified from OpenCV Sample Code) Input gray-scale image Optical point extraction Optical point mapping & filtering Motion detection criteria Reset criteria Interaction trigger Optical flow Resettrigger Previous gray-scale image Store Optical Flow: 1. A technique to create edge points then map them between adjacent images. (edge tracking) 2. Better to analyze group behavior rather than individual (not accurate) 3. Can be utilized as motion detection tool 1. Not enough effective optical points on screen 2. Time out then refresh 1. Average velocity of moving points > threshold 2. Number of moving points > threshold Moving points: points with velocity > 3*average velocity Yosen Chen, Copyright Reserved 52
BACKUP SLIDES END Yosen Chen, Copyright Reserved 53

More Related Content

PPTX
Object detection with deep learning
bySushant Shrivastava
 
PDF
GTSRB Traffic Sign recognition using machine learning
byRupali Aher
 
PPTX
Digital watermarking
byAnkush Kr
 
PPTX
Automatic number-plate-recognition
byDevang Tailor
 
PPTX
Vehicle registration plate recognition system
byshailendra92
 
PPT
Face detection ppt
byPooja R
 
PPTX
Object tracking
bySri vidhya k
 
PDF
Moving Object Detection And Tracking Using CNN
byNITISHKUMAR1401
 
Object detection with deep learning
bySushant Shrivastava
 
GTSRB Traffic Sign recognition using machine learning
byRupali Aher
 
Digital watermarking
byAnkush Kr
 
Automatic number-plate-recognition
byDevang Tailor
 
Vehicle registration plate recognition system
byshailendra92
 
Face detection ppt
byPooja R
 
Object tracking
bySri vidhya k
 
Moving Object Detection And Tracking Using CNN
byNITISHKUMAR1401
 

What's hot

PPTX
Automatic number plate recognition using matlab
byChetanSingh134
 
PDF
Deep learning based object detection basics
byBrodmann17
 
PDF
Object Detection and Recognition
byIntel Nervana
 
PPTX
Object Detection using Deep Neural Networks
byUsman Qayyum
 
PDF
Object tracking presentation
byMrsShwetaBanait1
 
PPTX
Real Time Object Tracking
byVanya Valindria
 
PPTX
Automatic Number Plate Recognition(ANPR) System Project
byGulraiz Javaid
 
PPTX
Yolov3
bySHREY MOHAN
 
PPTX
Face Recognition
byarvind carpenter
 
PPT
Digital Image Forgery
byMohamed Talaat
 
PPTX
Object detection presentation
byAshwinBicholiya
 
PPTX
Vehicle Number Plate Recognition System
byprashantdahake
 
PPTX
Signature recognition
byVijju Lakkundi
 
PPTX
Face detection and recognition
byPankaj Thakur
 
DOCX
Smart Traffic Monitoring System Report
byALi Baker
 
PPTX
MATLAB Based Vehicle Number Plate Identification System using OCR
byGhanshyam Dusane
 
PPTX
face detection
bySmriti Tikoo
 
PPTX
road lane detection.pptx
byTheMusicFever
 
PPTX
Vehicle number plate recognition using matlab
byKongara Sudharshan
 
PPT
AUTOMATIC LICENSE PLATE RECOGNITION SYSTEM FOR INDIAN VEHICLE IDENTIFICATION ...
byKuntal Bhowmick
 
Automatic number plate recognition using matlab
byChetanSingh134
 
Deep learning based object detection basics
byBrodmann17
 
Object Detection and Recognition
byIntel Nervana
 
Object Detection using Deep Neural Networks
byUsman Qayyum
 
Object tracking presentation
byMrsShwetaBanait1
 
Real Time Object Tracking
byVanya Valindria
 
Automatic Number Plate Recognition(ANPR) System Project
byGulraiz Javaid
 
Yolov3
bySHREY MOHAN
 
Face Recognition
byarvind carpenter
 
Digital Image Forgery
byMohamed Talaat
 
Object detection presentation
byAshwinBicholiya
 
Vehicle Number Plate Recognition System
byprashantdahake
 
Signature recognition
byVijju Lakkundi
 
Face detection and recognition
byPankaj Thakur
 
Smart Traffic Monitoring System Report
byALi Baker
 
MATLAB Based Vehicle Number Plate Identification System using OCR
byGhanshyam Dusane
 
face detection
bySmriti Tikoo
 
road lane detection.pptx
byTheMusicFever
 
Vehicle number plate recognition using matlab
byKongara Sudharshan
 
AUTOMATIC LICENSE PLATE RECOGNITION SYSTEM FOR INDIAN VEHICLE IDENTIFICATION ...
byKuntal Bhowmick
 

Viewers also liked

PDF
Color based image processing , tracking and automation using matlab
byKamal Pradhan
 
PDF
2012 mdsp pr05 particle filter
bynozomuhamada
 
PPTX
Using particle filter for face tracking
byМария Михисор
 
PDF
Particle Filter
byTakahiro Inoue
 
PPT
CS221: HMM and Particle Filters
byzukun
 
PDF
Particle filtering in Computer Vision (2003)
byzukun
 
PDF
Particle Filters and Applications in Computer Vision
byzukun
 
PPT
Single person pose recognition and tracking
byJavier_Barbadillo
 
PPT
Pose
byRobin Low
 
PDF
Particle Filter Tracking in Python
byKohta Ishikawa
 
Color based image processing , tracking and automation using matlab
byKamal Pradhan
 
2012 mdsp pr05 particle filter
bynozomuhamada
 
Using particle filter for face tracking
byМария Михисор
 
Particle Filter
byTakahiro Inoue
 
CS221: HMM and Particle Filters
byzukun
 
Particle filtering in Computer Vision (2003)
byzukun
 
Particle Filters and Applications in Computer Vision
byzukun
 
Single person pose recognition and tracking
byJavier_Barbadillo
 
Pose
byRobin Low
 
Particle Filter Tracking in Python
byKohta Ishikawa
 

Similar to Presentation of Visual Tracking

PDF
A novel enhanced algorithm for efficient human tracking
byIJICTJOURNAL
 
PDF
Robust Tracking Via Feature Mapping Method and Support Vector Machine
byIRJET Journal
 
PDF
An improved particle filter tracking
byijcsit
 
PPTX
Object Detection and Tracking using Statistical and Stochastic Techniques
byVasuhiSamydurai1
 
PDF
IRJET- Application of MCNN in Object Detection
byIRJET Journal
 
PDF
Real time object tracking and learning using template matching
byeSAT Publishing House
 
PDF
IRJET- Full Body Motion Detection and Surveillance System Application
byIRJET Journal
 
PPTX
Real Time Object Dectection using machine learning
bypratik pratyay
 
PPTX
slide-171212080528.pptx
bySharanrajK22MMT1003
 
PDF
IRJET- Border Security using Computer Vision
byIRJET Journal
 
PDF
19_bibliography.pdf
byutak1
 
PDF
Dq4301702706
byIJERA Editor
 
PDF
Single person pose recognition and tracking
byJavier_Barbadillo
 
PDF
Lecture-1-CVIntroduction.pdf
byTechEvents1
 
PDF
Crime Detection using Machine Learning
byIRJET Journal
 
PDF
Intelligent video monitoring
byivangomezconde
 
PDF
Long-Term Robust Tracking Whith on Failure Recovery
byTELKOMNIKA JOURNAL
 
PDF
Design and implementation of video tracking system based on camera field of view
bysipij
 
PDF
Schematic model for analyzing mobility and detection of multiple
byIAEME Publication
 
PDF
Object Tracking System Using Motion Detection and Sound Detection
byAM Publications,India
 
A novel enhanced algorithm for efficient human tracking
byIJICTJOURNAL
 
Robust Tracking Via Feature Mapping Method and Support Vector Machine
byIRJET Journal
 
An improved particle filter tracking
byijcsit
 
Object Detection and Tracking using Statistical and Stochastic Techniques
byVasuhiSamydurai1
 
IRJET- Application of MCNN in Object Detection
byIRJET Journal
 
Real time object tracking and learning using template matching
byeSAT Publishing House
 
IRJET- Full Body Motion Detection and Surveillance System Application
byIRJET Journal
 
Real Time Object Dectection using machine learning
bypratik pratyay
 
slide-171212080528.pptx
bySharanrajK22MMT1003
 
IRJET- Border Security using Computer Vision
byIRJET Journal
 
19_bibliography.pdf
byutak1
 
Dq4301702706
byIJERA Editor
 
Single person pose recognition and tracking
byJavier_Barbadillo
 
Lecture-1-CVIntroduction.pdf
byTechEvents1
 
Crime Detection using Machine Learning
byIRJET Journal
 
Intelligent video monitoring
byivangomezconde
 
Long-Term Robust Tracking Whith on Failure Recovery
byTELKOMNIKA JOURNAL
 
Design and implementation of video tracking system based on camera field of view
bysipij
 
Schematic model for analyzing mobility and detection of multiple
byIAEME Publication
 
Object Tracking System Using Motion Detection and Sound Detection
byAM Publications,India
 

Recently uploaded

PDF
Safer’s Picks: The 5 FME Transformers You Didn’t Know You Needed
bySafe Software
 
PDF
Writing GPU-Ready AI Models in Pure Java with Babylon
byAna-Maria Mihalceanu
 
PDF
Why Many Smart Device Platforms Fail to Scale?
byPromeraki Developments
 
PDF
CI CD Observability, Metrics and DORA - Shifting Left and Cleaning Up! - Febr...
byPeter Souter
 
PPTX
How Does an ICO Launchpad Work Step-by-Step Breakdown.pptx
byTom Hardy S
 
PDF
UiPath Automation Developer Associate Training Series 2026 - Session 1
byDianaGray10
 
PPTX
Introduction to Artificial Intelligence (AI).pptx
byammar414783
 
PDF
The Enterprise Web3 Landscape - a view from Kaleido based on the past 10 year...
byPeter Broadhurst
 
PDF
Understanding Foldable 3-Wheel Electric Scooters for Everyday Use
byTopmate
 
PDF
Track Phone Location by Number (2026)_ What’s Actually Possible for Free (and...
byEmily Woods
 
PPTX
TechSprint Inauguration at SJB Institute of Technology held on 18 December 2025
bysuhasspgdg
 
PDF
The Manifesto for AI-enabled Governance !
byYannis Charalabidis
 
PDF
TravelTech Paris 2025 | The EU Digital Identity Wallet and it’s impact on Tra...
byapidays
 
PDF
Configure and Manage Systemd Timers- RHCSA (RH134).pdf
byLinuxCert Guru
 
PDF
OpenCharacter AI Reviews: Features, Plans, and Best Alternative
byOpenCharacter AI
 
PDF
Title Installing Windows, Linux, MacOS.pdf
byGeraldAbadajos
 
PDF
UiPath Automation Developer Associate Training Series 2026 - Session 2
byDianaGray10
 
PPTX
Automation Without Apprentices: How AI Challenges the Open Source Way
byIcinga
 
PPTX
Beyond the Algorithm: Designing Human-Centric Public Service with AI
byAlan Dix
 
PDF
Rustici Software: eLearning standards in the age of AI
byRustici Software
 
Safer’s Picks: The 5 FME Transformers You Didn’t Know You Needed
bySafe Software
 
Writing GPU-Ready AI Models in Pure Java with Babylon
byAna-Maria Mihalceanu
 
Why Many Smart Device Platforms Fail to Scale?
byPromeraki Developments
 
CI CD Observability, Metrics and DORA - Shifting Left and Cleaning Up! - Febr...
byPeter Souter
 
How Does an ICO Launchpad Work Step-by-Step Breakdown.pptx
byTom Hardy S
 
UiPath Automation Developer Associate Training Series 2026 - Session 1
byDianaGray10
 
Introduction to Artificial Intelligence (AI).pptx
byammar414783
 
The Enterprise Web3 Landscape - a view from Kaleido based on the past 10 year...
byPeter Broadhurst
 
Understanding Foldable 3-Wheel Electric Scooters for Everyday Use
byTopmate
 
Track Phone Location by Number (2026)_ What’s Actually Possible for Free (and...
byEmily Woods
 
TechSprint Inauguration at SJB Institute of Technology held on 18 December 2025
bysuhasspgdg
 
The Manifesto for AI-enabled Governance !
byYannis Charalabidis
 
TravelTech Paris 2025 | The EU Digital Identity Wallet and it’s impact on Tra...
byapidays
 
Configure and Manage Systemd Timers- RHCSA (RH134).pdf
byLinuxCert Guru
 
OpenCharacter AI Reviews: Features, Plans, and Best Alternative
byOpenCharacter AI
 
Title Installing Windows, Linux, MacOS.pdf
byGeraldAbadajos
 
UiPath Automation Developer Associate Training Series 2026 - Session 2
byDianaGray10
 
Automation Without Apprentices: How AI Challenges the Open Source Way
byIcinga
 
Beyond the Algorithm: Designing Human-Centric Public Service with AI
byAlan Dix
 
Rustici Software: eLearning standards in the age of AI
byRustici Software
 

Presentation of Visual Tracking

  • 1.
    Presentation of PersonalWork Yosen Chen
  • 2.
    Yosen Chen, CopyrightReserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 2
  • 3.
    Implementation Tool Microsoft VisualC++ with OpenCV Library IPC (Interprocess Communication) Library open source @ Carnegie Mellon University Yosen Chen, Copyright Reserved 3
  • 4.
    Yosen Chen, CopyrightReserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 4
  • 5.
    Intelligent Interactive Engine Imageprocessing Webcam Image data Show message & animation Display Central Server Webcam Our goal is to construct an interactive engine which utilizes target motion as instruction input to the server. Yosen Chen, Copyright Reserved 5
  • 6.
    Yosen Chen, CopyrightReserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 6
  • 7.
    Basic Visual TrackingFramework (Color) Yosen Chen, Copyright Reserved Camera Real-time image Feature Extraction RGB to HSV Hue Saturation Value 256-to-16 color Qtz Similarity Evaluation Reference Image Reference 16-color Histogram Hue Saturation Value Reference Color Backprojection Probability distribution figure Probability Peak Search/Climbing Target Information Estimation (Position, size,...) If iterative Reference 16-color Histogram Reference Color Backprojection Input Hue Image Probability distribution figure Color similarity score Iterative Search (Motion Prediction) Previous Current Next Camera View Region of Interest 7
  • 8.
    Demo Video: BallTracking 追球後段 Yosen Chen, Copyright Reserved 8 Peak climbing: MeanShift
  • 9.
    Related Works Varieties ofFeature Extraction/Evaluation Feature evaluation: template matching Background subtraction Reference color model: YCrCb, RGB Varieties of Peak Search/Climbing Gravity center Max-Likelihood MeanShift Yosen Chen, Copyright Reserved Cb Cr 9
  • 10.
    Summary Advantages: Simple & Fast low-cost Each pixel (RGBHSV, backprojection) is processed independently  good for parallel processing Drawbacks: Hard to combine with more object features  EX: edge information (object shape, contour)  All object features 2D probability distribution figure Hard to cope with higher-dimension target state  EX: object orientation (rotation angle) Yosen Chen, Copyright Reserved ??suitable 10
  • 11.
    Yosen Chen, CopyrightReserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 11
  • 12.
    SIR+SIS Particle FilterTracking Yosen Chen, Copyright Reserved State candidates (Particle set) Color Possible region sampling SIR Real-time State Candidate Prediction Frame Multi-Cue Candidate Evaluation      1 m m t tp X X       1 1 , , , 1,..., mm t t X x y r m N       m t tp O X State Candidate Resampling            , equal weight m m t t t k t X p O X X tO SIS Shape Motion weighting Particles from last candidate set Particles generated from current image Tracking result on Frame tO  m tX Best State Candidate Sequential Importance Sampling (SIS) Particle Filter Sampling Importance Resampling (SIR) Particle Filter Color backproject + filtering + best contour finding Initial State Estimation State Prediction State Evaluation  Find Best Candidate Resampling Last Candidates based on weights State Prediction State Evaluation  Find Best Candidate Last candidate set {state candidate, weight} 12
  • 13.
    Multi-Cue Likelihood Evaluation YosenChen, Copyright Reserved For each state candidate Inner color extraction Color Similarity Correlation-based comparison Color weight Reference color modelColor Multi-Cue Candidate Evaluation    m t tp O XShape Motion weighting  m tX tO inner outer High continuity Low continuity id maxd ig      , , , mm t t X x y r  Shape Similarity Motion continuity (logic check) outer color extraction Shape distance State difference with last tracking result id  Ih i  Oh i  REFh i Edge strength ig Max matched size 1 null MAX Nr r N        Symmetry 1i N id d   , , ,x y r     Shape weight Motion weight  Overall candidate score    m t tp O X 13
  • 14.
    Demo Video: 4DBottle Tracking (x, y, r, θ) Yosen Chen, Copyright Reserved 14 Candidate group behavior Bottle tracking video Blue: SIS (from current color projection) Green: SIR (from last candidate set) Red: Best candidate as tracking result
  • 15.
    Yosen Chen, CopyrightReserved 15 Interaction System Interaction Triggering Object Checking Tracking & Interaction Losing Handling Start Motion detection by Optical flow Image Initialize/Reset all parameters Motion detected? Show Checking Box Record target information and features Target Info is robust? Tracking algorithm Quality index > threshold ? Wait until user puts target in “Losing Box” Is target put back? Visual reality interacting program image Yes No Yes Yes No No Yes No Image
  • 16.
    Demo Video: Interactive Game– Brick Breaker Yosen Chen, Copyright Reserved Interaction system: 4 status switching 4 degrees of freedom: location x & y, size, rotation angle 16 Motion Interaction Function Interactive Game Example
  • 17.
    Summary High flexibility State dimension Motionmodel Easy to add new features & tune weights in tracker for different applications and scenarios Trade-off: Feature description vs. Computation time Impoverishment(intensive) vs. Degeneracy(sparse)  Many particles concentrate on high-probability region  Many particles are evaluated at low-probability region  Number of particles ～ Complexity Yosen Chen, Copyright Reserved 17
  • 18.
    Yosen Chen, CopyrightReserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 18
  • 19.
    Related applications: Visual surveillance Remotemonitoring Intelligent security system Introduction 19Yosen Chen, Copyright Reserved
  • 20.
    Yosen Chen, CopyrightReserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 20
  • 21.
    Multi-Human Tracking Flow YosenChen, Copyright Reserved Prediction Evaluation Resampling Prediction Evaluation Resampling Prediction Evaluation Resampling … State joint evaluation SIR Head tracking Multi-human tracking result Real-time captured image Skin color blob finding SIS Face detection Shape & location filter ROI location Sampling Face detector (OpenCV) Losing Track Detector (score, counter) Delete tracker Create new tracker 1. SIS face detector  SIR human trackers 2. Multi-human state joint evaluation 3. Bad evaluated tracker  losing track detector  face detector  delete tracker 21
  • 22.
    Multi-Cue Likelihood Evaluation forHead Tracking Yosen Chen, Copyright Reserved High color likelihood Low color likelihood outer inner Skin color mask Hair color mask outer inner Skin color mask Hair color mask   1 , , t x y r   , , t x y r   1 , , t x y r   , , t x y r High continuity Low continuity 1n 2n 6n 9n 13n Shape distance 1n 2n 6n 9n 13n Num of Matched point ig Edge strength 1n 2n 6n 9n 13n symmetry Color •Masked skin color comparison •Masked hair color comparison Texture •Texture SAD similarity Edge •Shape distance •Number of matched point •Edge strength •Contour symmetry fitting Motion •State continuity 22
  • 23.
    Face Detection ImplementationDetails Yosen Chen, Copyright Reserved SIS Face Detection Flow =======================Pseudo code======================== cvCalcBackProject; cvEqualizeHist + cvThreshold; cvResize + cvErode + cvDilate; cvFindContours(LinkedList detect_contour); for( ; detect_contour != 0 ; detect_contour = detect_contour->next ) { if (Area < threshold) continue; Use 8 points average to approximate center (xA, yA); if (distance between all target T and (xA, yA) > threshold) { calculate geometric center (xC, yC), deviation σX, σY of detect_contour; if ((σX > 0.7*σY) && (σY > 0.8*σX) && (σX /sqrt(Area) < 0.3) && (σY /sqrt(Area) < 0.4)) { Draw k random samples (x, y) ~ N((xC, yC), (σX, σY)); where k = (Area/MIN_AREA). if (frontalFaceTest(x, y)->detect() || profileFaceTest(x, y)->detect()) return location (x, y) as start point of new tracker; } } } return NOT_DETECTED; Skin color blob finding SIS Face detection Shape & location filter ROI location Sampling Face detector (OpenCV) Losing Track Detector Create tracker Delete tracker 23
  • 24.
    Face Detection Demo YosenChen, Copyright Reserved Skin color backproject Skin color blob finding SIS Face detection Shape & location filter ROI location Sampling Face detector (OpenCV) Losing Track Detector Create tracker Delete tracker 24Skin color backproject (down-size & morphology) Last detecting region Blue region: Last detected face Red region: color is matched with skin color. Tracking System GUI panel
  • 25.
    People Overlapping Problem YosenChen, Copyright Reserved When people get close to each other… State candidates are attracted by other people because trackers don’t know each other!! Prediction Evaluation (individual) Resampling Original Head Trackers 25
  • 26.
    Solutions of PeopleOverlapping Yosen Chen, Copyright Reserved Prediction Evaluation (Individual + joint) ResamplingJoint state check Depth order check Body occlusion handling Target information storage Modified Head Trackers for People Overlapping For people re-entering problem 1. Use joint evaluation to avoid candidates merging for people overlapping 2. Solution#1 (left): give higher joint score for candidates further from other people  reduce risk of candidates merging but also lose tracking accuracy 3. Solution#2 (right): kill candidates too close to other people  better tracking accuracy 26 resVideo_FurtherIsGood2.mpg resVideo_CloseThenLowWeight2.mpg
  • 27.
    Implementation Yosen Chen, CopyrightReserved Prediction Evaluation (Individual + joint) ResamplingJoint state check Depth order check Modified Head Trackers for People Overlapping Image distance check “if △x2 + △y2 < threshold, …” è group together target1 target2 target3 target4 target5 Grouping (2nd-order tree) root leaf leafleaf Target A candidate 1 (far from B) Target B last best candidate Target A candidate 2 (near B) Pointer to target1 Candidate Evaluation (individual + joint) Individual: (as previous mentioned) Color, shape, motion Joint: (not be attracted by other close targets) Distances to all other targets in the same group Pointer to target2 ……… Depth order: deform 2nd-tree into array, then quick sort by depth features Sort by depth features Depth features: For best candidates of each target Texture changing rate Contour matching score Distance from camera to target ground points Image source 1:Front 2:Rear Candidate A2 has high penalty (since Candidate A2 are too close to B !!) 27
  • 28.
    Depth Order Evaluationwith Heights Depth order estimation with height knowledge is accurate Even when people are not overlapping. Yosen Chen, Copyright Reserved 28
  • 29.
    Summary of Multi-HumanTracker We extend the concept of single-object tracking to multi-object tracker We propose a entire tracking system Face detection  multi-human trackers  detect people leaving Address the solution of people overlapping Robust multi-human tracker is indeed the basis of correspondence process within multi-camera tracking system Yosen Chen, Copyright Reserved 29
  • 30.
    Yosen Chen, CopyrightReserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 30
  • 31.
    Multi-Camera Tracking System YosenChen, Copyright Reserved Captured image Local camera #A multi-head tracker  1 2,A B t tz z2tH 1 A tz 2 A tz 2 B tz 1 B tz Local camera #B multi-head tracker Captured image Central Server Multi-camera multi-observation correspondence Person#1 Person#2  1 2 1,A B t tz ztH Identify the relationship of observations in different cameras Local trackers: multi-human tracking and occlusion handling Central server: observation correspondence across cameras 31
  • 32.
    : image observation :3D line from image head center j rt j rt O L  “Are these cameras tracking the same people? ”  Let be the rth observation tracked by camera at time Problem of Observation Correspondence across Cameras  ,j j j rt rt rtO Lz j t How many people exist in surveillance area? Ground plane Camera j Camera k 1 j tO 2 j tO 3 j tO1 k tO 2 k tO 3 k tO 1 j tL 2 j tL 3 j tL 1 k tL 2 k tL 3 k tL Yosen Chen, Copyright Reserved 32
  • 33.
     Determine observationcorrespondence using Multi-Feature Similarity  Multi-Feature Observation Similarity contains: Head line intersection verification Body color matching by blocks Confidence level Label history consistency How to Determine Observation Correspondence across Cameras Correspondence Flow Calculate 3D geometric intersection Confidence level Are they from the same person? Camera#A Camera#B Body Color comparison by block Label history consistency 2D Body image block allocation No, if no reasonable intersection No, if color unmatched Yes, 2 observations are from the same person Yosen Chen, Copyright Reserved 33
  • 34.
     2D Positionon image plane  3D head line in real space  3D ground point in real space  2D position on image plane observation r Optic center Image plane of camera j Ground plane Optic center observation q Image plane of camera k 3D Geometric Correspondence across Cameras Calibrated camera system Image 2D↔3D coordinate transfer is constructed off-line Head Point height Estimated Ground Point Yosen Chen, Copyright Reserved 34
  • 35.
    How to Implement3D-2D Mapping Yosen Chen, Copyright Reserved Class: Plane3D Member functions: isPointOnPlane getDistFromPoint getProjPointFromPoint Class: line3D Member functions: getPositionFromX,Y,Z isPointOnLine getDistFromPoint getProjPointFromPoint getLinePointOnPlane getMinDistPointFromLine Class: CameraLine3D Member functions: setByImageLocation setBy3DPosition How to verify the mapping accuracy? 3D Optic point (≈ Camera location) Class: CameraLine3D 2D image plane Reference plane in 3D space 3D Normal vector Reference 3D plane center (0,0) derived 35 CAM2 CAM1 Locate 3D points/lines on 2D image planes
  • 36.
     Human BodyModel Based on Head Tracking Matching body color  Human Body is separated into 4 blocks ground-point head ground-point head Image boundary Observed ground-point Unobserved ground-point j rtz k qtz Camera kCamera j qtrt 3D Geometric-based Body Image Block Allocation Yosen Chen, Copyright Reserved 36 For some observations, not all the body blocks are unobservable. Here we take the number of observable blocks as confidence level of correspondence result
  • 37.
    Observation Correspondence acrossCameras Object Conf = 3 Object Conf = 2 Matching body color (Region-based body color comparison) Depth order: Front Depth order: RearBody Occlusion Handling (based on depth order and 3D-2D block mapping ) Yosen Chen, Copyright Reserved 37 Body appearance is available for correspondence process Body appearance is not available (Color turns darker)
  • 38.
    Body Occlusion Handling YosenChen, Copyright Reserved 38 Prediction Evaluation (Individual + joint) ResamplingJoint state check Depth order check Body occlusion handling Target information storage Modified Head Trackers for People Overlapping
  • 39.
    Labeling Consistency withConfidence Level Yosen Chen, Copyright Reserved Observation correspondence process should follow rules stated below Rule1. Claiming “match” for one pair will also imply claiming “collision” for all corresponding residual pairs, with the same confidence level Rule2. Only higher-confidence decisions can enforce changes of lower-confidence ones √ √ √ A B C √ 1 0 0 -4 √ 2 -4 -4 4 √ 3 0 0 -4 √ √ X A B C √ 1 0 0 0 √ 2 -4 -4 0 √ 3 0 0 0 √ √ A B √ 1 -3 0 √ 2 -4 -4 √ 3 3 -3 Camera#1 Camera#2 Conf = 4Conf = -4 1 2 3 A B C Camera#1 Camera#2 Conf = -4 1 2 3 A B Camera#1 Camera#2 Conf = 3 Conf = -4 Conf = -3 1 2 3 A B 39
  • 40.
    Yosen Chen, CopyrightReserved Human Machine Interaction Basic Visual Tracker Multi-Cue Particle Filter Tracker Consistent Labeling in Visual Surveillance System Multi-Human Tracker Multi-Camera Multi-Human Tracker Consistent Labeling with People Re-Entering Problem Outline 40
  • 41.
    Consistent Labeling withPeople Re-Entering Yosen Chen, Copyright Reserved Captured image Local camera #A multi-head tracker  1 2,A B t tz z2tH 1 A tz 2 A tz 2 B tz 1 B tz Local camera #B multi-head tracker Captured image Central Server Multi-camera multi-observation correspondence Person#1 Person#2  1 2 1,A B t tz ztH Identify the relationship of observations in different cameras 3tX2tX1tX associate with target databases 41
  • 42.
    Association Processes withMatching Confidence Orders Hierarchical Association with target databases  Objects is associated to target databases  1 m ht h H   1 n it i X  Consistent Labeling Using Target Database 3tX 2tX 1tX 4tX 1tH i TARL High Low 1 OBJL 1 TARL 2 TARL 3 TARL 4 TARL compare with descendent order : target confidence level (in database) : object confidence level (from observation) i TAR i OBJ L L if association success update related database else create new database PS: database association process is only for objects with conf. level ≧2 Yosen Chen, Copyright Reserved 42
  • 43.
    Association Process  Associationprocess:  To determine if observed object and database are from the same person  Features are similar to observation correspondence process  People height  Body color comparison by blocks  Association history consistency Yosen Chen, Copyright Reserved Observation-Database Association Flow: Check height difference Are they from the same person? Body Color comparison by block Association history consistency No, if diff > threshold No, if color unmatched Yes, observations & this database set are from the same person 2tX 1tH From Camera observations People information already stored in databases 43
  • 44.
    How to Model& Update Database Adaptive Gaussian Mixture Model 3 color candidates (histogram & weight) to describe 1 body block color Candidate’s histograms & weights are updated based on current color of MATCHED observations Color comparison with observed color (Define MATCHED or not)  Bhattacharyya distance from 3 candidates  3 weighted Gaussian kernels Yosen Chen, Copyright Reserved 44 Color histogram (1-D for demo) Candidate weight Candidate color Adaptive Gaussian Mixture Model Candidate#1 Candidate#3 Candidate#2 Candidate color & weight will be updated based on observations Candidate variance Inspired by [Stauffer., CVPR 2000]
  • 45.
    Database Operation Demonstration ConsistentLabeling Using Target Database Histogram candidatesCamera#2 histogramCamera#1 histogram Matched with camera#2 Matched with camera#1 Yosen Chen, Copyright Reserved 45
  • 46.
    Consistent Labeling UsingTarget Database Database Demonstration (cont’) 46Yosen Chen, Copyright Reserved
  • 47.
    Multi-People Correspondence Yosen Chen,Copyright Reserved 47 1. Multi-people depth order 2. body occlusion handling 3. Multi-camera multi-people correspondence is correct
  • 48.
    Multi-People Re-entering Yosen Chen,Copyright Reserved 48 System Operation Flow: 1. Observation correspondence & Body occlusion handling 2. Associate with current databases  (Create then) update new information 3. Return people IDs (show by color) to each tracker of each camera
  • 49.
    Future Works… Extend tothe 3rd Camera Challenge of communication load, algorithm complexity, and tracking/labeling stability Performance Evaluation of Target Database Comparison with related works More challenges: Outdoors Surveillance (human body detection) Uncalibrated multi-camera system (Training) Motive camera (rotation 3D-2D mapping) Yosen Chen, Copyright Reserved 49
  • 50.
  • 51.
    BACKUP SLIDES Yosen Chen,Copyright Reserved 51
  • 52.
    Motion Detection :Optical Flow (Modified from OpenCV Sample Code) Input gray-scale image Optical point extraction Optical point mapping & filtering Motion detection criteria Reset criteria Interaction trigger Optical flow Resettrigger Previous gray-scale image Store Optical Flow: 1. A technique to create edge points then map them between adjacent images. (edge tracking) 2. Better to analyze group behavior rather than individual (not accurate) 3. Can be utilized as motion detection tool 1. Not enough effective optical points on screen 2. Time out then refresh 1. Average velocity of moving points > threshold 2. Number of moving points > threshold Moving points: points with velocity > 3*average velocity Yosen Chen, Copyright Reserved 52
  • 53.
    BACKUP SLIDES END YosenChen, Copyright Reserved 53