Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Exploiting Multiple Component Representations for Person Re-Identification Ph.D. candidate: Riccardo Satta Annual report, ...
Outline <ul><li>Introduction to  Person Re-Identification </li></ul><ul><li>Problem formulation </li></ul><ul><li>The prop...
Introduction <ul><li>In  video surveillance , it is often desirable to determine if an individual </li></ul><ul><li>has al...
Main issues <ul><li>Low resolution of the frames  (all figs) </li></ul><ul><ul><li>Less data available </li></ul></ul><ul>...
Problem formulation <ul><li>We can model the problem as follows: </li></ul><ul><li>given a  gallery set of templates  T  =...
My contribution to this field <ul><li>A general framework to for people re-identification problems </li></ul><ul><ul><li>I...
Towards a Multiple Component Representation of the Human Body <ul><li>Human body peculiarities </li></ul><ul><ul><li>non-r...
Towards a Multiple Component Representation of the Human Body <ul><li>How to represent every single component? </li></ul><...
Multiple Component Matching Framework <ul><li>The proposed  Multiple Component Matching  framework integrates multiple ins...
Multiple Component Matching Framework <ul><li>Multiple Component Matching (MCM) framework </li></ul><ul><ul><li>Inspired t...
MCM Implementation <ul><li>Part subdivision </li></ul><ul><ul><li>Anti-symmetry properties of the human silhouette [1]  </...
MCM Implementation <ul><li>Part subdivision </li></ul><ul><ul><li>head-torso axis   is given by </li></ul></ul><ul><ul><li...
MCM Implementation <ul><li>Instances </li></ul><ul><ul><li>For each part, we extract  random overlapping patches  of [25% ...
MCM Implementation <ul><li>Instances - Simulation </li></ul><ul><ul><li>To asset lighting and contrast variations we gener...
MCM Implementation <ul><li>Distance between sets  d ( ) </li></ul><ul><ul><li>k-th Haussdorff distance [2] </li></ul></ul>...
Evaluation <ul><li>Evaluation on the ViPER dataset </li></ul><ul><ul><li>632  individuals, 1 template and 1 probe image pe...
Prototypes <ul><li>AmI LAB demo application </li></ul><ul><li>A prototype is being realized at the Ambient Intelligence La...
Further developments <ul><li>Research issues </li></ul><ul><ul><li>Dissimilarity-based representations </li></ul></ul><ul>...
Further developments <ul><li>New applications </li></ul><ul><ul><li>Personal Photo Re-Tagging </li></ul></ul>ID 1 ID 1 ID ...
Thanks! <ul><li>Questions? </li></ul>
Upcoming SlideShare
Loading in …5
×

Person re-identification, PhD Day 2011

1,805 views

Published on

Awarded presentation of my research activity, PhD Day 2011, February 23th 2011, Cagliari, Italy.

This presentation has been awarded as the best one of the track on information engineering.

Want to know more?
see my publications at
http://prag.diee.unica.it/pra/ita/people/satta

  • Be the first to comment

  • Be the first to like this

Person re-identification, PhD Day 2011

  1. 1. Exploiting Multiple Component Representations for Person Re-Identification Ph.D. candidate: Riccardo Satta Annual report, Year I
  2. 2. Outline <ul><li>Introduction to Person Re-Identification </li></ul><ul><li>Problem formulation </li></ul><ul><li>The proposed Multiple Component Matching (MCM) framework </li></ul><ul><li>Implementation of MCM , and experimental results </li></ul>
  3. 3. Introduction <ul><li>In video surveillance , it is often desirable to determine if an individual </li></ul><ul><li>has already been observed over a network of cameras. </li></ul><ul><li>This issue is called Person Re-Identification </li></ul><ul><li>In general, we can’t apply face recognition algorithms ( low resolution! ), </li></ul><ul><li>therefore we must consider the global “ appearance ” of the individual </li></ul><ul><ul><li> short term problem </li></ul></ul><ul><li>Scenarios : </li></ul><ul><ul><li>Tracking of the movements of an individual in large public places </li></ul></ul><ul><ul><li>monitored by several non-overlapping cameras </li></ul></ul><ul><ul><li>Use in conjunction with other common identification techniques </li></ul></ul><ul><ul><li>(RFID, biometric systems) </li></ul></ul>
  4. 4. Main issues <ul><li>Low resolution of the frames (all figs) </li></ul><ul><ul><li>Less data available </li></ul></ul><ul><ul><li>More subject to noise </li></ul></ul><ul><li>Changing lighting conditions (fig. 1) </li></ul><ul><ul><li>Brightness changes </li></ul></ul><ul><ul><li>Contrast changes </li></ul></ul><ul><li>Different sensor responses (fig. 2) </li></ul><ul><ul><li>Colour temperature </li></ul></ul><ul><ul><li>White balance </li></ul></ul><ul><li>Partial occlusions (fig. 3) </li></ul><ul><li>Pose variations (fig. 1, 4) </li></ul>figure 1 figure 4 figure 2 figure 3
  5. 5. Problem formulation <ul><li>We can model the problem as follows: </li></ul><ul><li>given a gallery set of templates T = { T 1 ,…, T n } and a probe Q , find the most similar template T * T with respect to a similarity measure D ( · , · ) : </li></ul><ul><li> T * = arg min D( T i , Q ) </li></ul>Descriptor generation MATCHING SCORE (similarity) Descriptor generation TEMPLATE PROBE T = Q T i
  6. 6. My contribution to this field <ul><li>A general framework to for people re-identification problems </li></ul><ul><ul><li>Inspired to common paradigms in Machine Learning and Computer Vision </li></ul></ul><ul><ul><li>Includes and generalizes ideas partly embedded in previous works </li></ul></ul><ul><ul><li>Aims at providing a common foundation for existing and future methods </li></ul></ul><ul><li>A method for person re-identification </li></ul><ul><ul><li>Simple, yet effective, implementation of the framework </li></ul></ul><ul><li>… and some interesting ideas for further developments </li></ul>
  7. 7. Towards a Multiple Component Representation of the Human Body <ul><li>Human body peculiarities </li></ul><ul><ul><li>non-rigid object </li></ul></ul><ul><ul><li>complex kinematics </li></ul></ul><ul><ul><li>composed of many quasi-rigid parts </li></ul></ul><ul><li>Possible approach: COMPONENT SUBDIVISION </li></ul><ul><li> </li></ul><ul><li> We can take into account an arbitrary m -component subdivision by fusing matching scores of every part </li></ul>
  8. 8. Towards a Multiple Component Representation of the Human Body <ul><li>How to represent every single component? </li></ul><ul><li>Multiple Instance Learning (MIL) paradigm : </li></ul><ul><li>the object of interest is represented by a bag of instances </li></ul><ul><li>MIL framework adapted to a matching problem </li></ul><ul><li>Every component is described by a series of instances </li></ul><ul><ul><li>Patches, point of interest… </li></ul></ul>{ x 1 , … , x p } { x 1 , … , x p } { x 1 , … , x p } { x 1 , … , x p } { x 1 , … , x p } { x 1 , … , x p } { x 1 , … , x p } Object
  9. 9. Multiple Component Matching Framework <ul><li>The proposed Multiple Component Matching framework integrates multiple instance representations with component subdivision in a matching paradigm </li></ul><ul><ul><li>every individual X i is decomposed into an ordered sequence of parts X j i </li></ul></ul><ul><ul><li>every part X j i is represented by a set of instances X j i,n </li></ul></ul><ul><ul><li>matching is performed at set level </li></ul></ul><ul><ul><li>global matching distance is a combination of set distances </li></ul></ul>Example (4 parts)
  10. 10. Multiple Component Matching Framework <ul><li>Multiple Component Matching (MCM) framework </li></ul><ul><ul><li>Inspired to MIL statistical learning framework </li></ul></ul><ul><ul><li>Embeds two concepts ( part subdivision and multiple instance representation ) that can be found, even if partly and implicitly, in the main part of the existing works </li></ul></ul><ul><ul><li>Aims at providing a solid foundation for existing and future works </li></ul></ul><ul><li>Well, nice theory… but in practice? </li></ul><ul><li>Implementation of MCM: </li></ul><ul><ul><li>choose part subdivision </li></ul></ul><ul><ul><li>choose what to treat as an “instance” </li></ul></ul><ul><ul><li>choose an appropriate distance between sets d( ), and how to combine set distances </li></ul></ul>
  11. 11. MCM Implementation <ul><li>Part subdivision </li></ul><ul><ul><li>Anti-symmetry properties of the human silhouette [1] </li></ul></ul><ul><ul><li>Let we first define two operators: </li></ul></ul><ul><ul><li>chromatic bilateral operator </li></ul></ul><ul><ul><li>spatial covering operator </li></ul></ul>[1] M. Farenzena, L. Bazzani, A. Perina, V. Murino, and M. Cristani . Person re-identification by symmetry-driven accumulation of local features. In Proc. IEEE Conf. on Comp. Vision and Patt. Rec., 2010
  12. 12. MCM Implementation <ul><li>Part subdivision </li></ul><ul><ul><li>head-torso axis is given by </li></ul></ul><ul><ul><li> separating regions that strongly differ in area </li></ul></ul><ul><ul><li>torso-legs axis is given by </li></ul></ul><ul><ul><li>separating regions with strongly different appearance and similar area </li></ul></ul><ul><li>Two parts: [torso] and [legs] , discarding the head </li></ul>
  13. 13. MCM Implementation <ul><li>Instances </li></ul><ul><ul><li>For each part, we extract random overlapping patches of [25% - 75%] width in respect of the part width and [25% - 75%] height in respect of the part height </li></ul></ul><ul><ul><li>Each instance is described by an HSV colour histogram (24-12-4 bins) </li></ul></ul>
  14. 14. MCM Implementation <ul><li>Instances - Simulation </li></ul><ul><ul><li>To asset lighting and contrast variations we generate synthetic patches for templates </li></ul></ul><ul><ul><li>Each RGB pixel is multiplied by different values </li></ul></ul><ul><ul><ul><li>we chose [1.4 1.2 1.0 0.8 0.6] </li></ul></ul></ul><ul><ul><ul><li>This changes both brightness (mean) and contrast (variance) of the patches </li></ul></ul></ul>
  15. 15. MCM Implementation <ul><li>Distance between sets d ( ) </li></ul><ul><ul><li>k-th Haussdorff distance [2] </li></ul></ul><ul><ul><li>Metric: </li></ul></ul><ul><ul><ul><li>dist B Bhattacharyya distance between the HSV histograms of the patches </li></ul></ul></ul><ul><ul><ul><li>y pos,a , y pos,b mean vertical position of the patch </li></ul></ul></ul><ul><li>Set distance combination </li></ul><ul><ul><li>Average distance between sets: </li></ul></ul>[2] Wang and J.-D. Zucker. Solving the multiple-instance problem: A lazy learning approach . In Proc. Int. Conf. Mach. Learn., 2000.
  16. 16. Evaluation <ul><li>Evaluation on the ViPER dataset </li></ul><ul><ul><li>632 individuals, 1 template and 1 probe image per individual, SvsS scenario </li></ul></ul><ul><ul><li>random selection of 316 individuals, average performance on 10 runs </li></ul></ul>SDALF refers to the state-of-the art on this dataset [1] [1] M. Farenzena, L. Bazzani, A. Perina, V. Murino, and M. Cristani . Person re-identification by symmetry-driven accumulation of local features. In Proc. IEEE Conf. on Comp. Vision and Patt. Rec., 2010
  17. 17. Prototypes <ul><li>AmI LAB demo application </li></ul><ul><li>A prototype is being realized at the Ambient Intelligence Lab of Sardegna Ricerche </li></ul>
  18. 18. Further developments <ul><li>Research issues </li></ul><ul><ul><li>Dissimilarity-based representations </li></ul></ul><ul><ul><li>Frame quality </li></ul></ul><ul><ul><li>Multiple frames accumulation </li></ul></ul>Prototype 1 Prototype 2 Prototype N d 1 d 2 d N Dissimilarity representation
  19. 19. Further developments <ul><li>New applications </li></ul><ul><ul><li>Personal Photo Re-Tagging </li></ul></ul>ID 1 ID 1 ID 2 ID 2 ID 3 ID 3
  20. 20. Thanks! <ul><li>Questions? </li></ul>

×