Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

of

FYP Research Presentation Slide 1 FYP Research Presentation Slide 2 FYP Research Presentation Slide 3 FYP Research Presentation Slide 4 FYP Research Presentation Slide 5 FYP Research Presentation Slide 6 FYP Research Presentation Slide 7 FYP Research Presentation Slide 8 FYP Research Presentation Slide 9 FYP Research Presentation Slide 10 FYP Research Presentation Slide 11 FYP Research Presentation Slide 12 FYP Research Presentation Slide 13 FYP Research Presentation Slide 14 FYP Research Presentation Slide 15 FYP Research Presentation Slide 16 FYP Research Presentation Slide 17 FYP Research Presentation Slide 18 FYP Research Presentation Slide 19 FYP Research Presentation Slide 20 FYP Research Presentation Slide 21 FYP Research Presentation Slide 22 FYP Research Presentation Slide 23 FYP Research Presentation Slide 24 FYP Research Presentation Slide 25 FYP Research Presentation Slide 26 FYP Research Presentation Slide 27 FYP Research Presentation Slide 28 FYP Research Presentation Slide 29 FYP Research Presentation Slide 30 FYP Research Presentation Slide 31 FYP Research Presentation Slide 32 FYP Research Presentation Slide 33 FYP Research Presentation Slide 34 FYP Research Presentation Slide 35 FYP Research Presentation Slide 36 FYP Research Presentation Slide 37 FYP Research Presentation Slide 38 FYP Research Presentation Slide 39 FYP Research Presentation Slide 40 FYP Research Presentation Slide 41 FYP Research Presentation Slide 42 FYP Research Presentation Slide 43 FYP Research Presentation Slide 44 FYP Research Presentation Slide 45 FYP Research Presentation Slide 46
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

0 Likes

Share

Download to read offline

FYP Research Presentation

Download to read offline

Presentation of my Final Year Research during undergraduate studies

  • Be the first to like this

FYP Research Presentation

  1. 1. Semantic Parsing and Saliency Scoring for Occluded Person Re-Identification Niruhan Viswarupan Madhushan Buwaneshwar Harishanth Sivakumaran Thivakaran Thanabalasingam Dr. Ranga Rodrigo Dr. Chamira Edussooriya Zheng, W.S., Li, X., Xiang, T., Liao, S., Lai, J. and Gong, S., 2015. Partial person re-identification. In Proceedings of the IEEE International Conference on Computer Vision (pp. 4678-4686).
  2. 2. Identity of Indiscernibles - Leibniz “there cannot be separate objects or entities that have all their properties in common.” Zheng, L., Yang, Y. and Hauptmann, A.G., 2016. Person re-identification: Past, present and future. arXiv preprint arXiv:1610.02984
  3. 3. Person Re-Identification Person re-identification (re-id) is to identify a person in a multi camera setting without overlapping fields of view. To solve re-identification the system needs to assign a stable id to pedestrians across multiple cameras.
  4. 4. Reformulation ● retrieving the ID of best matching image in a gallery given a query image of a person. ● The gallery usually consists of large number of images of persons with associated IDs ● It is also common to have multiple photos of the same person in the gallery
  5. 5. Popular Datasets for Person Re-ID - General Market 1501 CUHK 03 Li, W., Zhao, R., Xiao, T. and Wang, X., 2014. Deepreid: Deep filter pairing neural network for person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 152-159). Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J. and Tian, Q., 2015. Scalable person re-identification: A benchmark. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1116-1124). IDs: 1501 Train: 12936 Test: 19732 IDs: 1360 Images: 13164
  6. 6. Challenges in Person Re-ID ● Background clutter ● Varying illumination conditions ● Severe occlusions ● Pose variation ● Picking up or dropping objects, removing or wearing a jacket, cap etc
  7. 7. The Focus of This Paper 1. Can body part-wise feature extraction help to deal with occlusion? 2. Does semantic parsing help reduce background bias and hence make generalization better?
  8. 8. 1. Can body part-wise feature extraction help to deal with occlusion?
  9. 9. Occlusion An object of interest being partially or completely blocked from view due to the presence of other objects Object of Interest Occluding Object Image Source: How to introduce Dogs and Cats? https://www.insidedogsworld.com/how-to-introduce-dogs-and-cats/
  10. 10. Occluded Person Re-ID Datasets Partial Re-ID Dataset Partial iLIDS Zheng, W.S., Li, X., Xiang, T., Liao, S., Lai, J. and Gong, S., 2015. Partial person re-identification. In Proceedings of the IEEE International Conference on Computer Vision (pp. 4678-4686). Zheng, W.S., Gong, S., and Xiang, T. Person re-identification by probabilistic relative distance comparison. In Proceedings of the IEEE International Conference on Computer Vision, pages 649–656, 201
  11. 11. Existing Methods for Occluded Person Re-ID He, L., Liang, J., Li, H. and Sun, Z., 2018, January. Deep Spatial Feature Reconstruction for Partial Person Re-identification: Alignment-Free Approach. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 7073-7082).
  12. 12. Idea: Identify body parts like head, hands, torso, legs etc and take only the visible parts into account. Compare each part of query image with corresponding part from the gallery images Image Source: http://clipart-library.com/person-icon.html Image Source: https://www.freeiconspng.com/img/1927
  13. 13. Human Semantic Parsing Image Source: http://campar.in.tum.de/Students/MaReconstrSeg Image Source: https://people.cs.umass.edu/~kalo/
  14. 14. Overall Network Design Extract from our paper
  15. 15. Human Semantic Parsing Sub-Network Inception V3 [1] with 2 modifications, 1. Making stride of last grid reduction module 1 instead of 2 2. To deal with added computational cost making the following convolutions as dilated convolutions This approach from: ● Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. Rethinking atrous convolution for semantic image segmentation.arXiv preprint arXiv:1706.05587,2017 ● Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos,Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs.IEEE transactions on pattern analysis and machine intelligence, 40(4):834–848, 2018. ● Mahdi M Kalayeh, Emrah Basaran, Muhittin G ̈okmen,Mustafa E Kamasak, and Mubarak Shah. Human semantic parsing for person re-identification. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1062–1071, 2018 [1] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. InProceedings of the IEEE conference on computer vision and pattern recognition, pages2818–2826, 2016
  16. 16. Inception V3 Architecture ● “Network within a network” - Inception movie ● Salient parts have large variations in size ● Need different size filters at each level ● Uses 1x1 convolution for dimensionality reduction ● Auxiliary classifiers help deal with vanishing gradients Optional Slide
  17. 17. Image Source: https://hackathonprojects.files.wordpress.com/2016/09/74911-image03.png Inception V3 Architecture
  18. 18. Dilated Convolution Image Source: https://towardsdatascience.com/review-dilat ed-convolution-semantic-segmentation-9d5 a5bd768f5 Image Source: https://github.com/hassony2/inria-research-wiki/wiki/dilated-convolutions-vs-transposed-convolutions Yu, F. and Koltun, V., 2015. Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122.
  19. 19. Atrous Spatial Pyramid Pooling Chen, L.C., Papandreou, G., Schroff, F. and Adam, H., 2017. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587. Image Source: http://liangchiehchen.com/projects/DeepLab.html
  20. 20. Semantic Parsing Network Diagram (Keras Output Image)
  21. 21. Human Semantic Parsing Dataset: Look into Person (LIP) Source: http://sysu-hcp.net/lip/ Gong, K., Liang, X., Zhang, D., Shen, X. and Lin, L., 2017, July. Look into person: Self-supervised structure-sensitive learning and a new benchmark for human parsing. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 6757-6765). IEEE. 50000 images 20 labels From COCO dataset
  22. 22. Overall Network Design Extract from our paper
  23. 23. Choosing Backbone Architecture for Person Re-ID ● Training “vanilla” network for person re-identification with different architectures - ResNet50 [1], ResNet101 [1], DenseNet121 [2], Inception V3 ● Training and Testing dataset: Market 1501 [1] He, K., Zhang, X., Ren, S. and Sun, J., 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778). [2] Huang, G., Liu, Z., Van Der Maaten, L. and Weinberger, K.Q., 2017, July. Densely connected convolutional networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2261-2269). IEEE. Extract from our paper Competitive accuracy with less parameters
  24. 24. DenseNet Architecture ● Every layer takes input from ALL preceding layers ● Knowledge flow is high ● No need to replicate feature maps from layer to layer ● This allows for reduced number of channels ● So less number of parameters Huang, G., Liu, Z., Van Der Maaten, L. and Weinberger, K.Q., 2017, July. Densely connected convolutional networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2261-2269). IEEE. Optional Slide
  25. 25. Saliency Scoring ● If a body part is highly visible then its semantic parsing map will be brighter ● i.e the pixel values in the map will be high (white) ● If a body part is large it will have a larger semantic parsing map ● More pixels in the map will have a non-zero value ● So the sum of pixel values of each part map will give a relative visibility Xu, J., Zhao, R., Zhu, F., Wang, H. and Ouyang, W., 2018. Attention-Aware Compositional Network for Person Re-identification. arXiv preprint arXiv:1805.03344. Mi = Semantic parsing map of ith body part Si = saliency score for ith body part Chandra, S., Tsogkas, S. and Kokkinos, I., 2015. Accurate human-limb segmentation in rgb-d images for intelligent mobility assistance robots. In Proceedings of the IEEE International Conference on Computer Vision Workshops (pp. 44-50).
  26. 26. A = vector from query image B = vector from gallery image Distance between gallery and query computation A = Descriptive vector of query image Ag Af Ah At Al As “Partitioning the vector”
  27. 27. Network training ● Training dataset: Market 1501 ● Batch size: 11 ● GPU: NVidia tesla K80 ● Iterations: 10,000 (or 8.5 epochs) ● Time: 13.54 hours Sample training log. (This is from DenseNet training on Market1501)
  28. 28. Extract from our paper Results Ours Against Previous Works
  29. 29. Problem with Our Approach Partial re-id dataset has occlusion by people also. So cannot be sure for which person the semantic parsing network fires. Zheng, W.S., Li, X., Xiang, T., Liao, S., Lai, J. and Gong, S., 2015. Partial person re-identification. In Proceedings of the IEEE International Conference on Computer Vision (pp. 4678-4686).
  30. 30. 2. Does semantic parsing help reduce background bias and hence make generalization* better? 2nd contribution. Experiments yet to be done
  31. 31. *generalization ● Can be thought of as how well a model trained on a dataset performs in-the-wild when deployed in real world ● However, we define generalization as cross-dataset testing accuracy. i.e training on one dataset and testing on another
  32. 32. Tian, M., Yi, S., Li, H., Li, S., Zhang, X., Shi, J., Yan, J. and Wang, X., 2018. Eliminating Background-Bias for Robust Person Re-Identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 5794-5803). Different person in the same background as query has higher rank than the same person in different background Background Bias
  33. 33. Tian, M., Yi, S., Li, H., Li, S., Zhang, X., Shi, J., Yan, J. and Wang, X., 2018. Eliminating Background-Bias for Robust Person Re-Identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 5794-5803). Optional Slide
  34. 34. Tian, M., Yi, S., Li, H., Li, S., Zhang, X., Shi, J., Yan, J. and Wang, X., 2018. Eliminating Background-Bias for Robust Person Re-Identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 5794-5803). Observations ● Drop in performance when background is removed ● Trained on background only works better than random guess Implications ➢ Background info used by models for re-id ➢ Model fails to understand person from background (performance drop in random background) Optional Slide
  35. 35. Our Proposed Approach ● Use our semantic parsing model for cross dataset evaluation between Market1501, CUHK03 etc. and report accuracy Market 1501 CUHK 03 Train Test
  36. 36. Dead End Research Applying Masks at Feature Map Level with DenseNet 121 ● Motivation: remove background noise by using pre generated masks ● Approach: pass image through densenet and resize the masks to match feature map size and element-wise multiply ● Result: accuracy poorer than even without using masks ● Possible explanations: Masks are sometimes erroneous, zeroing out background pixels introduce “artificial discontinuities” Images Masks
  37. 37. Our Test Results
  38. 38. Appendix
  39. 39. Papers that we went through 1. Human Semantic Parsing for Person Re-identification. Mahdi M. Kalayeh et al. CVPR 2018 2. Deep Spatial Feature Reconstruction for Partial Person Re-identification: Alignment-free Approach. Lingxiao He et al. CVPR 2018 3. Attention-Aware Compositional Network for Person Re-identification. Jing Xu et al. CVPR 2018 4. Eliminating Background-bias for Robust Person Re-identification. Maoquing Tian et al. CVPR 2018 5. Mask-guided Contrastive Attention Model for Person Re-identification. Chunfeng Song et al. CVPR 2018 6. Learning Deep Context-aware Feature over Body and Latent Parts for Person Re-identification. Dangwei Li et al. CVPR2017 7. Mask R-CNN. Kaiming He et al. ICCV 2017 8. Densely Connected Convolutional Networks. Gao Huang et al. CVPR 2017 9. Harmonious Attention Network for Person Re-Identification. Wei Li et al. CVPR 2018 10. Occluded Person Re-Identification. Jiaxuan Zhuo et al. ICME 2018 11. Improved Person Re-Identification Based on Saliency and Semantic Parsing with Deep Neural Network Models. Rodolfo Quispe et al. CoRR 2018 12. Adversarially Occluded Samples for Person Re-identification. Houjing Huang et al. CVPR 2018
  40. 40. Other Relevant Papers 1. Resource Aware Person Re-identification across Multiple Resolutions. Yan Wang et al. CVPR 2018 2. Transferable Joint Attribute-Identity Deep Learning for Unsupervised Person Re-Identification. Jingya Wang et al. CVPR 2018 3. Dual Attention Matching Network for Context-Aware Feature Sequence based Person Re-Identification. Jianlou Si et al. CVPR 2018 4. Diversity Regularized Spatiotemporal Attention for Video-based Person Re-identification. Shuang Li et al. CVPR 2018 5. Exploit the Unknown Gradually: One-Shot Video-Based Person Re-Identification by Stepwise Learning. Yu Wu et al. CVPR 2018 6. Features for Multi-Target Multi-Camera Tracking and Re-Identification. Ergys Ristani et al. CVPR 2018 7. Deep Group-shuffling Random Walk for Person Re-identification. Yantao Shen. CVPR 2018
  41. 41. NN Architecture Selection Complete Results

Presentation of my Final Year Research during undergraduate studies

Views

Total views

62

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

5

Shares

0

Comments

0

Likes

0

×