Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Object Detection Beyond Mask R-CNN and RetinaNet II

21 views

Published on

ICME2019 Tutorial: Object Detection Beyond Mask R-CNN and RetinaNet II

Published in: Internet
  • Be the first to comment

  • Be the first to like this

Object Detection Beyond Mask R-CNN and RetinaNet II

  1. 1. AutoML for Object Detection Xiangyu Zhang MEGVII Research
  2. 2. AutoML for Object Detection • Advances in AutoML • Search for Detection Systems 2 1
  3. 3. AutoML for Object Detection • Advances in AutoML • Search for Detection Systems 2 1
  4. 4. Introduction v AutoML o A meta-approach to generate machine learning systems o Automatically search vs. manually design v AutoML for Deep Learning o Neural architecture search (NAS) o Hyper-parameters turning o Loss function o Data augmentation o Activation function o Backpropagation …
  5. 5. Revolution of AutoML v ImageNet 2012 - o Hand-craft feature vs. deep learning v Era of Deep Learning begins! 27 26.2 16.4 8.1 7.3 6.6 4.9 3.57 OXFORD ISI AlexNet SPPnet VGG GoogleNet PReLU ResNet 152 Classification Top-5 Error (%)
  6. 6. Revolution of AutoML (cont’d) v ImageNet 2017 - o Manual architecture vs. AutoML models 19.1 17.3 17.3 17.1 16.1 15.6 ResNeXt-101 SENet NASNet-A PNASNet-5 AmoebaNet-A EfficientNet Classification Top-1 Error (%) Era of AutoML?
  7. 7. Revolution of AutoML (cont’d) v Literature o 200+ since 2017
  8. 8. Revolution of AutoML (cont’d) v Literature o 200+ since 2017 v Google Trends
  9. 9. Recent Advances in AutoML (1) v Surpassing handcraft models o NASNet v Keynotes o RNN controller + policy gradient o Flexible search space o Proxy task needed Zoph et al. Learning Transferable Architectures for Scalable Image Recognition Zoph et al. Neural Architecture Search with Reinforcement Learning
  10. 10. Recent Advances in AutoML (2) v Search on the target task o MnasNet v Keynotes o Search directly on ImageNet o Platform aware search o Very costly (thousands of TPU-days) Tan et al. MnasNet: Platform-Aware Neural Architecture Search for Mobile
  11. 11. Recent Advances in AutoML (3) v Weight Sharing for Efficient Search & Evaluation o ENAS o One-shot methods v Keynotes o Super network o Finetuning & inference only instead of retraining o Inconsistency in super net evaluation Pham et al. Efficient Neural Architecture Search via Parameter Sharing Bender et al. Understanding and Simplifying One-Shot Architecture Search Guo et al. Single Path One-Shot Neural Architecture Search with Uniform Sampling
  12. 12. Recent Advances in AutoML (4) v Gradient-based methods o DARTS o SNAS, FBNet, ProxylessNAS, … v Keynotes o Joint optimization of architectures and weights o Weight sharing implied o Sometimes less flexible Liu et al. DARTS: Differentiable Architecture Search Xie et al. SNAS: Stochastic Neural Architecture Search Cai et al. ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware Wu et al. FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search
  13. 13. Recent Advances in AutoML (5) v Performance Predictor o Neural Architecture Optimization o ChamNet v Keynotes o Architecture encoding o Performance prediction models o Cold start problem Luo et al. Neural Architecture Optimization Dai et al. ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation
  14. 14. Recent Advances in AutoML (6) v Hardware-aware Search o Search with complexity budget o Quantization friendly o Energy-aware search … v Keynotes o Complexity-aware loss & reward o Multi-target search o Device in the loop Wu et al. Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search V´eniat et al. Learning Time/Memory-Efficient Deep Architectures with Budgeted Super Networks Wang et al. HAQ: Hardware-Aware Automated Quantization with Mixed Precision
  15. 15. Recent Advances in AutoML (7) v AutoML in Model Pruning o NetAdapt o AMC o MetaPruning v Keynotes o Search for the pruned architecture o Hyper-parameters like channels, spatial size, … Yang et al. NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications He et al. AMC: AutoML for Model Compression and Acceleration on Mobile Devices Liu et al. MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning
  16. 16. Recent Advances in AutoML (8) v Handcraft + NAS o Human-expert guided search (IRLAS) o Boosting existing handcraft models (EfficientNet, MobileNet v3) v Keynotes o Very competitive performance o Efficient o Search space may be restricted Howard et al. Searching for MobileNetV3 Tan et al. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Guo et al. IRLAS: Inverse Reinforcement Learning for Architecture Search
  17. 17. Recent Advances in AutoML (9) v Various Tasks o Object Detection o Semantic Segmentation o Super-resolution o Face Recognition … Liu et al. Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation Chu et al. Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Search Ramachandra et al. Searching for Activation Functions Alber et al. Backprop Evolution v Not only NAS, search for everything! o Activation function o Loss function o Data augmentation o Backpropagation …
  18. 18. Recent Advances in AutoML (10) v Rethinking the Effectiveness of NAS o Random search o Random wire network v Keynotes o Reproducibility o Search algorithm or search space? o Baselines Li et al. Random Search and Reproducibility for Neural Architecture Search Xie et al. Exploring Randomly Wired Neural Networks for Image Recognition
  19. 19. Summary: Trends and Challenges v Trends o Efficient & high-performance algorithm o Flexible search space o Device-aware optimization o Multi-task / Multi-target search v Challenges o Trade-offs between efficiency, performance and flexibility o Search space matters! o Fair benchmarks o Pipeline search Efficiency FlexibilityPerformance
  20. 20. AutoML for Object Detection • Advances in AutoML • Search for Detection Systems 2 1
  21. 21. AutoML for Object Detection v Components to search o Image preprocessing o Backbone o Feature fusion o Detection head & loss function …
  22. 22. AutoML for Object Detection v Components to search o Image preprocessing o Backbone o Feature fusion o Detection head & loss function …
  23. 23. AutoML for Object Detection v Components to search o Image preprocessing o Backbone o Feature fusion o Detection head & loss function …
  24. 24. AutoML for Object Detection v Components to search o Image preprocessing o Backbone o Feature fusion o Detection head & loss function …
  25. 25. AutoML for Object Detection v Components to search o Image preprocessing o Backbone o Feature fusion o Detection head & loss function …
  26. 26. Search for Detection Systems Feature Fusion Augmentation DetNAS Chen et al. DetNAS: Backbone Search for Object Detection
  27. 27. Challenges of Backbone Search v Similar to general NAS, but … o Controller & evaluator loop o Performance evaluation is very slow v Detection backbone evaluation involves a costly pipeline o ImageNet pretraining o Finetuning on the detection dataset (e.g. COCO) o Evaluation on the validation set
  28. 28. Related Work: Single Path One-shot NAS v Decoupled weight training and architecture optimization v Super net training Guo et al. Single Path One-Shot Neural Architecture Search with Uniform Sampling
  29. 29. Pipeline v Single-pass approach o Pretrain and finetune super net only once
  30. 30. Search Space v Single path super net o 20 (small) or 40 (large) choice blocks o 4 candidates for each choice block o Search space size: 420 or 440
  31. 31. Search Algorithm v Evolutionary search o Sample & reuse the weights from super net o Very efficient
  32. 32. Results v High performance o Significant improvements over commonly used backbones (e.g. ResNet 50) with fewer FLOPs o Best classification backbones may be suboptimal for object detection
  33. 33. Results v Search cost o Super nets greatly speed up search progress!
  34. 34. Search for Detection Systems Backbone Feature Fusion Augmentation NAS-FPN Ghaisi et al. NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection
  35. 35. Feature Fusion Modules v Multi-scale feature fusion o Used in state-of-the-art detectors (e.g. SSD, FPN, SNIP, FCOS, …) v Automatic search vs. manual design
  36. 36. First Glance v Searched architecture o Very different from handcraft structures
  37. 37. Search Space v Stacking repeated FPN blocks v For each FPN block, N different merging cells v For each merging cell, 4-step generations
  38. 38. Search Algorithm v Controller o RNN-based controller o Search with Proximal Policy Optimization (PPO) v Candidate evaluation o Training a light-weight proxy task
  39. 39. Architectures During Search v Many downsamples and upsamples
  40. 40. Results v State-of-the-art speed/AP trade-off
  41. 41. Search for Detection Systems Backbone Feature Fusion Augmentation Auto-Augment for Detection Zoph et al. Learning Data Augmentation Strategies for Object Detection
  42. 42. Data Augmentation for Object Detection v Augmentation pool o Color distortions o Geometric transforms o Random noise (e.g. cutout, drop block, …) o Mix-up … v Search for the best augmentation configurations
  43. 43. Search Space Design v Mainly follows AutoAugment v Randomly sampling from K sub-policies v For each sub-policy, N image transforms v Each image transform selected from 22 operations: o Color operations o Geometric operations o Bounding box operations Cubuk et al. AutoAugment: Learning Augmentation Strategies from Data
  44. 44. Search Space Design (cont’d)
  45. 45. Search Algorithm v Very similar to NAS-FPN v Controller o RNN-based controller o Search with Proximal Policy Optimization (PPO) v Evaluation o A small proxy dataset o Short-time training
  46. 46. Results v Significantly outperforms previous state-of-the-arts
  47. 47. Analysis v Better regularization
  48. 48. Future Work v More search dimensions o E.g. loss, anchor boxes, assign rules, post-processing, … v Reducing search cost v Joint optimization
  49. 49. Q & A

×