This document discusses different convolutional neural network architectures including traditional architectures using convolutional, pooling, and fully connected layers, siamese networks for learning visual similarity, dense prediction networks for tasks like semantic segmentation and image colorization, video classification networks, music recommendation networks, and networks for tasks like object localization, detection, and alignment. It provides examples of specific networks that have been applied to each type of architecture.

1. Deep Learning
Christian S. Perone
Jun 2016
Convolutional
Neural Networks
Architectural Zoo

2. Who
Christian S. Perone
Machine Learning Engineer
Software Engineer
Blog
http://blog.christianperone.com
Open-source
http://github.com/perone
Twitter @tarantulae

3. Agenda
 Traditional Architectures
 Siamese Networks
 Dense Predictions
 Video
 Music Recommendation
 Localization, Detection, Alignment
 Q & A

4. Convolutional
Neural
Networks
Traditional
Architectures
1

5. Traditional Building Blocks
Three main building blocks:
– Convolutional
– Pooling
– Fully Connected (Dense)
Image by Liu et al, 2016
Architecture Overview

6. Traditional Building Blocks
The input data
Color Image
(RGB)
28 pixels (width)
28 pixels (height)
3 channels
(RGB)

7. Traditional Building Blocks
Convolutions
Animation by Vincent Dumoulin et al, 2016
Image by Apple

8. Traditional Building Blocks
Pooling
Images by Karpathy, cs2311 Stanford, 2016

9. Traditional Building Blocks
Fully Connected

10. Traditional Architecture
The canonical CNNs
Images by Rob Hess et al, 2016

11. Convolutional
Neural
Networks
Siamese
Networks
2

12. Architectural Zoo
The Siamese Architecture
Learning Hierarchies of Invariant Features. Yann LeCun.

13. Architectural Zoo
The Siamese Architecture
Learning visual similarity for product design with convolutional neural networks, Sean Bell et al

14. Architectural Zoo
The Siamese Architecture
Learning visual similarity for product design with convolutional neural networks, Sean Bell et al

15. Architectural Zoo
The Siamese Architecture
Learning visual similarity for product design with convolutional neural networks, Sean Bell et al

16. Architectural Zoo
The Siamese Architecture
Learning Deep Representations for Ground-to-Aerial Geolocalization, Tsung-Yi Lin et al. 2016.

17. Architectural Zoo
The Siamese Architecture
Learning Deep Representations for Ground-to-Aerial Geolocalization, Tsung-Yi Lin et al. 2016.

18. Convolutional
Neural
Networks
Dense
Prediction
3

19. Architectural Zoo
Dense Prediction
Fully Convolutional Networks for Semantic Segmentation. Jonathan Long et al. 2015

20. Architectural Zoo
Dense Prediction
Fully Convolutional Networks for Semantic Segmentation. Jonathan Long et al. 2015

21. Architectural Zoo
Dense Prediction
Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification. Satoshi Iizuka et al. 2016

22. Architectural Zoo
Dense Prediction
Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification. Satoshi Iizuka et al. 2016

23. Architectural Zoo
Dense Prediction
Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification. Satoshi Iizuka et al. 2016

24. Architectural Zoo
Dense Prediction
Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification. Satoshi Iizuka et al. 2016

25. Architectural Zoo
Dense Prediction
Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup. Edgar Simo-Serra et al. 2016.

26. Architectural Zoo
Dense Prediction
Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup. Edgar Simo-Serra et al. 2016.

27. Convolutional
Neural
Networks
Video
4

28. Architectural Zoo
Video
Large-scale Video Classification with Convolutional Neural Networks . Andrej Karpathy. 2014

29. Architectural Zoo
Video
Large-scale Video Classification with Convolutional Neural Networks . Andrej Karpathy. 2014

30. Architectural Zoo
Video
Learning Spatiotemporal Features with 3D Convolutional Networks . Du Tran et al. 2015

31. Convolutional
Neural
Networks
Music
Recommendation
5

32. Architectural Zoo
Music Recommendation
Recommending music on Spotify with Deep Learning. Sander Dieleman. 2014.

33. Convolutional
Neural
Networks
Localization
Detection
Alignment
6

34. Architectural Zoo
Localization, Detection and Alignment
Rich feature hierarchies for accurate object detection and semantic segmentation. Girschick et al. 2014.

35. Architectural Zoo
Localization, Detection and Alignment
Selective Search for Object Recognition. Uijlings et al. IJCV 2013

36. Architectural Zoo
Localization, Detection and Alignment
Fast R-CNN. Girschick et al. 2015.

37. Architectural Zoo
Localization, Detection and Alignment
Image by Fei-Fei Li & Andrej Karpathy & Justin Johnson. Cs231n, 2016.
Fast R-CNN. Girschick et al. 2015.

38. Architectural Zoo
Localization, Detection and Alignment
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Ren et al. 2015.
Image by Ross Girschick / Slide from CS231n, Fei-Fei & Andrej Karpathy & Justin Johnson. 2016.

39. Architectural Zoo
Localization, Detection and Alignment
Right Whale Recognition Challenge, winner report. Robert Bogucki, Deepsense.io. 2016.

40. Architectural Zoo
Localization, Detection and Alignment
Spatial Transformer Networks. Max Jaderberg et al. 2015.