1. Clockwork ConvNets for Video
Semantic Segmentation
2018/02/10

2. Fully Convolutional Networks

3. The Idea – Adapt FCN to Video Segmentation
• Naïve Idea: segment video frames as still images
• Drawbacks
• Computation redundancy
• Ignoring the temporal continuity inherit
• Two Key observations:
• The speed of motion varies in video.
• Deeper layers in the feature hierarchy change more slowly.

4. Analysis

5. The Overall Framework

6. Clockwork ConvNets on Time

7. Clockwork
• Fix-rated clockwork
• Exponential clockwork: 1, 2, 4, 8, …
• Alternating clockwork: 0, 1, 0, 1
• Adaptive clockwork
• Threshold clockwork:
• Learned clockwork:

8. Fix-rated Clockwork – translated PASCAL

9. Adaptive Clockwork – YouTube-Objects

10. The Idea -- Clockwork
• Hardship in learning Long-term dependencies (Vanishing gradient)

11. The Clockwork RNN

12. The End