Recording of the meetup can be viewed here: https://youtu.be/hEPWZXWOj58 In this presentation, Simon goes through some themes and some papers from ICLR 2018 conference, which happened this May in Vancouver. In particular, he went through the advances in: - GAN training - Adversarial examples - New building blocks for deep learning - Learning more effectively with less labeled data - New domains to apply deep learning techniques A good way to get a download of some fun papers if you missed the conference! Simon's Bio: Simon Kozlov is a Machine Learning Engineer at Instrumental focusing on applying deep learning techniques to the manufacturing space. Prior to Instrumental, he was part of the machine learning team at Dropbox. Before that, he helped make deep learning possible by giving a reason for GPUs to exist - in other words, worked on games and real-time rendering.

2. Hi!
I lead ML things at
We’re ~20 people
Before that:

3. 2018: ~1000 2018: 2000-3000
Source

4. Generative Adversarial Networks (GANs)

5. Original paper
Generative Adversarial Networks’14

6. Better architecture
Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks’15

7. Spectral normalization
Spectral Normalization for Generative Adversarial Networks’18

8. Spectral Normalization for Generative Adversarial Networks’18

9. Finally!
Progressive Growing of GANs for Improved Quality, Stability, and Variation’17

10. Progressive GANs
Progressive Growing of GANs for Improved Quality, Stability, and Variation’17

12. Adversarial Examples
Intriguing properties of neural networks’14

13. Works in real world
Adversarial Patch’17Robust Adversarial Examples’17

14. Adversarial Subspaces
Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality, slides from ICLR 2018

15. Adversarial Subspaces
Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality’18

17. Learning efficiency

18. Domain Adaptation

19. DIRT-T’18
76.5% accuracy
on MNIST->SVHN

20. Memory-based Adaptation’18

21. Meta-Learning: Learning to Learn
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks’17

22. New blocks – Adam upgrade
Adaptive Subgradient Methods for Online Learning and Stochastic Optimization’11
accumulated = beta2 * accumulated + (1-
beta2) * gradient ** 2
adaptive_learning_rate = learning_rate /
sqrt(accumulated)
w = w - adaptive_learning_rate * velocity

23. Adam update
On Convergence of Adam and Beyound, ICLR slides’18

24. Progress on ConvNet architectures
Deep Layer Aggregation’17

25. Source
Reinforcement Learning

26. Reinforcement Learning

27. New tasks
Emergent Complexity via Multi-Agent Competition’17

28. Emergent Communication
Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input’18

29. Emergent Communication
Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input’18

31. Spherical CNNs’18

32. Multi-Task Learning for Document Ranking and Query Suggestion’18

33. THE EFFECTIVENESS OF A TWO-LAYER NEURAL NETWORK FOR RECOMMENDATIONS’18

34. Putting a bug in ML: The moth olfactory network learns to read MNIST’18

35. simon@instrumental.com