Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Random thoughts on
Paper Implementation
Taehoon Kim / carpedm20
Taehoon Kim / carpedm20 /
Paper Implementation
Release of
2015.11.08
Sukhbaatar, Sainbayar, Jason Weston, and Rob Fergus. "End-to-end memory networks." Advances in neural information processi...
Sukhbaatar, Sainbayar, Jason Weston, and Rob Fergus. "End-to-end memory networks." Advances in neural information processi...
Sukhbaatar, Sainbayar, Jason Weston, and Rob Fergus. "End-to-end memory networks." Advances in neural information processi...
https://github.com/facebook/MemNN/tree/mater/MemN2N-lang-model
https://github.com/carpedm20/MemN2N-tensorflow
Lua Torch →
Lua Torch →
Simple Translation
→
https://github.com/carpedm20/MemN2N-tensorflow/blob/master/model.py
Lua Torch 0.5.0
Translation
20+ implementations
900 days
https://carpedm20.github.io/#personal-projects
13,000+
Received questions about implementation
4 random thoughts
1. What to implement
2. What to read
3. TensorFlow or PyTorch?
4. Grants and Competitions
1. What to implement
Computer Vision
Natural Language Processing
Reinforcement Learning
Semi/Un-supervised Learning
…
Computer Vision
Natural Language Processing
Reinforcement Learning
Semi/Un-supervised Learning
…
GAN
Generative Adversarial Network
Too many GANs..
Relatively easier to implement many but not much of a difference
Reasoning
Question Answering
Controllable Text Generation
Unsupervised Embedding
Exploration Without Reward
Neural Program...
Meta (Reinforcement) Learning
Transfer Learning
https://blog.openai.com/evolved-policy-gradients/
Adaptive learning from prior experience
with Evolution Strategies (ES)
https://blog.openai.com/evolved-policy-gradients/
Easiest way : learning from pretrained network
But how can we do better?
https://blog.openai.com/evolved-policy-gradients/
𝜃
Loss : 𝑓 𝑥; 𝜃 − 𝑦 (
Neural Network : 𝑓 𝑥; 𝜃
https://blog.openai.com/evolved-policy-gradients/
𝜃𝜙
Loss : 𝐿 𝑥, 𝑦; 𝜙
Neural Network : 𝑓 𝑥; 𝜃
https://blog.openai.com/evolved-policy-gradients/
𝜃𝜙
https://blog.openai.com/evolved-policy-gradients/
https://blog.openai.com/evolved-policy-gradients/
Evolved Policy Gradient Policy Gradient
Ganin, Yaroslav, et al. "Synthesizing Programs for Images using Reinforced Adversarial Learning." arXiv preprint arXiv:180...
Ganin, Yaroslav, et al. "Synthesizing Programs for Images using Reinforced Adversarial Learning." arXiv preprint arXiv:180...
Ganin, Yaroslav, et al. "Synthesizing Programs for Images using Reinforced Adversarial Learning." arXiv preprint arXiv:180...
https://blog.openai.com/ingredients-for-robotics-research/
https://blog.openai.com/robots-that-learn/
https://blog.openai.com/competitive-self-play/
2. What to Read
“How to implement a paper FASTER?”
Read others code a lot
Recommendations
• NLP: End-to-End Memory Network
• https://github.com/carpedm20/MemN2N-tensorflow
• Vision: (Fast) Style T...
More (nerdy) recommendations
• from tensorflow.contrib.seq2seq import Helper:
• https://github.com/keithito/tacotron/blob/...
Two useful code snippets
https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/train.py
https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/train.py
https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/train.py
https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/image.py
https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/image.py
https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/image.py
64 batch image → 8 × 8 image grid summary
https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/image.py
64 batch image → 8 × 8 image grid summary
Read more recent and correct code
Never read carpedm20’s code (no joke. BAD examples)
3. TensorFlow or PyTorch?
or
or
Mom
Dad
• Dirty long code
• More stressful debugging
• Faster development (1.5.0: Jan. 18, 1.6.0-1.7.0: Mar. 18, 1.8.0: Apr. 18, p...
… compiling parts of the computational graph with XLA (a TensorFlow Just-In-Time compiler) and
…
Espeholt, Lasse, et al. "...
tensorboard --port 6006 --debugger_port 6064
https://github.com/tensorflow/tensorboard/tree/master/tensorboard/plugins/deb...
https://github.com/tensorflow/tensorboard/tree/master/tensorboard/plugins/debugger
https://github.com/tensorflow/tensorboard/tree/master/tensorboard/plugins/debugger
https://twitter.com/jordannovet/status/977645002200252416
• Clean short code
• Less stressful debugging
• Slower development (0.2.0: Aug. 17, 0.3.0: Dec. 17)
• Dirty (partial) save...
4. Grants and Competitions
This is not about
https://blog.openai.com/openai-scholars/
https://blog.openai.com/openai-scholars/
Open Challenges
https://blog.openai.com/retro-contest/
https://contest.openai.com/
Workshop Challenges
NIPS, ICML, ICLR
http://www.cs.mcgill.ca/~jpineau/ICLR2018-ReproducibilityChallenge.html
http://www.cs.mcgill.ca/~jpineau/ICLR2018-ReproducibilityChallenge.html
https://openreview.net/forum?id=r1dHXnH6-
Feedback from Authors
https://arxiv.org/abs/1802.03198
So,
Read a paper
Implement a code
Seize the day :)
Great opportunities await you
One more thing...
2015.06 ~
https://www.slideshare.net/carpedm20/ai-67616630
Teaching Machines to Understand Visual Manuals
via Attention Supervision for Object Assembly
Teaching Machines to Understand Visual Manuals via Attention Supervision for Object Assembly (work in progress)
https://www.slideshare.net/carpedm20/deview-2017-80824162
https://carpedm20.github.io/tacotron/
Person A
https://carpedm20.github.io/tacotron/
Person A
https://carpedm20.github.io/tacotron/
Person A
https://carpedm20.github.io/tacotron/
Person B
https://carpedm20.github.io/tacotron/
Person B
https://carpedm20.github.io/tacotron/
Person B
http://www.devsisters.com/jobs/
Thank you
@carpedm20
Random Thoughts on Paper Implementations [KAIST 2018]
Random Thoughts on Paper Implementations [KAIST 2018]
Random Thoughts on Paper Implementations [KAIST 2018]
Random Thoughts on Paper Implementations [KAIST 2018]
Random Thoughts on Paper Implementations [KAIST 2018]
Random Thoughts on Paper Implementations [KAIST 2018]
Upcoming SlideShare
Loading in …5
×

Random Thoughts on Paper Implementations [KAIST 2018]

54,154 views

Published on

Post: https://carpedm20.github.io/2018/paper-implementation/

Published in: Technology

Random Thoughts on Paper Implementations [KAIST 2018]

  1. 1. Random thoughts on Paper Implementation Taehoon Kim / carpedm20
  2. 2. Taehoon Kim / carpedm20 /
  3. 3. Paper Implementation
  4. 4. Release of 2015.11.08
  5. 5. Sukhbaatar, Sainbayar, Jason Weston, and Rob Fergus. "End-to-end memory networks." Advances in neural information processing systems. 2015.
  6. 6. Sukhbaatar, Sainbayar, Jason Weston, and Rob Fergus. "End-to-end memory networks." Advances in neural information processing systems. 2015.
  7. 7. Sukhbaatar, Sainbayar, Jason Weston, and Rob Fergus. "End-to-end memory networks." Advances in neural information processing systems. 2015.
  8. 8. https://github.com/facebook/MemNN/tree/mater/MemN2N-lang-model
  9. 9. https://github.com/carpedm20/MemN2N-tensorflow
  10. 10. Lua Torch →
  11. 11. Lua Torch → Simple Translation
  12. 12. → https://github.com/carpedm20/MemN2N-tensorflow/blob/master/model.py Lua Torch 0.5.0 Translation
  13. 13. 20+ implementations 900 days https://carpedm20.github.io/#personal-projects
  14. 14. 13,000+
  15. 15. Received questions about implementation
  16. 16. 4 random thoughts
  17. 17. 1. What to implement 2. What to read 3. TensorFlow or PyTorch? 4. Grants and Competitions
  18. 18. 1. What to implement
  19. 19. Computer Vision Natural Language Processing Reinforcement Learning Semi/Un-supervised Learning …
  20. 20. Computer Vision Natural Language Processing Reinforcement Learning Semi/Un-supervised Learning …
  21. 21. GAN Generative Adversarial Network
  22. 22. Too many GANs.. Relatively easier to implement many but not much of a difference
  23. 23. Reasoning Question Answering Controllable Text Generation Unsupervised Embedding Exploration Without Reward Neural Programmers Neural Architecture Search Hierarchical Representation Learning Video Generation Program Induction Self-Play Learning Adversarial Attack Multi-agent RL Model-based RL Meta Learning Learned Data Augmentation Speech Synthesis Music Generation
  24. 24. Meta (Reinforcement) Learning Transfer Learning
  25. 25. https://blog.openai.com/evolved-policy-gradients/ Adaptive learning from prior experience with Evolution Strategies (ES)
  26. 26. https://blog.openai.com/evolved-policy-gradients/ Easiest way : learning from pretrained network But how can we do better?
  27. 27. https://blog.openai.com/evolved-policy-gradients/ 𝜃 Loss : 𝑓 𝑥; 𝜃 − 𝑦 ( Neural Network : 𝑓 𝑥; 𝜃
  28. 28. https://blog.openai.com/evolved-policy-gradients/ 𝜃𝜙 Loss : 𝐿 𝑥, 𝑦; 𝜙 Neural Network : 𝑓 𝑥; 𝜃
  29. 29. https://blog.openai.com/evolved-policy-gradients/ 𝜃𝜙
  30. 30. https://blog.openai.com/evolved-policy-gradients/
  31. 31. https://blog.openai.com/evolved-policy-gradients/ Evolved Policy Gradient Policy Gradient
  32. 32. Ganin, Yaroslav, et al. "Synthesizing Programs for Images using Reinforced Adversarial Learning." arXiv preprint arXiv:1804.01118 (2018). Distributed RL + GAN
  33. 33. Ganin, Yaroslav, et al. "Synthesizing Programs for Images using Reinforced Adversarial Learning." arXiv preprint arXiv:1804.01118 (2018).
  34. 34. Ganin, Yaroslav, et al. "Synthesizing Programs for Images using Reinforced Adversarial Learning." arXiv preprint arXiv:1804.01118 (2018).
  35. 35. https://blog.openai.com/ingredients-for-robotics-research/
  36. 36. https://blog.openai.com/robots-that-learn/
  37. 37. https://blog.openai.com/competitive-self-play/
  38. 38. 2. What to Read
  39. 39. “How to implement a paper FASTER?”
  40. 40. Read others code a lot
  41. 41. Recommendations • NLP: End-to-End Memory Network • https://github.com/carpedm20/MemN2N-tensorflow • Vision: (Fast) Style Transfer • https://github.com/lengstrom/fast-style-transfer • RL: Asynchronous Methods for Deep Reinforcement Learning • https://github.com/openai/universe-starter-agent (TBH code is dirty but there are lots of things to learn) • https://github.com/ikostrikov/pytorch-a3c • Etc: Neural Turing Machine • https://github.com/loudinthecloud/pytorch-ntm
  42. 42. More (nerdy) recommendations • from tensorflow.contrib.seq2seq import Helper: • https://github.com/keithito/tacotron/blob/master/models/helpers.py • tf.while_loop: • https://github.com/melodyguan/enas/blob/master/src/ptb/ptb_enas_child.py • import tensorflow.contrib.graph_editor as ge: • https://github.com/openai/gradient-checkpointing/blob/master/memory_saving_gradients.py • Google’s production level code example: • https://github.com/tensorflow/tensor2tensor • The Annotated Transformer • http://nlp.seas.harvard.edu/2018/04/03/attention.html • Relatively safe-to-read codes: • https://github.com/tensorflow/models
  43. 43. Two useful code snippets
  44. 44. https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/train.py
  45. 45. https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/train.py
  46. 46. https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/train.py
  47. 47. https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/image.py
  48. 48. https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/image.py
  49. 49. https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/image.py 64 batch image → 8 × 8 image grid summary
  50. 50. https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/image.py 64 batch image → 8 × 8 image grid summary
  51. 51. Read more recent and correct code Never read carpedm20’s code (no joke. BAD examples)
  52. 52. 3. TensorFlow or PyTorch?
  53. 53. or
  54. 54. or Mom Dad
  55. 55. • Dirty long code • More stressful debugging • Faster development (1.5.0: Jan. 18, 1.6.0-1.7.0: Mar. 18, 1.8.0: Apr. 18, possibly more reliable) • Easier (partial) save and load models (tf.Supervisor) • Harder dynamic computation (tf.fold) • XLA, TensorBoard, TPU, tf.eager, Multi-node distribution, Documentation
  56. 56. … compiling parts of the computational graph with XLA (a TensorFlow Just-In-Time compiler) and … Espeholt, Lasse, et al. "IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures." arXiv preprint arXiv:1802.01561 (2018).
  57. 57. tensorboard --port 6006 --debugger_port 6064 https://github.com/tensorflow/tensorboard/tree/master/tensorboard/plugins/debugger
  58. 58. https://github.com/tensorflow/tensorboard/tree/master/tensorboard/plugins/debugger
  59. 59. https://github.com/tensorflow/tensorboard/tree/master/tensorboard/plugins/debugger
  60. 60. https://twitter.com/jordannovet/status/977645002200252416
  61. 61. • Clean short code • Less stressful debugging • Slower development (0.2.0: Aug. 17, 0.3.0: Dec. 17) • Dirty (partial) save and load models • Easier dynamic computation
  62. 62. 4. Grants and Competitions
  63. 63. This is not about
  64. 64. https://blog.openai.com/openai-scholars/
  65. 65. https://blog.openai.com/openai-scholars/
  66. 66. Open Challenges
  67. 67. https://blog.openai.com/retro-contest/
  68. 68. https://contest.openai.com/
  69. 69. Workshop Challenges NIPS, ICML, ICLR
  70. 70. http://www.cs.mcgill.ca/~jpineau/ICLR2018-ReproducibilityChallenge.html
  71. 71. http://www.cs.mcgill.ca/~jpineau/ICLR2018-ReproducibilityChallenge.html
  72. 72. https://openreview.net/forum?id=r1dHXnH6- Feedback from Authors
  73. 73. https://arxiv.org/abs/1802.03198
  74. 74. So,
  75. 75. Read a paper
  76. 76. Implement a code
  77. 77. Seize the day :) Great opportunities await you
  78. 78. One more thing...
  79. 79. 2015.06 ~
  80. 80. https://www.slideshare.net/carpedm20/ai-67616630
  81. 81. Teaching Machines to Understand Visual Manuals via Attention Supervision for Object Assembly
  82. 82. Teaching Machines to Understand Visual Manuals via Attention Supervision for Object Assembly (work in progress)
  83. 83. https://www.slideshare.net/carpedm20/deview-2017-80824162
  84. 84. https://carpedm20.github.io/tacotron/ Person A
  85. 85. https://carpedm20.github.io/tacotron/ Person A
  86. 86. https://carpedm20.github.io/tacotron/ Person A
  87. 87. https://carpedm20.github.io/tacotron/ Person B
  88. 88. https://carpedm20.github.io/tacotron/ Person B
  89. 89. https://carpedm20.github.io/tacotron/ Person B
  90. 90. http://www.devsisters.com/jobs/
  91. 91. Thank you @carpedm20

×