Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Music and Art with Machine Learning | GDG DevFest Bangkok 2017 (Oct 7th, 2017)

718 views

Published on

This talk discusses Google Brain's Magenta, a project using TensorFlow to generate art and music.

The slides are adapted from Douglas Eck's talk at Google I/O 2017 (https://www.youtube.com/watch?v=2FAjQ6R_bf0)

  • Login to see the comments

Music and Art with Machine Learning | GDG DevFest Bangkok 2017 (Oct 7th, 2017)

  1. 1. Music and Art with Machine Learning Part I Ta Virot Chiraphadhanakul (@tvirot) Google Developer Expert in Machine Learning Managing Director, Skooldio
  2. 2. AI and Creativity
  3. 3. Make Music and Art Using Machine Learning https://magenta.tensorflow.org/
  4. 4. Deep Learning A Quick Intro
  5. 5. Photo: NVIDIA Blog
  6. 6. x1 + x2 = 0
  7. 7. http://playground.tensorflow.org/
  8. 8. http://playground.tensorflow.org/
  9. 9. DEMO https://teachablemachine.withgoogle.com/ Teachable Machine
  10. 10. Machine Learning on Google Cloud Platform Pre-trained ML models Custom ML models
  11. 11. Understand the content of images ● Label Detection ● Optical Character Recognition ● Explicit Content Detection ● Face Detection etc. Google Cloud Vision API Photos: Google Cloud Platform / Kaz Sato @tvirot
  12. 12. Generative Models Training a model to create things
  13. 13. "What an AI cannot create, it does not understand." — Ian Goodfellow, Google Brain
  14. 14. Style Transfer https://research.googleblog.com/2016/10/supercharging-style-transfer.html
  15. 15. https://experiments.withgoogle.com/ai/sound-maker/view/
  16. 16. https://deeplearnjs.org/demos/performance_rnn
  17. 17. Sketch-RNN A Generative Model for Sketch Drawings
  18. 18. https://quickdraw.withgoogle.com/
  19. 19. https://quickdraw.withgoogle.com/data
  20. 20. Autoencoders ● A neural network with the target values set to be equal to the inputs ● The goal is to learn an approximation to the identity function -- a lower-dimensional feature representation ● By placing constraints on the network, such as by limiting the number of hidden units, we can discover interesting structure about the data
  21. 21. Autoencoders Input OutputZEncoder Decoder Latent variable / Embedding Reconstructed Input
  22. 22. Sequence Models ● Data points are related or dependent on each other (e.g., text, audio waves, videos) ● Challenges: ○ Order matters ○ Statistical Invariance ○ Long-term dependencies
  23. 23. Recurrent Neural Networks Networks with loops http://colah.github.io/posts/2015-08-Understanding-LSTMs/
  24. 24. Unfolded RNN http://colah.github.io/posts/2015-08-Understanding-LSTMs/
  25. 25. A Simple RNN http://colah.github.io/posts/2015-08-Understanding-LSTMs/
  26. 26. RNN Architectures http://colah.github.io/posts/2015-08-Understanding-LSTMs/
  27. 27. Sketch-RNN Sequence-to-Sequence Variational Autoencoder ZEncoder Decoder A Neural Representation of Sketch Drawings (David Ha, Douglas Eck)
  28. 28. Sketch-RNN Sequence-to-Sequence Variational Autoencoder Forward + Backward RNN Z Decoder A Neural Representation of Sketch Drawings (David Ha, Douglas Eck)
  29. 29. Sketch-RNN Sequence-to-Sequence Variational Autoencoder Latent vector input + noise Encoder Decoder A Neural Representation of Sketch Drawings (David Ha, Douglas Eck)
  30. 30. Sketch-RNN Sequence-to-Sequence Variational Autoencoder RNN + Mixture of Gaussians ZEncoder A Neural Representation of Sketch Drawings (David Ha, Douglas Eck)
  31. 31. Sketch-RNN Sequence-to-Sequence Variational Autoencoder A Neural Representation of Sketch Drawings (David Ha, Douglas Eck)
  32. 32. https://github.com/tensorflow/magenta-demos/blob/master/jupyter-notebooks/Sketch_RNN.ipynb
  33. 33. Conditional Reconstructions from a model trained on cat sketches https://research.googleblog.com/2017/04/teaching-machines-to-draw.html
  34. 34. Conditional Reconstructions from a model trained on cat sketches A Neural Representation of Sketch Drawings (David Ha, Douglas Eck)
  35. 35. Conditional Reconstructions from a model trained on pig sketches A Neural Representation of Sketch Drawings (David Ha, Douglas Eck)
  36. 36. DEMO https://magenta.tensorflow.org/assets/sketch_rnn_demo/multi_vae.html Variational Autoencoder
  37. 37. Latent Space Interpolation: how one image morphs into another? https://research.googleblog.com/2017/04/teaching-machines-to-draw.html
  38. 38. DEMO https://magenta.tensorflow.org/assets/sketch_rnn_demo/interp.html Sketch-RNN Interpolation
  39. 39. Latent Space Learned relationships between abstract concepts A Neural Representation of Sketch Drawings (David Ha, Douglas Eck)
  40. 40. Predicting endings of incomplete sketches A Neural Representation of Sketch Drawings (David Ha, Douglas Eck)
  41. 41. DEMO https://magenta.tensorflow.org/assets/sketch_rnn_demo/multi_predict.html Sketch-RNN Predictor
  42. 42. Creative Applications
  43. 43. Pattern designs Generate a large number of similar, but unique designs A Neural Representation of Sketch Drawings (David Ha, Douglas Eck)
  44. 44. Abstract designs A Neural Representation of Sketch Drawings (David Ha, Douglas Eck)
  45. 45. Thank you! #devfestbkk #machinelearning Ta Virot Chiraphadhanakul (@tvirot) Google Developer Expert in Machine Learning Managing Director, Skooldio

×