Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

machine learning and art hack day

948 views

Published on

My slides from the presentation I gave

Published in: Technology
  • Be the first to comment

machine learning and art hack day

  1. 1. SOME EXPERIMENTS I HAVE DONE WITH ART + DEEP LEARNING Jason Toy
  2. 2. THE PRESENTATION show some of my experiments, both testing the limits of other people’s models and training my own models overview of how the models work for artists and machine learners in the audience, i tried to make it so you will all learn a little hopefully inspire some of you to go out and build something awesome
  3. 3. MY BACKGROUND - JASONTOY my main passion is general artificial intelligence studied math and computer science generalists, program a little of everything, master of nothing founded a couple of companies: rubynow, socmetrics - using ML for mining social media CEO of filepicker,sold in beginning of 2016 exploring the intersection of machine learning,art, entrepreneurship
  4. 4. DEEP LEARNINGVSTRADITIONAL MACHINE LEARNING mostly automated feature extraction
  5. 5. DEEP LEARNINGVSTRADITIONAL MACHINE LEARNING much better at learning nonlinear relationships
  6. 6. WHAT IS GENERATIVE MODELING generative vs discriminative architect of models GM around for a long time - used in architect, design,games,etc miniature systems that mimic something in real life,“artist in a box” more fun; I'm not as interested to increase ad clickthrough rates
  7. 7. DISCRIMINATIVEVS GENERATIVE generative: naive bayes LDA deep learning discriminative: SVM random forest linear/logistic regression
  8. 8. GENERATIVE DEEP MODELS tweak-able output w/ vectors
  9. 9. EXPERIMENTS
  10. 10. CHAR-RNN “I'm not going anywhere. I will bring the poorly educated back bigger and better. It's an incredible movement. ” “We're losing companies, the economy.We are going to save it. We're going to bring the party. Let's Make America Great Again” “I want to thank the volunteers.They've been unbelievable, they work like endlessly, you know, they don't want to die. My leadership is good”
  11. 11. CHAR-RNN 1vanilla (image classification) 2 sequence output (image -> text) 3 sequence input ( sentiment analysis) 4 seq2seq (machine translation) 5 synced seq2seq (video classification)
  12. 12. CHAR-RNN RNN - recurrent because they perform the same task for every element of a sequence; typically 2-3 layers LSTM - long short term memory similar, state is calculated differently
  13. 13. MY CHAR-RNN EXPERIMENTS what does Hellen Keller think? seeing is like or inspirents of a kiss licks, in child, for the last decting of accomplish with me for the mistakes in silence is to keep the moments filled whiter, the chaps of the house language was sends a humanise. i wish i could presepred its repepenting and the days like the poor discuss of language of the poem in the letters, dotiment in the endless good and eager and over the charicality of the hall of rubbings that I hapmende the comprehend, the birds like your mind to perhaps the not wind I should do?
  14. 14. MY CHAR-RNN EXPERIMENTS “i love you. Now her before it just numberse idevening with the press over. I was probably ever need to ever admit? Right” - Trump “life is an economy. I was in the LGBT communities can to the worst of the gun not only the fight are of us safe and I start up these are not grow…” - Hillary
  15. 15. FUTURE CHAR-RNN EXPERIMENTS train a model to talk like a person with little data? transfer learning? could we train a model off of a standard “human” model ? could we train a model to talk in different emotions/styles?
  16. 16. DEEP DREAMING / INCEPTION
  17. 17. A MACHINE LEARNING IMAGE CLASSIFIER
  18. 18. LAYERS LEARNED FEATURES architects: imagenet googlenet alexnet
  19. 19. GOOGLENET
  20. 20. LAYER AMPLIFICATION objective function: activate as many neurons in a layer key trick: push back to image feedback loop choose different layers for different effects: conv2/3x3,inception_3a,etc
  21. 21. TEST IMAGE
  22. 22. –Johnny Appleseed “Type a quote here.”
  23. 23. TRAINING MY OWN DREAMS
  24. 24. INCEPTION FUTURE EXPERIMENTS train with different image sets - sea life, reptiles? different objective function - activate only 1 group of neurons? selective regions of hallucinating? testing different network architects
  25. 25. NEURALSTYLE Paint images in the style of any painting
  26. 26. A NEURAL ALGORITHM OF ARTISTIC STYLE paper: http://arxiv.org/abs/1508.06576 The key finding of this paper is that the representations of content and style in the CNNs are separable. CNNs - convolutional Neural Network
  27. 27. high layers in the network act as the content of the image style computed from multiple layers’ filter responses
  28. 28. ] ?{;. /ΠK ;
  29. 29. NEURALSTYLE FUTURE EXPERIMENTS can we automatically find the “good” images from a combination? can we know beforehand if a combo style/content will look good? currently trained on vggnet data, what happens if we train it on a different data set, will the art look different? will a different architect make better art?
  30. 30. MULTIMODAL! STORYTELLING
  31. 31. I ACCIDENTALLY GAVETHE ANIMAL BACK OF MY HEAD , BREATHING DEEPLY .THERE WAS NO DOUBT IN HER EYES ,AND I COULDTELL BY THE LOOK ON HIS FACETHAT HE DID N'T APPROVE OF WHAT WAS HAPPENINGTO ME . IN FACT , IT MUST HAVE BEEN ONE OFTHOSE RARE OCCASIONS ,AS WELL AS A PET ANIMAL . HER SCENT FILLED THE AIR .THAT 'S WHAT SHE WAS LOOKING FOR ,AND NOW SHE HADTO STAY AWAKE LONG ENOUGHTO DIG UPTHE LEASH
  32. 32. SKIP-THOUGHTVECTORS sentence -> vectors
  33. 33. TRUMP STORYTELLER
  34. 34. FUTURE NEURAL STORY EXPERIMENTS train with different text a “seeing” Hellen Keller version train on different visual features
  35. 35. AND MANY OTHER EXPERIMENTS……. HOPEFULLY INSPIRING
  36. 36. DATA IS ESSENTIAL many of these models are built on public datasets always has been a problem; bigger problem for DL and general models very hard to get data; how can this be solved? constantly on my mind ; lets connect me if interested
  37. 37. DL IS NOT ALL FUN AND UNICORNS data issue specialized software/hardware pipelines; GPUs be prepared to wait; think weeks, not hours model tuning architect tuning techniques and architects changing everyday
  38. 38. WHY? I dream of building larger models AGI and multi modal models larger experiments want to collaborate with cool artists and coders fun? lets talk!
  39. 39. LINK APPENDIX
  40. 40. STUDY LINKS what is deep learning: http://www.jtoy.net/2016/02/14/opening- up-deep-learning-for-everyone.html generative models: https://en.wikipedia.org/wiki/ Generative_model discriminative models: https://en.wikipedia.org/wiki/ Discriminative_model
  41. 41. TEST LIVE MODEL LINKS trump char-rnn model: http://somatic.io/models/WZmmBjZ9 neural style model: http://www.somatic.io/models/5BkaqkMR neural talk model: http://somatic.io/models/qoEGanRe romance story telling: http://somatic.io/models/2n6g7RZQ
  42. 42. LINKS VGG net data used: http://www.robots.ox.ac.uk/~vgg/research/ very_deep/ tensorflow version: https://github.com/anishathalye/neural-style neural style paper: http://arxiv.org/abs/1508.06576 char-rnn code: https://github.com/somaticio/char-rnn-tensorflow mscoco: http://mscoco.org imagenet: http://image-net.org/
  43. 43. LINKS char-rnn: https://github.com/somaticio/char-rnn-tensorflow tensorflow char-rnn tutorial: https://www.tensorflow.org/ versions/r0.9/tutorials/seq2seq/index.html#recurrent-neural- networks neuralstyle: https://github.com/anishathalye/neural-style
  44. 44. –John Dewey “Every great advance in science has issued from a new audacity of imagination.” Jason Toy jason@somatic.io I write here: http://jtoy.net http://somatic.io/bog my models here: http://somatic.io @jtoy QUESTIONS?

×