Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deep Learning and Fontli

770 views

Published on

Deep learning experiment with Fontli, using the layers of a deep convolution neural network (DCNN) and typography.

Published in: Technology
  • Be the first to comment

Deep Learning and Fontli

  1. 1. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. DEEP LEARNING & FONTLI
  2. 2. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. I would like to understand things better, but I don’t want to understand them perfectly. - Douglas Hofstadter
  3. 3. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. In Fluid Concepts and Creative Analogies, Douglas Hofstadter gives centre stage to analogy making as a foundation for intelligence Hofstadter talks about derivative projects – one of which (“Letter Spirit”) is about analogy making in the typography space : the question being “if <word1> : <insert-stylized-word1> then <other-letters> : ?”. In a sentence, if I show the system a particular stylized presentation of the letters “abcd”, how do you get it to construct other letters such as “wxyz” in the same style.
  4. 4. Deep learning experiment with Fontli
  5. 5. Fontli is a social network for typoholics. Capture typography around you and share it. Let the typetalk develop on Fontli.
  6. 6. OUR SEARCH FOR ANSWER … Is it possible to take a image style and apply to another image?
  7. 7. So we tried it with a bunch of images largely in three classes – type, patterns and semi-realistic pictures – with some surprisingly good and surprisingly crappy results. TYPE PATTERNS SEMI-REALISTIC
  8. 8. We wanted to try what “taking the style from an image” would do to a dead simple bland piece of text in black and white – the “content”, shown below. The image which was passed to neural- style as the “content” image in all the cases.
  9. 9. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. DEEP CONVOLUTION NEURAL NETWORK (DCNN) The layers of a deep convolution neural network (DCNN) that are closer to the input tend to look at spatially local features The layers deeper down tend to be concerned with higher level and even conceptual features
  10. 10. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. The images were rendered using the 19-layer network which is the default in the neural-style tool The amount of content has been kept small in these images deliberately – the proportion varying between 5:1000 and 50:1000 Each image took about 10mins to generate on a g2.2xlarge instance on AWS DCNN + FONTLI
  11. 11. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. THE RESULTS … A top favourite. The swirly strokes of the pattern have been beautifully woven to form the letters amidst the chaos of flowy colours.
  12. 12. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. THE RESULTS … The tree pattern has lots of whirlies all over, which appears to be the reason the whirlies have been used to express the letter shaping. Note that the “n” appears in bold and somewhat like the trunk of the tree.
  13. 13. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. THE RESULTS … The colours and texture of the mat pieces in the style image have been used pretty effectively to create the letters. Again, the “content” is set to low in this case. We’ve also lost some of the regularity of the whole pieces in the style image, which we might consider to be the “content” of the style image.
  14. 14. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. THE RESULTS … The colour palette and the texture of the colouring has transposed nicely on to the result image as expected.
  15. 15. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. THE RESULTS … This is interesting because the vertical strokes of the tree bark have been preserved more or less intact, while locally altering the lines to reveal the letters. The notches in the style image have been used to make the characters. See how the lines converge towards the dot of the “i”.
  16. 16. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. THE RESULTS … One of those exceptional cases where something interesting came out of using a typography image as the style. That said, I had no idea such an image would result – i.e. the result in unpredictable.
  17. 17. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. THE RESULTS … Another favourite. The strokes of the “kolam” pattern and the colours have been used beautifully to shape the letters.
  18. 18. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. THE RESULTS … The thorny pattern seems to have been used as the “pencil” to draw out the letters this time.
  19. 19. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. THE HEAD-SCRATCHERS … Realistic photographs did absolutely nothing. Also, since the ImageNet training set doesn’t attempt to teach typography to the neural net, we also didn’t have much success trying to use the letter styles from an image and apply it to new text. The results were mixed, with decent results some times, but head-scratchers at other times.
  20. 20. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. THE HEAD-SCRATCHERS … Using real-ish photographs does absolutely nothing artistically interesting to the text! This turned out to be the crappiest of what we tried :)
  21. 21. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. THE HEAD-SCRATCHERS … We thought the sketchiness would transfer over to the lettering, but that didn’t happen. As noted, this area is a bit unpredictable. Maybe the sketching is quite far away from what we might find in photographs.
  22. 22. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. LESSONS LEARNT What we’ve seen in this quick experiment is that the net trained on real world photos does not quite have the ability to read typography. This isn’t surprising since typography neither features strongly in the collection nor have such images been labelled well. That said, we can indeed make some very interesting images that present lettering and we hope to include this functionality in Fontli.
  23. 23. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. Are you looking to solve complex problems using Deep Learning? Explore Deep Learning with Imaginea… GET IN TOUCH WITH US >>
  24. 24. Pramati’s M&A’s of Leading products Serving from 5 Global Locations Over 200 Product Companies Unique Products & Services Agile Methodology User-centric design Open Source Contributions Products built from conception-code-cash Imaginea: Services Company with Product DNA Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.
  25. 25. Disclaimer This document may contain forward-looking statements concerning products and strategies. These statements are based on management's current expectations and actual results may differ materially from those projected, as a result of certain risks, uncertainties and assumptions, including but not limited to: the growth of the markets addressed by our products and our customers' products, the demand for and market acceptance of our products; our ability to successfully compete in the markets in which we do business; our ability to successfully address the cost structure of our offerings; the ability to develop and implement new technologies and to obtain protection for the related intellectual property; and our ability to realize financial and strategic benefits of past and future transactions. These forward-looking statements are made only as of the date indicated, and the company disclaims any obligation to update or revise the information contained in any forward-looking statements, whether as a result of new information, future events or otherwise. All Trademarks and other registered marks belong to their respective owners. Copyright © 2012-2015, Imaginea Technologies, Inc. and/or its affiliates. All rights reserved. Credits Images under Creative Commons Zero license. Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

×