Esin Yavuz from Cyanapse talks about style transfer, GANs and image data

1. Leveraging machine intelligence
to improve sensory experiences
Dr Esin Yavuz, Co-founder & Chief Scientist
MI Garage Meetup: “How can the creative industries use AI?”
7 November 2018

2. Augmenting sensory perception
with machine intelligence
Linking machine intelligence and human perception requires
real-time and automated image processing. To enable this, we
are building a platform that allows fast and automated
processing of images using AI.

3. Outline
●
Background & State of the art
●
AI for photorealistic image transformations
●
Automating image editing

4. What is AI?
●
“Science of making machines do things that would require intelligence if done by men.”
Minsky
●
Often used to refer to statistical methods that aim to fnd structure and regularities in data
●
Used for making deductions based on observations
●
Applied to problem solving, reasoning, natural language processing, machine perception

6. ●
2006: NVIDIA releases CUDA for general purpose GPU
computing
High-performance computing becomes easier to use
●
2010: ImageNet Large Scale Visual Recognition challenge
-Classifcation task based on a dataset of 10M images in
10K+ classes
Importance of number of examples and problem defnition
●
2012: Alexnet achieves top-5 error of 15.3%, more than 10.8
percentage points lower than that of the runner up on
ImageNet competition using a deep convolutional network
Beginning of the Deep Learning era
Breakthroughs in the last decade

7. https://distill.pub/2017/feature-visualization/

8. Deep Dream (Google) 2015
●
Initially designed as a tool to understand internal properties of neural networks
●
Whatever the network sees, it adjust the other pixels to see more of it
●
Allows creative image manipulations based on internal structure of a neural network

9. Neural Style Transfer – Gatys et al. 2016
Combining the
content of an image
with the style of
another image

10. https://deepart.io/

11. https://prisma-ai.com/

12. Image generation with Generative Adversarial Nets
(GAN) – Goodfellow et al. 2014

13. More recent GAN examples
Brock et al - BigGAN (Deepmind) 2018Karras et al. (NVIDIA) 2017
Evolving to be a new tool for artistic creation

16. Style Transfer with CycleGAN

17. Artistic style transfer with GAN-based methods
Drawbacks
●
While artistic style transfer can
be easily achieved, photorealistic
transformations are not easy
●
Mostly support low-res, if not
they need massive computational
power
●
Diffcult to generalise to different
content domains

18. Limitations of CycleGAN

19. Why do we need photorealistic transformations?
●
Good quality visual content is indispensable for a better
communication through visuals. However, creating good quality
visuals needs expertise and it is time consuming.
●
Existing tools for image flters provide synthetic results that look
similar, most are slow, do not scale for bulk image processing, and
still require substantial user expertise.
●
Apart from content creation, there is a need in other felds such as
data augmentation, video game development, rendering, enhancing
video streams.

20. Problems of applying AI to casual photography
Flickr/Mr Boss
Flickr/Yuma Hori
Flickr/Bex Walton

21. AI filers lhal work witlh any l pe of itmages
Transform any type of image
to refect different daytime
lighting conditions

22. AI-powered itmage filers

23. AI-powered itmage filers

24. AI-powered itmage filers
More examples here: cyanapse.com/ai-tech/photorealistic-image-flters

25. ●
Use AI-powered photorealistic flters
that work with any type of photos
●
Automate dull image editing tasks
●
Enable bulk image editing with
custom pipelines
Automated Image Editing Platform
1- Back-end API for fast and automated image processing
2- Front-end App for user-friendly interface

26. ●
RESTful API with intuitive
endpoints
●
Bulk processing that scales
to users’ needs
●
User-defned image editing
pipelines
●
Includes AI-enhanced
image styling flters
Automating image editing

27. Front-end
●
Mobile and web app to try out
AI flters
●
Use AI photo flters in just a
few clicks
●
Instantaneous generation of
various previews
●
Less than a second to
enhance 4k-res images, 40
ms for a low-res image

28. Advantages
●
4k resolution images are enhanced in less than a second
●
AI transformations run in the cloud (fexible and scalable)
●
We provide image enhancements that are photorealistic and that work for
every type of photos
●
Minimise the time spent on creating good quality content, allowing content
creators to focus on more important tasks
●
Enable real-time and photorealistic enhancements of video streams with
minimal human intervention
●
Generate multiple possible transformations to help content creators
preview the results beforehand, and to pass their messages better

29. Thank you!
Email: info@cyanapse.com
Register here:
cyanapse.com/beta-testing
@cyanapse