Lucas Theis - Compressing Images with Neural Networks - Creative AI meetup
•
0 likes•470 views
This talk by Lucas Theis from Twitter/Magic Pony on "Compressing Images with Neural Networks" was presented at the Learning Image Representations event on 30th August at Twitter as part of the Creative AI meetup.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Lucas Theis - Compressing Images with Neural Networks - Creative AI meetup
Similar to Lucas Theis - Compressing Images with Neural Networks - Creative AI meetup (20)
More from Luba Elliott
More from Luba Elliott (20)
Recently uploaded
Recently uploaded (20)
Lucas Theis - Compressing Images with Neural Networks - Creative AI meetup
- 1. Compressing Images with Neural Networks Lucas Theis @lucastheis 30 Aug 2017
- 2. Cisco VNI: Forecasts and Methodology, 2013-2018
- 3. Is compression still a problem? Demand for higher quality (4k, 60fps) New forms of media (VR, 360, stereo) Network congestion Emerging markets
- 4. Image compression A brief introduction to image compression Autoregressive models Lossless image compression with autoregressive models SRGANs Using GANs and super-resolution as a pragmatic approach to compression Compressive autoencoders End-to-end training of neural networks for compression
- 6. Entropy coding In information theory an entropy encoding is a lossless data compression scheme that is independent of the specific characteristics of the medium.
- 7. Entropy coding In information theory an entropy encoding is a lossless data compression scheme that is independent of the specific characteristics of the medium. …000011011100111101000000011… Optimal code:
- 8. Entropy coding In information theory an entropy encoding is a lossless data compression scheme that is independent of the specific characteristics of the medium. log2 P(n | encodi) bits
- 9. Autoregressive models 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 X ij log2 P(xij | x<ij) minimize
- 10. Recurrent image density estimator Pixels SLSTM units SLSTM units Pixels RIDE xij x<ij MCGSMB C he distribution of images such that the prediction of a pixel (black) he upper-left green region. (B) A graphical model representation of anTheis & Bethge, Generative Image Modeling Using Spatial LSTMs, 2015
- 11. Autoregressive models RIDE (3 layers) PixelRNN (12 layers) GAN Alec Radford (@AlecRad) van den Oord et al., 2016 Sohl-Dickstein et al., 2015 Deep Diffusion LAPGAN Denton et al., 2015 Deep Unsupervised Learning using Nonequilibrium (a) (b) Figure 3. The proposed framework trained on the CIFAR-10 [20] dataset. (a) Examp the diffusion model. will also be a Gaussian (binomial) distribution. The longer the trajectory the smaller the diffusion rate can be made. During learning only the mean and covariance for a Gaus- sian diffusion kernel, or the bit flip probability for a bi- nomial kernel, need be estimated. As shown in Table C.1, fµ x(t) , t and f⌃ x(t) , t are functions defining the mean and covariance of the reverse Markov transitions for a Gaussian, and fb x(t) , t is a function providing the bit flip probability for a binomial distribution. For all results in this paper, multi-layer perceptrons are used to define these functions. A wide range of regression or function fitting techniques would be applicable however, including nonpa- rameteric methods. then only a si to exactly ev substitution. process in sta 2.4. Training Training amo L = Z dx( = Z dx( log 2 4 CIFAR-10 Data Autoregressive
- 13. JPEG
- 14. Autoregressive models CIFAR-10 (van den Oord et al., 2016) Bit-rate[bit/px] 0 1.75 3.5 5.25 7 Gaussian NICE DeepDiffusion DeepGMM RIDE PixelCNN PixelRNN
- 15. Rate-distortion tradeoff DistortionNumber of bits Reconstruction
- 16. Rate-distortion tradeoff DistortionNumber of bits Reconstruction
- 21. Metrics PSNR: 28.6 SSIM: 0.84 PSNR: 17.9 SSIM: 0.24
- 22. Generative adversarial networks GAN“Compression” Kullback-Leibler divergence Maximum likelihood log2 Q✓(x) Data x1 x2 max D E log D(x) 1 D(g✓(z)) Theis et al., 2016
- 23. SRGAN log D(g✓(z)) + || (g✓(z)) (x)||2 Low-resolution image Super-resolution network Classifier Feature representation (VGG) Ledig et al., Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network, 2017
- 24. SRResNet (mean squared error) Original (high resolution)
- 27. Evaluation Ledig et al., 2017
- 29. Optimize a different autoencoder! [f(x)] ⇡ f(x) + u Uniform noise Ballé et al., 2016
- 30. Theis et al., 2017 x g([f(x)]) g(f(x) + u)
- 31. Bengio et al., 2013; Theis et al., 2017 Compressive autoencoder d dy [y] := d dy r(y) = 1
- 32. Compressive autoencoder d dy [y] := d dy r(y) x [f(x)] g([f(x)]) Bengio et al., 2013; Theis et al., 2017
- 33. Theis et al., 2017 Compressive autoencoder Q(z) = Z [ .5,.5[M q(z + u) du log2 Q(z) Z [ .5,.5[M log2 q(z + u) du
- 34. Theis et al., 2017 Compressive autoencoder log q([f(x)] + u) + · ||x g([f(x)]))||2
- 35. CAEToderici et al. (2016)JPEG 2000JPEG 4:4:4
- 36. CAEToderici et al. (2016)
- 37. CAEToderici et al. (2016)JPEG 2000
- 38. CAE
- 39. Evaluation
- 40. Evaluation
- 41. Rippel & Bourdev, 2017
- 42. Conclusion • End-to-end trained lossy image compression is now on par with the best hand-designed image compression algorithms (BPG) and quickly improving • Neural nets already offer many advantages: • Can be specialized to datasets • Can be adapted to other forms of content • Can be optimized for different metrics • Can be adapted for various settings (small/large encoder/decoder)
- 44. normalize mirror-pad conv conv conv conv sum conv conv sum conv input round GSM 128x5x5/2 64x5x5/2 128x3x3 128x3x3 128x3x3 128x3x3 96x5x5/2 subpix 512x3x3/2 conv conv 128x3x3 128x3x3 sumconv conv 128x3x3 128x3x3 sum subpix subpix 256x3x3/2 12x3x3/2 clip noise output code Model Decoder Encoder denormalize Compressive autoencoder
- 45. Evaluation