Jason joined 2018 PIXNET Hackathon Food-AI 智慧影像生成 https://github.com/pixnet/2018-pixnet-hackathon/tree/master/food-ai

1. Copyright © 2018 by AI4quant [TW] All rights reserved
AI4quant: AI智慧影像生成
PIXNET Hackathon 2018
Jason Chuang、呂沂樺、Kiki
2018-08-19

2. Copyright © 2018 by AI4quant [TW] All rights reserved
What is AI?
Big Data
Algorithm
Computing Power

3. Copyright © 2018 by AI4quant [TW] All rights reserved
Data preprocessing for training data
We use 5 different datasets for training
If the shortest width of the image is less than 256 pixels, ignore
Shrink the shortest width to 256 pixels, cut the center part of the image

4. Copyright © 2018 by AI4quant [TW] All rights reserved
Big data
Food-11: https://mmspg.epfl.ch/food-image-datasets
DataSet Size(jpg,
png)
Number of
Files
# Training
files
Training
size(bmp,
256x256)
# Validation
files
dataset100 1GB 14,467 10,044 1.83GB 0
dataset256 4GB 31,653 27,002 5GB 0
Food-11 1.1GB 16,643 16,635 3.04GB 0
food-101 4.76GB 101,008 100,956 18.4GB 0
pixfood20 2.03GB 31,305 19,322 3.53GB 19,322

5. Copyright © 2018 by AI4quant [TW] All rights reserved
Algorithms are motivated by research paper
DCNN based method
Image Inpainting for Irregular Holes Using Partial Convolutions, 2018, Guilin Liu,
Fitsum A. Reda, Kevin J. Shih, Ting-Chun Wang, Andrew Tao, Bryan Catanzaro
High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis, 2017,
Chao Yang, Xin Lu,Zhe Lin, Eli Shechtman, Oliver Wang, and Hao Li
Style Transfer (Harmonization)
A Neural Algorithm of Artistic Style, 2015, Leon A. Gatys, Alexander S. Ecker,
Matthias Bethge

6. Copyright © 2018 by AI4quant [TW] All rights reserved
Proposed model#1 - PConv
Hole mask: random shape

7. Copyright © 2018 by AI4quant [TW] All rights reserved
U-net like architecture(encoder-decoder)

8. Copyright © 2018 by AI4quant [TW] All rights reserved
PConv:During prediction-Erosion

9. Copyright © 2018 by AI4quant [TW] All rights reserved
Proposed model#2: Content network- central square
Model : pre-trained imagenet_inpaintCenter.t7

10. Copyright © 2018 by AI4quant [TW] All rights reserved
Final image: Image blending
g(x)=(1-ß)*f1(x)+ß*f2(x)
ß is (0,1)

11. Copyright © 2018 by AI4quant [TW] All rights reserved
Code library and computation platform
● Tensorflow 1.9.0
● Keras 2.2.2
● OpenCV 3.4.0
● Torch
Training(Oregon, U.S.) Model Evaluation/Competition (GCP in Taiwan)
● AWS p3.2xlarge
● Nvidia V100 GPU
● 8vCPU
● 61GB memory
● GCP Google Compute Engine
● Nvidia K80 GPU
● 4 vCPU
● 26 GB memory

12. Copyright © 2018 by AI4quant [TW] All rights reserved
Quantitative comparison(validation data loss)

13. Copyright © 2018 by AI4quant [TW] All rights reserved
Qualitative comparison

14. Copyright © 2018 by AI4quant [TW] All rights reserved
During prediction - Deep style transfer(harmonization)
(Thinking to use)
Iter 0 Iter 5 Iter 10