Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
#denatechcon
#denatechcon
#denatechcon
#denatechcon
#denatechcon
00./ 0
#denatechcon
00./ 0
#denatechcon
#denatechcon
#denatechcon
// . /
#denatechcon
// . /
#denatechcon
#denatechcon
#denatechcon
#denatechcon
#denatechcon
#denatechcon
AGENDA
#denatechcon
(@hamadakoichi)
Mobage
2010 6
Mobage2011 4
DeNA
DeNA2014 10
: ( )
TokyoWebmining
- 1500
- 2010 60
DeNA 8 AI A...
#denatechcon
50 /
#denatechcon
#denatechcon , , , , , .
“ ”. 232 (7/20/2017).
0
3
#denatechcon
5 4 /0 /
5 4 /0 : 6 / .
#denatechcon
Full-body High-resolution Anime Generation with
Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
AGENDA
#denatechcon
AGENDA
#denatechcon
#denatechcon
1 3 5 7
2 4 6 8
#denatechcon
1 3 5 7
2 4 6 8
#denatechcon
ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
1 3 5 7
2 4 6 8
#denatechcon
ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
#denatechcon
ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
#denatechcon
ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
#denatechcon
#denatechcon
Generative Adversarial Nets.
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-
Farley...
#denatechcon
#denatechcon
ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
#denatechcon
ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
#denatechcon
Progressive Growing of GANs for Improved Quality, Stability, and Variation.
Tero Karras, Timo Aila, Samuli La...
#denatechcon
.441 7 545 7 4 /
Progressive Growing of GANs for Improved Quality, Stability, and Variation
Tero Karras, Timo...
#denatechcon
/ 5. 44 5
Progressive Growing of GANs for Improved Quality, Stability, and Variation
Tero Karras, Timo Aila, ...
#denatechcon
ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
#denatechcon
ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
#denatechcon
+ Spectral Normalization on Generator
+ Self Attention
+ Two Time Scale Update Rule
(512x512)
+ Spectral Norm...
#denatechcon
(512x512)
Generator
Typical Architecture
Res Block
Architecture for ImageNet at 512x512
Large Scale GAN Train...
#denatechcon
(512x512)
Large Scale GAN Training for High Fidelity Natural Image Synthesis.
Andrew Brock, Jeff Donahue, Kar...
#denatechcon
Large Scale GAN Training for High Fidelity Natural Image Synthesis.
Andrew Brock, Jeff Donahue, Karen Simonya...
#denatechcon
Large Scale GAN Training for High Fidelity Natural Image Synthesis.
Andrew Brock, Jeff Donahue, Karen Simonya...
#denatechcon
Large Scale GAN Training for High Fidelity Natural Image Synthesis.
Andrew Brock, Jeff Donahue, Karen Simonya...
#denatechcon
(512x512)
Large Scale GAN Training for High Fidelity Natural Image Synthesis.
Andrew Brock, Jeff Donahue, Kar...
#denatechcon
(512x512)
Large Scale GAN Training for High Fidelity Natural Image Synthesis.
Andrew Brock, Jeff Donahue, Kar...
#denatechcon
Large Scale GAN Training for High Fidelity Natural Image Synthesis.
Andrew Brock, Jeff Donahue, Karen Simonya...
#denatechcon
ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
#denatechcon
ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
#denatechcon
ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
#denatechcon
ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
#denatechcon
#denatechcon
#denatechcon
#denatechcon
#denatechcon
#denatechcon
#denatechcon
#denatechcon
#denatechcon
#denatechcon
AGENDA
#denatechcon
ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
#denatechcon
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Netw...
#denatechcon
#denatechcon
#denatechcon
AGENDA
#denatechcon
n e
n 4 3 33 36 4
n 0231 33 36 4
n c
b a
)3 (
0 06I J@ N LT A F
( D T
#denatechcon
n
n
n
#denatechcon
n
n
#denatechcon
n C
n 0 1 .
n 0:: /1:0 6 ..:. 21 )()
#denatechcon
n CEAD A
n ) 17:
n 7 2 -( ) 17: 0/ 1 7 0 7 . :0. 0 7 1 7 :
#denatechcon
AGENDA
#denatechcon
#denatechcon
: ( . ) / .
#denatechcon
•
•
•
•
: ( . ) / .
#denatechcon
#denatechcon
O .10 2:: ( / /:7 10
* .) A L A
#denatechcon
O .10 2:: ( / /:7 10
* .) A L A
#denatechcon
•
O .10 2:: ( / /:7 10
* .) A L A
#denatechcon
•
•
O .10 2:: ( / /:7 10
* .) A L A
#denatechcon
O .10 2:: ( / /:7 10
* .) A L A
•
•
#denatechcon
#denatechcon
) ) 3
#denatechcon
) ) 3
#denatechcon
#denatechcon
#denatechcon
#denatechcon
#denatechcon
8 1 1
1
Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation.
Huaizu ...
#denatechcon
/30 480 6 2/81 4C
+ 60 2
• 8 ,
Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video...
#denatechcon
7
1
Video Frame Synthesis using Deep Voxel Flow.
Ziwei Liu, Raymond A. Yeh, Xiaoou Tang, Yiming Liu, Aseem Ag...
#denatechcon
N I 7 B7 =: B = P
77 = 7: :=D
• /0 (+ /0 ,
Video Frame Synthesis using Deep Voxel Flow.
Ziwei Liu, Raymond A....
#denatechcon
D
F 6 + 23C
• 1 76 , P SV P J IOM S R
• ( ,24 c SV P J ,24 cP
/ ++ C
• 1 76 , P J SV P
• 4 8 4 0 L a
Super Sl...
#denatechcon
2
#denatechcon
#denatechcon
F
8 0 O T PL
n S 4 )
n F 4 (
W4000301 32 ( 6 A 7
#denatechcon
F F F 0 -1 12 W
n
n S
L ***7* 7 A A
OT 0 -1 12
#denatechcon
#denatechcon
#denatechcon
#denatechcon
n
n
#denatechcon
n
n
n
#denatechcon
n
n
n
n F
#denatechcon
n
n
n
n F
#denatechcon
G
I0 I4
Optical Flow
I1, I2, I3
(Generated)
#denatechcon
G
I0 I4
D
Optical Flow
I1, I2, I3
(Generated)
Ground Truth
#denatechcon
G
I0 I4
D
Optical Flow
I1, I2, I3
(Generated)
Ground Truth
MSE
#denatechcon
G
I0 I4
D
Optical Flow
Ground Truth
I1, I2, I3
(Generated)
Ground Truth
Generated
MSE
MSE
#denatechcon
G
I0 I4
D
Optical Flow
Ground Truth
I1, I2, I3
(Generated)
Ground Truth
Generated
MSE
MSE
#denatechcon
2
#denatechcon
Conv-BN-ReLU
Conv-BN-ReLU
Conv-BN-ReLU
Conv-BN-ReLU
Local Discriminator
“Real” or “Fake”
Local Patch
(16×16pi...
#denatechcon
Conv-BN-ReLU
Conv-BN-ReLU
Generated Image
Sequense
Conv-BN-ReLU
Conv-BN-ReLU
Conv-BN-ReLU
Conv-BN-ReLU
Conv-B...
#denatechcon
#denatechcon
Video
#denatechcon
image0 image1 image2 image3 image4Video
#denatechcon
⁃ step size = 4 7FPS -> 30FPS 001.png, 005.png, 009.png, 013.png, 017.png
⁃ step size = 1 30FPS -> 120FPS 001...
#denatechcon
⁃ step size = 4 7FPS -> 30FPS 001.png, 005.png, 009.png, 013.png, 017.png
⁃ step size = 1 30FPS -> 120FPS 001...
#denatechcon
#denatechcon
Frame
Frame
Deep Voxel FlowInput
// . /
Experimental Results: “Anime Frame Generation with Structure-consiste...
#denatechcon
step size = 1 step size = 4 step size = 7 step size = 10
Input
SPGAN
(Ours)
Deep Voxel Flow
4
// . /
Experime...
#denatechcon
Deep Voxel Flow Ours
1.average PSNR/SSIM on test dataset step size=4
PSNR SSIM
Deep Voxel Flow 23.32 0.9294
S...
#denatechcon
AGENDA
#denatechcon
ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
#denatechcon
ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
#denatechcon
ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
#denatechcon
00./ 0
#denatechcon
00./ 0
#denatechcon
// . /
#denatechcon
// . /
#denatechcon
#denatechcon * 10 2 1
#denatechcon
#denatechcon
L0*** * 2 D 1 A 7
1 D 1 7
#denatechcon
TZ ... ./ 0 KD SL KA N O
N KD SL K N W
(3 Frame) AI (x16 )
Input Frames Generated Frames
78 : : 102*0DeNA AI :
#denatechcon
Upcoming SlideShare
Loading in …5
×

AIによるアニメ生成の挑戦

45,133 views

Published on

「AIによるアニメ生成の挑戦」。DeNAの「構造的生成学習」技術開発により、従来AI生成で難しかった “複雑な構造での高品質生成”、"構造変化が大きい画像間の補完生成・中割” 等の課題を解消し、多様なキャラクター全体でのアニメ生成・中割生成、を実現しています。DeNA TechCon 2019での DeNA "アニメ生成プロジェクト" の活動紹介、発表資料です。

[AIによる生成アニメ例]
https://www.youtube.com/watch?v=tOZW_KWb8b0

---
"AIによるアニメ生成の挑戦".
濱田晃一 , 李天琦.
DeNA TechCon 2019.
---
"Challenges toward Anime Generation with Deep Generative Models".
Koichi Hamada and Tianqi Li.
In DeNA Technology Conference 2019.

Published in: Engineering
  • Be the first to comment

AIによるアニメ生成の挑戦

  1. 1. #denatechcon #denatechcon
  2. 2. #denatechcon
  3. 3. #denatechcon
  4. 4. #denatechcon 00./ 0
  5. 5. #denatechcon 00./ 0
  6. 6. #denatechcon
  7. 7. #denatechcon
  8. 8. #denatechcon // . /
  9. 9. #denatechcon // . /
  10. 10. #denatechcon
  11. 11. #denatechcon
  12. 12. #denatechcon
  13. 13. #denatechcon
  14. 14. #denatechcon
  15. 15. #denatechcon AGENDA
  16. 16. #denatechcon (@hamadakoichi) Mobage 2010 6 Mobage2011 4 DeNA DeNA2014 10 : ( ) TokyoWebmining - 1500 - 2010 60 DeNA 8 AI AI AI
  17. 17. #denatechcon 50 /
  18. 18. #denatechcon
  19. 19. #denatechcon , , , , , . “ ”. 232 (7/20/2017). 0 3
  20. 20. #denatechcon 5 4 /0 / 5 4 /0 : 6 / .
  21. 21. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCV Workshop 2018. (ECCV: European Conference on Computer Vision)
  22. 22. #denatechcon AGENDA
  23. 23. #denatechcon AGENDA
  24. 24. #denatechcon
  25. 25. #denatechcon 1 3 5 7 2 4 6 8
  26. 26. #denatechcon 1 3 5 7 2 4 6 8
  27. 27. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019) 1 3 5 7 2 4 6 8
  28. 28. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  29. 29. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  30. 30. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  31. 31. #denatechcon
  32. 32. #denatechcon Generative Adversarial Nets. Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde- Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio. arXiv:1406.2661. In NIPS 2014.
  33. 33. #denatechcon
  34. 34. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  35. 35. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  36. 36. #denatechcon Progressive Growing of GANs for Improved Quality, Stability, and Variation. Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen. In ICLR 2018. (1024X1024) (256x256)
  37. 37. #denatechcon .441 7 545 7 4 / Progressive Growing of GANs for Improved Quality, Stability, and Variation Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen. In ICLR 2018.
  38. 38. #denatechcon / 5. 44 5 Progressive Growing of GANs for Improved Quality, Stability, and Variation Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen. In ICLR 2018.
  39. 39. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  40. 40. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  41. 41. #denatechcon + Spectral Normalization on Generator + Self Attention + Two Time Scale Update Rule (512x512) + Spectral Normalization on Discriminator + Projection Discriminator SNGAN with Projection (Miyato+, ICLR’18) SAGAN (Zhang+, 18) BigGAN (Brock+, ICLR’19) + Large Batch Size (256→2048) + Large Channel (64→96) + Shared Embedding + Hierarchical Latent Space + Truncation Trick + Orthogonal Regularization + First Singular Value Clamp + Zero-centered Gradient Penalty Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2018.
  42. 42. #denatechcon (512x512) Generator Typical Architecture Res Block Architecture for ImageNet at 512x512 Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019.
  43. 43. #denatechcon (512x512) Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019.
  44. 44. #denatechcon Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019. (512x512)
  45. 45. #denatechcon Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019. (512x512)
  46. 46. #denatechcon Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019. (512x512)
  47. 47. #denatechcon (512x512) Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019.
  48. 48. #denatechcon (512x512) Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019.
  49. 49. #denatechcon Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019. (512x512)
  50. 50. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  51. 51. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  52. 52. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  53. 53. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  54. 54. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0
  55. 55. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0
  56. 56. #denatechcon
  57. 57. #denatechcon
  58. 58. #denatechcon
  59. 59. #denatechcon
  60. 60. #denatechcon
  61. 61. #denatechcon
  62. 62. #denatechcon
  63. 63. #denatechcon
  64. 64. #denatechcon
  65. 65. #denatechcon
  66. 66. #denatechcon AGENDA
  67. 67. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  68. 68. #denatechcon
  69. 69. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0
  70. 70. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0
  71. 71. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 0 0 0 0 0 0 0 0 0 0 0
  72. 72. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018.
  73. 73. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018.
  74. 74. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 0 0 0 0 0 0 0 0 0 0 0
  75. 75. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018.
  76. 76. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018.
  77. 77. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018.
  78. 78. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0
  79. 79. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0
  80. 80. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0 Full-body anime generation at 1024x1024 with Progressive Structure-conditional GANs
  81. 81. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0
  82. 82. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. // . 0/0 Adding action to full-body anime characters with Progressive Structure-conditional GANs
  83. 83. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018.
  84. 84. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. (ICLR’18)
  85. 85. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. (ICLR’18)
  86. 86. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. (ICLR’18) (NIPS’17) (NIPS’17)
  87. 87. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. (ICLR’18) (NIPS’17) (NIPS’17)
  88. 88. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018.
  89. 89. #denatechcon
  90. 90. #denatechcon
  91. 91. #denatechcon AGENDA
  92. 92. #denatechcon n e n 4 3 33 36 4 n 0231 33 36 4 n c b a )3 ( 0 06I J@ N LT A F ( D T
  93. 93. #denatechcon n n n
  94. 94. #denatechcon n n
  95. 95. #denatechcon n C n 0 1 . n 0:: /1:0 6 ..:. 21 )()
  96. 96. #denatechcon n CEAD A n ) 17: n 7 2 -( ) 17: 0/ 1 7 0 7 . :0. 0 7 1 7 :
  97. 97. #denatechcon AGENDA
  98. 98. #denatechcon
  99. 99. #denatechcon : ( . ) / .
  100. 100. #denatechcon • • • • : ( . ) / .
  101. 101. #denatechcon
  102. 102. #denatechcon O .10 2:: ( / /:7 10 * .) A L A
  103. 103. #denatechcon O .10 2:: ( / /:7 10 * .) A L A
  104. 104. #denatechcon • O .10 2:: ( / /:7 10 * .) A L A
  105. 105. #denatechcon • • O .10 2:: ( / /:7 10 * .) A L A
  106. 106. #denatechcon O .10 2:: ( / /:7 10 * .) A L A • •
  107. 107. #denatechcon
  108. 108. #denatechcon ) ) 3
  109. 109. #denatechcon ) ) 3
  110. 110. #denatechcon
  111. 111. #denatechcon
  112. 112. #denatechcon
  113. 113. #denatechcon
  114. 114. #denatechcon 8 1 1 1 Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation. Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, Jan Kautz. In CVPR 2018.
  115. 115. #denatechcon /30 480 6 2/81 4C + 60 2 • 8 , Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation. Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, Jan Kautz. In CVPR 2018. https://youtu.be/MjViy6kyiqs Research at NVIDIA: Transforming Standard Video Into Slow Motion with AI
  116. 116. #denatechcon 7 1 Video Frame Synthesis using Deep Voxel Flow. Ziwei Liu, Raymond A. Yeh, Xiaoou Tang, Yiming Liu, Aseem Agarwala. In ICCV 2017.
  117. 117. #denatechcon N I 7 B7 =: B = P 77 = 7: :=D • /0 (+ /0 , Video Frame Synthesis using Deep Voxel Flow. Ziwei Liu, Raymond A. Yeh, Xiaoou Tang, Yiming Liu, Aseem Agarwala. In ICCV 2017. BB F=CBC 67 ?. / 3: 1 B Video Frame Synthesis using Deep Voxel Flow
  118. 118. #denatechcon D F 6 + 23C • 1 76 , P SV P J IOM S R • ( ,24 c SV P J ,24 cP / ++ C • 1 76 , P J SV P • 4 8 4 0 L a Super SloMo(Adobe) Super SloMo Deep Voxel Flow Video Frame Synthesis using Deep Voxel Flow. Ziwei Liu, Raymond A. Yeh, Xiaoou Tang, Yiming Liu, Aseem Agarwala. In ICCV 2017. Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation. Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, Jan Kautz. In CVPR 2018. F
  119. 119. #denatechcon 2
  120. 120. #denatechcon
  121. 121. #denatechcon F 8 0 O T PL n S 4 ) n F 4 ( W4000301 32 ( 6 A 7
  122. 122. #denatechcon F F F 0 -1 12 W n n S L ***7* 7 A A OT 0 -1 12
  123. 123. #denatechcon
  124. 124. #denatechcon
  125. 125. #denatechcon
  126. 126. #denatechcon n n
  127. 127. #denatechcon n n n
  128. 128. #denatechcon n n n n F
  129. 129. #denatechcon n n n n F
  130. 130. #denatechcon G I0 I4 Optical Flow I1, I2, I3 (Generated)
  131. 131. #denatechcon G I0 I4 D Optical Flow I1, I2, I3 (Generated) Ground Truth
  132. 132. #denatechcon G I0 I4 D Optical Flow I1, I2, I3 (Generated) Ground Truth MSE
  133. 133. #denatechcon G I0 I4 D Optical Flow Ground Truth I1, I2, I3 (Generated) Ground Truth Generated MSE MSE
  134. 134. #denatechcon G I0 I4 D Optical Flow Ground Truth I1, I2, I3 (Generated) Ground Truth Generated MSE MSE
  135. 135. #denatechcon 2
  136. 136. #denatechcon Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Local Discriminator “Real” or “Fake” Local Patch (16×16pix)
  137. 137. #denatechcon Conv-BN-ReLU Conv-BN-ReLU Generated Image Sequense Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU FC Local Discriminator Temporal Discriminator “Real” or “Fake” Local Patch (16×16pix) Image Sequense “Real” or “Fake”
  138. 138. #denatechcon
  139. 139. #denatechcon Video
  140. 140. #denatechcon image0 image1 image2 image3 image4Video
  141. 141. #denatechcon ⁃ step size = 4 7FPS -> 30FPS 001.png, 005.png, 009.png, 013.png, 017.png ⁃ step size = 1 30FPS -> 120FPS 001.png, 002.png, 003.png, 004.png, 005.png
  142. 142. #denatechcon ⁃ step size = 4 7FPS -> 30FPS 001.png, 005.png, 009.png, 013.png, 017.png ⁃ step size = 1 30FPS -> 120FPS 001.png, 002.png, 003.png, 004.png, 005.png
  143. 143. #denatechcon
  144. 144. #denatechcon Frame Frame Deep Voxel FlowInput // . / Experimental Results: “Anime Frame Generation with Structure-consistent Prediction GANs”
  145. 145. #denatechcon step size = 1 step size = 4 step size = 7 step size = 10 Input SPGAN (Ours) Deep Voxel Flow 4 // . / Experimental Results: “Anime Frame Generation with Structure-consistent Prediction GANs”
  146. 146. #denatechcon Deep Voxel Flow Ours 1.average PSNR/SSIM on test dataset step size=4 PSNR SSIM Deep Voxel Flow 23.32 0.9294 SPGAN(Ours) 24.27 0.9407
  147. 147. #denatechcon AGENDA
  148. 148. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  149. 149. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  150. 150. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  151. 151. #denatechcon 00./ 0
  152. 152. #denatechcon 00./ 0
  153. 153. #denatechcon // . /
  154. 154. #denatechcon // . /
  155. 155. #denatechcon
  156. 156. #denatechcon * 10 2 1
  157. 157. #denatechcon
  158. 158. #denatechcon L0*** * 2 D 1 A 7 1 D 1 7
  159. 159. #denatechcon TZ ... ./ 0 KD SL KA N O N KD SL K N W (3 Frame) AI (x16 ) Input Frames Generated Frames 78 : : 102*0DeNA AI :
  160. 160. #denatechcon

×