Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Style transfer for generation of realistically textured elastic subsurface models

170 views

Published on

We apply the neural style transfer technique (Gatys et al., 2015) to enrich a simplified prior content distribution with features extracted from a geological reference. We demonstrate an application of this approach by transferring the layered texture from the Marmousi II benchmark model to a distribution generated by random Gaussian field. This work offers another way for the generation of target-specific pseudo-random subsurface models.

Published in: Science

Style transfer for generation of realistically textured elastic subsurface models

  1. 1. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.sa O. Ovcharenko, V. Kazei, D. Peter, T. Alkhalifah Style transfer for generation of realistically textured subsurface models Sep 18th, 2019 San Antonio, TX
  2. 2. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models Team 2 Vladimir Kazei, Post-doctoral Fellow Oleg Ovcharenko, PhD student Tariq Alkhalifah, Professor Daniel Peter, Assistant Professor KAUST Saudi Arabia
  3. 3. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models Supervised deep learning 3 Subsurface model Seismic data Inverted model
  4. 4. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models Community random model generators 4 Deep-learning tomography Araya-Polo et al., 2017 Deep-learning inversion: a next generation seismic velocity-model building method Yang and Ma, 2019 Deep learning Inversion of Seismic Data Li et al, 2019 Generative Adversarial Networks for Model Order Reduction in Seismic Full-Waveform Inversion, Richardson, 2018 Velocity model building from raw shot gathers using machine learning Øye and Dahl, 2019 Stochastic Seismic Waveform Inversion using Generative Adversarial Networks as a Geological Prior Mosser et al, 2018
  5. 5. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models Backstory — Bandwidth extrapolation 5 “Deep learning for low-frequency extrapolation from multi-offset seismic data”, Ovcharenko et al., 2019. GEOPHYSICS
  6. 6. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 6Backstory — Random subsurface models
  7. 7. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models Backstory — Velocity model building 7 “Mapping seismic data cubes to vertical velocity profiles by deep learning: New full-waveform inversion paradigm?”, Kazei et al., 2019 submitted to GEOPHYSICS https://github.com/vkazei/deeplogs
  8. 8. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models Backstory — Elastic transform 8 “Mapping seismic data cubes to vertical velocity profiles by deep learning: New full-waveform inversion paradigm?”, Kazei et al., 2019 submitted to GEOPHYSICS https://github.com/vkazei/deeplogs
  9. 9. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models Textures –> wavelet packets 9 Kazei, V., et al. "Realistically Textured Random Velocity Models for Deep Learning Applications”, EAGE 2019
  10. 10. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models Neural style-transfer 10 Gatys, L.A., Ecker, A.S. and Bethge, M., 2015. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576. Neckarfront in Tubingen, Germany Der Schrei by Edvard MunchThe Starry Night by Vincent van Gogh The Shipwreck of the Minotaur by J.M.W. Turner
  11. 11. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models Color channels RGB = elastic isotropic parameters 11 RGB = Vp, Vs, Rho
  12. 12. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models Application to elastic models 12 + = Content Style Generated
  13. 13. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 13 Workflow Parametrization, losses, optimization
  14. 14. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 14 3. Extract feature maps Flowchart 2. Propagate through the network 4. Compute loss 5. Compute gradients ∂L ∂m 1. Init prior L = ws Ls + wc Lc + wTV LTV 6. Update
  15. 15. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 15Feature extractor — VGG16 (Simonyan and Zisserman, 2014) Going deeper Input image 64 128 256 512 512
  16. 16. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 16Content loss - 2 2 Lc =
  17. 17. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 17Style loss Texture representation builds from multiple scales Going deeper
  18. 18. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 18Gram matrix f h w h * w f f h*w f f * = G f f
  19. 19. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 19Style loss - 2 2 Ls = Gs G
  20. 20. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 20Objective function Total ContentStyle Smoothing
  21. 21. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 21 25% 50% 100% 200% 1000% Contentcontribution Content/style weight ratio
  22. 22. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 22 + Marmousi II“Salt” TensorFlow 1.12.0Python 3.6 Keras 2.2.4 Titan V Computational aspects Size: 100 x 300 x 3 L-BFGS: 100 iterations ~ 1 sec / iter Future: fast style transfer by GAN following (Johnson et al., 2016; Ulyanov et al., 2016) Demo notebook available at https://github.com/ovcharenkoo/geo-style-keras
  23. 23. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 23 +
  24. 24. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 24 +
  25. 25. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 25 +
  26. 26. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 26 +
  27. 27. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 27 +
  28. 28. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 28 + Target contrasts preserved, but need more smoothing
  29. 29. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 29 + Less sharp contrasts, more consistency
  30. 30. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 30 Would it cause strong variability? Starting optimization from random noise?
  31. 31. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 31A few samples + Sample 1/3
  32. 32. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 32 + Sample 2/3 A few samples
  33. 33. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 33 + Sample 3/3 A few samples
  34. 34. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 34 + Mean Depends on loss weights
  35. 35. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 35 + Standard deviation Depends on loss weights
  36. 36. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 36 Well-log constraints and wave propagation L2 loss for given well-log locations
  37. 37. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 37More constraints L = ws Ls + wc Lc + wTV LTV + wlog Llog Balance of smoothing and well-log penalties
  38. 38. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 38 + Well-log constraint - OFF MarmousiRandom Gaussian Field
  39. 39. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 39 + MarmousiRandom Gaussian Field Well-log constraint - ON
  40. 40. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 40 + MarmousiRandom Gaussian Field Well-log constraint - ON
  41. 41. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models Wave propagation 41 https://github.com/ar4/deepwave 3 km 10 km vmax = 3500 m/s vmin = 1500 m/s fc = 10 Hz t = 3 s
  42. 42. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models 42Test on benchmark models Layered style prior leads to diverse outputs
  43. 43. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models Test on benchmark models 43 Gradient or homogeneous content priors produce visually-plausible models
  44. 44. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models Acknowledgements 44 ovcharenkoo.com vkazei.com Frederik J. Simons Xiangliang Zhang
  45. 45. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models References 45 Adler, A., Araya-Polo, M. and Poggio, T., 2019, June. Deep Recurrent Architectures for Seismic Tomography. In 81st EAGE Conference and Exhibition 2019. Araya-Polo, M., Jennings, J., Adler, A. and Dahlke, T., 2018. Deep-learning tomography. The Leading Edge, 37(1), pp.58-66. Gatys, L.A., Ecker, A.S. and Bethge, M., 2015. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576. Johnson, J., Alahi, A. and Fei-Fei, L., 2016, October. Perceptual losses for real-time style transfer and super-resolution. In European conference on computer vision (pp. 694-711). Springer, Cham. Kazei, V., Ovcharenko, O., Zhang, X., Peter, D. & Alkhalifah, T. "Mapping seismic data cubes to vertical velocity profiles by deep learning: New full-waveform inversion paradigm?", Geophysics, submitted (2019) Li, S., Liu, B., Ren, Y., Chen, Y., Yang, S., Wang, Y. and Jiang, P., 2019. Deep learning inversion of seismic data. arXiv preprint arXiv: 1901.07733. Mosser, L., Dubrule, O. and Blunt, M., 2018, November. Stochastic seismic waveform inversion using generative adversarial networks as a geological prior. In First EAGE/PESGB Workshop Machine Learning. Ovcharenko, O., Kazei, V., Kalita, M., Peter, D. and Alkhalifah, T.A., 2019. Deep learning for low-frequency extrapolation from multi-offset seismic data. Øye, O.K. and Dahl, E.K., 2019, June. Velocity Model Building from Raw Shot Gathers Using Machine Learning. In 81st EAGE Conference and Exhibition 2019. Richardson, A., 2018. Generative adversarial networks for model order reduction in seismic full-waveform inversion. arXiv preprint arXiv: 1806.00828. Simonyan, K. and Zisserman, A., 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. Ulyanov, D., Lebedev, V., Vedaldi, A. and Lempitsky, V.S., 2016, June. Texture Networks: Feed-forward Synthesis of Textures and Stylized Images. In ICML (Vol. 1, No. 2, p. 4). Yang, F. and Ma, J., 2019. Deep-learning inversion: a next generation seismic velocity-model building method. Geophysics, 84(4), pp. 1-133.
  46. 46. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models Simplified priors + Geological models = target-textured datasets 46 Well-log constraints can be incorporated Demo notebook available at https://github.com/ovcharenkoo/geo-style-keras Outlook Test models in low frequency extrapolation and velocity model building Fast style transfer and Automated parameter selection + = Conclusions

×