We apply the neural style transfer technique (Gatys et al., 2015) to enrich a simplified prior content distribution with features extracted from a geological reference. We demonstrate an application of this approach by transferring the layered texture from the Marmousi II benchmark model to a distribution generated by random Gaussian field. This work offers another way for the generation of target-specific pseudo-random subsurface models.

1. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.sa
O. Ovcharenko, V. Kazei, D. Peter, T. Alkhalifah
Style transfer for generation of realistically
textured subsurface models
Sep 18th, 2019
San Antonio, TX

2. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
Team 2
Vladimir Kazei,
Post-doctoral Fellow
Oleg Ovcharenko,
PhD student
Tariq Alkhalifah,
Professor
Daniel Peter,
Assistant Professor
KAUST
Saudi Arabia

3. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
Supervised deep learning 3
Subsurface
model
Seismic
data
Inverted model

4. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
Community random model generators 4
Deep-learning tomography
Araya-Polo et al., 2017
Deep-learning inversion: a next generation
seismic velocity-model building method
Yang and Ma, 2019
Deep learning Inversion of Seismic Data
Li et al, 2019
Generative Adversarial Networks for Model
Order Reduction in
Seismic Full-Waveform Inversion,
Richardson, 2018
Velocity model building from raw shot
gathers using machine learning
Øye and Dahl, 2019
Stochastic Seismic Waveform Inversion
using Generative Adversarial Networks as
a Geological Prior
Mosser et al, 2018

5. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
Backstory — Bandwidth extrapolation 5
“Deep learning for low-frequency extrapolation from multi-offset seismic data”,
Ovcharenko et al., 2019.
GEOPHYSICS

6. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
6Backstory — Random subsurface models

7. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
Backstory — Velocity model building 7
“Mapping seismic data cubes to vertical velocity profiles by deep learning:
New full-waveform inversion paradigm?”, Kazei et al., 2019
submitted to GEOPHYSICS https://github.com/vkazei/deeplogs

8. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
Backstory — Elastic transform 8
“Mapping seismic data cubes to vertical velocity profiles by deep learning:
New full-waveform inversion paradigm?”, Kazei et al., 2019
submitted to GEOPHYSICS https://github.com/vkazei/deeplogs

9. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
Textures –> wavelet packets 9
Kazei, V., et al. "Realistically Textured
Random Velocity Models for Deep
Learning Applications”, EAGE 2019

10. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
Neural style-transfer 10
Gatys, L.A., Ecker, A.S. and Bethge,
M., 2015. A neural algorithm of artistic
style. arXiv preprint arXiv:1508.06576.
Neckarfront in Tubingen, Germany
Der Schrei by Edvard MunchThe Starry Night by Vincent van Gogh
The Shipwreck of the Minotaur
by J.M.W. Turner

11. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
Color channels RGB = elastic isotropic parameters 11
RGB = Vp, Vs, Rho

12. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
Application to elastic models 12
+
=
Content
Style
Generated

13. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
13
Workflow
Parametrization, losses, optimization

14. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
14
3. Extract feature maps
Flowchart
2. Propagate through
the network
4. Compute loss
5. Compute
gradients
∂L
∂m
1. Init prior
L = ws Ls + wc Lc + wTV LTV
6. Update

15. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
15Feature extractor — VGG16 (Simonyan and Zisserman, 2014)
Going deeper
Input
image
64
128
256
512
512

16. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
16Content loss
-
2
2
Lc =

17. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
17Style loss
Texture representation
builds from multiple
scales
Going deeper

18. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
18Gram matrix
f
h
w
h * w
f
f
h*w
f
f
* = G
f
f

19. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
19Style loss
-
2
2
Ls = Gs G

20. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
20Objective function
Total ContentStyle Smoothing

21. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
21
25%
50%
100%
200%
1000%
Contentcontribution
Content/style weight ratio

22. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
22
+
Marmousi II“Salt”
TensorFlow 1.12.0Python 3.6 Keras 2.2.4 Titan V
Computational aspects
Size:
100 x 300 x 3
L-BFGS:
100 iterations
~ 1 sec / iter
Future: fast style transfer by GAN following (Johnson et al., 2016; Ulyanov et al., 2016)
Demo notebook available at https://github.com/ovcharenkoo/geo-style-keras

23. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
23
+

24. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
24
+

25. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
25
+

26. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
26
+

27. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
27
+

28. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
28
+ Target contrasts
preserved, but need
more smoothing

29. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
29
+ Less sharp
contrasts, more
consistency

30. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
30
Would it cause strong variability?
Starting optimization from random noise?

31. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
31A few samples
+ Sample 1/3

32. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
32
+ Sample 2/3
A few samples

33. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
33
+ Sample 3/3
A few samples

34. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
34
+
Mean
Depends on loss
weights

35. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
35
+
Standard deviation
Depends on loss
weights

36. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
36
Well-log constraints and wave propagation
L2 loss for given well-log locations

37. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
37More constraints
L = ws Ls + wc Lc + wTV LTV + wlog Llog
Balance of smoothing and well-log penalties

38. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
38
+
Well-log constraint - OFF
MarmousiRandom Gaussian Field

39. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
39
+
MarmousiRandom Gaussian Field
Well-log constraint - ON

40. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
40
+
MarmousiRandom Gaussian Field
Well-log constraint - ON

41. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
Wave propagation 41
https://github.com/ar4/deepwave
3 km
10 km
vmax = 3500 m/s
vmin = 1500 m/s
fc = 10 Hz
t = 3 s

42. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
42Test on benchmark models
Layered style prior leads to diverse outputs

43. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
Test on benchmark models 43
Gradient or homogeneous content priors produce visually-plausible models

44. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
Acknowledgements 44
ovcharenkoo.com vkazei.com
Frederik J. Simons
Xiangliang Zhang

45. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
References 45
Adler, A., Araya-Polo, M. and Poggio, T., 2019, June. Deep Recurrent Architectures for Seismic Tomography. In 81st EAGE Conference and
Exhibition 2019.
Araya-Polo, M., Jennings, J., Adler, A. and Dahlke, T., 2018. Deep-learning tomography. The Leading Edge, 37(1), pp.58-66.
Gatys, L.A., Ecker, A.S. and Bethge, M., 2015. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576.
Johnson, J., Alahi, A. and Fei-Fei, L., 2016, October. Perceptual losses for real-time style transfer and super-resolution. In European
conference on computer vision (pp. 694-711). Springer, Cham.
Kazei, V., Ovcharenko, O., Zhang, X., Peter, D. & Alkhalifah, T. "Mapping seismic data cubes to vertical velocity profiles by deep learning:
New full-waveform inversion paradigm?", Geophysics, submitted (2019)
Li, S., Liu, B., Ren, Y., Chen, Y., Yang, S., Wang, Y. and Jiang, P., 2019. Deep learning inversion of seismic data. arXiv preprint arXiv:
1901.07733.
Mosser, L., Dubrule, O. and Blunt, M., 2018, November. Stochastic seismic waveform inversion using generative adversarial networks as a
geological prior. In First EAGE/PESGB Workshop Machine Learning.
Ovcharenko, O., Kazei, V., Kalita, M., Peter, D. and Alkhalifah, T.A., 2019. Deep learning for low-frequency extrapolation from multi-offset
seismic data.
Øye, O.K. and Dahl, E.K., 2019, June. Velocity Model Building from Raw Shot Gathers Using Machine Learning. In 81st EAGE Conference
and Exhibition 2019.
Richardson, A., 2018. Generative adversarial networks for model order reduction in seismic full-waveform inversion. arXiv preprint arXiv:
1806.00828.
Simonyan, K. and Zisserman, A., 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
Ulyanov, D., Lebedev, V., Vedaldi, A. and Lempitsky, V.S., 2016, June. Texture Networks: Feed-forward Synthesis of Textures and Stylized
Images. In ICML (Vol. 1, No. 2, p. 4).
Yang, F. and Ma, J., 2019. Deep-learning inversion: a next generation seismic velocity-model building method. Geophysics, 84(4), pp.
1-133.

46. {oleg.ovcharenko, vladimir.kazei}@kaust.edu.saStyle transfer for realistic subsurface models
Simplified priors + Geological models = target-textured datasets
46
Well-log constraints can be incorporated
Demo notebook available at https://github.com/ovcharenkoo/geo-style-keras
Outlook
Test models in low frequency extrapolation and velocity model building
Fast style transfer and Automated parameter selection
+ =
Conclusions