- 1. oleg.ovcharenko@kaust.edu.sa O. Ovcharenko, V. Kazei, D. Peter, T. Alkhalifah Transfer learning for low frequency extrapolation from shot gathers for FWI applications June 6th, 2019
- 2. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Team 2 Vladimir Kazei, Post-doctoral Fellow Oleg Ovcharenko, PhD student Tariq Alkhalifah, Professor Daniel Peter, Assistant Professor KAUST Saudi Arabia Temperature on June 6th Max: 41°C Min: 23°C Average: 32°C Today in London: ~18°C
- 3. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Team 3 Vladimir Kazei, Post-doctoral Fellow Oleg Ovcharenko, PhD student Tariq Alkhalifah, Professor Daniel Peter, Assistant Professor KAUST Saudi Arabia Temperature on June 6th Max: 41°C Min: 23°C Average: 32°C Today in London: ~18°C
- 4. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Previous work 4 2017 2018 Today “Low-Frequency Data Extrapolation Using a Feed- Forward ANN”, 80th EAGE Annual conference, 2018 “Neural network based low-frequency data extrapolation”, SEG FWI Workshop: Where are we getting? “Transfer learning for low frequency extrapolation from shot gathers for FWI applications”, 81st EAGE Annual conference, 2019
- 5. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Limitations of real-world acquisitions 5 Lack of low-frequency data - Due to instrumental limitations - Due to noise
- 6. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Low-frequency data in FWI 6 - Inverts large-scale velocity structures - Less chance to get stuck in local minima - Reveals deep model structures / below salt fHigh fLow Multiple local minima Smooth (Kazei et al., 2016)
- 7. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers FWI with different misﬁts 7 (Bozdag 2011, Choi & Alkhalifah 2013, Leeuwen & Herrmann, 2014 …) Pros: - Established workflow - Direct image quality control Cons: - Computational costs - Prone to event mismatching - Sensitivity hard to control Kalita et al., 2018
- 8. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers FWI with update conditioning 8 (Alkhalifah, 2015; Kazei, et al., 2016; Yao et al., 2018; Ovcharenko, et al., 2018 ) etc… Pros: - Easily accessible sensitivity - Direct image quality control - Can be used together with any misfit Cons: - Computational costs - Prone to event mismatching Ovcharenko, et al., 2018
- 9. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 9Extrapolation of low-frequency data Pros: - Cheaper computations Cons: - Not well explored robustness - Wavefield approximations Hu et al., 2014; Li & Demanet, 2015, 2016, Ovcharenko et al., 2018) etc…
- 10. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 10 This work: shot-to-shot extrapolation Beat tone inversion (Hu et al., 2014) Bandwidth extension for atomic events (Li & Demanet, 2015, 2016) Deep learning freq domain for CSG – Ovcharenko et al., 2017, Trace to trace deep learning – Sun & Demanet, 2018 Extrapolation of low-frequency data
- 11. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Why extrapolation should be possible? 11
- 12. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 12Why extrapolation should be possible?
- 13. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers One trace, one shot, one dataset 13 Accuracy Computational complexity Trace-to-Trace Shot-to-Shot Data-to-Data (Ovcharenko et al, 2017, 2018) (Sun & Demanet, 2018)
- 14. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 14Mono-frequency shot gather
- 15. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Supervised learning to link data in frequency domain 15 One network is trained to extrapolate for One frequency
- 16. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers ML to learn the phenomena 16 High Low ? Non-linear function
- 17. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 17 Training dataset
- 18. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Generators which didn’t work well 18
- 19. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Generator on 1D proﬁles 19
- 20. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Realistic random models based on wavelets and style transfer 20 “Style transfer for generation of realistically textured subsurface models”, Ovcharenko et al., SEG Technical Program Expanded Abstracts 2019 “Realistically Textured Random Velocity Models for Deep Learning Applications”, Kazei et al., 81st EAGE Conference and Exhibition 2019
- 21. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Size of training dataset 21 Total training samples = random models x shots in the model
- 22. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 22 Deep learning model
- 23. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Selection of network architecture 23 Custom design Ready-to-use design
- 24. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Selection of network architecture 24 Custom design Ready-to-use design
- 25. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Learned ﬁlters applied to a different dataset 25
- 26. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Benchmark of network architectures 26 (Bianco et al., 2018, "Benchmark Analysis of Representative Deep Neural Network Architectures”)
- 27. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Benchmark of network architectures 27 (Bianco et al., 2018, "Benchmark Analysis of Representative Deep Neural Network Architectures”)
- 28. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers MobileNet (Howard et al., 2017) 28 NVIDIA Titan V TensorFlow 1.12.0Python 3.6 Keras 2.2.4 Matlab R2016b Training for one frequency ~ 5 min Inference time < 1 sec Total params ~ 3.9M
- 29. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 29 Examples
- 30. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 30Input and target data (R x F x 2) (R x 1 x 2)
- 31. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 31Parametrization (R x F x 3) (R x 1 x 2)
- 32. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 32 Example 1/2. Central part of BP 2004
- 33. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers FWI for the central part of BP 2004 benchmark model 33 64 sources and receivers 32 known frequency in range 3-5 Hz https://github.com/vkazei/fastFWI Successive mono-frequency inversions at 0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz Acoustic
- 34. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Extrapolated low-frequency data for 0.25 Hz 34 True Pred Diﬀ Re Im Phase
- 35. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Extrapolated low-frequency data for 0.55 Hz 35 True Pred Diﬀ Re Im Phase
- 36. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Extrapolated low-frequency data for 0.93 Hz 36 True Pred Diﬀ Re Im Phase
- 37. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Initial model 37
- 38. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Conventional regularized FWI 38
- 39. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 39 FWI
- 40. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 40 What can go wrong with inversion?
- 41. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 41 What can go wrong with inversion? FWI Frequency Extrapolation
- 42. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 42 What can go wrong with inversion? FWI Frequency Extrapolation Normalization Training data Training Assumption Implementation
- 43. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 43 What can go wrong with inversion? FWI Frequency Extrapolation Normalization Training data Training Misﬁt Solver Initial guess Formulation Implementation Assumption Implementation
- 44. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 44 What can go wrong with inversion? FWI Frequency Extrapolation Normalization Training data Training Misﬁt Solver Initial guess Formulation Implementation Assumption Implementation True Pred Diﬀ Re Im Phase
- 45. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 45 What can go wrong with inversion? FWI Frequency Extrapolation Normalization Training data Training Misﬁt Solver Initial guess Formulation Implementation Assumption Implementation True Pred Diﬀ Re Im 0.25 Hz
- 46. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 46 What can go wrong with inversion? FWI Frequency Extrapolation Normalization Training data Training Misﬁt Solver Initial guess Formulation Implementation Assumption Implementation True Pred Diﬀ Re Im 0.25 Hz4.5 Hz
- 47. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 47 It is a rocky road Training Misﬁt Solver
- 48. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 48 FWI True vs Extrapolated
- 49. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 0.25 Hz 49 0.25Hz 0 5 10 15 20 km 0 2 4 6 km 2 3 4 km/s 0.25Hz 0 5 10 15 20 km 0 2 4 6 km 2 3 4 km/s True Extrapolated 0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz
- 50. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 0.55 Hz 50 0.55Hz 0 5 10 15 20 km 0 2 4 6 km 2 3 4 km/s 0.55Hz 0 5 10 15 20 km 0 2 4 6 km 2 3 4 km/s True Extrapolated 0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz
- 51. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 0.93 Hz 51 0.93Hz 0 5 10 15 20 km 0 2 4 6 km 2 3 4 km/s 0 5 10 15 20 km 0 2 4 6 km 2 3 4 km/s True Extrapolated 0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz
- 52. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 4.5 Hz 52 0 5 10 15 20 km 0 2 4 6 km 2 3 4 km/s 0 5 10 15 20 km 0 2 4 6 km 2 3 4 km/s True Extrapolated 0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz
- 53. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 53 Do we overﬁt the data?
- 54. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 54Left part of BP 2004 True Pred Diﬀ Re Im Phase
- 55. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 55Marmousi II True Pred Diﬀ Re Im Phase True Pred Diﬀ Re Im Phase
- 56. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 56 Example 2/2. One shot from ﬁeld
- 57. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Streamer acquisition 57
- 58. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Single shot 58
- 59. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Time domain 59
- 60. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Frequency domain 60
- 61. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Low-frequency part of spectrum 61
- 62. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Used range 4 - 10 Hz 62
- 63. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Regular sampling of the range 63
- 64. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Raw input data for extrapolation 64
- 65. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 65Normalized input data for extrapolation
- 66. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Train one net for each frequency 66 13 frequencies to extrapolate for: 0.1000 0.1334 0.1778 0.2371 0.3162 0.4217 0.5623 0.7499 1.0000 1.3335 1.7783 2.3714 3.1623 Hz
- 67. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Extrapolated low frequency 67 Synthetic Field A reference sample of low- frequency data for a random velocity model
- 68. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Extrapolated low frequency 68 Synthetic Field Extrapolated low-frequency data for a real-world shot
- 69. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Extrapolated low frequency 69 Synthetic Field Extrapolated low-frequency data for a real-world shot Expected patterns in low-frequencies
- 70. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 70 Trained on synthetic, tested on synthetic = OK
- 71. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 71 Trained on synthetic, tested on ﬁeld = ???
- 72. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Conclusions 72 Wavenumber analysis justiﬁes feasibility of bandwidth extrapolation Pre-trained networks are applicable with mild retraining Need a priori constraints on random velocity models Lowest frequencies are better extrapolated Regularized full-waveform inversion to tolerate inaccuracies
- 73. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Acknowledgements 73 Mahesh Kalita (KAUST/ION) Xiangliang Zhang (KAUST) Gerhard Pratt (UWO) Tristan van Leeuven (Utrecht University) Yunyue Elita Li (NUS) ovcharenkoo.com ai.vkazei.com
- 74. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers References 74 Alkhalifah, T., 2015. Conditioning the full-waveform inversion gradient to welcome anisotropy. Geophysics, 80(3), pp.R111-R122. Bozdağ, E., Trampert, J. and Tromp, J., 2011. Misfit functions for full waveform inversion based on instantaneous phase and envelope measurements. Geophysical Journal International, 185(2), pp.845-870. Choi, Y. and Alkhalifah, T., 2013. Frequency-domain waveform inversion using the phase derivative. Geophysical Journal International, 195(3), pp. 1904-1916. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M. and Adam, H., 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861. Hu*, W., 2014. FWI without low frequency data-beat tone inversion. In SEG Technical Program Expanded Abstracts 2014 (pp. 1116-1120). Society of Exploration Geophysicists. Kazei, V., Tessmer, E. and Alkhalifah, T., 2016. Scattering angle-based filtering via extension in velocity. In SEG Technical Program Expanded Abstracts 2016 (pp. 1157-1162). Society of Exploration Geophysicists. Kazei, V. and Alkhalifah, T., 2019. Scattering Radiation Pattern Atlas: What anisotropic elastic properties can body waves resolve?. Journal of Geophysical Research: Solid Earth. Li, Y.E. and Demanet, L., 2015. Phase and amplitude tracking for seismic event separation. Geophysics, 80(6), pp.WD59-WD72. Li, Y.E. and Demanet, L., 2016. Full-waveform inversion with extrapolated low-frequency data. Geophysics, 81(6), pp.R339-R348. Kalita, M., Kazei, V., Choi, Y. and Alkhalifah, T., 2018. Regularized full-waveform inversion for salt bodies. In SEG Technical Program Expanded Abstracts 2018 (pp. 1043-1047). Society of Exploration Geophysicists. Ovcharenko, O., Kazei, V., Peter, D. and Alkalifah, T., 2017. Neural network based low-frequency data extrapolation. In 3rd SEG FWI workshop: What are we getting. Ovcharenko, O., Kazei, V., Peter, D., Zhang, X. and Alkhalifah, T., 2018, June. Low-Frequency Data Extrapolation Using a Feed-Forward ANN. In 80th EAGE Conference and Exhibition 2018. Ovcharenko, O., Kazei, V., Peter, D. and Alkhalifah, T., 2018. Variance-based model interpolation for improved full-waveform inversion in the presence of salt bodies. Geophysics, 83(5), pp.R541-R551. Sun, H. and Demanet, L., 2018. Low frequency extrapolation with deep learning. In SEG Technical Program Expanded Abstracts 2018 (pp. 2011-2015). Society of Exploration Geophysicists. van Leeuwen, T. and Herrmann, F.J., 2014. 3D frequency-domain seismic inversion with controlled sloppiness. SIAM Journal on Scientific Computing, 36(5), pp.S192-S217.
- 75. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Conclusions 75 Wavenumber analysis justiﬁes feasibility of bandwidth extrapolation Pre-trained networks are applicable with mild retraining Need a priori constraints on random velocity models Lowest frequencies are better extrapolated Regularized full-waveform inversion to tolerate inaccuracies