Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Transfer learning for low frequency extrapolation from shot gathers for FWI applications

241 views

Published on

Slides for my talk at EAGE 2019 in London this June. We attempt to extrapolate for missing low-frequency content in seismic data using a deep learning (DL) approach. We generate a set of random subsurface models and use those to produce a synthetic training dataset. We train a supervised DL model to infer a mono-frequency representation of a common shot gather, given respective data on multiple high frequencies. In the end, we show an example of FWI on extrapolated synthetic data and an example of bandwidth extrapolation on a single shot from field data.

Published in: Science
  • Be the first to comment

  • Be the first to like this

Transfer learning for low frequency extrapolation from shot gathers for FWI applications

  1. 1. oleg.ovcharenko@kaust.edu.sa O. Ovcharenko, V. Kazei, D. Peter, T. Alkhalifah Transfer learning for low frequency extrapolation from shot gathers for FWI applications June 6th, 2019
  2. 2. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Team 2 Vladimir Kazei, Post-doctoral Fellow Oleg Ovcharenko, PhD student Tariq Alkhalifah, Professor Daniel Peter, Assistant Professor KAUST Saudi Arabia Temperature on June 6th Max: 41°C Min: 23°C Average: 32°C Today in London: ~18°C
  3. 3. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Team 3 Vladimir Kazei, Post-doctoral Fellow Oleg Ovcharenko, PhD student Tariq Alkhalifah, Professor Daniel Peter, Assistant Professor KAUST Saudi Arabia Temperature on June 6th Max: 41°C Min: 23°C Average: 32°C Today in London: ~18°C
  4. 4. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Previous work 4 2017 2018 Today “Low-Frequency Data Extrapolation Using a Feed- Forward ANN”, 80th EAGE Annual conference, 2018 “Neural network based low-frequency data extrapolation”, SEG FWI Workshop: Where are we getting?  “Transfer learning for low frequency extrapolation from shot gathers for FWI applications”, 81st EAGE Annual conference, 2019
  5. 5. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Limitations of real-world acquisitions 5 Lack of low-frequency data - Due to instrumental limitations - Due to noise
  6. 6. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Low-frequency data in FWI 6 - Inverts large-scale velocity structures - Less chance to get stuck in local minima - Reveals deep model structures / below salt fHigh fLow Multiple local minima Smooth (Kazei et al., 2016)
  7. 7. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers FWI with different misfits 7 (Bozdag 2011, Choi & Alkhalifah 2013, Leeuwen & Herrmann, 2014 …) Pros: - Established workflow - Direct image quality control Cons: - Computational costs - Prone to event mismatching - Sensitivity hard to control Kalita et al., 2018
  8. 8. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers FWI with update conditioning 8 (Alkhalifah, 2015; Kazei, et al., 2016; Yao et al., 2018; Ovcharenko, et al., 2018 ) etc… Pros: - Easily accessible sensitivity - Direct image quality control - Can be used together with any misfit Cons: - Computational costs - Prone to event mismatching Ovcharenko, et al., 2018
  9. 9. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 9Extrapolation of low-frequency data Pros: - Cheaper computations Cons: - Not well explored robustness - Wavefield approximations Hu et al., 2014; Li & Demanet, 2015, 2016, Ovcharenko et al., 2018) etc…
  10. 10. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 10 This work: shot-to-shot extrapolation Beat tone inversion (Hu et al., 2014) Bandwidth extension for atomic events (Li & Demanet, 2015, 2016) Deep learning freq domain for CSG – Ovcharenko et al., 2017, Trace to trace deep learning – Sun & Demanet, 2018 Extrapolation of low-frequency data
  11. 11. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Why extrapolation should be possible? 11
  12. 12. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 12Why extrapolation should be possible?
  13. 13. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers One trace, one shot, one dataset 13 Accuracy Computational complexity Trace-to-Trace Shot-to-Shot Data-to-Data (Ovcharenko et al, 2017, 2018) (Sun & Demanet, 2018)
  14. 14. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 14Mono-frequency shot gather
  15. 15. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Supervised learning to link data in frequency domain 15 One network is trained to extrapolate for One frequency
  16. 16. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers ML to learn the phenomena 16 High Low ? Non-linear function
  17. 17. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 17 Training dataset
  18. 18. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Generators which didn’t work well 18
  19. 19. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Generator on 1D profiles 19
  20. 20. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Realistic random models based on wavelets and style transfer 20 “Style transfer for generation of realistically textured subsurface models”, Ovcharenko et al., SEG Technical Program Expanded Abstracts 2019 “Realistically Textured Random Velocity Models for Deep Learning Applications”, Kazei et al., 81st EAGE Conference and Exhibition 2019
  21. 21. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Size of training dataset 21 Total training samples = random models x shots in the model
  22. 22. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 22 Deep learning model
  23. 23. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Selection of network architecture 23 Custom design Ready-to-use design
  24. 24. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Selection of network architecture 24 Custom design Ready-to-use design
  25. 25. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Learned filters applied to a different dataset 25
  26. 26. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Benchmark of network architectures 26 (Bianco et al., 2018, "Benchmark Analysis of Representative Deep Neural Network Architectures”)
  27. 27. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Benchmark of network architectures 27 (Bianco et al., 2018, "Benchmark Analysis of Representative Deep Neural Network Architectures”)
  28. 28. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers MobileNet (Howard et al., 2017) 28 NVIDIA Titan V TensorFlow 1.12.0Python 3.6 Keras 2.2.4 Matlab R2016b Training for one frequency ~ 5 min Inference time < 1 sec Total params ~ 3.9M
  29. 29. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 29 Examples
  30. 30. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 30Input and target data (R x F x 2) (R x 1 x 2)
  31. 31. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 31Parametrization (R x F x 3) (R x 1 x 2)
  32. 32. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 32 Example 1/2. Central part of BP 2004
  33. 33. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers FWI for the central part of BP 2004 benchmark model 33 64 sources and receivers 32 known frequency in range 3-5 Hz https://github.com/vkazei/fastFWI Successive mono-frequency inversions at 0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz Acoustic
  34. 34. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Extrapolated low-frequency data for 0.25 Hz 34 True Pred Diff Re Im Phase
  35. 35. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Extrapolated low-frequency data for 0.55 Hz 35 True Pred Diff Re Im Phase
  36. 36. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Extrapolated low-frequency data for 0.93 Hz 36 True Pred Diff Re Im Phase
  37. 37. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Initial model 37
  38. 38. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Conventional regularized FWI 38
  39. 39. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 39 FWI
  40. 40. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 40 What can go wrong with inversion?
  41. 41. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 41 What can go wrong with inversion? FWI Frequency Extrapolation
  42. 42. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 42 What can go wrong with inversion? FWI Frequency Extrapolation Normalization Training data Training Assumption Implementation
  43. 43. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 43 What can go wrong with inversion? FWI Frequency Extrapolation Normalization Training data Training Misfit Solver Initial guess Formulation Implementation Assumption Implementation
  44. 44. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 44 What can go wrong with inversion? FWI Frequency Extrapolation Normalization Training data Training Misfit Solver Initial guess Formulation Implementation Assumption Implementation True Pred Diff Re Im Phase
  45. 45. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 45 What can go wrong with inversion? FWI Frequency Extrapolation Normalization Training data Training Misfit Solver Initial guess Formulation Implementation Assumption Implementation True Pred Diff Re Im 0.25 Hz
  46. 46. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 46 What can go wrong with inversion? FWI Frequency Extrapolation Normalization Training data Training Misfit Solver Initial guess Formulation Implementation Assumption Implementation True Pred Diff Re Im 0.25 Hz4.5 Hz
  47. 47. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 47 It is a rocky road Training Misfit Solver
  48. 48. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 48 FWI True vs Extrapolated
  49. 49. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 0.25 Hz 49 0.25Hz 0 5 10 15 20 km 0 2 4 6 km 2 3 4 km/s 0.25Hz 0 5 10 15 20 km 0 2 4 6 km 2 3 4 km/s True Extrapolated 0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz
  50. 50. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 0.55 Hz 50 0.55Hz 0 5 10 15 20 km 0 2 4 6 km 2 3 4 km/s 0.55Hz 0 5 10 15 20 km 0 2 4 6 km 2 3 4 km/s True Extrapolated 0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz
  51. 51. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 0.93 Hz 51 0.93Hz 0 5 10 15 20 km 0 2 4 6 km 2 3 4 km/s 0 5 10 15 20 km 0 2 4 6 km 2 3 4 km/s True Extrapolated 0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz
  52. 52. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 4.5 Hz 52 0 5 10 15 20 km 0 2 4 6 km 2 3 4 km/s 0 5 10 15 20 km 0 2 4 6 km 2 3 4 km/s True Extrapolated 0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz
  53. 53. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 53 Do we overfit the data?
  54. 54. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 54Left part of BP 2004 True Pred Diff Re Im Phase
  55. 55. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 55Marmousi II True Pred Diff Re Im Phase True Pred Diff Re Im Phase
  56. 56. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 56 Example 2/2. One shot from field
  57. 57. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Streamer acquisition 57
  58. 58. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Single shot 58
  59. 59. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Time domain 59
  60. 60. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Frequency domain 60
  61. 61. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Low-frequency part of spectrum 61
  62. 62. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Used range 4 - 10 Hz 62
  63. 63. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Regular sampling of the range 63
  64. 64. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Raw input data for extrapolation 64
  65. 65. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 65Normalized input data for extrapolation
  66. 66. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Train one net for each frequency 66 13 frequencies to extrapolate for: 0.1000 0.1334 0.1778 0.2371 0.3162 0.4217 0.5623 0.7499 1.0000 1.3335 1.7783 2.3714 3.1623 Hz
  67. 67. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Extrapolated low frequency 67 Synthetic Field A reference sample of low- frequency data for a random velocity model
  68. 68. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Extrapolated low frequency 68 Synthetic Field Extrapolated low-frequency data for a real-world shot
  69. 69. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Extrapolated low frequency 69 Synthetic Field Extrapolated low-frequency data for a real-world shot Expected patterns in low-frequencies
  70. 70. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 70 Trained on synthetic, tested on synthetic = OK
  71. 71. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers 71 Trained on synthetic, tested on field = ???
  72. 72. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Conclusions 72 Wavenumber analysis justifies feasibility of bandwidth extrapolation Pre-trained networks are applicable with mild retraining Need a priori constraints on random velocity models Lowest frequencies are better extrapolated Regularized full-waveform inversion to tolerate inaccuracies
  73. 73. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Acknowledgements 73 Mahesh Kalita (KAUST/ION) Xiangliang Zhang (KAUST) Gerhard Pratt (UWO) Tristan van Leeuven (Utrecht University) Yunyue Elita Li (NUS) ovcharenkoo.com ai.vkazei.com
  74. 74. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers References 74 Alkhalifah, T., 2015. Conditioning the full-waveform inversion gradient to welcome anisotropy. Geophysics, 80(3), pp.R111-R122. Bozdağ, E., Trampert, J. and Tromp, J., 2011. Misfit functions for full waveform inversion based on instantaneous phase and envelope measurements. Geophysical Journal International, 185(2), pp.845-870. Choi, Y. and Alkhalifah, T., 2013. Frequency-domain waveform inversion using the phase derivative. Geophysical Journal International, 195(3), pp. 1904-1916. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M. and Adam, H., 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861. Hu*, W., 2014. FWI without low frequency data-beat tone inversion. In SEG Technical Program Expanded Abstracts 2014 (pp. 1116-1120). Society of Exploration Geophysicists. Kazei, V., Tessmer, E. and Alkhalifah, T., 2016. Scattering angle-based filtering via extension in velocity. In SEG Technical Program Expanded Abstracts 2016 (pp. 1157-1162). Society of Exploration Geophysicists. Kazei, V. and Alkhalifah, T., 2019. Scattering Radiation Pattern Atlas: What anisotropic elastic properties can body waves resolve?. Journal of Geophysical Research: Solid Earth. Li, Y.E. and Demanet, L., 2015. Phase and amplitude tracking for seismic event separation. Geophysics, 80(6), pp.WD59-WD72. Li, Y.E. and Demanet, L., 2016. Full-waveform inversion with extrapolated low-frequency data. Geophysics, 81(6), pp.R339-R348. Kalita, M., Kazei, V., Choi, Y. and Alkhalifah, T., 2018. Regularized full-waveform inversion for salt bodies. In SEG Technical Program Expanded Abstracts 2018 (pp. 1043-1047). Society of Exploration Geophysicists. Ovcharenko, O., Kazei, V., Peter, D. and Alkalifah, T., 2017. Neural network based low-frequency data extrapolation. In 3rd SEG FWI workshop: What are we getting. Ovcharenko, O., Kazei, V., Peter, D., Zhang, X. and Alkhalifah, T., 2018, June. Low-Frequency Data Extrapolation Using a Feed-Forward ANN. In 80th EAGE Conference and Exhibition 2018. Ovcharenko, O., Kazei, V., Peter, D. and Alkhalifah, T., 2018. Variance-based model interpolation for improved full-waveform inversion in the presence of salt bodies. Geophysics, 83(5), pp.R541-R551. Sun, H. and Demanet, L., 2018. Low frequency extrapolation with deep learning. In SEG Technical Program Expanded Abstracts 2018 (pp. 2011-2015). Society of Exploration Geophysicists. van Leeuwen, T. and Herrmann, F.J., 2014. 3D frequency-domain seismic inversion with controlled sloppiness. SIAM Journal on Scientific Computing, 36(5), pp.S192-S217.
  75. 75. oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers Conclusions 75 Wavenumber analysis justifies feasibility of bandwidth extrapolation Pre-trained networks are applicable with mild retraining Need a priori constraints on random velocity models Lowest frequencies are better extrapolated Regularized full-waveform inversion to tolerate inaccuracies

×