Managing the computational cost of Large PDE problems with multiple right hand side sources using source sketching algorithm

1. Randomized Source Sketching for Full Waveform Inversion (IEEE
Transactions on Geoscience and Remote Sensing)
K. Aghazade*, H. S. Aghamiry**, A. Gholami*, and S. Operto**
*University of Tehran, Institute of Geophysics, Tehran, Iran
**University of Côte d’Azur, Geoazur, CNRS - IRD - OCA, Valbonne, France

2. 2
How to tackle the computational burden of a multi-right
hand side partial differential equation?
The Helmholtz equation: Source terms
Direct Solvers (LU/MUMPS, etc)
Iterative Solvers (GMRES/PCG,
etc)
 Suitable for sparse matrices
 Challenging for multi-
parameter/3D applications
Computational burden of solving
the system is proportional to the
number of right-hand sides
Our Proposal:
Dimensionality reduction based
on Source Sketching
Randomized Source Sketching
Problem definition

3. 3
Matrix Sketching
Matrix Sketching: What is it? & How it works?
Computationally
challenging
Sketching
matrix
Dimension Reduction
𝑿 = 𝐗𝐒 ∙×𝒒
𝑞 ≪ 𝑝
 Problem: How to manage large
matrices?
 Find another matrix which is significantly smaller
than the original one, but still approximates it well.
Randomized Source Sketching
Wang (2015)

4. 4
Choices of Sketching matrices
Sketching Matrices Randomized Source Sketching
Gram Matrix
Sketching
matrices
Random
sketching
matrices
Gaussian Sketch
Bernoulli/
Rademacher Sketch
Random phase Sketch
Count Sketch
Randomized
orthogonal
matrices
Identity matrix
Discrete Fourier
transform
Discrete cosine
transform
Discrete wavelet
transform
Noiselet transform
Hadamard transform
Castellanos et al
(2015)
Haber et al (2011)
Ben-Hadj-Ali et al
(2011)

5. 5
Choices of Sketching matrices
(Part1: Random Sketching matrices)
G𝑖𝑗~𝒩(0,1)
φ𝑖𝑗 ∈ [0 2𝜋]
Sketching Matrices Randomized Source Sketching
First, initialize S to be the p × q all-zero
matrix. Second, for each column of original matrix, we
flip its sign with probability 50%, and add it
to a uniformly selected column of S (Wang 2015).
 Very efficient, especially when
the original matrix is sparse

6. 6
Choices of Sketching matrices
(Part2: Random Sketching matrices)
Sketching Matrices Randomized Source Sketching
Orthogonal matrix H of size 𝑝 × 𝑝
Permutation matrix
that reorders the rows of the
matrix H randomly
is chosen randomly from
the set of all p canonical
basis vectors (the columns
of the identity matrix).
More info:
Randomized source sketching for full waveform inversion
https://arxiv.org/abs/2108.03961

7. 7
Choices of Sketching matrices
Sketching Matrices
Samples of sketching matrices for p = 50 and q = 20.
The color range between -1 (black) and +1 (white)
Randomized Source Sketching

8. 8
Gram matrices associated with the samples of sketching matrices shown in (p = 50 and q = 20).
The color scale ranges between -1 (black) and +1 (white).
Sketching Matrices Randomized Source Sketching

9. 9
The error between identity matrix and truncated expectation of sketching matrices for p = 50
and q = 20 for different level of truncation
Sketching Matrices Randomized Source Sketching
Initialization:
E=0,
For i = 1:N do
Step1: Generate sketch
matrix S
Step2: e = 𝐈 − 𝔼 𝐒𝐒T
F
Step3: E=E+e
End

10. 10
Full waveform inversion: Augmented Lagrangian framework (IR-WRI)
ADMM iteration (Aghamiry et al., 2019a )
Data Assimilated System
Computationally Challenging
FWI (Augmented Lagrangian) Randomized Source Sketching
 Increasing the convergence rate (Decreasing the
total number of PDE solved) (Aghazade et. al, 2021)
 Decreasing the number of RHS terms at each
iteration (Randomized source sketching) (Aghazade
et al., 2021)

11. 11
Randomized Source Sketching
for Full Waveform Inversion
IF
AL function in original domain
Let:
AL function in transform domain
Sketching matrices
Source Sketched FWI Randomized Source Sketching
Because use a subset
of sketching matrices.

12. 12
Transform the original source/data
into randomized sketch domain
Back transform the dual variables
into original domain
Source Sketched FWI Randomized Source Sketching

13. 13
True models Initial models
A) SEG/EAGE 2D Overthrust model:
• Number of grids: 187 ×801
• Grid interval: 25 m
• Number of sources: p = 134
• Number of receivers: m = 801
• Inverted Frequencies: [3-6.5] Hz and
[3-13] Hz with a frequency interval of
0.5 Hz
B) 2007 BP model:
• Number of grids: 151 × 1050
• Grid interval: 75 m
• Number of sources: p = 263
Number of receivers: m = 1050
• Inverted Frequencies:
• [1-2] Hz, [1-3.5] Hz and [3-4] Hz
with a frequency interval of 0.5 Hz
Numerical Examples Randomized Source Sketching

14. 14
Case (i)
Number of sources: 134
Sketched sources: 2
Without Regularization
Numerical Examples Randomized Source Sketching
With Regularization
Reference
Example1:
Overthrust model

15. 15
Case (ii)
Number of sources: 134
Sketched sources: 10
Numerical Examples Randomized Source Sketching
Case (iii)
SNR=10 dB
Number of sources: 134
Sketched sources: 10
Example1:
Overthrust model
Noise-free data
Noisy data
Reference

16. 16
The MSE curve versus q for different methods
Accuracy analysis
Numerical Examples Randomized Source Sketching
Reference
Example1:
Overthrust model

17. 17
Case (i)
Number of sources: 263
Sketched sources: 2
Without Regularization
Numerical Examples Randomized Source Sketching
With Regularization
Aghamiry et al (2019b, 2021)
Reference
Example2:
BP 2007 model

18. 18
Case (ii)
Number of sources: 263
Sketched sources: 10
Numerical Examples Randomized Source Sketching
Example2:
BP 2007 model
Without Regularization
Reference
With Regularization

19. 19
Total number of PDE solved and speed up (in
percent) for the deterministic IR-WRI and its randomized
source sketching variant
Speed up gain
Numerical Examples Randomized Source Sketching
Castellanos et al (2015)
Less PDE solved, High speed up

20. 20
1) We developed a general algorithm based on randomized source
sketching to efficiently solve any PDE based problem with multi-right
hand side terms.
2) This randomized scheme admits the existing encoding
methods as special cases.
3) We demonstrate with numerical examples that the randomized
algorithm provides improved performance over existing deterministic
algorithms.
4) The method is very straightforward which honors both accuracy and
computational efficiency.
Conclusion
Randomized Source Sketching

21. Randomized Source Sketching 21
With Regularization
Example 3
BP2004 model

22. 22
[1] H. S. Aghamiry, A. Gholami, and S. Operto, “Improving full-waveform inversion by wavefield reconstruction with
the alternating direction method of multipliers,” Geophysics, vol. 84, no. 1, pp. R139–R162, 2019
[2] H. S. Aghamiry, A. Gholami, and S. Operto, “Compound regularization of full-waveform inversion for imaging
piecewise media,” IEEE Transactions on Geoscience and Remote Sensing, vol. 58, no. 2, pp. 1192–1204, 2019
[3] H. Aghamiry, A. Gholami, and S. Operto, “Full Waveform Inversion by Proximal Newton Methods using Adaptive
Regularization,” Geophysical Journal International, vol. 224, no. 1, pp. 169–180, 2021. [Online]. Available:
https://doi.org/10.1093/gji/ggaa434
[4] K. Aghazade, A. Gholami, H.S. Aghamiry, and S. Operto., Anderson accelerated augmented Lagrangian for
extended waveform inversion, Geophysics, 87 (1) 1-13.
[5] C. Castellanos, L. Metivier, S. Operto, R. Brossier, and J. Virieux, ´“Fast full waveform inversion with source
encoding and second-order optimization methods,” Geophysical Journal International, vol. 200, no. 2, pp. 720–744,
2015
[6] H. Ben-Hadj-Ali, S. Operto, and J. Virieux, “An efficient frequencydomain full waveform inversion method using
simultaneous encoded sources,” Geophysics, vol. 76, no. 4, pp. R109–R124, 2011.
[7] E. Haber, U. M. Ascher, and D. Oldenburg, “On optimization techniques for solving nonlinear inverse problems,”
Inverse problems, vol. 16, no. 5, p. 1263, 2000
[8] S. Wang, “A practical guide to randomized matrix computations with matlab implementations,” arXiv preprint
arXiv:1505.07570, 2015
Randomized Source Sketching
References