The slides of my Ph.D. defense

Efficient geometric random walks
for high-dimensional sampling from convex bodies
Apostolos Chalkis
Ph.D. supervisor: Prof. Ioannis Z. Emiris
National & Kapodistrian U. Athens, Greece
December 20, 2021

Problem: Truncated log-concave sampling
Definition. Let π(x) ∝ e−f (x), where f : Rd → R is a convex
function. π(x) is called log-concave (LC) probability density.
We consider the case where π(x) is restricted in a
convex body K ⊂ Rd .
Important examples: Uniform, Gaussian, Boltzmann.

Why is sampling an important problem?
Several problems can be reduced to sampling:
Convex optimization [Ma,Chen,Flammarion,Jordan’19]
Volume approximation [M. Dyer, A. Frieze, and R. Kannan’91]
Multivariate integration [Lovasz,Vempala’06]
Bayesian inference [Gelman, Carlin, Stern, Dunson, Rubin’95]
Sampling also appears in plenty of applications

Why geometric random walks?
Alternative methods:
Acceptance rejection sampling [Casella,Christian,Wells’04]
Adaptive Rejection sampling [Gilks,Wild’92]
Slice sampling [Neal’03]
Inverse transform sampling [Olver,Townsend’13]
typically, do not scale beyond dimension d ≥ 20.

Geometric Random Walks
A Geometric Random Walk starts at some interior point and at
each step moves to a ”neighboring” point, chosen according to
some distribution depending only on the current point.
Implementation of Billiard Walk
step.
Uniform sampling via the
Billiard Walk.

Goals of my Ph.D.
Problem: Sample from a LC distribution restricted to a
convex body
Algorithmic and complexity results.
Efficient implementations of geometric random walks.
Practical, randomized algorithms to address challenging
problems in:
– Computational geometry
– Computational Finance
– Systems biology

Summary of results
Support and develop random walks to sample from
log-concave distributions restricted in:
H-polytopes
V-polytopes
Zonotopes (Z-polytopes)
Spectrahedra
Applications of log-concave sampling:
Volume approximation of convex bodies
Systems biology (metabolic network analysis)
Finance (crisis detection, portfolio optimization/scoring)
Convex optimization
Open-source software:
C++/R package volesti
github.com/GeomScale/volesti
python package dingo
github.com/GeomScale/dingo

Open-source internships
Google Summer of Code:
State-of-the-art geometric random walks for sampling from
high dimensional bodies in R, 2018.
State-of-the-art algorithms in R for volume computation and
sampling in high dimensions, 2019.
Tweag - Software Innovation Lab:
Develop dingo, a Python package to analyze metabolic
networks, 2021.

GeomScale org
– GeomScale/volesti
volume approximation & sampling
from convex bodies
[C,Fisikopoulos’21]
– GeomScale/dingo
analyze metabolic networks with
MCMC sampling
Co-founders: V. Fisikopoulos & E. Tsigaridas
————————————————————————————
One of the 200 mentoring organizations supported by
iii Google Summer of Code (GSoC) 2020 & 2021.
Mentor 8 GSoC projects.
NumFOCUS Affiliated Project.
Support an open community (>130 members).
More than 15 000 lines of code.

Part II
Convex bodies &
important random walks

Convex polytope representations
H-polytope: P = {x | Ax ≤ b, A ∈ Rm×d , b ∈ Rm}.
P is given as a set of linear inequalities.
P is the feasible set of a linear program.

V-polytope: the convex hull of a set of points in Rd .
Applications:
Systems biology (metabolic networks) [Caso,Montañez’13]
Biogeography
[Barnagaud,Kissling,Tsirogiannis,Fisikopoulos,Villéger,Sekercioglu,Svenning’17]

Z-polytope: Minkowski sum of k d-dimensional segments.
A zonotope is a centrally symmetric convex body.
Applications:
Autonomous driving [Althoff,Dolan’14]
Human-robot collaboration [Pereira,Althof’15]
Neural networks [Anderson,Pailoor,Dillig,Chaudhuri’19]

Spectrahedron
A spectrahedron S ⊂ Rd is the feasible set of a linear matrix
inequality. If Ai are symmetric matrices in Rm×m and
F(x) = A0 + x1A1 + · · · + xd Ad ,
then S = {x ∈ Rd | F(x) 0}.
A 3D elliptope
S is the feasible set of a Semidefinite Program (SDP)

Random walks for
truncated log-concave sampling
Year Authors Random walk Mixing time∗
Distribution
[Smith’86] Hit-and-Run O∗
(d3
) any LC
[Berbee,Smith’87] Coordinate Hit-and-Run O∗
(d10
) any LC
[Lovasz,Simonovits’90] Ball walk O∗
(d3
) any LC
[Kannan,Narayanan’12] Dikin walk O∗
(d2
) uniform (H-polytope)
[Polyak,Dabbene’14] Billiard walk ?? uniform
[Afshar,Domke’15] Reflective HMC ?? any LC (polytopes)
[Lee,Vempala’16] Geodesic walk O(md3/4
[Lee,Vempala’17] Remannian HMC O∗
(md2/3
) uniform (H-polytopes)
[Chen,Dwivedi,Wainwright,Yu’19] John walk O∗
(d5/2
[Chen,Dwivedi,Wainwright,Yu’19] Vaidya walk O(m1/2
d3/2
Table 1
Cost per sample: cost per step × mixing time (#steps).
The cost per step depends on the convex body.
Hit-and-Run (HR): widely used well studied.
Coordinate Hit-and-Run (CDHR): has been proven more
efficient than HR in practice.
Existing software uses either CDHR or HR (H-polytopes).
∗
total variation mixing time

Contributions
For Reflective Hamiltonian Monte Carlo (ReHMC):
We prove that it converges to the target distribution when
the latter is truncated by a spectrahedron.
For both (ReHMC) and Billiard walk:
We improve the cost per step for H-polytopes.
We provide efficient operations for V-, Z-polytopes and
spectrahedra.
We offer efficient parameterizations.
We experimentally show that they outperform all the random
walks in Table 1.

Hit-and-Run(K, x, π): Convex body K ⊂ Rd , point x ∈ K,
PDF π : Rd → R+
1 Pick uniformly a line ` through x.
2 return a random point on the chord ` ∩ K chosen from
the distribution π`,π restricted in ` ∩ K.
P
x
B
`

Hit-and-Run(K, x, π): Convex body K ⊂ Rd , point x ∈ K,
PDF π : Rd → R+
1 Pick uniformly a line ` through x.
P
`
x

Coordinate Hit-and-Run(K, x, π): Convex body K ⊂ Rd ,
point x ∈ K, PDF π : Rd → R+
1 Pick a uniformly random axis direction ei .
2 Consider the line ` through x with direction ei .
`
p q

Hamiltonian Monte Carlo
Being at p ∈ K, HMC introduces an auxiliary random variable
v ∈ Rd and generates samples from the joint density
π(p, v) = π(v|p)π(p),
Marginalize out v, then recover the target dist. π(p).
Consider v ∼ N(0, Id );
PDF π(p, v) = e−H(p,v) defines a Hamiltonian,
H(p, v) = − log π(p, v) = − log π(p) +
1
2
|v|2
,

Hamiltonian Monte Carlo
HMC simulates a particle moving in a conservative field
determined by − log π(p) and −∇ log π(p).
HMC, starting from a position p, generates a new state:
1 Draw a value for the momentum, v ∼ N(0, Id )
2 (p, v) is given by the Hamilton’s system of ODE:
dp
dt
=
∂H(p, v)
∂v
dv
dt
= −
∂H(p, v)
∂p
⇒





dp(t)
dt = v(t)
dv(t)
dt = −∇ log π(p)
(1)
Solve (1) with
–Euler methods (e.g., Leapfrog) [Neal’12] or,
–Collocation method [Vempala,Lee,Song’18].

Reflective Hamiltonian Monte Carlo (ReHMC)
When the density is restricted in a convex body K then HMC
trajectory stays inside K by using boundary reflections.
Case of Leapfrog method
π(x) Discrete Hamiltonian
trajectory
We pre-select the number of Leapfrog steps
[C,Fisikopoulos,Papachristou,Tsigaridas’21]

Reflective Hamiltonian Monte Carlo (ReHMC)
When the density is restricted in a convex body K then HMC
trajectory stays inside K by using boundary reflections.
Case of collocation method
π(x)
Polynomial Hamiltonian
trajectory
We randomly select the integration time in each steps

Billiard walk - Uniform case
BW(P, x): Convex body K ⊂ Rd
, point x ∈ K
1 Generate the length of the trajectory L ∼ D.
2 Pick a uniform direction v to define the trajectory.
3 The trajectory reflects on the boundary if necessary.
4 return the end of the trajectory as pi+1.

Geometric and algebraic oracles
The implementation of a geometric random walk requires:
Membership oracle
Boundary (intersection) oracle
Reflection oracle

Membership oracle
membership(F, p): An LMI F(x) 0 ⇔ A0+x1A1+· · ·+xd Ad 0
representing a spectrahedron S and a point p ∈ Rd
.
1 λmin ← smallest eigenvalue of F(p).
2 if λmin ≥ 0 return true else return false.

Boundary oracle
intersection(F, Φ(t)): An LMI F(x) 0 ⇔ A0 + x1A1 + · · · +
xd Ad 0 representing a spectrahedron S,
Φ : t 7→ Φ(t) := (p1(t), . . . , pd (t)) parameterization of a polynomial
curve, where pi (t) =
Pni
j=0 pi,j tj
, and Φ(0) ∈ S.
1 Solve the polynomial eigenvalue problem
F(Φ(t)) x = 0 ⇔ (B0 + tB1 + · · · + td
Bd )x = 0,
where Bk =
Pd
j=1 pj,k Aj
2 Smallest positive and largest negative eigenvalues λ−
max , λ+
min
3 return the boundary points F(Φ(λ−
max )) and F(Φ(λ+
min))

Reflection oracle
reflection(F, Φ(t), λ+): An LMI F(x) 0 ⇔ A0 + x1A1 + · · · +
xd Ad 0 representing spectrahedron S,
Φ(t) parameterization of a polynomial curve,
λ+ s.t. Φ(λ+) ∈ ∂S
1 Let the boundary point p+ = Φ(λ+)
2 Let w = ∇ det(F(p+)) = c · (s
A1s, · · · , s
Ad s),
s vector in the kernel of F(p+)
3 return the direction of the reflection
s+ ← dΦ
dt (t+) − 2 h∇dΦ
dt (t+), wi w

Per-step complexity
Random walk per-step Complexity
HR O(mω + m log(1/) + dm2)
Coordinate HR O(mω + m log(1/) + m2)
Billiard walk e
O(ρ(mω + m log(1/) + dm2))
ReHMC (collocation) e
O(ρ((nm)ω + mn log(1/) + dnm2))
ReHMC (leapfrog) e
O(Lρ(mω + m log(1/) + dm2))
[C,Emiris,Fisikopoulos,Repouskos,Tsigaridas’20]
m: size of the matrices Ai in LMI
d: dimension
n: degree of the polynomial curve
ρ: number of reflections
: accuracy to approximate the intersection with the boundary
ω: exponent in the complexity of matrix multiplication
L: number of leapfrog steps

Optimization - Semidefinite Programs
In an SDP we minimize a linear function of a variable x ∈ Rd
subject to an Linear Matrix Inequality (LMI):
min cT
x
subject to F(x) = A0 + x1A1 + · · · + xnAn 0
The feasible set is a spectrahedron S = {x ∈ Rd | F(x) 0}.
Semidefinite programming generalizes Linear programming.

Optimization via Exponential sampling
Problem: Minimize a linear function f (x) = cT x in
spectrahedron S.
Answer: Sample from π(x) ∝ e−cT x/T restricted in S, for
T = T0 · · · TM.
T0 T1 T2 T3
Task: Compute a sequence of Ti ∈ R+ of length M s.t. a
sample from πTM
is close to the optimal solution
with high probability.

Simulated Annealing
Convergence to the optimal solution
πi (x) ∝ e−cT x/Ti
Starting with T0 = R, where S ⊂ RBd (uniform distribution).
Ti = Ti−1(1 − 1
√
d
), i ∈ [M] (Ti−1 is a warm start for Ti ).
M = O∗(
√
d) phases to obtain a solution |fM − f ∗| ≤
Only Hit-and-Run has been used in previous work
[Kalai,Vempala’06].

Hit-and-Run Vs ReHMC
ReHMC is faster than Hit-and-Run.

Faster scheduling
In practice, we speedup the temperature schedule by setting,
Ti = T0

1 −
1
dk
i
, i ∈ [I]. (2)
In theory, k = 1/2 (Hit-and-Run).
Experimentally, for ReHMC, Ti results to a warm starting
point for Ti+1, when k 1/2.

sdpa Vs volesti
Random generator of Spectrahedra [Polyak,Dabbene’14].
Experimental branch in GeomScale/volesti [C,Fisikopoulos,Tsigaridas]

In our cells...
We call both the inputs (reactants) and the outputs (products)
of a chemical reaction, metabolites.
In every cell of our body
thousands of chemical reactions are taking place!

The reactions interact
A small fragment of the human metabolic network
Q: How can we model all the interactions between chemical
sdf reactions in an organism?
A: Computational geometry can help!

Key Concept of a metabolic network:
Reaction Fluxes
The i-th reaction has a flux (rate) vi that is flowing.
vi multiplies each metabolite in the i-th reaction.

Matrix representation of a metabolic network
11 metabolites and 4 reactions
Use S ∈ Rm×n and flux vector v ∈ Rn to express the
change of the mass of each metabolite over time [Palsson’15],
dr
dt
= Sv = S




v1
v2
v3
v4



 = v1S(·,1) + v2S(·,2) + v3S(·,3) + v4S(·,4)

Steady states: The network in balance
When for each metabolite, the rate of production equals to the
rate of consumption, the reactions exactly balance each other.
When a flux vector v balances the network,
Sv = 0,
v is a steady state.

The region of steady states
As a low dimensional polytope.
Sv = 0,
vlb ≤ v ≤ vub ←→
v=Nx
S ∈ Rm×n
, v ∈ Rn
As a full dimensional polytope
P := {x ∈ Rd | Ax ≤ b}
N ∈ Rn×d the matrix of the right nullspace of S.

Sampling steady states
Sampling could lead to important biological insights [Palsson’15].
Explore the flux space [Schellenberger,Palsson’09].
We introduce a
Multiphase Monte Carlo Sampling algorithm
based on Billiard Walk

Difficulties - skinny polytopes
When the polytope is skinny:
The average number of reflections increases.
The mixing rate decreases.

Multiphase Monte Carlo Sampling
MMCS(P0, p, N, set i = 0)
1 Sample O(d) points from Pi with Billiard walk.
2 Estimate the Effective Sample Size (ESS) of the sample
in Pi .
3 Map the sampled points to an isotropic position and
apply the same transformation Ti to Pi , set
Pi+1 = Ti (Pi ).
4 i = i + 1; goto 1.
5 Stop when the sum of ESS N and PSRF 1.1.
Related work
(optimization)
1. [Bertsimas et al.’04]
2. [Kalai et al.’06]

Experiments
MMCS cobra
model (d) Time (sec) (Steps) Time (sec) (Steps)
iAB RBC 283 130 5.20e+01 1.07e+04 7.85e+03 4.05e+08
iAT PLT 636 289 3.25e+02 1.04e+04 1.73e+04 6.68e+08
iML1515 633 4.65e+03 5.65e+04 1.15e+05 3.21e+09
Recon1 931 8.09e+03 1.94e+04 3.20e+05 6.93e+09
Recon2D 2430 2.48e+04 5.44e+04 ∼ 140 days 1.57e+11
Recon3D 5335 1.03e+05 1.44e+05 – –
[C,Fisikopoulos,Tsigaridas,Zafeiropoulos’21]
Package cobra [Palsson, Thiele, Fleming et al.’19]:
state-of-the-art software to analyze metabolic networks,
Coordinate Directions Hit-and-Run [Cousins, Vempala et al.’17].
MMCS outperforms existing software.

Find possible anti-COVID19 targets
Not an anti-viral target Possible anti-viral target
Sample steady states when,
the growth rate of COVID-19 is optimized,
the host biomass production is optimized.
Check if the flux distribution of a reaction changes
[Renz,Widerspick,Dräger’20,’21].
[Open-source internship
Tweag’21]
Joint work:
[Tweag] Cheplyaka, Carstens
[GeomScale] Fisikopoulos, Tsigaridas,
ii Zafeiropoulos

Complexity
Computing the exact volume of P,
is #P-hard for all the representations [DyerFrieze’88]
is open if both H- and V- representations available
is APX-hard (oracle model) [Elekes’86]

Randomized approximation algorithms
Multiphase Monte Carlo
Theorem [Dyer, Frieze, Kannan’91]
For any convex body P and any 0 ≤ , δ ≤ 1, there is a
randomized algorithm which computes an estimate V s.t. with
probability 1 − δ we have (1 − )vol(P) ≤ V ≤ (1 + )vol(P),
and the number of oracle calls is poly(d, 1/, log(1/δ)).
Using randomness, we can go from an exponential
approximation to an arbitrarily small one.

State-of-the-art
Authors-Year
Complexity
random walk
(oracle calls)
[Dyer, Frieze, Kannan’91] O∗(d23) grid walk
[Kannan, Lovasz, Simonovits’97] O∗(d5) ball walk
[Lovasz, Vempala’03] O∗(d4) hit-and-run
[Cousins, Vempala’15] O∗(d3) ball walk
Can not be implemented as they are due to large constants in
the complexity and pessimistic theoretical bounds.
Practical algorithms:
Follow theory but make practical adjustments (experimental).
[Emiris, Fisikopoulos’14] Sequence of balls + coordinate
hit-and-run.
[Cousins, Vempala’16] Spherical Gaussians + hit-and-run

Practical Multiphase Monte Carlo scheme
Let Cm ⊆ · · · ⊆ C1 a sequence of scaled copies of a body C
intersecting P, s.t. Cm ⊆ P ⊆ C1 [C,Emiris,Fisikopoulos’21].
vol(P) = vol(P ∩ Cm) · vol(P∩Cm−1)
vol(P∩Cm)
· · · vol(P∩C1)
vol(P∩C2)
· vol(P)
vol(P∩C1)

Ratio estimation
Estimate ri = vol(P∩Ci+1)
vol(P∩Ci )
within some target relative error i .
Sample N uniform points from Pi = Ci ∩ P and count points
in Pi+1 = Ci+1 ∩ P ⊆ Pi .
Keep each ratio bounded, then N = O(1/2
i ) points suffices.
Use Billiard walk to sample uniformly from each body.

Keep volume ratios bounded
Statistical tests
Given convex bodies Pi ⊇ Pi+1, we define two statistical tests:
[U-test(Pi , Pi+1)] H0: vol(Pi+1)/vol(Pi ) ≥ r + δ
[L-test (Pi , Pi+1)] H0: vol(Pi+1)/vol(Pi ) ≤ r
The U-test and L-test are successful iff both H0 are rejected.
If both U-test and L-test are successful then
ri = vol(Pi+1)/vol(Pi ) ∈ [r, r + δ], with high probability.

Experiments
Total number of generated points: ∼ d2 (experimental).
Our method outperforms existing implementations

Experiments
We compute the volume of Birkhoff polytope for n 15 for
the first time.
Bn d Exact Vol. Asympt. Vol. Est. Vol m steps time (sec)
B10 81 8.78e-46 9.81e-46 8.56e-46 13.1 4.69e+04 1.9
B20 361 ?? 4.23e-312 3.31e-312 94.9 8.46e+05 297
B25 576 ?? 6.46e-554 5.56e-554 158.8 1.82e+06 1 060
B30 841 ?? 1.60e-875 1.35e-875 249.1 3.53e+06 4 982
B33 1024 ?? 1.56e-1108 1.37e-1108 314.1 4.97e+06 11 105
We perform computations up to thousands of dimensions for
the first time.
Polytope Vol. error m steps time
cube-500 3.38e+150 0.03 84.7 7.09e+05 458
cube-1000 2.11e+301 0.03 201.0 2.70e+06 6 180
simplex-500 7.55e-1135 0.08 149.1 1.65e+06 847.8
simplex-1000 2.67e-2568 0.07 328.4 5.34e+06 10 711
product-simplices-250-250 9.10e-986 0.05 151.3 1.69e+06 958.5
product-simplices-500-500 6.22e-2269 0.07 332.5 5.39e+06 9 432
iSDY 1059 [2966-509] 2.25e-350 ?? 136.0 1.43e+06 3 486
Recon1 [4934-931] 8.01e-5321 ?? 266.1 3.84e+06 18 225

Part VI
Computational Geometry Finance
Crises detection in stock markets
Portfolio strategies
Portfolio scoring

Crisis detection
Financial markets exhibit 3 types of behavior e.g.,
[Billio, Getmansky,Pelizzon’12]:
1 In normal times, stocks are characterized by slightly positive
returns and a moderate volatility.
2 In up-market times (typically bubbles) by high returns and low
volatility.
3 During financial crises by strongly negative returns and high
volatility.
These observations motivate us to describe the time-varying
dependency between portfolios’ returns and volatility.

Geometric representation of portfolios
Set of portfolios
{x ∈ Rd
|
P
i xi = 1, xi ≥ 0}
The set of portfolios can be seen
as the simplex/polytope in Rd
.
Ptf x= (0.3, 0.25, 0.45)

Portfolio return and volatility
Ptf x= (0.3, 0.25, 0.45)
– Asset returns R ∈ Rd
– R = (0.09, 0.13, −0.038)
– Return Rx = RT
x = 18%
– Covariance matrix Σ ∈ Rd×d
– Σ =


3.66 0.25 −0.66
0.25 6.36 0.50
−0.66 0.50 0.18


– Volatility Vx = xT
Σx = 73%

Computing the copula of a given time period
– Cut the simplex with bothqqqa
q hyperplanes and ellipsoids
– Compute the volume of the q
d bodies defined by the qdfxcvxc
d intersections
→
– Obtain a Copula
– Bivariate distribution
– Each marginal is uniform

Q: How to detect a crisis?
1st September 1999 1st September 2000
during dot-com bubble bubble burst
Left Copula corresponds to normal and right copula to crises times.
⇒ Normal times: The mass of portfolios on the up diagonal.
⇒ Crisis times: The mass of portfolios on the down diagonal.

Answer: Define an indicator I
I :=
mass in red area
mass in blue area

Detects past financial crises
Daily returns from Europe DJ 600 from 01/01/1990 to 31/11/2017
I 1 for 61-100 days , over 100 days
[Calès,C,Emiris,Fisikopoulos’18]
1. May ’90 - Dec. ’90: early 90’s recession.
2. May ’00 - May ’01: dot-com bubble burst.
3. Oct. ’01 - Apr. ’02: the stock market downturn of 2002.
4. Nov. ’05 - Apr. ’06: not listed.
5. Dec. ’07 - Aug. ’08: recent crisis.

Portfolio managers compute and propose portfolio allocations.
An investor decides which asset allocation proposal to select
and how much to modify it.
To model this procedure, we employ log-concave distributions

Let π be an LC distribution supported on the portfolio domain,
Then, Fπ is a portfolio allocation strategy:
“To build a portfolio with strategy Fπ sample a
point/portfolio from π”

Mixed strategy
Let π1, . . . , πM be a sequence of LC distributions.
We call Fπ the mixed strategy;
π(x) =
PM
i=1 wi πi (x) a mixture density;
wi ≥ 0,
PM
i=1 wi = 1.
Weight wi the proportion of the investors that build their
portfolios according to Fπi .

A new portfolio score
For given asset returns R ∈ Rd over a single period of time, the
score of a portfolio, providing a value of return R∗, is
s =
Z
P
g(x)π(x)dx, g(x) =

1. if RT x ≤ R∗,
0, otherwise.
(3)
The score s corresponds to the expected proportion of
portfolios that an allocation outperforms when the portfolios
are invested according to the mixed strategy Fπ.
Theorem. We can estimate the score within accuracy after
O∗(Md4/2) operations.

Portfolio strategies - special case
equal risk different risk
πα,q ∝ e−αφq(x), φ = xT Σx − qµT x ∗
µ: mean of assets’ returns
Σ: covariance matrix of assets’ returns
α: controls dispersion around manager’s proposal
q: controls the risk of the investment
∗Quadratic utility function in original Markowitz’s problem [’79,’84]; Nobel Prize in economics ’90.

Parametric score
[C,Christoforou,Dalamagas,Emiris’21]
We introduce behavioral functions to model how the investors
tend to behave (set the parameters α and q and the bias
vector r).
Parametric score, s(T) =
R
S hπ(w)e(rT w)/T , T 0.
T → ∞: investors are equally divided among strategies.
T → 0: all investors follow a single strategy.

The slides of my Ph.D. defense

Recommended

Recommended

More Related Content

What's hot

What's hot (10)

Similar to The slides of my Ph.D. defense

Similar to The slides of my Ph.D. defense (20)

Recently uploaded

Recently uploaded (20)

The slides of my Ph.D. defense