This document discusses methods for sampling and optimization within spectrahedra. Spectrahedra are the feasible sets of semidefinite programs and arise in applications like control theory and statistics. The document presents random walk algorithms like billiard walk and Hamiltonian Monte Carlo that can be used to sample from spectrahedra. It also discusses how exponential sampling via simulated annealing can be applied to optimize linear functions over spectrahedra.

1. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Sampling Spectrahedra:
Volume Approximation and Optimization
Apostolos Chalkis (NKUA),
Vissarion Fisikopoulos (NKUA), Elias Tsigaridas (INRIA)

2. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Why spectrahedra are important? 2/1
Feasible set of Semidefinite Programs (SDPs),
Bayesian statistics (sampling and volume estimation),
Control theory,
Combinatorial optimization (bounds of NP-hard problems),
Optimal experiment design (VLSI).

3. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Spectrahedra 3/1
Definition
A spectrahedron S ⊂ Rn is the feasible set of a linear matrix
inequality. If Ai are symmetric matrices in Rm×m and
F(x) = A0 + x1A1 + · · · + xnAn,
then S = {x ∈ Rn | F(x) 0}.
A 3D spectrahedron.

4. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Developing practical randomized methods 4/1
Sampling Spectrahedra
We address both solving SDPs and approximating the volume,
using sampling.
To sample we use geometric random walks.

5. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Random Walks 5/1
A Geometric Random Walk
starts at some interior point and at each step moves to a
”neighboring” point, chosen according to some distribution
depending only on the current point.
We employ
m Ball Walk (truncated MH)
[Vempala, ’05]
m Billiard Walk (uniform
dist.) [Gryzina, Polyak, ’05]
m Hamiltonian Monte
Carlo (HMC with
reflections) [Lee et al., ’18]
m Hit Run (most
common) [Smith, ’84]
Uniform sampling via the
Billiard Walk.

6. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Uniform sampling - Billiard Walk 6/1
A step of Billiard Walk:
À Pick uniformly a direction
and the total length of
the step.
Á Reflect the trajectory
when it hits the boundary.
 Repeat.
It converges asymptotically to the
uniform distribution [Polyak, Gryazina, ’14].

7. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Exponential sampling - HMC with reflections 7/1
A step of Hamiltonian Monte
Carlo with reflections:
À Pick uniformly a direction
as the tangent of a
quadratic polynomial
trajectory and the total
length of the step.
Á Reflect the trajectory
when it hits the boundary.
 Repeat.
It converges asymptotically to the
exponential distribution [Pion,Cazals’20].

8. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Random Walks in Spectrahedra 8/1
Each step of a geometric
random walk is based on
membership / boundary
oracles.
Specializing a random walk
for spectrahedra requires
geometric operations.

9. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Contributions on sampling 9/1
n Only HitRun has been used to sample spectrahedra
[Calafiore, ’04]
ä We develop geometric operations / oracles to employ
several random walks.
ä We give per-step complexity to sample from any
log-concave distribution restricted to a spectrahedron.
ä Our C++ implementation is open source and extends the
functionality of volesti1.
1
github.com/GeomScale/volume_approximation

10. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Geometric Operations on Spectrahedra 10/1
In general we consider
polynomial curves
intersecting the boundary
We compute the
intersection point p+
We compute the reflection
at p+
Computation of p+
requires solving a
polynomial eigenvalue
problem [Tisseur, ’00]
P(λ)u = 0,
P(λ)=B0 + λB1 + · · · + λd Bd

11. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Optimization - Semidefinite Programs 11/1
In an SDP we minimize a linear function of a variable x ∈ Rn
subject to an Linear Matrix Inequality (LMI):
min cT
x
subject to F(x) = A0 + x1A1 + · · · + xnAn 0
Ai ∈ Rm×m are symmetric matrices.
The feasible set is a spectrahedron.
Semidefinite programming generalizes Linear programming.

12. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Optimization via Uniform Exponential sampling
12/1
Problem: Minimize a linear function f (x) = cT x in
spectrahedron S.
Answer: Sample from πT (x) ∝ e−cT x/T restricted in S, for
T = T0 · · · TN.
T0 T1 T2 T3
Task: Find a sequence of Ti ∈ R+ of length N s.t. a
sample from πTN
is close to the optimal solution
with high probability.

13. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Simulated Annealing 13/1
Fix the sequence of Temperatures
T0 T1 T2 T3
Only HitRun has been used in previous work
[Kalai,Vempala’06].
The sequence T0 · · · TI is fixed s.t. the L2 norm of
πTi
w.r.t. πTi+1
is bounded by a constant,
||πTi
/πTi+1
|| = EπTi
dπTi
dπTi+1
= O(1)
Then a sample from πTi
is a warm start for πTi+1
.

14. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Simulated Annealing 14/1
Convergence to the optimal solution
Starting with T0 = R (uniform distribution).
Ti = T0(1 − 1
√
d
)i , i ∈ [I] (Ti is a warm start for Ti+1).
Knowing that for a temperature T,
EπT
[cT
x] ≤ dT + min
x∈K
cT
x
I = O∗(
√
d) phases to obtain a solution |fI − f ∗| ≤
[Kalai,Vempala’06].

15. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Hit-and-Run Vs Hamiltonian Monte Carlo 15/1
Hamiltonian Monte Carlo with reflections is faster than
HitRun.

16. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Faster scheduling 16/1
In practice, we speedup the temperature schedule by setting,
Ti = T0
1 −
1
dk
i
, i ∈ [I]. (1)
In theory, k = 1/2 (HitRun).
For HMC, Ti results to a good starting point for Ti+1,
when k 1/2.

17. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
sdpa Vs volesti 17/1
Random generator of Spectrahedra [Polyak,Dabbene’14].

18. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Volume approximation 18/1
Computing the exact volume of a polytope,
is #P-hard for all the representations [DyerFrieze’88]
is open if all representations available
is APX-hard (oracle model) [Elekes’86]

19. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Randomized approximation algorithms 19/1
Multiphase Monte Carlo
Theorem
[Dyer, Frieze, Kannan’91] For any convex body P and any
0 ≤ , δ ≤ 1, there is a randomized algorithm which computes
an estimate V s.t. with probability 1 − δ we have
(1 − )vol(P) ≤ V ≤ (1 + )vol(P), and the number of oracle
calls is poly(d, 1/, log(1/δ)).
Using randomness, we can go from an exponential
approximation to an arbitrarily small one.

20. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
State-of-the-art 20/1
Authors-Year Complexity random walk
[Dyer, Frieze, Kannan’91] O∗(d23) grid walk
[Kannan, Lovasz, Simonovits’97] O∗(d5) ball walk
[Lovasz, Vempala’03] O∗(d4) hit-and-run
[Cousins, Vempala’15] O∗(d3) ball walk
Can not be implemented as they are due to large constants
in the complexity and pessimistic theoretical bounds.
Practical algorithms:
Follow theory but make practical adjustments
(experimental).
The existing implementations can be applied only for
polytopes.

21. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Multiphase Monte Carlo 21/1
We employ billiard walk and the algorithm in
[C, Emiris, Fisikopoulos, ’19].
Let Cm ⊆ · · · ⊆ C1 a sequence of concentric balls
intersecting P, s.t. Cm ⊆ P ⊆ C1.
Construct a sequence of balls intersecting P, then:
vol(P) = vol(P ∩ Cm)
vol(P ∩ Cm−1)
vol(P ∩ Cm)
· · ·
vol(P ∩ C1)
vol(P ∩ C2)
vol(P)
vol(P ∩ C1)

22. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Approximating Volume 22/1
S-n-m µ ± tα,ν−1
s
√
ν
#Points Time (sec)
S-60-60 (1.23 ± 0.11)e-20 20370.9 28.5
S-80-80 (4.24 ± 0.26)e-33 31539.1 124.4
S-100-100 (1.21 ± 0.10)e-51 52962.7 362.3
10 experiments per S-n-m; µ average volume; #Point average
number of generated points. Second column: 95% confidence
interval.

23. Sampling
Spectrahedra:
Volume
Approximation
and
Optimization
Apostolos
Chalkis
(NKUA),
Vissarion
Fisikopoulos
(NKUA), Elias
Tsigaridas
(INRIA)
Open code - open science 23/1
Our methods are implemented in package volesti of
Geomscale Org.
github.com/GeomScale/volume_approximation.
geomscale.github.io/.
Thank you!