The document discusses scenario reduction techniques for approximating a discrete probability distribution with fewer support points. It introduces proximity measures between distributions, analyzes the tradeoff between accuracy and tractability of approximations, and derives worst-case error bounds. Both discrete and continuous scenario reduction problems are considered, with proofs that the approximation error of discrete reduction is within a constant factor of continuous reduction. Different scenario reduction algorithms are presented and tested on a color quantization problem.

1. Scenario Reduction Revisited:
Fundamental Limits and Guarantees
Napat RUJEERAPAIBOON
(joint work with K. Schindler, D. Kuhn, W. Wiesemann)
Risk Analytics and Optimization Chair
´Ecole Polytechnique F´ed´erale de Lausanne

2. Objectives
Ÿ Approximate a discrete probability distribution by another with
fewer support points.
Optimization

3. Objectives
Ÿ Approximate a discrete probability distribution by another with
fewer support points.
Ÿ Choose a representative sample from a population.
Optimization Clustering Facility Location

4. Motivation from Stochastic Optimization
min c x
s.t. x € X
P
¡
Ax ¥ bp˜ξq
©
¥ 1 ¡
CCP
min c x EP
¡
Qpx, ˜ξq
©
s.t. x € X
SP
Ÿ Both are hard optimization problems.
Ÿ Assuming P is approximated by
ˆPn 1
n
°n
i1 δξi
,
their complexity is directly influenced by n.
Römisch, W. (2003)

5. Outline
Ÿ Proximity measure between probability distributions.
Ÿ Trade-off between accuracy and tractability.
Ÿ Discrete versus continuous scenario reduction.
Ÿ Numerical experiment: color quantization.

6. Proximity Measure
Ÿ Proximity between P and Q is measured by type-2 Wasserstein
distance.
dpP, Qq inf
4
E
¡
}˜ξ ¡ ˜ζ}2
©1{2
: ˜ξ P, ˜ζ Q
B
Ÿ d2
pP, Qq equals the minimum cost of moving P to Q.

7. Proximity Measure
Ÿ Proximity between P and Q is measured by type-2 Wasserstein
distance.
dpP, Qq inf
4
E
¡
}˜ξ ¡ ˜ζ}2
©1{2
: ˜ξ P, ˜ζ Q
B
Ÿ Goal of scenario reduction: given ˆPn and m 3 n, solve
Q € argmin
Q
3
dpˆPn, Qq : |supppQq| m
A
.
Ÿ Results extend to type-1 Wasserstein distance and other norms.

8. Voronoi Partition
Proposition 1
Scenario reduction can be cast as a Voronoi partitioning problem.
min
|supppQq|m
d2
pˆPn, Qq min
tIk u€Ppmq
°m
k1
°
i€Ik
1
n }ξi ¡meanpIk q}2
Proof Sketch (m 3q:

9. Accuracy vs Tractability
Most accurate pm nq
Q ˆPn
No approximation error.
Difficult to handle.
Most tractable pm 1q
Q δ¯ξ
Crude approximation.
Easy to handle.

10. Accuracy/Tractability Trade-Off
Ÿ Quantify worst-case approximation error for each m 1, . . . , n.
Ÿ WLOG, we assume that }ξi } ¤ 1 for all i 1, . . . , n.
Cpn, mq max
}ξi }¤1
min
|supppQq|m
dpˆPn, Qq

11. Worst-Case Approximation Error
Theorem 1
We have the upper bound Cpn, mq ¤
˜
n¡m
n¡1 .
Proof Sketch: Gram matrix (sij ξt
i ξj ) reformulation.
C
2
pn, mq max}ξi }¤1 mintIk u€Ppmq 1
n
°m
k1
°
i€Ik
}ξi ¡meanpIk q}2
¤ 1 ¡ 1
n min}ξi }¤1,sij ξJ
i
ξj
maxtIk u€Ppmq
°m
k1
1
|Ik |
°
i,j€Ik
sij
looooooooooooooooooooooooooooooomooooooooooooooooooooooooooooooon
non-convex program
¤ 1 ¡ 1
n minS©0,sii ¤1 maxtIk u€Ppmq
°m
k1
1
|Ik |
°
i,j€Ik
sij
looooooooooooooooooomooooooooooooooooooon
:fpSqloooooooooooooooooooooooooooomoooooooooooooooooooooooooooon
semidefinite relaxation

12. Worst-Case Approximation Error
Theorem 1
We have the upper bound Cpn, mq ¤
˜
n¡m
n¡1 .
Proof Sketch: Reduction to a linear program.
fpSq : maxtIk u€Ppmq
°m
k1
1
|Ik |
°
i,j€Ik
sij
1 fpSq is convex.
2 fpSq is invariant under any permutation.

13. Worst-Case Approximation Error
Theorem 1
We have the upper bound Cpn, mq ¤
˜
n¡m
n¡1 .
Proof Sketch: Reduction to a linear program.
S
Permuted S
Average S

14. Worst-Case Approximation Error
Theorem 1
We have the upper bound Cpn, mq ¤
˜
n¡m
n¡1 .
Proof Sketch: Reduction to a linear program.
fpSq : maxtIk u€Ppmq
°m
k1
1
|Ik |
°
i,j€Ik
sij
1 fpSq is convex.
2 fpSq is invariant under any permutation.
3 The SDP admits an optimizer S αI β11t.
4 The SDP reduces to an LP.
5 The LP admits an analytical solution.

15. Worst Case and Normality
Theorem 2
The bound Cpn, mq ¤
˜
n¡m
n¡1 is sharp and can be attained if
}ξi } 1 di and ξi ξj arccosp ¡1
n¡1 q di $ j.
1 Deterministic attainment in Rd
,
d ¥ n ¡1.
2 Probabilistic attainment in Rd
,
d Ñ V, if ξi Np0, c
d,nIq.

16. Discrete vs Continuous Scenario Reduction
Ÿ So far, we have imposed no restrictions on supppQq.
continuous discrete
Ÿ Both variants are NP-hard optimization problems.
Ÿ Discrete scenario reduction

17. MILP.1
Ÿ Continuous scenario reduction

18. MINLP.2
1
Heitsch, H. Römisch, W. (2003)
2
Peng, J. Wei, Y. (2007)

19. Discrete vs Continuous Scenario Reduction
Ÿ Much of the literature emphasizes on discrete reduction.3
discrete continuous
Ÿ How does continuous compare to discrete reduction?
Ÿ In terms of approximation error.
Ÿ In terms of computational complexity.
3
Dupaˇcová, J. et al. (2003)

20. Discrete/Continuous Approximation Errors
Ÿ Continuous approximation error:
CpˆPn, mq min
Q
3
dpˆPn, Qq : |supppQq| m
A
Ÿ Discrete approximation error:
DpˆPn, mq min
Q
5
dpˆPn, Qq :
|supppQq| m
supppQq € tξ1, . . . , ξnu
C
Ÿ It immediately follows that 1 ¤ DpˆPn, mq{CpˆPn, mq.

21. Discrete/Continuous Approximation Errors
Ÿ Continuous approximation error:
CpˆPn, mq min
Q
3
dpˆPn, Qq : |supppQq| m
A
Ÿ Discrete approximation error:
DpˆPn, mq min
Q
5
dpˆPn, Qq :
|supppQq| m
supppQq € tξ1, . . . , ξnu
C
Ÿ It immediately follows that 1 ¤ DpˆPn, mq{CpˆPn, mq.
Ÿ Is the ratio DpˆPn, mq{CpˆPn, mq bounded?

22. Discrete/Continuous Approximation Errors
Theorem 3
We have DpˆPn, mq{CpˆPn, mq € r1,
c
2s.
Proof Sketch: The inequality
°
i€Ik
}ξi ¡meanpIk q}2
1
2|Ik |
°
i,j€Ik
}ξi ¡ξj }2
1
2 ¤ 1
|Ik |
°
j€Ik
°
i€Ik
}ξi ¡ξj }2
¥ 1
2
°
i€Ik
}ξi ¡ξi
k
}2
, hi
k € Ik
implies that
°m
k1
°
i€Ik
1
n }ξi ¡meanpIk q}2
¥ 1
2
°m
k1
°
i€Ik
1
n }ξi ¡ξi
k
}2
.

23. Discrete/Continuous Approximation Errors
Theorem 3
We have DpˆPn, mq{CpˆPn, mq € r1,
c
2s.
Proof Sketch: The inequality
°
i€Ik
}ξi ¡meanpIk q}2
1
2|Ik |
°
i,j€Ik
}ξi ¡ξj }2
1
2 ¤ 1
|Ik |
°
j€Ik
°
i€Ik
}ξi ¡ξj }2
¥ 1
2
°
i€Ik
}ξi ¡ξi
k
}2
, hi
k € Ik
implies that
mintIk u€Ppmq
°m
k1
°
i€Ik
1
n }ξi ¡meanpIk q}2
looooooooooooooooooooooooooomooooooooooooooooooooooooooon
C2pˆPn,mq
¥
1
2 mintIk u€Ppmq
°m
k1
°
i€Ik
1
n }ξi ¡ξi
k
}2
loooooooooooooooooooooomoooooooooooooooooooooon
¥D2pˆPn,mq
.

24. Discrete vs Continuous Scenario Reduction
Ÿ Reasons in favor of/against using discrete reduction.
Constant loss.
c
2 is tight.
Ÿ An algorithm with α-approximation guarantee for DpˆPn, mq
provides
c
2α-approximation guarantee for CpˆPn, mq.
Ÿ Both admit exact MILP reformulations.
Ÿ DpˆPn, mq

25. MILP with n binary variables.
Ÿ CpˆPn, mq

26. MILP with nm binary variables.

27. Procedures for Discrete Scenario Reduction
1 Greedy heuristic4
:
Fast (Opn2
q) but no approximation guarantee.
1. Initialize Qp1q δξi . 2. For i 1, . . . , m ¡1, solve
Qpi 1q € argmin
Q
6
98
97
dpˆPn, Qq :
|supppQq| i 1
supppQpiqq € supppQq
supppQq € tξ1, . . . , ξnu
D
GF
GE
.
3. Output Q Qpmq.
4
Dupaˇcová, J. et al. (2003)

28. Procedures for Discrete Scenario Reduction
2 Localsearch algorithm5
:
Fast (Opn3
q) with constant approximation guarantee.
1. Populate supppQq of size m randomly from tξ1, . . . , ξnu.
2. Determine an error-reducing swap pξin, ξoutq.
supppQq Ð supppQq‰tξinuztξoutu.
3. Repeat 2 until no error-reducing swap exists. Output Q.
5
Arya, V. et al. (2004)

29. Procedures for Discrete Scenario Reduction
3 MILP Reformulation 6
:
Slow (Opnm
q) but exact.
1. Solve
min
Π,λ
6
98
97
1
n
n¸
i,j1
πij }ξi ¡ξj }2
:
Π € Rn¢n
, λ € t0, 1un
Π1 1, λ 1 m
Π ¤ 1λ
D
GF
GE
1{2
.
2. Output Q 1
n
°n
j1
°n
i1 π
ij δξj
.
6
Heitsch, H. Römisch, W. (2003)

30. Color Quantization
Ÿ Let ξi rri , gi , bi s denote the color of the ith
pixel.
Ÿ Hence, ˆPn represents the color distribution of the image.
Ÿ Goal is to recover the image7
using only 16 colors.
7
Kodak image suite

31. Color Quantization
Ÿ Let ξi rri , gi , bi s denote the color of the ith
pixel.
Ÿ Hence, ˆPn represents the color distribution of the image.
Ÿ Goal is to recover the image7
using only 16 colors.
Bitmap
7
Kodak image suite

32. Color Quantization
Ÿ Let ξi rri , gi , bi s denote the color of the ith
pixel.
Ÿ Hence, ˆPn represents the color distribution of the image.
Ÿ Goal is to recover the image7
using only 16 colors.
Greedy heuristic (0.45 sec)
7
Kodak image suite

33. Color Quantization
Ÿ Let ξi rri , gi , bi s denote the color of the ith
pixel.
Ÿ Hence, ˆPn represents the color distribution of the image.
Ÿ Goal is to recover the image7
using only 16 colors.
Localsearch algorithm (1.38 sec)
7
Kodak image suite

34. Color Quantization
Ÿ Let ξi rri , gi , bi s denote the color of the ith
pixel.
Ÿ Hence, ˆPn represents the color distribution of the image.
Ÿ Goal is to recover the image7
using only 16 colors.
Exact MILP (224.62 sec)
7
Kodak image suite

35. Conclusions
Ÿ We derive tight bounds for the worst-case approximation errors.
Cpn, mq ¤
™
n ¡m
n ¡1
Ÿ We analyze optimality loss incurred from discrete reduction.
1 ¤ DpˆPn, mq{CpˆPn, mq ¤
c
2
Ÿ We propose constant-approximation algorithm for solving both
discrete and continuous reductions.
localsearch algorithm

36. References
Ÿ Dupaˇcová, J., Gröwe-Kuska, N., and Römisch, W.
Scenario reduction in stochastic programming.
Mathematical Programming 95, 3 (2003).
Ÿ Heitsch, H. and Römisch, W.
Scenario reduction algorithms in stochastic programming.
Computational Optimization and Applications 24, 2 (2003).
Ÿ Rujeerapaiboon, N., Schindler, K., Kuhn, D., Wiesemann, W.
Scenario reduction revisited: fundamental limits and guarantees.
Submitted for Publication.
napat.rujeerapaiboon@epfl.ch
Special thanks to artwork from tPopcorns Arts, Maxim Basinski, Business strategy,
Freepik, Prosymbols, Vectors Market, Madebyoliver, Alfredo Hernandezu@Flaticon.