This document discusses optimal transport and how it can be used to model complex systems. Optimal transport finds the most efficient way to transport mass from an initial distribution to a final distribution given a cost matrix, and can be formulated as an energy minimization problem with constraints. The solution is found using the Sinkhorn-Knopp algorithm, which scales rows and columns iteratively. Optimal transport has applications in distribution matching, interpolation, domain adaptation, and modeling developmental landscapes. It provides a simple framework for comparing distributions with constraints and competition between parts.

1. A TOUR IN OPTIMAL TRANSPORT
Michiel Stock
@michielstockKERMIT

4. DESSERTS MADE BY TINNE

5. PORTIONS PER KERMIT MEMBER

6. PREFERENCES FOR DESSERTS

7. FORMAL PROBLEM DESCRIPTION
r: vector containing portions of dessert per person (general: n-dim.)
c: vector containing portions of each dessert (general: m-dim.)
M: a cost matrix (negative preference)
U(r, c) = {P 2 Rn⇥m
>0 | P1m = r, P|
1n = c}
Polyhedral set containing all valid partitions:
Solve the following problem:
dM (r, c) = min
P 2U(r,c)
X
i,j
PijMij
Minimizer is the optimal distribution!P?
ij

8. dM (r, c) = min
P 2U(r,c)
hP, MiF
1
h(P)
OPTIMAL TRANSPORT WITH ENTROPIC REGULARIZATION
Cost:
Entropic regularization:
Tuning parameter:
hP, MiF =
X
i,j
PijMij
h(P) =
X
i,j
Pij log Pij
Constrain solution to possess a minimal ‘evenness’

9. GEOMETRY OF THE OPTIMAL TRANSPORT PROBLEM
M
P?
P?
rc|
U(r, c)

10. DERIVATION OF THE SOLUTION
@L(P, , {a1, . . . , an})
@Pij
|P ⇤
ij
= 0
Lagrangian of the problem:
Choose constants to satisfy constraints!
P?
ij = e ai bj 1
e Mij
= ↵i je Mij
L(P, , {a1, . . . , an}, {b1, . . . , bm}) =
X
ij
PijMij +
1 X
ij
Pij log Pij
+
nX
i=1
ai(ri
X
j
Pij) +
mX
j=1
bj(cj
X
i
Pij)
@L(P, , {a1, . . . , an})
@Pij
= Mij +
log Pij
+
1
ai bj

11. THE SINKHORN-KNOPP ALGORITHM
Init
Until converged
Scale rows
Scale columns
P = e M

12. SOLUTION (HIGH LAMBDA)
Solution is very good approximation of unregularized OT problem!
total average
preference:
36

13. SOLUTION (LOW(ER) LAMBDA)
Every person has to try a bit of everything!
total average
preference:
29.6

14. APPLICATIONS
➤ Matching distributions
➤ Interpolation
➤ Domain adaptation
➤ Color transfer
➤ Comparing distributions
➤ Modelling complex systems

15. MATCHING AND INTERPOLATING DISTRIBUTIONS

16. DOMAIN ADAPTATION WHEN DISTR, TRAIN AND TEST DIFFER

17. IMAGE COLOR TRANSFER BY MATCHING DISTRIBUTIONS

18. IMAGE COLOR TRANSFER BY MATCHING DISTRIBUTIONS

19. COMPARING DISTRIBUTIONS (WITH METRIC/COST)
➤ Comparing two
distributions with
cost
➤ Comparing two sets
of objects with
pairwise similarity
No equal
number of
bins required!

20. dM (r, c) = min
P 2U(r,c)
hP, MiF
1
h(P)
OPTIMAL TRANSPORT AS ENERGY MINIMISATION
OT can be seen as a physical system of interacting parts
Energy of the system
Physical constrains (i.e. mass balance)
Inverse temperature
Entropy of system

21. Interacting systems with competition.

22. COMPUTATIONAL FLUID DYNAMICS
Lévy, B. and Schwindt, E. (2017). Notions of optimal transport theory and how to implement them on a computer arxiv

23. LEARNING EPIGENETIC LANDSCAPES
Reconstruction of developmental landscapes by optimal-transport analysis of single-cell gene expression sheds light on cellular reprogramming.
doi: https://doi.org/10.1101/191056

24. IN SUMMARY
➤ OT is a simple framework for
thinking about distributions
➤ Powerful tool for modelling
complex systems (constraints
+ competition)
➤ Efficient solvers: O(n^2)
(when using entropic
regularization)

25. REFERENCES
Lévy, B. and Schwindt, E. (2017). Notions of optimal transport
theory and how to implement them on a computer arxiv
Courty, N., Flamary, R., Tuia, D. and Rakotomamonjy, A. (2016).
Optimal transport for domain adaptation
Cuturi, M. (2013) Sinkhorn distances: lightspeed computation
of optimal transportation distances