This slide introduces geometric aspect of continuous-time Markov chains and potential research direction.

1. Geometry of Continuous-Time
Markov Chains
Shuchang Zhang

2. Content
› Motivation
› Mathematical Background
› Geometric Flow of Markov Chain
› Conclusions

3. Motivation

4. Motivation
• Time reversible Markov chain (detailed balance) is known to have
symmetric probability flux and can be described by gradient system
• Symmetric flux contributes to the production of relative entropy,
whereas skew-symmetric flux doesn’t
• Skew-symmetric flux playing a very important role as circulation in
time evolution of chains is yet hardly understood
• Evolution of Markov chain can be characterized by differential
geometry, which is a powerful and indispensable tool in dynamics

5. Mathematical Background

6. Alpha representation
In information geometry, a probability distribution can be coded by a
parameter 𝛼 as the following,
𝑙 𝛼
=
2
1 − 𝛼
𝑝
1−𝛼
2
Important examples include:
𝛼 = −1, 𝑙(−1) = 𝑝 mixed representation
𝛼 = 1, 𝑙(1)
= log 𝑝 exponential representation
𝛼 = 0, 𝑙(0) = 2 𝑝 0-representation

7. Alpha representation
𝛼 = −1 𝛼 = 1

8. Alpha representation
𝛼 = −1 𝛼 = 0

9. Alpha representation
• Different representations are equipped with different geometric
structures and restrict the dynamics of Markov chains on different
manifolds.
• Particularly, 0-representation admits the flow of probability on the
(hyper)sphere, which has radical symmetry.

10. Lie group of 𝑆𝑂 𝑛
• The motion on the manifold of 𝑛-sphere 𝑀 = 𝕊 𝑛 can be seen as
continuous isometry (distance-preserving) transformation.
• Given an initial point 𝑝0 ∈ 𝑀, the trajectory of 𝑝 is given by 𝑝𝑡 = 𝑔𝑡 𝑝0,
where 𝑔0 = 𝑒, 𝑔𝑡+𝑠 = 𝑔𝑡 𝑔𝑠 form a Lie group 𝐺 = 𝑆𝑂𝑛.
• Under matrix representation, 𝐺 is the set of order-𝑛 orthogonal
matrices with determinant 1. i.e. 𝑆𝑂𝑛 = 𝑂 ∈ 𝑆𝐿 𝑛|𝑂 𝑇
𝑂 = 𝑂𝑂 𝑇
= 𝐼 𝑛

11. Lie algebra of 𝔰𝔬 𝑛
• Let 𝐺 action on the torsor (principal homogenous space) 𝑀 from the
left, we have
ሶ𝑔𝑡 = lim
𝑠→0
𝑔𝑠 − 𝑒
𝑠
𝑔𝑡 = X𝑔𝑡 ∈ 𝑇𝑔 𝑡
𝐺
𝑋 = lim
𝑠→0
𝑔s − 𝑒
𝑠
= ሶ𝑔𝑡 ∘ 𝑔𝑡
−1
∈ 𝑇𝑒 𝐺 = 𝔤
The tangent vector X is the right translation of ሶ𝑔𝑡 by 𝑔𝑡
−1
.
• Lie algebra 𝔤 can be identified as tangent space at the identity.
Given any vector 𝑋 ∈ 𝔤, there is a unique left-invariant vector field
𝑋 𝑔 = 𝑇𝐿 𝑔 𝑋 = 𝐿 𝑔∗
𝑋

12. Lie algebra of 𝔰𝔬 𝑛
• Note that for 𝑂𝑠 ∈ 𝑆𝑂𝑛 near the identity, we have
𝐼 = 𝑂𝑠
𝑇 𝑂𝑠 = 𝐼 + 𝑠Ω + 𝑜 𝑠
𝑇
𝐼 + 𝑠Ω + 𝑜 𝑠 = 𝐼 + 𝑠 Ω + Ω 𝑇 + 𝑜 𝑠
The matrix Lie algebra of 𝔰𝔬 𝑛 is the set of skew-symmetric matrices,
i.e. 𝔰𝔬 𝑛 = 𝑇𝑒 𝑆𝑂𝑛 = Ω ∈ 𝐺𝐿 𝑛| Ω + Ω 𝑇
= 0
• It can also be identified with vector space of dimension
𝑛(𝑛−1)
2

13. Adjoint and coadjoint representation of 𝔰𝔬 𝑛
• An important representation of Lie algebra, called adjoint
representation, is defined as
𝑎𝑑 ∶ 𝔤 → 𝔤𝔩 𝑛 = 𝐸𝑛𝑑 𝔤
𝑎𝑑 𝑋: 𝑌 ↦ 𝑋, 𝑌
• Choose a non-degenerate inner product , on Lie algebra 𝔤, the
coadjoint representation is defined as
𝑎𝑑 𝑍
∗
𝑋, 𝑌 = 𝑋, 𝑎𝑑 𝑍 𝑌

14. Riemannian metric
• The inner product , induces a right-invariant Riemannian metric
, 𝑔 on the whole Lie group 𝐺. Given two vectors 𝑋, 𝑌 ∈ 𝑇𝑔 𝐺, the
Riemannian metric is defined as
𝑋, 𝑌 𝑔: = 𝑇𝑅 𝑔
−1
∗
𝑋 , 𝑇𝑅 𝑔
−1
∗
𝑌
• The geodesic is defined as the extremal of the energy functional
𝐸 𝑔𝑡 = න
𝑎
𝑏
1
2
ሶ𝑔𝑡, ሶ𝑔𝑡 𝑔 𝑑𝑡

15. Geometric Flow of Markov Chain

16. 0-representation of Markov chain
• A continuous-time Markov chain (CTMC) is completely determined by
its infinitesimal generator 𝑄, admitting the first-order ODE.
ሶ𝑝𝑖 = ෍
𝑗
𝑄𝑖𝑗 𝑝𝑗
where σ𝑖 𝑄𝑖𝑗 = 0, 𝑄𝑖𝑗 ≥ 0 for 𝑖 ≠ 𝑗 and 𝑄𝑖𝑖 < 0
• Let 𝑞𝑖 = 2 𝑝𝑖 be 0-representation of probability 𝑝, we have
ሶ𝑞𝑖 =
1
2
෍
𝑗
𝑄𝑖𝑗 𝑞 𝑗
2
𝑞𝑖
= ෍
𝑗
Ω𝑖𝑗 𝑞 𝑗
where Ω𝑖𝑗 + Ω𝑗𝑖 = 0

17. Evolution of the same CTMC
𝛼 = −1 𝛼 = 0

18. Geometric flow of CTMC
• Let 𝑞𝑡 be a continuous trajectory on 𝕊 𝑛 such that 𝑞𝑡 = 𝑔𝑡 𝑞0, where
𝑔𝑡 ∈ 𝑆𝑂𝑛, then
ሶ𝑞𝑡 = ሶ𝑔𝑡 𝑞0 = ሶ𝑔𝑡 𝑔𝑡
−1
𝑞𝑡 = Ω𝑞𝑡
Ω = ሶ𝑔𝑡 𝑔𝑡
−1
∈ 𝔰𝔬 𝑛
• This establishes a bijection between the trajectory on 𝕊 𝑛 and that on
𝑆𝑂𝑛. This inspires us to investigate geodesic flow on 𝑆𝑂𝑛.

19. Geodesic flow on 𝑆𝑂 𝑛
• By requiring the first variation of energy functional 𝐸[𝑔𝑡] to vanish,
i.e. we have
𝛿𝐸 𝑔𝑡 =
1
2
𝛿 න
𝑎
𝑏
ሶ𝑔𝑡, ሶ𝑔𝑡 𝑔 𝑑𝑡 =
1
2
𝛿 න
𝑎
𝑏
ሶ𝑔𝑡 𝑔𝑡
−1
, ሶ𝑔𝑡 𝑔𝑡
−1
𝑑𝑡
= න
𝑎
𝑏
𝛿 ሶ𝑔𝑡 𝑔𝑡
−1
+ ሶ𝑔𝑡 𝛿𝑔𝑡
−1
, ሶ𝑔𝑡 𝑔𝑡
−1
𝑑𝑡 = න
𝑎
𝑏
ሶ𝛿𝑔𝑡 𝑔𝑡
−1
− Ω𝛿𝑔𝑡 𝑔𝑡
−1
, Ω 𝑑𝑡
= 𝛿𝑔𝑡 𝑔𝑡
−1
, Ω ቚ
𝑎
𝑏
+ න
𝑎
𝑏
𝛿𝑔𝑔𝑡
−1
, Ω + 𝛿𝑔𝑡 𝑔𝑡
−1 ሶΩ, Ω 𝑑𝑡
= න
𝑎
𝑏
𝛿𝑔𝑡 𝑔𝑡
−1 ሶΩ − 𝑎𝑑Ω 𝛿𝑔𝑔𝑡
−1
, Ω 𝑑𝑡 = න
𝑎
𝑏
𝛿𝑔𝑔𝑡
−1
, ሶΩ − 𝑎𝑑Ω
∗
Ω 𝑑𝑡 = 0

20. Geodesic flow on 𝑆𝑂 𝑛
• We obtain Euler-Poincare equation
ሶΩ = 𝑎𝑑Ω
∗
Ω
• Choose Frobenius inner product 𝑋, 𝑌 = 𝑡𝑟(𝑋 𝑇 𝑌), then
𝑋, 𝑎𝑑 𝑍 𝑌 = 𝑋, 𝑍, 𝑌 = 𝑡𝑟 𝑋 𝑇 𝑍𝑌 − 𝑌𝑍
= 𝑡𝑟 𝑋 𝑇 𝑍 − 𝑍𝑋 𝑇 𝑌 = 𝑍 𝑇, 𝑋 , 𝑌 = 𝑎𝑑 𝑍
∗
𝑋, 𝑌
• Rewrite Euler-Poincare equation as Lie-Poisson form
ሶΩ + Ω, Ω = 0

21. Geometric flow of CTMC again
• Euler-Poincare equation:
ሶΩ = 𝑎𝑑Ω
∗
Ω
Note that this equation doesn’t contain 𝑔𝑡 explicitly.
• We can reconstruct the equation of the motion of 0-representation
Markov chain by
Ω = ሶ𝑔𝑡 𝑔𝑡
−1
,
ሶ𝑔𝑡 = Ω𝑔𝑡

22. Conservation law in CTMC
• By Noether’s theorem, the right-invariant geodesic flow preserves
some quantities, which can be computed by momentum map 𝜇
𝜇: 𝔤 → ℝ, 𝑋 ↦ Ω, 𝑔𝑡 𝑋𝑔𝑡
−1
• Proof
ሶ𝜇 = ሶΩ, 𝑔𝑡 𝑋𝑔𝑡
−1
+ Ω, Ω, 𝑔𝑡 𝑋𝑔𝑡
−1
= 𝑎𝑑ΩΩ, 𝑔𝑡 𝑋𝑔𝑡
−1
+ Ω, 𝑎𝑑Ω 𝑔𝑡 𝑋𝑔𝑡
−1
= 𝑎𝑑ΩΩ, 𝑔𝑡 𝑋𝑔𝑡
−1
+ 𝑎𝑑Ω
∗
𝛺, 𝑔𝑡 𝑋𝑔𝑡
−1
= 0

23. Conclusions

24. Conclusions
In summary, we give a geometric formulation of 0-representation CTMC.
This view allows us to
• Investigate the dynamics on (hyper)sphere, from both intrinsic
and extrinsic view
• Reduce the dimension of infinitesimal generator by half (from
𝑛(𝑛 − 1) to
𝑛 𝑛−1
2
)
• The time evolution of Markov chains follows Euler-Poincare
equation, whose trajectory is always geodesic flow
• Conservation quantities can be found

25. Further questions
There are many problems to be solved yet
• How to distinguish skew-symmetric flux from symmetric one in
geometric view
• Geometric formulation of CTMC in other representations
• Find master equation of geodesic flows
• Etc..