Symmetry Reduction of Information Inequalities

Symmetry Reduction of Information Inequalities
Kai Zhang and Chao Tian
Department of Electrical Engineering and Computer Science
The University of Tennessee Knoxville (UTK)
Sep. 2016
K. Zhang and C. Tian (UTK) Symmetry Reduction of Information Inequalities Sep. 2016 1 / 16

Outline
Symmetry Reduction
Conclusion
Motivation and Backgrounds
Network Coded Caching
Extremal Pairwise Cyclically Symmetric Entropy Inequalities
Regenerating Code

The full message can be recovered by accessing nodes, a failure node can be repaired
by accessing nodes, each transmitting units of data.
nodes each with capacity , the message is stored (coded) across these nodes;
Regenerating Code
Proposed by Dimakis, et al (2010) to provide efficient repair:
e.g.
What is the fundamental limit of memory vs. transmission rate ?

Regenerating Code
e.g.
nodes each with capacity , the message is stored (coded) across these nodes;
The full message can be recovered by accessing nodes, a failure node can be repaired
by accessing nodes, each transmitting units of data.
Proposed by Dimakis, et al (2010) to provide efficient repair:

files, users, each user has a cache of size ;
Proposed by Maddah-Ali and Niesen (2014) to reduce network congestion:
e.g.
Placement phase: part of each file is pre-fetched to each user’s cache;
Delivery phase: users make requests, server multicasts information to fulfill requests,
the transmission rate is .

files, users, each user has a cache of size ;
Placement phase: part of each file is pre-fetched to each user’s cache;
Proposed by Maddah-Ali and Niesen (2014) to reduce network congestion;
e.g.
Delivery phase: users make requests, server multicasts information to fulfill requests,
the transmission rate is .

The Fundamental Limits
Fundamental limits are derived by its inner-bound and outer-bound:
A recent finding: A Computer-aided approach can be used. [Tian, 2014]
Outer-bound: find new inequalities by manipulating information inequalities (difficult);
Inner-bound: propose new coding schemes;
 Trial and error: almost impossible for long proofs.
Example: A new inequality found for (4, 3, 3) regenerating code problem:
 Tons of them to choose from;
Example: In (4, 3, 3) regenerating code, about 2 million of them.

The Fundamental Limits
Outer-bound: find new inequalities by manipulating information inequalities (difficult);
 Trial and error: almost impossible for long proofs.
Fundamental limits are derived by its inner-bound and outer-bound:
A recent finding: A Computer-aided approach can be used. [Tian, 2014]
Example: A new inequality found for (4, 3, 3) regenerating code problem:
Inner-bound: propose new coding schemes;
 Tons of them to choose from;
Example: In (4, 3, 3) regenerating code, about 2 million of them.

A Computer-aided Approach
Originally proposed by Yeung (1997), use Linear Program (LP) to prove information inequalities:
Use joint-entropy terms as LP variables, e.g. , etc.;
Use information inequalities as LP constraints, e.g.,
Only Shannon-type inequalities are used, but sufficient for most applications.
However, this method is cursed by dimensionality:
In our problem: many variables and constraints are symmetric Symmetry Reduction
In our problem: Regenerating Code: ; Caching: ;
Shannon-type Inequality (Yeung)
Almost all the information inequalities known to data are implied by the basic
inequalities (non-negativity of two elementary forms):
random variable LP problem: LP variables and LP constraints ;

A Computer-aided Approach
Use information inequalities as LP constraints, e.g.,
Only Shannon-type inequalities are used, but sufficient for most applications.
However, this method is cursed by dimensionality:
random variable LP problem: LP variables and LP constraints ;
Shannon-type Inequality (Yeung)
Almost all the information inequalities known to data are implied by the basic
inequalities (non-negativity of two elementary forms):
In our problem: Regenerating Code: ; Caching: ;
Originally proposed by Yeung (1997), use Linear Program (LP) to prove information inequalities:
Use joint-entropy terms as LP variables, e.g. , etc.;
In our problem: many variables and constraints are symmetric Symmetry Reduction

Outline
Symmetry Reduction
Conclusion
Regenerating Code

Color set: ,
Background: Pólya’s Counting Theorem
Example: Color a 2x2 chessboard using two colors , how many different ways?
Notice: we do not distinguish a chessboard with its rotations ;
Solved by Pólya (1937): a group of operations permutation group G.
In this example, the chessboard is labeled as ,
Base set: ;
Permutation group: ;
The different ways to color a group of objects can be given by the coefficients of the
cycle index , substituting each by the summation of all color symbol’s taking
power,
where .
Pólya’s Counting Theorem
Critical: “correctly define ” & “find its cycle index ”.

Color set: ,
Base set: ,
Permutation group: ,
power,
where .

In our problem:
Not easy.Color set: ,
Base set: ,
Permutation group: ,
power,
where .

Second layer: the random variable set , with the induced permutation group ;
A 3-layer General Framework
First layer has a trivial symmetry group;
A few notes:
Second layer is problem specific, difficult part is the study of cycle index of ;
Third layer is what we’re interested: LP variables are represented as elements in .
Study of second layer is already difficult, then third layer ??
Get some knowledge of third layer by studying second layer & coloring problem.
First layer: the (maybe multiple) base set(s) , with the symmetry group ;
Third layer: the power set , with the induced permutation group .
3-layer framework:

A 3-layer General Framework
Study of second layer is already difficult, then third layer ??
Get some knowledge of third layer by studying second layer & coloring problem.
Second layer: the random variable set , with the induced permutation group ;
First layer has a trivial symmetry group;
A few notes:
Second layer is problem specific, difficult part is the study of cycle index of ;
Third layer is what we’re interested: LP variables are represented as elements in .
First layer: the (maybe multiple) base set(s) , with the symmetry group ;
Third layer: the power set , with the induced permutation group .
3-layer framework:

For an regenerating code problem, the random variable set is ,
name change
name change
The “name change” can be represented mathematically by permutations ;
All possible ways of changing names can form a permutation group .
name change
Symmetry Reduction: (4,3,3) Regenerating Code

For an regenerating code problem, the random variable set is ,
The “name change” can be represented mathematically by permutations ;
All possible ways of changing names can form a permutation group .
Symmetry Reduction: (4,3,3) Regenerating Code

Induced Permutation
Base set has a permutation group and its cycle index ,
Group:
Cycle index:
induce
Random variable set has a permutation group and its cycle
index ,
Group:
Cycle index: check every permutation in ?
The cycle index of the group on base set
Cycle Index in Regenerating Code Problem
will induce a term cycle index of the group on base set
of them.

Induced Permutation
The cycle index of the group on base set
Theorem: Cycle Index in Regenerating Code
will induce a term cycle index of the group on base set
Base set has a permutation group and its cycle index ,
Group:
Cycle index:
induce
Random variable set has a permutation group and its cycle
index ,
Group:
Cycle index:

Example: (4,3,3) Regenerating Code
Cycle index:
Number of LP variables ( ):
Intuition: color red denotes , color green denotes ;
The number of variables is the summation of coefficients of minus 1 ;
Note: a simple trick to sum the coefficients is to assign ;
Number of LP constraints:
Type 1: is trivial, use two colors and result is the
coefficient before , result is 2;
Result is , far less than the original .
Type 2: , use three colors
and result is the coefficient before all terms containing , result is 83200;

Cycle index:
Number of LP variables ( ):
Intuition: color red denotes , color green denotes ;
The number of variables is the summation of coefficients of minus 1 ;
Note: a simple trick to sum the coefficients is to assign ;
Number of LP constraints:
Type 1: is trivial, use two colors and result is the
coefficient before , result is 2;
Type 2: , use three colors
and result is the coefficient before all terms containing , result is 83200;
Example: (4,3,3) Regenerating Code

Symmetry Reduction: Network Coded Caching
Variables: files , users , and transmissions
File index set , with the symmetric group ;
represents different demands.
Two base sets:
User index set , with the symmetric group ;
Induced set:
Example:
File index permutation : ;
with the induced permutation group ;
User index permutation : ;

This time, we have to check some permutations to derive .
Fortunately, not all ( ).
Example: N=4, K=3 Cache Network
The file index permutation group and the user index permutation group , each
will have its own cycle index,
The goal: finding the cycle index of the induced set :
, then the permutation induce by and will also have the
If two permutations have the same cycle index, and the same with
Proposition: Cycle Index in Caching
same cycle index.
Result: LP variables , LP constraints .

same cycle index.

same cycle index.
K. Zhang and C. Tian (UTK) Symmetry Reduction of Information Inequalities
Sep. 2016 13 / 16

same cycle index.
Sep. 2016 13 / 16

Outline
Symmetry Reduction
Conclusion
Regenerating Code

Conclusion
Contributions:
Symmetry is carefully defined in our problems;
A general framework for different research problems;
Counting the LP variables/constraints coloring problem.
Challenge:
Counting enumerating;
Counting symmetry reduction, there are other problem specific reduction.

Thank you!
Sep. 2016 16 / 16

Symmetry Reduction of Information Inequalities

Recommended

Recommended

More Related Content

Similar to Symmetry Reduction of Information Inequalities

Similar to Symmetry Reduction of Information Inequalities (20)

Recently uploaded

Recently uploaded (20)

Symmetry Reduction of Information Inequalities