Thesis_Presentation

low-density parity-check codes for the
relay channel
Panagiotis Chatzinikolaou
Telecommunications Division
Technical University of Crete
July 29, 2014
Panagiotis Chatzinikolaou (TUC) ldpc codes for the relay channel July 29, 2014 0 / 42

outline
1 ldpc codes
Introduction
Decoding
Design
Construction
2 bilayer ldpc codes
Relay Channel model
Decoding
Design
Construction

ldpc codes Introduction
channel coding
Source encoder Channel decoder sgn( ) Destination
u
{0,1}k
{0,1}n
c m
{LLRs}
L
{LLRs}
ûChannel Channel
{0,1}k
• The channel encoder maps each vector u ∈ {0,1}k
, representing the
information message, to longer vectors c ∈ {0,1}n
, known as codewords.
• The transmitted symbols may be corrupted in some way by the channel,
and it is the function of the channel decoder to use the added redundancy
to determine the corresponding k information bits.

ldpc codes
• An LDPC code C(n,k) is deﬁned as the nullspace of its own parity-check
matrix, i.e.,
C = {c ∈ {0,1}n
H(n−k)×ncT
= 0}.
• LDPC codes are linear block codes whose parity-check matrices are
sparse.
• Parity-check matrices of linear block codes can be visualized by Tanner
Graphs.

ldpc codes
• If we associate each column of H with a variable node, each row of H
with a check node and connect variable node j with check node i iﬀ
Hij = 1, we can represent each LDPC code as a graph.
• Consider the following parity-check matrix:
H =
x1 x2 x3 x4 x5 x6 x7 x8 x9 x10
⎡
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
⎤
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
c1 1 0 1 1 1 0 1 1 0 0
c2 0 1 1 0 0 1 1 0 1 1
c3 1 1 0 1 1 1 0 1 0 0
c4 0 1 1 0 1 0 1 0 1 1
c5 1 0 0 1 0 1 0 1 1 1
.

ldpc codes
• The associated Tanner graph is depicted below:
(3,6)-regular ldpc code

degree distributions
• Regular codes have constant column and row weights.
• Irregular codes have column and row weights which are chosen according
to some distribution.
• Variable/Check node degree distributions from a node perspective.
Λ(x) =
max
∑
i≥2
Λixi
, P(x) =
rmax
∑
i≥2
Pixi
,
where Λi/Pi is the number of variable/check nodes of degree i.
• Observe that ∑i Λi = n and ∑i Pi = n(1 − r) ≜ m where
r ≜ k
n
is the design rate of the code.

• We deﬁne normalized variable/check node degree distributions from a
node perspective as
L(x) =
Λ(x)
Λ(1)
=
max
∑
i≥2
Lixi
, R(x) =
P(x)
P(1)
=
rmax
∑
i≥2
Rixi
.
• Variable/Check node degree distributions from an edge perspective
λ(x) =
Λ
′
(x)
Λ′
(1)
=
max
∑
i≥2
λixi−1
, ρ(x) =
R
′
(x)
R′
(1)
=
rmax
∑
i≥2
ρixi−1
,
where λi,ρi denote the fraction of edges connected to nodes with degree i.

example
• λ(x) = 6
29
x + 15
29
x2
+ 8
29
x3
, ρ(x) = 4
29
x3
+ 5
29
x4
+ 6
29
x5
+ 14
29
x6
.
irregular ldpc code

• Given the degree distribution of an LDPC code and its number of edges
E = Λ
′
(1) = P
′
(1) the number of variable nodes n is
n = E ∑
i≥2
λi
i
= E ∫
1
0
λ(x)dx,
and the number of check nodes m is
m = E ∑
i≥2
ρi
i
= E ∫
1
0
ρ(x)dx.
• The design rate is given by
r(λ,ρ) ≜ 1 −
m
n
= 1 −
∫
1
0 ρ(x)
∫
1
0 λ(x)
= 1 −
∑i≥2
ρi
i
∑i≥2
λi
i
.

code ensemble
• Sets of codes that share certain characteristics or properties are called
code ensembles.
• For suﬃciently long codes, the error probability of each code in a
(λ(x), ρ(x)) ensemble, is “very close” to the average performance of the
ensemble (concentration theory).

ldpc codes Decoding
factor graphs
• Factor graphs are a way to represent graphically the factorization of a
function.
• Consider a function f that can be factorised as follows:
f(x1,...,x7) = f1(x1,x2,x3)f2(x1,x4)f3(x1,x6,x7)f4(x4,x5)f5(x4)f6(x2).
• Assume that we wish to compute marginal f(x1) then,
f(x1) = ∑
∼x1
f(x1,...,x7)
= ∑
∼x1
f1f2f3f4f5f6.
• By brute force method f(x1) computation requires O( X 7
) operations.

ldpc codes Decoding
factor graphs
• Applying the factorisation and the distributive law we get
f(x1) = ∑
x2,...,x7
f1(x1,x2,x3)f2(x1,x4)f3(x1,x6,x7)f4(x4,x5)f5(x4)f6(x2)
= [∑
x2
(f6(x2)∑
x3
f1(x1,x2,x3))][∑
x4
(f5(x4)f2(x1,x4)
∑
x5
f4(x4,x5))][ ∑
x6,x7
f3(x1,x6,x7)],
reducing complexity to O( X 3
) !
• Likewise for f(xi), i = 2,...,7.

ldpc codes Decoding
factor graphs
• Consider the aforementioned function and its factorization. We associate
with this factorization a factor graph as follows.
• For the aforementioned function f, we associate a variable node (circle)
with each variable and a factor node (square) with each factor.
• Connect a variable node to a factor node by an edge if, and only if, the
corresponding variable appears in this factor.

ldpc codes Decoding
marginalization on factor graphs
• Initial Step:
⋆ Leaf variable nodes send 1.
⋆ Leaf factor nodes send their corresponding function.
• Message Passing Rules:
. . . . . . . . . . . .
variable/check node
rules

ldpc codes Decoding
single message passing-example
• We will apply the message passing rules on the factor graph of f(x1).

ldpc codes Decoding
single message passing-example-step 1

ldpc codes Decoding

ldpc codes Decoding
belief propagation algorithm
• All marginals can be computed simultaneously on a single tree, if we
allow message passing to both directions.

ldpc codes Decoding
belief propagation algorithm-step 1

ldpc codes Decoding

ldpc codes Decoding
bi-awgn channel
+
• It is convenient to use log-likelihood ratios for messages; i.e.,
L ≜ ln
P(X = +1/y)
P(X = −1/y)
=
2y
σ2
.

ldpc codes Decoding
map decoding
• MAP decoding rule for a codeword bit is given by
ˆxi
MAP
(y) = arg max
xi∈{±1}
pXi/Y(xi/y)
= ⋯ (uniform codewords, bayes, memoryless channel)
= arg max
xi∈{±1}
∑
∼xi
(∏
j
pYj /Xj
(yj/xj))1x∈C(H).
• BP algorithm provides exact bit-wise MAP decoding on cycle free graphs.

ldpc codes Decoding
belief propagation decoding
• Messages are log-likelihood ratios (LLRs).
• Variable node update:
...
• Check node update:
...
out = ch +
K
∑
k=1
k.
out = 2 ⋅ tanh−1
(
J
∏
j=1
tanh( j/2)).

ldpc codes Decoding
belief propagation decoding-example
Figure: Initialisation.

ldpc codes Decoding
belief propagation decoding-round 1

ldpc codes Decoding
bit error rate using bp decoding
0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2
10
−6
10
−5
10
−4
10
−3
10
−2
10
−1
10
0
Eb/N0 (dB)
BitErrorRate
Regular vs Irregular
Regular, n=20000
Irregular, n=20000
Figure: Bit Error Rate of a (3, 6) − regular and a concentrated irregular LDPC
code with same block length over BI-AWGN channel.

ldpc codes Decoding
belief propagation decoding for the bec
• Variable node update:
If all incoming messages are erasures,
then the outgoing message is an
erasure. Otherwise, the outgoing
message is equal to the common value.
• Check node update:
If any incoming message is an erasure,
then the outgoing message is an
erasure. Otherwise, the outgoing
message is the mod-2 summation of
the incoming messages.
X Y
1
1
1
1 0 1 0 1
0

ldpc codes Design
density evolution
• In the limit of inﬁnite block length:
1 DE can predict the performance of a (λ(x), ρ(x)) ensemble under BP
decoding using only the degree distributions of the code.
2 We assume cycle-free graphs.
• Under certain symmetry conditions, the convergence behaviour of the
decoder is independent of the transmitted codeword.
1 For the BEC we can assume that the all-zero codeword was transmitted.
2 For the BIAWGN we can assume that the all-one codeword was
transmitted.

ldpc codes Design
density evolution for the bec
• DE tracks the mean variable-to-check message erasure probability of error
for each round of BP.
• The evolution of the expected fraction of erasure messages emitted in the
-th iteration, call it x , is given by
x = ⋅ λ(1 − ρ(1 − x −1)).
• The threshold ∗
is deﬁned as the maximum such that x
→∞
→ 0.
• There is a stability condition that must satisﬁed in order x converges to
0.

ldpc codes Design
density evolution for the bi-awgn channel
• We need to track messages’ densities instead of a single parameter.
• The check node message update rule can be rewritten as
m( )
cv = γ−1⎛
⎝
dc−1
∑
i=1
γ(m( )
vic)
⎞
⎠
,
where γ(x) = (γ1(x),γ2(x)) ≜ (sgn(x),−lntanh x
2
).
• Let P and Q denote the shorthand notations for the densities of random
variables m
( )
vc and m
( )
cv respectively.
• Let Γ denote the density transformation corresponding to γ(x), i.e., given
a random variable Z ∈ R with density fZ(z), the density of γ(Z) is
fγ(Z)(γ1(z),γ2(z)) = Γ(fZ(z)).
• The density of m
( )
cv is:
Q = Γ−1
(Γ(P ) ⋆ ⋯ ⋆ Γ(P )
(i−1)
).

ldpc codes Design
density evolution for the bi-awgn channel
• P0 denotes the density of the r.v.
describing the channel.
P0 ≡ N( 2
σ2 , 4
σ2 ).
• The density of the messages emanating from variable nodes at -th
iteration is
P( +1) = P0 ⊗ ∑
j≥2
λj
⎛
⎝
Γ−1⎛
⎝
∑
i≥2
ρiΓ(P )⋆(i−1)⎞
⎠
Q
⎞
⎠
⊗(j−1)
.
• The average probability of error at round ( + 1) is
Pe(P +1) = ∫
0
−∞
P +1(x) dx.

ldpc codes Design
design using density evolution
• We fix the SNR and the check node degree distribution.
• We want to optimise λ(x) = ∑
d
i=2 λixi−1
.
• The rate of the code is given by
r = 1 −
∑i
ρi
i
∑i
λi
i
.
• We maximize the rate of the code while maintaining some constraints
1 λ(1) = 1 and λi ≥ 0, 2 ≤ i ≤ d .
2 the new λ(x) is not significantly different from the old one.
3 the new λ(x) produces smaller probability of error than the old one.
• Initially, we choose λ(x) = xd −1
to make the initial rate low.
• DE has very high computational complexity.

ldpc codes Design
optimised ldpc code
• For the following simulation we used 1200 decoding iterations and
block-length 106
.
0.2 0.25 0.3 0.35 0.4 0.45 0.5
10
−6
10
−5
10
−4
10
−3
10
−2
10
−1
10
0
Eb/N0 (dB)
BitErrorRate
Optimized Irregular, n=1e6
DE threshold
Shannon limit

ldpc codes Design
gaussian approximation
• We assume that
1 the Tanner graph of the LDPC code is cycle-free;
2 we transmit the all-one codeword;
3 during each iteration, due to 1, the messages are i.i.d. and, more
speciﬁcally, they follow the symmetric Gaussian distribution.
• Symmetric Gaussian pdf means that the variance σ2
= 2µ. Hence, we only
need to track the mean of the densities.
• For decoding to converge to a low probability of error, we want µ
( )
v keep
increasing with each iteration.

ldpc codes Design
gaussian approximation
• At a variable node of degree i:
µ
( )
v,i = µu0 + (i − 1)µ( −1)
u .
• Averaging over λ(x):
µ( )
v = ∑
i≥2
λiµ
( )
v,i .
• At a check node of degree j:
µ
( )
u,j = φ−1⎛
⎝
1 − [1 − φ(µ( )
v )]
j−1⎞
⎠
.
• Averaging over ρ(x):
µ( )
u = ∑
j≥2
ρjµ
( )
u,j.

ldpc codes Design
gaussian approximation-regular ldpc codes
• We consider a (3,6) LDPC code. The threshold has been found to be
σ∗
= 0.8747. We will set ﬁrstly the noise variance equal to σ = 0,8511< σ∗
and secondly σ = 0,9411> σ∗
.
−10 0 10 20 30 40 50
0
0.05
0.1
0.15
0.2
Initial pdf, regular, n=2000
Density Evolution
assuming N~(m,2m)

ldpc codes Design
−20 −10 0 10 20 30 40 50
0
0.05
0.1
0.15
0.2
n=2000, variable2check iter No1
LLR magnitude
Density Evolution
assuming N~(m,2m)

ldpc codes Design
−20 −10 0 10 20 30 40 50
0
0.05
0.1
0.15
0.2
LLR magnitude
Density Evolution
assuming N~(m,2m)

ldpc codes Design
−4 −3 −2 −1 0 1 2 3 4
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
n=2000, check2variable iter No1
LLR magnitude
Density Evolution
assuming N~(m,2m)

ldpc codes Design
−4 −3 −2 −1 0 1 2 3 4
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
LLR magnitude
Density Evolution
assuming N~(m,2m)

ldpc codes Design
• Above threshold:
−10 0 10 20 30 40 50
0
0.05
0.1
0.15
0.2
Initial pdf, regular, n=2000
Density Evolution
assuming N~(m,2m)

ldpc codes Design
−20 −10 0 10 20 30 40 50
0
0.05
0.1
0.15
0.2
LLR magnitude
Density Evolution
assuming N~(m,2m)

ldpc codes Design
−4 −3 −2 −1 0 1 2 3 4
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
LLR magnitude
Density Evolution
assuming N~(m,2m)

ldpc codes Design
gaussian approximation-irregular ldpc codes
• The threshold in terms of noise variance, is σ∗
= 0,97704. In the following
simulation we use σ = 0,9441 < σ∗
.
−20 −10 0 10 20 30 40 50
0
0.05
0.1
0.15
0.2
Initial pdf−irregular−n=20000
experimentally
assuming N~(m,2m)

ldpc codes Design
−20 −10 0 10 20 30 40 50
0
0.05
0.1
0.15
0.2
variable2check iter No1
experimentally
assuming N~(m,2m)
GMM prediction

ldpc codes Design
−20 −10 0 10 20 30 40 50
0
0.05
0.1
0.15
0.2
experimentally
assuming N~(m,2m)
GMM prediction

ldpc codes Design
−4 −3 −2 −1 0 1 2 3 4
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
check2variable iter No1
experimentally
assuming N~(m,2m)

ldpc codes Design
−4 −3 −2 −1 0 1 2 3 4
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
experimentally
assuming N~(m,2m)

ldpc codes Construction
configuration model
• An optimised degree distribution pair (λ(x),ρ(x)) is given.
• From λ(x), we compute the probability that a variable node is of degree i
according to
Li =
1
∑j
λj
j
λi
i
, for i = 2,..., max.
• number of variable nodes per degree i is given by Li ⋅ n.
• number of edges connected to variable nodes of degree i is given by
Li ⋅ n ⋅ degree i.

configuration model
• “Generate” n variable nodes and m check nodes with their sockets.
• Label the variable and check node sockets separately with the set
[Λ′
(1)] = {1,...,∑i iΛi}.
.
.
.
.
.
.
v1
v2
vn
C2
C1
C m
v3
C3
1
2
3
4
5
6
7
1
2
.
.
.
total_edges
total_edges
.
.
.
.
.
.
total edges is the last socket (edge) which is equal to the total number of
edges (= ∑i iΛi).

configuration model
• Pick a permutation π on [Λ′
(1)] at random with uniform probability
from the set of all (∑i iΛi)! such permutations.
• Let π be a permutation on [Λ′
(1)]. Connect i-th variable node edge
socket with π(i)-th check node edge socket.
.
.
.
.
.
.
v1
v2
vn
C2
C1
C m
v3
C3
1
2
3
4
5
6
7
total_edges
.
.
.
π(1)
π(2)
π(3)
• Put “1” in the appropriate position of H.

bilayer ldpc codes Relay Channel model
relay channel
1 For the source-relay link, x ∈ C, with parity-check matrix H such that
Hx = 0.
2 rate(C) ≤ rate(source → relay). Hence, relay decodes x.
3 rate(C) > rate(source → destination). Hence, destination cannot decode x.
4 The parity-check matrix of Cbilayer denoted by Hbilayer is given by
Hbilayer = [
H
H1
], Hbilayer ⋅ x = [
0
p
],
where H1 is the upper layer of the bilayer graph.
5 Finally, rate(Cbilayer) ≤ rate(source → destination) and, therefore,
destination decodes x using Hbilayer and parity vector p sent by the relay.

bilayer expurgated ldpc codes
. . .
. . .
. . .
k1
k2
dc
d’
c
j
i
C1 C2 Ck1
C’1 C’k2
V1 V2 Vn
• C (lower graph) should be capacity approaching for source→relay link.
• Cbilayer (overall graph) should be capacity approaching for
source→destination link.

bilayer lengthened ldpc codes
1 For the source-relay link, x ∈ Cbilayer, with parity-check matrix Hbilayer
such that Hbilayerx = 0.
2 rate(C) ≤ rate(source → relay). Hence, relay decodes x over the bilayer
graph.
3 rate(C) > rate(source → destination). Hence, destination cannot decode x.
4 The parity-check matrix of Cbilayer denoted by Hbilayer is given by
Hbilayer = [H H1],
where H1 is the upper layer of the bilayer graph.
5 The relay forwards to the destination the upper n2 variable nodes.
Destination removes the upper variable nodes, reducing the resulting
code’s rate and decodes the lower graph.

. . .
. . .
. . .
n1
n2
dc
d’
c
V1 V2 Vn1V3
Vn1+1
Vn
• Lower code C should be capacity approaching for the source → destination
link.
• Cbilayer should be capacity approaching for the source → relay link.

bilayer ldpc codes Decoding
bilayer expurgated ldpc codes
1.6 1.8 2 2.2 2.4 2.6 2.8 3
10
−6
10
−5
10
−4
10
−3
10
−2
10
−1
10
0
SNR (dB)
BitErrorRate
n=100000
SL (Relay)
BL (destination)

bilayer ldpc codes Decoding
0 0.5 1 1.5 2 2.5 3 3.5
10
−6
10
−5
10
−4
10
−3
10
−2
10
−1
10
0
SNR (dB)
BitErrorRate
n=100003,n1=6.039430e+04,n2=3.960326e+04
BL(source−relay)
SL(source−destination)

bilayer ldpc codes Design
bilayer-expurgated ldpc code optimisation
• The design involves ﬁnding a variable degree distribution λi,j, a
parameter η, and a pair of check degree dc and d
′
c such that the lower
sub-graph is capacity-achieving at SNR+, while the bilayer graph is
capacity-approaching at SNR−.
• We ﬁx the lower graph to be an optimal capacity-approaching LDPC
code at SNR+.
• Find an overall degree distribution λi,j which is consistent with the lower
graph and capacity-approaching at SNR−.
• The rate of the bilayer code is 1 − k1+k2
n
. Maximizing the rate is
equivalent to maximize parameter η.

bilayer ldpc codes Design
bilayer-lengthened ldpc code optimisation
• The design involves finding a pair of variable degree distributions λ1
i and
λ2
i (i ≥ 2) and a pair of dc and d
′
c such that the overall graph is
capacity-approaching at SNR+, while the lower sub-graph is
capacity-approaching at SNR−.
• We fix the lower degree distribution λ1
i to be capacity-approaching at
SNR−.
• The design problem is now reduced to finding λ2
i such that the overall
graph be a capacity-approaching at SNR+.
• The rate of the bilayer code is 1 − k
n1+n2
, where n2 = d
′
c ⋅ k ⋅ ∑i≥2 λ2
i /i.
• To maximise the rate, it suffices to maximise ∑i≥2 λ2
i /i.

bilayer ldpc codes Construction
bilayer expurgated construction
• Construct the conventional lower code according to λi = 1
η ∑j≥0
i
i+j
λi,j.
• An extension of the equation we derived from Li is as follows:
Li,j =
1
∑k ∑
λk,
k+
⋅
λi,j
i + j
, i ≥ 2, j ≥ 0,
where Li,j represents the probability of a variable node be of lower degree
i and upper degree j.
• After computing Li,j table, we can calculate the number of upper edges
with respect to lower degrees i and upper degrees j, as n ⋅ Li,j ⋅ j.
• Construct upper layer using the single user conﬁguration model.
• Concatenate the lower and upper graph to create the bilayer graph.

conclusion
LDPC codes:
1 have low decoding complexity;
2 capacity-approaching in several scenarios;
3 strong design tools.

Thank You!

Thesis_Presentation

More Related Content

What's hot

Similar to Thesis_Presentation

Thesis_Presentation