Information theory

1. In telecommunication, a convolutional code is a type of error-
correcting code in which each m-bit information symbol (each m-bit-
string) to be encoded is transformed into an n-bit symbol, where m/n is
the code rate (n ≥ m).
Convolutional code
1

2. Fig. below shows a (2,1,8) coder with g1 = (1 1 1 1 0 1 0 1 1) and
g2 = (1 0 1 1 1 0 0 0 1) where g1 and g2 are generator sequences
C
Information
frame S
M3 M4 M5 M6 M7 M8
C2
+
+
M9
C1
M2
M1
Let the message sequence, S = 1010000100000101
The channel frame would be input sequence with m = 8 tail bits of ‘0’
2
(l, k, r)
l=no. of output terminal
K=no. of input terminal
r=no. of memory cells

3. 1 0 1 0 0 0 0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0
M =
Information 16 bits Tail bits
Clock
Pulse
M M1 M2 M3 M4 M5 M6 M7 M8 M9 C1 C2
1 1 1 0 0 0 0 0 0 0 0 1 1
2 0 0 1 0 0 0 0 0 0 0 1 0
3 1 1 0 1 0 0 0 0 0 0 0 0
4 0 0 1 0 1 0 0 0 0 0 0 1
5 0 0 0 1 0 1 0 0 0 0 1 0
6 0 0 0 0 1 0 1 0 0 0 0 1
7 0 0 0 0 0 1 0 1 0 0 0 1
8 1 0 0 0 0 0 1 0 1 0 0 0
9 0 1 0 0 0 0 0 1 0 1 0 0
10 0 0 1 0 0 0 0 0 1 0 0 0
11 0 0 0 1 0 0 0 0 0 1 0 0
12 0 1 0 0 1 0 0 0 0 0 0 0
13 0 0 1 0 0 1 0 0 0 0 1 1
14 1 1 0 1 0 0 1 0 0 0 1 0
15 0 0 1 0 1 0 0 1 0 0 0 1
16 1 1 0 1 0 1 0 0 1 0 1 1
17 0 0 1 0 1 0 1 0 0 1 0 0
18 0 0 0 0 1 0 1 0 1 0 1 0
19 0 0 0 0 0 1 0 1 0 1 1 1
20 0 0 0 0 0 1 0 1 0 0 1 0
21 0 0 0 0 0 0 1 0 1 0 0 0
22 0 0 0 0 0 0 0 1 0 1 1 1
23 0 0 0 0 0 0 0 0 1 0 1 0
24 0 0 0 0 0 0 0 0 0 1 1 1
3

4. C = 11 10 00 01 10 01 01 00 00 00 00 00 11 10 01 11 00 10 11 10 00
11 10 11
Length of C is 48 bits
The rate of the coder is n/m=16/48=1/3
Length of code sequence is determined using the relation,
L = l(S+r) = 2(16+8) = 48
; where l is the number of output; r is the number of shift register and
S is the length of message string.
4

5. Q: Design a convolution coder (2,1,2) with g1 = 1 1 1 and g2 = 1 0 1 with M = 2.
Determine output for message sequence, S = 1 0 0 1 1
Ans:
Here is S = 1 0 0 1 1
The channel frame would be input sequence with m = 2 tail bits of ‘0’ frame,
M = 1 0 0 1 1 0 0
C1 = M1+M2+M3
C2 = M1+M3
Here sum of C1 and C2 are X-OR sum
The circuit diagram of the coder is shown in fig. below.
M3
M2
M1
C1
S
C2
m1
m2
+
+
Clock M M1 M2 M3 C1 C2
1 1 1 0 0 1 1
2 0 0 1 0 1 0
3 0 0 0 1 1 1
4 1 1 0 0 1 1
5 1 1 1 0 0 1
6 0 0 1 1 0 1
7 0 0 0 1 1 1
5

6. C = 11 10 11 11 01 01 11 i.e. the length of codeword is14 bits.
Length of code sequence
L = l (S+r)
= 2(5+2) = 14
6

7. Q: A convolution coder with g1 = 1 1 1 and g2 = 1 0 1 with m = 2.
Determine output code sequence for S = 10011
Ans:
Here, g1(x)= 1+x+x2
g2(x)= 1+x2
S(x)= 1+x3+x4
C1(x)= S(x)g(1)(x)
= (1+x3+x4)( 1+x+x2)
= 1+x3+x4+x+x4+x5+x2+x5+x6
= 1+x+x2+x3+x6
↔ 1 1 1 1 0 0 1
C2(x)= S(x)g(2)(x)
= (1+x3+x4)( 1+x2)
= 1+x2+x3+x5+x4+x6
= 1+x2+x3+ x4+x5+x6
↔ 1 0 1 1 1 1 1
C = C1+ C2
= (11 10 11 11 01 01 11)
M3
M2
M1
C1
S
C2
m1
m2
+
+
7

8. 8
Q1. A convolution coder with g1 = 1 1 1 and g2 = 1 0 1 with m =
2. Determine output code sequence for S = 10011 using Matlab.
Ans. Here d1 = g1*S and d2 = g2*S
g1=[1 1 1];
g2=[1 0 1];
S=[1 0 0 1 1];
d1=rem(conv(g1,S),2); %Modulo-2 addition
d2=rem(conv(g2,S),2);% %Modulo-2 addition
d1 = 1 1 1 1 0 0 1
d2 = 1 0 1 1 1 1 1

9. 9
Q2. A convolution coder with g1 = 1 0 1 1 and g2 = 1 1 1 1 with
m = 2. Determine output code sequence for S = 1 0 1 1 1 using
Matlab.
g1=[1 0 1 1];
g2=[1 1 1 1];
S=[1 0 1 1 1];
d1=rem(conv(g1,S),2); %Modulo-2 addition
d2=rem(conv(g2,S),2);% %Modulo-2 addition
d1 = 1 0 0 0 0 0 0 1
d2 = 1 1 0 1 1 1 0 1

10. Graphical Presentation of Convoluational Coder
A convolutional coder can be expressed in three graphical form
a) code tree
b) state diagram
c) trellis diagram
10

11. 00
00
00
00
11
11
11
11
10
10
00
01
01
01
00
11
10
11
01
11
11
00 10
01
10
00
01
01
10
0
1
Fig. 1 Tree diagram of fig.1
M3
M2
M1
C1
S
C2
m1
m2
+
+
Fig.1
a) code tree For input message sequence 10011,
the encoder sequence could be 11 10
11 11 according to the dotted line of
tree diagram. At receiving end
reverse operation is performed on
coded data to retrieve the message.
11

12. b) state diagram
Let us consider the convolutional coder of the fig.below; where content of the two
memory units m1 and m2 can take any one of the four state 00, 01, 10 and 11.
M3
M2
M1
C1
S
C2
m1
m2
+
+
1
0
m1 m2
0 0
m1 m2
0 1
m1 m2
1 1
m1 m2
1 0
11
11
01
01 10
00
00
10
12

13. Let us consider a trellis between k and (k+1)th clock shown in fig. below
0
1
00
01
10
11
00
11
11
00
10
01 01
10
13
c) Trellis diagram
1
0
m1 m2
0 0
m1 m2
0 1
m1 m2
1 1
m1 m2
1 0
11
11
01
01 10
00
00
10

14. 14
00
11
11
00
10
01 01
10
00
11
11
00
10
01 01
10
00
11
11
00
10
01 01
10
00
01
10
11
00
11
11
00
10
01 01
10

15. Determine the correct codeword if, R = 00 10 10 11
15
00
11
11
00
10
01 01
10
00
11
11
00
10
01 01
10
00
11
01
10
00
01
10
11
00
11

16. MATLAB
Determine encoded data of the convolutional encoder of fig. 8.1,
where the message sequence is, m = [1 0 0 1 1 0 0]. Here the last two
zeros of m are for two FF (memory cell) of the encoder; not the part of
the message sequence.
m
C
M1 M2
+
+
m = [1 0 0 1 1 0 0];
t = poly2trellis( 3, [7,5] );
%Formation of the sequential circuit of fig
%length of generator sequence is 3
%generator g1 = 111 = 7 and g2 = 101 = 5
c = convenc(m, t)
The result of above code is,
c = 1 1 1 0 1 1 1 1 0 1 0 1 1 1
16

17. 17
trellis = poly2trellis([5 4],[23 35 0; 0 5 13])
Size of generator 5 bits
Size of generator 4 bits

18. Let us decode c by Viterbi algorithm.
tb=1% traceback depth
v=vitdec(c, t, tb, 'cont', 'hard')
Above code gives one bit delayed version of message vector, m.
v =
0 1 0 0 1 1 0
Determine bit error of above encoder and decoder system.
[b,r]=biterr(v(tb+1:end), m(1:end-tb))
b = 0
r = 0
18

19. 19
trellis = poly2trellis( 3, [7,5] );
spect = distspec(trellis,4)
berub = bercoding(1:10,'conv','hard',2/3,spect); % BER bound
berfit(1:10,berub); ylabel('Upper Bound on BER'); % Plot.
G=2; %channel gain
for SNR=1:12
t = poly2trellis([4 3],[4 5 17;7 4 2]); % Define trellis.
tb = 2; % Traceback length for decoding
% Create a ConvolutionalEncoder System object
hConvEnc = comm.ConvolutionalEncoder(t);
% Create a ViterbiDecoder System object
hVitDec = comm.ViterbiDecoder(t, 'InputFormat', 'hard', ...
'TracebackDepth', tb, 'TerminationMethod', 'Truncated');
data=randi(2,200000,1)-1; %random binary bits
code = step(hConvEnc, data); % Encode a string of ones.
decoded = step(hVitDec, awgn(G*code, SNR)); % Decode under awgn.
[a,b]=symerr(decoded, data);
e(SNR)=b;
end
semilogy(1:12,e, '*-')

20. 20
t = poly2trellis([4 3],[4 5 17;7 4 2]); % Define trellis.
tb = 2; % Traceback length for decoding
% Create a ConvolutionalEncoder System object
hConvEnc = comm.ConvolutionalEncoder(t);
% Create a ViterbiDecoder System object
hVitDec = comm.ViterbiDecoder(t, 'InputFormat', 'hard', ...
'TracebackDepth', tb, 'TerminationMethod', 'Truncated');
data=randi(2,20000,1)-1; %random binary bits
code = step(hConvEnc, data); % Encode a string of ones.
decoded = step(hVitDec, code+randerr(length(code),1,[0 1;0.9 0.1]));
% randerr produces bit string randomly with pr of 0 of 96% and 1 of 4%
% Decode under awgn.
[a,b]=symerr(decoded, data)