Channel Coding
Types of Coding
 Source Coding (Data Compression)- Code data to more efficiently
represent the information
 Reduces ―size‖ of data
 Analog - Encode analog source data into a binary format
 Digital - Reduce the ―size‖ of digital source data
 Channel Coding ( Error Control) - Code data for transmission
over a noisy communication channel
 Increases ―size‖ of data
 Digital - add redundancy to identify and correct errors
 Analog - represent digital values by analog signals
ERROR
CONTROL
CODING
2 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022

Channel Coding
Types of Coding
 Source Coding (Data Compression)- Code data to more efficiently
represent the information
 Reduces ―size‖ of data
 Analog - Encode analog source data into a binary format
 Digital - Reduce the ―size‖ of digital source data
 Channel Coding ( Error Control) - Code data for transmission over
a noisy communication channel
 Increases ―size‖ of data
 Digital - add redundancy to identify and correct errors
 Analog - represent digital values by analog signals
ERROR
CONTROL
CODING
11/24/2022

Channel Coding
Types of Coding
 Cryptographic Coding - Cryptography or cryptographic coding is the practice
and study of techniques for secure communication in the presence of third
parties (called adversaries).
 Cryptography makes sure that messages remain unreadable — except to the intended recipient.
 More generally, it is about constructing and analyzing protocols that block adversaries.
 Line Coding - A line code (also called digital baseband modulation or digital
baseband transmission method) is a code chosen for use within a communication
system for baseband transmission purposes.
 Line coding is often used for digital data transport.
 Line coding consists of representing the digital signal to be transported by an amplitude- and time-
discrete signal that is optimally tuned for the specific properties of the physical channel (and of the
receiving equipment).
 The common types of line encoding are unipolar, polar, bipolar, and Manchester encoding.
ERROR
CONTROL
CODING
11/24/2022

Channel Coding
Why Channel Coding?
 To increase the resistance of digital communication systems to
channel noise via error control coding.
How Channel Coding Works?
 By mapping the incoming data sequence into a channel input
sequence and inverse mapping the channel output sequence into
an output data sequence in such a way that the overall effect of
channel noise on the system is minimized
 Redundancy is introduced in the channel encoder so as to
reconstruct the original source sequence as accurately as
possible.
ERROR
CONTROL
CODING
11/24/2022

Channel Coding
The implication of Error Control Coding (Channel Coding)
 Addition of redundancy implies the need for increased
transmission bandwidth.
 It also adds complexity in the decoding operation.
 Therefore, there is a design trade-off in the use of
error-control coding.
 achieving acceptable error performance shall be traded for
bandwidth and system complexity.
ERROR
CONTROL
CODING
11/24/2022

Channel Coding
 Channel Coding Taxonomy
ERROR
CONTROL
CODING
11/24/2022

Channel Coding
Generally
 Coding is used for error detection and/or error correction.
 Coding is a compromise between reliability, efficiency, equipment
complexity.
 Channel Coding is mapping of binary source (usually) output sequences
of length k into binary channel input sequences n (>k).
 Binary coding produces 2k codewords of length n.
 Extra bits in codewords are used for error detection/correction.
ERROR
CONTROL
CODING
11/24/2022

Channel Coding
 The most popular channel coding techniques which are realized
by binary numbers are block codes and convolutional codes.
 Block codes: mapping of information source into channel
inputs is done independently: Encoder output depends only on
the current block of input sequence.
 Convolutional codes: each source bit influences n(L+1)
channel input bits. n(L+1) is the constraint length and L is the
memory depth. These codes are denoted by (n,k,L).
ERROR
CONTROL
CODING
11/24/2022

Block Codes
 It is in the form of (n,k) block code where n is the number of bits of the codeword
and k is the number of bits for the binary message (Dataword).
 To generate an (n,k) block code, the channel encoder accepts information in
successive k-bit blocks.
 For each block add (n-k) redundant bits to produce an encoded block of n-bits called
a codeword.
 The (n-k) redundant bits are algebraically related to the k message bits.
ERROR
CONTROL
CODING
11/24/2022

Block Codes
 The channel encoder produces bits at a rate called the channel
data rate, R0
ERROR
CONTROL
CODING
12
s
R
k
n
R 






0
Where Rs is the bit rate of the information
source and n/k is the code rate
HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022

Block Codes
Forward Error-Correction (FEC)
 The channel encoder accepts information in successive k-bit blocks and for
each block it adds (n-k) redundant bits to produce an encoded block of n-bits
called a codeword.
 The channel decoder uses the redundancy to decide which message bits
were actually transmitted.
 The central idea is the sender encodes the message with redundant
information in the form of an ECC (Error Correction Code).
 The redundancy allows the receiver
 to detect a limited number of errors that may occur anywhere in the message, and
 to correct these errors without retransmission.
ERROR
CONTROL
CODING
11/24/2022

Block Codes
 In this case, whether the decoding of the received code
word is successful or not, the receiver does not perform
further processing for retransmission.
 i.e, if an error is detected in a transmitted code word, the
receiver does not request for retransmission of the
corrupted code word.
 Appropriate for delay sensitive and one-way transmission
(e.g., broadcast TV) of data
ERROR
CONTROL
CODING
11/24/2022

Block Codes
 ECC is accomplished by adding redundancy to the transmitted
information using an algorithm.
 A redundant bit may be a complex function of many original
information bits.
 The original information may or may not appear literally in the
encoded output;
 codes that include the unmodified input in the output are systematic,
 while those that do not are non-systematic.
ERROR
CONTROL
CODING
11/24/2022

Block Codes
Backward Error-Correction (BEC)-
 BEC can be realized by Using Automatic-Repeat Request Scheme (ARQ)
 In this scheme, upon detection of error, the receiver requests a repeat
transmission of the corrupted codeword
 The major advantage of ARQ over Forward Error Correction (FEC)
schemes is that
 error detection requires much simpler decoding mechanisms and much less
redundancy than error correction.
 There are 3 types of ARQ scheme
• Stop-and-Wait
• Continuous ARQ with pullback
• Continuous ARQ with selective repeat
ERROR
CONTROL
CODING
11/24/2022

Block Codes
Backward Error-Correction (BEC)-Types
 Stop-and-wait
 A block of message is encoded into a code word and transmitted
 The transmitter stops and waits for feedback from the receiver
either an acknowledgement of a correct receipt of the codeword
or a retransmission request due to error in decoding.
 The transmitter resends the codeword before moving onto the
next block of message.
ERROR
CONTROL
CODING
11/24/2022

Block Codes
 Continuous ARQ with pullback (Go-back-N)
 Allows the receiver to send a feedback signal while the
transmitter is sending another codeword
 The transmitter continues to send a succession of code words
until it receives a retransmission request.
 It then stops and pulls back to the particular code word that was
not correctly decoded and retransmits the complete sequence of
code words starting with the corrupted one.
ERROR
CONTROL
CODING
11/24/2022

Block Codes
 Continuous ARQ with selective repeat
 Retransmits the code word that was incorrectly decoded only.
 Thus, eliminates the need for retransmitting the successfully
decoded code words.
ERROR
CONTROL
CODING
11/24/2022

Important Terminologies
 Modular Arithmetic- In modulo-N arithmetic, we use only the integers in the range
0 to N −1, inclusive. We define an upper limit, called a Modulus N. We then use only
the integers 0 to N −1, inclusive.
 Modulo-2 Arithmetic-The encoding and decoding functions involve the binary
arithmetic operation (XOR).
 Rules for modulo-2 operations are:
ERROR
CONTROL
CODING
11/24/2022

 Hamming Distance- The Hamming distance between two words d(x,y)
over the same alphabet is the number of places where the symbols
differ. It measures
 the minimum number of substitutions required to change one word into the other,
 the minimum number of errors that could have transformed one word into the
other.
 The Hamming distance can easily be found if we apply the XOR operation
( ) on the two words and count the number of 1s in the result.
 Minimum Hamming Distance- The minimum Hamming distance, dmin, is
the smallest Hamming distance between all possible pairs in a set of
words.
ERROR
CONTROL
CODING
11/24/2022

 Example- Find the minimum Hamming distance of the coding scheme
presented in table below.
 Solution- We first find all Hamming distances.
 Then, dmin in this case becomes 3.
ERROR
CONTROL
CODING
11/24/2022

 Therefore,
 The Hamming distance between the received cordword and the sent codeword
is the number of bits that are corrupted during transmission.
 To guarantee the detection of up to s errors in all cases, the minimum
Hamming distance in a block code must be
dmin = s + 1.
 Then, the received codeword does not match a valid codeword.
 Code strength is measured by Hamming distance that tells how
different codewords are:
 Codes are more powerful when their minimum Hamming distance dmin (over all
codes in the code family) is large.
ERROR
CONTROL
CODING
11/24/2022

 To guarantee the detection of up to s errors in all cases, the minimum
dmin = s + 1.
 Geometrical concept for dmin in error detection
ERROR
CONTROL
CODING
11/24/2022

 To guarantee correction of up to t errors in all cases, the minimum
dmin = 2t + 1.
 Geometrical concept for finding dmin for error correction
ERROR
CONTROL
CODING
11/24/2022

Hamming distance: The decision sphere interpretation
 It can be seen that we can detect s=dmin-1 errors in the codewords.
This is because the only way to NOT to detect the error is that the
error completely transforms the code into another codeword. This
requires the change of at least dmin code bits. Therefore the error
detection upper bound is dmin-1.
 Also, we can see that we can correct t=(dmin-1)/2 errors. If more
errors occur, the received word may fall into the decoding sphere of
another code word (see the above figure).
ERROR
CONTROL
CODING
11/24/2022

Hamming distance: The decision sphere interpretation
 Consider two block code (n,k) words c1 and c2 at the Hamming
distance in the n-dimensional code space:
ERROR
CONTROL
CODING
27
,
min ( , )
i j
i j
d d c c


1
c 2
c
/ 2
d
11/24/2022

Hamming distance: Example
0100→1001 has distance 3;
0110→1110 has distance 1
ERROR
CONTROL
CODING
11/24/2022

Example
 Look the previous example, this code has dmin=3, this code detects up to two errors.
Again, we see that when any of the valid codewords is sent, two errors create a
codeword which is not in the table of valid codewords.
 However, some combinations of three errors change a valid codeword to another
valid codeword. The receiver accepts the received codeword and the errors are
undetected.
ERROR
CONTROL
CODING
11/24/2022

Properties of Hamming Distance
Take away
 Hamming Distance-The Hamming distance between two
words over the same alphabet is the number of places
where the symbols differ.
ERROR
CONTROL
CODING
11/24/2022

 The binomial distribution with parameters n and p is the
number of successes in a sequence of n independent
experiments, each asking a yes–no question, and each with its
own Boolean-valued outcome:
 Success (with probability p)
 Failure (with probability q = 1 − p).
 A single success/failure experiment is also called a Bernoulli
trial or Bernoulli experiment, and
 A sequence of outcomes is called a Bernoulli process; for a
single trial, i.e., n = 1, the binomial distribution is a Bernoulli
distribution.
ERROR
CONTROL
CODING
11/24/2022

Binomial Distribution
 If the random variable X follows the binomial distribution with parameters
n ∈ ℕ and p ∈ [0,1], The probability of getting exactly k successes in n
independent Bernoulli trials is given by the probability mass function:
 The formula can be understood as follows: k successes occur with
probability pk and n − k failures occur with probability (1 − p)n − k.
 However, the k successes can occur anywhere among the n trials, and
there are different ways of distributing k successes in a sequence of
n trials.
ERROR
CONTROL
CODING
11/24/2022

 Def: The weight of a codeword ci , denoted by w(ci),
is the number of nonzero elements in the codeword.
 Def: The minimum weight of a code, wmin, is the
smallest weight of the nonzero codewords in the
code.
 Theorem: In any linear code, dmin = wmin
ERROR
CONTROL
CODING
11/24/2022

Linear Block Codes
 A code is said to be linear if any two valid codewords in the
code can be added in modulo-2 arithmetic to produce a
third valid codeword in the code.
 i.e, if m and m’ have respective codewords c(m) and c(m’),
then c(m + m’)=c(m) + c(m’) is a codeword for message (m
+ m’)
 Examples
ERROR
CONTROL
CODING
11/24/2022

Linear Block Codes
 Types of Linear Block Codes
 The Repetition Code
 Parity-check Codes
 Systematic Block Codes
 Hamming Code
 Cyclic Codes
ERROR
CONTROL
CODING
11/24/2022

Linear Block Codes-The Repetition Code
 The repetition code is one of the most basic error-correcting codes.
 The idea of the repetition code is to just repeat the message several
times.
 The hope is that the channel corrupts only a minority of these
repetitions.
 This way the receiver will notice that a transmission error occurred
since the received data stream is not the repetition of a single
message,
 The receiver can recover the original message by looking at the
received message in the data stream that occurs most often.
ERROR
CONTROL
CODING
11/24/2022

 It has bad error correcting performance and low code rate.
 The advantages of the repetition code is the ease of implementation,
and reliability.
ERROR
CONTROL
CODING
11/24/2022

 In the case of a binary repetition code, there exist two codewords - all
ones and all zeros - which have a length of n.
 Therefore, the minimum Hamming distance of the code equals its
length n.
 This gives the repetition code an error correcting capacity of (n−1)/ 2.
 If the length of a binary repetition code is odd, then it's a perfect
code.
 The binary repetition code of length n is equivalent to the (n,1)-Hamming
code.
 It uses the Maximum Likelihood Decoding-it decides on the most
probable
ERROR
CONTROL
CODING
11/24/2022

 Consider for instance (3,1) repetition code, yielding the code rate
 Encoded word is formed by the simple coding rule:
 Code is decoded by majority voting, e.g. for instance:
 the probability that a group of three repeated bits decoded correctly
is given by
pr (no error) + pr(one error)=p3+3p2(1-p)
ERROR
CONTROL
CODING
39
/ 1/3
C
R k n
 
1 111 0 000
 
001 0, 101 1
 
11/24/2022

 Assuming binomial error distribution, the bit error is given
by :
 From our example, Error in decoding is introduced if all the
bits are inverted or two bits are inverted (by noise or
interference), e.g. majority of bits are in-error
ERROR
CONTROL
CODING
40
( , ) (1 ) , 1
   

   
   
   
   
i n i i
n n
P i n
i i
2 3
(2,3) (3,3) 3 2
 
   
we
P P P
11/24/2022

 In a three bit codeword
 Always one error can be corrected, because majority voting can
detect and correct one codeword bit error.
 Always two errors can be detected, because all codewords must
be all zeros or all ones (but now the encoded bit can not be
recovered).
 Decoding error occurs if at least (n+1)/2 of the transmitted symbols
are received in error. Therefore the error probability can be
expressed as
 or
ERROR
CONTROL
CODING
41
( 1)/ 2
(1 )
n
k n k
e
k n
n
p
k
  
 
 
 
 
 

11/24/2022

Points for Discussion
ERROR
CONTROL
CODING
42
 For a transmitted message of length n there are n +1
situations the receiver has to distinguish between:
 no errors and a single error in a specified position along the string
of n received bits.
 Any linear code can be transformed into an equivalent
systematic code.
 A systematic code is one where every n-bit codeword can
be represented as the original k-bit message followed by
the n -k parity bits.
11/24/2022

Points for Discussion
ERROR
CONTROL
CODING
43
 Given a systematic code, how many parity bits do we absolutely need in
Single Error Correction Code?
 We need to choose n so that single error correction is possible and the
error occurs in any position.
 Since there are n -k parity bits, each combination of these bits must
represent some error condition.
 For (n,k) codes, there are 2n-k possible distinct parity bit combinations.
 Therefore, we can distinguish at most that many error conditions. We
therefore arrive at the constraint
 Taking the log (to base 2) of both sides, we can see that the number of parity
bits must grow at least logarithmically with the number of message bits
11/24/2022

Linear Block Codes-Parity-check Codes
 Codes are based on the notion of parity.
 The parity of a binary word is said to be even when the word
contains an even number of 1s and odd parity when it has
odd number of 1s.
 A group of n-bits codewords are constructed from a group
of n-1 message bits.
 One check bit is added to the n-1 message bits such that all
the codewords have the same parity
 When the received codeword has different parity, we know
that an error has occurred
ERROR
CONTROL
CODING
11/24/2022

 Advantages of Parity check codes- its simplicity and low
overhead.
ERROR
CONTROL
CODING
11/24/2022

 Example : n=3 and even parity
 The binary message are 00,01,10,11
 The check bit is added such that all the code words have even
parity
 So, the resulting code words are 000,011,101 and 110
ERROR
CONTROL
CODING
11/24/2022

 A simple parity-check code is a single-bit error-detecting
code in which n = k + 1 with dmin = 2.
 Single Parity check has
 Code Rate R = k/n = k/(k+1)
 Error detecting capability = 1
 Error correcting capability = 0
ERROR
CONTROL
CODING
11/24/2022

 Encoder and decoder for simple parity-check code
syndrome bit helps the receiver diagnose the ―illness‖ (errors) in the received data
ERROR
CONTROL
CODING
11/24/2022

Example
 Let us look at some transmission scenarios. Assume the sender sends
the dataword 1011. The codeword created from this dataword is 10111,
which is sent to the receiver. We examine five senarios:
1. No error occurs; the received codeword is 10111. The syndrome is 0.
The dataword 1011 is created.
2. One single-bit (error changes a1 ). The received codeword is 10011.
The syndrome is 1. No dataword is created.
3. One single-bit ( error changes r0 ). The received codeword is 10110.
The syndrome is 1. No dataword is created.
ERROR
CONTROL
CODING
11/24/2022

Example-Continued
4. Two bits ( An error changes r0 and a second error changes a3). The
received codeword is 00110. The syndrome is 0. The dataword 0011 is
created at the receiver. Note that here the dataword is wrongly created
due to the syndrome value.
5. Three bits—a3, a2, and a1—are changed by errors. The received
codeword is 01011. The syndrome is 1. The dataword is not created. This
shows that the simple parity check, guaranteed to detect one single error,
can also find any odd number of errors.
Therefore,
A simple parity-check code can detect an odd number of errors.
ERROR
CONTROL
CODING
11/24/2022

Linear Block Codes-Parity-check Codes (2-D)
 In Two-dimensional parity-check code, the data code is organized in a
table (rows and columns) form.
 The data to be sent are arranged in rows.
 For each row and each column, 1 parity-check bit is calculated.
 The whole table is then sent to the receiver, which finds the syndrome for each
row and each column.
 The two-dimensional parity check can detect up to three errors that occur
anywhere in the table.
ERROR
CONTROL
CODING
11/24/2022

Linear Block Codes-Parity-check Codes (2-D)
 Two-dimensional parity-check code
ERROR
CONTROL
CODING
11/24/2022

Linear Block Codes-Systematic Block Codes
 In (n,k) block codes each sequence of k information bits is mapped into a
sequence of n channel inputs in a fixed way regardless of previous
information bits.
 The formed code family should be selected such that the code minimum
distance is as large as possible -> high error correction or detection
capability
 Definition: A systematic block code:
– the first k elements are the same as the message bits
– the following r = n - k bits are the check bits
 Therefore, the encoded word is
 As partitioned representation
ERROR
CONTROL
CODING
53
1 2 1 2
message
check
( .... ... ),
r k
b b b m m m r n k
  
X
( | )
B

X M
11/24/2022

 Given the message vector M, the respective linear, systematic block
code X can be obtained by the matrix multiplication
 The matrix G is the generator matrix with the general structure
where Ik is kxk identity matrix and
P is called hamming code (or called parity check matrix), it is a kxr binary
submatrix ultimately determining the generated codes
ERROR
CONTROL
CODING
54

X MG
( | )
k

G P I
11 12 1
21 22 2
1 2
r
r
k k kr
p p p
p p p
p p p
 
 
 

 
 
 
P
11/24/2022

 For u message vectors M (each consisting of k bits) the respective n-bit block codes
X are therefore determined by

ERROR
CONTROL
CODING
11/24/2022

 The check vector B that is appended to the message in the encoded
word is thus determined by the multiplication
 The jth element of B on the uth row is therefore encoded by
 For the Hamming code (parity matrix), P matrix of k rows consists of
all r-bit words with two or more "1― is arranged in all orders! Hence P
can be (for instance)
ERROR
CONTROL
CODING
56
B  MP Note: X=(B|M)=MG = M(P|Ik)
Therefore: B = MP
, ,1 1, ,2 2, , , , 1...
u j u j u j u k k j
b m p m p m p j r
   
1 0 1
1 1 1
1 1 0
0 1 1
 
 
 

 
 
 
P
11/24/2022

Linear Block Codes-Hamming Code
 HAMMING CODES are types of linear block codes of (n, k) with the
following parameters
• Number of parity bits, m=n-k
• Codeword length, n = 2m - 1
• Dataword length, k =n-m= 2m – m -1
• Minimum distance dmin=3,
• m >= 3
i.e. Code with m=2 is Repetition Code.
 Therefore, the HAMMING CODE will have this general form of
expression
c(n, k)=(2m - 1, 2m – m -1)
ERROR
CONTROL
CODING
11/24/2022

 For each integer m ≥ 2 there is a code with block length n = 2m - 1 and
message length k = 2m− m− 1.
 Hence the rate of Hamming codes is R = k / n = 1 − {m / (2m − 1)},
 Hamming codes are perfect codes, that is, they achieve the highest
possible rate for codes with their block length and minimum distance
of three.
 Hamming Codes were originally designed with dmin =3, so they can be
used for single error correction (SEC) or dual error detection (DED).
 Therefore, Hamming codes are distance-3 linear block codes.
 The first four Hamming codes, for example, are (3,1), (7,4), (15,11), and
(31,26) codes.
ERROR
CONTROL
CODING
11/24/2022

 Drawbacks- it is used for single error correction and up to two errors detection
only.
 This is due to the limited redundancy that Hamming codes add to the data, they can only detect and
correct errors when the error rate is low.
 Advantages-it is easy and cost effective for single error detection and correction
applications and it is popular in computer memory because in computer memory
(usually RAM) the bit errors are extremely rare and Hamming codes are widely
used.
ERROR
CONTROL
CODING
11/24/2022

Hamming bound
 There is an inequality called the Hamming bound, which states that if there is an (n,k)
code with an alphabet of q elements and dmin = 2t + 1 then
 For each of the qk codewords there are exactly words that are exactly
distance i from it.
 Therefore, the number on the right of the inequality is the total number of words
that are at most a distance of t from a codeword.
 Since the total number of words cannot exceed qn , the inequality follows
immediately.
ERROR
CONTROL
CODING
11/24/2022

Linear Block Codes-Hamming Bound
 For a binary code becomes
 Codes that attain equality in the Hamming bound are called perfect codes.
 A perfect t –error correcting code has the property that every word in the Hamming
space lies within a distance of t from exactly one codeword.
 A Hamming space is usually the set of all 2 N binary strings of length N.
 More generally, a Hamming space can be defined over any alphabet (set) Q as the
set of words of a fixed length N with letters from Q.
ERROR
CONTROL
CODING
11/24/2022

 Assignment 3- Show How to generate/calculate Hamming
Codes? Develop Algorithm for Hamming Code.
ERROR
CONTROL
CODING
11/24/2022

 Example-Generate the Hamming Codes for parity bits m=3;
C(n, k)=(2m - 1, 2m – m -1)=C(7,4)
 let’s assume m be the message to be encoded , m=(m0, m 1 ,m2 ,m3) ,
and let x be the corresponding codedword, x=(x0, x 1 ,x2 ,x3 ,x4, x 5 ,x6)
 then we know that and
ERROR
CONTROL
CODING
63

X MG ( | )
k

G P I
 
k
I
|
P
G 






























k
k
k
n
k
1
0
0
0
0
1
0
0
0
0
1
0
0
0
0
1
1
0
1
1
1
1
1
1
0
0
1
1
)
(
11/24/2022

 for m=(m0, m 1 ,m2 ,m3) , codeword x=(x0, x 1 ,x2 ,x3 ,x4, x 5 ,x6)
becomes
x0= m0 + m2 +m3
x1 =m0 + m1 +m2
x 2=m1 + m2 +m3
x3 = m0
x4 = m1
x 5= m2
x6= m3
ERROR
CONTROL
CODING
11/24/2022

 Hamming Codes for C(7,4)
ERROR
CONTROL
CODING
11/24/2022

 The structure of the encoder and decoder for a Hamming code C(7,4)
ERROR
CONTROL
CODING
11/24/2022

 Generator creates 3 parity-check bit (r0 ,r1 , and r2)
 r0 = a2 + a1 + a0 modulo-2
 r1 = a3+ a2 + a1 modulo-2
 r2 = a1 + a0 + a3 modulo-2
 The checker in the decoder creates a 3bit syndrome bit
 s0 = b2 + b1 + b0 + q0 modulo-2
 s1 = b3 + b2 + b1 + q1 modulo-2
 s2 = b1 + b0 + b3 + q2 modulo-2
ERROR
CONTROL
CODING
11/24/2022

 For the Hamming code (parity matrix), P matrix of k rows
consists of all r-bit words with two or more "1― is
arranged in all orders! Hence P can be (for instance)
ERROR
CONTROL
CODING
68
1 0 1
1 1 1
1 1 0
0 1 1
 
 
 

 
 
 
P
11/24/2022

 .
ERROR
CONTROL
CODING
11/24/2022

Linear Block Codes-Hamming Code-Single Error Correction,
Double Error Detection
 Can achieve this by adding an overall parity bit.
 If parity checks are correct and overall parity bit are
correct, then no single or double errors occurred.
 If overall parity bit is incorrect, then single error has
occurred, can use previous to correct.
 If one or more of parity checks incorrect but overall
parity bit is correct, then two errors are detected.
ERROR
CONTROL
CODING
11/24/2022

Linear Block Codes-Extended Hamming Code (Reading
Assignment)
 Extended Hamming codes
 achieve a Hamming distance of four,
 allows the decoder to distinguish between when at most one one-bit
error occurs and when any two-bit errors occur.
 since it cannot distinguish a double bit error of some codeword
from a single bit error of a different codeword.
 Thus, some double-bit errors will be incorrectly decoded as if they
were single bit errors and therefore go undetected, unless no
correction is attempted.
 In this sense, extended Hamming codes are single-error correcting
and double-error detecting, abbreviated as SECDED.
ERROR
CONTROL
CODING
11/24/2022

Linear Block Codes-Cyclic Codes
 Cyclic code is a block code, where the circular shifts of each
codeword gives another word that belongs to the code.
 They are error-correcting codes that have algebraic properties that
are convenient for efficient error detection and correction.
Definition:- A code C is cyclic if
 C is a linear code;
 Any cyclic shift of a codeword is also a codeword,
 A cyclic shift of a vector [v0 v1 … vn-2 vn-1] is the vector
[vn-1 v1 … vn-2 ].
 Cyclically shifting v, i places to the right is equivalent to
cyclically shifting v, n − i places to the left.
ERROR
CONTROL
CODING
11/24/2022

Linear Block Codes-Cyclic Codes-Polynomial
Representation
Polynomial representation of vectors
 For every vector v=[v0 v1 … vn-2 vn-1] there is a polynomial
 Let v(i) be the vector resulting from i cyclic shifts on v
ERROR
CONTROL
CODING
11/24/2022

Representation
Consider vector v(X) andv(1) (X) and using Modulo 2 arithmetic
v(X) andv(i) (X) are related by
ERROR
CONTROL
CODING
11/24/2022

Representation
Proof
ERROR
CONTROL
CODING
11/24/2022

Representation
 Property 1- The nonzero code polynomial of minimum
degree in a linear block code (Cyclic code, C) is unique.
 Let be the
nonzero code polynomial of minimum degree in an (n, k)
cyclic code C. Then the constant term g0 must be equal to 1.
ERROR
CONTROL
CODING
11/24/2022

Representation
 Property 2- Consider the polynomial
Clearly, they are cyclic shifts of g(x) and hence code
polynomials in C. Since C is linear, a linear combination of
is also a code polynomial
where ui ∈ {0, 1}.
ERROR
CONTROL
CODING
11/24/2022

Representation
.
ERROR
CONTROL
CODING
11/24/2022

Representation
 Let be the
nonzero code polynomial of minimum degree in an (n, k)
cyclic code C.
 Property 3- A binary polynomial of degree n-1 or less is a code
polynomial if and only if it is a multiple of g(x).
 Proof: Let v(x) be a binary polynomial of degree n − 1 or less. Suppose
that v(x) is a multiple of g(x). Then
 Since v(x) is a linear combination of the code polynomials, it
is a code polynomial in C.
ERROR
CONTROL
CODING
11/24/2022

Representation
 Now let v(x) be a code polynomial in C. Dividing v(x) by g(x), we obtain
Where the degree of b(x) is less than the degree of g(x).
 Since v(x) and a(x)g(x) are code polynomials, b(x) is also a code
polynomial.
 Suppose b(x) not = 0. Then b(x) is a code polynomial with less degree
than that of g(x). Contradiction.
ERROR
CONTROL
CODING
11/24/2022

Representation
 Property 4- The number of binary polynomials of degree n − 1 or less
that are multiples of g(x) is 2n−r.
 Property 5- There are total of 2k = 2n−r code polynomials in C, i.e.,
r = n − k.
 Property 6- The polynomial g(x) is called the generator polynomial
of the code C. If
 The degree of g(x) is equal to the number of parity-check digits of the code.
 The generator polynomial g(x) of an (n, k) cyclic code is a factor of xn + 1.
ERROR
CONTROL
CODING
11/24/2022

Representation
 Property 7- If g(x) is a polynomial of degree n − k and is a factor of xn + 1, then g(x)
generates an (n, k) cyclic code.
 v(x) is a polynomial of degree n − 1 or less and is a multiple of g(x).
 There are a total of 2k such polynomial and they form an (n, k) linear code.
ERROR
CONTROL
CODING
11/24/2022

Linear Block Codes-Cyclic Codes-Systematic
Encoding of Cyclic Codes
 Since both xv(x) and xn + 1 are divisible by g(x), v(1)(x) must be divisible by g(x).
 Hence, v(1)(x) is a code polynomial and the code generated by g(x) is a cyclic code.
 Systematic Encoding of Cyclic Codes
 Suppose we have the message u = (u0, u1, . . . , uk−1) to be encoded, Then
 Since the degree of g(x) is n − k, the degree of b(x) must be n − k − 1 or less. Then
is a multiple of g(x) and therefore it is a code polynomial.
ERROR
CONTROL
CODING
11/24/2022

Linear Block Codes-Cyclic Codes-Systematic
Encoding of Cyclic Codes
 Then, the coded polynomial can be written as follows
 Therefore, a systematic encoding of the message can be obtained as follows
 First- Divide xn-ku(x) by g(x) to obtain remainder b(x), and then
 The code polynomial is given by b(x) + xn-ku(x)
 Given message bits, (mk-1…m1m0 ), the code is generated simply as:
 In other words, C(x) can be considered as the product of m(x) and g(x).
ERROR
CONTROL
CODING
11/24/2022

Linear Block Codes-Cyclic Codes-Shift Register
Circuit
 Shift Register Circuit
ERROR
CONTROL
CODING
11/24/2022

Circuit
 If the initial state polynomial is zero and the input is a sequence of bits am-1, am-2, …. a1, a0, the
final state polynomial is
 For systematic encoding we need Xn-ku(X) mod g(X) which corresponds to input bit sequence
ERROR
CONTROL
CODING
11/24/2022

Circuit
 Is there a way to avoid the delay of n - k clock ticks?
ERROR
CONTROL
CODING
11/24/2022

Circuit
 If the initial state polynomial is zero and the input is a sequence of bits am-1, am-2, …. A1, a0, the
final state polynomial is
ERROR
CONTROL
CODING
11/24/2022

Circuit
 Turn on the gate. Shift the message bits uk-1, uk-2, …, u0 into the circuit and channel
simultaneously. Only Output1 is fed to the channel.
 Turn off the gate and shift the contents of the shift register into the channel. Only Output2 is
fed to the channel.
ERROR
CONTROL
CODING
11/24/2022

Circuit
 Encoding circuit for an (n, k) cyclic code with generator polynomial
g(X) = 1 + g1X + g2X2 + · · · + gn−k−1Xn−k−1 + Xn−k
The operation of the encoding circuit is described as follows:
 1) Initially, the gate is turned on. The, k information digits, message polynomial u(X) = u0+u1X+
・・・+uk−1Xk−1 is fed to the circuit as well as transmitted into the channel. Feeding the k
information digits into the circuit is equivalent to pre-multiplying u(X) by Xn−k. When all k
information digits are shifted into the circuit, the (n − k) digits in the register form the
remainder.
ERROR
CONTROL
CODING
11/24/2022

Circuit
 Encoding circuit for an (n, k) cyclic code with generator polynomial
g(X) = 1 + g1X + g2X2 + · · · + gn−k−1Xn−k−1 + Xn−k
 2) The gate is then turned off, since the register now contains the desired (n − k) parity
check digits.
 3) Selector is changed to the left position to send parity check digits into the channel. These
(n−k) parity check digits along with k information digits form the cyclic codeword in
systematic form.
ERROR
CONTROL
CODING
11/24/2022
11/24/2022

Linear Block Codes-Cyclic Codes-Error Detection
 Errors are detected when the received vector is not a codeword
 For linear block codes, r is a codeword iff rHT = 0
 s = rHT is called the syndrome vector
 For cyclic codes, the received polynomial r(X) is a code polynomial iff
r(X) mod g(X) = 0
 s(X) = r(X) mod g(X) is called the syndrome polynomial, or
 r(X) = a(x) g(X) + s(X) , n-k cofficient of s(X) makes the syndrome
 The following circuit computes the syndrome polynomial
ERROR
CONTROL
CODING
11/24/2022
11/24/2022

 Property-10- Let s(x) be the syndrome of a received polynomial r(x).
Then the remainder s(1)(x) resulting from dividing xs(x) by the
generator polynomial g(x) is the syndrome of r(1)(x), which is a cyclic
shift of r(x).
 Property-11-The remainder s(i)(x) resulting from dividing xis(x) by the
generator polynomial g(x) is the syndrome of r(i)(x), which is the ith
cyclic shift of r(x).
ERROR
CONTROL
CODING
11/24/2022

Linear Block Codes-Cyclic Codes
 Example
ERROR
CONTROL
CODING
11/24/2022

 Example
ERROR
CONTROL
CODING
11/24/2022

.
ERROR
CONTROL
CODING
11/24/2022

Decoding of Cyclic Codes
 Decoding of linear codes consists of three steps:
1) Syndrome computation;
2) association of the syndrome to an error pattern;
3) error correction.
 The cyclic structure of a cyclic code allows us to decode a received
vector r(x) in serial manner.
 The decoding circuit checks whether the syndrome s(x) corresponds
to a correctable error pattern e(x) with an error at the highest-order
position xn−1 (i.e., en−1 = 1).
ERROR
CONTROL
CODING
11/24/2022

 If s(x) does not correspond to an error pattern with en−1 = 1,
 the received polynomial and the syndrome register are cyclically
shifted once simultaneously.
 By doing this, we have r(1)(x) and s(1)(x).
 The second digit rn−2 of r(x) becomes the first digit of r(1)(x).
 The same decoding processes.
 If the syndrome s(x) of r(x) does correspond to an error pattern with
an error at the location xn−1, the first received digit rn−1 is an
erroneous digit and it must be corrected by taking the sum rn−1 ⊕
en−1.
ERROR
CONTROL
CODING
11/24/2022

 This correction results in a modified received polynomial, denoted by
r1(x) = r0 + r1x + ・・・ + rn−2xn−2 + (rn−1 ⊕ en−1)xn−1.
 The effect of the error digit en−1 on the syndrome can be achieved by
adding the syndrome of e(x) = xn−1 to s(x).
 The syndrome s(1)
1 of r(1)
1 (x) is the remainder resulting from dividing
x[s(x) + xn−1] by the generator polynomial g(x).
ERROR
CONTROL
CODING
11/24/2022

Performance Analysis of Linear Block Codes: The Repetition Code,Parity-
check Codes, Systematic Block Codes, Hamming Code, Cyclic Codes.
Use MATLAB simulator to show the performance.
 Due Date 24/11/2022
102
Assignment 2
11/24/2022
11/24/2022 ERROR CONTROL CODING

Introduction to Channel Coding.pdf