9953056974 Call Girls In South Ex, Escorts (Delhi) NCR.pdf
Introduction to Channel Coding.pdf
1. Course Coordinator:-
Dr. Mulugeta Atlabachew (Ass. Professor)
HARAMAYA UNIVERSITY
HARAMAYA INSTITUTE OF TECHNOLOGY
SCHOOL OF ELECTRICAL AND COMPUTER
ENGINEERING
Course Coordinator:-
Dr. Mulugeta Atlabachew (Ass. Professor),
Guest Lecturer
Introduction to Channel Coding
2. Channel Coding
Types of Coding
Source Coding (Data Compression)- Code data to more efficiently
represent the information
Reduces ―size‖ of data
Analog - Encode analog source data into a binary format
Digital - Reduce the ―size‖ of digital source data
Channel Coding ( Error Control) - Code data for transmission
over a noisy communication channel
Increases ―size‖ of data
Digital - add redundancy to identify and correct errors
Analog - represent digital values by analog signals
ERROR
CONTROL
CODING
2 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
3. Channel Coding
Types of Coding
Source Coding (Data Compression)- Code data to more efficiently
represent the information
Reduces ―size‖ of data
Analog - Encode analog source data into a binary format
Digital - Reduce the ―size‖ of digital source data
Channel Coding ( Error Control) - Code data for transmission over
a noisy communication channel
Increases ―size‖ of data
Digital - add redundancy to identify and correct errors
Analog - represent digital values by analog signals
ERROR
CONTROL
CODING
3 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
4. Channel Coding
Types of Coding
Cryptographic Coding - Cryptography or cryptographic coding is the practice
and study of techniques for secure communication in the presence of third
parties (called adversaries).
Cryptography makes sure that messages remain unreadable — except to the intended recipient.
More generally, it is about constructing and analyzing protocols that block adversaries.
Line Coding - A line code (also called digital baseband modulation or digital
baseband transmission method) is a code chosen for use within a communication
system for baseband transmission purposes.
Line coding is often used for digital data transport.
Line coding consists of representing the digital signal to be transported by an amplitude- and time-
discrete signal that is optimally tuned for the specific properties of the physical channel (and of the
receiving equipment).
The common types of line encoding are unipolar, polar, bipolar, and Manchester encoding.
ERROR
CONTROL
CODING
4 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
5. Channel Coding
Why Channel Coding?
To increase the resistance of digital communication systems to
channel noise via error control coding.
How Channel Coding Works?
By mapping the incoming data sequence into a channel input
sequence and inverse mapping the channel output sequence into
an output data sequence in such a way that the overall effect of
channel noise on the system is minimized
Redundancy is introduced in the channel encoder so as to
reconstruct the original source sequence as accurately as
possible.
ERROR
CONTROL
CODING
5 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
6. Channel Coding
The implication of Error Control Coding (Channel Coding)
Addition of redundancy implies the need for increased
transmission bandwidth.
It also adds complexity in the decoding operation.
Therefore, there is a design trade-off in the use of
error-control coding.
achieving acceptable error performance shall be traded for
bandwidth and system complexity.
ERROR
CONTROL
CODING
7 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
7. Channel Coding
Channel Coding Taxonomy
ERROR
CONTROL
CODING
8 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
8. Channel Coding
Generally
Coding is used for error detection and/or error correction.
Coding is a compromise between reliability, efficiency, equipment
complexity.
Channel Coding is mapping of binary source (usually) output sequences
of length k into binary channel input sequences n (>k).
Binary coding produces 2k codewords of length n.
Extra bits in codewords are used for error detection/correction.
ERROR
CONTROL
CODING
9 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
9. Channel Coding
The most popular channel coding techniques which are realized
by binary numbers are block codes and convolutional codes.
Block codes: mapping of information source into channel
inputs is done independently: Encoder output depends only on
the current block of input sequence.
Convolutional codes: each source bit influences n(L+1)
channel input bits. n(L+1) is the constraint length and L is the
memory depth. These codes are denoted by (n,k,L).
ERROR
CONTROL
CODING
10 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
10. Block Codes
It is in the form of (n,k) block code where n is the number of bits of the codeword
and k is the number of bits for the binary message (Dataword).
To generate an (n,k) block code, the channel encoder accepts information in
successive k-bit blocks.
For each block add (n-k) redundant bits to produce an encoded block of n-bits called
a codeword.
The (n-k) redundant bits are algebraically related to the k message bits.
ERROR
CONTROL
CODING
11 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
11. Block Codes
The channel encoder produces bits at a rate called the channel
data rate, R0
ERROR
CONTROL
CODING
12
s
R
k
n
R
0
Where Rs is the bit rate of the information
source and n/k is the code rate
HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
12. Block Codes
Forward Error-Correction (FEC)
The channel encoder accepts information in successive k-bit blocks and for
each block it adds (n-k) redundant bits to produce an encoded block of n-bits
called a codeword.
The channel decoder uses the redundancy to decide which message bits
were actually transmitted.
The central idea is the sender encodes the message with redundant
information in the form of an ECC (Error Correction Code).
The redundancy allows the receiver
to detect a limited number of errors that may occur anywhere in the message, and
to correct these errors without retransmission.
ERROR
CONTROL
CODING
13 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
13. Block Codes
Forward Error-Correction (FEC)
In this case, whether the decoding of the received code
word is successful or not, the receiver does not perform
further processing for retransmission.
i.e, if an error is detected in a transmitted code word, the
receiver does not request for retransmission of the
corrupted code word.
Appropriate for delay sensitive and one-way transmission
(e.g., broadcast TV) of data
ERROR
CONTROL
CODING
14 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
14. Block Codes
Forward Error-Correction (FEC)
ECC is accomplished by adding redundancy to the transmitted
information using an algorithm.
A redundant bit may be a complex function of many original
information bits.
The original information may or may not appear literally in the
encoded output;
codes that include the unmodified input in the output are systematic,
while those that do not are non-systematic.
ERROR
CONTROL
CODING
15 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
15. Block Codes
Backward Error-Correction (BEC)-
BEC can be realized by Using Automatic-Repeat Request Scheme (ARQ)
In this scheme, upon detection of error, the receiver requests a repeat
transmission of the corrupted codeword
The major advantage of ARQ over Forward Error Correction (FEC)
schemes is that
error detection requires much simpler decoding mechanisms and much less
redundancy than error correction.
There are 3 types of ARQ scheme
• Stop-and-Wait
• Continuous ARQ with pullback
• Continuous ARQ with selective repeat
ERROR
CONTROL
CODING
16 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
16. Block Codes
Backward Error-Correction (BEC)-Types
Stop-and-wait
A block of message is encoded into a code word and transmitted
The transmitter stops and waits for feedback from the receiver
either an acknowledgement of a correct receipt of the codeword
or a retransmission request due to error in decoding.
The transmitter resends the codeword before moving onto the
next block of message.
ERROR
CONTROL
CODING
17 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
17. Block Codes
Backward Error-Correction (BEC)-Types
Continuous ARQ with pullback (Go-back-N)
Allows the receiver to send a feedback signal while the
transmitter is sending another codeword
The transmitter continues to send a succession of code words
until it receives a retransmission request.
It then stops and pulls back to the particular code word that was
not correctly decoded and retransmits the complete sequence of
code words starting with the corrupted one.
ERROR
CONTROL
CODING
18 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
18. Block Codes
Backward Error-Correction (BEC)-Types
Continuous ARQ with selective repeat
Retransmits the code word that was incorrectly decoded only.
Thus, eliminates the need for retransmitting the successfully
decoded code words.
ERROR
CONTROL
CODING
19 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
19. Important Terminologies
Modular Arithmetic- In modulo-N arithmetic, we use only the integers in the range
0 to N −1, inclusive. We define an upper limit, called a Modulus N. We then use only
the integers 0 to N −1, inclusive.
Modulo-2 Arithmetic-The encoding and decoding functions involve the binary
arithmetic operation (XOR).
Rules for modulo-2 operations are:
ERROR
CONTROL
CODING
20 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
20. Important Terminologies
Hamming Distance- The Hamming distance between two words d(x,y)
over the same alphabet is the number of places where the symbols
differ. It measures
the minimum number of substitutions required to change one word into the other,
the minimum number of errors that could have transformed one word into the
other.
The Hamming distance can easily be found if we apply the XOR operation
( ) on the two words and count the number of 1s in the result.
Minimum Hamming Distance- The minimum Hamming distance, dmin, is
the smallest Hamming distance between all possible pairs in a set of
words.
ERROR
CONTROL
CODING
21 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
21. Important Terminologies
Example- Find the minimum Hamming distance of the coding scheme
presented in table below.
Solution- We first find all Hamming distances.
Then, dmin in this case becomes 3.
ERROR
CONTROL
CODING
22 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
22. Important Terminologies
Therefore,
The Hamming distance between the received cordword and the sent codeword
is the number of bits that are corrupted during transmission.
To guarantee the detection of up to s errors in all cases, the minimum
Hamming distance in a block code must be
dmin = s + 1.
Then, the received codeword does not match a valid codeword.
Code strength is measured by Hamming distance that tells how
different codewords are:
Codes are more powerful when their minimum Hamming distance dmin (over all
codes in the code family) is large.
ERROR
CONTROL
CODING
23 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
23. Important Terminologies
To guarantee the detection of up to s errors in all cases, the minimum
Hamming distance in a block code must be
dmin = s + 1.
Geometrical concept for dmin in error detection
ERROR
CONTROL
CODING
24 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
24. Important Terminologies
To guarantee correction of up to t errors in all cases, the minimum
Hamming distance in a block code must be
dmin = 2t + 1.
Geometrical concept for finding dmin for error correction
ERROR
CONTROL
CODING
25 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
25. Important Terminologies
Hamming distance: The decision sphere interpretation
It can be seen that we can detect s=dmin-1 errors in the codewords.
This is because the only way to NOT to detect the error is that the
error completely transforms the code into another codeword. This
requires the change of at least dmin code bits. Therefore the error
detection upper bound is dmin-1.
Also, we can see that we can correct t=(dmin-1)/2 errors. If more
errors occur, the received word may fall into the decoding sphere of
another code word (see the above figure).
ERROR
CONTROL
CODING
26 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
26. Important Terminologies
Hamming distance: The decision sphere interpretation
Consider two block code (n,k) words c1 and c2 at the Hamming
distance in the n-dimensional code space:
ERROR
CONTROL
CODING
27
,
min ( , )
i j
i j
d d c c
1
c 2
c
/ 2
d
HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
27. Important Terminologies
Hamming distance: Example
0100→1001 has distance 3;
0110→1110 has distance 1
ERROR
CONTROL
CODING
28 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
28. Important Terminologies
Example
Look the previous example, this code has dmin=3, this code detects up to two errors.
Again, we see that when any of the valid codewords is sent, two errors create a
codeword which is not in the table of valid codewords.
However, some combinations of three errors change a valid codeword to another
valid codeword. The receiver accepts the received codeword and the errors are
undetected.
ERROR
CONTROL
CODING
29 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
29. Properties of Hamming Distance
Take away
Hamming Distance-The Hamming distance between two
words over the same alphabet is the number of places
where the symbols differ.
ERROR
CONTROL
CODING
30 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
30. Properties of Hamming Distance
The binomial distribution with parameters n and p is the
number of successes in a sequence of n independent
experiments, each asking a yes–no question, and each with its
own Boolean-valued outcome:
Success (with probability p)
Failure (with probability q = 1 − p).
A single success/failure experiment is also called a Bernoulli
trial or Bernoulli experiment, and
A sequence of outcomes is called a Bernoulli process; for a
single trial, i.e., n = 1, the binomial distribution is a Bernoulli
distribution.
ERROR
CONTROL
CODING
31 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
31. Binomial Distribution
If the random variable X follows the binomial distribution with parameters
n ∈ ℕ and p ∈ [0,1], The probability of getting exactly k successes in n
independent Bernoulli trials is given by the probability mass function:
The formula can be understood as follows: k successes occur with
probability pk and n − k failures occur with probability (1 − p)n − k.
However, the k successes can occur anywhere among the n trials, and
there are different ways of distributing k successes in a sequence of
n trials.
ERROR
CONTROL
CODING
32 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
32. Properties of Hamming Distance
Def: The weight of a codeword ci , denoted by w(ci),
is the number of nonzero elements in the codeword.
Def: The minimum weight of a code, wmin, is the
smallest weight of the nonzero codewords in the
code.
Theorem: In any linear code, dmin = wmin
ERROR
CONTROL
CODING
33 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
33. Linear Block Codes
A code is said to be linear if any two valid codewords in the
code can be added in modulo-2 arithmetic to produce a
third valid codeword in the code.
i.e, if m and m’ have respective codewords c(m) and c(m’),
then c(m + m’)=c(m) + c(m’) is a codeword for message (m
+ m’)
Examples
ERROR
CONTROL
CODING
34 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
34. Linear Block Codes
Types of Linear Block Codes
The Repetition Code
Parity-check Codes
Systematic Block Codes
Hamming Code
Cyclic Codes
ERROR
CONTROL
CODING
35 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
35. Linear Block Codes-The Repetition Code
The repetition code is one of the most basic error-correcting codes.
The idea of the repetition code is to just repeat the message several
times.
The hope is that the channel corrupts only a minority of these
repetitions.
This way the receiver will notice that a transmission error occurred
since the received data stream is not the repetition of a single
message,
The receiver can recover the original message by looking at the
received message in the data stream that occurs most often.
ERROR
CONTROL
CODING
36 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
36. Linear Block Codes-The Repetition Code
It has bad error correcting performance and low code rate.
The advantages of the repetition code is the ease of implementation,
and reliability.
ERROR
CONTROL
CODING
37 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
37. Linear Block Codes-The Repetition Code
In the case of a binary repetition code, there exist two codewords - all
ones and all zeros - which have a length of n.
Therefore, the minimum Hamming distance of the code equals its
length n.
This gives the repetition code an error correcting capacity of (n−1)/ 2.
If the length of a binary repetition code is odd, then it's a perfect
code.
The binary repetition code of length n is equivalent to the (n,1)-Hamming
code.
It uses the Maximum Likelihood Decoding-it decides on the most
probable
ERROR
CONTROL
CODING
38 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
38. Linear Block Codes-The Repetition Code
Consider for instance (3,1) repetition code, yielding the code rate
Encoded word is formed by the simple coding rule:
Code is decoded by majority voting, e.g. for instance:
the probability that a group of three repeated bits decoded correctly
is given by
pr (no error) + pr(one error)=p3+3p2(1-p)
ERROR
CONTROL
CODING
39
/ 1/3
C
R k n
1 111 0 000
001 0, 101 1
HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
39. Linear Block Codes-The Repetition Code
Assuming binomial error distribution, the bit error is given
by :
From our example, Error in decoding is introduced if all the
bits are inverted or two bits are inverted (by noise or
interference), e.g. majority of bits are in-error
ERROR
CONTROL
CODING
40
( , ) (1 ) , 1
i n i i
n n
P i n
i i
2 3
(2,3) (3,3) 3 2
we
P P P
HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
40. Linear Block Codes-The Repetition Code
In a three bit codeword
Always one error can be corrected, because majority voting can
detect and correct one codeword bit error.
Always two errors can be detected, because all codewords must
be all zeros or all ones (but now the encoded bit can not be
recovered).
Decoding error occurs if at least (n+1)/2 of the transmitted symbols
are received in error. Therefore the error probability can be
expressed as
or
ERROR
CONTROL
CODING
41
( 1)/ 2
(1 )
n
k n k
e
k n
n
p
k
HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
41. Points for Discussion
ERROR
CONTROL
CODING
42
For a transmitted message of length n there are n +1
situations the receiver has to distinguish between:
no errors and a single error in a specified position along the string
of n received bits.
Any linear code can be transformed into an equivalent
systematic code.
A systematic code is one where every n-bit codeword can
be represented as the original k-bit message followed by
the n -k parity bits.
HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
42. Points for Discussion
ERROR
CONTROL
CODING
43
Given a systematic code, how many parity bits do we absolutely need in
Single Error Correction Code?
We need to choose n so that single error correction is possible and the
error occurs in any position.
Since there are n -k parity bits, each combination of these bits must
represent some error condition.
For (n,k) codes, there are 2n-k possible distinct parity bit combinations.
Therefore, we can distinguish at most that many error conditions. We
therefore arrive at the constraint
Taking the log (to base 2) of both sides, we can see that the number of parity
bits must grow at least logarithmically with the number of message bits
HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
43. Linear Block Codes-Parity-check Codes
Codes are based on the notion of parity.
The parity of a binary word is said to be even when the word
contains an even number of 1s and odd parity when it has
odd number of 1s.
A group of n-bits codewords are constructed from a group
of n-1 message bits.
One check bit is added to the n-1 message bits such that all
the codewords have the same parity
When the received codeword has different parity, we know
that an error has occurred
ERROR
CONTROL
CODING
44 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
44. Linear Block Codes-Parity-check Codes
Advantages of Parity check codes- its simplicity and low
overhead.
ERROR
CONTROL
CODING
45 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
45. Linear Block Codes-Parity-check Codes
Example : n=3 and even parity
The binary message are 00,01,10,11
The check bit is added such that all the code words have even
parity
So, the resulting code words are 000,011,101 and 110
ERROR
CONTROL
CODING
46 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
46. Linear Block Codes-Parity-check Codes
A simple parity-check code is a single-bit error-detecting
code in which n = k + 1 with dmin = 2.
Single Parity check has
Code Rate R = k/n = k/(k+1)
Error detecting capability = 1
Error correcting capability = 0
ERROR
CONTROL
CODING
47 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
47. Linear Block Codes-Parity-check Codes
Encoder and decoder for simple parity-check code
syndrome bit helps the receiver diagnose the ―illness‖ (errors) in the received data
ERROR
CONTROL
CODING
48 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
48. Linear Block Codes-Parity-check Codes
Example
Let us look at some transmission scenarios. Assume the sender sends
the dataword 1011. The codeword created from this dataword is 10111,
which is sent to the receiver. We examine five senarios:
1. No error occurs; the received codeword is 10111. The syndrome is 0.
The dataword 1011 is created.
2. One single-bit (error changes a1 ). The received codeword is 10011.
The syndrome is 1. No dataword is created.
3. One single-bit ( error changes r0 ). The received codeword is 10110.
The syndrome is 1. No dataword is created.
ERROR
CONTROL
CODING
49 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
49. Linear Block Codes-Parity-check Codes
Example-Continued
4. Two bits ( An error changes r0 and a second error changes a3). The
received codeword is 00110. The syndrome is 0. The dataword 0011 is
created at the receiver. Note that here the dataword is wrongly created
due to the syndrome value.
5. Three bits—a3, a2, and a1—are changed by errors. The received
codeword is 01011. The syndrome is 1. The dataword is not created. This
shows that the simple parity check, guaranteed to detect one single error,
can also find any odd number of errors.
Therefore,
A simple parity-check code can detect an odd number of errors.
ERROR
CONTROL
CODING
50 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
50. Linear Block Codes-Parity-check Codes (2-D)
In Two-dimensional parity-check code, the data code is organized in a
table (rows and columns) form.
The data to be sent are arranged in rows.
For each row and each column, 1 parity-check bit is calculated.
The whole table is then sent to the receiver, which finds the syndrome for each
row and each column.
The two-dimensional parity check can detect up to three errors that occur
anywhere in the table.
ERROR
CONTROL
CODING
51 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
51. Linear Block Codes-Parity-check Codes (2-D)
Two-dimensional parity-check code
ERROR
CONTROL
CODING
52 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
52. Linear Block Codes-Systematic Block Codes
In (n,k) block codes each sequence of k information bits is mapped into a
sequence of n channel inputs in a fixed way regardless of previous
information bits.
The formed code family should be selected such that the code minimum
distance is as large as possible -> high error correction or detection
capability
Definition: A systematic block code:
– the first k elements are the same as the message bits
– the following r = n - k bits are the check bits
Therefore, the encoded word is
As partitioned representation
ERROR
CONTROL
CODING
53
1 2 1 2
message
check
( .... ... ),
r k
b b b m m m r n k
X
( | )
B
X M
HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
53. Linear Block Codes-Systematic Block Codes
Given the message vector M, the respective linear, systematic block
code X can be obtained by the matrix multiplication
The matrix G is the generator matrix with the general structure
where Ik is kxk identity matrix and
P is called hamming code (or called parity check matrix), it is a kxr binary
submatrix ultimately determining the generated codes
ERROR
CONTROL
CODING
54
X MG
( | )
k
G P I
11 12 1
21 22 2
1 2
r
r
k k kr
p p p
p p p
p p p
P
HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
54. Linear Block Codes-Systematic Block Codes
For u message vectors M (each consisting of k bits) the respective n-bit block codes
X are therefore determined by
ERROR
CONTROL
CODING
55 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
55. Linear Block Codes-Systematic Block Codes
The check vector B that is appended to the message in the encoded
word is thus determined by the multiplication
The jth element of B on the uth row is therefore encoded by
For the Hamming code (parity matrix), P matrix of k rows consists of
all r-bit words with two or more "1― is arranged in all orders! Hence P
can be (for instance)
ERROR
CONTROL
CODING
56
B MP Note: X=(B|M)=MG = M(P|Ik)
Therefore: B = MP
, ,1 1, ,2 2, , , , 1...
u j u j u j u k k j
b m p m p m p j r
1 0 1
1 1 1
1 1 0
0 1 1
P
HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
56. Linear Block Codes-Hamming Code
HAMMING CODES are types of linear block codes of (n, k) with the
following parameters
• Number of parity bits, m=n-k
• Codeword length, n = 2m - 1
• Dataword length, k =n-m= 2m – m -1
• Minimum distance dmin=3,
• m >= 3
i.e. Code with m=2 is Repetition Code.
Therefore, the HAMMING CODE will have this general form of
expression
c(n, k)=(2m - 1, 2m – m -1)
ERROR
CONTROL
CODING
57 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
57. Linear Block Codes-Hamming Code
For each integer m ≥ 2 there is a code with block length n = 2m - 1 and
message length k = 2m− m− 1.
Hence the rate of Hamming codes is R = k / n = 1 − {m / (2m − 1)},
Hamming codes are perfect codes, that is, they achieve the highest
possible rate for codes with their block length and minimum distance
of three.
Hamming Codes were originally designed with dmin =3, so they can be
used for single error correction (SEC) or dual error detection (DED).
Therefore, Hamming codes are distance-3 linear block codes.
The first four Hamming codes, for example, are (3,1), (7,4), (15,11), and
(31,26) codes.
ERROR
CONTROL
CODING
58 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
58. Linear Block Codes-Hamming Code
Drawbacks- it is used for single error correction and up to two errors detection
only.
This is due to the limited redundancy that Hamming codes add to the data, they can only detect and
correct errors when the error rate is low.
Advantages-it is easy and cost effective for single error detection and correction
applications and it is popular in computer memory because in computer memory
(usually RAM) the bit errors are extremely rare and Hamming codes are widely
used.
ERROR
CONTROL
CODING
59 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
59. Linear Block Codes-Hamming Code
Hamming bound
There is an inequality called the Hamming bound, which states that if there is an (n,k)
code with an alphabet of q elements and dmin = 2t + 1 then
For each of the qk codewords there are exactly words that are exactly
distance i from it.
Therefore, the number on the right of the inequality is the total number of words
that are at most a distance of t from a codeword.
Since the total number of words cannot exceed qn , the inequality follows
immediately.
ERROR
CONTROL
CODING
60 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
60. Linear Block Codes-Hamming Bound
For a binary code becomes
Codes that attain equality in the Hamming bound are called perfect codes.
A perfect t –error correcting code has the property that every word in the Hamming
space lies within a distance of t from exactly one codeword.
A Hamming space is usually the set of all 2 N binary strings of length N.
More generally, a Hamming space can be defined over any alphabet (set) Q as the
set of words of a fixed length N with letters from Q.
ERROR
CONTROL
CODING
61 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
61. Linear Block Codes-Hamming Code
Assignment 3- Show How to generate/calculate Hamming
Codes? Develop Algorithm for Hamming Code.
ERROR
CONTROL
CODING
62 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
62. Linear Block Codes-Hamming Code
Example-Generate the Hamming Codes for parity bits m=3;
C(n, k)=(2m - 1, 2m – m -1)=C(7,4)
let’s assume m be the message to be encoded , m=(m0, m 1 ,m2 ,m3) ,
and let x be the corresponding codedword, x=(x0, x 1 ,x2 ,x3 ,x4, x 5 ,x6)
then we know that and
ERROR
CONTROL
CODING
63
X MG ( | )
k
G P I
k
I
|
P
G
k
k
k
n
k
1
0
0
0
0
1
0
0
0
0
1
0
0
0
0
1
1
0
1
1
1
1
1
1
0
0
1
1
)
(
HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
63. Linear Block Codes-Hamming Code
for m=(m0, m 1 ,m2 ,m3) , codeword x=(x0, x 1 ,x2 ,x3 ,x4, x 5 ,x6)
becomes
x0= m0 + m2 +m3
x1 =m0 + m1 +m2
x 2=m1 + m2 +m3
x3 = m0
x4 = m1
x 5= m2
x6= m3
ERROR
CONTROL
CODING
64 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
64. Linear Block Codes-Hamming Code
Hamming Codes for C(7,4)
ERROR
CONTROL
CODING
65 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
65. Linear Block Codes-Hamming Code
The structure of the encoder and decoder for a Hamming code C(7,4)
ERROR
CONTROL
CODING
66 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
66. Linear Block Codes-Hamming Code
Generator creates 3 parity-check bit (r0 ,r1 , and r2)
r0 = a2 + a1 + a0 modulo-2
r1 = a3+ a2 + a1 modulo-2
r2 = a1 + a0 + a3 modulo-2
The checker in the decoder creates a 3bit syndrome bit
s0 = b2 + b1 + b0 + q0 modulo-2
s1 = b3 + b2 + b1 + q1 modulo-2
s2 = b1 + b0 + b3 + q2 modulo-2
ERROR
CONTROL
CODING
67 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
67. Linear Block Codes-Hamming Code
For the Hamming code (parity matrix), P matrix of k rows
consists of all r-bit words with two or more "1― is
arranged in all orders! Hence P can be (for instance)
ERROR
CONTROL
CODING
68
1 0 1
1 1 1
1 1 0
0 1 1
P
HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
68. Linear Block Codes-Hamming Code
.
ERROR
CONTROL
CODING
69 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
69. Linear Block Codes-Hamming Code-Single Error Correction,
Double Error Detection
Can achieve this by adding an overall parity bit.
If parity checks are correct and overall parity bit are
correct, then no single or double errors occurred.
If overall parity bit is incorrect, then single error has
occurred, can use previous to correct.
If one or more of parity checks incorrect but overall
parity bit is correct, then two errors are detected.
ERROR
CONTROL
CODING
70 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
70. Linear Block Codes-Extended Hamming Code (Reading
Assignment)
Extended Hamming codes
achieve a Hamming distance of four,
allows the decoder to distinguish between when at most one one-bit
error occurs and when any two-bit errors occur.
since it cannot distinguish a double bit error of some codeword
from a single bit error of a different codeword.
Thus, some double-bit errors will be incorrectly decoded as if they
were single bit errors and therefore go undetected, unless no
correction is attempted.
In this sense, extended Hamming codes are single-error correcting
and double-error detecting, abbreviated as SECDED.
ERROR
CONTROL
CODING
71 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
71. Linear Block Codes-Cyclic Codes
Cyclic code is a block code, where the circular shifts of each
codeword gives another word that belongs to the code.
They are error-correcting codes that have algebraic properties that
are convenient for efficient error detection and correction.
Definition:- A code C is cyclic if
C is a linear code;
Any cyclic shift of a codeword is also a codeword,
A cyclic shift of a vector [v0 v1 … vn-2 vn-1] is the vector
[vn-1 v1 … vn-2 ].
Cyclically shifting v, i places to the right is equivalent to
cyclically shifting v, n − i places to the left.
ERROR
CONTROL
CODING
72 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
72. Linear Block Codes-Cyclic Codes-Polynomial
Representation
Polynomial representation of vectors
For every vector v=[v0 v1 … vn-2 vn-1] there is a polynomial
Let v(i) be the vector resulting from i cyclic shifts on v
ERROR
CONTROL
CODING
73 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
73. Linear Block Codes-Cyclic Codes-Polynomial
Representation
Consider vector v(X) andv(1) (X) and using Modulo 2 arithmetic
v(X) andv(i) (X) are related by
ERROR
CONTROL
CODING
74 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
74. Linear Block Codes-Cyclic Codes-Polynomial
Representation
Proof
ERROR
CONTROL
CODING
75 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
75. Linear Block Codes-Cyclic Codes-Polynomial
Representation
Property 1- The nonzero code polynomial of minimum
degree in a linear block code (Cyclic code, C) is unique.
Let be the
nonzero code polynomial of minimum degree in an (n, k)
cyclic code C. Then the constant term g0 must be equal to 1.
ERROR
CONTROL
CODING
76 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
76. Linear Block Codes-Cyclic Codes-Polynomial
Representation
Property 2- Consider the polynomial
Clearly, they are cyclic shifts of g(x) and hence code
polynomials in C. Since C is linear, a linear combination of
is also a code polynomial
where ui ∈ {0, 1}.
ERROR
CONTROL
CODING
77 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
77. Linear Block Codes-Cyclic Codes-Polynomial
Representation
.
ERROR
CONTROL
CODING
78 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
78. Linear Block Codes-Cyclic Codes-Polynomial
Representation
Let be the
nonzero code polynomial of minimum degree in an (n, k)
cyclic code C.
Property 3- A binary polynomial of degree n-1 or less is a code
polynomial if and only if it is a multiple of g(x).
Proof: Let v(x) be a binary polynomial of degree n − 1 or less. Suppose
that v(x) is a multiple of g(x). Then
Since v(x) is a linear combination of the code polynomials, it
is a code polynomial in C.
ERROR
CONTROL
CODING
79 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
79. Linear Block Codes-Cyclic Codes-Polynomial
Representation
Now let v(x) be a code polynomial in C. Dividing v(x) by g(x), we obtain
Where the degree of b(x) is less than the degree of g(x).
Since v(x) and a(x)g(x) are code polynomials, b(x) is also a code
polynomial.
Suppose b(x) not = 0. Then b(x) is a code polynomial with less degree
than that of g(x). Contradiction.
ERROR
CONTROL
CODING
80 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
80. Linear Block Codes-Cyclic Codes-Polynomial
Representation
Property 4- The number of binary polynomials of degree n − 1 or less
that are multiples of g(x) is 2n−r.
Property 5- There are total of 2k = 2n−r code polynomials in C, i.e.,
r = n − k.
Property 6- The polynomial g(x) is called the generator polynomial
of the code C. If
The degree of g(x) is equal to the number of parity-check digits of the code.
The generator polynomial g(x) of an (n, k) cyclic code is a factor of xn + 1.
ERROR
CONTROL
CODING
81 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
81. Linear Block Codes-Cyclic Codes-Polynomial
Representation
Property 7- If g(x) is a polynomial of degree n − k and is a factor of xn + 1, then g(x)
generates an (n, k) cyclic code.
v(x) is a polynomial of degree n − 1 or less and is a multiple of g(x).
There are a total of 2k such polynomial and they form an (n, k) linear code.
ERROR
CONTROL
CODING
82 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
82. Linear Block Codes-Cyclic Codes-Systematic
Encoding of Cyclic Codes
Since both xv(x) and xn + 1 are divisible by g(x), v(1)(x) must be divisible by g(x).
Hence, v(1)(x) is a code polynomial and the code generated by g(x) is a cyclic code.
Systematic Encoding of Cyclic Codes
Suppose we have the message u = (u0, u1, . . . , uk−1) to be encoded, Then
Since the degree of g(x) is n − k, the degree of b(x) must be n − k − 1 or less. Then
is a multiple of g(x) and therefore it is a code polynomial.
ERROR
CONTROL
CODING
83 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
83. Linear Block Codes-Cyclic Codes-Systematic
Encoding of Cyclic Codes
Then, the coded polynomial can be written as follows
Therefore, a systematic encoding of the message can be obtained as follows
First- Divide xn-ku(x) by g(x) to obtain remainder b(x), and then
The code polynomial is given by b(x) + xn-ku(x)
Given message bits, (mk-1…m1m0 ), the code is generated simply as:
In other words, C(x) can be considered as the product of m(x) and g(x).
ERROR
CONTROL
CODING
84 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
84. Linear Block Codes-Cyclic Codes-Shift Register
Circuit
Shift Register Circuit
ERROR
CONTROL
CODING
85 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
85. Linear Block Codes-Cyclic Codes-Shift Register
Circuit
If the initial state polynomial is zero and the input is a sequence of bits am-1, am-2, …. a1, a0, the
final state polynomial is
For systematic encoding we need Xn-ku(X) mod g(X) which corresponds to input bit sequence
ERROR
CONTROL
CODING
86 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
86. Linear Block Codes-Cyclic Codes-Shift Register
Circuit
Is there a way to avoid the delay of n - k clock ticks?
ERROR
CONTROL
CODING
87 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
87. Linear Block Codes-Cyclic Codes-Shift Register
Circuit
If the initial state polynomial is zero and the input is a sequence of bits am-1, am-2, …. A1, a0, the
final state polynomial is
ERROR
CONTROL
CODING
88 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
88. Linear Block Codes-Cyclic Codes-Shift Register
Circuit
Turn on the gate. Shift the message bits uk-1, uk-2, …, u0 into the circuit and channel
simultaneously. Only Output1 is fed to the channel.
Turn off the gate and shift the contents of the shift register into the channel. Only Output2 is
fed to the channel.
ERROR
CONTROL
CODING
89 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
89. Linear Block Codes-Cyclic Codes-Shift Register
Circuit
Encoding circuit for an (n, k) cyclic code with generator polynomial
g(X) = 1 + g1X + g2X2 + · · · + gn−k−1Xn−k−1 + Xn−k
The operation of the encoding circuit is described as follows:
1) Initially, the gate is turned on. The, k information digits, message polynomial u(X) = u0+u1X+
・・・+uk−1Xk−1 is fed to the circuit as well as transmitted into the channel. Feeding the k
information digits into the circuit is equivalent to pre-multiplying u(X) by Xn−k. When all k
information digits are shifted into the circuit, the (n − k) digits in the register form the
remainder.
ERROR
CONTROL
CODING
90 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
90. Linear Block Codes-Cyclic Codes-Shift Register
Circuit
Encoding circuit for an (n, k) cyclic code with generator polynomial
g(X) = 1 + g1X + g2X2 + · · · + gn−k−1Xn−k−1 + Xn−k
2) The gate is then turned off, since the register now contains the desired (n − k) parity
check digits.
3) Selector is changed to the left position to send parity check digits into the channel. These
(n−k) parity check digits along with k information digits form the cyclic codeword in
systematic form.
ERROR
CONTROL
CODING
91 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
11/24/2022
91. Linear Block Codes-Cyclic Codes-Error Detection
Errors are detected when the received vector is not a codeword
For linear block codes, r is a codeword iff rHT = 0
s = rHT is called the syndrome vector
For cyclic codes, the received polynomial r(X) is a code polynomial iff
r(X) mod g(X) = 0
s(X) = r(X) mod g(X) is called the syndrome polynomial, or
r(X) = a(x) g(X) + s(X) , n-k cofficient of s(X) makes the syndrome
The following circuit computes the syndrome polynomial
ERROR
CONTROL
CODING
92 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
11/24/2022
92. Linear Block Codes-Cyclic Codes-Error Detection
Property-10- Let s(x) be the syndrome of a received polynomial r(x).
Then the remainder s(1)(x) resulting from dividing xs(x) by the
generator polynomial g(x) is the syndrome of r(1)(x), which is a cyclic
shift of r(x).
Property-11-The remainder s(i)(x) resulting from dividing xis(x) by the
generator polynomial g(x) is the syndrome of r(i)(x), which is the ith
cyclic shift of r(x).
ERROR
CONTROL
CODING
93 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
93. Linear Block Codes-Cyclic Codes
Example
ERROR
CONTROL
CODING
94 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
94. Linear Block Codes-Cyclic Codes-Error Detection
Example
ERROR
CONTROL
CODING
95 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
95. Linear Block Codes-Cyclic Codes-Error Detection
.
ERROR
CONTROL
CODING
96 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
96. Linear Block Codes-Cyclic Codes-Error Detection
.
ERROR
CONTROL
CODING
97 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
97. Linear Block Codes-Cyclic Codes-Error Detection
.
ERROR
CONTROL
CODING
98 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
98. Linear Block Codes-Cyclic Codes-Error Detection
Decoding of Cyclic Codes
Decoding of linear codes consists of three steps:
1) Syndrome computation;
2) association of the syndrome to an error pattern;
3) error correction.
The cyclic structure of a cyclic code allows us to decode a received
vector r(x) in serial manner.
The decoding circuit checks whether the syndrome s(x) corresponds
to a correctable error pattern e(x) with an error at the highest-order
position xn−1 (i.e., en−1 = 1).
ERROR
CONTROL
CODING
99 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
99. Linear Block Codes-Cyclic Codes-Error Detection
If s(x) does not correspond to an error pattern with en−1 = 1,
the received polynomial and the syndrome register are cyclically
shifted once simultaneously.
By doing this, we have r(1)(x) and s(1)(x).
The second digit rn−2 of r(x) becomes the first digit of r(1)(x).
The same decoding processes.
If the syndrome s(x) of r(x) does correspond to an error pattern with
an error at the location xn−1, the first received digit rn−1 is an
erroneous digit and it must be corrected by taking the sum rn−1 ⊕
en−1.
ERROR
CONTROL
CODING
100 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
100. Linear Block Codes-Cyclic Codes-Error Detection
This correction results in a modified received polynomial, denoted by
r1(x) = r0 + r1x + ・ ・ ・ + rn−2xn−2 + (rn−1 ⊕ en−1)xn−1.
The effect of the error digit en−1 on the syndrome can be achieved by
adding the syndrome of e(x) = xn−1 to s(x).
The syndrome s(1)
1 of r(1)
1 (x) is the remainder resulting from dividing
x[s(x) + xn−1] by the generator polynomial g(x).
ERROR
CONTROL
CODING
101 HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
101. Performance Analysis of Linear Block Codes: The Repetition Code,Parity-
check Codes, Systematic Block Codes, Hamming Code, Cyclic Codes.
Use MATLAB simulator to show the performance.
Due Date 24/11/2022
102
Assignment 2
HU, Haramaya Institute of Technology, School of Electrical and Computer Engineering
11/24/2022
11/24/2022 ERROR CONTROL CODING