(i) Information theory is a branch of mathematics dealing with quantifying information. It was developed by Claude Shannon and involves key concepts like entropy which measures uncertainty. (ii) Error coding schemes allow for reliable communication over unreliable channels by detecting and correcting errors. Common schemes include repetition codes, parity bits, checksums, and cyclic redundancy checks. (iii) Forward error correction adds redundant bits that allow the receiver to correct errors without retransmission. Applications of error coding include internet protocols, deep space communications, satellite broadcasting, data storage, and memory.

1. Topic:- Coding Scheme
(i).- Information theory
(ii).- Error coding scheme.
Submitted to:- Submitted By:-
Mr. Hemant Kumar Meena 1. Pawan Kumar Jangid
Assistant Professor 2. Sunil Kumar Yadav
Dept. of Electrical Engineering,MNIT, Jaipur 3. Manish Kumar Bagara
4. Chandan Kumar

2. What is information theory:-
Information theory is a branch of applied mathematics,
electrical engineering, and computer science involving the
quantification of information. Information theory was
developed by Claude E. Shannon to find fundamental limits on
signal processing operations such as compressing data and on
reliably storing and communicating data.

3. Concept of Information Theory:-
A key measure of information is entropy, which is
usually expressed by the average number of bits needed
to store or communicate one symbol in a message.
Entropy quantifies the uncertainty involved in
predicting the value of a random variable. For example,
specifying the outcome of a fair coin flip (two equally
likely outcomes) provides less information (lower
entropy) than specifying the outcome from a roll of a
die (six equally likely outcomes).

4. Application of information theory:-
Lossless data compression (e.g. ZIP files)
Lossy data compression (e.g. Mp3s and jpegs)
Channel coding (e.g. For digital subscriber line (DSL)).
Important sub-fields of information theory are source coding,
channel coding, algorithmic complexity theory, algorithmic
information theory, information-theoretic security, and
measures of information.

5. APPLICATIONS IN OTHER AREAS:-
Including statistical inference
Natural language processing,
Cryptography
Neurobiology
The evolution and function of molecular codes
Model selection in ecology thermal physics
Quantum computing
Linguistics, Plagiarism detection, Pattern Recognition, Anomaly Detection

6. Error coding scheme (Error detection and correction) :-
 Techniques that enable reliable delivery of digital data over unreliable communication
channels.
 Error detection techniques allow detecting errors,
while error correction enables reconstruction of the original data in many cases.
Definitions:-
The general definitions of the terms are as follows:-
1. Error detection is the detection of errors caused by noise or other impairments during
transmission from the transmitter to the receiver.
2. Error correction is the detection of errors and reconstruction of the original, error-free data.

7. Error detection schemes:-
1 Repetition codes
2 Parity bits
3 Checksums
4 Cyclic redundancy checks (CRCs)
5 Cryptographic hash functions
6 Error-correcting codes
1. Repetition codes:-
 Coding scheme that repeats the bits across a channel to achieve error-free communication.
 The data are divided into blocks of bits. Each block is transmitted some predetermined number of times.
For example, to send the bit pattern "1011", the four-bit block can be repeated three times, thus producing "1011 1011
1011". However, if this twelve-bit pattern was received as "1010 1011 1011" – where the first block is unlike the other
two – it can be determined that an error has occurred.

8. 2. Parity bits:-
A parity bit is a bit that is added to a group of source bits
to ensure that the number of set bits (i.e., bits with value
1) in the outcome is even or odd. It is a very simple
scheme that can be used to detect single or any other odd
number (i.e., three, five, etc.) of errors in the output. An
even number of flipped bits will make the parity bit appear
correct even though the data is erroneous.
Extensions and variations on the parity bit mechanism are
horizontal redundancy checks, vertical redundancy
checks, and "double," "dual," or "diagonal" parity (used in
RAID-DP).
Figure - Even-parity checking scheme

9. 3. Checksums:-
A checksum of a message is a modular
arithmetic sum of message code words of a
fixed word length. The sum may be
negated by means of a ones'-complement
operation prior to transmission to detect
errors resulting in all-zero messages.
Checksum schemes include parity bits,
check digits, and longitudinal redundancy
checks. Some checksum schemes, such as
the Damm algorithm, the Luhn algorithm,
and the Verhoeff algorithm, are specifically
designed to detect errors commonly
introduced by humans in writing down or
remembering identification numbers.
Figure- Calculating Checksum

10. 4. Cyclic redundancy checks (CRCs)
A cyclic redundancy check (CRC) is a non-secure
hash function designed to detect accidental changes
to digital data in computer networks; as a result, it
is not suitable for detecting maliciously introduced
errors. It is characterized by specification of what is
called a generator polynomial, which is used as the
divisor in a polynomial long division over a finite
field, taking the input data as the dividend, such
that the remainder becomes the result.
CRCs are particularly easy to implement in
hardware, and are therefore commonly used in
digital networks and storage devices such as hard
disk drives.
Figure - basic scheme for cyclic
redundancy checking

11. 5. Cryptographic hash functions
• The output of a cryptographic hash function, also known as a message digest, can provide
strong assurances about data integrity, whether changes of the data are accidental or
maliciously introduced.
• Any modification to the data will likely be detected through a mismatching hash value.
• Furthermore, given some hash value, it is infeasible to find some input data that will yield the
same hash value.
• If an attacker can change not only the message but also the hash value, then a keyed hash or
message authentication code (MAC) can be used for additional security. Without knowing the
key, it is infeasible for the attacker to calculate the correct keyed hash value for a modified
message.
6. Error-correcting codes
Any error-correcting code can be used for error detection. A code with minimum Hamming
distance, d, can detect up to d − 1 errors in a code word. Using minimum-distance-based error-
correcting codes for error detection can be suitable if a strict limit on the minimum number of
errors to be detected is desired.
Codes with minimum Hamming distance d = 2 are degenerate cases of error-correcting codes, and
can be used to detect single errors. The parity bit is an example of a single-error-detecting code.

12. Error Correction :-
The techniques that we have discussed
so far can detect errors, but do not correct them.
Error Correction can be handled in two ways:-
1 . Backward error correction:-
When an error is
discovered, the receiver can have the sender retransmit
the entire data unit.

13. 2 . Forward error correction(Error-correcting code):-
An error-
correcting code (ECC) or forward error correction (FEC)
code is a system of adding redundant data, or parity data,
to a message, such that it can be recovered by a receiver
even when a number of errors (up to the capability of the
code being used) were introduced, either during the
process of transmission, or on storage.
• Since the receiver does not have to ask the sender for
retransmission of the data, a backchannel is not
required in forward error correction.

14. • Error-correcting codes are frequently used in lower-
layer communication, as well as for reliable storage in
media such as CDs, DVDs, hard disks, and RAM.
• In theory it is possible to correct any number of errors
atomically.
• The number of bits required to correct multiple-bit or
burst error is so high that in most of the cases it is
inefficient to do so. For this reason, most error
correction is limited to one, two or at the most three-bit
errors.

15. Single-bit error correction:-
Concept of error-correction
can be easily understood by examining the simplest case
of single-bit errors.
• As we have already seen that a single-bit error can be
detected by addition of a parity bit with the data,
which needed to be send. A single additional bit can
detect error, but it’s not sufficient enough to correct
that error too. For correcting an error one has to know
the exact position of error. For this one has to know the
required number of redundant bits.

16.  To calculate the numbers of redundant bits (r) required
to correct d data bits .
 2r ≥ d+r+1
 Where d = no. of bits to be transmitted and
r = no. of redundant bits

17. Approach to a solution :-
1. Decide on the number of bits in the code word.
2. Determine the bit positions of the check bits.
3. Determine which parity bits check which positions
4. Calculate the values of the parity bits
The bit positions covered by each parity bit can be
calculated by writing each bit position as a sum of the
powers of 2:

18. 1 = 1
2 = 2
3 = 1 + 2
4 = 4
5 = 1 + 4
6 = 2 + 4
7 = 1 + 2 + 4
8 = 8
9 = 1 + 8
10 = 2 + 8
11 = 3 + 8
12 = 4 + 8

19. a
Check bit 1 governs positions 1, 3, 5, 7, 9
Check bit 2 governs positions 2, 3, 6, 7, 10
Check bit 4 governs positions 4, 5, 6, 7, 12
Check bit 8 governs positions 8, 9, 10, 11, 12

20. man

21. Applications of Error Coding Scheme:-
 Internet:-
 In TCP/IP stack, error control is performed at multiple levels:
 -In Ethernet carries a CRC checksum.
 -IPv4 header use checksum to protect the contain of header.
 - IPv6 header, use link layer technology.
 -UDP provide checksum for error correction.
 -TCP provides a checksum for protection of addressing information
from the IP headers.

22. Deep-space telecommunications:-
• The Voyager 1 and Voyager 2 missions, in 1977, uses
convolution codes and reed Muller codes to delivers color
imaging amongst scientific information of Jupiter and Saturn.
• The Voyager 2 craft additionally Reed–Solomon code allowed
for very powerful error correction.
• Space mission uses more powerful Turbo codes and LDPC
code.

23.  Satellite broadcasting :-
• The demand for satellite transponder bandwidth
continues to grow.
• It uses forward error correction .
• QPSK coupled with Reed Solomon codes.
• PSK ,QAM increase transponder efficiency.

24.  Data storage:-
• Error detection improve reliability of data storage.
• ORC use in group code recoding detect and correct the errors.
• Read Solomon code used compact discs to correct errors
caused by scratches.
• Modern hard drives uses CRC codes to detect and correct the
minor errors.

25.  Error-correcting memory:-
 DRAM memory provide protection against soft errors by relying on
error correcting codes.
 ECC used in deep space application due to increase radiation.
 ECC generally use Hamming codes.
 ECC supports check summing errors detected on PCI bus. .

26. Thank You!