How were the first error correcting codes constructed? A historical introduction to coding theory through Hamming's work

How were the ﬁrst error correcting codes
constructed?
A historical introduction to coding theory
through Hamming’s work
H. Gopalakrishna Gadiyar and R. Padma
V. I. T. University, Vellore
IIT Madras
6 February 2015
Gadiyar & Padma VIT
A Historical Introduction to Coding Theory

One of the principal objects of theoretical research is to
ﬁnd the point of view from which the subject appears in
the greatest simplicity.
—–Josiah Willard Gibbs (1839-1903)
Gadiyar & Padma VIT

Hamming’s Original Paper Rewritten in Symbolic Form:
A Preamble to Coding Theory
—-H. Gopalakrishna Gadiyar and R.Padma
arXiv:1401.5919v1
http:
//web.mit.edu/redingtn/www/netadv/Xinformati.html
Gadiyar & Padma VIT

Introduction
To introduce error correction coding we have to ﬁrst convince
the audience
Why digital?
The answer is in Shannon’s Information Theory
Gadiyar & Padma VIT

Using speakers, cameras etc. sound and light are converted
into electrical signals.
The essential point that one should know is that electrical
signals are functions of time x(t).
The aim of communication engineering is to get this signal
A) From here to there (communication)
B) From now to later (data storage)
Gadiyar & Padma VIT

There are two choices
1) Analog (Continuous)
2) Digital (Discrete)
Digital wins for reasons partly mathematical and partly
technological
Gadiyar & Padma VIT

Mathematical
It is well known that numbers, text, images, audio and video
can be transformed into a sequence of 0’s and 1’s
This is a consequence of the sampling theorem which
connects analog and digital signals
Gadiyar & Padma VIT

Sampling theorem
In the ﬁeld of digital signal processing, the sampling theorem
is a fundamental bridge between continuous signals (analog
domain) and discrete signals (digital domain).
Let x(t) whose Fourier transform is X(jω) represent a band -
limited signal, so that X(jω) = 0 for |ω| > ωm. If ωs > 2ωm,
where ωs = 2π
Ts
is the sampling frequency, then x(t) is uniquely
determined by its samples x(nTs), n = 0, ±1, ±2, · · · .
Reconstruction Theorem
x(t) =
n=∞
n=−∞
x(nTs)sinc(
ωs(t − nTs)
2π
)
where sinc(u) = sin πu
πu
Gadiyar & Padma VIT

Sampling Theorem - Contd...
As the samples x(nTs) are numbers known to a certain degree
of accuracy (may be 5 decimal places), these can be written
in binary form
These numbers x(nTs) are transmitted across and x(t) can be
reconstructed using the formula
Hence whether it is text, pictures, etc., we ﬁnally deal with
strings of 1’s and 0’s.
Gadiyar & Padma VIT

All this mathematics is hidden from public view because all we
need are what are called Analog - to - Digital converters and
Digital - to - Analog converters
Gadiyar & Padma VIT

Technological
Making devices is easier because digital essentially deals with
bits 1’s and 0’s which are high or low
Analog requires great accuracy
Gadiyar & Padma VIT

Digital also helps in data security (Encryption),
Data compression etc.
Gadiyar & Padma VIT

In this talk...
One of the important problems of communications is the
reliable transmission through a noisy channel
This is achieved by the techniques of encoding and decoding
The Channel model
Gadiyar & Padma VIT

Binary Symmetric Channel
0
1
0
1
1 −
1 −
From this the probability of sending a string of 1’s and 0’s
without error can be easily constructed
We want to make the probability of error as low as possible
and also have a good rate of transmission
This involves a trade-oﬀ
Gadiyar & Padma VIT

Origin of error correcting codes
Communication is imperfect
Even if the message is accurately stated, it may be garbled
during transmission
Consequences of a mistake in the interpretation of
ﬁnancial/diplomatic/military information may be serious.
Gadiyar & Padma VIT

When a message travels from an information source to a
destination both the sender and the receiver would want an
assurance that the received message either is free of errors
or
if it contains errors, these errors should be detectable
Ideally when an error is detected the receiver would like to be
able to correct it and recover the original message
Gadiyar & Padma VIT

How do we transmit strings of 0’s and 1’s of length n?
These strings are called binary n-blocks.
In this talk, a message is a sequence of binary n-blocks.
A message of six 3-blocks is
001 101 110 110 010 111
When there is no error in transmission, all such n-blocks may
be used to form messages without risk of misinterpretation.
But if errors can occur, the block received may diﬀer from the
one sent
Gadiyar & Padma VIT

One way to compensate this is to send only blocks that diﬀer
from each other so much that one or two errors cannot
change one of the selected blocks into another.
This restriction requires the use of longer blocks in order to
have the same variety of messages possible with the 2n
n-blocks.
One disadvantage is that this will reduce the amount of
information sent per unit time.
However, the gain is in reliability of data
There is a trade-oﬀ between the rate of transmission and
reliability
Gadiyar & Padma VIT

Simplest code
A set of selected n-blocks called codewords will make up a
(binary) code
The simplest code has just two codewords, 0 and 1
No errors in transmission can be detected as any error
changes one codeword to another codeword
Gadiyar & Padma VIT

Repetition code
Let us replace the code 0, 1 by 00, 11
We are repeating the digits of the old codewords.
This will allow us to detect any single error
For example, if 00 is sent and if the block 01 is received, then
we know that an error has occurred
But two errors will go undetected as the received word is also
a codeword
Gadiyar & Padma VIT

Repetition code - Contd.
By repeating each of the binary symbols n − 1 times we form
the repetition code.
This code consists of two codewords of length n, the all-zero
word (00 · · · 0) and all-one word (11 · · · 1)
Up to n − 1 errors in transmission are detectable
The ﬁrst digit of each word is the message digit while the
following n − 1 are the check digits
Gadiyar & Padma VIT

Repetition code - Contd.
These repetition codes allow the sending of only two diﬀerent
codewords
We can use a block repetition code which is copies of a block
of bits
We can have s-copies of r binary blocks
n = rs
There are 2r words of length n = rs in this code
For example, let us take r = 4 and s = 2.
The 16 words of length 8 are those whose ﬁrst 4 bits coincide
with their last 4 bits
Gadiyar & Padma VIT

In this case also all single errors are detectable.
For example, an error in the 6th digit of the codeword
11001100 yields
11001000 which is not a codeword.
But two errors are not always detectable
For example
let us consider 11001100 → 11101110
Gadiyar & Padma VIT

(n, r) Code
An (n, r) binary block code is a binary block code of length n
having r message digits.
We call this an (n, r) code
In repetition code the ﬁrst r message digits occupy the ﬁrst r
positions of a codeword
Gadiyar & Padma VIT

Rate of Information
For an (n, r) code, the rate of information is r
n
This rate gives you how much of information (in bits) is
carried on the average per symbol sent.
For example, for the (n, 1) repetition code, the rate is 1
n
For the (rs, r) block repetition code, the rate is 1
s
Increase in check digits will lower the rate of information
Gadiyar & Padma VIT

What is the goal of coding theory?
To send messages quickly and accurately
Accuracy over a noisier channel means more error detection
which means more check digits per codeword and thus longer
codewords
This slows the ﬂow of information
The repetition code is slow but reliable
Is there any other way?
Gadiyar & Padma VIT

Parity Check
A block of n-bits has even parity if the number of 1’s is even
For example, 101011 is a 6-bit block having even parity
101010 is a 6- bit block having odd parity
The summing of the digits is called the parity check
Gadiyar & Padma VIT

Error detection using parity check -(5, 4) code
Let us take all the 16 blocks of 4 bits.
We create a 5-block from each of the 4-block by appending a
1 or 0 which will make the 5-block to have even parity
This process is called encoding
0000 00000 0100 01001 1000 10001 1100 11000
0001 00011 0101 01010 1001 10010 1101 11011
0010 00101 0110 01100 1010 10100 1110 11101
0011 00110 0111 01111 1011 10111 1111 11110
If there is a single error in the transmission of an encoded
message, then the 5-block will have odd parity.
Gadiyar & Padma VIT

(5, 4) code - Contd.
For example 10001 → 10000 will indicate an error as the
received word has odd parity
(5, 4) code detects single error
Rate is 4
5 = 0.8
Remember that the rate for the (8, 4) block repetition code is
0.5
Any single error - detecting code must have at least one check
digit
For the 4-block code, (5, 4) code has the highest rate possible
among single error - detecting codes
Gadiyar & Padma VIT

In 1940’s....
The history of coding theory dates back to middle of 20th
century
Bell Telephone Laboratories used single parity check code in
two of its relay computers
It is similar to (5, 4) code
The 10 decimal digits were represented using 5 - block with
exactly two 1’s
Each decimal digit was stored on a group of ﬁve relays by
having exactly two of them switched on in the proper locations
Gadiyar & Padma VIT

2 Out of 5 Code
Decimal Codeword Decimal Codeword
Digit Digit
1 11000 6 00110
2 10100 7 10001
3 01100 8 01001
4 10010 9 00101
5 01010 0 00011
Any single relay malfunction was immediately detectable
In this case either exactly one or exactly three relays will be on
In case of parity check failure, the checking circuits
automatically and immediately halted the machine
Gadiyar & Padma VIT

3 Out of 5 Code
A later model relay computer used a three out of 5 code
This code also has 10 codewords
Mathematically, it is obtained by interchanging 0’s and 1’s in
the 2 out of 5 code
Decimal Codeword Decimal Codeword
Digit Digit
1 00111 6 11001
2 01011 7 01110
3 10011 8 10110
4 01101 9 11010
5 10101 0 11100
Gadiyar & Padma VIT

Bell Labs used this code in their relay computing machines
In case of parity check failure, the checking circuits
automatically and immediately halted the machine
When such a parity check failed, two results were possible
If it is daytime, a check failure stopped failure and sounded an
alarm
In this case the check failure was located by operating
personnel, whose eﬀorts were facilitated by an elaborate check
light panel
If it is night-time or weekend, a check failure switched the
machine immediately to other work
In this case the problem simply had to be rerun
This ineﬃciency led Richard Hamming to investigate the
possibility of automatic error-correction
Gadiyar & Padma VIT

Richard Hamming (1915 - 1998)
Hamming was a pure mathematician
He ﬁrst worked in atomic bomb project in Los Alamos
In 1946 he left Los Alamos to join Bell Laboratories
He had interest in large scale computing machines
Gadiyar & Padma VIT

Hamming did not have priority use of the Model V
He had only weekend access and that only when another
department was not using it
The weekend mode is the unattended mode and hence any
errors caused the machine to go on to the next problem
This annoyed Hamming and led him to invent the ﬁrst error
correcting code
Gadiyar & Padma VIT

In Hamming’s own words...
Two weeks in a row I came in and found that all my stuﬀ
had been dumped and nothing was done. I was really
aroused and annoyed because I wanted those answers and
two weekends had been lost. And so I said,
’Damn it, if the machine can detect an error, why can’t it
locate the position of the error and correct it?’
Gadiyar & Padma VIT

The first error correcting code
Very first solution appeared in the Memorandum dated July
27, 1947
Later published a year later in a slightly modified form
It was based on the idea of parity checks
Here the code consists of codewords of length n = (t + 1)2,
with r = t2 message digits and 2t + 1 check digits
Gadiyar & Padma VIT

Encoding of ((t + 1)2
, t2
) Code
Place the t2 message digits in a square array
Adjoin the 2t + 1 check digits on two adjacent sides including
the corner so that each row and column in the ﬁnal square
array has an even number of 1’s
Gadiyar & Padma VIT

Example: (9, 4) Code
Take t = 2
r = 22 = 4
To encode the 4-block, say 1101
1101 →
1 1
0 1
1101 →
1 1
0 1 →
1 1 0
0 1
Gadiyar & Padma VIT

Take t = 2
r = 22 = 4
1101 →
1 1
0 1
1101 →
1 1
0 1 →
1 1 0
0 1 1
Gadiyar & Padma VIT

Take t = 2
r = 22 = 4
1101 →
1 1
0 1
1101 →
1 1
0 1 →
1 1 0
0 1 1
1
Gadiyar & Padma VIT

Take t = 2
r = 22 = 4
1101 →
1 1
0 1
1101 →
1 1
0 1 →
1 1 0
0 1 1
1 0
Gadiyar & Padma VIT

Take t = 2
r = 22 = 4
1101 →
1 1
0 1
1101 →
1 1
0 1 →
1 1 0
0 1 1
1 0 1
The resulting codeword is 110011101
Gadiyar & Padma VIT

Decoding (9, 4) Code
Note that the 4 message digits are in positions 1, 2, 4 and 5
To decode, place the received codeword in the 3 × 3 array and
perform the row and column parity checks.
For t, we should place the codeword in the (t + 1) × (t + 1)
array
This code can locate a single error in transmission
The parity checks of the row and column containing that
position will fail.
Gadiyar & Padma VIT

Example
Suppose 110011101 is received as 010011101
Place the word on a 3 × 3 array as follows
0 1 0
0 1 1
1 0 1
Perform the parity check
Parity checks for the first row and the first column fail
The error has occurred at the first position
Gadiyar & Padma VIT

Rate of ((t + 1)2
, t2
) Code
Information rate of this code is t2
(t+1)2
For t = 2, rate =4
9 = 0.44
Can we improve this code?
Gadiyar & Padma VIT

1948 Memorandum
Hamming noted that the corner check symbol could be
deleted without losing the error correction capability.
An error in the message digit will cause two parity checks to
fail
An error in the check digit causes only one parity check to fail
For example, if 11001110 → 01001110, the checking array for
the received codeword is
0 1 0
0 1 1
1 0
Note that the first row and the first column parity checks fail
The error therefore has occurred in the intersection of the first
row and first column digit
Gadiyar & Padma VIT

If 11001110 → 11101110, the checking array is
1 1 1
0 1 1
1 0
Note that only the ﬁrst row parity check fails.
The error is in the right hand 1 in the top row.
The rate of this code is 4
8 = 0.5
Note that this is better than the (8, 4) block repetition code
(8, 4) repetition code can detect a single error, whereas the
above (8, 4) code can also correct the single error.
Gadiyar & Padma VIT

Can you ﬁnd a code with better rate?
Hamming wanted an eﬃcient code
That is a code with r message digits and as few check digits
as possible and yet could still correct single errors
He wanted each check digit to be a 0 or 1 so that it and the
message digits in certain positions would have even parity as
unit, sort of partial parity checks
The sequence of 0’s and 1’s thus calculated when written
from right to left, called the checking number or syndrome
would give the position in binary of any single error
Here the all - zero word means there is no error
Gadiyar & Padma VIT

Let us assume that k = n − r is the number of check digits
These k digits should be able to describe the
r + k + 1 = n + 1 possibilities
An error could occur in any one of the n positions or the
codeword has been transmitted without any error
Now note that the k check bits in binary will represent all the
2k integers from 0 to 2k − 1
Thus, 2k ≥ n + 1
Now multiplying both sides by 2r we get
2k+r
= 2n
≥ 2r
(n + 1)
Dividing both sides by n + 1 will give the fundamental
inequality
2n
n + 1
≥ 2r
This gives the minimum n for the given r
Gadiyar & Padma VIT

Hamming’s construction
There was a conceptual leap in his construction
The key idea can be summarized in the phrase “interleaved
partial parity check equations”
At the encoding end, the check digits are determined using
the partial parity check equations
At the decoding end, the same partial parity checks are
carried out to get the check digits.
These check digits when written from right to left give the
binary representation of the location of the error
If all check digits are 0, then there is no error
Gadiyar & Padma VIT

Hamming’s construction - Encoding
Hamming places the k check digits in positions
x20 , x21 , · · · x2k−1
He places the r = n − k message digits at the remaining
positions
The k check digits are calculated as follows
x1 = x20 is determined by the partial parity check equation
x1 + x3 + x5 + x7 + · · · = 0
Notice that all these bits have their position numbers
1, 3, 5, 7 · · ·
When these are written in their binary representation they
have the least signiﬁcant bit equal to 1
Gadiyar & Padma VIT

1 1
3 11
5 101
7 111
9 1001
11 1011
and so on
Hence if the single error has occurred in any one of the odd
positions, then at the decoding end, the partial parity check
equation will give
x1 + x3 + x5 + x7 + · · · = 1
Gadiyar & Padma VIT

Next, x2 is determined (at the encoding end) by the equation
x2 + x3 + x6 + x7 + · · · = 0
The binary representations
2 10
3 11
6 110
7 111
10 1010
11 1011
and so on have 1 as their second bit from the right and 2 is
the smallest of these numbers.
Hence if the single error has occurred in a position whose
second bit is 1, then at the decoding end, we would get
x2 + x3 + x6 + x7 + · · · = 1
Gadiyar & Padma VIT

Similarly x22 , . . . x2k−1 are determined by the corresponding
partial parity check equations
Notice that 1, 2, 4, 8, · · · , 2k−1 (1, 10, 100, 1000, · · · 100 · · · 0)
are the smallest numbers having 1 in the ﬁrst, second, third,
fourth, · · · kth positions in their binary representations and
hence the check bits are independent of one another
Thus the position number of the error bit is determined bit by
bit from right to left
Once the position of the error bit is found, the bit can be
corrected as a bit can take only two values: 0 or 1
Gadiyar & Padma VIT

Example: (7, 4) code
Let us take r = 4.
Let us ﬁnd the smallest n satisfying the inequality
2n
n+1 ≥ 24 = 16
n 2n
n+1 2r
5 25
5+1 = 5.333 16
6 26
6+1 = 9.143 16
7 27
7+1 = 16 16
n = 7 is the smallest number satisfying the fundamental
inequality
So, n = 7, r = 4 and k = 3.
The 3 check bits are in positions 1, 2 and 4
The four message bits are placed in the remaining positions
3, 5, 6 and 7
Gadiyar & Padma VIT

Let the encoded message be x1x2x3x4x5x6x7
The digit x1 is determined by the partial parity check equation
x1 + x3 + x5 + x7 = 0
Next we determine the bit x2 from the equation
x2 + x3 + x6 + x7 = 0
Finally, x4 is determined by the equation
x4 + x5 + x6 + x7 = 0
At the decoding end you perform the same partial parity
checks
If all three bits are zero, then there is no error
Otherwise the three bits will give the location of the error bit
Gadiyar & Padma VIT

Example
Let us take the 4-block 1010
We have x3 = 1, x5 = 0, x6 = 1, x7 = 0
x1 + x3 + x5 + x7 = 0 gives x1 = 1
x2 + x3 + x6 + x7 = 0 gives x2 = 0
x4 + x5 + x6 + x7 = 0 gives x4 = 1
Thus, 1010 is encoded as 1011010
Suppose at the decoding end, the message is received as
1001010
The parity check equations give 1, 1 and 0
The error has occurred in the position 011 = 3
Gadiyar & Padma VIT

Table for n, r and k
n r k = n − r
(total) (message) (checks)
1 0 1
2 0 2
3 1 2
4 1 3
5 2 3
6 3 3
7 4 3
8 4 4
Gadiyar & Padma VIT

Table for n, r and k
n r k = n − r
(total) (message) (checks)
9 5 4
10 6 4
11 7 4
12 8 4
13 9 4
14 10 4
15 11 4
16 11 5
Gadiyar & Padma VIT

Algebraic Structure of Hamming code
Consider the three equations
x4 + x5 + x6 + x7 = 0
x2 + x3 + x6 + x7 = 0
x1 + x3 + x5 + x7 = 0
This can be written as the matrix equation
HXT
=


0 0 0 1 1 1 1
0 1 1 0 0 1 1
1 0 1 0 1 0 1












x1
x2
x3
x4
x5
x6
x7










=


0
0
0


Gadiyar & Padma VIT

H is the binary matrix whose columns are the binary numbers
from 1 to 7
H determines a linear transformation from [GF(2)]7 to
[GF(2)]3
If X is a codeword, then HXT = 0
In the language of Linear Algebra this means that the
codeword X belongs to the kernel of the linear transformation
deﬁned by the matrix H and hence is a subspace of [GF(2)]7
In other words, X belongs to the null space of the matrix H
Since rank(H)=3, dim (Null space of H)=7-3=4
Hence the (7, 4) Hamming code is 4 dimensional subspace of
[GF(2)]7
Thus there are 24 = 16 codewords in [GF(2)]7
Gadiyar & Padma VIT

If the received word X is not a codeword, then by the
construction HXT gives the error position in binary, if there is
a single error
The matrix H is called the parity check matrix
HXT is called the syndrome of X
If there is no more than single error then the syndrome = 0
means that there is no error and the received word is a
codeword
Otherwise the syndrome indicates the exact location of the
error in the received word
Gadiyar & Padma VIT

Group Code, Linear Code
The sum of any two codewords is another codeword
Thus the codewords form a group
Hence the older name group codes
Recent books employ the fact that the codewords form a
linear space (vector space)
Hence the new name linear codes
Gadiyar & Padma VIT

Hamming’s construction was based on the idea that parity
check equations would write out the binary string
corresponding to the error location
Historically things developed in a diﬀerent way
The fact that the codewords form a group led to the fact that
occurrences of error would cause the codeword to become the
member of the coset
Hence the subject transformed into more mathematical form
far removed from its engineering roots: Algebraic coding
theory
This is the starting point of most books, not Hamming’s
original construction
Gadiyar & Padma VIT

Geometric Model
Hamming introduced geometric point of view to study error
correction codes
Consider the unit cube in an n- dimensional space consisting
of 2n n-tuples of 0’s and 1’s
A binary code with words of length n is a certain subset of the
vertices of this cube
Gadiyar & Padma VIT

(3, 2) Code
The four darkened vertices are the codewords of length 3 formed
by adding a parity check bit to each pair of binary digits
00 → 000, 01 → 011, 10 → 101 and 11 → 110
Gadiyar & Padma VIT

Here you see the three possible 3-blocks formed by a single error in
the transmission of the codeword 101
Gadiyar & Padma VIT

Hamming Distance
The figure also shows why two errors in a codeword are not
detectable
A second error sends the result of the first error to another
codeword
One error in a codeword changes one coordinate and two
errors change two coordinates
For any integer e, e errors change e coordinates
This observation motivated Hamming to define the distance D
Gadiyar & Padma VIT

Hamming Distance
The distance D between any two n- blocks of binary digits is
the number of coordinates in which the corresponding vertices
differ
For example, D(101, 011) = 2 and D(101, 010) = 3
Geometrically, this is the number of edges in a shortest path
between the two vertices
This distance is called the Hamming distance
Hamming distance satisfies all the axioms of a metric
1. D(x, y) ≥ 0 and D(x, y) = 0 iff x = y
2. D(x, y) = D(y, x)
3. D(x, z) ≤ D(x, y) + D(y, z) for any y
Gadiyar & Padma VIT

Minimum Distance of a Code
The minimum distance of a code is the minimum of all the
distances between two nonidentical codewords of the code
For the (3, 2) single parity check code whose codewords are
{000, 011, 101, 110}, the minimum distance is 2
Single error detection is possible with a code whose minimum
distance is 2
Single error correction is possible with a code with minimum
distance at least 3
Gadiyar & Padma VIT

Hamming distance of (3, 1) Repetition code
This ﬁgure shows the words that can be received if exactly one
error is made in the transmission of the codeword 000. Each of
these errors is still at a distance of 2 from (111)
Gadiyar & Padma VIT

If a maximum of single error occurs in the transmission of a
codeword
then the error can be found and corrected
If two errors occur in the transmission, then the error can be
detected but cannot be corrected
For example if 000 is received as 110 with two errors
then the receiver cannot decide whether the codeword sent is
111 or 000
Gadiyar & Padma VIT

(4, 1)RepetitionCode
The two codewords are 0000 and 1111
This code can correct any single error
Two errors can be detected but cannot be corrected as the
received word will be at a distance of 2 from both the
codewords
In general, a code with minimum distance d will detect up to
[d
2 ] errors and simultaneously correct up to [d−1
2 ] errors.
Some of the correction capability can be sacriﬁced for
additional detection capability.
For example, a code with minimum distance d can detect up
to d − 1 errors by scrapping all correction
Gadiyar & Padma VIT

Perfect Codes
This geometrical model of Hamming suggests the notion of a
perfect code
For non-negative integer l, Hamming deﬁned a sphere of
radius l with a vertex of the unit cube En as
the set of all vertices of the cube at a Hamming distance l
from the given vertex
This means that the sphere will contain
n
0 +
n
1 +
n
2 + · · · +
n
l vertices of the cube
Gadiyar & Padma VIT

1 - Spheres of the (3, 1) repetition code
The two 1- spheres centered at the two codewords (000) and (111)
exhaust all the vertices of unit cube
Gadiyar & Padma VIT

A code of length n is called perfect if there is a nonnegative
integer l such that
1. The l− spheres centered on the codeword vertices are
pairwise disjoint
2. Each vertex of the n-cube is contained in some l-sphere
Gadiyar & Padma VIT

Hamming’s (7, 4) code
Now let us go back to Hamming’s (7, 4) code
The minimum distance of this code is three
Hence the 1-spheres centered on the codewords are disjoint
Now the Hamming 1 sphere of any codeword must contain
7
0 +
7
1 = 8 vertices
There are 16 codewords
Hence the 16 Hamming 1-spheres with center at each of the
16 codewords must contain a total of 16 × 8 = 128 = 27
vertices which is all the vertices of the unit 7-cube
Hence this code is perfect
More generally, (n, r) = (2k − 1, 2k − 1 − k) Hamming codes
are perfect
Gadiyar & Padma VIT

One cannot escape the feeling that these mathematical
formulae have an independent existence and an
intelligence of their own, that they are wiser than we are,
wiser even than their discoverers, that we get more out of
them than we originally put in to them.
-Heinrich Hertz (1857-1894) German physicist
Gadiyar & Padma VIT

Directions in Pure Mathematics
Algebraic coding
Sphere packing
Simple groups
A good starting point for these topics is
T. M. Thompson, From error correcting codes through sphere
packings to simple groups, Carus Monograph No. 21, Math.
Assoc. Amer., Washington, D.C., 1983.
Quantum error correcting codes
Gadiyar & Padma VIT

Directions in Engineering
Algebraic codes
Convolutional codes
Turbo codes
Space - Time codes
A good starting point for these topics is
Todd K Moon, Error correction coding- Mathematical
methods and algorithms, Wiley - India Edition
Quantum error correcting codes
Gadiyar & Padma VIT

Representation of Hamming codes
Block codes
Cyclic codes
Trellises
Tanner graphs
Gadiyar & Padma VIT

There is nothing so practical as a good theory
—Kurt Lewin
There are two ways to do great mathematics. The ﬁrst is
to be smarter than everybody else. The second way is to
be stupider than everybody else - but persistent.
—Raoul Bott
Gadiyar & Padma VIT

Thank You
Gadiyar & Padma VIT

How were the first error correcting codes constructed? A historical introduction to coding theory through Hamming's work

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to How were the first error correcting codes constructed? A historical introduction to coding theory through Hamming's work

Similar to How were the first error correcting codes constructed? A historical introduction to coding theory through Hamming's work (20)

Recently uploaded

Recently uploaded (20)

How were the first error correcting codes constructed? A historical introduction to coding theory through Hamming's work