Presented a long (long) time ago in 2004, for AMUC for honors linear algebra.

1. Error Correcting Codes
A basic description of the Hamming (7,4)
Linear Error Correcting Code, and stacking
cannon balls in n-dimensions.
WW

2. Schedule
 Why to use them
 How they really work
 The Hamming Distance
 A geometer’s description
 Example
 Message, encode, and introduce error
 Detect, and repair
 Questions

3. Applications
 Satellites
 Computer Memory
 Modems
 PlasmaCAM
 And many more
 PlasmaCAM:
 300 Amp Discharge
 High Frequency
 Open connectors
 Shielding wire
 Embedded
Processor

4. The Problem: Noise
Information
Source
Message
Transmitter
Noise
Source
Destination
Message
Reciever
Received
Signal
Signal
Message = [1 1 1 1]
Noise = [0 0 1 0]
Message = [1 1 0 1]
WW
hh

5. Poor solutions
Single CheckSum -
 Truth table:
 General form:
Data=[1 1 1 1]
Message=[1 1 1 1 0]
 Repeats –
Data = [1 1 1 1]
Message=
[1 1 1 1]
[1 1 1 1]
[1 1 1 1]
A B X-OR
0 0 0
0 1 1
1 0 1
1 1 0
aa
WW
hh

6. Why they are poor
( )
rationoisetosignaltheisS/N
CapacityChannelrawisW
CapacityChannelisC
/1log2 NSWC +≤
Shannon Efficiency
Repeat 3 times:
•This divide W by 3
•It divides overall capacity by at least
a factor of 3x.
Single Checksum:
•Allows an error to be detected but
requires the message to be discarded
and resent.
•Each error reduces the channel
capacity by at least a factor of 2
because of the thrown away
message.
WW
hh
aa
tt

7. Hammings Solution
Encoding:
 Multiple Checksums
Message=[a b c d]
r= (a+b+d) mod 2
s= (a+b+c) mod 2
t= (b+c+d) mod 2
Code=[r s a t b c d]
Message=[1 0 1 0]
r=(1+0+0) mod 2 =1
s=(1+0+1) mod 2 =0
t=(0+1+0) mod 2 =1
Code=[ 1 0 1 1 0 1 0 ]
WW
hh
aa
tt
ii
Dr. Hamming made several families of error correcting codes. This
Is just a single member of one of the families.

8. Simulation
100,000 iterations
Add Errors to (7,4) data
No repeat randoms
Measure Error Detection
Error Detection
•One Error: 100%
•Two Errors: 100%
•Three Errors: 83.43%
•Four Errors: 79.76%
Stochastic Simulation:
Results:
Fig 1: Error Detection
50%
60%
70%
80%
90%
100%
1 2 3 4
Errors Introduced
PercentErrorsDetected(%)
WW
hh
aa
tt
ii
ss

9. How it works: 3 dots
Only 3 possible words
Distance Increment = 1
One Excluded State (red)
It is really a checksum.
 Single Error Detection
 No error correction
A B C
A B C
A C
⇒ Two valid code words (blue)
WW
hh
aa
tt
ii
ss
tt
This is a graphic representation of the “Hamming Distance”

10. Hamming Distance
Definition:
The number of elements that need to be changed
(corrupted) to turn one codeword into another.
The hamming distance from:
 [0101] to [0110] is 2 bits
 [1011101] to [1001001] is 2 bits
 “butter” to “ladder” is 4 characters
 “roses” to “toned” is 3 characters
WW
hh
aa
tt
ii
ss
tt
hh

11. Another Dot
The code space is now 4.
The hamming distance is still 1.
Allows:
Error DETECTION for
Hamming Distance = 1.
Error CORRECTION for
Hamming Distance =1
For Hamming distances greater than 1
an error gives a false correction.
WW
hh
aa
tt
ii
ss
tt
hh
ee

12. Even More Dots
Allows:
Error DETECTION for
Hamming Distance = 2.
Error CORRECTION for
Hamming Distance =1.
• For Hamming distances greater than
2 an error gives a false correction.
• For Hamming distance of 2 there is
an error detected, but it can not be
corrected.
WW
hh
aa
tt
ii
ss
tt
hh
ee
MM

13. Multi-dimensional Codes
Code Space:
• 2-dimensional
• 5 element states
Circle packing makes more
efficient use of the code-space
WW
hh
aa
tt
ii
ss
tt
hh
ee
MM
aa

14. Cannon Balls
 http://wikisource.org/wiki/Cannonball_stacking
 http://mathworld.wolfram.com/SpherePacking.html
Efficient Circle packing is the same
as efficient 2-d code spacing
Efficient Sphere packing is the same
as efficient 3-d code spacing
Efficient n-dimensional sphere packing is the same as n-code spacing
WW
hh
aa
tt
ii
ss
tt
hh
ee
MM
aa
tt

15. More on Codes
 Hamming (11,7)
 Golay Codes
 Convolutional Codes
 Reed-Solomon Error Correction
 Turbo Codes
 Digital Fountain Codes
WW
hh
aa
tt
ii
ss
tt
hh
ee
MM
aa
tt
rr

16. An Example
WW
hh
aa
tt
ii
ss
tt
hh
ee
MM
aa
tt
rr
ii
We will
 Encode a message
 Add noise to the transmission
 Detect the error
 Repair the error

17. Encoding the message
we multiply this matrix












=
1111000
0110100
1010010
1100001
H
But why?
You can verify that:
To encode our message
By our message
messagecode ×= H
Hamming[1 0 0 0]=[1 0 0 0 0 1 1]
Hamming[0 1 0 0]=[0 1 0 0 1 0 1]
Hamming[0 0 1 0]=[0 0 1 0 1 1 0]
Hamming[0 0 0 1]=[0 0 0 1 1 1 1]
Where multiplication is the logical AND
And addition is the logical XOR
WW
hh
aa
tt
ii
ss
tt
hh
ee
MM
aa
tt
rr
ii
xx
??

18. Add noise
 If our message is
Message = [0 1 1 0]
 Our Multiplying yields
Code = [0 1 1 0 0 1 1]
Lets add an error,
so Pick a digit to mutate
[ ]
[ ]
[ ]
[ ]
[ ]
[ ]
[ ]
[ ]1100110
0110100
1010010
11110000
01101001
10100101
11000010
0110
1111000
0110100
1010010
1100001
=
+
⇒
×+
×+
×+
×⇒
×












Code => [0 1 0 0 0 1 1]

19. Testing the message
 We receive the
erroneous string:
Code = [0 1 0 0 0 1 1]
 We test it:
Decoder*CodeT
=[0 1 1]
 And indeed it has an
error
The matrix used to decode is:
To test if a code is valid:
 Does Decoder*CodeT
=[0 0 0]
 Yes means its valid
 No means it has error/s










=
1010101
1100110
1111000
Decoder

20. Repairing the message
 To repair the code we
find the collumn in the
decoder matrix whose
elements are the row
results of the test vector
 We then change
 Decoder*codeT is
[ 0 1 1]
 This is the third
element of our code
 Our repaired code is
[0 1 1 0 0 1 1]










=
1010101
1100110
1111000
Decoder

21. Decoding the message
We trim our received code by 3 elements
and we have our original message.
[0 1 1 0 0 1 1] => [0 1 1 0]

22. Questions?