Channel coding


Published on

Channel coding
prepared by OLA MASHAQI

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Channel coding

  1. 1. CHANNEL CODING Introduction The mutual information I(A,B), with input A and output B, measures the amount of information that the channel is able to convey over the source. If the channel is noiseless, then this is equal to the information content of source A, I(A,B)=H(A). But if the channel is noisy, there is an uncertainty of equivocation that reduces the mutual information to I(A,B)=H(A) – H(A|B). The presence of noise manifests itself as random errors in the transmission of the symbols. The random errors may be tolerable ( eg. Occasional bit errors in the transmission of an image may not yield any noticeable degradation in picture quality) or fatal (e.g., with Huffman codes, any bit errors in the encoded variable-length output will create a cascade of errors in the decoded sequence due the loss of coding synchronization ) . When the errors introduced by the information channel are unacceptable, then channel coding is needed to reduce the error rate and improve the reliability of the transmission. Since the digital communication system uses a binary channel and the source coder will encode the source to a binary code, then the intervening channel coder will code binary msgs to binary codes. Our concern is the binary block codes, that is, both the channel coder inputs and outputs will be binary and fixed length.
  2. 2. Definition of channel coding The channel encoder separates or segments the incoming bit stream into equal length blocks of L binary digits and maps each L-bit message block into an N-bit code word where N > L and the extra N –L check bits provide the required error protection. There are M=2L messages and 2L code words of length N bits  The channel decoder has the task of detecting that there has been a bit error and (if possible) correcting the bit error  The channel decoder can resolve bit errors by two scheme of error-control . ARQ (Automatic-Repeat-Request ) If the channel decoder performs error detection then errors can be detected and a feedback channel from the channel decoder to the channel encoder can be used to control the
  3. 3. retransmission of the code word until the code word is received without detectable errors. FEC (Forward Error Correction) If the channel decoder performs error correction then errors are not only detected but the bits in error can be identified and corrected (by bit inversion) Code Rate  The extra N –L bits requires a higher bit-rate or bandwidth of a channel  Assume that the source coder generates messages at an average bit rate of ns bit per second  If the channel encoder maps each L-bit message into N-bit code word, then the channel bit rate will be nc = ( N/L )ns  It infers that more bits transmitted through the channel, so the channel must transmit bits faster than the source encoder The general expression for the code rate is R=H(M)/n Where H(M) is the average number of bits transmitted with each source message and n is the length of code word For the case of channel coding, we have M= 2L and each message is transmitted as the code word of length N, then H(M)=log2M=L and n = N R = L/N = ns/nc = Tc/Ts  The code rate measure the relative amount of information conveyed in each code word  Higher code rate implies fewer redundant check bits relative to the code word length
  4. 4.  The message rate and the channel rates are fixed design parameters, the code word length is then selected: N = Lnc/ns Example A system has bps and the channel bit rate is bps. This requires a channel coder with If L = 3, we have to choose N = 4 If L = 5, we have to choose N = 6, and the channel coder has code rate of 5/6 yields loss of synchronization, we have to compensate with 2 “dummy’ bits every 3 code word Decoding Rules The form of decoding rule will govern how the channel codes are used and the level of error protection provided Let ai = ai1ai2….aiN be the i-th N-bit code word transmitted through the channel Let b = b1b2….bN be the corresponding N-bit word produced at the output of the channel Let BM represent the set of M valid N-bit code words and its complement BCM is the remaining N-bit which are not code words The BN = BM BCM is the set of all possible N-bit binary sequences and b = BN whereas ai The channel decoder has to apply the decoding rule on the received word to decide which code word was transmitted.
  5. 5. If the decision rule is D(.) then ai = D(b) Let PN (b | ai) be the probability of receiving b given ai was transmitted For discrete memoryless channel, the probability is Let PN(ai) be the a priori probability for the message corresponding to the code word ai By the Bayes’ Theorem we get If the decoder decodes b into the code word ai then the probability that this is correct is PN(ai | b) and the probability that it is wrong is 1 – PN(ai| b) If we want to minimize the error, we have to maximize the PN(ai | b) (It is the minimum-error decoding rule) Definition of minimum-error decoding rule: We choose DME(b)=a* (a* ) such that PN (a* | b) >= PN(ai | b) i By using Bayes’ Theorem and noting that PN(b) is independent of the code word, the condition becomes PN (b | a*) PN (a*) >= PN(b | ai) PN (ai) i requiring both knowledge of the channel probabilities and channel input probabilities. This decoding rule guarantees minimum error in decoding
  6. 6. Another rule is called maximum-likelihood decoding rule, by maximizing PN(b | ai) , the likelihood that b is received given ai was transmitted The rule is simpler as the channel input probabilities are not required Definition of Maximum-Likelihood Decoding: We choose DML(b)=a* where a*  is such that PN (b | a*) >= PN(b | ai) i requiring only knowledge of the channel probabilities. This decoding rule does not guarantee minimum error in decoding The maximum-likelihood decoding rule is the same as the minimum-error decoding rule when the channel input probabilities are equal Assume we have decoding rule D(.) and Bi is the set of all possible received words b such that D(b) = ai The probability of decoding error given code word ai is transmitted is The overall probability of decoding error is Hamming Distance The important parameter in error coding is the number of bit errors or differences between two different code words ai, and aj
  7. 7. This measure is provided by Hamming distance on the space of binary words of length N The properties of Hamming distance are instrumental in establishing the operation and performance of channel codes for both error detection and error correction Definition of Hamming Distance: Consider the two N-length binary words a = a1a2….aN and b = b1b2….bN The Hamming distance between a and b, d(a,b) is defined as the number of bit positions in which a and b differ The Hamming distance is a metric on the space of all binary words of length N since for arbitrary words a, b, c, the Hamming distance obeys the following conditions: d(a,b)>=0 with equality when a= b d(a,b)= d(b,a) d(a,b)+ d(b,c)>=d(a,c) (triangle inequality) Hamming Distance for BSCs Consider the maximum likelihood decoding rule where the code word, ai , which maximizes the conditional probability PN(b | ai) has to be found Consider a BSC with bit error probability q Let the Hamming distance d(b,ai) = D represent the number of bit errors between the transmitted code word and received word This gives the following expression for the conditional probability
  8. 8. If q< 0.5, then eq. above is maximized when ai is chosen such that d(b,ai) is minimized Hamming Distance Decoding Rule The binary word, b, of length N is received upon transmission of one of the M possible N-bit binary code words, ai BM, through a BSC. Assuming the maximum likelihood decoding rule we choose the most likely code word as follows:  If b = ai , for a particular i then the code word ai was sent  If b  ai , for any i, we find the codeword a*  B which is closest to b in the Hamming sense: If there is only one candidate a* then the t-bit error, where t=d(a*,b), is corrected and a* was sent If there is more than one candidate a* then the t-bit error, where t=d(a*,b) , can only be detected Error Detection/Correction Using Hamming Distance Errors are detected when the received word is equidistance from more than one code word Errors are corrected when the received word is closest to only one code word Definition of minimum distance of a code: The minimum distance of a block code Kn, where Kn identifies the set of length n code words, is given by
  9. 9.  d(Kn) = min d(a,b) |a,b Kn and a b that is, the smallest Hamming distance over all pairs of distinct code words A block code Kn detects up to t errors if and only if its minimum distance is greater than t A block code Kn corrects up to t errors if and only if its minimum distance is greater than 2t Upper Bounds on M and the Hamming Bound Definition of Upper Bound on M: For a block code KN of length N and minimum distance d(KN) the maximum number of code words , and hence the msgs, M, to guarantee that such a code is exist is given by : M = B(N,d(KN)) The following are elementary results for B(N,d(KN)) : B(N,1) = 2N B(N,2) = 2N-1 B(N,2t+1) = B(N+1,2t+2) B(N,N) = 2
  10. 10. Where B(N,2t+1) = b(N+1,2t+2) indicates that if we know bound for odd d(KN) equivalent 2t+1 then we can obtain the bound for the next even d(KN)+1 = 2t+2, and vice versa Theorem: Hamming Bound If the block code of length N is a t-bit error correcting code, then the number of code words, M, must satisfy the following inequality : which is an upper bound (the Hamming bound) on the number of code words We have already known that for a t-bit error correcting code d(KN)>2t A t-bit error correcting code will have a minimum distance of d(KN)=2t+1 or more Consider the problem of finding the upper bound for a code with d(KN)=2t+1 Since d(KN)>2t this is a t-bit error correcting code and subject to the Hamming bound
  11. 11. Maximal Codes and Gilbert Bound The code K3= {000, 011} has d(K3)=2 and contains M= 2 code words of length N= 3 implying L= 1 and a code rate R= 1/3 The code can be made more efficient by augmenting it to the code K3= {000, 011, 101, 110} which is also d(K3) = 2 but contains M= 4 code words of length N= 3 implying L= 2 and a code rate R= 2/3 The second one is more efficient than the first code without sacrificing the minimum distance Definition of Maximal Codes: A code KN of length N and minimum distance d(KN) with M code words is said to be maximal if it is not part of, or cannot be augmented to, another code KN of length N, minimum distance d(KN) but with M+ 1 code words. It can be shown that a code is maximum if and only if for all words b of length N there is a code word ai, such that d(b,ai) <d(KN) Example The code K3= {000, 011} with d(K3)=2 is not maximal since for word b= 101, d(a1,b)=d(000,101)=2 , and d(a1,b)=d(011,101)=2 and thus there is no code word such that d(b,ai) < 2 If code KN is a maximal code then it satisfies the Gilbert bound Theorem: Gilbert Bound If a block code KN of length N is a maximal code with minimum distance d(KN) then the number of code words, M, must satisfy the following :
  12. 12. which provides a lower bound (the Gilbert bound) on the number of code words Error Probabilities There are two important measures for a channel code :  The code rate. Codes with higher code rate are more desirable  The probability of error. Codes with lower probability of error are more desirable. For error correcting codes the block error probability is important. For error detecting codes the probability of undetected error is important There is a tradeoff between the code rate and probability of error To achieve a lower probability of error it may be necessary to sacrifice code rate The bit error probability, Peb(Kn), is the probability of bit error between the message and the decoded message For a BSC system without a channel coder : Peb(Kn)=q The block error probability, Pe(Kn), is the probability of a decoding error by the channel decoder  The probability that the decoder picks the wrong code word when applying the Hamming distance decoding rule  For L = 1, the bit error and block error probabilities are the same
  13. 13.  For L > 1 the calculation of the block error probability is straightforward if we know the minimum distance of the code Example : Consider code K6 as shown below : Say message 000 is transmitted as code word 000000 Two bit errors occur in the last two bits, so the received word is 000011  The channel decoder will select the nearest code: 100011  The block error has occured  If this always occurs then Pe(Kn)=1  The code word 100011 is decoded as message 100 which has only the first bit wrong and the remaining two bits correct when compared with the original message  Then, the bit error is Pb(Kn)=1/3