The idea of similarity is through the Hamming

Transcription

1 Hamming distance A good channel code is designed so that, if a few bit errors occur in transmission, the output can still be identified as the correct input. This is possible because although incorrect, the output is sufficiently similar to the input to be recognizable. The idea of similarity is through the Hamming distance. Let X = [x 1,..., x i,..., x n ], x i = {0, 1} and Y = [y 1,..., y i,..., y n ], y i = {0, 1} be two binary sequences of the same length n. The Hamming distance between these two codes is the number of symbols that disagree. HD(X, Y ) = n i=1 x i y i If the transmitter and the receiver agree that the transmitter only uses a collection of codewords X 1, X 2, X 3,..., X M. 1

2 If Y is received, then some of X (e.g. X k ) has the minimum hamming distance (MHD) to Y, such that for all j k. HD(X k, Y ) HD(X j, Y ) The decoder will assign to Y the code X k that minimises the Hamming distance between X j, j = 1,.., M and Y. For example, consider the codewords: a: b: c: If the transmitter sends but there is a single bit error and the receiver gets 10001, it can be seen that the nearest codeword is in fact and so the correct codeword is found. 2

3 By designing a good code, we try to ensure that the Hamming distance between possible codewords {X j } is larger than the Hamming distance arising from errors. Given a set of code words with MHD = 2. A single binary will produce a word not in the list, so that the error will be detected, but there will be no way of knowing which code was actually transmitted. However for another set of code words with MHD = 3, a single binary will also produce a word not in the list, this error can however be corrected by generating the nearest code word. 3

4 It can be shown that to detect E bit errors, a coding scheme requires the use of codewords with a Hamming distance of at least (E + 1). It can also be shown that to correct E bit errors requires a coding scheme with at least a Hamming distance of (2E + 1) between the codewords. 4

5 Shannons Second Theorem If coding is in long groups of n binary digits, use only a small number M from the possible 2 n combinations. p(e) 0 as n provided M 2 nc, where C is the channel capacity. Assuming that the M selected code words are equiprobable, the information rate R = log 2 M log 2 2 nc = C n n Information can be transmitted up to and including the channel capacity without error (symbol errors in the decoded output) 5

6 Example: Suppose that a set of codewords with length n = 10 are used. The channel bit error rate is p = There are N = 2 n = 2 10 = 1024 all possible code words. The channel capacity C = 1 H(p) log 2 (0.01) log 2 (0.99) 0.9 We require R < C to ensure error free, so the maximum number of codewords is M = 2 nc = 2 9 = 512. Hence 512 used = 1 2 of all possible codewords are the error rate is high (not good). So we need to increase n so that there are more possible codewords to choose from. Unfortunately a very large n can be unpractical. 6

7 Example: Suppose that a set of codewords with length n = 100 are used. The channel bit error rate is p = There are N = 2 n = = all possible code words. The channel capacity C = 1 H(p) log 2 (0.01) log 2 (0.99) 0.9 We require R < C to ensure error free, so the maximum number of codewords is M = 2 nc = Hence 1 out of 1024 ( ) all possible codewords are used. error rate is low (very good), but M 10 27, which is an impractically large source alphabet. It is difficult to choose which code word to use. 7

8 Practical codes for error detection and correction In general to detect E errors MHD E + 1 To correct E errors MHD 2E + 1 Block codes (n, k): n=total length of code k=number of information digits n k=number of checking digits 8

9 (i) Double-difference codes for error detection A simple way of constructing such a code with n = 5, is to use all the combinations in which exactly two 1 s appear, which is known as two out of five code. There are C5 2 = 10 combinations, hence 10 code words can be constructed. MHD = 2, single errors can be detected. Such systems can be arranged to automatically request retransmission of an incorrect symbol ARQ (automatic request repeat) Alternatively, 3 out of 7 code with n = 7, which has 35 combinations, can be used. 9

10 (ii)parity codes Parity is the sum of binary digits (even or odd). Parity digit is an extra digit added to make parity even or odd as desired. Parity digit can be generated with exclusive ORs A B = AB + AB Code word with 3 information digits is x 1 x 2 x 3 p, in which the parity digit is p = x 1 x2 x 3 10

11 For example: 000 is coded as is coded as is coded as is coded as 1111 Odd number of errors changes parity. The code will fail to detect an even number of error, but will detect any odd number of errors. 11