Error Correction with Hamming Codes

Transcription

1 Hamming Codes Error Correction with Hamming Codes Forward Error Correction (FEC), the ability of receiving station to correct a transmission error, can increase the throughput of a data link operating in a noisy environment. The transmitting station must append information to the data in the form of error correction bits, but the increase in frame length may be modest relative to the cost of re transmission. (sometimes the correction takes too much time and we prefer to re transmit). Hamming codes provide for FEC using a "block parity" mechanism that can be inexpensively implemented. In general, their use allows the correction of single bit errors and detection of two bit errors per unit data, called a code word. The fundamental principal embraced by Hamming codes is parity. Hamming codes, as mentioned before, are capable of correcting one error or detecting two errors but not capable of doing both simultaneously. You may choose to use Hamming codes as an error detection mechanism to catch both single and double bit errors or to correct single bit error. This is accomplished by using more than one parity bit, each computed on different combination of bits in the data. The number of parity or error check bits required is given by the Hamming rule, and is a function of the number of bits of information transmitted. The Hamming rule is expressed by the following inequality: p d + p + 1 < = 2 (1) Where d is the number of data bits and p is the number of parity bits. The result of appending the computed parity bits to the data bits is called the Hamming code word. The size of the code word c is obviously d+p, and a Hamming code word is described by the ordered set (c,d). Codes with values of p< =2 are hardly worthwhile because of the overhead involved. The case of p=3 is used in the following discussion to develop a (7,4) code using even parity, but larger code words are typically used in applications. A code where the equality case of Equation 1 holds is called a perfect code of which a (7,4) code is an example. A Hamming code word is generated by multiplying the data bits by a generator matrix G using modulo-2 arithmetic. This multiplication's result is called the code word vector (c1,c2.c3,...cn), consisting of the original data bits and the calculated parity bits. The generator matrix G used in constructing Hamming codes consists of I (the identity matrix) and a parity generation matrix A: G = [ I : A ] An example of Hamming code generator matrix: G = The multiplication of a 4-bit vector (d1,d2,d3,d4) by G results in a 7-bit code word vector of the form (d1,d2,d3,d4,p1,p2,p3). It is clear that the A partition of G is responsible for the generation of the actual parity bits. Each column in A represents one parity calculation computed on a subset of d. The Hamming rule requires that p=3 for a (7,4) code, therefore A must contain three columns to 1 of :29

2 Hamming Codes produce three parity bits. If the columns of A are selected so each column is unique, it follows that (p1,p2,p3) represents parity calculations of three distinct subset of d. As shown in the figure below, validating the received code word r, involves multiplying it by a parity check to form s, the syndrome or parity check vector. T H = [A I] * 1 = H*r = s If all elements of s are zero, the code word was received correctly. If s contains non-zero elements, the bit in error can be determined by analyzing which parity checks have failed, as long as the error involves only a single bit. For instance if r=[ ], s computes to [101], that syndrome ([101]) matches to the third column in H that corresponds to the third bit of r - the bit in error. OPTIMAL CODING From the practical standpoint of communications, a (7,4) code is not a good choice, because it involves non-standard character lengths. Designing a suitable code requires that the ratio of parity to data bits and the processing time involved to encode and decode the data stream be minimized, a code that efficiently handles 8-bit data items is desirable. The Hamming rule shows that four parity bits can provide error correction for five to eleven data bits, with the latter being a perfect code. Analysis shows that overhead introduced to the data stream is modest for the range of data bits available (11 bits 36% overhead, 8 bits 50% overhead, 5 bits 80% overhead). A (12,8) code then offers a reasonable compromise in the bit stream. The code enables data link packets to be constructed easily by permitting one parity byte to serve two data bytes. 2 of :29

3 EE4253 Digital Communications Error Correction and the Hamming Code The use of simple parity allows detection of single bit errors in a received message. Correction of such errors requires more information, since the position of the bad bit must be identified if it is to be corrected. (If a bad bit can be found, then it can be corrected by simply complementing its value.) Correction is not possible with one parity bit since any bit error in any position produces exactly the same information - "bad parity". If more bits are included with a message, and if those bits can be arranged such that different errored bits produce different error results, then bad bits could be identified. In a 7-bit message, there are seven possible single bit errors, so three error control bits could potentially specify not only that an error occured but also which bit caused the error. Similarly, if a family of codewords is chosen such that the minimum distance between valid codewords is at least 3, then single bit error correction is possible. This distance approach is "geometric", while the above error-bit argument is 'algebraic'. Either of the above arguments serves to introduce the Hamming Code, an error control method allowing correction of single bit errors. The Hamming Code Consider a message having four data bits (D) which is to be transmitted as a 7-bit codeword by adding three error control bits. This would be called a (7,4) code. The three bits to be added are three EVEN Parity bits (P), where the parity of each is computed on different subsets of the message bits as shown below D D D P D P P 7-BIT CODEWORD D - D - D - P D D - - D P - D D D P (EVEN PARITY) (EVEN PARITY) (EVEN PARITY) Why Those Bits? - The three parity bits (1,2,4) are related to the data bits (3,5,6,7) as shown at right. In this diagram, each overlapping circle corresponds to one parity bit and defines the four bits contributing to that parity computation. For example, data bit 3 contributes to parity bits 1 and 2. Each circle (parity bit) encompasses a total of four bits, and each circle must have EVEN parity. Given four data bits, the three parity bits can easily be chosen to ensure this condition. It can be observed that changing any one bit numbered 1..7 uniquely affects the three parity bits. Changing bit 7 affects all three parity bits, while an error in bit 6 affects only parity bits 2 and 4, and an 1 of :29

4 error in a parity bit affects only that bit. The location of any single bit error is determined directly upon checking the three parity circles. For example, the message 1101 would be sent as , since: BIT CODEWORD (EVEN PARITY) (EVEN PARITY) (EVEN PARITY) When these seven bits are entered into the parity circles, it can be confirmed that the choice of these three parity bits ensures that the parity within each circle is EVEN, as shown here. It may now be observed that if an error occurs in any of the seven bits, that error will affect different combinations of the three parity bits depending on the bit position. For example, suppose the above message is sent and a single bit error occurs such that the codeword is received: transmitted message received message > BIT: BIT: The above error (in bit 5) can be corrected by examining which of the three parity bits was affected by the bad bit: BIT CODEWORD (EVEN PARITY) NOT! (EVEN PARITY) OK! (EVEN PARITY) NOT! 1 In fact, the bad parity bits labelled 101 point directly to the bad bit since 101 binary equals 5. Examination of the 'parity circles' confirms that any single bit error could be corrected in this way. The value of the Hamming code can be summarized: Detection of 2 bit errors (assuming no correction is attempted); Correction of single bit errors; Cost of 3 bits added to a 4-bit message. The ability to correct single bit errors comes at a cost which is less than sending the entire message twice. (Recall that simply sending a message twice accomplishes no error correction.) Hamming Distance = 3 The Hamming Code allows error correction because the minimum distance between any two valid codewords is 3. In the figure below, two valid codewords in 8 possible 3-bit codewords are arranged to have a distance of 3 between them. It takes 3 bit changes (errors) to move from one valid 2 of :29

5 codeword 000 to the other 111. If the codeword 000 is transmitted and a single bit error occurs, the received word must be one of {001,010,100}, any of which is easily identified as an invalid codeword, and which could only have been 000 before transmission. Sixteen Valid Codewords The Hamming Code code essentially defines 16 valid codewords within all 128 possible 7-bit codewords. The sixteen words are arranged such that the minimum distance between any two words is 3. These words are shown in this table, where it is left as an exercise to check that from any codeword N={0..F} in the table to any other word M, the distance is at least A B C D E F Example: For N=3, codeword 3 = is expected to be a distance of at least 3 from all the other codewords. The distance is 4 between 3 = and 0 = The distance is 3 between 3 = and 1 = The distance is 3 between 3 = and 2 = The distance is 4 between 3 = and D = The distance is 4 between 3 = and E = The distance is 3 between 3 = and F = Therefore, codeword 3 is a distance of at least 3 from any other valid codeword. The Distance Argument Looking again at the Venn diagram (at right) it can be observed that a change in any of the data bits (3,5,6,7) necessary changes at least 3 of :29

6 two other bits in the codeword. For example, given a valid Hamming codeword, a change in bit 3 changes three bits (1,2,3) such that the new codeword is a distance (d=3) from the initial word. The clever arrangement of the Hamming codewords ensures that this is the case for every valid codeword in the set. A Final Note Any set of codewords is useful for error control provided that the minimum distance between any two of them is some value D. (some may be more that D but none will be less than D) So there is no unique set of codewords with L=7 and D=3. The Hamming code shown here (L=7,D=3) is useful because it is easy to generate and to check this particular set of codewords. The distances would still be the same if you swapped two columns or complemented the bits in any column, but the codewords would look very different (and the Venn diagrams would not work!). To explore this subject further, visit the Online Hamming Code Tool The value of carefully choosing error control schemes is self-evident in the Hamming Code. Still, for very long messages another approach is desirable. 19 SEP 05 - tervo@unb.ca University of New Brunswick, Department of Electrical and Computer Engineering 4 of :29