Channel Coding/Decoding Hamming Method
INFORMATION TRANSFER ACROSS CHANNELS Sent Received messages symbols messages source encoder Source coding Channel coding Channel Channel Source decoder decoding decoding receiver Compression Source Entropy Rate vs Distortion Error Correction Channel Capacity Capacity vs Efficiency Decompression
INFORMATION TRANSFER ACROSS CHANNELS Sent Received messages messages symbols Information source Source coding Channel coding Channel Channel decoding Source decoding receiver Compression Source Entropy Rate vs Distortion Error Correction Channel Capacity Capacity vs Efficiency Decompression
Channel Coding/Decoding The purpose of channel coding/decoding is to detect and correct errors in noisy channels
Channel Coding/Decoding Error detection and correction In a noisy channel errors may occur during the transmission of data from information source to destination, so we need a method to detect there errors and then correct them.
Error-Detecting and -Correcting Codes Add extra bits to the data bits for the sole purpose of detecting when errors have been introduced and correcting the errors if possible
Repetition Code Repeat every bit 3 times 0110 ==> 000,111,111,000 Error detected if all 3 bits are not the same 000,110,111,000
Analysis of Repetition Code 200% overhead (code is 3x size of data) Error can go undetected only if 3 consecutive bits are in error 0110 => 000,111,111,000 => 000,000,111,000 If probability of one-bit error is p, then probability of undetected error is p 3 E.g. one-bit error = 10-5 => undetected error = 10-15 (Assumes independence)
What if an Error is Detected? Strategy #1: Throw the data out and ask for it to be sent again Slow but very low odds of erroneous data Strategy #2: Majority rules 0110 => 000,111,111,000 => 000,110,111,000 => 000,111,111,000 Quicker, but higher odds of error Was it actually 0010 => 000,000,111,000 => 000,110,111,000?
Parity Codes Add one bit to every block of, say, 4 bits Parity bit = 0 or 1 so that total number of 1 bits is even Detects all 1-bit errors, no 2-bit errors 00000 00011 00101 00110 01001 01010 01100 01111 10001 10010 10100 10111 11000 11011 11101 11110
Channel Coding/Decoding Hamming Method - It was the first complete error-detecting and error-correcting procedure. - It represents one of the simplest and most common method for the transmission of information ( in the presence of noise ). - It assumes that the source transmits binary messages ( i.e. The information source alphabet is { 0, 1 }) - It uses the parity checker method to detect an error - It assumes that the channel is a binary symmetric channel (BSC)
Hamming Codes (4,7) Hamming Code detects all one- and two-bit errors Corrects all 1-bit errors Magic: Any two different codewords differ in at least 3 places! 0000000 0001011 0010111 0011100 0100110 0101101 0110001 0111010 1000101 1001110 1010010 1011001 1100011 1101000 1110100 1111111
Hamming Distance Number of places in which two bit strings differ 1 0 0 0 1 0 1 = Hamming distance 3 1 0 0 1 1 1 0 Acts like a distance: a b = a+b
Error Correcting Codes ECC design is a kind of geometry problem: Find 16 bit strings of length 7, no two of which are separated by distance less than 3
Hamming Distance as Geometry 1100100 1010100 1000100 1000101 1001101 1100101 1001111 1100111 1001110 1101110
Definitions Hamming distance between x and y is d H := d(x, y) is the # of positions where x i y i The minimum distance of a code C is d min = min { d(x, y) x C, y C, x y} Hamming weight of a vector x is - w(x) := d(x, 0 ) is the # of positions where x i 0
Example Hamming distance d( 1001, 0111) = 3 Minimum distance (101, 011, 110) = 2 Hamming weight w(0110101) = 4
Performance A code with minimum distance d min is capable of correcting t errors if d min 2 t + 1. Proof: If t errors occur, then since d min 2 t + 1 an incorrect code word has at least t+1 differences with the received word.
Hamming codes Minimum distance 3 Construction G = I m All k-tuples of Hamming weight > 1 where m = 2 k k 1
Example k = 4, n = 7 1000 110 G = 0100 101 0010 011 0001 111 m = 2 3 3 1 = 4
Syndrome decoding Let G = [ I k P ] then construct H T = P I n-k For all code words c = xg, ch T = xgh T = 0 Hence, for a received noisy vector ( c n ) H T = c H T n H T = n H T = : S
Example 100 110 G = 010 101 001 011 H T = 110 101 011 100 x = 1 0 1 c = 1 0 1 1 0 1 c H T = 0 0 0 010 n = 0 1 0 0 0 0 001 c n = 1 1 1 1 0 1 [c n] H T = S = 1 0 1 Obvious fast decoder: precalculate all syndromes at receiver for correctable errors
In system form c n Calculate syndrome c n c n n* [c n] H T = S n* Precalculated syndromes when n = n* then n n* = 0
Structure: Reed Solomon Codes (CD, DVD) m k information symbols n-k check symbols Properties: minimum distance = n-k+1 (symbols) length 2 m -1
General remarks The general problem is the decoding RS codes can be decoded using Euclids algorithm Berlekamp Massey algorithm
Why error correction? Systems with errors can be made almost error free CD, DvD would not work without RS codes