Channel Coding/Decoding. Hamming Method

Transcription

1 Channel Coding/Decoding Hamming Method

2 INFORMATION TRANSFER ACROSS CHANNELS Sent Received messages symbols messages source encoder Source coding Channel coding Channel Channel Source decoder decoding decoding receiver Compression Source Entropy Rate vs Distortion Error Correction Channel Capacity Capacity vs Efficiency Decompression

3 INFORMATION TRANSFER ACROSS CHANNELS Sent Received messages messages symbols Information source Source coding Channel coding Channel Channel decoding Source decoding receiver Compression Source Entropy Rate vs Distortion Error Correction Channel Capacity Capacity vs Efficiency Decompression

4 Channel Coding/Decoding The purpose of channel coding/decoding is to detect and correct errors in noisy channels

5

6 Channel Coding/Decoding Error detection and correction In a noisy channel errors may occur during the transmission of data from information source to destination, so we need a method to detect there errors and then correct them.

7 Error-Detecting and -Correcting Codes Add extra bits to the data bits for the sole purpose of detecting when errors have been introduced and correcting the errors if possible

8 Repetition Code Repeat every bit 3 times 0110 ==> 000,111,111,000 Error detected if all 3 bits are not the same 000,110,111,000

9 Analysis of Repetition Code 200% overhead (code is 3x size of data) Error can go undetected only if 3 consecutive bits are in error 0110 => 000,111,111,000 => 000,000,111,000 If probability of one-bit error is p, then probability of undetected error is p 3 E.g. one-bit error = 10-5 => undetected error = (Assumes independence)

10 What if an Error is Detected? Strategy #1: Throw the data out and ask for it to be sent again Slow but very low odds of erroneous data Strategy #2: Majority rules 0110 => 000,111,111,000 => 000,110,111,000 => 000,111,111,000 Quicker, but higher odds of error Was it actually 0010 => 000,000,111,000 => 000,110,111,000?

11 Parity Codes Add one bit to every block of, say, 4 bits Parity bit = 0 or 1 so that total number of 1 bits is even Detects all 1-bit errors, no 2-bit errors

12 Channel Coding/Decoding Hamming Method - It was the first complete error-detecting and error-correcting procedure. - It represents one of the simplest and most common method for the transmission of information ( in the presence of noise ). - It assumes that the source transmits binary messages ( i.e. The information source alphabet is { 0, 1 }) - It uses the parity checker method to detect an error - It assumes that the channel is a binary symmetric channel (BSC)

13 Hamming Codes (4,7) Hamming Code detects all one- and two-bit errors Corrects all 1-bit errors Magic: Any two different codewords differ in at least 3 places!

14 Hamming Distance Number of places in which two bit strings differ = Hamming distance Acts like a distance: a b = a+b

15 Error Correcting Codes ECC design is a kind of geometry problem: Find 16 bit strings of length 7, no two of which are separated by distance less than 3

16 Hamming Distance as Geometry

17 Definitions Hamming distance between x and y is d H := d(x, y) is the # of positions where x i y i The minimum distance of a code C is d min = min { d(x, y) x C, y C, x y} Hamming weight of a vector x is - w(x) := d(x, 0 ) is the # of positions where x i 0

18 Example Hamming distance d( 1001, 0111) = 3 Minimum distance (101, 011, 110) = 2 Hamming weight w( ) = 4

19 Performance A code with minimum distance d min is capable of correcting t errors if d min 2 t + 1. Proof: If t errors occur, then since d min 2 t + 1 an incorrect code word has at least t+1 differences with the received word.

20 Hamming codes Minimum distance 3 Construction G = I m All k-tuples of Hamming weight > 1 where m = 2 k k 1

21 Example k = 4, n = G = m = = 4

22 Syndrome decoding Let G = [ I k P ] then construct H T = P I n-k For all code words c = xg, ch T = xgh T = 0 Hence, for a received noisy vector ( c n ) H T = c H T n H T = n H T = : S

23 Example G = H T = x = c = c H T = n = c n = [c n] H T = S = Obvious fast decoder: precalculate all syndromes at receiver for correctable errors

24 In system form c n Calculate syndrome c n c n n* [c n] H T = S n* Precalculated syndromes when n = n* then n n* = 0

25 Structure: Reed Solomon Codes (CD, DVD) m k information symbols n-k check symbols Properties: minimum distance = n-k+1 (symbols) length 2 m -1

26 General remarks The general problem is the decoding RS codes can be decoded using Euclids algorithm Berlekamp Massey algorithm

27 Why error correction? Systems with errors can be made almost error free CD, DvD would not work without RS codes