Channel Coding. Baltic Summer School Tartu (Estonia) August 2008

Transcription

1 Baltic Summer School 2008, Tartu, Estonia, Channel Coding Baltic Summer School Tartu (Estonia) August 2008 Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 1

2 Introduction Baltic Summer School 2008, Tartu, Estonia, Contact Prof. Dr.-Ing. habil. Andreas Ahrens Hochschule Wismar University of Technology, Business and Design Department of Electrical Engineering and Computer Science Philipp-Müller-Straße Wismar Germany Phone: (+49) / Fax: (+49) / andreas.ahrens@hs-wismar.de Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 2

3 Introduction Baltic Summer School 2008, Tartu, Estonia, Contents (1) Introduction Error-detection and error-correction Channel characteristics, error-structures with and without memory Examples Fundamentals of linear block codes Encoding and decoding Systematic and non-systematic codes Examples Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 3

4 Introduction Baltic Summer School 2008, Tartu, Estonia, Contents (2) Fundamentals of convolutional codes Description (code tree, state diagram, trellis diagram) Encoding, Decoding, Termination, Puncturing Interleaving, Error Probability Digital Channels with Memory Exercises & Solutions Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 4

5 Introduction Baltic Summer School 2008, Tartu, Estonia, Literature (1) Anderson, J. B.: Digital Transmission Engineering. München, New York: IEEE Press, 1999 Bingham, J. A. C.: The Theory and Practice of Modem Design. New York: Wiley, 1988 Bossert, M.: Channel Coding for Telecommunications. New York: Wiley, 1999 Bossert, M.: Kanalcodierung. Stuttgart: Teubner, 1998 Clark, G. C.; Cain, J. B.: Error-Control Coding for digital Communications. New York: Plenum, 1981 Friedrichs, B.: Kanalcodierung - Grundlagen und Anwendungen in Kommunikationssystemen. Berlin: Springer, 1995 Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 5

6 Introduction Baltic Summer School 2008, Tartu, Estonia, Literature (2) Goldsmith, A.: Wireless Communications. New York: Cambridge, 2005 Richardson, T.; Urbanke, R.: Modern Coding Theory. New York: Cambridge, 2008 Huber, J.: Trelliscodierung. Berlin, Heidelberg: Springer, 1992 Larrson, R.; Stoica, P.: Space-Time Block Coding for Wireless Communications. New York: Cambridge, 2003 Öberg, T.: Modulation, Detection and Coding. Chichester: Wiley, 2001 Paulraj, A.; Nabar, R.; Gore, D.: Introduction to Space-Time Wireless Communications. New York: Cambridge, 2003 Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 6

7 Introduction Baltic Summer School 2008, Tartu, Estonia, Literature (3) Proakis, J. G.: Digital communications. Boston: McGraw-Hill, 2000 Proakis, J. G.; Salehi, M.: Grundlagen der Kommunikationstechnik. München: Pearson-Education, 2004 Rohling, H.: Einführung in die Informations- und Codierungstheorie. Stuttgart: Teubner, 1995 Schneider-Obermann, H.: Kanalcodierung Theorie und Praxis fehlerkorrigierender Codes. Braunschweig/Wiesbaden: Vieweg, 1998 Hamming, R. W.: Information and Coding. Prentice Hall, 1988 Hanzo, L.; Liew, T. H.; Yeap, B. L.: Turbo Coding, Turbo Equalisation and Space-Time Coding for Transmission over Fading Channels. Chichester: Wiley, 2002 Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 7

8 Introduction Baltic Summer School 2008, Tartu, Estonia, Introduction Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 8

9 Introduction Baltic Summer School 2008, Tartu, Estonia, Channel Coding Principles Three main areas of coding Source coding (entropy coding) Channel coding Cryptography Channel Coding Encoder adds redundancy (additional bits) to information bits in order to detect or even correct transmission errors. Distinguish between: Forward Error Correction (FEC) Automatic Repeat Request (ARQ) Hybrid FEC/ARQ Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 9

10 Introduction Baltic Summer School 2008, Tartu, Estonia, Channel Coding Definitions (Definition 1) Channel coding allows bit errors introduced by transmission of a modulated signal through a wireless or wireline channel to be either detected or corrected by a decoder in a receiver. (Definition 2) The task of channel coding is to represent the source information in a manner that minimizes the error probability in decoding. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 10

11 Introduction Baltic Summer School 2008, Tartu, Estonia, Historical Perspective The history of channel coding or Forward Error Correction (FEC) coding dates back to Shannon s pioneering work in Shannon founded the information theory 1950 Single error correcting Hamming code 1955 Convolutional FEC codes (Elias) 1959/60 Multiple error correcting codes (Bose, Chaudhuri, Hocquenghem) 1960 Burst-error correcting codes (Reed, Solomon) 1967 Maximum Likelihood sequence estimation (Viterbi) 1987 Trellis coded modulation (Ungerboeck) 1993 Turbo codes (Berrou, Glavieux, Thitimajshima) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 11

12 Introduction Baltic Summer School 2008, Tartu, Estonia, Channel Coding Applications Importance of channel coding increased with digital communications First use for deep space communications AWGN channel, no bandwidth restrictions, only few receivers (costs negligible) Examples: Viking (Mars), Voyager (Jupiter, Saturn), Galileo (Jupiter) Mass storage Compact disc, digital versatile disc, magnetic tapes Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 12

13 Introduction Baltic Summer School 2008, Tartu, Estonia, Channel Coding Applications Furthermore, channel coding algorithms can be found, for example, in in the family of Digital Video Broadcasting (DVB) schemes, in satellite communication, in wireless communication (e. g. GSM, UMTS, Bluetooth, WLAN (Hiperlan, IEEE )) and in wireline communication (e. g. DSL). Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 13

14 Introduction Baltic Summer School 2008, Tartu, Estonia, Other Possibilities of Coding Encryption Cryptography,... Codes for spectral adaptation (e. g. Line codes) Pseudo-ternary codes (e. g. HDB-3, AMI), Manchester code, partial-response codes,... Codes for frame synchronization or channel identification Barker codes, pseudo-noise codes,... Codes for spreading Pseudo-noise codes, Gold codes, orthogonal Gold codes,... Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 14

15 Introduction Baltic Summer School 2008, Tartu, Estonia, BP/LPtransform Structure of Digital Transmission System binary sequence binary sequence sampling quantization compression channel encoder modulator source encoding analogue signal disturbances physical channel Likelihood information signal reconstr. decompression DAconverter channel decoder demodulator source decoding digital channel Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 15

16 Introduction Baltic Summer School 2008, Tartu, Estonia, Applications without distortions with distortions Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 16

17 Introduction Baltic Summer School 2008, Tartu, Estonia, Source Coding (1) Representation of a message or a signal by a value-discrete and a time-discrete sequence (e. g. a binary sequence) with an as small as possible bit rate. Speech signal: Frequency range: 300 Hz to 3.4 khz (in telephone networks) Sample rate: 8 khz Quantization: 8 bit/sample Result: 64 kbit/s (not available in wireless communication systems such as cordless telephones or cellular radio networks) Research: CELP (Codebook Exited Linear Predictive Coding) kbit/s with approximately the same quality MELP (Mixed Excitation Linear Prediction) kbit/s Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 17

18 Introduction Baltic Summer School 2008, Tartu, Estonia, Source Coding (2) Audio signal: Frequency range: 20 Hz to 20 khz Sample rate: 44 khz Quantization: 17 bit/sample Result: > 700 kbit/s Research: MPEG-1 Audio Layer 3 (MP3) Advanced Audio Coding (AAC) Bit stream is highly sensitive against transmission errors channel coding. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 18

19 Introduction Baltic Summer School 2008, Tartu, Estonia, Structure of Digital Transmission System (1) Channel encoder: Channel encoder adds redundancy so that errors can be detected or even corrected Consists of several constituent codes Modulator: Maps discrete vector onto analog waveform and moves it into the transmission band Physical channel: Represents transmission medium - multipath propagation and time varying fading Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 19

20 Introduction Baltic Summer School 2008, Tartu, Estonia, Mobile Channel Characteristic acements direct path reflection dispersion (scattering) diffraction Time- and frequency dispersion of wireless transmission channels through multipath propagation and mobile objects. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 20

21 Introduction Baltic Summer School 2008, Tartu, Estonia, Structure of Digital Transmission System (2) Demodulator: Moves signal back into baseband and performs lowpass filtering, sampling, quantization Channel decoder: Error detection or error correction on the basis of received vector Since encoder may consist of several parts, decoder may also consist of several modules Discrete Channel analog part of modulator, physical channel and analog part of demodulator Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 21

22 Introduction Baltic Summer School 2008, Tartu, Estonia, Factors effecting the design of channel coding and modulations schemes bit-error rate delay coding gain bandwith coding/ modulation scheme delay throughput channel characteristic complexity Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 22

23 PSfrag replacements Introduction Baltic Summer School 2008, Tartu, Estonia, Basic Terms The digital Channel (1) {e k } error sequence input sequence {x k } digital channel {y k } output sequence Example: {x k } = {y k } = Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 23

24 Introduction Baltic Summer School 2008, Tartu, Estonia, Bit-Error Rate and Binary Symmetric Channel Bit-Error Rate p e = E{number of error bits per transmission} E{number of bits per transmission} In a binary channel four types of communication events can occur: 0 transmitted and 0 received no error 0 transmitted and 1 received error 1 transmitted and 1 received no error 1 transmitted and 0 received error. If the probability, p e, of a transmitted 0 being received as a 1 is equal to the probability of a transmitted 1 being received as a 0, then the binary channel is said to be symmetric. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 24

25 Introduction Baltic Summer School 2008, Tartu, Estonia, The finite Field GF(2) Addition Table Multiplication Table Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 25

26 Introduction Baltic Summer School 2008, Tartu, Estonia, PSfrag replacements Basic Terms The digital Channel (2) error sequence {e k } input sequence output sequence {x k } {y k } digital channel Example: {x k } = {y k } = {e k } = Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 26

27 Introduction Baltic Summer School 2008, Tartu, Estonia, Basic Terms The digital Channel (3) x x - x x x - - x x x x x x - x x x x x x - x - - x x - - x x x x x x x x x x x x x - - x x x x x x x x x x x x x x x x x x x x - - x x x x x x - x x x x x x x Parameter: x bit error - correct bit Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 27

28 Introduction Baltic Summer School 2008, Tartu, Estonia, Basic Terms The digital Channel (4) x x x x x x x x x x x x x x x x x x x x - - x x x x x x x x x x x x x x x x x x x - x x x x x x x x x - - x x x x x x x x x x x x Parameter: x bit error - correct bit Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 28

29 Block Codes Baltic Summer School 2008, Tartu, Estonia, Basic Terms Block Codes Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 29

30 Block Codes Baltic Summer School 2008, Tartu, Estonia, Basic Idea input channel digital channel output (m bit) encoder (n bit) channel (n bit) decoder (m bit) binary data block of the length n with N = 2 n possible combinations but only M = 2 m < N are chosen for the data transmission remaining combinations are forbidden M = 2 m valid codewords Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 30

31 Block Codes Baltic Summer School 2008, Tartu, Estonia, Basic Idea Example input channel digital channel output (m bit) encoder (n bit) channel (n bit) decoder (m bit) Let us analyse a (7, 4) code n = 7 block length and m = 4 information bits 2 7 possible combinations, but only 2 4 valid codewords How many bit-errors are correctable? Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 31

32 Block Codes Baltic Summer School 2008, Tartu, Estonia, Basic Terms Error Detection (1) input channel digital channel output (m bit) encoder (n bit) channel (n bit) decoder (m bit) possible transmit vectors (0 0 0) (0 0 1) (0 1 0) (0 1 1) (1 0 0) (1 0 1) (1 1 0) (1 1 1) Can we detect or correct transmission errors? possible receive vectors (0 0 0) (0 0 1) (0 1 0) (0 1 1) (1 0 0) (1 0 1) (1 1 0) (1 1 1) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 32

33 Block Codes Baltic Summer School 2008, Tartu, Estonia, Basic Terms Error Detection (2) input channel digital channel output (m bit) encoder (n bit) channel (n bit) decoder (m bit) possible transmit vectors (0 0 0) ( ) ( ) (0 1 1) ( ) (1 0 1) (1 1 0) ( ) Can we detect or correct transmission errors? possible receive vectors (0 0 0) (0 0 1) (0 1 0) (0 1 1) (1 0 0) (1 0 1) (1 1 0) (1 1 1) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 33

34 Block Codes Baltic Summer School 2008, Tartu, Estonia, Basic Terms Definitions (Hamming) weight The (Hamming) weight w{ } of a vector (codeword) c is defined as the number of non-zero vector coordinates. This number ranges from a minimum value of zero to the length of the vector. (Hamming) distance The (Hamming) distance h between two vectors c µ and c ν is the number of coordinates where c µ and c ν differ h = d{c µ, c ν } = w{c µ c ν }. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 34

35 Block Codes Baltic Summer School 2008, Tartu, Estonia, Basic Terms Examples Example - I c 1 = ( ) c 2 = ( ) Find the (Hamming) weights of c 1 and c 2 and the (Hamming) distance between c 1 and c 2! Example - II c 1 = ( ) c 2 = ( ) c 3 = ( ) Find the (Hamming) weights and the (Hamming) distances! Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 35

36 Block Codes Baltic Summer School 2008, Tartu, Estonia, Basic Terms Definitions Minimum (Hamming) distance The minimum (Hamming) distance h min of a code C is the minimum distance between any two codewords. Example The code consists of the following codewords: c 1 = (0 0 0) c 2 = (0 1 1) c 3 = (1 0 1) c 4 = (1 1 0) Find the minimum (Hamming) distance of the code! Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 36

37 Block Codes Baltic Summer School 2008, Tartu, Estonia, Basic Terms Example Minimum (Hamming) Distance The code consists of the following codewords: c 1 = (0 0 0) c 2 = (0 1 1) c 3 = (1 0 1) c 4 = (1 1 0) Combination distance Combination codeword weight c 1 + c 2 = (0 1 1) 2 c 1 + c 3 = (1 0 1) 2 c 1 + c 4 = (1 1 0) 2 c 2 + c 3 = (1 1 0) 2 c 2 + c 4 = (1 0 1) 2 c 3 + c 4 = (0 1 1) 2 c 1 + c 2 = (0 1 1) c 2 2 c 1 + c 3 = (1 0 1) c 3 2 c 1 + c 4 = (1 1 0) c 4 2 c 2 + c 3 = (1 1 0) c 4 2 c 2 + c 4 = (1 0 1) c 3 2 c 3 + c 4 = (0 1 1) c 2 2 Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 37

38 Block Codes Baltic Summer School 2008, Tartu, Estonia, Basic Terms Definitions A desirable structure for a block code to possess is linearity, which greatly reduces the encoding complexity. (Linearity) A binary block code is linear if and only if the modulo-2 sum of any two codewords is also a codeword. (Minimum (Hamming) distance) The minimum (Hamming) distance of a linear block code is equal to the minimum weight of its nonzero codewords. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 38

39 Block Codes Baltic Summer School 2008, Tartu, Estonia, Basic Terms How many bit-errors are detectable? Example I : h min =? Example II : h min =? Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 39

40 Block Codes Baltic Summer School 2008, Tartu, Estonia, Basic Terms Definitions How many bit-errors are detectable? A block code with a minimum distance h min guarantees detection of all the error patterns of or fewer errors. f e = h min 1 Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 40

41 Block Codes Baltic Summer School 2008, Tartu, Estonia, Basic Terms How many bit-errors are correctable? Example I : h min =? Example II : h min =? Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 41

42 Block Codes Baltic Summer School 2008, Tartu, Estonia, Basic Terms Definitions Random-error-correcting capability A block code with a minimum distance h min guarantees correction of all the error patterns of hmin 1 f k = 2 errors, where hmin 1 2 denotes the largest integer no greater than (h min 1)/2. The parameter f k is called the random-errorcorrecting capability of the code. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 42

43 Block Codes Baltic Summer School 2008, Tartu, Estonia, Visualizing Distance Properties with Code Cube h min = 1 h min = 2 h min = 3 no error detection no error correction detection of 1 error no error correction detection of 2 errors correction of 1 error Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 43

44 Block Codes Baltic Summer School 2008, Tartu, Estonia, Distance Properties and IOWEF of Block Codes Distance Spectrum n A(D) = a d D d d=0 Parameter a d : number of codewords with the weight d Input Output Weight Enumerating Function (IOWEF) m n A(W, D) = a w,d W w D d, w=0 d=0 Parameter a w,d : number of codewords with the weight d and the weight w of the corresponding input bits Placeholder: W weight of (uncoded) input word D weight of (coded) output word Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 44

45 Block Codes Baltic Summer School 2008, Tartu, Estonia, Distance Spectrum Example: (3,2,2) SPC-Code codeword weight info word weight code word w{i} w{c} c 0 = (0 0 0) 0 0 c 1 = (1 0 1) 1 2 c 2 = (0 1 1) 1 2 c 3 = (1 1 0) 2 2 Distance Spectrum A(D) = n a d D d = a 0 D 0 + a 1 D a n D n d=0 = 1 + 3D 2. Interpretation: 3D 2 3 codewords with the weight d = 2 Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 45

46 Block Codes Baltic Summer School 2008, Tartu, Estonia, Distance Spectrum and IOWEF Example Distance Spectrum A(D) IOWEF A(W, D) d A d IOWEF d = 0 d = 1 d = w = w = w = A(D) = D 2 A(W, D) = W D W 2 D 2 Interpretation: 2W D 2 2 codewords with the weight d = 2 and the weight w = 1 of the corresponding input bits Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 46

47 Block Codes Baltic Summer School 2008, Tartu, Estonia, Basis Principles of Error-Control Coding Forward Error Correction (FEC) The redundancy added in the transmitter is used to correct transmission errors in the receiver. Automatic Repeat Request (ARQ) The redundancy added in the transmitter is used to detect but not to correct transmission errors. Hybrid FEC/ARQ Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 47

48 Block Codes Baltic Summer School 2008, Tartu, Estonia, Summarizing Basic Terms (1) i c channel digital channel (m bit) encoder (n bit) channel (n bit) decoder (m bit) d î Parameter Description Meaning codeword c = (c 0, c 1, c 2,, c n 1) sequence of n bits (valid codeword, sequence of n binary symbols) information word i = (i 0, i 1, i 2,, i m 1) sequence of m bits (sequence of m binary information symbols) number of parity bits k = n m Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 48

49 Block Codes Baltic Summer School 2008, Tartu, Estonia, Summarizing Basic Terms (2) i c channel digital channel (m bit) encoder (n bit) channel (n bit) decoder (m bit) d î Parameter Description Meaning weight w{c} non-zero vector coordinates distance h = d{c µ, c ν} minimum distance h min = min d{cµ, cν} µ ν code rate R = m n = n k n = 1 k n efficiency of the code Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 49

50 Block Codes Baltic Summer School 2008, Tartu, Estonia, Summarizing Basic Terms (3) Code: set of codewords Encoder: device that maps an information word onto a codeword by adding redundancy (Hamming) distance: number of differing elements between two codewords (Hamming) weight: number of non-zero elements in a codeword Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 50

51 Baltic Summer School 2008, Tartu, Estonia, Exercises Consider a (3, 1) linear block code. a) Find all codewords of this code! b) Find the following code parameters: n, m and k! c) Find the minimum Hamming distance of the code! d) Is it a linear code? e) Find the following parameters: f e and f k! Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 51

52 Baltic Summer School 2008, Tartu, Estonia, Exercises Given is a simple 2-out-of-5-code of the length n = 5 that is composed of any possible words with the weight w{c} = 2. a) Specify the code C. Is it a linear code? b) Determine the distance properties of the code. What has to be considered? c) Find the following parameters: f e and f k! Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 52

53 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Linear Block Codes Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 53

54 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Linear Block Codes For a binary block code with 2 m codewords and length n, unless it has a certain special structure, the encoding will be prohibitively complex for large m and n. (Solution) Linearity A binary block code is linear if and only if the modulo-2 sum of any two codewords is also a codeword. Results: The all-zero vector is a valid codeword. The minimum distance of the linear block code is equal to the minimum weight of its nonzero codewords. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 54

55 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Linear Block Codes Example (1) The 2 m codewords of a (n, m) code arise by linear combination of the basic codewords. Basic codewords of a (7, 2) code a = ( ) b = ( ) The code has 2 m = 2 2 = 4 valid codewords. These are all-zero codeword : ( ) a : ( ) b : ( ) a b : ( ) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 55

56 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Linear Block Codes Example (2) Basic codewords of a (7, 3) code a = ( ) b = ( ) c = ( ) The code has 2 m = 2 3 = 8 valid codewords. These are all-zero codeword : ( ) a : ( ) b : ( ) c : ( ) a b : ( ) a c : ( ) b c : ( ) a b c : ( ) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 56

57 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Generator Matrix (1) The generator matrix is a compact description of how codewords are generated from information bits in a linear block code. All basic codewords generate a (m n) matrix g 0,0 g 0,1 g 0,n 1 g 1,0 g 1,1 g 1,n 1 G =.... = g m 1,0 g m 1,1 g m 1,n 1 g 0 g 1. g m 1. The rows of G are the m linearly independent basic vectors (codewords) g 0, g 1,, g m 1 of the code. Each codeword c can be constructed by c = i G. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 57

58 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Generator Matrix (2) Each codeword c can be constructed by with c = i G, i = (i 0, i 1, i 2,, i m 1 ) describing the vector of the information bits. The codeword results in g 0 g 1 c = (i 0, i 1, i 2,, i m 1 ). g m 1 = i 0 g 0 i 1 g 1 i m 1 g m 1. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 58

59 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Generator Matrix Example (1) Let us consider a linear (7, 3) code with a given generator matrix G G = Each codeword c can be constructed by with c = i G, i = (i 0, i 1, i 2,, i m 1 ) describing the vector of the information bits. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 59

60 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Generator Matrix Example (2) Basic codewords of a (5, 2) code g 0 = ( ) g 1 = ( ) Generator matrix of the (5, 2) code ( ) ( ) g G = = g 1 The code has 2 m = 2 2 = 4 valid codewords. all-zero codeword : ( ) g 0 : ( ) g 1 : ( ) g 0 g 1 : ( ) i 0 = 0, i 1 = 0 : ( ) i 0 = 1, i 1 = 0 : ( ) i 0 = 0, i 1 = 1 : ( ) i 0 = 1, i 1 = 1 : ( ) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 60

61 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Generator Matrix (3) A systematic linear block code is described by a generator matrix of the form a 0,0 a 0,1 a 0,k a G = 1,0 a 1,1 a 1,k = (I m A) a m 1,0 a m 1,1 a m 1,k 1 Note that any generator matrix of a (n, m) linear block code can be reduced by row operations and column permutations to a generator matrix in systematic form. Definition: I m (m m) Identity matrix Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 61

62 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Generator Matrix Example (3) Generator matrix of the systematic (5, 2) code ( ) ( ) g G = = = (I m A). g 1 The code has 2 m = 2 2 = 4 valid codewords. all-zero codeword : ( ) g 0 : ( ) g 1 : ( ) g 0 g 1 : ( ) i 0 = 0, i 1 = 0 : ( ) i 0 = 1, i 1 = 0 : ( ) i 0 = 0, i 1 = 1 : ( ) i 0 = 1, i 1 = 1 : ( ) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 62

63 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Linear Block Codes Example (1) Consider a (5, 2) code with a given generator matrix G ( ) G = and parity check matrix H H = Is it a systematic code? Is it a linear code? Is the vector ( ) a valid codeword? Determine the minimum (Hamming) distance! Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 63

64 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Linear Block Codes Example (2) Consider a linear code with a given generator matrix G G = Find the following code parameters: n, m and k! Find all codewords of the code! Determine the minimum (Hamming) distance! Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 64

65 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Modification of Linear Block Codes Expansion: appending additional parity check symbols n > n, m = m, R < R, h min h min Puncturing: removing redundancy from code word n < n, m = m, R > R, h min h min Lengthen: appending additional information symbols n > n, m > m, R > R, h min h min Shortening: removing information symbols from code word n < n, m < m, R < R, h min h min Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 65

66 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Principle of Channel Coding Code: set of code words Encoder: device that maps information word i onto codeword c by adding redundancy Systematic encoder: codeword c explicitly contains information word i Non-systematic encoder: codeword c does not contain information word i (Hamming) distance: number of differing elements between two codewords (Hamming) weight: number of non-zero elements in a codeword Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 66

67 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Encoding Summary input channel digital channel output (m bit) encoder (n bit) channel (n bit) decoder (m bit) How can we detect and correct transmission errors? Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 67

68 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Error Detection and Correction Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 68

69 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Block Codes Error Detection Let d be the received codeword resulting from the transmission of the codeword c. In the absence of channel errors, it yields: d = c. However, if the transmission is corrupted, one or more of the codeword elements in d will differ from those in c. It holds d = c f, where f = (f 0, f 1,, f n 1 ) is the error vector indicating which elements are corrupted by the channel. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 69

70 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Block Codes Error Detection Error Detection: receiver determines, if d is a valid codeword if d is an invalid codeword, error has been detected if d is valid, assumes no error Undetectable Errors: occurs when the transmitted codeword is changed by the channel and appears as another valid codeword at the receiver side Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 70

71 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Parity Check Matrix (1) Codewords c = (c 0, c 1,, c n 1 ) of a linear code can be defined by the following equation H c T = 0 (n m) 1 or c H T = 0 1 (n m) with H as parity check matrix. The parity check matrix is used to decode linear block codes with generator matrix G. With c = i G the test condition c H T = 0 1 (n m) results in c H T = i G H T = 0 1 (n m). Between the generator matrix and parity check matrix is valid. G H T = 0 m (n m) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 71

72 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Parity Check Matrix (2) The ((n m) n) parity check matrix H corresponding to a generator matrix G = (I m A) of a systematic code is defined as H = ( A T I k ). Parity check matrix of a systematic code H = a 0,0 a 1,0 a m 1, a 0,1 a 1,1 a m 1, a 0,k 1 a 1,k 1 a m 1,k = ( A T ) I k. Definition: I k (k k) Identity matrix Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 72

73 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Parity Check Matrix Example (1) (5, 2, 3) Code with G = (I m A) ( ) 1 0 a0,0 a 0,1 a 0,2 G = = (I 2 A) 0 1 a 1,0 a 1,1 a 1,2 and H = ( A T ) I k H = a 0,0 a 1, a 0,1 a 1, a 0,2 a 1, = ( A T I 3 ). With c = i G the test condition c H T = 0 1 (n m) results in c H T = i G H T = 0 1 (n m). Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 73

74 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Parity Check Matrix Example (2) The relationship between the generator matrix and the parity check matrix is given by: G H T = 0 m (n m). The expression G H T results for the (5, 2, 3) code in: a 0,0 a 0,1 a 0,2 a 1,0 a 1,1 a 1, ( 1 0 a0,0 a 0,1 a 0,2 ) a 1,0 a 1,1 a 1,2 with G H T = 0 m (n m) = ( ). Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 74

75 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Parity Check Matrix Example (3) (5, 2, 3) Code with G and H ( G = ) H = The codeword c = ( ) fulfills the following equation H c T = = Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 75

76 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Syndrome Decoding Let d be the received codeword resulting from the transmission of the codeword c. If the transmission is corrupted, one or more of the codeword elements in d will differ from those in c. It holds d = c f, where f = (f 0, f 1,, f n 1 ) is the error vector indicating which elements are corrupted by the channel. We define the syndrome S as S = d H T = (c f) H T = f H T. Note that the syndrome S 1 (n m) is a function only of the error pattern f and not the transmitted codeword c. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 76

77 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Syndrome Decoding Example (1) Question: Prove that if the sum of two error patterns is a valid codeword, then each pattern has the same syndrome. Answer: Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 77

78 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Syndrome Decoding Example (2) Question: A linear (5, 2) code has the codewords c 0 = ( ), c 1 = ( ), c 2 = ( ) and c 3 = ( ). Find the set of error patterns corresponding to non-detectable errors! Answer: Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 78

79 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Syndrome Decoding Example (3) (5, 2, 3) Code with G and H ( ) G = H = (5, 2, 3) Code Syndrome table assuming one bit-error error position error vector syndrome j f S 1 ( ) (1 1 0) 2 ( ) (1 0 1) 3 ( ) (1 0 0) 4 ( ) (0 1 0) 5 ( ) (0 0 1) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 79

80 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, (5, 2, 3) Code Example Let us transmit the codeword c = ( ) and receive (one bit-error) d = c f = ( ) ( ) = ( ). Let us calculate the syndrome The syndrome will result in: S = d H T = (c f) H T = f H T. S = (c f) H T = ( ) H T ( ) H T = (0 1 0). }{{} 0 Result: error-position: j = 4 Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 80

81 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, (5, 2, 3) Code Example Let us transmit the codeword c = ( ) and receive (two bit-errors) d = c f = ( ) ( ) = ( ). Let us calculate the syndrome S = d H T = (c f) H T = f H T. Now, the syndrome will result in: S = (c f) H T = ( ) H T ( ) H T = (1 0 0). }{{} 0 Result: error-position: j = 3? Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 81

82 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Syndrome Decoding Example (4) (5, 2, 3) Code Syndrome table assuming two bit-errors error position error vector syndrome j k S 1 2 ( ) (0 1 1) 1 3 ( ) (0 1 0) 1 4 ( ) (1 0 0) 1 5 ( ) (1 1 1) 2 3 ( ) (0 0 1) 2 4 ( ) (1 1 1) 2 5 ( ) (1 0 0) 3 4 ( ) (1 1 0) 3 5 ( ) (1 0 1) 4 5 ( ) (0 1 1) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 82

83 Linear Block Codes Baltic Summer School 2008, Tartu, Estonia, Transmission System with Syndrome Decoding replacements i encoder c channel d = c f f Determination syndrome Determination error-pattern ˆf î demapper c f ˆf d = c f Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 83

84 Convolutional Codes Baltic Summer School 2008, Tartu, Estonia, Convolutional Codes Structure, algebraic and graphical presentation Distance properties and termination Viterbi algorithm Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 84

85 Convolutional Codes Baltic Summer School 2008, Tartu, Estonia, Encoder of a Convolutional Code 1 2 n to Modulator eplacementsm bits 1 2 m 1 2 m 1 2 m Stage 1 Stage 2 Stage K The maximum span of output words that can be influenced by a given input bit is K and called the constraint length. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 85

86 Convolutional Codes Baltic Summer School 2008, Tartu, Estonia, Basics of Convolutional Codes Shift register structure with m K elements Memory leads to statistical dependence of successive code words In each cycle m bits are shifted Each bit affects the output K times K is called the constraint length, memory depth K 1 Coded symbols are calculated by modulo 2 additions of memory contents Codewords contain n bit code rate R = m/n (n, m, K) convolutional code Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 86

87 Convolutional Codes Baltic Summer School 2008, Tartu, Estonia, Equivalence of Block Codes and Convolutional Codes Convolutional codes m Information bits are mapped onto a codeword, consisting of n bits Codewords are interdependent due to the memory Block codes Codewords are independent Block codes are convolutional codes without memory Only sequences of finite length are viewed in practice Finite convolutionally encoded sequence can be viewed as a single code word generated by a block code Convolutional codes are a special case of block codes Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 87

88 Convolutional Codes Baltic Summer School 2008, Tartu, Estonia, Encoder of a (2, 1, 3) Convolutional Code Example to Modulator eplacements m = 1 bit Stage 1 Stage 2 Stage 3 Code is non-systematic and non-recursive (NSC-Code). Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 88

89 Convolutional Codes Baltic Summer School 2008, Tartu, Estonia, Encoder of a (2, 1, 3) NSC-Code Example input encoder following output bit state encoder state bits Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 89

90 Convolutional Codes Baltic Summer School 2008, Tartu, Estonia, Encoder of a (2, 1, 3) NSC-Code Example Alternative Description (1) eplacements information sequence code sequence memory elements Code is non-systematic and non-recursive (NSC-Code). Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 90

91 Convolutional Codes Baltic Summer School 2008, Tartu, Estonia, Properties of Convolutional Codes Only a small number of simple convolutional codes are of practical interest. Convolutional codes can not be constructed by algebraic methods but by computer search. Similar to block codes, systematic and non-systematic encoders are distinguished for convolutional codes. mostly non-systematic convolutional codes are of practical interest Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 91

92 Convolutional Codes Baltic Summer School 2008, Tartu, Estonia, Algebraic Description (1) Description by generators g i (octal) Example code with K = 3 and R = 1/2 Sfrag replacements g 1 = (g 1 0 g 1 1 g 1 2 ) = (1 1 1) 7 8 g 2 = (g 2 0 g 2 1 g 2 2 ) = (1 0 1) 5 8 c 1 (l) i(l) g 1 0 = 1 g 1 1 = 1 g 1 2 = 1 i(l 1) i(l 2) g 2 0 = 1 g 2 1 = 0 g 2 2 = 1 c 2 (l) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 92

93 Convolutional Codes Baltic Summer School 2008, Tartu, Estonia, Algebraic Description (2) Description by generators g i (octal) Example code with K = 3 and R = 1/2 g 1 = (g 1 0 g 1 1 g 1 2 ) = (1 1 1) 7 8 g 2 = (g 2 0 g 2 1 g 2 2 ) = (1 0 1) 5 8 Encoding by discrete convolution g 1 (x) = 1 + x + x 2 g 2 (x) = 1 + x 2 Encoding c(x) = [c 1 (x) c 2 (x)] = [i(x) g 1 (x) i(x) g 2 (x)] Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 93

94 Convolutional Codes Baltic Summer School 2008, Tartu, Estonia, Algebraic Description (3) Example code with K = 3 and R = 1/2 i = ( ) i(x) = 1 + x 3 + x 4 Generator polynomials g 1 (x) = 1 + x + x 2 and g 2 (x) = 1 + x 2 Encoding c(x) = [c 1 (x) c 2 (x)] = [i(x) g 1 (x) i(x) g 2 (x)] = [(1 + x 3 + x 4 ) (1 + x + x 2 ) (1 + x 3 + x 4 ) (1 + x 2 )] = [1 + x + x 2 + x 3 + x x 2 + x 3 + x 4 + x 5 + x 6 ] Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 94

95 Convolutional Codes Baltic Summer School 2008, Tartu, Estonia, Encoder of a (2, 1, 3) NSC-Code Example Encoding i = ( ) i(x) = 1 + x 3 + x 4 Sfrag replacements c(x) = [c 1 (x) c 2 (x)] = [1 + x + x 2 + x 3 + x x 2 + x 3 + x 4 + x 5 + x 6 ] c 1 (l) i(l) i(l 1) i(l 2) c 2 (l) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 95

96 Convolutional Codes Baltic Summer School 2008, Tartu, Estonia, Graphical Presentation of Convolutional Codes In order to describe a convolutional code, we must describe how the codeword generation depends on the m input bits and the encoder states, which have 2 m (K 1) possible values. There are multiple ways to describe a convolutional code: a) Code tree b) Finite state diagram c) Trellis diagram Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 96

97 rag replacements Convolutional Codes Baltic Summer School 2008, Tartu, Estonia, Graphical Presentation in the Code Tree Example: Code tree of a (2, 1, 3) NSC-code state Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 97

98 Convolutional Codes Baltic Summer School 2008, Tartu, Estonia, Graphical Presentation in Finite State Diagram Example: State diagram of a (2, 1, 3) NSC-code 1/ / /10 1/00 0/ / /00 0/11 Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 98