Channel Coding and Cryptography

Transcription

1 Hochschule Wismar Channel Coding and Cryptography Baltic Summer School Technical Informatics & Information Technology (BaSoTi) Tartu (Estonia) July/August 2012 Prof. Dr.-Ing. habil. Andreas Ahrens Communications Signal Processing Group Hochschule Wismar, University of Technology, Business and Design August 2012, Tartu, Estonia Andreas Ahrens 1

2 Outline Hochschule Wismar Digital Communications Introduction and principles of digital communications Channel Coding Introduction Fundamentals of linear block codes Encoding and decoding of linear block codes Interleaving Cryptography Introduction Fundamentals of cryptographic schemes RSA cryptosystem August 2012, Tartu, Estonia Andreas Ahrens 2

3 Hochschule Wismar Channel Coding and Cryptography Part I: Channel Coding August 2012, Tartu, Estonia Andreas Ahrens 3

4 Channel Coding (Intro.) Hochschule Wismar Problem Statement: 1) Message x to be transmitted is disturbed by the underlying transmission channel (e.g wireless or wireline channel). 2) Received message y differs from the transmitted message Unreliable Transmitted channel Message x (e.g. Wireless) y Received Message Solution Reliable communication over an unreliable channel requires some redundancy to be added at the transmitter side in order to be able to detect or even correct transmission errors August 2012, Tartu, Estonia Andreas Ahrens 4

5 Further Reading and Information Hochschule Wismar Understanding error-detecting and error-correcting coding schemes Bossert, M.: Channel Coding for Telecommunications. New York: Wiley, Öberg, T.: Modulation, Detection and Coding. Chichester: Wiley, Goldsmith, A.: Wireless Communications. New York: Cambridge, August 2012, Tartu, Estonia Andreas Ahrens 5

6 Hochschule Wismar Digital Communications Introduction and Principles of Digital Communications Prof. Dr.-Ing. habil. Andreas Ahrens Communications Signal Processing Group Hochschule Wismar, University of Technology, Business and Design August 2012, Tartu, Estonia Andreas Ahrens 6

7 Introduction Hochschule Wismar Internet broadband local area network switching centre Digital Radio Mondiale (DRM) access network mobile radio network WLAN copper cable August 2012, Tartu, Estonia Andreas Ahrens 7

8 Introduction Hochschule Wismar Applications such as Telelearning, Musicloads, Teleworking, accessnetwork bottleneck: transmission channel Challenges - Increased desire for communication and information interchange - Efficient and reliable information transport August 2012, Tartu, Estonia Andreas Ahrens 8

9 Message, Information, Signal Hochschule Wismar Message Information that contains some news for the receiving side Picture: Pope election, 2005 Information Transmitted by signals Signals carry information Source: Signals Signs with a predefined meaning August 2012, Tartu, Estonia Andreas Ahrens 9

10 Message, Information, Signal Hochschule Wismar Railway signals Light signals Possible signals for information transport Source: Source: Disturbance of the information transport (e.g. fog) August 2012, Tartu, Estonia Andreas Ahrens 10

11 Message, Information, Signal Hochschule Wismar Communications Engineering Electric signal Possible signals for information transport Voltage Voltage time time Disturbance of the information transport (e.g. noise) time August 2012, Tartu, Estonia Andreas Ahrens 11

12 Early History of Wireless Communication Hochschule Wismar Wireless Communication Transmission of information without wires Many people in history used light for communication Signaling towers (flags) (China, Han-Dynasty, 206 BC 24 AC) Smoke signals for communication (Greece, 150 BC) Optical telegraph, Claude Chappe (1794) Source: Source: August 2012, Tartu, Estonia Andreas Ahrens 12

13 History of Communication Hochschule Wismar Application to Communication 1837 Morse First Telegraph 1861 Reis First Telephone (Patent Bell 1876) Wireless Communication 1901 Marconi First transatlantic transmission first demonstration of wireless telegraphy (digital!) long wave transmission, high transmission power necessary (> 200 kw) Digital Communication 1948 Shannon A Mathematical Theory of Communication August 2012, Tartu, Estonia Andreas Ahrens 13

14 The Birth of Digital Communications Hochschule Wismar Claude Elwood Shannon ( ) Foundation of Information Theory Pioneering paper 1948: A Mathematical Theory of Communication Claude Elwood Shannon ( ) August 2012, Tartu, Estonia Andreas Ahrens 14

15 Message, Information, Signal Hochschule Wismar Classical Situation: Point-to-Point-Communication Transmitter Sender (TX) 1 1 Receiver (RX) Information Theory Channel Coding Theorem [Shannon, 1948] Claude Elwood Shannon ( ) With a given transmit power P S (in W), a maximum data rate (in bit/s) can be transmitted error-free (C = Channel Capacity) over a channel with a given bandwidth B HF and given noise power spectral density N 0 (in W/Hz). August 2012, Tartu, Estonia Andreas Ahrens 15

16 Message, Information, Signal Hochschule Wismar Challenges of Communications Engineering and Communications Technology Efficient and reliable information transport Increased desire for communication and information interchange Available Ressources are limited - Transmit Power - Regulatory control, - Bandwidth - Width of the usable frequency band - Complexity - Size and weight August 2012, Tartu, Estonia Andreas Ahrens 16

17 Hochschule Wismar Data Transmission August 2012, Tartu, Estonia Andreas Ahrens 17

18 Data Transmission Hochschule Wismar Data Transmission??? Mixer Amplifier Data, Speech, (baseband) Shifting into transmission band (khz GHz) Shifting back into baseband??? Mixer Amplifier August 2012, Tartu, Estonia Andreas Ahrens 18

19 Data Transmission Hochschule Wismar Transmitter: Generation of an analog signal; matched to the channel Analog modulation Amplitude or frequency of a carrier will be modulated (manipulated in the rhythm of the source signal) Source signal Transmitted signal Examples: Short-wave communication Television August 2012, Tartu, Estonia Andreas Ahrens 19

20 Digitale Übertragung Hochschule Wismar Receiver: Reconstruction of the transmitted signal Received signal Reconstructed source signal Examples: Short-wave communication Television August 2012, Tartu, Estonia Andreas Ahrens 20

21 Digital Data Transmission Hochschule Wismar Digital Transmission Digitalization of analog source signal binary data stream Binary data stream will again be transformed into an analog transmit signal Source signal Analog-to-Digital Converter (ADU) Manipulation data stream Transmitter for digital signals Transmitted signal Examples: GSM, UMTS Television August 2012, Tartu, Estonia Andreas Ahrens 21

22 Digital Data Transmission Hochschule Wismar Digital Transmission Source signal Analog-to-Digital Converter (ADU) Manipulation data stream Transmitter for digital signals Transmitted signal Voltage time August 2012, Tartu, Estonia Andreas Ahrens 22

23 Digital Data Transmission Hochschule Wismar Digital Transmission Source signal Analog-to-Digital Converter (ADU) Manipulation data stream Transmitter for digital signals Transmitted signal Source coding Compression, Data reduction Channel coding Error detection, error correction Cryptography Encryption August 2012, Tartu, Estonia Andreas Ahrens 23

24 Digital Data Transmission Hochschule Wismar Digital Transmission Source signal Analog-to-Digital Converter (ADU) Manipulation data stream Transmitter for digital signals Transmitted signal Voltage 0 1 Voltage time time Transmitted signal: Sinusoidal signal with a given frequency but variable amplitude and phase August 2012, Tartu, Estonia Andreas Ahrens 24

25 Digital Modulation Hochschule Wismar The transmitted signal can be constructed by using basic signal elements Example: Voltage 0 1 Voltage Transmitted signal time time or Voltage 0 1 Voltage time time August 2012, Tartu, Estonia Andreas Ahrens 25

26 Digital Modulation Hochschule Wismar Tool: Representation of the basis elements as points in a complex plane Q Voltage time Voltage time I Voltage Voltage time time Sinusoidal signal with a given frequency and amplitude but variable phase August 2012, Tartu, Estonia Andreas Ahrens 26

27 Digital Modulation Hochschule Wismar Tool: Representation of the basis elements as points in a complex plane Voltage Q Voltage time time I Voltage Voltage Voltage time time time Sinusoidal signal with a given frequency but variable amplitude and phase August 2012, Tartu, Estonia Andreas Ahrens 27

28 Digital Modulation Hochschule Wismar Digital Modulation 2 signal points 1 bit per symbol Q Transmitted signal 0 1 I August 2012, Tartu, Estonia Andreas Ahrens 28

29 Digital Modulation Hochschule Wismar Digital Modulation 4 signal points 2 bits per symbol Imag. Q 10 Transmitted signal I 01 August 2012, Tartu, Estonia Andreas Ahrens 29

30 Digital Modulation Hochschule Wismar Digital Modulation Quadrature Amplitude Modulation (QAM) 2 bits per symbol 4 bits per symbol 6 bits per symbol August 2012, Tartu, Estonia Andreas Ahrens 30

31 Digital Data Transmission Hochschule Wismar Additive White Gaussian Noise (AWGN) Most communication systems disturbed by AWGN Comprises all noise sources at receiver Gaussian Gaussian distributed by central limit theorem assumption White successive noise samples are statistically independent power density spectrum is flat in considered frequency band August 2012, Tartu, Estonia Andreas Ahrens 31

32 Digital Data Transmission Hochschule Wismar Gaussian Noise Gaussian Noise (random signal or stochastic signal) Scatterplot instead of exact signal points Noise Transmitted signal Received signal Q Q I I August 2012, Tartu, Estonia Andreas Ahrens 32

33 Probability of Error: AWGN (1) Hochschule Wismar Transmitter Sender (TX) QPSK 1 1 Receiver (RX) To double the data rate + 6 db is required! Transmitter Sender (TX) 16-QAM 1 1 Receiver (RX) To double the data rate + 12 db is required! Transmitter Sender (TX) 256-QAM 1 1 Receiver (RX) If existing systems use higher order modulation, unacceptable huge transmission power is required to double the data rate at a given quality. August 2012, Tartu, Estonia Andreas Ahrens 33

34 Probability of Error: AWGN (2) Hochschule Wismar Bit-error probability assuming different QAM constellation sizes August 2012, Tartu, Estonia Andreas Ahrens 34

35 Introduction Baltic Summer School 2012, Tartu, Estonia, Introduction to Channel Coding Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 35

36 Introduction Baltic Summer School 2012, Tartu, Estonia, Channel Coding Principles Three main areas of coding Source coding (entropy coding) Channel coding Cryptography Channel Coding Encoder adds redundancy (additional bits) to information bits in order to detect or even correct transmission errors. Distinguish between: Forward Error Correction (FEC) Automatic Repeat Request (ARQ) Hybrid FEC/ARQ Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 36

37 Introduction Baltic Summer School 2012, Tartu, Estonia, Channel Coding Definitions (Definition 1) Channel coding allows bit errors introduced by transmission of a modulated signal through a wireless or wireline channel to be either detected or corrected by a decoder in a receiver. (Definition 2) The task of channel coding is to represent the source information in a manner that minimizes the error probability in decoding. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 37

38 Introduction Baltic Summer School 2012, Tartu, Estonia, Historical Perspective The history of channel coding or Forward Error Correction (FEC) coding dates back to Shannon s pioneering work in Shannon founded the information theory 1950 Single error correcting Hamming code 1955 Convolutional FEC codes (Elias) 1959/60 Multiple error correcting codes (Bose, Chaudhuri, Hocquenghem) 1960 Burst-error correcting codes (Reed, Solomon) 1967 Maximum Likelihood sequence estimation (Viterbi) 1987 Trellis coded modulation (Ungerboeck) 1993 Turbo codes (Berrou, Glavieux, Thitimajshima) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 38

39 Introduction Baltic Summer School 2012, Tartu, Estonia, Channel Coding Applications Importance of channel coding increased with digital communications First use for deep space communications AWGN channel, no bandwidth restrictions, only few receivers (costs negligible) Examples: Viking (Mars), Voyager (Jupiter, Saturn), Galileo (Jupiter) Mass storage Compact disc, digital versatile disc, magnetic tapes Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 39

40 Introduction Baltic Summer School 2012, Tartu, Estonia, Channel Coding Applications Furthermore, channel coding algorithms can be found, for example, in in the family of Digital Video Broadcasting (DVB) schemes, in satellite communication, in wireless communication (e. g. GSM, UMTS, Bluetooth, WLAN (Hiperlan, IEEE )) and in wireline communication (e. g. DSL). Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 40

41 Introduction Baltic Summer School 2012, Tartu, Estonia, BP/LPtransform Structure of Digital Transmission System binary sequence binary sequence sampling quantization compression channel encoder modulator source encoding analogue signal disturbances physical channel Likelihood information signal reconstr. decompression DAconverter channel decoder demodulator source decoding digital channel Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 41

42 Introduction Baltic Summer School 2012, Tartu, Estonia, Applications without distortions with distortions Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 42

43 Introduction Baltic Summer School 2012, Tartu, Estonia, Source Coding (1) Representation of a message or a signal by a value-discrete and a time-discrete sequence (e. g. a binary sequence) with an as small as possible bit rate. Speech signal: Frequency range: 300 Hz to 3.4 khz (in telephone networks) Sample rate: 8 khz Quantization: 8 bit/sample Result: 64 kbit/s (not available in wireless communication systems such as cordless telephones or cellular radio networks) Research: CELP (Codebook Exited Linear Predictive Coding) kbit/s with approximately the same quality MELP (Mixed Excitation Linear Prediction) kbit/s Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 43

44 Introduction Baltic Summer School 2012, Tartu, Estonia, Source Coding (2) Audio signal: Frequency range: 20 Hz to 20 khz Sample rate: 44 khz Quantization: 17 bit/sample Result: > 700 kbit/s Research: MPEG-1 Audio Layer 3 (MP3) Advanced Audio Coding (AAC) Bit stream is highly sensitive against transmission errors channel coding. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 44

45 Introduction Baltic Summer School 2012, Tartu, Estonia, Structure of Digital Transmission System (1) Channel encoder: Channel encoder adds redundancy so that errors can be detected or even corrected Consists of several constituent codes Modulator: Maps discrete vector onto analog waveform and moves it into the transmission band Physical channel: Represents transmission medium - multipath propagation and time varying fading Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 45

46 Introduction Baltic Summer School 2012, Tartu, Estonia, Mobile Channel Characteristic acements direct path reflection dispersion (scattering) diffraction Time- and frequency dispersion of wireless transmission channels through multipath propagation and mobile objects. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 46

47 Introduction Baltic Summer School 2012, Tartu, Estonia, Structure of Digital Transmission System (2) Demodulator: Moves signal back into baseband and performs lowpass filtering, sampling, quantization Channel decoder: Error detection or error correction on the basis of received vector Since encoder may consist of several parts, decoder may also consist of several modules Discrete Channel analog part of modulator, physical channel and analog part of demodulator Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 47

48 Introduction Baltic Summer School 2012, Tartu, Estonia, Factors effecting the design of channel coding and modulation schemes bit-error rate delay coding gain bandwith coding/ modulation scheme delay throughput channel characteristic complexity Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 48

49 PSfrag replacements Introduction Baltic Summer School 2012, Tartu, Estonia, Basic Terms The digital Channel (1) {e k } error sequence input sequence {x k } digital channel {y k } output sequence Example: {x k } = {y k } = Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 49

50 Introduction Baltic Summer School 2012, Tartu, Estonia, Bit-Error Rate and Binary Symmetric Channel Bit-Error Rate p e = E{number of error bits per transmission} E{number of bits per transmission} In a binary channel four types of communication events can occur: 0 transmitted and 0 received no error 0 transmitted and 1 received error 1 transmitted and 1 received no error 1 transmitted and 0 received error. If the probability, p e, of a transmitted 0 being received as a 1 is equal to the probability of a transmitted 1 being received as a 0, then the binary channel is said to be symmetric. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 50

51 Introduction Baltic Summer School 2012, Tartu, Estonia, The finite Field GF(2) Addition Table Multiplication Table Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 51

52 Introduction Baltic Summer School 2012, Tartu, Estonia, PSfrag replacements Basic Terms The digital Channel (2) error sequence {e k } input sequence output sequence {x k } {y k } digital channel Example: {x k } = {y k } = {e k } = Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 52

53 Introduction Baltic Summer School 2012, Tartu, Estonia, Basic Terms The digital Channel (3) x x - x x x - - x x x x x x - x x x x x x - x - - x x - - x x x x x x x x x x x x x - - x x x x x x x x x x x x x x x x x x x x - - x x x x x x - x x x x x x x Parameter: x bit error - correct bit Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 53

54 Introduction Baltic Summer School 2012, Tartu, Estonia, Basic Terms The digital Channel (4) x x x x x x x x x x x x x x x x x x x x - - x x x x x x x x x x x x x x x x x x x - x x x x x x x x x - - x x x x x x x x x x x x Parameter: x bit error - correct bit Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 54

55 Block Codes Baltic Summer School 2012, Tartu, Estonia, Basic Terms Block Codes Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 55

56 Block Codes Baltic Summer School 2012, Tartu, Estonia, Basic Idea input channel digital channel output (m bit) encoder (n bit) channel (n bit) decoder (m bit) binary data block of the length n with N = 2 n possible combinations but only M = 2 m < N are chosen for the data transmission remaining combinations are forbidden M = 2 m valid codewords Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 56

57 Block Codes Baltic Summer School 2012, Tartu, Estonia, Basic Idea Example input channel digital channel output (m bit) encoder (n bit) channel (n bit) decoder (m bit) Let us analyse a (7, 4) code n = 7 block length and m = 4 information bits 2 7 possible combinations, but only 2 4 valid codewords How many bit-errors are correctable? Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 57

58 Block Codes Baltic Summer School 2012, Tartu, Estonia, Basic Terms Error Detection (1) input channel digital channel output (m bit) encoder (n bit) channel (n bit) decoder (m bit) possible transmit vectors (0 0 0) (0 0 1) (0 1 0) (0 1 1) (1 0 0) (1 0 1) (1 1 0) (1 1 1) Can we detect or correct transmission errors? possible receive vectors (0 0 0) (0 0 1) (0 1 0) (0 1 1) (1 0 0) (1 0 1) (1 1 0) (1 1 1) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 58

59 Block Codes Baltic Summer School 2012, Tartu, Estonia, Basic Terms Error Detection (2) input channel digital channel output (m bit) encoder (n bit) channel (n bit) decoder (m bit) possible transmit vectors (0 0 0) ( ) ( ) (0 1 1) ( ) (1 0 1) (1 1 0) ( ) Can we detect or correct transmission errors? possible receive vectors (0 0 0) (0 0 1) (0 1 0) (0 1 1) (1 0 0) (1 0 1) (1 1 0) (1 1 1) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 59

60 Block Codes Baltic Summer School 2012, Tartu, Estonia, Basic Terms Definitions (Hamming) weight The (Hamming) weight w{ } of a vector (codeword) c is defined as the number of non-zero vector coordinates. This number ranges from a minimum value of zero to the length of the vector. (Hamming) distance The (Hamming) distance h between two vectors c µ and c ν is the number of coordinates where c µ and c ν differ h = d{c µ, c ν } = w{c µ c ν }. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 60

61 Block Codes Baltic Summer School 2012, Tartu, Estonia, Basic Terms Examples Example - I c 1 = ( ) c 2 = ( ) Find the (Hamming) weights of c 1 and c 2 and the (Hamming) distance between c 1 and c 2! Example - II c 1 = ( ) c 2 = ( ) c 3 = ( ) Find the (Hamming) weights and the (Hamming) distances! Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 61

62 Block Codes Baltic Summer School 2012, Tartu, Estonia, Basic Terms Definitions Minimum (Hamming) distance The minimum (Hamming) distance h min of a code C is the minimum distance between any two codewords. Example The code consists of the following codewords: c 1 = (0 0 0) c 2 = (0 1 1) c 3 = (1 0 1) c 4 = (1 1 0) Find the minimum (Hamming) distance of the code! Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 62

63 Block Codes Baltic Summer School 2012, Tartu, Estonia, Basic Terms Example Minimum (Hamming) Distance The code consists of the following codewords: c 1 = (0 0 0) c 2 = (0 1 1) c 3 = (1 0 1) c 4 = (1 1 0) Combination distance Combination codeword weight c 1 + c 2 = (0 1 1) 2 c 1 + c 3 = (1 0 1) 2 c 1 + c 4 = (1 1 0) 2 c 2 + c 3 = (1 1 0) 2 c 2 + c 4 = (1 0 1) 2 c 3 + c 4 = (0 1 1) 2 c 1 + c 2 = (0 1 1) c 2 2 c 1 + c 3 = (1 0 1) c 3 2 c 1 + c 4 = (1 1 0) c 4 2 c 2 + c 3 = (1 1 0) c 4 2 c 2 + c 4 = (1 0 1) c 3 2 c 3 + c 4 = (0 1 1) c 2 2 Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 63

64 Block Codes Baltic Summer School 2012, Tartu, Estonia, Basic Terms Definitions A desirable structure for a block code to possess is linearity, which greatly reduces the encoding complexity. (Linearity) A binary block code is linear if and only if the modulo-2 sum of any two codewords is also a codeword. (Minimum (Hamming) distance) The minimum (Hamming) distance of a linear block code is equal to the minimum weight of its nonzero codewords. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 64

65 Block Codes Baltic Summer School 2012, Tartu, Estonia, Basic Terms How many bit-errors are detectable? Example I : h min =? Example II : h min =? Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 65

66 Block Codes Baltic Summer School 2012, Tartu, Estonia, Basic Terms Definitions How many bit-errors are detectable? A block code with a minimum distance h min guarantees detection of all the error patterns of or fewer errors. f e = h min 1 Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 66

67 Block Codes Baltic Summer School 2012, Tartu, Estonia, Basic Terms How many bit-errors are correctable? Example I : h min =? Example II : h min =? Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 67

68 Block Codes Baltic Summer School 2012, Tartu, Estonia, Basic Terms Definitions Random-error-correcting capability A block code with a minimum distance h min guarantees correction of all the error patterns of hmin 1 f k = 2 errors, where hmin 1 2 denotes the largest integer no greater than (h min 1)/2. The parameter f k is called the random-errorcorrecting capability of the code. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 68

69 Block Codes Baltic Summer School 2012, Tartu, Estonia, Visualizing Distance Properties with Code Cube h min = 1 h min = 2 h min = 3 no error detection no error correction detection of 1 error no error correction detection of 2 errors correction of 1 error Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 69

70 Block Codes Baltic Summer School 2012, Tartu, Estonia, Basis Principles of Error-Control Coding Forward Error Correction (FEC) The redundancy added in the transmitter is used to correct transmission errors in the receiver. Automatic Repeat Request (ARQ) The redundancy added in the transmitter is used to detect but not to correct transmission errors. Hybrid FEC/ARQ Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 70

71 Block Codes Baltic Summer School 2012, Tartu, Estonia, Summarizing Basic Terms (1) i c channel digital channel (m bit) encoder (n bit) channel (n bit) decoder (m bit) d î Parameter Description Meaning codeword c = (c 0, c 1, c 2,, c n 1) sequence of n bits (valid codeword, sequence of n binary symbols) information word i = (i 0, i 1, i 2,, i m 1) sequence of m bits (sequence of m binary information symbols) number of parity bits k = n m Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 71

72 Block Codes Baltic Summer School 2012, Tartu, Estonia, Summarizing Basic Terms (2) i c channel digital channel (m bit) encoder (n bit) channel (n bit) decoder (m bit) d î Parameter Description Meaning weight w{c} non-zero vector coordinates distance h = d{c µ,c ν} minimum distance h min = min µ ν d{cµ,cν} code rate R = m n = n k n = 1 k n efficiency of the code Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 72

73 Block Codes Baltic Summer School 2012, Tartu, Estonia, Summarizing Basic Terms (3) Code: set of codewords Encoder: device that maps an information word onto a codeword by adding redundancy (Hamming) distance: number of differing elements between two codewords (Hamming) weight: number of non-zero elements in a codeword Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 73

74 Block Codes Baltic Summer School 2012, Tartu, Estonia, Exercises Consider a (3, 1) linear block code. a) Find all codewords of this code! b) Find the following code parameters: n, m and k! c) Find the minimum Hamming distance of the code! d) Is it a linear code? e) Find the following parameters: f e and f k! Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 74

75 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Linear Block Codes Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 75

76 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Linear Block Codes For a binary block code with 2 m codewords and length n, unless it has a certain special structure, the encoding will be prohibitively complex for large m and n. (Solution) Linearity A binary block code is linear if and only if the modulo-2 sum of any two codewords is also a codeword. Results: The all-zero vector is a valid codeword. The minimum distance of the linear block code is equal to the minimum weight of its nonzero codewords. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 76

77 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Linear Block Codes Example (1) The 2 m codewords of a (n, m) code arise by linear combination of the basic codewords. Basic codewords of a (7, 2) code a = ( ) b = ( ) The code has 2 m = 2 2 = 4 valid codewords. These are all-zero codeword : ( ) a : ( ) b : ( ) a b : ( ) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 77

78 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Linear Block Codes Example (2) Basic codewords of a (7, 3) code a = ( ) b = ( ) c = ( ) The code has 2 m = 2 3 = 8 valid codewords. These are all-zero codeword : ( ) a : ( ) b : ( ) c : ( ) a b : ( ) a c : ( ) b c : ( ) a b c : ( ) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 78

79 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Generator Matrix (1) The generator matrix is a compact description of how codewords are generated from information bits in a linear block code. All basic codewords generate a (m n) matrix g 0,0 g 0,1 g 0,n 1 g 1,0 g 1,1 g 1,n 1 G =.... = g m 1,0 g m 1,1 g m 1,n 1 g 0 g 1. g m 1. The rows of G are the m linearly independent basic vectors (codewords) g 0, g 1,, g m 1 of the code. Each codeword c can be constructed by c = i G. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 79

80 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Generator Matrix (2) Each codeword c can be constructed by with c = i G, i = (i 0, i 1, i 2,, i m 1 ) describing the vector of the information bits. The codeword results in g 0 g 1 c = (i 0, i 1, i 2,, i m 1 ). g m 1 = i 0 g 0 i 1 g 1 i m 1 g m 1. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 80

81 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Generator Matrix Example (1) Let us consider a linear (7, 3) code with a given generator matrix G G = Each codeword c can be constructed by with c = i G, i = (i 0, i 1, i 2,, i m 1 ) describing the vector of the information bits. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 81

82 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Generator Matrix Example (2) Basic codewords of a (5, 2) code g 0 = ( ) g 1 = ( ) Generator matrix of the (5, 2) code ( ) ( ) g G = = g 1 The code has 2 m = 2 2 = 4 valid codewords. all-zero codeword : ( ) g 0 : ( ) g 1 : ( ) g 0 g 1 : ( ) i 0 = 0, i 1 = 0 : ( ) i 0 = 1, i 1 = 0 : ( ) i 0 = 0, i 1 = 1 : ( ) i 0 = 1, i 1 = 1 : ( ) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 82

83 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Generator Matrix (3) A systematic linear block code is described by a generator matrix of the form a 0,0 a 0,1 a 0,k a G = 1,0 a 1,1 a 1,k = (I m A) a m 1,0 a m 1,1 a m 1,k 1 Note that any generator matrix of a (n, m) linear block code can be reduced by row operations and column permutations to a generator matrix in systematic form. Definition: I m (m m) Identity matrix Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 83

84 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Generator Matrix Example (3) Generator matrix of the systematic (5, 2) code ( ) ( ) g G = = = (I m A). g 1 The code has 2 m = 2 2 = 4 valid codewords. all-zero codeword : ( ) g 0 : ( ) g 1 : ( ) g 0 g 1 : ( ) i 0 = 0, i 1 = 0 : ( ) i 0 = 1, i 1 = 0 : ( ) i 0 = 0, i 1 = 1 : ( ) i 0 = 1, i 1 = 1 : ( ) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 84

85 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Linear Block Codes Example (1) Consider a (5, 2) code with a given generator matrix G ( ) G = and parity check matrix H H = Is it a systematic code? Is it a linear code? Is the vector ( ) a valid codeword? Determine the minimum (Hamming) distance! Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 85

86 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Linear Block Codes Example (2) Consider a linear code with a given generator matrix G G = Find the following code parameters: n, m and k! Find all codewords of the code! Determine the minimum (Hamming) distance! Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 86

87 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Principle of Channel Coding Code: set of code words Encoder: device that maps information word i onto codeword c by adding redundancy Systematic encoder: codeword c explicitly contains information word i Non-systematic encoder: codeword c does not contain information word i (Hamming) distance: number of differing elements between two codewords (Hamming) weight: number of non-zero elements in a codeword Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 87

88 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Encoding Summary input channel digital channel output (m bit) encoder (n bit) channel (n bit) decoder (m bit) How can we detect and correct transmission errors? Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 88

89 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Error Detection and Correction Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 89

90 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Block Codes Error Detection Let d be the received codeword resulting from the transmission of the codeword c. In the absence of channel errors, it yields: d = c. However, if the transmission is corrupted, one or more of the codeword elements in d will differ from those in c. It holds d = c f, where f = (f 0, f 1,, f n 1 ) is the error vector indicating which elements are corrupted by the channel. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 90

91 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Block Codes Error Detection Error Detection: receiver determines, if d is a valid codeword if d is an invalid codeword, error has been detected if d is valid, assumes no error Undetectable Errors: occurs when the transmitted codeword is changed by the channel and appears as another valid codeword at the receiver side Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 91

92 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Parity Check Matrix (1) Codewords c = (c 0, c 1,, c n 1 ) of a linear code can be defined by the following equation H c T = 0 (n m) 1 or c H T = 0 1 (n m) with H as parity check matrix. The parity check matrix is used to decode linear block codes with generator matrix G. With c = i G the test condition c H T = 0 1 (n m) results in c H T = i G H T = 0 1 (n m). Between the generator matrix and parity check matrix is valid. G H T = 0 m (n m) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 92

93 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Parity Check Matrix (2) The ((n m) n) parity check matrix H corresponding to a generator matrix G = (I m A) of a systematic code is defined as H = ( A T I k ). Parity check matrix of a systematic code H = a 0,0 a 1,0 a m 1, a 0,1 a 1,1 a m 1, a 0,k 1 a 1,k 1 a m 1,k = ( A T ) I k. Definition: I k (k k) Identity matrix Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 93

94 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Parity Check Matrix Example (1) (5, 2, 3) Code with G = (I m A) ( ) 1 0 a0,0 a 0,1 a 0,2 G = = (I 2 A) 0 1 a 1,0 a 1,1 a 1,2 and H = ( A T ) I k H = a 0,0 a 1, a 0,1 a 1, a 0,2 a 1, = ( A T I 3 ). With c = i G the test condition c H T = 0 1 (n m) results in c H T = i G H T = 0 1 (n m). Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 94

95 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Parity Check Matrix Example (2) The relationship between the generator matrix and the parity check matrix is given by: G H T = 0 m (n m). The expression G H T results for the (5, 2, 3) code in: a 0,0 a 0,1 a 0,2 a 1,0 a 1,1 a 1, ( 1 0 a0,0 a 0,1 a 0,2 ) a 1,0 a 1,1 a 1,2 with G H T = 0 m (n m) = ( ). Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 95

96 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Parity Check Matrix Example (3) (5, 2, 3) Code with G and H ( G = ) H = The codeword c = ( ) fulfills the following equation H c T = = Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 96

97 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Syndrome Decoding Let d be the received codeword resulting from the transmission of the codeword c. If the transmission is corrupted, one or more of the codeword elements in d will differ from those in c. It holds d = c f, where f = (f 0, f 1,, f n 1 ) is the error vector indicating which elements are corrupted by the channel. We define the syndrome S as S = d H T = (c f) H T = f H T. Note that the syndrome S 1 (n m) is a function only of the error pattern f and not the transmitted codeword c. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 97

98 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Syndrome Decoding Example (1) Question: Prove that if the sum of two error patterns is a valid codeword, then each pattern has the same syndrome. Answer: Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 98

99 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Syndrome Decoding Example (2) Question: A linear (5, 2) code has the codewords c 0 = ( ), c 1 = ( ), c 2 = ( ) and c 3 = ( ). Find the set of error patterns corresponding to non-detectable errors! Answer: Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 99

100 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Syndrome Decoding Example (3) (5, 2, 3) Code with G and H ( ) G = H = (5, 2, 3) Code Syndrome table assuming one bit-error error position error vector syndrome j f S 1 ( ) (1 1 0) 2 ( ) (1 0 1) 3 ( ) (1 0 0) 4 ( ) (0 1 0) 5 ( ) (0 0 1) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 100

101 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, (5, 2, 3) Code Example Let us transmit the codeword c = ( ) and receive (one bit-error) d = c f = ( ) ( ) = ( ). Let us calculate the syndrome The syndrome will result in: S = d H T = (c f) H T = f H T. S = (c f) H T = ( ) H T ( ) H T = (0 1 0). }{{} 0 Result: error-position: j = 4 Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 101

102 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, (5, 2, 3) Code Example Let us transmit the codeword c = ( ) and receive (two bit-errors) d = c f = ( ) ( ) = ( ). Let us calculate the syndrome S = d H T = (c f) H T = f H T. Now, the syndrome will result in: S = (c f) H T = ( ) H T ( ) H T = (1 0 0). }{{} 0 Result: error-position: j = 3? Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 102

103 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Syndrome Decoding Example (4) (5, 2, 3) Code Syndrome table assuming two bit-errors error position error vector syndrome j k S 1 2 ( ) (0 1 1) 1 3 ( ) (0 1 0) 1 4 ( ) (1 0 0) 1 5 ( ) (1 1 1) 2 3 ( ) (0 0 1) 2 4 ( ) (1 1 1) 2 5 ( ) (1 0 0) 3 4 ( ) (1 1 0) 3 5 ( ) (1 0 1) 4 5 ( ) (0 1 1) Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 103

104 Linear Block Codes Baltic Summer School 2012, Tartu, Estonia, Transmission System with Syndrome Decoding replacements i encoder c channel d = c f f Determination syndrome Determination error-pattern ˆf î demapper c f ˆf d = c f Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 104

105 Interleaving Baltic Summer School 2012, Tartu, Estonia, Interleaving Interleaving is a method of converting error bursts to single independent errors through permutation of bit positions. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 105

106 Interleaving Baltic Summer School 2012, Tartu, Estonia, Error Sequences with and without Memory with Memory without Memory xx-xxx- -xxxxxx-xx xxxx-x- -xx- -xx xxxx xx xxxx- -x- -xx xxx- -xxx xxx- -xx xxxxx- -xx- -xxxxxx-x xxxxxx x x- --x --x- -x xx x x-- -x xx x x x x x x x x--x x- --x x - --x xxxxx--- x x x- - -x x x x-- -x x-x x x xx x x x x- -x x x x x x x x x xx x Burst Error: Erroneous bits are concentrated in a certain part of the received word. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 106

107 Interleaving Baltic Summer School 2012, Tartu, Estonia, Block Interleaver Transmitter Side Transmitter side Interleaver s e read in d t n t. Interleaver read out Array with x rows and n columns Codewords are read into the interleaver by rows so that each row contains a (n,m) codeword Interleaver contents are read out by columns into the modulator for subsequent transmission Codeword symbols in the same codeword are separated by x 1 other symbols Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 107

108 Interleaving Baltic Summer School 2012, Tartu, Estonia, Block Interleaver Receiver Side s Receiver side Deinterleaver r d n t error-burst Deinterleaver read in. read out Deinterleaver is an array identical to the interleaver Elements are read into the deinterleaver from the demodulator by column so that each row of the deinterleaver contains a codeword Deinterleaver output is read into the decoder by rows Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 108

109 Interleaving Baltic Summer School 2012, Tartu, Estonia, Block Interleaver ments transmitter side error-burst receiver side read in.. read out read out read in Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 109

110 Interleaving Baltic Summer School 2012, Tartu, Estonia, Block Coding with Interleaving input channel encoder Interleaver Π channel De-Interleaver Π 1 channel decoder output superchannel A block interleaver designed for an (n, m) code is an array with x rows and n columns. Codeword symbols in the same codeword are separated by (x 1) other symbols. Prof. A. Ahrens, Hochschule Wismar, University of Technology, Business and Design, Germany Channel Coding 110