To compress or not to compress? Gabriele Buch, Frank Burkert, Joachim Hagenauer. Department of Communications Engineering,

Size: px

Start display at page:

Download "To compress or not to compress? Gabriele Buch, Frank Burkert, Joachim Hagenauer. Department of Communications Engineering,"

Martina Webster
5 years ago
Views:

1 To compress or not to compress? Gabriele Buch, Frank Burkert, Joachim Hagenauer and Bernhard Kukla Department of Communications Engineering, Technical University of Munich, Arcisstr 1, D-8090 Munich, Germany Abstract For practical communications which transmit nite blocks of source data over noisy channels, we uestion the common practice to compress (C) the source and then to add redundancy for error control Rather we exploit the redundancy of the noncompressed source (NC) at the channel decoder by source-controlled channel decoding For a simple binary Markov source and a Rayleigh fading channel we simulated in a fair comparison the systems (C and NC) using an ARQ/FEC scheme with RCPC codes, Lempel- Ziv compression and a modied Viterbi decoder We indicate parameter regions where it is better not to compress I Introduction In today's digital storage and transmission systems for data, text, speech, audio, fax, image and video the source is compressed as much as possible and the resulting bits are transmitted For noisy channels one has to add redundancy for error correction and detection Supposedly this two step method is supported by Shannon's famous separation theorem [1], which states that source compression and channel coding should be separated and that error free transmission is possible as long as the entropy of the source is less than the capacity of the channel This is true in the information theoretical sense with long source coded blocks and channel coding using a seuence of random block codes with length tending to innity In a practical scenario very often the situation is dierent The popular variable length source coding schemes (VLC) such as Human, Lempel-Ziv and arithmetic coding are very sensitive to channel errors Usually a single error blows up the whole scheme Arti- cial blocking methods, comma and sync seuences have to be employed On bad channels as in mobile communications or on bad telephone lines the reuired redundancy used by FEC or retransmission is high Do we really have a net gain? Is it worth to compress a fax and then the errorfree reuirement forces us to perform numerous ARQ retransmissions? Shouldn't we leave some redundancy in the source data, be tolerant to a few transmission errors and let the sink use it for error detection and concealment? Joachim Hagenauer, Department of Communications Engineering, Technical University of Munich, Arcisstrasse 1, D{8090 Munich, Germany, phone: , fax: , hag@lnte{techniktu{muenchende Recently even more sophisticated methods became available [6] where the channel decoder uses the residual redundancy of the uncompressed or partly compressed source data to improve channel decoding The conjecture of this paper is, that there are situations with bad or medium channels where it is not worth to compress We propose in certain cases to leave the redundancy in the source and to use it at the channel decoder to derive an a priori information for the next source bit This method is called source-controlled channel decoding (SCCDEC or APRI decoding) We are aware that there are still good reasons to compress, because the source is stored or transmitted by error free media such as RAMs or optical bers For bad channels however, we shall describe situations where it is not worth to compress Let us start a fair competition: We will transmit bits from a skewed binary source with an entropy less than one source bit per symbol Since there is no way to rate residual source errors, we nally have to achieve errorfree bits at the sink A type II FEC/ARQ system [3] with RCPC codes as described in [5] achieves this for two schemes: Method C compresses with the lossless universal LZ algorithm and method NC skips compression Both transmit with the same variable rate RCPC codes by sending more and more redundancy until a CRC check conrms errorfree decoding The NC channel decoder however estimates the source state resp the correlation of the source bits and uses it in a modied Viterbi algorithm for better decoding In such a way less redundancy is reuired for FEC Both methods have no knowledge of the source statistics; Method C exploits the source at the transmit side and method NC does the same at the receive side The gure of merit is in both cases the throughput or the overall eective code rate R = number of uncompressed source bits total number of reuired channel bits : Depending on channel conditions we shall give ranges of source parameters where R C < R NC means it is better not to compress II The source We will use a simple skewed binary source model with three

2 The probability of X k is dependent on the actual state of the source 1? 1? r G 1 P (1) = p 0 r Fig 1: The source model P (1) = 0:5 r 1? B G P (1) = 1? p 0 In state B the two source symbols x 1 = 1 and x = 0 are eually probable In state G 1 x 1 occurs with probability P (x 1 ) = p 0 As the probabilities of occurance in state G are complementary to those in state G 1 we can write P (X = x 1 jstate = B) = P (X = x jstate = B) = 0:5; (1) P (X = x 1 jstate = G 1 ) = P (X = x jstate = G ) = p 0 ; () P (X = x jstate = G 1 ) = P (X = x 1 jstate = G ) = 1? p 0 : (3) The state of the source changes at time k according to the probabilities or r respectively (Fig 1) Let the changing probabilities be time invariant, then we have a time invariant Markov chain rst order with the stationary distribution and Note that on the average P (B) = r + P (G 1 ) = P (G ) = The entropy of the source is 1 X k?1 H(X) = H1(X) = lim k!1 k (4) r r + : (5) P(0) = P (1) = 0:5: (6) i=0 p(qs i ) ld 1 p(qs i ) ; (7) where QS i is one of the k possible source seuences with length k If the respective state of the source would be known, the entropy would be the average of the entropies H (i) (X) of the three states E fh (i) (X)g = H 0(p 0 )r + r + H(X) ; (8) where H 0 (p 0 ) denotes the binary entropy function III Source coding The Lempel-Ziv algorithm in the version described in [] is applied for source coding The source seuence is seuentially parsed into strings as short as possible that have not appeared so far Since this is the shortest string, all its pre- xes, especially the string consisting of all but the last bit of this string, must have occurred earlier Thus the phrase is coded by giving the location of the prex and the value of the last bit Instead of describing the position of the prex with a pointer of constant length, we used in our implementation a pointer, where the length is variable and depends only on the number of already encoded phrases This leads to a further improvement of the compression properties The compression rate depends on the source parameter p 0 ; r; as well as on the length n of the seuence to be coded Generally the compression becomes more ecient with decreasing p 0 and and increasing n and r IV APRI-Viterbi algorithm To use the redundancy of the source at the receive side a channel decoder is reuired which accepts besides soft inputs y and channel state information also an a priori information L c = 4 E S N 0 a; (9) P (u = +1) L(u) = log P (u =?1) (10) about the information bits u, where L C is called the reliability value of the channel, a denotes the fading amplitude and E S =N 0 the signal{to{noise ratio for the binary channel symbols Taking into account the a priori L{value, the metric of the VA can be modied in such a way [6], that M (m) j X = M (m) j?1 + n 0 l=1 x (m) j;l L cj;l y j;n + u (m) j L(u j ) (11) denotes the metric of the m{th path with the coded bits x (m) j;l V Transmission system In order to answer our uestion raised in the introduction we compare two communication sytems both exploiting the redundancy of the source System C will compress the soure seuence to minimize the number of bits to transmit, whereas the other system (system NC) will ommit source coding trying to use the redundancy to improve channel

3 transmission seems to us to be a fair deciding factor we applied a so{called type II hybrid ARQ/FEC protocol [3] to both systems Using RCPC codes for the FEC allows transmission of incremental redundancy where needed, so it can be avoided to repeat information and parity bits as it is done in other ARQ schemes In the following we will briey describe how the ARQ/FEC protocol takes advantage of the property of RCPC codes, that each code with rate r i of a family of RCPC codes is part of every code with a rate r j < r i of this family [5] The code with the minimal rate of such a family is called the mother code First the data block of size N is encoded with an error detecting CRC block code, then extended by M tail bits and nally passed to a M state convolutional encoder The so obtained rate 1=n 0 mother code is stored in a matrix as shown in Fig The transmission starts with the highest achievable code rate, ie only those bits remaining after the puncturing according to rule (1) are transmitted The decoder stores the received symbols and starts the rst decoding trial If the CRC code indicates an error the code rate will be lowered by transmitting only additional redundancy obtained through the puncturing of the mother code according to () The code rate will be decreased further and further as long as erroneous decoder decisions are assumed by the CRC If decoding is even not successful for the minimal rate 1=n 0 the transmission restarts with the highest code rate and code combining can be applied as the decoder has stored all received symbols The signal processing in system C is schematically shown in Fig 3 First the n source symbols are passed to the Lempel{Ziv coder which reduces the redundancy of the source The then remaining n c bits are subdivided into blocks of size N (segmentation), where N is dependent on the communication protocol After the error free transmission which is achieved with the above described ARQ/FEC method the decoded bits are reassembled to blocks and - nally decompressed by the source decoder Special control units manage segmentation, reassembly and the ARQ/FEC protocol in the transmitter and the receiver respectively B System NC without compression As system NC aims to use the redundancy of the source for improving decoding to increase the overall eective code rate the signal processing is uite dierent to the one of sytem C (unshaded blocks in Fig 4) Due to the decision depth of the APRI{VA it is necessary to perform a reordering of the source seuence to enable the receiver to estimate the state seuence of the source Ideally the source seuence is interleaved in such a way that the distance between two successive source symbols becomes greater or eual the decision depth As soon as the rst decision is made by the APRI{VA the source state estimator (SSE) is able to give a rst estimate about the state of the source Thus, knowing the statistics of the source it can provide the channel decoder with an a priori L{value (en (10)) about the successing bit The SSE works according to a principle very 1=4 - x k;1 x mother k; xk;3 code x Memory k;4 data bits CRC M l Rate Puncturing Rule =9 (1) =5 (1) () =4 (1) : : : (13) Fig : RCPC coding for the ARQ/FEC scheme (n 0=4)

4 Source Segmenting Lempel-Ziv- ui Convolutional Puncturing Rate Selection Transmitter Feedback for ARQ / FEC Receiver Error Detection Rayleigh P (a) Channel State Estimator CSI: ^a i ai ni Interleaver xi = yi Deinterleaver N 0 E S Sink Lempel-Ziv- Reassembly Viterbi- ^ui Depuncturing Fig 3: Signal processing in system C (parts common to system NC are shaded) Source Interleaver Segmenting ui Convolutional Puncturing Rate Selection Transmitter Feedback for ARQ / FEC Receiver Error Detection Rayleigh P (a) Channel State Estimator CSI: ^ai ai ni Interleaver xi = yi Deinterleaver N 0 E S Sink Deinterleaver Reassembly ^ui APRI- Viterbi- Depuncturing L(u) Source State Estimator Fig 4: Signal processing in system NC (parts common to system C are shaded)

5 of this L{value depends on the estimated state, whereas the magnitude is dependent on the reliability of the estimation given by the state of the SSE This state is determined by taking into account not only the direct but all already decoded predecessors of the source symbol to predict VI Simulation results According to our purpose of evaluating and comparing the performance of system C and system NC the overall eective code rate of the two communication systems has been determined by computer simulations for a fully interleaved Rayleigh fading channel using optimal Channel State Information (CSI) For the simulation the source parameters p 0 and have been varied and r = 0:05 has been set constant to reduce the number of variables According to section II p 0, and r determine the redundancy of the source as follows: A decreasing p 0 and result in a increase of the redundancy The number of source bits which are compressed by the LZ-encoder has been xed to bits which is eual to the size of the interleaver in system NC The remaining data bits after source coding were subdivided into blocks of size N = 1000 For error detection polynom according to CCITT The channel encoder in the ARQ-system which is basedon RCPC Codes, as described in section V, has been a rate 1=4{encoder with memory M = 4 The mother code was punctured according to [5] We always started with a with the code of rate 8/9 The maximal rate of the channel coding system therefore is :871: The results of the computer simulations are shown as 3dplots in Fig 6 and 5 1? 1? r 1? r = 0:05 G 1 B G r = 0:05 P (1) = p 0 P (1) = 0:5 P (1) = 1? p effective code rate R system NC system C do compress Generally it could be seen according to the diagrams that if the source is very redundant, particularly if p 0 < 0:1 and < 0:01, the attainable eective code rate of system C is higher than the rate of system NC In this range the LZ-algorithm is very ecient and, because of the good compression, even eective code rates above one are possible, especially if p 0 < 0:01 and moderate channel noise (not shown in the diagrams) Within this range it is of course worthy to compress particularly if it is considered that system NC cannot achieve rates above the maximal code rate But if the redundancy of the source decreases the LZ-algorithm gets step by step more inecient As a condo not compress log (1/) p0 Fig 6: Eective code rate (throughput) at ES =N 0 = 1 db

6 = 0:01 r = 0:05 G 1 B G r = 0:05 = 0:01 P (1) = p 0 P (1) = 0:5 P (1) = 1? p 0 effective code rate R do compress system C system NC do not compress 0 4 Es / No in db p Fig 5: Eective code rate (throughput) depending on the SNR seuence R C decreases rapidly if the source becomes less and less redundant On the other side the redundancy of the source is still high enough to improve channel decoding in system NC, as we nd out by analyzing R NC depending on the source parameters According to Fig 6 and 5 this method becomes more and more appropriate if p 0 > 0:1 and > 0:01 In this case it is indeed not worthy to compress because R NC becomes higher than R C The points of intersection of the two graphs are mainly dependent on the source parameters which determine the compression of the LZ algorithm and depend only slightly on the channel Signal-to-Noise Ratio (SNR) as shown in Fig 5 Since the code rate is variable we describe the SNR by the average E S =N 0 of the Rayleigh fading channel rather than by E b =N 0 = E S =RN 0 VII Conclusion As we have shown in our fair comparison of the two systems, it can be promising for the transmission of redundant data to skip the compression and to exploit the redundancy in the channel decoder As a basic rule it can be specied that if the LZ algorithm compresses only by a factor of 085 and the channel is noisy, source{controlled channel decoding can sucessfully be applied The decision whether to compress or not to compress depends on the source statistics and on the type of the channel Under same source conditions, one would use the system NC rather for bad channels than for good channels Furthermore we used only a simple Markov source and an ad hoc estimator, which could surely be optimized Thus, the results for a real application like data, text, fax or image transmission may be slightly dierent But all things considered, there are scenarios where it is really worth to exploit the redundancy of the source in the decoder instead of extracting it by source coding References [1] CE Shannon, Collected Papers, edited by NJA Sloane and A Wyner, IEEE press, New York, 1993, p 40 [] ThM Cover and JA Thomas, \Elements of Information Theory," Wiley-Interscience Publication, New York, pp , 1991 [3] Shu Lin et al, \Automatic repeat{reuest error control schemes, IEEE Commun Mag, vol 1, pp 5-17, December 1984 [4] JJ Metzner and D Chang, \Ecient selective repeat ARQ strategies for very noisy and uctuating channels," IEEE Trans Commun, vol COM 33, pp , May 1985 [5] J Hagenauer, \Rate-Compatible Punctured Convolutional Codes (RCPC Codes) And Their Applications," IEEE Trans Commun, vol COM 36, no 4, pp , April 1988 [6] J Hagenauer, \Source-led Channel Decoding" IEEE Trans Commun, vol 43, no 9, pp , September 1995 [7] B Kukla, \Vergleich verschiedener Strategien zur Maximierung der eektiven Coderate bei ARQ-Verfahren und korrelierten Quellensymbolen," Master Thesis, Department of Communications Engineering, TU Munich, January 1996

_ MAPequalizer _ 1: COD-MAPdecoder. : Interleaver. Deinterleaver. L(u)

_ MAPequalizer _ 1: COD-MAPdecoder. : Interleaver. Deinterleaver. L(u) Iterative Equalization and Decoding in Mobile Communications Systems Gerhard Bauch, Houman Khorram and Joachim Hagenauer Department of Communications Engineering (LNT) Technical University of Munich e-mail: