The Lempel-Ziv (LZ) lossless compression algorithm was developed by Jacob Ziv (AT&T Bell Labs / Technion Israel) and Abraham Lempel (IBM) in 1978;

Transcription

1 Georgia Institute of Technology - Georgia Tech Lorraine ECE 6605 Information Theory Lempel-Ziv Lossless Compresion General comments The Lempel-Ziv (LZ) lossless compression algorithm was developed by Jacob Ziv (AT&T Bell Labs / Technion Israel) and Abraham Lempel (IBM) in 978; The LZ algorithm does not require any knowledge of the source probabilities (universal algorithm). The algorithm is inherently adaptive in that it handles sources with any probability law; It is possible to show that the LZ algorithm achieves the entropy rate of any stationary source, and hence it is an optimum universal code; for the proof, see [, Chapter ] LZ is the basis of virtually all commercially available compression softwares (PK Zip, stuff-it, etc.); There are hundreds of variation of LZ algorithm; the version presented here is the simplest, and is not as good as those used in commercial softwares. Basic idea of LZ algorithm Use simple adaptation rule to construct the table of source outpus to encode. Huffman code probabilities of source known encoding table selected ahead of time fixed-to-variable length code The LZ algorithm Two step procedure LZ code probability law unknown encoding table constructed as encoding progresses variable-to-fixed length code. Given a source output x = (x,..., x n ) (n large), the encoder looks for the longest sourceword w in the table that prefixes x (that is x = (w, x )) and sends it.. Table is updated with a new sourceword (w, x), where x is the source ouput following w. Example Suppose we must encode and decode the following source output that takes values in an alphabet {a, b}. x = abbbbba

2 . Encoding algorithm Step 0. Start with table of two entries: a and b. Codewords and are L bit longs with L fixed (say 0 bits) a = ( ) b = ( ) Step. Begin encoding.. We look for the longest codeword that prefixes x: it is a. We send codeword.. We remove the prefix from x a bbbbba bbbbba. We add to the table since a is the next source output a = ( ) b = ( ) = ( ) Step.. Look for the longest source codeword that prefixes x: is is. We send codeword.. We remove the prefix from x bbbbba bbbbba. We add b to the table since b is the next source output a = ( ) b = ( ) = ( ) b = ( )

3 Step.. Look for the longest source codeword that prefixes x: is is b. We send codeword.. We remove the prefix from x b bbbba bbbba. We add ba to the table since a is the next source output a = ( ) b = ( ) = ( ) b = ( ) Step.. Look for the longest source codeword that prefixes x: is is b. We send codeword.. We remove the prefix from x b b bbba bbba. We add ba to the table since a is the next source output a = ( ) b = ( ) = ( ) b = ( ) ba 6 =...

4 Step 5.. Look for the longest source codeword that prefixes x: is is ba. We send codeword 6.. We remove the prefix from x b b ba abba 6 6. We add b to the table since a is the next source output abba a = ( ) b = ( ) = ( ) b = ( ) ba 6 =... b 7 =... We continue until data symbols in x are exhausted.. Decoding algorithm The decoder must be able to accept the fixed length codewords produced by the encoder and contruct the original sequence of a s and b s. To do so, it must also reconstruct the encoding table on the fly. In the previous example, the decoder receives the sequence of codewords 6. Step 0. Decoder starts with the same table as encoder at Step 0. a = ( ) b = ( ) Step. Decoder receives codeword. Therefore it decodes an a. It must add a? to the list, where? is the next source output. This is what the encoder did, but the decoder does not know yet what? will be. The decision is deferred until next time. x = a a = ( ) b = ( ) a? = ( )

5 Step. Decoder receives codeword, which cannot be decoded yet. But since it received a rather than a, it knows the blank must be filled with an a. Thus it replaces a? by in the table, and adds? to the table, deferring the decision on? until next symbol. x = a a = ( ) b = ( ) = ( )? = ( ) Step. Decoder receives codeword, which is decoded as b. Knowing this, the decoder can decide that? = b. Encoder adds b? to the table and defers the decision until next symbol. x = ab a = ( ) b = ( ) = ( ) b = ( ) b? 5 = (... ) Step. Decoder receives codeword, which is decoded as b. Knowing this, the decoder can decide that b? = ba. Encoder adds b? to the table and defers the decision until next symbol. And so on... References x = abb a = ( ) b = ( ) = ( ) b = ( ) ba 5 = (... ) b? 6 = (... ) [] T. M. Cover and J. A. Thomas, Elements of Information Theory, st ed. Wiley-Interscience, 99. 5