Side-Match Vector Quantizers Using Neural Network Based Variance Predictor for Image Coding

Transcription

1 Sde-Match Vector Quantzers Usng Neural Network Based Varance Predctor for Image Codng Shuangteng Zhang Department of Computer Scence Eastern Kentucky Unversty Rchmond, KY 40475, U.S.A. Abstract Sde-match vector quantzer reduces bt-rates n mage codng by usng smaller-szed state codebooks generated from a master codebook through explotng the correlatons of neghborng vectors. Ths paper presents a new neural network based sde-match vector quantzaton method for mage codng. In ths method, based on the varance of a vector whch s predcted by a neural network, a subset of the codewords n the master codebook s selected for the sde-matchng to construct the state codebook for the encodng of the vector. Ths technque generates a lower encodng bt rate wth a hgher reconstructed mage qualty. Expermental results demonstrate that n terms of PSNR (Peak Sgnal-to-Nose Rato) of the reconstructed mages, the proposed method sgnfcantly outperforms the regular sde-match vector quantzer, especally at lower codng bt-rates. Keywords: Vector Quantzaton, Image Codng, Sde Match, Neural Network.. INTRODUCTION Image data compresson s very mportant for varous mage and vdeo processng applcatons whch requre reduced bt-rate/channel bandwdth. Such applcatons nclude dgtal televson, vdeo conferencng, telemedcne, multmeda, and remote sensng mages from satellte and reconnassance arcrafts, as well as the storage systems of multspectral mage data from space programs, medcal mages, fnger prnts, and facal mages. There are many approaches to data compresson. Among them, vector quantzaton (VQ) [-3], whch acheves data compresson through mappng n-dmensonal vectors onto a fnte set of representatve vectors called a codebook, s an mportant and heavly conducted research area. In VQ mage compresson [4], the mages are usually parttoned nto blocks wth each block formng an n-dmensonal vector. Each of these vectors s then coded as an ndex of ts best matchng vector n the codebook. The reconstructed mages are obtaned smply by selectng the correspondng codeword vectors from the codebook usng the ndces. VQ mage compresson takes advantage of the correlaton of the mage pxels wthn a block (vector) for the codng bt-rate reducton and yelds acceptable performance at low bt-rates. To further mprove VQ s performance at low bt-rates, a sde-match vector quantzaton method (SMVQ) [5] has been proposed. Ths method explores not only the redundancy wthn a vector but also the strong correlaton between the neghborng vectors for hgh qualty mage codng at low bt-rates. In practce, SMVQ assumes the contnuty of edges across neghborng mage blocks boundares and generates a smaller-szed state codebook from the master codebook for each block s encodng. The selecton of each state codebook s a subset of codewords n the master codebook whch are the best matches of the upper and left blocks of the block to be encoded. In other words, SMVQ reduces the codng bt-rates by predctng the current block usng ts upper and left boundary blocks. Therefore, SMVQ s performance reles on the accuracy of the predcton. Inaccurate predcton, whch occurs often when the block s wthn an edge area, may result n reduced qualty of reconstructed mages. Internatonal Journal of Image Processng (IJIP),Volume (5) : Issue () : 0 9

2 To ncrease the accuracy of the predcton and therefore further mprove the performance of SMVQ, varous SMVQ schemes have been developed. Chang and Chen [6] proposed varablerate sde-match fnte-state vector quantzaton wth a block classfer. We et al. [7] reported a three-sded sde match technque whch uses not only the upper and left sdes, but one of the bottom and rght sdes for the predcton. Yang and Tseng [8] developed a smooth sde-match classfed vector quantzer whch selects state codebook accordng to the smoothness of the gray levels between neghborng blocks. Chang proposed a gradent match quantzaton method [9] usng gradent match error for the selecton of the codewords n the state codebook. The performance of the gradent sde match vector quantzaton method was further mproved by combnng the non-teratve fractal block codng technque [0,]. Some other technques have also been reported for the enhancement of the orgnal SMVQ method. Examples nclude patternbased sde match VQ [], sde match VQ usng gradent based classfer [3], smooth sde match weghted method [4] and sde match vector quantzers wth varable rates accordng to both the codng qualty and the bt rates [5]. In ths paper, a neural network based sde-match vector quantzaton method for mage codng s presented. In ths method, a neural network s desgned to predct the varances of the vectors, whch are n turn used to select the codewords n the master codebook for the generaton of state codebooks for the encodng of the vectors. Ths technque generates a lower encodng bt rate wth a hgher reconstructed mage qualty. Expermental results demonstrate that n terms of PSNR (Peak Sgnal-to-Nose Rato) of the reconstructed mages, the proposed method sgnfcantly outperforms the regular sde-match vector quantzer at smlar low codng bt-rates.. SMVQ FOR IMAGE CODING As mentoned n prevous secton, SMVQ takes advantage of both the redundancy wthn a block and strong correlaton between the neghborng blocks for hgh qualty mage codng at low btrates. The orgnal SMVQ encodes each mage block by a smaller-szed state codebook generated from a master codebook usng a sde-match selecton functon as shown n Fgure. Assume that the master codebook has N codewords wth each codeword an m x n vector denotng by C, =,,, N. Also assume that the mage to be encoded s parttoned nto blocks of sze m x n. SMVQ encodes the mage blocks n an order from left to rght and top to bottom. For each block beng encoded, SMVQ uses the sde nformaton of ts upper and left neghborng blocks to produce the state codebook. The block s encoded as the ndex of the codeword n the state codebook whch s the best match to the block. Image to be encoded Current block Quantzer: Nearest neghbor search Code (ndex) block block Sde Match State Codebook Master Codebook FIGURE : Block dagram of SMVQ encoder Internatonal Journal of Image Processng (IJIP),Volume (5) : Issue () : 0 0

3 Let x(, j) (=,,, m and j=,,, n) be the pxel n the current mage block X, and u(, j) (=,,, m and j=,,, n) and l(, j) (=,,, m and j=,,, n) be ts upper block U and left block L, respectvely. The state codebook s generated and the block s encoded accordng the followng steps: () For each of the codewords C n the master codebook, calculate the sde-match error e, e = n k = ( u( m, k ) c (, k)) + m k = ( l( k, n) c ( k,)) () Select M (M N) codewords n the master codebook wth the smallest sde-match errors as the state codebook codewords S, =,,, M, (3) For each of the codewords S n the state codebook, calculate ts dstorton measure d (S, X), d ( S, X ) = m n k = r= ( s ( k, r) x( k, r)) (4) The mage block X s encoded as the ndex j of the codeword S j whch produces the smallest dstorton d j (S j, X). For the decodng of each mage block, SMVQ frst generates the state codebook whch was used to encode t accordng to step () and () n the encodng process. Once the state codebook s generated, the reconstructed block s obtaned by smply selectng the correspondng codeword n the state codebook usng the ndex whch s the code of the block generated n the encodng process. The block dagram of SMVQ decoder s shown n Fgure. () () Reconstructed Image Reconstructed block Table Lookup Code (ndex) block block Sde Match State Codebook Master Codebook FIGURE : Block dagram of SMVQ decoder 3. PROPOSED NEURAL NETWORK BASED SMVQ 3. Image Block Varance for SMVQ SMVQ mage codng acheves low bt-rates through usng smaller-szed state codebooks generated from master codebook for the codng of the mage blocks. The qualty of the reconstructed mages coded at low bt-rates reles on the accuracy of the predcton of the blocks. An accurate predcton of a block may keep the codeword, whch s the best match of the block when a full search n the master codebook s conducted, n the state codebook generated by the sde match functon. The more accurate predcton makes t possble to select a smaller-szed state codebook for the codng so that the mage can be coded at a lower bt-rate wth a qualty Internatonal Journal of Image Processng (IJIP),Volume (5) : Issue () : 0

4 smlar to the one obtaned when the master codebook s used. However, the predcton may be naccurate, especally when the block s wthn an edge area. An naccurate predcton on the other hand may keep the codeword whch s the best match of the block n a full search out of the generated state codebook when a small sze s chosen, and therefore results n a degraded reconstructed mage qualty. Snce edge and non-edge blocks may have dfferent varances wth an edge block hghly lkely to have a hgher varance, the best matched codeword for each block n a full search may scatter around the codeword n the master codebook whose varance s equal to or close to the varance of the block to be encoded, assume that the master codebook s sorted accordng to the varances of the codewords. Fgure 3 shows the dstrbuton of the best matched codewords n full search for the encodng of mage Lena usng a codebook of sze 56 whch s sorted by varance of the codewords. The x-axes s the dfference value of the ndex of codeword whose varance s equal or close to the varance of the block beng encoded and the ndex of ts best matched codeword n a full search, and the y-axes s the number of mages blocks assocated wth the same ndex dfference value Number of Image Blocks Poston of the Best Matched Codewords n Full Search FIGURE 3: Dstrbuton of the best matched codewords n full search From Fgure 3, t can be seen that more than 90% of the best matched codewords are dstrbuted n the range from 6 codewords before to 54 codewords after the codeword whose varance s equal to or close to the varance of the block to be coded. In other words, f the 60 codewords around the codeword whose varance matches the varance of the block beng coded are used to be sde-matched to generate the state codebook, then there s a much hgher possblty that the best matched codeword wll be n the state codebook even f the sze of the state codebook s selected to be smaller than that selected n the regular SMVQ. Ths wll result n a faster processng speed snce a subset of codewords n the master codebook nstead of the whole s used for sde matchng. Furthermore, the codng bt rate can be further lowered due to the fact that smaller sze of state codebooks can be used whle preservng smlar reconstructed mage qualty. Based on the above observaton, n our proposed SMVQ method, the varances of the mage blocks, whch are usually senstve to mage edges, are combned nto the sde matchng and state codebook generatng process for more accurate predcton and therefore lowered codng btrate wth preserved mage qualty. Internatonal Journal of Image Processng (IJIP),Volume (5) : Issue () : 0

5 3. Neural Network Based Varance Predctor In the proposed method, the varance of the block beng coded s used to select a subset of the codewords n the master codebook for the sde matchng and state codebook generaton. To mplement the process, the varance of the block must be known. In the proposed algorthm, the varance s predcted by a three-layered feed-forward neural network [6] as shown n Fgure 4. W V H H... H q... R... I I... I p FIGURE 4: Neural network structure of the varance predctor Ths neural network contans an nput layer wth p external nputs, a hdden layer wth q neurons and an output layer wth one neuron. The external nputs of the network consst of the mean value k of the pxels n the last two rows of the upper block and last two columns of the left block of the current block and the dfference values z (=,,, p-) between the mean k and each pxel value n those rows and columns, denoted as I=(I, I,, I p )=(k, z, z,, z p- ), where p = *(m+n)+, k s calculated as, k = ( *( m + n) m n = m j= u(, j) + m n = j= n l(, j)) The output of the network s the predcted varance value of the current block, whch s formulated as, (3) V = f ( q j= w j f ( p = r j I )) (4) where V s the output of the network, I s the -th element of the nput vector I, r j s the weght of the connecton between the -th neuron n the nput layer and the j-th neuron n the hdden layer, w j s the connecton weght between the j-th neuron n the hdden layer and the neuron n the output layer, and f(.) s the actvaton functon of the neurons, whch s f ( x) = + e λ x where λ > 0 s the neuron actvaton functon coeffcent determnng the steepness of the functon. The network s traned usng back-propagaton learnng algorthm [6] wth sample mages. After traned, t can be used to predct the varance of each block beng encoded. 3.3 The Encodng and Decodng Algorthms Gven the nformaton presented n the prevous sectons, the proposed neural network based SMVQ encoder and decoder algorthms are descrbed n ths secton. The encoder conssts of four elements ncludng neural network varance predctor, canddate codeword selector, sde (5) Internatonal Journal of Image Processng (IJIP),Volume (5) : Issue () : 0 3

6 match functon, and quantzer as shown n Fgure 5. The encoder algorthm can be mplemented as follows: () Sort the master codebook by the varances of the codewords n an ascendng order, () Use the neural network varance predctor to predct the varance of the current block, (3) Select T (predefned) (T < N) codewords n the master codebook around the codeword whose varance s equal to or s closest to the predcted varance value, (4) For each of the codewords C selected n step (3), calculate the sde-match error e usng equaton (), (5) Select M (M T) codewords wth the smallest sde-match errors from the T codewords selected n step (3) as the state codebook codewords S, =,,, M, (6) For each of the codewords S n the state codebook, calculate ts dstorton measure d (S, X) usng equaton (), (7) If the smallest dstorton d (S, X) s greater than a predefned threshold, the block s coded usng full search. Otherwse, t s encoded as the ndex j of the codeword S j whch produces the smallest dstorton d j (S j, X), (8) Repeat step () (7) for next block encodng. Image to be encoded Current block Quantzer: Nearest neghbor search Code (ndex) block block Sde Match State Codebook Neural Network Varance Predctor Predcted varance of the Current block Master Codebook Canddate Codeword Selector FIGURE 5: Block dagram of the proposed neural network based SMVQ encoder The structure of the decoder s same as that of the encoder except that the quantzer element n the encoder s replaced by the table lookup element. The block dagram of the decoder s shown as n Fgure 6. For the decodng of each mage block, the coder frst generates the state codebook whch was used to encode t accordng to step () and (5) n the encodng process. Once the state codebook s generated, the reconstructed block s obtaned by smply selectng the correspondng codeword n the state codebook usng the ndex whch s the code of the block generated n the encodng process. Internatonal Journal of Image Processng (IJIP),Volume (5) : Issue () : 0 4

7 Reconstructed Image Reconstructed block Table Lookup Code (ndex) block block Sde Match State Codebook Neural Network Varance Predctor Predcted varance of the current block Master Codebook Canddate Codeword Selector FIGURE 6: Block dagram of the proposed neural network based SMVQ decoder 4. EXPERIMENTS AND DISCUSSION In ths secton, the performance of the proposed algorthm s evaluated by experments. The peak sgnal-to-nose rato (PSNR) s used as a quanttatve measure of the qualty of the reconstructed mages. PSNR can be calculated as, z PSNR = 0* log 0 B B * B B = j= 55 ( z(, j) z(, j)) where (, j) and z(,j) are the (,j)th pxels n the reconstructed and orgnal mages, respectvely, and the mages are of sze B x B. In our experments, the master codebook of sze 56 was generated usng LBG algorthm [7] and the neural network based varance predctor was traned usng 8-bt gray level mage Lena of sze 5 x 5. The traned neural network predctor and codng algorthm were then appled to code the mage Lena as well as some other gray mages of dfferent objects at varous bt rates. For comparson purpose, the correspondng mages were also coded by usng the regular SMVQ method. Table shows the PSNRs of the reconstructed mages coded around bt rate 0.4bpp (bt per pxel), 0.3bpp and 0.37bpp by usng the regular SMVQ and the proposed method. Ths table ndcates that the proposed algorthm mproves the PSNR of the reconstructed mage over the regular SMVQ average by.66db,.06db and 0.4dB when bt rate s around 0.4bpp, 0.3bpp, and 0.37bpp, respectvely. These results demonstrate that n terms of PSNR, the proposed algorthm sgnfcantly outperforms the regular SMVQ method, especally when the codng bt rate s low. (6) Internatonal Journal of Image Processng (IJIP),Volume (5) : Issue () : 0 5

8 Lena Peppers Elane Arplane Flowers Kds Bt rate: around 0.4bpp SMVQ Proposed Method Bt rate: around 0.3bpp SMVQ Proposed Method Bt rate: around 0.37bpp Proposed SMVQ Method Average TABLE : PSNRs (db) comparson (Codng Bt Rate around 0.4bpp, 0.3bpp and 0.37bpp) Fgure 7 shows the curves of PSNRs as a functon of codng bt-rate for the proposed and regular SMVQ methods. From the fgure, t can be observed that from low to hgh bt-rates, the PSNRs of the mages coded by usng the proposed method approach much faster than the regular SMVQ method to a constant value, whch s the hghest PSNR value SMVQ methods can reach and can be obtaned when the mages are coded usng the master codebook. Ths also confrms that the proposed method has a more accurate predcton of the block beng coded and therefore allows to use smaller state codebooks whch generate lower codng bt-rates wth hgher mages qualty. FIGURE 7: PSNR as a functon of codng bt-rate The vsual qualty of the reconstructed mages s also examned. Fgure 8 shows the orgnal and correspondng Lena mages coded by master codebook at 0.5bpp, the proposed method at 0.30bpp and the regular SMVQ method at 0.3bpp. Magnfed face portons of the orgnal and the reconstructed Lena mages are shown n Fgure 9 for further comparson. These fgures show that the mages coded by the proposed method are at the qualty smlar to those coded by the master codebook and they are sharper and vsually look much more pleasant than the ones Internatonal Journal of Image Processng (IJIP),Volume (5) : Issue () : 0 6

9 coded by the regular SMVQ method at the smlar codng bt-rate. Ths s consstent wth the PSNR results shown n Table I. (a) (b) (c) (d) FIGURE 8: (a) Orgnal Lena mage; (b) Lena mage coded by usng the master codebook at 0.5bpp; (c) Lena mage coded by usng the proposed method at 0.3bpp; (d) Lena mage coded by usng the regular SMVQ at 0.3bpp Internatonal Journal of Image Processng (IJIP),Volume (5) : Issue () : 0 7

10 (a) (b) (c) (d) FIGURE 9: Magnfed face porton from (a) Orgnal Lena mage; (b) Lena mage coded by usng the master codebook at 0.5bpp; (c) Lena mage coded by usng the proposed method at 0.3bpp; (d) Lena mage coded by usng the regular SMVQ at 0.3bpp 5. CONCLUSION We have presented a new sde-match vector quantzaton method for mage codng usng a neural network-based varance predctor. In ths method, a neural network s used to predct the varances of the mage blocks. The predcted varances are n turn used to select a subset of the codewords n the master codebook for the sde matchng to generate the state codebooks for the encodng of the mage blocks. Wth the nvolvement of the mage block varances, the regular SMVQ s enhanced wth more accurate predcton of the current block by usng sde nformaton and therefore generates smaller state codebooks whch encode the mages at a lower bt rate wth hgher mage qualty. Expermental results demonstrate that n terms of PSNR (Peak Sgnalto-Nose Rato) of the reconstructed mages, the proposed method sgnfcantly outperforms the regular sde-match vector quantzer at smlar low codng bt-rates. 6. REFERENCES [] R. M. Gray, Vector quantzaton, IEEE ASSP Magazne, pp. 4-9, 984. [] A. Gersho, R. M. Gray, Vector Quantzaton and Sgnal, Compresson, Kluwer Academc Publshers, 99. [3] N. M. Nasrabad, R. A. Kng, Image codng usng vector quantzaton: a revew, IEEE Tran. Communcatons, vol. 36, no. 8, pp , 988. [4] K. Sayood, Introducton to data compresson, Morgan Kaufmann Publshers, San Francsco, CA 996. Internatonal Journal of Image Processng (IJIP),Volume (5) : Issue () : 0 8

11 [5] T. Km, Sde match and overlap match vector quantzers for mages, IEEE Trans. Image Process., vol., no., pp , 99. [6] R. F. Chang and W. -T. Chen, Image codng usng varable-rate sde-match fnte-state vector quantzaton, IEEE Tran. Image Processng, vol., no., pp , 993. [7] H. We, P. Tsa and J. Wang, Three-sded sde match vector quantzaton, IEEE Trans. Crcuts and Systems for Vdeo Technology, vol. 0, no., pp. 5 58, 000. [8] S. B. Yang and L. Y. Tseng, Smooth sde-match classfed vector quantzer wth varable block sze, IEEE Tran. Image Processng, vol. 0, no. 5, pp , 00. [9] H. T. Chang, Gradent match vector quantzers for mages, Opt. Eng., vol. 39, no. 8, pp , 000. [0] H. T. Chang, Gradent match and sde match fractal vector quantzers for mages, IEEE Trans. Image Process., vol., no., pp. -9, 00. [] H. T. Chang and C. J. Kuo, Iteraton-free fractal mage codng based on effcent doman pool desgn, IEEE Tran. Image Processng, vol. 9, pp , 000. [] C. C. Chang, F. C. Shue, T. S. Chen, Pattern-based sde match vector quantzaton for mage compresson, Imagng Scence Journal, vol. 48, no., pp , 000. [3] Z. M. LU, B. Yang, S. H. SUN, Image compresson algorthms based on sde-match vector quantzer wth gradent-based classfers, IEICE TRAN. Informaton and Systems, vol. E85- D, no.9, pp , 00. [4] S. B. Yang, Smooth sde-match weghted vector quantser wth varable block sze for mage codng, IEE Proc. Vs. Image Sgnal Processng, vol. 5, no. 6, pp , 005. [5] S. B. Yang, New varable-rate fnte state vector quantzer for mage codng, Opt. Eng., vol. 44, no. 6, 06700, 005. [6] M. H. Hassoun, Fundamentals of Artfcal Neural Network, MIT Press, Cambrdge, MA, 995. [7] Y. Lnde, A. Buzo and R. M. Gray, An algorthm for vector quantzaton desgn, IEEE Trans. Communcatons, vol. 8, pp , 980. Internatonal Journal of Image Processng (IJIP),Volume (5) : Issue () : 0 9