CONFIDENCE FEATURES EXTRACTION FOR WYNER-ZIV VIDEO DECODING

Transcription

1 CONFIDENCE FEATURES EXTRACTION FOR WYNER-ZIV VIDEO DECODING Ralph Hänsel, and Eria Müller Institute of Counications Engineering, University of Rostoc Richard-Wagner-Straße, Rostoc, Gerany phone: + (), fax: + (), eail: {ralph.haensel,eria.ueller}@uni-rostoc.de web: ABSTRACT Distributed video coding (DVC) is well suited for low coplexity encoding and error robust transission. The feedbac channel is the ain handicap of state-of-the-art DVC systes. We previously proposed error locating coding (ELC) in conjunction with iage inpainting as solution. This paper is focused on ELC with confidence inforation. It is shown that the proposed confidence feature iproves the ELC perforance and adapts well to the data rate.. INTRODUCTION Video copression is an iportant topic in odern ultiedia counication systes. The conventional video codecs (e.g. MPEG-, H./AVC) are well suited for broadcast video transission. Hence, the encoding is done once with a very high coputational coplexity, whereas every client can decode the video with low coplexity. That behavior is not well suited for video encoding on obile devices, where the encoder is very liited in coputing power and energy. Distributed Video Coding (DVC) gives the ability to design video codecs with low encoder coplexity. The coplex otion estiation part is shifted fro the encoder to the decoder. Thus, the teporal correlation is exploited only at the decoder. The theories of D. Slepian and J. Wolf [] as well as A.D. Wyner and J. Ziv [] proof that distributed coding can reach the perforance of conventional coding. Further applications for DVC are given in []. The ain handicap for state-of-the-art DVC [, ] is the feedbac channel []. It is necessary for rate control, to guaranty high rate distortion (RD) perforance. If no feedbac channel is available, state-of-the-art DVC codecs will have very low RD perforance. Data transitted to the decoder will not increase the decoding quality, until it reaches a critical liit. The Slepian-Wolf (SW) decoder only decodes a valid quantisation sybol beyond the critical rate. SW decoding can be seen as forward error correction (FEC, turbo code). FEC only wors beyond the critical rate. In contrast, Error Locating Coding (ELC) adapts to the data rate. Depending on the rate it is ore or less fine grained or reliable. Thus ELC is woring beneath the critical rate without a andatory feedbac channel. In [] we proposed ELC to ar unreliable regions in a frae, which were filled in by inpainting, obtaining the reconstructed frae. In this paper we will focus on an iproved error locating and confidence feature extraction. An overview of the proposed DVC codec is given in the next section, followed by a review of the related wor in section. The proposed error locating ethods and confidence features are described in section. The corresponding results and conclusion are given in section and. X split K Wyner-Ziv Encoder H. Encoder Y Wyner-Ziv ELC, IIp SI- Gen H. ˆX ux Figure : The proposed DVC architecture. SYSTEM The input sequence is split up into ey fraes K and Wyner- Ziv (WZ) fraes X at the encoder (fig. ). Whereas the ey fraes are encoded by a conventional H. intra encoder, the WZ fraes are encoded by the WZ encoder. The pixel doain WZ encoder follows the DVC principles and has very low coplexity. It quantizes every pixel and does SW encoding (based on systeatic turbo code []). The decoded ey fraes are used for teporal interpolation (BiMESS,[]) at the decoder, gaining the side inforation Y. The side inforation is corrected by the WZ decoder to get the decoded WZ frae ˆX. On the one hand, the decoder can request parity bits fro the encoder until the critical rate for successful decoding is reached. On the other hand, the proposed error locating and inpainting can be applied to generate the reconstructed frae, whereas no feedbac channel is needed. Typically, a DVC codec uses a DCT transfor to iprove the RD perforance. But transfor doain DVC has two drawbacs: at first the encoder coplexity is increased and secondly, the rate control and thus the feedbac channel proble is ore coplex. The rate control has to operate on every band (DCT). Therefore, we propose a pixel doain DVC codec. The ajor iproveents in the proposed syste are the error locating coding (ELC) and the iage inpainting (IIp) (see fig. ). During the decoding process the SW decoder is observed by the error locating odule, which ais to detect error-prone pixels. This inforation is used in an iage inpainting process, which iproves the quality of the side inforation Y and the quality of the reconstructed WZ frae ˆX. In [] we were focused on the ELC and IIp. But pixel are only classified in erroneous and non erroneous pixel. In this paper we propose ELC algoriths which give an additionally confidence inforation. No extra data is transitted for ELC, because it only observes the SW decoding process. ˆK

2 p, p Slepian-Wolf ˆq R ˆX WZ Dec. error locating eap conf. feat. H. inpainting Y SI- Gen ˆK, ˆK + Figure : DVC consisting of SW and the proposed error location with confidence feature extraction. RELATED WORK. Error Locating in Distributed Source Coding Error location in conjunction with distributed source coding (DSC) was proposed in []. Here, the DSC decoder locates the tapering in a wateraring scenario. A two state channel odel (tapered / non tapered ) is applied, where the lielihood of each state is estiated on a bloc basis. The underlying SW coder is ipleented by an LDPC code, which is as powerful as a turbo code.. Stopping criteria for Turbo Codes In [] several stopping criteria for turbo codes are evaluated in conjunction with DVC. The entropy coding in DVC is done by the SW coder, which is ipleented by e.g. LDPC or turbo code. For a high RD perforance of the codec, it is iportant to now, whether SW decoding was successful or not. Based on the evaluated stopping criteria the decoder can decide whether to request ore data fro the encoder or to stop decoding. The odified stopping criteria are the basis for the proposed confidence feature. Therefore, the criteria are reviewed briefly. The sign-change ratio (SCR) criterion [] counts the sign-changes in the extrinsic inforation L e. If there are only soe sign-changes, the decoding is entioned to be error free and is stopped. The hard-decision-aided (HDA) criterion [] counts the sign-changes in the a posteriori inforation L (hard output) between subsequent iterations and also stops decoding if the nuber of sign-changes is low. The sign-difference ratio (SDR) [] does not copare the sign in subsequent iteration, but copare the sign of the a priori L a and extrinsic inforation L e. Also the decoding is stopped if only a few sign-changes occur. Account of the agnitude of the a posteriori inforation L is taen by the ean-estiate (ME) criterion []. If the ean of the agnitude of the a posteriori inforation L is greater than a threshold, the decoding is considered successful. The fifth criterion is used in the DISCOVER codec []. If the nuber of sybols, with the a posteriori inforation ˆK BER forean (a) line (b) bloc (c) rando coastguard Figure : Pixel scan order soccer line bloc rando (a) BER vs. data rate Sequence line bloc rando forean, bpp, bpp, bpp, bps, bps, bps coastguard, bpp, bpp, bpp, bps, bps, bps soccer, bpp, bpp, bpp, bps, bps, bps (b) critical data rate Figure : Bit Error Rate (BER) vs. Data Rate, Wyner Ziv Fraes - bloc, line and rando pixel scan order (coastguard, forean, soccer, QCIF, fps) L is below a threshold, decoding is stopped. Additionally a CRC is applied. In our proposed error location and confidence estiation algorith we use the entioned stopping criteria in a odified for to estiate, whether a bit is correct decoded or not. Therefore, we are aiing to get a confidence inforation for each bit and not only for the decoded whole frae.. PIXEL SCAN ORDER IN DVC In a pixel-doain Wyner-Ziv encoder the D frae needs to be transfored into an one-diensional vector for SW encoding. The pixel are typically scanned line by line (fig. (a)). In [] bloc by bloc ( ) pixel scan order (fig. (b)) was proposed for iproved error locating capabilities. Furtherore, rando pixel scan order (fig. (c)) is also possible (sae perforance as a second TC interleaver). The bit error rate (BER) vs. data rate of the first bit plane is shown in figure (a). One can see, that the BER is not significantly decreased if the data rate is increased. At least when the critical rate (fig. (b)) is reached the BER will drop to zero. Furtherore, if rando pixel scan order is used, successful decoding is possible at the lowest rate. The rando scan of pixel will scatter bloc errors, which produces a good error pattern for turbo code based SW decoding. Therefore, we will apply the rando pixel scan order in this paper. The pixel order is fixed and don t need to be transitted.

3 . PROPOSED ERROR MAP AND CONFIDENCE FEATURE ESTIMATION The stopping criteria reviewed in section give inforation, whether the decoding was successful or not. Successful decoding eans, that the bit plane u of the original frae X is equal to the bit plane û of the decoded frae ˆX. If the reviewed criteria can give inforation whether the whole sequence is decoded successfully, it can also give inforation about successful decoding of each single bit û. The ey idea of the proposed algorith is to generate an estiated error ap eap est in the first step, by coparing the SW decoder output bits with the input bits. In a second, step a odified stopping criterion (confidence feature) is estiated to separate the predicted error ap into reliable and non reliable segents. This two step approach iproves the error locating perforance. The proposed error ap prediction and confidence feature estiation does not rely on a transission overhead, because it only uses inforation available fro the turbo decoder (SW decoder). Furtherore, the WZ encoder is left untouched, preserving the low encoder coplexity.. Error Map Estiation The estiated error ap eap est () is generated by coparing the hard SW decoder output û with the hard decoder input y (b) (eq. ). The hard decoder input is the corresponding bit plane b of the side inforation Y. For perforance evaluation the real error ap eap real () defined in equation is also calculated. As shown in figure (a), the BER is not decreased before the critical rate is reached. So the nuber of errors in the decoder input y (b) (fig. (a),data rate bps) and output û is nearly the sae. But the erroneous bits are in different positions. Therefore, the predicted error ap is generated by coparing the hard decoder input and output. eap est () = û y (b) () eap real () = u y (b) (). Confidence Feature Estiation The estiated confidence feature is used to separate the estiated error ap into true positives (tp, estiated and real error position) and false positive (fp, estiate error but no real error). The confidence feature generation is based on the odified stopping criteria. In this section we present several confidence features, which are evaluated in section. At first, we focus on the sign-change stopping criteria. The SCR, HDA and SDR criteria are used to generate a signchange vector for each iteration criterion (). A axiu nuber of iteration is used (eq. -). SCR () = sgn { HDA () = sgn { SDR () = sgn { a (u ) e (u ) sgn L (i+) e (u ) (u ) sgn L (i+) (u ) sgn e (u ) Where a (u ), e (u ) and (u ) are the a priori, extrinsic and a posteriori inforation of the bit u in the i-th iteration. () () () iteration sign-change vector s() = p() = non sign-change s() = p() = criterion () s() = p() = sign-change Figure : Calculation of the su feature s criterion () and position feature p criterion () based on the sign-change vector () Based on the sign-change vector criterion () two confidence features for each criterion are extracted. On the one hand the su of sign-changes s criterion () (eq. ) is calculated for a bit u over all iterations. On the other hand the iteration of the last sign-change p criterion () (position, eq. ) is extracted (fig. ). Corresponding to the stopping criteria fewer sign-changes or only sign-changes in first iteration indicate lower error probability and thus a reliable error ap. { } s criterion () = i; i, () criterion () = true { } p criterion () = ax i; i criterion () = true () Based on the ME criterion, the e() feature (eq. ) is calculated. It separates the error ap into n = segents, corresponding to the absolute value of the a posteriori inforation after the last iteration L (u ). The greater the absolute value of the a posteriori inforation, the higher the probability that the decoded bit is equal to the original one û = u and thus the estiated error ap is right. e() = {r; L border,r L (u ) L border,r+ } () L border,r = a r j= b j b =. a = ax{ L (u ) } n j= b j We used a saller interval size for sall L-values L (u ) than for bigger ones, because we want to have a ore finegrain confidence feature in unreliable regions and coarsegrain feature in reliable regions. The proposed confidence features are the su signchange features (s HDA (),s SCR (),s SDR ()), the position features (p HDA (),p SCR (),p SDR ()) and the ME feature (e()). The features divide the error ap into (HDA, SCR) or (ME, SDR) segents. The error ap in every segent is ore or less accurate, thus the feature indicates whether the error ap is accurate or not. The significance of each feature is analysed in the siulation results (sec. ).

4 (a), HDA, s HDA () (b), SCR, s SCR () (c), SDR, s SDR ().... (d), HDA, s HDA ().. (e), SCR, s SCR ().... (f), SDR, s SDR () Figure : HDA, SCR and SDR su feature s criterion (), forean, QCIF, fps (colored lines feature values, eq. ). SIMULATION RESULTS The siulation results are based on the QCIF sequences coastguard, forean and soccer at fps and a KWK GOP structure. At first the su feature s criterion () for the odified HDA, SCR and SDR criteria are evaluated. In figure (a-c) the P[eap est = eap real = ] for the forean sequence is shown. The blac line () indicates the true positive rate without applying any feature, whereas the other colored lines depend on the feature value (see legend). For the HDA criterion (fig. (a)) there is a big difference in the depending on the feature value. But the perforance (higher ) is not increased with an higher data rate. The SCR criterion (fig. (b)) shows only sall capability to separate reliable error ap segents and non reliable, because the is nearly independent of the feature. At least, the SDR criterion (fig. (c)) shows good properties, because the su feature can separate reliable and non reliable error ap segents. Furtherore, the perforance of this feature is increased with an increased data rate. On the other hand the P[eap est = eap real = ] (error estiate but no real error, fig. (d-f)) is also iportant. The SDR su feature also shows the best perforance, because the error ap is divided into segents with high difference in the, depending on the feature. Furtherore, the feature values... shows high and low for ediu to high data rates. This is a very good property. For low data rate the SDR su feature and the estiated error ap is not accurate. Therefore, the error ap estiation needs to be iproved before an accurate feature can be selected. In conclusion a low value of the feature indicates a good error ap and thus high confidence. In contrast to the su feature s criterion () the perforance of position feature p criterion () is slightly worse. Fro a coputational coplexity perspective su and position feature are equal, therefore we propose to use the SDR su feature. The ME feature is only dependent on the absolute value of the a posteriori inforation L (u ). A higher feature value corresponds to a higher L-value and thus to a ore reliable decoding and error ap. The true positive, false positive and bit error rate (BER) are shown in figure. The true positive rate depends on the feature value, but for a ediu data rate all the curves intersect. Therefore, there is no advantage of this feature for ediu data rates ( bps, forean). Furtherore, the feature is well suited to detect segents with high BER (fig. ). That is outstanding for the evaluated features, because non of the other features is well suited to separate low and high BER regions. Figure shows the for the sequences coastguard and soccer. The SDR su feature and ME feature shows the sae behavior as for the forean sequence. Thus, the ajor properties of the features are independent of the video sequence. At least, the pixel scan order has also ipact on the features presented here. The rando pixel scan order shows the best perforance of the confidence estiate features. Thus we recoend it for ELC and SW coding. Furtherore, the SW coding perforance do not suffer fro additional ELC coding (no transission overhead, no negative constrains).. CONCLUSIONS An error ap and confidence feature estiation for turbo code based DVC were proposed in this paper. It was shown that the proposed SDR su confidence feature gives the best perforance for true positive detection. Furtherore, the ME feature is best suited for BER classification. In conjunction with iage inpainting the proposed algoriths will help to eliinate the feedbac channel in future DVC systes. Further wor will include the research on a odified iage inpainting algorith, which can handle confidence infor-

5 BER (a) (b) (c) bit error rate Figure : ME feature e(), forean, QCIF, fps (colored lines feature values, eq. ) (a) coastguard, SDR, s SDR () (b) soccer, SDR, s SDR () (c) soccer, ME, e() Figure :, SDR su s SDR () and ME e() feature, QCIF, fps (colored lines feature values, eq. ) ation to iprove the quality of the reconstructed fraes. REFERENCES [] X. Artigas, J. Ascenso, M. Dalai, S. Klop, D. Kubasov, and M. Ouaret. The Discover Codec: Architecture, Techniques and Evaluation. In Proc. Picture Coding Syposiu (PCS), Lisboa, Noveber. [] J. Ascenso, C. Brites, and F. Pereira. Iproving Frae Interpolation with Spatial Motion Soothing for Pixel Doain Distributed Video Coding. In Proc. EURASIP, Slova Republic, July. [] C. Brites, J. Ascenso, and F. Pereira. Feedbac Channel in Pixel Doain Wyner-Ziv Video Coding: Myths and Realities. In Proc. European Signal Processing Conference (EUSIPCO), Florence/Italy, Septeber. [] B. Girod, A. Aaron, S. Rane, and D. Rebollo- Monedero. Distributed Video Coding. Proc. of the IEEE, ():, Jan.. [] R. Hänsel and E. Müller. Error Locating for plausible Wyner-Ziv Video Decoding using Turbo Codes. In Proc. IEEE Int. Worshop on Multiedia Signal Processing (MMSP), Rio de Janeiro, October. [] D. Kubasov, K. Lajnef, and C. Guilleot. A Hybrid Encoder/ Rate Control for a Wyner-Ziv Video Codec with a Feedbac Channel. In Proc. IEEE Multiedia Signal Processing Worshop (MMSP), Chania, Crete, October. [] Y.-C. Lin, D. Varodayan, T. Fin, E. Bellers, and B. Girod. Localization of Tapering in Contrast and Brightness Adjusted Iages Using Distributed Source Coding and Expectation Maxiization. In Proc. IEEE Int. Conf. on Iage Processing (ICIP), San Diego, October. [] F. Pereira, L. Torres, C. Guilleot, T. Ebrahii, R. Leonardi, and S. Klop. Distributed Video Coding: Selecting the Most Proising Application Scenarios. Signal Processing: Iage Counication, :,. [] D. Rowitch and L. Milstein. On the perforance of hybrid fec/arq systes using rate copatible punctured turbo (rcpt) codes. IEEE Transactions on Counications, ():, June. [] R. Y. Shao, S. Lin, and M. P. C. Fossorier. Two Siple Stopping Criteria for Turbo Decoding. IEEE Transactions on Counications, ():, August. [] J. Ŝorupa, J. Slowac, S. Mys, P. Labert, R. V. de Walle, and C. Grecos. Stopping Criterions for Turbo Coding in a Wyner-Ziv Video Codec. In Proc. Picture Coding Syposiu (PCS), Chicago, May. [] D. Slepian and J. Wolf. Noiseless Coding of Correlated Inforation Sources. IEEE Trans. on Inforation Theory, ():, July. [] Y. Wu, B. D. Woerner, and W. J. Ebel. A Siple Stopping Criterion for Turbo Decoding. IEEE Counications Letters, ():, August. [] A. D. Wyner and J. Ziv. The Rate-Distortion Function for Source Coding with Side Inforation at the. IEEE Trans. on Inforation Theory, ():, Jan. [] F. Zhai and I. J. Fair. Techniques for Early Stopping and Error Detection in Turbo Decoding. IEEE Transactions on Counications, ():, October.