UNEQUAL ERROR PROTECTION FOR DATA PARTITIONED H.264/AVC VIDEO STREAMING WITH RAPTOR AND RANDOM LINEAR CODES FOR DVB-H NETWORKS

Transcription

1 UNEQUAL ERROR PROTECTION FOR DATA PARTITIONED H.264/AVC VIDEO STREAMING WITH RAPTOR AND RANDOM LINEAR CODES FOR DVB-H NETWORKS Sajid Nazir, Vladimir Stankovic, Dejan Vukobratovic Department of Electronic and Electrical Engineering University of Strathclyde, 16 Richmond Street, Glasgow G1 1XQ, Scotland, United Kingdom {sajid.nazir, ABSTRACT Application layer forward error correction is becoming a popular addition to protocols for real-time video delivery over IP-based wireless networks. Since each part of video data is not equally important for video reconstruction, it is beneficial to divide video data based on its importance. Such partitioned data could then be provided with different degree of protection, with the important data having more protection against channel erasures. Data partitioning (DP) is one such low-cost feature in H.264/AVC enabling partitioning of video data based on its importance. In this paper, we propose an Unequal error protection (UEP) scheme to protect the DP H.264/AVC coded video data with Raptor and Random linear codes (RLC). The simulations have been performed using error traces which depict physical-layer Transport Stream (TS) packet losses in DVB- H. The results highlight that for broadcasting applications with varying channel conditions, better results can be obtained with dynamic probability of selection of different importance layers. Index Terms Forward error correction, data partitioning, random linear coding, Raptor codes 1. INTRODUCTION Two key challenges of multimedia broadcast/multicast applications over IP-based wireless networks are high and varying error characteristics of underlying communications channels and huge heterogeneity of users equipment. To combat these challenges effectively, efficient error-resilient and layered video compression and forward error correction (FEC) must continuously adapt to channel characteristics and receivers demands. H.264/AVC [1] is the latest video coding standard achieving significant compression efficiency and gaining widespread use in the emerging standards and applications. H.264/AVC was recently adopted for the mobile broadcasting communication standard Digital Video Broadcasting-Handheld (DVB-H) [2]. DVB-H is primarily intended for broadcast of digital TV signals to mobile devices. The DVB-H standard uses physical layer of DVB- T, and additionally at the link layer specifies Multi Protocol Encapsulation Forward Error Correction (MPE-FEC) [3] for real-time services and the use of Raptor codes [4] at the application layer (AL) for IP datacasting services. Apart from Raptor codes, another packet erasure protection code that has been gaining increased popularity is Random Linear Code (RLC) [5], which shows near-capacity performance regardless of codeword length, but suffers from high decoding complexity as codeword length increases. The focus of this study is to analyse the use of data partitioning (DP) error resilient feature of H.264/AVC tailored with AL-FEC codes for unequal error protection (UEP) in the DVB-H standard. The DP feature of H.264/AVC is chosen because it effectively prioritizes video stream by partitioning it into classes of decreasing importance to video reconstruction with a very small decrease in overall performance. Note that the alternative is to use Scalable Video Coding (SVC), which by itself introduces performance penalty, plus error resilience. For FEC, we use Raptor and RLC codes, which are increasingly being considered for real-time services at the AL [11]. In our simulations we partition video stream into two priority groups, which we term as HPS (high priority segment) and LPS (low priority segment). The error resilience through DP is used in conjunction with AL-FEC with either systematic 3GPP Raptor code or RLC. The encoded data is encapsulated in RTP/UDP/IP packets, which are then placed in TS packets as specified in the DVB-H standard. In our simulations, we use realistic DVB-H channel TS packet drops trace obtained by real-world measurements. Our simulation results show that Raptor and RLC can be used to effectively protect the HPS and LPS for reliable video transmission over packet erasure channels in DVB-H. Furthermore, for video broadcast/multicast applications, it can be advantageous to relate the degree of protection to various DP segments depending on the varying channel conditions. The proposed schemes are general enough to be applied to other video transmission standards /11/$ IEEE

2 Although the loss of a low priority NAL unit enables successful decoding of high priority NAL unit, it could result in error propagation to the subsequent frames. This degrades the video quality beyond the effect of lost NAL units alone. To offset the error propagation, insertion of periodic I (intra) frames could be used but is costly. A relatively low cost solution is to insert periodic MB intra updates (MBIU) which can arrest the error propagation to a certain extent. The error resilience features generally are associated with a decrease in compression efficiency, and overhead. However, as shown in [7], the DP has the least overhead as compared to the other common error resilience techniques in H.264. Hence, it is an attractive low-cost option for splitting the data into various layers with unequal importance Raptor codes and RLC Fig.1. DVB-H Protocol Layers. The rest of the paper is structured as follows. Section 2 briefly covers the necessary background. The experimental setup used for this study is explained in Section 3. Section 4 contains the results and their analysis. Finally Section 5 concludes the paper. 2. BACKGROUND 2.1. H.264/AVC and Data Partitioning H.264/AVC is widely deployed [1] in the latest multimedia applications and this trend is likely to grow. The H.264/AVC design covers a Video Coding Layer (VCL) for coding of video content and Network Abstraction Layer (NAL) which formats the video data enabling it to be transported over various channels. Each frame or slice is encapsulated in a separate NAL unit. H.264/AVC provides many error-resilience features to mitigate the effect of lost packets during transmission. One such scheme available in the extended profile is DP [6] which supports the partitioning of a slice in up to three partitions (NAL units), based on the importance of the encoded video syntax elements for video reconstruction. Partition A contains the most important data comprising slice header, quantization parameters, and motion vectors. Partition B contains the intra-coded macroblocks (MB) residual data, and partition C contains inter-coded MB residual data. It is thereby possible to assign different protection levels to different partitions based on importance. The decoding of DP A is always independent of DP B and C. However, if DP A is lost the remaining partitions cannot be utilized. The decoding of DP B is possible without DP C, but not other way around. To make DP B independent of DP C, Constrained Intra Prediction (CIP) parameter in H.264/AVC encoder must be set. Raptor codes [4] are rateless codes, i.e., they provide a flexibility to generate as many encoded symbols as desired from the source symbols. The Raptor decoder can recover the original source symbols from any set of encoded symbols, as long as their number is at least equal or slightly exceeds the number of source symbols. These have recently been adopted for use in various standards including DVB-H. As an Application layer Forward error correction (AL- FEC) solution in DVB-H, systematic Raptor codes provide improved system reliability and a large degree of freedom in the choice of transmission parameters [8]. For an explanation of the encoding and decoding algorithms and implementation guidelines, [9] is a detailed reference. The encoding process consists of a pre-code followed by reduced-complexity LT coding. The decoding process results in maximum-likelihood decoding performance and it can be considered successful if the received generator matrix at the decoder is invertible. Raptor codes have constant encoding and linear decoding complexity. Another class of rateless codes which has become popular is RLC [5]. RLC applied over a source message produces encoded symbols as random linear combinations of source symbols with coefficients randomly selected from a given finite field. RLC as a packet-level AL-FEC solution are simple to implement and perform as optimal erasure codes for sufficiently large finite field used for creating linear combinations of source symbols (one-byte field GF(256) is usually good [5]). This makes RLC an attractive alternative to Raptor codes as a universal FEC/network coding solution for emerging wireless communication systems, such as LTE-A, WiMAX and DVB-NGH. The major limitation of the application of RLC is the decoding complexity of Gaussian Elimination (GE) decoding, which is polynomial in the number of symbols. However, for short lengths of the source messages, the decoding complexity is acceptable (see [5, 11] and references therein).

3 2.3. DVB-H Mobile multimedia broadcasting is an emerging technology and is expected to grow. DVB-H is the European standard for digital TV signals broadcast to handheld devices and is based on the DVB-Terrestrial (DVB-T) standard which uses MPEG-2 packet streams. DVB-H is IP-based and has two important new features introduced in it. Firstly, the transmission takes place in intermittent bursts of maximum data rate of up to 2Mbps. Secondly, to mitigate errors in mobile and wireless environments the DVB-H standard introduces a link layer FEC mechanism, called MPE-FEC. The H.264/AVC standard is used for encoding the video data. DVB-H protocol layers, which successively add headers and prepare the video data for transmission, are shown in Fig. 1. The H.264/AVC encoder generates NAL units which are then encapsulated into RTP/UDP packets. These packets are then placed into IP packets, which are inserted into MPE sections. The MPE sections are then divided into TS packets for transmission over the DVB-H physical layer. Clearly, the header overhead could be substantial especially for small IP packet sizes. The header overhead and unused bytes in a data packet are detrimental to the performance of FEC scheme. Hence it is important to provide FEC configurations which can be adapted to the lower layers of the protocol structure. 3. EXPERIMENTAL SETUP The video sequence Foreman in CIF format encoded using the H.264/AVC software JM version 16.2 [10] has been used in all simulations. All the simulations have been performed using a group of pictures (GOP) size of 64 frames, one slice per frame, and a frame rate of 25 frames per second (fps). This configuration implies that the average GOP size is 94,400 bytes, corresponding to 2.95 seconds of video data, and thus the source data rate is 295 kbps. This rate ensures that sufficient amount of video data is transmitted during one burst of DVB-H transmission. Similar source parameters have been used by in [11, 12]. In order to enable UEP, in addition to coding of a single-layer H.264/AVC stream, we have coded the Foreman sequence with the DP enabled feature with all the other parameters being the same as in the single-layer case. For the purpose of evaluation, two different DP schemes have been used. In the first scheme, referred to hereafter as SEQCIP, the Constrained Intra Prediction (CIP) feature of the encoder has been set so that the decoding of Partition B could be made independent of Partition C. In the second scheme, named as SEQMBI, in addition to the CIP flag, the MB intra update feature is used to additionally limit the effect of error propagation. The partitions with their relative sizes and PSNR contribution are shown in Table 1 for SEQCIP and Table 2 for SEQMBI. For this study, we have grouped IDR, DP-A Table 1. Relative partition sizes SEQCIP. Partition Size (bytes) Size Cumulative (%) PSNR IDR DP-A DP-B DP-C Total Table 2. Relative partition sizes SEQMBI. Partition Size (bytes) Size Cumulative (%) PSNR IDR DP-A DP-B DP-C Total and DP-B as HPS and remaining data as LPS. As can be seen from the table the size of HPS is about 57% of the total size for SEQCIP and about 72% of the total size for SEQMBI. The selection of a large variation in proportion of HPS adequately tests the proposed scheme. It is expected that the UEP scheme will work well if the size of HPS is smaller relative to LPS. After the H.264/AVC encoded data is segmented into the HPS and LPS, each of the segments is packetized into equal-length source symbols. The length of the source symbols determines the number of source symbols contained in the source message, k, and is an important parameter in our setup. For Raptor codes, ideally, short source symbols are selected resulting in higher values of source message length k, typically in the order of thousands of symbols. In the case of RLC, large source symbols are favoured in order to yield small values of k, typically in the order of hundreds, because of high decoding complexity of RLC. After AL-FEC coding, encoded symbols are grouped to fit the content of a single IP packet. This process also takes into consideration the header overhead. It is important to underline that IP packets are either correctly received or lost in the transmission process. Finally, IP packets are placed into the MPE frames where each IP packet is encapsulated within a single MPE section with its own overhead containing the error-detection field. Note that we do not employ the option of using Reed-Solomon codes. An MPE frame is transmitted within a single transmission burst by mapping MPE frame data onto 188-bytes long physicallayer TS packets. We always set the IP packet size to be an integer multiple of 184 bytes (the content size of TS packet), i.e., more precisely, each MPE section is transmitted over the integer number of TS packets. For simplicity, we assume that MPE sections and TS packets are aligned, i.e., the

4 Table 3. System configurations. FEC SEQCIP (symbols) SEQMBI (symbols) Symbol size Symbols in IP HPS LPS HPS LPS (bytes) packet RTC RTC RLC RLC Fig. 3. PSNR vs Rate for UEP versions of RLC68 at SNR (12dB). Fig. 2. PSNR vs SNR for selected Raptor and RLC codes. borders between MPE sections at the MPE layer are borders between TS packets at the physical layer. The performance of Raptor and RLC has been compared in [11]. We selected two best performing schemes from each code class (Raptor and RLC) reported in [11], and designed the system configuration for evaluation as shown in Table 3. The number within the FEC scheme s name denotes the source message length k, e.g., RTC1111 is a Raptor code of source message length k = 1111 symbols. The next columns show the distribution of HPS and LPS message lengths for each of SEQCIP and SEQMBI. Note that the sizes of HPS and LPS come purely from H.264/AVC video coding. The symbol size for each configuration and the number of symbols to be put into one IP packet are also shown. 4. RESULTS AND ANALYSIS The trace files representing TS packet losses at different signal-to-noise ratio (SNR) for the DVB-H channel have been obtained from [12] where real-world measurements were conducted. Each of the configurations shown in Table 3 is simulated with 1000 runs for each channel SNR value. The simulations are repeated for different data rates to evaluate the UEP performance as compared to single-layer AVC. As in [11, 12], we use performance criteria called ESR5 (20) (Erroneous Second Ratio) which defines a limit of at most 1 sec of erroneous transmission within each 20 sec segment of video transmission. In all the schemes, if the entire GOP is lost, we resort to a simple error concealment method based on using the last frame of the previously decoded GOP to replace all frames of the lost GOP. The results are evaluated based on varying the available data rate and the probability of selection of HPS. Note that the same rateless codes are used for protection of both HPS and LPS, and UEP is achieved by probabilistically selecting at the transmitter for each output symbol whether it should come from the HPS or LPS stream. Thus instead of two different fixed code rates, we use soft code rates via defined selection probability of HPS. If we increase the selection probability of HPS, we improve its robustness for the price of a decrease robustness of LPS. Figure 2 shows PSNR vs. channel SNR for the four selected configurations when transmission data rate is fixed at 383 kbps and with protection probability of HPS of 55%. As can be seen from the figure, the RLC configurations provide slightly better performance as compared to the RTC. The PSNR difference between RLC68 and RLC136 is marginal. Similarly, the difference between RLC136 and RTC136 is negligible. However, RTC1111 provides a poor result overall. Having in mind the limited complexity of GE decoding for RLC68 (due to small code length), it could be a good candidate for real-world implementations. The PSNR at various transmission data rates has been evaluated for the RLC68 scheme and the results are as shown in Fig. 3. We show averaged PSNR vs. overall data rate. As can be seen from the figure, the single-layer AVC has PSNR advantage over the UEP schemes for higher data rates. For low data rates, the performance of UEP schemes is better than the single-layer one. This is because at lower rates the contribution to PSNR by successful decoding of HPS for UEP is significant. The UEP schemes with higher protection for the HPS have higher PSNR at low rates. As the available data rate is increased, the UEP curves get a performance penalty, with UEP schemes with higher protection of HPS being the ones which are severely affected. This is because at such high rates the successful

5 Table 4: Decoding Performance for RLC68 (SEQCIP) for SNR12 at HPS Protection Probability (60%) Data Single Layer Segmented Data (HPS + LPS) Rate % % PSNR(dB) Percentage of PSNR(dB) (kbps) Success Failure Success (HPS+LPS) Success (HPS ) Failure Table 5: PSNR Results for RLC68 (SEQCIP) for SNR12 at Different HPS Protection Probability (%) % Probability of Protection (HPS) Single Layer Rate (kbps) Table 6: PSNR Results for RLC68 (SEQMBI) for SNR12 at Different HPS Protection Probability( %) Rate % Probability of Protection (HPS) Single (kbps) Layer decoding of both HPS and LPS is less probable due to higher selection probability of HPS. From Table 4, it can be seen that the probability of cumulative successful decoding of both HPS and LPS, and only HPS is higher than the successful decoding for singlelayer scheme. This means that although for some data rates the average PSNR value for UEP may be lower than that of the single-layer scheme, the percentage of overall failure is smaller with UEP since often it would provide at least the basic HPS service while the single-layer fails. The PSNR results of SEQCIP for RLC68 are shown in Table 5 for different probabilities of protection for HPS. As can be seen from the table, if the channel conditions are good then it is better to use single-layer coding which does not have performance penalty. However, in broadcasting scenarios, where some of the receivers are likely to suffer bad channel conditions, it makes more sense to use DP partitioning with UEP. The degree of protection being afforded to the HPS could be made dynamic with the changing channel conditions. The grayed boxes in the table show as how the protection could vary to provide best visual experience at the receiver. The column with the border portrays the case where protection probability is same as the HPS data size ratio. In Table 6 we present the results for SEQMBI for RLC68. The results support the analysis of Table 4 and 5. It is interesting to note that the best results for a UEP scheme occur for protection probability of HPS being slightly more than the percentage size ratio of HPS to LPS. Also to be noted is the fact that with a change in the data rate, the protection probability for HPS may as well be adjusted for the optimum results. Such a scheme could be easily incorporated in transmission channels providing periodic feedback. The results for Raptor codes for SEQCIP for SNR = 12 db for RTC136 at 60% probability of protection for HPS are shown in Table 7. In Table 8 the results for same configuration are plotted for different data rates. Again it can be seen that at low data rates the performance for Raptor codes with UEP could be better than the single layer case.

6 Table 7: Decoding Performance for RTC136 (SEQCIP) for SNR12 at HPS Protection Probability (60%) Data Single Layer Segmented Data (HPS + LPS) Rate % % PSNR(dB) Percentage of PSNR(dB) (kbps) Success Failure Success (HPS+LPS) Success (HPS ) Failure Table 8: PSNR Results for RTC136 (SEQCIP) for SNR12 at Different HPS Protection Probability (%) % Probability of Protection (HPS) Single Layer Rate (kbps) CONCLUSIONS AND FUTURE WORK The study has extensively compared the performance of RLC and Raptor codes by assigning different degree of protection to the data partitions of H.264/AVC video under different packet loss scenarios at different data rates. RLC are seen to perform very close to but somewhat better than Raptor codes. Within RLC configurations, it is seen that the scheme with less number of symbols results in better performance. This is promising result for the use of RLC for multimedia broadcast applications, as the decoding complexity for such configurations is manageable. The average PSNR of UEP schemes is not as good as that of single-layer AVC at high data rates. However, it is interesting that at lower data rates UEP schemes provide good results with smaller number of decoding failures. The UEP schemes would be extended to H.264/SVC and comparison with EWF codes [13]. 6. REFERENCES [1] T. Wiegand, G. Sullivan, G. Bjøntegaard, and A. Luthra, Overview of the H.264/AVC video coding standard, IEEE Trans. Cir. Sys. Video Techn., vol. 13, no. 7, pp , Jul [2] ETSI TS , digital video broadcasting (DVB): Specification for the use of video and audio coding in DVB services delivered directly over IP protocols, ETSI Tech. Spec., [3] ETSI TR , digital video broadcast (DVB); IP datacast: Content delivery protocols (cdp) implementation guidelines part 1: IP datacast over DVB, ETSI Tech. Spec., [4] A. Shokrollahi, Raptor codes, IEEE Trans. Inform. Theory, vol. 52, no. 6, pp , [5] M. Wang and B. Li, How practical is network coding?, IEEE IWQoS 2006, Intl. Workshop on Quality of Service, pp [6] ITU-T Recommendation H.264 (03/2010) Advanced video coding for generic audiovisual services. [7] R. Razavi, M. Fleury, M. Altaf, H. Sammak, and M. Ghanbari, H.264 video streaming with data-partitioning and growth codes, IEEE ICIP-2009, Intl. Conf. on Image Proessing, Cairo, Egypt, Oct [8] D. Gomez-Barquero, D. Gozalvez, and N. Cardona, Application layer FEC for mobile TV delivery in IP datacast over DVB-H systems, IEEE Trans. Broadcasting, vol. 55, no. 6, pp , [9] ETSI TS , universal mobile telecommunications system (umts); multimedia broadcast/multicast service (mbms); protocols and codecs, ETSI Tech. Spec., 2005 [10] H.264/AVC Reference Software, [11] S. Nazir, D. Vukobratović, and V. Stanković, Performance evaluation of Raptor and Random Linear Codes for H.264/AVC video transmission over DVB-H networks, in Proc. IEEE ICASSP-2011, Intl. Conf. on Acoustics, Speech, and Signal Processing, Prague, Czech Republic, May [12] K. Nybom, S. Gronroos, and J. Bjorkqvist, Expanding window fountain coded scalable video in broadcasting, in Proc. IEEE ICME-2010, Intl. Conf. on Multimedia and Expo., Singapore, July [13] D. Vukobratović, V. Stanković, D. Sejdinović, L. Stanković, and Z. Xiong, Scalable video multicast using Expanding window fountain codes, IEEE Trans. Multimedia, vol. 11, pp , Oct