G-CAST: Gradient Based Image SoftCast for Perception- Friendly Wireless Visual Communication

Transcription

1 G-CAST: Gradent Based Image for Percepton- Frendly Wreless Vsual Communcaton Journal: Transactons on Image Processng Manuscrpt ID TIP-- Manuscrpt Type: Regular Paper Date Submtted by the Author: -Oct- Complete Lst of Authors: Xong, Ruqn; Pekng Unversty, School of EE & CS Wu, Feng; Unversty of Scence and Technology of Chna, College of Informaton Scence; Gao, Wen; Insttute of Dgtal Meda, Pekng Unversty EDICS:. COM-MMC Image and Vdeo Multmeda Communcatons < Image & Vdeo Communcatons,. SMR-Rep Image and Vdeo Representaton < Image & Vdeo Sensng, Modelng, and Representaton

2 Page of SUBMITTED TO IEEE TRANS. ON IMAGE PROCESSING G-CAST: Gradent Based Image for Percepton-Frendly Wreless Vsual Communcaton Ruqn Xong, Feng Wu, Fellow, IEEE, and Wen Gao, Fellow, IEEE Abstract For vsual sgnal communcaton n wreless broadcast scenaros, there s recently a resurgence of nterest n uncoded transmsson (e.g. ). Instead of usng conventonal complcated encodng technques to mprove compresson rato, uncoded transmsson schemes utlze only a few smple operatons such as lnear transform and sgnal scalng to pursue power-optmzed transmsson. They provde nherent capablty to handle channel varaton and can acheve graceful qualty transton n wde channel SNR range. One problem s that they are mostly desgned wth the objectve beng to maxmze the fdelty of receved mages measured by mean square errors (MSE), whch s known for not well reflectng the vsual qualty perceved by human eyes. Inspred by the recent advancements n mage qualty assessment that structural smlarty (SSIM), partcularly the gradent smlarty, reveals the perceptual fdelty of mages more relably, ths paper proposes a new framework that conveys the vsual nformaton of an mage by transmttng ts mage gradents n a pseudo-analog way. The mage s recovered at decoder sde from the receved nosy gradent data, usng a statstcal gradent pror model to pursue better qualty. Expermental results show that the proposed scheme can acheve vsual communcaton wth much better perceptual qualty. The advantage n perceptual qualty s verfed by the mprovement n SSIM and gradent sgnal-to-nose rato (GSNR). Index Terms Vsual communcaton, wreless broadcast, uncoded transmsson, pseudo-analog transmsson, perceptual mage qualty, mage gradent, mage reconstructon I. INTRODUCTION The vsual communcaton systems nowadays are mostly bult upon the Shannon s separaton theorem [], []. A typcal framework of ths knd s llustrated by Fg.. Input mages are frst compressed nto a bt stream of much smaller sze, usng varous source codng technques [] [] to reduce the redundancy n sgnal. The coded bts are protected by channel codes and then modulated and transmtted over channels. The recever frst extract the data packets and correct possble bt errors by demodulaton and channel decodng. It then reconstruct the mages from the compressed bt stream by nvertng the operatons appled n the source codng procedure. Manuscrpt receved Oct,. Ths work was supported n part by the Natonal Basc Research Program of Chna under Grant CB, the Natonal Natural Scence Foundaton of Chna (, and ), the Bejng Natural Scence Foundaton (), and also by the Cooperatve Medanet Innovaton Center. R. Xong and W. Gao are wth the Natonal Engneerng Laboratory for Vdeo Technology (NELVT), and the Insttute of Dgtal Meda, School of Electronc Engneerng and Computer Scence, Pekng Unversty, Bejng, Chna (emal: {rqxong, wgao}@pku.edu.cn). F. Wu s wth the Unversty of Scence and Technology of Chna, Hefe, Chna (e-mal: fengwu@ustc.edu.cn). Prelmnary results of the work were reported n the IEEE Data Compresson Conference (DCC ). Such framework works perfectly for the deal case of pontto-pont vsual communcaton over stable channel wth known capacty. However, t faces bg challenges n heterogeneous or dynamc communcaton scenaros. Frstly, such framework lacks of flexblty n handlng channel varaton. It requres the channel capacty to be known n advance by the encoder. It cannot serve multple users wth satsfacton smultaneously n heterogeneous scenaros, e.g. wreless broadcast, where dfferent users may have dfferent channel capactes. Secondly, the framework s senstve to bt errors. When the channel qualty drops below a threshold, bt errors not correctable by channel codes may appear. In that case, the qualty of decoded mages wll drop sgnfcantly as a result of extreme source compresson, because the coded stream s extremely vulnerable to errors. In the worst cases, the decodng procedure may even crash. Such phenomenon s known as clff effect. Wth the prevalence of moble devces n recent years, the weakness of ths framework becomes more evdent. To address the ssues, varous technques have been proposed n lterature, ncludng jont source channel codng [] [], unequal error protecton [] [], hybrd FEC (forward error correcton) and retransmsson [] [], and scalable mage or vdeo codng [] []. However, these schemes usually ether make the system desgn complcated or ncur a nonneglgble overhead. Recently there s a resurgence of nterest n uncoded (or pseudo-analog) vsual communcaton. The poneerng work n ths drecton s [] []. As llustrated n Fg., such framework skps the conventonal codng operatons such as quantzaton and entropy codng. Instead, t utlzes only lnear decorrelaton transform and sgnal scalng to mplement power-dstorton optmzed transmsson. Compared wth the conventonal approaches, such transmsson has two dstnctve features. Frstly, t uses contnuous numbers nstead of dscrete bts to represent vsual sgnals. Secondly, t pursues approxmate nstead of exact delvery of nformaton varables va analog-lke modulaton and transmsson. The transmsson s always lossy n nature, wth ts nose level commensurate wth the channel condton. Thus t provdes an elegant way to deal wth transmsson errors and channel varaton. Sgnfcant works have been done followng the work of. Fan et. al. [] propose a dstrbuted wreless vdeo broadcast scheme utlzng the dea of coset codng wth pseudo-analog transmsson. Lu et. al. [] propose a parallel soft vdeo delvery scheme explotng the subchannel dversty of MIMO-OFDM channel. Wu et. al. [] propose a lnebased pseudo-analog transmsson for broadcastng satellte mages. Fan et. al. [] and Cu et. al. [] propose to

3 Page of SUBMITTED TO IEEE TRANS. ON IMAGE PROCESSING Sender Transform Quantzaton & Entropy Codng { } Channel Codng Modulaton Fg.. The coded transmsson framework for vsual communcaton. Sender Transform Scalng Whtenng Modulaton Fg.. The uncoded transmsson framework for vsual communcaton. transmsson of / bts { } Demodulaton Channel Decodng {.,-.,.,.,.,...} {.,-.,.,.,.,...} lossy transmsson of real-number coeffcents use wavelet-based moton-compensated temporal transform to better explot the redundancy among successve frames for soft vdeo broadcast. Yu et. al. [] propose a hybrd scheme usng dgtal channel to send a conventonal coded baselayer and usng pseudo-analog channel to send an uncoded enhancement layer. Xong et. al. [], [] study the problems of ncorporatng wavelet decomposton and base-layer btrate selecton n hybrd dgtal-analog schemes. Yun et. al. [] propose a layered and ntegrated soft codng scheme to handle MIMO antenna heterogenety. Fan et. al. [] propose a layered soft vdeo transmsson scheme to handle bandwdth heterogenety. Although very smple, uncoded transmsson has demonstrated very promsng performance [] [], [], [], [], []. Uncoded transmsson s not unprocessed transmsson. In fact, sgnal decorrelaton and power optmzaton are key procedures that determne ts effcency. In [], Xong et. al. present a theoretcal analyss for the decorrelaton transform gan n uncoded transmsson. The analyss reveals that a decorrelaton transform can brng sgnfcant performance gan by boostng the energy dversty n sgnal representaton, and ths gan can be up to more than db for natural mages. In [], Xong et. al. further show that achevng the full potental of decorrelaton transform requres the sgnal energy dstrbuton to be well modeled and properly exploted by both the sender and recever. Although sgnfcant advancements have been acheved n ths feld, there are stll a few ssues to be addressed. The frst ssue s that, based on the analyss presented n [], the overall dstorton of receved sgnal s determned by the sum square-root energy of all frequency components of the mage. In other words, the transmsson of the low-frequency components of mage s not hghly effcent. These components consume a major part of the total transmsson power but they stll consttute the major source of dstorton n the ultmate reconstructon mage. The second ssue s that the exstng uncoded transmsson schemes are mostly desgned wth objectve beng to maxmze the fdelty of receved mages measured by mean square errors (MSE). MSE s wdely used as the fdelty measure for the desgn and optmzaton of vsual communcaton systems. For example, to predct a pxel block from already-reconstructed pxels, the predcton mode s usually selected n such a way that mnmal square predcton error s acheved. Smlarly, De- Modulaton De- Whtenng Dequantzaton & Entropy Decodng LLSE Estmaton Recever Inverse Transform Recever Inverse Transform rate-dstorton optmzaton (RDO) s usually performed usng MSE as the dstorton metrc. In addton, the usng of orthogonal transform (e.g. DCT) n mage and vdeo codng schemes may be nterpreted as an example of usng MSE metrc mplctly, because orthogonal transform keeps the MSE dstorton unchanged so that a good approxmaton n transform doman s guaranteed to be a good approxmaton n sgnal doman, n terms of MSE. MSE may be a good dstorton metrc ndeed for some sgnal processng tasks. However, t has also been long crtczed for not reflectng the perceptual vsual qualty perceved by human eyes, especally when dealng wth perceptually mportant sgnals [], []. Recently, a great deal of effort has been put nto the development of mage qualty assessment (IQA) methods. The famous work of Wang et. al. [] tells us that the structural smlarty (SSIM) ndex s more correlated wth the perceptual mage qualty than the wdely used MSE metrc. The more recent works [] and [] suggest that mage gradents convey mportant vsual nformaton and hence gradent smlarty s even more relevant to perceptual mage qualty assessment. Based on these advancements, some perceptual mage/vdeo codng schemes were proposed n lterature (e.g. [], []). However, most of the exstng perceptual codng schemes focus on codng parameter selecton or codng mode optmzaton, and do not change the codng framework and codng tools. Inspred by the gradent based IQA metrcs, ths paper proposes a new mage communcaton approach, whch conveys the vsual nformaton n an mage by transmttng ts mage gradent. In partcular, we consder the wreless broadcast scenaro and transmt the mage gradent n a pseudo-analog way usng -lke scheme. At the recever sde, the scheme reconstructs the mage from the receved nosy gradent data, usng statstcal pror knowledge about the gradent data and the mage tself to pursue a better reconstructon. To assst the reconstructon, we also send a base-layer descrpton of the mage va a small number of ts low-frequency coeffcents. The proposed scheme has the same advantages as that t provdes graceful qualty transton for very wde channel SNR range, and t can serve multple recevers wth dfferent channel qualtes smultaneously va a sngle sgnal transmsson. More mportantly, we expect the proposed scheme to acheve better perceptual qualty, snce the scheme s desgned wth the objectve beng to reproduce (as accurately

4 Page of SUBMITTED TO IEEE TRANS. ON IMAGE PROCESSING as possble) the mage gradent, whch s beleved to be hghly relevant to the vsual structure perceved by human eyes. The paper s organzed as follows. Secton II revews the uncoded transmsson procedure and ts power-dstorton effcency. Secton III explans the motvatons and the basc dea of gradent-based mage transmsson. Secton IV descrbes the proposed scheme wth detals of the sender and recever. Secton V presents a soluton for gradent-based mage reconstructon at recever. Expermental results are reported n Secton VI to verfy the analyss and evaluate the proposed schemes. Fnally Secton VII concludes the paper. II. REVIEW OF SOFTCAST TRANSMISSION A. The Transmsson Scheme We brefly summarze the processng n [] [] to provde a background for our dscusson. adopts a framework as llustrated by Fg.. At the sender, nput mage s frst processed by a lnear decorrelaton transform such as DCT, producng a stream of transform coeffcents. In the subsequent stage, nstead of beng processed by quantzaton and entropy codng, the coeffcents are scaled ndvdually, each scaled by a separate factor, and then further processed by a Walsh-Hadamard Transform (WHT) to whten the whole stream and reduce ts Peak-to-Average Rato (PAR). The coeffcents after WHT are transmtted as follows. The underlyng communcaton module n only strves to make the contnuous numbers get through a channel wth mnmum dstorton. Ths s mplemented va dense or even contnuous modulaton. A par of coeffcent numbers are mapped to a pont n the constellaton, usng the two numbers as the I- and the Q- components drectly. The constellaton ponts are grouped and sent out n OFDM packets, usng nverse Fourer transform to convert data nto the waveforms to transmt over the ar [] []. The recever extracts the constellaton ponts n OFDM packet from the arrved waveforms. It then recovers the coeffcent numbers usng some optmal lnear estmator and reconstructs the mages va nverse transform. B. The Power Allocaton Strategy We consder a general uncoded transmsson process. Suppose x = (x,x,...,x N ) R N s a random sgnal to transmt. Each x may represent a sngle pxel or coeffcent, dependng on the pre-processng before transmsson. The sender scales each x ndvdually and sends out y = g x drectly usng dense constellaton. After demodulaton, the sgnal that arrves at the recever s ŷ = y +n, where n s channel nose, typcally assumed to be whte Gaussan. The decoder nverses the scalng operaton and gets an estmaton of x by ˆx = ŷ /g. In ths transmsson procedure, the expected dstorton of x s D = E[(ˆx x ) ] = E[n ]/g = σ n/g, where σ n s the power of channel nose. The transmsson power for We gnore the WHT transform n Fg. snce t does not change the power-dstorton performance. See [] for a theoretcal proof. sendng x s P = E[y ] = g E[x ]. Combnng the two formulatons, we have D P = σn E[x ] or D (P ) = σ n P E[x ]. () The Eq. () holds rrespectve of the g value we choose. The scalng factor g controls both the transmsson power P and the dstorton D. To acheve optmal performance, the transmsson power should be allocated among {x } by mnmzng D subject to P P total. Ths can be solved by settng D / P to be constant for all element x. Ths leads to { P = Kσ n E[x ] D = K σ () n E[x ] Substtutng the Eq. () back nto P = g E[x ], we can see that the strategy for choosng {g } can be formulated as g λ /, () where λ s a shorthand notaton for E[x ]. C. The Power-Dstorton Effcency The constant K n () s determned subject to P = P total so that K = P total /(σ n E[x ]). Therefore, the overall dstorton under optmal power allocaton s ( D total = D = σ n P total E[x ] ) () Ths ndcates the power-dstorton performance of uncoded transmsson. We can see that the overall dstorton s determned by two factors. One s the condton of communcaton channel, measured by channel SNR,.e. P/σ n = P total /(Nσ n). The other one s the complexty of source sgnal, measured by the sum square-root energy of all elements {x },.e. H(x) E[x ]. Manfestly, the uncoded transmsson would become more challengng f a random sgnal has hgher energy (.e. hgher varance or uncertanty) n ts elements. Another observaton, whch s probably not so manfest but surely more mportant, s that the transmsson would become more effcent f the energy of source sgnal can be concentrated to a small fracton of the elements. The mathematcal reason for ths s that H(x) s the sum of an upper convex functon (.e. f(x) = x) appled on each of the sgnal elements. If λ = λ s fxed, f(λ ) acheves ts maxmum value only when all λ are equal. The value of f(λ ) would become smaller f the λ values become more dversfed. Ths observaton also sheds lght on the benefts of applyng decorrelaton transform n uncoded transmsson. Please refer to [] for more analyss and dscussons. III. MOTIVATIONS AND BASIC IDEAS Now we explan the motvatons and ntroduce the basc deas of ths work. Accordng to the analyss and observatons presented n Secton II-C, an effcent uncoded mage transmsson scheme would decorrelate the mage sgnal frst before

5 Page of SUBMITTED TO IEEE TRANS. ON IMAGE PROCESSING λ... Lena ( ) x Index of Sorted Coeffcents P Lena ( ) x... Index of Sorted Coeffcents D Lena ( ) x.... Index of Sorted Coeffcents (a) (b) (c) Fg.. The power and dstorton behavor of uncoded transmsson () for Lena (, CSNR= db, σ n = ). (a) The energy λ = E[x ] of each transform coeffcent (sorted n the descendng order). (b) The consumed transmsson power of each coeffcent. (c) The reconstructon dstorton of each coeffcent. sendng t out usng the pseudo-analog transmsson procedure (.e. scalng, WHT whtenng and dense modulaton). In [] [], dscrete cosne transform (DCT) s employed for ths purpose, and n that case the random vector x = (x,x,...,x N ) n Secton II-B actually corresponds to the DCT coeffcents. The current optmzaton strategy, as presented n Secton II-B, s developed wth the objectve beng to mnmze the total mean square error. As a result, the transmsson of low frequency mage components would become the part that consumes most transmsson power and meanwhle ncurs most reconstructon dstorton. To see ths, recall the power and dstorton behavor formulated n Eq. (). As s well known, for natural mages, the energy of low frequency components s usually hgher than that of hgh frequency components for several orders of magntude. We sort the transform coeffcents n descendng order of energy and llustrate the trend of rapd decay n Fg. (a). Accordng to Eq. (), the transmsson power consumed by a coeffcent wll be proportonal to the square-root energy of that coeffcent (.e. P λ ), as llustrated by Fg. (b). Fg. (a) shows the accumulatve transmsson power and we can observe that the % lowest frequency coeffcents consume about % of the overall transmsson power. On the other hand, however, these low frequency components stll produces a major part (about %) of the overall dstorton (recall D λ ), as shown n Fg. (c) and Fg. (b). In other words, the transmsson of low frequency components s not so effcent, and we expect to mprove the effcency of ths part. The recent advancements n the research of mage qualty assessment tell us that the structure smlarty, partcularly the gradent smlarty, s much more correlated wth the perceptual vsual qualty than the MSE metrc. Therefore, we conjecture that the vsual nformaton contaned n an mage can be well represented by ts gradent feld. Based on ths conjecture, we come up wth the dea of communcatng an mage by transmttng ts gradent feld, whch ndcates the local ntensty changes n the horzontal and the vertcal drectons, respectvely. Indeed, the orgnal mage can be perfectly recovered from the gradent feld, f the global lumnance (.e. the DC component) of the mage s gven. The advantage of usng gradent feld for mage commu- Accumulatve Transmsson Power (%) Lena ( )... Index of Sorted Coeffcents x Accumulatve Dstorton (%) Lena ( )... Index of Sorted Coeffcents x (a) (b) Fg.. The percentage of accumulatve transmsson power and accumulatve reconstructon dstorton. The % lowest-frequency coeffcents consume about % of the overall transmsson power, but produces about % of the overall dstorton. ncaton can also be nterpreted from the aspect of power dstorton effcency. As formulated by Eq. () n Secton II-C, gven the channel condton, the performance of uncoded transmsson s determned by the complexty of source sgnal measured by H(x) E[x ]. By transformng the sgnal to transmt from mage doman to ts gradent doman, the sgnal complexty can be sgnfcantly reduced. We llustrate ths by Fg. and Fg.. Fg. llustrates the orgnal mage Lena and ts horzontal and vertcal gradent. Fg. (a) shows the DCT spectrum of orgnal Lena. Fg. (c) shows the DCT spectrum of the gradent feld of Lena. Comparng Fg. (a) and Fg. (c), we can see that the energy n low frequency components has been attenuated sgnfcantly. The H(x) has actually been reduced from. to.. The reason for such reducton s that the gradent operator s essentally a hghpass operator. Fg. (b) llustrates the frequency response of the gradent operator. We can see that t can retan the hgh frequency components whle attenuatng the low frequency components. The ampltude of frequency response s close to one for the hghest frequences, and s close to zero for the lowest frequences. IV. GRADIENT BASED IMAGE SOFTCAST (G-CAST) Ths secton descrbe the proposed gradent based mage scheme, wth detals of the sender and recever.

6 Page of SUBMITTED TO IEEE TRANS. ON IMAGE PROCESSING (a) (b) (c) Fg.. An example of mage gradent feld. From left to rght: (a) orgnal mage (Lena ), (b) horzontal gradent, and (c) vertcal gradent. The DCT Spectrum of Image Frequency Response of Gradent Operator The DCT Spectrum of Gradent Feld (a) (b) (c) Fg.. The spectrum attenuaton characterstc of the mage gradent operator. From left to rght: (a) the spectrum of orgnal mage (H(x) =. ), (b) frequency response of gradent operator, and (c) the spectrum of mage gradent feld (H(x) =. ). The colors n (a) and (c) llustrate the spectrum ampltude n a logarthm scale. A. The Sender The proposed scheme transmts an mage over a nosy wreless channel usng one base-layer and one gradent enhancement layer. The purpose of the base layer s to delver the DC and a small part of the low-frequency components of the mage so that the recever can fgure out the global lumnance of the whole mage and the local lumnance of each regon. The base layer also provdes a coarse reconstructon of the mage whch can be refned by the gradent nformaton n the enhancement layer. The base layer wll be coded nto a bnary representaton of very low bt rate and sent out usng dgtal transmsson technques wth FEC protecton so that the recever can get the base layer wth very hgh probablty, even when the channel SNR s very low. The purpose of the enhancement layer s to delver the gradent nformaton of the mage so that a vewer can observe the vsual detals. As stated before, the scheme strves to reproduce the mage gradents nstead of the pxel ntenstes, as accurately as possble at the recever sde. The enhancement layer wll be represented by a stream of real numbers and sent out usng the pseudo-analog transmsson procedure, whch can acheve graceful qualty transton when the channel qualty fluctuates n a wde SNR range. The detal of the proposed scheme s llustrated n Fg.. In the base layer, the whole nput mage s frst transformed nto frequency doman usng dscrete cosne transform (DCT). Then a small set, say M M, of low-frequency coeffcents are retaned whle all the other coeffcents are dscarded. The retaned coeffcents are then quantzed and encoded nto a btstream usng entropy codng technques. An alternatve codng scheme for base layer, f DCT s used to produce the coeffcents, s to convert the retaned M M low-frequency coeffcents back to a small mage (of sze M M) usng an nverse DCT of sze M M (see [], []), and then encode the small mage to a hgh qualty level usng a state-of-the-art mage coder (such as HEVC Intra). The coded btstream s then sent to the OFDM module for transmsson, usng FEC codes for error protecton and quadrature ampltude modulaton (QAM) (such as BPSK, QPSK, etc.) for modulaton. In the enhancement layer, the mage gradent s frst extracted from the nput mage usng a gradent transform (GT). Then the gradent mage (as shown n Fg. (b) and Fg. (c)) s transformed nto DCT doman, wth each coeffcent scaled ndvdually for power optmzaton, and further processed by by Walsh-Hadamard transform (WHT) to reduce the peak-tomean rato (PMR) of the gradent data stream. The WHTtransformed gradent data s drectly modulated to a dense

7 Page of SUBMITTED TO IEEE TRANS. ON IMAGE PROCESSING Fg.. The Sender. OFDM Channel QAM - Demodulaton Fg.. The Recever. Base-layer DCT Transform Gradent-layer Gradent Transform k-qam - Demodulaton Channel Decodng Lowpass Selecton Power Allocaton... Quantzaton & Entropy Codng Walsh-Hadamard Transform Entropy Decodng... bt stream real numbers..., -.,.,.,... constellaton (e.g. usng k-qam) for raw OFDM transmsson, n the same way as done n [], [], []. For each OFDM sub-carrer, a par of numbers s extracted from the WHT-transformed gradent data stream and mapped to a pont n the constellaton, usng the two numbers as the I- and the Q- components respectvely, whch ultmately controls the ampltude and phase of the sub-carrer. B. The Recever Sgnals transmtted n the ar wll be nfluenced by the nterferences from other transcevers. The recever recovers the mage from the nosy OFDM sgnal va a base layer decoder and an enhancement layer decoder, as llustrated n Fg.. The base layer decoder reconstructs the DC and lowfrequency coeffcents of the mage (or equvalently, the correspondng low-resoluton mage). To do ths, t frst recovers the base layer btstream va demodulaton and FEC decodng. Then t performs entropy decodng and dequantzaton. As long as the transmsson of base layer s successful, the decoded base layer should be error free and be dentcal wth the base layer at the sender. The enhancement layer decoder frst retreves the gradent feld from the arrved nosy OFDM sgnal va demodulaton and nverse WHT transform. Obvously, the obtaned gradent feld s nosy, wth ts nose level commensurate wth the channel condton. The decoder wll create a fnal estmaton of the mage va a gradent based reconstructon (GBR) procedure, utlzng both the nosy gradent nformaton at the enhancement layer and the clean low-frequency coeffcents provded by the base layer. The GBR procedure wll be descrbed wth detals n the next secton. V. GRADIENT BASED RECONSTRUCTION (GBR) A. The Overall Reconstructon Procedure Suppose u s the lexcographcally stacked representaton of the orgnal mage, g h = D h u and g v = D v u are the bt stream real numbers..., -.,.,.,... Dequantzaton Walsh-Hadamard Inverse Transform Channel Codng QAM Modulaton k-qam Dense Modulaton Gradent-Based Reconstructon OFDM Channel horzontal and the vertcal gradent of u, respectvely. The matrces D h and D v represent the gradent operators n the horzontal and the vertcal drectons, respectvely. Suppose m = Tu denotes the low-frequency coeffcents of u, where T s the M M rows of DCT transform matrx, correspondng to the retaned M M low-frequency coeffcents. The sender encodes m n the base layer and transmts g h and g v n the enhancement layer. The recever gets m, g h and g v due to the quantzaton effect and the exstence of channel noses. For the convenence of dscusson, we wrte D = [D h D v ], g = [g h g v ] and g = [ g h g v ] so that g = Du. For any vector v (or matrx V ), we use v (or V ) to represent ts th element (or row). The GBR procedure recovers u from g h, g v and m. To reduce the nfluence of channel nose, ths paper consders the statstcal feature that gradent data usually conforms to zero-mean Laplacan dstrbutons [] []. The optmzaton objectve can be formulated as: µ mn u D u + µ + D v u g v ( D h u g h ) + µ Tu m () wth µ = σ, µ = σ, µ n = Q. Here σ and σ n are the varance of gradent data and the channel nose, respectvely, Q s the quantzaton step for codng m. Ths above optmzaton problem s complcated and dffcult to solve drectly. But t can be solved wth the assstance of varable splttng and augmented Lagrangan methods [] []. Usng the auxlary varable g = Du, () can be reformulated as mn u,g µ g + µ g g + µ Tu m s.t. g = Du ()

8 Page of SUBMITTED TO IEEE TRANS. ON IMAGE PROCESSING The augmented Lagrangan functon for () s J(u,g) = µ g + µ g g + µ Tu m + ( ) β D u g vt (D u g ) () Here the Lagrangan varable v has the same dmenson wth g. The problem () can be solved by an teratve algorthm, mnmzng () wth respect to u and g and then updatng v by v v β(d u g ) n each teraton. The mnmzaton of () can be easly handled by solvng the followng two subproblems. B. The g Sub-Problem Wth u fxed, the problem () s reduced to ( µ mn g g + µ g g +β D u g v ) β () whch has a closed-form mnmzer ( µ g +β(d u v β g = Shrnk ) ) µ, () µ +β (µ +β) Here the shrnkage operaton s defned by ( ) v Shrnk(v,s) = max v s, v. () C. The u Sub-Problem Wth g fxed, the problem () s reduced to µ mn u Tu m + β D u g v β () The second term n () s exactly (β/) Du g v/β. By wrtng t = g v/β and reorganzng t (for all ) nto t h and t v, () can be reformulated as µ mn u Tu m + β ( ) D h u t h + D vu t v () Ths s a smple least square problem. Snce both the operators D h and D v are convolutons, they can be effcently calculated n the frequency doman. To be specfc, f we use perodc extenson at sgnal boundary, the convoluton can be calculated by pontwse multplcaton n the DFT doman. Also, we note that T s essentally data maskng n the frequency transform, the problem () can be solved n frequency doman by Eq. (). Here T s the forward DFT transform and T ( ) s the nverse DFT transform. T ( ) s conjugate of the forward transform. If we use DCT (nstead of DFT) as the transform n base layer and employ symmetrc extenson for the convoluton D h and D v, the problem can be smlarly solved n the DCT doman [], wth mnor changes to (). VI. EXPERIMENTAL RESULTS A. Overall Performance of To evaluate the performance of our proposed scheme, we compare t wth two anchor schemes. One anchor s the drect transmsson (DrectTx) scheme, n whch the pxels are drectly sent out usng analog transmsson, wthout any transform and power allocaton. Ths s smlar to the tradtonal analog-tv. Another anchor s the baselne scheme n [], [], whch employs DCT transform and power allocaton. For our scheme, we set M = (.e., the base layer ncludes DCT coeffcents of the lowest frequences). To make the comparson far, the data n the DrectTx and scheme s transmtted twce (and averaged at the recever sde) so that they use the same amount of bandwdth and consume the same amount of total transmsson power as does. The three schemes are tested under varous channel SNR condtons, rangng from -db to db. For smplcty, n ths paper the OFDM transmsson s smulated by an Addtve Whte Gaussan Nose (AWGN) channel. Fg. shows the PSNR results of the reconstructed mages. The proposed scheme turns out to be better than DrectTx, but nferor to n many cases, n terms of PSNR values. Ths s not surprsng because our scheme s not optmzed w.r.t. MSE. In fact, we are more nterested n the perceptual qualty or structural smlarty. Fg. shows the SSIM results of the reconstructed mages. Clearly, outperforms both and DrectTx for all CSNR condtons. The advantage of over s partcularly sgnfcant at low CSNR condtons. To gan deeper nsght, we also evaluate the schemes by gradent SNR (GSNR). In other words, we transform both the orgnal mage and the reconstructed mage nto gradent doman and measure the sgnal fdelty by the sgnal-to-nose rato n the gradent doman. Fg. shows the GSNR results of the reconstructed mages. We can see that outperforms both and DrectTx n GSNR values, for all CSNR condtons. That means, under the same channel condton, the scheme preserves the gradent nformaton more accurately than both and DrectTx. Perceptual qualty s the ultmate target of our desgn. Fg. and Fg. show the reconstructed mages of the three tested schemes, under the channel condton CSNR=dB. We see that the output of DrectTx contan whte noses whle the output of contan low-frequency noses. The output of are much clear, wth most of mage detals reserved and most noses removed. Experments on more test mages gve smlar observatons. Based on these results, we argue that the proposed gradent-based mage scheme s desrable for percepton orented vsual communcaton, partcularly for wreless broadcast scenaros. B. The Beneft of Utlzng Gradent Pror A key pont of the GBR algorthm proposed n Secton V s the utlzaton of Laplacan pror model to assst the recovery of gradent feld. Now we evaluate the performance mprovement brought by ths pror. In Fg., we compare the reconstructon mages produced by the scheme wthout usng gradent pror (.e. by settng the µ n Eq. () to ) and

9 Page of SUBMITTED TO IEEE TRANS. ON IMAGE PROCESSING PSNR of Reconstructon (db) PSNR of Reconstructon (db) Lena ( ) DrectTx Salboats ( ) DrectTx Fg.. The PSNR results of reconstructed mages. ( βt u = T ( ) ( ) (D h ) T (t h )+βt ( ) (D v ) T (t v )+µ T ( ) ) (T) T ( m) βt ( ) (D h ) T (D h )+βt ( ) (D v ) T (D v )+µ T ( ) (T) T (T) PSNR of Reconstructon (db) Peppers ( ) DrectTx Parrot ( ) by the scheme wth gradent pror (.e. µ = σ ). Ths comparson s conducted at dfferent channel qualtes, e.g. CSNR=dB, db and db. It can be easly observed from Fg. that, f the gradent pror knowledge s not taken nto consderaton, the mage reconstructed drectly from the nosy gradent feld wll be qute nosy and annoyng. By ncorporatng the pror dstrbuton of gradent feld, such nose has been reduced sgnfcantly. Note that the mages n Fg. have a dmenson of. Please enlarge the fgures to observe detals. The beneft of usng ths pror s partcularly remarkable when the channel qualty s low. PSNR of Reconstructon (db) VII. CONCLUSION AND DISCUSSIONS The power dstorton characterstc of uncoded transmsson ndcates that the low frequency mage components consttute the major part that consumes the most transmsson power and meanwhle produces the most reconstructon dstorton. The recent advancements n mage qualty assessment ndcate that the perceptual fdelty of mages can be measured more relably by the gradent smlarty rather than the wdely used conventonal dstorton metrc MSE. Ths suggests that the perceptble vsual nformaton of an mage can be descrbed by the mage gradent more effcently than the ntenstes of pxels themselves. Inspred by ths observaton, we proposed a gradent-based mage transmsson scheme whch communcates an mage by sendng ts gradent feld wth ts achevable mnmum dstorton. Expermental results show that the proposed scheme provdes promsng perceptual qualty, and t s partcularly sutable for wreless broadcast scenaros. In order to reconstruct the orgnal mage from the nosy gradent data, ths paper utlzed a gradent-sparsty pror model to DrectTx PSNR of Reconstructon (db) PSNR of Reconstructon (db) Cap ( ) () DrectTx Salboats ( ) DrectTx reduce the nfluence of channel noses. In future works, more advanced mage pror models may be ncorporated to further mprove the effcency of the proposed scheme. REFERENCES [] C. E. Shannon, A mathematcal theory of communcaton, Bell Syst. Tech. J., vol., pp.,, July. [] S. Vembu, S. Verdu, and Y. Stenberg, The source-channel separaton theorem revsted, IEEE Transactons on Informaton Theory, vol., no., pp., Jan.. [] W. P. Pennebaker and J. L. Mtchell, JPEG: stll mage compresson standard, New York: Van Nostrand Renhold,. [] D. Taubman and M. Marcelln, JPEG : mage compresson fundamentals, standards, and practce, Norwell, MA: Kluwer,. [] T. Wegand, G. Sullvan, G. Bjontegaard, and A. Luthra, Overvew of the H./AVC vdeo codng standard, IEEE Transactons on Crcuts and Systems for Vdeo Technology, vol., no., pp., July. [] G. Sullvan, J. Ohm, W.-J. Han, and T. Wegand, Overvew of the hgh effcency vdeo codng (HEVC) standard, IEEE Transactons on Crcuts and Systems for Vdeo Technology, vol., no., pp., Dec. [] P. Sherwood and K. Zeger, Progressve mage codng for nosy channels, IEEE Sgnal Processng Letters, vol., no., pp., July. [] V. Chande and N. Farvardn, Progressve transmsson of mages over memoryless nosy channels, IEEE Journal on Selected Areas n Communcatons, vol., no., pp., June. [] H. Yousef zadeh, H. Jafarkhan, and F. Etemad, Transmsson of progressve mages over nosy channels: An end-to-end statstcal optmzaton framework, IEEE Journal of Selected Topcs n Sgnal Processng, vol., no., pp., Apr.. [] A. Mohr, E. Rskn, and R. Ladner, Unequal loss protecton: graceful degradaton of mage qualty over packet erasure channels through forward error correcton, IEEE Journal on Selected Areas n Communcatons, vol., no., pp., June.

10 Page of SUBMITTED TO IEEE TRANS. ON IMAGE PROCESSING SSIM of Reconstructon (db) SSIM of Reconstructon (db) Lena ( ). DrectTx. Salboats ( ) DrectTx. Fg.. The SSIM results of reconstructed mages. GSNR of Reconstructon (db) GSNR of Reconstructon (db) Lena ( ) DrectTx Salboats ( ) DrectTx Fg.. The GSNR results of reconstructed mages. SSIM of Reconstructon (db) SSIM of Reconstructon (db) Peppers ( ).. DrectTx. Parrot ( ). [] V. Stankovc, R. Hamzaou, Y. Charf, and Z. Xong, Real-tme unequal error protecton algorthms for progressve mage transmsson, IEEE Journal on Selected Areas n Communcatons, vol., no., pp., Dec.. [] R. Hamzaou, V. Stankovc, and Z. Xong, Optmzed error protecton of scalable mage bt streams [advances n jont source-channel codng for mages], IEEE Sgnal Processng Magazne, vol., no., pp., Nov.. [] R. Pur, K. Ramchandran, and A. Ortega, Jont source channel codng wth hybrd FEC/ARQ for buffer constraned vdeo transmsson, n IEEE Workshop on Multmeda Sgnal Processng, Dec, pp.. [] P. A. Chou, A. Mohr, A. Wang, and S. Mehrotra, FEC and pseudo- ARQ for recever-drven layered multcast of audo and vdeo, n IEEE Data Compresson Conference (DCC),, pp.. [] T. Stockhammer, H. Jenkac, and C. Wess, Feedback and error protecton strateges for wreless progressve vdeo transmsson, IEEE GSNR of Reconstructon (db) GSNR of Reconstructon (db) DrectTx. Peppers ( ) DrectTx Parrot ( ) DrectTx SSIM of Reconstructon (db) SSIM of Reconstructon (db) GSNR of Reconstructon (db) GSNR of Reconstructon (db) Cap ( ).. DrectTx. Salboats ( ) DrectTx. Cap ( ) DrectTx Salboats ( ) DrectTx Transactons on Crcuts and Systems for Vdeo Technology, vol., no., pp., Jun. [] D. Taubman and J. The, Optmal erasure protecton for scalably compressed vdeo streams wth lmted retransmsson, IEEE Transactons on Image Processng, vol., no., pp., Aug.. [] R. Xong, D. Taubman, and V. Svaraman, Optmal PET protecton for streamng scalably compressed vdeo streams wth lmted retransmsson based on ncomplete feedback, IEEE Transactons on Image Processng, vol., no., pp., Sep.. [] R. Xong, D. S. Taubman, and V. Svaraman, PET protecton optmzaton for streamng scalable vdeos wth multple transmssons, IEEE Transactons on Image Processng, vol., no., pp., Nov. [] D. Taubman, Hgh performance scalable mage compresson wth EBCOT, IEEE Transactons on Image Processng, vol., no., pp., July. [] W. L, Overvew of fne granularty scalablty n MPEG- vdeo stan-

11 Page of SUBMITTED TO IEEE TRANS. ON IMAGE PROCESSING dard, IEEE Transactons on Crcuts and Systems for Vdeo Technology, vol., no., pp., Mar.. [] J.-R. Ohm, Three-dmensonal subband codng wth moton compensaton, IEEE Transactons on Image Processng, vol., no., pp., Sep.. [] S.-J. Cho and J. Woods, Moton-compensated -D subband codng of vdeo, IEEE Transactons on Image Processng, vol., no., pp., Feb.. [] R. Xong, J. Xu, F. Wu, and S. L, Barbell-lftng based -D wavelet codng scheme, IEEE Transactons on Crcuts and Systems for Vdeo Technology, vol., no., pp., Sep.. [] H. Schwarz, D. Marpe, and T. Wegand, Overvew of the scalable vdeo codng extenson of the H./AVC standard, IEEE Transactons on Crcuts and Systems for Vdeo Technology, vol., no., pp., Sep.. [] S. Jakubczak, H. Rahul, and D. Katab, : One vdeo to serve all wreless recevers, n MIT Techncal Report, MIT-CSAIL-TR--,. [], One-sze-fts-all wreless vdeo, n Proc. Egth ACM SIGCOMM HotNets Workshop, New York, NY, USA,. [] S. Jakubczak and D. Katab, A cross-layer desgn for scalable moble vdeo, n Proc. nternatonal conference on Moble computng and networkng (MobCom ), New York, NY, USA,, pp.. [] X. Fan, F. Wu, D. Zhao, and O. C. Au, Dstrbuted wreless vsual communcaton wth power dstorton optmzaton, IEEE Transactons on Crcuts and Systems for Vdeo Technology, vol., no., pp.,. [] X. L. Lu, W. Hu, C. Luo, Q. Pu, F. Wu, and Y. Zhang, ParCast+: Parallel vdeo uncast n MIMO-OFDM wlans, IEEE Transactons on Multmeda, vol., no., pp.,. [] F. Wu, X. Peng, and J. Xu, LneCast: Lne-based dstrbuted codng and transmsson for broadcastng satellte mages, IEEE Transactons on Image Processng, vol., no., pp.,. [] X. Fan, R. Xong, F. Wu, and D. Zhao, WaveCast: Wavelet based wreless vdeo broadcast usng lossy transmsson, n IEEE Vsual Communcatons and Image Processng (VCIP),, pp.. [] H. Cu, R. Xong, C. Luo, Z. Song, and F. Wu, Denosng and resource allocaton n uncoded vdeo transmsson, IEEE Journal of Selected Topcs n Sgnal Processng, vol., no., pp.,. [] L. Yu, H. L, and W. L, Wreless scalable vdeo codng usng a hybrd dgtal-analog scheme, IEEE Transactons on Crcuts and Systems for Vdeo Technology, vol., no., pp., Feb. [] Z. Song, R. Xong, X. Fan, S. Ma, and W. Gao, Transform doman energy modelng of natural mages for wreless softcast optmzaton, n IEEE Internatonal Symposum on Crcuts and Systems (ISCAS), Melbourne, Australa, June, pp.. [] Z. Song, R. Xong, S. Ma, X. Fan, and W. Gao, Layered mage/vdeo softcast wth hybrd dgtal analog transmsson for robust wreless vsual communcaton, n IEEE Internatonal Conference on Multmeda and Expo (ICME), Chengdu, Chna, Jul.. [] S. Yun, D. Km, X. Lu, and L. Qu, Optmzed layered ntegrated vdeo encodng, IEEE Conference on Computer Communcatons (INFOCOM ), pp., Aprl. [] X. Fan, R. Xong, D. Zhao, and F. Wu, Layered soft vdeo broadcast for heterogeneous recevers, IEEE Transactons on Crcuts and Systems for Vdeo Technology, vol., no., pp., Nov. [] H. Cu, C. Luo, C. W. Chen, and F. Wu, Robust lnear vdeo transmsson over raylegh fadng channel, IEEE Transactons on Communcatons, vol., no., pp.,. [] R. Xong, F. Wu, J. Xu, X. Fan, C. Luo, and W. Gao, Analyss of decorrelaton transform gan for uncoded wreless mage and vdeo communcaton, IEEE Transactons on Image Processng, vol., no., pp., Aprl. [] R. Xong, J. Zhang, F. Wu, and W. Gao, Power-dstorton optmzaton for uncoded lnear-transformed transmsson of mages and vdeos, IEEE Transactons on Image Processng, accepted n Oct., Manuscrpt ID: TIP--. [] B. Grod, What s wrong wth mean-squared error, Dgtal Images and Human Vson, pp., MIT Press, Cambrdge, MA, USA,. [] Z. Wang and A. Bovk, Mean squared error: Love t or leave t? a new look at sgnal fdelty measures, IEEE Sgnal Processng Magazne, vol., no., pp.,. [] Z. Wang, A. Bovk, H. Shekh, and E. Smoncell, Image qualty assessment: from error vsblty to structural smlarty, IEEE Transactons on Image Processng, vol., no., pp.,. [] J. Zhu and N. Wang, Image qualty assessment by vsual gradent smlarty, IEEE Transactons on Image Processng, vol., no., pp.,. [] A. Lu, W. Ln, and M. Narwara, Image qualty assessment based on gradent smlarty, IEEE Transactons on Image Processng, vol., no., pp.,. [] A. Rehman and Z. Wang, SSIM-nspred perceptual vdeo codng for hevc, n IEEE Internatonal Conference on Multmeda and Expo (ICME),, pp.. [] S. Wang, A. Rehman, Z. Wang, S. Ma, and W. Gao, Perceptual vdeo codng based on SSIM-nspred dvsve normalzaton, IEEE Transactons on Image Processng, vol., no., pp.,. [] K. N. Ngan, Experments on two-dmensonal decmaton n tme and orthogonal transform domans, Sgnal Processng, vol., no., pp.,. [] S. Martucc, Image reszng n the dscrete cosne transform doman, n Internatonal Conference on Image Processng, vol.,, pp., vol.. [] R. Xong, F. Wu, J. Xu, and W. Gao, Performance analyss of transform n uncoded wreless vsual communcaton, n IEEE Internatonal Symposum on Crcuts and Systems (ISCAS), Bejng, Chna, May, pp.. [] L. I. Rudn, S. Osher, and E. Fatem, Nonlnear total varaton based nose removal algorthms, Physca D: Nonlnear Phenomena, vol., no., pp.,. [] A. Chambolle and P.-L. Lons, Image recovery va total varaton mnmzaton and related problems, Numersche Mathematk, vol., no., pp.,. [] S. Osher, M. Burger, D. Goldfarb, J. Xu, and W. Yn, An teratve regularzaton method for total varaton-based mage restoraton, Multscale Modelng & Smulaton, vol., no., pp.,. [] Y. Wang, J. Yang, W. Yn, and Y. Zhang, A new alternatng mnmzaton algorthm for total varaton mage reconstructon, SIAM Journal on Imagng Scences, vol., no., pp.,. [] M. Afonso, J. Boucas-Das, and M. Fgueredo, Fast mage recovery usng varable splttng and constraned optmzaton, IEEE Transactons on Image Processng, vol., no., pp., Sept. [] J. Zhang, R. Xong, S. Ma, and D. Zhao, Hgh-qualty mage restoraton from partal random samples n spatal doman, n IEEE Vsual Communcatons and Image Processng (VCIP),, pp.. [] S. Martucc, Symmetrc convoluton and the dscrete sne and cosne transforms, IEEE Transactons on Sgnal Processng, vol., no., pp.,.

12 Page of SUBMITTED TO IEEE TRANS. ON IMAGE PROCESSING ew ev rr Fo ly On Fg.. Reconstructed mages of the three evaluated schemes (CSNR=dB). From left to rght: the orgnal mage, the DrectTx scheme, the scheme, and the proposed scheme. Please enlarge the fgures for vsual detals.

13 Page of SUBMITTED TO IEEE TRANS. ON IMAGE PROCESSING ev rr Fo Fg.. Reconstructed mages of the three evaluated schemes (CSNR=dB). From left to rght: the DrectTx scheme, the scheme, and the proposed scheme. Please enlarge the fgures for vsual detals. ew ly On Fg.. The effect of usng gradent pror knowledge n reconstructon. Top: wthout usng gradent pror. Bottom: wth gradent pror. From left to rght: CSNR= db, CSNR= db and CSNR= db, respectvely. Please enlarge the fgures for vsual detals.