View Synthesised Prediction with Temporal Texture Synthesis for Multi-View Video

Transcription

1 View Synhesised Prediion wih Temporal Texure Synhesis for Muli-View Video D M Moiur Rahaman Charles Sur Universiy Shool of Compuing and Mahemais Bahurs, NSW 2795 Ausralia drahaman@su.edu.au Manoranjan Paul Charles Sur Universiy Shool of Compuing and Mahemais Bahurs, NSW 2795 Ausralia mpaul@su.edu.au Absra In free viewpoin video (FVV) framework, a large number of viewpoins from limied number of views are generaed o redue he amoun of ransmiing, reeiving and proessing video daa signifianly. To generae a virual view, he dispariy among adjaen views or emporal orrelaion beween differen frames of he inermediae views are normally exploied. Those ehniques may onern poor rendering qualiy by missing pixel values (i.e. reaing holes) due o he oluded region, rounding error and dispariy disoninuiy. To address hese problems reen ehniques use inpaining, however, hey sill suffer qualiy degradaion due o lak of spaial orrelaion on he foreground-bakground boundary areas. The bakground updaing ehniques wih Gaussian mixure based modelling (GMM)) an improve qualiy in some oluded areas, however, due o he dependenies on warping of bakground image and spaial orrelaion hey sill suffer qualiy degradaion. In his paper, we propose a view synhesized prediion using Gaussian model (VSPGM) ehnique using he number of GMM models raher han he bakground image o idenify bakground/foreground pixels. The missing pixels of bakground and foreground are reovered from he bakground pixel and he weighed average of warped and foreground model pixels respeively. The experimenal resuls show ha he proposed approah provides 0.50~2.14dB PSNR improved synhesized view ompared wih he sae-of-he-ar mehods. To verify he effeiveness of he proposed synhesized view, we use i as a referene frame wih immediae previous frame of urren view in he moion esimaion for muli-view video oding (MVC). The experimenal resuls onfirm ha he proposed ehnique is able o improve PSNR by 0.17 o 1.00 db ompared o he onvenional hree referene frames. Keywords Deph image based rendering; spaial orrelaion; emporal orrelaion; inerpolaion; muli-view video oding; muliview video plus deph I. INTRODUCTION Laes hree dimensional (3D) ehnologies suh as free viewpoin elevision (FTV) videos have beome inreasing ineres for generaing a more realisi impression of a sene in everyday appliaions suh as eleonferene sysems. A large number of views wih smaller baseline disane are required o failiae his luxury, whih inreases ransmission bandwidh and proessing ime for video daa signifianly. Therefore, he bes poliy would be o enode a subse of views and he reeiver will synhesis he desired view from he limied deoded views. The mos popular view synhesis ehnique is deph image based rendering (DIBR) whih relies on deph map or he geomery of a sene [1]-[4]. DIBR ehnique akes a deph map ogeher wih exure image o synhesis a virual view [3]-[6]. However, if he rendering views onain holes due o olusion, i is no possible o overome by using a single referene amera view [7][8]. Two adjaen ameras an over relaively wider viewing angle. Therefore, adjaen views and heir orresponding deph maps are used o minimize he olusion problem for generaing a virual view [8]-[10]. However, in he praial appliaion, a small numbers of views are ransmied due o he limiaion of bandwidh. Therefore, he rendered view from a limied number of exures and deph maps would miss some pixels informaion [12]. The exising ehniques based on a small number of views suffer from several problems suh as oluded region, low preision rounding error and dispariy disoninuiy [11][13]. Mos of he ases, an inpaining ehnique is used o reover oluded areas by exploiing spaial orrelaion [14]. The inpaining ehnique ompues he prioriy of every pixel on he hole s boundary and hen opies he orresponding porion of he soure pah in he arge region of he arge pah [14][15]. However, he suess of he inpaining depends on how effiienly he prioriy of he hole is seleed and on how he soure pahes seleed for a arge pah [14]. Moreover, he mixing up of he foreground and bakground prioriy pahes an deeriorae he inpaining proess. There are a number of ehniques for overoming hose problems, however, due o he low spaial orrelaion in foreground/bakground boundaries his ehnique are no effeive [16]. The above problem an be solved by reaing foreground and bakground areas separaely based on Gaussian mixure modelling (GMM) [17][18]. The ehniques exploi emporal orrelaion using GMM-based bakground updaing o reover missing pixels of he oluded areas. GMM was used o generae a bakground exure frame and hen refine i by removing roaional foreground areas using lusering ehniques wih he orresponding deph image. The refine bakground image is used for reovering missing oluded sai bakground pixels of he synhesised view. They used a onvenional inpaining mehod for he missing pixels of sai foreground areas. The experimenal resuls show ha he

2 bakground updaing mehod provides a signifian improvemen ompared o oher mehods inluding popular inpaining mehods [18]. Due o he dependeny on he warping of a bakground image and inpaining mehods, if here is any error in bakground frame i propagaes o he lusering ehnique as well. As hey need hole-filling in wo onseuive proesses, any inauray of he firs sep also deerioraes he qualiy of he final sep. Moreover, he bakground pixel generaion based on he mean value of he model sored by he raio of weigh and sandard deviaion does no represen he reen hanges of he pixel. As a resul poor bakground pixel has been generaed for heir approah [19]. In he proposed VSPGM ehnique, we uilized he inheren apaiy of he GMM o idenify foreground and bakground pixels o reover oluded areas. To address he above menioned problems, we use he number of Gaussian mahemaial models represening a pixel raher han he end produ i.e. bakground o idenify bakground and foreground he pixel. Aording o he GMM ehnique only one model is inrodued for a pixel if he pixel experienes similar inensiies over he ime, whih indiaes ha he pixel is a bakground pixel. On he oher hand, a pixel may experiene bakground and foreground objes if a pixel is represened by more han one GMM models. Thus, our assumpion is ha he number of GMM models would be a good indiaor o idenify bakground/foreground pixels. In he proposed ehnique, we reover he missing pixels of he foreground and bakground areas from he adapive weighed average of warped image and he bakground image and bakground image respeively. We apply GMM on he images of he inerpolaed view raher han on he adjaen view assuming ha we have already synhesized previous images of he inerpolaed view. Thus, in he proposed mehod we an ge beer pixel orrespondenes. The experimenal resuls onfirm ha he proposed ehnique ould provide promising resuls. View synhesized ehniques reognized as a promising ool for rendering new views from muli-view video plus deph (MVD) for supporing advaned 3D video oding [20]. This ehnique provides an exra referene for muli-view video oding (MVC) by exploiing dispariy among adjaen views. Due o he high similariy of he proposed synhesised view wih he urren view, his ehniques provide beer prediion ompare o he onvenional hree referenes (i.e. wo frames from adjaen views and he previous frame of he urren view) sysem. To verify he effeiveness of he proposed synhesized view, we use wo referene frames using he proposed synhesized view and he previous frame. The resuls show ha we an improve he PSNR ompared o he hree referene sheme in mul-view ompression. The main onribuion of his paper is o inrodue a new hole-filling mehod based on he mahemaial model using GMM insead of inpaining ehnique in he oluded areas of he inerpolaed views. The res of his paper is organized as follows: seion II desribes he proposed view synhesis ehnique wih deails of foreground and bakground pixel modelling. Seion III desribes experimenal resuls, while Seion IV onludes he paper. II. PROPOSED VIEW SYNTHESIZED PREDICTION TECHNIQUE In his paper, we propose a VSPGM ehnique for improving he qualiy of virual view for 3D video and FVV by using sandard muli-view video sequenes. In his ehnique, we have aken wo exure images and orresponding dephs maps and amera parameers as inpus. Then, we render a virual view, bu his view onains many raks, ghoss and holes. To redue hese missing pixels of he oluded region, we model eah pixel using he GMM wih available previous frames of he inerpolaed view. We assume ha when we inerpolae n h frame of a virual view, 1 o (n-1) h frames of he virual view are available for he GMM. We lassify eah pixel as a foreground or bakground based on he number of models in he GMM. Pixel inensiies of he oluded areas are filled based on eiher ompleely from he pixel of he bakground model or from he weighed inensiy beween he rendered image and he foreground model(s). The following sub-seion desribes he ehnique of inerpolaing virual view(s) wih GMM based hole-filling ehnique. A. Inerpolaing Virual View Le 1 and 2 be he view 1 and view 2 exure images and 1 and 2 are he orrespondene deph maps of he same sene apured by wo ameras a he same ime. Generally, deph maps represen disane of objes from he amera whih are quanized ino 256 differen values where 0 and 255 represen he farhes and neares disane respeively. The rue deph values Z are onvered from he enoded deph map, he farhes deph in he sene Z max, and he neares deph in he sene Z min as by using (1) [10][12][21]: 1 Z Zmin Z max Zmax ( 1) Then he dispariy ( ) beween he referene view and he virual view is alulaed by using (2) f. Z where f is he amera foal lengh and is he baseline disane, i.e. he horizonal disanes beween he referene view and he virual view posiion. Afer adjusing he rue deph values and alulaed dispariy, he exure image is aligned in he new posiion [21][22]. However, his aligned exure onains many holes due o he quanizaion error, disoninuiy of dispariy and olusion problem. To minimize he hole problems, we warped adjaen exure by using orresponding deph images o a virual posiion [14][22][23]. Afer ha, blending boh warped images based on four ondiions are as follows: (2)

3 1 ' ' ' ' 1 3, if no holes in 1 and 3 ' ' ' 1, if no holes in 1, bu holes in 3 (2) ' ' 3, if no holes in 3, bu holes in 1 ' ' 0, if holes in 1 and 3. ' ' where 1 and 3 are he view 1 and view 3 warped images, is he rendered view and is a weighed faor (o generae a middle view, he value of is 0.5). This proedure redues he number of holes, bu does no help o reover all missing pixel inensiies. To reover missing pixel, we model eah pixel by using GMM ehnique. B. GMM Tehnique Eah pixel posiion of a sene is modeled independenly by a mixure of K (generally 3 models [24][25]) Gaussian disribuions. Le a ime a pixel inensiy of k-h Gaussian represening, k,, wih mean, k,, weigh in he mixure, k,, 2 and he variane, k, suh ha k, 1. The fixed iniial parameers suh as sandard deviaion, k = 2.5, iniial weigh, k =0.001 and learning rae, =0.1 are used in he proposed ehnique. For balaning he onribuion beween presen and previous values of variane, mean, and weigh, a learning parameer 0<α<1 is used [19][26]. Afer iniializaion, a he ime for every new observaion, he new pixel inensiy is X suh ha X k, 2. 5 k, for he firs mah agains exising models. If a model mahes, assoiaed parameers are updaed as follows: k, ( 1) k, 1 X ; ( 3) 2 k, ( k, 1 k, k, 2 T 1) ( X ) ( X ) ; ( 4) k, ( 1) k, 1, ( 5) and he weighs of he remaining Gaussians are updaed as k, ( 1) k, 1. ( 6) Afer ha he weighs are re-normalized among all models so ha he oal value is 1. On he oher hand, if a model doesn mah, a new model is inrodued using iniial parameer values. If i has already reahed a he maximum allowable number of models, he new model replaes an exising model based on he value of /. When a pixel inensiy of a olor () saisfies a model (k), we also keep he pixel inensiy as he reen value k, of he orresponding model and olor. The value of pixel reovering ehnique. k, will be used in he missing Thus, eah pixel an be represened by a number of Gaussian models. If a pixel represens a sai bakground over he ime, hen i migh have only one model, on he oher hand, if a pixel experienes foreground/bakground i migh have more han one model where one model represens a sable bakground and oher models migh represen foreground/bakground. The model wih he highes value of / represening he bakground and oher models represen foreground. As he GMM an suessfully apure foreground and bakground pixel inensiies by exploiing emporal orrelaion, missing pixels of an oluded area are suessfully reovered. C. Hole Filling Approah In he GMM ehnique, he Gaussian models of a pixel are always ordered based on he / in desending order, assuming ha he op Gaussian will provide mos sable bakground. In he proposed ehnique if a pixel is modelled using only one Gaussian model, he pixel inensiy of he inerpolaed final image is aken from he reen value i.e. k, of he model. Oherwise, he pixel inensiy of he inerpolaed image is aken as a weighed average from he rendering image (i.e. oupu image afer warping) and he reen value of he model whih provides lowes value in erms of /. The deail of he inerpolaed image reovering ehnique using GMM is desribed below [27]: Case 1: If a pixel has only one model for a given olor, we assign he reen value k, of he olor of he inerpolaed image using 1,.. ( 7) Case 2: If a pixel has wo models for a given olor a pixel experienes foreground/bakground, herefore, we hoose a weigh faor ( ) for seleing he fraion of he rendering image and he reen value of he seond models as follows: 1 ). ( 8) ( 2, Case 3: If a pixel has hree models for a given olor, we have used equaion (9) for seleing fraion of he inerpolaed image as follows: 1 ). ( 9) ( 3, III. VIEW SYNTHESIS FOR MVC Adjaen views of muli-view video sequenes are apured by muliple ameras wih slighly differen angles. Therefore, here are dispariies among differen views. Moreover, oloaed pixels/bloks a differen insan of he same views are predied by moion esimaion ehnique. However, finding o-loaed pixels/bloks on differen frame by using moion esimaion and dispariy esimaion is ime onsuming [28]. Therefore, reduion of ompuaion for searhing moion parameers suh as moion veor is an imporan aspe of

4 Synhesised Curren Frame n-h Frame of View 1 Curren Blok (X, Y) Synhesising (n-1)-h Frame of View 2 Dynami Bakground Modelling Referene Blok (Xr3, Yr3) Moion Esimaion Referene Frame 2 Referene Frame 1 Synhesising Curren Blok (X, Y) urren researh [29]. Thus, he bes poliy is reduing he number of referene views. Tradiionally, hree referenes suh as already enoded frames of adjaen views (referene frames 1 and 2) and previous frame of he urren view (referene frame 3) are used o enode eah frame of dependen views [20]. In his ehnique, a dispariy is used o find a urren blok (CB) on adjaen referene views ((, ) and ( )) where and. This mehod only onsiders he horizonal omponen as muli-view video sequenes are reified [9]. Furhermore, moion veors are predied o find a CB on he previous frame of he urren view i.e. ( ) [9][13]. Insead of ypial approahes, we will use a view synhesis ehnique o generae a synhesized urren frame i.e. referene frame 2. This synhesized frame is almos similar in erms of obje posiion and is moion o he expeed urren frame. If we would onsider wo referenes suh as referene 1 and referene 2 as shown in Fig. 1, i provides beer prediion ompared o he radiional approahes. IV. EXPERIMENTAL RESULTS n-h Frame of View 2 / Curren Frame n-h Frame of View 3 Fig. 1: Proposed MVC oding ehnique by using wo referenes suh as he previous frame of he urren view and a synhesised frame. In his seion he performane of he proposed VSPGM ehnique analysed and ompared wih sandard ehniques suh as HTM Renderer (HEVC Tes Model) [30], inpaining ehnique [14] and bakground updae ehnique [18] based on peak-signal-o-noise-raio (PSNR). We apply he inpaining mehod on he rendering image reaed afer warping. We also use he same ehnique and hen apply bakground updae ehnique and he VSPGM ehnique for he refinemen of he inerpolaed view. Fig. 2 shows he PSNR omparison beween VSPGM, HTM, inpaining and bakground updae ehniques. The figure reveals ha he proposed ehnique ouperforms he exising hole-filling ehnique for all video sequenes. The improvemen range varies from 0.1dB o 9.20dB for HTM, Fig. 2: Average PSNR (db) of he VSPGM ehnique and four sae-of-he-ar ehniques for hree sandard video sequenes. 0.50dB o 7.66dB for inpaining and 0.50dB o 2.14dB for bakground updae ehnique respeively. Fig. 3 illusraes he frame differenes beween he original image and he reonsrued images of he proposed and he sae-of-he-ar mehods. The figure reveals ha he proposed ehnique is able o generae more similar images ompared o he sae-of-he-ar mehods. To undersand he onribuion beween he GMM models and he rendering image in he proposed sheme o reonsru he final inerpolaed image, we analysis PSNR agains differen values of in Fig. 4. The figure reveals ha he onribuion of rendering image and he GMM based pixel inensiy in he foreground areas varies for differen sequenes. I an be easily observed from he figure ha boh rendering image and GMM have some onribuion o generae an inerpolaed image for eah video. Noe ha if we ge he maximum PSNR value of a given image where he value of is 1.0, i means ha he pixel inensiies of he inerpolaed image for foreground is enirely aken from he rendering image. However, he bakground pixel of he inerpolaed image is always aken from he reen pixel value of he sable bakground model in he GMM. Newspaper video sequene has maximum foreground areas as well as olusion areas whih are no able visible in he warped images. Thus, inreasing he onribuion of he warped images redue overall PSNR. The onen of he Lovebird1 video sequene i.e. foreground and bakground areas are almos balaned, herefore he onribuion of he warped images and GMM model are varied slighly wih varying he values of. Poznan Sree video sequene has less olusion areas, herefore i inreases onribuion of he warped images whih has less olusion areas and improved PSNR wih inreasing he values of. Fig. 5 illusraes he subjeive qualiy for Newspaper video sequene. Fig. 5 (a) shows he original images, i.e. 11 h original frame of he virual view and he green reangular boxes are used o mark he ropped and zoomed porion whih are shown in Fig. 5 (b) and (). Similarly, Fig. 5 (d), (g), (j) and (m) shows he inerpolaed view by HTM, inpaining, bakground updae and VSPGM ehnique and Fig. 5 (e), (f), (h), (i), (k), (l), (n) and (o) shows orresponding ropped and zoomed images. The figures reveal ha he proposed mehod is able o generae a beer virual view ompared o he sae-of-he-ar mehods.

5 (a) Original Newspaper frame (b) Original Lovebird1 frame () Original Poznan Sree frame (d) HTM [30] (e) HTM [30] (f) HTM [30] (g) Frame differene using Inpaining [14] (h) Frame differene using Inpaining [14] (i) Frame differene using Inpaining [14] (j) Frame differene using Bakground Updae [18] (k) Frame differene using Bakground Updae [18] (l) Frame differene using Bakground Updae [18] (m) Frame differene using VSPGM mehod (n) Frame differene using VSPGM mehod (o) Frame differene using VSPGM mehod Fig. 3: Comparison of frame differenes beween he original image (11 h frame) and he orresponding generaed virual image using sandard muli-view video sequenes by he proposed mehod and hree sae-of-he-ar mehods.

6 Fig. 4: PSNR (db) vs weigh for 11 h frames To undersand he reover areas in he inerpolae virual image aken from he rendering image and he bakground image, we provide pixel aegorizaion based on he soure of refinemen. In his regard Fig. 5 shows he pixels ha are seleed from he bakground image (blak porion) and rendered image (non-blak porion) for generaing final virual view for he Newspaper and Lovebird1 video sequenes. The figure demonsraes ha he moving areas i.e. foreground areas are normally aken from rendering image and he sai bakground/oluded areas are aken from he bakground image. The resuls onfirm our hypohesis on he effeiveness of he foreground/bakground pixel idenifiaion using he number of GMM models. Table 1: PSNR omparison for he proposed MVC sheme Three referenes Two referenes Newspaper (PSNR) Lovebird (PSNR) Poznan Sree (PSNR) To enode differen resoluions and wide range of video onens for differen views eah frame is divided ino he number of bloks wih various sizes suh as 8 8, 16 16, and [9] pixels and he searh lengh beome 8, 16, 32, 64 and 128 pixels. In our experimen, we have onsidered pixel blok sizes and 64 pixels searh lenghs. Due o he beer prediion of synhesized view, he proposed ehniques provides beer PSNR ompare o he (a) Fig. 6: Pixels are seleed from bakground image (blak porion) and rendered image (non-blak porion) by he proposed mehod: (a) Newspaper sequenes and (b) Lovebird1 sequenes. onvenional approahes as shown in Table 1. I reveals ha he average PSNR improvemen for wo referenes is 0.70dB. V. CONCLUSION In his paper, we presen a new VSPGM ehnique ha explois emporal orrelaion for improving he qualiy of synhesised views ompared o he exising mehods. Virual images are generaed from a exure image and is orresponding deph map. Inerpolaed virual images onain many holes due o he olusion and rounding error problem. To address hese issues, inpaining and/or bakground updaing mehods are used for he mos ases for refining he virual image. Due o he lak of spaial orrelaion in he bakground-foreground areas and warping/lusering problems of exising mehods, hey fail o provide good virual views. In he proposed mehod, we reover he missing pixel inensiies using he number of Gaussian mixure-based models whih have he apaiy o idenify foreground and bakground pixels based on he emporal orrelaion. The bakground pixels are reovered from he sable bakground model and he foreground pixels are reovered as he weighed pixel inensiies of he rendering image and he pixel inensiies of he foreground models from GMM. The experimenal resul shows ha he proposed mehod improves 5.2dB, 1.60dB, and 5.84dB PSNR on average ompared o he inpaining, bakground updae and HTM ehniques respeively. To evaluae he performane of he proposed ehnique we used synhesised frame as a referene wih immediae previous frame of he urren view for MVC, i improves 0.70dB PSNR on average ompare o he sandard ehniques. (b)

7 (a) Original frame (b) Crop and zoom image of original frame () Crop and zoom of original frame (d) HTM [30] (e) HTM [30] (f) HTM [30] (g) Inpaining [14] (h) Crop and zoom image of inpaining ehnique [14] (i) Crop and zoom image of inpaining ehnique [14] (j) Bakground Updae [18] (k) Crop and zoom image of bakground updae ehnique [18] (l) Crop and zoom image of bakground updae ehnique [18] (m) VSPGM (n) Crop and zoom image of VSPGM ehnique (o) Crop and zoom image of VSPGM ehnique Fig. 5: Original image (a), synhesised images (d, g, j and m), rop and zoom images (b,, e, f, h, i, k, l, n and o) for Newspaper video sequene by he VSPGM mehod and hree sae-of-he-ar mehods.

8 REFERENCES [1] M. Tanimoo, M. P. Tehrani, T. Fujii, and T. Yendo, Free-Viewpoin TV, IEEE Signal Proessing Magazine, vol. 28, pp , [2] P. Merkle, A. Smoli, K. Muller, and T. Wiegand, Muli-view video plus deph represenaion and oding, IEEE Inernaional Conferene on Image Proessing, vol. 1, pp , [3] C. Fehn, A 3D-TV approah using deph-image-based rendering, Conferene on Visualizaion, Imaging, and Image Proessing, pp , [4] H. Shum and S. B. Kang, Review of image-based rendering ehniques, SPIE Visual Communiaions and Image Proessing, vol. 4067, pp. 2 13, [5] L. Zhan-Wei, A. Ping, L. Su-xing, and Z. Zhao-yang, Arbirary view generaion based on DIBR, Inernaional Symposium on Inelligen Signal Proessing and Communiaion Sysems, pp , [6] C. M. Cheng, S. J. Lin, S.-H. Lai, and J. C. Yang, Improved novel view synhesis from deph image wih large baseline, Inernaional Conferene on Paern Reogniion, pp. 1 4, [7] Y. Morvan, F. Dirk, and H. Peer, Sysem arhieure for freeviewpoin video and 3D-TV, IEEE Trans. On Consumer Eleronis, vol. 54, no. 2, pp , [8] M. Xi, L.-H. Wang, Q.-Q. Yang, D.-X. Li, and M. Zhang, Dephimage-based rendering wih spaial and emporal exure synhesis for 3DTV, EURASIP Journ. on Image and Video Proess., pp. 1-18, [9] K. Muller, H. Shwarz, D. Marpe, C. Barnik, S. Bosse, H. Brus, T. Hinz, H. Lakshman, P. Merkle, F. H. Rhee, G. Teh, M. Winken and T. Wiegand, 3D high effiieny video oding for muli-view video and deph daa, IEEE Transaions on ImageProess., vol. 20, no. 9, pp , [10] S. Farid, M. Luenefore, and M. Grangeo, Deph image based rendering wih inverse mapping, IEEE Inernaional Workshop on Mulimedia Signal Proessing, pp , [11] F. Zou, D. Tian, A. Vero, H. Sun, O. C. Au, and S. Shimizu, View Synhesis Prediion in he 3-D Video Coding Exensions of AVC and HEVC, IEEE Trans. on Ciruis and Sysems for Video Tehnology, 24(10), pp , [12] A. I. Puria, E. G. Mora, B. P. Popesu, M. Cagnazzo, and B.Ionesu, Muliview plus deph video oding wih emporal prediion view synhesis, Ciruiss Sys. Video Tehnol., DOI: /TCSVT [13] F. Shao, G. Jiang, M. Yu, K. Chen, and Y. S. Ho, Asymmeri oding of muli-view video plus deph based 3-D video for view rendering, IEEE Transaions on Mulimedia, vol. 14, no. 1, pp , [14] A. Oliveira, G. Fikel, M. Waler, and C. Jung, Seleive hole-filling for deph-image based rendering, IEEE Inernaional Conferene on Aousis, Speeh and Signal Proessing, pp , April [15] A. Criminisi, P. Pérez, and K. Toyama, Region filling and obje removal by exemplar-based image inpaining, IEEE Trans. on Image Proess., vol. 13, no. 9, pp , [16] L. Zhu, Y. Zhang, M. Yu, G. Jiang, and S. Kwong, View-spaialemporal pos-refinemen for view synhesis in 3D video sysems, Signal Proessing: Image Communiaion, vol. 28, no. 10, pp , [17] C. Yao, Y. Zhao, and H. Bai, View synhesis based on bakground updae wih gaussian mixure model, In Advanes in Mulimed. Inform. Proess. PCM, pp , [18] C. Yao, T. Tillo, Y. Zhao, J. Xiao, H. Bai, and C. Lin, Deph Map Driven Hole Filling Algorihm Exploiing Temporal Correlaion Informaion, IEEE Transaions on Broadasing, vol. 60, no. 2, pp , [19] M. Paul, W. Lin, C. T. Lau, and B. S. Lee, Explore and model beer I- frames for video oding, IEEE Transaions on Ciruis and Sysems for Video Tehnology, vol. 21, no. 9, pp , [20] C. Zhu, and S. Li, Deph Image Based View Synhesis: New Insighs and Perspeives on Hole Generaion and Filling, IEEE Transaions on Broadasing, vol. 62, no. 1, pp , [21] H. Yuan, Y. Chang, J. Huo, F. Yang, and Z. Lu, Model-based join bi alloaion beween exure videos and deph map for 3-D video oding, IEEE Transaions on Ciruis and Sysems for Video Tehnology, vol. 21, no.4, pp , [22] P. Gao, and W. Xiang, Rae-disorion opimized mode swihing for error resilien muli-view video plus deph based 3-D video oding, IEEE Transaions on Mulimedia, vol. 16, no. 7, pp , [23] J. Oh, S. Yea, A. Vero, and Y.S. Ho, Virual view synhesis mehod and self-evaluaion meris for free viewpoin elevision and 3D video, Inernaional Journal of Imaging Sysems and Tehnology, vol. 20, no. 4, pp , [24] S. Lee, Effeive Gaussian mixure learning for video bakground subraion, IEEE Transaions on Paern Analysis and Mahine Inelligene, 27(5), pp , [25] M. Paul, Effiien muli-view video oding using 3D moion esimaion and virual frame, Neuroompuing, pp , [26] M. Haque, M. Murshed, and M. Paul, Improve Gaussian mixures for robus obje deeion by adapive muli-bakground generaion, in Pro. IEEE Inernaional Conferene on Paern Reogniion, pp. 1-4, [27] D. M. M. Rahaman and M. Paul, Hole-filling for single-view plusdeph based rendering wih emporal exure synhesis, 7h IEEE In. Workshop on Ho Topis in 3D - Ho 3D (in onjunion wih ICME), (Aeped) [28] X. Xu, and Y. He, Fas dispariy moion esimaion in MVC based on range prediion, 15h IEEE Inernaional Conferene on ICIP, pp , [29] J. Koniezny and M. Domanski, Deph-based iner-view prediion ofmoion veors for improved muliview video oding, in Pro. IEEE True Vis., Capure, Transmiss. Display 3D Video, pp. 1 4, [30] HTM, Renderer referene sofware, version 16, hps://hev.hhi.fraunhofer.de/svn/svn_3dvcsofware/, 2016.