Application of Image Fusion to Wireless Image Transmission

Transcription

1 Applicatio of Image Fusio to Wireless Image Trasmissio Liae C. Ramac ad Pramod K. Varshey EECS Departmet, Syracuse Uiversity 121 Lik Hall, Syracuse, NY Abstract - Image fusio is proposed as a method to combat errors durig trasmissio of images o wireless chaels. For images represeted i the wavelet domai, diversity is used to obtai multiple data streams correspodig to the trasmitted image at the receiver. These idividual image data streams are fused to form a composite image with higher perceptual quality. Diversity combiig methods usig image fusio exploit the characteristics of the wavelet trasform. Simulatio results demostrate that the perceptual quality of the received image ca be sigificatly improved. Key Words: image fusio, diversity combiig, image trasmissio 1. Itroductio The use of multiple images i fields such as remote sesig, medical imagig ad automated machie visio has icreased i the past decade. As a result of this, several image fusio techiques have bee developed to produce a composite image with more useful iformatio cotet for automatic computer aalysis tasks as well as for huma perceptio [1,2]. This paper applies image fusio cocepts to a ew area, amely to image trasmissio systems that employ wireless chaels. For image trasmissio over wireless chaels, several methods have bee proposed i the literature [3-8] that use differet types of error-correctio codig, ARQ, or post-processig to deal with the chael errors. The goal of this paper is to itroduce a ovel image trasmissio method based o image fusio that ca produce a image of high perceptual quality at the receiver. Before a image is trasmitted over a wireless chael, it is desirable to implemet a method for represetig the image that is resiliet to chael errors. For a error resiliet represetatio, wavelet based decompositio will be utilized for trasmittig the image i its ucompressed state. Durig trasmissio, the image will be subject to bursty chael errors. Therefore, a techique is eeded at the receiver to correct or coceal ay errors that may degrade the perceptual quality of a image beyod acceptable limits. Diversity is a commuicatio method used to improve wireless trasmissio that utilizes idepedet (or highly ucorrelated) commuicatio sigal paths to combat chael oise. The idepedet sigal paths provide the receiver with multiple sigals for appropriate diversity processig of the received sigals. For image trasmissio, a diversity techique has bee employed i cojuctio with ARQ [4]. This approach ivolves switched atea diversity that operates i the data domai. Ulike data domai diversity combiig methods, the diversity combiig method we propose here operates i the image domai by usig the properties of the origial image or its wavelet trasform. Our ovel approach to wireless image trasmissio combats the effects of fadig ad other chael impairmets by employig a diversity combiig method that attempts to directly improve image quality. This diversity combiig method was ispired by the image fusio work of Burt [9] where he produced oe composite image from multiple source images with differet iformatio cotet. Burt implemeted his fusio method by takig a Laplacia pyramid trasform of each source image, combiig the trasforms based o measures i the trasform coefficiet eighborhoods, ad performig the iverse trasform to obtai the composite image. Later, Li et al [10] used this same image fusio methodology but with the wavelet trasform. For image trasmissio over wireless chaels, two or more diversity chaels ca be utilized to obtai multiple bit streams at the receiver, with each bit stream idepedetly represetig the image data. The these bit streams ca be fused i the image domai to improve the perceptual quality of the received image. Due to the radom ature of radio propagatio, we expect the errors o the idividual chaels to be idepedet or at least highly ucorrelated. This allows for a fusio method that yields excellet quality images i the presece of wireless chael errors. The orgaizatio of the rest of the paper is as follows. Sectio 2 briefly describes the chael model used for simulatios. Our diversity combiig method based o image fusio is discussed i Sectio

2 3 alog with some results ad coclusios are give i Sectio Chael Model Wireless chaels are corrupted by errors that are bursty i ature. Modelig of the physical chael is a complex problem that depeds upo the movemet of the trasmitter, receiver, ad other objects i the sigal path. While a umber of models that characterize the physical pheomea have bee proposed i the literature, here we employ a chael model to geerate error sequeces that attempts to represet the iput-output relatioships of the wireless chael. Oe popular iput-output error model is i terms of a fiite state Markov chai. I this model, each state represets a differet chael coditio ad the associated error behavior. These models are specified i terms of trasitio probabilities betwee the idividual states ad the correspodig error probability for each state. The model we use for our simulatios is a two-state Gilbert-Elliott chael [11, 12]. The two-state Gilbert-Elliott chael has oe good state ad oe bad state, represeted by 0 ad 1 respectively as show i Figure 1. This chael ca also be described by its burst error legth ad error rate parameters, which are related to the trasitio probabilities betwee states ad the error probabilities of the idividual states. The average error rate is the proportio of errors to the total umber of trasmitted bits ad the average burst error rate is the time spet i the bad state. While i the good state the bits are trasmitted icorrectly with probability P e (0), ad while i the bad state the bits are trasmitted icorrectly with probability P e (1). For this model it is assumed that P e (0) << P e (1). The two-state chael model ca be described by the biary Markov process γ with the followig trasitio matrix: P P = P p = r ( γ = 0γ = 0) P( γ = 1γ = 0) ( γ = 0γ = 1) P( γ = 1γ = 1) 1 p 1 r where γ =0 if the chael is i the good state at time, ad γ =1 if the chael is i the bad state at time. The average burst legth L is a geometric radom variable with mea 1/r, ad the average time the chael is i the good state is also a geometric p 1-p 0 1 radom variable with mea 1/(1-p). The steady state probability of the chael beig i a bad state is π 1 = ( 1 p) ( r + 1 p). Also, the steady-state error rate is give as ε = ( Pe (0) r + Pe (1) (1 p) ) ( r + 1 p) [13]. This model will be used to geerate errors to corrupt the images i our simulatios i order to evaluate the performace of our diversity combiig method. 3. Image Fusio for Diversity Combiig Our diversity combiig method for ucompressed images ivolves computig the twodimesioal wavelet decompositio of the source image ad quatizig the resultig wavelet coefficiets. The coefficiets are the trasmitted as a bit stream over a wireless commuicatios system employig diversity without ay error cotrol. Diversity is used to obtai multiple copies of the decomposed image data at the receiver. At the receiver, the idividual decomposed images are fused to form a composite wavelet decompositio ad the the fial received image is recostructed. This diversity combiig method based o image fusio is depicted i Figure 2. The first step i wireless image trasmissio is to cosider how the image will be represeted for trasmissio. The two-dimesioal wavelet decompositio of a image is implemeted with traditioal subbad filterig [14] usig oe- Source Image r Figure 1. Two-state Gilbert-Elliott chael Received Image Wavelet Trasform Iverse Wavelet Trasform Quatize Image Fusio Figure 2. Image trasmissio method. 1-r Wireless Chael M

3 dimesioal low-pass (H) ad high-pass (G) quadrature mirror filters. First, the iput image is covolved with H ad G i the horizotal directio ad the the output rows are dow-sampled by two. The the two resultig sub-images are further filtered alog the vertical directio followed by dow samplig of the colums. At the output, the source image at resolutio k is decomposed ito four subimages: a image at lower resolutio level k-1, a horizotally orieted detail image, a vertically orieted detail image, ad a diagoally orieted detail image. The filterig ca be repeated by usig the low-resolutio image as the source image util the desired decompositio level is reached. The image at resolutio k is recostructed from the four subimages at resolutio k-1 usig recostructio filters H ad G. The rows are up-sampled by two (oe row of zeros is iserted betwee each row) ad filtered i the vertical directio. The the same procedure is followed i the horizotal directio. At the output, a recostructed image at resolutio k is obtaied. Repeatig the same procedure, the origial level at which the decompositio was started ca be reached. I this paper, we use images trasformed i the wavelet domai with uiform scalar quatizatio of the coefficiets. The results obtaied will help demostrate the usefuless of image domai diversity combiig for image trasmissio over wireless chaels. For images without compressio, the wavelet represetatios are obtaied from the bit streams received o the idividual diversity chaels. I geeral, the low-resolutio subbad is more importat perceptually ad a large error i pixel itesity ca seriously affect image quality. A error i the high frequecy subbad is ot as importat to the overall image quality. Because the characteristics of the subbads are differet, the diversity-combiig rule for the low-resolutio subbad differs from the combiatio rule for the high frequecy subbads. After obtaiig the composite decomposed image from fusig the idividual trasformed images, the iverse wavelet trasform is performed to obtai the fial image. The idea behid diversity combiatio is to sigificatly reduce visible errors i the received image without ecessarily usig techiques such as ARQ or error correctio codig. The diversity combiig method is demostrated here usig two idepedet chaels, chael oe ad chael two, but the idea ca easily be exteded to more chaels. Whe the bit streams cotaiig the decomposed images are received, a decisio is made as to whether to take the data from chael oe, chael two, or from a combiatio of both. Depedig upo the chael state the two received bit streams will cotai the same values for may of the coefficiets. The low frequecy subbad ad high frequecy subbads have differet sesitivities to bursty chael errors. Therefore, the rules for the two types of subbads are differet. For both of the differet subbad types there are two combiatio modes: selectio ad coefficiet combiig. I the selectio mode, oe coefficiet is selected from the two decomposed images ad placed i the composite. I the coefficiet-combiig mode, groups of coefficiets from eighborhoods of both decomposed images are examied ad a value is placed i the composite decomposed image based o measures from both coefficiet eighborhoods. The combiatio method is similar to usig both image averagig ad spatial filterig to remove chael oise. Sice the low-resolutio subbad is more perceptually importat to the image, more care must be take whe dealig with detected chael errors i the low-resolutio subbad. First, the coefficiets from the two diversity bit streams are compared as they arrive at the receiver. If the received wavelet coefficiet values are the same, we assume that the value is correct ad select the coefficiet from either chael to place i the combied trasform. If the coefficiet values are differet, the receiver waits util a m by eighborhood of coefficiets surroudig the coefficiet of iterest is available from both chaels. Small eighborhoods (i.e. 3 by 3) of a image are geerally smooth. Therefore, the itesity values usually do ot vary sigificatly withi these eighborhoods. Whe the two received coefficiets at locatio (i, j) are differet, the m by eighborhoods of coefficiets aroud them are grouped ito a set of 2m values. The the media value is chose as the coefficiet to place i the combied low-resolutio sub-image at locatio (i, j). I geeral, this media-based method teds to be more robust to large chael errors tha averagig the coefficiets i order to obtai a combied coefficiet value. Therefore, for each (i, j), the coefficiet placed i the combied low resolutio subbad image is defied as follows (assumig m ad are odd): c (i, j) Lc = R S T c (i, j) if c (i, j) = c (i, j) l ql q L1 L1 L2 med c (k, l), c (k, l) if c (i, j) c (i, j) F R HG S L1 L2 L1 L2 U V R S U V I W KJ for m 1 m 1 (k, l) i,, i +, j,, j T W 1 1 K K T 2 2 where c Lc represets the wavelet coefficiets i the low-resolutio subbad of the combied trasform,

4 ad c L1 ad c L2 are the low-resolutio coefficiets obtaied from two diversity chaels. A error i the high frequecy subbads does ot affect the quality of the fial recostructed image as much as i the low frequecy subbads. Also, most of the coefficiets have magitudes close to zero. Therefore, the errors i the detail subbads are processed differetly whe the received wavelet coefficiets are ot the same. Agai, if the received wavelet coefficiet values are the same, we assume that the value is correct ad place this value i the combied trasform. However, if the received coefficiets are differet, the coefficiet with the miimum absolute value is chose ad placed i the fial combied trasform. The idea behid this selectio method is that a coefficiet that implies a strog edge where oe does ot exist will visually degrade the image more tha a coefficiet that implies o edge where oe really exists. Sice most of the coefficiets i the high frequecy subbads are ear zero, there is a better chace that the coefficiet with the miimum absolute value will be correct. Eve if we set the coefficiets to zero i the high frequecy subbads, the quality of the fial image will still be acceptable. The combied coefficiet values for each locatio (i,j) i the high frequecy subbads are give as follows: c Hc ( i, j) = R S T c ( i, j) if c ( i, j) = c ( i, j) H1 H1 H2 c ( i, j) if c ( i, j) < c ( i, j) H1 H1 H 2 c ( i, j) if c ( i, j) < c ( i, j) H 2 H 2 H1 500 bits ad various bit error rates (.0001,.0005,.001,.005,.01). The error probabilities withi the idividual states were set to P e (0) = 0.0 ad P e (1) = 0.5. Performace is measured usig peak sigal to oise ratio (PSNR): PSNR = 10log 10 where ( i j) image ad p ( i, j) 1 N ( p( i, j) pˆ ( i, j) ) i j 255 p, are the pixel values of the origial ˆ are the pixel values of the received image. For our simulatios we tested our diversity combiig method o the two images show i Figure 3. Both are 8-bit graylevel images with 256 by 256 pixels. First, the source images were decomposed to two levels usig the wavelet trasform. The the wavelet coefficiets were uiformly quatized to 8 bits per pixel i order to maitai the same umber of bits as i the origial image. But for the bit stream with BCH codig, the total umber of trasmitted bits is greater tha 8 bits per pixel. For ucompressed images we did ot attempt to match bit rates for performace comparisos. The give PSNR results were averaged over twety rus. 2 2, where c Hc represets the wavelet coefficiets i the detail subbads of the combied trasform, ad c H 1 ad c H 2 are the detail subbad coefficiets obtaied from two diversity chaels. I order to show the feasibility of usig diversity combiatio for wireless image trasmissio, simulatios were performed usig ucompressed images. The results are compared to a system that uses error cotrol codig for error protectio. I our experimets, images were trasmitted usig a BCH(255, 179) code with error correctio capability of 10 bits. For each simulatio, two bit error patters were geerated usig the twostate Markov model described i Sectio II. Both error patters were applied to the image data bit streams for the diversity combiatio method ad oe of the error patters was used for the error codig method. The parameters used for geeratig the bit error patters were a average burst error legth of (a) (b) Figure 3: Origial test images for wireless image trasmissio: (a) Peppers ad (b) Lea. Table 1 gives the PSNR results for the Peppers image usig image fusio versus BCH(255, 179). I this table, we see that the PSNR results for image fusio are about 11 to 13 db higher tha error codig. Examples of the received Peppers images are show i Figure 4 for bit error rates of ad Table 2 gives the PSNR results for the Lea image usig image fusio versus BCH codig where the image fusio method exceeds the error codig by about 15 to 16 db. Examples of the received Lea images are show i Figure 5 for bit error rates of ad These examples demostrate that

5 image fusio ca sigificatly improve performace compared to usig BCH error correctio codig. Table 1: PSNR (db) for Peppers Bit error rate Image Fusio BCH(255, 179) (a) (b) Table 2: PSNR (db) for Lea Bit Error Rate Image Fusio BCH(255, 179) (a) (b) (c) (d) Figure 5: Received images for BER = (a) BCH codig, (b) image fusio ad BER = 0.01 (c) BCH codig ad (d) image fusio. 4. Coclusios A image domai diversity method has bee preseted for the trasmissio of images over wireless chaels. For images represeted i the wavelet domai, diversity is used to obtai multiple data streams of the image at the receiver where these data streams are fused to obtai a composite image. The methods proposed here use some of the properties of the wavelet trasform to sigificatly improve the perceptual quality of the received image. Our results showed that image domai diversity could be used to improve performace for images trasmitted over wireless chaels. We have also implemeted similar image fusio methods for compressed images to improve image quality ad have obtaied excellet results [15]. 5. Ackowledgemets (c) (d) Figure 4: Received images for BER = (a) BCH codig, (b) image fusio ad BER = 0.01 (c) BCH codig ad (d) image fusio. We would like to thak Mucahit Uer, Mark Alford ad Dave Ferris for their help ad support durig this research. This work was supported by Air Force Research Laboratory, Air Force Materiel Commad, USAF, uder grat umber F The U.S. Govermet is authorized to reproduce ad distribute reprits for govermetal purposes otwithstadig ay copyright aotatio thereo. The views ad coclusios cotaied herei are those of the authors ad should ot be iterpreted as ecessarily

6 represetig the official policies or edorsemets, either expressed or implied, of Air Force Research Laboratory or the U.S. Govermet. Refereces [14] J.W. Woods ad S.D. O Neil, Subbad codig of images, IEEE Tras. o Acoust., Speech, Sigal Processig, vol. ASSP-34, Oct [15] L.C. Ramac, Ph.D. dissertatio i progress. [1] A. Toet, "Hierarchical image fusio," Machie Visio Applicatios, pp. 1-11, March [2] M. Pavel, J. Larimer ad A. Ahumada, "Sesor fusio for sythetic visio," i Proceedigs of AIAA Coferece o Computig i Aerospace, (Baltimore, MD), October [3] P. Burlia ad F. Alajaji, A Error Resiliet Scheme for Image Trasmissio over Noisy Chaels with Memory, IEEE Tras. o Image Processig, Vol. 7, o. 4, pp , April [4] V. Weerackody ad W. Zeg, ARQ schemes with switched atea diversity ad their applicatios i JPEG image trasmissio, IEEE GLOBECOM, vol. 3, pp , [5] P.G. Sherwood ad K. Zeger, Error Protectio for Progressive Image Trasmissio Over Memoryless ad Fadig Chaels, IEEE Tras. o Commuicatios, vol. 46, o. 12, pp , December [6] W.M. Lam ad A.R. Reibma, A Error Cocealmet Algorithm for Images Subject to Chael Errors, IEEE Trasactios o Image Processig, vol. 4, o. 5, pp , May [7] Y. Koyama ad S. Yoshida, Error Cotrol for Still Image Trasmissio over a Fadig Chael, IEEE 45 th Vehicular Techology Coferece, Vol. 2, pp , [8] Z. Su, J, Luo, C.W. Che ad K.J. Parker, Aalysis of a Wavelet-based Compressio Scheme for Wireless Image Commuicatio, Proc. of SPIE, Vol. 2762, pp , [9] P.J. Burt ad R.J. Lolczyski, Ehaced image capture through fusio, Proc. of Fourth Iteratioal Coferece o Computer Visio (Berli, Germay), pp , May [10] H. Li, B.S. Majuath ad S.K. Mitra, Multisesor image fusio usig the wavelet trasform, Graphical Models ad Image processig, vol. 57, pp , May [11] E.N. Gilbert, Capacity of a Burst-Noise Chael, The Bell Systems Techical Joural, pp , September [12] E.O. Elliott, Estimates of error rates for codes o burst error chaels, Bell Systems Techical Joural, vol. 42, p. 1977, Sept [13] J.R. Yee ad E.J. Weldo, Evaluatio of the performace of error-correctig codes o a Gilbert chael, IEEE Tras. o Commuicatios, vol. 43, pp , Aug.