Testing of methods for blind estimation of noise variance on large image database

Transcription

1 Testig of methods for blid estimatio of oise variace o large image database Vladimir V. Luki a, Sergey K. Abramov a, Mikhail L. Uss a,b, Igor Marusiy a, Nikolay N. Poomareko a, Alexader A. Zelesky a, Beoit Vozel b, Kacem Chehdi b a Natioal Aerospace Uiversity, 617, Kharkov, Ukraie; b Uiversity of Rees I, Laio, Frace 1. Itroductio Curretly there is a obvious eed i applyig blid (automatic), accurate, reliable (robust) ad fast methods ad algorithms for estimatio of oise type ad characteristics, at least, variace or stadard deviatio for images at had. This is eeded for optical grayscale ad color images (see [1, ] ad refereces therei), radar images [3, 4], multispectral ad hyperspectral remote sesig (RS) data [5, 6], video ad surveillace [7], etc. Determiatio of oise type ad evaluatio of oise characteristics is desirable sice origial images formed by differet types of systems are usually oisy due to various pheomea ad oise statistics is ofte ot kow i advace ad/or ca chage i a upredictable way. At the same time, may image processig methods iteded for filterig, recostructio, edge detectio, segmetatio, ad compressio require a priori kowledge o oise type ad statistics [1]. Thus, oise statistics is ofte to be estimated for a image at had. I geeral, sometimes this ca be doe i iteractive maer if a highly qualified expert has the correspodig software at disposal. However, first, this caot be doe always whe eeded. For example, such estimatio is impossible if oise is to be aalyzed o-board of a spacebore carrier of a remote sesig system. It is also impossible if such estimatio has to be carried out very quickly as i video ad surveillace applicatios. Secod, estimatio becomes very complicated ad/or laborcosumig if amout of images or its compoets is large ad processig has to be performed operatively. Such situatios take place i multi- ad hyperspectral remote sesig whe each image cotais tes or hudreds of sub-bad compoets ad RS data are exploited for ecological moitorig, catastrophe predictio, flood cotrol ad other similar purposes [8]. Note that image pre-processig (pre-filterig, recostructio, etc.) that relies o pre-estimated type ad parameters of oise ad other degradatios is ofte required i practice. For example, i may cases it leads to better classificatio of RS data [9, 1]. It might result i better visual quality of images [11, 1] ad so o. Oe ca take advatage of the fact that blid methods for determiatio (idetificatio) of oise type have bee already proposed [1, 13-15]. It is possible to determie oise type ad to estimate oise characteristics joitly or as separate stages of image aalysis [1, 13]. As it has bee show i experimets [13, 16], correct determiatio of oise type ca be carried out with rather high probability. Thus, it is possible to cocetrate o estimatig oise characteristics for a give type of oise. Methods for blid evaluatio of oise characteristics, mostly variace, were started to be desiged i early 9-th of the previous cetury [3, 4, 17]. Let us explai what is meat by the aforemetioed requiremet to them to be robust. I may applicatios, it is ukow i advace what ca be such characteristics of oise to be met i a give image as, e.g., oise variace (or SNR). It is ofte ot kow a priori is oise spatially correlated or ot. Moreover, image structure (how may details, edges, textures, homogeeous regios a image cotais) is also frequetly ukow i advace eve approximately. However, a give method for oise characteristic estimatio should perform reasoably well for a wide class of images at had, differet level (variace) ad spatial correlatio of oise, i.e. it has to be robust i wide sese accordig to P. Huber s defiitio. Examples of such lack of robustess for the method [18] are demostrated i the paper [19]. It is show that the method [18] produces large bias of additive oise variace evaluatio if oise is spatially correlated. Cosiderable efforts have bee also spet o providig appropriate accuracy of blid methods of variace evaluatio i textured images [, 1] that are the most complex (ufavorable) case of image structure for the cosidered task. Oe questio is what is appropriate accuracy? It has bee demostrated [, 3] that if peak sigal-to-oise ratio (PSNR) i a image is withi the limits...34 db, the it is desirable to provide a estimate of (additive or multiplicative) oise variace withi the limits of of its true value. I this sese, there are various ways to characterize a method accuracy ad robustess. For example, a method accuracy ca be described by some covetioal statistical parameters as mea (or bias) ad stadard deviatio (or variace) of oise variace estimates obtaied for a appropriately large set of oise

2 realizatios ad test images. Takig ito accout that estimate distributio for this approach is ot Gaussia (see data i Sectio 4) ad keepig i mid above metioed practical requiremet to a method accuracy, we prefer to aalyze how ofte blid estimates occur to be out the limits of oise variace true value. This approach also gives imagiatio about robustess of a method sice the fact of oise variace estimate out of the limits ca be cosidered as iappropriate robustess. However, a problem is that methods described i literature are commoly tested o few images (at least, the results are preseted i this way, probably, due to lack of space). These images are either artificially sythesized or they are the stadard oes i image processig like Lea, Barbara, Baboo, House, etc. Sometimes, oe or two real life images are processed as examples of method applicability but the it is hard to establish how accurate the obtaied estimates are. To partly fill this gap, we have decided to test some methods for blid evaluatio of oise variace for cosiderably greater umber of atural images. Note that recetly a database TID8 was created [11, 4]. This database cotais 5 color images (mostly take from Kodak image database corrupted by 17 types of distortios each with 4 levels. Two types of distortios are spatially ucorrelated ad spatially correlated additive oises. This allows testig ay blid method for 15 images corrupted by additive oise with the same variace (R, G, ad B compoets of each of 5 test images with either i.i.d. or spatially correlated oise). I tur, oe ca the aalyze robustess of a cosidered method to image cotet, average accuracy of a method. It also becomes possible to fid image ad oise situatios for which a cosidered method performs poorly ad so o. This is the mai goal of this paper. The paper structure is the followig. I Sectio, we briefly describe the image set i the database TID8 ad discuss how oise was simulated. Sectio 3 deals with cosiderig the blid methods for additive oise variace evaluatio used i our study. Sectio 4 presets the mai results of this study ad aalysis of the cosidered methods accuracy. The reasos why the required accuracy is ot provided i some cases are discussed. Fially, the coclusios are draw.. Test image set ad oise characteristics TID8 cotais 5 oise-free (high quality) color images where first 4 are the fragmets of origial Kodak database ad the 5-th image is the artificial image with differet textures (see Fig. 1). As see, the images are quite differet; there are atural scees, portraits, houses, etc. All color images are RGB 4-bit oes.

3 Fig. 1. Test oise-free color images of TID8 All images i the database are of size 51x384 pixels. This choice has bee suggested for uificatio purpose (the images i the Kodak test set have o-equal sizes) ad for coveiece of carryig out subjective experimets (see details i [11, 4]). Accordig to the methodology of subjective experimets proposed i [11] ad iteded for aalysis of visual quality of images distorted by differet types of degradatios, it was ecessary to provide four values of PSNR approximately equal to 1, 4, 7, ad 3 db. For additive oise it was ot a problem, a required PSNR of a distorted image ca be easily produced by simulatig oise with such variace that PSNR db = σ, (1) req ( ) 1lg(55 / ) where σ is variace of oise. Therefore, oise variace had to be 65 (for PSNR req =3 db), 13 (PSNR req =7 db), 6 (PSNR req =4 db), ad 5 (PSNR req =1 db). I our studies described i this paper we do ot cosider all these values of oise variace. The first two values (65 ad 13) are of more practical iterest sice two other oes (6 ad 5) are rarely met i practice of 8-bit image processig. Besides, for most methods of blid evaluatio of additive oise variace it is more difficult to provide appropriate accuracy of estimatio for smaller values of oise variace tha for large oes []. Thus, let us below cocetrate o aalysis of images corrupted by oise with variaces 65 ad 13. Availability of TID8 ( allows a iterested researcher to carry out his/her ow experimets for other sets of images. Obviously, simulatio of i.i.d. oise is ot a problem at all. Spatially correlated oise with variaces give above has bee simulated i a simple way by first applyig the 3x3 mea filter to a array of i.i.d. oise ad the adjustig a required variace to the obtaied oise. Oe ca argue that this is oly oe particular case of spatially correlated oise. This is really so. However, a reader should keep i mid that our purpose was oly to verify performace of the cosidered blid methods for spatially correlated oise case i order to kow does a method fail to work well or o. More details cocerig performace of several methods ca be foud i the paper [19]. Note that spatially correlated oise (with the same variace as i.i.d. oise) has more upleasat appearace [5]. The correspodig images are perceived as havig worse visual quality (compare the image i Fig.,b to the image i Fig.,a). Moreover, spatially correlated oise is much more difficult to filter out [6]. Fig.. The same image corrupted by i.i.d. (a) ad spatially correlated oise (b) with variace 13

4 Certaily, it was possible that a simulated value I true ij = Iij + ij occurred to be out of the limits true,,55 where I ij is the true value i the ij-th pixel of a give compoet of RGB image, I ij is the oisy value, ij deotes simulated additive oise. The we retured the simulated values back to the limits,,55 by assigig the closer limit value to keep 8-bit represetatio of data. This could slightly chage a practically achieved PSNR i compariso to the correspodig required oe, but such saturatio correspods to practice, e.g., how this is doe to fit image data to pre-determied limits [7]. Here we metio clippig effects sice, as it will be show below, they might lead to some specific problems i blid estimatio of oise variace. Also ote that oise has bee modeled as idepedet for differet compoets of RGB color image. This also correspods to practice [, 8]. The method of oise type blid idetificatio [13] has bee applied to all images (totally, 6 oes sice idetificatio of oise type has bee performed compoet-wise [1] for all 5 test images corrupted by four values of oise variace, both spatially ucorrelated ad correlated). For all cosidered images the oise has bee idetified as additive. This demostrates very good performace of the method [14] ad its practical applicability, at least, for images corrupted by additive oise. I future, we pla to test the method [14] to images of TID8 corrupted by other types of oise ad degradatios. 3. The used blid methods for evaluatio of additive oise variace It is impossible to cosider all kow methods for blid evaluatio of additive oise variace (see [1] ad refereces therei). Because of this, let us aalyze several particular methods that belog to differet groups. Oe, probably the largest, group icludes the methods operatig i spatial domai [9]. The methods that belog to this group are based o assumptio that blocks of a certai size tessellate a image ad there is a set of blocks that belog to image homogeeous regios. The local estimates of oise variace obtaied for these blocks are quite close to a true value of oise variace. These ormal local estimates form a distributio mode that ca be foud (estimated). A example of histograms of local variace estimates obtaied for the kow (stadard) test image Barbara corrupted by spatially correlated additive oise with variace for o-overlappig blocks of three differet sizes is preseted i Fig. 3. As it is see, they all have maximum (mode) i the eighborhood of the true values of oise variace although the positios ad widths of these maximums deped upo the block size (the width is the smallest for 9x9 blocks). Besides, all distributios (characterized by their histograms) have heavy right-had tails (these tails are depicted ot totally, cosiderably larger values of local estimates have bee observed). These large (abormal) local estimates are obtaied for image heterogeeous blocks that correspod to edges, details ad texture. Fig. 3. Examples of histograms of the local estimates { σ ˆ k, k = 1,..., K} (K is the umber of blocks used) for o-overlappig blocks for oisy image Barbara corrupted by spatially correlated oise There ca be particular differeces for the cosidered group of methods i how to estimate local variaces, how to fid the distributio mode (this priciple is put ito basis of the cosidered group of methods), to pre-process a image or ot. However, the geeral idea of these methods works well eough if there are eough image homogeeous blocks [1, 19, 1, 9, 3]. Amog the methods that belog to this group we have decided to test the method [1] i the first order. This method is quite simple ad fast; it exploits fidig miimal iter-quatile distace of sorted

5 (i ascedig order) local estimates of oise variace for fidig a prelimiary estimate ˆ σ pr. The the estimates i the eighborhood of prelimiary estimate are approximated ad a fial estimate of distributio mode ˆ σ is foud. Note that if a aalyzed image is rather large, i.e., cotais hudreds of thousads of pixels, a differece i fial estimates obtaied for differet realizatios of oise with the same variace is relatively small, commoly cosiderably smaller tha estimatio bias absolute value [19]. This allows aalyzig data for oe or few realizatios of oise to get imagiatio o provided accuracy of blid evaluatio. Aother group of methods is based o exploitig differece of image cotet ad oise i spatial spectrum domai [1, 18,, 31]. The basic idea is that oise is spread amog all spectral compoets (if oise is i.i.d., it is spread uiformly) whilst iformatio is maily cotaied (cocetrated) i a limited umber of spectral compoets. Wavelets [18] ad DCT [] as well as other orthogoal trasforms ca be used for obtaiig ad further processig of data i spectral domai. Processig of spectral coefficiets has to be doe so that large amplitude spectral compoets are practically eglected (cosidered as outliers). This meas that robust data processig methods are to be applied. The methods of this group, especially [, 31], perform rather well eve for very textural images, but their commo drawback is that this takes place oly if oise is i.i.d. or it exhibits very small spatial correlatio. For the methods [18, ] some results will be preseted i Sectio 4. Recetly, methods based o maximum likelihood (ML) estimatio of oise ad image parameters have bee proposed [3, 33]. The mai goal i desigig these methods is to achieve better performace for highly textural images. For wavelets or DCT-based methods to operate well oe mai coditio should be satisfied: stadard deviatio (SD) of a umber of texture high-spectral coefficiets has to be small compared to the oise stadard deviatio. The it is possible to detect (select) these coefficiets ad use them to estimate oise variace. However, this coditio ca be violated for highly textural images ad/or low oise level. The coditio ca be softeed if we ca predict ad elimiate texture high-spectral coefficiets based o low- ad middle-spectral oes. This was doe i [3, 33] by itroducig texture parametric model, amely D fractal Browia motio (fbm) model. This model ca be locally adjusted i statistical sese to the texture with respect to texture amplitude ad roughess. The, oise variace estimatio task is stated as maximum likelihood problem of joit estimatio of texture parameters (locally) ad oise variace (globally). This approach has proved to be effective for ucorrelated additive ad multiplicative oise variace estimatio [3, 33]. But it ca lead to biased estimatio whe texture model is sigificatly differet from fbm-model. Detectio of such differeces is of great importace for providig reliable oise variace estimatio. If oise is correlated, the its correlatio matrix should be itroduced i ML scheme, otherwise oise variace estimatio will be biased. I this paper, to cope with ukow oise correlatio structure, the followig modificatio is itroduced: the oise is made spatially ucorrelated by resamplig (dowsamplig) a origial image uder processig. I practice, it is commoly eough to use dowsamplig by a factor 3 for both directios (takig ito accout that oise correlatio legth is commoly less tha 1- pixels i practice). After this, the ML method [3, 33] for blid oise variace estimatio ca be applied. I order to provide comparable results, the modified versio of the ML algorithm is used below for estimatig both correlated ad ucorrelated oise variace. 4. Accuracy aalysis of the cosidered methods 4.1. Variace estimatio for spatially ucorrelated oise Let us cosider the results obtaied for the method [1]. Fig. 4 presets the plots for red, gree, ad blue compoets of the test images for oise variace 65. The block size is 5x5 pixels. Nooverlappig blocks have bee used.

6 18 Blue compoet Gree compoet Red compoet Test image idex Fig. 4. Noise variace estimates obtaied by the method [1] with ooverlappig 5x5 pixels block size for red, gree ad blue compoets for the test image set corrupted by spatially ucorrelated additive oise with σ =65 There are several observatios that follow from Fig. 4: 1) i most cases, the estimates ˆ σ obtaied for differet compoets of the same color image are quite close; the oly exceptio is the -th test image; the reaso will be aalyzed later; ) i most cases, the estimates ˆ σ are larger tha the true value of variace (equal to 65); the oly exceptio is the 5-th test image as well as gree ad red compoets of the -th test image; 3) although ˆ σ are commoly larger tha σ, the estimates ˆ σ are maily withi the required limits from 65x.8=5 till 65x1.=78 (the required limits are idicated by two horizotal lies i Fig. 4 ad later i other plots); the exceptios are the test images 1, 5, 8, 1, 13, 14, 18, ad ; ote that all these images are either textural (especially the 13-th image which is extremely textural) or cotai may details as the 5-th ad 8-th images. Let us cosider the same blid estimatio method but for 7x7 blocks. The results are preseted i Fig. 5. The coclusios are practically the same as for the plots i Fig. 4. The differece of the estimates for blocks of sizes 5x5 ad 7x7 is ot large. This meas that for spatially ucorrelated oise there is practically o differece is the block size 5x5 or 7x Blue compoet Gree compoet Red compoet Test image idex Fig. 5. Noise variace estimates obtaied by the method [1] with ooverlappig 7x7 pixels block size for red, gree ad blue compoets for the test image set corrupted by spatially ucorrelated additive oise with σ =65

7 Let us aalyze what happes to the gree ad red compoets of the -th test image. The histogram of local variace estimates for red compoet is depicted i Fig. 6. As it is see, it really has maximum for the argumet about (aother maximum correspods to the argumet about 65 but its amplitude is smaller). Thus, the mode determiatio algorithm that fids the largest maximum coordiate performs correctly. The reaso why quite may local estimates are cosiderably smaller tha the true value of oise variace deals with clippig effects. I the red ad gree compoets of the origial oisefree -th image, there are may positios of blocks (placed i the upper part of the image that correspods to sky, see Fig. 1) for which the block meas are close to 55. Thus, after addig simulated oise ad returig the obtaied values ito the limits from to 55 quite may oisy values occur to be equal to 55. The, the local variace estimates for the correspodig blocks are distorted (smaller tha they should be) due to such clippig. ˆσ lock Fig. 6. Histogram of local estimates for the red compoet of the th test image The clippig effect has bee already metioed ad cosidered i the paper of A. Foi [34]. Note that the effects of clippig might happe i practice due to differet reasos [34]. Their egative ifluece o the fial estimate ˆ σ demostrated above meas that local variace estimates obtaied i blocks where clippig is observed should be removed from aalysis before the algorithm for determiatio of ˆ σ is applied. For example, it is possible to cotrol how may values I ij i a give block are equal to 55 or. If their amout N clip is larger tha β clipsbl (where S bl is the umber of pixels i oe block, β clip is a preset parameter), the a give block is removed from further cosideratio. A rough recommedatio is to set βclip.15. However, this recommedatio eeds additioal verificatio. Cosider aother value of oise variace, amely, 13. The obtaied results are preseted i Fig. 7 (the required limits are 18 ad 156 for σ = 13 ). The block size is 5x5 pixels. First, it is possible to compare the estimates ˆ σ i Fig. 4 (for σ = 65 ) ad i Fig. 7 (for σ = 13 ). Such compariso shows that for a give particular test image ad the same color compoet the estimates ˆ σ i Fig. 7 are almost twice larger tha i Fig. 4. Thus, all coclusios draw from aalysis of the plots i Fig. 4 are also valid for the plots i Fig. 7. Agai the estimates are quite accurate except the estimates for several images metioed earlier which are either textural or for which clippig effects are observed. The results for 7x7 blocks are similar to those oes preseted i Fig. 7.

8 3 5 Blue compoet Gree compoet Red compoet Test image idex Fig. 7. Noise variace estimates obtaied by the method [1] with ooverlappig 7x7 pixels block size for red, gree ad blue compoets for the test image set corrupted by spatially ucorrelated additive oise with σ =13 Let us preset some results for the method [18] that belogs to the secod group of blid methods. Fig. 8,a shows the estimates obtaied for i.i.d. oise with variace equal to 65. I geeral, they are similar to those oes preseted i Figures 4 ad 5. Larger estimates are observed for images that are more textural. Almost all estimates are larger tha the true value. Approximately half of them are larger tha the upper boud of the limits cosidered appropriate (5 78). There are two exceptios whe the estimates are smaller tha the lower boud. They are observed for greed ad red compoets of the th test images for which the clippig effects have impact o the fial estimates. It seems that clippig effects caot be easily take ito accout i the method [18]. Fig. 8,b shows the estimates for the case of i.i.d. oise with σ = 13 for the method [18]. They are at similar level as those oes preseted i Fig. 7. Agai, large estimates are observed for textural images 1, 5, 8, 13, 14, 18. Meawhile, there are quite may estimates withi the required limits from 14 to 156. Due to clippig effects, the estimates for the gree ad red compoets of the th test images are cosiderably smaller tha they should be Blue compoet Gree compoet Red compoet 4 18 Blue compoet Gree compoet Red compoet Test image idex Test image idex a b Fig. 8. Noise variace estimates obtaied by the method [18] for red, gree ad blue compoets for the test image set corrupted by spatially ucorrelated additive oise with σ =65 (a) ad σ =13 (b) The method [] modified i [31] for which removal of clipped areas has bee doe produces cosiderably more accurate estimates. They are preseted i Fig. 9,a for oise variace equal to 65 ad i Fig. 9,b for σ = 13. As see, oly for two compoets of the highly textural 13 th test image the estimates are out the required limits but oly slightly.

9 85 8 Blue compoet Gree compoet Red compoet Blue compoet Gree compoet Red compoet Test image idex Test image idex a b Fig. 9. Noise variace estimates obtaied by the method [31] for red, gree ad blue compoets for the test image set corrupted by spatially ucorrelated additive oise with σ =65 (a) ad σ =13 (b) Cosider ow the methods based o maximum likelihood estimatio of oise ad image parameters [3, 33]. Sice here we are ot iterested i results of image parameters estimatio, let us cocetrate o the obtaied estimates of oise variace. Data for σ = 65 are preseted i Fig. 1,a. Fragmets with clippig effects have bee prelimiarily removed. As it is see, the estimates are mostly withi the required limits from 5 to 78. Besides, the obtaied estimates ˆ σ are slightly smaller tha the true value i most cases. Oly the 5-th ad, i some sese, the 13-th (blue ad gree compoets) ad 11-th (blue compoet) test images are problematic for accurate estimatio of oise variace. The reaso is that after dowsamplig these images become very textural Blue compoet Gree compoet Red compoet Blue compoet Gree compoet Red compoet Test image idex Test image idex a b Fig. 1. Noise variace estimates obtaied by the method [3, 33] for red, gree ad blue compoets for the test image set corrupted by spatially ucorrelated additive oise with σ =65 (a) ad σ =13 (b) The same method has bee also tested for σ = 13. The obtaied data are preseted i Fig. 1,b. For most cases, the obtaied estimates satisfy the requiremet to their accuracy (to be withi the limits from 14 to 156). The exceptios are the 5-th test image (all compoets), the 11-th ad 13-th test images (blue ad gree compoets). The reasos have bee explaied earlier. Summarizig the testig results preseted above, we ca state the followig. Although the cosidered methods have some differeces i their mai properties ad theory put ito their basis, they all mostly satisfy the requiremets to the provided accuracy of blid estimatio. However, the problem of oise variace estimatio i highly textural images as the test images 5, 11, ad 13 remais.

10 4.. Variace estimatio for spatially correlated oise Cosider the more complicated case of spatially correlated oise. Similarly to previous subsectio, let us start from the method [1]. Fig. 11 represets the plots for red, gree, ad blue compoets of the test images for oise variace 65. Let the block size be 5x5 pixels. Blocks are ooverlappig. Several mai coclusios ca be draw from aalysis of the plots i Fig. 11: 1) agai for most test images the estimates ˆ σ for differet compoets of the same color image are quite close; the oly exceptio is agai the -th test image; the reaso is the clippig effect discussed above; ) i most cases, the estimates ˆ σ are smaller or oly slightly larger tha the true value of variace, the oly exceptio is the 13-th, very textural test image ad, partly, the 14-th test image which is also textural; 3) for quite may cases, the estimates ˆ σ are withi the required limits from 5 till 78, although quite may estimates are out of these limits, commoly smaller tha the lower limit Blue compoet Gree compoet Red compoet Test image idex Fig. 11. Noise variace estimates obtaied by the method [1] with ooverlappig 5x5 pixels block size for red, gree ad blue compoets for the test image set corrupted by spatially correlated additive oise with σ =65 Thus, it is desirable to improve the performace of the method [1] for spatially correlated oise. Followig the recommedatio give i the paper [19], let us use a larger size blocks (7x7). The obtaied results are preseted i Fig. 1. As ca be see from compariso of the plots i Figures 11 ad 1, due to usig the block size 7x7 istead of 5x5 the accuracy has, i geeral, improved. I particular, the errors ˆ σ σ has decreased for all compoets of the test images 3, 4, 6, 7, 9, 1, 11, 15, 16, 17, 19, 1,, 3, 4, 5 ad some color compoets of other test images. The reaso for this improvemet deals with the followig property of ormal local estimates of oise variace. The maximum of their histogram is located for σ max < σ. But if the block size icreases, the differece σ σ max reduces as follows from the theory of variace estimatio for data samples of limited size. This pheomeo ca be also observed from aalysis of the histograms i Fig. 3 i the eighborhoods of their maximums.

11 5 Blue compoet Gree compoet Red compoet Test image idex Fig. 1. Noise variace estimates obtaied by the method [1] with ooverlappig 7x7 pixels block size for red, gree ad blue compoets for the test image set corrupted by spatially correlated additive oise with σ =65 Note that improvemet of accuracy due to usig the 7x7 blocks has take place for images that are ot too textural. At the same time, the accuracy for textural test images like the 5-th, 13-th, ad 14-th has worseed. I aggregate, for spatially correlated oise there exists differece what block size to apply: 5x5 or 7x7. The preseted results clearly show that 7x7 is a better choice. The same tedecies have bee observed for other values of oise variace, i particular, 13. Because of this, below i Fig. 13 oly the results for 7x7 block size are give. Mostly the obtaied estimates are withi the required limits. The obvious exceptios are the test images 13 ad 14 (which are textural) ad (due to clippig effect). The estimates are also ot accurate eough for some compoets of the textural images 1 ad Blue compoet Gree compoet Red compoet Test image idex Fig. 13. Noise variace estimates obtaied by the method [1] with ooverlappig 7x7 pixels block size for red, gree ad blue compoets for the test image set corrupted by spatially correlated additive oise with σ =13 Let us ow preset the results obtaied for the method [18] if oise is spatially correlated ad its variace is equal to 65. The plots are give i Fig. 14. Their aalysis clearly shows the drawbacks of this method it totally fails for the case of spatially correlated oise with rather wide mai lobe of D autocorrelatio fuctio. The same level of estimates is observed for the methods [] ad [31] (ot preseted i plots). This shows the problems of the methods that relate to the secod group.

12 8 7 Blue compoet Gree compoet Red compoet Test image idex Fig. 14. Noise variace estimates obtaied by the method [18] for red, gree ad blue compoets for the test image set corrupted by spatially correlated additive oise with σ =65 Beig applied without ay modificatio to blid estimatio of spatially correlated oise variace, the origial method [33] based o maximum likelihood estimatio also produces cosiderably biased estimates ˆ σ < σ. However, a alterative modificatio used i this paper that carries out resamplig (dowsamplig) a origial image performs well eough. The data obtaied for the proposed modificatio for σ = 65 are preseted i Fig. 15. The estimates are mostly withi the required limits. However, there are test images for which the obtaied estimates ˆ σ are obviously out of them. These are the 5-th ad 13-th test images for which the modified method fails. The reaso is that these images are textural. More i detail, the test image 13 is highly textural both o base scale ad after resamplig. At the base scale, the test image 5 cotais homogeeous fragmets but of very small area. After resamplig, liear size of these fragmets decrease below scaig widow size ad they are ot recogized as homogeeous. Thus, after resamplig the test image 5 becomes highly textural too. The outlyig estimate of oise variace is also obtaied for the blue compoet of the 11-th test image. There are two reasos behid this. The first reaso is that clippig effects i dark homogeeous areas that occupy rather large space of etire image are observed. The secod reaso is that after resamplig the image also becomes highly textural. Very similar results have bee obtaied for σ = 13. Therefore, some problems with highly textural images are typical for both origial ad modified methods based o ML estimatio. 1 Blue compoet Gree compoet Red compoet Test image idex Fig. 15. Noise variace estimates obtaied by the modified method [33] for red, gree ad blue compoets for the test image set corrupted by spatially correlated additive oise with σ =65

13 4.3. Problems of variace estimatio The aalysis carried out for three groups of methods for blid evaluatio of additive oise variace applied to TID8 images has demostrated the followig: 1) for most images which are atural (ot artificial), all three groups (uder certai coditios) are able to provide the required accuracy; ) the most complex situatio is with highly textural images especially if they are corrupted by spatially correlated oise with rather small variace; the it is difficult to fid regios or features by exploitig which it is possible to separate image iformatio cotet ad oise ad, hece, to estimate oise variace with proper accuracy; 3) aalysis of plots of oise variace estimates for differet images shows that their distributio is ot Gaussia (for a give variace of oise); outlyig estimates ca be observed for images which are, e.g., highly textural. This shows that it is desirable to desig special idicators of such image/oise situatios as well as to cotiue elaboratio of methods applicable to such situatios. There are several ideas that ca occur fruitful i this sese; i particular, recetly a ew etropy-like measure [35] able to characterize image cotet has bee itroduced, a coordiate of robust kurtosis-like parameter histogram maximum [36] seems to cotai iformatio o oise ad image properties, the parameter p used i the method [1] for characterizig the percetage of blocks belogig to image homogeeous regios might be useful; however, these ad other parameters eed additioal studies to give fial practical recommedatios. These are geeral coclusios. But there are also particular coclusios ad cosideratios cocerig aforemetioed problems. The first group of methods icludes quite may methods although oly oe of them is studied i details above. It is worth sayig that several methods that belog to the first group have bee desiged by the authors of this book chapter [1, 4, 14, 19, 1, 3, 37]. Durig several years we were steadily improvig performace of our methods by the followig directios. They iclude improvig the robustess of the estimators used at the fial stage [4, 1, 3], removal of abormal estimates by pre-segmetatio [19, 9], method parameter selectio (optimizatio) [19], etc. Comparisos have bee doe to earlier versios as well as to other methods of the first group like the method [17]. Compariso has bee also doe to the methods of other groups [19], e.g., to the method [18]. Aalysis has bee carried out for several stadard gray scale test images as Barbara, Baboo, Goldhill, Peppers, etc. Practically i all cosidered cases the accuracy provided by our methods was better or at the same level as for other techiques. The latest versio of our techiques is the method [19] that exploits image pre-segmetatio. Let us see does it produce some improvemets for the case where the method [1] has produced ot accurate estimates. Cosider the most problematic (textural) test images # 5, 8, 13, ad 14 for spatially ucorrelated oise with σ = 65. The results for the method [19] are preseted i Table 1 (the estimates obtaied without pre-segmetatio are give i paretheses for coveiece). As see, the use of presegmetatio leads to cosiderably more accurate estimatio. I may cases, the estimates have become fittig the required limits. Oly for the 13 th test image ad the red compoet of the 14 th test image the estimates are out the limits. After pre-segmetatio, oly a very small part of the 13 th test image is recogized as homogeeous. It is placed i the top part of the image where there is a small piece of sky, However, eve this fragmet is textural i blue compoet ad subject to clippig for the gree ad red compoets. These effects explai the properties of the estimates obtaied for the compoets of this image. Note that for the first group of methods [19, 1] we have ot applied detectio ad removal of clipped fragmets. Thus, aalysis carried out for the method [19] shows that pre-segmetatio is worth applyig. Table 1. Variace estimates for the test images # 5, 8, 13, ad 14, σ = 65, i.i.d. oise Chael Blue Gree Red 5 6. (89.8) 66.5 (96.9) 74.1 (91) (97.8) 77.8 (9.3) 61.7 (84.) (185) 5.3 (53) 38.4 (3) (9.9) 69.6 (15) 8.1 (98.3)

14 Oe advatage of the methods of the first group is that they are quite fast. Less tha 1s is eeded for applyig them to 51x51 image usig middle level moder computers to get the fial estimate eve if full overlappig of blocks is used. Let us cosider the method recetly proposed by Barducci et al [6]. I geeral, it caot be referred to ay of the three groups metioed above. This method is based o a ovel priciple: aalysis of oise i bit-plaes of cosidered images. Our prelimiary aalysis has show that the origial method [6] is very sesitive to clippig effects. If these effects are preset for eve a small percetage of pixels, the algorithms fails to perform appropriately well. The reaso for this is that each bit-plae becomes oradom i clipped areas due to saturatio. This violates the mai idea put behid the bit-plae method [6] that less sigificat bit-plaes are affected oly by oise ad are radom. Because of this, to improve the algorithm performace we had to itroduce the followig modificatio. First, we have detected clipped areas followig the rule described above i subsectio 4.1. The, these areas have bee removed from further cosideratio whe calculatig bit-plae radomess criterio δ ( k ) [6]. Thus, we avoid fial variace estimatio from beig affected by clippig effect. These modificatios have cosiderably decreased the algorithm sesitivity to clippig effects. However, the algorithm accuracy has ayway remaied ot well eough. To prove this, Fig. 16 presets the plots of oise variace estimates for the TID8 images corrupted by spatially ucorrelated oise with variace 65. Similarly, the plots for i.i.d. additive oise with σ = 13 are demostrated i Fig. 17. The followig coclusios ca be draw: 1) most estimates are cosiderably larger tha the true value of oise variace, oly few estimates are withi the required limits; ) especially large estimates (cosiderably larger tha they have to be) are observed for highly textural images. Aalysis for spatially correlated oise has bee carried out as well. The estimates are, i geeral, smaller ad more estimates satisfy accuracy requiremets. However, ayway the accuracy of the method [6] is worse tha the accuracy of the methods of the earlier cosidered groups. I particular, this follows from compariso of the plots preseted i Figures 18 ad Blue compoet Gree compoet Red compoet Test image idex Fig. 16. Noise variace estimates obtaied by the modified method [6] for red, gree ad blue compoets for the test image set corrupted by spatially ucorrelated additive oise with σ =65

15 Blue compoet Gree compoet Red compoet Test image idex Fig. 17. Noise variace estimates obtaied by the modified method [6] for red, gree ad blue compoets for the test image set corrupted by spatially ucorrelated additive oise with σ = Blue compoet Gree compoet Red compoet Test image idex Fig. 18. Noise variace estimates obtaied by the modified method [6] for red, gree ad blue compoets for the test image set corrupted by spatially correlated additive oise with σ =65 As it has bee already stated ad demostrated, the methods of the secod group fail if oise is spatially correlated. Oe way out is to recogize images corrupted by spatially correlated oise. This is still a ope problem. If it will be solved, the it will become possible to apply, for example, the methods of the first group if oise is recogized as spatially correlated ad to apply a secod group method, e.g., the method [31] if oise is i.i.d. Aother way out seems to be applyig dowsamplig of a origial image corrupted by spatially correlated oise as it occurred useful for the methods of the third group. The methods of the secod group are quite fast sice they are based o orthogoal trasforms that usually have fast algorithms ad o differet sortig operatios that ca be performed rather quickly as well. The mai advatage of the third group of methods is that they provide more accurate ad stable results for highly textural images, thus wideig the class of images for which reliable oise variace estimatio ca be obtaied. Yet, abormal biases for images 5, 13 ad 11 show that further advacig of these methods is eeded. Oe drawback of the third group of methods from practical poit of view is that they are rather slow i compariso to the most methods of the first ad secod groups. The reaso for this is extesive computatios for uderlyig texture parameters estimatio. This meas that the methods of the third group should be used oly for images with complex structure. I other cases, faster methods of the first or secod group are worth applyig. To implemet this idea i a fast ad blid way, some methods for image complexity estimatio are to be developed.

16 Let us characterize accuracy ad robustess of the cosidered methods by the parameter N out the umber of estimates that are out the required limits for 75 images (three color compoets of 5 test images for fixed variace of oise). These data are collected i Table. Their aalysis cofirms the mai coclusios give above. I particular, it shows that better accuracy (smaller N out ) is provided if oise variace is larger. The methods of the third group produce the best accuracy. The methods of the first ad the secod group are characterized by approximately the same accuracy for spatially ucorrelated oise with large variace, but if the oise is spatially correlated the methods of the secod group fail. Table. The values N out for differet estimatio methods ad oise properties Method Method Noise variace i.i.d. or spatially N out parameters correlated Of the first group Block size - 7x7, 65 i.i.d. 5 from the paper o-overlappig 13 i.i.d. [1] blocks 65 spat. correlated 14 Of the secod 65 i.i.d. 41 group from the 13 i.i.d. 5 paper [18] 65 spat. correlated 7 Of the secod 65 i.i.d. group from the 13 i.i.d. paper [31] 65 spat. correlated 75 Of the third group With image dowsamplig 65 i.i.d. 7 from the paper by two 13 i.i.d. 7 [33] times 65 spat. correlated 8 by Barducci et al Modified to avoid 65 i.i.d. 75 [6] clippig effects 13 i.i.d spat. correlated 49 If oise is spatially correlated, oe should keep i mid that blid estimatio of oise variace or stadard deviatio ca be oly a prelimiary step to further aalysis of oise properties. The ext step could be blid estimatio of oise spatial spectrum or D autocorrelatio fuctio. Oe approach how to do this is described i the paper [38]. Obviously, a priori iformatio o oise stadard deviatio allows determiig image homogeeous regios for which quite accurate estimates of oise spatial spectrum ca be obtaied. The, these estimates ca be aggregated (processed joitly i some robust maer) to produce more accurate estimates of spatial spectrum to be further used for image filterig, restoratio, edge detectio, etc. Coclusios ad future work I this chapter, we have tested several methods for blid evaluatio of additive oise variace that belog to differet groups (accordig to operatio priciples put behid them). Testig has bee performed for TID8 that cotais 5 color images with differet structure ad properties. Moreover, we have cosidered two variaces of oise as well as both spatially ucorrelated ad correlated oise. The testig has demostrated that although cosiderable efforts have bee spet i recet years o desig ad performace improvemet of these methods, the work is far from completeess. First, oly the methods of the first group ca be sought as those oes providig appropriate accuracy i about 8% of the cosidered situatios uder coditio that 7x7 blocks are used. The methods of the secod ad the third groups provide good accuracy if oise is spatially ucorrelated. However, if oise is spatially correlated this should be kow i advace to udertake the correspodig decisios ad modificatios. Thus, a particular future task is desig of methods for blid determiatio is a image corrupted by i.i.d. or by spatially correlated oise. The aalysis carried out has cofirmed oe more time that the most complex ad ufavorable practical situatio is whe a aalyzed image is textural, oise is spatially correlated ad has rather small variace. The it is the most difficult situatio i the sese of providig the required accuracy of oise variace estimatio i blid (automatic) maer. Note that i such situatios it is difficult to do this i iteractive maer as well sice image homogeeous regios ca be hardly selected. Sayig hardly, we mea that eve a experieced expert caot be absolutely cofidet that selected homogeeous fragmets are really homogeeous.

17 The study has also show that clippig effects have to be take ito accout. Although it is uderstood that this should be so, desigers of practical realizatios (codes) of the proposed methods ofte forget that clippig should be take ito cosideratio. Moreover, it ca be a good optio to idicate image regios where clippig takes place. This ca be useful for further image processig sice clippig ca ifluece differet operatios of image processig as edge detectio, classificatio, filterig, restoratio i a egative way. Although we have carried out compoet-wise aalysis of oise statistics of color images, oe iterestig observatio is the followig. If a estimate of oise variace is larger (smaller) tha the true value i oe compoet, the estimates i other compoets are, most likely, larger (smaller) tha the true value as well. I other words, the estimates i differet compoets of color images are highly correlated. This deals with two facts. The first kow fact is that compoets of color images are correlated [39]. The other fact is that although all methods of blid evaluatio of oise variace attempt to separate image cotet ad oise, image cotet ayway iflueces estimatio of oise parameters. Fially, we would like to metio the followig. All methods cosidered above are based o iitial assumptio that oise is additive with spatially ivariat (costat) variace. Although i may fudametal books (see, e.g., [4]) it is stated that this is a typical model of oise i color ad other types of images, this is oly a simplified model. Due to a set of operatios carried out with iput data i imagig systems, oise statistical properties ca be quite complicated [, 5, 41]. The, the use of blid estimatio methods adapted to pure additive oise to aalysis of images corrupted by real life oise might lead to uexpected ad hardly tractable results. I other words, we would like to say be coviced that oise is pure additive before applyig the methods cosidered above to real life images. Refereces 1. B. Vozel, S. Abramov, K. Chehdi, V. Luki, N. Poomareko, M. Uss, Blid methods for oise evaluatio i multi-compoet images, book chapter i Multivariate image processig, ISTE Ltd, Frace, 9 (i press).. A. Foi, Poitwise Shape-Adaptive DCT Image Filterig ad Sigal-Depedet Noise Estimatio: Thesis for the degree of Doctor of Techology, Tampere Uiversity of Techology, Tampere, Filad, Dec J.-S. Lee, K. Hoppel ad S.A. Mago, Usupervised Estimatio of Speckle Noise i Radar Images, Iteratioal Joural of Imagig Systems ad Techology, Vol. 4, 199, pp Zelesky A.A., Totsky A.V., Luki V.V. et al, Airbore multichael remote sesig data processig techiques ad software, Proceedigs of the Secod Iteratioal Airbore Remote Sesig Coferece ad Exhibitio ERIM, Sa Fracisco, CA, USA, Jue 1996, Vol. III, pp A. Barducci, D. Guzzi, P. Marcoioi, ad I. Pippi, CHRIS-Proba performace evaluatio: sigal-to-oise ratio, istrumet efficiecy ad data quality from acquisitios over Sa Rossore (Italy) test site, Proceedigs of the 3-rd ESA CHRIS/Proba Workshop, Italy, 11 p., March Barducci, A.; Guzzi, D.; Marcoioi, P.; Pippi, I. Assessig Noise Amplitude i Remotely Sesed Images Usig Bit-Plae ad Scatterplot Approaches; IEEE Trasactios o Geosciece ad Remote Sesig, Vol. 45, No 8, Aug. 7, pp Zlokolica V., Pizurica A., Philips W. Noise estimatio for video processig based o spatio-temporal gradiets, IEEE Sigal Processig Letters, Vol. 13, No 6, 6, pp Hyperspectral Data Exploitatio: Theory ad Applicatios, Edited by Chei-I Chag, Wiley-Itersciece, Luki V.V., Methods ad Algorithms for Pre-processig ad Classificatio of Multichael Radar Remote Sesig Images, Proceedigs of Iteratioal Coferece "Avaces e Ciecias de la Computacio", Mexico, Oct. 3, pp N. Reard, S. Boureae, J. Blac-Talo, Deoisig ad dimesioality reductio usig multiliear tools for hyperspectral images, IEEE Geosciece ad Remote Sesig Letters, Vol. 5, No, 8, pp K. Egiazaria, J. Astola, N. Poomareko, V. Luki, F. Battisti, Metrics Performace Compariso for Color Image Database, CD-ROM Proceedigs of the Secod Iteratioal Workshop o Video Processig ad Quality Metrics, Scottsdale, USA, 9, 6 p. 1. E. Vasteekiste, D. Va der Weke, W. Philips, E. Kerre, Perceived image quality measuremet of stateof-the-art oise reductio schemes, Lecture Notes o Computer Sciece, Spriger, Vol. 4179, 6, pp M.-P. Carto-Vadecadelaere, B. Vozel, L. Klaie, K. Chehdi, Applicatio to Multispectral Images of a Blid Idetificatio System for Blur, Additive, Multiplicative ad Impulse Noises, Proceedigs of EUSIPCO, Toulouse (Frace), Vol. III, pp ,. 14. B. Vozel, K. Chehdi, L. Klaie, V.V. Luki, S.K. Abramov, Noise idetificatio ad estimatio of its statistical parameters by usig usupervized variatioal classificatio, Proceedigs of ICASSP, 6, Toulouse, Frace, Vol. II, pp