Digital Image Analysis and Enhancement

Transcription

1 Digital Image Analysis and Enhanement S. Battiato, G. DiBlasi, G. Gallo, A. LoGiuoo, M. Madonia, S. Niotra, F. Stano DIPATIMENTO DI MATEMATICA ED INFOMATICA UNIVESITA DI CATANIA {battiato, gdiblasi, gallo, logiuoo, madonia, sniotra, Abstrat. This note reports the main researh results of the Imaging and Graphi esearh Group in the field of image analysis and enhanement. Partiular emphasis, reporting also some major details, is given to methods and tehniques devoted to unsupervised image analysis and lassifiation, automati text disrimination, adaptive interpolation and zooming, olor and ontrast enhanement. Introdution This report desribes researh ativities arried out by the Imaging and Graphi esearh Group woring at Dipartimento di Matematia ed Informatia of the University of Catania. Two main topis have been mainly foused: image proessing (analysis, enhanement and ompression) and pattern reognition (unsupervised lassifiation) both applied to typial mahine vision tass. esearh ontrats have been ahieved by means of Progetti d Ateneo, ooperation with STMiroeletronis-Catania and by MIU projet Trattamento delle Immagini Digitali, luster 15. Two main lines of researh have been followed: image analysis (medial, sat and aerial) and image enhanement of digital pitures aquired by onsumer devies. Of ourse, the best results in terms of quality enhanement an be obtained by using effetively all the information olleted in the analysis phase. The rest of the paper will desribe main results obtained in the various topis; some slightly details will be reported only for two reent tehniques regarding respetively aerial image lassifiation and expeted olor rendition. Image Analysis Publi and private organizations involved in the aquisition of aerial and satellite images, need suitable analysis tehniques to effetively tae advantage of the great amount of raw information. The speifi expertise derives from partiipation in European projets and by various partnership with ompanies operating in the GIS area ([2]) The analysis of aerial images presents some speifi harateristis. There exist large databases of these images, produed in the past deades using analogial devies. These images are typially aquired storing only the luminane hannel. The analysis, lassifiation and segmentation annot be performed using lassial multihannel tehniques already available for satellite images. For these reasons to perform the lassifiation of these images, we introdued some features oming from the textural analysis: Laws filtering, morphologial analysis and Gabor filtering. These features have been the starting point for different lassifiation strategies, partially based on some previous wors in the textural analysis and proessing ([1],[3],[9]). It has been used a real ase study: the analysis of a temporal sequene of aerial images of the Stelvios natural par. The first investigated tehnique is based on the learning of a membership fuzzy funtion relative to different territorial prototypes previously extrated from some test images by an human expert. The features used for the lassifiation are the Laws filters. Using Laws filters for gradient evaluation, the overall analysis is not influened by the luminane variation, obtaining also a good invariane with respet to rotation and translation. The gradient omputation allows to manage the miro-struture of the textures present in the neighborhood of eah pixel. The final lassifier shows performane similar to the state of the art for this ind of problem, as an be seen in Figure 1. The robustness of the system is also inreased from the fuzzy (not risp) lassifiation. The algorithm is very effiient whenever it requires a reliable set of prototypes used as referene ([12]). Together with the fuzzy approah, another tehnique is atually under development. This method is mainly based on bayesian models (i.e. probabilisti distanes derived from Gabor filtering). In partiular using a suitable set of diretional Gabor filters we obtained a good lassifiation on the same set of images used in previous tehnique ([13]). To have a robust lassifier for temporal analysis and segmentation on images aquired in different deades, with different sensors and limati ondition, the generalization of these tehniques is an important point of our researh. eently, we extended the above strategies to the lassifiation of bad mirohip. We obtained very good results in the field of the disrimination of doument images ([6]). By properly using a blo partitioning of a digital image (typially aquired by a onsumer devie) we have designed a proessing pipeline able to

2 filtering the input data isolating in an effetive way the text areas, without using lassial omplex methodologies and tehniques suh as segmentation, textual analysis, et. To disriminate the semanti ategory of an image text allows to selet a suitable pipeline able to outperform the enhanement and ompression steps. For example this ind of image an be effiiently ompressed using an ad ho indexed palette ([11]). Original Image Automati Classified Image Manually Classified Image Figure 1: Some examples of lassified images obtained with our strategy. First olumn is the original image, seond olumn is the automati lassified image, third olumn is the manually lassified image. Image Enhanement Due to the inreasing apaity, both in terms of proessing power and memory resoures, the enhanement of images aquired by onsumer devies (Digital Still Cameras, PDA, Mobile phone, et.) an be designed using advaned ad-ho tehniques. The relative algorithms are implemented in an embedded way, aording to the speifi HW platform. The enhanement proess is devoted to improve the global visual appearane of the input data, woring on some speifi peuliarities (resolution, visual artefats, ontrast, sharpness, hromatiity, ). It has been developed an adaptive interpolation algorithm able to outperform the lassial zooming algorithm (biubi, bilinear, et.) with respet to the overall pereptual quality. The loal adaptiveness taes into aount magnitude and diretion of the loal gradient to properly guide the interpolation proess ([4]). In ooperation with the AST (Advaned System Tehnology) researh group of STMiroeletronis of Catania, we have investigated about the performanes of several lassi and advaned zooming tehniques in the speifi ontext of Super-esolution, reporting a series of omparative studies ([5],[8]). Among others, it has been also designed an enhanement proessing pipeline devoted to mas typial artefats of images aquired by low-ost imaging devies ([7]) and a more advaned tehnique able to enhane the global ontrast appearane, maing use of a loal analysis able to detet some visual signifiane areas (e.g. texture, edge, sin, et.) needed to properly tuning the global ontrast setting ([10]). A new enhanement algorithm semanti based has been reently proposed ([14]). While most enhanement tehniques are ompletely blind to sene ontent, the proposed tehnique aims to improve the visual quality of natural sene images by strongly relying on atual, and expeted, image olour appearane. The design of the algorithm has been preeded by the olletion of reliable statistis on sample images, to be used for the purpose of image lassifiation, allowing at the same time the identifiation of target olours for the various lasses. It is based on a two

3 steps proess: a sene lassifier aimed to label eah pixel as belonging to a partiular hromati lass, followed by an automati olour orretion step with dynami range and intensity level preserving apabilities. A series of subjetive experiments, in whih involved people were ased about the pereived quality of the olour orreted images, onfirm the effetiveness of the proposed algorithm even when ompared with some of ommerial available tools. The algorithm an be viewed as omposed of two different proesses, image lassifiation and olor enhanement. The lassifiation analyzes the input image, and relying on experimentally derived statistis and rules, assigns pixels to a set of fixed semanti lasses. The lassifiation, after a further refinement proessing, is feed to the enhanement blo to drive the adaptive, modulated, lass based, olor rendition proess. The design of the algorithm relies on onsistent olor statistis for eah onsidered lass that have been obtained from a olleted sample images database. Figure 2 and 3 show a blo based desription of the algorithm. Input Puntual Expansion efinement Mas M image lassifiation Figure 2. A blo desription of the various steps employed to generate the lassifiation mas M. Mas M GB to rg transform Centroids of lasses Compute offsets Constrained Linear system Modulation Input image Output image Figure 3. A blo desription of the steps performed by the olor enhanement step. The mas image M is used both in the lassifiation and modulation phase. The proess, mainly driven by the olleted olor statistis, leading from lassifiation of the input image to the final pseudo image mas representing the membership, and the degree of membership of eah pixel to the various lasses, an be outlined as omposed of different steps (see Figure 2). Puntual lassifiation The image is first lassified on a per pixel basis using rules that have been easily derived from the olleted statistis. In order to avoid dealing with ambiguous values oming from de-saturated and/or low-lit pixels, only pixels satisfying the following ondition are onsidered: ( s L S > T ) ( L > T ) (1) where S, L are respetively saturation and lightness values for pixel in position, and T s, T L are experimentally fixed thresholds. The assignment to eah pixel P to the available lasses is handled by three mutual exlusive rules: (( H L ) ( H )) { sin, veg, sy} P lass (2) For sae of larity we hoose to ode with GB triplets the various lasses: (255,0,0) for lass sin, (0,255,0) for lass veg, and (0,0,255) for lass sy, oding with (0,0,0) the unlassified pixels. Thus in our onvention, G and B, are respetively representative of sin, vegetation and sy/sea. Figure 4 shows a sample image and the oded provided by puntual lassifiation step.

4 (a) (b) Figure 4. An input image (a), and the output of its puntual lassifiation (b). Blue and green pixels are representative of sy/sea and landsape. Expansion Puntual lassifiation is liely to be not always perfet, sine several pixels, even if identifiable by visual inspetion as belonging to the available lasses ould not be properly reognized due to high deviation from expeted hue values. In order to expand the results oming from the puntual lassifiation step, the mas is subjeted to a relevant low pass filtering step. The filtering has been performed by employing a Gaussian ernel, whih an be defined by Eq. (3): g 2 2 ( x + y ) 2 s ( x, y) e (3) Experimentally we have found that Eq. (3) needs very high s (spae onstant) values, thus maing filtering in the spatial domain omputationally expensive. To avoid this, the filtering is performed on a down-sampled mas image followed by suessive up sampling by means of bilinear interpolation. An example of suh filtering an be seen in Figure 5 (a). The sampling ratio and the ernel size where hosen to be proportional to input image resolution. (a) (b) Figure 5. The puntual lassifiation of Figure 3 after the expansion step (left), and the refinement step (right). efinement The refinement step produes the final mas M{, w }, indiating for eah pixel in position the lass to whih it belongs, and the degree of membership w.. Sine the filtering step will ause the results of the puntual lassifiation to overlap (e.g. multiple assignments will be available for the same pixel), a max rule is used to obtain one lass and one degree of membership for eah pixel. w { lass : max(, G, B )} (4) max(, G, B ) Figure 5(b) shows an example of mas. Adaptive Color Corretion The enhanement is aimed to redue the distane of olors belonging to the various lasses from the target values by means of proper, lightness preserving, olor shifting. The mas M{, w } is used to guide this proess, by assigning a lass related target to the lassified pixels, and by modulating the amount of olor orretion.

5 Color targets For eah lass (sin, vegetation, sy-sea) the targets were obtained by mapping the entroids of the olleted statistis on the rg (GB normalized) hromatiity plane. Given an GB olor, the mapping on the rg plane an be defined as: r + G + B G g + G + B (5) The omputed olor targets for eah lass will be indiated as (r,g ). Shift omputation After onverting the input image into the rg olor spae employing (5), the mean value on the olor plane of eah identified olor lass is omputed as follows: µ µ r g ( r : ) ( g : ) ard ard (6) with ard representing the ardinality of lass. For eah lass, the offset from the target olor is defined as: r µ (7) r g g r µ g Modulated olor enhanement The olor enhanement is arried out by shifting eah pixel value (r, g ) by the omputed offset and then onverting ba in the standard GB olor spae. The ambiguity, due to the one to many mapping, of the inverse of Eq.(5) an be advantageously used to define a lightness preserving, onstrained linear system: (8) r + r + G + B G + G + G 3 + B + B g + + G 3 g + B where (,G,B ) is the input olor for pixel, and (,G,B ) its output value. In order to avoid the appearane of unpleasant artifats and/or exessive olor distortions, the final olor orretion is modulated by using the omputed membership values w of the mas M, and two modifiable parameters a and b. The final values (,G,B ) are thus defined as follows:

6 " a + b a + b " ag + b wg G a + b " ab + b B a + b [ w + ( 1 w ) ] [ + ( 1 w ) G ] [ w B + ( 1 w ) B ] Parameters a and b allow to perform a linear ombination between original and olor orreted pixel values, while weights w derease or inrease the amount of orretion depending on the reliability of the lassifiation. This approah allows us to preserve the dynami range of the lassified regions avoiding also a naturalness modifiation. Two examples of input-output ouples relative to the entire proess are reported in Figure 6. The Figure also ontains an image of the absolute differene between the original and the proessed image, that shows how the orretion is spatial variant. (9) (a) (d) (b) (e) () (f) Figure 6. A landsape image (a), its enhaned version (b) and the differene image (). A portrait image (d) its enhaned version (e) and the differene image (f). Conlusions and Future Wors In this paper we have briefly summarized the main researh results of the Imaging and Graphi esearh Group, woring at Dipartimento di Matematia ed Informatia, University of Catania. Main topis have regarded the investigation of innovative solution for digital image analysis and enhanement. Some detail has been provided about a novel tehnique able to enhane the olour appearane of digital images. Future wors will inlude investigation about SVG raster to vetor onversion and advaned rate-ontrol tehniques for still/video appliation.

7 eferenes [1] S. Battiato, G. Gallo - Multi-resolution Clustering of Texture Images Chapter in Texture Analysis in Mahine Vision, Ed. M. Pietiäinen - Series in Mahine Pereption and Artifiial Intelligene - Vol. 40, pp.41-51, World Sientifi, Otober 2000; [2] G. Gallo, G. Grasso, S. Niotra, A. Pulvirenti - emote Sensed Images Segmentation Through Shape efinement In Proeedings of IEEE ICIAP 01, Int. Conf. on Image Analysis and Proessing, 2001 Palermo, Italy; [3] S. Battiato, G. Gallo, S. Niotra Glyph epresentation of Diretional Texture Properties 10-th International Conferene in Central Europe on Computer Graphis, Visualization and Computer Vision, WSCG 2002 Plzen, Czeh epubli, February 2002 Published in Journal of WSCG, Vol. 10, No.1-3, pp , 2002; [4] S. Battiato, G. Gallo, F. Stano A Loally-Adaptive Zooming Algorithm for Digital Images Image Vision and Computing Journal Elsevier Siene In. - Vol. 20, Issue 11, pp , September 2002; [5] S. Battiato, G. Gallo, M. Manuso, G. Messina, F. Stano - Analysis and Charaterization of Super-esolution eonstrution Methods In Proeedings of SPIE Eletroni Imaging Sensors, Cameras, and Appliations for Digital Photography, Vol. 5017B-37 Santa Clara, CA USA, January 2003; [6] N. Alessi, S. Battiato, G. Gallo, M. Manuso, F. Stano Low Level Feature s Set for Text Image Disrimination In Proeedings of IEEE-EUASIP Worshop on Non Linear Signal and Image Proessing NSIP 2003 Grado, Italy - June 2003; [7] S. Battiato, A. Castorina, M. Guarnera, P. Vivirito A Global Enhanement Pipeline for Low-ost Imaging Devies IEEE Transations on Consumer Eletronis Vol. 49, Issue 3, pp , August 2003; [8] S. Battiato, G. Gallo, F. Stano - Smart Interpolation by Anisotropi Diffusion In Proeedings of IEEE ICIAP 03 Int. Conf. on Image Analysis and Proessing pp Mantova, Italy September 2003; [9] S. Battiato, G. Gallo, S. Niotra - Pereptive Visual Texture Classifiation and etrieval In Proeedings of IEEE ICIAP 03 Int. Conf. on Image Analysis and Proessing pp Mantova, Italy September 2003; [10] S. Battiato, A. Boso, A. Castorina, G. Messina Automati Image Enhanement by Content Dependent Exposure Corretion EUASIP Journal on Applied Signal Proessing In press (2004); [11] S. Battiato, G. Gallo, G. Impoo, F. Stano An Effiient e-indexing Algorithm for Colour-Mapped Images IEEE Transations on Image Proessing In press (2004); [12] G. DiBlasi, G. Gallo, S. Moshetto Texture Classifiation in Aerial Photos - Preprint (2004); [13] S. Battiato, A. Carini, G. Gallo, M. Madonia Gabor filtering for Texture Classifiation in Geographial Data Preprint (2004); [14] F. Naari, S. Battiato, A. Bruna, S. Cariolo, A. Castorina Natural Senes Enhanement by Adaptive Color Corretion In Proeedings of IEEE ISCE 2004 Int. Symposium on Consumer Eletronis, eading (UK), September 2004.