Linear Regression Based Global Thresholding

Transcription

1 Linear Regression Based Global ing Khalid Aboura Centre for Built Infrastructure Research University of Technology Sydney 5 Broadway, Ultimo, NSW 27, Australia kaboura@eng.uts.edu.au Abstract A large number of thresholding methods exist. These methods perform differently in various image analysis problems where they are often used locally. They are not as effective globally. Global thresholding is a difficult task in most problems. We highlight a linear regression based global thresholding method that performed well in an engineering problem. This same approach can be used in biomedical applications where the environment is better controlled.. Introduction In image processing, there is almost always the need to segment the image, that is separate an object from the rest of the image. A natural way to segment is through thresholding, using the intensities of the pixels to create binary images. It is done by turning all pixels below an intensity threshold to zero and all pixels above that threshold to one. This reduction of information facilitates the application of mathematical algorithms in the treatment of the image for the localization of objects or assessment of characteristics of the image. This facilitation is important in image processing where volume, real-time processing, and reliability needs impose the requirement for automatic processing. ing can be effective. But this simple and easy to conduct operation requires a careful choice of the threshold. The almost infinite variety of pixel color combinations that can exist in an image presents a challenge in determining the threshold value. A large number of thresholding algorithms exist but their success is often local, where the algorithms exploit the local properties of the object being sought in the image. Effective global thresholding methods are rare. While success in any approach is measured within the context of the image processing task at hand, it remains that few global approaches are attempted. In this article we review briefly the operation of thresholding and see its importance in biomedical applications. We then highlight a global method that proved effective in an engineering problem. The approach uses linear regression on transformations of the colors of the image to produce a clear segmentation of the desired foreground. We conclude by encouraging the use of the approach in the treatment of biomedical images where the environment is better controlled. 2. ing Image thresholding is a classification problem where the pixels of an image are put into two classes: foreground and background. ing creates binary images from grey-scale ones by turning all pixels below some intensity threshold (gray level) to zero (black) and all pixels above that threshold to one (white). A digital image is a matrix of values consisting of the colors of the pixels. For a n m image, let I i,j represents the color of pixel (i, j), i =,..., n, j =,..., m. In the RGB (Red Green Blue) image color space, I i,j can take one of = 6, 777, 26 possible colors. This range creates an extremely large number of combinations of pixel values making up the image. To work in such image space makes it nearly impossible to have reasonable processing times. A reduction in dimension is unavoidable and it comes first in the form of a grayscale, or gray level, transformation. The color image is reduced to a grayscale image through the application of one of a number of transformations that can take the pixel value from [ 255] 3 to [ 255]. One such transformation is J i,j =.2989 R G +.4 B if I i,j = (R, G, B) where R, G and B are in [ 255]. This mathematical operation turns the color image into an image with shades of gray, where only the intensity of the pixel is shown. However, even in this gray-level, one dimensioned pixel space, the application of some search algorithms can be tedious and would require long processing times. In some image analysis problems such as the search for an object in the image, or the definition of a contour, the difficulty may be greatly reduced by switching to a black and white image, without too much loss in efficiency. The reduction

2 Level =.3 in information is compensated by the large reduction in the size of the search problem. In a black and white image, each pixel can have only one of two values, for black and for white, as compared to a whole interval [ 255], or 256 values for a gray pixel. When this difference is taken to a n m image, where n and m are in the hundreds, it makes a great difference. In a black and white image, it is possible to apply mathematical algorithms in realistic processing times. The operation of turning a gray-scale image into a black and white image is often done through a thresholding operation, by letting the pixel value be if it corresponding gray-scale value, or intensity, J is above some threshold T. Often, T is normalized and taken to be in the range [ ], and the intensities J of the pixels are compared to 255 T. Bellow this value, the pixels are blackened. Above it, they are whitened. To illustrate the thresholding problem, we take the image of cells used in biomedicine (Figure a) and taken from the image bank of the National Institutes of Health, US Department of Health and Human Services []. The actin (purple), microtubules (yellow), and nuclei (green) are labeled in these cells by immunofluorescence. We first take the image though a grayscale transformation using the formula J =.2989 R+.587 G+.4 B on the RGB components of the image. Then we use a well known thresholding method, the Otsu method [2] and apply it to the grayscale image to obtain the black and white image in Figure b. The (a) Original Image (b) Otsu ing Figure. Cell image (NIH []) Otsu method is a clustering-based method. The algorithm assumes two classes of pixels, say foreground and background. It then computes the optimal threshold that minimizes the weighted sum of within-class variances of the foreground and background pixels. This method gives satisfactory results when the numbers of pixels in each class are close to each other. The Otsu method still remains one of the most referenced thresholding methods. In the case of Figure a, it yielded an optimal threshold of To understand threshloding better, we vary the threshold T from its optimal value and set it to.,.3,.7 and.9. The resulting black and white images are show in Figure 2. Varying the threshold T from to takes the resulting image from an all white image to a completely dark (a) T=. (c) T=.7 (b) T=.3 (d) T=.9 Figure 2. Different thresholds one. There is an optimal threshold level that separates best the desired features in the image. In this example, the Otsu level of.4275 seems to provide a good black and white image, although good here depends on the context, that is the goal of the image analysis problem. Another biomedical example of thresholding is the automatic segmentation of the caudate nucleus (CN) region from human brain magnetic resonance images (MRI) [3]. The issue of automation is important in the analysis of MRIs. Manual segmentation requires significant time on the part of expert medical staff. In addition, manual inspection is prone to human errors [4]. In the case of this particular biomedical problem, existing solutions such as SnAP [5], a software package for CN segmentation available in the public domain, require manual input for a number of tasks including thresholding. This highlight the importance of thresholding. It is a significant step in any image analysis. All ensuing results depend on the effectiveness of the separation of pixels in the image before applying any search or localization algorithm. Undeman and Lindeberg [6] discuss the difficulty of segmenting MRI brain images. Figure 3 shows a 2D section of a MRI brain image and its corresponding Otsu thresholded binary image. Another biomedical image thresholding application is the skin lesion segmentation problem for automated early skin cancer detection and diagnosis [7]. Accurate segmentation is critical for early detection. 2. Local thresholding A common problem with global thresholding is the changes in illumination across an image. Parts of the image

3 second the ground truth (hand segmentation) and the third the segmentation result. (a) Original Image (b) Otsu ing Figure 3. MRI brain image (NIH []) appear to be brighter and some parts darker regardless of the objects being photographed. This illumination can be natural or man made and has to do with the angle of the shot, the time of the day and other factors, some random. This variation of illumination renders the application of a global threshold difficult. A commonly used solution is to apply thresholds locally. Instead of having a single global threshold, the threshold is allowed to vary across the image. A number of thresholding methods exist. They are classified by Sezgin and Sankur (24) [8] into i) histogram shapebased methods, (ii) clustering-based methods, (iii) entropybased methods (iv) object attribute-based methods (v) spatial methods that use higher-order probability distribution and/or correlation between pixels and (vi) local methods that adapt the threshold value on each pixel to the local image characteristics. An earlier survey is that of Sahoo et al. (988) [9]. Recently, Chang et al. (26) [] provide a survey and comparative analysis of entropy-based thresholding techniques. The thresholding methods effectiveness varies over the applications and new problems often necessitate the development of new thresholding techniques. New methods are constantly being developed (see [, 2]). 2.. Hysteresis thresholding The hysteresis thresholding approach [3] is a local thresholding method that can be used efficiently in a number of problems, for example in angiography where vessel imaging is conducted after the injection of a radiopaque substance [4]. Angiography is a widely used procedure for vessel observation in both clinical routine and medical research. Angiograms are X-ray images of blood vessels in situ with contrast medium in them. Vessel extraction from angiogram images is useful for blood vessel measurement and computer visualization of the coronary artery tree [5]. To segment the vessels in the image, the hysteresis thresholding is applied. It is a bi-threshold procedure typically used for two class object-background pixel segmentation. Figure 4 shows the segmentation results obtained for a retina image. The first column shows the original image, the Figure 4. The hysteresis thresholding approach (Condurache and Aach [4]) 3. Linear regression based thresholding Aboura (28) [6] introduces a linear regression based global thresholding for the segmentation of characters of a license plate in the image of a car. The method uses colorbased histogram classification information and leads to a 93.4% success rate. In the license plate recognition problem, thresholding is essential in all steps [7, 8]. However, the issue is often not mentioned, assuming an effective or at the very least an adequate foreground separation. In general, thresholding methods are not mentioned in the literature. A typical article reports a solution on one of the steps in the image analysis problem. The focus is more on the solution than on the thresholding method used for the data set of images. Some authors do admit to the difficulty, for example [7]. It is not simple to automatically determine a threshold value that would separate the letters from a license plate for any image. Brightness and illumination are factors that affect the performance of a binarization (thresholding) method. Some other authors, Anagnostopoulos et al.(26) [9], also find the use of global threshoding difficult to apply satisfactorily and instead choose to divide the image into blocks, and use a threshold for each block. ing locally each block of an image can be costly computationally, particularly when one is looking for an object that is much smaller than the image. In using a global approach, the computational savings are obvious. In this article, we highlight the approach of Aboura (28) [6] where the linear regression method is fast and its application renders the approach implementable in real time systems. 3. Major Color Histogram Piccardi and Cheng (25) [2] use image color information to introduce a Major Color Spectrum Histogram Representation (MCSHR) to identify a moving object in images. Using a normalized geometric distance between the

4 colors of two pixels in the RGB space, they rank the colors by creating clusters of colors and counting the sizes of the clusters. In doing so, they scale down the possible colors to a very limited number of major colors (for example 5 to ) without losing much accuracy on representing an object in an image. Colors within a given mutual distance threshold are dealt with as a single color. This approach results in a histogram that ranks major colors found in the image according to their frequency in pixels. Aboura (28) [6] uses the MCSHR metric to define a new thresholding approach. The approach mimics at first the Bayesian combination of prior information and image data to provide an estimate of the optimal thresholding level of the image, in what is called a Likelihood Global ing method. The author then refines the approach by applying a formal solution in the form of a linear regression solution. 3.2 Likelihood global thresholding A set of images was studied and the license plate regions in the images were identified visually and thresholded using the hysteresis method. When applied locally, the hysteresis threshold provides an optimal binary image, separating adequately the characters in the image. Then each license plate image was run through the straightforward thresholding where a level T [, ] separates the grayscale version of the image into two sets. T was varied from to, with. increments. The minimization of squared errors was used to select that T opt value that yielded a thresholded image that most resembles the result of the hysteresis approach. This T opt value is considered to be the optimal threshold level and provides a good binary image in all license plate region images. In the second step, the MCSHR is calculated from the image of a vehicle. The histogram MCSHR of the image is obtained, as described in [2]. Let c represent the vector of colors on the x-axis of the MCSHR histogram. These are the major colors found in the image. c is a vector of RGB colors. Let J be its grayscale transform, that is J =.2989 c(., ) c(., 2) +.4 c(., 3), where c(., ), c(., 2) and c(., 3) are the RGB colors, respectively, of the corresponding color c. J is then sorted in a descending order into u = {u,..., u n }, where u i > u j for i > j. Given the u i s, the image data is generated the following manner. Let Rs =.6,..., 2, with increments of.4. Let T (Rs) be the threshold value u m /255, such that m is the first i that satisfies Eq. (). m n i= u i > u i, () Rs i= Then these image sampled values, T (Rs), Rs =.6,.64,..., 2, are the image data that is combined with the prior information to generate a thresholded image. The prior information is the discrete probability distribution derived from the histogram of optimal thresholds, p = {p i }, such that p i = Prob(T opt = T i ), T i =,.,,...,. The image data T (Rs), Rs =.6,.64,..., 2 is rounded off to match the discrete sample space T i =,.,,...,, and its frequency is summarized in a probability distribution q = {q i }, such that q i = Prob(T (Rs) = T i ), T i =,.,,...,. In an ad-hoc manner that mimics the combination of likelihood and prior in a Bayesian analysis, [6] multiplies these two distribution in Eq. (2) to obtain a distribution Q, Q = p.q i p. (2) i.q i Using this distribution, the author generates an image he calls Likelihood ed image by simply thresholding the original image for each threshold level T i =,.,,...,, and multiplying the image with Q i = p i.q i / j p j.q j, then summing all the resulting images. This image has pixel values between and. To make it a fully thresholded image, he thresholds again, this time in the middle, by turning all pixels of value less than.5 into and the others into. This approach yielded good results. In about 89.8% of the cases, this thresholding gave a high level of separation of the characters in the license plate. However the method is ad-hoc and does not make full use of the color information contained in J. It uses the intensities found in the image, but not the frequencies at which these intensities exist in the image. To incorporate the full color information, a formal statistical approach is adopted by developing a thresholding method based on a linear regression model. 3.3 Regression global thresholding A Linear regression Y = β + β X β p X p + ɛ is used next by [6] to model the optimal threshold Y of an image with characteristics X,..., X p. Over 4 different explanatory variables were tried leading to a selection of 9 variables. The first source of explanation was the same one used to generate the image data in the likelihood thresholding method. Instead of varying Rs from.6 to 2 and sampling within, the intensity and therefore the threshold T (Rs) is taken at 6 values Rs = 4, 2.66, 2,.6,.33,.4. These variables provide the information about the intensities in the image, using the MCSHR. To use the full color information, the histogram values corresponding to the intensities given by the MCSHR are accumulated from the smallest intensity to the highest. Then the three intensities that provide the 25%, 5% and 75% of the accumulated sum are used as the remaining 3 variables in the linear regression. These 9 variables plus the intercept proved to be the most effective regression in the case of the data. A training set of 2 images was used to estimate the β parameters.

5 Figure 5 shows the regressed threshold within the training data. The red line is the plot of the optimal threshold for the 2 images of the training set. The black line is the corresponding Otsu threshold, and the blue line is the regressed threshold. One can easily see that the linear regression solution outperforms both a fixed threshold such as the T =.5 represented by the dashed line, and the Otsu threshold. Depending on its colors and brightness, an image may or may not be that sensitive to a change in the threshold value. Each of the thresholded images in the 3 cases was checked visually, for all original images in the training set. The regression method performed even better than reflected in figure 5, except for the obvious cases. Looking closely at some of these cases, the problem was identified as being that of excessive brightness in the image, due to the sun shining strongly on parts of the image. But in general, the regression method fits well the training data. Figure 6 shows the a 93.4% reliability figure. Figure 7 shows the example of a thresholded image using the regression approach. Figure 7. Regression thresholded image Value =.5 Otsu Method Optimal Regression Method Figure 5. s within the training data performance of the approach over the 8 images outside the training data for which the optimal threshold was computed. All images were inspected visually for each of the =.5 Otsu Method Optimal.8 Regression Method Figure 6. s for the validation data three thresholding methods. The regression approach yields 4. Application to medical imaging There is a wide variety of segmentation techniques proposed for medical imaging (see [2]). However, the problem of global thresholding remains open as not one single method can offer a satisfactory solution in all medical imaging applications. Global thresholding is fast and effective but often fails when there is a low contrast between the background and the foreground. The problem is further complicated in some cases where not one but multiple thresholds are sought for an image. For example, in the segmentation of MRI brain images, several tissues are sought in the image; white matter, grey matter, bone and cerebrospinal fluid. Undeman and Lindeberg (23) [6] discuss several approaches to brain segmentation and present a fully automatic method for segmenting the brain from other tissue in a MR image of the human head. While each method has its own merit, none provides a fully satisfactory approach to all problems. The reason we propose the use of the method of [6] in medical imaging is that the technique solved a problem in a different context where the environment is less controlled than in a bio-medical setup and yet performed reasonably well enough. Brightness and illumination are factors that affect the performance of a binarization method in license plate recognition. While this effect is observed in the reported methodology through its failure in a percentage of tested cases, some of the effects of brightness and illumination are attenuated by the regression model. For example, in studying the failed cases in the license plate example, we observed that () the image brightness is a factor that affect the global threshold in some images more than others, (2) the area of the license plate in the image is strongly illuminated in comparison to the rest of the image due to a random distribution of the sunlight and the position of the car and (3) the license plate wears

6 a protective sheet of plastic that is transparent but creates a reflection. These sort of illumination problems do not occur in bio-medical imaging or are better controlled. The power of the regression method could be well utilized in some thresholding problem in that context. 5. Conclusion Converting an image into a binary image has several applications in the biomedical field. There is a large variety of methods, but the problem remains. We highlight an approach based on the application of a linear regression model where the explanatory variables are color/intensity informative variables extracted from the original images. The method succeeds in the good segmentation of the image in an engineering problem where outdoor conditions increase the difficult of separating the foreground from the background. The purpose of this article is to introduce a thresholding method that can be considered a generic approach. The idea is is to bring the success of the method to the attention of researchers in the bio-medical imaging field. References [] National Institutes of Health, US Department of Health and Human Services, NIH Image Bank, [2] N. Otsu, A threshold selection method from gray-level histogram, IEEE Trans. System Man Cybernetics 9(), 979, pp [3] Y. Xia, K. Bettinger, L. Shen, and A. L. Reiss, Automatic Segmentation of the Caudate Nucleus From Human Brain MR Images, IEEE Transactions on Medical Imaging 26(4), 27, pp [4] W.-H. Chao, Y.-Y. Chen, S.-H. Lin, Y.-Y. I. Shih and S. Tsang, Automatic segmentation of magnetic resonance images using a decision tree with spatial information, Computerized Medical Imaging and Graphics 33, 29, pp. 2. [5] SnAP, University of Pennsylvania, Univ. North CarolinaChapel Hill, ETH Zurich, [6] C. Undeman and T. Lindeberg, Fully Automatic Segmentation of MRI Brain Images using Probabilistic Anisotropic Diffusion d Multi-Scale Watersheds, In L. Griffin and M. Lillholm (Eds), Proc. Scale-Space3, Isle of Skye, Scotland, Springer Lecture Notes in Computer Science 2695, 23, pp [7] X. Yuan, N. Situ and G. Zouridakis, A Narrow Band Graph Partitioning Method for Skin Lesion Segmentation, Pattern Recognition 42, 29, pp [8] M. Sezgin and B. Sankur, Survey over image thresholding techniques and quantitative performance evaluation, Journal of Electronic Imaging 3(), 24, pp [9] P.K. Sahoo, S. Soltani, A.K.C. Wong and Y. Chen, A Survey of ing Techniques, Computer Vision, Graphics and Image Processing 4, 988, pp [] C.-I Chang, Y. Du, J. Wang, S.-M. Guo and P.D. Thouin, Survey and comparative analysis of entropy and relative entropy thresholding techniques, IEE Proceedings Vision, Image and Signal Processing 53(6), 26, pp [] Y. Qiao, Q. Hu, G. Qian, S. Luo, W. L. Nowinski, ing based on variance and intensity contrast, Pattern Recognition 4, 27, pp [2] B. Saha and N. Ray, Image thresholding by variational minimax optimization, Pattern Recognition 42, 29, [3] J. Canny, A computational approach to edge detection, IEEE Transactions on Pattern Analysis and Machine Intelligence 8(6), 986, pp [4] A.P. Condurache and T. Aach, Vessel segmentation in angiograms using Hysteresis thresholding, Proc. of the Ninth IAPR Conference on Machine Vision Applications, 25, pp [5] D. Guo and P. Richardson, Automatic Vessel Extraction from Angiogram Images, Computers in Cardiology l25, 998, pp [6] K. Aboura, Automatic ing of License Plate, International Journal of Automation and Control 2(2-3), 28, pp [7] J.R. Parker and P. Federl, An approach to licence plate recognition, Computer Science Technical Report , The laboratory for Computer Vision, Calgary University, 996. [8] X. Zhang, L. Liu, Z. Chen and Z. Yuan, ing based on the fuzzy entropy and genetic algorithm of vehicle license plate, Proc. Fourth International Conference on Virtual Reality and Its Applications in Industry, Eds Jizhou Sun, Zhigeng Pan, Proceedings of the SPIE 5444, 24, pp [9] C.N.E Anagnostopoulos, I.E. Anagnostopoulos, V. Loumos and E. Kayafas, A License Plate-Recognition Algorithm for Intelligent Transportation System Applications, IEEE Trans. Intelligent Transportation Systems 7(3), 26, pp [2] M. Piccardi and E.D. Cheng, Multi-frame moving object track matching based on an incremental major color spectrum histogram matching algorithm, Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 5), 3, 25, pp [2] J. Rogowska, Overview and Fundamentals of Medical Image Segmentation, In Handbook of Medical Imaging: Processing and Analysis, By Issac N. Bankman, Academic Press, 2.