Thresholding and segmentation concerns in spray imaging

Transcription

1 Thresholding and segmentation concerns in spray imaging D. Sedarsky * Applied Mechanics Department Chalmers University of Technology SE Gothenburg, Sweden Abstract Continuing advances in fast digital detection, high-resolution imaging hardware, and new diagnostic methods have provided a wealth of valuable experimental imaging data for the study of spray breakup and atomization. The availability of higher quality visualizations of spray phenomena is an undeniably positive, but taking full advantage of these data requires automated image processing methods. In general, these analysis tools implicitly filter and distort the image source data and care must be taken to ensure that the assumptions required to apply the processing methods are valid. In this work, we briefly discuss the basis of a spatially resolved intensity signals subject to sampling and Nyquist limits, and examine thresholding, clustering, and segmentation of image regions for identifying spray fluid structures. Image data compiled for the study of spray breakup under well-controlled conditions using the Chalmers steady spray are analyzed to provide context for discussing concerns for basic thresholding and more sophisticated segmentation of spray images. * sedarsky@chalmers.se

2 Name Reference Comments Cluster Variance Maximization Otsu (1979). Exhaustive class optimization; Requires roughly uniform intensity. IsoData Ridler & Calvard (1978). Iterative heuristic method; Can be computationally expensive. Minimum Cross Entropy Li & Lee (1993). Requires good to moderate contrast. Maximum Entropy Kapur, Sahoo, & Wong (1985). Maximizes Shannon entropy subject to range constraints. Moments Preserving Tsai (1985). Determines grouping based on matching initial graylevel moments. Minimum Error Kittler & Illingworth (1986). Assumes normally distributed graylevels. Robust to unequal class counts. Percentile / Black Fraction Doyle (1962). Assumes FG and BG occupy a distinct portions of the intensity range. Fuzzy Minimization Huang & Wang (1995). Monitors and minimizes Shannon entropy. Table 1. Useful global threshold selection algorithms. Introduction Advances in fast digital detection, high-resolution imaging hardware, and new optical measurement methods have made a wealth of new experimental imaging data available for the study of spray breakup and atomization. However, taking full advantage of these data requires automated image processing methods which, in general, implicitly filter or alter the image source data and care must be taken to ensure that the assumptions underlying the applied image processing methods are valid. To this end, we present a limited survey of basic image signal considerations related to identifying background and fluid structure in spray images. We review approaches for determining global threshold criteria for binarization and segmentation applied to transillumination images of sprays, including a cluster-based segmentation scheme based on a k-means spatial distance function. Remarks on image data In modern imaging measurements, a light pattern created from the object of interest is focused by the optical system onto the image sensor. In the common arrangement, this image sensor is made up of a regular array of microscale light-sensitive elements whose measurements are combined to form a 2-D array of spatial data. Each so-called pixel (picture-element) produces one sample of light intensity which is filtered by the efficiency, dynamic range, and time-response of the sensor. The position and light collection geometry of each pixel together with the exposure and readout timings allow these individual intensities to be organized to represent a corresponding 2-D spatial extent. This is an explicit sampling of the continuous optical signal which is discretized in space and time. According to the Nyquist-Shannon sampling theorem: a n,i = f s,i w(x i ) sin[π f s,i(x i n/f s,i )] π f s,i (x i n/f s,i ) (1) and Fourier analysis, if the signal of interest, w(x i ) is bandlimited to B Hz, and f s,i 2B, then the sampled values give the orthogonal Fourier coefficients directly and the original continuous signal can be reconstructed with perfect fidelity [1]. Digital sampling of any signal with power in frequency bands above the Nyquist limit results in apparent signals (aliasing) which manifest as power in lower frequencies. It is possible for given discrete signal to arise from an infinite number of aliases, which correspond to all the possible sine waves that could have produced the set of sampled points. Thus, in general it is prudent to prevent aliasing by low pass filtering the input signal to attenuate all signal content above the critical frequency, B. To illustrate the issue of spatial aliasing, it is useful to consider a pure signal variation in just one dimension (as detected by a single row of pixels) and momentarily neglect the timing and response effects. Figure 1 shows

3 a sinusoidally varying continuous signal with points representing intensity sampling by a line of sensor elements with fixed positions. In this case, if the sampling frequency, f s, is less than twice the maximum frequency of the signal, then the signal is under sampled and it is not possible to determine and reconstruct the original continuous signal from the discrete samples. This point is illustrated in fig. 1, which shows two sine waves, one below and one above the critical (Nyquist) frequency for the sampling system. In this case, two different frequencies result in the same set of sampled points: the blue curve which is below the critical frequency is reproduced accurately, while the higher frequency green curve aliases the blue signal, giving the appearance of a lower frequency result. Figure 1. A set sample points (shown in red) correspond to both a properly sampled sine wave which oscillates with a frequency below the critical frequency (half the sampling frequency), and an aliased sine wave at a frequency higher than the critical limit. Each sensor pixel produces a point sample which is defined only at a point. Although the square pixel model is convenient to visualize, one should avoid thinking of discrete pixel values as little squares [2]. This abstraction is sometimes convenient for problems in geometry based computer graphics, but often introduces unnecessary artifacts and limitations. The actual image result should be viewed as a model-based recovery of continuous data from a set of discrete samples within a device specific range. The familiar pixelated view of the raw data actually constitutes a crude box-filter broadened reconstruction. While other approaches are possible, in the common arrangement for spray imaging we can assume the discrete data are regularly sampled on a Cartesian grid and the underlying continuous signal takes on the same values as the sampled data at the grid locations. Likewise, the model-based recovery from sampled points is assumed to be linear, such that the sum of two interpolated functions is equal to the interpolation of the sum of the two functions. Finally, for high quality imaging devices we assume a linear, shift-invariant response across the sensor, such that signal from one region of the active area is directly comparable to any other region. Under these assumptions, the image data constructed from the signal is firmly grounded to the imaging system optical transfer function (OTF) and the physical context which generated the optical signal. Aliasing can also lead to problems when repetitive patterns of high spatial frequency are sampled at low resolution. Intense moiré-like patterns can be formed by spatial intensity beating between superimposed regular periodic structures. Such structure can be present in the sampling function (as a result of the imaging hardware) as well as the imaged spray. Figure 2 shows such a moiré pattern resulting from a low resolution image of a water surface. Figure 3. (a). Grayscale colorbar showing 256 levels; appears continuous due to limited contrast sensitivity of human vision. (b). Grayscale color bar with 64 levels; bands of colors are perceptible due to the larger grayscale tone differences in adjacent bands. Figure 2. Low resolution image of water surface illustrating the formation moiré-like patterns arising from aliasing and interactions between regular periodic surface wave patterns. Another caveat which bears mentioning regarding the display and manipulation of grayscale image data is the limited ability for the human eye to differentiate similar graylevels. This is a consequence of the eye s nonlinear response to luminosity which allows it to cope with a wide range of intensities, but limits our ability to separate small differences in intensity. For most people, this limit is in the range of ~256 levels from pure black to pure white. Note that the color bar shown in Figure 3(a). is made up of 256 bands of different grayscale tones, but this banding is not discernible. Figure

4 3(b). displays wider bands of 64 tones, and the band structure is readily apparent. Thus, the perceived contrast when viewing an image differs from contrast computed from linear differences in luminance values [3]. Unsurprisingly, the display systems designed for humans to a large extent only show the level of color detail that we can actually see. This is an issue for viewing and visual interpretation of scientific measurement data which routinely record information without regard to the limits of human perception. Thresholding It is often useful to identify and separate regions of fluid features in spray images from the surrounding background field, for instance to determine spray angle, penetration depth, or to isolate different spray morphology. Thresholding is a very basic segmentation method which classifies image regions into foreground and background groups based on simple pixel value criteria. In grayscale spray images this is often accomplished by globally dividing pixels into groups based only on their intensity values. Naturally, when one threshold value is used, the range of image intensity levels is divided into two classes. This is a fairly standard step in many image processing algorithms used to obtain a binary image from a source image by applying an intensity threshold. The effectiveness of the threshold operation depends on the intensity distribution of the signal of interest; under the right circumstances this approach can select complicated features in an image with little computational effort. Factors which may complicate thresholding include limited contrast in source images, excessive random noise, widely varying illumination across the source image, and strong variance of graylevels within the foreground and background classes. When the background and foreground pixels are well-separated in intensity level, i.e. the distributions of values for each group do not overlap, and the threshold is set at a value between the two distributions this division is very accurate. In this case the pixels below the threshold will be correctly categorized as background, and pixels above the threshold will be correctly categorized as foreground. More commonly in spray images, however, the distributions of the two groups overlap, resulting in classification errors. A useful view of the global distribution of image pixel values is provided by an image histogram (a binned frequency plot of pixel graylevels). This plot is a bar chart which displays pixel counts summed over some bin size versus the graylevel value of each bin. Here the area of each bar indicates the number of counts for its corresponding bin. The image histogram doesn t include any direct spatial or temporal information, but it gives a clear picture of contrast and brightness, and can also indicate saturation or clipping of the image data. The image histogram is essentially an estimate of the probability density function (PDF) of the image data graylevels. Thus it follows that analysis of the histogram can in many cases provide justification for choosing a proper threshold value. Ideally, when a threshold is chosen for spray segmentation, it should have some physical basis related to the collected signal. A number of histogram based global threshold determination approaches can provide such a basis, provided the image PDF conforms to the requirements of the threshold procedure. A very simple approach can be applied for images with well-behaved PDF s in which the foreground occupies a distinct portion of the intensity range, i.e. fully brighter or fully darker than the background. Here the appropriate intensity percentile, which can be read directly from the cumulative probability function (CDF), is the proper threshold value. One of the most successful and widely applied global threshold computations is the cluster variance maximization approach by Otsu [4]. This approach minimizes the overlap of foreground and background regions by maximizing the within-class variance, i.e., the sample-weighted sum of the foreground and background variances. The procedure performs an exhaustive categorization of image pixel values, to find the globally optimal threshold. In figures 5, 6, and 7, the threshold results labeled Cluster correspond to the threshold computed by this method. The Otsu method performs reasonably well for all of these test images, but exhibits the best performance for the spray image of fig. 5. From the image histogram of fig. 5 one can see that here the image data is divided into a (somewhat lopsided) bimodal distribution with fairly even intensity across the image. Since these data conform fairly well to the underlying assumptions of the intra-class variance metric, the computed threshold segments the spray structure accurately. ISODATA is a common clustering heuristic that can be used to compute a global threshold criteria from image data [5]. Here the set of image data points are taken together with an integer k, which specifies the initial number of groups and a parameter limiting the maximum iterations of the procedure. The overall goal is to minimize what is termed average distortion the squared distance of each pixel to the center of its group. The Shannon entropy is a measure of the information content of a datastream, and can be used as a similarity indicator for comparing groups of data. H(x) = P(x i ) log b P(x i ) i (2) If we dividing the pixel values into foreground and background classes we can compute the absolute difference in H(x) for the two classes. When this difference is larger, the groups are less similar. Thus, by optimizing this difference we should be able to derive a globally op-

5 timal threshold according to this similarity metric. Figures 5, 6, and 7 show the results of this approach in the images labeled Entropy. This procedure can perform well on high-resolution images with lots of intricate texture, but it performs poorly compared to the other approaches for our raw spray test images. Figure 4 shows the result of the entropy procedure when we artificially equalize the histogram. Here the increased visibility of the texture improves the performance of the entropy metric substantially. avoid the temptation to tune image thresholds individually. In addition, the trends of the analysis result versus selected threshold should be examined, as sensitivity to the particular threshold value are often indicative of large uncertainties in the analysis results. Software resources Python skimage and matplotlib. ImageJ open-source scientific imaging suite. OpenCV computer vision API. Matlab image processing toolbox. Mathematica image processing functions. Nomenclature f s w(x) n sampling frequency intensity signal sample Subscripts i constituent dimension Figure 4. Entropy threshold procedure applied to compute global threshold value for a spray image after image histogram equalization (Diesel spray in high P&T chamber at 450 K and 30 bar). The Minimum Error method is a global threshold determination procedure that optimizes a mean-squared error criterion related to the pixel classification. It operates under the assumption that the proper class groupings are normally distributed with independent means and standard deviations. This approach has been shown to be robust to unequal class populations. Figures 5, 6, and 7 show threshold results from this method, labeled MinimimError. The procedure exhibits acceptable performance for all the test images, but the result in fig. 5 is especially noteworthy. In this case, the image histogram is heavily skewed with a large population occupying the lowest bin. This leads to classification errors in the Entropy (Kapur) and Cluster (Otsu) threshold computations; note the large circular band of background on the left side in the Entropy and Cluster results. Figure 8 shows the results of a threshold procedure accomplished by computing a 15-level k-means division of the original image and selecting a combination of the two largest connected components. This approach to clustering divides image data into k clusters each sample is grouped to the cluster with the nearest mean. In instances where a suitable computed threshold based on image graylevel statistics cannot be determined, an arbitrary user-selected threshold cannot be avoided. In this case, it is preferable to select a consistent value for analysis of the appropriate group of images and References 1. Couch, L.W., Modern Communication Systems, Prentice Hall, 1995, p Smith, A. R. A Pixel Is Not A Little Square! (And a Voxel is Not a Little Cube, Technical Memo 6, Microsoft Research, E. Peli, "Contrast in complex images," J. Opt. Soc. Am. A, 7, (1990). 4. Otsu, N., A threshold selection method from graylevel histograms, IEEE Trans. Syst. Man. Cybernetics, vol. 9, no. 1, pp (1979). 5. Ridler, T. and Calvard, S., Picture thresholding using an iterative selection method, IEEE Trans. Syst. Man Cybernetics, vol. SMC-8, no. 8, pp , (1978). 6. Kapur, J. N., Sahoo, P. K., Wong, A. K. C., A new method for gray-level picture thresholding using the entropy of the histogram, Computer Vision, Graphics Image Processing, vol. 29, no. 3, pp (1985). 7. Kittler, J., Illingworth, J., Minimum error thresholding, Pattern Recognition, Vol. 19, Issue 1, pp (1986). 8. Li, C. H. and Lee, C. K., Minimum cross entropy thresholding, Pattern Recognition, vol. 26, no. 4, pp (1993).

6 Figure 5. Image histogram and global threshold results of water spray from a 6 mm nozzle issuing into ambient conditions. The MinimumError approach outperforms the Entropy and Cluster Variance methods in on this spray image which exhibits large intensity differences in the foreground and background classes. Figure 6. Image histogram and global threshold results of diesel spray image in spray chamber at ambient conditions. Injection pressure is 800 bar.

7 Figure 7. Image histogram and global threshold results of diesel spray image in spray chamber at 450 K and 30 bar. Figure 8. Image histogram and global threshold results of diesel spray image in spray chamber at ambient conditions. Injection pressure is 800 bar. Original and labeled image computed by k-means connected component selection. The 2 Component Threshold represents the threshold result derived by combining the two largest labeled regions selected in the k-means grouping procedure.