The Unique Role of Lucis Differential Hysteresis Processing (DHP) in Digital Image Enhancement Brian Matsumoto, Ph.D. Irene L. Hale, Ph.D. Imaging Resource Consultants and Research Biologists, University of Southern California, Santa Barbara, CA Table of Contents Goals..Page 1 Problem of Spatial Detail Recovery..Page 1 Comparison of Sigmoid LUT Re-Map, Unsharp Mask and Lucis Processing. Page 4 Real World Examples Page 10 Conclusions Page 11
1 Goals Lucis is a new image-processing program that increases our perception of details in images; however, its acceptance in the market has been hindered by the belief that standard digital algorithms, such as the unsharp mask, a sharpening filter, perform a similar function. Considering that these algorithms are readily available in consumer-grade products, it has been argued that Lucis does not provide significant additional advantages. This turns out not to be the case. A detailed exploration of the effects of this algorithm and a rigorous comparison with the results obtained using the unsharp mask filter reveal that Lucis is capable of recovering greater information from the digital image. This paper will analyze the unique role of the Lucis Differential Hysteresis Processing (DHP) algorithm in digital processing of scientific images by providing a pictorial illustration of its power. It will not attempt to provide a detailed mathematical analysis; but, instead, will use test images to show the relative utility of the two image-processing methods. To simplify the illustration, this report will use a defined test-pattern image. Problem of Spatial Detail Recovery Image display has always been plagued with the problem of clearly and fully exhibiting the variations in light intensity present in the subject. In photography, it often was found that photographic films were able to record ranges of light intensity that could not be displayed in prints. In regions of highlight and shadow it was difficult for the viewer to discern the fine variations in gray tones. To display such subtle changes in intensity, the photographer employed analog processing to generate an enhanced photographic print. This was accomplished during printing either by underexposing the highlights (dodging) or by overexposing the shadows (burning in). Digital processing with the computer provides a more refined method by which these manipulations can be performed. Since each picture element (pixel) has a numerical value for intensity (gray value), the photographer can use defined arithmetic operations to adjust graylevel relationships. At a simple level, this can be done by generating a look-up table (LUT) in which the slope of the line or curve relating input and output gray levels is made steeper in regions of interest to make subtle changes in intensity evident. Such an operation is similar, in principle, to the analog processes of dodging and burning; however, it has greater precision because arithmetic adjustment can be defined exactly and its application is reproducible. Below is the basic image model that will be used in this paper to illustrate the effects of manipulating intensity values (Figure 1). It is a high-resolution, 8-bit grayscale image (1500 x 1000 pixels). The most important features are the simple and repeating line arrangements, which represent spatial detail. In the very center, a set of white lines forms an asterisk and a square. These lines are 4 pixels wide, whose maximum value is 255. They lie on a field of pixels whose intensity value is 0. To the left of the black central field is a series of alternating black and gray vertical lines. The finest vertical lines are 4 pixels wide and the lines become progressively thicker toward the left edge of the image. The gray lines possess an intensity gradient with black (0) at the top and white (255) at the bottom. On the right side of the image, a similar pattern is oriented horizontally, with the intensity gradient running from black (0) on the left to white (255) on the right. At the top of the black central field is a strip possessing a full gradient of
2 intensities, while at the bottom there is a step gradient of 16 densities. The finely inscribed lines and adjacent intensity gradients simulate spatial detail at various levels of resolution and contrast, as found in many scientific images. The virtue of the model is that it provides such detail in a defined and relatively simple pattern for objective analysis. Figure 1. Image model showing fine details and intensity levels from 0 to 255. The test image can be modified in a controlled way to simulate imaging problems that obscure detail. For example, if one generates a LUT to re-map the input grayscale range of 0 to 255 to a compressed output range of 0 to 15, the image appears very dark (Figure 2A). (Image brightness was increased after re-mapping to facilitate display.) Such an image simulates the effects of digital imaging under low light levels. In spite of the loss of brightness, the image retains a high degree of spatial information.
3 Figure 2A. Shadow image obtained by re-mapping pixels to bottom 16 gray levels. A similar exercise can be performed by adjusting the test image to concentrate all of the gray values in the bright end of the scale. For this example, the input grayscale range of 0 to 255 was re-mapped to the top 16 gray levels: 241-255 (Figure 3). As can be seen, the lines are faint, and again, much of the fine spatial information is hidden. This condition is equivalent to a highlight area that possesses subtle features. Such an overexposed region could result from capturing an image of a field in which some objects are much brighter than the surround. Figure 3. Highlight image obtained by re-mapping pixels to top 16 levels of brightness.
4 Comparison of Sigmoid LUT Re-Map, Unsharp Mask and Lucis Processing The applications for simple re-mapping, for example, applying a nonlinear LUT, unsharp mask and Lucis DHP are viewed by many customers as equivalent. Use of the shadow and highlight test images allows accurate observation and comparison of the effects of these algorithms, and makes it evident that their effects are not equivalent. First, a sense of their operation will be given by a verbal description. A simple re-map operation that can be used to enhance subtle intensity changes is a nonlinear gamma curve. For example, if one increases the slope of this curve for the darkest and lightest gray values and decreases the slope for the midrange, one generates a sigmoid LUT. Intensity variations are exaggerated in the shadows and highlights and reduced in the midtones. The unsharp mask algorithm is a digital means of implementing the photographic process of generating a blurred (unsharp) positive and then applying that as a mask to the original negative. The blurring at edges of structures or areas, when superimposed over the original, produces an edge-defining effect, which shows details. Functionally, this process is used to select the high spatial frequencies and recombine that component with the original image. In digital image processing, the operation can be accomplished using a convolution kernel to first, select the high spatial frequencies by blurring the original and then subtracting this low-pass filtered output from the original, and second, scale and add the high-frequency output to the original. Thus, the unsharp mask algorithm is used as a sharpening filter. This filter frequently is used after an image has been scanned into a digital file, since edges sometimes are blurred during the process of scanning, or after an image is resized by interpolation. In a similar manner, Lucis image processing can be thought of as generating two digital masks and then subtracting one from the other. In this case, modification of the original by hysteresis processing can either suppress or preserve local contrast variations. One contrast-modulated data set is subtracted from either a second modified data set or the original data, after which the data values are shifted so that all are positive and the gray-value range is expanded to fill the 0 to 255 gray scale. The algorithm operates on the principle that in order to emphasize fine contrast variations, it is necessary to sacrifice larger ones, and vice versa. The masking effect of Lucis processing is dependent on how extensively intensity values change along lines passing through each pixel at many angles. Selective blurring (removal) of contrast ranges is imparted by two factors. First, the size of a contrast window sets the amplitude range selected by the operation; the midpoint in the window is the output value. The window does not shift in amplitude in response to intensity values that lie within the window. Second, the sensitivity of the window s shift response to each intensity variation along the line is less if the intensity change is in the opposite direction as the previous shift (hysteresis). This is the critical difference between the Lucis and unsharp mask algorithms. The unsharp mask filter operates on spatial frequency information, while Lucis processing operates on amplitude information it could be said to work in the contrast space. While both can be used to extract spatial detail, it is evident that the unsharp mask is useful primarily as a means of enhancing edge definition, whereas Lucis can be used to enhance subtle regional contrast changes by suppressing the extreme changes.
5 The effects of processing the shadow test image with the unsharp mask algorithm are illustrated in Figure 4A. The Unsharp Mask filter in Photoshop (Adobe) was used with settings of Amount equal = 400%, Radius = 3 and Threshold = 0. (Image brightness then was increased to facilitate display.) This treatment with the Unsharp Mask extracted some of the spatial detail. The edge sharpening effect generated light borders that make the lines visible to the observer. But, as expected, the resultant image does not have a full intensity profile. Detail in the gradient strips is largely obscured. Figure 4A. Unsharp mask filter sharpens image by enhancing the edge definition of lines.
6 Figure 4B. Histogram stretch of Figure 4A. Performing a histogram expansion after applying Unsharp Mask generates the image in Figure 4B. Now the edge-sharpening effects produced by unsharp masking can be better appreciated. If one studies the edges of the vertical and horizontal bars adjacent to the black field, one can clearly see light borders. In addition, dark borders separate the gray patches in the step gradient, and horizontal, light borders demarcate the gradient strips. Note that when the unsharp mask filter is employed prior to histogram expansion, the border or halo artifacts it generates are exaggerated by re-mapping. Most importantly, it is evident that gradient information still is poorly recovered, although the image now contains gray values up to 255. The halos and the central thin lines comprise the brightest pixels. When Unsharp Mask is applied to the highlight test image, using the same settings as above, one obtains the image in Figure 5. Again, the unsharp mask filter does extract high-frequency detail, but the contrast is not improved. In this case, since the starting image was very light, dark edgeenhancement borders are most evident. Figure 5. Unsharp mask filter sharpens image by enhancing the edge definition of lines. In contrast, if the highlight test image is processed by Lucis DHP, the image in Figure 6 is obtained. For these exercises, the small cursor was set at 1, the large cursor at 17. A small cursor setting of 1 gives the original image. A large cursor setting of 17 blurs small contrast variations. When the large cursor result is subtracted from the small cursor result and a histogram stretch is performed, slight contrast variations are heightened. This includes most of the variation present in the highlight image. This processing recovered the spatial detail and a full gray-value range, similar to re-mapping of the image by a histogram stretch alone. The gradient information is well recovered. Only the local contrast where the lightest pixels are nearby the darkest is
7 somewhat reduced, for example the lines and surrounding dark central field are no long pure white and black. Figure 6. Lucis DHP reveals the spatial data of the highlight image. The same result occurs if the shadow image is processed. It is significant that an identical result is obtained if the shadow test image is processed using the same settings (data not shown). This is because hysteresis processing operates on the differential in intensity along lines through each pixel. The method by which the highlight and shadow images were generated produced proportionately equivalent intensity variations. Thus, a single setting allows Lucis DHP to recover the same spatial information and display it identically for each. In the previous section, the test image was subjected to single-pixel processing to illustrate highlight and shadow regions of an image that contain hidden detail. In essence, those single images represent the problems that arise when the dynamic range of the subject cannot be captured by the digital sensor. If the digital imager has properly acquired the image and has obtained a full grayscale, some of the pixels already are at full brightness and full darkness but areas still are over or underexposed. To illustrate this point, a model image was constructed that emulates an image properly captured to digital format and containing details in the highlight as well as the shadow regions. This multipart test image is shown in Figure 7. The image is a composite, comprising the overexposed image in the upper-left quadrant, the underexposed image in the lower-right quadrant and the original model image in the two remaining quadrants for reference. It is an 8-bit grayscale image. The thinnest lines are 4 pixels wide. Because it is impossible to expand the gray-level information in any region by a simple histogram stretch of the whole image, processing needs to be done on selected regions.
8 Figure 7. Test image that simulates regions of highlight (upper- left quadrant) and shadow (lowerright quadrant) within a properly captured (exposed) digital image. If one applies a sigmoid LUT to this multipart test image, one obtains the image in Figure 8. Expanding the output gray-value range for the highlights and shadows brings out spatial detail in those regions, but the gradient information still is not adequately displayed. It is difficult to generate an appropriate LUT for this type of image. As discussed above, the unsharp mask algorithm, by itself, cannot handle these extremes in exposure. If the Lucis DHP algorithm is employed in the attempt to enhance subtle intensity changes, its advantages are strikingly apparent. Lucis processing with a small cursor setting of 1 and a large cursor setting of 17 produced the image in Figure 9. The recovery of spatial detail in the upperleft and lower-right quadrants is dramatic. The asterisk and its field are restored, the thin black lines can be distinguished from the darkest sections of the neighboring gray lines and the 16 gray steps in gradient regions can be discerned. In contrast to the result achieved by altering the gamma curve (Figure 8) or by unsharp mask filtering (Figures 4A and 5), all four quadrants now utilize the full 8-bit gray scale. As expected, the larger contrast variations in the gradient regions of the upper-right and lower-left quadrants have been suppressed. Although gray values are not identical to those in the original single test image, relative relationships are well recovered. Similarly, the sigmoid LUT compressed intermediate intensities, but without the degree of spatial recovery in the highlight and shadow regions.
9 Figure 8. Applying a sigmoid LUT partially recovers data in the highlights and shadows. Figure 9. Lucis DHP dramatically displays spatial details in both highlights and shadows.
10 Real World Example Dinoflagellates are good examples of a subject containing structures that are obscured in shadow and highlight regions. Their large single cell has a network of delicate cytoplasmic strands (low optical density) surrounding a cluster of chloroplasts (high optical density). The organism was photographed using brightfield illumination and an Optronics MagnaFire camera (24-bit RGB image, 1280 x 1024 pixels). Photoshop (Adobe) was used to perform the sigmoid LUT re-map and Unsharp Mask filtration (Amount = 400%, Radius = 1.5 and Threshold = 3). Lucis DHP settings were small cursor = 1, large cursor = 29. Figure 10 shows the results of these processing methods. Of the three methods, the Lucis processed image most clearly defines the transparent reticulum of the cell and reveals details within the dark central mass of the chloroplasts. In comparison, the unsharp mask while showing some details in the dark mass fails to reveal the full extent of the central fissure that divides the mass into two halves. Additionally, Lucis shows fine details, such as varicosities, within the transparent strands at the highest contrast over background of the methods.
11 Figures 10. Comparison of the effects of processing a digital microscope image using a sigmoid LUT re-map, Unsharp Mask filter or Lucis DHP. The raw image is properly exposed, but the full dynamic range of the specimen could not be displayed. Lucis processing reveals features of a dinoflagellate, including transparent cytoplasmic strands and densely packed chloroplasts, that are not fully displayed by the other methods. Conclusions The sigmoid LUT, unsharp mask and Lucis DHP processing methods all allow display of the fine lines that are principle elements of the test image. This may appear to raise questions about the utility of Lucis relative to a full-featured image-processing program, such as Photoshop. One might suspect that the other methods are sufficient for most purposes, making Lucis an expensive add-on program. However, this argument does not take into account the striking difference in the degree of recovery of spatial detail by these methods.
Two aspects of Lucis DHP make it a powerful tool that possesses advantages for extracting spatial detail in a properly exposed image with fine features hidden in highlights and shadows. Although a sigmoid LUT re-map can enhance slight contrast variations in shadows and highlights, the expansion of the difference between darkest and lightest intensity values in these regions still is confined to a limited range. Spatial detail is only partially recovered. Regions of interest often must be treated separately to achieve optimal display of information. The unsharp mask filter s design for edge enhancement makes it unable, by itself, to handle extremes in over or underexposure. Edges may be enhanced, but the gray scale is not expanded, making it difficult to detect the sharpening effect. These handicaps are not shared by Lucis. First, it can more completely extract detail in the highlight and shadow regions simultaneously, because it operates on relative intensity information to enhance contrast locally. Lucis can be viewed as an algorithm that provides a balanced presentation of spatial information in all regions of the gray scale. Second, it can substantially expand the gray scale for regions of subtle contrast variation, by suppressing larger variations. Additionally, both advantages can be implemented in a single processing step. 12