Target identification performance as a function of low spatial frequency image content

Transcription

1 Target identification performance as a function of low spatial frequency image content Ronald G. Driggers Richard H. Vollmerhausen Keith Krapels U.S. Army Night Vision and Electronic Sensors Directorate Burbeck Road Ft. Belvoir, Virginia rdrigger@nvl.army.mil Abstract. Current imaging system performance models use either the minimum resolvable temperature difference or the minimum resolvable contrast concepts to predict target identification performance. Both of these performance functions describe the limiting frequency that can be viewed through the imaging system at a particular contrast. No credit is given to the system for the amount of low frequency (lower than the limiting frequency) information that is passed through the system. We determine whether the low spatial frequency information is important in the target identification task. Previous experiments show that no degradation is seen on character recognition if a high-pass, edge enhancing filter is applied to character images. This is not the case in target identification performance, where the targets of interest are military tanks. A number of filters (six levels of blur at four bandwidth configurations) are applied to tank imagery including high-pass filters to reduce the low frequency image content. A perception experiment is performed to determine whether target identification performance was degraded with a reduced amount of low spatial frequency image content. The probability of target identification is calculated from the observer responses and the identification performance is evaluated as a function of low spatial frequency image content. Low frequency information is shown to contribute to the overall system performance Society of Photo-Optical Instrumentation Engineers. [S (00) ] Subject terms: target identification; imager performance; spatial frequency. Paper received Oct. 18, 1999; revised manuscript received Mar. 17, 2000; accepted for publication Mar. 17, Introduction In current imager performance models, the limiting frequency determines the overall target acquisition performance of the imager. The limiting frequency is described by the minimum resolvable temperature difference 1 3 MRT or the minimum resolvable contrast 4,5 MRC and the corresponding intersection with target contrast. Consider the two MRT curves shown in Figure 1. Two sensors are shown: a platinum silicide PtSi sensor and a more sensitive indium antimonide InSb sensor. The InSb has less noise and passes low frequency information with higher fidelity than the PtSi sensor. However, both these sensors begin to limit the spatial frequencies in a similar manner at the higher spatial frequencies. The load line for a cool target is shown and describes the contrast of the target as a function of range. Where the contrast of the target becomes equivalent to the MRT is where the limiting frequency of the system is seen. Under the conditions shown, the limiting frequencies for both sensors are identical. No credit is given to the InSb sensor for passing higher fidelity low frequency information. Also, lower noise values cause the differences in the MRT curves at lower frequency to be more prominent. Typically, sensor models are based on the concept that high frequency image transfer describes the overall performance of a system. 6,7 Therefore, all sensors that pass the same amount of high frequency information are evaluated in a similar manner. This concept was reasonable when all sensors, say first generation IR imagers, were fundamentally the same in transfer function characteristics and noise characteristics. That is, the modulation transfer functions MTF and noise characteristics of these systems were about the same. With enhancements in technology and the large variations in sensor types, the shape of the MTF and amount of relative noise is becoming important and limiting frequency is no longer sufficient to describe the overall performance of the imager. 8 Take the case shown in Figure 2. Two sensors are shown with two different MTF shapes. The systems are not noise limited i.e., the noise levels are small so the highest frequency seen through the sensor is limited by the eye contrast threshold function CTF. Note that the limiting frequency is the intersection of the CTF and the MTF and is identical for both of these sensors. Under the current sensor performance concepts, the two sensors would perform in an identical manner, where it is obvious that the MTF on the right degrades frequencies lower than the limiting frequency much less than the sensor on the left. Sadot et al. 9 suggest a new target transfer model that considers the overall image-spectrum target received by the human visual system. Hadar et al. 10 suggest that the entire MTF be incorporated into the target acquisition model. In this paper, we are interested in whether the lower frequency information is really a benefit in the task of target identification. It has been shown that a high-pass filter 2458 Opt. Eng. 39(9) (September 2000) /2000/$ Society of Photo-Optical Instrumentation Engineers

2 Fig. 3 Filtering operation. Fig. 1 MRT and target load line determine the limiting frequency of the sensor. applied to characters before recognition tasks does not degrade the recognition performance. However, this is a specialized case, where the information is either black or white and edge enhancements are sometimes desired. In the identification task of military vehicles, many shades are present and many frequency components are present that are used in the identification task. A number of filters are applied to tracked armored vehicle imagery to determine whether the low frequency information was used in the target identification process. These filters were both low-pass and bandpass filters where the low-pass filters provided a baseline and the bandpass filters reduced the low frequency information. The imagery was presented to military observers and a probability of identification was obtained. The results are analyzed to determine the identification performance degradation with imagery that had been stripped of low frequency information. 2 Image Content Experiment An experiment was performed at Fort Knox during the month of September 1999 to investigate the degradation in identification performance with low frequency target content. The experiment was prepared at the U.S. Army Night Vision and Electronic Sensors Directorate NVESD and conducted at Fort Knox with 12 soldiers as imagery observers. The preparation included processing high-fidelity imagery such that target images were filtered, thus tailoring the spectral image content. The sensor model for this experiment is depicted in Figure 3. The input images were convolved with a blurring function as shown. Even though the blurring function was represented in a digital fashion, the input image was of such high fidelity that a large number of samples represented the blurring function. The images were then reconstructed for display. The MTFs used were of the Vollmerhausen shape Vollmerhausen after the person who first started using this filter at NVESD. This type is rectangular with a Gaussian cutoff at the end. Analytically, it is a rect function convolved with a Gaussian function, 11 described in Equation 1. The width of the Gaussian function was one third of the rectangle width to suppress ringing artifacts from sharp edges in the images. The baseline, or unfiltered, MTF is shown in Figure 4. The filtered MTFs are the difference of the baseline MTF and three narrower MTFs 1/2, 3/4, and 7/8 as wide. The resulting filtered MTFs are also shown. Numerically, the degradation of the images was accomplished in the spatial plane. The kernels used were impulse response functions related to the MTFs by the Fourier transform. The 1-D representation for the MTF of the Vollmerhausen shape is described by: MTF total rect a *gaus b, where the rect and gaus functions are defined by Gaskill 12 and the * denotes the convolution operation. The Fourier transform pair for this function is described by: h total 1 ab sinc x a gaus x b. The values a and b are the scaling factor in samples. Note that the area of the impulse response function is 1, and the 1 2 Fig. 2 Same limiting frequency with different MTF shapes. Fig. 4 Experimental filters. Optical Engineering, Vol. 39 No. 9, September

3 Fig. 6 Target set. Fig. 5 Filtered Images (a) with full BW, (b) with upper half of BW, (c) with upper 25% of BW, and (d) with top 12.5% of BW. area of the MTF function was 1/ab. The impulse response function in Equation 2 is the convolution kernel used to degrade the images. The space and frequency functions were written in one dimension. All images were processed in terms of two separable functions, where the vertical function was identical to the horizontal function. All functions were processed in the space domain, where the impulse response function or point spread function given in Equation 2 was convolved with the input image to give an output image. Examples of the degraded images used in the experiment are shown in Figure 5. Figure 5 a shows an image which has been blurred, yet has the entire MTF bandwidth BW. Figure 5 b shows an image that has been processed with the bandpass filter described as the upper one half of the frequency content. Figures 5 c and 5 d show the upper one fourth and one eighth images, respectively. Note that the full MTF BW image contains many lower spatial frequencies, where the bandpass filter images have eliminated many of these lower frequencies. Also note that as the BW converges toward the upper end of the MTF, that the image content that is left becomes more like an edge enhancer that outlines the target and objects in the image. The image set in this experiment was a group of 12 tracked armored vehicles Figure 6. The individual vehicles were chosen as both representative of typical battlefield types and for their confusability. There were views from 12 different aspects of each vehicle for a total of 144 different images. Each filter operated on every image once. Four different MTFs one full MTF and three bandpass degraded these images at six different scaling levels blur levels. The result was 24 cells of 24 images each. These cells were arranged randomly for presentation to the observers. The images within the cells were selected so there were two views of each vehicle in the cell; each cell had the same number of images at a particular aspect ratio and was randomized in order. The input imagery was pixels at 12 bits of gray levels, which was then down sampled spatially by a factor of 2. These images were taken with an Agema long wave IR radiometer with a mrad sample spacing. The distance to the targets was 125 m. Aspect angles of 0, 90, 180, and 270 deg were used for 0 deg elevation. Additionally, there were eight aspects at an elevation of 15 deg. Figure 6 is the right side aspect with 0 deg elevation for each of the vehicles in the set. The set were all tracked armored vehicles. Five of the vehicles were non-u.s. Army vehicles. The set included tanks, armored fighting vehicles, and self-propelled guns. The images were taken in the same place to minimize the use of scene based cues by the observers. All of the vehicles were driven into place, so the engine, exhaust and drive trains had operational thermal signatures. The baseline imagery was grouped into six cells to correspond to six levels of blur. The scaling factors used to cover a wide range of P ID performance were 5, 7.5, 10, 12.5, 15, and 17.5 samples. Table 1 illustrates the organization and naming convention for the image cells. Each row in the table represents a different filter configuration or degradation. The rows each contain 144 unique images. Table 1 Image cell layout and naming convention. Image Cells Matrix (Naming Convention) A Blur (7.5) B Blur (10) C Blur (12.5) D Blur (15) E Blur (17.5) F Blur (5) A, full MTF AA BA CA DA EA FA B, high 50% of MTF BW AB BB CB DB EB FB C, high 25% of MTF BW AC BC CC DC EC FC D, high 12.5% of MTF BW AD BD CD DD ED FD 2460 Optical Engineering, Vol. 39 No. 9, September 2000

4 Each cell contains 2 images of each vehicle and 2 of the same aspect i.e., the targets and aspects were distributed equally across all cells. The performance of each row can be directly compared to the other rows since they operated on the same images. During the testing, observers were shown one image at a time, in random presentations. The responses were timed, but the observers were given unlimited time to make their recognition choices. The menu consisted of 12 options presented in a forced choice method. The test required approximately 60 to 90 min of observer time. The observers, soldiers from the 2nd Armored Training Brigade, learned to identify the vehicles in the image set before the actual experiment. The recognition of combat vehicles ROC-V multimedia training program was the training package. Each individual trained until they were able to attain a correct identification level of 96% on a 50 image test. To reach this level depended upon the particular observer s background and ability, some requiring more time than others. Typical training time for this group was 2 to3h. The dimensions of the display monitor were in. and the viewing distance was approximated at 15 in. The target image size was 3 4 in. for a pixel image. This results in a spatial frequency limit of approximately 1.14 cycles/mrad. The contrast/level for each monitor was set prior to the experiment, using a black and white target square, to a maximum value of 70 Cd/m 2 for the white pixels. Using these dimensions and the display brightness, the contrast threshold at the monitor s limiting frequency was estimated 13 to be 0.5%. The monitor display pixel MTF was not a limiting MTF in the experiment since the blur levels were much larger than the display element size. This operation made the MTF of the monitor wide in the frequency domain compared to the other MTF widths 14 pixel very narrow in space. The monitors used for the experiment were tested for their linearity in intensity. There was a noticeable gamma applied to the outputs of the monitor. A correction was applied to linearize the output with respect to the input gray level. An 8 bar gray level target was displayed on the monitor. The gray levels in the image file were encoded in 12 bits. A luminance contrast meter was used to measure the output of the monitor for each gray bar. A linear luminance output was the goal, so a correction was applied to each gray level. 3 Results The probability of identification was determined for each of the filter cells described in the previous section. The results are shown in Figure 7. The full bandwidth MTF, or the MTF including the low frequency information, appeared to have a monotonically decreasing degradation with the amount of blur imposed on the image. The average probability ranged from 20 to 80%. A clear and consistent degradation was seen for the filter that included only the upper 50% of the full bandwidth MTF. It can be generally observed that further degradation was seen for the upper 25% and upper 12.5% filters. However, the degradation was seen more at the higher probabilities and less for the lower probabilities. That is, a higher difference was seen when the full filter corresponded to a high probability. When the Fig. 7 Experimental results. full filter was at low probabilities, eliminating the lower frequencies only degraded the performance a small amount. It made sense that when little information was present to begin with i.e., a low probability, then removing more changed the performance only a small amount. The standard error for the measurements taken in Figure 7 was around 4%. 4 Discussion A number of characteristics can be observed from the results of the experiment. The first and most important observation is that overall target identification performance degrades with a reduction in the low frequency content of a typical tank image. This characteristic supports the notion of a target acquisition model that changes with sensor throughput over all frequencies including low spatial frequencies. While the performance is likely to be dependent on the target power spectrum, the spectrum must be able to pass the sensor over all frequencies and the performance should reflect the sensor transfer function over all frequencies. Therefore, a parameter such as an MTF area-weighted metric 15,16 would be a better descriptor of target acquisition performance than a limiting frequency approach. The results describe only that there is degradation in performance with elimination of lower frequencies in the target image; that is, all of the target spectrum is used in the identification task. However, these results do not describe the relative importance of target spectral regions in the identification task. It could be that, while lower frequencies are used in target identification, they are not as important as higher spatial frequencies. Further experimentation would be required to determine whether the regions are not equal in importance. Currently, target acquisition models apply only to a general class of targets such as tanks in an aggregate sense. Target acquisition models do not accurately describe the field performance of a particular sensor against a particular target. It is the opinion of the authors that target acquisition modeling could be improved significantly if a model were developed based on the target power spectrum and its modification while passing through a sensor. 5 Conclusions The need for this work lies in the use of limiting frequency in current acquisition models to predict sensor perfor- Optical Engineering, Vol. 39 No. 9, September

5 mance. This approach is a simple single value descriptor, which historically has been adequate. With the introduction of new sensor technologies, sensor MTF curves and noise or MRT/MRC curves do not have sufficiently similar characteristics to be compared in this manner. While the location of the intersection between CTF and MTF may be the same for two sensors, the way they pass lower frequencies may be very different. Historically, high spatial frequency content has been deemed most important to the identification task and hence justified the use of limiting frequency. We questioned the current validity of this assumption and tested to determine if the lower frequency information benefited identification performance. The results were analyzed to determine the degradation in target identification performance with imagery stripped of lower spatial frequency content. Target identification performance degrades as the lower frequency components are removed. This degradation is more significant at lower blur levels than at higher levels. The degradation in performance is postulated to result from the complex nature of the military targets. This characteristic is different from characters, where recognition performance may not be affected as significantly by image low frequency content. We conclude that sensor performance metrics should be weighted with respect to the amount of low spatial frequency information passed by the sensor. This inclusion in the acquisition modeling process would improve the accuracy of the performance predictions. References 1. J. M. Lloyd, Thermal Imaging Systems, p. 184, Plenum Press, New York R. Driggers, P. Cox, and T. Edwards, Introduction to Infrared and Electro-optical Systems, Artech House, Boston G. Holst, Electro-Optical Imaging System Performance, pp , JCD Publishing, Winter Park, FL W. Frame, Minimum resolvable and minimum detectable contrast prediction for monochrome solid state imagers, SMPTE J., May W. Lawson, Electro-optical system evaluation, in Photoelectronic Imaging Devices, I, Vol. 1, p. 375, L. Biberman, ed., Plenum Press, New York R. Vollmerhausen, R. Driggers, and B. O Kane, Influence of sampling on target identification and recognition, Opt. Eng. 38 5, R. Vollmerhausen and R. Driggers, Sampled Imaging Systems, SPIE Tutorial Series, SPIE Press, Bellingham, WA R. Vollmerhausen and R. Driggers, NVTherm: next generation night vision thermal model, IRIS Proc. 1, Feb D. Sadot, N. Kopeika, and S. Rotman, Target acquisition modeling for contrast-limited imaging: effect of atmospheric blur and image restoration, J. Opt. Soc. Am. A 12 11, O. Hadar, S. Rotman, N. Kopeika, and M. Kowalczyk, Incorporating the entire modulation transfer function into an infrared target acquisition model, Infrared Phys. Technol. 39, J. D. Gaskill, Linear Systems, Fourier Transforms, and Optics, pp , John Wiley and Sons, New York J. D. Gaskill, Linear Systems, Fourier Transforms, and Optics, Chap. 6, John Wiley and Sons, New York I. Overington, Vision and Acquisition, Crane, Russak, & Co., New York R. Vollmerhausen and R. Driggers, Chap. 9 in Sampled Imaging Systems, SPIE Tutorial Series, SPIE Press, Bellingham, WA Dec Y. Feng, O. Ostberg, and B. Lindstrom, MTFA as a measure for computer display screen image quality, Displays 11 4, C. Infante, Numerical methods for computing modulation transfer function area, Displays 12 2, Ronald G. Driggers has 12 years of electro-optics experience and has worked for or consulted for Lockheed Martin, SAIC, EOIR Measurements, Amtec Corporation, Joint Precision Strike Demonstration Project Office, and Redstone Technical Test Center. He is currently with the U.S. Army s Night Vision and Electronic Sensors Directorate and is the U.S. representative to the North Atlantic Treaty Organization (NATO) panel on advanced thermal imager characterization. Dr. Driggers is the author of two books on IR and electro-optics systems and has published over 30 refereed journal papers. He is coeditor of Marcel Dekker s Encyclopedia of Optical Engineering and is an associate editor of Optical Engineering. Richard H. Vollmerhausen currently heads the Model Development Branch at the U.S. Army s Night Vision Lab (NVL). The branch is updating the U.S. Army s target acquisition models to include the effect of sampling on performance and to make other model enhancements to predict the performance of advanced technology sensors. During his tenure at NVL, he was a project engineer and electro-optical systems analyst for numerous U.S. Army weapon systems. His previous work included designing air-to-air missile seekers for the Navy and working as an instrumentation engineer for Douglas Aircraft on the Saturn/Apollo Program. Mr. Vollmerhausen is the author of two books on electro-optical systems analysis and has published numerous journal and symposium papers. Keith Krapels is an electrical engineering doctoral COOP, from the University of Memphis, at the Night Vision and Electronic Sensors Directorate. From 1990 to 1996, he was an electronic warfare officer in the EA-6B aircraft and from 1996 to 1998 a special project officer at the Tactical Electronic Warfare Division of the Naval Research Laboratory. His current interests include electro-optical, IR, and electronic warfare sensor modeling and countermeasures development and testing Optical Engineering, Vol. 39 No. 9, September 2000