Amplitude spectra of natural images

Transcription

1 Amplitude spectra of natural images D.. Tolhurst, Y. Tadmor and Tang Chao The Physiological Laboratory, University of Cambridge. Downing Street, Cambridge CB2 3EG, UK (Received 8 October 1991) Several studies have suggested that the amplitude spectra of photographs of tiatural scenes are remarkably similar and have the form: amplitude x spatial frequency"'" This is, of course, a straight line with slope of 1.0 when plotted on double logarithmic coordinates. We have examined the amplitude spectra of 135 digitized photographs of natural scenes and have found that relatively few images conform exactly to the suggestion. About 25yi> of the images in our sample have spectra which show significant curvature when plotted on log-log coordinates. The best-fitting regression lines have slopes that range from 0.8 to 1.5; the average slope is - 1.2, rather steeper than previously suggested. Images of natural scenes have immense diversity and yet several studies have suggested that the amplitude spectra of photographs of such scenes are all remarkably similar in form' '. The amplitude of each spatial frequency coefficient in the Fourier spectrum, averaged across orientation, has been reported to have the form: average amplitude x spatial frequency (1) It was noted in these studies that, in fact, not all the digitized pictures of natural scenes conformed exactly to this equation. However, the exceptions seemed to be infrequent and seemed not to be very marked, so that it has been tempting to ignore them in favour of retaining the benefits of generalization. This generalizing description of the amplitude spectra of natural images has several attractions, especially because the exponent has the particular value of 1.0. It implies that natural scenes contain the same amount of detail, regardless of the viewing scale, and that spatial frequency channels with constant logarithmic bandwidth* would be presented with the same energy, regardless of their optimal spatial frequency. Further, natural scenes contain objects with luminance edges; the amplitude spectrum of a single edge has a spectrum conforming exactly to Equation I. The supposed consistency of the amplitude spectra has made it possible to examine computationally the efficiency of the coding schemes implied by neurophysiological studies of the visual cortex^'^. It has also allowed the development of psychophysical paradigms for studying the processing of natural images by the human visual system'' ^. However, our psychophysical results^ imply that it may be an over-simplification to regard the amplitude spectra of all natural images as being nearly identical. We have, therefore, examined the amplitude spectra of 135 digitized photographs of natural scenes {many more than in the previous studies) in order to determine whether their amplitude spectra are, indeed, as similar as has been believed previously. The digitized photographs Negatives were made on Kodak Tmax-IOO 35 mm film. This film has good spatial resolution (fine grain) and has a low y, allowing representation of the very wide range of luminances in the photographed scenes. All photographs were taken at f/8. allowing large depth of focus; exposure was adjusted by changing the shutter speed. To allow later correction for the / of the photographic process, each film included several negatives of a flood-lit test chart comprising 15 different grey-level squares, covering a 20 X range of reflectance. The luminances of the squares were measured concurrently. The grey-level chart was photographed at several shutter speeds and through a 10 X neutral-density filter. It was possible, then, to relate the optical density of the negatives to the actual luminance over a range of >3 log units. Each negative was digitized by imaging it onto a linear CCD (charge-coupled device) array housed in a camera body; the negative was moved past the array using a stepper-motor. The output of the CCD array was digitized using an analogue-to-digital converter with over 1000 nominally different levels. The negatives were digitized into 256 x 256 square pixels, each of dimension of ^100/im. The digitized transmission values were converted to luminance values using the information provided by the calibration test chart. The combined line-spread function of the photographic and digitizing equipment was estimated from the digitized images of the edges of the squares in the test chart, and the spectra of the digitized images were corrected for the optical degradation. In theory, each digitized image should have been represented to more than 1000 grey levels. However, due to noise in the digitizing apparatus, this was reduced C' 1992 Butterworth-Heinemann for British College of Optometrists / Ophthal. Physiol. Opt., 1992, Vol. 12. April 229

2 Amplitude spectra of natural images: D. J. Tolhurst et al. to about 300 truly distinguishable grey levels, which were approximately equally spaced on a logarithmic scale. Amplitude spectra of the images The 135 digitized images covered a wide range of subjects: animals, plants, the English countryside, buildings, vehicles, and laboratory equipment. Figure I presents four of these images to show something of the variety, while Figure 2 shows their averaged amplitude spectra, plotted on double logarithmic axes. The graphs have been displaced vertically by arbitrary amounts for clarity. Each image was subjected to a 2-dimensional Fourier transform and the amplitude of each spatial frequency coefficient in the transform was averaged across orientation. It is evident that the amplitude falls more or less monotonically with spatial frequency, as reported previously^ ^. However, amplitude falls at different rates for the four images. implying that the spectra cannot all conform to Equation J. The suggested form of the amplitude spectrum {Equation I) would be represented on log log axes by a straight line with slope of To determine the precise slopes of the spectra, we have fitted them with functions of the form: averaged amplitude x spatial frequency"" (2) This is, of course, a more general version of Equation 1 and, on log-log axes, would be represented as a straight line with slope of - a. We fitted (by least squares) simple regression lines to the amplitude spectra on log-log coordinates, and the best-fitting regression lines are shown for the four spectra in Figure 2. For two images (A, B), a straight line was a good description of the spectra, showing that they do indeed conform to Equation 2. However, it should be noted that the Figure I Four examples of our digitized photographs, represented to only 64 grey levels 230 Ophthal. Physiol. Opt Vol. 12, April

3 Amplitude spectra of natural images: D. J. Tolhursi et al of imag Z3 ^ 4 Q. E "O CD > o O A B C D anumber E "o Jumber ^ r I I I I n i l I Mill Frequency (cycles/picture) Figure 2 The averaged amplitude spectra of the four images shown in Figure I. The four graphs have been displaced on the amplitude axis by arbitrary amount.s for clarity. The lines show the best-fitting regression lines, calculated on log - log coordinates. The slopes of the lines are: A ; B : C, -1.00; D Note Ihat ihe ordinate and abscissa are drawn lo different scales «400 CD E 300 QJ 'B 200 E Slope of amplitude spectrum -1.6 regression lines titted to these two spectra have slopes considerably steeper (i.e and 1.42) than the value of 1.0 suggested previously^ ^. For the other images, a simple regression line was not an adequate description of the amplitude spectrum. The spectrum of image C has distinct curvature. The spectrum of image D shows a different kind of failure to conform to Equation 2\ the decline in amplitude with spatial frequency is not monotonic. The spectrum shows small local peaks, presumably associated with the regularities (the brickwork) in the original image. We have applied this analysis to all 135 images and. for many. Equation J? was a reasonable description of their amplitude spectrum (as for images A and B). However, for 39 out of the 135 images, the spectra showed significant curvature, since a quadratic curve provided a better description of the spectrum than did a simple regression line: an F-test showed that the residual variance of the simple linear fit was significantly reduced [P = 0.01) by addition of a quadratic term. For most of the images, the slope of the best-fitting regression line was not particularly close to the suggested value of 1.0, as is evident from Figure 3a. which shows the distribution of the slopes of the best-fitting regressioti lines for the 135 images. The slopes varied considerably, from 0.8 to with an average of (SD 0.13); in fact. Figure 3 The distribution of the slopes of the bcst-fiiting regression lines for all 135 images: (t; the original images, mean = 1.20, SD = 0.13; (/)) Lifter multiplying each image by a circularly-symmetric Gaussian envelope, mean = SD = 0.13; (((after breaking each image inio 25 overlapping segmenls of 12K x 128 pixels, mean = SD = 0.15 relatively few of the images in our sample had slopes close to 1.0 [Equation I). There are a number of possible artefacts that might have led us to find steeper slopes than previously reported. First, the steeper slopes might arise if the majority of photographs were poorly focussed. We ruled out this possibility by manipulating the Fourier spectra of some images that had steep spectra in order to construct test images whose amplitude spectra had slopes different from the originals. We examined whether the image quality could be improved by "image sharpening". Given a choice amongst test images with a variety of slopes to their amplitude spectra, human observers chose the originals as having better quality than the manipulated images. The test images with slopes of 1.0 appeared poor quality compared to the original images. These results imply that the original images were not out of focus. Secondly, it is possible that edge effects introduced by the Discrete Fourier Transform may have significantly Ophthal. Physiol. Opt., 1992, Vol. 12, April 231

4 Amplitude spectra of natural images: D. J. Tolhurst et al. distorted the spectra. We controlled for this by multiplying each image by a circularly-symmetric Gaussian envelope of standard deviation 55 pixels (21% of the image width), centred on the image; the spectra were then recomputed. Figure 3b shows the distribution of slopes after removing the edges of the images in this way; there is little difterence from the original distribution shown in Figure 3a. Lastly, it is possible that the images were unrepresentative of natural scenes because of deliberate (perhaps unnatural) composition of the photographs. To examine this possibility, each image of 256 x 256 pixels was broken into 25 overlapping segments of 128 x 128 pixels. The segments mostly contained incomplete fragments of any objects in the original scenes. The spectra of these segments were computed, and Figure 3c shows the distribution of the slopes of these spectra. Again, the averaged slope is about (but with a slightly larger standard deviation), implying that the slopes of the amplitude spectra have not been influenced by the particular composition of the original photographs. Thus, we have been unable to confirm the suggested generalization that the averaged amplitude spectra of the images of most natural scenes are almost identical in their form '"^. It is true that amplitude invariably dechnes with spatial frequency, but the spectra of different images show more variety than has been accepted previously. Many spectra conform reasonably well to Equation 2. but the slopes vary considerably and are usually steeper than 1.0. However, the spectra of some images show significant curvature in addition to being steep. The supposition that the amplitude spectra of natural images are almost identical is implicit in the demonstrations'*"'' that the amphtude spectrum of one image can be replaced by that of another image without serious detriment to image quahty. However, the generality of such demonstrations has been questioned' ^ and our own unpublished observations show that it is easy to find pairs of images where exchange of the amplitude spectra results in hybrid images of very poor quality. Thus, it is not safe to maintain that the amplitude spectra of natural images are so similar as to be regarded as identical. Acknowledgements We should like to thank Mr P. Starling for assistance with the photography and Dr D. Roberts of the Engineering Department, University of Cambridge, for the loan of the digitizing apparatus. Y. T. received grants from the Daniel Falkner Charitable Trust and the Leo Baeck (London) Lodge 1593, and was later employed on a Wellcome Trust project grant to D. J. T.. T. C. received a scholarship from the British Council. The research was supported by a project grant awarded to D. J. T. under the Image Interpretation Initiative of the Science and Engineering Research Council, UK. References 1. Carlson. C. R. Thresholds for perceived image sharpness. Photog. Sci. Kngng. 22,69-71 (1978), 2. Burton. G. J. and Moorhcad, 1, R. Color and spatial structure in natural scenes. AppL Opt. 26, ( Field, D. J. Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A 4, (1987). 4. Blakemore. C. and Campbell, F. W. On the existence of neurones in the human vi.sual system seleetively sensitive to the orientation and size of retina! images. J. Physioi (1969). 5. Field. D. J. What the statistics of natural images tell us about visual coding. SPtE (1989). 6. Kni!!, D. C, Field, D. J. and Kersten. D. Human discrimination of fractal images. J. Opt. Soc. Am. A 1, (1990). 7. Tadmor, Y. and Tolhurst, D. J. Is the human visual system optimized for processing natural images? Perception 19, 398A (1990). 8. Tadmor. Y. and Tolhurst, D. J. The amplitude speelra of natural images (Abstract). Ophthal. Physiol. Opt. \1. 97 (1992). 9. Oppenheim, A. V. and Lim, J. S. The importance of phase in signals. Proc. IEEE ( Piotrowski, L. N. and Campbell, F. W. A demonstration of the visual importance and flexibility of spatial-frequency amplitude and phase. Perception II, (1982). 11. Morgan, M. J., Ross. J. and Hayes, A. The relative imporlance of local phase and local amplitude in patchwise image reconstruction. Bml. Cyhern. 65, (1991). 12. Brettel, H., Caelli, T., Hilz, R. and Rentschler, I. Modelling perceptual distortion: amplitude and phase transmission in the human visual system. Human Neurohiol. 1, (1982). 232 Ophthal. Physiol. Opt., 1992, Vol. 12, April

5