IMAGE PROCESSING IN THE SPATIAL DOMAIN: IMAGE ENHANCEMENT

Transcription

1 Image processing in the spatial domain IMAGE PROCESSING IN THE SPATIAL DOMAIN: IMAGE ENHANCEMENT This section considers some image characteristics such as spatial resolution, mixed pixels and scale. It then looks at low- and high-pass filters, Fourier transforms and the variogram as techniques to manipulate the spatial variation in an image. Remote sensing and spatial resolution Image resolution is an important consideration in operational remote sensing and the analysis of remotely sensed data. The resolving power of sub-orbital and satellite imagery ultimately controls the detection of phenomenological structures present at or on the surface of the earth or ocean bodies. Spatial resolution is, in particular, an important element of image resolution. This is because earth surface objects and/or spatial processes operative at scales below the sensor s spatial resolution cannot be detected. Therefore, sensor spatial resolution must be a vital consideration for the image analyst for a successful remote sensing application. It may be defined as: The ability to distinguish between point targets, the ability to measure the periodicity of repetitive targets and the ability to measure the spectral properties of small targets. Mather, PM (999), p.30 The spatial resolution of a remote sensing system is characterised either by a) line pairs per millimetre, for airborne imagery (this measure is typically employed when dealing with analogue or hardcopy prints); or b) the pixel (picture element), the basic unit or building block of digital imagery. Given that this unit is concerned solely with the digital processing and analysis of remotely sensed data, the first part of this section will discuss resolution in terms of pixel size and properties. For a discussion of aerial photography and its spatial resolution parameters the reader is directed to Lillesand et al. (2004). Recent technological developments have seen the emergence of finer spatial resolution instruments, for example, IKONOS II available at and 4 metres in panchromatic and multispectral modes respectively. It would be completely wrong, however, to suggest that the optimal remote sensing system is that with the smallest available pixel size. Tso & Mather (200) note that sensor

2 Image enhancement spatial resolution is only truly dependent upon what is we want to see and interpret from the imagery.. The pixel and digital imagery The pixel is the basic building block of a remotely sensed image (Figure ). The dimensions of the pixel, however, are arbitrarily defined. Its size (resolution) is controlled by a number of different elements, including:. Sensor technologies 2. Elevation or orbital height Figure : The image and the pixel. This image is a false colour composite of a Landsat Thematic Mapper scene for an area of south Manchester. The data have a nominal spatial resolution of 30m. The Local View is very pixelated, showing evidence of high spectral variation over a few pixels. Global View Local View 2

3 Image processing in the spatial domain Students who want to explore some of these issues in further detail are directed to classic papers by Fisher (997) and Cracknell (998) on the parameters of the image pixel and a discussion of its implications for digital image analysis. Image sensor characteristics and the Point Spread Function (PSF) The spatial resolution of an airborne or satellite sensor is primarily determined by the technologies of the imaging system. This includes the viewing geometry and sensor elevation above ground surface. Many widely available satellite sensors, such as Landsat TM and SPOT XS, employ scanners which again can affect image resolution and quality. The Instantaneous Field Of View (IFOV) of the sensor is a common measure of spatial resolution. IFOV is a measurement of the footprint (area of ground surface) within the view of one detector of the sensor (Mather 999). If this is difficult to grasp then think of the detector being a torch and emitting radiation rather than receiving it. The IFOV is the area illuminated at that detector at any one moment (Figure 2). Figure 2: Instantaneous Field Of View (IFOV) for a sensor from its elevated platform above the ground surface. The area marked by the arrowhead line indicates the width of the surface area viewed from this elevation. Additionally, there is the Point Spread Function (PSF) which provides another way to measure image (spatial) resolution. PSF is a more reliable measure as it accounts for the fact that radiance from the pixel footprint may be 3

4 Image enhancement affected by radiation from the surrounding landscape as well as the brightness properties of any features present within the pixel. In other words, provided the feature present within the pixel is highly contrasted with the background it can be readily detected, regardless of whether the land surface object is smaller than the IFOV. This phenomenon explains why road and river networks can often be seen on medium-resolution imagery such as Landsat, even when these objects are less than 30m across. While this function may seem helpful in enabling the detection of objects smaller than the dimensions of a single pixel, it also signifies the additional problem of defining the true pixel composition, which we will turn to next. The Mixed Pixel Problem The importance and impact of spatial resolution is also witnessed by the mixed pixel problem. Mixed pixels, sometimes known as 'mixels', occur where the image pixels are not homogenous, or pure. Instead a pixel contains a measure of the energy reflected or emitted from several different materials or land surface objects and the sensor records a composite of these responses (Figure 3). Figure 3: The mixed pixel problem. Many of the pixels in this 9 cell grid contain more than one class (or spectral material). The sensor will simply mix these different spectral components, generating a type of composite signature. In many cases this spectral mixing can make it very difficult for the image analyst to identify the different sub-pixel fractional components that serve to make up the landscape under observation. Under these circumstances the analyst may wish to employ a finer resolution data set, in order that a greater number of pure pixels may be recorded. Even with very fine resolution, however, there is still the issue of edge pixels (Campbell 987), where pixels can show the boundaries between different land surface properties. Land 4

5 Image processing in the spatial domain surface features do not follow the arbitrary confines of the pixel and therefore even very fine spatial resolution data of the order of a few metres will still experience some degree of confusion and spectral missing. As you can imagine, mixed pixels can cause great difficulties in the stages of image analysis and interpretation. One of the common tasks in this process is image classification. Classification is widely used as it allows users to easily discriminate information from images presented as a series of categories (classes) rather than raw digital number (DN) values. Images are classified on the basis of their spectral properties and so this will be discussed in more detail in the later section on image processing in the spectral domain. However, if we forget about the spectral properties of images for one moment and consider the implications of the spatial properties of images: how can we be sure we are accurately allocating pixels to their appropriate classes if we are unsure of the exact nature of image pixel composition? As you can imagine, the difficulty of the mixed pixel problem has led to considerable research into how image analysts can improve classification accuracy. Where pixels contain heterogeneous land cover properties standard statistical classifiers, which rely upon pure pixels, cannot cope very well. In these circumstances, the use of sub-pixel classifiers such as linear spectral mixing methods, artificial neural networks or fuzzy C means is recommended (Atkinson et al. 997). In essence these different techniques are employed to identify the degree (or percentage) of membership each available category has to the pixel being classified. Each pixel can possess multiple classes, and therefore can be considered a mosaic of land cover types. Given that we know the dimensions of the pixel we can then estimate the fraction of subpixel land cover occupied by each class. For example, say we have a SPOT HRV image with 20m pixels, and the pixel contains 3 classes: parkland grass (0.45), tarmac surface (0.30) and woodland vegetation (0.25). A 20m pixel approximately represents 0.04 hectares. Therefore, we would expect the pixel to contain 0.08 hectares of parkland grass, 0.02 hectares of tarmac and 0.0 hectares of woodland. While these classification methods will be examined in greater depth later in this unit, the issue of mixed pixels and the selection of appropriate analytical methods for their interpretation is an important consideration regarding sensor spatial resolution for operational remote sensing. 5

6 Image enhancement.2 Scale in remote sensing With spaceborne sensors now providing image resolution at scales from m to km, the selection of an appropriate sensor is increasingly becoming an important issue in operational remote sensing. Woodcock & Strahler (987) carried out an important study on the factor of scale in remote sensing. Using measures of local variance (the frequency of changes within the image scene) they were able to identify that variance within the images was directly related to the size of objects present within the imagery and sensor spatial resolution. Their original hypothesis was they would identify greatest local variance at the pixel (spatial) resolution that coincides with the average size of objects present within the image scene. This would obviously have very beneficial results for remote sensing as the choice of sensor (and optimal resolution) could be more easily decide upon, depending upon what land surface scene was being imaged. For example, in an agricultural scene of crops grown in linear fields with standard spacing between rows of 20 to 30 cm, one might reasonably expect the graph of local variance to show a peak at the same resolution (Figure 4). Figure 4: Crop spacing within an agricultural field and hypothetical graph of local variance. However, using a series of aerial photographs and simulated Landsat TM data Woodcock & Strahler (987) identified that the greatest level of local variance occurs between ½ to ¾ of the size of prevalent objects present within the image scene. This is because as the pixel size approaches the dimensions of the object the effect of that object spans across several contiguous pixels. This high spatial frequency is characterised by a peak on the local variance graph (Woodcock & Strahler 987). 6

7 Image processing in the spatial domain Obviously, the ability to accurately detect land surface objects from satellite scenes is therefore strongly tied into its image resolution. We will explore this issue further in the next section in our discussion of operational remote sensing and the selection of an appropriate sensor for any particular application..3 Optimal remote sensing: selecting the appropriate sensor From our discussion in the previous sections it is plain to see that the spatial resolution of satellite (and indeed airborne) remote sensing instruments is a vital consideration and indeed may place constraints on the suitability and application of imagery. Added to this confusion is the fact that the new generation of satellite sensors have finer spatial resolution (Figure 5). DigitalGlobe, which at the time of writing acquires the finest ground resolution of civilian EO missions, has a ground resolution of 0.6 making the data only one order of magnitude coarser than standard aerial photography. Figure 5: Satellite sensors and image resolution. IKONOS Pan SPOT HRV Pan Landsat ETM+ Pan SPOT HRV XS Landsat ETM+ -5, 7 However attractive the new generation satellite sensors appear, we should remember the words of Tso & Mather (200) concerning image resolution and interpretation. Rather than employ unnecessarily fine spatial resolution 7

8 Image enhancement sensors we should instead attempt to match the spatial resolution properties of the phenomenon under investigation with that of the available imagery. This work takes place on an application-by-application basis, though there has been some literature published on the matter (Millington et al. 995, Jensen & Cowan 999). Figure 6, for example, highlights the spatial and temporal scales of different geomorphological processes and may form the basis on which to select the most appropriate remote sensing data. The creation of such a graph would be a useful exercise when starting out on an investigation that may necessarily require the application of remote sensing. Figure 6: The spatial and temporal resolution limitations of satellite remote sensing applications in geomorphology. Adapted from Millington et al. (995) 8

9 Image processing in the spatial domain 2 Image Enhancement Once the image analyst has selected the appropriate type of imagery, the next step is digital image processing. One of the major components of this analysis is the enhancement of the imagery in order to improve its information content. Image enhancement involves the manipulation of imagery for the purposes of improved visual interpretability (Lillesand & Kiefer 2000). There is a wide range of enhancement tasks, and these processes are typically employed after the early image pre-processing stages (such as noise removal or the geometric correction of the image scene). This section will provide a discussion of spatial enhancement procedures only. The reader is directed to the growing body of introductory digital image processing textbooks for a wider review of radiometric and spectral digital image enhancement techniques. The spatial properties of a remotely sensed image can be readily improved by a variety of different processing techniques, including spatial frequency filtering, Fourier analysis and the use of variograms. Each of these methods will now be introduced and explained in turn. There are also IDRISI 32 practical exercises for you to complete associated with the sections on spatial frequency filtering and Fourier analysis. 2. Spatial Filtering Remotely sensed images contain particular data elements that we commonly wish to find out more about. A particular form of important information is the spatial detail present within the image scene. This detail can be defined as high and low frequencies and edges. All of these spatial frequency components are identified by their tonal characteristics (the changes within image brightness across the image scene), what Lillesand & Kiefer (2000) describe as image roughness. There are many different circumstances in which we may want to emphasise or suppress these particular image properties, and one of the ways in which we do this is the process of spatial filtering. Before we go on to examine this image processing method in further detail, however, it first be would useful to identify exactly how low and high spatial frequencies are characterised within remotely sensed imagery (Lillesand & Kiefer 2000). 9

10 Image enhancement Low Frequency Image Characteristics: These areas are presented within the imagery as large, relatively homogeneous sectors. There is relatively little change in pixel (DN) brightness values over a large geographic region. Minimal tonal variation is often present in images that contain large water bodies where there is little DN change across the feature(s). High Frequency Image Characteristics: These areas are presented within the imagery as zones of very localized detail. In other words there is a great deal of change in pixel brightness values over a very short distance. This is the roughness described by Lillesand & Kiefer (2000) earlier, and is strongly present in images where there is an abrupt change from one type of landscape to another, for example, rural to urban landscapes. Given these image characteristics we may wish to employ some form of enhancement technique that teases these features out more or alternatively goes someway to reduce their visual impact. The spatial filtering process involves the use of a local mathematical operation [the convolution filter] on the original imagery. We do this by employing a two dimensional moving window, more commonly known as a kernel, across the raw imagery to generate a new output image which is composed of the results of the kernel or averaging process (Figure 7). The kernel matrix has very specific dimensions; typically digital image processing software offer odd-number sized kernels, from 3x3 upwards. Each cell in the matrix contains a weighting coefficient, as shown in Figure, which is used in the mathematical operation. Some of the newer software programmes may also allow you to programme your own kernel dimensions. All kernels should be of equal dimensions in X and Y. Otherwise there is the possibility that they may generate image scene artifacts (of spurious value or origin) which are open to misinterpretation by the analyst within the newly created images (Drury 993). For the purposes of this unit, we limit our exploration of spatial filtering to the techniques most commonly used in remote sensing applications. Please do note, however, that there are many more different filter techniques available to the image analyst, and as always the analyst should be sure to employ the most appropriate filter for that particular application. For example, one type of 0

11 Image processing in the spatial domain Figure 7: The Filter Kernel, and the filtering operation. A smoothing or low-pass filter has been applied to the original image data. Raw Image A smoothing (low-pass) filter Output Image

12 Image enhancement filter not discussed here is the Sigma filter used to remove speckle (image noise) present within SAR (RADAR) data. Should you wish to learn more about the different filter techniques you should read Mather (999). Alternatively, you should take time to explore the different filters available to the IDRISI 32 image processing software. The Low Pass Filter The simplest of the filtering techniques is the Low Pass filter, often known as the smoothing filter. This method is employed to block out and remove the high spatial frequency detail that may be present within the image scene. In other words, the very fine local detail that may be present, such that may be characterised by a typical urban scene, where changes from buildings to pavements to roads to urban trees and parkland (each with highly distinctive DN values) can occur over the space of a few pixels. Instead this type of filter is employed when we wish to emphasise the broader changes present within the image scene. For example, if we are interested in mapping the agricultural fields within a rural scene, characterised by very large cropped areas, with only minimal changes in image brightness identifiable at the field boundaries, it would make sense to try and emphasise these features, whilst also suppressing the tonal properties of any rural settlements that may be present within the image scene. As detailed in the previous sub-section, a spatial filter comprises of a two dimensional kernel. Figure 8 describes the nature of the mathematical operation of this moving kernel matrix. Figure 8: The Low Pass Filter Kernel /9 * Figure 9b shows the effect of running this filter on a Landsat TM scene of South Manchester. Visual comparison of images A and B shows just what an impact the low pass filter has had on the original imagery. The filtered image contains a smoothly varying pattern of grey tones, and it is a relatively simple 2

13 Image processing in the spatial domain procedure to delineate the urban and rural areas from the imagery. The intraurban features that could be picked out from the original imagery are no longer discernable from the filtered data. Figure 9: Effects of the Spatial Filtering Process on a Landsat TM scene of South Manchester. Image A shows a raw TM band 4 image; B shows a Low Pass filtered image; C shows a High Pass filtered image; D shows an Edge Enhanced filtered image. A B C D This type of filter is very useful for the simplification of the data set and to remove noise (any unwanted image scene components) from the original imagery. There are also circumstances, however, where we may wish to 3

14 Image enhancement emphasise other image components and this is where techniques such as the high pass and edge enhancement filters prove valuable to the image analyst. Each of these techniques will be discussed in turn next. The High Pass Filter This filter operation is the event of subtracting the results of the low pass filter from the original image scene in order to generate a new image scene that enhances the local contrast present within the imagery (Drury 993). The high pass filter very effectively enhances spatial frequencies that are less than the size of the kernel matrix used in the operation (Curran 985, Drury 993). Figure 9C shows just how effective this type of filter is for highlighting local detail, in this case the peri-urban features of south Manchester, including road networks. Given that the spatial frequencies preferentially enhanced using this technique are directly related to the properties of the filter kernel; the choice of kernel size acts as a strong control on the level of detail generated from the process. For instance, very large geological faults may require much larger kernels in order to effectively enhance their properties within the imagery (Drury 993). Edge Enhancement and Orientation Filters On closer inspection you may not think so much of the high pass filter operation, despite its obvious value in generating information about linear features such as road networks. First, it shows the greatest level of transformation from the original imagery making the new filtered image appear markedly different, and second; there are many situations where you may want to keep the lower frequencies as well as emphasise the high spatial frequency patterns present within an image. When the latter is the case then image analysts typically employ another cosmetic operation known as edge enhancement. This is another filtering procedure, but it serves to maintain, and enhance, both low and high spatial frequency components that may be present within the original image scene (Lillesand & Kiefer 2000). The superior value of this technique as an all-rounder is clearly evident in Figure 9d. In addition to the edge enhancement filter, there are a number of directional filters available these allow the image analyst the opportunity to explore the preferential orientation of linear features or edges within a remotely sensed image. Some of the most commonly employed of these filters are the Sobel and Laplacian operators. 4

15 Image processing in the spatial domain 2.2 Fourier Analysis Fourier analysis is another form of image data enhancement, though this technique works in the frequency domain, unlike the spatial filtering methods. The Fourier process involves the transformation of the original image (it works on individual bands only) into an alternative co-ordinate space, namely a two dimensional spectrum, (or image) composed of the low, medium and high spatial frequencies present within the original image (Figure 0). Figure 0: The Power Spectrum created from the Fourier Transform. Adapted from Mather (999) Not only that, the Fourier Transform highlights the amplitude and orientation of these different image frequencies (Drury 993). Figure 0 also usefully gives prominence to several key elements of a Fourier spectrum image noted in the IDRISI users guide, Volume 2, p.67 (Eastman 999):. The spectrum images show a radial frequency pattern, moving from the centre point (DC) of the image which shows zero frequency (the average image value) to higher frequencies towards the outer edges of the image. 2. Noise, such as image striping, or features showing a strong linear orientation can be seen in the spectrum image. The spectrum, however, shows these features at 90 degrees to their direction in the original remotely sensed image scene. Filtering methods can be employed on Fourier Spectrum images, in the same way that they can on standard image scenes. The filters are employed to 5

16 Image enhancement reduced the amplitude of the particular image frequency to zero in the frequency domain, and then an inverse transform is performed to generate a standard image scene for display and visualisation purposes. The procedure is computationally intensive, although there are newer algorithms such as the Fast Fourier Transform (FFT) which is limited to applications of imagery with pixel dimensions to the power of 2 (for example, 256, 52 etc.) but can process the image data much more quickly. Despite these computational demands, Fourier analysis is increasingly being employed in digital image processing, most commonly in for the removal of noise from the image scene. This probably reflects the increasing processor capabilities of PC systems. Figure : Sixth-Line Banding in Landsat MSS band image of the Senegal River Flood Plain in Mauritania. Note the regularized horizontal line pattern. Working with Fourier Transforms: Worked example As noted in the previous section, the Fourier Transform is a powerful image processing tool for the analysis of remotely sensed data. In the following exercise you will explore how you can use this method to filter out and remove image noise (of a particular frequency and orientation) often found 6

17 Image processing in the spatial domain within some of the earlier satellite sensors. The removal of image noise is a common procedure in preliminary image analysis. Landsat MSS imagery can sometimes present what is known as six-line banding, a radiometric error caused by deterioration of the scanning system employed by the sensor (Campbell 987, Mather 999). MSS uses an onboard scanner with 6 detectors for each image waveband. Normally, this would provide the advantage of a shorter dwell time (the time necessary to capture the image scene) for the satellite, which in turn would enable a higher signal-to-noise ratio (SNR). Unfortunately, data from one of MSS sensors is missing because of a recording problem (Sabins 987). This image data error can cause difficulties for further image analysis if not attended to, but fortunately is merely a cosmetic problem (Figure ), however, and be quite easily resolved (i.e. the image line restored) using one of the available destriping methods. 3 Variograms and semivariograms Another approach widely employed in remote sensing applications since the 980s is geostatistics. This body of methods allows the analyst to estimate, measure and study the spatially correlation variations often present within data such as remotely sensed images. Treitz & Howarth (2000), p.269 note reflectance values are a function of spatial position and can therefore be considered as values of a regionalized variable. Much of this work has recently concentrated on variogram (also known as semivariogram) analysis (Curran 988), the basic tool of geostatistics (Oliver et al. 989). In fact, we briefly touched upon a similar procedure earlier in our discussion of Woodcock & Strahler s (987) work on scale in remote sensing images. Spatial resolution-dependent variance within images scenes has been widely used to define the most appropriate scale or image resolution for any particular application (Atkinson & Curran 997, Collins & Woodcock 999). This spatial component of image variance has also been established as a valuable technique for the estimation and measurement of a range of biophysical properties (e.g. Atkinson & Curran 997, Hay et al. 996). The variogram (Figure 2) is used to describe the correlation between image pixels in close proximity to one another (Treitz & Howarth 2000). The variogram is presented as a graphical plot that contains a variety of spatial descriptors of the remotely sensed imagery (Table ). 7

18 Image enhancement Figure 2: The Classic Variogram Adapted from Curran (988) Table : The Semivariogram Descriptors. Spatial Lag Distance between sampling points (in metres or pixels) 2. Range Maximum level of semivariance (distance to sill); pixels closer than the range are related, further apart they are not 3. Sill The maximum level of semivariance (height of the semivariogram) 4. Nugget Variance Represents spatially independent variance The variogram plot is fairly straightforward to interpret once you know the different descriptors, although very rarely do you obtain such a classic plot. Indeed, Curran (988) notes that it is more typical to obtain a variety of different plots for different image scenes, for instance a common urban image 8

19 Image processing in the spatial domain scene is more likely to reproduce a multifrequency variogram plot, where there are noticeable peaks in the graphical plot associated with regular patterns, such as street networks and buildings. The peak(s) of this plot identifies the spatial frequency of the observed feature. Figure 3 Exemplar variogram plots. A: a periodic variogram; B: an unbounded variogram. Adapted from Curran (988) Semivariance may be derived using the following equation: S2 = ½ [z(x) z(x + h)]2 Where z is the DN of pixels x extracted at regular intervals (h) along a transect within the remotely sensed image under analysis. It is important to additionally note that lag length has a strong control over ability to discriminate similarity and dissimilarity between pixels. The range is possibly the most valuable component of the semivariogram, as it can help to determine whether there are repetitive spatial patterns present within the imagery. Figure 3 illustrates examples of a) repetitive patterns and b) a surface with a definite trend where range has not been reached along the transect line within the image scene. Combining the range and spatial lag information the analyst can begin to establish distance element of the spatial patterns within an image, for example the spacing between trees or agricultural crops. The variogram measure is therefore exceedingly important in allowing us to establish the degree, or scale of spatial variability and the directional pattern of this variance within the remotely sensed imagery. Treitz & Howarth (2000) in a 9

20 Image enhancement study of forest ecosystems have also identified that range estimates calculated from the semivariogram appeared to be directly related to forest canopy diameter. Elsewhere, Atkinson & Curran (997) have shown how the variogram may be employed to estimate dry biomass within an agricultural landscape, and; St-Onge & Cavayas (995) used directional variograms to estimate tree size and forest stand structure. 20