International Journal of Remote Sensing Vol. 29, No. 2, 20 January 2008, 459 470 Technical Note Automatic relative radiometric normalization using iteratively weighted least square regression L. ZHANG*{, L. YANG{, H. LIN{ and MINGSHENG LIAO{ {Institute of Space and Earth Information Science, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong {State Key Laboratory for Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, P.R. China (Received 10 July 2006; in final form 19 January 2007 ) Relative radiometric normalization among multiple remotely sensed images is an important step of preprocessing for applications such as change detection and image mosaicking. In this paper we present a new automatic normalization approach that uses the iteratively weighted least square regression technique. This approach does not require selection of the pseudo-invariant features beforehand as in some other traditional methods, and is robust to outliers since it adaptively places different weights on different pixels according to their probabilities of no-change. This approach is mainly applicable to cases where primary spectral differences between the two images are caused by variations in imaging conditions rather than phenological cycle or land cover changes. The effectiveness of this approach was demonstrated by two experiments using both artificially constructed data and remotely sensed images respectively. The experimental result seems promising and our approach shows accuracy comparable to normalization methods. 1. Introduction Relative radiometric normalization (RRN) is a procedure that attempts to reduce the numeric differences between two remotely sensed images that are induced by disparities in the acquisition conditions (e.g. sensor performance, solar illumination, atmospheric effects) rather than changes in surface reflectance (Yuan and Elvidge 1996). RRN has been widely used in applications such as change detection and image mosaicking as a practical alternative for absolute radiometric normalization (ARN) techniques that try to remove the radiometric differences by calibrating the DN values to surface reflectance independently (Du et al. 2002). One of the major advantages of RRN over ARN is that RRN does not require data of atmospheric properties or laboratory spectral curves that are essential for ARN (Furby and Campbell 2001) but usually difficult to acquire or even unavailable for calibrating historical satellite images (Jensen 2005). Numerous approaches have been proposed for RRN and the common approach is to take one image as a reference image and then adjust the radiometric properties of another image (subject image) to match the reference image (Hall et al. 1991, Yang and Lo 2000). According to the radiometric adjustments adopted, most of *Corresponding author. Email: zhanglu@cuhk.edu.hk International Journal of Remote Sensing ISSN 0143-1161 print/issn 1366-5901 online # 2008 Taylor & Francis http://www.tandf.co.uk/journals DOI: 10.1080/01431160701271990
460 L. Zhang et al. these RRN approaches can be categorized as either pairwise pixel-based or distribution-based. The strategy of the former is to derive a linear regression from a set of subjectively chosen invariant pixels from the two images. Typical methods of this category include pseudo-invariant feature (Schott et al. 1988, McGovern et al. 2002), dark set bright set (Hall et al. 1991), and automatic scattergram-controlled regression (Elvidge et al. 1995). While being useful, the major disadvantages of such methods are the subjectivity in choosing invariant features and their sensitivity to image misregistrations. The distribution-based methods, such as histogram matching (Chavez and MacKinnon 1994), min max normalization and meanstandard deviation normalization (Yuan and Elvidge 1996), use global image statistics to determine adjustment on radiometric properties. Performance of such methods is often not as good as that of pixel pairwised methods in terms of normalization accuracy. However, subjectivity problem could be eliminated and dependency on high-accuracy registration could be reduced. Recently a few novel methods have been devised for RRN. Canty et al. proposed to use the multivariate alteration detection (MAD) transformation (Nielsen et al. 1998) with thresholding to obtain invariant pixels from two multispectral images to derive linear regression for normalization (Canty et al. 2004). Nelson et al. put forward to match global characteristics of two images by equalizing the pixel ordinal ranks based on sorting of pixel values (Nelson et al. 2005), which is in fact a distribution-based method. In this paper we present a new approach for RRN that uses the iteratively weighted least squares regression (IWLSR) technique. The major advantages of this method over traditional pairwise pixel-based methods are as follows: (1) unlike those methods using a few pseudo-invariant pixels, IWLSR takes all pixels into computation, so that the problem of subjectivity in choosing invariant features could be avoided, and the RRN procedure can be performed automatically; (2) traditional methods consider equally all pixels used in the regression analysis, while IWLSR puts different weights on different pixels adaptively, so that the regression result is predominantly determined by pixels with higher weights that are largely invariant features, consequently the result is expected to be more robust to outliers such as pixels whose land cover had changed over time or pixels that were covered by clouds and their shadows. 2. Methodology The proposed IWLSR method is based on the weighted least square (WLS) linear regression analysis, which is the generalized form of the ordinary least square (OLS) linear regression analysis. In the following sections, OLS and WLS are firstly introduced, followed by the description of the IWLSR method. 2.1 OLS and WLS The OLS linear regression is a procedure to determine the optimal prediction of one variable y as a linear function of another variable x from n joint observations of both variables as (x i, y i )(i51,, n). The linear function is assumed to be y5a + bx, where a and b are the intercept and slope parameters of the regression line respectively. Considering the residuals, for each pair of observation (x i, y i ) there should be the following relationship: y i 5a + bx i + e i, where e i is the regression residual. By minimizing the root mean square (RMS) of residuals, OLS derives
optimal estimates of parameters a and b as: ( â~y{ˆbx where x~ 1 n x i, y~ 1 n respectively, and ŝ xy ~ 1 and ŝ 2 x ~ 1 n{1 y i n{1 ˆb~ŝxy ŝ 2 x ð1þ are the mean values of x and y observations ðx i {xþðy i {yþ is the covariance between x and y, ðx i {xþ 2 is the variance of x. It is well known that OLS is sensitive to outlier observations, as a result there may be large biases in parameter estimates from true values (Rousseeuw and Leroy 2003). Aiming at this problem, the WLS was designed to reduce the unfavourable impact of outliers on regression by placing different weights w i on different observation pairs (x i, y i ). Observation pairs with large regression residuals are prone to be considered as outliers and thus should be given low weights, whereas high weights should be assigned to observation pairs with small residuals. To avoid extreme imbalances in weighting, weights w i are usually limited to vary between 0 and 1. According to the WLS, parameters a and b are still solved using equation (1), however, weighted forms of mean values and variance/covariance as below should be adopted instead of ordinary ones, i.e. 8 >< x~ Pn ŝ 2 x ~ P >: 1 n An automatic radiometric normalization method 461 n{1 n w i x i w i, y~ Pn w i x i {x w i w i y i w i ð Þ 2,ŝ xy ~ P 1 w n i ðx i {xþðy i {yþ w i n{1 n ð2þ 2.2 IWLSR method for relative radiometric normalization A key issue for applying WLS to relative radiometric normalization is to select an appropriate weighting scheme. There are numerous weighting schemes available for selection, and here we chose one by taking into account the approximate distribution of regression residuals proposed by Canty et al. (2004). In general the residual of a linear regression can be regarded as a random variable that is subject to Gaussian distributions with zero centres if most observation pairs are scattered along the regression line, i.e. e~y{ âzˆbx *N 0, s 2 e ð3þ where s e is the standard deviation. Under above hypothesis, the variable t5(e/s e ) 2 is approximately chi-square distributed with only one degree of freedom, i.e. t,x 2 (1). In this way the weight can be constructed using the following equation: ~1{P x 2 ð1þƒt w~p x 2 ð1þwt Note that the item P{x 2 (1)(t} is the chi-square cumulative probability function, which is a monotonic increasing function of variable t and varies between 0 and 1. ð4þ
462 L. Zhang et al. Consequently, the weight decreases monotonically with t, so that the weight is near to 1 when t is approaching 0, and is very close to 0 when t becomes very large. Usually for a pixel whose land cover has not changed, the regression residual e should be close to 0, so that the value of variable t should be small, as a result the corresponding weight must be high; while for a changed pixel, variable t ought to have a large value, thus yielding a low weight. In this way, above weight could be seen as a measurement of no-change probability for each pixel. As the weights in WLS are determined by regression residuals which are unknown a priori, the RRN procedure must be initialized with an OLS linear regression. After initial regression residuals are obtained from the OLS, the weights can be established using equation (4), and then the WLS regression can be carried out with new regression residuals produced, which in turn will determine new weights, and so on. Such a procedure will continue until the iteration converged, i.e. the maximum difference of weights between two successive iterations is below a preset threshold. This is why the method is referred as Iteratively Weighted Least Square Regression. A flowchart for this procedure is shown in figure 1. There are several merits for this iterative weighting scheme. First, all pixels will be used in computation and there is no need to select a set of pseudo-invariant features beforehand as required by most traditional pairwise pixel-based methods. In this way, the subjectivity problem in choosing invariant features could be avoided. Second, unlike traditional methods in which all selected pixels are considered to have equal contributions to the regression result, the IWLSR method adaptively puts higher weights on pseudo-invariant pixels and lower weights on possible outliers, so that the linear regression parameters are predominantly determined by the former and consequently the result is expected to be more accurate and reliable. Finally, the iterations can improve the stability of the regression result compared with those produced by traditional methods. The result of IWLSR RRN can be quantitatively evaluated in terms of two statistical measurements. The first one is the weighted correlation coefficient r between the reference and the subject images, calculated using equation (5). The second is the weighted RMS regression residual e in equation (6). Note that x and ȳ are weighted means of x and y respectively. In general the better the IWLSR result is, the larger r value and the smaller e value are achieved. w i ðx i {xþðy i {yþ r~ sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi w i ðx i {xþ 2 Pn w i ðy i {yþ 2 ð5þ Figure 1. Flowchart of the proposed IWLSR RRN procedure.
An automatic radiometric normalization method 463 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi h i 2 w i y i { âzˆbx i e~ u t n{1 n w i ð6þ 3. Data and results The experimental data used for RRN is a pair of Landsat-7 ETM + panchromatic images of size 5126512 pixels covering the Three Gorge Dam area that is located at the middle of Yangtze River. The two images were acquired on 1 September 1999 and 25 September 2002 respectively, as shown in figure 2. It is well known that the Three Gorge Dam Project (the largest hydroelectric project in the world) was under construction during that period between the two image acquisitions. Major land cover types in the area consist of vegetation, water body and manmade features. As the two images were acquired at nearly anniversary times of different years, the impact of phenological variation on radiometric characteristics should be small enough to be omitted. The two images have been coregistered to each other with an RMS error less than 0.5 pixels before carrying out RRN using a multi-step matching strategy (Liao et al. 2004). For convenience, the two images are called image 1999 and image 2002 respectively. Two experimental scenarios were designed to explicitly illustrate the effectiveness of the proposed IWLSR method. The first one is a scenario in which the reference image is artificially constructed from the two original images. In the second the two original images were used. 3.1 Using artificially constructed data In this experiment the original image 2002 is selected as the subject image, and the reference image is constructed by replacing the left 3/4 part (384 columns) of image (a) (b) Figure 2. 2002. Landsat-7 ETM + images over the Three Gorge Dam area for (a) 1999 and (b)
464 L. Zhang et al. 1999 with that of image 2002, as shown in figure 3. In this way there should be no radiometric difference in the left 3/4 part of the study area between the subject image and the reference image. Provided that pairwise pixel-based methods are adopted and all invariant pixels are chosen from the left 3/4 part, the regression residuals for pixels in that part must be zeros. A good RRN method should make the magnitude of the regression residuals in that part as small as possible, i.e. close to zero. Figure 4(a) shows the weights obtained by OLS, which were used as initial values of weights in the IWLSR procedure. Figure 4(b) shows the weights obtained by IWLSR when converged. As described in section 2.2, these weights can be used to measure the no-change probability at each pixel. Hence we call these two figures as the no-change probability images of OLS and IWLSR respectively. In these images bright pixels mean large weights and low probabilities of change, whereas dark ones correspond to small weights and high probabilities of change. Obviously IWLSR outperforms OLS in distinguishing no-change vs. change, since the overall brightness of left 3/4 part area in 4(b) is much higher than that of 4(a). The images of resultant regression residual for OLS and IWLSR (converged) are shown in figure 5(a) and 5(b) respectively. By visual inspection the left 3/4 part of 5(b) is much smoother than that of 5(a), which implies that the variation of residual in that area for IWLSR is much smaller than OLS. Figure 6(a) and 6(b) separately show evolutions of mean and standard deviation of absolute value of the residual in the left 3/4 part area over IWLSR iterations. Both statistical variables decrease rapidly after the first iteration, which indicates that a substantial improvement was achieved by WLS over OLS regression. Then both variables undergo a stage of smaller variation until they are converged after five iterations. The final values are 0.19 for mean and 0.05 for standard deviation, which are much closer to zero compared with those of OLS, i.e. 2.12 for mean and 0.76 for standard deviation. This confirmed the fact that the IWLSR outperforms OLS. The weighted correlation coefficient and weighted RMS regression residual for the entire image are shown in figure 6(c) and 6(d) respectively. Once again rapid changes in value are observed after the first iteration for both statistical variables, i.e. increase in correlation and decrease in residual. The values by convergence are 0.99 for weighted correlation coefficient and 1.11 for weighted RMS regression Figure 3. The artificially constructed reference image.
An automatic radiometric normalization method 465 (a) (b) Figure 4. No-change probability images for (a) OLS and (b) IWLSR using constructed data. residual, compared with initial values of 0.87 and 4.59 respectively. This also proves the advantage of IWLSR over OLS in performance. 3.2 Using original images The original image 1999 and image 2002 are selected as the reference and subject images respectively in this scenario. By visual inspection of the two images one can see that the most significant land cover changes from 1999 to 2002 are changes in manmade features, while most vegetation and water body have little changes. According to the IWLSR method, one can infer that pixels of the former will be associated with lower weights while pixels of the latter will be attached with higher weights. (a) (b) Figure 5. Regression residual images for (a) OLS and (b) IWLSR using constructed data.
466 L. Zhang et al. Figure 6. Statistics over iterations of IWLSR RRN using constructed data: (a) and (b) are mean and standard deviation of absolute residual in the left 3/4 part respectively; (c) and (d) are weighted correlation coefficient and weighted RMS regression residual for the entire image separately. Figure 7(a) and 7(b) show the no-change probability images produced by OLS and IWLSR respectively. In figure 7(a) and 7(b) bright pixels have high probabilities of no-change, while dark pixels have low probabilities of no-change. Through visual interpretation we can identify that most of the bright pixels are covered with vegetation or water body, and dark pixels correspond to either land cover changes caused by a construction project or radiometric differences induced by clouds and their shadows. This shows agreement with previous theoretical inference. Figure 7(c) is the difference image between 7(a) and 7(b). In figure 7(c) bright tone that is marked with A represents a notable increase in value of no-change probability from OLS to IWLSR, dark tone that is marked with B stands for significant decrease, and grey tone that is marked with C or D means very little or even no change. Evidently no-change probability for most pixels remained unchanged or changed little, and there are more pixels with no-change probability increased than those with no-change probability decreased. In this way the overall no-change probability of the area was improved, which shows the effectiveness of the iterative procedure of the IWLSR method. It is noteworthy that there are little differences between figure 7(a) and 7(b) in the lowest and highest no-change probabilities, because such weights are primarily determined by the OLS procedure and then undergo little variations in the following iterations. Figure 8(a) and 8(b) illustrate the development of the weighted correlation coefficient and weighted RMS regression residual over 12 iterations until converged. The weighted correlation coefficient increases rapidly from 0.75 to 0.9 after the first iteration, then increases gradually to 0.92 in the following 11 iterations. For the
An automatic radiometric normalization method 467 (a) (b) (c) Figure 7. No-change probability images for (a) OLS and (b) IWLSR using original image data. (c) The difference between (a) and (b). residual, a remarkable decrease from 5.89 to 3.03 after the first iteration can also be observed, showing the same tendency as that of using artificially constructed data. A comparative experiment was also designed and carried out to further interpret the effectiveness of the IWLSR method compared with OLS and PIF methods. In this experiment, 7286 sample pixels with no-change probabilities obtained by IWLSR higher than 0.98 were selected from the original image 1999 and image 2002. These samples can be assumed to be invariant pixels with high confidence. Then according to the pseudo-invariant feature (PIF) method, 2/3 of these samples, i.e. 4857 invariant pixels, were used as training samples to derive the regression parameters for RRN, then the other 2429 samples were used as test samples to validate the regression result. Four statistics were adopted as test criteria: minimum,
468 L. Zhang et al. Figure 8. Statistics over iterations of IWLSR RRN using original image data: (a) and (b) are weighted correlation coefficient and weighted RMS regression residual respectively. maximum, mean and standard deviation of regression residuals. At the same time such statistics were calculated on the same test samples in the regression residual images produced by OLS and IWLSR respectively. To reduce bias in statistical estimation, the above procedures were repeated 10 times and each time training and test samples were randomly selected, and finally the average values of 10 times for the four statistics were used in comparison. Such a procedure is a cross-validation to a certain extent. The results are listed in table 1. Obviously OLS is inferior to the other two methods since it produced much higher values of all statistics than that of PIF and IWLSR. The minimum and mean values of residual for IWLSR are a little higher than those for PIF, while the maximum value and standard deviation of residual for IWLSR are lower than PIF. Therefore we conclude that IWLSR can produce comparable RRN results to the PIF method. Considering the efforts and cost needed by selection of pseudo-invariant features in the PIF method, IWLSR does provide a more practical and reliable approach than traditional methods for RRN even at the cost of considerable computation time for weighted iterations. 4. Summary and conclusions An automatic approach for relative radiometric normalization using iteratively weighted least squares regression is presented. Unlike traditional pairwise pixelbased methods, this approach takes all pixels into computation, so that it does not need to subjectively choose pseudo-invariant features. In this approach each pixel is adaptively weighted according to its probability of no-change during iterative estimation of linear regression parameters, consequently the result is more robust to outliers than traditional methods. Effectiveness of this approach has been validated by two experiments using artificially constructed data and original image data respectively. Table 1. Comparison using statistics of residuals among PIF, OLS and IWLSR. Minimum Maximum Mean Standard deviation PIF 0.0223 0.1356 0.0795 0.0421 OLS 0.0913 2.8746 0.6872 0.5269 IWLSR 0.0152 0.1472 0.0799 0.0424
An automatic radiometric normalization method 469 For the experiment using a set of artificially constructed data, the IWLSR method outperforms the OLS in two aspects. First, the result of IWLSR shows much lower regression residuals than that of OLS in the invariant region, i.e. the left 3/4 part of the study area. Second, the IWLSR method produces a better magnitude of two statistical measurements than OLS, i.e. higher in weighted correlation coefficient r and lower in weighted RMS regression residual e. When the original images are used as test data, the result again proves the advantage of the IWLSR method over OLS. The value of regression residual e is remarkably decreased by using IWLSR together with significant improvement in weighted correlation coefficient r. An independent assessment using 7286 samples with very high probabilities of no-change further illustrates the effectiveness of the IWLSR method, in that it can produce comparable results to the traditional PIF method. The proposed IWLSR method is promising for enabling automation of radiometric normalization between multiple remotely sensed images in applications such as change detection and image mosaicking. In this paper we have only applied it to normalizing single-band images, and we will expand its application to multispectral images in further investigations. Acknowledgements This work was supported by the Research Grant Council (Ref. No: CUHK4642/ 05H) of Hong Kong SAR, P.R.China and the National Key Basic Research and Development Program of China (Contract No. 2006CB701300), Direct Grant for Research 2005 2006 of the Chinese University of Hong Kong (Project Code: 2020881). The authors also wish to thank Prof. Yong Wang of the East Carolina University, USA for providing the Landsat ETM + images used in the experiments. References CANTY, M.J., NIELSEN, A.A. and SCHMIDT, M., 2004, Automatic radiometric normalization of multitemporal satellite imagery. Remote Sensing of Environment, 91, pp. 441 451. CHAVEZ, P.S. Jr. and MACKINNON, D.J., 1994, Automatic detection of vegetation changes in the southwestern United States using remotely sensed images. Photogrammetric Engineering and Remote Sensing, 60, pp. 571 583. DU, Y., TEILLET, P.M. and CIHLAR, J., 2002, Radiometric normalization of multitemporal high-resolution satellite images with quality control for land cover change detection. Remote Sensing of Environment, 82, pp. 123 134. ELVIDGE, C.D., YUAN, D., WEERACKOON, R.D. and LUNETTA, R.S., 1995, Relative radiometric normalization of Landsat Multispectral Scanner (MSS) data using an automatic scattergram-controlled regression. Photogrammetric Engineering and Remote Sensing, 61, pp. 1255 1260. FURBY, S.L. and CAMPBELL, N.A., 2001, Calibrating images from different dates to likevalue digital counts. Remote Sensing of Environment, 77, pp. 186 196. HALL, F.G., STREBEL, D.E., NICKESON, J.E. and GOETZ, S.J., 1991, Radiometric rectification: toward a common radiometric response among multidate, multisensor images. Remote Sensing of Environment, 35, pp. 11 27. JENSEN, J.R., 2005, Introductory Digital Image Processing: A Remote Sensing Perspective, 3 rd Ed, pp. 213 222 (New Jersey: Prentice Hall Press). LIAO, M.S., LIN, H. and ZHANG, Z.X., 2004, Automatic registration of InSAR data based on least- square matching and multi-step strategy. Photogrammetric Engineering and Remote Sensing, 70, pp. 1139 1144.
470 An automatic radiometric normalization method MCGOVERN, E.A., HOLDERN, N.M., WARD, S.M. and COLLINS, J.F., 2002, The radiometric normalization of multitemporal Thematic Mapper imagery of the midlands of Ireland a case study. International Journal of Remote Sensing, 23, pp. 751 766. NELSON, T., WILSON, H.G., BOOTS, B. and WULDER, M.A., 2005, Use of ordinal conversion for radiometric normalization and change detection. International Journal of Remote Sensing, 26, pp. 535 541. NIELSEN, A.A., CONRADSEN, K. and SIMPSON, J.J., 1998, Multivariate alteration detection (MAD) and MAF post-processing in multispectral, bitemporal image data: new approaches to change detection studies. Remote Sensing of Environment, 64, pp. 1 19. ROUSSEEUW, P.J. and LEROY, A.M., 2003, Robust Regression and Outlier Detection, pp. 1 60 (New Jersey: John Wiley & Sons Press). SCHOTT, J.R., SALVAGGIO, C. and VOLCHOK, W.J., 1988, Radiometric scene normalization using pseudo-invariant features. Remote Sensing of Environment, 26, pp. 1 16. YANG, X.J. and LO, C.P., 2000, Relative radiometric normalization performance for change detection from multi-date satellite images. Photogrammetric Engineering and Remote Sensing, 66, pp. 967 980. YUAN, D. and ELVIDGE, C.D., 1996, Comparison of relative radiometric normalization techniques. ISPRS Journal of Photogrammetry & Remote Sensing, 51, pp. 117 126.