Orthorectified imagery is widely

Transcription

1 Geometric Quality Assessment of Or By Simon Kay, Peter Spruyt, Abstract Two QuickBird images delivered in basic and standard processed formats and an IKOOS Geo-processed image, covering an area of 108km 2 near to Aradippou, Cyprus, were orthorectified using a Digital levation Model derived from medium scale map data and off-the-shelf software. The resulting orthoimages were tested for geometric correction quality using a standardized protocol. A network of ground control points (GCPs) was surveyed using a dual-frequency carrier-phase Global Positioning System receiver, with an estimated accuracy of better than 0.1m. The GCPs were divided into two sets: a calibration set (of 18 points) was used for a repetitive orthocorrection trial using between one and four GCPs, and a validation set (of 28 points) was used to evaluate the precision of each orthoimage. The system corrected Standard QuickBird image delivered by the supplier was found to meet the documented specification of 14m RMS, and is suitable for medium scale mapping applications. The upper 95% confidence limits of the one-dimensional RMS of orthoimages made from IKOOS Geo and QuickBird Basic images were 1.65 and 1.27 meters respectively: Both systems are well within 1:10,000 scale Class I accuracy requirements of the U Common Agricultural Policy. Orthorectified imagery is widely recognized as a data source for a variety of thematic mapping applications; in the uropean Union (U), information systems for the management of the Common Agricultural Policy are now routinely based upon digital orthophoto coverage with a nominal geometric quality of 1:10,000 map scale equivalent and a 1m pixel size, or better (Kay et al., 1997). evertheless, airborne image acquisition demands a certain degree of access that is not always feasible, and the availability of so-called very high resolution (VHR) satellite sensors permits the acquisition of data with image content quality characteristics aimed at the demands of rural area or agricultural mapping and monitoring (Petrie, 2002). Three main approaches for producing orthoimagery from QuickBird and IKOOS data are in current use: rigorous sensor models (for example, Toutin and Cheng, 2003), the Rational Polynomial Coefficient (RPC) method computed using ground control points, or the use of RPC information provided by image vendors. The first two require the use of substantial numbers of ground control points (typically 7 to 39, for different orders of correction), while the last may in theory function without any new ground control (Cheng, pers. comm., Dial and Grodecki, 2002; see Toutin, 2003 for a review of geometric correction approaches). While the above studies mostly demonstrate results of a very precise nature, we consider that for large area applications such as those initiated by the uropean Commission the use of more than a few points per image would render the widespread use of current VHR satellite imaging systems uneconomical, due to the relatively small scene footprint. Furthermore, any requirement to use a very precise terrain model would again pre-suppose the existence of high quality mapping, a consideration that would place many areas in urope outside the scope of application. The use of a limited amount of ground control has been attempted previously to improve the geocoding of IKOOS imagery (see Kaichang et al, 2003), and this is the philosophy that we have followed. The purpose of this study, therefore, was to evaluate the geometric quality of orthorectified QuickBird and IKOOS imagery, for a typical agricultural area, using a restricted number of high precision ground control points (GCP) and a medium-scale externally-derived digital elevation model (DM). For the purposes of this test, the Cyprus Ministry of Agriculture Table 1: Images used in the study. The two QuickBird images are identical in terms of acquisition characteristics but are processed to different levels. Both the IKOOS and QuickBird images present moderate off-nadir viewing that is typically selected for space mapping applications. Instrument QuickBird QuickBird IKOOS Product Type Standard Basic Geo Sensor elevation angle Pixel size 0.70 cm, resampled Ground sampling distance 0.62m/0.61m Resampled 1m (Ground Sampling Distance 0.83m) Date of acquisition May 2003 PHOTOGRAMMTRIC GIRIG & RMOT SSIG

2 thorectified VHR Space Image Data and Kyriacos Alexandrou acquired a QuickBird image with the characteristics shown in Table 1; we also received an IKOOS image of the same scene for comparison. In addition, the digitization of the 25m contour from the 1:50,000 national mapping series was undertaken to provide elevation data of adequate quality, nominally with an accuracy of 1/3 rd of the contour interval, or around 8m Root Mean Square rror (RMS) (ASPRS, 1989) in elevation. The technical work that followed consisted of: Selection of GCPs and field survey work using dual-frequency carrier-phase GPS; The creation of the DM, from the contour lines, using GIS software, and subsequent quality control; Assessment of the delivered Standard QuickBird image product geometric quality; Image orthorectification, using off-the-shelf software, of the three image types under test; Geometric quality assessment, using guidelines developed for use in uropean Commission project work. This article describes each of the five elements in turn. orthoimage products (uropean Commission, 1999). These guidelines provide a standardized approach for the quality testing of orthorectified imagery, derived from either airborne or spaceborne sensors, and have been used by our organization in a number of previous operational studies. In particular, we applied the approach for the testing by external quality control of orthoimage products, which is the more general method among those outlined in the guidelines. The QuickBird Standard scene was divided into 25 sections, with the objective of acquiring two GCPs in each section (a target of 50 GCPs in total); a diagram of the GCP positions is given in Figure 1. Candidate points were identified on large scale print-outs of the QuickBird Standard image in preparation for the field visit, and their continued on page 486 General Description of the Test Site and Images The location of the test site is in Cyprus, near the city of Larnaca, centred on the village of Aradippou. The scene extent covered by all images was just over 100km 2, and the zone is typical for agricultural regions in Cyprus with differences in relief of around 250m over the test site. The QuickBird image was acquired following normal ordering procedures, and delivered as both basic and standard corrected products (see Table 1). The former product is corrected for radiometric and sensor characteristics, but has no geometric corrections applied; the latter product, by contrast, is geometrically corrected to a nominal accuracy of 14m one-dimensional RMS 1, using a coarse DM (1km spacing), and projected into UTM using the WGS84 ellipsoid. However, the image product geometric specification does not account for relief displacement (urimage, 2003). The IKOOS image was provided as a Geo product, the lowest processing level of product available from the supplier 2. These data are processed into a system projection, in this case UTM projection and WGS84 ellipsoid (Space Imaging urasia, 2003). All three images were supplied with vendor-produced rational function data files (see Kaichang et al., 2003), and these were used as the primary means of orthocorrection. Ground Control Point Survey and Measurement A number of high precision control points were measured during a field mission conducted from 16 th -23 rd September 2002, throughout the test zone. The selection procedure was carried out using a set of guidelines adopted by the uropean Commission for testing of Figure 1: Distribution of ground control points used for calibration (red) and validation (green) overlaid on the QuickBird image of 21 June The grid shows the partitioning of the scene into equal sized areas, to ensure that the distribution of these points was well spread over the scene. The GCP selection was done on-screen, and printed documents produced to aid the exact survey of the points in the field. PHOTOGRAMMTRIC GIRIG & RMOT SSIG May

3 Definition of Root-Mean-Square rror When checking the geometry of orthoimagery, a measurement is required that can take into account the positive and negative values of the recorded residual errors. ormally, a simple averaging or mean value would give a result close to zero, without any indication of the size of the residuals observed. For this reason, mapping science generally uses the root-meansquare rror, or RMS, which gives an objective indication of the goodness of fit of the image to the check points. The RMS is, literally, the square root of the mean of the sum of the squares of the error residuals. The residuals are squared, to change them all to positive signed values, and then averaged; by taking the square root of this average, we get a single figure that describes the residuals behavior in a standardized manner. The form of the equation often used (for example, ASPRS 1989) is: where n is number of check points observed and d 1 to d n represent the check point residuals. However, classical textbook theory states that, for the interpretation to be generalized to a statistical statement for all populations (and not just that under test), the mean should be computed using (n-1) and not n. The RMS roughly and conveniently corresponds to the 1-σ level in statistics, i.e., around 68% of the residuals observed in an image would be expected to be less than the RMS, with the remainder being worse. Sometimes in statistical testing it is useful to estimate the 90%, 95% or 99% levels, and these can be approximated by multiplying the RMS value by 1.65, 1.96, or 2.58, respectively (Kachigan, 1991). Map Scales and Map Class Map scales can be broadly classified by two characteristics: their geometric quality and their information content. When producing orthoimagery, both are of primary consideration, and are related to the radiometric quality of the image (number and type of bands) and the image resolution. In practice, the two characteristics are highly correlated, an orthoimage of high information content usually capable of meeting a similar quality geometric product specification. We have adopted the term Class 1, originally used in a draft ASPRS scheme, to describe an orthoimage product that has an information content broadly compatible with its geometric quality. An image with a high level information content, but a lower geometric quality content, is described by us as having a Class 2 quality description. A good example in this study is the QuickBird Standard product after our rectification, which clearly has the information content of a typical 1:10,000 scale product but for which the geometric classification is closer to a 1:20,000 map (better than 5m RMS). In this study, we have focused upon the geometric quality of the images: it is necessary still that we evaluate the information content of these images for the agricultural scope of our application in more detail, and this will be part of a future study. Analysis of Variance Analysis of Variance, or AOVA, is a common statistical tool used to analyze datasets for which we are interested in evaluating the importance of several factors at once. In essence, the principles of AOVA are the same as those for the comparison of two datasets using the well-known Student s t-test, but for which the application is extended to groups of three or more datasets (in our case two kinds of QuickBird rectification and one IKOOS dataset, or different numbers of GCPs used in that rectification process). Both these tests examine the significance of the differences of the mean values of each dataset, with respect to the variability of the observations (in our case, RMS estimates) measured for that dataset. Furthermore, several factors can be tested at the same time, such as in our experiment where we have tested the number of GCPs used in parallel with the type of image used. More complicated experiments where the effect of factors such as test site, DM quality, and so forth, could be incorporated into the design. continued from page 485 approximate positions located on a medium-scale navigation plan of the area. The selection was limited to well-defined points, visible on the image during preparation for the field phase, with examples given in Figure 2a-b. Our guidelines for orthoimage quality control stipulate that ground control points must have an accuracy specification three times better than the orthoimage product specification, the latter established in this case as 2.5 m RMS (Kay et al., 1997, uropean Commission, 1999). It was thus necessary to acquire points with an RMS of better than 0.8m; in practice this is most reliably achieved with the use of carrierphase GPS equipment. A 12-channel dual frequency carrier phase system was used, together with associated post-processing software. The goal was reliable measurement of points with better than decimeter accuracy. For this project, GPS survey baselines were relatively short (less than 15km): nevertheless, GCP occupation times of up to 15 minutes were made, in order to ensure that uncertainty in the GPS positioning was eliminated as a factor in the image geometry assessment. All survey work was carried out in the WGS84 system, using the UTM projection (zone 36). It can be seen from Table 2 that the repositioning of the check reference points was very successful, with no planimetric error exceeding 0.07m. One survey point (Piagina) produced a significant elevation error with respect to the survey documentation, but still within the acceptable error of the project survey requirements. We thus concluded that we were able to locate GCPs within the local reference system with the decimeter horizontal accuracy targeted for the study. GPS measurement of GCPs was performed using the same equipment as mentioned above. Out of the planned 50 points, four (points 5, 8, 19, 22) were abandoned due to access problems. However, each of the 25 sections had at least one GCP that was successfully measured. 486 May 2003 PHOTOGRAMMTRIC GIRIG & RMOT SSIG

4 Figure 2a. Figure 2b. Figure 2. xamples of ground control point locations, with the image sub-sections taken from QuickBird (June 2002, left image) and IKOOS (January 2002, right image), and the survey photograph. GCP selection was made exclusively from well defined points on the QuickBird image. Some differences between the QuickBird and IKOOS images can be observed; it should be recalled that the images were acquired six months apart. Table 2. Base station check points. The results presented in this table show that the location of surveyed check points, when compared to positions in the official national survey network, have horizontal residual errors better than decimetre accuracy (maximum error 0.07m). The same survey standard was used for the GCPs collected for testing image correction accuracy. asting (m) orthing (m) MSL levation (m) d (m) d (m) dz (m) Alona Arradipou Diplopotamos n/a n/a Mouti_Farangou Survey point Piagina Quality Control of DM and Delivered QuickBird Standard Image The DM was derived from digitized contour lines with an interval of 25 meters, extracted from the national 1:50,000 map series. The contour lines were transformed into a triangulated irregular network (TI) and then converted to a 25m grid using ArcGIS 8.0 (SRI, 2003). The surveyed GCPs were superimposed on the raster grid, and the Mean Sea Level (MSL) elevation value selected from the corresponding pixel in the DM. The residual result between the DM elevation and the surveyed elevation was then calculated on all 46 measured GCPs; the result obtained was less than 5m RMS, confirming the suitability of the DM for production of an orthorectified image with a nominal 1:10,000 scale, in accordance with our guidelines. All 46 GCPs were used for an initial assessment of the delivered Standard corrected image product; the position for each point was observed in the image and the coordinates (asting, orthing) compared with those collected by GPS, and the residual differences computed. Following the uropean Commission quality assessment guidelines, blunder errors were defined as points with a residual of more than three times the RMS of observed data set; however, no blunder errors were identified and all check points validated. The quality of the delivered Standard corrected image was thus estimated as RMS 6.89 m, RMS 8.70 m, on the basis of these 46 independent points. This is well inside the specified accuracy of the QuickBird Standard product; as expected, however, it is insufficient to meet the geometric demands of our application and so a full orthorectification with the use of ground control is required. This was the next step in our study. continued on page 488 PHOTOGRAMMTRIC GIRIG & RMOT SSIG May

5

6 Table 3. Results of RMS computed for each orthoimage. Orthocorrection was based upon the use of one or more calibration GCPs, and up to 28 validation GCPs. The one-dimensional (either ast or orth) RMS value was calculated from the residuals observed on the validation points. From these results it is possible to estimate the mean expected RMS when using between one and four calibration GCPs, for each of the three image types. QB Standard QB Basic IKOOS 1 GCP RMS RMS 1 GCP RMS RMS 1 GCP RMS RMS GCP RMS RMS 1 GCP RMS RMS 1 GCP RMS RMS GCP RMS RMS 1 GCP RMS RMS 1 GCP RMS RMS GCP RMS RMS 1 GCP RMS RMS 1 GCP RMS RMS dimensional RMS, so the residual data, measured and presented in the Table 3 as a one-dimensional RMS value in either the orthing or asting directions, were converted to a two-dimensional RMS (RMS 2D ) using the formula: An initial two-factor AOVA analysis of the RMS 2D residual showed that both the number of GCPs used and the image type were significant factors (>99.9% level). Inspection of the results (Figure 4, page 490) revealed an important difference for the QuickBird Standard image that we had rectified, which clearly produced a lower quality orthoimage product compared to the other two images. However, the rectification approach applied does not substantially alter the relief displacement corrections already made during the Standard processing of the data by the image provider, using a coarse DM: consequently, the result of the corrections is broadly equivalent to a continued on page 490 PHOTOGRAMMTRIC GIRIG & RMOT SSIG May

7 continued from page 489 simple translation 3. Since the main result was outside our target specification of 1:10,000 Class 1 mapping, the data for this product were excluded from further analysis. A further two-factor (GCP, image type) AOVA was performed on the reduced data set of the QuickBird Basic image and the IKOOS Geo image; results were again significant for both factors. However, the interaction term (Kachigan, 1991) of the analysis was also very significant, again indicating that the result was dependent upon both the GCP numbers and the image type. Inspection of the results shows that for one GCP, the RMS obtained is more variable (Figure 5). Final results were then calculated on the basis of the data collected for image orthorectification using two, three or four GCPs only. In this mode, the difference in RMS 2D visible in Figure 4 between the products is still significant (p < ); however, no significant differences were computed for the number of GCPs used, implying that 2 or more points are suitable for obtaining a good correction. It should be noted that the results presented here are only statistically valid for the test site and image orthocorrection approaches (combinations of sensor, GCP and DM data) investigated; further work will be required to reliably determine the true performance potential of these spaceborne sensors in a broader context for the agricultural mapping application proposed. Additionally, the reported constraints of the RPC method with respect to image extent and method limitations (Toutin, 2003) need to be recognized. everthe- less, we believe that the results do indicate strongly the potential for both these imaging systems to produce mapping at the scale required for agricultural applications within the uropean Union. To Summarize, the results showed satisfactorily that: The external quality control approach applied helps the objective evaluation of geometric quality, and permitted the comparison of the different image data products on a single test site. The QuickBird Standard product delivered was inside the announced geometric specification of 14m RMS. The QuickBird Standard product, further rectified using the vendor supplied RPC data and 2 to 4 GCPs, remained outside the specification required for U rural area mapping (1:10,000 Class 1), with a mean one dimensional RMS of 2.72m and a RMS of 2.43m (maximum limit of the 95% confidence interval was 2.93m). However, a Class 2 specification (same information content, but with a 1:20,000 mapping scale equivalent) was easily achieved using this approach. A mean RMS of 1.60m and 1.13m, and mean RMS of 1.29m and 1.15m were observed for the IKOOS and QuickBird (Basic product) orthoimages respectively: a significant difference in the performance of the two systems was detected. The upper 95% confidence limit of the one-dimensional RMS of orthoimages made from IKOOS Geo and QuickBird Basic images were 1.65 and 1.27 meters respectively: Both systems are Figure 4. Plot of mean estimate of two dimensional RMS plotted by image type. The black diamond represents the mean value, and the error bars represent the 95% confidence interval for the estimate. Some suspected statistical outliers were identified in this analysis in nearly all cases these correspond to results obtained using just one GCP. It can clearly be seen that the QuickBird Basic and IKOOS orthoimage products were superior to that produced from the QuickBird Standard image. evertheless, the QuickBird Basic image produced the highest quality product in this test. Both the IKOOS and QuickBird Basic orthoimage products were well within the 1:10,000 Class 1 quality specification. Figure 5: Plot of two dimensional RMS variation with GCP number, QuickBird Basic and IKOOS Geo results combined. The black diamond represents the mean value, and the error bars represent the 95% confidence interval for the estimate. Only a weak decreasing trend is observed with an increase in the number of GCPs used: the slight increase in mean value (and variability) for the use of one GCP is due to the reduced reliability of operator precision when a single point is taken for control. The use of more than two GCPs is not considered worthwhile in order to meet the 1:10,000 Class 1 specification used for this application. 490 May 2003 PHOTOGRAMMTRIC GIRIG & RMOT SSIG

8 well within 1:10,000 scale Class I accuracy requirements of the U Common Agricultural Policy. For the orthorectified QuickBird basic product, the two-dimensional RMS result (mean 1.62, upper limit of 95% confidence interval 1.72m) was just inside the equivalent 1:5,000 Class 1 mapping requirement of 1.77m. However, the 1-D RMS exceeded the allowed error budget for this specification. A DM with 5m RMS (vertical) was sufficient to achieve orthocorrection within target specification for the conditions of this study (agricultural landscape, moderate sensor view angles). Increasing the number of ground control points from two to four made no significant difference to the quality of the correction result for either of the systems tested. It is recommended that more than one GCP is used, since errors in control point data cannot be compensated for or detected without any redundancy. Footnotes 1 For more information on Quickbird products see website: or For information on IKOOS products, see website: It should be noted that since this study was carried out, a new QuickBird Standard product destined for orthocorrection has been announced by Digital Globe. References ASPRS, ASPRS interim accuracy standards for large scale maps, Photogrammetric ngineering and Remote Sensing, 56: Dial, G., and Grodecki, J. (2002) IKOOS accuracy without ground control, Integrated Remote Sensing at the Global, Regional and Local Scale, ISPRS Commission I Mid-Term Symposium in conjunction with Pecora 15/Land Satellite Information IV Conference, ovember 2002, Denver, CO. ( pdf) SRI, urimage (2002). Products and Services catalogue, uropean Commission, (1999) Guidelines for Quality Checking of orthoimagery, Kachigan, S.K. (1991) Multivariate statistical analysis, Radius Press, ew York. Kaichang Di, Ruijin Ma, and Ron Li, Rational functions and potential for rigorous sensor model recovery, Photogrammetric ngineering and Remote Sensing, 69(1): Kay, S., Léo, O., Meyer Roux, J., Delincé, J, and Van de Steene, M. (1997), Operational activities involving airborne remote sensing related to the Common Agricultural Policy, Proceedings of the 3 rd International Airborne Remote Sensing Conference and xhibition, 7-10 July 1997, pp I Petrie, G., (2002) Optical Imagery from Airborne & Spaceborne Platforms: Comparisons of Resolution, Coverage & Geometry for a Given Ground Pixel Size, GeoInformatics, 5(1): Space Imaging urasia (2003) Online products catalogue, Toutin, T (2003), Geometric processing of remote sensing images: models, algorithms and methods, IJRS 25, (in press). Toutin, T, and Cheng, P., (2002) QuickBird a milestone for high resolution mapping, arth Observation Magazine, Vol. 11, o. 4. Acknowledgments The authors would like to thank the following persons who helped undertake this study: The anonymous reviewer, whose comments and suggestions much improved the submitted draft of the article; Livio Rossi, Bruno Biagini and Fabio Volpe of urimage for QuickBird image provision, technical advice and comments on the draft manuscript; iek Schaap, of Space Imaging urasia and Gene Dial, Jacek Grodecki and Adrian Zevenberger of Space Imaging, Inc. for their assistance in providing images, technical advice and comments on the draft manuscript; Thierry Toutin (CCRS, Canada) for information on image rectification approaches. Simon Kay, Peter Spruyt, and Kyriacos Alexandrou Mars Unit Institute for the Protection and Security of the Citizen, TP266 Joint Research Centre of the uropean Commission, Ispra (VA) Italy simon.kay@jrc.it, peter.spruyt@jrc.it Kyriacos Alexandrou Department of Agriculture, Ministry of Agriculture, atural Resources and nvironment icosia, Cyprus PHOTOGRAMMTRIC GIRIG & RMOT SSIG May