GEOINFORMATION VIA PARALLEL IMAGE ENGINEERING AND IMAGE INFO RMATION MINING

Transcription

1 GEOINFORMATION VIA PARALLEL IMAGE ENGINEERING AND IMAGE INFO RMATION MINING Dr.-Ing. Werner Mayr (1), Dr.-Ing. Timm Ohlhof (2) (1) CONPIE GmbH, Oskar-Frech-Str. 15, D Schorndorf, Germany (2) ESG Elektroniksystem- und Logistik -GmbH, Einsteinstraße 174, D Munich, Germany ABSTRACT Earth observation is a fundamental source of information in today s society. A multitude of spaceborne and airborne sensors collect daily an enormous amount of information in the form of imagery of the earth. It is evident that systematic approaches for information mining in such imagery are required. New sensors and image processing techniques extend the image information base as well as improve quality and speed of availability. In this paper we give information on a new, European, high-resolution optical sensor, which is currently implemented for airborne use and will contribute to geo-referenced earth observation. The project includes also a new approach for image processing. Via parallel computing for the purpose of fully automated generation of digital surface models and orthomosaics, fundamental geometric and visual geoinformation is built up. All of this would be cumbersome without a metadata information system, which allows for archiving and retrieval, dissemination and workflow management. Thus, the paper presents aspects of image generation as well as aspects of image information mining. 1 INTRODUCTION There are a multitude of European initiatives for progressing Europe s needs in social, economical, technological and other domains towards a better, more balanced, competitive and safer environment for Europeans and people in all continents. One of these domains is Earth Observation. Various sensors and thus technologies are applied. In many cases images are somehow generated and then processed i.e. measured, interpreted, visualized, analyzed, fused and by other means. Common to all these procedures is the generation of new, timely concurrent information about the earth. Some of the applications are of European or national interest, such as global monitoring, change detection and disaster management support or land mine detection. Some of them are also of interest in regional or local domains. Thus, there is a fluent transition from global to local and vice versa. Optimization and effective utilization of the full information content requires systematic archiving, data management, standards for information interchange and means of image information mining (IIM). The first step in the processing chain is image generation, which is done in many different ways. Newly generated images often go into an interim database to be used for primary image data processing. After this step, value-added images are stored i.e. archived. Later retrieval will use search criteria generated during the value-adding processing step, also known as metadata of the imagery. This paper concentrates on the early steps of IIM. Image generation, first level image processing with automated information extraction and storing results in a webbased archiving and retrieval system for imagery and other geospatial data are these first IIM steps to be presented. We will report on a novel large format digital camera system, which is now tuned for aerial applications but may also be adapted for space. It opens a swath-width of up to 20,000 pixels across flight track. With its current 10,000 pixels across track, a volume of about 200GB is recorded in 1 hour when flying at 2,000m above ground level (AGL), delivering a ground resolution of 20cm. Accordingly, a volume of nearly 400GB per hour is generated from 1,000m AGL and 10cm ground resolution. Samples of such imagery will be shown. As flying times of 1h are usually the lower limit of a flight mission, one can imagine the amount of raw image data generated during a flight season. In the first processing level, all of this image data is georeferenced and primary information like digital surface models and orthomosaics is extracted automatically and made available for later use in various, independent applications. It is obvious that the above-mentioned quantities of raw and processed data require highly automated, reliable and fast processing procedures. Our implementation is carried out in a parallel computing environment. Some samples and results of parallel image data processing on a PC cluster are described. The paper will report on a scenario, within which all of these aspects can become implemented with standard off-the-shelf systems and products. This is a rather demanding engineering task and requires joint efforts of various disciplines. The combination of aerial images, satellite images, derived products such as nationwide Digital Elevation Models (DEM), and orthomosaic databases are the data foundation of IIM. To exploit petabytes of image information, geocoding and automated workflows for a metadata information system (MIS) for imagery and other relevant geospatial data appears to be a necessary step. To this end, well-proven standards in image archiving and information interchange are of vital importance to IIM. A discussion of these aspects will conclude the paper. Paper-ESA-WM-TO-16mar04.doc Page 1 of 6

2 2 DIGITAL IMAGE GENERATION Ubiquitous sensors increasingly support our daily life. Examples are the domains of health, sports, engineering or geosciences. Of particular and increasing interest is our environment. Many applications are based on environmental monitoring. Earth Observation (EO) is the key e.g. for ecologic interest, forestry, agriculture, geological surveys, mapping, mega-city growth monitoring, fugitive movement monitoring, disarmament and proliferation control or homeland security surveillance. Quite a number of EO programs and related EO satellites are well known. Their characteristics are large ground coverage and a geographically often very wide operational area. Also, high-resolution satellite imagery enlightens enthusiasts in the spaceborne EO domain. The higher the ground resolution is, however, the fewer are the satellites and the harder is it to capture imagery of a selected region. Not contrary, but complementary to this fact, it can be stated that about 1,000 to 1,500 aerial cameras are in commercial operation worldwide [1]. Nearly all of them are (wet) film-based aerial cameras and they deliver excellent high-quality images, up to 1cm ground resolution level, anywhere, any (day) time, under direct control and without operational restrictions. Thus, mo st of the commercial requirements could be fulfilled up to now. This is changing and it coincides with the increasing multidisciplinary demands for information and knowledge derived from EO, which is part of the geoinformation (GI) domain. Advances in the semiconductor and computer industries bring powerful sensors and electronics to the range of many commercial users, in particular mapping companies equipped with mapping airplanes. Consequently, the device manufacturing industry started the rollout of the paradigm change from film-based cameras to sensor-based imaging devices. A small number of digital aerial large format cameras at the product level are commercially available on the market. However, the U.S. industry dominates this device supply industry nearly to 100%. Nevertheless, a group of six European small and medium enterprises teamed up with two research institutes to investigate the options for design and development of a new metric camera (NMC) as a large format digital aerial camera system. In order to achieve best possible degrees of freedom in the hardware design, we had the whole system including the image exploitation services in mind. Some user requirements could be shifted from hardware into software. However, this increased the computing demands. To cope with this, we implemented the image processing tasks in a parallel computing PC environment. The European team formed the project GeoPIE Geoinformation via Parallel Image Engineering. The European Commission funds GeoPIE under contract ID IST of the 5th Framework Program. NMC an EO sensor The NMC camera head is a digital camera suitable for airborne use, preferably in fixed wing vehicles. NMC is a modular system and comprises one or more identical camera heads (CH). At present, NMC is planned to be available in configurations of 1 Camera Head (1CH), 2 Camera Heads (2CH) and 3 Camera Heads (3CH). Sketches of the three configurations are displayed in Fig But a 4 CH configuration and even more CHs are also feasible. With less than 5kg and about 295 x 105 x 110 mm 3, a NMC CH can fit in many cabin compartments of flying vehicles. CH1 Nadir Flight Direction Fig. 1. Schematic 1CH NMC configuration Flexibility The NMC CH operates in color, nm, with a fixed focal length of close to 50mm. By the use of a filter, alternatively the NIR band, nm, is accessible with co-registration of the red and green bands. The combination with another NMC CH allows for two different 2CH system configurations. In a symmetric assembly, one CH is forward looking (FWL), while the other one is backward looking (BWL), see Fig. 2. One can, however, also place one CH into the nadir looking position and thus obtain either a forward looking and nadir looking or vice versa a nadir looking and backward looking 2CH configuration. For many applications, this might be suitable. For higher demands, a third, nadir looking NMC CH is configurable, see Fig. 3. Each CH is equipped with 2 ruggedized industrial PCs. The Control Computer Unit interfaces to a flight management system (FMS), controls exposure and synchronizes image capture for the 2CH or 3CH configurations. NMC sustains an exposure frame sequence 0.55sec per image capture. FWL BWL Flight Direction Fig. 2. Schematic 2CH NMC configuration Paper-ESA-WM-TO-16mar04.doc Page 2 of 6

3 CH - Nadir CH2 FWL CH3 BWL Flight Direction Fig. 3. Schematic 3CH NMC configuration the frame camera model. When complementing it in the 3CH configuration, it acts similar to the TLS approach. There, one CH looks forwards, one nadir and one backwards. The inclination angles of forward and backward looking CHs are ±22.5. This guarantees a favorable and constant 45 intersection angle at any given point in object space i.e. on earth. In order to reduce the amount of required images for contiguous stereo overlap, in particular in high altitude flights, we also developed special suspension mount for lightweight loads. Fig. 5 shows the 2CH NMC installation in its suspension mount. No moving parts The NMC system concept follows the strict design constraint of applying as many off-the-shelf components as possible. Some worldwide active and experienced mapping companies influence and support the ongoing development of the NMC system. Only optical and electronic components find use in NMC. There are no moving parts inside the camera head and a suspension mount is also not required for operation. Even the electro-mechanical shutters elsewhere are implemented electronically. No moving parts are required in order to achieve Forward Motion Compensation (FMC), since NMC implements an Optical FMC. Fig. 4 shows a 2CH NMC system installed into an aircraft. Fig. 5. 2CH NMC mounted in its suspension mount inside a mapping airplane Fig. 4. 2CH NMC mounted into a mapping airplane, seen from below Best of both There are two competing sensor concepts for photogrammetric purposes: 1D linear arrays like HRSC or ADS40 and 2D area arrays like DMC. Inspired by both principles, 1D linear-arrays and 2D area-arrays, NMC follows the reliable frame camera imaging geometry model proven over decades and combines it with the new-age 3-line-scanner (TLS) principle, thus offering the well-proven TLS -along-track-stereo imaging feature as well. The imaging geometry of a single NMC CH is Many pixels one projection center NMC combines in one CH the typical wide across track resolution of a TLS with about 10,000 pixels and extends this width towards a 2-dimensional detector array of about 1,600 lines along flight track. To achieve this, NMC applies the multiple detectors in one focal plane technique and operates 9 CCD detector arrays per focal plane, each having a pitch of less than 5 microns. They comprise a 10,000 x 1,600 pixel sized image, which the GeoPIE team calls an Exposure Frame (EF). Simultaneous exposure of all 9 CCDs and thus one EF is guaranteed. Each EF comprises about 25MB data and may be taken within 0.2msec. Unlike others, NMC uses only one projection center per EF. This allows for two unique but geometrically important properties. First, a rigorous restoration of a frame image consisting of several single images is only Paper-ESA-WM-TO-16mar04.doc Page 3 of 6

4 possible if all projection rays pass through the identical projection center, as in NMC. Second, due to its special way of usage of the single detectors, NMC can geometrically calibrate itself in-situ. This unique feature allows for various automated control mechanisms in NMC. Computer and storage One Control Computer Unit (CCU) manages NMC, which is an off-the-shelf configured, 2U high, 19 rack mounted industry PC suitable for harsh flying environments. The CCU communicates with the FMS and reports to it. The on-board FMS triggers the multiple camera head NMC, which appears as one frame camera to it, and all other camera related events. The CCU controls all NMC CHs and guarantees simultaneous exposure of 2CH or 3CH configurations, see Fig. 2 and Fig. 3. For data storage, each NMC CH connects to a 19 rack mounted dual computer configuration, called Camera Head Computer and Storage (CHCS). Two computers, technically identical to the CCU, build up the CHCS for one NMC CH. Each CHCS connects via Ethernet to the CCU. The CH uses a broadband interface to CHCS. This configuration allows per CH for up to 1.8 EFs/sec to be stored onto the two external 250GB PC hard disks of the two CHCS computers. Each CCD image of each EF is stored with a unique identification schema. The radiometric resolution is 12 bits per channel. Raw image data are stored. Several EFs are bundled into one file for easier handling. Each stored EF is supplemented with metadata such as time tags, dark current, check bits and other NMC registrations. Fig consecutive exposures of 2 adjacent raw CCD images 3 IMAGE EXPLOITATION SOFTWARE NMC imagery is of the nature of the frame camera model. Existing standard image processing and photogrammetric exploitation software is applicable in most cases. NMC s CCDs are arranged in such a way, that they optically overlap in across-track direction, which is displayed in Fig. 6. Exposure time control triggers and guarantees the contiguous overlap in along-track direction. When shifting the raw CCD images into their overlapping positions, we obtain a configuration as shown in Fig. 7. It demonstrates the sophisticated contiguous and overlapping schema, as well as clarifying that these central perspective CCD images cannot fit together smoothly without post-processing. Terrain undulation varies the amount of overlap of consecutive EFs. An along-track overlap of 20% to 60% is achieved depending on NMC exposure interval and undulation. Image exploitation techniques and methods may be grouped into: Fig. 7. Above 4 raw CCD images in overlapping position Georeferencing Surface model and orthomosaic generation 3D visualization Feature extraction Image information mining (IIM) Knowledge-based systems Imagery intelligence (IMINT) Decision support tools Paper-ESA-WM-TO-16mar04.doc Page 4 of 6

5 GeoPIE focuses on surface model and orthomosaic generation via a parallel computing approach, where several tested with up to 16 PCs connected via Ethernet and running under LINUX process a block of imagery. This environment is a testing phase and currently operating on scanned aerial photography. Fig. 8 shows an overview of a fully automatically processed image block of about 60GB input data amounting to one orthomosaic of about 14GB at 0.5m ground sampling distance and a surface model of 2.5m grid spacing. Fig. 9 shows a detail of a regarding automated elevation modeling complex overpass / underpass situation of a highway crossing. Fig. 8. Fully automatically generated orthomosaic from 42 aerial images Fig. 9. Complex highway crossing, automated elevation modeling (circled in Fig. 8) 4 IMAGE ARCHIVING AND DATA MANAGEMENT In order to take advantage of the full image information content, systematic geo-referenced archiving and retrieval, data and workflow management, standards for information interchange and image information mining (IIM) techniques are needed. The combination of aerial images, satellite images, digital maps, GIS data, control points, derived products such as nationwide DEM, and orthomosaic databases are the data foundation of advanced image exploitation. To handle petabytes of image information, automated production workflows and an efficient metadata information system (MIS) for imagery and other relevant geospatial data are necessary. This MIS should offer the following features: Data model based on international ISO/TC211 and OGC (Open GIS Consortium) standards Handling of centralized or distributed databases of different types, processing level, data formats, contents, size and availability Automated archiving, requesting and dissemination Streamlined end-to-end workflows High-speed viewing Online consultation Web-based delivery of geospatial information on time There are only a few operational MIS available worldwide which fulfil the above-mentioned requirements. One such MIS is ESG's GeoBroker which is an intelligent, high-performance solution for the archiving, management, retrieval, display and dissemination of all kinds of imagery and geospatial information. GeoBroker presently supports 53 different data types, including the standardized NATO formats for imagery, elevation data, raster maps and vector data. During data ingest, the original data are automatically converted into standard formats like TIFF and GeoTIFF. All data in the GeoBroker archive are geo-referenced, so that the query results are displayed both in the form of footprints located in a world map and in an attribute list. The geospatial data can be queried, ordered and downloaded in a LA N (local area network), an Internet or Intranet environment. GeoBroker can be implemented as a stand-alone tool or as an integrated application within a GIS software package like GeoMedia [2] or ArcView. Fig. 10 shows the 3-tier system architecture of GeoBroker. The metadata and vector data are stored in an Oracle database, whereas the imagery and other mass data are stored in a file system. The archive data are usually stored in a SAN (storage area network) environment on RAID systems and tape libraries. GeoBroker can access data on a central server or on distributed servers at different locations. An interface to a special raster data server, which also stores its data in a database, is under development. The following data formats are supported: Aerial image data: TIFF, GeoTIFF Paper-ESA-WM-TO-16mar04.doc Page 5 of 6

6 Satellite image data: Ikonos, Quickbird, EROS, SPOT, Landsat, IRS, ERS, JERS, Radarsat Raster data: ADRG, CADRG, ASRP, CRP, USRP, RLE, MRG, KMRG, GeoTIFF, MilGeoTIFF Vector data: VPF (VMap, FFD, DNC), S57, DFAD, DLM, DGN, GeoMedia Oracle Dataserver Elevation data: DTED, GTOPO, DHM, DSM, SRTM Simulation data: OpenFlight, SIF, SEDRIS (in preparation) Point data: TP (trigonometric point), NavP (navigation point), MP (military point), GCP (ground control point), RP (reference point) Maps and charts: Atlas, topographic map, aeronautical chart, marine chart, thematic map, city map, world map, continent map, region map Other data: Slide, video sequence (MPEG, AVI), dossier databases and decision support during civil and military crisis operations. GeoBroker is the central catalogue system of the Bundeswehr Geoinformation Office for all imagery and geospatial data. In the future, it is planned to add a knowledge database to GeoBroker as well as to appoint rules, to design models and methods and to deploy software tools for an IIM workflow, which assist the operator in image exploitation e.g. for feature extraction, target recognition, change detection or other IMINT tasks. 5 CONCLUSIONS The presented GeoPIE project possesses the potential to contribute to geo-referenced image databases and valueadded products. The complete chain from image generation to geoinformation is covered. In combination with a MIS such as GeoBroker, all of GeoPIE s geospatial information immediately becomes valuable geo-related information and is available for IIM. 6 ACKNOWLEDGEMENTS We would like to express our thanks to the European Commission, which funds the GeoPIE project under IST This demanding project, however, would not have its advanced status without the dedicated excellence of its team members. Parts of the work related to GeoBroker were funded by the German Federal Office of Defence Technology and Procurement (BWB). Fig. 10. GeoBroker system architecture The GIS server provides the full GeoBroker functionality for an effic ient data ingest and retrieval. We used an object-oriented modular software design, based on (D)COM technology. This scaleable multiuser system guarantees maximum data consistency and security in order to protect the archive data from unintentional damage or intentional systems hacking. The web server software generates the HTML pages using ASP (active server pages). For the dynamic display of vector data in a web browser, an internet map server is required. To this end, we presently use Intergraph's GeoMedia WebMap Professional. An alternative solution with an open source map server and SVG (scaleable vector graphics) is planned. The web client needs Internet Explorer and the Active CGM control for the vector data display. 7 REFERENCES 1. Mayr W., New Exploitation Methods and Their Relevance for Traditional and Modern Imaging Sensors, Presentation at 22 nd Scientific Annual Meeting of the German Society of Photogrammetry, Remote Sensing and Geoinformation DGPF, Neubrandenburg, Germany, Ohlhof T., GeoBroker: Management of Geospatial Data Powered by GeoMedia, Proceedings GeoSpatial World 2004 Conference, Miami, FL, (in preparation). During the past few years, ESG has customized GeoBroker for several civil, military and intelligence agencies. Typical applications of GeoBroker include imagery archiving, retrieval and dissemination, the build-up and management of large GIS databases (TBytes...PBytes), the generation of simulation terrain Paper-ESA-WM-TO-16mar04.doc Page 6 of 6