GIS Test Data y. Gabriele Neyer z. September Geographic Information Systems (GIS) handle objects embedded in the

Transcription

1 GIS Test Data y Gabriele Neyer z September Introduction Geographic Information Systems (GIS) handle objects embedded in the space. Digital satellite imagery, scanned aerial photos, elevation models and scanned maps are typical sources of geographic data sets. Graphics data is traditionally divided into two classes: vector data and raster data which is also called bitmap data. Vector data consists of points, lines, arcs and areas with topographic information associated. Often vector data sets are classied according to the data they contain. Typical classes are boundaries of states, rivers, buildings and vegetation. Raster data are a collection of pixels, organized in a series of rows. Although many organizations try to create a unique vector format and a unique raster format, up to now each country uses its own format. For an overview of the spatial data standards around the world see: spatial.html This web page is provided by the International Cartographic Association (ICA) which is working on a world spatial data standard. Thus, since there is no European spatial data standard we had to choose some important le formats for our \representative" test data sets. For raster data our choices are TIFF and GeoTIFF which is TIFF compatible. This format is a standard le format which is used and known all over the y This work was partially supported by grants from the Swiss Federal Oce for Education and Science (Projects ESPRIT IV LTR No CGAL). z Institute for Theoretical Computer Science, ETH Zurich, Switzerland, neyer@@inf.ethz.ch 1

2 world. Supplying representative vector data was a harder problem. We decided to provide vector data in AutoCAD DXF format which can be interpreted by most CAD programs. The DXF format is widely used as a least-common-denominator format. Additionally, we provide vector data in TIGER format and SDTS format since these are standard US vector data formats, the data sets cover the whole USA and are freely available. We do not \really" provide the test data sets, we only provide addresses, where the testdata can be found (for free). Since geographic maps often need a lot of space (several MB) and usually people only work with one or at most two dierent le formats, only a few people would install a library several MB large, containing a few examples of each le format. This description consists of 6 parts. In Section 2 we describe the vector data format. Section 3 describes the raster data format. In Section 4 we discuss format conversion between dierent raster and vector data formats. Section 5 describes the selected vector formats: The DXF format is described in Section 5.1, the TIGER format in Section 5.2 and the SDTS format in Section 5.3. In Section 6 we describe the selected raster data formats: The TIFF format is described in Section 6.1 and the GeoTIFF format in Section 6.2. The description is based on J.D. Murrays and W. vanrypers Encyclopedia of Graphics File Formats [MV94] and on the particular documentation of a le format. 2 Vector Data In computer graphics, vector data usually refers to a means of representing lines, polygons, or curves or any other object that can easily be drawn with lines by numerically specifying key points. The job of a program rendering this key-point data is to regenerate the lines by somehow connecting the key points or by drawing using the key points for guidance. Always associated with vector data is attribute information (such as color and line thickness information) and a set of conventions or rules allowing a program to draw the desired objects. These conventions can be either implicit or explicit and, although designed to accomplish the same goals, are generally dierent from program to program. Although vector les vary considerably in design, most contain the same basic structure: a header, a data section, and an end-of-le marker. Some structure is needed in the le to contain information global to the le and to correctly interpret the vector data at render time. The header contains information that is global to the vector le and must be read before the 2

3 remaining information in the le can be interpreted. Such information can include identication number, version number, color information as well as default attributes, which will apply to any vector data elements in the le. In the data section the vector data is organized according to certain rules. Such a vector data element can consist of a type (e.g. line, point or arc), coordinates, thickness, color, etc. Thus, vector les are useful for storing images composed of line-based elements such as lines and polygons, or those that can be decomposed into simple geometrical objects, such as text. More sophisticated formats can also store three-dimensional objects such as polyhedrons and wire frame models. The vector data can be easily scaled and manipulated to accommodate the resolution of a spectrum of output image. Additionally, they can easily be modied with simple text editing tools. Individual elements can be added, changed or removed. Usually, it is easy to render vector data and save it to a bitmap format le or to convert the data to another vector format. The disadvantages of vector formats are that vector les cannot space economically be used to store extremely complex images, such as some photographs, where color information is paramount and may vary on a pixel by pixel basis. Since each image element must be drawn individually and in sequence, reconstruction of images in vector data format may take considerably longer than reconstruction of images in raster data format. 3 Raster Data Historically, the term raster has been associated with cathode ray tube (CRT) technology and has referred to the pattern of rows the device makes when displaying an image on a picture tube. Raster-format images are therefore a collection of pixels, organized in a series of rows, which are called scan lines. Because raster output devices, by far the most popular kind available today, display images as patterns of pixels, pixel values in a bitmap are usually arranged so as to make them easy to display on certain common raster devices. A newer term for raster data is bitmap data. We will use this term from time to time. Raster data can be produced when a program renders graphics data and writes the corresponding output image to a le instead of displaying it on an output device. This is one of the reasons bitmaps and bitmaps data are often referred to as images, and bitmap data is referred to as image data. Other sources of raster data are raster devices used to work with images in the traditional sense of the word, such as scanners, cameras and other 3

4 digitizing devices. Raster les consist of a header, raster data and other information. The header typically contains information about the le version, identier, the color palette and image specic data like the number of lines per image, the number of pixels per line, the number of bits per pixel, compression type, etc. For the organization of the raster data in the le there exist various methods. The simplest is the organization of pixel values into rows or scan lines. Two other methods of le organization are strips and tiles. In the rst case the images are stored in strips, which consist of rows stored contiguously. The total image is represented by more than one strip and individual strips may be widely separated. Tiles are similar to strips in that each is a delineation of a rectangular area of an image. However, the width of a tile is variable. The pros of raster formats against vector formats are the following: Raster formats can be easily created from existing pixel data stored in an array. Furthermore, pixel values may be modied individually or as large groups by altering a palette if present. The cons is that raster les can be very large, particularly if the image contains a large number of colors. Data compression can shrink the size of pixel data, but the data must be expanded before it can be displayed and this can slow down the reading and rendering process considerably. Additionally, raster les do not scale very well. Shrinking an image by decimation can change the image in an unacceptable manner. 4 Format Conversion File conversion is a big problem when you try to convert between les of dierent basic format types raster to vector, for instance. Successful conversion between basic format types is not always possible due to the great dierences in the ways data is stored. For converting one raster format to another raster format there exist several software packages. pbmplus is a specially good tool, freely available, and designed for UNIX systems. The pbmplus software can be retrieved at the following address: ftp://ftp.x.org/r5contrib/pbmplus10dec91.tar.z Two problems can occur when converting between vector formats. The rst comes about due to dierences in the number and type of objects available in dierent vector formats. Some formats, for instance, provide support for only a few simple image elements, such as circles and rectangles. Richer 4

5 formats may also provide support for more complex elements, such as b- splines and Bezier curves. Attempting to convert a le written in a complex format rich in elements to a simpler format will result in an approximation of the original image. The second problem comes from the fact that each vector format has its own interpretation of measurements and the appearance of image elements and primitives. Rarely do two formats agree exactly on the placement and appearance of even simple image elements. Common problems are those related to line joint styles and end styles, and to centerline and centerpoint location. 5 Selected Vector Data Formats In the following we give a short overview over the vector data formats we selected: 5.1 DXF File Format The AutoCAD DXF and the AutoCAD DXB formats are associated with the CAD application AutoCAD, created and maintained by Autodesk. DXB is a binary version of a DXF le used for faster loading and is apparently tailored for use by AutoCAD. DXF supports the ability to store three-dimensional objects and to handle associative dimensioning. It is a vector format which is supported by AutoCAD, Corel Draw and many other CAD programs. Because DXF was created in support of a CAD program, there is good support for included text. In spite of this, DXF is widely used as a least-common-denominator format for the exchange of simple line data. The Open Inventor 3d Toolkit which is a standard toolkit on Silicon Graphics, oers conversion routines from DXT to Inventor (DxfToIv). The Inventor 3d File Format is SGI's standard for 3d data. The Inventor toolkit includes numerous programs besides a SceneViewer, which demonstrates Inventor scene graph les in three dimensions. Note that the rendering mechanism inside Inventor objects employs Open GL. For further information about Open Inventor read the manual page (man inventor) or check out the following WWW address: See directory CGAL/examples/Polyhedron IO/ (of your local cgal installation) for the source and description of more conversion routines that are provided by CGAL. Each DXF le consists of ve sections: a header, tables, blocks, entities sections, and an end-of-le marker. The header contains zero or more groups 5

6 of header variables that contain information applying to the entire image. The tables section contains organized data that is referenced by other sections of data. Table data may include information on font sizes and styles, line type descriptions, and layer information. The blocks section describes each block of information that is found in the image. The entities section contains the actual object data of the image. The end-of-le marker is the string EOF; it appears as the last line in the le. A full documentation of the DXF le format can be retrieved at: toc.htm Further information about DXF is given by: Sample data sets including digital elevation models in DXF format of the swiss region Albis/Tuerlersee can be found at: They have been published by the Swiss Bundesamt fur Landestopographie. The testdata consists of points and polylines. One testdata set is scaled 1:25'000 and contains 8 dierent categories of data. The other set is scaled 1:200'000 and contains 11 categories of data. A large collection of sample data sets from the United States in DXF format can be found at the following address: Further data sets from the United States in DXF format can be found at: ftp://ftpmcmc.cr.usgs.gov/release/dxf/ 5.2 Census TIGER File Format The TIGER/Line les, 1995, are extracts of selected geographic and cartographic information from the Census TIGER (Topologically Integrated Geographic Encoding and Referencing) data base. The le format is a vector le format. The Census TIGER System provides support for the creation and maintenance of a digital geographic data base that includes complete coverage of the United States. The design of the Census TIGER data base adapts the theories of topology, graph theory, and associated elds of mathematics. The Census TIGER les contain data describing three types of features: line features, landmark features and polygon features. Line features are roads, railroads, hydrography, transportation features, selected power lines 6

7 and boundaries. Landmark features are point landmarks such as schools and churches; area landmarks such as parks and cemeteries and key geographic locations such as apartment buildings and factories. Polygon features are geographic entity codes for areas; locations of area landmarks and locations of key geographic locations. The Census TIGER les contain information about the spatial objects distributed over a series of record types called the topology of the data set. The topology explains how points, lines, and areas relate to each other and is used as the foundation for organizing spatial objects in the Census TIGER database. A le consists of 17 record types that collectively contain geographic information. Each record type specializes a certain attribute of an object. E.g. record 1 contains a data record of an object and record 2 contains the shape coordinates. Files in TIGER data format for all 102 Illinois Counties can be retrieved at: A full documentation can be retrieved from (pdf-format): The Census TIGER homepage has the following address: 95.html 5.3 SDTS Vector File Format Spatial Data Transfer Standard (SDTS) is a standard which by denition is \a document that species a set of rules". The SDTS provides a way of facilitating the transfer of digital spatial data between dissimilar computer systems. It also preserves the information meaning and minimizes the need for any external information. This standard not only allows the transfer of spatial data, attributes and georeferencing but also the data quality report, data dictionary and any other associated metadata. SDTS was approved as Federal Information Processing Standard (FIPS). Publication 173 in 1992 after 12 years of development and testing and in 1994 became mandatory for federal agencies. SDTS is available for use also by state and local governments, the private sector and research and academic organizations. Promoting and facilitating the transfer of spatial data between dissimilar computer systems provides users and producers of spatial data with a way to gain access to a greater amount of otherwise inaccessible data. It also promotes data exchange and data sharing, which in turn increase the quality and the integrity of existing spatial data. 7

8 SDTS is designed to support all types of spatial data. A single translator that could support all the dierent types and options of data is probably not practical. It is much better to implement SDTS through the use of proles. A prole is a subset of SDTS which was created to transfer a specic type of spatial data with as few SDTS options as possible. The Topological Vector Prole (TVP) was the rst developed and applies to geographic vector data with planar graph topology. This prole will handle both USGS DLG-3 and DLG-F data as well as the Census Bureau's TIGER data. The Raster Prole was developed to accommodate image data, digital terrain models, gridded GIS layers, and other gridded data. This prole will accommodate USGS DEM's and DOQ's. SDTS information including the Standard itself and much more is available via ftp at: ftp://sdts.er.usgs.gov/pub/sdts More information on SDTS is also available by visiting the SDTS web page located at: Sample data can be retrieved at: 6 Selected Raster Data Formats In this section we describe the TIFF raster data format and the GeoTIFF raster data format. 6.1 TIFF TIFF is a standard le format found in most paint, imaging, and desktop publishing programs and is a format native to the Microsoft Windows GUI. TIFF's extensible nature allowing storage of multiple bitmap images of an pixel depth, makes it ideal for most image storage needs. The TIFF imagery le format can be used to store and transfer digital satellite imagery, scanned aerial photos, elevation models, scanned maps or the results of many types of geographic analysis. TIFF is a full-featured format in the public domain, capable of supporting compression, tiling, and extension to include geographic meta-data. The description in this chapter covers the current TIFF version 6.0 which was released in June TIFF les are organized in three sections: the Image File Header (IFH), the Image File Directory (IFD), and the bitmap data. Of these three sec- 8

9 tions, only the IFH and IFD are required. A TIFF le which contains multiple images has one IFD and one bitmap per image stored. TIFF has a reputation for being a complicated format in part because the location of each Image File Directory and the IFD points to including the bitmapped data may vary. In fact, the only part of a TIFF le that has a xed location is the Image File Header, which is always the rst eight bytes of every TIFF le. All other data in a TIFF le is found by using information found in the IFD. Each IFD and its associated bitmap are known as a TIFF suble. Each IFD contains one ore more data structures called tags or eld. Each tag is a 12-byte record that contains a specic piece of information about the bitmapped data. A tag may contain any type of data, and the TIFF specication denes over 70 tags that are used to represent specic information. TIFF supports the following types of data compression: JPEG compression method for use with continuous-tone color and gray-scale images; LZW compression, Run Length Encoding and CCITT T.4 and T.6 compression. More information including a TIFF specication, mailing lists and more can be retrieved at: GeoTIFF GeoTIFF refers to TIFF les which have geographic (or cartographic) data embedded as tags within the TIFF le. The geographic data can then be used to position the image in the correct location and geometry on the screen of a geographic information display. GeoTIFF is a metadata format, which provides geographic information to associate with the image data. GeoTIFF fully complies with the TIFF 6.0 specications, and its extensions do not in any way go against the TIFF recommendations, nor do they limit the scope of raster data supported by TIFF. A full version of the GeoTIFF specication can be retrieved from: or, for those using only ftp: ftp://mtritter.jpl.nasa.gov/pub/tiff/geotiff USGS maintains a mirror site of this spec at: ftp://ftpmcmc.cr.usgs.gov/release/geotiff 9

10 A le containing frequently asked questions concerning GeoTIFF can be retrieved at Data sets from the United States in GeoTIFF format can be retrieved at the following address: ftp://ftpmcmc.cr.usgs.gov/release/geotiff/images/ These les have various scales and pixel sizes. Information about the specic meta data of each le is given in the README le of that directory. The Swiss Bundesamt fur Landestopographie also provides sample data sets in geoti format. The pixelmaps have a resolution of 508 dpi. The pixelsize has been adjusted such that small lines of 0.05mm are still plotted and the quantity of data is minimum. The pixelsize of 0.05mm corresponds to 1.25m in the nature with scale 1:25. The language of the maps and the description (c.f. pkdocu.doc, pkdocu.rtf and pkread.txt) is written in german. The data sets and further information about the data can be retrieved at: References [MV94] J.D. Murray and W. VanRyper. Encyclopedia of Graphics File Formats. O'Really and Asoociates, Inc., Reading,