Semi-Automated Road Extraction from QuickBird Imagery. Ruisheng Wang, Yun Zhang

Transcription

1 Semi-Automated Road Extraction from QuickBird Imagery Ruisheng Wang, Yun Zhang Department of Geodesy and Geomatics Engineering University of New Brunswick Fredericton, New Brunswick, Canada. E3B 5A3 <Ruisheng KEY WORDS: Kalman Filtering, Profile Matching, Auto-Tuning, System Noise ABSTRACT: Kalman filtering has found a wide application in navigation and computer vision. Recent years it has been also used in road extraction from aerial photograph. However, a rigorous application of Kalman filtering turns out to be difficult due to the difficulty of determination of system noise and measurement noise. In this paper, we use Kalman filtering for road extraction from QuickBird imagery. A new and effective method for determination of system noise and measurement noise is presented, which leads to a significant improvement in road extraction. The method is efficient even for the extraction of partially shadowed and occluded road. high-resolution satellite imagery, especially QuickBird imagery.. INTRODUCTION The launch of commercial high-resolution satellites, such as IKONOS and QuickBird marked a revolution in the remote sensing industry. The high-resolution satellite imageries (m or less) are available periodically since the September of 999. Compared to aerialphotograph, it has many advantages: large coverage, high temporal resolution and availability of multi-spectral information. To effectively utilize the information inherent in such valuable data sources, automatic information extraction is important, such as road extraction, building extraction, and other object extractions. Road is one of the important manmade objects, whose information is essential for cartography, traffic management and planning of urban and industrial areas. In this paper, the research is therefore focusing on the development of efficient methods to extract roads from the Roads look like elongate regions in the highresolution satellite imagery. But, in fact, a road has more complex appearance in a image due to other disturbing objects beside or on the road, such as trees, buildings, vehicles, and shadows. More complex situations are often in the urban area. Such irregularities limit the effectiveness of the existing road extraction approaches such as edge based (Nevetia and Babu, 98), profile matching based (Quam, 978; Vosselman and de Knecht, 995), and snake or dynamic programming based techniques (Gruen and Li, 997). Because of the complexity of roads, the measurement of road center is subject to various influences so that an accurate measurement is not always available. But a road has its inherent regularity like smoothness and continuity, which can be employed to predict the next road position. Therefore, Quam (978) used this predicted value to guess the road center position

2 in the case that the accurate measurement is unavailable. McKeown and Denlinger (988) integrated the similar method into their cooperative way for road tracking, which combined the edge-based approach with profile matching based approach. Vosselman and de Knecht (995) first utilized the Kalman filtering and least square profile matching for road tracing, which combined the measurement with the predicted value to obtain an optimal road estimate. Baumgartner et al. (22) used the same style of the approach (Vosselman and de Knecht, 995) to build an efficient tool for semiautomated road extraction. The paper of Vosselman and de Knecht (995) showed a promising research direction in road extraction. However, in their paper the road model assumed that all the roads have a constant curvature. The variances of the system noise are derived from the possible curvature changes in road design, and estimated by using human knowledge on the road design. This means the system noise is set as a constant during the road tracing. In reality, however, roads always have arbitrary shapes, and the deviation between the real measurement and the system prediction often changes during the tracing. In this paper, therefore, we present a new method to determine the system noise of the Kalman filtering, in which the system noise is altered according to the feedback of current tracing condition. The new system model of Kalman filtering is closer to the real road situation. The implementation results show that the presented method is efficient even for the extraction of partially shadowed and occluded roads. 2. BACKGROUND Kalman filtering has found many applications in navigation and computer vision as a predictorcorrector algorithm for estimating the parameters of a dynamic system. It can achieve an optimal estimate by using all available information in a recursive fashion. The road extraction can be simulated as a dynamic process by considering the distance along the road as a time variable (Vosselman et al. 995). In this case the road position is the parameter to be estimated as the state. The Kalman filtering consists of two steps: the time update and the measurement update. The first step can be considered as a predicting procedure, in which the system model needs to be known. The more accurate system model is, the better results are. But in reality, an accurate model is normally difficult to be determined. The error between the predicted value and the actual position often exists, and it is considered as the system noise. An incorrect determination of the system noise will cause an inaccurate final result. The second step can be thought of as a correcting procedure because the final result is a weighted average between the predicted value and the current measurement. The weight is derived using the Kalman filter. For example, if the predicted value is affected by the system noise, the corresponding measurement update will improve the predicted value by an actual measurement at that time. Whereas, the predicted value will be used as the final result when the measurement is unavailable, or combined with measurement to get the final result when the measurement is not accurate. If the measurement noise correctly reflects the accuracy of the measurement, the final result will respond the measurement correctly. In road extraction sometimes the predicted value and actual measurement are the same, especially in the road without any shadow and occlusion. In this case whatever value are taken for the variance of the system noise, this will have no effect on the final result. But when the road has some occlusions like vehicles or small changes like unclear edge, the current measurements will be inaccurate due to the disturbance. For getting accurate road tracing results, correct variances of

3 system and measurement noise are very important. If the weight is not proper, it will cause an incorrect Kalman filtering result, which will result in an inaccurate road tracing. Therefore, it is important to adjust the Kalman filtering result according to the road situation during the tracing. In this paper, the system noise is adjusted according to the distance difference between the predicted road center point and the measured road center point for semi-automated road extraction from the QuickBird imagery. 3 ROAD EXTRACTION The core of the proposed road extraction algorithm is a profile matching combined with an auto-tuning Kalman filtering to optimize the final results. The auto-tuning Kalman filtering consists of a conventional Kalman filtering and the proposed system noise adjustment. The human operator just needs to click two points along the centerline of the road to set up an initial model profile of the road. The center point and axis direction of the road within the model profile are then calculated based on the initialization, which are used for the prediction of the next road center at the beginning of a road tracking. After the calculation, a quadratic curve fitting is employed for estimating the road direction. In the end, the profile matching at the predicted position achieves a more accurate measurement result of road center. The modified system model of the Kalman filter with noise adjustment for road extraction is described as: rt + rt + dt * cos( α t ) x = = () c ct + dt *sin( α t ) where r, c are the row and column coordinates of the road center position at time. r t, c t are the row and column coordinates of the road center position at time t. dt is the interval of these two center points, normally pixel. α t is the road direction at time t and it is estimated by the curve fitting. α t will not be involved in the recursive updating of the Kalman filter parameters. It is determined directly from a quadratic curve fitting based on the previous road information. The covariance matrix of the predicted state vector is T P = APt A + Qt (2) where A consists of the coefficients of the linearized time update equation. Q t is the variance of the system noise. It can be a constant and can also be changed according to the update or measurement each time. In this paper the Q t changes during the road tracking procedure. Because the system noise reflects the error between the predicted value and actual value, the distance between the predicted value and the filtered value of the Kalman filtering is a good approximation for the determination of the system noise. In order to avoid the error between the filtered value and the actual value, an average value, determined using the distances between 2 predicted points and the corresponding filtered points, is used for determining the system noise. On the other hand, the measurement noise of the Kalman filtering is derived from the correlation coefficient of the profile matching and the variance of the road center position within the model profile. The measurement model E is E = H x = x (3) The final filtered output x^ is x = x + K z Hx ) (4) ( where Z is the measurement value, and k i+ is the Kalman gain and which is described as: T T K = P H ( R HP H (5) + )

4 with R being the measurement noise. 4. RESULTS AND CONCLUSION The method based on the optimal estimation of Kalman filtering, proposed in this paper, can effectively reduce the influence of measurement noise. In the case that a road has a junction, contains vehicles on it or be partially shadowed, the tracking can still achieve a satisfactory result in spite of these influences. Figure.a and.b show different tracking results of a road containing other objects and having various road widths and unclear edges. Figure.a is the result using a conventional Kalman filtering without the adjustment of system noise, and Figure.b shows the result with the autotuning Kalman filtering (conventional Kalman filtering plus system noise adjustment). It can be clearly seen that the road central line cannot be accurately extracted without the adjustment (Figure.a), but it can be very well extracted with the adjustment (Figure.b). Figure. Comparison of road extractions without and with the adjustment of the Kalman filtering. (a) Road central line extracted without the adjustment, (b) Road central line extracted with the adjustment. Figure 2 shows the Kalman gain K values for the circled road line in Figure.a. Figure 2.a shows that the K values for row coordinates are all above.5, and those for column coordinates are fluctuated along.5 This tells that after the Kalman filtering the tracing result is still very close to the measurement result in raw direction. That means the Kalman filtering did not achieve an effective filtering result. Because the road line in the circled area is not horizontal, the measurements usually contain errors. Hence, the tracing errors of the road extraction still remain. On the other hand, most of K values for column direction are below.5. This shows that most of column coordinates are from the prediction. Compared to Figure.a, it be seen that the filtering in column direction does not result in an accurate correction. Kalman gain K value for row coordinates.2.8 Series (a) (a) Kalman gain value for column coordinates (b) (b) Series

5 Figure 2. Kalman gain K without the system noise adjustment for row and column coordinates Figure 3 shows the Kalman gain K values for the circled road line in Figure.b, which is extracted after the integration of the system noise adjustment for row and column coordinates. It can be seen that the K values are fluctuated along.5 in both row and column directions. This means that the integration of the system noise adjustment into the Kalman filtering has resulted in a strong noise adjustment, which effectively removes the error from the measurements (a) Kalman gain K value for row coordinates Kalman gain K value for column coordinates Series Figure 4. Tracing in a partially shadowed road 5. Conclusion The proposed integration of a system noise adjustment into the Kalman filtering has shown a clear improvement in road extraction. The tracing can be successful in the curved and shadowed road as well as crossing a road with a junction. With the adjustment of the system noise, the error from measurement due to the disturbing influences can be effectively eliminated. Hence, this greatly improves the tracing s stability. However, the determination of the system noise and measurement noise in this paper is also an approximation. A more rigorous determination of the system and measurement noise is further required..8.4 Series References (b) Figure 3. Kalman gain K with the system noise adjustment for row and column coordinates Figure 4 illustrates that the tracking is successful for curved road with partial shadow. Kalman, R.E. (96): A New Approach to Linear Filtering and Prediction Problems. ASME Journal of Basic Engineering, vol. 82D, pp Quam, L.H. (978). Road Tracking and Anomaly Detection in Aerial Imagery. Proceedings of DARPA Image Understanding Workshop, pp McKeown, D.M. and J.L. Denlinger (988): Cooperative Methods for road tracing in aerial imagery. IEEE Proceedings on Computer Vision and Pattern Recognition, Ann Arbor, MI, pp

6 Vosselman, G., Knecht, J.D. (995). Road Tracing by Profile Matching and Kalman Filtering. Automatic Extraction of Man-made Objects from Aerial and Space Images, Birkhäuser Verlag, Basel, pp Baumgartner, A., Hinz,S., Wiedemann, C. (22). Efficient Methods and Interfaces for Road Tracking. International Archives of Photogrammetry and Remote Sensing, Vol. 34, Part 3B, pp Nevatia, R., Babu, K. (98). Linear Feature Extraction and Description. Computer Graphics and Image Processing, 3 (3): Grün, A., Li, H. (997a). Linear Feature Extraction with 3-D LSB-Snakes. Automatic Extraction of Man-made Objects from Aerial and Space Images (II), Birkhäuser Verlag, Basel, pp Grün, A., Li, H. (997b). Semi-Automatic Linear Feature Extraction by Dynamic Programming and LSB-Snakes. Photogrammetric Engineering and Remote Sensing, 63(8):