Sensor Data Fusion in Inland Navigation

Sensor Data Fusion in Inland Navigation Andrzej Stateczny, Witold Kazimierski Faculty of Navigation, Chair of Geoinformatics Maritime University of Szczecin 70-500 Szczecin, Waly Chrobrego 1-2, Poland a.stateczny@am.szczecin.pl, w.kazimierski@am.szczecin.pl Abstract: Safety of inland navigation has become for many years more and more important problem because there is tendency to move cargo from land and train transportation to inland shipping. To improve inland navigation River Information Services (RIS) have been established. In the European Union the implementation process of RIS is steel in progress. For example first in Poland RIS for Lover Odra River will be completed to the end of this year. Problems of sensor data fusion in the process of inland navigation are described in the paper. 1. Introduction Radar is one of the most important navigational sensor in the process of navigation not only at the sea but also in much more difficult environment inland waterways. Inland navigation is much more complicated than maritime because of depth and gap under the bridge restriction, where draft and air draft should be compare with local conditions. Important problem in inland navigation is target detection and tracking in narrow and curve waterways with very close distance of approach. Tracking of vessels in inland shipping is the key matter in the aspect of navigational safety and traffic management. In the European Union this issue is a part of wider concept of River Information Services (RIS), being a basis of Tactical Traffic Image (TTI) and partly Strategic Traffic Image (STI) [1]. The main idea of RIS is to harmonize the activities of all inland water users and organizations dealing with them in order to ensure the most possibly effective and safe inland shipping. One of the main services in RIS is tracking and tracing. The requirements for tracking and tracing in inland waterways stated both in European Community documents as well as in IALA regulations, focus on AIS, as the most effective source of information about targets. Simultaneously however radar is still indicated as the main source of such information for the needs of navigational safety. Target Tracking in inland shipping can be defined in three different ways: estimation of target s approach and movement parameters, based on information received from sensors; monitoring movement of the vessel for the needs of transport planning and logistics often referred to as tracing; focusing on the target and continuously showing it on the screen this approach is known from computer vision and is applied to cameras. The first one is the most important from the point of view of safety of navigation. Typically used river radars does not provide a function of target tracking in the way it is understood at sea, namely estimation of state vector. The cause of it is the characteristics of inland shipping. Because of its bathymetry and topography, inland waters are very specific areas. IN fact they are more similar to roads than to sea environment. Large and frequent course and speed alterations requires waiting for vector stabilization. Additionally great number of small boats, which manoeuvre a lot, confirms the need of fast and accurate estimation. Simultaneously it

shall be pointed out, that requirements for bearing and range measurements accuracy for river radars are stricter than for sea radars, although they do not affect tracking directly. As it was shown on earlier research presented also in IRS 2011 [2,3] and IRS 2012 [4,5,6,7] the problem of sensor planning for tracking in inland waters requires special solutions [8,9,10,11 Traditionally used in maritime VTS radars shall be supplemented with other systems, like CCTV or smaller radars along the river. One of the concepts designed in Maritime University of Szczecin and presented in IRS 2011 [12], assumed the use of small broadband X-band radar in extremely important and dangerous part of rivers. Very important is also proper fixing of own ship s position usually done by DGPS although another idea - comparative navigation methods are especially useful because of nearness to riverside and chart information compare with radar images could give possibility to find proper way in restricted areas [13,14]. Especially useful for inland navigation is to use FMCW radars according to their known advantages [15,16,17,18]. 2. Comparative navigation on inland waterways. The problem of positioning has been dominated in recent years by satellite systems, the GPS in particular. There is, however, a need to apply methods of determining the position of ships which are autonomous and independent from external sources of information. This concerns particularly inland navigation. The basic merit of this method is that it obtains direct geodetic control of the riverside and connected to the information from electronic chart about bottom, which is the basic requirement for navigation in waters with limited depths. In recent years more navigational accidents caused by striking the bottom or various submerged obstacles than by collisions with other ships have been observed. Problems of comparative navigation on the basis of radar image compared with electronic chart was described in several works [19,20,21,22]. 3. Radar and AIS data fusion. Basic source of information for tracking and tracing, according to inland waters standards is AIS. The confidence to Inland AIS is so great, that in practice some systems do not include radar stations at all. Properly functioning AIS, based on satellite derived positions, which are more precise than radar positions, is more reliable if all the ship s systems are in force. Also the fact of relatively high cost of establishing many radar stations to cover with range all area of mostly curved inland waters cannot be excluded. Another advantage of AIS against radar is supplementing dynamic navigational data with static information such as identification of ship. The subject of operational reliability of AIS is still open, especially interesting is the question: what is the inaccuracy of AIS if the supporting facilities, like ship s GPS or log are malfunctioning? AIS relay their errors. It is commonly noticed that one target has two completely different movement vectors. One is from radar and one from AIS. Usually the other one is wrong. It is then thought, that radar station can be of important matter in RIS and they should be posted at least in the areas of high risk of collisions. Key problem in radar-ais fusion is the process of target association in which system has to decide if both AIS and

radar targets presents the same physical object. When solving this problem, various fusion algorithms may by applied, usually based on multisensory Kalman filter. 4. Radar and camera data fusion. To increase possibilities of tracking system, it may be worth of implementing the video cameras in it [23]. Lately, the huge progress in the area of computer vision can be seen. Based on this, it can be stated, that usage of these sensors should allow not only increasing of tracking accuracy, but also supplementing of dynamical data from radar with static information about object from camera. It can be notice, that none of the sensors (radar, camera) separately is able to provide complete information about traffic. Thus, only integration of these two sensors into one system can make it up to requirements of modern tracking. The integration of radar and camera in inland navigation can be done in a few layers [23]. First, it can be understood as the fusion of dynamic navigational data about movement of targets, coming from different sensors. In this way, the information about target s course and speed enters the filter, where one common movement vector is estimated with the help of suitable mathematical algorithm. Provided that efficient filter is used, for example filter based on neural networks [24,25,26,27], the estimated vector shall be more accurate and stable, than inputs. The second level of integration is to combine CCTV and radar stations as supplementing sensor in a geographical sense to fill in lacks of coverage. In the third level, the camera provides additional information about target tracked by radar. Aspects of planning of combined system of radars and CCTV cameras for inland waterways was presented in [12]. 5. FMCW radar and automotive radar data fusion The new idea is to use two radars in the process of inland navigation FMCW (Frequency Modulated Continuous Wave) radar and automotive radar. Tasks for FMCW radar is to use radar image for comparative navigation for ship s positioning between river banks and task for automotive radar is moving target detection. Situation on narrow inland waterways is quite similar like on the road. Ships are passing each other at very close distance. Automotive radar will control area in front of the ship in narrow angle for moving target detection. Broadband radars, known also as a FMCW radars are more and more popular alternative for the pulse radars at sea. The technology is usually used on small boats and yachts (small antennas) or in VTS systems (large antennas). In case of boat the antennas are part of multifunctional navigation systems and in case of VTS they are a part of radar monitoring system chain integrated in VTS center. In both situations tracking of targets is enabled for the user and is provided by software manufacturer. Particularly interested from the tracking point of view are the systems provided for boats, which has very small scanners (ab. 0,6 m), which means horizontal beam width of 5,2. This can be however reduced to 2,6 with digital signal processing. Such radars can be used in RIS systems for traffic monitoring. In the developed concept broadband radars can be mounted on or under the bridge and monitor the traffic along the river. An example of screenshot, while working with radar on the bridge is presented in figure 1. The barge approaching a bridge can be noticed.

Figure 1 Example of screen of broadband radar mounted onto the bridge (prior to digital bean reduction) Alternatively radars can be mounted at shore but this would usually mean building of towers or other constructions. It has to be however pointed out that due to inland waters shape, the effective range of observation is usually a few hundred meters ( up to the nearest corner). This means that it is pointless to use traditionally used tracking algorithms [28], which requires up to 3 minutes waiting time for vector estimation with suitable requirements (according to IMO). Thus alternative possibilities are needed. Even use advance neural networks filters (like the one presented in [29])would not be good enough. The authors propose to use automotive radars for this purpose. The idea arose while analyzing different possibilities. The similarities between roads and rivers (inland fairways) were noticed, like: short straight distances; a lot of corners (course and speed alteration); small passing distances (comparing to sea conditions); movement along the river/ road (usually). All this encouraged authors to consider implementation of automotive radars for tracking of vessels in inland waters, next to already included broadband radars. Thus the need of integration broadband and automotive radar appeared. Some problems of integration of such different radar s solution should be foreseen. Problems of using Automotive Radar Systems was described in previous IRS conferences [30,31]. 6. Final remarks The paper presents the research results and new idea of sensor data fusion in several aspects of inland navigation. A new concept of using automotive radar in inland waters itself and the basics of proposed integration as well is presented. At the beginning the basics of both technologies automotive radars (pulse Doppler and CW/FM) as well as broadband X band

radar is analyzed. The first tests are very promising. It has been noticed, that inland traffic is more similar to road traffic than to maritime environment. Barges usually use right side of the waterway and most of the collision situations are head-on situations. This is significantly different than at sea were maneuverable area is almost not restricted and most of collision threats are crossing situations. Thus the idea of using automotive radar arose. It is assumed that this radar will be mounted on the bow and will alert if any target approaches from bow sectors. The ranges of a few hundred meters offered by automotive radars should be sufficient especially for relatively slow inland ships. The paper presented a concept and not full solution, thus an analysis of strengths and weaknesses of this idea is also included. Empirical research are to be performed to confirm the idea. References: [1] Guidelines and Recommendations for River Information Services. (coauthor Stateczny A.) PIANC, Brussels 2011. [2] Kazimierski W., Łubczonek J., Analysis of broadband radar picture in the aspect of marine target tracking, International Radar Symposium (IRS 2011), Leipzig, Germany. [3] Kazimierski W., Stateczny A., Adjusting multiple model neural filter for the needs of marine radar target tracking. International Radar Symposium (IRS 2011), Leipzig, Germany, 2011 [4] Kazimierski W., Łubczonek W. Vertification of marine multiple model neural tracking filter for the needs of shore radar stations. 13th International Radar Symposium (IRS 2012), 5th Microwave and Radar Week. Warszawa, 2012, pp. 554-559. [5] Kazimierski W., Stateczny A. Optimization of multiple model neural tracking filter for marine targets. 13 th International Radar Symposium (IRS 2012), 5 th Microwave and Radar Week. Warszawa, 2012, pp. 543-548. [6] Kazimierski W., Zaniewicz G., Kozak M. Mobile radar set for the needs of target tracking algorithms analysis. The European Navigation Conference on Navigation for People (ENC 2012). Gdańsk, Poland,2012. CD edition. [7] Kazimierski W., Zaniewicz G., Stateczny A. Verification of multiple model neural tracking filter with ship s radar. 13 th International Radar Symposium (IRS 2012), 5 th Microwave and Radar Week. Warszawa, 2012, pp. 549-553. [8] Łubczonek J. The accuracy of determination of radar shadows. Annual of Navigation, 19/2012/part 2, Gdynia 2012, ISSN 1640-8632, pp.41-52. [9] Łubczonek J., Application of Modified Method of Viewshed Analysis in Radar Sensor Network Planning on Inland Waterways., 13th International Radar Symposium (IRS 2012), 5th Microwave and Radar Week., Warszawa, 2012, pp. 269-274. [10] Łubczonek J., Stateczny A., Spatial planning of land-based marine radar sensors by using 3D cartographical model of the port and urbanized areas. Coast GIS, 2011, Oostend, Belgium. [11] Lubczonek, J., Stateczny, A., Aspects of spatial planning of radar sensor network for inland waterways surveillance. 2009 EUROPEAN RADAR CONFERENCE (EURAD 2009) Book Series: European Radar Conference-EuRAD, 2009, Rome, ITALY. [12] Łubczonek J., Kazimierski W., Pałczyński M., Planning of combined system of radars and CCTV cameras for inland waterways surveillance by using various methods of visibility analyses. International Radar Symposium (IRS 2011), Leipzig, Germany. [13] Stateczny A., Comparative Navigation. Gdansk Science Society, Gdańsk 2001 (in polish). [14] Stateczny A., Methods of comparative plotting of the ship's position. MARITIME ENGINEERING & PORTS III Book Series: WATER STUDIES SERIES Volume: 12, 2002. Editor(s): Brebbia CA; Sciutto G. [15] Adamski, M.E.; Kulpa, K.S.; Nalecz, M.; Wojtkiewicz, A., Phase noise in two-dimensional spectrum of video signal in FMCW homodyne radar. Microwaves, Radar and Wireless

Communications. 2000. MIKON-2000. 13th International Conference on, vol.2, no., pp.645-648 vol.2, 2000. [16] Kulpa, K.S. Novel method of decreasing influence of phase noise on FMCW radar. Radar, 2001 CIE International Conference on, pp.319-323, 2001. [17] Kulpa, K.S., Focusing range image in VCO based FMCW radar. Radar Conference, 2003. pp. 235-238, 3-5 Sept. 2003. [18] Kulpa, K.S.; Wojtkiewicz, A.; Nalecz, M.; Misiurewicz, J., The simple method for analysis of nonlinear frequency distortions in FMCW radar. Microwaves, Radar and Wireless Communications. 2000. MIKON-2000. 13th International Conference on, vol.1, no., pp.235-238 vol.1, 2000. [19] Stateczny A., (ed.), Methods of Comparative Navigation. Gdańsk Science Society, Gdansk 2004 (in polish). [20] Stateczny A., Comparative Navigation as an Alternative Positioning System, Proceedings of the 11th IAIN World Congress Smart navigation Systems and Services, 2003, Berlin, Germany. [21] Stateczny A., Praczyk T., Artificial Neural Networks in Maritime Objects Recognition. Gdansk Science Society, Gdańsk 2002 (in polish). [22] Stateczny, A., Artificial neural networks for comparative navigation. ARTIFICIAL INTELLIGENCE AND SOFT COMPUTING -ICAISC 2004 Book Series: Lecture Notes in Artificial Intelligence Volume: 3070, 2004. Editor(s): Rutkowski L; Siekmann J; Tadeusiewicz R; Zadeh LA. [23] Stateczny, A., Kazimierski, W., Integration of radar and video sensors for the needs of target tracking in inland shipping. International Radar Symposium (IRS 2009), Hamburg, Germany. [24] Stateczny, A., Kazimierski, W., A comparison of the target tracking in marine navigational radars by means of GRNN filter and numerical filter. IEEE RADAR CONFERENCE VOLS. 1-4, 2008 Rome, ITALY. [25] Stateczny, A., Kazimierski, W., Determining Manoeuvre Detection Threshold of GRNN Filter in the Process of Tracking in Marine Navigational Radars. 2008 PROCEEDINGS INTERNATIONAL RADAR SYMPOSIUM, (IRS 2008),Wroclaw, POLAND. [26] Stateczny, A., Kazimierski, W., Selection of GRNN network parameters for the needs of state vector estimation of manoeuvring target in ARPA devices. Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments IV Book Series: PROCEEDINGS OF THE SOCIETY OF PHOTO-OPTICAL INSTRUMENTATION ENGINEERS (SPIE) Volume: 6159, 2006. [27] Stateczny, A., Neural manoeuvre detection of the tracked target in ARPA systems. CONTROL APPLICATIONS IN MARINE SYSTEMS 2001 (CAMS 2001) Book Series: IFAC. Editor(s): Katebi R. PROCEEDINGS SERIES. [28] Stateczny A., (ed.), Radar navigation. Gdansk Science Society, Gdańsk 2011 (in polish) [29] Kazimierski W. Analysis of tracking errors of multiple model neural filter for marine navigational radar. Elektronika. No 7/2012, ISSN 0033-2089, SIGMA, Warszawa 2012, pp. 47-50. [30] Heuel S., Rohling H., Two-Stage Pedestrian Classification in Automotive Radar Systems. International Radar Symposium (IRS 2011), Leipzig, Germany, 2011. [31] Meinecke, M. M., Rohling, H., Combination of FSK and LFMCW Modulation Principles for Automotive. Radars, German Radar Symposium GRS2000, Berlin, Germany, 2000.