Localization and Place Recognition Using an Ultra-Wide Band (UWB) Radar

Transcription

1 Localization and Place Recognition Using an Ultra-Wide Band (UWB) Radar Eijiro Takeuchi, Alberto Elfes and Jonathan Roberts Abstract This paper presents an approach to mobile robot localization, place recognition and loop closure using a monostatic ultra-wide band (UWB) radar system. The UWB radar is a time-of-flight based range measurement sensor that transmits short pulses and receives reflected waves from objects in the environment. The resulting echo waveform is the sum of the reflected waves, and the delay of each component reflected wave depends on the distance between the sensor and the object. Consequently, the received waveform is a superposition of individual echoes that depend on ranging distance, material, shape and radar reflectance of the obstacles in the environment. The echo waveform measured by the UWB radar has more information than what can be provided by an optical sensor of comparable range, as it provides data from inside and beyond obstacles. At the same time, the sensor data is consistent and repeatable for a given location and a static environment. This paper proposes a waveform matching method for place recognition, presents a number of experiments that illustrate the high position estimation accuracy of our UWB radarbased localization system, and shows the resulting loop detection performance in a typical indoor office environment and forest. 1 Introduction An ultra-wide band (UWB) radar is a time-of-flight based range measurement sensor that transmits short pulses and receives reflected waves from objects in the en- Eijiro Takeuchi International Research Institute of Disaster Science, Tohoku University, takeuchi at rm.is.tohoku.ac.jp Alberto Elfes Autonomous Systems Lab, CSIRO ICT Centre, Alberto.Elfes at csiro.au Jonathan Roberts Autonomous Systems Lab, CSIRO ICT Centre, Jonathan.Roberts at csiro.au 1

2 2 Eijiro Takeuchi, Alberto Elfes and Jonathan Roberts vironment. The resulting echo waveform is the sum of the reflected waves, and the delay of each component reflected wave depends on the distance between the sensor and the object. Consequently, the received waveform is a superposition of individual echoes that depend on ranging distance, material, shape and radar reflectance of the obstacles in the environment. UWB radar has a number of potential advantages over other sensors, particularly optical devices such as stereo systems and laser scanners, that are extensively used on mobile robots. The echo waveform measured by an UWB radar has more information than what can be provided by an optical sensor of comparable range, as it provides data from inside and beyond obstacles. UWB radar system is able to see through and beyond obstacles, walls and vegetation, and can detect objects through smoke and fog. Hence, they can be used for reliable localization under a broad range of conditions, which is especially promising for field, search and rescue, and service robotics. Applications of UWB radars have included imaging the environment [1, 2], detecting and tracking people through walls [3], and localization. Several localization methods using UWB radar have been proposed [4, 5, 6, 7, 8]. However, almost all of these methods rely on augmenting the environment with artificial beacons. This paper proposes a place recognition method using an UWB radar based on waveform matching. The full radar echo signal is used as a signature for a given location. These signatures are collected and stored in a database of places, and are subsequently used for place recognition and robot localization when the same stop is revisited by the robot. The proposed method results in accurate estimates of location in real environments without the use of any artificial beacons. The paper also presents a number of experiments that illustrate the high position estimation accuracy obtained by our UWB radar-based localization system, shows the resulting localization and loop detection performance in a typical indoor office environment and forest. 1.1 Related work One of the major applications of UWB radars is penetration imaging of objects using synthetic aperture radar (SAR) techniques [1, 2]. These techniques are used for ground and underground imaging from satellites, subsurface imaging from airborne platforms, and building imaging from ground-based systems. The environment model is estimated using reflected waves measured from different positions. Other applications include through-wall moving object detection, which can be used for tracking of people [3]. Since the UWB radar receives the sum of reflected waves from various objects, moving objects can be detected from temporal changes in the detected waveform. Segura et al. implemented a functional SLAM and localization system using UWB radar sensor [4]. The receiver on a mobile robot measured waves from multi-

3 Localization and Place Recognition Using an Ultra-Wide Band (UWB) Radar 3 ple fixed transmitting antennas and estimated the transmit antenna positions and the robot position using an extended Kalman filter (EKF). Blanco et al. proposed a range-only SLAM-based method using a Rao-Blackwellized particle filter and a sum of Gaussians approximation [5]. The paper reported SLAM experiments using an UWB transceiver and radio beacons. Several other UWB radar-based localization methods have been proposed for mobile robots, including [6, 7] and [8], who demonstrated localization using active nodes or base stations. 1.2 Waveform matching for localization In contrast to previous research, this paper proposes a reflected radar waveform matching method for localization and loop closing. The method estimates positions in real environments without artificially introduced beacons, reflectors or any other special devices. Data retrieval based place recognition methods have been actively studied in the computer vision community [9, 1]. These methods recognize a location by comparing it to similar images retrieved from a database. Similarly, we use the measured reflected waveform as a feature vector that encodes the signature of a place, and localize a mobile robot by comparing this signature to other signatures (measured waveforms) retrieved from a wave signature database. The remainder of this paper discusses the waveform matching based place recognition method and shows that it leads to a very high precision, high recall rate, and accurate localization system. We also show how wave-based database building and localization are done in indoor environments and forest without beacons. 2 UWB Radar The UWB radar used in this paper and shown in Fig. 1(a) is a Time Domain Inc. P4 system [11] with a range of approximately 3 m. The system is small, low power, and has no moving parts. The radar unit has two antennas, the transmit antenna and the receive antenna. The antennas are approximately omnidirectional in the azimuth direction [12], leading to a doughnut-shaped beam pattern. The radar transmits a short pulse and receives reflected waves from various objects in the environment. The resulting echo waveform is the sum of the reflected waves, and the delay of each component reflected wave depends on the distance between the sensor and the object. The received waveform is a superposition of individual echoes that depend on ranging distance, material, shape and radar reflectance of the obstacles in the environment. Since the radio waves can penetrate objects, the received waveform will depend on a large 3D volume of the environment around the radar.

4 4 Eijiro Takeuchi, Alberto Elfes and Jonathan Roberts Transmit antenna Receive antenna (a) UWB Radar P4 Amplitude Time of arrival [ns] (b) Received wave Fig. 1 The UWB radar used in this paper and a measured reflected waveform example. 2.1 Reflected waveform for the UWB radar Figure 1(b) shows a typical received echo for the TD P4 UWB radar unit. As can be seen, the return signal shows a very complex waveform. The echo or received waveform is the sum of the waves reflected by various objects. This is expressed by the following simplified wave function [2]: N 1 f (t) = i= (a i δ(t τ i )) In this equation, δ(t) is the transmitted wave, a i is the strength or amplitude of the reflection from a single object i, τ i is the time of arrival, and N is the number of objects, which will depend on the distance between the radar and the objects in the environment, as well as these shape and radar reflectance. 3 Waveform matching method for UWB radar In this section, a correlation based matching and retrieval method for UWB radars is described. For a given measured echo, the proposed method searches for similar waves in a place signature (measured echo waveforms) database. 3.1 Wave correlation The echo signature database M contains the reflected waveform f i measured at each position x i : M = ( f i, x i ) (i =,...,N 1) where f i = ( f i,, f it,, f it 1 ), x i = (x i,y i )

5 Localization and Place Recognition Using an Ultra-Wide Band (UWB) Radar 5 In these equations, t is discretized time and N is the id of the signature waveform (and of the associated location) in the database. In this paper, the similarity between waveforms i and j is defined as: T 1 E i j = η (w t f it w t f jt ) 2 t= where w t is a weighing coefficient and η is a normalization factor defined as: T 1 η = 1/min( (w t f it ) 2 T 1, (w t f jt ) 2 ) t= If the radar echoes are received at the same position, they will be similar and consequently E i j will be low. Conversely, if E i j is large, the signature echoes will likely have been measured at different locations. The normalization factor is used to keep the values of E i j commensurate. t= 3.2 Finding the peak of the correlation The proposed method finds corresponding locations using the similarity function as follows: 1. measure new echo waveform f j 2. calculate similarities E i j with all possible signatures i in the signature database M 3. calculate minimum value E min of the similarities E i j 4. find additional signature candidates by selecting those k such that E k j E min ε, where ε is a predefined tolerance 5. check the position variance of candidates Var(x k ) In the results shown later in the paper, the acceptable value of ε is.5 and the acceptable variance of position is.3 m. Figure 2 shows examples where the radar signature from a specific position is correlated against the signatures in the database. For this dataset, the database contains signatures. Figure 2(a) shows the correlations for an echo measured very close to position number 1554, which is the matched location. There were two additional candidates within the defined tolerance, which are locationally very close. Figure 2(b) shows the results for a location that had not been previously visited, where no acceptable match was found.

6 6 Eijiro Takeuchi, Alberto Elfes and Jonathan Roberts Correlation Correlation Minimum value Wave ID (a) Query wave is measured at close position of wave number Acceptable value Minimum value Acceptable value Wave ID (b) Query wave is measured at outside of database positions Fig. 2 Correlation of a signature against the wave signature database. Standard deviation Distance[m] Standard deviation Distance [m] (a) Standard deviation of received echo for a moving human (b) Standard deviation of echoes measured at various positions Fig. 3 Standard deviation of received echoes for a moving human. 3.3 Sensitivity weight Due to power dispersion, echoes received from obstacles become weaker with increasing distance to the obstacle. However, the more distant components of place signatures change more slowly, and are therefore more robust to small changes in receiver position. To compensate the diminishing intensity of the echoes from more distant targets, we use the sensitivity weights w t. In this research, the sensitivity weight is designed to flatten the effect from unknown obstacle with distance. To define the sensitivity weight, consider that we have a human moving within the range of the UWB radar. Figure 3(a) shows the standard deviation of the echoes for a moving person. In these experiments, the human moves in front of radar with constant velocity. The red plot is the standard deviation of the echoes, while the dashed black line is a fit using an inversely proportional function. In this figure, the standard deviation shows only effects of moving objects because reflected waves from static objects are almost constant value. The figure illustrates that the standard deviation decreases with distance and the relationship is almost inversely proportional. At the near distances, the standard deviation is decreased that caused by saturation.

7 Localization and Place Recognition Using an Ultra-Wide Band (UWB) Radar 7 Rate of standard deviation Distance [m] Fig. 4 Products of the sensitivity weight and information intensity of echoes This is compensated using the following proportional function as the sensitivity weight: w t = kt For the later results, the parameter k was set to Range and sensitivity To accurately find similar signatures in the database, it is necessary to use the portions of the signature that correspond to more distant objects as they provide more robust information. At the same time, the signature search and matching time depends on the effective range of the signature that was used. This section discusses this effective range. Similarly to what was shown in Fig. 3(b), shows the standard deviation of signatures measured at various position in a static indoor environment, while the dashed line is an exponential function fitted using a least square method. The standard deviation indicates breadth of the value change over the environments, and the high deviation distances are more effective to identify the waves. Again, this shows that the intensity of the information decreases exponentially with distance. The sensitivity weight is increase propotionally with distance and the environment information decreases exponentially with distance. As a result, we can find the sensitive distance of the echoes by multiply these characteristics. The Fig. 4 shows the sensitive distance. As can be seems, the most useful information is in the middle range of the signatures. This research uses a range out to 9 m for signature matching. 4 Experiments This section describes several experimental localization results using the proposed signature matching method. Figure 5 illustrates localisation and loop closure. In the mapping phase, the robot moves around an environment using a meandering motion. This motion increases the number of potential crossing points and consequently of signature candidates for localization and loop closure. Once the robot finds a matching signature in the database, it closes the trajectory using a graph-based SLAM approach [13] [14][15] to keep consistency of database. In the localization phase,

8 8 Eijiro Takeuchi, Alberto Elfes and Jonathan Roberts loop detection position correction Wave data base Mapping with loop detection Localization with retrieve position Fig. 5 Loop detection and localization using radar signature matching. UWB antenna: BroadSpec UWB Radar: PulseON 4 115cm Mobile robot: PatrolBot Experimental environments Fig. 6 Experimental vehicle and indoor environments used for testing at the Autonomous Systems Lab, Brisbane. if the robot finds several similar signatures in the database, it corrects the position using a position estimation method such as an EKF [14]. 4.1 Experimental setup Figure 6 shows the experimental setup and the experimental environment. The mobile robot was a PatrolBot developed by Mobile Robots Inc., and the UWB radar was a PulsOn 4 MRM developed by Time domain Inc. The radar was mounted on the robot at a height of 1.15 m. The experimental environment was a typical indoor office area with meeting rooms, desks and people. The UWB radar has a range of up to 3 m, of which the first 9 m were used as discussed previously. The sampling rate at which signature were collected is 25 Hz. The mobile robot moved at speed under 1 cm/s, and consequently the inter-measurement distances were under 4 mm. 4.2 Waveform retrieval with unknown objects and a moving target The first experiment was a robustness test with unknown objects. The robot repeated a move and stop motion in the environment, and a human moved around the robot randomly when the robot stopped. The distance of the person from the robot was

9 Localization and Place Recognition Using an Ultra-Wide Band (UWB) Radar 9 id of waveform id of waveform Similarity value red: high correlation blue: low correlation Matching results red: true positive green: false negative blue: false positive black: true negative Fig. 7 Waveform retrieval results using radar signature matching. between 1 m and 1 m. Figure 7 shows the similarity and the obtained results. In this figure, the upper triangle shows the similarity values, while the bottom triangle is the matched results. The red points on the upper triangle side indicate high correlation. On the lower triangle the red points show true positive matches, the green points are false negatives, the blue points are false positives, and the black points are true negatives. The true positives mean that the distance between the retrieved signature wave position and the query wave position was under 1 cm. The red squares show places where the robot stopped for varying amounts of time. In this experiment, the precision of retrieval was.999 and the recall rate was.955, showing that the method was robust to the disturbance of having a person moving around the vehicle in an indoor environment. 4.3 Loop detection and closing The following experiments demonstrate how the technique can be used for loop detection and closing to build maps. Here, the robot moved around the laboratory using a meandering route. Figure 8 shows the loop detection results using the proposed method. The red line shows the route taken by the robot, and the blue lines are corresponding points detected by the signature matching method. The route has 23 crossing points and the proposed method detects loops on 22 crossing points. The corresponding pairs are found irrespective of the direction that the vehicle is pointing at, as the radar antennas have omnidirectional sensitivity in the azimuth direction. For this experiment, these were no corresponding pairs with large distance errors. However, there were cases where multiple corresponding pairs were found clustered close to the actual position, leading to small errors in the estimated position. The total number of signatures in the database is 36858, and the mean

10 1 Eijiro Takeuchi, Alberto Elfes and Jonathan Roberts 25 2 y[m] 15 1 Odometry Corresponding pair Goal 5 Start x[m] Fig. 8 Loop detection results. matching/retrieval time is 25ms on an Intel core i7 2.8 GHz processor. The graph had 1263 nodes and 146 constraints, and the graph closing time is approximately.4 s. Figure 9 shows loop closing results using a graph-based SLAM method [15]. The red line is the corrected graph, the green points are laser scanner mapping results using the corrected positions, circles are detected loops and the square is a nondetected loop. Note that the laser scanner was used only for reference, but not for loop detection and localization. The results show that the trajectory was corrected and the laser point clouds overlap well. 4.4 Localization using the radar echo signature database The next experiment was conducted to demonstrate the localization performance achieved using our radar echo signature matching and retrieval method. This experiment estimated the position of the robot using an Extended Kalman Filter [14]. The motion model used vehicle odometry and the measurement model is the retrieved position match using UWB radar signatures. The localization method uses as a map the radar signature database obtained from previous loop closing experiments, and finds the position using radar signature matching and retrieval. Figure 1(a) shows the localization results obtained using signature matching. The experiment was conducted three days after the mapping experiment discussed earlier. The red line is the estimated position using an EKF with UWB radar data, the cyan line is the position associate with the signature stored in the database, the

11 Localization and Place Recognition Using an Ultra-Wide Band (UWB) Radar 11 1 y[m] 5 False negative Mapped points measured by laser scanner True positives Start / goal Corrected trajectory x[m] Fig. 9 Loop closing results. The laser scanner was used for mapping, but not for localization. purple crosses are corrected points, and the blue dashed line is the reference position estimated using a scan matching method with laser scanner data [16]. The laser-based scan matching method estimates the vehicle position using the point clouds that were obtained in the previous experiment, and as a result the reference position has its own, relatively small errors. This figure illustrates that in almost all cases the estimated positions using the UWB radar are close to the reference positions. For this experiment, the total number of signatures in the database is 36858, the search area is 2 m, and the mean matching and retrieval time is 9 ms. Figure 1(b) shows the difference between the estimated position and the reference position. The red line shows the difference in the x-axis and the green line the difference in the y-axis, indicating that almost all position differences were smaller than.2 m. The results show that the localization accuracy achieved using the UWB radar signature retrieval method is close to the laser scanner localization performance. 4.5 Loop detection and closing in forest The final experiment was conducted to demonstrate that the proposed method works in heavy vegetation and forest environments. Figure 11(a) shows the forest area used for the experiments, which is located within the area of the Queensland Centre for Advanced Technologies (QCAT). The PatrolBot used for the experiments is not an outdoor vehicle, but we took advantage of a boardwalk through the forest which provided a route that was traversable by the vehicle. The route is surrounded by trees, heavy vegetation, and some light posts.

12 12 Eijiro Takeuchi, Alberto Elfes and Jonathan Roberts y[m] Reference position Mapped points Wave database points Corrected positions Estimated position Start / goal x[m] (a) Trajectory of estimated position difference[m] time[s] (b) Difference between radar-based vehicle position estimates and reference positions derived from laser scanner estimates. Fig. 1 Localization results. The laser scanner was used for mapping, but not for localization Green points: scan points measured on return path Goal 15 y[m] 1 (a) Experimental environments 5 Odometry Corresponding pair Red points: scan points Start measured on outward path x[m] (b) Loop detection results Fig. 11 Forest area used for outdoor experiments and loop detection results. In this experiment, a straight trajectory following the boardwalk was used on the outward path, while a meandering trajectory was used on the return path. The parameters used in our method were the same as in the indoor experiments. Figure 11(b) shows the loop detection results, and Fig. 12 the loop closing results. The blue line shows the route taken by the robot, the purple lines are corresponding points detected by the signature matching method, the red points are laser scanner mapped results for reference using the estimated positions on the outward path, and the green points are mapped points on the return path. In Fig. 12, the circles are detected loops and the squares are non-detected loops. The route has 25 crossing points and our method detects loops on 21 crossing points. The graph had 19 nodes and 1235 constraints. After loop closing, Fig. 12 shows that the outward and inbound point clouds overlap well. The results show the proposed method can detect crossing points in natural environments.

13 Localization and Place Recognition Using an Ultra-Wide Band (UWB) Radar 13 5 False negatives Goal Start y[m] Corrected trajectory Green points: scan points measured on return path Red points: scan points measured on outward path True positives x[m] Fig. 12 Loop closing results in forest. 5 Conclusions This paper presents an approach to robot loop detection and localization using an UWB radar and an echo waveform signature matching and retrieval method. We have shown that the radar signature waveform is highly correlated with the position of the sensor, that the correlation of waves have strong minima, and that the global minimum can be found using a computationally efficient method. The experiments illustrate the robustness of the proposed method in the presence of a moving human, and show loop detection and closing, as well as localization results. The robustness experiments show that the method retrieves the correct signature with a high level of precision and recall rate, even with a moving target in the scene. The loop detection results show that the method detects almost all crossing points on a route, and that an accurate map is obtained from the loop closing result using graph-based SLAM. Finally, the localization results using radar are compared with laser scan-matching localization, showing that the differences between UWB radar-based localization and laser scanner-based localization are typically below.2 m. Accurate results were obtained for both indoor and outdoor environments. In summary, the experimental results confirm the usefulness of an UWB radar for place recognition. Future work will include a more accurate evaluation of the proposed method, the extension of the technique to more complex indoor and outdoor environments, and the development of higher position localization accuracy using predictive modelling of the radar echo signatures. The proposed localization method may have the ability to localize within a millimeter-order accuracy, but the experiments outlined in the paper are not precise enough to assess this potential accuracy. The UWB radar can detect objects through walls, smoke, fog, and vegetation. Consequently, the sensor and the methods discussed are applicable to domains such as forest traversal or search and rescue in smoky environments. However, the proposed method has only been used to detect crossing points along a 2D trajectory,

14 14 Eijiro Takeuchi, Alberto Elfes and Jonathan Roberts and while it is also applicable to 2 1/2 D environments, it will need further research to be generalized to 3D trajectories. Acknowledgements The authors acknowledge the support of Dr. Pavan Sikka and other members of the Autonomous Systems Lab, ICT Centre, CSIRO. References 1. Michal Aftanas, Through-Wall Imaging with UWB Radar System, Lambert Academic Publishing, A. Judson Braga and Camillo Gentile, An Ultra-Wideband Radar System for Through-the- Wall Imaging Using a Mobile Robot, Proceedings of the 29 IEEE International Conference on Communications, pp , SangHyun Chang, Michael Wolf, and Joel W. Burdick, Human Detection and Tracking via Ultra-Wideband (UWB) Radar, Proc. of ICRA21, pp , Marcelo J. Segura, Fernando A. Auat Cheein, Juan M. Toibero, Vicente Mut and Ricardo Carelli, UltraWide-Band Localization and SLAM: A Comparative Study for Mobile Robot Navigation, Sensors 211, 11, pp , Jose-Luis Blanco, Juan-Antonio Fernández-Madrigal, Javier González, Efficient Probabilistic Range-Only SLAM, Proc. of IROS 8, pp , Guowei Shen, Rudolf Zetik, Honghui Yan, Snezhana Jovanoska, Elke Malz and Reiner S. Thomä, UWB Localization of Active Nodes in Realistic Indoor Environments, 211 Loughborough Antennas & Propagation Conference, Marcelo J. Segura, Vicente A. Mut, Hector D. Patiño, Mobile Robot Self-Localization System using IRUWB Sensor in Indoor Environments, In Proceedings of IEEE International Workshop on Robotic and Sensors Environments, pp , J. González, J.L. Blanco, C. Galindo, A. Ortiz-de-Galisteo, J.A. Fernández-Madrigal, F.A. Moreno, and J.L. Martínez, Mobile Robot Localization based on Ultra-Wide-Band Ranging: A Particle Filter Approach Robotics and Autonomous Systems Volume 57, Issue 5, pp , Josef Sivic and Andrew Zisserman, Video Google: A Text Retrieval Approach to Object Matching in Videos, Proc. of ICCV 23, pp , M. Cummins and P. Newman. Accelerated Appearance-Only SLAM, Proc. of ICRA 28, pp , Time Domain, PulsOn 4, Time Domain, BroadSpec UWB Antenna, F. Lu and E. Milios, Globally Consistent Range Scan Alignment for Environment Mapping., Autonomous Robots, 4, , S. Thrun, W. Burgard, D. Fox: Probabilistic Robotics, MIT Press, E. Takeuchi and T. Tsubouchi, Multi Sensor Map Building based on Sparse Linear Equations Solver, Proc. of IROS 8, pp , Peter Biber and Wolfgang Straßer: The Normal Distributions Transform: A New Approach to Laser Scan Matching, Proceedings of the 23 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp , 23.