Applicability and Improvement of Underwater Video Mosaic System using AUV

Transcription

1 Applicability and Improvement of Underwater Video Mosaic System using AUV Hiroshi Sakai 1), Toshinari Tanaka 1), Satomi Ohata 2), Makoto Ishitsuka 2), Kazuo Ishii 2), Tamaki Ura 3) 1) Port and Airport Research Institute 2) DBSE, Kyushu Institute of Technology 3) IIS, University of Tokyo Nagase, Yokosuka, Kanagawa 2-4 Hibikino, Wakamatu, Kitakyusyu, Fukuoka Komaba, Meguro, Tokyo , Japan , Japan , Japan Abstract - In various inspections for the construction and the maintenance of port facilities, the underwater observation is a very important and an effective method. In general, the view of underwater video images is narrowly limited due to the turbidity of the sea water and the camera field angle. Therefore, the data processing such as mosaicing is necessary to observe the object over the wide range. However, the automization of mosaicing work is difficult, because of the insufficiency of the position information about underwater images. We have proposed an efficient mosaicing system of underwater images using AUV (Autonomous Underwater Vehicle) mounting the video camera with the line-laser. It is preferable to take a movie from the same direction by a constant distance as much as possible to make an accurate mosaicing image efficiently. In proposed system, the attitude of AUV is controlled based on the distance and the inclination to the object calculated from the position of the laser line in obtained images. To make mosaicing image by proposed method, the image of the port structure with the laser line is acquired in a real sea area. And, the affine transformation, the projective transformation, and the method of using information about the laser line are used as a transformation method of images extracted from a movie of an actual port structure. In this paper, applicability and the improvement method of mosaicing procedure are discussed by comparing mosaicing images obtained by these methods. system using AUV which can recognize the observation object in a wide-ranging image was proposed [3]. It is possible to contribute to an economical and efficient maintenance of port facilities by using this system to observe the state of the port structure. II. MEASUREMENT SYSTEM The method of mosaicing images taken by the underwater video camera with the line-laser installed in AUV was developed based on the optical flow. To acquire the suitable images for accurately mosaicing, the system to control AUV keeping a constant distance and facing perpendicularly to the observation object was proposed. A. Estimation of distance and direction to object The method which the position of AUV to the object was calculated from information on the laser line projected in images of the video camera was as follows. The lateral geometric relationship of the video camera and an observation object (a wall) is illustrated in Fig. 1. L is the distance from the camera to the wall, y is the distance between the top of the image field and the laser line, and y means the camera field angle in the lateral plane. The coordinates on the image field are represented by the pixel number and the entire image field is pixels. The laser line falls below as AUV parts from the wall. And parameters, y and H are constant depending on the initial setting condition of the apparatus. Accordingly, the distance L is obtained from pixel number y in the y-axis of the image using (1). I. INTRODUCTION The observation of port facilities such as the caisson, the steel pipe pile, and the gravel mound is generally carried out by divers. However, when the operation sea area is wide or deep, it is necessary to automate the observation work using by ROV (Remotely Operated Vehicle) or AUV. Since in the turbid water, it is needed to approach the object to take a video movie, only the image with the narrow range is obtained. Such an image is insufficient to observe a large-scale structure. Therefore, the data processing such as mosaicing of many images is necessary to observe the object over the wide range [1] [2]. However, there are many technical difficulties in the automization of the mosaicing work, because the underwater image is featureless and the accuracy of the position information is not sufficient. Accordingly, an efficient underwater video mosaicing Fig. 1. Geometric relationship in lateral plane. 1

2 H L (1) y tan y tan tan 2 y 2 11 The geometric relationship in the horizontal plane is illustrated in Fig. 2. If AUV is not facing to the wall in the perpendicular direction, the laser line in the image is projected slantingly as shown in Fig. 3. is the relative angle to the wall, L l and L r are the distance form the camera to the wall at both ends of the image field, D is the distance between points b and d, and x is the camera field angle in the horizontal plane. The direction of AUV to the wall can be calculated by (2). Fig. 4. Surface image of caisson with laser line D L l L r x 9 cos 2DLl 2 (2) B. Image Processing of laser line It is necessary to extract laser line information from the image taken by the video camera to calculate the position of AUV. In addition, it is needed to acquire the value of the y coordinate of extended straight line on extracted laser line at both ends of image. Therefore, the source image is binarized and the straight line is extracted from this image using Hough Transformation. Fig. 4 shows the sample image of the caisson surface with the laser line taken by the underwater video camera in the real sea area, and the clear straight line can be extracted by proposed image processing as shown in Fig. 5. Fig. 2. Geometric relationship in horizontal plane. Fig. 3. Position of laser line in image. 2 Fig. 5. Line detection using Hough transformation. C. Control method of AUV AUV used in this study has the four degrees of freedom, surging, yawing, heaving, and swaying. Surging and yawing motion are controlled to keep the constant distance and right angle to the underwater object, heaving motion is to keep constant depth, and the constant force is given to AUV for swaying motion. AUV is controlled to keep an angle is and the target distance to the object is L. If AUV cannot recognize the laser line, AUV stops for a certain time and turn around. If AUV can recognize the laser line again, the angle and the distance are adjusted. If AUV cannot recognize the laser line, it turns once. Nevertheless, if AUV cannot recognize the laser line, AUV goes forward direction for a certain distance. This procedure is repeated until AUV can recognize the laser line. D. Mosaicing method The mosaicing image is generated by the following procedure. 1) Capture images from video movie Images of JPEG type are obtained from the video movie at a constant time span. 2) Determination of characteristic points Characteristic points are selected in the same part of two consecutive images [4]. 3) Transformation of images Each image is transformed to overlap to the previous state image by the appropriate transformation method. 4) Composition of images Mosasicing image is produced by the composition of several images based on the coordinate values of corresponding characteristic points.

3 Fig. 6. Mosaicig image of tank test. E. Experimental result of tank test Fig. 6 shows the mosaicing image generated by the video movie of the picture put on the test tank s wall. 23 images were obtained from the movie taken by the video camera mounting on AUV. The affine transformation was used as the method of the image transformation. This method is capable of the extension, reduction, rotation, parallel transformation, and mirror reversal transformation. The coordinate transformation is expressed in the following equation with 6 parameters. x a y c 1 b d e X f Y 1 1 (3) III. ACQUIREMENT OF UNDERWATER IMAGE OF PORT FACILITIES A. Experimental method The image of the wall of the breakwater was acquired by ROV in a real sea area. The validity of the method developed in this study was confirmed by mosaicing by using the image of this actual port structure. RTV1 was used as ROV to move the video camera. The video camera with the line-laser was installed on ROV as shown in Fig. 7. The experiment was carried out around Izumi construction base and Izumi breakwater in the port of Kamaishi located in Iwate prefecture, northern part of Japan in November 23 (see Fig.8). Izumi breakwater is composed of block part and caisson part, and is about 1m in depth in a deepest part. The profile of the breakwater structure is shown in Fig. 9. RTV1 was controlled to keep a constant speed, a constant depth, and a constant distance from the wall shown in Fig. 1. The video movie of the wall of the breakwater with the laser line was taken by the video camera mounting on ROV. Fig. 7. Experimental device. Fig. 8. Port of Kamaishi. Fig. 9. Profile of Izumi breakwater. 3

4 Fig. 1. Control of ROV. B. Experimental result Though it was planned to take a movie in the distance of about 1m from the wall at first, it was necessary to approach the wall up to.5m in order to obtain clear images of the laser line. The method to determine the appropriate output, color, and shape of the laser according to the condition and the weather in the real sea area should be developed in the future. The part which images of the wall and the laser line were taken comparatively clearly was extracted from DV recorded all experimental results and edited in VTR. Fig. 11 shows the result of the mosaicing of continuous still pictures extracted from VTR data. IV. IMPROVEMENT OF MOSAICING METHOD A. Examination of transformation method Since images for mosaicing work were taken by AUV controlled to keep right angle vertically and horizontally to the wall in this proposed system, it was thought that the affine transformation was effective. However, each image contains the gap which cannot be transformed by the affine transformation only, because AUV actually moves in three dimensions. A vertical, left and right inclination of the image to the wall cannot be transformed by the affine transformation, though the distance from the wall and rotating angle in the plain of the image can be transformed by the affine transformation. A general projective transformation which a rectangular frame of obtained image is transformed into a convex trapezoid is examined as the transformation method. The coordinate transformation is expressed in (4). Fig. 11. Mosaicing image of surface of breakwater in Kamaishi. a1 X b1y c1 x a X b Y c a 2 X b2y c2 y a X b Y c (4) The mosaicing process using images of the wall of the breakwater acquired in the pot of Kamaishi was executed to confirm the effectiveness of this transformation. Fig. 12 shows the mosaicing image using the affine transformation and Fig. 13 shows the mosaicing image using the projective transformation. By comparing Fig. 12 with Fig. 13, it is understood that the distortion of image caused by the three dimensional inclination of AUV to the wall of the breakwater is improved. Fig. 12. Mosaicing image using affine transformation. 4

5 Fig. 15. Original image of Fig. 14. Fig. 13. Mosaicing image using projective transformation. B. Addition of image correction process It is examined to add the process to correct the distance and the inclination of the image to the wall before the image transform process as a method of improving the distortion of the mosaicing image. Information on the laser line used to control AUV can be used as information to correct the image. The distance and the inclination to the wall of AUV calculated from the position information of the laser line projected in the image show the distance and the inclination from the camera point of the image at the same time. Therefore, if each image is corrected based on this information, the image which does not incline in same distance from the wall will be obtained. Afterwards, the image with less distortion is obtained by the mosaicing of these images. Fig. 14 shows the example of correcting the image of the port structure by this method and Fig. 15 is the original image. First of all, the mosaicing image generated images corrected by this method without transformation process is shown in Figure 16. And next, Fig. 17 shows the mosaicing image using the affine transformation. Moreover, Fig. 18 shows the mosaicing image using the projective transformation as the transformation method. It can be recognized that there is an effect of the image correction process using the laser line information by comparing images of Fig. 16-Fig. 18 with the images of Fig. 12 and Fig. 13. The effect of the difference of the image transformation method seems to be less than the effect of the addition of the image correction process. Fig. 16. Mosaicing image without transformation. Fig. 17. Mosaicing image using affine transformation. Fig. 14. Image corrected by information of laser line. Fig. 18. Mosaicing image using projective transformation. 5

6 V. CONCLUSION 1) The system using AUV mounting the video camera with the line-laser was developed to acquire an wide underwater image of port facilities. 2) The system which can calculate the distance and the angle to the observation object based on the position of the laser line in the image was proposed. 3) The autonomous control method of AUV using the laser line information to acquire suitable images for mosaichig process was investigated. 4) The mosaicing method through the image correction process using the laser line information and transformation process using the affine transformation and the projective transformation was examined. 5) The effectiveness of proposed mosaicing method was shown by the mosaicing the images extracted from an underwater video movie of the port structure in real sea area. 6) Our future work is to develop the transformation method with less distortion and the method which can automatically extract characteristic points from the image. Acknowledgments The authors would like to thank Kamaishi Port Office, Tohoku Regional Development Bureau, Ministry of Land, Infrastructure and Transport, and TOA CORPORATION for support of sea area experiment. This study was sponsored by Ministry of Land, Infrastructure and Transport. REFERENCES [1] Y. Kanazawa, K. Kanatani, Image Mosaicing by Stratified Matching, IEICE. D-, Vol.J86-D-, No.6, 23, pp [2] N. Terashima, S. Takeuchi, H. Tominaga, High Resolution Image Digitization by Layered Image Mosaicing using Projected Mask, IEICE. Vol. J83-D-, No.4, April 2. [3] H. Sakai, T. Tanaka, T. Mori, S. Ohata, K. Ishii, T.Ura, Underwater Video Mosaicing Using AUV and Its Application to Vehicle Navigation, IEEE Proc. of 24 Int. Symp. on Underwater Technology, 24, pp [4] N. Chiba, H. Kano, M. Minoh, M. Yasuda, M. Osumi, Feature-Based Image Mosaicing, IEICE. D-, Vol.J82-D-, No.1, pp