Using Echo Ultrasound from Schooling Fish to Detect and Classify Fish Types

Transcription

1 Journal of Bionic Engineering 6 (2009)?????? Using Echo Ultrasound from Schooling Fish to Detect and Classify Fish Types Yeffry Handoko 1, Yul.Y. Nazaruddin 1, Huosheng Hu 2 1. Department of Engineering Physics, Bandung Institute of Technology, Jl. Ganesa 10 Bandung 40132, Indonesia 2. School of Computer Science and Electronic Engineering, University of Essex, Wivenhoe Park, Colchester CO4 3SQ, UK Abstract Fish finders have already been widely available in the fishing market for a number of years. However, the sizes of these fish finders are too big and their prices are expensive to suit for the research of robotic fish or mini-submarine. The goal of this research is to propose a low-cost fish detector and classifier which suits for underwater robot or submarine as a proximity sensor. With some pre-condition in hardware and algorithms, the experimental results show that the proposed design has good performance, with maximum value 100% detection and 94% classification of two types of fish. Both the existing type of fish and the group behavior can be revealed by statistical interpretations such as hovering passion and sparse swimming mode. Keywords: fish detection, classification, artificial neural network, ultrasound sensor Copyright 2009, Jilin University. Published by Elsevier Limited and Science Press. All rights reserved. doi: /S (08) Introduction It is common for fisherman to find fish using some kind of fish finder devices [1]. Fish detection technique has been widely deployed in fishing industry to improve fishing productivity. Up to now, many researchers have been tried to produce advanced fish detection devices to locate the location of individual or schooling fish. A variety of techniques and algorithms has been proposed and implemented, including ultrasound-based fish finders combined with GPS. For a robotic fish or a mini-submarine with a limited area for the detector, a compact detection device has to be deployed, which is smart enough but light weight [2]. In general, a fish finder consists of an ultrasonic transmitter and a receiver. It can detect the existence of both individual fish and schooling fish based on examining the reflected ultrasound waves transmitted. Brehmer et al. [3,4] developed new technologies to interpret and analyze sonar data in terms of the fish schools swimming speed, kinematics in terms of diffusion and migration, school density and fisher behaviors, etc. based on two standard omni-directional sonars. Moreover, its performance can be improved by using pattern classification techniques such as Hidden Markov Models (HMMs), clustering algorithms, fuzzy logic [5] and Artificial Neural Network (ANN). In other words, not only the existence of fishes but also the fish type can be identified. For instance, Lee, et al. [6] proposed a novel location estimation approach based on maximum likelihood approach to underwater localization based on signals collected by the sound receivers. The current fish detection devices are expensive and cannot be deployed in many small underwater robots or mini-submarine under the limit of space [7]. Therefore, it is necessary to develop new type of fish detection devices which are cheap and compact in size. This is the main issue to be addressed in this paper. The rest of the paper is organized as follows. Section 2 presents the design for new fish detection and classification device, including the sensor system. In Section 3, some general procedures are presented for testing and evaluating the proposed fish detector. Section 4 presents some experimental results to show the feasibility and performance of the proposed fish detection device. Finally, a brief conclusion and future work are summarized in Section 5. Corresponding author: Huosheng Hu hhu@essex.ac.uk

2 136 2 Design of fish detection and classification device 2.1 Fish classification There are various classification methods being developed to classify patterns in real-world applications, such as Hidden Markov Models (HMMs). An HMM is a statistical model under assumption that the system being modeled is a Markov process with unobserved state. It can be considered as the simplest dynamic Bayesian network. Estévez [8] proposed another classification method, namely the fuzzy min-max neural network classification, which combines the fuzzy logic as inferential tools with ANN. The benefit of this method is easy to train. The feature extraction process also becomes the important step to achieve successful classification. Some methods of feature extraction run on time domain other run on frequency domain. Kulinchenko [9] proposed two types of this feature extraction. In the time domain of feature extraction, some features such as the signal shape, peak present, local maximum or minimum, average maximum amplitude and duration are concerned and examined. In the frequency domain, the feature is attained by transforming time domain echo signals to frequency domain ones using fast Fourier transform. The range of maximum amplitude in the frequency domain becomes an observation variable. Hu [10] also used a fuzzy classifier, namely fuzzy C-means and fuzzy nearest prototype and his detection tool was running in the time domain. All echo ultrasound data is extracted to significant features and then inserted to ANN. 2.2 Sensory system We proposed a new sensory system based on modification of traditional ping ultrasound sensor, Maxsonar EZ-1 [11], which is widely used in mobile robots as a proximity sensor (Fig. 1). To make the sensor becomes waterproof, it needs a protecting layer which does not affect the transmitssion and receiving of ultrasound signal. A condom is used here as waterproof media for this device. The sensor output data has three types i.e. analog, digital and Pulse Width Modulation (PWM). Digital serial data can be easily transferred to other digital devices such as PCs or microcontrollers. Fig. 2 shows the Journal of Bionic Engineering (2009) Vol.6 No.3 schematic of connection. This sensor is able to detect small fish from range 10 cm to 100 cm. Its detection range is shorter than the ones used in air because of the condom and water. The sensor will produce echo signals in serial data when there is obstacle in front of the sensor. The sensor is used as a proximity sensor for obstacle detection device of underwater robots or submarine. The problems for feature extraction algorithm are to distinguish between fish and other obstacles such as wall and other floating objects, and to compensate the distance information. 3 Experimental setup Fig. 1 Ultrasound sensor. Fig. 2 Wiring diagram of sensor to microcontroller. 3.1 General procedures Experimental procedures were divided into three stages. The first stage consists of data acquisition and preparation of feature extraction from raw sensory data. The second stage is a learning stage based on using MATLAB toolbox. ANNs are used offline for supervised learning. Note that both feature extraction and data acquisition are carried out on a microcontroller Basic Stamps II (Fig. 3). A microcontroller in the preparation stage will transfer echo signals to computer which already installed with visual basic programs. The last stage is to test the detection and classifier algorithm which has already been downloaded into the EEPROM of the microcontroller.

3 As shown in Fig. 4, a rotating shaft allows the board to be rotated in its axes, and the sensor. The rotating mechanism is mounted on the fixed thin plywood. Other plywood divides the tank into two areas, i.e. fish area and non fish area. To make the fish existence or not, is simply done by driving a rotating shaft. If the sensor faces to the non fish area, it will be conditioned as no fish. On the other hand, if the sensor faces to fish, fish existence signal will be issued. Sensor Down-loading Sensor PWM signal PWM signal ANN algorithm for microcontroller Handoko et al.: Using Echo Ultrasound from Schooling Fish to Detect and Classify Fish Types 137 y f b xw, (1) Feature extraction (Basic stamp II) Preparation stage Feature extraction + classification (Basic stamp II) Testing Stage Weighted factor Hard disk E, D Learning classification using ANN Learning Stage Fish type viewed in LCD monitor Fig. 3 Schematic of fish detection and classification. = ( + ) j i i ij 1 f (.) =, (2) x 1 + e σ df Δ wij = α ( t y), (3) dx where α is learning rate, α = 0.1; σ=0.003; t is target vector; f(.) is activation function. To implement ANN on a microprocessor, some continuous nonlinear activation functions are represented by using linear-fractions of function. In this study, the exponential function in sigmoid function (Eq. 2) can be represented as e x = 1+ x + x + x. (4) 2! 3! 3.2 Feature extraction Feature extraction procedure will process raw data, i.e. echo sensor signal c(t), as shown in Fig. 5. Some features are extracted from raw data to represent the existence of fish. In contrast, other features representing obstacles can be eliminated. Kulinchenko [9] used the range of minimum to maximum of temporal data as a feature representing fish existence, namely shape parameter. The shape of the echo provides a great deal of information about the object detected. The slope of the echo s leading edge reveals how hard the reflecting surface is. The trailing edge reveals information about the absorption of the echo by the target and the target s resonant structure. In this paper, the output of the feature extraction process is a existence vector with size [2 1] and a directional vector with size [4 1], which are symbolized as E and D respectively. N E = ci ; c u k, v i= 1 [ ) k, k = 1, 2, 3, (5) Fig. 4 Experiment condition. Feature extraction has the duty to locate any big amplitudes of echo signal, which represents the existence of fish, and separate them from raw data as independent signal. Then it produces statistical values such as means or variances. To achieve learning ability, the device should have some kind of learning algorithms embedded. Here, simple back propagation ANN with no hidden layers are deployed with min( τ ) max( τ ) D =, (6) f1( d) f2( d) N 2 f1( d) = di ; f2( d) = d, i= 1 N i= N where τ represents the time distance between two neighbor amplitudes and d is the amplitude difference between two neighbor amplitudes, as shown in Fig. 6. i

4 138 Journal of Bionic Engineering (2009) Vol.6 No.3 4 Results Fig. 5 Echo sensor signal c(t) obtained by directly connecting ping sensor to computer serial port. The amplitude range classifier is based on observation about the amplitude range of temporal data whose magnitude is normally in the range of 0 to 1 (Table 1). It should be noticed that this range, represented by E vector, also contains the distance information of the fish since the ping ultrasound sensor itself is used as a proximity sensor. So the amplitude of echo signal measured in this vector is true distance between the sensor and the fish. However, this information is not useful for fish type detection, but very useful for fish existence detection. Table 1 Cumulative amplitudes of echo signals Class Normally amplitude range Number of amplitude E Fish Type I Fish Type II [0.1, 0.3] [0.3, 0.6] [0.6, 0.7] [ 0, 0.2] [0.2, 0.5] [0.5, 0.9] Fig. 6 Calculation of τ and d value. 3.3 Detection and classification procedure The existence of fish is detected by checking the E vector and fish type is detected by inserting the D vector, produced by feature extraction, to ANN algorithms. Fig. 7 shows the flowchart for the whole process. The amplitude value is obtained by recording echo amplitude in multi interval and ignoring fish-swimming direction. Using simple statistics, every range is sorted into a table subsequently; the detection can be done by using a table matching approach. No Ultrasound echo signal Amplitude detection and localization Feature extraction Detect fish existence Fish =? Yes Learned ANN Decision system Type I Type II Unknown Fig. 7 Detection and classification procedures. The values of τ and d in Eq. (6) are the interval time and the amplitude difference of moving objects respectively. The directional vector D cannot be exactly detected to reveal the real fish direction. It only expresses the changing direction caused by fish schooling pattern, i.e. approaching near or going away from the sensor. The number of echo signals is counted for the same sampling time for each measurement within one minute. The range of the maximum amplitude and the number of the (echo) amplitude is used in the separation process. The range is defined simply by dividing the full range of amplitude into three equal parts of percentile gained from 20 experiments. Signal disturbance from the edge of aquarium and the wall of aquarium can be eliminated by increasing the starting boundary value and decreasing the final boundary value respectively Then, Table 1 becomes a template or match table to detect the existence of fish. Actually the numbers of amplitude as mention before represent the variety of distance obtained from schooling fish. These numbers can also represent the change of distance in some certain time which indirectly represents the motion parameter of fish in a group, such as schooling direction, mobility, randomness and agility. Using 20 experiment data, the ANN supervised learning algorithm, which is implemented in MALTAB,

5 Handoko et al.: Using Echo Ultrasound from Schooling Fish to Detect and Classify Fish Types 139 has achieved successful optimization with MSE = 0.1. The weight variable as the product of this process is inserted into the ANN algorithm in the microcontroller. As a result of 20 real-time tests, it brings successful achievement as shown in Table 2. The successful criteria of classification are represented in the percentage of the total test. Table 2 The results of classification. Fish Type Detection Successful Classification from detection Fish Type I 90 % 94 % Fish Type II 100 % 80 % The experimental results show that using an existent vector E and the cumulative amplitude pre-designed table, the design of low-cost fish detection device has been successful. The classification of Type II fish is 80%, which is less than the classification of Type I fish, i.e. 94%. The pattern of fish direction, which is represented by the directional vector D, is more reliable for Type I fish. The ANN is easy to obtain an optimization value in tendentious similar patterns or non similar patterns. Hence, for the case of Type II fish, the pattern of directional vector for each learning sample together does not have the power to obtain significant distinction. As shown in Table 3, the mean value of d has almost the same value between two types. It indicates that the average of amplitude neighborhood distances in two types is similar. In reality, it means that two groups of fish have the tendency to gather in the steady location and have little passion to wander or making hovering swim. Type I fish has the mean value of τ greater than Type II fish since the size of Type I fish is bigger than Type II fish. Therefore, schooling Type I fish is detected more often with smaller standard deviation of τ. The standard deviation of d represents the sparse group of fish. In a group, schooling Type II fish has more density (individual per area) than Type I fish. Fish Type Table 3 Statistical values of two fish types. Mean Standard deviation τ d τ d Fish Type I 0,36 0,45 0,001 0,002 Fish Type II 0,02 0,41 0,3 0,15 5 Conclusion and future work This paper proposes a low-cost fish detector based on using existence and directional vectors and some pre-condition mechanisms. Although some noise interfered in echo signals, the experiment results show that the proposed sensor device achieved 100% fish detection and 94% classification of two types of fish. Furthermore, apart from providing information about the existence and the type of fish, the device can also reveal the fish behavior in group by statistical interpretations such as hovering passion and sparse swimming mode [12]. In the future research, the variation of mobility and agility can be quantized by observing thoroughly a directional vector. Hence, it can become a good starting point to have complex classification of more than two types of fish. More aggressive fish with high agility swim ability will also be investigated. Acknowledgement This research was fund by Fundamental Research Fund Program from Indonesia Ministry of Research and Technology in years ID Number: RD References [1] Fish Finder, [ ], [2] Zhi Y J, Kui C E, Shuo W, Min T. Motion control algorithms for a free-swimming biomimetic robot fish. Acta Automatica Sinica, 2005, 31, [3] Brehmer P, Lafont T, Georgakarakos S, Josse E, Gerlotto F, Collet C. Omnidirectional multibeam sonar monitoring: Applications in fisheries science. Fish and Fisheries, 2006, 7, [4] Brehmer P, Georgakarakos S, Josse E, Trygonis V, Dalen J. Adaptation of fisheries sonar for pelagic fish school monitoring: dependency of the schooling behaviour on the fish finding efficiency. Aquatic Living Resources, 2007, 20, [5] Handoko Y, Nazaruddin Y Y, Riyanto B, Leksono E. Implementing artificial catching-prey behavior using fuzzy logic on rish robot. Proceeding of The 9th Conference in Instrumentation and Control, Bandung Institute of Technology, Indonesia, 2007, 1 4. [6] Lee K C, Ou J S, Huang M C, Fang M C. A novel location estimation based on pattern matching algorithm in underwater environments. Applied Acoustics, 2009, 70, [7] Tidd R A, Wilder J. Fish detection and classification system, Journal of Electronic Imaging, 2001, 10, [8] Estévez P B, Ruz G A, Perez C A. Fuzzy min-max neural network for image segmentation. The 7th Joint Conference on Information Science, North Carolina, USA, 2003.

6 140 Journal of Bionic Engineering (2009) Vol.6 No.3 [9] Kulinchenko A B, Simpson P K, Denny G F. Tethered fish data collection and species classification: Prince William Sound bottomfish. Proceedings of IEEE International Geoscience and Remote Sensing Symposium, 2004, 2, [10] Hu B G, Gosine R G, Cao L X, de Silva C W. Application of fuzzy classification technique in computer grading of fish products. IEEE Transactions on Fuzzy Systems, 1998, 6, [11] MaxSonar, [ ], [12] Sfakiotakis M, Lane D M, Davies J B. Review of fish swimming modes for aquatic locomotion. IEEE Journal of Oceanic Engineering, 1999, 24,