THERMAL IMAGING ANALYSIS OF POTENTIALLY HARMFUL SUBJECT FOR NIGHT VISION SYSTEM Noor Amira Syuhada Mahamad Salleh 1, Kamarul Hawari Ghazali 2 Faculty of Electrical and Electronics Engineering, Universiti Malaysia Pahang, MALAYSIA 1 nooramirasyuhada@gmail.com 2 kamarul@ump.edu.my Abstract Thermal imaging is now widely used in important and high risk field. With the good performance to detect the object in poor lighting condition, thermal imaging is used as the advanced technology in the critical surveillance activity. Surveillance system become a critical issue when it comes to a critical area cover by the responsible party such as military, police and others security organization.besides, the computing technology contributes the easiness of implementing digital image processing in many applications. This can greatly improve the effectiveness of the surveillance activity done at the critical surveillance area. Hence, this present paper proposed few techniques on image processing to enhance the surveillance efficiency based on the thermal image dataset. The selection of the correct technique is followed the specification, process and output from this thermal surveillance system. The method of frame difference is used since the detection is on the moving subject. Image processing is fully used to fulfil the identification of the subject. By using the edge characteristic of the subject, the identification and image analysis use the boundary information. The feature of subject detected is analysed to improve the identification of the existence of the potentially harmful subject. The algorithm developed able to identify the subject come across the critical surveillance area. Analysis of the study is using a dataset of night surveillance activities which include the possibilities of harmful and harmless subject. Finally, the system will automatically differentiate and identify the subject and indicate the existing subject is harmful or harmless subject. The experimental results show the detection with better accuracy. Keywords: surveillance, thermal imaging, identify, potentially harmful subject 1. Introduction Thermal Imaging Analysis of Human Detection for Night Vision System to be part of military technology is considered very important and appropriate to the changing times. This thermal camera application has been used in the developed countries as in the United States. Yet applications used thermal camera is just a thermal camera that displays images from the temperature based where the vision is more accurate compared to the visible image. On that basis, we propose a thermal camera technology in an automated system which enhances the efficiency of thermal camera especially in this critical surveillance application. Thermal Imaging Analysis of Human Detection for Night Vision System able to increase the detection level of surveillance on intruders at the surveillance area. This will become an aide to the border control activities carried out by the military. The ability of thermal camera which overcome the Organized by WorldConferences.net 265
ambient light problem is the factor why thermal camera can provide enormous benefit for night vision, especially in very dark restricted area. Until now, the research on automatic surveillance for military is still new and still under study. This field of study is a great opportunity in the development of image processing technologies as well as the military technology. The research works in thermal imaging surveillance are continuing even not in the military field. The study still use thermal camera technology this proposed two simple and fast detection algorithms into a cost effective thermal imaging surveillance system [1][2][3]. The thermal imaging detection overcome the visible camera with the limitation of daytime only operation. Therefore, the thermal image captured from the thermal camera does see heat and does not reflected with light energy. Besides, the images produce by thermal camera do not reflected with shadow [4] [5][6]. Hence, make the thermal imaging detection has higher persistent. Thermal image come with a presentation of indexed color corresponding to the temperature of the object focused [7]. Hence the surveillance demands on the automatic system as the aided to enhance the surveillance activity especially at the borderline. Since the advancement of image processing technology, there is a big potential to embark research on the smart automation surveillance focusing military surveillance. 2. Research Methodology The data acquisition process was done using thermal camera by FLIR A615 series because of the specification of the resolution. Data collection was done at the suitable places similar to surveillance area. The empty field area was chosen to take the thermal video. Significant to the scope of analysis for this paper which to detect the human is exist or non-exist; the data collected are the images of human in thermal images. The data collection done involved one to two target objects. Figure 1: List of data contain zero intruder Figure 2: List of data contain one intruders Figure 3: List of data contain three intruders Organized by WorldConferences.net 266
3. Result and Discussion There are lots of techniques in image processing tools. The selection of the correct technique is followed the specification, process and output from that system. In this study, the method of frame difference is used since the detection is on the moving target. With the increase of input frame, still background does not change and the target change by frame. In practical, iteration of 100 frames for each number of target intruders images obtained better result of detection. The way to detect the existence of the object flow the image processing flow as shown in Figure 4. Figure 4: Image processing flow chart All the frames extracted from the video taken by FLIR A615 are put into the dataset.the total images in the dataset are 300 images. The method continues with the detection algorithm techniques.the steps of detection are follows, 1. Converting image to grayscale 2. Color thresholding 3. Binarization 4. Closing,Opening 5. Edge Detection Boundaries Figure 5 : Flow chart of the algorithm Image processing is fully used to fulfil the identification of the target from the thermal image. By using the edge characteristic of the target, the identification and image analysis use the boundary information for the analysis to get better identification [8][9]. We use the color thresholding technique do define the chosen color respect to the target color characteristic. The threshold value to detect the object is really important to make sure the object is detected unless the detection is failed [10]. Hence, we do the binarization to enhance the clarity of the target object (Figure 5). The further steps of image processing continue with the mathematical morphology and classical edge detection. The mathematical morphology used are the opening and closing operation which are also known as dilation and erosion process. The size and shape of the structuring element give different results of detecting the object [11]. Organized by WorldConferences.net 267
The detection algorithm follows with the classical edge detection. The edge detection commonly used as the basic step of image segmentation and recognition [12]. The combination of classical edge detection and mathematical morphology used to achieve the goal of identification. In this paper, the fusion method of classical edge detection and mathematical morphology detection is used to identify the existence of the target from the thermal image from the dataset obtained. Classical edge detection used are includes Prewitt, Sobel and Laplacian edge detector. Prewitt is a good operator with the advantages for simple, fast processing speed, relatively smooth and continuous edge [13]. The Prewitt method finds edges using the Prewitt approximation to the derivative. It returns edges at those points where the gradient of I is maximum. BW = edge(i,'prewitt') (1) While the second edge detector approached is Sobel. The Sobel method finds edges using the Sobel approximation to the derivative. It returns edges at those points where the gradient of I is maximum. W = edge(i,'sobel') (2) And the final edge detector been used is The Laplacian of Gaussian method.this method finds edges by looking for zero crossings after filtering I with a Laplacian of Gaussian filter [9]. BW = edge(i,'log') (3) Figure 6: Result from the Binarization, Closing and opening, Edge Detection, and Boundary detection. Hence the target object had been classified either the target is exist on not exist in the scene by measurement of the existence pixel of that target object which is specifically the human. The values of the existence pixel differentiate whenever the intruder is exist or not. Organized by WorldConferences.net 268
Table 1: Sample of measurement of the existence pixel. Number of Images Number of intruders Zero One Two 1 0 384 1600 2 0 320 1481 3 0 329 1397 4 0 407 1554 5 0 449 1548 6 0 577 1528 7 0 552 1613 8 0 363 1783 10 0 349 1437 11 0 355 1596 12 0 354 1643 13 0 329 1443 14 0 319 1655 15 0 387 1556 16 0 389 1345 17 0 376 1567 18 0 356 1435 19 0 366 1528 20 0 321 1548 21 0 345 1457 22 0 350 1674 23 0 322 1566 24 0 313 1653 25 0 345 1587 26 0 355 1484 27 0 366 1856 28 0 388 1684 29 0 379 1673 30 0 355 1647 Figure 7 : Measurement of existence pixel by number of intruders Organized by WorldConferences.net 269
1 10 19 28 37 46 55 64 73 82 91 100 The result shows by the graph indicates the measurement of existence pixel for one intruder is in the range of 400-600, for two intruders is 1400-1600, while consistent result 0 shown for zero intruder. Among this 100 frames respectively used, there are five images that out of the pixel range. From the detected object, we enhance the programme to detect the object to be the harmful or the harmless subject. Figure 8: Sample of the subject tracked The tracking result used as the measurement of the subject movement to classify the subject to be the harmful or the harmless subject. In this scenario, we set the target to be a harmless subject when the subject is tends to move near the borderline. The result of the subject movement is display on the Variance Values as for threshold. A threshold level is set to compare the variances values. The frame which has the variance value exceed the threshold value, the frame is then detected and classified as the potentially has the harmful subject. 0.006 0.004 0.002 Variance Value 0 Figure 9: Result of the subject movement The value of 0.002 is set to be the threshold to define the subject is harmful. This threshold also indicates the subject is getting near the borderline. We have tested this programme for 100 frames. This algorithm used shows that the harmful subject is successful detected and tracked. 4. Conclusion Smart Night Vision System for Detecting Intruders in Restricted Area is developed using thermal images captured from Thermal Camera FLIR A 615.The method from image processing techniques involved pre-processing, morphology, edge detection, and boundary detection used to perform the develop system. From the extracted frame by frame from the thermal video, the images of subject (human) in thermal images display are detected and tracked. Organized by WorldConferences.net 270
References [1] Wai Kit Wong, Poi Ngee Tan, Chu Kiong Loo, Way Soong Lim, "An Effective Surveillance System Using Thermal Camera," Signal Acquisition and Processing, 2009. ICSAP 2009. International Conference on, vol., no., pp.13,17, 3-5 April 2009 [2] Wai Kit Wong, Zeh Yang Chew, Chu Kiong Loo, Way Soong Lim, "An Effective Trespasser Detection System Using Thermal Camera," Computer Research and Development, 2010 Second International Conference on, vol., no., pp.702,706, 7-10 May 2010 [3] Wai Kit Wong, Joe How Hui, Chu-Kiong Loo, Way Soong Lim, "Off-time swimming pool surveillance using thermal imaging system," Signal and Image Processing Applications (ICSIPA), 2011 IEEE International Conference on, vol., no., pp.366,371, 16-18 Nov. 2011 [4] Davis, J.W., Sharma, V., "Robust detection of people in thermal imagery," Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference on, vol.4, no., pp.713,716 Vol.4, 23-26 Aug. 2004 [5] (May,2013,29). What s The Difference between Thermal Imaging and Night Vision? Available: http://www.flir.com [6] Padole, C.N., Alexandre, L.A., "Motion Based Particle Filter for Human Tracking with Thermal Imaging," Emerging Trends in Engineering and Technology (ICETET), 2010 3rd International Conference on, vol., no., pp.158,162, 19-21 Nov. 2010 [7] Chan Wai Ti, Sim Kok Swee, Tso Chih Ping, "A Proposed approach in applying optical character recognition for thermal image processing," Electronic Manufacturing Technology Symposium (IEMT), 2010 34th IEEE/CPMT International, vol., no., pp.1,4, Nov. 30 2010- Dec. 2 2010 [8] Tian kechun, Ou yang ning, Chen zuxi. The target recognition and research on auto based on the fuzzy recognition.the information on microcomputer.,6(3):309-311, 2008 [9] Zhang Qi-li,Kang Bao-sheng. Image edge detection algorithm based on multi-structural elements morphology.computer Engineering andapplication.,45(34):166-168, 2009 [10] Ramli, S.B., Ghazali, K.H.B., Bin Mohd Ali, M.F., Hisahuddin, Z.H.B., "Human motion detection framework Suzaimah Bt Ramli," System Engineering and Technology (ICSET), 2011 IEEE International Conference on, vol., no., pp.158,161, 27-28 June 2011 [11] Lei Yang, Dewei Zhao, Xiaoyu Wu, Hui Li, Jun Zhai, "An improved Prewitt algorithm for edge detection based on noised image," Image and Signal Processing (CISP), 2011 4th International Congress on, vol.3, no., pp.1197,1200, 15-17 Oct. 2011 [12] GUO Jun,PAN Shen,HU Xiao-jian.Edge Detection in Tobacco Leaf Image Based on GrayscaleMorphology. Computer Engineering.,33 (21):163-165, 2007 [13] K.Hawari, N.Wahidah, Faradila, Rosyati, Nazriyah, Rohana, Farizan, Falfazli, and Lailatul, The Computer Vision System Module, 1 st Edition, p.60-64, 2011. Organized by WorldConferences.net 271