Designing and construction of an infrared scene generator for using in the hardware-in-the-loop simulator

Transcription

1 124 Designing and construction of an infrared scene generator for using in the hardware-in-the-loop simulator Mehdi Asghari Asl and Ali Reza Erfanian MSc of Electrical Engineering Electronics, Department of Electrical Engineering, Malek Ashtar University of Technology, Tehran, Iran Assistant Professor, Department of Electrical Engineering, Malek Ashtar University of Technology, Tehran, Iran Abstract One of the main components of the hardware in the loop simulator system is the infrared scene generator. The generator produces thermal images using heated elements. In this study, an infrared scene generator has been designed and constructed to use in the hardware in the loop simulator. After completion of the construction, patterns and images have been sent to infrared scene generator to be tested. These images have been displayed by infrared scene generator. It is one of the strengths of this design to construct the infrared scene generator as fixed. The average cooling time for infrared images in this generator is 180s. Error of the infrared scene generator was investigated and its error function was obtained. The result was that increasing the average brightness, the error of the generator will increases. Minimum PSNR for each pixel is 37/2dB. Key-words: Infrared scene generator, hardware in the loop, infrared seeker, FPGA, generator error 1. Introduction In parallel with the scientific advances in infrared technology, the application of infrared detectors in military industries has had significant development and new generations of IR guided missiles have been produced and used [1, 2]. One of the most important components of the missile is its infrared seeker. Target coordinate finder, which constitute the major part of infrared seeker, is to detect radiation from target, determine the relative position of the target and the missile and track the target (locking on purpose). Testing of the missile guidance system performance is very important. Testing missiles guided by infrared seeker requires the actual environment. Due to the high costs and other problems, doing these tests in a real environment is impractical. Therefore, hardware in the loop systems are used to test missiles guided by infrared seeker. The main advantage of this method is that control systems can be tested under conditions as realistic as possible with no need to build all components of the system. One of the main subsystems used in hardware in the loop simulation system of missile is the infrared scene generator system. In order to investigate the performance of IR seeker, it is required to install right infrared radiation source on target motion simulator. Infrared radiation frequency changes in accordance with the temperature of the object [3]. 2. Infrared radiation Infrared radiation in physics is the portion of the electromagnetic wave spectrum that its wavelength is longer than visible light and shorter than radio waves. Such radiation wavelength range is approximately between 1 mm to 750 nm (7800 to 1,500,000 angstroms). Maximum frequency of infrared waves is measured 400 trillion times per second (very close to the range of visible red color i.e. being below red) to 800 billion times per second (final range of microwaves). Figure 1 shows the frequency spectrum including infrared and other waves. Infrared radiation is not visible by conventional means. Any object emits electromagnetic waves in the wavelength between μm according to its temperature. The vision of the human eye is in the visible range of the electromagnetic waves i.e. 0/38-0/76 μm. Therefore, in the absence of visible light, human uses detector to see around [4]. 3. Infrared seeker Array seekers have sensors that produce electrical signals while infrared light is irradiated on them. Since these seekers only distinguish between normal and very hot objects, the infrared system is much simpler than visible light detection system (such as a conventional video camera). Furthermore, infrared seekers do not need an external light source and as a result they function at night as well as the day. In some infrared-guided missiles, infrared sensors have been paired with a conical scanning system. Thus, the series of lenses and mirrors that direct the light to the sensor rotate around a small mirror and scan a large part of the sky. When the rotating set crosses with a source of high heat on one side of the missile, search system finds a target and guides the missile to the direction. As can Manuscript received March 5, 2017 Manuscript revised March 20, 2017

2 125 be seen in Figure 2, seeker of the missile is in front part of it. The seeker should send Infrared data to the guidance control [4]. Fig. 1 Frequency spectrum [4] Fig. 2 infrared seeker on the missile front [4] 4. The hardware in the loop Hardware in the loop simulation is kind of simulation in which one or more subsystems of a closed loop system have been placed as hardware in the simulation loop and the rest of sub-systems placed as software models in the simulation loop. The main purpose of the hardware in the loop simulation is to investigate the performance effect of one or more hardware of subsystems of the main system on the behavior of the entire closed loop system. So, the hardware in the loop simulation is important for closed loop systems, but it is not important in the open-loop system. In a closedloop system, given that the performance of the hardware of a subsystem along with other sub-systems is effective on its inputs, it is necessary to do the simulation as hardware in the loop to be able to investigate the whole system performance in the presence of system hardware. Consider the closed-loop system shown in Figure 3. As can be seen subsystems of G1, G2 and G3 have formed a closed loop and the performance of each subsystem affects its inputs and inputs of other subsystems via the feedback. This means that each of the inputs and outputs of r1, r2 and y change in accordance with the subsystem performance at any point in time. So, a series of predetermined data cannot be used for them [5]. Fig. 3 Closed loop system G [5] 5. Infrared scene generator Infrared scene simulation includes dynamic infrared simulation technology and infrared scene generator. Infrared simulation is performed by methods of resistive array, fluorescent tubes of liquid crystal, laser writers, laser diodes array, CRT and digital mirror device (DMD). All mentioned technologies are evolving and progressing. For example, consider a system that demands high dynamic range and high data transfer speed; flashing is not important in this system. In this case the laser diode array can be an appropriate option. If speed of data transfer was low, the

3 126 system would no longer offer. If infrared scene generator is scanning detector, CRT system is proposed. Infrared seeker dynamic scene simulation technology includes dynamic infrared simulation technology and infrared scene generator technology. Infrared scene simulator is a tool that can convert image signals to infrared images. Optimal performance for infrared scene simulator includes features such as high-speed, real-time, the high temperature differences between different parts of the image, high resolution, reasonable bandwidth, no flicker, image consistency and low cost of construction and maintenance [3]. Hardware Description Language) is used as hardware description of FPGA. VHDL is one of the hardware description languages. VHDL language was first designed and used by the Defense Department of America in order to design and describe the high-speed integrated circuits. As mentioned in the previous section, corresponding to each of the columns there is a byte that in fact determines the brightness of each pixel. A PWM signal is generated for each of the columns. The generated PWM signal applied to the both ends of its corresponding pixel and the current is generated as proportional to the voltage generated across the resistor. Figure 5 shows the overall circuit schematic that is synthesized on the FPGA. 6. The control circuit The main chip which is used to control the pixels of infrared scene generator is FPGA. FPGA is the next generation of programmable digital integrated circuits that is used as the central processing unit. Execution speed of logic functions in FPGA is very high (about nano seconds). FPGA is simply a chip consisted of a high number of logical blocks, communication lines and bases of input / output (IO) which are next to each other as an array. A simplified FPGA block diagram is shown in Figure 4. [6]. Fig. 5 overall circuit schematic that synthesized on FPGA 7. The control circuit components The control circuit of infrared scene generator is composed of three main components including sending and receiving information, main processing and guiding rows and columns. Fig. 4 Block diagram of an FPGA [7] Of course, many logic cells are made based on LUT (Look up Table). LUT is made up of a number of SRAM memory cells which are initialized while programming the FPGA. In short, LUT is to produce ready functions to use in logic cells. There are 2500 pixels in the construction of infrared scene generator that need to be controlled separately. 50 pixels which are in one row of infrared scene generator must be controlled simultaneously. This needs to apply 50 PWM signals at the same time. For the reasons mentioned above, the main chip which controls the pixels of infrared scene generator is FPGA. In this paper, VHDL (VHSIC 7.1. Send and receive information. Initial information of images are organized first by MATLAB software on a computer. Infrared images which are to be simulated are turned into black and white. This is done in order to get a picture as a matrix. This matrix contains the numbers 1 to 255. The number 1 represents the darkest point and the number 255 represents the brightest one. Turning the image into black and white, the three matrixes become one matrix. Due to the black and white nature of infrared images, we just need one black and white matrix for each image. After the process of turning images into black and white, matrixes size need to be minimize to

4 The purpose of this process is to match the size of matrixes with the size of the resistive array. After preparing the images in the desired format, information about the images need to be sent. This is done by using RS-232 serial port. To match the output voltage of the computer with input voltage of FPGA, MAX is used. This converts the output voltage of computer in serial port to 3 / 3 V. This IC and side circuits can be seen in Figure 6. 7). In order to perform the switching for rows, IRFZ-44 MOSFET is used. One of the things that must be considered in the selection of MOSFET is the attention to its RDS (ON). The lower the value of this parameter, the lower the voltage drop across the turned on MOSFET. This MOSFET with RDS (ON) of about 0/024 ohm is suitable for switching. Fig. 6 send data to the FPGA by MAX The main processing After receiving the information, the central processing unit separates the data as bytes. Next, the bytes are stored in a 50 byte array. 50 I / O stands for rows and 50 have been used for columns. A stand for clock and a stand for resetting have been considered. Circuit clock is provided by the oscillator 14/7456 MHZ. This oscillator has been chosen because firstly it is a multiple of the Baud Rate, secondly, it generates PWM according to the demanded clock. The main processing part controls the rows relating to resistive array according to the clock pulse. Sweeping images in control circuit is so that firstly the first row is activated by the connection to the land. With the activation of the first row, PWM signal is applied to the columns of this row compatible with the brightness of each pixel. This process continues until the last row. At any moment only one row is activated and PWM signals relating to each column are applied simultaneously. In the main processing, data of each pixel are given to PWM units concurrently. Data relating to each row are stored as 50 byte data in an array and remain in the array until the completion of the data of the next row. Fig. 7 bias circuit designed to control the rows. 8. The findings of the study 8.1 Thermal camera TBIR90 thermal camera has been used to test the infrared scene generator. This camera is a third-generation IR camera with Micro bolometer detectors made of silicon amorphous which works in spectral range of 8-14 micrometer. Figure 8 shows the appearance of the camera. To transfer images from the camera to the computer, the convertor ADVANTECH model DVP 7010B is used. The converter converts the video output of the camera to digital and transfers it to the computer. This converter has an application that is installed on the computer. Transferred thermal camera images can be recorded and stored with this application lead the rows and columns It was told in the previous section that the activation of each of the rows is performed in the main processing. It is done by using zero and one (logical) for each of the outputs of each row. Given the maximum flow through each row (in the case of full brightness of the corresponding row), it is required a drive circuit with high current tolerance (Figure

5 Testing phase Fig. 8 thermal camera TBIR90 In this section, the information is transmitted by computer to control circuit. This is done by MATLAB. Information transferred to the serial port and from there to the FPGA. FPGA chip is programmed by VHDL. The control circuit has been boarded on the board of fiberglass printed circuit with copper thickness of 0/35 mμ. Figure 9 shows the built control circuit. Fig. 9 control circuit Figure (10-a) shows the resistive array The connection between the control circuit and resistive array has been established by using flat cables. Placement of the resistive array in front of the camera is so that the proper distance with camera should be observed for proper image coverage (figure 10-B). This distance depends on the characteristics of the camera. Minimum distance to laboratory thermal cameras is 3/2 meters which has been obtained empirically. If the distance of camera from infrared scene generator is longer, the camera must be adjusted to the new distance. This is done using the options for adjusting the camera lenses. After setting the distance, back of the resistive array seen in the images must be uniform in order not to damage the original image. The work has been done in the laboratory by a wooden plate on the back of the resistive array on the table. If the camera has a zoom feature, it is not required to do so. Fig. 10 a) original resistive array, b) placement of original resistive array in front of the camera. After proper placement of the resistive array in front of the camera, it is time for the final stage i.e. to test infrared scene generator. To test infrared scene generator, simple patterns are used to display. So, a square pattern with the same characteristics were sent. Figure 11 shows this pattern and the pattern reconstructed on the infrared scene generator. As can be seen, this model has been implemented satisfactorily by infrared scene generator. In order to test the performance of infrared scene generator in the differentiation of bright and semi-bright section, the pattern of the figure 12 has been submitted to the generator. As is clear from Figure 12, the response of infrared scene generator to this pattern is also satisfactory Fig. 11 square pattern a) original image, b) reconstructed image Fig. 12 difference between light and semi-light image, a) original image, b) reconstructed image

6 129 Next, to test the performance of infrared scene generator, normal infrared image of a jet in flight (figure 13-a), a moving tank (figure 14-a) and a standing man (figure 15-a) was sent to the generator. The reconstructed images have been shown below. found that on average it takes approximately 180 seconds for an image to fade completely on the resistive array. Average intensity of brightness were prepared for each image. This was done using MATLAB software. Calculated brightness intensity to initial image brightness has provided as percentage. Figure 17 shows the reduction of average brightness intensity versus time for a picture. Fig. 13 A jet in flight: a) original image, b) reconstructed image Fig. 14 a moving tank, a) original image, b) reconstructed image Fig. 16 fading of the image Fig. 15 a standing man, a) original image, b) reconstructed image 9. Reduction of the brightness of the reconstructed image After scanning images, infrared image is seen for a while. Because resistive array needs a little more time to be cold for showing the next image. Cooling time for the image remained on the resistive array depends on the intensity of brightness of its pixels. In other words, this time has a direct relation with the average intensity of image s brightness. Figure 16 shows the cooling process of sample image. In this figure the infrared image is displayed in the first moments after scans up to 180 seconds after the scan. Each of the images tested and their cooling time recorded. It was Figure 17- Process of the reduction of the average brightness intensity versus time As the graph shows, brightness reduction is almost with a constant gradient. It is worth noting that in the case of using cooling elements this process will be faster. In order to show the moving images by infrared scene generator, this time should be in milliseconds.

7 Error of the infrared scene generator In order to estimate the error of the built infrared scene generator, generated images by the generator need to be compared with the original image. So, devices to generate original image need to by the same as those to generate infrared image of resistive array. To do this, using thermal cameras in the lab (with which resistive array images taken) thermal images were taken from the surroundings. An image selected for error estimation. Image taken by the thermal camera was sent to the infrared scene generator. By comparing the histograms presented in Figure 18, it is concluded that the reconstructed image contrast and brightness are less than those of the original image. A B A Fig. 18 a) image taken by the thermal camera b) Histogram of the image taken by thermal camera, c) reconstructed image, d) Histogram of the reconstructed image. B 11. Study the infrared scene generator error Conventional standards in processing images for error estimation and the comparison of the two sample images are Mean Squared Error (MSE) and Peak Signal to Noise Ratio (PSNR). To calculate the MSE and PSNR, formulas (1) and (2) are used [8]. MSE = M,N (x ij y ij ) 2 M.N (1) PSNR = db MSE value when the two compared images are quite contrasting (one white and the other black) will be equal to 650/25. Comparing the number with the number obtained above, the obtained value will be more tangible. Valid values of PSNR for lossy compression is between 30 and 50 [9]. The obtained error includes camera error and analog to digital converter (ADC) error and is not related only to infrared scene generator. Figure 19 shows this. PSNR = 10Log 10 ( R2 MSE ) (2) In formula (1) M and N is the number of rows and columns of a matrix related to the two images to be compared. xij and yij are elements of the matrix. R for 8-bit data is equal to 255. PSNR value is based on db. The lower the MSE value, the closer the compared two images. Calculated MSE and PSNR for the image taken by the thermal camera are as follows: Fig. 19 reduction related to the first and second images. The calculated error is related to the reconstructed image. Because the comparison is between images taken by the thermal camera and reconstructed image. MSE =

8 Error related to one pixel of infrared scene generator In order to examine precisely the error of infrared scene generator, a one pixel image was taken into consideration. So, a one pixel image with various light intensity sent to infrared scene generator. The brightness intensity increased by 10 percent in each image at every step. The applied voltage across the resistor SMD was measured and its radiation was recorded by the thermal camera. Then, the error of infrared scene generator calculated by comparing the original image and the reconstructed image. MSE error was recorded for each image. Figure 20 shows the changes of error of infrared scene generator with a one pixel image based on the voltage applied across it. Fig. 21 PSNR changes based on the applied voltage across the infrared scene generator resistor As is clear from Figure 21, the value of the PSNR (peak signal to noise ratio) will decrease with voltage increase. In other words, PSNR will decrease and the signal will weaken with the enhanced infrared scene generator radiation. The MSE, PSNR and radiant power for different images which presented in the final test have been shown in Table 1. It should be noted that radiant power has been obtained by calculation. Fig. 20 the changes of error of infrared scene generator with a one pixel image based on the voltage applied across it. Because of the relatively large distance of the pixels of infrared scene generator from each other and their little impact on each other, results for several pixels are very close to the results of one pixel. So, the results of a single pixel are sufficient. Figure 20 gives an important result. By increasing the voltage across the resistor of infrared scene generator and consequently increasing the resistor radiation, the error rate based on the MSE will increase. In other words, the more the bright spots of the infrared image, the more the error rate of the infrared scene generator. Because the MSE error criteria is normalized, it is concluded that in Figure 20 it is true for every number of pixels. Figure 20 can be the function of the infrared scene generator's error. Therefore, by calculating the mean radiation of an image and its corresponding average voltage, MSE error rate can be obtained based on Chart 20. Figure 21 also shows PSNR based on the applied voltage across the resistor. Table 1: MSE and PSNR values for images sent to infrared scene generator Row Numbe r-title of the image A jet in flight A moving tank A standing man 13. Conclusion radiant power(w) MSE PSNR (db) One important result of this research is the construction of infrared scene generator system. The generator was built by using resistive array. Different images and patterns were observed by using the thermal camera. Using ways to transfer heat on the board like constructing thermal highways had significant effect on improving the response of infrared scene generator. The intervals between resistors were good and heat exchange between them did not take place. In total, the response of the built infrared scene

9 132 generator was appropriate and satisfactory. Cooling process of the infrared image was analyzed. One of the important results of this study was the investigation of the infrared scene generator error. The generator with an acceptable error that is mentioned in the fifth chapter simulates the infrared images. The error has been obtained by using standards of MSE and PSNR. Another important result of this study is that the generator error will increase by increasing the brightness of its pixels. References [1] Global defence, rockets, global defence describes how defene instruments work. Last modified: March [2] Milnet, rockets, website about rockets. Last modified: July [3] Zhou Qiong, Dynamic scene simulation techonlogy used for infrared seeker, SPIE, Vol. 7383, [4] How Stuff Works, Sidewinder rocket, How Stuff Works explains thousands of topics. Last modified: June [5] Mohammad Sajjad Ghasemi, designing of hardware in the loop simulation lab system. Flight Simulator Conference, Institute of aerospace, Tehran, [6] Maxfield, Clive. The Design Warior's Guide to FPGAs: Devices, Tools and Flows, Elsevier. ISBN , [7] Welstead, Stephen T, Fractal and wavelet image compression techniques. SPIE Publication. pp ISBN , [8] Welstead, Stephen T, Fractal and wavelet image compression techniques. SPIE Publication. pp ISBN , 1999.