Using Infrared Array Devices in Smart Home Observation and Diagnostics

Transcription

1 Using Infrared Array Devices in Smart Home Observation and Diagnostics Galidiya Petrova 1, Grisha Spasov 2, Vasil Tsvetkov 3, 1 Department of Electronics at Technical University Sofia, Plovdiv branch, Bulgaria 2 Department of Department of Computer systems and technologies Sofia, Plovdiv branch, Bulgaria 3 Department of Department of Computer systems and technologies Sofia, Plovdiv branch, Bulgaria ABSTRACT This paper describes a low-cost solution for smart home observation and diagnostics with the use of a new IR array sensor. The developed system is based on PIC32-MAXI-WEB development board with integrated infrared sensor MLX The system structure and consequent steps of data processing in order to construct thermal image are described. Some details in the process of raw data conversion to the assigned RGB values for every pixel in the field of view of IR sensor are discussed. The developed system is considered to be with reasonable cost and affordable for the potential users in their everyday environment. The paper demonstrates the benefits of having an infrared sensor as a part of the developed monitoring module for detection of thermal losses and water leakage in a system for smart home diagnostics. Keywords: smart home diagnostics, home automation, infrared sensor, thermography. 1. INTRODUCTION There are many currently existing fields where infrared emission of the objects is measured, but with the increasing of the capabilities of the affordable microcontrollers and IR sensors it is possible to develop low-cost solutions for applications of infrared measurements in everyday environment. In a smart home network the IR sensor could be used as a presence device or temperature sensor [1]. It also could alert if there is any appliances left turn on, like stoves, heating devices, coffee makers, toasters and others that could potentially cause fire and having a way of preventing this in the earliest stages is something that must be a feature of the smart home network. IR array sensor could serve as a presence detector in security systems or in Heating, Ventilation and Air Conditioning (HVAC) systems, with the ability to determine the number and positioning of the people. The people detection, counting and localization are considered important and key control variables in smart sensing and modern HVAC on demand systems [2]. Having the precision of a fraction of a degree Celsius and based on the size of the object it is possible to distinguish different objects like animals that could potentially falsely alert that there is an intruder. Another possible useful application is in hospitals, personal and ambient assisted leaving (AAL) systems where is necessary to have observation if the patient has fallen and is not moving. In all these cases because of privacy issues it is recommended not using a normal camera for patient monitoring [3]. However, the usage of the traditional thermal cameras that provide very high-resolution thermal images for all these applications is very expensive. It is important these systems to have reasonable cost and to be affordable for potential users in their everyday environment. In comparison of human-sensing methods for detecting presence, count, location and tracking [4] using an inexpensive thermal infrared sensor gives promising alternative [5]. The goal of this paper is to present and demonstrate a low-cost solution based on application of a comparatively new IR array sensor in the field of smart home diagnostics and observation. 2. OVERVIEW AND DESCRIPTION OF THE DEVELOPED SYSTEM The developed solution is based on integration of an open source hardware platform and a small size IR array sensor. The basic structure of the developed system is shown on Figure 1. It consists of 16x4 array infrared and temperature sensor MLX90620, microcontroller development board PIC32-MAXI-WEB, router and PC. The Melexis MLX90620 IR array sensor is easy to integrate, small in size and with low power consumption [6]. Due to the factory calibrated temperature measurement with calibration parameters stored in the internal EEPROM it has high precision of measurement. All these features are a good reason for selection of this reliable sensor to be used in the developed solution for smart home diagnostics. In addition to 64 (16x4) IR pixels with dedicated low noise chopper Volume 4, Issue 1, January 2016 Page 9

2 stabilized amplifier and integrated fast ADC, a Proportional To Absolute Temperature (PTAT) sensor is integrated to measure the ambient temperature of the sensor chip. The output data of both IR and PTAT sensors are stored in the internal RAM (16-bit result of IR measurement for each individual sensor (64 words) and 16-bit result of PTAT sensor) and are accessible through I 2 C digital interface that supports clock speed up to 1MHz. The microcontroller connected to the MLX90620 sensor can read the different RAM data and, based on the calibration data stored in the internal EEPROM memory (256x8) of the sensor, compensate for difference between sensors to build up a thermal image, or calculate the temperature at each spot of the imaged scene. Figure 1 Basic structure of the system Open source hardware platform PIC32-MAXI-WEB produced by Olimex is a development board equipped with a high performance microcontroller (PIC32MX795F512L) [7]. It is easy for programing and supports the needed communication protocols and the necessary ports. It has embedded 100Mbit Ethernet module that allows to develop easily Ethernet connectivity applications. Along with the PIC KIT3 programmer there are some other useful features of the microcontroller which are in help in the process of program development and debugging, like the 240x320 TFT LCD with touchscreen, trimmer potentiometer and the buttons and leads. The microcontroller periodically samples the sensor, gathers the information from the 64 infrared sensors (pixels) ant the temperature sensor, then performs calculations for each pixel individually using the result from the temperature sensor and the calibration constants obtained in the process of sensor manufacturing which are stored in its internal EEPROM memory. The raw data from the sensor first are converted to human readable values, which in turn are translated to values for red, green and blue for each corresponding temperature with the help of a look-up table of values. The RGB values are stored in the microcontroller memory. Those values could be sent through a network to a client with the help of the Microchip TCP/IP Stack. The stack has all the necessary network management responsibilities and facilitates the process of establishing connection with the IR sensor via Internet. After the stack is configured a small web application which is stored in the memory of the microcontroller services all external requests. Upon request from the client side the developed web application provides the RGB data. The microcontroller is connected to the network using RJ45 connector (Ethernet LAN). The microcontroller could be accessed via web browser through the router using the fixed IP address. It is possible to enter a mode where measurements are taken only at the user s request. In this way the power consumption of the IR sensor is greatly decreased and the microcontroller has resources to do some other useful work. 3. DATA PROCESSING The data gathering and further processing is described in this section. As it was mentioned earlier inside of the TO-39 package the MLX90620 sensor contains the 16x4 IR array, thermometer, 65 words (16 bit) of RAM, 256 words of EEPROM and two internal configuration registers. After the microcontroller establishes communication with the sensor and initializes the configuration registers the sensor starts measuring the temperature of its field of view and the proportional to ambient temperature of the package. The data is then being read periodically by the microcontroller using the I 2 C digital interface protocol with clock speed supported from the sensor up to 1MHz. In order to extract some human readable data from the raw 16 bit sequence of zeros and ones obtained from each IR pixel, the microcontroller has to perform some heavyweight calculations, such as: offset compensation, thermal gradient compensation, pixel to pixel normalization and emissivity compensation. This step could potentially become a threat for a bottleneck if a microcontroller with lower computation power and limit of the frames per second visualization was selected, despite the refresh rate of the sensor being of 0,5Hz to 512 Hz. At that point the raw data is an array of 64 Volume 4, Issue 1, January 2016 Page 10

3 floating point values that represent the temperature that each pixel sees. That itself is useful information, if the goal is to target the temperature of a certain point and simply display it as a floating point value or to alert the system and start executing some other application specific functions. In the applications where the goal is to construct a thermal picture based on the values from the array it is necessary to assign RGB values to each pixel. This process is very flexible as there are many ways of doing that. A very straightforward approach is used in our system, because the colors of the table are chosen as follows: blue for the low temperatures, red for the hot, with the transition from light-blue, blue-green, green, yellow and orange in between. However, the range of the temperature that is measured is selected by the user, depending on the specific preferences and the current application. The number of the entries of the RGB look-up table and each separate value for red, green and blue is fixed. Determining which entry of the look-up table to pick is done by determining where inside the user selected range is the value of the current pixel if it is below the low value we assume RGB values as if it is the lowest in the table. If it is above the high boundary we assign the highest RGB values from the table. Anything value in between means assigning RGB values for the corresponding temperature. This data processing is done by using the following two relations: S = (H - L) / N, (1) where: S is the step, temperature value that separates two entries in the table, H is the high boundary, L is the low boundary and N is the number of entries in the table. T = (P L) / S, (2) where T is rounded to integer and is the entry of the look-up table and P is the temperature of the pixel. An option exists that allows the high and low boundaries to be selected automatically for each frame based on the current highest and lowest measured temperature. By constantly collecting information for each pixel we can construct a real time representation of the temperature of the surrounding area. Sometimes in the manufacturing process or due to errors in the calibration process errors could visibly appear in some of the pixels. A practical approach of correcting these types of errors is applied in the developed system that corrects those pixels and brings them closer to the real value with minimal deviation. In that case a special calibration sequence is started for the damaged pixel. The calibration procedure consists of taking measurements from object with consistent temperature up close so that each pixel should see the same value. The value seen from the good pixels is considered the correct one and the value of the bad pixel is considered with error. At least two measurements need to be taken for two different temperatures and all the values are saved. Then the values are plotted on a Cartesian plane and the function y = f(x), where f(x) is a polynomial so that when we plug in for x we get the approximated real value for y. As higher the degree of the polynomial as closer will get approximation but also will add more complex calculation load and could slow the system down if it gets too high. Finally, the information from the 64 pixels is stored in the microcontroller memory in the form of RGB values. An example of thermal picture constructed from the RGB values and the pixels position in the whole field of view is shown on Figure 2. Figure 2 Thermal picture and the pixels position in the sensor field of view Volume 4, Issue 1, January 2016 Page 11

4 4. APPLICATION OF THE IR SENSOR IN SMART HOME DIAGNOSTICS Having a way of obtaining a thermal image of a certain area presents a wide variety of applications in smart home, building observation and diagnostics. Very useful and attractive applications in building diagnostics are thermal loss and water leakage. The high precision measurement could detect the small temperature gradient throughout the wall, floor or ceiling and determine the source of thermal leakage. In case of water leakage IR sensor could detect the source even before the water breaks to the surface and damages the interior. In order to demonstrate these possible applications of our solution we have aligned spatially and matched the image from the web camera and the thermal image from the IR array sensor. The results of this procedure for the two described cases for building diagnostics are presented on the next figures. On Figure 3 an example for application of MLX90620 sensor in detection of thermal loss is shown, while Figure 4 shows an example for application of the sensor in detection of water leakage. Figure 3 Example for application of MLX90620 sensor in detection of thermal loss Figure 4 Example for application of MLX90620 sensor in detection of water leakage In general, with the employment of IR array sensors more complete information for the parameters of certain objects and processes in home environment could be obtained. This will increase the understanding and simplifies the process of picking the best reaction to the changes of important objects parameters. Additionally, the usage of infrared sensors instead of simple everyday camera will help to avoid privacy invasion issues but still maintain the necessary observation duties. 5. CONCLUSION In this paper a low-cost solution for smart home diagnostics and observation with the use of a new IR array sensor is described. Based on an open source hardware microcontroller and an inexpensive IR sensor the developed system is considered to be with reasonable cost and affordable for the potential users in their everyday environment. The paper demonstrates the benefits of having an infrared sensor as a part of the developed monitoring module for detection of thermal losses and water leakage in a system for smart home diagnostics. In the future work the IR sensor will be separated away from the main microcontroller thus having a battery powered device that will communicate wirelessly with the access point. This will give additional flexibility for measurement of objects in difficult to reach places without the need of wires. The further step is to build a network of those devices in order to have a way of observing and tracking the subjects in hospital and home environment. Acknowledgments The work presented in this paper is supported by project ДФНИ Е02/12 Investigation of methods and tools for application of cloud technologies in the measurement and control in the power system and Melexis Microelectronic Integrated Systems. Volume 4, Issue 1, January 2016 Page 12

5 References [1] Li, N., Calis, G., and Becerik-Gerber, B., Measuring and monitoring occupancy with an RFID based system for demand-driven HVAC operations, Automation in construction 24 (2012), pp [2] Melexis Microelectronic Integrated Systems, White paper MLX90620 for smart & green HVAC, April [3] Hnat, T. W., Griffiths, E., Dawson, R., and Whitehouse, K. Doorjamb, Unobtrusive room-level tracking of people in homes using doorway sensors, In Proceedings of the 10 th ACM Conference on Embedded Network Sensor Systems (2012), ACM, pp [4] Teixeira, T., Dublon, G., and Savvides, A., A survey of human-sensing: Methods for detecting presence, count, location, track, and identity, Technical report, Embedded Networks and Applications Lab (ENALAB), Yale University, [5] Kumar, S., Marks, T., Jones, M., Improving Person Tracking Using an Inexpensive Thermal Infrared Sensor, CVPRW 2014 Intl. Conf. on Computer Vision and Pattern recognition Workshops, Columbus, USA, June 2014, pp , doi: /CVPRW [6] Melexis, Datasheet IR thermometer 16X4 sensor array MLX90620, Thermometer-Sensors/ Infrared-Thermometer-Sensors/MLX aspx, [7] Olimex, Web server PIC32-MAXI-WEB with 32 bit PIC microcontroller PIC32MX795F512L datasheet, AUTHORS Galidiya Petrova received MSc. degree in Electronics at Technical University of Sofia in 1985 and PhD degree in Biomedical engineering in She is presently Associate professor in the Department of Electronics, Technical University of Sofia, Plovdiv branch. Her research interests are within: data acquisition systems, applications of distributed systems in medicine, personalized healthcare and ambient assisted leaving systems, wireless body sensor networks. Grisha Spasov received MSc. degrees in Computer engineering at Technical University of Sofia in 1983 and PhD degree in Computer networks in He is presently Professor in the Department of Computer systems and technologies, Technical University of Sofia, Plovdiv branch. His research interests are within: Distributed embedded systems and distributed applications, Internet of Things, Service oriented architectures for distributed measurements, Computer networks and Wireless sensor networks. Vasil Tsvetkov received BSc. degrees in Computer systems and technologies Technical University of Sofia, Plovdiv branch, Bulgaria in He is presently MSc student in the Department of Computer systems and technologies, Technical University of Sofia, Plovdiv branch. His research interests are in the area embedded systems, firmware, ASIC design, FPGA prototyping and Internet of Things. Volume 4, Issue 1, January 2016 Page 13