Non-Contact Heart Rate Monitoring Using Lab Color Space

Transcription

1 46 phealth 2016 N. Maglaveras and E. Gizeli (Eds.) IOS Press, The authors and IOS Press. All rights reserved. doi: / Non-Contact Heart Rate Monitoring Using Lab Color Space Hamidur RAHMAN a,1, Mobyen Uddin AHMED a, and a Shahina BEGUM a School of Innovation, Design and Engineering Mälardalen University, SE Västerås, Sweden Abstract. Research progressing during the last decade focuses more on non-contact based systems to monitor Heart Rate (HR) which are simple, low-cost and comfortable to use. Most of the non-contact based systems are using RGB videos which is suitable for lab environment. However, it needs to progress considerably before they can be applied in real life applications. As luminance (light) has significance contribution on RGB videos HR monitoring using RGB videos are not efficient enough in real life applications in outdoor environment. This paper presents a HR monitoring method using Lab color facial video captured by a webcam of a laptop computer. Lab color space is device independent and HR can be extracted through facial skin color variation caused by blood circulation considering variable environmental light. Here, three different signal processing methods i.e., Fast Fourier Transform (FFT), Independent Component Analysis (ICA) and Principal Component Analysis (PCA) have been applied on the color channels in video recordings and blood volume pulse (BVP) has been extracted from the facial regions. In this study, HR is subsequently quantified and compare with a reference measurement. The result shows that high degrees of accuracy have been achieved compared to the reference measurements. Thus, this technology has significant potential for advancing personal health care, telemedicine and many real life applications such as driver monitoring. Keywords. Heart Rate, Signal Processing, Lab Color Space 1. Introduction Non-contact HR monitoring systems using camera needs to measure color variation in skin caused by cardiac pulse. The two factors that affect the color values are: the first one is the blood volume variations caused by cardiac pulse and the second one is the (temporal) environmental illumination variations during the video recording [1]. Almost all the non-contact methods are applying RGB color facial video which is suitable for laboratory environment or where the illumination of source light is constant [2-4]. In 2011, Poh et al. have proposed an algorithm to extract underlying source signals from R, G, and B bands [5]. The experiments were conducted using built-in webcam (isight camera) in indoor environment however their method is not real time and unable to extract HR where the ambient light is not constant. Similar experiments were taken place using different cameras for physiological parameters extraction which are referred in [6]. One of the main drawback of these systems are that they are not applicable for outdoor 1 Hamidur Rahman: Mälardalen University, SE Västerås, Sweden, hamidur.rahman@mdh.se

2 H. Rahman et al. / Non-Contact Heart Rate Monitoring Using Lab Color Space 47 environment where light source is not constant. A novel motivation has been observed in 2014 by Xiaobi et al. to reduce environmental illumination from RGB facial images using Normalized Mean Least Square (NLMS) adaptive filter [1]. NMLS filter is very sensitive to the scaling of the input light source and the calculation is also very complex. Another method has been proposed by Zhang et al. to eliminate environmental illuminance during real time HR monitoring using YCbCr images of the facial skin [7]. This paper presents a non-contact HR monitoring system based on Lab color space using a web camera. The environmental illuminance has a significant contribution in the color variation of the body blood. The proposed method could eliminate the illuminance effect during HR extraction from facial images. In this paper, first the RGB color space is converted into Lab color space consisting of three independent signals such as L, a and b where L represents the lightness of the images and a (red/green) and b (yellow/blue) represents the combination of other color channels. These two color channels have been separated from the Lab color space and using those color channels HR has been extracted. Three signal processing algorithm (FFT, ICA and PCA) have been applied on the color channels to extract HR and a high degree of accuracy is attained while comparing with a reference signal. Though this experiment has been conducted in lab environment but it is useful for the applications like [8, 9]. The rest of the paper is organized as follows: chapter 2 describes materials and methods including Data Collection procedures, Feature Extraction, Approach and transformation of RGB color space into Lab color space, chapter 3 presents experimental works. Finally, chapter 3 summarizes the work. 2. Materials and Methods Data acquisition was conducted on 10 participants (8 Male, 2 female) of different ages (25 to 50 years) and skin colors. The experiments were carried out in Lab environment and with a varying amount of ambient sunlight and artificial light. The participants were informed the aim of the study and they seated in front of a laptop computer at a distance of approximately 0.5 m from the built-in webcam (HP HD webcam). During the experiment, participants were asked to keep still, breathe spontaneously, and face the webcam while their video were recorded for 5 minutes at 30 frames per second (fps) with pixel resolution of 640 x 480 and saved in PNG (Portable Network Graphics) format in the laptop. Here, the RGB videos are converted into Lab videos as described in chapter 3 and finally L (luminance) were subtracted from Lab images to build ab pure color images without any environmental illumination. HR was extracted in offline using the converted Lab images and saved in an excel file. Simultaneously, HR was also recorded using ECG sensors and cstress system 2 and saved in another excel file. The main features of the Lab image is L, a and b. The system reads image frames one by one and for each image frame Lab features are extracted from the cropped facial image and saved in a temporary database which are used for further processing. Feature extraction for HR monitoring using RGB videos are described in [10, 11]. For the Lab features that have been extracted using RGB videos is shown in Fig

3 48 H. Rahman et al. / Non-Contact Heart Rate Monitoring Using Lab Color Space Figure 1. Step diagram for Feature Extraction and HR monitoring Three algorithms such as FFT, ICA and PCA have been applied to extract HR. The average of a and b signals were calculated using the FFT method [12]. For ICA [13], normalized raw traces were decomposed into two independent source signals (a and b) using the joint approximate diagonalization of Eigen matrices (JADE) algorithm. Similarly, normalized raw traces were also decomposed by PCA to find the principal components [14]. Finally, FFT is applied on the selected source signal to obtain the power spectrum. The pulse frequency was designated as the frequency that corresponds to the highest power of the spectrum within an operational frequency band. To extract HR, the number of peaks in frequency domain was calculated for first 50 image frames in the beginning and the required time is recorded. Therefore, HR is calculated as HR = 60*f h bpm (beat per minute) where f h is the extracted frequency of the HR. In the proposed method, HR extraction begins after the first 50 image frames and after that each image frame has been added to the database and the method provides a new HR. The Graphical User Interface (GUI) has been developed using Matlab to monitor the extracted HR. Fig. 2 shows the GUI which displays the Lab Color image, the detected face, pulse peak and current HR using the three methods. Figure 2. HR Monitoring GUI.

4 H. Rahman et al. / Non-Contact Heart Rate Monitoring Using Lab Color Space RGB to Lab Transformation RGB image of an object may look different when they are captured by different camera and different environmental light [8]. On the other hand, Lab color space is device independent where L represents the lightness of the image and the color channels a and b represents the other color combinations. a axis represents red/green opponent color with green at negative a values and red at positive a values. b axis represents yellow/blue opponent colors with blue at negative b values and yellow at positive b values. A true neutral gray values at a = 0 and b = 0 and L = 0 represents completely black and L = 100 represents the brightest white [15]. Let, r, g and b are the red, blue and green color values in RGB color space. It s chromaticity coordinates (x r,y r ), ( x g,y g ) and ( x b,y b ) and it s reference white coordinate is ( X w,y w,z w ). RGB color space is transformed into Lab color space by two steps calculations. First, RGB color space is transformed into XYZ color space [16] and later XYZ color space is converted into Lab color space [17]. These equations have been modified to fit for our expectation and later the algorithm has been implemented using MATLAB. Where, M is the transformation matrix and ( ) is the source color values related as: Here, Using the equation (2)-(5), equation (1) implies: Now Lab color space can be calculated as: Where,

5 50 H. Rahman et al. / Non-Contact Heart Rate Monitoring Using Lab Color Space Where Using equation (8)-(11) the Lab color space is derived from equation (7). 3. Experimental Work HR was extracted and recorded for 5 minutes for all the 10 test persons using webcam and cstress system. For each test subject there are 2 separate excel files for the extracted HR; one was for non-contact proposed method for Lab images and another one for cstress system as reference. For the statistical analysis minimum, maximum, average, median and standard deviations were calculated for the extracted HR both for Lab images and cstress system. Then the two important statistical parameters such as RSQ (Rsquared) and CORREL (Correlation Coefficient) for all three methods. RSQ is used to see how close the data are to the fitted regression line. RSQ = 0 indicates that the model explains none of the variability of the response data around it s mean and RSQ = 1 indicates that the model explains all the variability of the response data around it s mean. CORREL (also known as The Pearson product-moment correlation coefficient) is another statistical measurement of the correlation (linear association) between two sets of values. The CORREL value +1 indicates a strong positive correlation and -1 indicates a strong negative correlation. The statistical analyses of HR using Lab images for the 10 test subjects are presented in Table 1. As can be seen from Table 1, both the RSQ and CORREL values of the three proposed methods are presented in the table where the range of RSQ values are 87% to 99% and CORREL values are 93% to 99%.

6 H. Rahman et al. / Non-Contact Heart Rate Monitoring Using Lab Color Space 51 Table 1. Statistical Analysis of Proposed Methods Considering cstress System Statistical Subject Terms Method RSQ FFT ICA PCA CORREL FFT ICA PCA It is also noted that CORRELvalues seem to higher than RSQ for all the three methods. The average RSQ and CORREL values for the 10 subjects are calculated and presented using bar charts in Fig.3. Figure 3. Comparison among the three proposed methods for Lab color space considering cstress System 4. Conclusion A non-contact based HR extraction method is described in this paper using facial Lab color space which is easy to implement, low cost and comfortable for real life applications both in indoor and outdoor environment. Here, the methodology for recovering the cardiac pulse rate from video recordings of the human face and an implementation using a simple webcam with ambient daylight providing illumination are demonstrated. According to the experimental work, both the RSQ and CORREL values show highest closeness (i.e. 97%) with the reference measurements. From the table and the figures presented in the earlier chapter it is noted that CORREL seems to represent better than RSQ. It is also seen from Fig.3 that FFT method shows the best results among the three algorithms. More precise results can be achieved by taking the average of the three methods and using HD (High Definition) video (1280 x 720 or even 1980 x 1080). Given the low cost and widespread availability of webcams, this technology is promising for extending and improving access to medical care, real time driver monitoring. However, more experiment should be done to obtain more verifying systems. Although this paper only addresses the measurement of the cardiac HR, many other important physiological parameters such as, Respiration Rate, Heart Rate Variability and arterial Blood Oxygen Saturation can potentially be estimated using the proposed technique. Creating a real-time, multi-physiological parameters measurement platform with higher

7 52 H. Rahman et al. / Non-Contact Heart Rate Monitoring Using Lab Color Space resolution of videos based on this technology in driving situation will be the subject of future work. Acknowledgement The authors would like to acknowledge the Swedish Knowledge Foundation (KKS), Hök instrument AB, Volvo Car Corporation (VCC), The Swedish National Road and Transport Research Institute (VTI), Autoliv AB, Prevas AB Sweden, Hässlögymnasiets, Västerås and all the test subjects for their support of the research projects in this area. References [1] L. Xiaobai, C. Jie, Z. Guoying, and M. Pietikainen, "Remote Heart Rate Measurement from Face Videos under Realistic Situations," in Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, 2014, pp [2] G. D. Costa, "Optical remote sensing of heartbeats," Optics Communications, vol. 117, pp , 6/15/ [3] F. Zhao, M. Li, Y. Qian, and J. Z. Tsien, "Remote Measurements of Heart and Respiration Rates for Telemedicine," PLoS ONE, vol. 8, p. e71384, [4] W. Verkruysse, L. O. Svaasand, and J. S. Nelson, "Remote plethysmographic imaging using ambient light," Optics express, vol. 16, pp , [5] P. Ming-Zher, D. J. McDuff, and R. W. Picard, "Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam," Biomedical Engineering, IEEE Transactions on, vol. 58, pp. 7-11, [6] D. Shao, C. Liu, F. Tsow, Y. Yang, Z. Du, R. Iriya, et al., "Noncontact Monitoring of Blood Oxygen Saturation Using Camera and Dual-Wavelength Imaging System," Biomedical Engineering, IEEE Transactions on, vol. PP, pp. 1-1, [7] Z. Qi, X. Guo-qing, W. Ming, Z. Yimin, and F. Wei, "Webcam based non-contact real-time monitoring for the physiological parameters of drivers," in Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), 2014 IEEE 4th Annual International Conference on, 2014, pp [8] H. Rahman, S. Begum, and M. U. Ahmed, "Driver Monitoring in the Context of Autonomous Vehicle," in 13th Scandinavian Conf. on Artificial Intelligence, Halmstad, Sweden, [9] H. Rahman, S. Barua, and S. Begum, "Intelligent Driver Monitoring Based on Physiological Sensor Signals: Application Using Camera," in Intelligent Transportation Systems (ITSC), 2015 IEEE 18th International Conference on, 2015, pp [10] H. Rahman, M. U. Ahmed, and S. Begum, "Non-contact Physiological Parameters Extraction using Camera," in The 1st Workshop on Embedded Sensor Systems for Health through Internet of Things (ESS-H IoT), Oct., [11] H. Rahman, S. Begum, M. U. Ahmed, and P. Funk, "Real Time Heart Rate Monitoring From Facial RGB Color Video Using Webcam," in 29th Annual Workshop of the Swedish Artificial Intelligence Society (SAIS) 2016, Gothenburg, Sweden, [12] H. Buijs, A. Pomerleau, M. Fournier, and W. Tam, "Implementation of a fast Fourier transform (FFT) for image processing applications," Acoustics, Speech and Signal Processing, IEEE Transactions on, vol. 22, pp , [13] P. Comon, "Independent Component Analysis, a new Concept," Signal Process., vol. 36, pp , [14] M. A. Kramer, "Nonlinear principal component analysis using autoassociative neural networks," AIChE Journal, vol. 37, pp , [15] A. Ciobanu, T. Barbu, M. Costin, S. I. Bejinariu, and P. Radu, "A novel iris clustering approach using LAB color features," in Electrical and Electronics Engineering (ISEEE), th International Symposium on, 2013, pp [16] S. Kaur, E. R. Gill, and R. Kaur, "Comparative analyses of YCbCr color space and CIELab color space based on DWT and SVD," in Next Generation Computing Technologies (NGCT), st International Conference on, 2015, pp

8 H. Rahman et al. / Non-Contact Heart Rate Monitoring Using Lab Color Space 53 [17] C. Connolly and T. Fleiss, "A study of efficiency and accuracy in the transformation from RGB to CIELAB color space," Image Processing, IEEE Transactions on, vol. 6, pp , 1997.