Design and Implementation of E-skin and Optical Sensors Heartbeat Detection Glove

Transcription

1 Department of Electrical & Computer Engineering ECE 4600 Group Design Project Final Project Report Design and Implementation of E-skin and Optical Sensors Heartbeat Detection Glove by Group 02 HanCheng Kim Mingkun Wang Yankai Shao Guangyu Zhang Academic Advisor Dr. Jun Cai Department of Electrical and Computer Engineering University of Manitoba Co- Advisor Malcolm Xing Department Mechanical Engineering University of Manitoba Date of Submission March 9, 2018 Copyright 2018 HanCheng Kim, Yankai Shao, Mingkun Wang, Guangyu Zhang

2 Abstract The following report will cover the design and implement of a heartbeat detection purpose glove system with using e-skin and optical sensor. Heart rate, or pulse, is one of the vital health parameter used to measure basic function of human body. Heart rate is the number of times the heart beats per minute. (BPM). Instead of manually counting the number of pulses at an exposed artery on the arm for 30 seconds or 1 minute, using sensors digitally measure the heart rate is more direct and efficient. The goal of this project is to develop a heartbeat detection purpose glove by using e-skin and optical sensors that results in accurate heart rate of a person. Electronic skin sensor design is based on its external force sensitive property. By attaching on human body such as the wrist, the electrical properties such as resistance can be changed based on the physical changes of human skins such as heartbeat. Optical sensor design is based on the principle of photoplethysmography (PPG). It is a technique that measures the variation in blood volume by measuring the absorption of light by skin tissue using a light source and detector as the pumping action of each heartbeat changes the blood volume in any part of the body The e-skin and optical sensor are placed on a glove for convenient use application. The testing method is to attach the e-skin sensor onto the wrist or attach the optical sensor onto the finger or earlobe. The glove also used e-skin attached on the finger as a switch to select the sensor operations such as choosing the senor and selecting the algorithm modes to find out the heart rate. The heart rate from which sensor with the respect algorithm mode is displayed on the OLED screen attached on the glove. The heart rate can also be demonstrated on an Android mobile device through the Bluetooth connection. The heart rate data can be saved on the database in the mobile device. i

3 Guangyu Zhang HanCheng kim Yankai Shao Mingkun Wang G02 Final Report 2018 Contributions Research E-skin Pulse Dectection Optical Pulse Detection PCB Implementation E-skin Switch Power Management System MCU Programming Andriod Software Sytem Integration Sytem Validation and Testing Final Report Legend Contributed Lead task ii

4 Acknowledgements Our design team would like to thank several people for their support in this project. We would like to take this opportunity to give special thanks to our project advisor Prof. Jun Cai, for all the supports and guidance he provides throughout the course of this project. We would like to thank our project co-advisor Prof. Malcolm Xing and his team, for providing us the e-skin sensor to achieve the project. We would like to thank to electrical engineering tech-shop staff who helped us in acquiring all the components required to implement the project. Finally, we would like to thank Dr. Derek Oliver, Daniel Card and Aidan Topping for providing valuable feedbacks that allow us to improve our written and oral communication skills. iii

5 Table of Contents Abstract... i Contributions... ii Acknowledgements... iii Table of Contents... iv List of Figures... vii List of Tables... ix List of Abbreviations... x 1.Introduction E-skin pulse detection circuit design Introduction Objectives Motivation Limitation Block Diagram System Development Testing Method Simulation Circuit Passive High Pass Filter Non-inverting Amplifier Band Pass Filter Heartbeat Output Waveform on Oscilloscope Optical Pulse Detection Circuit Design Introduction Objective Theory Motivation Limitation System Development Block Diagram iv

6 3.2.2 PPG Sensor Sensor Circuit using TCRT Filter and Amplifier Circuit using LM Filter and Amplifier Design Heartbeat Output Waveform on Oscilloscope Testing Method Advantages Hardware components Introduction Objective Motivation Block Diagram PCB Microcontroller: Bluno Beetle E-skin Switch OLED Screen Battery Management System Hardware Design and Implementation Theory of E-skin Switch Limitation of the Design System Block Diagram Implementation MCU Programs Design Software Filter OLED Display E-skin Switch Bluetooth communication Sending the information Checking the Bluetooth Connection Heartbeat Rate Calculation Mode 1: Instantaneous Reading Algorithm Mode 2: Moving Average Algorithm Mode 3: Accumulating Average Algorithm Main Program for MCU v

7 6.Android Software Design Android Introduction Architecture Application components development environment Bluetooth Communication introduction protocol stack connection and pairing Application design Functions block diagram instruction Discussion Conclusion References Appendix A Budget Appendix B OLED Screen Interface Appendix C Gantt Chart Appendix D MCU Program code Appendix E LM324N Datasheet vi

8 List of Figures Figure 2-1 Block diagram of e -s kin system 3 F i g u r e 2-2 T e s t i n g m e t h o d 3 F i g u r e 2-3 E - s k i n s e n s o r c i r c u i t 4 F i g u r e 2-4 H P F w i t h b o d e p l o t t e r 4 F i g u r e 2-5 N o n -i n v e r t i n g a mp l i f i e r c i r c u i t 5 F i gure 2-6 Bandpass f i l t er d e s i gn p u r p o s e 6 F i gure 2-7 Bandpa ss F i lter s i mu lation circuit 6 Figure 2-8 Heartbeat output waveform on oscilloscope 6 F i gure 3-1 Block d i a gr a m o f o p t i c a l s ys t e m 8 F i g u r e 3-2 T r a n s mi t t a n c e P P G s e n s o r 9 F i g u r e 3-3 R e f l e c t a n c e P P G s e n s o r 9 F i gure 3-4 T CRT ( Reflect i ve Optical S ensor) 9 F i g u r e 3-5 S e n s o r Circuit u s i n g T CRT F i g u r e 3-6 T y p i c a l P P G w a v e f o r m 10 Figure 3-7 First Stage of Filter and amplifier circuit using LM Figure 3-8 Filter and a mplifier circuit using LM F i gure 3-9 Output w a ve f o r m o n o s c i l l o scope 12 Figure 4-1 Block diagram of the digital signal processing system 14 F i gure 4-2 E -s k i n a n d o p t i c a l s e n s o r c i r c u i t 16 F i g u r e 4-3 P C B t o p a n d b o t t o m l a y o u t 16 F i g u r e 4-4 B l u n o b e e t l e mi c r o c o n t r o l l e r 17 F i g u r e 4-5 E - s k i n s w i t c h o n t h e g l o v e 17 F i g u r e 4-6 S S D O L E D s c r e e n 18 vii

9 F i gu re 4-7 Li Po b attar y and the voltage p roperty 18 F i g u r e 4-8 P o w e r B o o s t C 19 F i g u r e 4-9 P o r t C o n n e c t i o n o f t h e M C U 20 F i g u r e D e v i c e t o p v i e w 21 F i g u r e C o m p o n e n t s o n e a c h l a y e r 21 Figure 5-1 Software filter for e -skin detected heartbeat signal 23 Figure 5-2 The e-skin and optical sensor detected pulse signals before and after software filter 23 Figure 5-3 Two e-skin switches signals after software filter 24 F i g u r e 5-4 P r o g r a m c o d e s t r u c t u r e 24 Figure 5-5 Program flow chart of the switching function 25 Figure 5-6 Algorithm for determine different calculation modes 27 Figure 5-7 Program flow chart for instantaneous algorithm 28 Figure 5-8 Program flow cha rt for moving average algorithm 29 Figure 5-9: Program flow chart for accumulating average algorithm 30 F i gure P rogram flow chart o f ma in l oop function 32 F i g u r e 6-1 A n d r o i d s o f t w a r e s t a c k 35 Figure 6-2 The Bluetooth low energy protocol stack architecture. 36 Figure 6-3 Bluetooth searching device program flow chart 37 F i g u r e 6-4 P r o g r a m f l o w c h a r t 38 Figure B-1 Optical detection in (a)m1 mode (b)m2 mode (c)m3 mode 43 Figure B-2 During the thumb e -skin sensor is being press ed 44 Figure B-3 Mode being switched 43 Figure B-4 E-skin detection in (a)m1 mode (b)m2 mode (c)m3 mode 43 Figure C-1 Gantt Chart 45 viii

10 List of Tables Table 5-1 Complete prefix table for Bluetooth communication 26 Table A-1 Budget 43 ix

11 List of Abbreviations Symbol AC BPF BPM DC ECG EDA E-skin HPF LPF MCU OLED PCB PCM SCL SDA LED PPG Description Alternating Current Band Pass Filter Beat Per Minute Direct Current Electrocardiography Electronic Design Automation Electronic skin High Pass Filter Low Pass Filter Microcontroller Unit Organic Light-Emitting Diode Printed Circuit Board Powertrain Control Module Serial Clock Line Serial Data Line Light Emitting Diode Photoplethysmoggraphy x

12 Chapter 1 Introduction Since the advanced materials developing in chemistry field, its influences have a tremendous impact on medical care, signal processing, energy storage and other things. The engineering community is undergoing a great revolution in the 21st century. Nanomaterials are contributing enormous power to the development of quantum computers. Biomaterials are changing the way we repair and detect broken body organs. The new materials allow engineer to develop and create unlimited possibilities. Graphene and GO paper, a kind of biomaterials, have been successfully fabricated via filtration. Graphene based paper can be made into actuators that responds to different stimuli. [1] Graphene, the carbon allotrope, a flat monolayer of carbon atoms tightly packed into a 2D honeycomb lattice with the thickness at the nanometer level, has recently excited scientific researchers with its excellent mechanical and electronic properties. [2] Look at it from an electrical engineer point of view. The resistance of Graphene paper decreases as the force increase act on it, it is a special force sensor. In addition, the Graphene has super thin, light, flexible and electroconductibility properties. Attach it on human wrist to detect pulse signal due to heartbeat become possible, therefore a technologic name is given to the advanced material, E-skin. E-skin is a kind of electrical skin that working on human skin. The capstone project centered on the development of the E-skin, research and design a device base on its electrical properties to measure the human heartrate. Simultaneously, using optical sensor to measure the human heartrate as a parallel design. Researching and comparing the advantages and disadvantages both of them

13 Chapter 2 E-skin pulse detection circuit design 2.1 Introduction Objectives The goal of capstone design is to emphasize and prove the e-skin is an available source to detect heartbeat and versatility. The method of using e-skin sensor is to collect the pulse signal from wrist and amplifier the signal. Then using hardware filter to reduce noise so that the signal can be read by microcontroller and further processing the available signal to get the heartbeat. Next, using a Bluetooth device to transmit the heartrate value wirelessly to smartphone and display the result Motivation Pulse contains sufficient physiological and pathological information, especially in the pulse wave. It has some definite physical meanings of feature points, the main wave, tidal wave and fall in the edges. Collecting the wave characteristics accurately, it can provide diagnostic value of clinical information and has important medical value. The e-skin sensor has super thin, light and flexible properties, attach it on the human skin is weightless and harmless. Design and implantation a wearable and tiny device by using e-skin to collect heartbeat signal is a big challenge to overcome. Finally, the device is convenient for people, such as patients and athletes, to record instant heartrate data and further determine the body healthy condition. That is effective to act accurately and expeditiously Limitation The E-skin sensor is not a commercial product that provide by instructor. Its electrical properties are not ideal. The resistance is not linear responsible to pressures. It is kindly negative correlation with force. Therefore, e-skin collected pulse signal is not as accurate as Electrocardiography(ECG). While attaching e-skin on human wrist, due to skin sickness of different populations and pressure on e-skin sensor by fingers, the intensity of pulse signal is different. Consider the other influencing factors, body waggle, contact surface changes and other unavoidable conditions, that all would cause the signal deficiency and distortion

14 Another disadvantage condition is that the e-skin has a strict requirement. Since it is a pressure sensitive sensor, which need to be place in a certain way of the human body. To receive more accurate data result, an intensity and clearness signals are needed. The e-skin deformation causes the resistance change. The largest e-skin deformation happens at the neck and wrist since those are the thinnest skin on the body. Therefore, the device would consider place at those two locations to receive the most distinct signal. 2.2 Block Diagram The E-skin heartbeat detector system contains seven parts. E-skin sensor setting, passive High Pass Filter, Non-inverting Amplifier, second order band pass filter, MCU processing, OLED display and smartphone. In Figure 2-1 showed the whole e-skin system which indicated the flowchart and main elements. 2.3 System Development Figure 2-1 Block diagram of e-skin system Testing Method An obvious skin deformation is from human wrist or neck. In order to make the whole circuit more integrate, a glove that contain each element has been created. Using thermal gel to stick one e-skin (0.7cm x 1.3cm) on the middle finger. This sensor is going to collect the pulse signal and further heartrate calculation. The photo of e-skin sensor position is shown on Figure 2-2. Figure 2-2 Testing method - 3 -

15 2.3.2 Simulation Circuit As shown on the figure. the e-skin is attached on wrist, through cascading with a normal resistance, the voltage signal that between e-skin and resistance would carry the pulse information. The peak voltage value due to heartrate is higher than the DC offset. Therefore, collecting the peak signal and amplifier it is essential method in order to go further processing. From starting increase the pulse signal by using LM324N operational amplifier [3], the noise would be increasing concurrently. Then using second order bandpass filter to decrease the noise. The whole e-skin circuit is shown on Figure 1-1 Figure 2-3 E-skin sensor circuit Passive High Pass Filter The resistor of e-skin (0.7cm x 1.3cm) in a steady condition is 550Ω. To receive the pulse signal, cascade the e-skin with a 387Ω resistor. Then output the middle voltage signal to passive high pass filter which is a RC circuit. The HPF cut-off frequency is fc = 1/(2*pi*R5*C1) = Hz The simulation circuit and bode diagram which are using Mutisim are shown on Figure 1-3. This main propose of the whole subsystem is to remove DC offset and let AC signal pass and block DC signal. The result of cut-off frequency is mHz which is at dB. Figure 2-4 HPF with bode plotter - 4 -

16 2.3.4 Non-inverting Amplifier At output of HPF, the HPF remove the DC offset and pass small AC signal. In order to make the pulse signal distinct, adding an operational amplifier is necessary. The circuit is shown on Figure 1-4. The AC signal is applied to the non-inverting input of the op-amp. In this way the signal at the output is not inverted when compared to the input. the voltage gain of the circuit Av can be taken as: Av = 1+R2/R1 = 1001 The amplifier circuit shown on Figure 1-4. Figure 2-5 Non-inverting amplifier circuit [4] Band Pass Filter The American Heart Association stated that the normal resting adult human heart rate is bpm. [5] Therefore, by using the Wizard Analog which is a filter design tools, set up a bandpass filter which bandwidth is 0.95Hz to 1.65Hz corresponding to heartrate 57 to 99 bpm as shown on Figure2-6. The circuit simulation is shown on Figure

17 Figure 2-6 Bandpass filter design purpose [6] Figure 2-7 Bandpass Filter simulation circuit Heartbeat Output Waveform on Oscilloscope The amplification of whole system is a thousand times. The output waveform signal as shown on Figure 1-11 is synchronous with pulse signal. Its voltage level is in range from 0.5V to 1V. Figure 2-8 Heartbeat output waveform on oscilloscope - 6 -

18 Chapter 3 Optical Pulse Detection Circuit Design 3.1 Introduction Optical heartbeat sensor is an electronic device that using a light source and a detector to measure the heart rate. The light source such as infrared emitting diode transmit through the finger and detect the amount of light that pass through the finger using a photodiode detector. The photodiode is a semiconductor that produces a current proportional to the amount of light that hits it. Which means that when the blood volume in the fingers increases, less light is getting through the finger and hitting the photodiode. [7] With each heartbeat, the pumping action of the heart will force blood flowing inside of the body and it will changes the blood volume in any part of the body. Therefore, the optical sensor that consists of an infrared LED and a photodiode can measure the heartbeat by measuring the variation of the light inside a finger artery due to the variation of the blood volume. The output signal from the detector is equivalent to the pulse signal. The optical heartbeat sensor can be used by placing the index finger onto the infrared LED and a photodiode on the same side Objective The aim of the optical sensor is to collect the pulse signal from the detector and remove the noise by using the filter. Then, amplifying the output PPG signal is necessary so that the microcontroller can read it and process the PPG signal to the heartbeat Theory The optical heartbeat sensor is based on the principle of Photoplethysmography (PPG). It measures the change in volume of blood through any organ of the body which causes a change in the light intensity through that vascular region. The flow of blood volume is decided by the rate of heartbeat and since light is absorbed by blood, the signal pulses are equivalent to the heartbeat pulses. [7] - 7 -

19 3.1.3 Motivation Monitoring heart rate is very important for people, especially for athletes and patients as it determines the condition of the heart. Heart rate can be monitored in two ways: one way is to manually check the pulse at the wrist (the radial pulse) or neck (carotid pulse) and the other way is to use a heartbeat sensor. [7] Using the sensor will save time and the results from the sensor can be saved and processed for furthermore application. The main motivation of the optical sensor design in this project is that we need this as a parallel design to compare the results with e-skin sensor and find the advantages or disadvantages of e-skin sensor Limitation It detects the ambient light which produces the error in heart rate measurement. 3.2 System Development Block Diagram Figure 3-1 Block diagram of optical system [8] PPG Sensor There are two types of PPG sensor: Transmission: light emitted from LED is transmitted through any vascular region of the body such as finger and received by the photodiode. [7] - 8 -

20 Figure 3-2 Transmittance PPG sensor [9] Reflection: light emitted from the LED is emitted into the tissue of the finger and then the reflected light is measured by the detector [7] Figure 3-3 Reflectance PPG sensor [9] Sensor Circuit using TCRT5000 In this project, we choose TCRT5000 reflective optical sensor for Photoplethysmography. Instead of using the emitter diode and detector separately, TCRT5000 reflective optical sensor combines both the infrared light emitter diode and the detector side by side in a leaded package so that there is a minimum effect of surrounding visible light which reduces the error in the output signal. Figure 3-4 TCRT5000 (Reflective Optical Sensor) [10] - 9 -

21 The circuit diagram below shows the external biasing circuit for the TCRT5000 sensor. By placing the fingertip over the sensor, the emitting LED will be transmitted into finger artery and the amount of light reflected back from the fingertip is detected by the phototransistor. Figure 3-5 Sensor Circuit using TCRT5000 [10] The output signal from the detector has two components: AC component and DC component. The AC component is synchronous with the heartbeat which it is mainly caused by pulsatile changes in arterial blood volume. Therefore, the AC component can be used as a source of heart rate measurement information. This AC component is superimposed onto a large DC component that relates to the tissues and to the average blood volume. [11] To measure the AC waveform with a high signal-to-noise ratio, the DC component must be removed. Since the useful AC signal is only a very small portion of the whole PPG signal, an effective amplification circuit is also required to extract desired information from it. [11] Figure 3-6 Typical PPG waveform [12]

22 3.2.4 Filter and Amplifier Circuit using LM324 The Figure 3.7 shows the first stage of the filter and amplifier circuit design. The objective is to block the DC component and amplifying the weak AC component that is synchronous with the heartbeat. Figure 3-7 First Stage of Filter and amplifier circuit using LM324 In Figure 3.7, the sensor output is first passed through a passive high-pass filter (HPF) to block the DC component. The cut-off frequency of the HPF is set to 0.72 Hz. The signal then passed through an active low-pass filter (LPF) that is made of an LM324 Op-Amp circuit. The cut-off frequency of the LPF is set to 2.34 Hz and the gain is set to 101 to amplify the signal. To ensure that we get a clear signal synchronous with the heartbeat, a similar HPF/LPF combination is used as the second stage for further filtering and amplification that is shown in the Figure 3.8. Therefore, the total gain from the two cascaded stages is 101*101 = The final output signal will be connected to an ADC channel of a microcontroller for further processing. Figure 3-8 Filter and amplifier circuit using LM

23 3.2.5 Filter and Amplifier Design Passive High Pass Filter: Choose the value for the low cut-off frequency fc= 0.72 Hz Choose the value of capacitor C = 1 uf Calculate the value of R using equation fc = 1/(2πRC), R = 220 KΩ Active Low Pass Filter: Choose the value for the high cut off frequency fc = 2.34 Hz Choose the value of capacitor C = 0.1 uf Calculate the value of R using equation fc = 1/(2πRC), R = 680 KΩ Gain = (680K / 6.8K ) + 1 = Heartbeat Output Waveform on Oscilloscope expected. As shown on Figure 3-9 the output wave of the optical sensor is synchronous with the heartbeat as Figure 3-9 Output waveform on oscilloscope Testing Method The TCRT5000 reflective optical sensor is placed on index finger of the e-skin glove. To detect the examinee s pulse, the examinant will wear the e-skin glove and use the index finger with optical sensor to touch the examinee s wrist or finger. The heartbeat result will display on the OLED screen of the e-skin glove

24 3.2.8 Advantages PPG optical sensor has low cost with long useful life. The way to test the PPG optical sensor is more effective. It saves a lot of time to detect examinee s pulse with wearing the e-skin glove. Chapter 4 Hardware components 4.1 Introduction Objective The aim of this processing system is to firstly filter the heartbeat signals which coming from the hardware filter circuit, so the microcontroller would be able to calculate the BPM value based on the three different algorithms. The e-skin switch will choose whether the optical sensor or the e-skin sensor to detect the heartbeat signal. Furthermore, the e-skin switch will also control the microcontroller to circulate through the three different heartbeat calculation algorithms Motivation By having the MCU, the extensibility of the design can be extended. Also, with the help of the MCU, software filter can be designed and programmed to make the heartbeat signal clear. The e-skin switch makes the project possible to be one hand controlled. And the OLED screen displays the BPM results and the current state of the program, this makes the project integrated and to be a wearable device

25 4.2 Block Diagram Figure 4-1 Block diagram of the digital signal processing system As shown on the above diagram, the e-skin switch tells the MCU which heartbeat signal to take (whether from optical or e-skin sensor). Then the signal will be filtered by software filter and processed to the calculation algorithm. The e-skin switch is also able to select from three different calculation algorithms. Finally, the calculated BPM value together with the input heartbeat signal will be send to an Android smart phone through the on-board Bluetooth module. 4.3 PCB Cadence Design Systems, Inc. is an American multinational electronic design automation (EDA) software and engineering services company. The company produces software, hardware and silicon structures for designing integrated circuits, systems on chips (SoCs) and printed circuit boards. [13] The schematic of hardware system that contains e-skin and optical sensor shown on Figure

26 Figure 4-2 E-skin and optical sensor circuit

27 Through using Cadence V17.0, the top and bottom PCB layout shown on Figure On top PCB which shown on Figure 4-1(a) indicates the wiring path of 5V supply and electric components, on bottom PCB layout which shown on (b) indicates the GND wiring approach. (a) Top wiring path (b) Bottom wiring path Figure 4-2 PCB top and bottom layout The below figures show top and bottom PCB layout. (a)pcb Top layout (b) PCB Bottom layout 4.4 Microcontroller: Bluno Beetle Figure 4-3 PCB top and bottom layout Since the project is a small wearable device and the project requires Bluetooth communication. After searching online, the Bluno Beetle (DFR0339) microcontroller chip has been chosen. As shown in Figure

28 Figure 4-4 Bluno beetle microcontroller There are several benefits of using this MCU. Firstly, it has an on-board Bluetooth module, which does not require much programing for setting up the Bluetooth. Secondly, this MCU is an Arduino UNO based MCU. Since the Arduino has a lot of program code library support, this could simplify the programming process. Lastly, this MCU is really small and compact as shown above, it is ideal for wearable device design. 4.5 E-skin Switch The e-skin switch is designed by connect the e-skin sensor in series with a resistor to form a voltage divider circuit. In order to have more control functions for the project. Two e-skin switches have been used in the project. As shown in the below figure. Figure 4-5 E-skin switch on the glove More detail on the e-skin switch design and implementation will be discussed later

29 4.6 OLED Screen For choosing a suitable display screen for the project. The screen has to be small for this wearable device and low power consumption. The SSD1306 OLED screen has been chosen for the project, as shown in Figure Battery Management System Figure 4-6 SSD1306 OLED screen Lithium ion polymer batteries are thin, light and powerful. The output ranges from 4.2V when completely charged to 3.7V. The Lipo battery size is 3mm x 20mm x 30mm, with PCM board, with 2 wires which are positive and negative anodes to install to device. (a) A LiPo battery (b) Voltage output over discharge capacity [15] Figure 4-7 LiPo battary and the voltage property As shown on Figure 2-1 (b), the voltage starts at 4.2 maximum and quickly drops down to about 3.7V. Once you hit 3.4V the battery is dead and at 3.0V the cut-off circuitry disconnects the battery. When

30 using 3.7V LiPo battery, keep the output voltage steady is a major element to make capstone project successful. Therefore, an voltage booster module would be used to achieve the function. PowerBoost 500C, as shown on Figure 1-8, is the power supply that working properly with capstone project. Dimensions is 22mm x 37mm x 2mm. With a built-in battery charger circuit, that would be able to keep the whole system running even while recharging the LiPo battery. This little DC/DC boost converter module can be powered by any 3.7V LiIon/LiPoly battery and convert the battery output to 5.2V DC for running 5V projects. [15] Figure 4-8 PowerBoost 500C 4.8Hardware Design and Implementation Theory of E-skin Switch Since the e-skin sensor behaves similar with a pressure sensitive resistor. By connecting the e-skin sensor in series with a resistor to form a voltage divider circuit and using MCU to read the voltage of the e-skin sensor, the pressure change of the e-skin sensor can be detected. Low pass filter is needed when the e-skin is working as a switch Limitation of the Design As shown on the Figure 4-5, there are two e-skin switches used in the project. The e-skin on the thumb is implemented using the voltage divider circuit as mentioned above. However, the e-skin on the middle finger is mainly for heartbeat detection and its circuit is the filter circuit for heartbeat detection, which is not a voltage divider circuit. Thus, it behaves differently than the other one. A major difference is when applying a constant force on both e-skin sensors, the thumb e-skin s voltage will stay high, since the force is applied constantly. However, the middle one will only generate a small pulse even the force is applied constantly, this is due to the bandpass filter circuit

31 Because of this characteristic and the heartbeat detection purpose of the middle e-skin, two different software filters are applied for the e-skin switches. For the thumb e-skin, the software filter is to decrease the sensitivity of the e-skin. So only an intended pressing by user will cause a voltage change, thus activate the switch. This helps to prevent mistake pressing. For the middle finger e-skin, software filter will filter out the noise. And the MCU will only consider the middle finger e-skin as a switch input when the thumb e-skin switch is active. Another limitation is the interrupt technology for programming the switch function could not be used. Because the external interrupt port on the Bluno beetle MCU is the digital port, without additional comparator circuit, if the e-skin switch is connected to the digital port, the port reading will always be high. Thus, instead of design a comparator circuit for the e-skin switch, the polling technology for switching function will be used. This meets the design requirement also helps to keep the circuit small System Block Diagram Figure 4-9 Port Connection of the MCU

32 4.8.4 Implementation Figure 4-10 Device top view (a)top layer view (b)base layer view Figure 4-11: Components on each layer As shown in the above figures, the e-skin pulse detection sensor had been attached on the middle finger of the glove. The optical pulse detection sensor had been attached on the index finger of the glove. And another piece of e-skin sensor had been attached on the thumb finger of the glove to implement the e- skin switch

33 Chapter 5 MCU Programs Design In this section of the report, the program code and algorithm in the MCU will be discussed. With all the hardware components discussed above, the MCU is able to perform several tasks as listed below: Software filters: for further filter out the input heartbeat signals as well as the e-skin switch signals. Heartbeat rate calculation: three different calculation algorithms have been implemented for both sensor to calculate the BPM values. E-skin switch control: by using two switch functions, the e-skin switches are able to select optical or e-skin sensor for heartbeat detection as well as control the MCU to circulate through three BPM calculation algorithms. Display information on the OLED: by using the existing SSD1306 library, the OLED screen can be controlled to display information, such as: current detection sensor, calculated BPM value and current algorithm. Bluetooth communication: MCU will send the raw heartbeat signal together with the calculated BPM value to an Android smart phone for further analysis. Next, the detail explanations on each task will be performed accordingly. Since the heartbeat rate calculation function involves other functions, such as updating the OLED screen, it will be discussed after other functions have been introduced. Finally, the main program for the MCU will be introduced. 5.1 Software Filter Although the input heartbeat signals into the MCU have already been filtered by the hardware filter circuit. But in order to have a clear and steady heartbeat signal for BPM calculation, the software filter is needed. The software filter that has been implemented is based on first order low pass filter. H(s) = 1 1+ s wc where w c = 2π f c [16]. With a chosen cut-off frequency, the expression of H(s) could be obtained. And by using hand calculation or MATLAB, the transfer function could be change to Z domain (discrete model). And lastly, based on the H(z) function, the time domain expression could be obtained and implemented on Arduino program [17]. By using this method, the software filter structure is always in this form:

34 rawinput = analogread(sensor) A; FilteredInput = (int)(b FilteredInput + C rawinput) where A, B and C are constant numbers Thus, by changing the values of A, B and C, the software filter can be adjusted to meet the requirement. The below figure shows the actual implementation of the software filter in the program. Figure 5-1 Software filter for e-skin detected heartbeat signal All the three software filters (for e-skin heartbeat signal, for optical heartbeat signal and for e-skin switch signal) in the program are developed using this method. The detail program code can be found in the appendix. The below figure shows the signals after the software filter. (a) Signal from e-skin sensor (b) Signal from the optical sensor Figure 5-2: The e-skin and optical sensor detected pulse signals before and after software filter The above two outputs are from e-skin (left side) and optical (right side) sensor detected pulse signals. The blue curves are the pulse signals before software filters and the red curves are the pulse signals after software filters. It is noticeable that the signals after the software filters are clear and stable

35 Figure 5-3 Two e-skin switches signals after software filter The above outputs are obtained by pressing two e-skin sensors simultaneously to acting as e-skin switches. The red curve is from the thumb finger e-skin sensor, it shows that the signal is clear and stable. And the signal stays high while the sensor is being pressed. The blue curve is from the middle finger e-skin sensor, it shows that the signal is clear and stable. However, as expected, even the e-skin is being pressed, it only generates a pulse signal due to the bandpass filter circuit. 5.2 OLED Display The OLED display is programmed based on the existing SSD1306 display library. This library has a general code structure for displaying on the screen. The below figure shows the example from the heartbeat calculation function. Figure 5-4 Program code structure

36 The five draw function shown above are defined at the beginning of the program code as void functions, every time MCU process through this code structure will update the OLED screen once. The complete program code can be found in appendix D. 5.3 E-skin Switch As mentioned above, the e-skin switch is designed to select which heartbeat detection sensor to use. And the program flow chart is shown blow. Also, e-skin switch is designed to circulate through 3 different calculation modes. For this purpose, the program flow chart can be found in Figure 5-5. Since the program logic is the same, the program for selection different sensors will be used as an example for detail explanation. Figure 5-5 Program flow chart of the switching function As shown in the program flow chart, the program will call the switchinput() function to update the value of y1 and y2. The variable y1 is the software filtered e-skin switch signal from the middle finger e-skin sensor. The variable y2 is the software filtered e-skin switch signal from the thumb e-skin sensor. When y1 and y2 are both bigger than zero, it means both e-skin switch sensors have been pressed, the pulse detection sensor should be switched. After switched to another sensor, the program will command the OLED to draw the word Switched! on the OLED screen. And the word on the OLED screen will stay, until y2 back to zero. That is wait for the user to remove the force on the thumb e-skin switch sensor. The

37 program does not check y1 values after switching. Because of the bandpass filter circuit connected with the middle finger e-skin, y1 always back to zero automatically. 5.4 Bluetooth Communication Sending the Information With the help of the on-board Bluetooth module on the Bluno Beetle MCU, sending the information through Bluetooth is easy to implement. By using the command Serial.print() or Serial.println(), the information will be transmitted through Bluetooth. The information that will be transmitted through Bluetooth including the heartbeat signals of the two sensors (before the software filter), and the calculated BPM values coming from two sensors three different algorithms. In order to make the android software easy to sort different information, some prefixes have been added to information. For example, if the MCU wants to send the e-skin detected BPM value, the prefix e will be add in the front. If the value is calculated based on instantaneous algorithm (mode 1), the prefix 1_ will be added after e. If the number is 68, the transmitted string will be e1_68. The complete prefix table are shown below: Table 5-1 Complete prefix table for Bluetooth communication E-skin Sensor Optical Sensor Heartbeat Signals ERI_xxx ORI_xxx BPM values based on Instantaneous Algorithm (mode 1) e1_xxx o1_xxx BPM values based on Moving Average Algorithm (mode 2) BPM values based on Accumulating Average Algorithm (mode 3) e2_xxx e3_xxx o2_xxx o3_xxx Checking the Bluetooth Connection To check the Bluetooth connection, Android program will send a string 1 every time it receives a BPM value. And when MCU receives the string from the Android smart phone in the serial buffer, the MCU will command the OLED to draw the Bluetooth logo and clear the serial buffer

38 5.5 Heartbeat Rate Calculation After the heartbeat signals filtered by the software filter, the heartbeat rate calculation function will calculate the BPM value. There are two the heartbeat rate calculation functions in the program, one for e- skin detected pulse signal, one for optical sensor detected pulse detected pulse signal. The two functions have the same algorithms for calculating the BPM, but since two functions use different global variables and call different OLED display function, two separate functions are decided to be used, instead of having one heartbeat rata calculation function for both sensors. And in this report, the heartbeat rate calculation function for e-skin detected pulse signal will be used as an example for explanation. There are three different heartbeat rate calculation algorithms for each the heartbeat rate calculation function. And by input integer 1, 2 or 3, the function can select three different algorithms. The number 1 stands for mode 1 Instantaneous reading mode. The number 2 stands for mode 2 Moving average mode. The number 3 stands for mode 3 Accumulating average mode. Figure 5-6 shows the algorithm for determining different calculation mods. Figure 5-6 Algorithm for determine different calculation modes

39 5.5.1 Mode 1: Instantaneous Reading Algorithm For the instantaneous reading algorithm, the time difference, delta t, between two heartbeats is being calculated, the unit of this result is in millisecond. And by using one minute, which is milliseconds to divide this delta t, the BPM value could be obtained. Since the BPM value is always being calculated based on the one new obtain delta t, this algorithm is call instantaneous algorithm. The program flow chart for instantaneous reading algorithm is shown below. Figure 5-7 Program flow chart for instantaneous algorithm

40 5.5.2 Mode 2: Moving Average Algorithm For the moving average algorithm, the time differences, delta t, between two heartbeats of the recent six heartbeats are being calculated, where six is the sample number that the program uses. And by calculating the average delta t value and divided by milliseconds, the BPM value could be obtained. Since it always calculates the average delta t based on the recent six heartbeats, it called the moving average algorithm. The program flow chart for moving average algorithm is shown below. Figure 5-8 Program flow chart for moving average algorithm

41 5.5.3 Mode 3: Accumulating Average Algorithm Comparing to the moving average algorithm, instead of calculating the average delta t value based on the recent six heartbeats, the accumulating average algorithm calculates the average delta t value based on all the detected heartbeats. And similarly, the BPM value could be obtained by using the average delta t divides milliseconds. The program flow chart for moving average algorithm is shown below. Figure 5-9: Program flow chart for accumulating average algorithm

42 5.6 Main Program for MCU Before discussing the main program loop, two small function used in the main program will be explained firstly. The first function is called restdata function. This function simply set some global variables back to zero. These global variables include the calculated BPM values, the arrays of detected heartbeat interval times and so on. This rest reset function is necessary because every time the user switches to another sensor or selects another calculation mode, it is important to rest the storage to guarantee the accuracy of the BPM calculation. The second function is called sswitchtriggered function. This function is a software filter based on the previously discussed technology. This function filters the signal coming from the thumb finger e- skin sensor. If the user presses the e-skin sensor, this function will return a Boolean value true. As mentioned above the polling method is used for the switching. By calling this sswitchtriggered function, the MCU is able to check if the user has pressed the switch, if so stop the current program and execute the switching function. With the above explanations, the main loop function for the MCU could be discussed. The program flow chart is shown below

43 Figure 5-10: Program flow chart of main loop function The main program will firstly initialize all the required global variables. Then the MCU will check whether the user selected e-skin sensor for heartbeat detection or optical sensor for heartbeat detection. And then keep running the selected calculation algorithm and updating the OLED screen with according information. If the user pressed the thumb finger e-skin switch, the current running loop will be stopped. And the MCU will call the drawstop function to display information on the screen. While the thumb finger e-skin is being pressed, user could press the middle finger e-skin sensor to change another pulse detection sensor. After the thumb finger e-skin has been released, the MCU will go through three if command to switch to another calculation mode

44 In summary, the user can control the device in two ways. To select the other heartbeat detection sensor by holding or pressing the thumb finger once. And by holding the thumb finger e-skin sensor while pressing the middle finger e-skin sensor, the device will change to another BPM calculation algorithm. Chapter 6 Android Software Design 6.1 Android Introduction Android original meaning is robots. Nowadays, for people familiar, it is a one of most popular mobile operating system. Which have 85.9% global market share in smartphone 2017[19]. Not only for smartphones, but also for tablets, minicomputer, smart TV, watches and even car stereo. Android platform include an operating system, middleware and also key applications for use on mobile devices. The Android platform allows developers to write code using Java to design applications Architecture Android can be model as a leveled software stack. Which have 5 layers, each layer has a set of components. In Figure 6-1, from low to high are following, Linux Kernel, hardware abstraction layer (HAL), native libraries and android runtime, java API framework and system app. The whole architecture in simply words is combine Linux kernel and C/C++ libraries to manage and service the android runtime and apps.[19] The Linux Kernel Android platform is based on the Linux kernel. Android 1.5 to 3.2 is based on Linux 2.6 kernel. The newest version Android 8.0 is based on 4.10 kernel. Applications safety, memory management, processing management are all provided by Linux kernel. Hardware Abstraction Layer (HAL) The hardware abstraction layer (HAL) provides standard interface between device hardware and the software. Which means when a framework API makes a call to access device hardware, the Android system loads the library module contains in side HAL to hardware. Android Runtime

45 Android runtime is the application running power, include ART (Dalvik was android runtime before android 5.0) and core libraries. Native C/C++ Libraries Many core Android system components and services, such as ART and HAL, are built from native code that require native libraries written in C and C++. [20] java API Framework java API framework is the module which application direct use to simplifying the reuse. It includes following: View System, visual system to build app UI; Resource Manager, manage resources data; Notification Manager, let apps display alerts in the status bar; Activity Manager, manages the lifecycle; Content Providers, let apps can access data from other apps; System Apps This is the top layer of android Architecture. Android provide some core apps such as , SMS messaging, calendars, internet browsing, contacts, and more. And the most important, user can use 3-party apps

46 Figure 6-1 Android software stack[19] Application Components App components are the essential building blocks of an Android app. There are four different types of app components: activities, services, broadcast receivers and content providers Development Environment Android applications are usually developed in Java, but not like normal java compiler. It has to set a development environment. In 2013, Android Studio integrated development environment (IDE) was announced Google I/O conference. It based on IntelliJ IDEA, as replacement for the Eclipse Android Development Tools (ADT).[20] On 2016, google end support to Eclipse ADT. So, here we use android studio, also Java SDK. After coding, the whole application will package as apk file. 6.2 Bluetooth Communication

47 6.2.1 Introduction Bluetooth, as one of the popular way to transfer data over short distances, almost every mobile phone support it. Bluetooth Low Energy (BLE) was announced in 2011 as Bluetooth 4.0. compare with classic Bluetooth, BLE has lower power consumption. Because BLE remains in sleep mode constantly until there is a connection initiate. But, BLE cannot handle the large data that in this project the exchange data will not be large. BLE is a better choice Protocol Stack Figure 6-2 The Bluetooth low energy protocol stack architecture. [22] Connection and Pairing Creating a Bluetooth connection between two devices is a multi-step process involving three progressive states: Inquiry, Paging and Connection. If two Bluetooth devices know absolutely nothing about each other, one must run an inquiry to try to discover the other. One device sends out the inquiry request, and any device receiving such a request will respond with its address, and possibly its name and other information. Paging is the process of forming a connection between two Bluetooth devices. Before this connection can be initiated, each device needs to know the address of the other. After a device has completed the paging process, it enters the connection state. While connected, a device can either be actively participating or it can be put into a low power sleep mode. [23] 6.3 Application Design

48 6.3.1 Functions The android application can get data which transfer though Bluetooth by Arduino. Then recognition and classify each information. There s 2 kinds of sensor and 4 varieties of information which raw data, moving average algorithm, accumulating average algorithm and instantaneous reading algorithm. App will use raw data to draw a visual diagram. Showing all 3 heart beat values on the screen at same time. And save the accumulating average data using SQLite. User can search the history data Block Diagram Figure 6-3 Bluetooch searching device program flow chart

49 Figure 6-4 Program flow chart Instruction First, we need connecting the android device to the Arduino by Bluetooth. Press the scan button to start scan the nearby open Bluetooth device. Choosing our Arduino board name. the In the graph section, the raw data collected by sensor will be display. And in data section, all three of the calculate heart beat will be display. by click history button, user can search the data saved

50 Chapter 7 Discussion Comparing the results obtained from the testing on e-skin and optical pulse detection sensors, the heart rate is slightly more accurate with using the optical sensor. The main issue of the e-skin sensor is that e-skin is very sensitive to the pressure. With each testing, attaching the e-skin sensor on wrist, we cannot be able to press the sensor with constant equal force every time. On the other hand, we cannot attach the sensor on wrist with same position every testing as well. These two factors will result in big different resistance changes due to heartbeat. Since we need to measure the resistance change due to heartbeat to analyze the heart rate, the different force and testing position will produce some error while the output waveform from the e-skin sensor circuit is not a perfect pulse signal. Optical sensor is more reliable as a parallel design of e-skin sensor. We choosed TCRT5000 reflective optical sensor for Photoplethysmography (PPG). Instead of using the emitter diode and detector separately, TCRT5000 reflective optical sensor combines both the infrared light emitter diode and the detector side by side in a leaded package so that there is a minimum effect of surrounding visible light which reduces the error in the output signal. In addition, optical sensor is easier to use. In the reflectance PPG, the light source and the light detector are both placed on the same side of a body part. The light is emitted into the tissue and the reflected light is measured by the detector. As the light doesn t have to penetrate the body, the reflectance PPG can be applied to any parts of human body. [11] The considers of force and position do not matter while testing with optical sensor. For three algorithm modes to calculate the heart rate: the instantaneous mode is fastest to give you heart rate at the instantaneous time, but the result might not be accurate. The moving average mode calculates the heart rate slower than the instantaneous mode, but the result is more accurate. Lastly, the accumulating average mode is slowest in order to give you an accurate heart rate as it keeps calculating the average heart rate while testing

51 Chapter 8 Conclusion The resulted e-skin and optical heartbeat detection design met all performance matrix introduced in the proposal of the project. The capstone project achieved the goal of measuring the accurate heart rate in both e-skin sensor and optical sensors. The e-skin sensor pulse detection has error within the range of +/- 10 BPM; The optical sensor pulse detection has error within the range of +/- 5 BPM. The heart rate is displayed on the OLED screen attached on the glove. It can clearly show the heart rate from the selected sensor and the algorithm mode used. The glove used e-skin attached on the finger as a switch to select the sensor operations. We used three algorithm modes that are instantaneous mode, moving average mode and accumulating average mode in order to calculate the heart rate. The heart rate can also be demonstrated on an Android mobile device through the Bluetooth connection. The heart rate data can be saved on the database in the mobile device. For future improvement, the e-skin will have more extensive application instead of using as a pulse signal collector and a switch in this project. The e-skin imitates the human body and it not only can sense the pressure, but also the temperature as well. Now, e-skin is well-used in the medical area for health care. Therefore, e-skin can replace lots of tradition force sensitive sensor, to reduce the sensor product weight and size

52 References [1]D. Han, Y. Zhang, Y. Liu, Y. Liu, H. Jiang, B. Han, X. Fu, H. Ding, H. Xu and H. Sun, "Bioinspired Graphene Actuators Prepared by Unilateral UV Irradiation of Graphene Oxide Papers", Advanced Functional Materials, vol. 25, no. 28, pp , [2]A. Geim and K. Novoselov, "The rise of graphene", Nature Materials, vol. 6, no. 3, pp , [3]"LMx24-N, LM2902-N Low-Power, Quad-Operational Amplifiers", Ti.com. [Online]. Available: [Accessed: 06- Mar- 2018]. [4]"Non-Inverting Amplifier Op Amp Circuit Radio-Electronics.com", Radio-electronics.com, [Online]. Available: [Accessed: 06- Mar- 2018]. [5]"Heart rate", En.wikipedia.org. [Online]. Available: [Accessed: 06- Mar- 2018]. [6]"Filter Wizard Analog Devices", Analog.com. [Online]. Available: [Accessed: 06- Mar- 2018]. [7] Tarun Agarwal. (2014).Heartbeat Sensor Working $ Application [Online]. Available: [Accessed: 04- Mar- 2018]. [8] Kartik Rakhunde. (2017, Jan. 7). Heart Rate Measurement Using PPG Sensor TCRT pdf [Online]. Available: [Accessed: 04- Mar- 2018]. [9] Kaveh Mohamadabadi. (2017, March. 26).Photoplethysmography (PPG) [Online]. Available: [Accessed: 04- Mar- 2018]

53 [10] Dimitris Xarir. (2016). How To Use TCRT5000 With Arduino_DIY RPM Meter [Online]. Available: [Accessed: 04- Mar- 2018]. [11] Rajendra Bhatt. (2013, April. 20). Introducing Easy Pulse: A DIY photoplethysmographic sensor for measuring heart rate [Online]. Available: asy%20pulse%20%20a%20diy%20photoplethysmographic%20sensor%20for%20measuring%20he art%20rate%20%20embedded%20lab.html [Accessed: 04- Mar- 2018]. [12] Rajendra Bhatt. (2015, Oct. 13). Basic theory of PPG [Online]. Available: [Accessed: 04- Mar- 2018]. [13]"Cadence Design Systems", En.wikipedia.org. [Online]. Available: [Accessed: 06- Mar- 2018]. [14]"Voltages Li-Ion & LiPoly Batteries Adafruit Learning System", Learn.adafruit.com. [Online]. Available: [Accessed: 06- Mar- 2018]. [15]A. Industries, "PowerBoost 500 Charger - Rechargeable 5V Lipo USB 500mA+", Adafruit.com. [Online]. Available: [Accessed: 06- Mar- 2018]. [16]M. Kamenetsky, "Filtered Audio Demo", Web.stanford.edu. [Online]. Available: [Accessed: 06- Mar- 2018]. [17]"Digital filter Low-pass filter Using Arduino", YouTube, [Online]. Available: [Accessed: 06- Mar- 2018]. [18]"Smartphone OS market share worldwide Statistic", Statista, [Online]. Available: [Accessed: 06- Mar- 2018]. [19]R. Meier, Professional Android application development, 1st ed. Hoboken, N.J.: Wiley, 2010, pp

54 [20]"Platform Architecture Android Developers", Developer.android.com, [Online]. Available: [Accessed: 05- Mar- 2018]. [21]A. Notes, "Android Studio Release Notes Android Studio", Developer.android.com, [Online]. Available: [Accessed: 05- Mar- 2018]. [22]R. Heydon, Bluetooth low energy. Upper Saddle River, NJ: Prentice Hall, 2014, pp [23]"Bluetooth Basics - learn.sparkfun.com", Learn.sparkfun.com, [Online]. Available: [Accessed: 07- Mar- 2018]

55 Appendix A Budget Category Item Unit Cost Sensors E-skin sensors SFE Infrared Emitter and Detector Predicted Quantity Actual Quantity Actual Total Cost Supplier Bioengineering student Group Member Breadboard SOLDERLESS BREADBOARD Tech Shop Microcontroller Bluno Beetle Group Member OLED Screen 0.96 inch 128*86 resolution I2C interface OLED Display Group Member Power Supply Lithium Ion Polymer Battery - 3.7v 150mAh Group Member PCB PCB for the filter circuit Power Supply Circuit Component Adafruit PowerBoost 500 Charger (Rechargeable 5V Lipo USB 500ma+) Group Member Shipping 30 Total (Before Tax) Tax Predicted Total

56 Appendix B OLED Screen Interface The displays for three different BPM calculation algorithms while using optical sensor for pulse detection: Figure B-1: Optical detection in (a)m1 mode (b)m2 mode (c)m3 mode The displays during the thumb e-skin sensor is being pressed: Figure B-2: During the thumb e-skin sensor is being pressed The displays if the user press the middle finger e-skin while the thumb e-skin sensor is being pressed: (switched to another pulse detection sensor) Figure B-3: Mode being switched The displays for three different BPM calculation algorithms while using e-skin sensor for pulse detection: Figure B-4: E-skin detection in (a)m1 mode (b)m2 mode (c)m3 mode