ECG Biometrics using Intuitive Bases and Support Vector Machines. Peter Sam Raj

Transcription

1 ECG Biometrics using Intuitive Bases and Support Vector Machines by Peter Sam Raj A thesis submitted in conformity with the requirements for the degree of Master of Applied Science Graduate Department of Electrical and Computer Engineering University of Toronto c Copyright 214 by Peter Sam Raj

2 Abstract ECG Biometrics using Intuitive Bases and Support Vector Machines Peter Sam Raj Master of Applied Science Graduate Department of Electrical and Computer Engineering University of Toronto 214 In this thesis, a parts-based intuitive customized basis has been proposed using the non-negative sparse coding method for feature templates of the electrocardiogram(ecg) signal for biometrics. Combining with two classifier choices, two algorithms are proposed with complexity considerations for practical application scenarios using dependent classification. Using the proposed methods, a large scale fingertips ECG database was evaluated for biometrics potential, while observing the effect of user population size variation. The lowest known equal error rate (EER) for large-scale ECG biometrics was reported, which is 2.59% for 112 users. Two unprecedented databases containing signals from a single arm were collected and analysed for biometrics application, following evidence of ECG pulses in the upper left arm. They reported encouraging EERs, based on the proposed methods. Best EERs obtained were 3.26% in the single-lead, and 6.97% for the multi-lead signals, both for sitting posture. ii

3 Dedication To my family, Sherin Enocki, my sister, Stella and Suresh, my parents, I dedicate this to you. To Him who made me worthy of love, The Source of Joy I love You. iii

4 Acknowledgements My sincere thanks go to my advisor Prof. Dimitrios Hatzinakos, without whose guidance, direction and confidence this work would not have been possible. His support and encouragement for open-ended thinking in research has helped me learn much during my time doing research. I appreciate his help and feedback during my research and the writing of this thesis. I am grateful to the Department of Electrical and Computer Engineering - for the teaching and guidance they provided over the past two years. I thank them and the NSERC for the financial support. I also gratefully thank members of my thesis committee: Prof. Ravi Adve, Prof. Stark Draper and Prof. Mireille Broucke - for being in my committee and also for taking the time to provide insightful feedback on my work. I enjoyed their questions and comments. My heartfelt thanks go out to my colleagues and friends, who have made my experience at University of Toronto a memorable one. You have inspired, challenged and encouraged me to grow in these two years. My friends in Prof. Hatzinakos group - Gagan, Sahar, Petros, Saeid and Shahrzad, my coffee-buddies JP, Kaustav, Zhe and Masoud, Pratik and Gokul - you will all be enduring flavours of my time here. I thank my friends at Knox Presbyterian Church in Toronto for their love and prayers - through good times and bad. I would like to thank my close friends in lands near and far, especially Sukanya - for the friendship, collaboration and support, and Asher - for always being a loving elder brother. And finally most significantly, I would like to thank my parents and my sister, for their unwavering love, encouragement and confidence in me - I am thankful for you in my life. iv

5 Contents List of Acronyms List of Tables List of Figures viii x xi 1 Introduction Motivation for Biometrics ECG as a Biometric Single Arm ECG Biometrics Taxonomy of Errors Research Goals Contributions Related Publications Outline of Thesis Databases for Single Arm ECG Introduction Single Arm ECG: Related Work Single Lead Single Arm ECG Database Motivation Configuration and Procedure Challenges faced Multiple Lead Single Arm ECG Database Motivation Configuration and Procedure Challenges faced Note on Multiple Session Single Lead ECG v

6 3 Towards an Intuitive Domain for ECG Introduction Previous approaches : Autocorrelation/LDA Dimensionality Reduction Dictionary Learning and Sparse Coding Theory Motivations to Upgrade Towards Dictionary Learning The CoLD-kNN and CoLD-SVM algorithms k-nearest-neighbour based Classifier Support Vector Machine Classifier Application scenarios : Complexity Analysis Large Scale Scenario : Fingertip ECG Introduction Related Work System Model Database Preprocessing and Template Formation Algorithms Used Experimental Results Large-scale performance using full database Scalability : Effect of population size Summary Small Scale Scenario : Single Lead Single Arm ECG Introduction System Model Experiment Process Preprocessing, Segmentation and Outlier Detection Algorithms Used Experimental Results Discussion Multi-Sensor Single Arm ECG Terminology Used System Requirements and Challenges Feature Extraction and Fusion vi

7 6.4 Algorithms Used Experimental Results Discussion Conclusions and Future Work Summary of Thesis Areas for Future Work Bibliography 86 vii

8 List of Acronyms AC ECG EMG EEG NNSC LDA AC/LDA SVM knn Autocorrelation Electrocardiogram Electromyogram Electroencephalogram Non-negative sparse coding Linear Discriminant Analysis Autocorrelation/LDA Support Vector Machines k Nearest Neighbors CoLD-kNN AC-SparseCoded-LDA-kNN CoLD-SVM AC-SparseCoded-LDA-SVM SA-ECG FT-ECG EPIC PQRST QRS FAR FRR Single Arm Single Lead ECG Fingertip ECG Electric Potential Integrated Circuit Complex made of P,Q,R,S, and T fiducial points on the ECG heartbeat Complex made of Q,R, and S fiducial points on the ECG heartbeat False Acceptance Rate False Rejection Rate viii

9 EER MAT PCA ICA RBF Equal Error Rate Microsoft Access Table Principal Component Analysis Independent Component Analysis Radial Basis Function ix

10 List of Tables 2.1 Single Lead Single Arm ECG Database Multiple Lead Single Arm ECG Database Summary of Results: Mean EERs and standard deviation for different N Mean ± deviation of number of outlier windows per subject in collected SA-ECG database EER and corresponding system parameters. Coloured cells indicate lowest EER for each case Summary of Results: Sitting Case for Single-Lead Database Summary of Results: Standing Case for Single-Lead Database Summary of Results: After-exercise Case for Single-Lead Database Summary of Results: Sitting Case for Multi-Lead Database Summary of Results: Standing Case for Multi-Lead Database Summary of Results: Flexing Case for Multi-Lead Database Summary of Results: After-exercise Case for Multi-Lead Database x

11 List of Figures 2.1 Single lead ECG acquisition configuration. A and B are the sensors used Unfiltered raw and filtered processed signal comparisons of two subjects chosen at random An example of acquisition of signals. Note that apart from the sensors on the arm, there are sensors on the forearms and fingers Filtered signals for a single subject from the multiple lead database. Sensors 1-8 represent signals from the arm, whereas sensor 9 is from the fingers. This figure shows signals for the sitting case Filtered signals for a single subject from the multiple lead database. This figure shows signals for the standing case Filtered signals for a single subject from the multiple lead database. This figure shows signals for the flexing case Filtered signals for a single subject from the multiple lead database. This figure shows signals for the after-exercise while sitting case AC and basis for different values of k using the NNSC Using generated bases to represent ACs - an example Heartbeat and basis for k = 1 using the NNSC Heartbeat and basis for k = 2 using the NNSC Block Diagram for CoLD-kNN algorithm Block Diagram for CoLD-SVM algorithm (Left to Right) Electrode placement for ECG acquisition from fingertips, single heartbeat from fingertips Percentage of windows caught as outliers, for each user in the large-scale database Each trace is a template of subject 1(red) or subject 2(black). Mean(bold), minimum and maximum of templates are shown. First two dimensions out of 1 are omitted for clarity xi

12 4.4 ROC curves showing performance of CoLD-SVM and CoLD-kNN compared to AC/LDA for N = Mean EER values and standard deviation (shown using length of whiskers) for various database population sizes Signal Characteristics of SA-ECG signals and performance of system in various cases EER performance analysis plots for different methods and ROC curves for comparison of the methods for sitting case EER performance analysis plots for different methods and ROC curves for comparison of the methods for standing case EER performance analysis plots for different methods and ROC curves for comparison of the methods for after-exercise case Possibilities for sensor placement using the designed acquisition methodology. Each shaded square represents a sensor attached to the armband on the under-side i-th channel in multi-sensor ECG database Processing of i, j-th channels in multi-sensor ECG database Formation of comb ij Diagram for score calculation for comb ij Final selected signal calculation Results for four different subjects for automatic selection of the best combination of channels, compared with the fingertip ECG signals EER performance analysis plots for different methods and ROC curves for comparison of the methods for sitting case EER performance analysis plots for different methods and ROC curves for comparison of the methods for standing case EER performance analysis plots for different methods and ROC curves for comparison of the methods for flexing case EER performance analysis plots for different methods and ROC curves for comparison of the methods for after-exercise case xii

13 Chapter 1 Introduction 1.1 Motivation for Biometrics Validation of a person s identity is an age old problem that has been faced by people over the centuries. This problem though has become one that is more important now in the present day than it was ever before in history because of the rapid globalization of technology and invention of various new access methodologies which make it hard to perceive the flow of control from a person to the task the person wants to do. In such a world, sensitive transactions, service requests, security clearances, etc. become intrinsically linked with the identity of a person and the verification of that identity with the claim made by the person. A simple example of this may be the process of airport check-in where the identity is confirmed by the possession of a passport and also by a human security officer who makes sure that the person in the passport represented by a picture is really the person who possesses it. There are many such ways to verify the identity of a person. Many common ones that come to mind are passwords, fingerprints, iris scans, pre-chosen questions, CAPTCHAs, PIN numbers, credit cards, and so on. All these methods have been invented and widely used because of their feasibility and low chances of being circumvented or manipulated upon. However, the fact that they are either materials or information stored in memory of the person or on a physical memory device makes them highly susceptible to being forged, faked or simply being stolen. Recent news of discoveries of government sponsored spying on the general population is hence unremarkable as efficient methods exist to break down such authentication methods based access barriers. The presence of an individual s identity almost in entirety on the internet or other networks in today s society necessitates new ways of verifying the person s identity - for better ways to control the flow of information concerning the identity of the person and 1

14 Chapter 1. Introduction 2 his/her activities. For this, it is logical from the above discussion to conclude that the new way should be independent of a piece of information or a material object like an identity card with RFID on it. This is where the concept of biometrics comes in. Recognition of individuals using biometric signatures has been an area of interest in the past decade among the research community. Use of biometrics for authentication has gained popularity in recent years with the use of fingerprints, iris and face recognition systems in various practical applications. This approach to recognition is closer to the actual person than indirect means such as a password, which is memorized by the user who wishes access to a system. Another advantage of using certain biological signals for biometrics is that they are almost universally present. Though the advantage for applications in security is that it uses the biological characteristics of the very person being recognized, simple but effective falsification efforts have also emerged, e.g.replay attacks, biometric obfuscation and circumvention. One solution to such attacks is the use of the body s internal physiological signals that are difficult to mimic or alter, i.e., the notion of medical biometrics, one modality of which is elucidated in this thesis. Also, privacy concerns are important in such systems because once a biological identity is stolen, it is usually hard to change for a user. A criteria proposed by Jain et al. [21] is that of the admissibility of a biometric modality if it satisfies the following criteria. 1. Universality : It can be found in almost all living humans 2. Uniqueness : Authentication capabilities for groups of individuals have been proven 3. Measurability : Ease of acquisition using suitable devices 4. Performance : Has been shown to perform accurately for subsets of population 5. Acceptability : Ease of use compatible with normal functioning of the user 6. Circumvention : Difficulty of forging the modality used. 7. Permanence : Temporal invariance of the modality. 1.2 ECG as a Biometric With these perspectives, the electrocardiogram (ECG) signal has been proposed as a modality for biometrics [2, 4]. An ECG is a trace of the electric activity of the heart

15 Chapter 1. Introduction 3 obtained through a configuration of electrodes placed on the body at specific locations. It is a quasi-periodic signal with pulses corresponding to cycles of the body s cardiac functions. Studies by Biel et al. [4] and Israel et al. [2] first showed that the ECG signal has enough specificity and sensitivity in it to be useful for biometrics. More of the related work in the area of ECG biometrics, more specific to the work in this thesis is discussed in the subsequent chapters. However, it should be noted that the area of feature representation of the ECG signals is still in infancy and the literature in ECG biometrics currently contains works that use rather simple domains for feature representation. This is explained in more detail in Chapter 3. The work in this thesis is consequently focussed on exploration of appropriate signal representation for biometrics. Furthermore, the ECG signal satisfies the criteria of universality, measurability, performance, acceptability, permanence and circumvention. It satisfies the criteria of universality as all living human beings have a beating heart and hence an ECG signal. It satisfies measurability as all one needs to collect ECG signals is a sensor electrode which non-invasively records the surface voltage potential. Two such electrodes are enough to acquire good ECG signals which may be used for biometrics. It satisfies the performance criteria as shown by the above works and as will be shown in the next chapter. It is acceptable for use, specially when the device and acquisition design for a proposed system is non-invasive and doesn t interfere with the normal functioning of a person who uses the system. This criteria is in fact one of the motivations for this thesis, as this work builds on previous work in this area of ECG biometrics that used ECG signals from two fingers from different hands, thereby using both sides of the body for the acquisition process. The present work aims to improve on such a system, as discussed more in the following sections. The permanence criteria for ECG biometrics is still being researched and various methods have been proposed to extract the more permanent features from the ECG signals. The ECG signal is highly susceptible to the physiological and psychological changes of the user. However, there have been works recently which have reported good performances using long term data for ECG signals. Finally, circumvention is not possible with ECG signals with appropriately secured devices which make up the authentication system because of the fact that ECG is produced by the heart and the nervous system which are impractical to be forged for a person. The physical make up of a person is what gives him or her the unique characteristics, one of which is the ECG, and hence this is hard to fake. ECG also offers possibilities for continuous authentication(if it can be obtained continuously) and liveness detection(to

16 Chapter 1. Introduction 4 verify if the user is normal and alive) making forgery or mimicking even more difficult compared to traditional modalities. In the following subsection, we briefly discuss the new ECG signals that are used in this thesis Single Arm ECG Biometrics A recent trend in the use of electronic devices and all forms of computational device that people use is the arrival of wearable computing and hence wearable electronics in the market. Google Glass has probably been one of the more popular of the devices that aims to bring the internet and all the information interesting to the user right before his/her eyes, literally. The advantage of wearable computing is said to be the ultra-personalization of the services that technology provides, to the user involved. This is part of the larger trend in computer science and engineering towards ubiquitous computing where computing is made to appear everywhere and anywhere. Here, computing is present in any device, any location and in any format. Such pervasiveness and ease of interaction with the devices also leads to the need for devices and designs for device-user interaction that facilitate similarly easy methods of authentication. It is here that in context of the above discussion of ECG as a biometric modality, acceptability becomes a deal-breaker criteria. This is because in a market of devices where the ease of use is almost universal, ECG biometrics must have ease of use too, otherwise it will not be accepted in the user base where users are interested in interfacing with their devices through an authentication mechanism. Placement locations of acquisition electrodes on the body is key to obtaining a usable ECG signal. This configuration for the electrodes is based on both experimental observation and knowledge of biological facts like the axis of the heart and location of the atrioventricular and sinoatrial nodes. Moreover, the number of electrodes/locations can also be varied. Examples include the traditional 12-lead ECG and the 3-lead Frank XYZ ECG. In this context, new ways of acquiring ECG become more interesting and pertinent to the question of designing user friendly, almost invisible methods which may be more acceptable to users. One of the main drawbacks in the present ECG biometrics methods in literature is the need for the signal to be acquired using both sides of the body. This is not user friendly at all and the best current method which is least intrusive and easy to use is the finger ECG signals, collected and studied at University of Toronto. Even

17 Chapter 1. Introduction 5 these signals are not practical for day-to-day use in a wearable computing scenario - for the main reason that the signals have to be collected using both sides of the body. Hence, a method to collect the signals using a single side of the body, in a way which is possibly easy to use along with other activities is highly preferred. In this thesis, a simple but effective and user friendly way of acquiring ECG from a single side of the body is proposed. The signal is collected (explained in much detail in Chapter 2) from the upper left arm, for the reason that it is closer to the heart. Also, experimentally it has been found recently that the signals acquired at that location contained representation of the QRS complex of the heartbeats. In this work, these signals are acquired using different methodologies and analysed for their applicability for biometrics use. 1.3 Taxonomy of Errors Performance analysis of any biometrics algorithm is highly dependent on the error criteria that is used. In this section we define a few basic error types and describe the metrics that will be used for performance evaluation throughout this thesis. In this thesis, only the verification scenario has been assumed and analyzed for the different methods and signals. Hence, we define the different errors present in such a scenario. They are: 1. Authentication leads to a denial of access to a legitimate user, measured in false rejection rates (FRR). 2. Authentication leads to a acceptance of access to an intruder i.e., an illegitimate user, measured in false acceptance rates (FAR). These error rates are computed as ratios of the corresponding error s set divided by the complete trial set. i.e., FAR = FRR = Number of falsely authenticated intruders Total number of intruders Number of rejected legitimate users Total number of users (1.1) (1.2) Furthermore, the equal error rate (EER) is defined as the error value when the false acceptance is equal to false rejection i.e., EER = FAR = FRR. Lower equal error rate translates as better authentication performance of the system that is being evaluated.

18 Chapter 1. Introduction Research Goals Following the above discussion, the the goals for the work presented in this thesis are as follows. Firstly, the methods used for the feature extraction stage in ECG biometrics systems have not focussed on the idea of generation and use of a basis set custom designed for the ECG signals themselves. Work has been done on the same lines (for example, use of PCA as in eigenpulses). However, a parts based intuitive basis generating method has not been investigated for ECG signals in the context of biometrics. In this thesis, one of the main goals is to investigate use of such a basis generation method for ECG feature extraction. In addition, the designed system has to be application and resource specific. Secondly, most of the present work in ECG biometrics literature uses signals from publicly available databases that contain medical diagnosis quality signals obtained in hospitals [15]. Recent efforts by researchers [11],[51] have focussed on using differently acquired signals for ECG biometrics. However, a large-scale study of feasibility of the signals thus obtained has only been recently done [39] using the AC/LDA algorithm. Hence, this thesis also aims to evaluate the large scale fingertips ECG database available at University of Toronto using the designed system. Thirdly, in context of the previous section s discussion, there is a need to collect signals from a single arm and then analyse the signals thus acquired for their applicability in ECG biometrics. This is another goal for the work in this thesis - for the signals thus obtained to be analyzed using the proposed system design and also previous work from the present literature. The next section lists the major contributions which came about as part of this work. 1.5 Contributions Major contributions from this thesis are summarized as follows: 1. Proposed and designed the twin CoLD-kNN and CoLD-SVM algorithms as an improvement on existing techniques. These algorithms involve the use of dictionary learning in the form of non-negative sparse coding, for a parts based intuitive basis for ECG signals. 2. Complexity dependent choice between SVM and knn classifiers using dependent classification was introduced.

19 Chapter 1. Introduction 7 3. Collected the first database for single arm ECG signals. This was done in two stages. The first stage involved collection of a preliminary database using single lead configuration. A second database which contained a larger population size and used multiple lead ECG acquisition configuration was collected in the second stage. 4. Implemented and compared the above proposed methods with one of the current best methods for ECG biometrics available in literature, i.e., the AC/LDA algorithm. For the large-scale finger ECG database containing 112 subjects, lowest known equal error rate (EER) for large-scale application of finger ECG biometrics was obtained, i.e., 2.59%, using one of the the proposed methods. 5. The effect of increasing population size on performance of the different algorithms was studied using the large scale finger ECG database. 6. For the single lead single arm ECG database and the multiple lead single arm ECG database, promising results for verification performance were reported. CoLD-SVM outperformed existing methods in almost all scenarios that were tested. For the single lead signals, an EER of 3.26% for sitting posture, 4.34% for standing posture, and 5.55% for the after-exercise case were obtained. For the multiple lead signals, EERs of 6.97% for sitting posture, 8.32% for the standing posture, 24.56% for the muscle-flexing case, and 7.63% for the after-exercise case were obtained. This work is the first to use signals from only one side of the body, i.e. the left arm, for ECG biometrics. 1.6 Related Publications Part of the work presented in this thesis has also been accepted for publications in the form of the following accepted papers. [4] Peter Sam Raj and Dimitrios Hatzinakos. Feasibility of Single-Arm Single-Lead ECG biometrics. In EUSIPCO 214 (22nd European Signal Processing Conference 214) (EUSIPCO 214), Lisbon, Portugal, September 214, accepted for publication. [41] Peter Sam Raj, Sukanya Sonowal, and Dimitrios Hatzinakos. Non-Negative Sparse Coding based Scalable Access Control using Fingertip ECG. In IJCB 214 (IEEE

20 Chapter 1. Introduction 8 /IAPR International Joint Conference on Biometrics) (IJCB 214), Clearwater (Tampa), Florida, USA, September 214, accepted for publication. 1.7 Outline of Thesis This thesis is organized as follows: In Chapter 2, we discuss in length the databases collected - their collection methodology, the signals characteristics and other details regarding the procedure of collection, devices used, etc. This is intended to help any future work in the area of database expansion of the collected signals. In Chapter 3, we propose new algorithms using the dictionary learning method, the NNSC, and discuss the various parts of the system using the proposed algorithms for the authentication system. In Chapter 4 and 5, we deal with the fingertips ECG and the single lead single arm ECG signals and evaluate their feasibility for ECG biometrics using the proposed methods. Finally, in Chapter 6, we propose methods to use the multi-lead ECG signals collected using multiple sensors and evaluate the signals based on the authentication performance using the proposed methods. In Chapter 7, we conclude this thesis and give directions for future work.

21 Chapter 2 Databases for Single Arm ECG 2.1 Introduction This chapter presents the single arm ECG databases collected for analysis of single arm ECG signals for ECG biometrics. Also, detailed descriptions of the protocols that were designed for the individual experiments are presented here. The database has been anonymized for privacy and has been made available for use in the Biometric Security Laboratory, from where it can be requested for use in research. The need for a new database, building upon the already present large-scale database with the Biometric Security Lab, is because of the fact that the signals from the arm are very distinct from those collected at the finger tips. The large-scale database contains ECG from the finger tips from over a thousand subjects and is arguably the largest ECG database collected using a user-friendly acquisition methodology. However, in our preliminary experiments with observing the similarities between the ECG signals from the arm and those from the fingers, we observed significant differences and reduced quality of the signals from the arm. Hence, for evaluation of any proposed algorithms, a database containing characteristic signals containing ECG from the arm is necessary. Such comparisons have been done previously, as discussed in the next section. However, there is currently no publicly available database that contains ECG signals from a reasonable number of users collected from the upper left arm. 2.2 Single Arm ECG: Related Work We call the single lead single arm ECG signals as SA-ECG and the fingertips/finger ECG signals as FT-ECG. The existing methodology in all ECG biometrics literature has as yet 9

22 Chapter 2. Databases for Single Arm ECG 1 required sensors to be placed on either side of the body (e.g. fingers from both hands). This requirement becomes a major problem in user friendly applications as both sides of the body have to be in contact with the sensors. It is highly preferable instead to obtain ECG from only a single side of the body. This would pave the way for comfortable and user-friendly biometrics, applicable in devices such as a smartwatch. Placement criteria for the electrodes is key to obtaining a usable ECG signal and is based on both empirical observations and biological facts such as the axis of the heart and location of nodes. Recently, 1-lead ECG has been used in [51, 28, 7, 11], obtaining ECG from fingertips whereas in [48], both 1-lead and 2-lead signals obtained from Holter monitoring are used. Very limited work exists in literature that reports or uses the single arm ECG signals. Hannula et al. [17] showed that it was possible to get ECG signal from only a single arm. Their work involved solutions to amplifier and analysis software which was built to get ECG signals. One test subject was used for the experiments and methods of regular ECG measurement was compared to the proposed single-arm one electrode system. While they proposed this system and showed that heart-rate values measured by this new system correlated with actual heart-rate values, the biometric aspect of it wasn t studied. Later, Yang et al. [47] furthered the work by studying ECG acquisition using only one arm with flexible electrodes. They confirmed the fact that ECG can be obtained from a single arm and also showed that it is more beneficial to use electrodes on the upper arm of the subject than lower arm. Additionally, the subject was assumed to be at rest to reduce EMG (i.e. Muscle Motion Artifacts) interference. It was also noted in this study that ECG produced at this arm-position was very weak and required various signal processing techniques to denoise. Plessey Semiconductors in [38] have also shown a method of ECG acquisition from single-arm using their EPIC sensors. They reported important observations which agreed with [47] that the sensors used should be positioned far up the arm on the underside and that the left arm is preferable than the right. They showed that positioning of both sensors on the underside of arm is better for applications where motion of the arm may be expected. This was determined experimentally and a system was proposed. They showed that the ECG of a subject who had just taken a brisk walk could be acquired this way and could be detected after simple signal processing. Again, the magnitude of acquired signal was very small. In these works, the signals were not studied for use in biometrics, which is the motivation for work on single-arm signals in this thesis. Additionally, SA-ECG is extremely convenient to acquire with access only needed to a single location on the body. As already discussed in Chapter 1, this is an important advantage in commercial biometric

23 Chapter 2. Databases for Single Arm ECG 11 applications where comfort of use is key to success of new technology. Hence, the evidence from previous works for the presence of ECG signals at the upper arm location, combined with the lack of availability of such signals in a database motivated the experiments to record such signals. For design of an ECG acquisition system from the arm, a bracelet-like circular strip of multiple sensors around the upper left arm is proposed. As we know, there are two major muscles in the upper arm region: the triceps and biceps. The hypothesis is that when the arm is used, only some sensors will be affected by EMG interference from these muscles, whereas the other sensors will still record traces of the ECG signal, which could be used for ECG biometrics. During any database collection of signals to be used in biometrics, it is desirable to gather signals from as many people as possible. This is because a larger database helps in a reliable evaluation of biometric algorithms which would be applied in a large-scale application scenario. Also, a larger database is useful in testing for uniqueness of the biometric modality used, thereby providing a more stringent evaluation environment than when a smaller database is used. For the finger tips ECG signals, a large database has been collected by the Biometric Security Lab with 112 individuals. However, collection of a large database for single arm ECG signals is an ongoing work. For the presented work, a modestly sized database has been collected as described in the following sections. There exist the following desirable features of the database of signals collected from the arm for the present work. Not all of these were strictly followed in the experiments, and have been mentioned accordingly. However, these were maintained as much was possible during the course of the experimentation process. Acquisition Location: The location of signal collection, i.e., the sensors placement location, should be the same for all subjects on the arm. Signal Normalcy: The signals should be collected from subjects without heart abnormalities and other anomalies. Unconstrained Environment: The signals should be collected in normal office environment, where the intended application of the system is imagined to be. Hence, signals should be collected even though simultaneous noise due to vibrations caused by people walking around, using the keyboard, and powerline interference at 6Hz is present. Unconstrained Subject: The subject during the acquisition should not be unreasonably constrained. While it is required for the subject to be still, acquisition should not be too specific on posture, breathing pattern, etc.

24 Chapter 2. Databases for Single Arm ECG 12 Multiple Session: For biometrics applications, it is generally preferred that signals recordings from multiple sessions are available for evaluation purposes for permanence testing. In the following sections, the two different experimental sessions are described and the resulting databases are introduced. For each database, the experimentation process, device and parameters used, and database organization are given. Some examples of the configuration of sensors in each are shown through pictures taken during the experimentation process. Also, challenges faced during the experiments and that could be faced in subsequent efforts in similar data collection are described for each database. Finally, each database discussion is summarized with all the indispensable information. 2.3 Single Lead Single Arm ECG Database Motivation The motivation for this database collection session was to collect a preliminary database using only a single lead configuration. This was intended for feasibility analysis for single arm ECG signals biometric potential. These experiments took place in the Biometrics Security Laboratory at the University of Toronto, under the ethics protocol # participants participated in this experiment voluntarily and their consent was obtained using an approved consent form. These were all graduate students, male and female, with an age range of 18-3 years Configuration and Procedure The acquisition device used was a Vernier ECG sensor (EKG-BTA), along with the Go!Link software and interface with the device. This device had three electrode leads: positive, negative and ground. The device was interfaced with the software on a computer, where it recorded the potential difference between the positive and negative electrode leads, with respect to the ground. Thus, as the signal recorded by the device was a potential difference between two leads, it was a single lead configuration for acquisition of ECG. The electrode configuration used is shown in Figure 2.1. One of A or B consisted of the positive electrode connected to the sensor, whereas the other sensor had both the positive electrode and ground attached to it. The sensor electrodes used were Kendall AgCl gel electrodes. The sampling rate was set to be the highest the device permitted, i.e., 2 Hz.

25 Chapter 2. Databases for Single Arm ECG 13 Figure 2.1: Single lead ECG acquisition configuration. A and B are the sensors used. For each subject, the preparation started with sitting at rest for 5-1 minutes to relax and bringing the body into a state typical of a sitting while resting condition. During this time, the subjects were briefed about the procedure and schedule of the experiment, and were given the consent form to be read and signed. The crucial part of this experiment of collecting data was to find the best location to put the sensor electrodes on the upper left arm. It was observed that the signal quality considerably changed on relocating the electrodes, with some locations giving signals which showed the R-peak on the ECG pulse and other locations showing just noise. An elastic flexible armband was designed and a prototype was created which could be used to mount the electrodes easily. It was designed to fit most human users, supporting arms with a minimum circumference of 9 inches being taut and a maximum of 15 inches stretching to its maximum. The subject was made to wear this armband on his/her upper left arm as far up as was possible and the sensor electrodes were positioned on the armband, while observing the signal on the computer display. The best combination of electrodes was chosen to be the one which gave the best defined ECG pulses in the acquired signal. The found location was marked on the armband. Following this, the experiment was conducted in three stages, corresponding to the three different postures/states of the body that were chosen. These three cases, as we refer to them, were chosen to represent most postures/states possible for human beings in an workspace or home environment. Following were the cases: 1. Sitting (Sitting posture, subject at rest): The recording was done for 12 seconds duration from the arm using the found location of electrodes. During this time, the subject was reasonably still though normal breathing with rising and lowering

26 Chapter 2. Databases for Single Arm ECG 14 of chest, and conversation with movement of head was allowed. 2. Standing (Standing posture, subject at rest): The recording was done for 12 seconds duration from the arm, the subject being reasonably still though normal breathing with rising and lowering of chest, and conversation with movement of head was allowed. Also, the hands were allowed to touch the body while standing, allowing for the most comfortable standing posture preferred by the subject concerned. 3. After-exercise (Sitting posture, after 2 seconds of exercise): After the above, the electrodes were removed. The subject was then made to perform a cardio exercise of choice for a total of 2 seconds with the aim of increasing the heart rate and bringing the body into stress. After this exercise, a few seconds were given while the subject was made to sit down and the electrodes were reattached at the same locations. Then, the recording was done for 12 seconds duration from the arm using the found location of electrodes. During this time also, the subject was reasonably still though normal breathing with rising and lowering of chest, and conversation with movement of head was allowed. The total time duration for each subject s recordings was 6 minutes, however, the whole experiment took approximately 15 minutes to complete, combining the time for preparation and exercise. Of the 23 subjects that participated, 2 were female and the rest were male. Also, the experiments were done at Bahen Centre building at the University of Toronto, with an average office environment, i.e., allowing for people to walk in and out of the room, people walking around and powerline noise. The database was organized in a.mat file containing no names, hence the storage was anonymized. The MAT file contains the time stamps with values of the potential difference at each time stamp for the three cases recordings. Some examples of the signals obtained are shown in Figure 2.2, representing a 5 second slice of the recordings. The left side figures show the unprocessed raw signals obtained from two subjects chosen at random. It can be seen that the signals are distinct for these two subjects and differ in the heart rate as well as the pulse shapes themselves. It can also be seen that the ECG pulses differ considerably for a single subject, when compared between the different cases, i.e., the sitting and standing pulses are quite different, whereas after a short period of exercise, the heart rate changes too.

27 Chapter 2. Databases for Single Arm ECG Sitting Case : Raw Signal.8 Sitting Case : Filtered Signal Subject Subject Subject 2 1 Subject Time (s) Sitting, raw Time (s) Sitting, processed 1.2 Standing Case : Raw Signal.4 Standing Case : Filtered Signal Subject 1 1 Subject Subject Time (s) Standing, raw Subject Time (s) Standing, processed 1.3 After exercise Case : Raw Signal.15 After exercise Case : Filtered Signal Subject Subject Subject Time (s) After-exercise, raw Subject Time (s) After-exercise, processed Figure 2.2: Unfiltered raw and filtered processed signal comparisons of two subjects chosen at random.

28 Chapter 2. Databases for Single Arm ECG Challenges faced The following were identified as the challenges associated with the collection of single lead signals from the arm of a subject. These should be considered while any attempt to create another such database for research purposes is undertaken. Motion Artifacts: It was observed that even minute vibrations, such as those caused by people walking in the same room as the subject while recording, added noise to the recording. This was interesting to observe and the likely cause attributed is the vibration being translated to the sensors on the arm, which are highly sensitive to them. Electrode Location: As already discussed in the previous subsection, electrode location is very important in obtaining a signal which contains the ECG pulses clearly without much interference. As this is dependent upon the specific subject involved, the location of electrodes must be found empirically for every subject and kept the same throughout the experiment session. Skin Conductivity and Muscle Noise: Skin conductivity and the presence of muscle noise also changes the signal to noise ratio in the acquired signal. Though we have not defined a rigorous SNR metric for the ECG containing signals, empirically, it is seen that a dry skin leads to weaker ECG pulse in the signal. Also, muscle activity causes non-stationary interference which is non-trivial to remove with processing. Caffeine and Psychological State: Another factor that leads to change in the signals is caffeine consumption and a difference in psychological states. It was observed in the experiments that caffeine changed the heart rate and the ECG pulse shape, and thus is preferred to be avoided during any data collection. Also, severe psychological stress can cause the same effects. Before summarizing the database information, it should be noted that this database satisfies all the desirability features mentioned in the introduction to this chapter, except the multiple session one. This is of course a weakness of using this database for evaluation of long-term biometric systems. Also, the problem of finding out location for the electrodes is a major obstacle in the use of this database directly for practical systems. This is because this database consists of signals which are taken from the locations on the arm which are best for collection of the signals, and as such, the location has to be determined empirically. However, in a practical scenario, the user will not have access to a device to find out such a location automatically, and that will complicate the use of

29 Chapter 2. Databases for Single Arm ECG 17 Database Single lead from upper left arm Collected on October 213 Number of subjects 23 Acquisition device Vernier ECG sensor (EKG-BTA) Anonymized Yes Sampling Frequency 2 Hz Gain None, offset = 1mV. Number of sessions 1 Number of cases 3 Cases Sitting, Standing, After-exercise sitting Duration 12 seconds per case Table 2.1: Single Lead Single Arm ECG Database the biometric system. Once such a location is determined however, the database can be used for reliable biometrics assessments for a user. Table 2.1 summarizes the indispensable details of the single lead single arm ECG database. 2.4 Multiple Lead Single Arm ECG Database Motivation Following collection of the single lead single arm database and analysis of the signals based on their biometric quality and feasibility (as will be discussed later in Chapter 5, it was natural to proceed with collection of a larger, though modestly sized, database which contained multiple sensor signals from the arm. The idea was to eliminate the main shortcoming of the single lead single arm database, i.e., the requirement of empirically finding out the locations of the electrodes for each user and at the start of each use. That limited the use of such a system and as discussed above, was a roadblock in the usage of single arm signals. The main motivation of using multiple leads, i.e., multiple sensor electrodes, was to treat the whole upper left arm as valid locations for signal acquisition, positioning multiple sensor electrodes around the arm s circumference and then processing the obtained signals to infer information about the ECG. The processing of the obtained signals is discussed in Chapter 6. While designing the experiment, the following aspects were considered: Location Bias: The sensor electrodes were preferred to be placed on the arm randomly

30 Chapter 2. Databases for Single Arm ECG 18 to eliminate the location specificity of the signal from a sensor. This would allow for positioning the sensor electrodes in any way as long as they are on the arm of the user, thereby improving the ease of the system s use. Number of Sensor Electrodes: This is important as too few sensor electrodes will likely miss the locations that are good for a user, whereas too many sensor electrodes may make the system costlier, complicated and also lead to redundant information from adjacent sensors. The number of sensor electrodes was chosen to be 8 considering these and for practical reasons. Synchronization: One of the main reasons for using multiple sensor electrodes was to use the information from the different signals to derive information about the ECG pulses. This is possible only if we have time synchronization between them. Thus, the acquisition device chosen was one where all the multiple sensor electrodes were sensed and recorded simultaneously. Reference: An interesting aspect that was identified in these experiments was to see how the arm signals correlated with signals from the finger, i.e., the signals contained in the large scale finger ECG database that the Biometric Security Lab previously collected. Thus, two more sensor electrodes were used to obtain thumb signals with the 8 signals from the arm. Also, a reference electrode for all the 1 (8 arm + 2 fingers) electrodes was attached to the skin on the forearm (where it was empirically verified that there was no ECG containing signal present) which acted as the ground for all the other electrodes. Keeping in mind all the above consideration, the experiment procedure was designed and is described in detail in the following subsection. This experiment was also done under the same protocol # 2318 at the Biometrics Security Laboratory at University of Toronto and attracted 43 participants who signed a consent form. The age range was years Configuration and Procedure The device chosen for this experiment was the Nicolet EEG Wireless Device. This was chosen because of the availability of the multiple sensor electrode simultaneous recording option available with it. The device is designed and originally intended for the use of EEG signals recording, with the various ports for electrodes meant to be collecting data from a specific location on the head of a patient. However, the capabilities of the device

31 Chapter 2. Databases for Single Arm ECG 19 for precise signal measurement (high resolution) and time synchronization between the various electrodes are also perfect for any multi sensor signal acquisition where time synchronization is important. Thus, the device configuration was modified and the inbuilt filters calibrated accordingly to allow for certain ports to be used in ECG signal acquisition. The sampling rate was set to be 512 Hz. The preparation for each subject before the recordings involved the same procedure as described in Section Following this, the same armband used in Section was used, though now containing the attached 8 sensor electrodes. The sensor electrodes used were those supplied with the Nicolet device which were used for EEG recording, as it was found that these worked well for ECG recordings as well. The location of the electrodes was random, except that they were evenly placed along the circumference of the arm for each subject. The experiment was conducted in four stages, corresponding to the four different postures/states of the body that were chosen. Apart from the three cases chosen in Section 2.3.2, a fourth case was chosen which involved movement of muscle. Briefly, they were: 1. Sitting (Sitting posture, subject at rest): The recording was done for approximately 18 seconds duration with conditions as described earlier. 2. Standing (Standing posture, subject at rest): The recording was done for approximately 18 seconds duration with conditions as described for standing posture earlier. 3. Flexing (Sitting posture, subject at rest): The subject was made to flex the arm to contract and release the biceps, for a period of 12 seconds approximately. This was incorporated into the experiment to account for a signal which contains the noise from muscle movement, to study the effect of movement of the arm on the signal obtained. 4. After-exercise (Sitting posture, after 2 seconds of exercise): After the above, the electrodes kept attached, the subject was made to perform a cardio exercise of choice for a total of 2 seconds followed by a few seconds of resting while sitting. Then, the recording was done for 18 seconds approximately from the arm. The total time duration for each subject s recording was approximately 11 minutes, however, the whole session lasted 2 minutes with the preparation, breaks and exercise. Of the 43 subjects who participated, 13 were female. The collection environment was the same as described in the previous subsection, i.e., that of an average office environment.

32 Chapter 2. Databases for Single Arm ECG 2 Figure 2.3: An example of acquisition of signals. Note that apart from the sensors on the arm, there are sensors on the forearms and fingers. Also, it should be noted that of the 43 participants, 16 also participated earlier in the single lead single arm database collection. The database was stored as a.mat file, anonymizing the identity of the subjects who participated as discussed in the earlier section. Figure 2.3 shows a picture of one acquisition procedure. Figure 2.4 to 2.7 show the different sensor recordings for a single subject for the four different acquisition cases. The signals shown are a 5 second slice of the whole recording. Notice that some sensor signals completely lack information about the ECG signal (more visible in the sensor 9 plot in each case, which comes from the fingers), whereas some other signals seem to be related to it. However, this is expected, as the sensor plots shown here are potential differences between the sensor and the reference. Combination of any two sensor recordings yields signal combinations which contain more pronounced ECG pulses. This is described more in detail in Chapter Challenges faced The challenges faced in the collection of this multiple lead signals from the arm of a subject are identical to the ones faced with the single lead database. However, the challenge related to the electrode location determination is eliminated. This is because the whole point of collecting this database was to automate the process of finding the electrode

33 Chapter 2. Databases for Single Arm ECG 21 Sensor1 Sensor2 Sensor3 Sensor4 Sensor5 Sensor6 Sensor7 Sensor8 Sensor9 Sitting Case : Filtered Signal Time (s) Figure 2.4: Filtered signals for a single subject from the multiple lead database. Sensors 1-8 represent signals from the arm, whereas sensor 9 is from the fingers. This figure shows signals for the sitting case

34 Chapter 2. Databases for Single Arm ECG 22 Sensor1 Sensor2 Sensor3 Sensor4 Sensor5 Sensor6 Sensor7 Sensor8 Sensor9 Standing Case : Filtered Signal Time (s) Figure 2.5: Filtered signals for a single subject from the multiple lead database. This figure shows signals for the standing case

35 Chapter 2. Databases for Single Arm ECG 23 Sensor1 Sensor2 Sensor3 Sensor4 Sensor5 Sensor6 Sensor7 Sensor8 Sensor9 Flexing Case : Filtered Signal Time (s) Figure 2.6: Filtered signals for a single subject from the multiple lead database. This figure shows signals for the flexing case

36 Chapter 2. Databases for Single Arm ECG 24 Sensor1 Sensor2 Sensor3 Sensor4 Sensor5 Sensor6 Sensor7 Sensor8 Sensor9 After exercise Case : Filtered Signal Time (s) Figure 2.7: Filtered signals for a single subject from the multiple lead database. This figure shows signals for the after-exercise while sitting case

37 Chapter 2. Databases for Single Arm ECG 25 Database Multiple lead from upper left arm Collected on November-December 213 Number of subjects 43 Acquisition device Nicolet EEG Wireless Device Anonymized Yes Sampling Frequency 512 Hz Gain None, values in terms of µv Number of sessions 1 Number of cases 4 Cases Sitting, Standing, Flexing, After-exercise sitting Duration 18 seconds for Sitting, Standing, Afterexercise 12 seconds for Flexing Table 2.2: Multiple Lead Single Arm ECG Database locations that provide signals which contain the ECG signals. Also, this database similar to the earlier database has the weakness of not having multiple session signals. Table 2.2 summarizes the indispensable details of the multiple lead single arm ECG database. 2.5 Note on Multiple Session Single Lead ECG The two databases introduced can be merged in certain ways to form a larger single lead database and also a small multiple session single lead database. Firstly, as we noted in the discussion on multiple lead database experiment, 16 subjects were common participants among the two experiments. Hence, though we have different electrodes, device and acquisition methodology for these two databases, multi-session signals are available for 16 subjects. Hence the database signals can be resampled and combined for these 16 subjects to form a small sized multiple session database for long term variability analysis of the single arm signals. However, these are single lead signals, which puts a constraint on their use in algorithms for biometrics. The time period between the experiments was 3-45 days. Moreover, secondly, if the whole of the multiple sensor database was transformed by resampling, a total of 49 unique subjects for an improved, larger single lead database is formed for biometrics analysis, rather than the 23 subject database presented above. In this work, we do not make these mergers, and treat the two databases quite separately.

38 Chapter 3 Towards an Intuitive Domain for ECG 3.1 Introduction In this chapter, a new system for ECG biometrics is proposed that builds on previous approaches used in ECG biometrics literature. A feature representation for the ECG which has not been previously used in ECG biometrics literature is introduced here. For an ECG biometric system, often-times the first crucial step consists of choosing a representation for the signals. This is usually the choice of a domain using methods such as fourier transforms, wavelets etc. Many approaches have been adopted for use of domains for ECG signals. One such representation was proposed by Irvine et al. [19], where Principal Component Analysis was used to create what they called eigenpulses, representative of the basis for ECG signals. Use of the short-time frequency analysis[33], wavelets[7, 8], principal component analysis (PCA) [19] and the autocorrelation coefficients (ACs) [1] have yielded good results. Similar ideas for intuitive domains or basis for signals have been explored in other areas of biometrics research such as face recognition. Works by Shastri et al. [42], Casalino et al. [6], and the method using K-SVD by Zhang et al. [5] demonstrate the feasibility of the idea. However, an intuitive basis for ECG signals has not been explored yet, to the best of our knowledge. Dictionary learning and Sparse Coding is an approach which gives a parts-based representation that has not been applied to ECG in literature. It has been, however, known to the signal processing community for some time, and has been used (such as in Mairal et al. [31] and as described in the article by Tosic and Fossard [45]) with success in natural signals such as face images and speech. It leads to interesting intuitive basis 26

39 Chapter 3. Towards an Intuitive Domain for ECG 27 which can be further investigated for ECG signals, as in what the parts mean, and if they can be used for privacy-preservation. Also, a customized basis can be used to encode ECG templates making fake queries infeasible without access to this basis. This is impossible with fixed domains like AC, wavelets, etc. Here, it is proposed that such a basis using Dictionary Learning and Sparse Coding of the ACs be used for the feature template representation, thereby combining ACs non-fiducial aspect with the intuitive decomposition aspects of Dictionary Learning. As will be described later in this chapter, the version of dictionary learning and sparse coding for ECG signals that performs the best in the biometric aspect is the non-negative sparse coding. After coding the ECG signals into a representation that is intuitive, linear discriminant analysis is used, whereafter the classifiers that can be used in this proposed system have been discussed. The use of classifiers is also abundant in literature for ECG biometrics, but in this chapter, a complexity specific choice is provided between two classifier choices. This is crucial in case of large scale databases and application scenarios where resources are constrained. In the next section, we start with a brief discussion of a method available in literature which has been shown to be better performing for ECG biometrics than the rest of the algorithms, as shown by Pouryayevali et al. [39]. This method, i.e., Autocorrelation/LDA (AC/LDA) is used as the baseline for comparison with our proposed method for various ECG biometrics analysis that is done throughout this work. Thereafter, theory of Dictionary Learning and Sparse Coding is discussed followed by a discussion of the classifiers and complexity analysis of the proposed methods. 3.2 Previous approaches : Autocorrelation/LDA The Autocorrelation Linear Discriminant Analysis method is a non-fiducial method successfully used in ECG biometrics that uses the autocorrelation of the ECG signals as a feature vector for classification (described in Agrafioti et al. [1]). It does so by projecting the AC feature vectors to a new space with lower dimensionality, as shown in Li et al., [27]. Firstly, the ECG signal is pre-processed to remove noise by using a band pass filter. The noise removed is baseline wander, powerline interference and other artifacts due to movement caused by the body motion, electrode motion relative to the location of attachment of electrodes and other factors such as EMG interference by muscle motion. The filter order and pass band are determined empirically. Thereafter, the signal is divided into fixed length windows, by using overlapping segments of the ECG signal. These segments are long enough (chosen empirically to yield better recognition accuracy)

40 Chapter 3. Towards an Intuitive Domain for ECG 28 to contain a few ECG pulses. The windows thus obtained are passed to the AC/LDA algorithm which comprises of the following steps: 1. Normalized autocorrelation: Each window is processed to calculate the normalized autocorrelation. The normalized autocorrelation (AC) is calculated as: ˆR xx [m] = 1 N m 1 x[i]x[i + m] (3.1) ˆR xx [] where x[i] is the window in question. N is the length of the window and m is the time lag with m =, 1,..., (M 1) where M is the total number of time lags. This is chosen to be low, i.e. M << N, as the useful discriminative information in the ECG AC is concentrated in the first few time lags [1]. This time lag chosen actually corresponds to the QRS complex which is the most unique part of the ECG pulse. 2. Linear Discriminant Analysis: This step of the AC/LDA is important for increasing the inter-user variation among the feature templates and also to decrease the intrauser variation between different users. Also, another function that is performed by this step is that of dimensionality reduction. In a multi-class discrimination problem, LDA gives projections to a subspace where the inter-class scatter Σ b is maximized whereas the intra-class scatter Σ w is minimized [27]. In other words, we want to find the projection matrix W such that where Σ b = 1 i n i Σ w = 1 i n i W = arg max W i= W T Σ b W W T Σ w W. (3.2) N ( x i x)( x i x) T (3.3) i=1 N n i (x ij x i )(x ij x i ) T (3.4) i=1 j=1 (3.5) with N is the total number of users in the system, n i is the number of windows for the ith user, and the set of ACs of windows of the ith user is represented as x i. The bar indicates mean and x is the set of all ACs of the windows from all users. Also, x ij indicates the AC of jth window of the ith user.

41 Chapter 3. Towards an Intuitive Domain for ECG 29 It is then used to project each AC set x to the new subspace by the operation x proj = W x. The matrix W is calculated using the training templates and it is used to project the testing templates during evaluation, as discussed in the next chapter. The LDA leads to the projection matrix which is formed by the N eig significant eigenvectors of (Σ w ) 1 Σ b which correspond to its N eig largest eigenvalues. Here, N eig is chosen based on performance. 3. Classification: Using projections from the LDA, the query ECG is classified. Firstly, the query ECG signal is subjected to a similar preprocessing procedure as already described, until the AC windows are obtained for the user in question who is querying the system to authenticate herself. Then, the LDA projection matrix is used to project the AC window and a classifier is used. In the original work by Agrafioti et al. [1], the classifier used is a Nearest Neighbor classifier using Euclidean distance. Through work of Pouryayevali et al. [39], we know that the AC/LDA performs better than other methods in literature for the finger ECG signals. In [39], the authors compared methods such as the short time fourier transform (STFT) proposed by Odinaka et al. [33], wavelet distance based method proposed by Chan et al. [7] and the eigenpulse based method, that is basically using principal component analysis, proposed by Irvine et al. [19]. They found using the application of these methods on the large scale database that the AC/LDA outperformed them all. Hence, the AC/LDA is used as a baseline for the method proposed in this thesis Dimensionality Reduction The AC/LDA algorithm in the form described above does not perform well in applications to ECG biometrics using signals of the type and length that are used in this thesis. For good performance, we leverage the AC/LDA algorithm described above with a dimensionality reduction step before the LDA step. Bad performance of the application of AC/LDA to ECG biometrics happens due to the use of smaller length signals because of the small sample size problem. In the case of the large-scale database available for fingertip ECG, the recordings are roughly 12 seconds long. When the windows are formed and divided for training and testing purposes, we reach a scenario where we have lower number of samples for each user than the dimensions of the AC window feature template, i.e.,, n i < M which is undesirable for optimal operation of LDA. LDA performs the best when we have a high number of samples for each class it is finding projections to, i.e., in the case of n i >> M. This problem has been dealt with in literature in various contexts, such as in [18] and [49].

42 Chapter 3. Towards an Intuitive Domain for ECG 3 In the case of AC/LDA used in this work, we use the approach used in [3] where this is tackled in a simple way by using principle component analysis [22] before the use of ACs in LDA to reduce dimensionality. Thus, the PCA step before LDA reduces the effective dimension of the feature templates to, say M, that is much lower than n i - thereby solving the small sample size problem. This is done for the AC/LDA that is applied for comparison in this work. 3.3 Dictionary Learning and Sparse Coding Theory Motivations to Upgrade Motivation for another method as an upgrade to AC/LDA comes from two reasons. The first one is that ACs are chosen in the AC/LDA algorithm because of the fact that they blend the fiducial information to make the features essentially non-fiducial. However, there can be several such ways of blending in the fiducial information to obtain nonfiducial features. Hence, there is possibility for other features that could be explored for feature template representation for ECG signals. In this regard, using inspiration from face recognition to use parts based intuitive basis for ECG is reasonable, as ECG is a natural signal and contains structure. Also, the basis generated, say using PCA called eigenpulses or those generated later using LDA, do not make physical sense. In other words, when a particular ECG feature template in the form of a window s AC is taken, it would be correct to say that it is a linear combination of the basis vectors. This could involve subtraction of two basis vectors too, which doesn t make intuitive sense, in a similar argument to face image applications. A basis would be interesting where just adding more bases and different combination of bases will result in valid ECG ACs. More on this will be discussed later in this section. Secondly, the AC/LDA algorithm is fairly simple from the discrimination stage onwards, and involves a rather simple knn classifier. This makes the algorithm hard to scale well in large-scale scenarios as we will see in the complexity discussion later in this chapter over application scenarios. Moreover, in recent works where other classifiers have been used, such as SVM in Silva et al. [11], it has been seen that they outperform the knn classifier Towards Dictionary Learning The representations of a signal based on dictionary learning are based on the principle that a sparse subset of atoms taken from a redundant dictionary representing the causes

43 Chapter 3. Towards an Intuitive Domain for ECG 31 of the signal observations can also be used to described the observations [45]. Methods such as PCA (Principal Component Analysis) and ICA (Independent Component Analysis) look for transformations that respect some orthogonality conditions such that the variance is maximized (in case of PCA) or mutual information between observations is minimized (in case of ICA). However, these representations do not necessarily find the causes of the observations but rather only find a set of bases which satisfy the criteria of optimization chosen. Also the number of bases found by these methods are limited by the dimension of the signal. However, it could be that the sources which contribute to the signal s formation or its characteristics are more than the number of dimensions required to represent it while satisfying the criteria of variance/mutual information. In fact, in natural signals such as face images, speech, and other audio signals, it is quite likely that the number of sources is higher than the number of dimensions that maybe required to represent the signal quite accurately with PCA, ICA and similar methods. Thus a method which allows for more sources than the number of dimensions of the signals is required for a more intuitive and physically meaningful representation of such data which are likely to have come from a higher number of sources. Also, the sources calculated through methods such as PCA and ICA have to follow the orthogonality constraint. However, in real practical problem scenarios, such as the heart beat and the sources involved, there is no reason to assume that one source of the cause of heartbeats is orthogonal to the other source. Orthogonality thus is a statistical condition which makes for more compact lower dimensional signal representations, but makes no physical sense. Thus, a class of methods called Dictionary Learning has been developed to address the issue of deriving such bases which have more physical intuitive meaning and which are compatible with having more sources, i.e.,, bases than the number of dimensions in the signal. The problem of Dictionary Learning can be summarized as follows. For a set of signals X = [x 1,..., x n ] R D n, where D is the dimension of each signal x and the number of signals available for learning the dictionary is n, we want to find an approximation of this set of signals matrix X by matrix factorization. It has to find this as the product of a dictionary matrix D R D L and a coding matrix C R L n. This can be accompanied by sparsity constrains on the coding matrix C. The problem essentially becomes the following minimization problem (also described in [31]): min D C 1 n n (1 min c i C 2 x i Dc i f(c i ) ). (3.6) i=1

44 Chapter 3. Towards an Intuitive Domain for ECG 32 Here, f is a sparsity inducing regularizer on C and C is a constraint set for the dictionary. Different combinations of f and C can lead to different aspects of the approximate representation being solved for. For example, in the work presented here, we use the constraint that the coefficients of the coding matrix, i.e.,, the weights for the representation of the signal as a combination of the dictionary s columns, be non-negative. This leads to the interesting parts-based representation that we have talked about until now. For the sparsity inducing regularizer, we choose the function f(c i ) = λ c i 1 which is the l 1 norm of the coded matrix s column, i.e.,, the coded representation of x i. This essentially makes the dictionary learning problem same as the non-negative sparse coding problem. The constraint set for the dictionary is chosen to be the following set C = {D R D L, s.t., j, d j 2 2 1} (3.7) where d j are the columns, also called atoms, of the dictionary. There are two steps to solving the above problem, since there are two minimizations corresponding to finding the optimal dictionary for the signal representation and the optimal coding matrix for a particular dictionary. The natural way to go about this is to alternate between the following two minimizations. The algorithm used in this work for doing this is the one developed by Mairal et al. [31], as an online technique for large number of training data. As described above, in case of ECG signals, the dictionary learning method that was found to give best results in representation of the ACs was essentially Non-Negative Sparse Coding. In Mairal et al. [31], it is shown that NNSC can be implemented as a part of the online dictionary learning framework they propose which leads to nonbatch updates for the dictionary formation and thus to lower computation times. As argued in similar lines to above by Lee et al. [25], it is not consistent with the intuitive understanding of combining parts to form a whole object to say that features cancel each other out. The NNSC is a signal representation method for noise-robust feature extraction which aims to minimize the error between the signal and its approximation, subject to a sparsity constraint. Given a non-negative m n matrix X, NNSC aims to decompose it into the non-negative m k matrix W and the non-negative k n matrix H that minimize the reconstruction error J(W, H) = X WH 2 F + λ i,j H (ij) (3.8)

45 Chapter 3. Towards an Intuitive Domain for ECG 33 where F denotes the Frobenius norm of a matrix and λ controls the sparsity of H. The columns of W are the basis vectors and the values along the column in H denote the contribution of the columns of W, in other words, the decomposition of the signal values on the basis set W. It should be noted that NNSC is equivalent to Non-Negative Matrix Factorization (NMF) when λ =. NMF thus gives the parts based, linear representations of non-negative data as only additive combinations of the basis vectors are allowed. Usually, in case of NMF, k is chosen to be smaller than m or n, making it a lower-rank representation, only possible if there is inherent structure in the signals used. W can be learned according to (3.8) using the NNSC algorithm proposed in [31] as a special case of Dictionary Learning. H is obtained by using the same algorithm but with a fixed W. In our work, columns of X are the feature templates. Since these contain AC values in the range [ 1, 1], we add 1 to it in order to make its values non-negative. The decomposition of a feature template on the basis W is then treated as the new coded feature template and is used by the classifier. The number of vectors in the basis is crucial in signal reconstruction using the selected basis vectors and hence also in the authentication performance. Using more basis vectors compromises the dimensionality reduction aspect leading to worse performance of LDA as we described earlier in the section discussing the dimensionality reduction aspect. However, using more bases helps in better representation of the feature templates, i.e.,, the reconstruction improves when we use selected bases and do an inverse transformation using the coded templates to reobtain the original feature templates. Though reconstruction error is low for k = 1, we found in our experiments over the large scale database that the minima of recognition error occurred at this value. Since k < m, which is the dimension of the feature template, this effectively reduces the original dimension of the feature template m to k. In the experiments with NNSC using the finger ECG large-scale database, we started with a high number of bases which made the dictionary over-complete, i.e., 2 basis vectors. However, due to the small sample size problem of the LDA, we can not use such high number of bases for the representation. Hence, it was desired to use a very low number of basis vectors among the bases that were obtained by NNSC. Following this, we experimented on using a lower number of basis vectors for the representation of the signal, as it was observed from the 2 basis vectors case, that most of the bases found contained high frequency noise and no information related to the ACs. Also, only a few of these bases were used in the representation of most ACs and the rest were used only in some AC templates representation. Hence, we chose lower values of k and found the minimum value as described above. It could be said that the representation thus obtained after using a low value of

46 Chapter 3. Towards an Intuitive Domain for ECG Choosing k = Choosing k = 1 Figure 3.1: AC and basis for different values of k using the NNSC. k is not essentially sparse. This is a valid point. However, the choice of lower k is similar to PCA where we choose only a few eigenvectors to represent the signal when the remaining eigenvectors have very low eigenvalues. Here, the variation in the bases that were discarded may not be low in the same notion, but they are in a similar notion undesirable for use. Thus, we use a smaller basis set for representation, thereby achieving reasonably good representation of the signal, while also keeping the dimensionality of the resulting coded representations low for efficient use of the LDA. Figure 3.1 shows the bases generated for two different values of k, using the NNSC algorithm. On the top, for both sets of bases, is shown an AC generated by averaging a set of ACs for a particular subject. This is shown just for comparison s sake, as the bases are actually learned from all the AC templates of the whole enrollment set. As can be seen, k = 9 leads to a much smoother basis, whereas k = 1 generates bases that contain

47 Chapter 3. Towards an Intuitive Domain for ECG Contribution of each basis vector towards the AC formation Example of AC template and reconstructed from generated basis Reconstructed AC Original AC Weight Magnitude Base # Weights corresponding to each base Lag Original and reconstructed AC template Figure 3.2: Using generated bases to represent ACs - an example. more high frequency variations. As seen before, the reconstruction or representation error for k = 9 is higher than for k = 1, presumably because the smoother bases can not capture the high frequency variations in the ACs in the dataset. The figure on the right shows the 1 bases that were finally used to form the basis W, leading to a parts-based representation for the ACs. These bases can be combined additively to form many possible AC feature templates and thus provide a intuitive physically meaningful representation for the ECG signal. An example of this for the k = 9 case is shown in Figure 3.2 in the comparison of an example AC template and one reconstructed using the weights applied to the bases. Such different combinations of the generated bases yield different possible AC templates. It is not straightforward to see why the generated bases are intuitive. However, this is because of the embedding and blending of information related to the heartbeat in the autocorrelation coefficients, which are the feature templates we are dealing with. Since they essentially represent the second order statistics of heartbeats, the representation using generated basis contains information related to the heartbeats too. To show the parts-based representation property of the NNSC more clearly, it is directly applied to the heartbeats segmented from the large-scale database for different users. Figures 3.3 and 3.4 show two cases of representation where k = 1 and k = 2 are used. It can be seen in Figure 3.3 that some of the bases show different regions of interest in the heartbeat shown on the top. For example, the second base from the top shows the R- peak component, whereas the fourth base from the bottom contains the QRS region with an attenuated R-peak. In similar fashion, for a higher number of bases selected, in Figure 3.4, it can be

48 Chapter 3. Towards an Intuitive Domain for ECG 36 1 Hearbeat and Bases Generated k = 1, all bases Figure 3.3: Heartbeat and basis for k = 1 using the NNSC.

49 Chapter 3. Towards an Intuitive Domain for ECG k = 2, bases # 1 to 1 k = 2, bases # 11 to 2 Figure 3.4: Heartbeat and basis for k = 2 using the NNSC. seen that similar information is present, but also many bases with noise are present. Thus, it can be seen by these two examples that optimal selection of k is important for representation of the signal. In this section, it has been shown through examples for AC feature templates and also heartbeats directly that the NNSC gives parts-based representation of the signal which lead to intuitive bases which contain information about the original signal that can be intuitively understood and used. 3.4 The CoLD-kNN and CoLD-SVM algorithms After the above encoding of the ECG feature templates to a coded form using NNSC, there is still the step remaining of increasing the inter-class distances while reducing the intra-class distances. For this, we use the Linear Discriminant Analysis algorithm as was used in the case of AC/LDA as described earlier in this chapter. The LDA in this context does not face the high dimensionality induced small sample size problem because of the reason that we selected only a few number of bases in NNSC. Thus, LDA efficiently finds projections in the space of the encoded signals which increases the discrimination among the different classes, i.e., in our case, different users ECG templates.

50 Chapter 3. Towards an Intuitive Domain for ECG 38 After the encoding by NNSC and projecting by the LDA, we use one of two different classifiers depending on the application scenario, and computational requirements and availability. We propose two different classifier methods for our system: the k-nearest- Neighbour(kNN) based classifier and the Support Vector Machine(SVM) based classifier. Whereas nearest neighbor (NN) based classifiers have been used abundantly in ECG biometrics, SVM has been used recently [29, 1, 23, 48]. In [29], the authors propose a system based on two classification approaches, studied considering their computational and spatial complexities. We present similar aspects, and build on their work by introducing methods with lower complexity and dependent classification [1]. Gürkan et al. [16] used a subject size of N = 3 and achieved a Frame Recognition Rate = 97.31% They used AC/DCT, which is the Autocorrelation/ Discrete Cosine Transform method, MFCC(Mel-Frequency Cepstrum Coefficients) and QRS points from the heartbeat as the feature set. Then, an LDA based classifier was used on the AC/DCT and MFCC features, and a 3-NN based classifier on the QRS. The authentication performance was evaluated based on identification scenario. In the work by Silva et al. [11], the SVM was used in the process where recognition performance was evaluated in identification and also in authentication scenarios. For authentication they calculate the False Acceptance Rate (FAR) and the False Rejection Rate(FRR) for each operating point, which are used to determine the Equal Error Rate (EER) - the chosen metric for performance assessment. They use a 3-NN classifier specifically for the knn implementation and a linear kernel for the SVM using the LIBSVM implementation that is publicly available with the one-vs-one configuration for classification. This configuration leads to N 1 models for each user enrolled in the biometric system for authentication and thus leads to a total of N(N 1) binary SVM models. Their work presents the best known results, in our knowledge, of authentication results for single-session data, with an impressive EER of.99% achieved for a total of N = 63 subjects. In the work by Luz et al. [1], low frequency sampled ECG are used with a database size of 193 subjects compiled from various publicly available databases containing conventionally acquired ECG signals (for example, the MITDB available through Physionet [15]). Identification performance is evaluated and after applying SVM with RBF kernel, a classification approach using majority voting using multiple samples per subject is used. This is compared to a traditional single samples per subject method. Identification rates of 95% for 3Hz sampling and close to 1% for 6Hz are achieved. The authors of the work call the use of multiple samples as dependent classification and the single sample use as independent classification and show how the dependent classification ap-

51 Chapter 3. Towards an Intuitive Domain for ECG 39 proach improves performance. In this thesis, we adopt this approach in the design of our biometric system. Many other works also use SVM based classifiers for ECG biometrics, though with conventional medical quality signals available with small database sizes such as those by Boumbarov et al. [34], Kaveh et al. [23], Ming Li et al. [26] and Bugdol et al. [5]. In combination with earlier stages, the acronym CoLD is used for AC-SparseCoded- LDA, thereby naming the proposed methods CoLD-kNN or CoLD-SVM based on the classifier used. In both methods, the test templates are classified using dependent classification, based on the majority voting scheme used in [1]. In their work as described already, dependent classification was shown to improve performance. This method basically involves firstly, the assumption that a query from a user involves more than just one feature template. This is reasonable to assume, as it depends on the acquisition device and the design of the system. In security intensive systems, it would not be unreasonable to assume that users have to provide a longer signal duration for authentication. When multiple feature templates are available, say n t templates are taken from a user and this whole set of templates is considered as a query. Dependent classification involves classifying each individual feature template and then using voting among this batch of n t templates to classify the user. This accounts for the lower error in recognition, as it is more unlikely for a majority of templates in a query to be misclassified than for a single template to be misclassified k-nearest-neighbour based Classifier One of the main advantages of using this classifier is that it is simple to implement and easy to interpret its operation. knn classifier classifies the feature templates based on their similarity (or distance) to the feature templates in the training data, that is feature templates obtained from the users during the enrollment session. In the implementation of the knn classifier, choice of distance becomes crucial in its behaviour. This choice of distance may be informed by the noise present in the data, if known. Otherwise, as in our case, empirically the best distance is chosen among many options to get higher classifier accuracy. Here in this work, each test template is compared to all training templates from the enrollment set using a Euclidean distance measure, which performed better than the other distance measures tested. Let the k closest training templates to the test template (a single template from the n t templates in the query of user) be x i, i = 1,..., k. The

52 Chapter 3. Towards an Intuitive Domain for ECG 4 Figure 3.5: Block Diagram for CoLD-kNN algorithm classifier consequently assigns the test template to user u such that k u = arg max δ(u, L(x i)) (3.9) u i=1 where L( ) gives the label of its argument and δ is the Kronecker delta function. Hence this is basically a majority voting among the k nearest neighbors of the template from the query. This classifier can be very advantageous in situations where the distance measure is appropriate for the data at hand and also when ease of implementation and model complexity is to be kept low. However, there are disadvantages of knn as follows. The first is its higher complexity (as explained in the last subsection of this chapter) which leads to infeasibility in situations when there are too many training templates. Also, as it is a majority based scheme, with smaller values of k, there can be high susceptibility to noise in the training data. Moreover, when the distance measure is not selected appropriately, the classifier may fail to have high classification accuracy. The CoLDkNN algorithm is shown through a block diagram in Figure Support Vector Machine Classifier Support Vector Machines are arguably the most popular classification method [9] in literature and has been used in ECG biometrics recently as discussed extensively in this

53 Chapter 3. Towards an Intuitive Domain for ECG 41 section. It is essentially a binary classifier which works by providing the optimal decision hyperplane that best separates the two classes present in the training data. It does so by using the training data points at the edge of the class descriptors. With {(x i, y i ) i = 1,..., m} being the training data templates and {y i {±1}} being the corresponding class labels for a binary classification scenario, the problem to maximize decision margin becomes the minimization problem given by 1 min w,b,ξ i 2 w 2 + C m ξ i (3.1) i=1 s.t. y i (w T x i b) 1 ξ i (3.11) where ξ i are slack variables for the misclassification of x i and C is the penalty cost for optimal separation in non-separable cases. For optimization, the objective function is usually converted to its dual form involving only the inner product information of the input features which is specified by a kernel function. SVM is thus transformed to a non-linear classifier by applying a kernel, thereby mapping the data to a higher dimensional space. The kernel used in our work is the radial basis function, given by k(x i, x j ) = exp( x i x j 2 ). (3.12) 2σ 2 Parameters {σ, C} determine the classifier performance and are determined through cross-validation. The CoLD-kNN algorithm is shown through a block diagram in Figure 3.6, essentially differing from Figure 3.5 in the classifier stage. In the proposed system having more than two classes, we use a one-vs-all approach for dealing with the multi-class problem [12]. This means that for each user, a single binary SVM classifier model is trained which classifies a test template as belonging to either that user or to one of the rest of the users. Thus, a total of N models are trained. This is a feasible approach for large-scale applications. Compared to this, the approach used by Silva et al. [11] implemented through LIBSVM i.e., the one-vs-one approach leads to N 1 models for each user. This costs O(N 2 ) models in total for the system which significantly increases the computational and spatial complexity of the implementation. Thus, a simpler solution is to use the one-vs-all approach that is adopted throughout this work.

54 Chapter 3. Towards an Intuitive Domain for ECG 42 Figure 3.6: Block Diagram for CoLD-SVM algorithm 3.5 Application scenarios : Complexity Analysis In this section, the complexity requirements of the proposed two classifiers are discussed briefly. This is important from the point of view of application scenario, namely, the availability of computational and spatial resources for the proposed system to work. For the knn based classifier, for classification of each test template, we require the whole set of training feature templates from the enrollment data. This costs O(N T ) space, where T is the average number of training templates for a user of the system. Also, in terms of computational complexity, we require NT n t comparisons for the test templates from the query, thus costing O(NT ) time. In contrast, the SVM based classifier does not require the entire training feature templates set, but only N SVM binary models for each of the users, thereby costing O(N) space. Moreover, computationally speaking, a single model is used while classifying a query with n t comparisons costing O(1) time. In all these considerations, SVM emerges as the choice for the classifier used. However, the caveat is that it is undesirable for a situation where complexity for training the binary SVM models is a constraint. This is because while knn requires essentially no training, SVM is known to cost over O(N 2 T 2 ) time. Thus, the above considerations for the classifier choice depending on the application

55 Chapter 3. Towards an Intuitive Domain for ECG 43 scenario is important. In a scenario where training of the system is required more than once and the system doesn t have high computational capacity on-board the device, e.g.for new users on a device that performs authentication which is a quite common scenario which requires system flexibility, the SVM based classifier is too costly because of the high training complexity. In such scenarios, a knn based classifier is the solution as it essentially requires no training, though the system would still have to calculate the projections for NNSC and LDA. However, in all other scenarios with fixed user base and constrained spatial and computational resources, the SVM based classifier seems to be the classifier of choice.

56 Chapter 4 Large Scale Scenario : Fingertip ECG 4.1 Introduction In this chapter, we deal with the large scale database from the fingers containing 112 subjects, for the purpose of evaluating the feasibility of finger ECG for biometrics and also to verify the utility of the proposed system using the twin CoLD-kNN and CoLD-SVM methods. Population size of the users using the access control system is an important factor in its design and viability. Since in an authentication system, the system parameters and the performance are dependent on size, scalability is preferred for practical applications of ECG biometrics. Thus, large-scale databases ought to be used for evaluation for such applications. As discussed in [33], most studies have used small database sizes and hence there is need for feasibility studies in large databases. This present chapter evaluates fingertip ECG from 112 subjects in an uncontrolled acquisition environment and hence provides reliable performance measures on large scale applications. Also provided is a comparison of the proposed methods with AC/LDA over different population sizes. The only other large scale study known is Pouryayevali et al. [39] where the fingertip ECG large-scale database was introduced and AC/LDA was used. The usage scenario could consist of any application where ECG signals from the fingers are used. This could be very identical to a fingerprint recognition system in terms of use. Instead of touching a single fingerprint sensor, the user of the system would be required to touch two sensors with his/her thumbs and the system would record the ECG signals from the thumbs which would them be used for biometrics. As discussed in the 44

57 Chapter 4. Large Scale Scenario : Fingertip ECG 45 early chapters of this thesis, the main disadvantage of this system is that of having to use both sides of the body. In this chapter, firstly, through the use of proposed CoLD methods large-scale application of fingertips ECG biometrics is evaluated for performance. This utilizes the intuitive bases for ECG signals as discussed in the previous chapter. Also, the effect of increasing population size on performance is studied Related Work Non-intrusive acquisition of ECG signals has been proposed by usage of fingertips ECG, which has been pursued only recently. Chan et al. [7] first used fingers ECG in 28 and achieved 89.1% identification rate (IR) using a wavelet-based distance measure. Thereafter, Shen et al. [43] used signals from palms and obtained an IR of 95.3% using prescreening. Zhao et al. [51] also used fingers ECG and reported an EER of 8.7% using Autocorrelation/ Linear Discriminant Analysis (AC/LDA) for cross session matching. Finally, Silva et al. [11] used finger/palm ECG for a 63 subjects database and reported an across and single session EER of 9.1% and.99%. Biometric recognition using ECG consists of two broad approaches, namely the fiducial points dependent and the non-fiducial methods. Fiducials are specific points on the ECG heartbeat which can be used to extract features based on its temporal and amplitude characteristics. Fiducial approaches use specific points on the ECG heartbeat whereas the latter adopts methods which are blind to location of these points. Approaches using fiducials are abundant in literature such as [2, 4, 48, 24, 13]. Notably, [2, 4, 13] report 1% identification accuracy using fiducial methods on modestly sized databases using conventional electrode configurations whereas [48] reports 99.6% and 88.2% identification accuracy using 2-lead fusion and 1-lead respectively. Nonfiducial methods used in [37, 46, 2, 32, 33] do not rely on specific points on the ECG curve but rather use statistical characteristics. For e.g., autocorrelation, which contains the same information as fiducials blended holistically is used in [37]. The use of nonfiducial methods are preferable and adopted in this work as they lead to less computations for real time processing, in addition to introducing lesser errors. Many approaches have been adopted for use of domains for ECG signals. Use of the short-time frequency analysis[33], wavelets[7, 8], principal component analysis (PCA) [19] and the autocorrelation coefficients (ACs) [1] have yielded good results. However, an intuitive basis for ECG signals has not been explored yet, and has been discussed in length in Chapter 3. Also, a customized basis can be used to encode ECG templates

58 Chapter 4. Large Scale Scenario : Fingertip ECG 46 making fake queries infeasible without access to this basis. This is impossible with fixed domains like AC, wavelets, etc. In the following sections, the system model describing the signal preparation and then the algorithms used are described. Then the experimental results are discussed with the parameters used and interpretations of the plots. A weakness of this work is that we use data from a single session alone. However, a large database for multi-session finger ECG is not available. Other works [11, 51] have also used modestly sized multisession databases only. This work instead aims at the large-scale scenario, although multi-session analysis is an ongoing work. In this regard, note that it is possible to update templates[14] to decrease the performance degradation over time, which could be further explored for finger ECG signals. In this chapter, present work builds on [11] by evaluating our methods in a large-scale scenario and by using the proposed methods according to the application scenario. 4.2 System Model We use two methodologies as described earlier in Chapter 3, the CoLD-kNN and the CoLD-SVM, which differ from each other in the classification stage, where either a k- Nearest Neighbour or a Support Vector Machines based classifier are used Database In this chapter, the publicly available finger ECG signal database collected at the Biometric Security Lab at University of Toronto with 112 subjects is used. The proposed methodology is evaluated for performance on this database. This database is chosen as it has: 1. Signals from fingertips that were collected in an uncontrolled environment. This makes these analysis useful for real world applications of ECG biometrics. 2. A considerably large database with 112 subjects for single session in sitting posture. The large size helps in reliable feasibility study of large-scale applications of fingertips ECG biometrics. A typical fingertip heartbeat and the acquisition method are shown in Figure 4.1[39]. The signals were collected through a 1-lead configuration from the thumbs and were sampled at 2Hz with 12 bits per sample. We use single session data with

59 Chapter 4. Large Scale Scenario : Fingertip ECG 47 signal lengths varying from 2 to 5 minutes. Each subject s signal was divided into training and testing signals for performance evaluation. 1.2 Single Heartbeat 1.8 Amplitude (mv) Time (s) Figure 4.1: (Left to Right) Electrode placement for ECG acquisition from fingertips, single heartbeat from fingertips Preprocessing and Template Formation Typical noise present in ECG signals are baseline wander(<.5hz), powerline interference(6hz), contact noise from electrodes and EMG interference(>4hz) [44]. A zerophase fourth order butterworth bandpass filter with passband in [.5, 4]Hz is used to remove these. In previous works in ECG biometrics literature, it was found that a simple bandpass filter works well for removal of such noise and hence the same approach is adopted, after confirmation through experiments that a butterworth filter works well. Firstly, we segment the ECG signals into segments using overlapping windows, leading to a fixed length preliminary feature vector, important for the future operations. Adopting a purely non-fiducial approach, we use the autocorrelation coefficients of segments of ECG signals as our feature templates, as in [1]. For this, we segment the signal blindly using overlapping windows chosen long enough to contain a few heartbeats. The normalized autocorrelation coefficients (ACs) for each window are then calculated as: ˆR xx [m] = 1 N m 1 x[i]x[i + m] (4.1) ˆR xx [] where x[i] is the window, L is the length of the window and m is the time lag with m =, 1,..., (M 1) and M is the total number of time lags. This is chosen to be low, i.e. M L, as the useful discriminative information in the ECG AC is concentrated in i=

60 Chapter 4. Large Scale Scenario : Fingertip ECG 48 the first few time lags [1]. Outlier Removal These ACs are then passed through an outlier removal stage that rejects windows whose ACs are statistically outliers in the set of ACs for each user. These outlier ACs are generally those which result from the windows that have sharp peaks and artifacts that survived the preprocessing stage. This is an important stage as bad windows create anomalies that propagate to the learning phase of the system, i.e., the LDA, and also the formation of a dictionary using NNSC which depends on the structure in the data. Using a variance dependent threshold τ σ, the set of valid templates, S i, for user i is obtained by retaining windows ˆR i such that: S i = { ˆR i : ˆR i mean( ˆR i ) 2 < τ σ } (4.2) In simple words, this is the removal of those AC windows for a user which are more than τ σ away from the user s mean AC window, when the distance used is the l 2 norm. This construct assumes that the windows follow a multivariate normal distribution. However, this is not the case all the time, as is observed during the experiments. However, this outlier removal method is simpler than those used by Lourenco et al. [3] and works well, hence used in this work. Figure 4.2 shows the percentage of windows caught as outliers, for each user, at best performance parameters. The number of outlier windows caught per user is ± 9.24 (mean ± deviation) windows. Also, as a percentage of available windows for each user, there are ± percentage outliers Algorithms Used The biometric quality of the signals are evaluated using the algorithms described in Chapter 3, i.e., the CoLD-kNN and the CoLD-SVM methods. Also, they are compared with the AC/LDA. These analysis are done with respect to the performance at the maximum possible database size, i.e.,, 112 subjects. Also, the effect of changing the database size from smaller sizes to larger sizes was studied to provide more insights into the operation of the algorithms. In case of NNSC, it was found empirically that a basis size of k = 1 was ideal for signal representation through the dictionary and for recognition performance. Similarly, in case of AC/LDA, it should be noted that the LDA stage was preceded by a PCA stage for dimensionality reduction, facilitating operation of the LDA. In this stage, the

61 Chapter 4. Large Scale Scenario : Fingertip ECG Outlier Windows Percent = ± Percent Outliers Mean Percent of outliers Outlier Windows Percent User Figure 4.2: Percentage of windows caught as outliers, for each user in the large-scale database. number of eigenvectors used in the formation of a projection matrix was also 1, thereby providing a reasonable comparison environment between the NNSC and PCA s influence. It should be noted that in application scenarios such as the present one, where the database size is large, the complexity issues are well seen. For example, consider a onevs-one approach as used in [11] which gave very good results for a smaller database of 63 subjects. When such a system is applied in a large scale setting as the presented database, the number of models comes to ( N 2 ) = N(N 1)/2 = models. This is considerably higher than the N = 112 models which need to be stored for a one-vs-all approach. 4.3 Experimental Results The performance was evaluated in verification mode where users are enrolled in a system. During use, a user makes an identity claim and the access control system decides whether to reject or accept the claim. This leads to two types of errors: False Acceptance (FA) -

62 Chapter 4. Large Scale Scenario : Fingertip ECG 5 Methods\N AC/LDA 3.17 ± ± ± ± ± CoLD-kNN 2.91 ± ± ± ± ± CoLD-SVM.63 ± ± ± ± ± Table 4.1: Summary of Results: Mean EERs and standard deviation for different N when a false claim is accepted and False Rejection (FR) - when a legitimate user is rejected access. These lead to the FA-rate (FAR) and FR-rate (FRR) error metrics which are the ratio of such errors with the total number of false and true claims made, respectively. Since there is a trade-off between FAR and FRR, the Equal Error Rate (EER), which is the error at the point where FAR=FRR is chosen to evaluate the performance of our system. In the following experiments, we compared our methods with the AC/LDA algorithm[1]. In [39], it was shown that AC/LDA performs better for large-scale systems compared to the short term frequency analysis [33] and wavelet distance method [7]. Hence, AC/LDA is used for evaluation of our methods for large-scale performance and scalability. Firstly, all the three methods were evaluated using the whole database containing 112 subjects in terms of their receiver operating characteristic(roc) curves. Secondly, the effect of increasing the number of users N on system performance was studied. For each N, multiple batches(disjoint subsets of database) of subjects were created. Then the methods were applied on each of these batches separately, hence using independent validation for evaluation. Results are reported in terms of mean and standard deviation of EER for each N. e.g.for N = 1 subjects, 4 batches of 1 subjects each were used. For template formation, overlapping windows of 4s duration were chosen with 5% overlap. Number of lags for ACs was M = 5, corresponding to a time lag of 25ms. We used 75% of the templates to train and the rest to test our system. The NNSC reduced the dimension of the AC feature templates from 5 to 1, but the LDA projected it to another 1 dimensional space where the user separability was maximized. Figure 4.3 shows two subjects templates having distinctiveness after LDA, even with low dimensionality. Parameters for SVM {σ, C} were tuned empirically using cross-validation. For experimental purposes, the nearest-neighbor classifier was used for the knn stage, i.e., k = 1.

63 Chapter 4. Large Scale Scenario : Fingertip ECG 51 Figure 4.3: Each trace is a template of subject 1(red) or subject 2(black). Mean(bold), minimum and maximum of templates are shown. First two dimensions out of 1 are omitted for clarity Large-scale performance using full database Figure 4.4 shows the ROC curves for the compared methods for maximum population size possible using the database. The ROC curve closer to origin along the FAR = FRR line is considered to be better. As we can see, the CoLD-SVM algorithm clearly outperforms the other methods whereas the CoLD-kNN performs only marginally better than the AC/LDA [1]. The CoLD-SVM also performs best for the low FAR region, i.e., FRR values at low FAR are lowest among other methods. This shows potential utility in high-security applications where accepting a false claim is extremely hazardous though a reasonably low FRR is required. On the other hand, in the low FRR region, FAR values for CoLD-SVM are highest among the methods, suggesting that the CoLD-SVM may not be suitable for applications where convenience is important and hence very low FRR is necessary, while FAR being reasonably low. Similarly, the CoLD-kNN performs best for low FRR region where it has lowest FAR