USING SUPPORT VECTOR MACHINES TO DISTINGUISH USERS THROUGH TOUCH GESTURE RECOGNITION

Transcription

1 USING SUPPORT VECTOR MACHINES TO DISTINGUISH USERS THROUGH TOUCH GESTURE RECOGNITION A Thesis submitted to the Faculty of the Graduate School of Arts and Sciences of Georgetown University in partial fulfillment of the requirements for the degree of Master of Arts in Communication, Culture, and Technology By Stevie N. Chancellor, B.A. Washington, DC April 3, 2014

2 Copyright 2014 by Stevie N. Chancellor All Rights Reserved ii

3 USING SUPPORT VECTOR MACHINES TO DISTINGUISH USERS THROUGH TOUCH GESTURE RECOGNITION Stevie N. Chancellor, B.A. Thesis Advisor: Evan Barba, PhD. ABSTRACT Security and authentication are essential components of modern-day computing systems. Yet the advantages of traditional passwords do not extend to touchscreen technology where others can easily discern passwords (Kim et al 2010) or the passwords are shorter and therefore less secure (Findlater et al 2011). Research has provided fruitful authentication alternatives for touchscreens, such as facial recognition and fingerprint scanning, but these have limited adoption on these devices. Gesture recognition is a biometric identification method with much potential for adoption onto touchscreen devices yet limited research in the subject area. I offer a two-factor gesture identification system for the Microsoft PixelSense Table. The program incorporates a gesture symbol ( what you know password) that also identifies a user based on their behavioral-based biometric password inputs on a touchscreen device ( what you are password). Eighteen features were extracted from the raw data and incorporated into a support vector machine, a supervised machine learning algorithm that classifies users based on their consistent input of a touch password. In addition to the program, I also conduct data collection of 19 participants and their unique gestures to determine the most distinct feature set for the model. I provide a proof of concept that gestures can be used as passwords on the PixelSense table that avoids many of the problematic elements of alphanumeric passwords on iii

4 touchscreens. I also provide a new list of features that can be incorporated into future learning models on other touchscreen devices. iv

5 Acknowledgements I m indebted to many friends and colleagues who have helped made this process successful. Thank you to all the participants in my research study your touchscreen doodles were invaluable in proving my initial research question right and proving my program a success. Thanks to the many colleagues and friends who have encouraged, listened, and challenged my ideas and the CCT community at large. A special thanks goes to Sam F. and Deepa R. - between writing parties, editing sessions, and practice presentations, your support and laughter has been invaluable in this whole process. I also want to thank Yianna Volvides for advice, both professional and personal - she has made this process enjoyable and reaffirmed my own faith in my ideas. And a very special thanks to Evan Barba for pushing me to take up this research despite my initial hesitations in my own abilities. I never thought taking a small summer research position would become a turning point in my academic career and lead to so many different projects and such a wonderful working relationship. Thanks to both my parents, Mom and Ed, for being my cheerleaders and teaching me the value of passion, ambition, and hard work. And, to Owen, without whom I would surely have gone crazy without your love and support. Many thanks, Stevie Chancellor v

6 TABLE OF CONTENTS Chapter 1 Introduction... 1 Contributions to the Field... 5 Organization... 6 Chapter 2 Literature Review... 7 Designing an Efficient Security System... 7 Current Password Systems... 9 Areas of Knowledge-based Authentication Research Graphical Passwords Biometric Password Research Facial Recognition Fingerprint Recognition Areas of Behavioral Biometric Research Location/Continuous Authentication Gesture Recognition Current Research on Gesture Authentication Systems Machine Learning Summary Chapter 3 Implementation and Methods Implementation Specifications Anatomy of a Gesture Building a Feature Set Chapter 4 Results Results of Passwipe User Study and Data Collection Chapter 5 Discussion Insights Problems and Limitations Chapter 6 Conclusion Future Research Bibliography vi

7 CHAPTER 1 INTRODUCTION Security and authentication are essential components of modern-day computing systems. These systems are hubs for self-expression, content creation, financial decision-making, and dating - computers distinctively mediate many of our life experiences. Users of these devices want control over who has access to data about their activities, which could be anything from personal communications to financial records (and more). Therefore, authentication on such devices is a necessary step for the protection of such information. Verifying one s knowledge of a cryptographic key, which could take the form of a passphrase, pattern, or a secret code, is the most common method to identify and authenticate users into systems. The most common authentication mechanism used today is the alphanumeric password, most frequently encountered as a login box on websites. This is an artifact of the older web, in which desktop and laptop computers were the primary means of online access. However, touchscreen interfaces are increasingly more popular and are now available as touch-screen PCs, handheld tablet computers, mobile phones, and larger touchscreen tables and devices. Because of the unique input factor of a touchscreen, password systems for touchscreens have made new combinations of the traditional alphanumeric password. Most devices have several options for passwords, including short personal identification numbers (PIN), dot password systems where a user traces a pattern across dots, or longer alphanumeric passwords. As more personal data moves onto these devices and they become increasingly portable (so much so that they fit in your 1

8 pocket or on your wrist), security of the device is crucial to help users maintain privacy and keep their data away from prying eyes. Yet, the outlook of these kinds of alphanumeric, knowledge-based password systems on touchscreen devices is bleak. The unique interface of a touchscreen poses new challenges to concealing a password. Users have more difficulty with traditional keyboard typing and thus select simpler passwords to compensate (Findlater et al 2011). In addition, these kinds of passwords are particularly vulnerable to unique attacks like a nearby shoulder surfer watching someone input a password (Kim et al 2010) or smudge marks revealing a swipe password (Aviv et al 2010). Users opt to use shorter and easier passwords than the ones they would use to secure their PCs, laptops, or online accounts (Findlater et al 2011), caused in part by the difficulty of using these kinds of passwords on touchscreens. The dominance of weak security measures does not stems not from an indifference to security but from difficulty, as nearly half of mobile users say that they dislike their current password system (Clark and Furnell 2007). A fruitful area in research incorporates biometric identification to authenticate users on their touch-screen devices. Biometrics use data about peoples bodies (physiological biometrics) or distinguishing behavioral patterns to differentiate between people (De Luca et al 2012). This is known as the what you do password system, opposed to the what you know approaches that alphanumeric passwords use. Areas of recent research in physiological biometrics include iris and fingerprint readers and facial recognition (Jain et al 2006). In some instances, these authentication techniques are more optimal as a user may need high levels of security and accuracy with instantaneous authentication. However, such technologies require additional hardware (such as an iris or fingerprint scanners), are cumbersome to use, and drain a device s 2

9 battery. These kinds of facial and fingerprint characteristics are also immutable there is no simple way to change one s facial structure and as soon as these characteristics can be spoofed or copied, the security of the system is impossible to restore. More importantly, most of these devices are inaccessible to the average consumer because accurate implementations of these systems can be prohibitively expensive for mobile technologies. Behavioral biometrics, on the other hand, identify users based on their habits of movement, intonation, gestures, or other behavioral traits. For mobile devices, behavioral biometrics can be used to identify users based on their movement and interactions with the devices. A novel application of this idea to mobile phone security is processing finger gestures on touchscreens. A gesture is an expressive body motion that provides meaningful information (Mitra and Acharya 2007). For my thesis, a gesture refers to the finger movements across the touchscreen of a computer. Research incorporating gesture recognition onto smartphone technology has been fruitful in the last few years; De Luca et al recorded and analyzed a preselected set of gestures and identified users with dynamic time warping (2012) and Sae-Bae et al incorporated simple multi-touch finger gestures to ipad authentication (2012). However, both of these studies had low accuracy rates (De Luca s study achieved as low as a 55% accuracy rate at times), did not enable customized password inputs, and did not incorporate machine learning algorithms into their analyses. A complementary development in another field of computer science is machine learning and artificial intelligence. The increasing power and focus of data-processing algorithms has led to an explosion of efforts to analyze data programmatically for predictive purposes, passwords notwithstanding. Different kinds of machine learning algorithms can analyze different kinds of 3

10 data, and their focuses can be geared towards toward classifying or labeling data. Some areas where this approach has been used in similar research for biometric passwords include facial identification, (Hjelmas and Low 2001) and gait recognition (Gafurov et al 2006; Mantyjarvi et al 2006). One type of algorithm useful in binary classification a is a support vector machine (SVM). An SVM is a supervised machine learning algorithm that classifies data based on a function computed from training data (Duda et al 2012). b A very recent study successfully applied support vectors c to mobile phone identification to a predefined set of finger gestures on mobile phone technology (Shahzad et al 2013). However, this research has not been extended to different kinds of touchscreen technologies or attempted with a richer set of features. This thesis will examine the implementation of machine learning algorithms to gestures passwords on a touchscreen device. Specifically, I ask the question: Can support vector machines be used to distinguish between users through a gesture password system on the Microsoft PixelSense table? I hypothesize that SVMs can be used to distinguish an authenticated user from a non-authenticated one on the Microsoft PixelSense table. The Microsoft PixelSense table (formerly called the Microsoft Surface table) is a tabletop computer with a touchscreen interface released in 2007 that allows more than one user to interact a Binary classification organizes data into two groups, whereas multiclass machine learning can organize data into multiple groups b Broadly speaking, machine learning techniques can be classified into supervised and unsupervised techniques. An unsupervised algorithm combs through the raw data to find conclusions rather than being explicitly trained by a labeled training set. For more information on this type of algorithm and the benefits and drawbacks of unsupervised learning, see Chapter 10 of Duda, Hart, and Stork s Pattern Classification: 2 nd Edition. c Shahzad et al applies the notion of support vector distribution estimations to gesture passwords, which uses the fundamental concepts of support vectors and applies the results to a probability distribution. A Support Vector Distribution Estimation (SVDE) is similar to an SVM in that they create a function that clumps data. Rather than classify into two piles, their system produced a probabilistic estimation that the likelihood that, given the features provided, the user was authorized to access the device. 4

11 with the device at the same time. Most modern touch screen interfaces are capacitive touch screens, which track the location of finger touches across a grid of electric circuits. As a finger moves across the screen, the modification of electrical charges between a finger and the touchscreen tracks your movement along a grid of circuits (White 2008). However, the PixelSense table uses near-infrared cameras to monitor the position of gestures across the table. This allows it to track multiple touches at once (up to 52) where most touch screens can only track two to five ( Hardware Overview 2009). More importantly, these touches contain a much richer set of data than touches made on a capacitive touch screen. Capacitive screens do not recognize the size of the touch or its orientation on the device; they merely track the coordinates of the touchscreen over a coordinate plane over time (White 2008). The PixelSense table provides a richer set of data about gestures and hopefully inform what additional features can be incorporated into future research about gesture passwords. Contributions to the Field This thesis aims to show support for other researcher s approaches to gesture recognition by incorporating a machine learning model into the authentication system. By bringing in an SVM, I demonstrate that gestures with the data provided by the PixelSense table do contain unique features that can be extracted for individual users. I created Passwipe, an application that distinguishes between an authorized user and a non-authorized user. In addition to providing a proof of concept of the system, I also provide extensive analysis of a rich feature set for gestures from 19 volunteers. This research may help other researchers and companies develop gesture password systems by providing example gesture shapes, features, and implementation techniques 5

12 for other touchscreen devices. In addition, the robust feature set provided by the PixelSense table and the accompanying features I extract will be useful in the near future as the sensitivity and raw data from capacitive touchscreens increase. I expect mobile devices to adopt these screens in the future, and my research would provide prototyping information and implementation details to professionals and researchers interested in implementing new feature sets to gesture password systems. Organization The rest of this thesis will unfold as follows: Chapter 2 will review the relevant literature on password security of touch-screen devices, the relevance of machine learning and SVMs to this approach, and justification for the approach taken. Chapter 3 will outline the methods and design goals of Passwipe, the program created to demonstrate a gesture authentication system. Chapter 4 will discuss the results of the thesis, including an analysis of Passwipe and the data collection. Chapter 5 discusses the impacts of Passwipe and such a password system, including new discoveries, limitations, and other relevant discoveries. Finally, Chapter 6 will provide a conclusion and areas for future research in gesture password systems. 6

13 CHAPTER 2 LITERATURE REVIEW In this chapter, I provide an overview of the field of password security with a review of the relevant literature. I begin with a survey of key components to devising new password systems and then provide an overview of the current types of systems available while also comparing their advantages and disadvantages. Then I assess the application of machine learning, specifically SVMs, to the problem of gesture passwords and conclude with a summary of why my approach is a good idea. Designing Efficient Security Systems When designing a new authentication system, two key factors surface in literature surrounding good design security and usability (Kim et al 2010; Klein 1998). Security is an obvious concern and the primary contribution of this thesis. The ubiquity of mobile devices and the sorts of transactions that occur on them necessitate a secure system that is not easily cracked, brute-forced, or reset. An insufficiently secure device exposes consumers personal, financial, and medical data to the risk of being stolen. Correspondingly, a user s method of verifying their identity, whether that be with a passphrase or fingerprint, should be uniquely identifiable and not easily compromised. 7

14 Yet, authentication systems must strike a balance between the most secure authentication techniques and usability advanced password systems like iris scanners may be effective at protecting high-security data but be wholly inconvenient for everyday touchscreen and mobile phone use. If users do not find the system intuitive and quick to use, they may become frustrated and choose a less complicated password to compensate (Findlater 2006). In fact, a poorly-chosen system could encourage users to turn off their security settings on their devices, making them more vulnerable to data theft than before! An example of this is when Microsoft introduced its User Account Control features in its Vista operating system. The alerts for seemingly normal activities on a computer (opening a program) were so intrusive that many users turned off the alerts entirely, despite its strong ability to catch dangerous viruses. Another concern for usability is how impaired audiences can engage with the device; visually or physically-impaired users may have difficulty with systems that presuppose that all humans have sight and sufficient use of their limbs to dexterously input a password. Both security and usability make a dichotomy for passwords that needs to be balanced when selecting or developing a new touchscreen authentication system. How do we know when something is sufficiently secure or usable? Are there ways to measure security and usability? Security is a blanket term that encompasses both theoretical concerns of algorithms - measured in time to crack the algorithm or mathematical operations needed to expose a vulnerability - as well as practical aspects, such as the need for a memory aid to recall a password. This dichotomy of security is apparent in our understanding of the failures of text passwords - even though the encryption algorithms for our passwords may be theoretically secure, no security is gained when someone uses an easy-to-guess password or 8

15 leaves it on a Post-It note next to their computer. Similar standards apply to usability: we can measure time to completions or number of taps, but that does not capture when something is or is not accessible, frustrating or relieving. Therefore, in the analysis that follows of different systems and their potential benefits and drawbacks, I have tried to illuminate certain characteristics that may be more abstract and not typically quantifiable by traditional measures of security. Current Password Systems The most common password systems are knowledge-based schemes, or something you know authentication approaches. The verification of knowing a cryptographic key, whether that be a passphrase, pattern, or a secret code, is the most common tool to check identify and authenticate users into systems. On face, this type of password system is the simplest yet is incredibly powerful: knowing a password verifies your authenticity and identity as an authorized user or member of a community (Jain et al 2005). More importantly, nothing about the user needs be known other than that they have the proper password the system requires no other input other than the passphrase to confirm identity and grant access. These what you know passwords are the most commonly used system on touch-screen devices and include PINs used on Android, iphone, and ipad devices as well as the Android swipe dot password system. Yet, there are drawbacks to these kinds of passwords. Users are often responsible for the quality of their own passwords. They typically choose easy to remember and thus easy to guess passwords and use the same phrases across multiple devices and accounts. A comparison of two dumps of hacked passwords found that, between common users in the lists, nearly 50% had identical usernames and passwords on both sites (Honan 2012). This illustrates that users will opt 9

16 for the path of least resistance with their passwords and choose easier passwords at the cost of security. Mobile devices also present a unique set of problems that make alphanumeric passwords easily compromised. One known vulnerability of knowledge-based passwords is shoulder surfing, where another person in the vicinity can see the user enter the password (Kim et al 2010). Mobile users enter text about 30% slower on their phones than on a keyboard (Findlater et al 2011) and choose shorter text passwords to compensate. Taken together, these two factors make mobile passwords a prime target for shoulder surfing. This coupled with the fact that many users use automated timed locks for their phone to increase security means that they are entering their passwords more often and are more likely to become a victim of shoulder surfing. Another unique factor in mobile phone password systems is the infamous smudge attack, where finger oils or residue left on the screen can give away or provide information on a PIN or swipe password (Aviv et al 2010). Given that many mobile users keep not only personal but also financial and business data on their phones, these security concerns are reason alone to search for alternatives to traditional knowledge-based passwords for touchscreen environment. Areas of Knowledge-Based Authentication Research Although traditional methods of knowledge-based authentication systems have some increasingly pressing security vulnerabilities, research in improving these systems is ongoing. Some scholarly efforts have overcome the vulnerabilities of traditional alphanumeric passwords, and effort in the arena of introducing new passphrases and identifiers has proven successful. 10

17 Graphical Passwords Graphical password systems rely on human s capacity for image recognition and memory recall to implement an alternative to text passwords (Shoemaker and Inkpen 2001). These systems require that the user identify secret images out of a set of decoys, and this process repeats several times to fully authenticate the user. One such application leverages the human ability to quickly recall and identify faces to accelerate response time (Passfaces 2013); another relies on a set of images inlaid on a grid (Dunphy and Yan 2007). Graphical passwords are good at preventing shoulder surfing and password snooping, as the location and generation of the correct images is randomized. However, time to authentication for graphical password systems is much higher than traditional approaches (some approaches take over a minute) and can become tedious for the user (Suo et al 2005). Even though graphical passwords are more resistant to brute force techniques, they are still weaker than other options to shoulder surfing and hacking as they suffer from similar downfalls as alphanumeric passwords. Biometric Password Research In contrast to the something you know password systems is the something you are system of biometric authentication and verification. The premise of this system verifies an inherent biological trait that is not memorized or recalled like a traditional password to determine identity. Familiar systems that use biometric identification systems in use already include fingerprint scanners and DNA tests (Jain et al 2006). While these systems can be more cumbersome to enact and use by the average consumer, advances in computational performance 11

18 have made biometric authentication worthy of consideration for mobile technologies and not just the implementation of science fiction writers. Biometric security can be divided into two subfields, each focused on a type of trait about the individual inputting data. Physiological biometrics uses data about the conditions of people s bodies to identify users, whereas behavioral biometrics analyze patterns of movement, location, and similar data to authenticate into a system (De Luca et al 2012). Examples of Physiological Biometric Research Facial Recognition Facial detection takes a photo of the frontal face and runs image detection algorithms on the face to take measurements that are compared against the valid user (Hjelmas and Low 2001). Facial recognition is generally accurate for most users and the algorithm to analyze the images runs very quickly. This system has robust research applications in alternative contexts, like targeted advertising in public places and security identification. However, facial recognition works poorly in many situations, such as non-frontal angles of the face to the camera phone, unique, or dark lighting. It has also had problems in the past identify users with darker complexions perceived as having less contrast by the recognitions system (Kawamoto 2010). Fingerprint Readers Fingerprint readers and scanners are one of the more familiar forms of physiological biometric recognition, as many people have had their fingerprints taken for identification purposes. These systems work by scanning a fingerprint or a set of prints and comparing many unique points and distortions of the image of the print to a stored record (Jain and Feng 2011). 12

19 The introduction of fingerprint scanners to the marketplace started with laptop and personal computers with swipe scanners and has only begun to move into the mobile device market. The new iphone5s includes the Touch ID system that analyzes a user s fingerprint for unlocking their phones and making itunes purchases (Dickey 2013). The fingerprint system is built natively into the device and quickly authenticates many users for these kinds of transactions. However, Touch ID has issues that reflect problems with the use of fingerprint scanners more broadly. The additional hardware component of the scanner means that it is more likely to be damaged; in addition, hackers cracked the checking algorithm just two days after the device hit the market (Arthur 2013). Capacitive touchscreen technology cannot read the fine details of a user s fingerprint and it is unlikely that the two will be integrated in the near future. Areas of Behavioral Biometric Research Location/Continuous Authentication Continuous authentication is one of the latest identification techniques as it relies entirely on an individual s location behaviors during the day. This form of authentication presumes that a user does not modify their weekly routine and is often in similar places at similar times of the day (Shi et al 2011). The phone keeps track of the GPS location, activity on the phone, and based on a user s patterns of usage automatically and continuously unlocks the phone for the user. Continuous authentication actively authenticates users regardless of the actions a user takes with their phone and avoids the constant log-in screen that many other password systems require. This approach works well for those with strong patterns to their lives and is incredibly convenient for continuous authentications. However, Shi et al note that this system is likely to be compromised 13

20 by nosy adversaries (spouses, children, and co-workers) and does not protect against someone stealing the device while a user is engaged in their daily rituals of working or commuting. Moreover, deviations from phone usage and location patterns could lock the phone down similar to strange charges will lock a credit card when made in a new city. Similar to facial recognition, users may be bothered by continuous monitoring of their movement patterns and be unwilling to adopt this system because they find it creepy. Shi et al suggest this type of authentication system as a secondary, underlying approach done in conjunction with other forms of verification. Gesture Recognition Gesture recognition is one of the up-and-coming forms of behavioral biometric recognition, as its application crosses from mobile technology to fields like gaming (the Microsoft Kinect), security and video footage (surveillance footage), and robotics. Gesture recognition assumes that the patterns that humans take to make a body gesture are unique and larger inferences can be deduced from these movements (Mitra and Acharya 2007). Applications of this technology to touchscreen devices ask users to make a symbol of motion using finger movements on a touchscreen that is then analyzed against a trained data set of true positives (de Luca et al 2012; Sae-Bae et al 2012; Shahzad et al 2013). The advantage of this technology is that no additional hardware is required and gesture recognition can happen within a few seconds given the processing power on most mobile devices. Gestures are also easy to remember and accurate authentication has been achieved with symbols as simple as an infinity sign. However, the best gesture authentication uses supervised machine learning techniques in the background and requires a training period of several days to a week to be accurate. 14

21 Gesture passwords provide a robust set of opportunities for exploration into the authentication space. They are quick to draw and complete, meaning that access to devices can be granted quickly and without repeated log-in attempts. Preliminary research suggests that gesture passwords are distinguishing and can be used for security systems. Finally, using gestures for a password mimic the natural interaction patterns that people have with touch screen devices. Current Research on Gesture Authentication Systems Despite their potential for authentication, gesture passwords are a surprisingly underexplored area in biometric password security. Only three publications have researched the potential and applications of touch gestures, and only in the area of mobile and tablet touchscreens. These studies are also incredibly recent, only happening since Although these studies are not perfect examples of a study involving the PixelSense table, their research methodologies and user studies offer insight into how a gesture password system can be applied to the PixelSense table. The most cited of the proceedings, De Luca et al presented a gesture recognition system that improved on the Android swipe-to-unlock and dot password system (De Luca et al 2012). In both their iterations, the authors measured differences in the time and location of gestures on the screen across different participants. The authors opted to use dynamic time warping, a technique that measures the similarity of two or more sequences of temporal data in time, speed, and location. They found that, while imposing dynamic time warping was only somewhat accurate for swipe-to-unlock systems (between 50-70%), accuracy for the dot password system was much 15

22 higher (~95% for the best three users). De Luca et al s study gave the first indication that touchscreen gestures were distinct enough between users to be used as a password system on touchscreen devices. Presented at the same time as De Luca et al s work, Sae-Bae et al also confirmed that touch gestures were distinct between touchscreen users, this time using an ipad and multi-touch gestures (Sae-Bae et al 2012). Rather than rely on single touch gestures, the authors combined five finger multi-touch gestures with dynamic time warping to create a password system. One of their unique contributions to the literature was their proposal and user testing of many kinds of gestures. Rather than recreating the dot password system on Android, Sae-Bae et al tested twenty two multi-finger gestures and found that different gestures produced differing levels of accuracy for distinction (between 80-90%). Shahzad et al implemented the ideas of the previous studies onto a mobile phone interface and was the first to incorporate machine learning into their authentication system. First, they directly extended the work of De Luca et al and Sae-Bae et al to implement a gesture recognition system on a Windows phone. What is unique about their system was that they chose not to impose gesture recognition onto common passwords systems that exist already. Rather, they created an open gesture input system where users were prompted to draw their gestures without another password system to conform to. Although they limited their gestures to the ten most effective in the study, Shahzad et al prompted users to interact with the device in unique ways. Second, and more importantly for gesture password systems, their implementation incorporated a support vector distribution estimation, a type of machine learning implementation. 16

23 From this system, they proposed a set of seven features to be used with multi-touch and single touch gestures which classified a user from an intruder with an equal error rate of 0.5%. Although these three works make good progress in developing a gesture authentication system, there are a few limitations to their research. First, all of the authors gather data through capacitive touchscreens. Capacitive touchscreens are widely deployed on mobile touchscreen devices, but their data output is only expressed through the x-y coordinate plane. This means that other relevant information that could be descriptive of a touch gesture, like finger width or height, is left out. Because of this, the feature set is limited to data that can be calculated from coordinate data. This leaves out a rich dataset that could be measured, like finger pressure, size, or orientation. Another problem of capacitive touch screens is that they are prone to noisy data collection. Fingers are large and clumsy for inputting precise data and, as a result, they touch groups of circuits on the capacitive touch screen. These large areas of touch produce lots of erratic data under the finger pad as different circuits are registered as the touch itself. Noisy data necessitates preprocessing, as done by Shahzad et al for their application, and some data will be lost with each processing iteration. Finally, these papers are limited in their application to current uses of touchscreen and do not address the potential for the commercialization of more robust touchscreens to consumers in the near future. Despite these problems, these three papers are the only ones in the field that address both gesture recognition and their applications to authentication systems on touchscreens. More research is needed to explore the validity of gesture password systems. 17

24 Machine Learning Another key component of this study is building an authentication system that can recognize and distinguish authorized users from non-authorized users. Most modern systems that attempt to accomplish this goal incorporate machine learning into their designs. Machine learning is a discipline within computer science that focuses on building models that can learn from data. The field combines aspects of statistics, artificial intelligence, data mining, text analysis, and other fields to produce models that, once generated, can predict outcomes on unexamined data sets. Tom Mitchell, a pioneer in the field, defined machine learning as "A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E" (Mitchell 1997). Many kinds of machine learning algorithms exist to fit different kinds of data to unique models. Two of the main distinctions in machine learning are between unsupervised and supervised learning. Unsupervised learning takes unlabeled, unorganized data and attempts to find hidden meaning and trends (Duda et al 2012). Supervised machine learning, on the other hand, uses labeled data to generate labeled predictions and classifications. Said in a different way, unsupervised learning is like clustering into unnamed piles whereas supervised learning gives names and meanings to those piles as specified by the researcher. A support vector machine (SVM) is a supervised machine learning algorithm that builds a function that classifies data into two piles, or classes. Supervised machine learning algorithms rely on a set of training data that is labeled and classified. An SVM classifies by creating a function, called a hyperplane, that divides the data based on how well it can organize the training 18

25 set into distinctive groups. SVMs draw this hyperplane by calculating the maximum margin between the training sets of data. The points that are closest to each other are what define this margin. Features are the distinctive attributes of a data set that the function uses to group the data together and are calculated and extracted from the data itself. SVMs allow a functionally unlimited number of features to be included because the learning algorithms computes based on matrix dot products within the vector space (Duda et al 2012; Cortes and Vapnik 1995). Dot products calculate the products across multiple features, replacing the laborious process of comparing Figure 1 - This example of an SVM shows how the hyperplane (or function line) is positioned at the maximum margin between the two training sets. The data points that create the hyperplane are known as support vectors. (Image taken from OpenCV Development, 2014). against every individual coordinate. What this means is rather than model the feature set of any given data set on a coordinate plane and do math based on the coordinates, matrix math replace these calculations, increasing the speed and efficiency of the model. Other kinds of machine learning algorithms could be applied to this task, but the SVM provide several benefits that other models do not. Several kinds of machine learning techniques have been applied to image recognition problems similar to gesture recognition, like neural networks, hidden Markov models, and decision trees. Although the PixelSense table uses image processing to calculate the location of 19

26 users touches on the table, applying these same techniques to the problem of gesture authentication has some drawbacks. An authentication system benefits from having a robust and expansive set of features to offer more potential points of security and identification. Even though an SVM cannot expand and create its own features like a hidden Markov model that investigates the data, SVMs can accommodate a functionally infinite number of features (limited only by processing power). More features generally means a more robust and (potentially) accurate model, so a higher feature set is desirable in many applications of research, especially ones concerned with security. SVMs can also be transformed to have non-linear classification systems. This means that, rather than draw a single line, plane, or n-dimensional divider through the data, the hyperplane can be whatever curve is best suited to separating the data. What these two advantages means in practical terms is that SVMs are very expansive and useful for unique problems and can include many kinds of data to create a decision-making system uniquely tailored to the scope of the problem. Summary A good password system must balance the need for security as well as usability on a touchscreen device. Current authentication systems that use alphanumeric passwords are limited, especially as they are used on touchscreen devices. Research in the area of new authentication systems is apt for scholarly discovery, but many of these systems fail to balance usability in a variety of circumstances. Gesture passwords, on the other hand, are a type of password that is quick to input on a touchscreen, distinctive between users, and easy to remember as a single symbols or pattern to 20

27 draw. Only three research teams have previously explored the area of gesture passwords and only one has incorporated machine learning into their models. SVMs in particular are a good candidate for my research because they are easy to implement, easy to expand with a robust set of features, and easily tuned to a variety of circumstances. The literature review suggests the following: gestures are a strong contender as a replacement for traditional alphanumeric passwords; a gap exists in the knowledge of gesture passwords; and incorporating a machine learning approach into the design of a gesture authentication system would ameliorate some of the inaccuracies of the previous research in this topic area. Passwipe hopes to fill this knowledge gap by creating an application that uses an SVM to distinguish between users gestures on a touchscreen system. 21

28 CHAPTER 3 IMPLEMENTATION AND METHODS Review of the current literature suggests ample opportunity to build an authentication system on a touchscreen surface, and that SVMs would be a good starting point for applied machine learning to this problem. The PixelSense table provides several unique advantages to traditional capacitive touchscreen displays. The unique touchscreen of the PixelSense table allows richer raw data to be extracted and, correspondingly, more features and gestures can be tested across different users. Derived from these broad research ideas, I hope to accomplish two goals with Passwipe and the accompanying data analysis. First, the project is a confirmation of the work done by the three previously mentioned sister studies (see Chapter 2) on a new touchscreen device specifically the Microsoft PixelSense table. Passwipe not only offers confirmation of these studies but also expands the research by testing new features and gestures on a different type of touchscreen. Second, I gathered user data about the authentication system and the corresponding SVM on the PixelSense table. Data collection on user interaction was necessary to test not only the distinctiveness of specific gestures but also whether the system would produce an SVM that could distinguish between users. This data should illustrate the diverse methods of drawing gestures and illustrates the potential for more robust learning algorithms to be applied in the future to authentication systems. 22

29 Implementation Specifications Passwipe is based on features extracted from a user s input of a gesture captured by the PixelSense table and its camera system. From these gestures, features are extracted that describe the gesture more abstractly, like gesture velocity and finger variance. An SVM is built from these features derived from sets of training data given through a data capture program. The proof-of-concept program implements this SVM and classifies between two people. There are eighteen features that characterize the data. For my study, I used the Microsoft PixelSense 1.0 table, formerly called the Microsoft Surface. The PixelSense table contains an integrated PC running Windows Vista 32-bit and a with a 30-inch screen at a 1024 x 768 resolution. One of the unique features of the table is its use of five near-infrared (IR) cameras rather than a capacitive touchscreen. These cameras record movement at approximately 60 times per second and can see fingers, hands, objects, and tags placed on the horizontal touch display. The cameras can track up to 52 unique touch points on the table ( Hardware Overview 2009). Passwipe used the software development kit (SDK) called Microsoft Surface SDK 1.0 for interacting with the table. Anatomy of a Gesture Gestures are the main interaction system for touchscreen devices. Broadly, gestures are body motions that convey meaning. For my thesis and for touchscreens, gestures are an aggregated set of finger movements on a touchscreen that indicates a specific actions or intention. Examples like swipe to scroll and pinch to zoom are some of the more common gestures on touchscreen devices like tablets and mobile phones. These kinds of gestures compose 23

30 the majority of user interaction with a touchscreen. What is unique about touchscreen gestures is that they are user-driven and discreet units of meaning, whereas other forms of biometric password systems do not require such explicit interaction with the device. A user confirms that they want to zoom into a picture by pinching the corners of the photo and dragging their fingers together; likewise, a user could make a gesture that indicates that they want to unlock their device. On the PixelSense table, gestures can be single or multi-touch, but for my study, I focus on single touch gestures. For this study, a single finger is used to create a gesture and a gesture consists of the actions recorded while the finger is touching the tabletop. Single finger interactions represent the majority of interaction points users have with touchscreens and also discourage noise from entering the dataset as the input mechanism is a single point of contact users don t have to worry about the positions of multiple fingers on a display while also focusing on completing an authentication gesture. The lift-off indicates that a user is finished with their gesture to the system and that the user is ready for the system to respond to their actions. Much like users hit Enter to submit their passwords, a similar submission system is necessary for a touch gesture password system to indicate that the password is indeed finished. Another check implemented is a minimum timeframe to register a full gesture. If a gesture took less time than the minimum, it was invalid and the data was discarded. This number varied depending on the kind of gesture being implemented a single loop, for example, had a minimum timeframe of 400 milliseconds and while an infinity sign had a minimum timeframe of 600 milliseconds. The PixelSense table will, on occasion, deregister a touch and mark it as removed from the table while the user is in the process of completing their gesture. This 24

31 deregistration is caused by a flaw in the system architecture of the Microsoft Surface SDK and the methods it uses to handle event registration. I found that this registration error would occur midway through completing a gesture and so I set this minimum to be slightly longer than half the average total time for a specific gesture. Each gesture is broken into individual data points called touches that measure specific characteristics of a user s finger interactions. A gesture contains touches according to its complexity and the length of time of the gesture. Touches on the PixelSense table include the following raw data: x-y coordinates, major and minor axis length of the finger touch made on the table, and Figure 2 - An abstract example of the raw data the PixelSense table provides. Each touch contains this data that is accessible through the SDK. orientation in radians relative to the positive x-axis of the program (see Figure 2). A touch is recorded roughly every 14 milliseconds and the timestamp for these touches are included in the raw data from the start of the gesture. For this program, I chose three gestures: a loop, an infinity sign, and an at sign (Figure 3). These three gestures were chosen because they provided the smallest amount of gesture recognition errors and dropped contacts in early testing. These were also chosen in the hopes that they would be more memorable and more closely replicate an actual authentication environment deployed in an actual authentication environment. 25

32 Figure 3: Examples of the touches used in my study. Building a Feature Set After a user has completed a gesture, the raw data is processed and the features are calculated. In deciding what features to include, I tried to create representations of what I thought would be distinctive characteristics of a gesture, such as how long their gestures took or how much their finger angle changed while completing the gesture. Here are the 18 features included in the model: 26

33 Gesture Total gesture time X-Y ratio Average X velocity Average Y velocity Substroke velocity magnitude Substroke velocity direction Average major finger width Average minor finger width Major finger width variance Minor finger width variance Average finger orientation Finger orientation variance Description Time taken to complete gesture Ratio of the total width of gesture on the x-axis to total height of gesture on y-axis Average velocity magnitude along x-axis Average velocity magnitude along y-axis The gesture is divided into four subsections and the velocity vector of each substroke subsection is determined. The magnitude of that velocity vector is recorded. (counts as four features) The gesture is divided into four subsections and the velocity vector of each subsection is determined. The direction of that velocity vector is recorded (in degrees) (counts as four features). Across the major axis created by a finger, average the length of the major axis across total gesture Across the minor axis created by a finger, average the length of the minor axis across total gesture Variance across major axis across total gesture Variance across minor axis across total gesture Across entire gesture, measure the average orientation/angle of the finger (in radians) Variance across finger orientation Table 1 - The list of the eighteen features included in the SVM and a description of their implementation. In addition to including metrics that summarized the total gesture, I also included eight features that described movement patterns across particular portions, or substrokes, of the gesture. This idea was taken from Shahzad et al s work, where they implemented the system to analyze a different set of gestures. Other features replicated some gestures from Shahzad et al, including total gesture time and overall velocity. PixelSense-specific gestures calculated from finger width, height, and finger orientation were created by me. 27