Human-Computer Interaction for Biometrics

Transcription

1 Human-Computer Interaction for Biometrics Prof. Julian FIERREZ Universidad Autonoma de Madrid - SPAIN Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 1 / 50 Funding Acknowledgements Public Private Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 2 / 50 1

2 Outline 1. Biometrics in Access Control Scenarios 2. Authentication in On-Line Applications 3. Biometrics based on HCI Handwriting and Signature Graphical Passwords Swipe Biometrics Mouse Dynamics Keystroke Dynamics 5. Future Trends & Conclusions Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 3 / 50 Biometrics in Access Control Scenarios Biometric access control deployments have concentrated much efforts and resources in the last 15 years Led the market of biometric authentication technologies In those scenarios, the user is physically present (in-situ) No specific actions from the user are needed (other than placement) Once checked, access granted / denied If denied, human supervision as an alternative Physiological (morphological) modalities are in general better adapted to these type of tasks Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 4 / 50 2

3 Human Activity Patterns Human activity patterns are clearly stablished from childhood As patterns, they are stable and reproducible, though subject to variability Neuromotor coordination of gestures, movements and speech Continuous identity monitoring possible User is an active part of the play Multilevel strategy: from dynamic trajectories to expressions, context, habits, stylometry, experiences Not fixed patterns but changing and adapting ones Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 5 / 50 Human-Computer Interaction for Biometrics User is monitored / connected at least for several minutes Not only security checking but user convenience scenarios Human-device interaction (HDI) is rich in terms of human activity and behavioral patterns Keystroking Mouse dynamics On-Line Handwritting & Signature Graphical Passwords Swipe and Gesture Biometrics Speech Behavioral biometrics, but not neuromotor HDI Gait Not online HDI (just sensing) for authentication Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 6 / 50 3

4 Biometric Market by Modality Decreasing: Fingerprint, from 48% to 15% (18% excluding AFIS) Growing: Iris from 9% to 16% (19%) and Face from 12% to 15% (18%) Huge growing: Speech from 6% to 13% (15,5%) and Signature, from 2% to 10% (12%) Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 7 / 50 Active Authentication by DARPA Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 8 / 50 4

5 x y z 11/04/2018 On-line Signature Verification azimuth altitude sample index Dynamic signature matching J. Fierrez and J. Ortega-Garcia, "On-line signature verification", A.K. Jain et al. (Eds), Handbook of Biometrics, Feature-based (Global Features) Distance-based classifiers Mahalanobis Euclidean [Nelson et al., 1994] Statistical/other classifiers Gaussian Mixture Models (GMM) Parzen Windows Function-based (Local Features) M. Martinez-Diaz and J. Fierrez, "Signature Databases and Evaluation", Stan Z. Li and Anil K. Jain (Eds.), Encyclopedia of Biometrics, Springer, pp. Julian , Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 9 / 50 Time-Sequence matching techniques Hidden Markov Models (HMM) [Dolfing et al., 1998] Gaussian Mixture Models (GMM) [Richiardi et al., 2005] Dynamic Time Warping (DTW) [Sato and Kogure, 1982] Signature Acquisition and Pre-processing Key step to greatly reduce sensor interoperability issues due to heterogeneous devices and writing tools (stylus/finger) Preprocessing Size normalization and centering - Pressure normalization - Resampling M. Martinez-Diaz, J. Fierrez and S. Hangai, "Signature Features", Stan Z. Li and Anil K. Jain (Eds.), Encyclopedia of Biometrics, Springer, pp , Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 10 / 50 5

6 Similarity Computation Hidden Markov Models (HMM) Statistical modeling of signature regions Dynamic Time Warping (DTW) Point-to-point correspondence M. Martinez-Diaz, J. Fierrez and S. Hangai, "Signature Matching", Stan Z. Li and Anil K. Jain (Eds.), Encyclopedia Julian of Fierrez Biometrics, Seminar at Springer, CIMAT, Guanajuato, pp , MEXICO April Slide 11 / 50 Use of Recurrent Neural Networks Deep Neural Network- Based System accomplishes: Linear Transformations Non-Linear Transformations (tangh, sigmoid, ReLU, etc) -> θ INPUT θ θ OUTPUT θ θ OUTPUT θ θ θ θ J. Schmidhuber, Deep Learning in Neural Networks: An Overview, Neural Networks, vol. 61, pp , Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 12 / 50 6

7 x y z 11/04/2018 Use of Recurrent Neural Networks Recurrent Neural Networks, well adapted to time sequences (speech, handwritting ) Different topologies: Long Short-Term Memory (LSTM) Gated Recurrent Unit (GRU) Also bidirectional versions: BLSTM y BGRU Siamese arquitecture Memory! Time Sequence A. Graves, A.R. Mohamed, and G. Hilton, Towards End-to-End Speech Recognition with Recurrent Neural Networks, in Proc. International Conference on Machine Learning, vol. 14, pp , R. Tolosana, R. Vera-Rodriguez, J. Fierrez and J. Ortega-Garcia, "Exploring Recurrent Neural Networks for On-Line Handwritten Signature Biometrics", IEEE Access, Vol. 6, pp , February Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 13 / 50 Traditional Acquisition Scenario ( ) Altitude (0-90 ) Azimuth (0-359 ) 0 M. Martinez-Diaz and J. Fierrez, "Signature Databases and Evaluation", Stan Z. Li and Anil K. Jain (Eds.), Encyclopedia of Biometrics, Springer, pp , Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 14 / 50 azimuth altitude sample index 7

8 False Rejection Rate (%) 11/04/2018 Benchmarks: SVC 2004 SVC-04 random (zero-effort, casual) impostors SVC-04 skilled forgeries Our HMM implementation Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 15 / 50 BioSec Signature Evaluation Campaign, BSEC 09 DTW, HMM and Global Systems Score normalization Fusion of systems False Acceptance Rate (%) N. Houmani, et al., "BioSecure signature evaluation campaign (BSEC2009): Evaluating online signature algorithms depending onthe quality of signatures", Pattern Recognition, March Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 16 / 50 8

9 BIOTRACE100 Performance (2015) Accuracy (SLT Database): Random Forg. Skilled Forg. 4 training signatures 16 signatures 31 signatures 41 signatures 97.2 % 99.3 % 99.9 % 99.9 % 88.3 % 93.1 % 95.9 % 99.3 % State of the art performance Template and system configuration update strategies in order to minimize the aging effect R. Tolosana, R. Vera-Rodriguez, J. Ortega-Garcia and J. Fierrez, "Preprocessing and Feature Selection for Improved Sensor Interoperability in Online Biometric Signature Verification", IEEE Access, Vol. 3, pp , May Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 17 / 50 e-biosign Database ( ) 70 users, 2 capturing sessions 5 devices (3 Wacom, 2 Samsung) 8 genuine signatures and 6 skilled forgeries per user and device Stylus and finger as writing tools (Samsung) R. Tolosana, R. Vera-Rodriguez, J. Fierrez, A. Morales, J. Ortega-Garcia, Benchmarking Desktop and Mobile Handwriting across COTS Devices: the e-biosign Biometric Database PLOS ONE, Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 18 / 50 9

10 Challenges on Handheld Devices Lack of space [Simsons et al., 2011] Stylus (or finger) Sampling quality Ergonomics [Blanco-Gonzalo et al., 2013b] Lack of pressure and orientation signals [Muramatsu and Matsumoto, 2007] Higher client-entropy [Garcia-Salicetti et al., 2008] Standing position Lack of in-air trajectories [Sesa-Nogueras et al., 2012] F. Alonso-Fernandez, J. Fierrez and J. Ortega-Garcia, "Quality Measures in Biometric Systems", IEEE Sec. & Privacy, Dec Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 19 / 50 From Signature to Touch Gestures Graphical Password-based User Authentication with Free-form Doodles M. Martinez-Diaz, J. Fierrez and J. Galbally, "Graphical Password-based User Authentication with Free-Form Doodles", IEEE Trans. on Human-Machine Systems, August M. Martinez-Diaz, J. Fierrez, and J. Galbally. The DooDB graphical password database: Data analysis and benchmark results. Julian IEEE Fierrez Access, Seminar September at CIMAT, Guanajuato, MEXICO April 2018 Slide 20 / 50 10

11 Graphical Passwords Gesture-based authentication on touch-screens Slow typing in touchscreens Biometric-rich gestures Revocability Physiological Biometrics Behavioral Biometrics Graphical Passwords M. Martinez-Diaz, J. Fierrez and J. Galbally, "The DooDB Graphical Password Database: Data Analysis and Benchmark Results", IEEE Access, Sept Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 21 / 50 Graphical Passwords: Related Works 1 2 (2,2) (3,2) (3,3) (2,3) (2,2) (2,1) Draw a Secret [Jermyn et al., 1999] US Patent B2 Pass-Go [Tao et al., 2008] Pattern Lock [Google] US Patent A1 Picture Gesture Authentication [Microsoft] US Patent A1 Multi-touch gestures [Sae-Bae et al., 2012] US Patent A1 Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 22 / 50 11

12 Graphical Examples Doodles Pseudo-signatures Genuine samples Forgeries Genuine samples Forgeries M. Martinez-Diaz, J. Fierrez and J. Galbally, "The DooDB Graphical Password Database: Data Analysis and Benchmark Results", IEEE Access, Sept Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 23 / 50 Current Work: Swipe Biometrics Continuous user authentication through touch biometrics: - Security beyond the entry-point Situation: - Freely interacting with the touchscreen while reading or viewing images Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 24 / 50 12

13 Session 2, test set High inter-user variability: distinctiveness High intra-user variability: Difficult to model the user USER A USER B Example of strokes captured for two different users Session 1, training set Abdul Serwadda, Vir V. Phoha, and Zibo Wang. Which verifiers work?: A benchmark evaluation of touch-based authentication algorithms. In 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS), pages 1 8, Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 25 / 50 Current Work: Swipe Biometrics A. Pozo, J. Fierrez, M. Martinez-Diaz, J. Galbally and A. Morales, "Exploring a Statistical Method for Touchscreen Swipe Biometrics", in Proc. Julian Fierrez Intl. Carnahan Seminar at Conference CIMAT, Guanajuato, on Security MEXICO Technology, April 2018 Slide ICCST 26 / , October

14 Distance (pixels) 11/04/2018 Current Work: Mouse Dynamics Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 27 / 50 Mouse Dynamics Mouse dynamics analyze behavior pattern of users for classification, verification and identifications. Mouse dynamics is a behavioral biometric characteristic Intra-class variability (example) Sample 1 Sample 2 Challenges High error rates High intra-class variability Advantages No intrusive Easy acquisition Continuous monitoring Distance (pixels) Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 28 / 50 14

15 Mouse Dynamics: Features S. Chao, C. Zhongmin, G. Xiaohong and R. Maxion, Performance evaluation of anomaly-detection algorithms for mouse dynamics, Computers and Security, vol. 45, pp , Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 29 / 50 Mouse Dynamics: Performance S. Chao, C. Zhongmin, G. Xiaohong and R. Maxion, Performance evaluation of anomaly-detection algorithms for mouse dynamics, Computers and Security, vol. 45, pp , Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 30 / 50 15

16 Keystroke Dynamics Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 31 / 50 Keystroke Dynamics - History Telegraphist style during World War II Keystroke dynamics authentication emerges with the massive deployment of computers Transparent, low-cost solution Easy to implement in any keypad device R. S. Gaines, W. Lisowski, S. J. Press, and N. Shapiro, Authentication by keystroke timing: Some preliminary results, tech. rep., DTIC Document, Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 32 / 50 16

17 Keystroke Dynamics - History Telegraphist style during World War II Keystroke dynamics authentication emerges with the massive deployment of computers Transparent, low-cost solution Easy to implement in any keypad device Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 33 / 50 Keystroke Dynamics: Scenarios Mobile Tablet Traditional Keyboard Text-dependent / passwords In a village of La Mancha, the name of which I have no desire to call to mind, there lived not long since one of those gentlemen that keep a lance in the lance-rack, an old buckler, a lean hack, and a greyhound for coursing Text-independent Buschek D, De Luca A, Alt F Improving accuracy, applicability and usability of keystroke biometrics on mobile touchscreen devices. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI 15. Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 34 / 50 17

18 From Password-Based to Free-Text Authentication Substitute password-based authentication + keystroke dynamics Biometric system as a secondary security level to detect intruders Free text usually divided into small strings (digraphs and trigraphs) List of passwords Biometric Templates DDBB DDBB Password Match? Keystroke Biometric Accept/Denied STEP 1 STEP 2 Text level Biometric level Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 35 / 50 From Password-Based to Free-Text Authentication Substitute password-based authentication + keystroke dynamics Biometric system as a secondary security level to detect intruders Free text usually divided into small strings (digraphs and trigraphs) Biometric Templates DDBB Free Text Keystroke Biometric Accept/Denied Matching at biometric feature level Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 36 / 50 18

19 Keystroke Verification: Sequence Alignment Identity claim Input sample Keystroke Verification Enrolled models Preprocessing Feature Extraction Similarity Computation Score normalization Decision Threshold Accepted or rejected Sequence alignment through DTW M. L. Ali, J. V. Monaco, C. C. Tappert, and M. Qiu. "Keystroke biometric systems for user authentication." Journal of Signal Processing Systems 86, no. 2-3 (2017): Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 37 / 50 Keystroke Verification: Features Identity claim Input sample Keystroke Verification Enrolled models Preprocessing Feature Extraction Similarity Computation Score normalization Decision Threshold Accepted or rejected A. Morales, J. Fierrez and J. Ortega-Garcia, "Towards Predicting Good Users for Biometric Recognition based on Keystroke Dynamics", Proc. of European Conference on Computer Vision Workshops, Springer LNCS-8926, Sept A. Morales, J. Fierrez, R. Tolosana, J. Ortega-Garcia, J. Galbally, M. Gomez-Barrero, A. Anjos and S. Marcel, "Keystroke Biometrics Ongoing Competition", IEEE Access, Vol. 4, Nov Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 38 / 50 19

20 Keystroke Verification: Similarity Computation Identity claim Input sample Keystroke Verification Enrolled models Preprocessing Feature Extraction Similarity Computation Score normalization Decision Threshold Accepted or rejected Classifiers based of algorithmic fusion (GMM, SVM, normalized Manhattan distance ) Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 39 / 50 Keystroke Dynamics: Performance and Benchmark Performance, strongly user-dependent and scenario-dependent. Password Authentication (CMU benchmark, 2012) : 8% EER (only genuine samples during training) Free-Text, ICB 2015 Competition: 83% Identification Rate (100 characters as query samples, 500 characters as development samples) New Benchmark: Keystroke Biometrics Ongoing Competition* J. V. Monaco, G. Perez, C. C. Tappert, P. Bours, S. Mondal, S. Rajkumar, A. Morales, J. Fierrez and J. Ortega-Garcia, "One-handed Keystroke Biometric Identification Competition", in Proc. IEEE/IAPR Int. Conf. on Biometrics, ICB, May *A. Morales, J. Fierrez, R. Tolosana, J. Ortega-Garcia, J. Galbally, M. Gomez-Barrero, A. Anjos and S. Marcel, "Keystroke Biometrics Ongoing Competition", IEEE Access, Vol. 4, pp , November Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 40 / 50 20

21 Biometrics Research: A Look into the Future Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 41 / 50 The Future of Biometrics based on HCI Challenge 1: Adapting to New Application Scenarios Knowledge Base + Experiments + Experiments - Domain adaptation - Transfer learning - Inductive transfer -... Bayesian adaptation* Discriminative adaptation** * J. Fierrez-Aguilar, D. Garcia-Romero, J. Ortega-Garcia and J. Gonzalez-Rodriguez, "Bayesian adaptation for user-dependent multimodal biometric authentication", Pattern Recognition, August **J. Fierrez-Aguilar, D. Garcia-Romero, J. Ortega-Garcia and J. Gonzalez-Rodriguez, "Adapted user-dependent multimodal biometric authentication exploiting general Julian Fierrez information", Seminar Pattern CIMAT, Recognition Guanajuato, Letters, MEXICO December April Slide 42 / 50 21

22 The Future of Biometrics based on HCI Challenge 2: Incorporating Contextual Information Signal Quality, Environmental Data, etc. Knowledge Base P. Aleksic, M. Ghodsi, et al. Bringing Contextual Information to Google Speech Recognition, Interspeech, F. Alonso-Fernandez, J. Fierrez, et al, Quality Measures in Biometric Systems, IEEE Sec. & Privacy, Dec F. Alonso-Fernandez, J. Fierrez, et al., "Quality-Based Conditional Processing in Multi-Biometrics: application to Sensor Interoperability", IEEE Trans. on Systems, Man and Cybernetics A, Vol. 40, n. 6, pp , J. Fierrez, et al., "Multiple Julian Fierrez Classifiers Seminar in at Biometrics. CIMAT, Guanajuato, Part 2: MEXICO Trends and April Challenges", 2018 Slide 43 Information / 50 Fusion, Nov Q-based enhancement [Hong et al. 98] Q-based feature weighting [Chen et al. 05] Q-based fusion [Bigun et al. 97, 03] [Fierrez et al. 05, 06] [Nandakumar et al. 06, 08] Failure to acquire event [Simon-Zorita et al. 03] [Chen et al. 05] J. Fierrez-Aguilar, J. Ortega-Garcia, J. Gonzalez-Rodriguez and J. Bigun, "Discriminative multimodal biometric authentication based on quality measures", Pattern Recognition, May J. Fierrez, A. Morales, R. Vera-Rodriguez and D. Camacho, "Multiple Classifiers in Biometrics. Part 2: Trends and Challenges", Information Fusion, November Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 44 / 50 22

23 The Future of Biometrics based on HCI Challenge 3: Adapting to the User (e.g., Aging) User-specific behaviour Knowledge base J. Fierrez, J. Ortega-Garcia and J. Gonzalez-Rodriguez, "Target Dependent Score Normalization Techniques and their Application to Signature Verification", IEEE Trans. on Systems, Man and Cybernetics-C, August J. Galbally, M. Martinez-Diaz and J. Fierrez, "Aging in Biometrics: An Experimental Analysis on On-Line Signature", PLOS ONE, July J. Fierrez, A. Morales, R. Vera-Rodriguez and D. Camacho, "Multiple Classifiers in Biometrics. Part 2: Trends and Challenges", Information Fusion, Nov Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 45 / 50 e.g., good/bad users [Hicklin et al., The myth of goats: how many people have fingerprints that are hard to match, NISTIR 7271, 2005]. Auxiliary Information Obtained during Enrollment: Exploitation of the enrollment data (multiple samples) not only to create the templates/models but also to adjust in a userdependent way some parameters of the system during verification Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 46 / 50 23

24 Exploiting the Zoo (User-Dependent Processing) UD features [Fairhurst et al., IPRAI 94] UD modeling [Martinez et al., ICFHR 08] UD score normalization [Fierrez et al., TSMC 05] [Poh et al., MMUA 06, TASLP 08] UD fusion [Jain et al., ICIP 02] [Toh et al., TSP 04] [Snelick et al., PAMI 05] [Fierrez et al., PR 05] UD decision (e.g., UD tresholds [Jain et al., PR 02], failure to enroll events and exception handling) J. Fierrez, A. Morales, R. Vera-Rodriguez and D. Camacho, "Multiple Classifiers in Biometrics. Part 2: Trends and Challenges", Information Fusion, Nov Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 47 / 50 GLOBAL: Set of training scores from a pool of users (genuine and impostor) LOCAL: Set of training scores from the user at hand (genuine and impostor) Bayesian and SVM user-dependent fusion algorithms J. Fierrez, A. Morales, R. Vera- Rodriguez and D. Camacho, "Multiple Classifiers in Biometrics. Part 2: Trends and Challenges", Information Fusion, Nov Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 48 / 50 24

25 11/04/2018 The Future of Biometrics based on HCI Challenge 4: Exploiting Big Data Ruben Tolosana, Ruben Vera, Julian Fierrez, Javier Ortega, Exploring Recurrent Neural Network for Handwriting Signature Biometrics IEEE Access, Anonymized info Big Data Signer 1 Deep Learning Signer N Knowledge Base Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 49 / 50 Conclusions Revocability Easy of use, user acceptance Less sensor-interoperability issues Easy to integrate at low-cost Continuous ID User intra-variability Multi-sample training Model updating Multilevel strategies Data scarcity Matureness of technologies (signature, keystroke) Major role in on-line and mobile remote applications User convenience to drive application development Room for substantial industry-applicable research Julian Fierrez Seminar at CIMAT, Guanajuato, MEXICO April 2018 Slide 50 / 50 25