A Survey of Synthetic Biometrics: Capabilities and Benefits

Transcription

1 Accepted for publication in The 2004 International Conference on Artificial Intelligence (IC-AI 04) A Survey of Synthetic Biometrics: Capabilities and Benefits Nicholas M. Orlans The MITRE Corporation 7515 Colshire Drive McLean, VA Douglas J. Buettner The Aerospace Corporation P.O. Box M1/112 Los Angeles, CA Joe Marques The MITRE Corporation 7515 Colshire Drive McLean, VA Abstract Within the Association for Computing Machinery (ACM) Special Interest Group on Computer Graphics (SIGGRAPH) community, a long-standing goal has been held for photo-realism in the generation of synthetic images a goal that some feel has been achieved [7]. This body of work, spanning over three decades, documents achievements in modeling, animation, and rendering human subjects. The visual products range from feature films, commercial art, to video games. Computer generated synthetic biometrics are not widely used within the biometrics community beyond their current use as a research tool. Yet they offer a number of potential advantages that can be developed further to support the science and practical use of biometrics. They can be used to improve the understanding of a biometric system s robustness and as an engineering tool to predict system performance. This paper surveys the state of synthetic biometrics generation, provides a glimpse at some benefits that can be obtained from their use, and discusses the issues retarding their adoption by the biometrics community. 1. Introduction This survey explores the current state of computer generated (synthetic) biometric images, and discusses areas where synthetic biometrics can provide added value to the biometrics community and ultimately to the public and private sector interests that it serves. Also discussed are the areas of research that need to be conducted before their possible wide spread adoption and use by the biometrics community. The most advanced synthetic images come from the movie industry and have yet to be fully adapted to suit biometric needs. A synthetically rendered face is illustrated in Figure Synthetic Biometrics Synthetic image generation, as shown in Table 1, has been achieved for the most widely recognized image-based biometrics of fingerprint, face, and iris [1, 2, 3, 4, 5, 6]. Table 1. Synthetic Biometric Data Generation Synthetic image generation Statistical feature model Validated model Environment effects Fingerprint Face Iris yes yes yes yes- level 2 minutiae yes - (method) yes (theoretical) no no no partial partial partial Figure 1. Rendering Of A Synthetic Face. 1 The quest for realistic computer-generated artificial-persons has created an art and science that includes physics-based models to control physical form, motion and illumination properties of materials [8]. Today, computer generated characteristics address detailed aspects of facial features, skin, hair, gait, as well as body and eye movement [3, 4, 5, 8, 9, 10, 11, 12, 13, 14]. Statistical feature models are empirical or mathematical models that provide a statistically valid method for a computer algorithm to generate 1 Image Source: reproduced with permission from the author The MITRE Corporation and The Aerospace Corporation All Rights Reserved. Page 1 of 7

2 biometric features across a target population. Statistical feature models should, in general, be derived from the physics. However, they can also be generated from empirical methods. For example, the optimal statistical derivation model for fingerprints would be derived from the biological factors that create the unique friction ridge and pore patterns in the human fetus. The method of empirically determining equivalent statistical parameters however suffices, where the analysis of anthropometric, fingerprint or other private/public databases of biometrics can be utilized [15, 16, 17, 18, 19, 20, 21]. The Pankanti et al. derivation of fingerprint uniqueness was for fingerprint minutia from ridge flow [22]. No statistical derivation has been proposed for finger ridge/trough patterns or level 3 information [21, 22]. Level 3 information includes pores and more subtle details about friction ridge width, shape and deviation. Hence for fingerprints, a partial theoretical mathematical model and empirical data exist that could be used to validate the synthetic model generator. Methods for empirically deriving statistical shape models for the face or body exist [23, 24]. Daugman has published treatments addressing the statistical uniqueness of the human iris [25, 26]. Empirical and mathematical models are validated by comparing their statistical distributions to the statistical distributions from real biometrics. The closest testing that approaches what is needed to validate synthetic biometrics generators against real biometrics is the University of Bologna Fingerprint Verification Competition (FVC) Tests [27, 28]. These tests use synthetic images from the University of Bologna s SFinGe tool (shown below in Figure 2) [29]. Very realistic images simulate a low-cost sensor with rotation, displacement and distortion characteristics approximating images from real biometrics. Figure 2. The SFinGe Fingerprint Generation Tool The National Institute of Standards and Technology s (NIST) Facial Recognition Verification Tests (FRVT) are equivalent to the FVC tests; the NIST tests, however, do not utilize synthetic faces [30]. There is no equivalent public testing of iris recognition vendors due to the limited number of vendors. Environmental affects not only refer to the ability to control through parametric means (while maintaining calibrated results) the effect of nature on acquired biometric image data, it also refers to the influences from the image acquisition physics. As an example, SFinGe, contains controls for pressure, moisture, skin elasticity, and the ability to choose specific sensor types to mimic sensor specific image acquisition affects [1, 2, 27]. Methods for synthesizing potential environmental affects on the acquisition of synthetic face and iris images can be gleaned from the radiosity and global illumination methods [8, 31]. Additional models for environmental factors such as temperature on human subjects can eventually be used to account for the effects of physical changes like sweat on biometrics [32]. Complete accounting for image acquisition affects is also possible using physical optics theory and the physics of the response by the imaging device to electromagnetic radiation. Image processing methods exist for measuring image acquisition distortion among other effects [see for example The Biometrics Community Within the biometrics community, the use of synthetic biometric images for research and testing is not a new concept or practice. As previously discussed, research groups have utilized synthetic biometric images for testing and parametrically controlled data generation [27, 28]. Of the mainstream biometrics, synthetic fingerprint and facial images have perhaps received the most attention by biometrics researchers [1, 2]. Using synthetic biometrics is generally not considered a best practice for testing biometrics systems [27, 33]. Additionally, the requirements of Common Criteria testing provide a further roadblock to the adoption of these methods [34]. Common Criteria requires the use of real images obtained from the sensor in an operational scenario for security certification testing, as the focus is on operational security. Hence, using synthetic images is not desirable in this case since they are not integrated into the biometric system; where the fully operational system s security vulnerabilities need to be uncovered The MITRE Corporation and The Aerospace Corporation All Rights Reserved. Page 2 of 7

3 Another issue retarding the adoption of synthetic biometrics pertains to the availability of real biometrics from engineering staff members working on the development of these systems, and from the understandable desire to simply use biometrics from real people captured by the actual system. Finally, the current synthetic biometric generation systems do not fully mimic real sensors, and do not provide statistically valid representations of real populations. Hence, despite the availability of cost-effective commercially available tools to synthesize complete human subjects, their use has not been fully realized within the biometrics community. This brings up the fundamental question of how these issues retarding their adoption can be removed to facilitate their use within this community. For example, how could the biometrics community best incorporate these tools to potentially further refine the testing and evaluation of any system, which incorporates biometrics (a.k.a. biometric systems)? What are the benefits? This introduction has provided some initial background into the state of computer generated (synthetic) biometrics. We now review the hurdles facing the adoption of synthetic biometrics within the biometrics community and ponder some potential advantages for using synthetic biometrics within that community. 2. Benefits for Biometric Systems While not a substitute for the biological diversity provided by live biometrics, computer generated biometrics do offer several advantages. The potential benefits for using synthetic biometrics, as well as research areas that should be explored are provided in this section. 2.1 Parametric Biometric Systems Testing The Common Criteria testing issue, mentioned in section 1.2, is circumvented if the use of synthetic images for testing biometric systems is restricted to testing the artificial intelligence underpinnings (i.e. the image processing algorithm s robustness factors and environmental boundaries). The system s biometric inputs are synthetically generated up front using any number of carefully controlled parameters. The synthetic images are then processed in a simulated operational mode, where the biometrics processing logic that will be employed by the operational system remains intact. System robustness issues can be investigated from the parametric control provided by these synthetic scenes. For example, synthetic face images for facial recognition tests can include carefully controlled changes to pose angle, illumination, expression, and obstructions [35]. Additional relevant parameters include motion and location information. Differences in biometric recognition performance across demographic populations and sub-populations (such as age) have been reported in recent tests, but the causes are not always fully understood [36]. Synthetic techniques may play a role in isolating demographic differences and associated specific parameters, which might play a role in effecting biometric recognition performance [37]. Figure 3. Synthesized Aging Sequence The illustration in Figure 3 is an example of using hybrid synthetic images to study aging, where a photographic image of a live person is added to a 3D face model. In this example, the model is aged forwards and backwards in a progressive sequence using the FaceGen application from Singular Inversions, Inc. [36]. In the case of fingerprinting, friction ridge quality and finger size correlations to age, ethnicity and gender effects on performance could be readily tested using validated synthetic images. Differences across environments and sensors are known to result in differences in performance. Some biometric systems address some of the environmental or sensor differences. However, the majority of systems have yet to effectively account for all differences. Here, the ability to parametrically control environmental difference is a necessary step towards the ability to account for (or cancel) its particular influences. 2.2 Operational Scenario Testing Synthetic scenes with injected biometrics provide the potential to construct arbitrary representations of real-world scenarios using a fixed 2004 The MITRE Corporation and The Aerospace Corporation All Rights Reserved. Page 3 of 7

4 set of model "subjects" or randomly generated populations of subjects and scenarios. Ground truth, that is, the actual layout of the potential operational location, is known in advance, eliminating the need to re-identify and re-orient human subjects or physical objects. Variations on a basic theme, such as a video of subjects walking through a chokepoint, can be systematically modeled with parametric settings. Parameters range from global, environmental factors such as illumination and camera angle, to local aspects such as hair length, pose angle, and facial expressions. Furthermore, specific behavioral outliers to normal chokepoint transit behavior can be examined to reveal system vulnerabilities. Biometric systems engineers could run a vast array of operational scenario tests to define the optimum layout of biometrically enabled security devices by testing the system in the computational realm prior to deployment. As biometric systems are deployed in support of national security, border control, and immigration applications, understanding the affects of international populations becomes very significant. Countries with population groups that lack sufficient diversity in age and ethnicity for thorough testing of border management applications would certainly benefit. For example, the FaceGen software permits control of several ethnic characteristics [36]. A recognition system s inability to properly discriminate between controlled variations in ethnic or racial attributes for example, suggests there may be an inadequate body of system training data used to fine-tune the system or a fundamental problem with the recognition algorithms. 2.3 Enhancing privacy Another important benefit for synthetic biometrics is their lack of association with a specific individual s identity. Hence, they provide the unique ability to be anonymous in origin by design and thus allow researchers to simulate populations without having to worry about security arrangements for the handling and use of biometrics from real people [38]. An example that highlights the touchy subject of biometrics use by governments in the area of privacy was the proposed use of biometrics in a Department of Defense anti-terrorism total information awareness system that attracted congressional and public scrutiny concerning privacy, policy, and potential abuse issues. Those concerns, ultimately led to the cancellation of that program, are summarized in a December 2003 audit report from the Inspector General of the U.S. Department of Defense [39]. Synthetically generated biometrics provide a benefit to security systems testing by providing a privacy sensitive method, where synthetics are not subject to the same privacy and legal issues that must be addressed when personal data is collected and maintained. Moreover, there should be few if any restrictions for distributing, publishing, and sharing synthetic datasets. There are these types of issues, however, when attempting to analyze real datasets from private or government sources. With fewer restrictions, there are additional benefits for information sharing for research as well as cost reductions from storage and data handling. 2.4 Cost and Time Once necessary software is in place and control parameters are configured, synthetically generated biometrics can be quickly produced. Typically, database production times are on the order of minutes or hours as opposed to months or even years for the collection of the same number of natural images. The synthetic fingerprint generator, SfinGe, reportedly generates 10,000 realistic prints in ten hours using a single Pentium IV CPU [1]. A MITRE experiment experienced similar generation times for face images [36]. Synthetic generators are functionally capable of scaling to produce very large databases or simply generating large numbers of randomly selected biometrics from the specified parametric controls. Collecting medium to large databases of biometrics from real people for testing (or any purpose) is expensive and challenging. Actual costs vary according to the collection protocol and logistics of the target population. Most collections undertaken by universities tend to incorporate small numbers of test subjects for the test population, typically from the young university population [17, 18]. Larger government or industry sponsored collection efforts, on the order of thousands or tens of thousands of images are extremely costly. The only alternative to carefully controlled biometrics collection protocols for research is use of private or government databases. This use restricts the publication visibility of the research results, due to privacy concerns. The ultimate scalability to large quantities of valid synthetic images hinges not only on the ability to create realistic synthetic biometric images, but also is directly dependent on the availability of valid statistical models. 3. Statistical modeling Statistical models are essentially mathematical equations or empirically derived algorithms, which 2004 The MITRE Corporation and The Aerospace Corporation All Rights Reserved. Page 4 of 7

5 are fed randomly generated numbers to create data that are statistically equivalent to real data. Mathematical models provide researchers and engineers with the ability to predict the performance of larger populations based on the modeled performance of smaller populations [40, 41]. Match score distributions from biometric systems contain many underlying behavioral and physical population uncertainties. Figure 4 shows the match score distributions from a corpus of input images (see [42] for details on the corpus). Figure 4. A Fingerprint Algorithm s Raw Scores The distribution is dependent on the conditions of the test and the subjects used. For example, high humidity and temperature causes sweat, where low humidity causes dry skin. This environmental influence on our skin, in turn influences the acquisition of the fingerprint image differently for different types of fingerprint sensors. Even though the corpus in Figure 4 was accumulated in a climate controlled indoor environment, some of the younger test subjects provided sweaty fingerprints. Future physics-based or empirically derived models from biometric technology testing can be combined with Monte Carlo simulations. This provides a powerful tool for scientists and engineers working with biometric systems, in the same manner that these methods help researchers in other disciplines [43]. 4. Conclusions and recommendations It is evident that the realm of biometrics is exploding and the integration of various biometrics into sophisticated and robust models is occurring. The ability to increase the reliability and accuracy of these systems is critical, as biometrics become an essential part of law enforcement and security communities. The use of synthetic biometrics by researchers in simulations provides a potential method for testing these systems in a privacyenhancing manner. Simulations provide controlled statistical sampling through the use of a model. The models in turn are used to provide an approximation about complex, multi-variable problems. The lack of statistically valid models remains one of the main problems faced by the biometrics community. The authors recommend that targeted studies and prototyping efforts be conducted to progress the state of the art as presented previously in Table 1. Additionally, a demonstration of how real biometrics samples can be accurately transformed across two or more diverse synthetic environments is an important and achievable next step for the advancement of biometrics. For example, face images with indoor illumination versus outdoor illumination can be simulated and then used to accurately compensate for the undesired variances from live images. An example of a biometrics deployment that may have benefited from the use of synthetic biometrics is an operational scenario test of the highly publicized face recognition system test at Boston s Logan Airport. The deployment reportedly failed to match the identities of 38% of a test group of employees [44]. Had this deployment been modeled and checked synthetically, by a third party, this highly public failure with its associated publicity debacle may have been avoided. A much better example of where the technology could be utilized in the future is in the testing of border management scenarios where there is a wide range of atmospheric and operational conditions. The U.S. alone must contend with the extreme cold wintertime conditions in Alaska to the hot humid summertime conditions in Florida [45]. As discussed, the ultimate scalability to large quantities of valid synthetic images hinges directly on the statistical models underpinning the generated data, and this is an area where questions remain. A synthetic generator for biometric data must contain a model, where currently it must make assumptions about the distinguishable phenomenology of the biometric trait, its natural variances, and also its responses to the environment. The creation, formalization and assessment of such models are important next steps towards allowing an understanding of performance derived from testing with synthetic biometrics The MITRE Corporation and The Aerospace Corporation All Rights Reserved. Page 5 of 7

6 Bibliography 1. Computer Science Department, University of Bologna, Synthetic Fingerprint Generator (SfinGe) Home, Version 2.5, January 2004, 2. R. Cappelli, A. Erol, D. Maio, D. Maltoni, Synthetic fingerprint-image generation, Proceedings 15th International Conference on Pattern Recognition, Vol. 3, Sept V. Blanz, and T.Vetter, A Morphable Model for the Synthesis of 3D Faces, Computer Graphics Proceedings of SIGGRAPH, pp Los Angeles, J. Yan, H. Zhang, Realistic Virtual Face and Body Synthesis, IAPR Workshop on Machine Vision Applications, Tokyo, Japan, Nov K. Waters, A muscle model for animating threedimensional facial expression, Computer Graphics Proceedings of SIGGRAPH, pp , A. Lefohn et al., "An Ocularist's Approach to Human Iris Synthesis." IEEE Computer Graphics, Vol. 23, No. 6. pp , November/December D. P. Greenberg et al., A Framework for Realistic Image Synthesis, Computer Graphics Proceedings of SIGGRAPH, pp , G. Ward. Radiance Synthetic Imaging System. Lawrence Berkeley National Laboratory, present, 9. P. Debevec et al., Acquiring the reflectance field of a human face, Computer Graphics Proceedings of SIGGRAPH, pp , Tae-Yong Kim, and U. Neumann, Interactive multiresolution hair modeling and editing, Computer Graphics Proceedings of SIGGRAPH, pp , B. Allen, B. Curless, and Z. Popović. The Space of Human Body Shapes: Reconstruction and Parameterization from Range Scan, Computer Graphics Proceedings of SIGGRAPH, July, H. C. Sun, and D. N. Metaxas, Automating gait generation, Computer Graphics Proceedings of SIGGRAPH, pp , P. Faloutsos, M. van de Panne, and D. Terzopoulos, Composable Controllers for Physics-Based Character Animation, Computer Graphics Proceedings of SIGGRAPH, pp , S. P. Lee, J. B. Badler, N. I. Badler, Eyes Alive, in Computer Graphics Proceedings of SIGGRAPH, pp , Civilian American and European Surface Anthropometry Resource Project CAESAR TM, available online at ome.htm 16. Anthropometric data of Japanese, available online at Japanthropdat.html 17. The AR Face Database, available online at abase.html 18. The Yale Face Database, available online at B.html 19. The UMIST Face Database, available online at The Database of Faces, An archive of the AT&T Cambridge Laboratories, available online at l 21. S. Meagher, An Update on Daubert Hearings on Fingerprints, International Association for Identification (IAI) 87th International Educational Conference, Las Vegas, NV, August 4-10, 2002, available online at S. Pankanti, S. Prabhakar, and A. K. Jain, "On the Individuality of Fingerprints", IEEE Transactions on PAMI, Vol. 24, No. 8, pp , A shorter version also appears in Fingerprint Whorld, pp , July I. L. Dryden and K. V. Mardia, Statistical Shape Analysis, John Wiley & Sons, A. El-Hussuna, Statistical variation of Three Dimensional face models, M. Sc. Thesis, IT- University of Copenhagen Multimedia Technologies, March J. Daugman, The importance of being random: Statistical principles of iris recognition, Pattern Recognition, vol. 36, no. 2, pp , J. Daugman and C. Downing, Epigenetic randomness, complexity, and singularity of 2004 The MITRE Corporation and The Aerospace Corporation All Rights Reserved. Page 6 of 7

7 human iris patterns, Proceedings of the Royal Society, B, 268, Biological Sciences, pp , D. Maio, D. Maltoni, R. Cappelli, J. L. Wayman, A. K. Jain, FVC2000: Fingerprint Verification Competition, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, No. 3, pp , March 2002, the full report is available online at 00_report.pdf 28. D. Maio, D. Maltoni, R. Cappelli, J. L. Wayman, A. K. Jain, FVC2002: Second Fingerprint Verification Competition, 16th International Conference on Pattern Recognition (ICPR'02) Volume 3, p , August D. Maltoni, D. Maio, A.K. Jain and S. Prabhakar, Handbook of Fingerprint Recognition, Springer (New York), P. J. Phillips, P. Grother, R.J. Micheals, D.M. Blackburn, E. Tabassi, and J.M. Bone, Face recognition vendor test 2002: Overview and Summary, Technical report, NIST, March 2003, available online at D. S. Ebert, Advanced modeling techniques for computer graphics, ACM Computing Surveys (CSUR), Vol. 28, Issue 1, pp , March Y. Daun, Fuzzy Role Based Expert System for Human Thermoregulation Model, U.C. Berkley Mechanical Engineering ME290M Final Project, 1999, available online at me290final.pdf 33. A. J. Mansfield and J. L. Wayman, Best Practices in Testing and Reporting Performance of Biometric Devices, Version 2.01, NPL Report CMSC 14/02, August P. Zatychec, Evaluation & Certification of Biometric Technologies to ISO Common Criteria Standards, Biometrics Consortium Conference 2001, available online at L_BCFEB02/FINAL_4_Final%20Paul%20Zatyc hec%20brief.pdf 35. J. Marques, N. Orlans, and A. Piszcz, Effects of Eye Position on Eigenface-Based Face Recognition Scoring, available online at ers_03/marques_eigenface/marques_eigenface.p df 36. N. Orlans, A. Piszcz, R. Chavez, Parametrically Controlled Synthetic Imagery Experiment for Face Recognition Testing, ACM SIGMM 2003 Proceedings, October N. Furl, P. J. Phillips, and A. J. O Toole, Face recognition algorithms and the other-race effect: computational mechanisms for a developmental contact hypothesis, Cognitive Science 26, pp , C. J. Hill, Risk of Masquerade Arising from the Storage of Biometrics, B.S. Thesis, Australian National University, 2001, available online at Department of Defense Office of the Inspector General (Information Technology Management), Terrorism Information Awareness Program (D ). December A. Y. Johnson, J. Sun, and A. F. Bobick, Predicting large population data cumulative match characteristic performance from small population data, 4th International Conference on Audio- and Video Based Biometric Person Authentication (AVBPA 2003), University of Surrey, Guildford, UK, June A. Y. Johnson, J. Sun, and A. F. Bobick, Using similarity scores from a small gallery to estimate recognition performance for larger galleries, IEEE International Workshop on Analysis and Modeling of Faces and Gestures held in conjunction with the International Conference on Computer Vision (ICCV 2003), Nice, France, October D. J. Buettner, Effects Of WSQ Compression On Client-Server Biometric Identification Performance, Proceedings of the 2002 International Conference on Artificial Intelligence (IC-AI 02), Vol. 1, pg , June R. Saucier, "Computer Generation of Statistical Distributions", Army Research Laboratory Technical Report, ARL-TR-2168, March S. Murphy, and H. Bray, Face recognition devices failed in test at Logan, The Boston Globe, Sept. 3, 2003, available on line at 09/03/face_recognition_devices_failed_in_test_a t_logan/ 45. See U.S. ports of entry documents found at editorial_0333.xml 2004 The MITRE Corporation and The Aerospace Corporation All Rights Reserved. Page 7 of 7