Research Statement Arunesh Sinha aruneshs/

Transcription

1 Research Statement Arunesh Sinha aruneshs/ Research Theme My research lies at the intersection of Artificial Intelligence and Security 1 and Privacy. Security and Privacy continues to remain a top concern in the physical and cyber world. Indeed, the sheer volume, diversity and persistence of cyber-attacks and crimes in the physical world point to the greater reality that defense mechanisms must be agile and adaptive to be effective against intelligent and adaptive adversaries. A critical component of adaptive defense is the defender-adversary interaction. Game theory provides a rich framework to capture such interactions between the defender and the adversary. However, there are significant challenges that need to be overcome to apply game theory in practice; I have addressed these challenges in my work 2. More specifically, I have explored the following broad areas: Scalability in game theory: A realistic model of the defender-adversary interaction results in large games that requires novel algorithmic techniques to solve such games. Machine learning in games: The availability of data about past interaction of defenders and adversaries offers an opportunity to learn the adversary behavior. My research interests cut across theory and practice. I believe in working on theory problems that are inspired from real-world issues and using principled theoretical techniques in practice. Indeed, my work reflects this underlying theme, having produced results of both theoretical interest and practical use in four major projects: (1) Audit Games: My work on privacy audits in hospitals yielded public policy guidelines that could aid the Health and Human Services (HHS) in designing appropriate oversight to encourage privacy audits in hospitals, (2) DARMS: The project for effective screening of passengers at US airports started as a real-world project named Dynamic Aviation Risk Management Solution (DARMS) initiated by the Transport Security Administration (TSA); it yielded theoretical advances in randomized allocation along with a pilot deployment which is planned for January 2016, (3) Urban Crime: This work started as a project with USC police and resulted in a novel Dynamic Bayesian Network (DBN) based approach to crime prevention; this technique is now part of the software tool used by USC police to aid in crime patrolling and (4) PAC model of adversary behavior: This primarily theoretical project yielded a Probabilistic Approximately Correct (PAC) formalization of learning the follower behavior in Stackelberg games; in particular, it led to a novel non-parametric learning approach that outperformed all existing approaches on predicting poaching activity in Queen Elizabeth National Park (QENP), Uganda. Security & Privacy Audits Airport Screening Urban Crime Wildlife Crime Figure 1: Domains of application of my research 1 By security I mean both physical and cyber security. Also, Security and Privacy refers to a single discipline. 2 All citations for my work are present in my CV.

2 Arunesh Sinha 2/5 Scalability in Game Theory My work on game models of defender-adversary interaction has focused on the Stackelberg (leader-follower) game model. The large number of possible allocations of defense resources results in significant scalability challenges in solving the optimization problem required to compute the Stackelberg equilibrium. Broadly, my approach to scalability has focused on compactly representing the optimization problem with provable solution guarantees. Scalability in audit games: The HIPAA privacy law was passed in 1996 to prevent inappropriate flow of private health information. Hospitals have adopted post-hoc audits of health record accesses as the preferred means of detection and remediation of HIPAA breaches, mainly due to the undesirable consequences of restricting flow of health information (e.g., in a medical emergency). However, the complicated privacy policies do not permit completely automated auditing, thus, the few human auditors are faced with a large number of suspicious cases to audit. Indeed, ad-hoc audit practices have failed to detect many HIPAA breaches in a timely fashion 3. As part of my Ph.D. work [IJCAI 2013,AAAI 2015], I proposed a Stackelberg game model of the interaction between the auditor and auditee. An important component of this auditorauditee interaction is punishments; judicious use of punishments can shape the behavior of the auditee resulting in desirable outcomes. The scalability challenge can be gauged by noting that even a small example of 100 audit cases to be inspected by 10 audit resources, results in many ( 100 ) possible allocations. Adding to this scalability problem, the decision variable for punishments makes the resultant Stackelberg equilibrium computation problem a non-convex optimization problem. I proposed a fixed parameter tractable (FPT) and a fully polynomial time approximation scheme (FPTAS) for this non-convex problem, thereby allowing for efficient computation of the equilibrium. Further improving scalability, I proposed a technique to transform the optimization problem into an equivalent compact optimization by reducing the number of variables and instead adding more constraints. I identified the conditions under which only polynomially many new constraints were added, which led to upto 3X improvement in runtime when these conditions were satisfied. These conditions are not restrictive as they capture many common auditing scenarios such as centralized auditing by a group of auditors and localized auditing of employees by immediate managers. Based on the model, I could provide an economic guided explanation for known occurrences such as hospitals not punishing top doctors for privacy transgressions. I also identified the external incentives that could encourage hospitals to conduct more thorough privacy audits, thereby providing policy tools to stimulate more rigorous auditing in hospitals. Scalability in DARMS: The TSA has launched a new initiative known as DARMS which aims to enhance aviation security. Dubbed as the future of aviation security, this ambitious project aims to provide a mathematically sound basis for intelligent screening of passengers. The TSA utilizes a number of screening resources for screening passengers, e.g., x-ray machines, metal detectors, pat-downs, etc. However, the most effective screening resources cannot be used to screen every passenger as there exists capacity bounds on the usage of each screening resource, with lower capacity for more effective resources. Thus, given the risk category of each passenger, the number of passengers and the efficacy of resources in detecting threats, the goal is to maximize the quality of security screening subject to the underlying passenger flow constraints. As part of the passenger screening component of the DARMS project, I proposed a novel Stackelberg game model for screening of threats [AAAI 2016]. Combining team formation aspects for the capacity-constrained screening resources with the need to screen every passenger in every risk category as effectively as possible resulted in a large (exponentially large) linear program (LP) required to compute equilibrium of the Stackelberg game. For example, assuming 900 and 3 See Information Week report

3 Arunesh Sinha 3/5 100 people arrive in one hour in two risk categories respectively and with 10 screening teams results in ( 909)( 109 ) variables in the LP; typically this number is much larger as the number of risk categories and teams are higher. I provided techniques for scaling up the equilibrium computation; the techniques include a novel temporal decomposition of the game and a novel alternate representation of the LP obtained by using a convex combination of a number of polytopes to represent the feasible space of the LP. In proposing this alternate representation of the LP, I advanced the techniques available in randomized allocation 4. Since I do prove the NP hardness of the optimization problem (even after temporal decomposition), the guaranteed optimal algorithm using the alternate representation of the LP has large running time for the worst case. By trading off between the worst case solution quality and running time, I provide an efficient and practical algorithm that is not worst case optimal but performs extremely efficiently in practice with practically no solution quality loss (no loss observed in random tests). This work is being tested as a pilot in one of the biggest airports in the country starting January 2016 and will hopefully be adopted by all airports in future. Learning in Games Distinct from classic game theory, the ready availability of data on defender-adversary interaction in domains such as wildlife and urban crime has brought forth the opportunity to learn adversary behavior and predict outcomes based on the learned models. My work on learning in games has been guided by the philosophy that adversary behavior should be directly learned from data with minimal prior assumptions, e.g., not assuming rationality. I believe this philosophy provides fundamental new approaches for planning optimal defender actions in adversarial game interactions. A learning theory approach to audit games: As part of my Ph.D. work, I also explored the audit scenario (described earlier) in which no assumptions are made about the auditee behavior [CSF 2011]. In such adversarial interaction scenarios, the notion of regret has been used in online learning literature to provide guarantees for the defender s utility. I formulated the audit process as a regret minimization problem for the auditor. I provided an efficient regret minimizing algorithm with stronger regret bounds for the partial information regret model than the current state of the art regret minimizing algorithms by exploiting estimatability properties of the underlying problem. A learning approach to urban crime: The Department of Public Safety (DPS) at the University of Southern California (USC) is tasked with ensuring security in and around the USC campus and, similar to other security agencies, the number of patrol officers available is quite limited. DPS has records for past crime and patrol allocations over multiple years, and their goal is to conduct preventive patrols intelligently by predicting crime. I recognized that it is almost impossible to estimate utilities for petty criminals and moreover, it has been shown empirically that criminals do not exhibit perfectly strategic behavior in practice. Also, while it is amazing that regret based approaches provide guarantees against adversaries with arbitrary behavior, I recognized that real adversaries are not completely arbitrary and are more likely to exhibit patterns in their behavior which can be exploited to obtain better guarantees. Thus, rather than using poor estimates of utility or uncertain models of bounded rationality or assuming that the adversary is completely arbitrary, I proposed a technique that directly learns the behavior of the criminals [AAMAS 2015]. I proposed a Dynamic Bayesian Network (DBN) based model for predicting crimes from available crime and patrol data; the parameters of the model capture the behavior of the criminals. Distinct from the crime predicts crime approaches in 4 E. Budish, Y.-K. Che, F. Kojima, and P. Milgrom, Designing random allocation mechanisms: Theory and applications, The American Economic Review, 2013.

4 Arunesh Sinha 4/5 criminology that rely only on crime data to predict future crime, this model learns the interaction between patrol officers and criminals using the patrol and crime data, resulting in better crime prediction than prior approaches. The learning approach used the Expectation Maximization algorithm, and used a number of independence assumptions to compactly represent the DBN model in order to avoid over-fitting and to scale up the computation. We also propose a dynamic programming based patrol planning algorithm, which projects a crime reduction of up to 50% as compared to the existing patrol strategy used by DPS. This project is now part of the software used by USC police to aid in preventive patrolling. Learning adversary response in Stackelberg games: Following from my earlier work on urban crime, I recognized that with the increasing availability of data about players past interactions any Stackelberg game can be played by directly learning the response function of the follower. This response function is the adversary s response as a function of the strategy committed to by the leader. This approach contrasts with previous approaches in Stackelberg games that use traditional game-theoretic reasoning about the follower s response through payoffs and rationality. I formalized the problem of learning the complicated stochastic response function of the adversary using any functional representation by appealing to the PAC model of learning. As an instance of the general representation, I proposed the first non-parametric class of response functions for Stackelberg games and also the learning techniques to learn the best function in this class; the richness of the non-parametric response functions (only restriction is Lipschitzness) allows for the capturing of complex human behavior in many domains. Another instance of this general representation turned out to be a classic discrete choice model that has been used previously in literature. I derived sharp sample complexity bounds for PAC learnability for this non-parametric model and the discrete choice model. When applied to the domain of wildlife crime, the nonparametric approach outperformed all known approaches in learning the poacher s response from poaching data obtained from QENP in Uganda albeit using more computational resources. Other research. The other significant pieces of work that I have done include predicting sharing policy of status messages on Facebook in order to warn and prevent users from posting privacy sensitive information and proposing a logging protocol that guarantees log-integrity (proved using formal logic) of Windows OS logs using the functionalities of the TPM2.0 chip. Vision and Future Research The recent rapid growth of AI technologies presents both an opportunity and challenge for researchers to tackle foundational problems at the intersection of AI and Security and Privacy. Cyber-security and privacy are among the top concerns hindering the large scale adoption of AI technologies. On the flip side, the rich set of techniques from AI such as game theory, machine learning and planning under uncertainty could lead to better defense mechanisms in the domain of Security and Privacy. Following my overarching goal of understanding and implementing adaptive defense against intelligent adversaries in any adversarial interaction, I aim to continue exploring the rich fundamental problems in this domain such as: 1. Adaptive Defense in Information Systems: The increased reliance on interconnected cyber and physical systems and sophisticated technologies for controlling physical systems, while definitely beneficial, has opened the door to new vulnerabilities. Examples of such systems abound: electric smart grids, the DARMS system as described above in my research, various control systems (water, electricity) controlled by software and in general the emerging phenomenon of the internet of things. A fundamental problem in both cyber and cyber-physical security is the ability of the attacker to control the information received by the defender, i.e., the defender can be fooled into believing that there is no attack in progress. I believe that two broad ap-

5 Arunesh Sinha 5/5 proaches to address this problem are: (a) Utility-Information tradeoff : The defender can invest in gathering more information about the system, possibly at the cost of efficiency. This trade-off in an adversarial environment leads to games of imperfect and incomplete information, areas that have not been studied a lot in computational sciences. Apart from the basic modeling challenge, large scale games of these types may require approximation methods such as game abstraction. (b) Intelligent use of information: More information is not useful if not acted upon in an intelligent manner. Many defense tools in use today, be it cyber (anti-virus, firewalls) or physical (access controlled doors, physical sensors, human guards), work independently and not as part of a co-ordinated plan, e.g., it should be natural that alarms going off in one part of the system should lower the threshold for alerts in other parts of the system. However, the question of how to make the best decision in the face of uncertain and possibly adversarial inputs is quite challenging. The need of the hour is an automated defense tool that can make optimal security decisions by analyzing the security inputs from many distributed sources and considering the efficiency-detectability trade-off of part (1a). 2. Improving Security Decision Making by Humans: Most automated systems in use today are administered by humans and possibly rely on human actions in their execution, e.g., networks in organizations, electric grids, etc. The increasing difficulty of attacks on software and protocols has led to attackers adapting their strategy and targeting the human in the loop. There are known issues with humans in the loop: (a) Social engineering attacks: A recognized problem, especially in cyber security, is the lack of expected behavior by human actors resulting in security vulnerabilities that are exploited by social engineering attacks, e.g., the recent cyber-attack on Sony started as a phishing attack. An end-to-end defense of such systems requires some guarantees from the human actors. An unexplored research issue centers around mechanisms to incentivize the human users of any system to act in an expected manner. Indeed, practical design of incentive mechanisms should consider behavioral models instead of the classic rational choice theory for robust results in the real world. (b) Burden of information: A reason for people not making the right security decision is because they are unable to process the vast amount of information presented to them in a security context. Automated tools using machine learning or planning techniques can guide human users in their decision making process by reducing their computational burden and narrowing down the choices to make. Further, as already stated in part (1), automated tools can also help trained security experts prioritize their investigation since overwhelming amounts of warnings lead to missed alerts and ignored signs of attacks, e.g., the ignored alerts in the recent Target credit card breach. I believe questions such as those above can be addressed by bringing together concepts from AI and Security and Privacy. Addressing such problems can lead to real world impact such as making emerging AI technologies secure and providing end-to-end security for currently deployed cyber-physical systems. I aim to build a research group that works towards the goal of solving these complex inter-disciplinary problems using techniques from various disciplines such as game theory, security and privacy, machine learning, optimization and economics.