SECURING WIRELESS LOCALIZATION AGAINST SIGNAL STRENGTH ATTACKS

Transcription

1 SECURING WIRELESS LOCALIZATION AGAINST SIGNAL STRENGTH ATTACKS BY YINGYING CHEN A dissertation submitted to the Graduate School New Brunswick Rutgers, The State University of New Jersey in partial fulfillment of the requirements for the degree of Doctor of Philosophy Graduate Program in Computer Science Written under the direction of Prof. Richard P. Martin and Prof. Wade Trappe and approved by New Brunswick, New Jersey October, 27

2 c 27 Yingying Chen ALL RIGHTS RESERVED

3 ABSTRACT OF THE DISSERTATION Securing Wireless Localization against Signal Strength Attacks by Yingying Chen Dissertation Director: Prof. Richard P. Martin and Prof. Wade Trappe Accurately positioning nodes in wireless and sensor networks is important because the location of devices and sensors is a critical input to many higher-level applications. However, the localization infrastructure can be subjected to non-cryptographic attacks, such as signal attenuation and amplification, that can not be addressed by traditional security services. This thesis aims to provide secure and accurate location information in wireless and sensor networks by characterizing the response of localization algorithms to attacks, detecting attacks, localizing adversaries, and additionally, improving localization performance. First we studied the robustness of localization algorithms to signal strength attacks. We found the performance of localization algorithms degrades significantly under attacks when signals are attenuated or amplified by an adversary. We then formulated a theoretical foundation for the attack detection problem using statistical significance testing. We proposed attack detection schemes for two broad localization approaches: signal strength and multilateration. We found that different localization systems all contain similar attack detection capabilities. Next, we examined the applicability of localization methods to localize adversaries participating in identity-based spoofing attacks. We proposed a spoofing detector for wireless spoofing that utilizes K-means cluster analysis. We integrated our K-means attack detector into a realtime indoor localization system, which is capable of localizing the positions of attackers. Our ii

4 experiments using both an (WiFi) network as well as an (ZigBee) network in two office buildings provide strong evidence of the effectiveness of our approach in attack detection and localizing the positions of the adversaries. In addition, we investigated the impact of landmark placement on localization performance using a combination of analytic and experimental analysis. We developed a novel algorithm called maxl mine algorithm that finds an optimized landmark deployment. Our experimental results show that our landmark placement algorithm is generic because the resulting placements improve localization performance significantly across a diverse set of algorithms, networks, and ranging modalities. Finally, we presented our general purpose real time localization infrastructure which targets to localize any radio-enabled wireless devices at anywhere and at anytime. iii

5 Acknowledgements This dissertation is an accumulation of much work, guidance, support, and the friendship of many people. First and foremost, I am deeply thankful for my two advisers, Prof. Richard P. Martin and Prof. Wade Trappe. Without their invaluable guidance and support, I would not be able to make rapid progress in my research work. Especially, I would like to express my gratitude to them for attending numerous discussions, listening to my practice talks, and helping me improve my writing and presentation skills. I have been very fortunate to have the opportunity to work with both of them. I must also thank the many professors and researchers that provided me advice and helped my research development during my studies at Rutgers. In particular, I would like to thank Prof. David Madigan, Prof. Dipankar Raychaudhuri, Prof. Yanyong Zhang, Prof. Larry Greenstein, Prof. Naftaly Minsky, and Dr. Wei Su for their advice and feedback throughout different stages of my study. Further, I would like to thank my colleagues in the Computer Science Department and in WINLAB at Rutgers University who have helped me along the way with stimulating discussions and interesting talks. My parents, who both are professors in the Physics Department at Nanjing University, deserve much credit. I would like to thank them for the continuous encouragement, support, and advice through these long years. Finally, to my husband, Jerry Cheng, and lovely son, Jeffrey Cheng, I must thank for giving me the love, support, understanding, and patience to keep me going through the difficult times. iv

6 Dedication To my parents, Kunji Chen and Xinfan Huang To my husband, Jerry Cheng To my lovely son, Jeffrey Cheng v

7 Table of Contents Abstract Acknowledgements Dedication ii iv v List of Tables xi List of Figures xii 1. Introduction Background and Motivation Thesis Organization Contributions Robustness Analysis of Localization Algorithms to Signal Strength Attacks Introduction Localization Algorithms Point-based Algorithms Area-based Algorithms Conducting Signal Strength Attacks Signal Strength Attacks Experimental Results of Attacks Attack Model Measuring Attack Susceptibility A Generalized Localization Model Attack Susceptibility Metrics Experimental Results vi

8 Experimental Setup Localization Angle Bias Localization Error Analysis Linear Response Precision Study Discussion about Hölder Metrics Conclusion Attack Detection in Wireless Localization Introduction Feasibility of Attacks Localization Attacks Signal Strength Attacks Experimental Methodology Generalized Attack Detection Model Localization Attack Detection Effectiveness Using Least Squares Localization The Residuals The Detection Scheme Experimental Evaluation Distance In Signal Space Overview Finding Thresholds Experimental Evaluation Other Test Statistics Nonlinear Least Squares (NLS) Area Based Probability (ABP) vii

9 Bayesian Networks (BN) Discussion Conclusion Detecting and Localizing Identity-based Spoofing Attacks Introduction Feasibility of Attacks Spoofing Attacks Experimental Methodology Attack Detector Formulation of Spoofing Attack Detection Test Statistic for Spoofing Detection Determining Thresholds Performance Metrics Experimental Evaluation Localizing Adversaries Localization System Attack Localizer Experimental Evaluation Discussion Conclusion Performance Improvement Using Optimal Landmark Placement Introduction Theoretical Analysis Background: Localization with LS Error Analysis Deployment Patterns Finding an Optimized landmark Deployment Evaluation Metrics viii

10 5.4. Landmark Position and Quantity Simulation Methodology Evaluation of Estimation Error Impact of Landmark Deployment Impact of Landmark Quantity Experimental Study Algorithms Experimental Setup and Methodology Localization Accuracy Evaluation of Performance and Sensitivity Using Time of Arrival Conclusion General Purpose Localization System Introduction Architecture Design Bayesian Networks Related Work Introduction Wireless Localization Secure Localization Coping with Identity Fraud Localization Performance Localization Infrastructure Conclusions and Future Work Dissertation Conclusions Future Research Directions References ix

11 Vita x

12 List of Tables 2.1. Algorithms under study CoRE: Slopes of Average Error from Linear Regression for attenuation attacks on all landmarks and individual landmark Industrial: Slopes of Average Error from Linear Regression for attenuation attacks on all landmarks and individual landmark CoRE: Slopes of Average Error from Linear Regression for mixed attacks of signal attenuation and amplification on multiple landmarks Analysis of (worst-case) H and (average-case) H Detection rate and false positive rate of the spoofing attack detector Localization error (ft) and Hölder metrics when standard deviation of noise on rss is 3dB Location estimation error (ft) and Hölder parameters across algorithms xi

13 List of Figures 2.1. The Bayesian Network under analysis Signal strength when going through a barrier Interpretation of distances in location estimation Deployment of landmarks and training locations on the experimental floors ABP: Localization estimation relative to the true locations for the Industrial Lab Error CDF across localization algorithms when attacks are performed on all the landmarks Error CDF across localization algorithms when attacks are performed on an individual landmark. The attack is 25dB of signal attenuation and signal amplification respectively CoRE: Error CDF across localization algorithms when attenuation attacks are performed on multiple landmarks CoRE: Error CDF across localization algorithms when amplification and attenuation attacks are simultaneously performed on multiple landmarks Average location estimation error across localization algorithms under signal strength attenuation attack CoRE: Average location estimation error across localization algorithms under simultaneous signal strength attenuation and amplification attacks on multiple landmarks CoRE: Maximum error as a function of attack strength from an all-landmark attack Contribution of each Landmark during sampling in the BN algorithm under attenuation attacks xii

14 2.14. CoRE: Comparison of localization results from the area-based algorithms for a testing point Analysis of precision CDF across area-based algorithms. The attack is performed on all the landmarks Precision vs. perturbation distance under attenuation attack Linear attack model on received signal strength for various media Layout of the experimental floor CoRE 82.11: (a) Ranging errors under the signal strength attacks (b) LLS residuals: Receiver Operating Characteristic (ROC) curves CoRE 82.11, LLS residuals: effectiveness of attack detection CoRE 82.11: (a) Cumulative Distribution Function (CDF) of minimum distance D s in signal space. (b) Minimum distance D s : Receiver Operating Characteristic (ROC) curves Minimum distance in signal space D s : attack detection across different networks and buildings CoRE 82.11, NLS: (a) Cumulative Distribution Function (CDF) of E. (b) E: Receiver Operating Characteristic (ROC) curves CoRE 82.11, NLS using E: effectiveness of attack detection CoRE 82.11, ABP: effectiveness of attack detection CoRE 82.11, ABP: Receiver Operating Characteristic (ROC) curves CoRE 82.11, BN: (a) Using fraction of contribution of each landmark for attack detection with threshold =.15. (b) Using likelihood in Bayesian inference for attack detection with threshold = CoRE 82.11: Comparison between generic and specific test statistics for attack detection CoRE 82.11: Relationships among Detection Rate (DR), ranging error, and localization error Landmark setups and testing locations in two networks Cumulative Distribution Function (CDF) of D c in signal space xiii

15 4.3. Receiver Operating Characteristic (ROC) curves Detection rate as a function of the distance between the spoofing node and the original node Relationships among the original node, the spoofing node, and their location estimation through localization system Localization error CDF across localization algorithms and networks Relationship between the true distance and the estimated distance for the original node and the spoofing node across localization algorithms and networks Packet-level localization: relationship between the true distance and the estimated distance for the original node and the spoofing node when using RADAR in the network Patterns for optimal landmark deployments The maxl-mine algorithm In 2x2ft area: (a) Location estimation error vs. random noise in RSS (b) Location estimation error vs. ranging error Performance of LS algorithms across different number of landmarks in 2x2ft area Deployment of landmarks and training locations on the experimental floors Localization accuracy CDFs across algorithms for network Localization accuracy CDFs across algorithms for network Linear regression on TOA data Localization accuracy CDFs using TOA GRAIL system architecture Solver scalability and flexibility Bayesian Networks xiv

16 1 Chapter 1 Introduction 1.1 Background and Motivation Wireless networks are changing the way we work, study, and interact with each other. As wireless networks become increasingly prevalent, they make integrating new information types into applications possible. Location information is one such information source that is very important for many applications. Localization refers to determining the physical position of a wireless device or a sensor node which can be either static or mobile. The location information can be one-dimensional (e.g., location on a long airport corridor), two-dimensional (e.g., location on one floor in a hospital), or three-dimensional (e.g., location within a multi-level shopping mall). For example, in the public arena, doctors want to use location information to track and monitor patients in medical facilities; for wild life observation, biologists can put tags on animals and perform habitat tracking; first responders can track victims and each other during an emergency. In the enterprise domain, location-based access control is needed for accessing the proprietary corporate materials in restricted areas or rooms. For example, during meetings, certain documents may need to be sent only to laptops within the involved conference rooms, which requires location-aware content delivery. In addition, asset tracking also relies on location information. These examples show that accurately positioning nodes in wireless and sensor networks is important as the location of sensors is a critical input to many high-level networking tasks and applications. With recent great advances in wireless technology, there are three wireless communication standards that have conjoined with our everyday life and have further promised to realize location-based services: First, Wireless Local Area Networks (WLANs) usually refer to networks based on WiFi technology functioning according to IEEE standards [4]. The normal infrastructure for a WiFi network consists of one or more Access Points (APs), which

17 2 has the ability to do wireless transmission and also serves as a gateway to a wired network. WiFi devices can thus connect to the Internet and talk to each other through APs. The most popular WiFi devices are laptops and Personal Digital Assistants (PDAs). Second, Bluetooth technology uses IEEE standards [5] and is designed for lower power consumption, and thus has a relatively shorter range around 1 meters. It is mostly used for communication between devices closely located to each other. Currently many devices support Bluetooth including laptops, cell phones, headsets, mouses, and digital cameras. Finally, ZigBee implements IEEE standards [6] and targets for sensor networks with embedded applications such as environmental monitoring, data collection, and intruder detection. Because of the nature of embedded applications, the corresponding devices utilizing ZigBee protocol are required to be small. The current available ones are about the size of a quarter [3, 7]. In wireless and sensor networks, there are various physical modalities can be employed to perform localization such as Received Signal Strength (RSS), Time of Arrival (TOA), Angle of Arrival (AOA), Hop Counts, and etc.. Among the localization techniques, utilizing RSS is especially attractive since it can reuse the existing deployment of wireless communication networks, rather than require a specialized localization infrastructure such as ultrasound or infrared methods. This provides tremendous cost savings. Also, all current standard commodity radio technologies, such as 82.11, , and Bluetooth provide it, and thus the same algorithms can be applied across different platforms. Further, based on the information obtained from physical modalities, different principles can be used to determine the positions of sensors. There has been active research in developing localization algorithms using lateration [23, 28, 44, 49, 54], angulation [53], probabilistic approaches [59, 69], and statistical supervised learning techniques [12, 27, 52]. We detail these efforts in Chapter 7. However, in spite of the utility of the location information, it is only useful if the location information is accurate and trustworthy. As more location-dependent services are deployed, they will increasingly become tempting targets for malicious attacks. Unlike traditional systems, the localization infrastructure is sensitive to a variety of attacks, ranging from conventional to non-cryptographic, that can subvert the utility of location information. Conventional attacks, where an adversary injects false messages, can be isolated and protected against using

18 3 traditional cryptographic methods such as authentication. However, there is a completely orthogonal set of attacks that are non-cryptographic, where the measurement process itself can be corrupted by adversaries. For instance, an adversary could introduce an absorbing barrier between the transmitter and the target, changing the underlying propagation physics. As the signal propagates through the barrier, it is attenuated, and hence the target would observe a significantly lower received signal strength. Consequently, the receiver would conclude that it is further from the transmitter than it actually is. On the other hand, wormhole attacks tunnel through a faster channel to shorten the observed distance between two nodes. Unfortunately, these non-cryptographic attacks can not be addressed by traditional security services. Thus, it is desirable to study the impact of these attacks on localization algorithms and explore methods to detect and further to eliminate these attacks from the network. This is the focus of this thesis. We are motivated to develop solutions that can be integrated into early generations of localization systems, so that we will not have to apply patchwork solutions to solve security threats that arise after localization systems are deployed. Specifically, in this thesis work we first performed a thorough study on the robustness of a broad array of localization algorithms to attacks that corrupt signal strength readings. The characterization of the response of algorithms provides important insights to be taken into consideration by system designers when choosing localization systems for deployment. From the robustness study, we observed that attackers can cause large localization errors using simple techniques. Hence, we must detect the presence of attacks in the network. We then formulate the attack detection problem as a generic statistical significance testing problem and proposed several attack detection schemes to broad classes of multilateration and signal strength-based methods. After a localization attack is detected in a wireless network, the next important and challenging step is to localize the positions of adversaries and further to eliminate the attack from the network. We proposed to use K-means cluster analysis to detect identity-based spoofing attacks and applied localization methods to locate adversaries participating in a spoofing attack. During the course of the security analysis for localization systems, we found that the landmark (a node with known position) placement plays an important role on localization performance. It is desirable to obtain higher location accuracy. In this investigation, we focused on

19 4 improving localization performance by repositioning the landmarks, rather than improving the localization algorithms or searching for new algorithms, and this should help a wide variety of algorithms. By using a combination of analytic and experimental analysis, we found geometric descriptions for the optimal deployment of landmark placement to maximize the location accuracy for indoor localization. Finally, one of the primary goals of localization research is to provide a scalable, general purpose, and real time localization infrastructure that can integrate location information into any computing radio-enabled devices. We are designing and developing a general purpose localization system prototype called GRAIL (Generalized Real-time Adaptable Indoor Localization), which can simultaneously position multiple devices using Bayesian Networks. The deployment of such a system in academic and research environments will allow researchers to explore issues beyond just algorithms and simulation tools, which can facilitate a variety of research topics such as privacy studies, security services, and policy enforcements. In addition, the practical usage of such an approach is significant as it can be applied to a broad array of applications such as monitoring, tracking, routing, and security services. 1.2 Thesis Organization This thesis is focused on to provide accurate and trustworthy location information to locationbased applications, and toward a general purpose localization infrastructure. The structure of the thesis is organized as follows. In Chapter 2, we characterize the response of localization algorithms to attacks where an adversary attenuates or amplifies the signal strength at one or more landmarks. We study both point-based and area-based methods that employ received signal strength for localization, and propose several performance metrics that quantify the estimator s precision and error, including Hölder metrics, which quantify the variability in position space for a given variability in signal strength space. We then conduct a trace-driven evaluation of several point-based and area-based algorithms, where we measured their performance as we applied attacks on real data from two different buildings. We observed both strong experimental and theoretic evidence that all the algorithms have similar average responses to signal strength attacks.

20 5 Next, in Chapter 3, we propose several attack detection schemes for wireless localization systems. We first formulate a theoretical foundation for the attack detection problem using statistical significance testing. Next, we define test metrics for two broad localization approaches: multilateration and signal strength. We then derived both mathematical models and analytic solutions for attack detection for any system that utilizes those approaches. We also studied additional test statistics that are specific to a diverse set of algorithms. Our trace-driven experimental results provide strong evidence of the effectiveness of our attack detection schemes with high detection rates and low false positive rates across both an (WiFi) network as well as an (ZigBee) network in two real office buildings. Further, wireless and sensor networks are especially vulnerable to identity-based spoofing attacks, which allows for many other forms of attacks in networks. It is desirable to detect the presence of spoofing and eliminate them from the network. Although the identity of a node can be verified through cryptographic authentication, authentication is not always possible because it requires key management and additional infrastructural overhead. In Chapter 4, we take a different approach by using the physical properties associated with wireless transmissions to detect spoofing. Specifically, we proposed a scheme for both detecting spoofing attacks, as well as localizing the positions of the adversaries performing the attacks. Our approach utilizes the Received Signal Strength (RSS) measured across a set of access points to perform spoofing detection and localization. We describe how we integrated our attack detector into a real-time indoor localization system, which is also capable of localizing the positions of the attackers. We show that the positions of the attackers can be localized using either area-based or point-based localization algorithms with the same relative errors as in the normal case through experimentation using both an (WiFi) network as well as an (ZigBee) network. In Chapter 5, We investigate the impact of landmark placement on localization performance using a combination of analytic and experimental analysis. Our analysis of landmark placement can find an optimal placement of landmarks in well-defined regular regions, thus making it quite suitable for indoor localization. The analysis places an upper bound of the maximum localization error given a set of landmark placements. We can show that this upper bound is minimized by a combination of minimizing the distance estimation error together with the employment of the optimal patterns for landmark placement. Using this result, we can compare

21 6 the maximum error between any two placements. We can then constrain a search of placements to minimize the maximum error. We have developed a simple algorithm called maxl mine algorithm that finds an optimized landmark deployment for the LLS algorithm. In summary, to ensure the trustworthiness of the location information, first we characterize the response of localization algorithms to signal strength attacks in Chapter 2. Then, in Chapter 3, we propose attack detection mechanisms in wireless localization. We next propose a method for both detecting and localizing spoofing attacks in Chapter 4. In addition, in order to improve the localization performance, we investigate the impact of landmark placement in Chapter 5. Moreover, we present the system architecture of our general purpose localization infrastructure in Chapter 6. In Chapter 7, we compare and contrast our work to the previous research work. Finally, we conclude our thesis in Chapter 8 and present future research directions. 1.3 Contributions Our contributions in this thesis are: We first characterized the response of localization algorithms to signal strength attacks. Specifically, we proposed a new set of metrics, Hölder metrics, which relate the magnitude of the perturbation in signal space to its effect on the localization result in physical space and thus measure the susceptibility of localization algorithms to signal attacks. We found that all the algorithms degraded gracefully, with a linear response as a function of the attack strength. And as a rule of thumb, it is easy to attack by 15 db and cause localization errors by 2-3 feet. We observed that the localization using Bayesian Networks is more robust than other algorithms under attacks that target individual landmarks. We developed a theoretical foundation of attack detection using statistical significance testing. We built test statistics for two broad localization approaches: multilateration and signal strength. For multilateration that uses Linear Least Squares, we derived a closed-form representation for the attack detector. Moreover, for localization schemes that employ signal strength, we showed that by utilizing the signal strength as a common feature, the minimum Euclidean distance in the signal space can be used as a test statistic for attack detection independent of

22 7 the localization process. The key advantage of our approach for signal strength based methods is that the detection phase can be performed prior to localization and thus results in localization computation cost savings under attack. Further, we derived additional test statistics for a selection of representative localization algorithms. We validated the effectiveness and generality of our attack detection schemes using a tracedriven evaluation across a diverse set of algorithms, networks, and buildings. we found that the performance of the different attack detection schemes are more similar than different. This result shows that different localization systems have similar attack detection capabilities, and consequently that system designers can focus on using algorithms that provide the highest localization accuracy rather than having to trade off position accuracy against attack detection abilities. Further, we developed a method for detecting spoofing attacks as well as localizing the adversaries in wireless and sensor networks. We applied our generic statistical significance testing formulation for spoofing detection problem. We then utilized the K-means cluster analysis to derive the spoofing detector. Moreover, we have built a real-time localization system and integrated our K-means spoofing detector into the system to locate the positions of the attackers and as a result to eliminate the adversaries from the network. our experimental results provide strong evidence of the effectiveness of our approach in detecting the spoofing attacks and localizing the positions of the adversaries. In addition, we took a distinctive approach to investigate the impact of landmark placement on localization performance. By analyzing the Linear Least Squares algorithm, we derived an upper bound on the maximum location error given the placement of landmarks. Based on this theoretical analysis, we found optimal patterns for landmark placement and further developed a novel algorithm, maxl mine, for finding optimal landmark placement that minimizes the maximum localization error. We found that the performance of a wide variety of algorithms showed significant improvements, about 3%, when using landmarks placed according to our algorithm, as opposed to alternate deployments. The experimental results provide strong evidence that our analysis and algorithm for landmark placement is very generic as the resulting placement has improved localization performance across a diverse set of algorithms, networks,

23 8 and ranging modalities. Finally, we presented a system prototype of a general purpose, real time, and scalable localization infrastructure. It aims to incorporate different localization properties and radios. In university research communities, this general purpose localization infrastructure enables researchers to explore issues beyond theoretical algorithms and simulation approaches. It makes further higher-level integrated research investigation possible including privacy studies and security services.

24 9 Chapter 2 Robustness Analysis of Localization Algorithms to Signal Strength Attacks 2.1 Introduction Out of the myriad of localization methods proposed over the last few years, algorithms that use Received Signal Strength (RSS) as the basis of localization are very attractive options as using RSS allows the localization system to reuse the existing communication infrastructure, rather than requiring the additional cost needed to deploy specialized localization infrastructure, such as ceiling-based ultrasound, GPS, or infrared methods [35, 56, 62]. In particular, all commodity radio technologies, such as 82.11, , and Bluetooth provide RSS values associated with packet reception, and thus localization services can easily be built for such systems. Further, RSS-based localization is attractive as the techniques are technology-independent: an algorithm can be developed and applied across different platforms, whether or Bluetooth. In addition, it provides reasonable accuracy with median errors of 1 to 5 meters [27]. However, as more location-dependent services are deployed, they will increasingly become tempting targets for malicious attacks. Adversaries may alter signal strength measurements for the purpose of accessing services that are based on location information (e.g. WLAN access may only be granted to devices inside of a building.). In this chapter, we thus investigate the susceptibility of a wide range of signal strength localization algorithms to attacks on the Received Signal Strength (RSS). Specifically, we examine the response of several localization algorithms to unanticipated power losses and gains, i.e. attenuation and amplification attacks. Conventional attacks, where an adversary injects false messages, can be isolated and protected against using traditional cryptographic methods, such as authentication. However, there is a completely orthogonal set of attacks that are non-cryptographic, where the measurement process itself can be corrupted by adversaries. Unfortunately, these non-cryptographic attacks

25 1 cannot be addressed by traditional security services. Thus, it is desirable to study the impact of these attacks on localization algorithms and explore methods to detect and further to eliminate these attacks from the network. Although there has been recent research on securing localization [17 19, 49, 51, 61], to date there has been no study on the robustness of the existing generation of RSS-based localization algorithms to physical attacks. Our evaluation is a valuable contribution to a wireless sensor network designer. Because it helps drive protocol decisions, and allows the engineer to decide whether more complicated secure localization algorithms are truly necessary. In the physical attacks that we study, the attacker modifies the RSS of a sensor node or landmark, for example, by placing an absorbing or reflecting material around the node or landmark. Notably, we study the set of representative attack scenarios on amplification or attenuation attacks on single landmark, all-landmark and combinations of landmarks. Further we analyze the results of simultaneous amplification and attenuation on multiple landmarks. We investigate both point-based and area-based algorithms that utilize RSS to perform localization. In order to evaluate the robustness of these algorithms, we provide a generalized characterization of the localization problem, and then present several performance metrics suitable for quantifying performance, including estimator angle bias, estimator distance error, and estimator precision. Additionally, an essential contribution of our work is the introduction of a new family of localization performance metrics, which we call Hölder metrics. These metrics quantify the susceptibility of localization algorithms to perturbations in signal strength readings. We use worst-case and average-case versions of the Hölder metric, which describe the maximum and average variability as a function of changes in the RSS. We then experimentally evaluate the performance of a wide variety of localization algorithms after applying attenuation and amplification attacks to real data measured from two different office buildings. Using experimentally observed localization performance, we found that the error for a wide variety of algorithms scaled with surprising similarity under attack. The single exception was the Bayesian Networks algorithm, which degraded slower than the others in response to attacks against a single landmark. In addition to our experimental observations, we found a similar average-case response of the algorithms using our Hölder metrics. However, we observed that methods which returned an average of likely positions had less variability and are thus less

26 11 susceptible than other methods. We also observed that all algorithms degraded gracefully, experiencing linear scaling in localization error as a function of the amount of loss or gain (in db) an attack introduced. This observation applied to various statistical descriptions of the error, leading us to conclude that no algorithm collapses in response to an attack. This is important because it means that, for all the algorithms we examined, there is no tipping point at which an attacker can cause gross errors. In particular, we found the mean error of most of the algorithms for both buildings scaled between ft/db when all the landmarks were attacked simultaneously, and.5-.8 ft/db when attacked a single landmark. Additionally, the performance of the mean response of algorithms with multiple landmarks under attack is between the all-landmarks attack and the single landmark attack, which scaled at ft/db. Further we observed that mixed attacks with simultaneous attenuation and amplification cause the mean response of algorithms to move faster, ranging from ft/db. More powerful affects were witnessed when the mixed attack was applied to landmarks that were further apart from each other. We also showed experimentally that RSS can be easily attenuated by 15 db, and that, as a general rule of thumb, very simple signal strength attacks can lead to localization errors of 2-3 ft. Finally, we conducted a detailed evaluation of area-based algorithms as this family of algorithms return a set of potential locations for the transmitter. Thus, it is possible that these algorithms might return a set with a larger area in response to an attack and could have less precision (or more uncertainty) under attack. However, we found all three of our area-based algorithms shifted the returned areas rather than increased returned area. Further, one of the algorithms, the Area Based Probability (ABP) scheme, significantly shrank the size of the returned area in response to very large changes in signal strength. The rest of this chapter is organized as follows. We begin, in Section 2.2, by giving an overview of the algorithms used in our performance study and discuss how signal strength attacks can be performed in Section 2.3. In Section 2.4, we provide a formal model of the localization problem as well as introduce the metrics that we use in this chapter. We then examine the performance of the algorithms through an experimental study in Section 2.5, and discuss the Hölder metrics for these algorithms in Section 2.6. Finally, we conclude in Section 2.7.

27 Localization Algorithms Signal strength is a common physical property used by a widely diverse set of algorithms. For example, most fingerpriting approaches utilize the RSS, e.g. [12, 14], and many multilateration approaches [52] use it as well. In spite of its several meter-level accuracy, using the RSS is an attractive approach because it can re-use the existing wireless infrastructure this feature presents a tremendous cost savings over deploying localization-specific hardware. In this chapter we thus focus on localization algorithms that employ signal strength measurements. In this section, we provide an overview of a representative set of algorithms selected for conducting performance analysis under attack. These algorithms use either deterministic or probabilistic methods for location estimation. There are several ways to classify localization schemes that use signal strength: rangebased schemes, which explicitly involve the calculation of distances to landmarks; and RF fingerprinting schemes whereby a radio map is constructed using prior measurements, and a device is localized by referencing this radio map. In this work, we focus on indoor signal strength based localization algorithms utilizing these approaches. We can further break down the algorithms into two main categories: point-based methods, and area-based methods Point-based Algorithms Point-based methods return an estimated point as a localization result. Here we describe a few representative point-based schemes for our study. RADAR (R1): A primary example of a point-based method is the RADAR scheme [12]. In R1, multiple base stations are deployed to provide overlapping coverage of an area, such as an office building. During set up, a mobile host with known position broadcasts beacons periodically, and the signal strength readings are measured at a set of fixed landmarks. Collecting together the averaged signal strength readings from each of the landmarks for different transmitter locations provides a radio map. After training, localization is performed by measuring a wireless device s RSS at each landmark, and the vector of RSS values is compared to the radio map. The record in the radio map whose signal strength vector is closest in the Euclidean sense

28 13 to the observed signal strength vector is declared to correspond to the location of the transmitter. Variations of RADAR, such as Averaged RADAR (R2) which returns the average of the closest 2 fingerprints and Gridded RADAR (GR) that uses the Interpolated Map Grid (IMG) as a set of additional fingerprints over the basic RADAR have been proposed in [27]. Highest Probability (P1): The P1 method uses a probabilistic approach by applying the statistical Bayes rule to return the point with the highest probability in the pre-constructed radio map as the location estimation result [59]. There are variations of Highest Probability. Averaged Highest Probability (P2) returns the mid-point of the top 2 training fingerprints. And like GR, Gridded Probability (GP) uses fingerprints based on an IMG [27] Area-based Algorithms On the other hand, area-based algorithms return a most likely area in which the true location resides. One of the major advantages of area-based methods compared to point-based methods is that they return a region, which has an increased chance of capturing the transmitter s true location. We study 3 area-based algorithms [27, 52], two of them, Simple Point Matching (SPM) and Area Based Probability (ABP), use an Interpolated Map Grid (IMG) and perform scene matching (fingerprint matching) for localization; and the other, Bayesian Networks (BN), is a multilateration algorithm. Simple Point Matching (SPM): In SPM, the floor is divided into small tiles. The strategy behind SPM is to find a set of tiles that fall within a threshold of the RSS for each landmark independently, then return the tiles that form the intersection of each landmark s set. We define the threshold as s i ± q, (2.1) where s i is the expected value of the RSS reading from Landmark i and q is an expected noise level. One way to choose q is to use the maximum of the standard deviation σ with σ = max{σ ij ;i {1..numberoflandmarks},j {1..numberofpoints}}. (2.2) SPM [27] is an approximation of the Maximum Likelihood Estimation (MLE) method. Area Based Probability (ABP): ABP returns a set of tiles bounded by a probability that the transmitter is within the returned tile set. The probability is called the confidence α and

29 14 Figure 2.1: The Bayesian Network under analysis. it is adjustable by user. ABP assumes the distribution of RSS for each landmark follows a Gaussian distribution with mean as the expected value of RSS reading vector s. The Gaussian random variable from each landmark is independent. ABP then computes the probability of the transmitter being at each tile L i on the floor using Bayes rule: P(L i s) = P(s L i) P(L i ). (2.3) P(s) Given that the transmitter must be at exactly one tile satisfying L i=1 P(L i S l ) = 1, ABP normalizes the probability and returns the most likely tiles up to its confidence α [27]. Bayesian Networks (BN): BN is a multilateration algorithm that encodes the signal-todistance propagation model into the Bayesian Graphical Model for localization [52]. In BN, the overall joint density of x X, where x is a random variable, only depends on the parents of x, denoted pa(x): p(x) = x X p(x pa(x)). (2.4) Once p(x) is computed, the marginal distribtution of any subset of the variables of the network can be obtained as it is proportional to overall joint distribution. Figure 2.1 shows the basic Bayesian Network used for our analysis. The vertices X and Y represent location; the vertex s i is the RSS reading from the ith landmark; and the vertex D i represents the Euclidean distance between the location specified by X and Y and the ith landmark. The value of s i follows a signal propagation model s i = b i + b 1i log D i, where b i,b 1i are the parameters specific to the ith landmark. The distance D i = (X x i ) 2 + (Y y i ) 2 in turn depends on the

30 15 location (X,Y ) of the measured signal and the coordinates (x i, y i ) of the ith landmark. The network models noise and outliers by modeling the s i as a Gaussian distribution around the above propagation model, with variance τ i : s i N(b i + b 1i log D i,τ i ). (2.5) The initial parameters (b i,b 1i,τ i ) of the model are unknown, and the training data is used to adjust the specific parameters of the model according to the relationships encoded in the network. Through Markov Chain Monte Carlo (MCMC) simulation, BN returns the sampling distribution of the possible location of X and Y as the localization result. The algorithms we have described in this section are summarized in Table 2.1. Although there are a variety of other signal strength based localization algorithms that may be studied, our results are general and can be applied to other point-based and area-based methods. 2.3 Conducting Signal Strength Attacks In this section, we study the feasibility of conducting signal strength attacks. We first discuss the possible attacks on signal strength. We then provide experimental results for signal strength going through various materials. Finally, we derive an attack model for our performance analysis of the robustness of localization algorithms Signal Strength Attacks The first step to tackle a security problem is to put oneself in the role of the adversary and attempt to understand the attacks. To attack signal-strength based localization systems, an adversary must attenuate or amplify the RSS readings. This can be done by applying the attack at the transmitting device, e.g. simply placing foil around the card; or by directing the attack at the landmarks. For example, we may steer the lobes and nulls of an antenna to target select landmarks. A broad variety of attenuation attacks can be performed by introducing materials between the landmarks and sensors [49]. In order to support the claim that physical attacks on received signal strength are feasible and capable of significantly affecting the results of a localization algorithm, we first examined the possibility of signal strength attacks. Next, we report results of actual experiments to

31 16 Table 2.1: Algorithms under study Algorithm Abbreviation Description Area-Based Simple Point Matching SPM Maximum likelihood matching of the RSS to an area using thresholds. Area Based Probability ABP-α Bayes rule matching of the RSS to an area probabilistically bounded by the confidence level α%. Bayesian Network BN Returns the most likely area using a Bayesian network approach. Point-Based RADAR R1 Returns the closest record in the Euclidean distance of signal space. Averaged RADAR R2 Returns the average of the top 2 closest records in the signal map. Gridded RADAR GR Applies RADAR using an interpolated grid signal map. Highest Probability P1 Applies maximum likelihood estimation to the received signal. Averaged Highest Probability P2 Returns the average of the top 2 likelihoods. Gridded Highest Probability GP Applies likelihoods to an interpolated grid signal map.

32 17 Attacked signal strength (dbm) Thin book Thick book Foil More foil Water Human body Glass Metal Signal Attenuation through Various Materials Non attacked signal strength (dbm) Figure 2.2: Signal strength when going through a barrier. quantify the effectiveness of various ways of attenuating/amplifying signal strength Experimental Results of Attacks Our experiments were performed in our laboratory in the 3rd floor of the CoRE building at Rutgers University, as shown in Figure 2.4 (a). There are 4 landmarks deployed in the 3rd floor of CoRE. We measured the RSS of beacon signals coming from each of the landmark. The RSS readings were collected using a laptop with an Orinoco Silver wireless card, using iwlist to sample the signal strength. In order to mitigate the effect of fluctuations, we collected samples once every second for 1 minutes, and averaged the signal strength over 6 samples. As noted earlier, an adversary may attack the signal strength by attenuating or amplifying the RSS readings. This can be done either at the receiver or at the transmitter. Our aim is to find the results of power loss in db by simple attacks. Therefore, in the experiments, we placed various obstruction materials close to the laptop s wireless card and measured the RSS values from each landmark at the laptop. The following obstructions were used: a thin book, a thick book, a layer of metal foil, three layers of foil (referred to as more foil), a mug filled with water (referred to as water), a glass mug (referred to as glass), a metal cabinet (referred to as metal), and a human body. These materials are easy to access and attacks utilizing these kind of materials can be simply performed with low cost. The original signal strength values, together with the signal strength measurements in the presence of these objects, are provided in Figure 2.2.

33 18 The points represent the measured data from experimental results for various materials, while the lines are the linear least-squares fitting. From these results, we have the following general observations: placing a blocking object between the transmitter and receiver can attenuate the signal strength; and, across different blocking objects, metal, foil, and human body are more effective than other blocking materials. Interestingly, we found that glass has the amplification effect on the signal strength. A more comprehensive study of propagation loss through common materials can be found in [58], and we note that more powerful attenuation loss is possible by using more advanced materials (such as RF-absorptive carbon fabric). Finally, we note that these results also imply that amplification is possible by removing a barrier (e.g. a door) of the corresponding material or through antenna-based methods Attack Model Based upon the results in Figure 2.2, we further see that there is a linear relationship between the unattacked signal strength and the attacked signal strength in db for various materials. The linear relationship implies that there is an easy way for an adversary to perform and control the effect of an attack on the observed signal strength by appropriately selecting different materials. Specifically, we envision that an adversary may suitably introduce and/or remove barriers of appropriate materials so as to attenuate and amplify the signl strength readings at one or more landmarks. Due to the observed linear relationship illustrated in Figure 2.2, we refer to this as the linear attack model. In the remainder of the chapter, we will use the linear attack model to describe the effect of an attack on the RSS readings at one or more landmarks. The resulting attacked readings are then used to study the consequent effects on localization for the algorithms surveyed above. In particular, in this study, we apply our attacks to individual landmarks, which might correspond to placing a barrier directly in front of a landmark, as well as to the entire set of landmarks, which corresponds to placing a barrier around the transmitting device. Similar arguments can be made for amplification attacks, whereby usually barriers are removed between the source and receivers. Moreover, we apply attenuation, amplification, or a mixture of simultaneous

34 19 attenuation and amplification attacks to multiple landmarks and study the performance of localization algorithms. The broad collection of our attack scenarios has covered the set of possibilities that an adversary could attempt to accomplish. Although there are many different and more complex signal strength attack methods that can be used, we believe their effects will not vary much from the linear signal strength attack model we use in this paper, and note that such sophisticated attacks could involve much higher cost to perform. 2.4 Measuring Attack Susceptibility The aim of a localization attack is to perturb a set of signal strength readings in order to have an effect on the localization output. When selecting a localization algorithm, it is desirable to have a set of metrics by which we can quantify how susceptible a localization algorithm is to varying levels of attack by an adversary. In this section, we shall provide a formal specification for an attack, and present several measurement tools for quantifying the effectiveness of an attack A Generalized Localization Model In order to begin, we need to specify a model that captures a variety of RF-fingerprinting localization algorithms. Let us suppose that we have a domain D in two-dimensions, such as an office building, over which we wish to localize transmitters. Within D, a set of n landmarks have been deployed to assist in localization. A wireless device that transmits with a fixed power in an isotropic manner will cause a vector of n signal strength readings to be measured by the n landmarks. In practice, these n signal strength readings are averaged over a sufficiently large time window to remove statistical variability. Therefore, corresponding to each location in D, there is an n-dimensional vector of signal readings s = (s 1,s 2,,s n ) that resides in a range R. This relationship between positions in D and signal strength vectors defines a fingerprint function F : D R that takes our real world position (x,y) and maps it to a signal strength reading s. F has some important properties. First, in practice, F is not completely specified, but rather a finite set of positions (x j,y j ) is used for measuring a corresponding set of signal strength vectors s j. Additionally, the function F is generally one-to-one, but is not onto. This

35 2 Figure 2.3: Interpretation of distances in location estimation. means that the inverse of F is a function G that is not well-defined: There are holes in the n-dimensional space in which R resides for which there is no well-defined inverse. It is precisely the inverse function G, though, that allows us to perform localization. In general, we will have a signal strength reading s for which there is no explicit inverse (e.g. perhaps due to noise variability). Instead of using G, which has a domain restricted to R, we consider various pseudo-inverses G alg of F for which the domain of G alg is the complete n-dimensional space. Here, the notation G alg indicates that there may be different algorithmic choices for the pseudo-inverse. For example, we shall denote G R to be the RADAR localization algorithm. In general, the function G alg maps an n-dimensional signal strength vector to a region in D. For point-based localization algorithms, the image of G alg is a single point corresponding to the localization result. On the other hand, for area-based methods, the localization algorithm G alg produces a set of likely positions. An attack on the localization algorithm is a perturbation to the correct n-dimensional signal strength vector s to produce a corrupted n-dimensional vector s. Corresponding to the uncorrupted signal strength vector s is a correct localization result p = G alg (s), while the corrupted signal strength vector produces an attacked localization result p = G alg ( s). Here, p and p are set-valued and may either be a single point or a region in D Attack Susceptibility Metrics We wish to quantify the effect that an attack has on localization by relating the effect of a change in a signal strength reading s to the resulting change in the localization result p. We

36 21 shall use p to denote the correct location of a transmitter, p to denote the estimated location (set) when there is no attack being performed, and p to denote the position (set) returned by the estimator after an attack has affected the signal strength. Figure 2.3 illustrates the relationship between the true location and the estimated locations. There are several performance metrics that we will use: Estimator Angle Bias: The perturbation on the signal strength vector caused by an attack will result in the variability of location estimation in the physical space. We want to investigate the bias along the angular dimension. That is, if we plot the relative error position in polar coordinates, for an unbiased estimator the error would have an equal probability of falling along any angle. However, when attacking a single landmark, we may expect an angular bias to be introduced. The estimation angle bias is studied by calculating the estimated position for different experimental trials, and comparing these results, in a spatial sense, to the true position. An angularly-unbiased algorithm should uniformly cover the 36 degrees around the true location. For area-based methods, we replace p, which is a set, with its median (along the x and y dimensions separately) to get a point. The angular bias is an important metric as it can serve as an indication as to whether an attacker can skew the localization result in a specific direction - algorithms with more angular bias are more skewable and hence worse choices for deployment since an adversary can use this knowledge to its advantage. Estimator Distance Error: An attack will cause the magnitude of p p to increase. For a particular localization algorithm G alg we are interested in the statistical characterization of p p over all possible locations in the building. The characterization of p p depends on whether a point-based method or an area-based method is used, and can be described via its mean and distributional behavior. For a point-based method, we may measure the cumulative distribution (cdf) of the error p p over the entire building. For area-based metrics, we calculate the CDF of the distance between the median of the estimated locations p med and the true location, i.e. p p med. The CDF provides a complete statistical specification of the distance errors. It is often more desirable to look at the average behavior of the error. For point-based methods,

37 22 the average distance error is simply E[ p p ], which is just the average of p p over all locations. Area-based methods allow for more options in defining the average distance error. First, for a particular value of p, p is a set of points. For each p, we get a collection of error values p q, as q varies over points in p. For each p, we may extract the minimum, 25th percentile, median, 75th percentile, and maximum. These quartile values of p q are then averaged over the different positions p. Estimator Precision: An area-based localization algorithm returns a set p. For localization, precision refers to the size of the returned estimated area. This metric quantifies the average value of the area of the localized set p over different signal strength readings s. Generally speaking, the smaller the size of the returned area, the more precise the estimation is. When an attack is conducted, it is possible that the precision of the answer p is affected. Precision vs. Perturbation Distance: The perturbation distance is the quantity p med p med. The precision vs. perturbation distance metric depicts the functional dependency between precision and increased perturbation distance. Hölder Metrics: In addition to error performance, we are interested in how dramatically the returned results can be perturbed by an attack. Thus, we wish to relate the magnitude of the perturbation s s to its effect on the localization result, which is measured by G alg (s) G alg ( s). In order to quantify the effect that a change in the signal strength space has on the position space, we borrow a measure from functional analysis [43], called the Hölder parameter (also known as the Lipschitz parameter) for G alg. The Hölder parameter H alg is defined via H alg = max s,v G alg (s) G alg (v) s v (2.6) where s and v are all the possible combinations of signal strength vectors in signal space. For continuous G alg, the Hölder parameter measures the maximum (or worst-case) ratio of variability in position space for a given variability in signal strength space. Since the traditional Hölder parameter describes the worst-case effect an attack might have, it is natural to also provide an average-case measurement of an attack, and therefore we

38 23 introduce the average-case Hölder parameter G alg (s) G alg (v) H alg = avg s,v. (2.7) s v These parameters are only defined for continuous functions G alg, and many localization algorithms are not continuous. For example, if we look at G R for RADAR, the result of varying a signal strength reading is that it will yield a stair-step behavior in position space, i.e. small changes will map to the same output and then suddenly, as we continue changing the signal strength vector, there will be a change to a new position estimate (we have switched over to a new Voronoi cell in signal space). In reality, this behavior does not concern us too much, as we are merely concerned with whether adjacent Voronoi cells map to close positions. We will revisit this issue in Section 2.6. Finally, we emphasize that Hölder metrics measure the perturbability of the returned results, and do not directly measure error. 2.5 Experimental Results In this section we present our experimental results. We first describe our experimental method. Next, we examine the impact of attacks on the RSS to localization bias and localization error under different attacking scenarios. We then quantify the algorithms linear responses to RSS changes. Finally, we present a precision study that investigates the impact of attacks on the returned areas for area-based algorithms Experimental Setup Figure 2.4 shows our experimental set up. The floor map on the left, (a) is the 3rd floor of the CoRE building at Rutgers, which houses the computer science department and has an area of 2x8ft (16 ft 2 ). The other floor shown in (b) is an industrial research laboratory (we call the Industrial Lab), which has an area of 225x144ft (324 ft 2 ). The stars are the training points, the small dots are testing points, and the larger squares are the landmarks, which are access points. Notice that the 4 CoRE landmarks are more co-linear than the 5 landmarks in the Industrial Lab.

39 Landmark Landmark 4 Y (feet) 4 Landmark 2 Landmark X (feet) (a) CoRE Building Y (feet) Landmark 1 Landmark 2 Landmark Landmark 4 Landmark X (feet) (b) Industrial Lab Figure 2.4: Deployment of landmarks and training locations on the experimental floors For both attenuation and amplification attacks, we ran the algorithms but modified the RSS of the testing points. We altered the RSS by +/-5 db to +/-25 db, in increments of 5 db. We experimented with different ways to handle signals that would fall below the detectable threshold of -92 dbm for our cards. We found that substituting the minimal signal (-92 dbm) produced about the same localization results and did not require changing the algorithms to special case missing data. We experimented with different training set sizes, including 2, 35, 6, 85, 115, 145, 185, 215, 245, 253, and 286 points. Experimental data was collected at a total of 286 locations in the CoRE building and at a total of 253 locations in the Industrial Lab. Although there are some small differences, we found that the behavior of the algorithms matches previous results [27] and varied little after using 115 training points. We therefore chose to use a training set size of 115 for this study.

40 25 8 Relative Position Estimation 8 Relative Position Estimation Y (feet) Y (feet) X (feet) X (feet) (a) Normal (b) 25dB attack on all landmarks 8 Relative Position Estimation 8 Relative Position Estimation Y (feet) Y (feet) X (feet) X (feet) (c) 25dB attack on Landmark 1 (d) 25dB attack on Landmark 3 Figure 2.5: ABP: Localization estimation relative to the true locations for the Industrial Lab Localization Angle Bias In this section, we study the angular bias of the localization schemes introduced by signal strength attacks. For the Industrial Lab, Figure 2.5(a) shows the localization result of ABP under no attack for the relative estimation positions to the true locations, setting as the origin, over all the localization attempts. The normal performance of the algorithms are unbiased with the localization results uniformly distributed around the true locations. Figure 2.5(b) is the relative position estimation results under 25dB attenuation attack on all landmarks, while Figure 2.5(c) and Figure 2.5(d) show the attacked results on single landmarks, landmark 1 and landmark 3, respectively. Figure 2.4(b) shows that landmark 1 and landmark 3 are placed in diagonal positions across the Industrial Lab. We have observed that signal strength attacks have affected the localization schemes by introducing angular bias on the results with the location estimation more likely to be in the fourth quadrant relative to the true location

41 26 when landmark 1 is attacked, as shown in Figure 2.5(c). Because landmark 1 is placed in the upper left corner in the building floor map shown in 2.4, signal attenuation on landmark 1 made the localization system think the sensor node is farther away from landmark 1, and thus the resulting localization results under attack have been pushed into the fourth quadrant. This effect has been proved by examining the localization results when landmark 3 is under attack. As presented in Figure 2.5(d), the relative localization results are mostly in the second quadrant since landmark 3 is placed in the lower right corner of the building floor map. Further, as expected, for simultaneous landmark attacks, the localization results are distributed around the true locations randomly, but with much larger estimation errors as presented in Figure 2.5(b). We have observed similar effects for the other algorithms in the Industrial Lab and the CoRE building.

42 27 1 Error CDF Across Algorithms 1 Error CDF Across Algorithms Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) (a) CoRE: No attack (b) CoRE: 1dB attenuation 1 Error CDF Across Algorithms 1 Error CDF Across Algorithms Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) (c) CoRE: 25dB attenuation (d) CoRE: 1dB amplification 1 Error CDF Across Algorithms 1 Error CDF Across Algorithms Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) (e) Industrial: No attack (f) Industrial: 1dB attenuation 1 Error CDF Across Algorithms 1 Error CDF Across Algorithms Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) (g) Industrial: 25dB attenuation (h) Industrial: 1dB amplification Figure 2.6: Error CDF across localization algorithms when attacks are performed on all the landmarks.

43 28 1 Error CDF Across Algorithms 1 Error CDF Across Algorithms Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) (a) CoRE: Attenuation, Landmark 1 (b) CoRE: Attenuation, Landmark 2 1 Error CDF Across Algorithms 1 Error CDF Across Algorithms Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) (c) CoRE: Amplification, Landmark 1 (d) CoRE: Amplification, Landmark 2 1 Error CDF Across Algorithms 1 Error CDF Across Algorithms Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) (e) Industrial: Attenuation, Landmark 4 (f) Industrial: Attenuation, Landmark 5 1 Error CDF Across Algorithms 1 Error CDF Across Algorithms Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) (g) Industrial: Amplification, Landmark 4 (h) Industrial: Amplification, Landmark 5 Figure 2.7: Error CDF across localization algorithms when attacks are performed on an individual landmark. The attack is 25dB of signal attenuation and signal amplification respectively.

44 29 1 Error CDF Across Algorithms 1 Error CDF Across Algorithms Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) (a) 1dB: Landmark 1 and 2 (b) 25dB: Landmark 1 and 2 1 Error CDF Across Algorithms 1 Error CDF Across Algorithms Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) (c) 1dB: Landmark 1 and 3 (d) 25dB: Landmark 1 and 3 1 Error CDF Across Algorithms 1 Error CDF Across Algorithms Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) (e) 1dB: Landmark 2 and 4 (f) 25dB: Landmark 2 and 4 Figure 2.8: CoRE: Error CDF across localization algorithms when attenuation attacks are performed on multiple landmarks.

45 3 1 Error CDF Across Algorithms 1 Error CDF Across Algorithms Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) (a) 1dB: Landmark 1 amplification (b) 25dB: Landmark 1 amplification and landmark 2 attenuation and landmark 2 attenuation 1 Error CDF Across Algorithms 1 Error CDF Across Algorithms Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) (c) 1dB: Landmark 1 amplification (d) 25dB: Landmark 1 amplification and landmark 3 attenuation and landmark 3 attenuation 1 Error CDF Across Algorithms 1 Error CDF Across Algorithms Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) Probability.6 SPMmed.5 BNmed ABP75min.4 ABP75med ABP75max.3 R1 R2.2 GR P1.1 P2 GP Error (feet) (e) 1dB: Landmark 2 amplification (f) 25dB: Landmark 2 amplification and landmark 4 attenuation and landmark 4 attenuation Figure 2.9: CoRE: Error CDF across localization algorithms when amplification and attenuation attacks are simultaneously performed on multiple landmarks.

46 Localization Error Analysis In this section, we analyze the estimator distance error through the statistical characterization of p p by presenting the error CDFs of all the algorithms as a function of attenuation and amplification attacks. The CDF provides a complete statistical specification of the distance errors. Specifically, we study the localization error under four attack scenarios: an all-landmark attack; a single landmark attack; attacks involving multiple landmarks; and attacks involving simultaneous amplification and attenuation on multiple landmarks. As a baseline, Figure 2.6(a) shows the normal performance of the algorithms for the CoRE building and (e) shows the results for the Industrial Lab. For the area-based algorithms, the median tile error is presented, as well as the minimum and maximum tile errors for ABP-75. As in previous work, the algorithms all obtain similar performance, with the exception of BN, which slightly under-performs the other algorithms. First, we look at the performance of localization algorithms under an all-landmark attack. Figures 2.6(b) and 2.6(c) show the error CDFs under simultaneous landmark attenuation attacks of 1 and 25 db for CoRE, respectively, while Figure 2.6(f) and 2.6(g) show the similar results in the industrial lab. First, the bulk of the curves shift to the right by roughly equal amounts: no algorithm is qualitatively more robust than the others. Comparing the two buildings, the results show that the industrial lab errors are slightly higher for attacks at equal db, but again, qualitatively the impact of the building environment is not very significant. Figures 2.6(d) and 2.6(h) show the error CDFs for the CoRE and Industrial Lab under a 1 db amplification attack. The results are qualitatively symmetric with respect to the outcome of the 1dB attenuation attack. We found that, in general, comparing amplifications to attenuations of equal db, the errors were qualitatively the same. An interesting feature is that in CoRE the minimum error for ABP-75 also shifts to the right by roughly the same amount as the other curves. Figures 2.6(a) and 2.6(e) show that, in the non-attacked case, the minimum tile error for ABP-75 is quite small, meaning that the localized node is almost always within or very close to the returned area. However, under attacks, the closest part of the returned area moves away from the true location at the same rate as the median tile. We observed similar effects for the SPM and BN algorithms. We noticed

47 32 that under large attacks around 25dB, the median error CDF curves in the Industrial Lab have similar performance to those from the CoRE building, but there are two curves that seem to be outliers, namely ABP75min and ABP75max. These two curves represent the best and the worst cases from the ABP algorithm and we see that they are not moving at the same speed as the median errors, when compared with the results of the CoRE building. This tells us that the variance/spread of the performance of area-based algorithms in the Industrial Lab has increased under an all-landmark attack, but that the average behavior is consistent across the two buildings. We then examine attacks against a single landmark. We found attacks against certain landmarks had a much higher impact than against others in the CoRE building. Figure 2.7(a) and 2.7(b) show the difference in the error CDF by comparing attacks of landmarks 1 and 2. Figure 2.4(a) shows that landmark 1 is at the left end of the building, while landmark 2 is in the center and is close to landmark 4. The tail of the curves in Figure 2.7(a) are much worse than for 2.7(b), showing that when landmark 1 is attacked, significantly more high errors are returned. Figures 2.7(c) and 2.7(d) show a similar effect for amplification attacks. This is because landmark 1 is at one end of the building alone. The contribution of the signal strength reading from landmark 1 plays an important role in localization, while the contribution of landmark 2 can be reduced by the contribution from the nearby landmark 4 when under attack. The Industrial Lab results in Figures 2.7(e)-(h) show much less sensitivity to landmark placement compared to the CoRE building. Figure 2.4(b) shows that landmark 5 is centrally located and we initially suspected this would result in increased attack sensitivity. However, the error CDFs show that the remaining 4 landmarks provide sufficient coverage: as landmark 5 is attacked, the error CDFs are not much different from attacking landmark 4. The landmark placement in the CoRE building is colinear (to maximize the signal coverage in the floor), while the landmark placement in the Industrial Lab is more close to the optimal landmark placement for location accuracy. We believe that the better landmark placement for localization [2] in the Industrial Lab can account for the localization performance being less sensitive to landmark placement under attack. Next, we study attacks on more than one landmark, but not on all landmarks. Figure 2.8 present the localization results in the CoRE building when attenuation attacks are performed on

48 33 multiple landmarks, specifically on landmark pairs, 1 and 2, 1 and 3, and 2 and 4. We found that attacks on landmark pair 1 and 3 shown in Figure 2.8(d) cause larger errors compared to results in Figure 2.8(b) and (f) when attacking landmark pairs 1 and 2, and 2 and 4. Since landmarks 1 and 3 are placed at two ends of the building alone, the contribution of the RSS reading from these two landmarks is significant compared to the readings from landmark 2 and 4, which are closely placed and can cover each other. In general, the impact of the multiple landmark attacks on the localization performance is between the performance of a single landmark attack and an all-landmark attack. Fourthly, we look at the attack scenario that the adversary simultaneously performs both amplification and attenuation attacks on multiple landmarks. The localization results are presented in Figure 2.9 for the CoRE building. For a direct comparison, we present results when mixed attacks are applied on landmark pairs, 1 and 2, 1 and 3, and 2 and 4. we should expect that such an attack would be more effective in falsifying the location results, and this is what we observe. But, beyond this, we observe that the performance depends heavily upon which landmarks are attacked. We found that if the attacked landmarks are close to each other such as landmark 2 and 4, which are located in the center of the building, the effects of amplification and attenuation attacks are canceled out. Thus the impact of mixed attacks does not lead to significant perturbation in the localization results, as shown in Figure 2.9(f), which is about the same as under single landmark attacks displayed in Figure 2.7. However, if the attacked landmarks are farther away from each other, such as landmark 1 and 3, which are located at opposite ends of the building, the simultaneous amplification and attenuation attacks can be very harmful and cause larger localization errors for all the algorithms presented in Figure 2.9(d). The behavior of the error CDFs in Figure 2.9(d) is qualitatively different than others with very long tails. The effect of the amplification attack on landmark 1 and the attenuation attack on landmark 3 pushed the localization results further in one direction, and thus introduced large localization bias. The four attack scenarios we studied have covered a broad collection of possible combinations of signal strength attacks. We found that simultaneously attacking all landmarks has more impact on localization performance than attacking an individual landmark. Further, simultaneous amplification and attenuation attacks on certain landmarks can cause qualitatively larger

49 34 errors than other kinds of attacks. Most importantly, we observed that none of the localization algorithms outperforms the others for the attacks we examined Linear Response In this section, we show that the average distance error, E[ p p ], of all the algorithms scales in a linear way to attacks. That is, the mean localization error changes linearly with respect to the size of the signal strength change introduced in db (recall db is a log-scaled change in power). Figure 2.1 plots the median error vs. RSS attenuation for an all-landmark attack in Figure 2.1(a) and 2.1(e), and for individual landmarks in the other figures. Figure 2.11 plots the median localization error under simultaneous signal strength attenuation and amplification attacks on multiple attacks. Points are data derived from experimental results, and the lines are linear least-squares fits. The most important feature is that, in all cases, the median responses of all the algorithms fits a line extremely well, with an average R 2 -statistic of.97 for both the CoRE and Industrial Lab. The mixed attacks with amplification attack on landmark 1 and attenuation attack on landmark 3 in CoRE shown in Figure 2.11(d) is an exceptional case with R 2 of.86 as the worst case. Comparing the slopes across all the algorithms presented in Tables 2.2, 2.3, and 2.4, we found a mean change in positioning error vs. signal attenuation of 1.55 ft/db under an alllandmark attack with a minimum of 1.3 ft/db and maximum of 1.8 ft/db. For the single landmark attack, the slope was substantially less,.64 ft/db, although BN degrades consistently less than the other algorithms at.44 ft/db. Under attenuation attacks on multiple landmarks, the localization algorithms move at the speed of.9 ft/db to 1.4 ft/db, which is between the results of a single landmark attack and an all-landmark attack. However, the median error moves faster under simultaneous amplification and attenuation attacks on landmark 1 and 3, at the speed of ft/db as shown in Table 2.4. We note the mean error tops out when the attack strength is 25dB. This confirms our analysis in Figure 2.9(d) that applying simultaneous amplification and attenuation attacks on landmarks that are farther apart causes larger impacts on the performance of localization schemes, although in practice it is hard for an adversary to

50 35 Error (feet) SPM ABP BN R1 R2 GR P1 P2 GP Median Mean Error Error (feet) SPM ABP BN R1 R2 GR P1 P2 GP Median Mean Error Signal attenuation (db) Signal attenuation (db) (a) CoRE: all Landmarks (b) CoRE: Landmark 1 Error (feet) SPM ABP BN R1 R2 GR P1 P2 GP Median Mean Error Error (feet) SPM ABP BN R1 R2 GR P1 P2 GP Median Mean Error Signal attenuation (db) Signal attenuation (db) (c) CoRE: Landmark 2 (d) CoRE: Landmark 3 Error (feet) SPM ABP BN R1 R2 GR P1 P2 GP Median Mean Error Error (feet) SPM ABP BN R1 R2 GR P1 P2 GP Median Mean Error Signal attenuation (db) Signal attenuation (db) (e) Industrial: all Landmarks (f) Industrial: Landmark 1 Error (feet) SPM ABP BN R1 R2 GR P1 P2 GP Median Mean Error Error (feet) SPM ABP BN R1 R2 GR P1 P2 GP Median Mean Error Signal attenuation (db) Signal attenuation (db) (g) Industrial: Landmark 2 (h) Industrial: Landmark 5 Figure 2.1: Average location estimation error across localization algorithms under signal strength attenuation attack.

51 36 Error (feet) SPM ABP BN R1 R2 GR P1 P2 GP Median Mean Error Error (feet) SPM ABP BN R1 R2 GR P1 P2 GP Median Mean Error Signal attenuation (db) Signal attenuation (db) (a) Simultaneous attenuation attacks (b) Amplification attack on Landmark 1 on landmarks 1 and 2 and attenuation attack on Landmark 2 Error (feet) SPM ABP BN R1 R2 GR P1 P2 GP Median Mean Error Error (feet) SPM ABP BN R1 R2 GR P1 P2 GP Median Mean Error Signal attenuation (db) Signal attenuation (db) (c) Simultaneous attenuation attacks (d) Amplification attack on Landmark 1 on landmarks 1 and 3 and attenuation attack on Landmark 3 Error (feet) SPM ABP BN R1 R2 GR P1 P2 GP Median Mean Error Error (feet) SPM ABP BN R1 R2 GR P1 P2 GP Median Mean Error Signal attenuation (db) Signal attenuation (db) (e) Simultaneous attenuation attacks (f) Amplification attack on Landmark 2 on landmarks 2 and 4 and attenuation attack on Landmark 4 Figure 2.11: CoRE: Average location estimation error across localization algorithms under simultaneous signal strength attenuation and amplification attacks on multiple landmarks.

52 37 Table 2.2: CoRE: Slopes of Average Error from Linear Regression for attenuation attacks on all landmarks and individual landmark Buildings CoRE: attenuation attack Landmarks All Area-Based SPM ABP BN Point-Based R R GR P P GP Average Table 2.3: Industrial: Slopes of Average Error from Linear Regression for attenuation attacks on all landmarks and individual landmark Buildings Industrial Lab: attenuation attack Landmarks All Area-Based SPM ABP BN Point-Based R R GR P P GP Average conduct simultaneous amplification and attenuation attacks without using sophisticated equipment. In general, the linear fit results are quite important as it means that no algorithm has a cliff where the average positioning error suffers a catastrophic failure under attack. Instead, it remains proportional to the severity of the attack. While the median error characterizes the overall response to attacks, it does not address whether an attacker can cause a few, large errors. We examined the response of the maximum error as a function of the strength of the attack on an all-landmark attack, i.e. how the 1 th percentile error scales as a function of the change in db under an all-landmark attack. The all-landmark attack corresponds to a common attack scenario. It is thus desirable to study the worst-case situation under an all-landmark attack. We note that this characterization is not

53 38 Table 2.4: CoRE: Slopes of Average Error from Linear Regression for mixed attacks of signal attenuation and amplification on multiple landmarks Buildings attenuation attacks amplification and attenuation attacks Landmarks 1 and 2 1 and 3 2 and 4 1 and 2 1 and 3 2 and 4 Area-Based SPM ABP BN Point-Based R R GR P P GP Average Error (feet) SPM ABP BN R1 R2 GR P1 P2 GP Worst case Error Signal attenuation (db) Figure 2.12: CoRE: Maximum error as a function of attack strength from an all-landmark attack. the same as, nor is directly related to, the Hölder metrics. Those metrics define the rates of change between physical and signal space within the localization function itself, while here we characterize the change in the estimator error to the change in signal, i.e. p p / s v. Figure 2.12 plots the worst-case error for each algorithm as a function of signal db for the CoRE building under an all-landmark attack. The figure shows that almost all the responses are again linear, with least-squares fits of R 2 values of.84 or higher, though SPM does not have a linear response. The second important point is the algorithms responses vary, falling into three groups. BN, R1 and R2 are quite poor, with the worst case error scaling at about 4 ft/db. P1 and P2, are in a second class, scaling at close to 3 ft/db. The gridded algorithms, GP and GR,

54 39 Sampling x coordinates Sampling x coordinates.5 Landmark 1 Landmark 2 Landmark 3 Landmark 4.5 Landmark 1 Landmark 2 Landmark 3 Landmark Percentage.3 Percentage Signal attenuation (dbm) Signal attenuation (dbm) (a) CoRE: all landmarks (b) CoRE: Landmark 1 Figure 2.13: Contribution of each Landmark during sampling in the BN algorithm under attenuation attacks. as well as ABP-75 fair better, scaling at 2 ft/db or less. Finally, SPM is in a class by itself, with a poor linear fit (R 2 of.61) and the maximum error topping out at about 85 ft after 15 db of attack. Examining the error CDFs and the maximum errors, we can see that most of the localizations move fairly slowly in response to an attack, at about 1.5 ft/db. However, for some of the algorithms, particularly BN, R1 and R2, the top part of the error CDF moves faster, at about 4 ft/db. What this means is that, for a select few points, an attacker can cause more substantial errors of over 1 ft. However, at most places in the building, an attack can only cause errors with much less magnitude. Figure 2.1 show that BN is more robust compared to other algorithms for individual landmark attacks. Recall BN uses a Monte-Carlo sampling technique (Gibbs sampling) to compute the full joint-probability distribution for not just the position coordinates, but also for every node in the Bayesian network. Under a single landmark attack we found the network reduces the contribution of network nodes directly affected by the attacked landmark to the full jointprobability distribution while increasing other landmarks contributions. In effect, the network discounts the attacked landmark s contribution to the overall joint-density because the attacked data from that landmark is highly unlikely given the training data. To show this effect we developed our own Gibbs sampler so that we could observe the relative contributions of each node in the Bayesian network to the final answer. Figure 2.13

55 Y (feet) 4 Y (feet) X (feet) X (feet) (a) SPM: no attack (b) SPM: under 25dB attack Y (feet) 4 Y (feet) X (feet) X (feet) (c) ABP: no attack (d) ABP: under 25dB attack Y (feet) 4 Y (feet) X (feet) X (feet) (e) BN: no attack (f) BN: under 25dB attack Figure 2.14: CoRE: Comparison of localization results from the area-based algorithms for a testing point. shows the percentage contribution for each landmark to overall joint-density. For instance, in CoRE, the contribution of each landmark starts almost uniformly. When Landmark 1 under attack, the contribution of Landmark 1 goes from.25 down to Precision Study In this section, we examine the area-based algorithms precision in response to attacks. Figure 2.14 shows a localization example of the area-based algorithms in the CoRE building. The actual point is shown as a big dot and the convex hulls of the returned areas are outlined. Normally, the SPM and ABP algorithms perform similarly, while the BN algorithm has a much

56 41 1 Precision CDF 1 Precision CDF Probability Probability normal 5 db.2 1 db 15 db.1 2 db 25 db Area (feet 2 ).3 normal 5 db.2 1 db 15 db.1 2 db 25 db Area (feet 2 ) (a) CoRE: SPM - Attenuation (b) CoRE: BN - Attenuation 1 Precision CDF 1 Precision CDF Probability Probability normal 5 db.2 1 db 15 db.1 2 db 25 db Area (feet 2 ).3 normal 5 db.2 1 db 15 db.1 2 db 25 db Precision (feet 2 ) (c) CoRE: ABP - Attenuation (d) Industrial: ABP - Amplification Figure 2.15: Analysis of precision CDF across area-based algorithms. The attack is performed on all the landmarks. different profile by returning the sampling distribution of the possible estimation. Under signal strength attacks, We observed that the returned areas are reduced and shifted from the true location. Figure 2.15 shows the CDF of the precision (i.e. size of the returned area) for different areabased algorithms under attack for all the landmarks in CoRE and Industrial Lab. We found that overall the algorithms did not become less precise in response to attacks, but rather, the algorithms tended to shift and shrink the returned areas. Figure 2.15(a) shows a small average shrinkage for SPM in the CoRE building, and likewise, 2.15(b) shows a similar effect for BN. ABP-75 had the most dramatic effect. Figures 2.15(c) and 2.15(d) show the precision versus the attack strength for both buildings. The shrinkages are quite substantial. We found that, under attack, the probability densities of the tiles shrank to small values that were located on a few tiles reflecting the fact that an attack causes there not to be a likely position to localize a node. We also found that this effect held for amplification attacks, as is shown in

57 SPMmed ABPmed BNmed 35 3 SPMmed ABPmed BNmed Average precision (feet 2 ) Average precision (feet 2 ) Distance difference (feet) Distance difference (feet) (a) CoRE: all landmarks (b) CoRE: Landmark 1 Figure 2.16: Precision vs. perturbation distance under attenuation attack. Table 2.5: Analysis of (worst-case) H and (average-case) H Algorithms CoRE: H LAB: H CoRE: H LAB: H Area-Based SPM ABP BN Point-Based R R GR P P GP Figure 2.15(d). The shrinking precision behavior may be useful for attack detection, although a full characterization of how this effect occurs remains for future work. Examining this effect further, Figure 2.16 presents the precision vs. the perturbation distance p med p med, with a least squares line fit. Figure 2.16(a) shows the effect when attacking all landmarks on the CoRE building. Figure 2.16(b) shows a downward trend, but much weaker, when one landmark is under attack. We observed similar results for the Industrial Lab. We see mostly linear changes in precision in response to attacks, although with great differences between the algorithms. The figures show that the decrease in precision as a function of db is particularly strong for ABP-75.

58 Discussion about Hölder Metrics In the previous section we examined the experimental results, and looked at the performance of a set of representative localization algorithms in terms of error and precision. We now focus on the performance of these localization algorithms in terms of the Hölder metrics. The Hölder metrics measure the variability of the returned answer in response to changes in the signal strength vectors. We first discuss the practical aspects of measuring H and H for different algorithms. In Section 2.4, the Hölder parameters are defined by calculating the maximum and average over the entire n-dimensional signal strength space. In practice, it is necessary to perform a sampling technique to measure H and H. Additionally, as noted earlier, the definition of H and H are only suitable for (Hölder) continuous functions G alg. In reality, several localization algorithms, such as RADAR, are not continuous and involve the tessellation of the signal strength space into Voronoi cells V j, and thus only a discrete set of localization results are produced (image of V j under G alg ). Hence, for any s V j we have G R (s) = (x j,y j ). Unfortunately, for neighboring Voronoi cells, we may take s V j and v V i such that they are arbitrarily close (i.e. s v ), while G R (s) G R (v). In such a case, the formal calculation of H and H is not possible. However, for our purposes, we are only interested in measuring the notion of adjacency of Voronoi cells in signal space yielding close localization results. Thus, our calculation of H and H is only performed over the centroids of the various Voronoi cells for localization algorithms that tessellate of signal strength space. The Hölder parameters for the different localization algorithms are presented in Table 2.5. Examining these results, there are several important observations that can be made. First, if we examine the results for H we see that, for each building, all of the algorithms have very similar H values. Hence, we may conclude that the average variability of the returned localization result to a change in the signal strength vector is roughly the same for all algorithms. This is an important result as it means, regardless of which RF fingerprinting localization system we deploy, the average susceptibility of the returned results to an attack is essentially identical. However, if we examine the results for H, which reflects the worst-case susceptibility, then we see that there are some differences across the algorithms. First, comparing H and H for

59 44 both point-based and area-based algorithms, we see that the worst-case variability can be much larger than the average variability. Additionally, the point-based methods appear to cluster. Notably, RADAR (R1) and Gridded Radar (GR) have similar performance across both CoRE and the Industrial Lab, while averaged RADAR (R2) and averaged Highest Probability (P2) have similar performance across both buildings. A very interesting phenomena is observed by looking at the algorithms that returned an average of likely locations (R2 and P2). Across both buildings these algorithms exhibited less variability compared to other algorithms. This is to be expected as averaging is a smoothing operation, which reduces variations in a function. This observation suggests that R2 and P2 are more robust from a worst-case point-of-view than other point-based algorithms. 2.7 Conclusion In this chapter, we analyzed the robustness of RF-fingerprinting localization algorithms to attacks that target signal strength measurements. We first examined the feasibility of conducting amplification and attenuation attacks, and observed a linear dependency between non-attacked signal strength and attacked signal strength readings for different barriers placed between the transmitter and a landmark receiver. We provided a set of performance metrics for quantifying the effectiveness of an attenuation/amplification attack. Our metrics included localization angular bias, localization error, the precision of area-based algorithms, and a new family of metrics, called Hölder metrics, that quantify the variability of the returned location results versus change in signal strength vectors. We conducted a trace-driven evaluation of several point-based and area-based localization algorithms where the linear attack model was applied to data measured in two different office buildings. We found that the localization error scaled similarly for all algorithms under attack. Further, we found that, when attacked, area-based algorithms did not experience a degradation in precision although they experienced degradation in accuracy. We then examined the variability of the localization results under attack by measuring the Hölder metrics. We found that all algorithms had similar average variability, but those methods returned the average of a set of

60 45 most likely positions exhibited less variability. This result suggests that the average susceptibility of the returned results to an attack is essentially identical across point-based and area-based algorithms, though it might be desirable to employ either area-based methods or point-based methods that perform averaging in order to lessen the worst-case effect of a potential attack.

61 46 Chapter 3 Attack Detection in Wireless Localization 3.1 Introduction In this chapter, we examine the problem of detecting attacks on wireless localization. We present a general formulation for attack detection using statistical significance testing and then build tests that are applicable to broad classes of multilateration and signal strength-based methods, as well as several other test statistics that are unique to a variety of different localization algorithms. Multilateration is a popular localization approach that uses Least Squares (LS) techniques to perform localization [23, 28, 44, 49, 54], and has the desirable property of supporting mathematical analysis, in part because LS-based regression has well-known statistical descriptions when operating near ideal conditions. By examining Linear Least Squares (LLS), we build a mathematical model and derive an analytic solution for attack detection using the residuals of an LLS regression. We show that attack detection using LLS is easy to conduct and is suitable for both single-hop and multi-hop ranging methods because it is independent of the ranging modality used by the localization system. On the other hand, many signal strength based algorithms [12, 27] rely on either statistical inference or machine-learning in the context of scene matching to perform localization, and consequently do not yield closed-form solutions. However, for algorithms based on signal strength, we found that the minimum distance between an observation and the database of signal strength vectors is a good test statistic to perform attack detection. One key advantage of our approach for signal strength based methods is that the detection phase can be performed before localization.

62 47 To evaluate the effectiveness of our attack detection mechanisms we first present experimental results illustrating the feasibility of physical attacks on localization. We then conducted a trace driven evaluation using both an (WiFi) network as well as an (Zig- Bee) network in two real office buildings. In particular, we applied signal strength attenuation and amplification, using a linear attack model obtained from our experiments, to the Received Signal Strength (RSS) readings collected from these two office buildings. We evaluated the performance of our attack detection schemes using detection rates and receiver operating characteristic curves. Our experimental results provide strong evidence of the effectiveness of our attack detection schemes with high detection rates, over 95%, and low false positive rates, often under 5%. Surprisingly, we found that most of the attack detection schemes provide qualitatively similar performance. This shows that the different localization systems have similar attack detection capabilities. The rest of the chapter is organized as follows. We study the feasibility of attacks and present our experimental methodologies in Section 3.2. We present our generalized theoretical formulation for the attack detection problem in Section 3.3. We next derive an analytic solution for attack detection using Least Squares in Section 3.4. Using common features for attack detection in signal strength based algorithms is presented in Section 3.5. We study the test statistics that are specific to a variety of different algorithms in Section 3.6. Then, we provide a discussion in Section 3.7. Finally, we conclude in Section Feasibility of Attacks In this section we provide background on how attackers can impact the localization system. We next discuss the feasibility of conducting these attacks on signal strength, and provide the experimental methodology that we use to evaluate our attack detection mechanisms later in this chapter.

63 Localization Attacks Localization mechanisms are built upon different ranging modalities, such as RSS, TOA, AOA, and hop count. These all rely on the measurement of the physical properties of the wireless system. Adversaries can apply non-cryptographic attacks against the measurement processes, bypassing conventional security services, and as a result can affect the localization performance. For example, wormhole attacks tunnel through a faster channel to shorten the observed distance between two nodes [36]. An attenuation attack would decrease the radio range, and thus potentially lengthen the hop-count. Compromised nodes may delay response messages to disrupt distance estimation [49]. RSS readings can be altered due to attenuation or amplification of the signal strength by an adversary [22]. A broad survey of the potential non-cryptographic attacks that are unique to localization can be found in [49] Signal Strength Attacks We choose to use RSS as the ranging modality for localization algorithms. An adversary can attack the wireless node directly or compromise the landmarks involved in localization by attenuating or amplifying the signal strength readings. Based on our experimental attacks using real materials, we will use the linear attack model [22] (i.e. a material causes a constant percentage power loss independent of distance) as shown in Figure 3.1 to describe the effect of an attack on the RSS readings at the wireless device or at the landmarks. As presented in the figure, these attacks are easy to conduct with low cost materials. The linear relationship implies that it is easy for an adversary to control the effect of an attack on the observed signal strength by appropriately selecting different materials Experimental Methodology In order to study the generality of our attack detection approaches, we have conducted experiments in two office buildings, one is the 3rd floor of the Computer Science building at Rutgers University (CoRE) as shown in Figure 3.2 (a) and the other is in a floor of an industrial research lab (Industrial Lab) as presented in Figure 3.2 (b). In Figure 3.2 (a), the experiments are performed for both an (WiFi) network as well as an (ZigBee) network. For

64 49 the (WiFi) network, there are 4 landmarks shown in red squares deployed in a collinear manner to maximize signal strength coverage. While for the (ZigBee) network, there are 4 landmarks shown in magenta circles placed in a square set to maximize localization accuracy [2]. For experiments conducted in the industrial lab, as depicted in Figure 3.2 (b), we only used an (WiFi) network with 5 landmarks. The small green dots are the localization testing points and the small blue stars are the training points. We will present the results of our experiments for each of the proposed attack detection cases in its associated section in this chapter. Across all experiments, we have performed a trace-driven evaluation by either attenuating or amplifying RSS readings collected from these two buildings. 3.3 Generalized Attack Detection Model In this section we first propose a general formulation for the localization attack detection problem. We then introduce metrics for evaluating the effectiveness of our approaches Localization Attack Detection In general, the error of a localization algorithm is defined as the distance between the true location x = [x,y] T and the estimated location ˆx, D err = x ˆx. We found in prior work that under physical attacks, the localization error D err increases significantly [22]. However, D err is not directly available during run-time, and the challenge in attack detection is to devise strategies for detecting localization attacks that do not use localization errors. Attacked signal strength (dbm) Thin book Thick book Foil More foil Water Human body Glass Metal Signal Attenuation through Various Materials Non attacked signal strength (dbm) Figure 3.1: Linear attack model on received signal strength for various media.

65 5 8 7 Landmark 2 Landmark 1 6 Landmark 1 5 Landmark 4 Y (feet) 4 Landmark 2 Landmark Landmark 4 Landmark X (feet) (a) CoRE Building Y (feet) Landmark 1 Landmark 2 Landmark Landmark 4 Landmark X (feet) (b) Industrial Lab Figure 3.2: Layout of the experimental floor We propose to formulate location attack detection as a statistical significance testing problem, where the null hypothesis is H : normal (no attack). In significance testing, a test statistic T is used to evaluate whether observed data belongs to the null-hypothesis or not. For a particular significance level, α (defined as the probability of rejecting the hypothesis if it is true), there is a corresponding acceptance region Ω such that we declare the null hypothesis valid if an observed value of the test statistic T obs Ω, and reject the null hypothesis if T obs Ω (i.e. declare an attack is present if T obs Ω c, where Ω c is the critical region of the test). In our attack detection problem, the region Ω and decision rule is specified according to the form of the detection statistic T (for example, when using distance in signal strength space for T, the decision rule becomes comparison against a threshold), and

66 51 rejection of the null hypothesis corresponds to declaring the presence of an attack Effectiveness In order to evaluate the effectiveness of our attack detection methods, we will utilize the following performance metrics: Cumulative Distribution Function (CDF): The CDF of the test statistic T provides the sensitivity of T under attack. Based on the CDF, we can study the feasibility of using T for attack detection. Detection Rate (DR): An attack may cause the significance test to reject H. We are thus interested in the statistical characterization of the attack detection attempts over all the localization attempts. The Detection Rate is defined as the percentage of localization attempts that are determined to be under attack, i.e.: DR = N attack N total (3.1) where N total is the total number of localization attempts and N attack is the number concluded under attack by detection. Note that when the signal is attacked, the detection rate corresponds to the probability of detection P d, while under normal (non-attack) conditions it corresponds to the probability of declaring a false positive P fa. We will examine DR as a function of the attack strength. Receiving Operating Characteristic (ROC) curve: To evaluate an attack detection scheme we want to study the false positive rate P fa and probability of detection P d together. The ROC curve is usually used to measure the tradeoff between false-positives and correct detections. The ROC curve is a plot of attack detection accuracy against the false positive rate. It can be obtained by varying the detection thresholds. 3.4 Using Least Squares In this section we provide mathematical analysis for attack detection in multilateration algorithms. We first provide background in using LS to perform localization. Next, based on the properties of the LLS estimator, we define an attack detection scheme that utilizes regression

67 52 residuals, and give an analytic formulation to specify the acceptance region Ω. Finally, the experimental results are presented to evaluate the effectiveness of the detection scheme Localization To perform localization with LS requires 2 steps: ranging and lateration. Ranging Step: Recent research has seen a host of variants on the ranging step such as RSS, TOA, TDOA, and hop count. Our attack detection approach works with any ranging modality. Lateration Step: From the estimated distances d i and known positions (x i, y i ) of the landmarks, the position (x, y) of the localizing node can be found by finding (ˆx,ŷ) satisfying: (ˆx,ŷ) = arg min x,y n [ (x i x) 2 + (y i y) 2 d i ] 2 (3.2) i=1 where n is the total number of landmarks. We call solving the above problem Nonlinear Least Squares, or NLS. Solving the NLS problem requires significant complexity and is difficult to analyze. We may approximate the NLS solution and linearize the problem [2] into the system Ax = b, where: and x 1 1 P n n i=1 x i y 1 1 P 1 n n i=1 y i A = B.. A x n 1 P n n i=1 x i y n 1 P n n i=1 y i (x P n n i=1 x2 i ) + (y2 1 1 P 1 n n i=1 y2 i ) (d b = P n n i=1 d2 i ) 2.. (3.4) B 2 n 1 P n n i=1 x2 i ) + (y2 n 1 P n n i=1 y2 i ) C A (d 2 n 1 P n n i=1 d2 i ) (3.3) Note that A is described by the coordinates of landmarks only, while b is represented by the distances to the landmarks together with the coordinates of landmarks. We call the above formulation of the problem Linear Least Squares, or LLS. The estimate of x = [x,y] T is done via x = (A T A) 1 A T b (3.5) In addition to its computational advantages, the LLS formulation allows for tractable statistical analysis, as we shall now see.

68 The Residuals In practice, there are estimation errors from the ranging step. The LLS formulation can be refined as a linear regression, b = Ax+e, where e corresponds to model errors. The localization result is then ˆx = (A T A) 1 A T b, and the fitted values ˆb corresponding to the observed values b are given by ˆb = Aˆx = A[(A T A) 1 A T b] = A(A T A) 1 A T b. (3.6) Further, we define the vector of residuals ê as ê = b ˆb = [1 A(A T A) 1 A T ]b. (3.7) When the regression model is performing well we may assume that the model errors are Gaussian [41,65]. Under this assumption, the residuals also follow a Gaussian distribution, N(µ, Σ), since the residuals are a linear combination of the elements of b and e. Here, µ is the mean vector and Σ is the covariance matrix. We choose the residuals ê as the test statistic T, and will build our attack detection scheme by using the statistical properties of ê when LLS is operating in a desirable performance regime The Detection Scheme The LLS attack detection is performed after localization. The residuals are correlated Gaussian random variables and the multivariate Gaussian distribution of ê can be expressed as: 1 f(ê) = ( 2π) n e 1 Σ 1 2 (ê µ)t Σ 1 (ê µ). (3.8) 2 In order to determine whether the location result is compromised by adversaries, we perform attack detection through significance testing. We can define an acceptance region in ê space by Ω = {ê : Pr({T : (T µ) T Σ 1 (T µ) > (ê µ) T Σ 1 (ê µ)}) > α}. In practice, after performing localization using LLS, we have an observed value of residuals ê obs. Testing the null hypothesis, we can decide that the localization is under attack if the probability P = 1 M < α, where 1 M = ( 2π) n Σ e 1 2 (ê µ)t Σ 1 (ê µ) dê 1...dê n (3.9) E

69 54 and E is the integration region defined by (ê µ) T Σ 1 (ê µ) X 2 with X 2 = (ê obs µ) T Σ 1 (ê obs µ). We can express the term (ê µ) T Σ 1 (ê µ) = (ê µ) T D T D(ê µ) = (D(ê µ)) T (D(ê µ)) = y T y. (3.1) Substituting y = D(ê µ) into Equation (3.9), we get 1 M = ( 2π) n... e 1 2 yty dy 1...dy n E 1 = ( 1 P n 2π) n... 2 i=1 y2 i dy1...dy n (3.11) E e with E defined by y T y X 2. Changing to polar coordinates, we get M = = = with and Using v = r 2 /2, we have 1 ( 2π) n X 2π π... π sinφ 2 dφ 2...sin n 2 φ n 1 dφ n 1 ] 1 ( 2π) n n 1 i=2 π X e r2 2 r n 1 dr sin i 1 φ i dφ i [e r2 2 r n 1 drdφ 1 2π dφ 1 n 1 2 ( π) n 2 A r,n B i (3.12) i=2 A r,n = 1 X ( e r2 2) n 2 r n 1 dr B i = π sin i 1 φ i dφ i. A r,n = 1 2 X 2 2 where Γ is the incomplete gamma function. Since e v v n 2 2 dv = 1 2 Γ(n 2, X2 2 ) (3.13) B i = β( i 2, 1 2 ) = Γ( i 2 ) Γ( i+1 2 ) π. (3.14)

70 55 Through further simplification, we can get n 1 i=2 B i = ( π) n 2 1 Γ( n (3.15) 2 ). Hence, substituting Equations (3.13) and (3.15) into (3.12), we obtain the probability mass M = Γ(n/2,X2 /2). Γ(n/2) We then further obtain the probability by P = 1 M. Based on the definition in Section 3.3, if the probability is sufficiently low, i.e. P < α, then ê obs belongs to the critical region Ω c and we can conclude that the location result is under attack Experimental Evaluation In this section we present the evaluation of the effectiveness of the attack detection scheme. We chose RSS as the ranging modality and performed signal strength attacks according to the experimental methodologies described in Section 3.2. The average ranging error as a function of the severity of signal strength attacks is shown in Figure 3.3(a). We know that the relationship between the RSS error and the ranging error is multiplicative with distance [2]. Even small random perturbation in RSS readings can cause large ranging errors due to this multiplicative factor. We observed this effect in Figure 3.3(a); the ranging error increases superlinearly to attack severity. Figure 3.4 presents DR vs. the ranging errors when tested against significance level α =.1 and α =.5. We found that under a normal situation, where the ranging errors are less than 15 feet, the false alarm probability, P fa, is less than 1.5% and 2.5% for α =.1 and α =.5 respectively. Large signal strength attacks, greater than 15dB, can cause ranging errors larger than 9 feet, and then the detection rates are more than 9%. These results strongly indicate that using residuals in LS as a test statistic for attack detection is effective. Further, the ROC curves in Figure 3.3(b) show that for false positive rates less than 1%, the detection rates are above 9% and close to 99% when the attack strength increases to 2dB and 25dB. This shows that if the adversary wants to cause a large localization error, it is almost certain that our attack detection mechanism will detect it. For small attacks of less than 5dB, the detection rates are about 4%. In this case, it is difficult to distinguish whether the anomaly

71 Ranging error (ft) Signal strength attack (db) (a) Ranging errors 1.9 Probability of detection, P d dB 1dB 15dB 2dB 25dB False positive rate, P fa (b) ROC curves Figure 3.3: CoRE 82.11: (a) Ranging errors under the signal strength attacks (b) LLS residuals: Receiver Operating Characteristic (ROC) curves Detection rate, DR Detection rate, DR Ranging error (ft) Ranging error (ft) (a) LOA: α =.1 (b) LOA: α =.5 Figure 3.4: CoRE 82.11, LLS residuals: effectiveness of attack detection

72 Probability normal 5dB attack.2 1dB attack 15dB attack.1 2dB attack 25dB attack Minimum distance in signal space (db) (a) CDF Probability of detection, P d dB.2 1dB 15dB.1 2dB 25dB False positive rate, P fa (b) ROC curves Figure 3.5: CoRE 82.11: (a) Cumulative Distribution Function (CDF) of minimum distance D s in signal space. (b) Minimum distance D s : Receiver Operating Characteristic (ROC) curves. in the test statistic is caused by attacks or by measurement errors since the RSS readings can fluctuate around 5dB due to environmental effects. However, for such small attacks, because the resulting impact on the final localization result was shown to be small [22], the consequences of failing to detect such attacks would likely be small as well. 3.5 Distance In Signal Space RSS is a common physical property used by a widely diverse set of algorithms. For example, most scene matching approaches utilize the RSS, e.g. [12, 14], and many multilateration approaches [52] use it as well. In spite of its several meter-level accuracy, using the RSS is

73 58 an attractive approach because it can re-use the existing wireless infrastructure this feature presents a tremendous cost savings over deploying localization-specific hardware. In this section, we thus derive an attack detection scheme applicable to any signal strength based localization system Overview All of the above algorithms take a vector s of n RSS readings to (or from) n landmarks for the node to be localized. Note that s corresponds to a point in a n-dimensional signal space [22]. Under normal conditions, the RSS vectors obtained from the physical positions in a floor form a surface S in the n-dimensional signal space; we can think of this surface as comprising valid points in signal space. Due to measurement noise, multipath effects, and unknown biases, s will fluctuate around this idealized RSS surface. A localization attacker would perturb the correct s to produce a corrupted n-dimensional RSS vector s. In signal space, s will be moved away from the ideal surface constructed by the correct RSS vectors. The stronger the attack, the more likely the vector s will be distant from the RSS surface. We thus choose the minimum distance to the surface S, i.e. D s = min{ s s j : where s j S}, as the test statistic for signal strength based attack detection. The key advantage of this approach is that the attack detection is independent of the localization algorithms and can be performed before the localization process. Although it is possible to devise a statistical model for D s based on models for normal measurement errors, in this section we shall take a different approach and apply empirical methodologies from training data to determine thresholds for defining the critical region Finding Thresholds Choosing an appropriate threshold τ will allow the detection scheme to be robust to false detections. In order to obtain the thresholds, we don t need to know the exact RSS surface in the signal space (in practice, it is hard to determine and exhibits discontinuities due to wall boundaries). Instead, we can obtain the thresholds through empirical training. During the offline phase, we can collect the RSS vectors for a set of known positions over the floor and construct

74 59 a radio map. During the localization phase, we get an observed vector s obs, and we can then determine whether the s obs is being attacked by calculating the D s using the pre-constructed radio map. We define that if D s > τ, (3.16) the signal strength readings are under attack. We use the distribution of the training data to help decide on the thresholds. Figure 3.5 (a) shows the CDF of the D s in signal space. We found that the curve of D s shifted to the right under signal strength attacks, especially for larger attacks, thereby suggesting that we can use D s as a test statistic for detecting attacks, and also that we can use the non-attacked CDF to obtain τ for a given α value Experimental Evaluation We next present the evaluation of the effectiveness of using minimum distance D s for attack detection. Figure 3.6 presents the Detection Rate under different threshold (TH) levels as a function of signal strength attacks for both the and the networks in CoRE and the network in the Industrial Lab. Figure 3.5 (b) is the corresponding ROC curves under signal attenuation attacks for the network in CoRE. We found that, in general, the effectiveness of the attack detection scheme is similar across the different networks and buildings. Interestingly, we found that the performance of the attack detection scheme under signal amplification attacks is uniformly better than those for signal attenuation attacks, although the shapes of the DR curves are similar. Because of the higher detection rates under amplification attacks, we do not present additional amplification results in the remainder of the chapter. All these results are highly encouraging because they show our methods are quite general and do not depend on a specific network or environment. Further, we observed that the DR under the network in CoRE outperformed the DR under the networks in both CoRE and Industrial Lab for the signal attenuation attacks as well as the signal amplification attacks. For attack strengths of 15dB or larger, the DR in the network is over 95% and equals 1% when attack severity reaches 2dB and larger. We believe that the better landmark placement for localization [2] of the

75 6 network can account for its higher detection rates, although further investigation of this effect is required.

76 61 Detection rate, DR Attenuation, TH = 8dB Attenuation, TH = 9dB.2 Attenuation, TH = 1dB Amplification, TH = 9dB.1 Amplification, TH = 1dB Amplification, TH = 11dB Signal strength attack (db) (a) CoRE Detection rate, DR Attenuation, TH = 8dB Attenuation, TH = 9dB.2 Attenuation, TH = 1dB Amplification, TH = 8dB.1 Amplification, TH = 9dB Amplification, TH = 1dB Signal strength attack (db) (b) CoRE Detection rate, DR Attenuation, TH = 11dB Attenuation, TH = 12dB.2 Attenuation, TH = 13dB Amplification, TH = 11dB.1 Amplification, TH = 12dB Amplification, TH = 13dB Signal strength attack (db) (c) Industrial Figure 3.6: Minimum distance in signal space D s : attack detection across different networks and buildings.

77 Other Test Statistics In this section, we examine algorithm-specific test statistics, which use properties specific to a particular localization algorithm. We have chosen a representative set of diverse algorithms. For the multilateration category, we investigate the NLS algorithm, while for signal strength based algorithms, we study both Area Based Probability (ABP) and Bayesian Networks (BN) algorithms. Detailed descriptions of these can be found in [2, 27, 52] Nonlinear Least Squares (NLS) As presented in Section 3.4, NLS is a multilateration algorithm that tries to satisfy the condition shown in Equation (3.2). The estimated (ˆx,ŷ) is the solution that minimizes the Sum of Squared Errors E 2 : n E 2 = [ (x i ˆx) 2 + (y i ŷ) 2 d i ] 2. (3.17) i=1 We define a test statistic E = E 2 because E will likely increase under the attack. The CDF of E presented in Figure 3.7 (a) confirms that the E grows rapidly with the attack severity. Figure 3.7 (b) and Figure 3.8 show that the performance of attack detection when using E for the network in CoRE is comparable to that using residuals in Section 3.4. The thresholds are also obtained from training Area Based Probability (ABP) Turning to signal strength based algorithms, ABP is an area-based algorithm that uses Bayes Rule to return an area which has the highest likelihood of capturing the true location [27]. ABP divides the floor into a set of tiles. The total likelihood that the wireless node resides at each tile is calculated using: n P = P i (3.18) i=1 where n is the total number of landmarks and P i is the likelihood of observing the measured RSS reading at landmark i which is usually modeled as a Gaussian random variable. The total likelihood is calculated at each tile, and the returned location estimation is either a region whose likelihood is above a certain level, or is the tile with the maximum likelihood.

78 Probability normal 5dB attack.2 1dB attack 15dB attack.1 2dB attack 25dB attack Square root of sum of squared error (ft) (a) CDF Probability of detection, P d dB.2 1dB 15dB.1 2dB 25dB False positive rate, P fa (b) ROC curves Figure 3.7: CoRE 82.11, NLS: (a) Cumulative Distribution Function (CDF) of E. (b) E: Receiver Operating Characteristic (ROC) curves. When under attack, the corrupted RSS readings reduce the set of likely positions on the floor to localize a node. We found that the highest tile-likelihood denoted as likelihood max decreases significantly under attack, as well as the sum of the likelihoods over all the tiles, likelihood sum. We explored both likelihood sum and likelihood max as test statistics. The thresholds are learned from the training data by taking the negative log of the values of the highest likelihood and the sum of the likelihoods. The effectiveness of using likelihood sum and likelihood max for attack detection in ABP are presented in Figure 3.9 and Figure 3.1. We found that using likelihood sum under threshold equal to 2 had better performance than others in detecting larger attacks, but on the other hand resulted in slightly higher false positive rates around 7%.

79 64 Detection rate, DR TH = 5ft.1 TH = 6ft TH = 7ft Ranging error (ft) Figure 3.8: CoRE 82.11, NLS using E: effectiveness of attack detection Detection rate, DR likelihood sum, TH = 2 likelihood sum, TH = 3.1 likelihood max, TH = 4.5 likelihood max, TH = Signal attenuation (db) Figure 3.9: CoRE 82.11, ABP: effectiveness of attack detection Probability of detection, P d dB.2 1dB 15dB.1 2dB 25dB False positive rate, P fa Figure 3.1: CoRE 82.11, ABP: Receiver Operating Characteristic (ROC) curves

80 Bayesian Networks (BN) Another representative signal strength based algorithm, BN, utilizes Bayesian networks [52]. With Bayesian statistical inference, BN predicts the probability distribution of the unknown positions. BN uses a Monte-Carlo sampling technique (Gibbs sampling) to compute the full joint-probability distribution for not just the position coordinates, but also for every random variable in the Bayesian network. Without an attack, the contribution from each landmark to the full joint-probability distribution is almost uniform. Under an attack, we found that the contribution from each landmark can become significantly reduced as the attack severity increases. Thus, we can use the fraction of contribution to the joint probability as a test statistic in BN. Another method we explored is to use the probability likelihood because the conditional probability distribution of the coordinates in BN relies on the prior and the likelihood. We observed that under an attack, the value of the likelihood became significantly smaller. During the sampling process, the calculation of the likelihood uses the same approach as in Equation (3.18). Because the absolute value of the likelihood is very small, we take the negative log of the likelihood and use it as a test statistic for attack detection in BN. Figure 3.11 shows the effectiveness of using the fraction of contribution and the likelihood for attack detection in BN. The detection rates are over 9% for attack strength of 2dB or larger. The false positive rates are about 1%. Comparing the absolute performance of these two methods with the other schemes we proposed in this chapter, the performance of these two methods is qualitatively worse. 3.7 Discussion Comparing all of our detection schemes, Figure 3.12 shows the DR as a function of the signal attenuation attacks for the network in the CoRE building. Surprisingly, we found that the performance of all the schemes provided qualitatively similar detection rates, although utilizing the residuals in LLS and the sum of likelihoods in ABP slightly outperformed the others, while using the fraction of contribution and the likelihood in BN underperformed the others. Based on these similar performance characteristics, it is advantageous to use the minimum

81 66 Detection rate, DR Signal attenuation (db) (a) Detection rate, DR Signal attenuation (db) (b) Figure 3.11: CoRE 82.11, BN: (a) Using fraction of contribution of each landmark for attack detection with threshold =.15. (b) Using likelihood in Bayesian inference for attack detection with threshold =.25. Detection rate, DR LLS: α =.1 LLS: α =.1.3 NLS: TH = 6ft Minimum distance Ds: TH = 8dB.2 BN likelihood: TH =.25 BN fraction: TH =.15.1 ABP likelihood sum: TH = 2 ABP likelihood max: TH = Signal attenuation (db) Figure 3.12: CoRE 82.11: Comparison between generic and specific test statistics for attack detection. distance in the signal space D s for signal strength based algorithms. Since the attack detection can be performed prior to the localization process and thus results in localization computation cost savings under attack. Additionally, the attack detection performance under the network when using D s outperforms the network with 1% detection rate for large attacks as shown in Figure 3.6. Moving to examine the relationship between attack detection and localization error, Figure 3.13 shows the DR when using residuals in LLS for attack detection, and the localization errors under the corresponding signal attacks with different localization algorithms. The figure shows that detection rates are more than 9% for attack strength equal to or greater than 15dB,

82 67 Detection rate, DR Detection rate, α =.5 RADAR.1 Area Based Probability Bayesian Networks Ranging error (ft) Localization error (ft) Figure 3.13: CoRE 82.11: Relationships among Detection Rate (DR), ranging error, and localization error. and at this attack strength the average localization error is about 35ft. The above result is quite encouraging, as it shows that an attacker cannot cause gross localization errors without there being a very high probability of detection ( 95%). In the case of RSS, with mean errors of 1-15 ft [27], an attacker can not cause errors of about 2-3 times over the average error without a very high probability of detection. Even for detection rates as low as 5%, the attacker s position error is limited to about 2 ft. 3.8 Conclusion In this chapter, we analyzed the problem of detecting non-cryptographic attacks on wireless localization. We proposed a theoretical foundation by formulating attack detection as a statistical significance testing problem. We then concentrated on test statistics for two broad localization approaches: multilateration and signal strength. For multilateration that uses Linear Least Squares, we derived a closed-form representation for the attack detector. Further, for localization schemes that employ signal strength, we showed that by utilizing the signal strength as a common feature, the minimum Euclidean distance in the signal space can be used as a test statistic for attack detection independent of the localization process. Further, we derived additional test statistics for a selection of representative localization algorithms. We studied the effectiveness and generality of our attack detection schemes using a tracedriven evaluation involving both an (WiFi) network and an (ZigBee) network in two real office buildings. We evaluated the performance of our attack detection schemes in

83 68 terms of detection rates and receiver operating characteristic curves. Our experimental results provide strong evidence of the effectiveness of our attack detection schemes with high detection rates, over 95% and low false positive rates, often below 5%. Also, our approach is generic across a diverse set of algorithms, networks, and buildings. Interestingly, we found that the performance of the different attack detection schemes are more similar than different. This result shows that different localization systems have similar attack detection capabilities, and consequently that system designers can focus on using algorithms that provide the highest localization accuracy rather than having to tradeoff position accuracy against attack detection abilities. After a localization attack is detected in a wireless network, the next important and challenging step is to localize the positions of the adversaries and further to eliminate the attack from the network. In the next chaper, we illustrate this idea further by examining the applicability of localization methods to locate an adversary participating in a spoofing attack. A spoofing attack is an attack where the attacker forges its identity and masquerades as another device, or even creates multiple illegitimate identities. Although the identity of a node can be verified through cryptographic authentication, authentication is not always desirable or possible because it requires key management and additional infrastructure overhead. We will take a different approach by using the physical properties of the radio signal and propose a scheme using K-means cluster analysis for both detecting spoofing attacks as well as localizing the positions of the adversaries without adding any overhead to the wireless devices and sensor nodes.

84 69 Chapter 4 Detecting and Localizing Identity-based Spoofing Attacks 4.1 Introduction Due to the openness of wireless and sensor networks, they are especially vulnerable to spoofing attacks where an attacker forges its identity to masquerade as another device, or even creates multiple illegitimate identities. Spoofing attacks are a serious threat as they represent a form of identity compromise and can facilitate a variety of traffic injection attacks, such as evil twin access point attacks. It is thus desirable to detect the presence of spoofing and eliminate them from the network. The traditional approach to address spoofing attacks is to apply cryptographic authentication. However, authentication requires additional infrastructural overhead and computational power associated with distributing, and maintaining cryptographic keys. Due to the limited power and resources available to the wireless devices and sensor nodes, it is not always possible to deploy authentication. In addition, key management often incurs significant human management costs on the network. In this chapter, we take a different approach by using the physical properties associated with wireless transmissions to detect spoofing. Specifically, we propose a scheme for both detecting spoofing attacks, as well as localizing the positions of the adversaries performing the attacks. Our approach utilizes the Received Signal Strength (RSS) measured across a set of access points to perform spoofing detection and localization. Our scheme does not add any overhead to the wireless devices and sensor nodes. By analyzing the RSS from each MAC address using K-means cluster algorithm, we have found that the distance between the centroids in signal space is a good test statistic for effective attack detection. We then describe how we integrated our K-means spoofing detector into a real-time indoor localization system. Our K-means approach is general in that it can be applied to almost all RSS-based localization algorithms. For two sample algorithms, we show that

85 7 using the centroids of the clusters in signal space as the input to the localization system, the positions of the attackers can be localized with the same relative estimation errors as under normal conditions. To evaluate the effectiveness of our spoofing detector and attack localizer, we conducted experiments using both an network as well as an network in a real office building environment. In particular, we have built an indoor localization system that can localize any transmitting devices on the floor in real-time. We evaluated the performance of the K-means spoofing detector using detection rates and receiver operating characteristic curve. We have found that our spoofing detector is highly effective with over 95% detection rates and under 5% false positive rates. Further, we observed that, when using the centroids in signal space, a broad family of localization algorithms achieve the same performance as when they use the averaged RSS in traditional localization attempts. Our experimental results show that the distance between the localized results of the spoofing node and the original node is directly proportional to the true distance between the two nodes, thereby providing strong evidence of the effectiveness of both our spoofing detection scheme as well as our approach of localizing the positions of the adversaries. The rest of the chapter is organized as follows. In Section 4.2, we study the feasibility of spoofing attacks and their impacts, and discuss our experimental methodologies. We formulate the spoofing attack detection problem and propose K-means spoofing detector in Section 4.3. We introduce the real-time localization system and present how to find the positions of the attackers in Section 4.4. Further, we provide a discussion in Section 4.5. Finally, we conclude our work in Section Feasibility of Attacks In this section we provide a brief overview of spoofing attacks and their impact. We then discuss the experimental methodology that we use to evaluate our approach of spoofing detection.

86 Spoofing Attacks Due to the open-nature of the wireless medium, it is easy for adversaries to monitor communications to find the layer-2 Media Access Control (MAC) addresses of the other entities. Recall that the MAC address is typically used as a unique identifier for all the nodes on the network. Further, for most commodity wireless devices, attackers can easily forge their MAC address in order to masquerade as another transmitter. As a result, these attackers appear to the network as if they are a different device. Such spoofing attacks can have a serious impact on the network performance as well as facilitate many forms of security weaknesses, such as attacks on access control mechanisms in access points [1], and denial-of-service through a deauthentication attack [15]. A broad survey of possible spoofing attacks can be found in [29, 47]. To address potential spoofing attacks, the conventional approach uses authentication. However, the application of authentication requires reliable key distribution, management, and maintenance mechanisms. It is not always desirable to apply authentication because of its infrastructural, computational, and management overhead. Further, cryptographic methods are susceptible to node compromise a serious concern as most wireless nodes are easily accessible, allowing their memory to be easily scanned. It is desirable to use properties that cannot be undermined even when nodes are compromised. We propose to use received signal strength (RSS), a property associated with the transmission and reception of communication (and hence not reliant on cryptography), as the basis for detecting spoofing. Employing RSS as a means to detect spoofing will not require any additional cost to the wireless devices themselves they will merely use their existing communication methods, while the wireless network will use a collection of base stations to monitor received signal strength for the potential of spoofing Experimental Methodology In order to evaluate the effectiveness of our spoofing detection mechanisms, which we describe in the next section, we have conducted experiments using both an (WiFi) network as well as an (ZigBee) network on the 3rd floor of the Computer Science Department at Rutgers University. The floor size is 2x8ft (16 ft 2 ). Figure 4.1 (a) shows the 82.11

87 Landmark Landmark 4 Y (feet) 4 Landmark 2 Landmark X (feet) (a) network 8 7 Landmark 3 Landmark Y (feet) Landmark 1 Landmark X (feet) (b) network Figure 4.1: Landmark setups and testing locations in two networks. (WiFi) network with 4 landmarks deployed to maximize signal strength coverage, as shown in red squares. The (ZigBee) network is presented in Figure 3.2 (b) with 4 landmarks distributed in a squared setup in order to achieve optimal landmark placement [2] as shown in red triangles. The small blue dots in the floor map are the locations used for spoofing and localization tests. For the network, we used 3 packet-level RSS samples for each of the 1 locations. We utilized the actual RSS values attached to each packet. We have 286 locations in the deployment. Unlike the data, the RSS values are partially synthetic. We had access to only the mean RSS at each location, but to perform our experiments we needed an RSS value per packet. To generate such data for 2 simulated packets at each location, we used random draws from a normal distribution. We used the measured RSS mean for the mean

88 73 of the distribution. For the standard deviation, we computed the difference in the RSS from a fitted signal to distance function, and then calculated the standard deviation of the distribution from these differences over all locations. To keep our results conservative, we took the maximum deviation over all landmarks, which we found to be 5 db. Much work has gone into characterizing the distributions of RSS readings indoors. It has been shown that characterizing the per-location RSS distributions as normal, although not often the most accurate characterization, still results in the best balance between algorithmic usability and the resulting localization error [27, 33]. In addition, we built a real-time localization system to estimate the positions of both the original nodes and the spoofing nodes. We randomly selected points out of the above locations as the training data for use by the localization algorithms. For the network, the size of the training data is 115 locations, while for the network, the size of the training data is 7 locations. The detailed description of our localization system is presented in Section 4.4. To test our approach s ability to detect spoofing, we randomly chose a point pair on the floor and treated one point as the position of the original node, and the other as the position of the spoofing node. We ran the spoofing test through all the possible combinations of point pairs on the floor using all the testing locations in both networks. There are total pairs for the network and 4371 pairs for the network. The experimental results will be presented in the following sections for the spoofing detector and the attack localizer. 4.3 Attack Detector In this section we propose our spoofing attack detector. We first formulate the spoofing attack detection problem as one using classical statistical testing. Next, we describe the test statistic for spoofing detection. We then introduce the metrics to evaluate the effectiveness of our approach. Finally, we present our experimental results Formulation of Spoofing Attack Detection RSS is widely available in deployed wireless communication networks and its values are closely correlated with location in physical space. In addition, RSS is a common physical property

89 74 used by a widely diverse set of localization algorithms [12, 27, 6, 69]. In spite of its several meter-level localization accuracy, using RSS is an attractive approach because it can re-use the existing wireless infrastructure. We thus derive a spoofing attack detector utilizing properties of the RSS. The goal of the spoofing detector is to identify the presence of a spoofing attack. We formulate the spoofing attack detection as a statistical significance test, where the null hypothesis is: H : normal (no attack). In significance testing, a test statistic T is used to evaluate whether observed data belongs to the null-hypothesis or not. If the observed test statistic T obs differs significantly from the hypothesized values, the null hypothesis is rejected and we claim the presence of a spoofing attack Test Statistic for Spoofing Detection Although affected by random noise, environmental bias, and multipath effects, the RSS value vector, s = {s 1,s 2,...s n } (n is the number of landmarks/access points(aps)), is closely related with the transmitter s physical location and is determined by the distance to the landmarks [27]. The RSS readings at different locations in physical space are distinctive. Each vector s corresponds to a point in a n-dimensional signal space [22]. When there is no spoofing, for each MAC address, the sequence of RSS sample vectors will be close to each other, and will fluctuate around a mean vector. However, under a spoofing attack, there is more than one node at different physical locations claiming the same MAC address. As a result, the RSS sample readings from the attacked MAC address will be mixed with RSS readings from at least one different location. Based on the properties of the signal strength, the RSS readings from the same physical location will belong to the same cluster points in the n-dimensional signal space, while the RSS readings from different locations in the physical space should form different clusters in signal space. This observation suggests that we may conduct K-means cluster analysis [34] on the RSS readings from each MAC address in order to identify spoofing. If there are M RSS sample

90 75 readings for a MAC address, the K-means clustering algorithm partitions M sample points into K disjoint subsets S j containing M j sample points so as to minimize the sum-of-squares criterion: K J min = s m µ j 2 (4.1) j=1 s m S j where s m is a RSS vector representing the mth sample point and µ j is the geometric centroid of the sample points for S j in signal space. Under normal conditions, the distance between the centroids should be close to each other since there is basically only one cluster. Under a spoofing attack, however, the distance between the centroids is larger as the centroids are derived from the different RSS clusters associated with different locations in physical space. We thus choose the distance between two centroids as the test statistic T for spoofing detection, D c = µ i µ j (4.2) with i,j {1,2..K}. Next, we will use empirical methodologies from the collected data set to determine thresholds for defining the critical region for the significance testing. To illustrate, we use the following definitions, an original node P org is referred to as the wireless device with the legitimate MAC address, while a spoofing node P spoof is referred to as the wireless device that is forging its identity and masquerading as another device. There can be multiple spoofing nodes of the same MAC address. Note that our K-means spoofing detector can handle packets from different transmission power levels. If an attacker sends packets at a different transmission power level from the original node with the same MAC address, there will be two distinct RSS clusters in signal space. Thus, the spoofing attack will be detected based on the distance of the two centroids obtained from the RSS clusters Determining Thresholds The appropriate threshold τ will allow the spoofing detector to be robust to false detections. We can determine the thresholds through empirical training. During the off line phase, we can collect the RSS readings for a set of known locations over the floor and obtain the distance between two centroids in signal space for each point pair. We use the distribution of the training information to determine the threshold τ. At run time, based on the RSS sample readings for

91 Probability normal spoofing attack Distance between two centroids in signal space (db) (a) network Probability normal spoofing attack Distance between two centroids in signal space (db) (b) network Figure 4.2: Cumulative Distribution Function (CDF) of D c in signal space a MAC address, we can calculate the observed value Dc obs. Our condition for declaring that a MAC address is under a spoofing attack is: D obs c > τ. (4.3) Figure 4.2 (a) and (b) show the CDF of the D c in signal space for both the network and the network. We found that the curve of D c shifted greatly to the right under spoofing attacks, thereby suggesting that using D c as a test statistic is an effective way for detecting spoofing attacks Performance Metrics In order to evaluate the performance of our spoofing attack detector using K-means cluster analysis, we use the following metrics:

92 77 Network, Threshold Detection Rate False Positive Rate 82.11, τ = 5.5dB , τ = 5.7dB , τ = 6dB , τ = 8.2dB , τ = 1dB , τ = 11dB.9577 Table 4.1: Detection rate and false positive rate of the spoofing attack detector. Detection Rate and False Positive Rate: A spoofing attack will cause the significance test to reject H. We are thus interested in the statistical characterization of the attack detection attempts over all the possible spoofing attacks on the floor. The detection rate is defined as the percentage of spoofing attack attempts that are determined to be under attack. Note that, when the spoofing attack is present, the detection rate corresponds to the probability of detection P d, while under normal (non-attack) conditions it corresponds to the probability of declaring a false positive P fa. The detection rate and false positive rate vary under different thresholds. Receiver Operating Characteristic (ROC) curve: To evaluate an attack detection scheme we want to study the false positive rate P fa and probability of detection P d together. The ROC curve is a plot of attack detection accuracy against the false positive rate. It can be obtained by varying the detection thresholds. The ROC curve provides a direct means to measure the trade off between false-positives and correct detections Experimental Evaluation In this section we present the evaluation results of the effectiveness of the spoofing attack detector. Table 4.1 presents the detection rate and false positive rate for both the network and the network under different threshold settings. The corresponding ROC curves are displayed in Figure 4.3. The results are encouraging showing that for false positive rates less than 1%, the detection rates are above 95%. Even when the false positive rate goes to zero, the detection rate is still more than 95% for both and networks. We further study how likely a spoofing node can be detected by our spoofing attack detector when it is at varying distances from the original node in physical space. Figure 4.4 presents the detection rate as a function of the distance between the spoofing node and the original node. We found that the further away P spoof is from P org, the higher the detection rate becomes. For the

93 78 Probability of detection, P d False positive rate, P fa (a) network Probability of detection, P d False positive rate, P fa (b) network Figure 4.3: Receiver Operating Characteristic (ROC) curves network, the detection rate goes to over 9% when P spoof is about 13 feet away from P org under τ equals to 5.5dB. While for the network, the detection rate is above 9% when the distance between P spoof and P org is about 2 feet by setting threshold τ to 9dB. This is in line with the average localization estimation errors using RSS [27] which are about 1-15 feet. When the nodes are less than 1-15 feet apart, they have a high likelihood of generating similar RSS readings, and thus the spoofing detection rate falls below 9%, but still greater than 6%. However, when P spoof moves closer to P org, the attacker also increases the probability to expose itself. The detection rate goes to 1% when the spoofing node is about 45-5 feet away from the original node.

94 Spoofing detection rate (%) τ = 5.5dB τ = 6dB Distance between the spoofing node to the original node (feet) (a) network 1 9 Spoofing detection rate (%) τ = 9dB τ = 1dB Distance between the spoofing node to the original node (feet) (b) network Figure 4.4: Detection rate as a function of the distance between the spoofing node and the original node. 4.4 Localizing Adversaries If the spoofing attack is determined to be present by the spoofing attack detector, we want to localize the adversaries and further to eliminate the attackers from the network. In this section we present a real-time localization system that can be used to locate the positions of the attackers. We then describe the localization algorithms used to estimate the adversaries position. The experimental results are presented to evaluate the effectiveness of our approach Localization System We have developed a general-purpose localization system to perform real-time indoor positioning. The detailed system architecture is presented in Chapter 6. Here we provide a brief system

95 8 overview. This system is designed with fully distributed functionality and easy to plug-in localization algorithms. It is built around 4 logical components: Transmitter, Landmark, Server, and Solver. The system architecture is shown in Figure 6.1 in Chapter 6. Transmitter: Any device that transmits packets can be localized. Often the application code does not need to be altered on a sensor node in order to localize it. Landmark: The Landmark component listens to the packet traffic and extracts the RSS reading for each transmitter. It then forwards the RSS information to the Server component. The Landmark component is stateless and is usually deployed on each landmark or access point with known locations. Server: A centralized server collects RSS information from all the Landmark components. The spoofing detection is performed at the Server component. The Server summarizes the RSS information such as averaging or clustering, then forwards the information to the Solver component for localization estimation. Solver: A Solver takes the input from the Server, performs the localization task by utilizing the localization algorithms plugged in, and returns the localization results back to the Server. There are multiple Solver instances available and each Solver can localize multiple transmitters simultaneously. During the localization process, the following steps will take place: 1. A Transmitter sends a packet. Some number of Landmarks observe the packet and record the RSS. 2. Each Landmark forwards the observed RSS from the transmitter to the Server. 3. The Server collects the complete RSS vector for the transmitter and sends the information to a Solver instance for location estimation. 4. The Solver instance performs localization and returns the coordinates of the transmitter back to the Server. If there is a need to localize hundreds of transmitters at the same time, the server can perform load balancing among the different solver instances. This centralized localization solution also makes enforcing contracts and privacy policies more tractable.

96 P org 6 L org 5 Y (feet) P spoof 1 L spoof X (feet) Figure 4.5: Relationships among the original node, the spoofing node, and their location estimation through localization system Attack Localizer When our spoofing detector has identified an attack for a MAC address, the centroids returned by the K-means clustering analysis in signal space can be used by the server and sent to the solver for location estimation. The returned positions should be the location estimate for the original node and the spoofing nodes in physical space. Using a location on the testing floor as an example, Figure 4.5 shows the relationship among the original node P org, the location estimation of the original node L org, the spoofing node P spoof, and the localized spoofing node position L spoof. In order to show the generality of our localization system for locating the spoofing nodes, we have chosen two representative localization algorithms using signal strength from pointbased algorithms and area-based algorithms. RADAR: Point-based methods return an estimated point as a localization result. A primary example of a point-based method is the RADAR scheme [12]. In RADAR, during the off line phase, a mobile transmitter with known position broadcasts beacons periodically, and the RSS readings are measured at a set of landmarks. Collecting together the averaged RSS readings from each of the landmarks for a set of known locations provides a radio map. At runtime, localization is performed by measuring a transmitter s RSS at each landmark, and the vector of RSS values is compared to the radio map. The record in the radio map whose signal strength vector is closest in the Euclidean sense to the observed RSS vector is declared to correspond to

97 82 the location of the transmitter. In this work, instead of using the averaged RSS in the traditional approach, we use the RSS centroids obtained from the K-means clustering algorithm as the observed RSS vector for localizing a MAC address. Area Based Probability (ABP): Area-based algorithms return a most likely area in which the true location resides. One major advantage of area-based methods compared to point-based methods is that they return a region, which has an increased chance of capturing the transmitter s true location. ABP returns an area, a set of tiles on the floor, bounded by a probability that the transmitter is within the returned area [27]. ABP assumes the distribution of RSS for each landmark follows a Gaussian distribution. The Gaussian random variable from each landmark is independent. ABP then computes the probability of the transmitter being at each tile L i on the floor using Bayes rule: P(L i s) = P(s L i) P(L i ). (4.4) P(s) Given that the transmitter must reside at exactly one tile satisfying L i=1 P(L i s) = 1, ABP normalizes the probability and returns the most likely tiles up to its confidence α. Both RADAR and ABP are employed in our experiments to localize the positions of the attackers Experimental Evaluation In order to evaluate the effectiveness of our localization system in finding the locations of the attackers, we are interested in the following performance metrics: Localization Error CDF: We obtain the cumulative distribution function (CDF) of the location estimation error from all the localization attempts, including both the original nodes and the spoofing nodes. We then compare the error CDF of all the original nodes to that of all the possible spoofing nodes on the floor. For area-based algorithms, we also report CDFs of the minimum and maximum error. For a given localization attempt, these are points in the returned area that are closest to and furthest from the true location. Relationship between the true and estimated distances: The relationship between the true distance of the spoofing node to the original node P org P spoof and the distance of the location estimate of the spoofing node to that of the original node L org L spoof evaluates

98 83 how accurate our attack localizer can report the positions of both the original node and the attackers. We first present the statistical characterization of the location estimation errors. Figure 4.6 presents the localization error CDF of the original nodes and the spoofing nodes for both RADAR and ABP in the network as well as the network. For the area-based algorithm, the median tile error ABP med is presented, as well as the minimum and maximum tile errors, ABP min and ABP max. We found that the location estimation errors from using the RSS centroids in signal space are about the same as using averaged RSS as the input for localization algorithms [27]. Comparing to the network, the localization performance in the network is qualitatively better for both RADAR and ABP algorithms. This is because the landmark placement in the network is closer to that predicted by the optimal and error minimizing placement algorithm as described in [2]. More importantly, we observed that the localization performance of the original nodes is qualitatively the same as that of the spoofing nodes. This is very encouraging as the similar performance is strong evidence that using the centroids obtained from the K-means cluster analysis is effective in both identifying the spoofing attacks as well as localizing the attackers. The challenge in localizing the positions of the attackers arises because the system does not know the positions of either the original MAC address or the node with the masquerading MAC. Thus, we would like to examine how accurate the localization system can estimate the distance between P org and P spoof. Figure 4.7 displays the relationship between L org L spoof and P org P spoof across different localization algorithms and networks. The blue dots represent the cases of the detected spoofing attacks. While the red crosses indicate the spoofing attack has not been detected by the K-means spoofing detector. Comparing with Figure 4.4, i.e. the detection rate as a function of the distance between P org and P spoof, the results of the undetected spoofing attack cases represented by the red crosses are in line with the results in Figure 4.4, the spoofing attacks are 1% detected when P org P spoof equals to or is greater than about 5 feet. Further, the relationship between L org L spoof and P org P spoof is along the 45 degree straight line. This means that L org L spoof is directly proportional to P org P spoof and indicates that our localization system is highly effective for localizing the attackers. At a

99 Probability ABPmin original ABPmed original.3 ABPmax original RADAR original.2 ABPmin spoofing ABPmed spoofing.1 ABPmax spoofing RADAR spoofing Error (feet) (a) network Probability ABPmin original ABPmed original.3 ABPmax original RADAR original.2 ABPmin spoofing ABPmed spoofing.1 ABPmax spoofing RADAR spoofing Error (feet) (b) network Figure 4.6: Localization error CDF across localization algorithms and networks.

100 85 2 Estimated distance from localization (feet) Estimated distance from localization (feet) True distance between two nodes (feet) (a) 82.11: RADAR, τ = 6dB 2 Estimated distance from localization (feet) Estimated distance from localization (feet) 2 (b) 82.11: ABP, τ = 6dB True distance between two nodes (feet) True distance between two nodes (feet) 2 (c) : RADAR, τ = 9dB True distance between two nodes (feet) 2 (d) : ABP, τ = 9dB Figure 4.7: Relationship between the true distance and the estimated distance for the original node and the spoofing node across localization algorithms and networks. fixed distance value of Porg Pspoof, the values of Lorg Lspoof fluctuate around the true distance value. The fluctuation reflects the localization errors of both Porg and Pspoof. The larger the Porg Pspoof is, the smaller the fluctuation of Lorg Lspoof becomes, at about 1 feet maximum. This means that if the attacker is farther away from the original node, it is extremely likely that the K-means spoofing detector can detect it. In addition, our attack localizer can find the attacker s position and estimate the distance from the original node to the attacker at about 1 to 2 feet maximum error.

101 86 Distance between centroids from localization (feet) True distance between two points (feet) Figure 4.8: Packet-level localization: relationship between the true distance and the estimated distance for the original node and the spoofing node when using RADAR in the network. 4.5 Discussion So far we have conducted K-means cluster analysis in signal space. Our real-time localization system also inspired us to explore packet-level localization at the server, which means localization is performed for each packet received at the landmarks. The server utilizes each RSS reading vector for localization. Over a certain time period (for example, 6 seconds), for a MAC address there will be a cluster of location estimates in physical space. Intuitively, we think that, during a spoofing attack there will be distinctive location clusters around the original node and the spoofing nodes in physical space. Our intuition was that the clustering results from the per-packet localization would allow the detection and localization of attackers in one step. However, we found that the performance of clustering packet-level localization results for spoofing detection is not as effective as deriving the centroids in signal space. The relationship between P org P spoof and L org L spoof is shown in Figure 4.8. Although it also has a trend along the 45 degree line, it shows more uncertainties along the line. Therefore, we believe that given a set of RSS reading samples for a MAC address, working with the signal strength directly preserves the basic properties of the radio signal, and this in turn is more closely correlated with the physical location, and thus working with the RSS values directly better reveals the presence of the spoofing attacks.

102 Conclusion In this chapter, we proposed a method for detecting spoofing attacks as well as localizing the adversaries in wireless and sensor networks. In contrast to traditional identity-oriented authentication methods, our RSS based approach does not add additional overhead to the wireless devices and sensor nodes. We formulated the spoofing detection problem as a classical statistical significance testing problem. We then utilized the K-means cluster analysis to derive the test statistic. Further, we have built a real-time localization system and integrated our K- means spoofing detector into the system to locate the positions of the attackers and as a result to eliminate the adversaries from the network. We studied the effectiveness and generality of our spoofing detector and attacker localizer in both an (WiFi) network and an (ZigBee) network in a real office building environment. The performance of the K-means spoofing detector is evaluated in terms of detection rates and receiver operating characteristic curves. Our spoofing detector has achieved high detection rates, over 95% and low false positive rates, below 5%. When locating the positions of the attackers, we have utilized both the point-based and area-based algorithms in our real-time localization system. We found that the performance of the system when localizing the adversaries using the results of K-means cluster analysis are about the same as localizing under normal conditions. Usually the distance between the spoofing node and the original node can be estimated with median error of 1 feet. Our method is generic across different localization algorithms and networks. Therefore, our experimental results provide strong evidence of the effectiveness of our approach in detecting the spoofing attacks and localizing the positions of the adversaries. During the course of the security analysis for localization systems, we found that the landmark placement plays an important role on localization performance. While most research has focused on improving the localization algorithm, we took the viewpoint that it is perhaps just as important to improve the deployment of the localization system. In the next chapter, we will investigate the impact of landmark placement on localization performance using a combination of analytic and experimental analysis.

103 88 Chapter 5 Performance Improvement Using Optimal Landmark Placement 5.1 Introduction Although recent efforts have resulted in a plethora of methods to localize sensor nodes, little work to date has systematically investigated how the placement of the nodes with known locations, or landmarks, impacts localization performance. In this chapter we investigate the impact of landmark placement on localization performance using a combination of analytic and experimental analysis. Our analytic approach focuses on the Least Squares (LS) algorithm, and in particular, a variant we call Linear Least Squares (LLS). Our analysis centers on the algorithm for two reasons. First, LS is a widely used multilateration algorithm, as is evidenced by its application as a step in many recent localization research works [23, 28, 44, 49, 54]. Second, mathematical analysis of LLS is tractable, resulting in equations with closed-form solutions. For a myriad of other algorithms, closed form solutions that describe the localization error as a function of landmark placement are not tractable and as a result heuristic search strategies must be used to find an optimal placement, as was done in [13]. Our analysis of landmark placement can find an optimal placement of landmarks in welldefined regular regions, thus making it quite suitable for indoor localization. The analysis begins with LLS and places an upper bound of the maximum localization error given a set of landmark placements. We can show that this upper bound is minimized by a combination of minimizing the distance estimation error together with the employment of the optimal patterns for landmark placement. Using this result, we can compare the maximum error between any two placements. We can then constrain a search of placements to minimize the maximum error. We have developed a

104 89 simple algorithm called maxl mine algorithm that finds an optimized landmark deployment for the LLS algorithm. We show that our placement minimizing the upper bounds of LLS also reduces the Hölder parameter for a variety of algorithms. The Hölder parameter [22] describes the maximum change in physical space that can arise from a change in signal space. This is strong evidence that our maxl mine algorithm finds a landmark placement that minimizes the errors due to noise, bias, and measurement error. Another interesting result of our analysis is that for a small number of landmarks, simple shapes such as equilateral triangles and squares result in placements with better localization performance. Interestingly, for higher number of landmarks, we can show that extensions of shapes with equal sides, e.g. a hexagon, are non-optimal. Rather, the simple shapes enclose one another, for example, two enclosing equilateral triangles. We detail these geometries and describe rule-of-thumb for landmark placement in Section 5.2. To show the generality of our results, we conducted localization experiments with both an (WiFi) network as well as an (ZigBee) network in a real building environment. For the network, we used two ranging modalities, Received Signal Strength (RSS) to distance, and Time of Arrival (TOA). In the network, we used only RSS-to-distance. We compared the accuracy of a suite of localization algorithms using landmarks placed according to our analysis as well as landmarks placed in positions that provide good signal coverage but ignore localization concerns. While we found that all algorithms improved their performance, over a non-optimal placement for localization, we also observed that LS became competitive with the other algorithms, and that coarse-grained TOA ranging was less accurate than RSS-based approaches. The remainder of the chapter is as follows. We provide the theoretical analysis in Section 5.2. Then Section 5.3 describes the metrics that we use to characterize the localization performance. The investigation of the number of landmarks and their positions is provided in Section 5.4. Section 5.5 presents the experimental results across localization algorithms, networks, and ranging strategies. Finally we conclude in Section 2.7.

105 9 5.2 Theoretical Analysis In this section we first provide background on using LS algorithms for localization, and then describe the LLS variant. We next present our theoretical analysis of an upper bound on the error, and then discuss our maxl mine placement algorithm Background: Localization with LS To perform localization with LS requires 2 steps: ranging and lateration. Ranging Step: Recent research has seen a host of variants on the ranging step. For example, in the APS algorithm [54], hop counts are used to estimate ranges. Other approaches are also possible, [56] used the time-difference of arrival between an ultrasound pulse and a radio packet. In this work, we focus on RSS and TOA as ranging strategies. Lateration Step: From the estimated distances d i and known positions (x i, y i ) of the landmarks, the position (x, y) of the localizing node can be found by finding (ˆx,ŷ) satisfying: (ˆx,ŷ) = arg min x,y N [ (x i x) 2 + (y i y) 2 d i ] 2 (5.1) i=1 where N is the total number of landmarks. We call solving the above problem Nonlinear Least Squares, or NLS. It can be viewed as an optimization problem where the objective is to minimize the sum of the error squared. Solving the NLS problem requires significant complexity and is difficult to analyze. We may approximate the NLS solution and linearize the problem by introducing a constraint in the formulation. We start with the N 2 equations: (x 1 x) 2 + (y 1 y) 2 = d 2 1 (x 2 x) 2 + (y 2 y) 2 = d 2 2 (5.2). (x N x) 2 + (y N y) 2 = d 2 N Now, subtracting the constraint 1 N N [(x i x) 2 + (y i y) 2 ] = 1 N i=1 N d 2 i (5.3) i=1

106 91 from both sides, we obtain the following set of linear equations (x 1 1 NX x i )x + (y 1 1 NX y i )y = N N i=1 i=1 1 2 [(x2 1 1 NX x 2 i N ) + (y2 1 1 NX yi 2 N ) (d2 1 1 NX d 2 i N )] i=1 i=1 i=1. (5.4) (x N 1 NX x i )x + (y N 1 NX y i )y = N N i=1 i=1 1 2 [(x2 N 1 NX x 2 i N ) + (y2 N 1 NX yi 2 N ) (d2 N 1 NX d 2 i N )]. i=1 i=1 i=1 The above can be easily solved linearly using the form Ax = b with: and x 1 1 P N N i=1 x i y 1 1 P 1 N N i=1 y i A = B.. A x N 1 P N N i=1 x i y N 1 P N N i=1 y i (x P N N i=1 x2 i ) + (y2 1 1 P 1 N N i=1 y2 i ) (d b = P N N i=1 d2 i ) 2.. (5.6) (x 2 N 1 P N N i=1 x2 i ) + (y2 N 1 P N N i=1 y2 i ) C A (d 2 N 1 P N N i=1 d2 i ) (5.5) Note that A is described by the coordinates of landmarks only, while b is represented by the distances to the landmarks together with the coordinates of landmarks. We call the above formulation of the problem Linear Least Squares, or LLS. NLS trades higher computational complexity for better accuracy. The introduction of the constraint collapsed the nonlinear problem into a linear problem, which greatly simplifies the computation needed to arrive at a location estimate. In addition to its computational advantages, the LLS formulation allows for tractable error analysis, as we shall soon provide Error Analysis Our objective is to minimize the location estimation error introduced by LLS. we have matrix A and vector b presented in Equations (5.5) and (5.6). In an ideal situation solving for x = [x,y] T

107 92 is done via x = (A T A) 1 A T b (5.7) However, the estimated distances are impacted by noise, bias, and measurement error. We express the resulting distance estimation error e in terms of b with estimated distances and b with true distances as b = b + e, and hence the localization result is x = (A T A) 1 A T b. (5.8) The location estimation error is thus bounded by x x A + e, (5.9) where the matrix A + is the Moore-Penrose pseudo-inverse of A. It can be shown that, under the 2-norm, A + = 1 γ 2, where γ 1 γ 2 are the singular values of A. This means that for a certain size on error e the LS estimation error is stretched by 1 γ 2. It can be proved that the eigenvalues of A T A are the squares of the singular values of A. Therefore, we can limit our concern to the eigenvalues of A T A, where A T A is a matrix of the form: A T A = a b b c with: a = b = c = N (x i 1 N x i ) 2 (5.1) N i=1 N [(x i 1 N i=1 N (y i 1 N i=1 i=1 N x i )(y i 1 N i=1 N y i )] (5.11) i=1 N y i ) 2. (5.12) i=1 Note that a, b and c are only related to the coordinates of landmarks (x i, y i ). The eigenvalues of A T A can be found as the roots of: Thus, we have: λ 2 (a + c)λ + (ac b 2 ) =.

108 93 where the discriminant, (a c) 2 + 4b 2, is non-negative. λ = (a + c) ± (a c) 2 + 4b 2, (5.13) Deployment Patterns Our goal in this section is to minimize the total error. Recall there are two terms on the right side of Equation (5.9). Our approach is to choose x i and y i so as to make λ 2 (the smaller eigenvalue) as close to λ 1 as possible, because this will minimize the first term, A +. Given the first term is minimized, we then minimize the second term. Having minimized the second term given the first term is minimized is clearly a local minima. We call such a local minima an optimal deployment, because no movement of a single landmark can improve the error bound. However, our piecewise minimization approach still leaves open a proof that this local minima is the true minima over all possible landmark positions. We leave such a proof as future work. Returning to minimizing the first term A +, to minimize 1 λ2, a general strategy would be to make (a c) small or to make b small or both. Interestingly, this is determined only by the coordinates of the landmarks. Then our next task is to find the landmark positions that satisfy λ 1 = λ2. We found that the optimal landmark deployment setup follows some simple and symmetric patterns. This makes it not only possible to achieve but also easy to deploy practically. Figure 5.1 shows the patterns for an optimal landmark deployment setup when utilizing 3, 4, 5, 6, 7, 8 landmarks in the indoor environment. These patterns consist of squares, equilateral triangles, or the enclosing of them. We observe that for higher number of landmarks, the extensions of shapes with equal sides, e.g. a hexagon, do not satisfy λ 1 = λ2, and thus are not optimal. Instead, simple shapes that enclose one another present optimal solutions Finding an Optimized landmark Deployment The above discussion dealt with deploying the landmarks without considering the physical constraints of the building and, as such, only provide a general guideline as to the shape of the deployment. Placing the landmarks within a particular building requires stretching/shrinking

109 94 Figure 5.1: Patterns for optimal landmark deployments the deployment shape so that it fits within the confines of the building. The stretching/shrinking should be done so as to minimize localization errors. Recall in Equation (5.9), the location estimation error is also contributed by e, and that b = b + e. The term e is a result of distance estimation errors introduced by ranging. We have developed an iterative algorithm, called maxl mine (i.e. maximum lambda and minimum error), which helps to find the real landmark coordinates given the floor size, number of landmarks, and the optimal landmark deployment pattern. Figure 5.2 shows the pseudo-code that implements maxl mine. The algorithm first minimizes A + using geometry, then uses an iterative search. The search begins with a maximal sized optimal pattern (e.g. a square) and simply keeps reducing the size of the pattern until such movements stop reducing the distance estimation error e. We observe the algorithm usually converges very quickly within a number of iterations. 5.3 Evaluation Metrics In this section we describe the three metrics we use throughout the rest of the paper.

110 95 input f loorsize, numof Landmark output optimized landmark coordinates [initialize] get optimal pattern based on geometry fit optimal pattern into maximum floorsize generate initial landmark coordinates calculate λ 1 and λ 2 minerror = maxnum thiserror = maxn um loop until thiserror > minerror generate random localizing nodes for each localizing node begin apply random noise or bias B = b b end for thiserror = avg(b) λ2 if thiserror < minerror, minerror = thiserror [landmark adjustment] move towards the center of mass one step end loop return optimized landmark coordinates Figure 5.2: The maxl-mine algorithm Average error: All of our observations are the results of many localization trials. This metric takes the average of the distances between the localized result and the true location over all trials. In area-based algorithms, as opposed to point-based ones, the result is a returned area. To compare these two kinds of algorithms, we use the median X and Y of the returned area to the true location to generate a point and then average these distance errors. Accuracy CDF: We also return the entire cumulative density function (CDF) of all our localization attempts. We simply report all attempts in sorted order, and then normalize the Y axis by the total number of attempts to obtain a domain of [, 1]. For area-based algorithms, we also report CDFs of the minimum and maximum error. For a given attempt, these are points in the returned area that are closest to and furthest from the true location. Hölder Metrics: In addition to error performance, we are also interested in how dramatically the localization results can be perturbed by changes in signal strength. Hölder metrics for RSS based localization were introduced in a previous work [22]. Intuitively, these metrics

111 96 relate the magnitude of a perturbation to its effect on the localization result. The idea here is that certain landmark placements can reduce the impacts of perturbations due to noise or bias, and we should be able to observe these as lower Hölder parameters. The Hölder parameter H p alg for a given placement and algorithm is defined as Hp alg = L p alg max (s) Lp alg (v) s,v s v, where L p alg is the result of a localization algorithm alg given placement p, with s as a signal strength vector and v as a perturbed vector. Since the traditional Hölder parameter describes the maximum effect a signal perturbation might have, it is natural to also provide an average-case measurement. We therefore examine the average-case Hölder parameter, H p alg, as well. In both cases, we measure the metrics by statistical sampling in the case of simulation, or direct computation over all localization attempts for experimentally measured data. 5.4 Landmark Position and Quantity In this section we investigate the impact of landmark position and quantity on localization performance. Because the data collection process using many real deployments is prohibitively time-consuming, we use a trace-driven simulation methodology for this section. We first describe our methodology, then present our results investigating both the impact of landmark deployment and quantity using our previously defined metrics Simulation Methodology Our simulation methodology requires we generate a simulated RSS reading for any point on the floor of a building from any landmark. We first begin with the path loss equation that models the received power as a function of the distance to the landmark: P(d)[dBm] = P(d )[dbm] 1nlog( d ) (5.14) d We choose the parameters d = 1m, P(d ) = and n = from [12]. We then apply a random noise factor to perturb the RSS readings. This corresponds to the random model described in [48], which represents an upper bound on the signal variability. In many cases, we found that the localization error is large enough such that the estimated

112 97 deployment optimal horizontal vertical diagonal Topology 2x2ft Linear LS error H H Nonlinear LS error H H Topology 23ftx15ft Linear LS error H H Nonlinear LS error H H Table 5.1: Localization error (ft) and Hölder metrics when standard deviation of noise on rss is 3dB position is well outside the floor. This was particularly true for LLS. Because such results are unrealistic in our scenario, we apply a simple truncation rule in these cases: if the X or Y coordinate is outside the floor, we truncate to the maximum or minimum value along that dimension Evaluation of Estimation Error Table 5.1 presents the average location estimation error after the application of truncation and the Hölder metrics for both LS algorithms under 5 landmarks for our two simulated floors. The optimized landmark deployment setup is obtained from the maxl mine algorithm. It is encouraging that both NLS and LLS provide smallest estimation errors using our placement algorithm. By comparing the values of the Hölder parameters, the LS algorithm is the least susceptible to random noise with the optimized landmark deployment, which has 4 landmarks positioned as the vertex of a square plus the fifth landmark placed at the center of the mass. When under the diagonal landmark deployment, the localization results suffer the largest estimation errors and the algorithm is the most susceptible. The following results presented in this section are bounded by the floor boundary.

113 Estimation error (ft) square, LLS square, NLS 4 diagonal, LLS diagonal, NLS 2 horizontal, LLS horizontal, NLS Standard deviation of noise (db) (a) Estimation error (ft) square, LLS square, NLS diagonal, LLS diagonal, NLS Standard deviation of distance estimation error(ft) (b) Figure 5.3: In 2x2ft area: (a) Location estimation error vs. random noise in RSS (b) Location estimation error vs. ranging error Impact of Landmark Deployment In this section we describe the impact of 3 different deployments on localization performance. We use a representative situation of 5 landmarks deployed in 3 ways to demonstrate the impact of our algorithm in a typical case. The first deployment we call square, and in the 5 landmark case it is an optimal deployment when the shape is a square plus one landmark at the center of the mass. Next, the horizontal deployment is the one where all the landmarks placed in a line along the longest dimension; this will give better signal coverage than the square for rectangular buildings. Finally, we also examine the impact of a poor deployment, in this case diagonal, which equally spaces the landmarks along a diagonal line. Figure 5.3(a) shows the average accuracy of 1 random trials across the floor for the 3

114 99 deployments as a function of increasing the standard deviation σ rss of the noise term applied to each point. The six curves correspond to the NLS and LLS for each deployment. First, NLS always significantly outperforms LLS. When the σ rss is less than 4dB, which is typical based on our experimental experience, both algorithms under the optimized landmark deployment outperform the two other deployments. When the σ rss is larger than 4dB, under the optimized landmark deployment, the NLS still performs better, while the performance of the LLS is compatible with the performance of the NLS for horizontal and diagonal landmark deployments. Constant sized deviations in the RSS readings result in wide differences in the distance estimation depending on the distance to the landmark. Note that the relationship between the RSS error and ranging error is multiplicative with distance, i.e., d = d1 ss ss 1n. For example, in our simulation a 3dB error corresponds to a multiplicative factor of 1.5, at 1ft distance, d = 15ft with an error of 5ft, while at 1ft distance, d = 15ft with an error of 5ft, a factor of ten larger. We are motivated to study the magnitude of distance estimation error caused by the deviation of the RSS readings. Figure 5.3(b) shows the location estimation error vs. the standard deviation σ d of distance estimation error. We observe that a noise σ rss of 2dB corresponds to a distance error σ d of 32ft. Further, the estimation results when the σ rss is 4dB and 5dB translate to the σ d of 65ft and 82ft respectively. Thus, even small random perturbation in RSS readings cause large ranging estimation errors due to this multiplicative factor Impact of Landmark Quantity In this section we observe the impact of adding more landmarks. We compare the performance of the LS algorithms with 4, 6 and 2 landmarks under square and diagonal deployments. We use our optimized placement in the case of 4 and 6 landmarks, and a uniform randomized deployment for 2 landmarks. Figure 5.4 shows a promising result that when deploying 4 landmarks and 6 landmarks under their optimized deployments, the localization results using LS are compatible with the results using a much higher number landmarks, 2, in this case. If a small number of landmarks

115 1 Estimation error (ft) landmarks LLS 4 landmarks NLS 6 landmarks LLS 6 landmarks NLS 2 landmarks LLS 2 landmarks NLS Standard deviation of noise (db) (a)optimized case Estimation error (ft) landmarks LLS 4 landmarks NLS 4 6 landmarks LLS 6 landmarks NLS 2 2 landmarks LLS 2 landmarks NLS Standard deviation of noise (db) (b)worst case Figure 5.4: Performance of LS algorithms across different number of landmarks in 2x2ft area provide sufficient coverage, this is an encouraging observation because good localization performance can be achieved without a large number of landmarks. 5.5 Experimental Study In this section we present our experimental study by using PCMCIA cards and Telos Sky motes. The objective is to compare the impact of our landmark deployment analysis on a variety of algorithms and different ranging modalities. Although the mathematics of our analysis is based on LLS, we show that deployments based on maxl mine algorithm improve localization accuracy in widely diverse scenarios. We first give a brief description of a set of representative RSS-based localization algorithms. We then describe our experimental method. Next, we quantify the performance across the algorithms provided different landmark deployments. We also compare the localization accuracy and Hölder metrics for these algorithms. Finally, we provide a comparison between the RSS-based and TOA-based LS algorithms using our deployment strategy Algorithms In this study, our main focus is the localization algorithms that employ signal strength measurements. To demonstrate the general applicability of our landmark deployment algorithm, we test

116 11 our placement strategy on three widely different localization algorithms, RADAR, ABP, and BN. Although there are many other RSS-based localization algorithms, this set spans various strategies, and given all algorithms have qualitatively similar performance [27] we feel this set is sufficiently representative. RADAR is a point-based, scene-matching algorithm. The user first builds a training set of RSS values from landmarks matched to known locations. To localize, the object creates a vector of RSS values from the landmarks and the algorithm returns the training point closest to the vector using Euclidean distance as the discriminating function [12]. ABP uses Bayes rule combined with scene-matching to return an area the object is likely to reside in and probabilistically bounds the likelihood with a confidence level [27]. Taking the Bayesian network approach, the BN algorithm uses a Bayesian graphical model based on lateration to find the estimated location [52] Experimental Setup and Methodology A series of experiments are conducted in our Computer Science Department which resides the whole 3rd floor of the CoRE building. The floor size is 2x8ft (16 ft 2 ). The experiments are performed using 4 landmarks setup in the floor. Figure 5.5(a) shows the original collinear landmark deployment setup in triangles and our optimized landmark deployment as squares for the network. The networking staff of the department deployed the APs in the collinear deployment specifically to maximize signal strength coverage. The first set of RSS data was collected under this collinear deployment by using a Dell laptop running Linux equipped with an Orinoco silver card (82.11 card). The data was collected at 286 locations on the 3rd floor. Then we used a trace-driven approach to generate the RSS data set under the optimized landmark deployment. We first performed a least squares fit of the measured data and obtained the parameters of the path loss model in Equation (5.14). Then we directly used measured variance to generate the RSS readings. Finally, we applied environmental bias using the Ray- Sector model described in [48] to obtain the new RSS data set for the optimized deployment case. To validate that our trace-driven strategy generated realistic radio signal readings, we placed

117 Y (feet) X (feet) (a) Y (feet) X (feet) (b) Figure 5.5: Deployment of landmarks and training locations on the experimental floors 4 simulated landmarks at the same positions as the real collinear deployment and then generated synthetic RSS values. We compared the localization performance of using this synthetic data set against the real data. We found the estimation CDFs nearly identical for all of our algorithms under study. Thus we have confidence that our combination of path-loss model fitting, variance application, and bias generation result in RSS readings that generate realistic localization results. Our second experimental setup was an network which utilized 4 Telos Sky mote landmarks and deployed two sets of landmark placement positions. Figure 5.5 (b) shows the mote landmarks under an optimized square deployment as squares and a horizontal landmark deployment (again, to maximize signal strength coverage) as triangles. Unlike the case, no RSS data was generated; for both deployments the measured data is used in the algorithms.

118 Probability.6.5 linear LS bounded.4 nonlinear LS bounded linear LS.3 nonlinear LS BN.2 ABPmin ABPmed.1 ABPmax RADAR Error (feet) (a)collinear case Probability.6.5 linear LS bounded.4 nonlinear LS bounded linear LS.3 nonlinear LS BN.2 ABPmin ABPmed.1 ABPmax RADAR Error (feet) (b)square case Figure 5.6: Localization accuracy CDFs across algorithms for network We have experimented with different training set sizes for constructing the radio map for RADAR and ABP. For data sets, we show the results with 115 training points. While for data sets, we use 7 training points. The small stars in Figure 5.5 are the randomly selected training points. The localization at each testing point is performed by using the leaveone-out method Localization Accuracy Figure 5.6 (a) and (b) present the accuracy CDF under collinear and square landmark deployments, respectively. A bounded result means we applied truncation. ABP is calculated with confidence level 75%. ABP-med is the error of the median distance of the area, together with ABP-min and ABP-max are the closest and furthest points of the returned area. Figure 5.6(a) shows that under the horizontal-like deployment, LLS always fairs very poorly, while NLS, RADAR, ABP and BN are qualitatively similar. All the algorithms have long tails. Figure 5.7(a) shows a similar result when using the motes, although in here the perfect collinear deployment, the horizontal case, reduces the performance of the lateration approaches (BN, NLS, and LLS) compared to Figures 5.6(b) and 5.7(b) show the key impact of our work. All of the CDFs have shifted up and to the left compared to those in Figures 5.6(a) and 5.7(a). Thus, a significant fraction of the results are more accurate using the optimized deployments generated by maxl mine

119 Probability.6.5 linear LS bounded.4 nonlinear LS bounded linear LS.3 nonlinear LS BN.2 ABPmin ABPmed.1 ABPmax RADAR Error (feet) (a)horizontal case Probability.6.5 linear LS bounded.4 nonlinear LS bounded linear LS.3 nonlinear LS BN.2 ABPmin ABPmed.1 ABPmax RADAR Error (feet) (b)square case Figure 5.7: Localization accuracy CDFs across algorithms for network algorithm. In addition, for ABP, the gap between the min and max CDFs is much narrower, implying the returned areas are on average smaller than those in the horizontal deployments Evaluation of Performance and Sensitivity Table 5.2 summarizes the average error for each algorithm to further investigate the improvements gained by using an optimal deployment. The table shows the average error improves for all the algorithms. For data sets, the LLS algorithm improves over 35% and NLS gains 25% in performance. Both ABP and RADAR have improved over 2% in localization accuracy, while BN has gained 1%. Looking at the network, the performance improvement results are compatible to the results from the network. The Hölder metrics presented in Table 5.2 for each algorithm under the optimized landmark deployment is smaller than the horizontal deployment. Recall that the Hölder parameter is a measurement of the sensitivity of the algorithm to perturbations of inputs such as RSS, which can model random noise, environmental bias, and measurement errors. The lower Hölder values are strong evidence that an optimized landmark deployment not only can improve the localization performance, but also can make an algorithm less susceptible to the above classes of perturbations.

120 15 Average location estimation error (ft) Algorithms Linear LS Nonlinear LS BN ABP RADAR w trun w/o trun w trun w/o trun collinear square w trun w/o trun w trun w/o trun horizontal square Hölder (worst-case) H Algorithms Linear LS Nonlinear LS BN ABP RADAR w trun w/o trun w trun w/o trun collinear square w trun w/o trun w trun w/o trun horizontal square Hölder (average-case) H Algorithms Linear LS Nonlinear LS BN ABP RADAR w trun w/o trun w trun w/o trun collinear square w trun w/o trun w trun w/o trun horizontal square Table 5.2: Location estimation error (ft) and Hölder parameters across algorithms Using Time of Arrival In this section we experimentally investigate how well our deployment algorithm works for an alternate ranging modality. In this second modality, we compute the distance to a landmark by measuring many round trip times between a node and a landmark, and then calculate the time-of-flight (ToF) of a packet. Given the ToF and the speed of light, we can estimate the range. This is a Time-of-Arrival (TOA) based approach because the actual time-of-flight is estimated. Space limitations prevent us from describing this approach in more details, but a full description of the technique and an analysis of it can be found in [32]. We used a similar trace-driven based methodology in our TOA investigation as for the RSS one. We estimated the TOA based on the round trip times for packets and derived the distance between the localizing node to each landmark. We then built an error distribution of the true distance vs. the estimated distance, and used that to drive a simulation where we could place the landmarks in the same positions as the RSS study. The same hardware is used as for the RSS study. The linear regression model applied to the distance estimation error of TOA data with 63

121 16 Difference between true and estimated distances (ft) Distance (ft) (a)toa Distance estimation error (ft) Distance (ft) (b)rss Figure 5.8: Linear regression on TOA data experimental distances is shown in Figure 5.8(a). We observe that shorter the distance to a landmark results in estimated distance longer than the true distance, while longer the distance to a landmark results in estimation distance shorter than the true distance. The corresponding distance estimation error of RSS data is presented in Figure 5.8(b). Comparing the TOA results to RSS distance estimation errors, while the magnitude of the distance estimation error grows with lengthening distance, unlike in TOA the resulted estimation in RSS is either longer or shorter with near equal probability. With the mean and variance estimated from linear regression, we have modeled distance estimation error of TOA as a Gaussian distribution defined in Equation (5.15): error N(µ,σ 2 ) (5.15) with ˆµ = b + b 1 d i n and ˆσ 2 i=1 = ( d i ˆµ) 2, n 1 where d i is the true distance and d i is the estimated distance. n is the total number of distances under experimentation. b and b 1 are the coefficients of the linear regression. We further conducted a trace-driven approach to localize 286 positions on the floor using 4 landmarks setup with collinear and square deployment respectively according to Figure 5.5(a) for the network.

122 Probability Probability linear LS nonlinear LS.1 linear LS bounded nonlinear LS bounded Error (feet) (a)collinear case.2 linear LS nonlinear LS.1 linear LS bounded nonlinear LS bounded Error (feet) (b)square case Figure 5.9: Localization accuracy CDFs using TOA Figure 5.9 plots the localization accuracy CDF of the LS algorithms using TOA. The figure shows that as with RSS, the performance of LS increases under an optimized deployment as compared to a horizontal deployment designed for coverage. Quantitatively, the performance improvement is over 3%. Comparing the absolute performance of this technique with RSS, our TOA approach is qualitatively worse. This is likely due to the very coarse grained microseconds-level clocks currently available in standard Additional clocks with much higher frequencies would help to reduce much of the measurement uncertainty. 5.6 Conclusion In this chapter, by analyzing the Linear Least Squares algorithm, we derived an upper bound on the maximum location error given the placement of landmarks. Based on this theoretical analysis, we found optimal patterns for landmark placement and further developed a novel algorithm, maxl mine, for finding optimal landmark placement that minimizes the maximum localization error. To show the generality of our results, we conducted experiments using both an (WiFi) network and an (ZigBee) network. Based on the experimental data, we investigated the impact of landmark position and quantity on localization performance using both the measurements of RSS in an actual building as well as trace-driven simulations that used the RSS measurements. In addition, we apply the trace-driven approach to an alternate ranging

123 18 modality, in this case, TOA. We found that the performance of a wide variety of algorithms showed significant improvements when using landmarks placed according to our algorithm, as opposed to alternate deployments. We evaluated these improvements under several different metrics. The experimental results provide strong evidence that our analysis and algorithm for landmark placement is very generic as the resulting placement has improved localization performance across a diverse set of algorithms, networks, and ranging modalities. Our results also point out that there is a tension between the ideal landmark deployment for localization vs. deployments that optimize for signal coverage. We found that in our building, the better coverage deployment was very collinear, and this had pronounced negative impact on localization performance. Future work would conversely investigate the impact of a deployment optimized for localization on signal coverage, as well as try to find a method of trading one kind of deployment for another depending on the users needs. In the previous chapters we have explored methods and solutions to provide accurate and trustworthy localization results. Further, we would like to provide a scalable, general purpose, and real time localization infrastructure that can localized any radio-enabled devices at any where and any time. In the next chapter, we present a general purpose localization system prototype called GRAIL (Generalized Real-time Adaptable Indoor Localization) which can simultaneously position multiple devices.

124 19 Chapter 6 General Purpose Localization System 6.1 Introduction Utilizing the same infrastructure for both communication and positioning would provide a tremendous cost and deployment savings over a specific localization infrastructure. Thus one of the primary goals of localization research is to provide a scalable, general purpose, and real time localization infrastructure that can integrate location information into any computing radio-enabled devices. We are designing and developing a general purpose localization system prototype called GRAIL (Generalized Real-time Adaptable Indoor Localization) which can simultaneously position multiple devices using Bayesian Networks. The deployment of such a system in academic and research environments will allow researchers to explore issues beyond just algorithms and simulation tools. It would make it possible to conduct higher-level interaged research investigation including privacy studies, security services, and policy enforcement. For instance, we utilized the GRAIL system to conduct research on spoofing attacks in Chapter 4. In addition, the practical usage of such an approach is significant because it can be applied to a broad array of applications such as monitoring, tracking, routing, and security services. Localization is a diverse area covering everything from lower-layer physical problems to application-level services. GRAIL assumes the localization area to be about the size of a building, where devices have access to gateway nodes, and these in turn can access wired networks. Additional properties of the GRAIL system include: General Purpose. A primary goal of the GRAIL system is that it should work over a variety of physical modalities and networks. Much as a networking system should support multiple media access layers, a general purpose localization system should support multiple physical modalities and methods of localization.

125 11 Figure 6.1: GRAIL system architecture GRAIL is designed to localize using any wireless network that supports physical layer measurements of packet data. It supports to use Received Signal Strength (RSS) as the physical modality and can be easily extended to support Angle-of-Arrival (AoA), and time-of-arrival (TOA). Real-time. Latency is a key property of localization systems because it defines the maximum mobility that can be supported. Our system can return results in less than one second, allowing us to support both stationary devices as well as those moving at walking speeds (about 1m/s). Adaptable. A common problem with many systems is that they are too brittle; they require specific environments, hardware, or much training data related to a specific set-up. GRAIL uses real-time feedback to dynamically calibrate its parameters due to changing radio conditions. Indoor. Indoor environments are especially challenging due to reflections, refractions, and scattering, which result in substantial multi-path effects. GRAIL manages the uncertainly of these effects. GRAIL can expand and contract the possible set of locations as we introduce or reduce the uncertainty in the environment. Specifically, we can show how modifying the antennas can either increase or decrease spatial uncertainty.

126 111 Figure 6.2: Solver scalability and flexibility. 6.2 Architecture Design The GRAIL system is designed with fully distributed functionality and easy to plug-in localization algorithms. As shown in Figure 6.1, The main software components are Transmitters, Landmarks, the Server, and Solvers. The Landmarks collect the RSS reading for each transmitting device and send it to the Server together with the coordinates of the landmarks. Landmarks are stateless, which greatly simplifies the design, and are deployed in known loations. Often, in other research works, landmarks are called Anchor Points or Access Points. However, we use the term landmark because an access point also provides access to the wired network, and in GRAIL, landmarks do no provide this function. Upon receiving the RSS readings from each Landmark module, the Server collects the complete set of RSS readings for each node, decides on which localization algorithm to use, then forwards the RSS information to the corresponding Solver module. The Solver is flexible and easy to scale. Many different kinds of localization algorithms can be plugged in as illustrated in Figure 6.2. The Server and Solver components are fully decoupled. For localization using Bayesian networks, in addition to the WinBugs Solver which utilizes the statistical WinBugs tool [9], we have implemented the Fast Solver [4]. The Fast Solver is developed by using a novel real-time sampling technique which reduces computational cost significantly and solves Bayesian networks 9 to 17 times faster than the WinBugs Solver. The

127 112 GRAIL system can localize 1 to 1 sensor nodes in less than half a second, and scales to localize 51 objects simultaneously with no location information in the training data within 6 seconds. Although our current GRAIL system uses Bayesian networks, its distributed and scalable architecture was designed for maximum flexibility. GRAIL can thus easily accommodate alternative localization modalities and algorithms by using replaceable components. Thus other localization algorithms [12, 27] such as RADAR (R), Simple Point Matching (SPM), and Area Based Probability (ABP) can be added on easily with this flexible architecture design. During localization, there are multiple Solver instances available and each can represent one type of algorithm. Once the localization results are returned by the Solver, the Server displays the positions of the unknown transmitting devices. If there is a need to localize hundreds of transmitting devices simultaneously, the Server can perform load balancing among the different Solver instances as shown in Figure 6.2. Plus, this centralized server also makes enforcing contracts and privacy policies more tractable. 6.3 Bayesian Networks In this section, we give a brief overview of Bayesian Networks that are used in the GRAIL system. The Bayesian network is a graphical model that encodes dependencies and relationships among a set of random variables. The vertices of the graph correspond to the random variables and the edges represent dependencies. Bayesian inference in conjunction with Bayesian networks offers an efficient and principal approach for avoiding the over-fitting of data. In the GRAIL system, we have developed several Bayesian graphical models to encode the relationship between the RSS and the location based on signal-to-distance propagation model. We have built both non-hierarchical (M1) and hierarchical (M2) Bayesian graphical models as presented in Figure 6.3 (a) and (b). The location measurement process is slow and labor-intensive. By contrast, gathering RSS readings without the corresponding locations does not require human intervention. For example, sniffing devices can perform RSS measurements repeatedly at essentially no cost. So, we pursue the idea that different access points behave similarly and the prior knowledge may provide sufficient constraints to obviate the need to know the actual locations of the training data

128 113 (a) Non-hierarchical Bayesian graphical model (b) Hierarchical Bayesian graphical model Figure 6.3: Bayesian Networks observations. As a result, we have extended the M2 model and built a new model called M3 whose training data comprise solely of signal strengths of unknown locations [52]. This leads to a truly adaptive, zero-profiling technique for location estimation. The GRAIL system fully supports M1, M2 and M3 models for performing localization using either WinBugs Solver or Fast Solver.